12 – Evaluating Subsidies on energy efficiency in Non-residential buildings using Energy-intensity indicators

The primary audience for this guide is energy efficiency programme designers, implementers or supervisors, and evaluators looking for guidance on the evaluation process of energy savings in the scope of this guide.

Although the application of the specific guide will generally concern the (sub)national level, account will be taken of issues at EU level when relevant (e.g. the specific format of saving figures for the EED).

This guide is not about the preceding step in the evaluation process, the choice of the method. About this previous step in the evaluation process, see the guidance provided here, table 1 (page 3). However, after presenting the capabilities and limitations of the guide at hand, the user will be offered alternatives for the method within this guide (see section 6).

The objective of this guide is to provide:

• Information on the scope of the guide that enables the user to decide whether this guide is suited to his/her needs, and whether complementary or additional method(s) could be needed or useful (section 2);
• Guidance about specifying the evaluation objectives and requirements (section 3);
• Guidance about key methodological choices to calculate energy savings (section 4);
• Guidance about the inputs (data requirements) and outputs of the method (energy savings metrics) (section 5);
• Possible alternative methods (with pros and cons) (section 6)
• Background about evaluation results other than energy savings (section 7);
• Relevant examples, case studies and/or good practices (section 8);
• Relevant references for further reading (section 9).

The guide is intended for assessing realised (ex-post) energy savings. However, account is taken of earlier (ex-ante) evaluations of expected savings, if available (see section 4).

The focus of the specific guide is on impact evaluation, i.e. determining the energy savings, but not on how this has been reached through a step by step process with intermediate results (process evaluation).

Readers looking for the basic and general principles of energy efficiency evaluation may find the following link useful.

2.1 About Subsidies

More information and examples on the different subtypes residing under the main type Grants & Subsidies (GS) can be found in the Odyssee-Mure platform and in the EPATEE Knowledge Base. The focus of this specific guide can be extended to tax-rebates (part of the main type Fiscal / Tariffs) that stimulate energy savings in the same way as subsidies.

Subsidy schemes on non-residential buildings generally concern grants covering part of the investments needed for various saving actions, such as insulation and more efficient heating systems. The grant will often concern a fraction of the total investment for saving actions. The grant can also be dependent on the quality of the saving actions, expressed as the amount of savings per Euro invested.

The subsidy scheme is often managed by an agency that reports to the government about provided subsidies, the number of executed saving actions and the savings realized. Often an evaluation of the effectiveness of the scheme is performed as part of the general control on government spending.

More detailed information on the evaluation of Grants can be found for example in the EMEEES report here.

2.2 Evaluation for a combination of policy measure types

Grants are often combined with information (part of main policy type Information & Education), in order to inform building owners on the possibilities for saving energy. These grants are also combined with the policy type Voluntary Agreements between a group of end-users and government/agency. For such combination it is assumed that the overall savings are mainly due to the grants, although the evaluation concerns the combined savings effect of both policy measures.

The guide is not capable of attributing part of the (overall) calculated savings to each of the policy measures (see also Double counting in the section on Calculating Gross and Net savings, paragraph 4.5).

2.3 Evaluation when combined with energy taxes

Energy taxes may also be present, which stimulate to invest into all kind of saving actions because taxes raise the savings on energy costs. The calculated savings effect for grants will overlap with that of the energy tax.

This guide is not capable of attributing part of the (overall) calculated savings to either the policy measures at hand or the energy tax. For dealing with this overlap, see section on Gross to Net savings.

2.4 About Buildings/non-residential

Information on (sub)sectors defined in the Toolbox ) can be found in the Odyssee-Mure platform and in the EPATEE Knowledge Base.
Non-residential buildings concern various building types, such as offices, shops, hotels, schools and hospitals. Most of these buildings fall under the sector Services, but offices can also occur in the industry sector or the sector Transport.

The focus of the subsidy scheme is often on existing buildings, as savings for new buildings are generally realized with the deployment of minimum efficiency standards. The scheme can focus on owners of buildings and/or on real estate companies that rent offices, shops and etcetera.

2.5 About energy-intensity indicators

Information about the various evaluation methods can be found here, table 1 and 2. This source also covers the combination of the method at hand with other methods, which will be dealt with below.

Energy-intensity indicators describe the ratio between energy consumption and the output/production/performance for which energy is used. If the yearly calculated ratio decreases this can be seen as energy efficiency improvement.

Unit consumption indicators are defined similar to energy-intensity indicators but differ as to scale of energy consumption. The unit consumption indicator concerns the equipment level and the energy-intensity indicator the (sub)sector level. This specific guide concerns subsectors of the sector Services, where energy consumption of buildings can be related to activity quantities, such as yearly turnover, number of employees or floor area.

The energy-intensity can be affected by the different characteristics of subsectors. E.g. energy consumption in office buildings is mainly related to working hours, while in hotel buildings it is related to the occupation rate of the rooms. In order to avoid unintended effects, the intensity method should be applied at the lowest possible aggregation level with one uniform building type.

For a specific subsector a lower energy intensity can be due to other factors than energy efficiency, e.g. changes in the occupation rate of the buildings, for which corrections should be made (see section on normalization factors in chapter 4).

The indicator is calculated yearly on the basis of statistical or other sources (see paragraph 5.1 on main data requirements).

The indicator method provides gross energy savings due to all saving actions, whether due to e.g. higher energy prices or in response to specific policy measures. Therefore, the (net) savings due to policy cannot be calculated generally. However, the introduction of a policy with a large impact on the savings can create a trend break in the structural development of the indicator value, from which policy derived (net) savings could be calculated.

2.6 Complementary methods to determine total savings

Complementary methods are methods that are required, in addition to the primary selected method, to calculate the total energy savings.

The method at hand provides directly the total savings (instead of combining unitary savings (per saving action) with the number of actions as in most guides of the Toolbox). Hence no complementary method is needed. For more information about methods to calculate unitary savings, see table 2 in this link.

2.7 Additional methods to increase reliability of the results

An additional method can be applied on top of the chosen method to improve the reliability of the evaluation results and/or the cost-effectiveness of the evaluation approach.

The method of energy-intensity indicator is based on aggregated data on energy consumption and a related quantity for a subsector. The yearly savings are calculated from changes in energy consumption and the related quantity that are small compared to the accuracy margin in the available data. Thus, the reliability is very dependent on accuracy of the yearly data and should be checked with additional methods.

As to the effect of stimulating savings through policy, calculated from a trend break in the structural decrease for the indicator value, the reliability of the results is very dependent on the accuracy of the yearly data as well.

In order to verify the energy indictor method, it may be checked with an additional method based on (sample wise) measurement. The alternative billing analysis can also increase the reliability of the savings figures, but in a more cost-effective way. The method of engineering estimates can unravel the changes in energy-intensity into various saving actions for buildings, thus enabling to calculate also the net savings. However, this method needs detailed data for a sufficient set of buildings and is rather costly.

For possible combinations with an additional method see chapter 6 in this link.

3.1 Meeting evaluation goals and ambition

The table shows whether this guide can be used to report on general evaluation goals or criteria. See also this document.

General types of evaluation goals or criteria	Level of  ambition	Remarks
Calculation of realized energy savings from saving actions	Fair	Depends on the accuracy of the annual data used to calculate the savings
Calculation of energy savings attributed to the policy measure(s)	Low	The method calculates savings from all saving actions, policy savings can only be derived from a trend break in the trend for the indicator value
Cost-effectiveness of saving action (for end-users)	Absent	Aggregated savings, no information on saving actions (savings and investment costs)
Cost-effectiveness of policy (government spending)	Low	Given spending per subsector, only in case of a trend break (see energy savings)
CO2-emission reduction from saving actions	Fair	See energy savings (calculated per energy carrier)
CO2-emission reduction attributed to the policy measure(s)	Low	See energy savings (calculated per energy carrier)

For more information on verification of actual energy savings and attribution/baseline/corrections, see section 4, and for cost-effectiveness and emission reduction see section 7.

3.2 Reporting expectations

Possible reporting options:

• Gross savings (Net savings only in some cases depending on the policy impact)
• Yearly savings and cumulative of discounted savings

No free rider effects can be reported.

3.3 Time frame for evaluation

The length of the evaluation period is dependent in general on:

• the active period of the policy measure (without major changes),
• the need to monitor developments before the implementation of savings actions
• the time needed to present (reliable) results or impacts that fit into the decision-making process. In some cases, the periodicity of evaluation can be set by law.

For the guide at hand the calculation of the savings can start in the year of introduction of the policy. The time needed to present reliable results depends on the accuracy of the applied data but will generally last several years.

If savings due to policy should be calculated as well, the indicator values should be calculated for several years before the introduction of the policy. If the (statistical) data is normally available, the indicator values before introduction of the policy can be calculated in retrospect. If not, this data has to be gathered some years before the start of the policy.

The planning of evaluation activities also concerns intermediate checks of calculated savings through additional methods, for which a time frame is valid as well; see planning of evaluation in in the in the link here.

3.4 Expertise needed for chosen method

The expertise concerns the availability of (statistical) data and its accuracy, and their appropriateness as to calculating energy savings, i.e., the presence of other factors than savings that influence energy consumption (see sections on normalization and adjustment factors in chapter 4).

3.5 Boundaries for the evaluation

In this guide on non-residential buildings, the energy-intensity indicator method can only be applied to subsectors with a rather uniform building stock. The guide covers energy consumption that is connected to the building, given its specific function (e.g. for hotels including energy consumption for showers, preparing meals, etc.).

This section deals with key methodological choices to be considered when calculating energy savings: consistency between ex-ante and ex-post evaluation, baseline, normalization and adjustment factors. These choices are important to document when reporting energy savings, to ensure the transparency of the results.

General principles of calculating realized savings using different methods can be found for example here and here.

Energy-intensity indicators describe the ratio between energy consumption and the output/production/performance for which energy is used. From the yearly decrease in the calculated ratio, and the level of output/performance, the energy savings can be calculated.

Calculating energy savings with the method energy-indicators only works for subsectors with a sufficiently uniform building stock and with yearly data available on energy consumption and related quantities.

The decrease in the ratio concerns both the effect of the subsidy scheme as well as other factors, like higher energy prices. The effect of introducing a subsidy scheme can be derived from a trend-break in the trend for the indicator value.

4.1 Matching method with earlier ex-ante evaluation

If an ex-ante evaluation has been performed before the start of the subsidy scheme, the choice of the ex-post evaluation method can be matched with that of the ex-ante evaluation. This will depend on the evaluation objectives, timeframe and data available.

From the viewpoint of methodological consistency and data availability, using the same method in the ex-ante evaluation and in this guide on ex-post evaluation might be an obvious choice. However, for an ex-ante evaluation usually no energy-intensity indicator method is applied because this method is based on observed data (see table 4 in this link).
The energy-intensity indicator method can only be applied ex-ante if a detailed scenario study supplies the same data as used in the ex-post evaluation.

The (ex-post) energy-intensity indicator method could be matched with a different method in the ex-ante evaluation. But the generally available ex-ante methods deemed savings or engineering estimate concern the calculation of unitary savings (per saving action) and cannot be matched with the method at hand on total savings, equal to unitary savings times number of actions (see “Different methods ex-ante and ex-post in chapter 7 here).

It is observed that other specific guides using the methods deemed savings or engineering estimates on unitary savings also apply complementary methods on number of actions. In this way, total savings can be calculated which can be compared with the total savings calculated with this guide. In this respect, some other guides on subsidy schemes and non-residential buildings can be matched with this guide.

4.2 Calculation baselines

Energy savings are defined in general as the difference between the actual situation and a reference situation without the saving actions (and without the policy measures that influence these saving actions). The reference situation can be defined using various calculation baselines: Before/after, With/without, Trend, Target/control group and Minimum efficiency standards; see further here.

The baselines With/without, Target/control group and Minimum efficiency standards concern only unitary savings per action (which multiplied by number of actions lead to total savings). The baselines Before/after and Trend are also applicable to calculate directly total savings using the energy-intensity indicator method; see table 1 in this link.

With the Before/After option, the energy-intensity after introduction of the subsidy scheme is compared with that before, that acts as the baseline. The difference between the two is used to calculate the energy savings.

In case the application of the indicator method provides a trend break in the development of the indicator value, the Trend option can be applied as baseline. The (steeper) decreasing trend for the intensity indicator after introduction of the scheme is compared with the gradual intensity improvement before the introduction that acts as baseline.

Depending on the requirements in section 3, preference should be given to one or the other baseline.

See further information on baselines in this document.

4.3 Normalization factors

The calculation, using the baseline(s) considered in the previous section, provides a change in energy consumption that should be corrected for influences on energy consumption other than saving actions. These so-called normalization factors can be weather (with effect on consumption), the rebound effect and changes in energy using activities, such as production (industry), occupation rate (buildings) or car usage (transport).

For this specific guide on energy-intensity indicators in non-residential buildings the normalization factors concern weather and changes in occupation. Weather concerns a correction of energy consumption data for differences in outdoor temperature during the heating season or cooling season. All energy consumption data are corrected for yearly deviations from long term mean values, expressed in heating degree days for heating or cooling degree days for cooling. The changes for occupation take into account occupancy of hotels, but also opening hours of shops or schools and beds use in hospitals.

4.4 Adjustment factors

Adjustment factors define which part of the calculated energy savings can be attributed to a policy measure or meets the definition of savings specified in the evaluation objectives or reporting requirements (see next section on “Calculating Gross and net savings”).

Adjustment factors can concern the Free rider effect, the Spill-over/multiplier effect, Additionality, Non-compliance and Double counting; for more information on adjustment factors see also the link here and here or here.

As the energy-intensity indicator method normally only calculates gross savings (and not the net savings), no adjustment factors have to be applied. The energy-intensity development can incorporate autonomous savings (or technological progress) and price-induced energy efficiency progress. However, the specific guide does not offer a way to correct for this adjustment factors.

In case the policy effect (net savings) can be calculated from a trend break in de development for the indicator value, some adjustment factors might be relevant; see table 1 this link .

The free rider effect concerns the number of realised saving actions that are not due to the subsidy scheme. This is already taken account of through the use of the baseline option trend. The same holds for the spill-over effect. Non-compliance will lead to lower savings than expected and will automatically affect the value of the intensity indicator “after”. Therefore, no correction is needed for this adjustment factor.

Finally, an adjustment may be needed for double counting, in case of another policy focusing on the same subsector as evaluated here. But double counting can only be accounted for at a higher level than individual specific guides (see section on Gross and Net savings).

4.5 Calculating Gross and Net energy savings

Gross savings concern the calculated savings from saving actions using a chosen baseline and normalization factors. Net savings concern the savings attributed to policy measures or eligible for a stakeholder (e.g. an energy company with an obligation to realise savings at their customers).

When calculating savings with a (top-down) method like energy intensity indicators no distinction is made to the unitary savings and number of actions; see for example here, because the methods calculates total savings directly.

The gross savings can be calculated using the appropriate baseline and correct for normalization factors (see section on normalization).

Net savings follow from gross savings by applying adjustment factors. But because generally no adjustment factors are deployed in this guide no separate net savings are calculated. Only in case of applying the baseline option Trend adjustment factors might be relevant, but these are already taken account of in the indicator method with this baseline.

The savings should be corrected for the double counting effect, i.e. the overlap between the savings due to grants and savings due to other policy measures. The overlap in the calculated savings of both policy measures cannot be processed at the level of this guide but must be corrected at the level of savings due to overall policy portfolios. For addressing double counting, see IEA/Evaluating energy efficiency policy measures & DSM programmes publication or this EMEEES report.

See also section 8 on concrete examples.

5.1 Main data requirements, sources and collection techniques

Data requirements specified in the table below correspond to the calculation of energy savings, when using the baseline option before/after. Requirements are more stringent when calculating the savings with the baseline option trend.

Calculation subject	Data requirements	Possible data sources and collection technics
Energy consumption	Fuel and electricity consumption at the level of subsectors with uniform buildings	Energy consumption statistics
Explaining quantities for energy consumption	Turnover subsector, floor area buildings (yearly change)	Production statistics, branch data or building surveys
Normalization factors weather and occupation rate	Yearly deviations for weather conditions (heating degree days and cooling degree days). Occupancy, opening hours, intensity of use of buildings	Weather data, branch data or surveys per subsector or building  (categories)
Primary energy factors	Conversion factor for electricity	Statistics on input and output of power production

Data issues with alternative method

For the data issues concerning the alternative methods (see chapter 6) reference is made to the guides using these methods.

5.2 Energy savings in final terms or in primary terms

Energy savings can be expressed in final terms or in primary terms See definitions about primary and final energy here.

This guide enables the calculation of savings in final terms. Savings in primary terms can also be calculated, provided that savings per subsector are calculated for each energy carrier apart, and primary factors are available to transform the savings in final terms to savings in primary terms.

5.3 Energy savings over time

Implemented saving actions in a year lead to savings over a number of consecutive years. E.g. a more efficient boiler can save gas over its lifetime of about 15 years, insulation over 30 years and more efficient computers up to 5 years. Energy savings can be calculated in different metrics in terms of time reference, for example: year-to-year, annual, cumulated annual, cumulative.

See the definitions here.

The energy intensity indicator method calculates a change in intensity for a number of years that encompasses all saving effects of all saving actions, whether due to the policy evaluated or other influences. Therefore, no results on individual saving actions, with their introduction year and lifetime, are available. Thus, this specific guide cannot deliver cumulative or discounted savings, nor cumulative savings according to the Article 7 in Energy Efficiency Directive.

With the method applied in this guide, only the cumulative yearly savings since a base year are provided, from which yearly savings can be derived from the year-to-year decrease in the intensity value.

6.1 Alternatives for the chosen method

Often other savings calculation methods can be applied as well, although they will all have pros and cons regarding various aspects dealt with in preceding sections.

The table below presents the pros and cons, both for the method at hand and for commonly used alternative methods for the same combination of policy measure and sector.

The method at hand uses a combination of diffusion indicators to calculate number of actions and a complementary method (deemed savings) to calculate unitary savings. The alternative method applies billing analysis and is already part of the 30 specific guides.

Type of method	Pros	Cons
Energy-intensity indicators for buildings	·         Calculation of ratios relatively simple ·         No data needed for individual buildings ·         Some adjustments incorporated in calculated savings	·         Reliable saving results only after some years of evaluation ·         Aggregated results for all buildings ·         Generally, only total savings (gross), not savings due to policy (net)
Billing analysis (alternative method)	·         Data relatively easy available ·         Results available for individual buildings, and per subsector ·         Both gross and net savings can be calculated	·         Billing does not show building connected energy consumption ·         More data needed for net savings

7.1 Calculating avoided CO2 emissions

Avoided CO2 emissions can be evaluated from the energy savings by applying emission factors. Four key aspects are to be taken into account when choosing the emission factor(s):

1. Emission factors vary according to the energy type, so the data about energy savings need to be available per energy type.
2. Emission factors for a given type of energy can vary over time (especially for electricity).
3. Emission factors can take into account:
   1. Direct emission factors: the emissions generated when producing the energy used;
   2. Lifecycle emission factors: all emissions generated from the extraction of the energy resources up to the dismantling of the energy plant.

Due to the differences that the choice of emission factor(s) can induce, it is important to document what emission factor(s) has(have) been used.

For this guide on savings for non-residential buildings, the savings will generally concern both fuel and electricity (and sometimes delivered heat). In order to be able to calculate the direct reduction of CO2-emissions the energy savings should be specified per energy carrier.

The reduction in CO2-emissions due to electricity savings can be calculated by combining these savings with an emission factor for electricity that takes into account the different inputs of power production. The actual factor to be applied can vary, depending on saving action(s) and sector, year of implementation, policy considerations, etcetera (see here).

The avoided emission of other greenhouse gasses due to energy savings are not taken into account here, as these emissions are generally negligible compared to CO2.

IPCC (Intergovernmental Panel on Climate Change) provides a detailed database of peer-reviewed emission factors.

7.2 Calculating cost-effectiveness

Cost-effectiveness is the ratio between costs to achieve energy savings and the amount of savings and possibly other benefits.

A distinction can be made according to the point of view adopted to assess cost-effectiveness:

• Cost-effectiveness for the end-user or participant
• Cost-effectiveness for society at large
• Cost-effectiveness for the party that takes responsibility for saving targets (government or actor with an Energy Efficiency Obligation)

See here.

The calculation of cost-effectiveness for end-users demands, next to the savings, data on investments made, subsidies on investments, interest rates, lifetimes of the saving actions, energy prices (including taxes) per type of end-user and discount factors per type of end-user.

Because this specific guide concerns aggregated savings, without data on specific saving actions, it does not offer the possibility to present cost-effectiveness.

7.3 Calculating other co-benefits

Possible co-benefits from saving energy concern:

• Extra employment
• Reduction of energy poverty
• Other emission reductions (NOx, SO2, fine particles, etc.)
• Better indoor climate
• Reduced dependency on (insecure) energy import

Due to the aggregated nature of the results this specific guide cannot calculate other co-benefits except reduced dependency on imports due to the calculated total savings for the subsectors with non-residential buildings.

7.4 Other aspects of importance

None identified.

Energy efficiency and savings calculation for countries, regions and cities, ISO 17742, chapter 4 on Indicator based savings calculations, 2015

About Energy-intensity indicators

• Top-down evaluation methods of energy savings – Summary report, B. Lapillonne et al, EIE 06 128 EMEEES, WI, March 2009
• Annex to the summary report on top-down evaluation methods: ODYSSEE and ODEX indicators that can be used in top-down evaluation of energy savings, Report EIE_06_128 EMEEES EIE_06_128 EMEEES, Bruno Lapillonne, ENERDATA, March 2009.
• Definition of ODEX indicators in ODYSSEE database-Ademe, 2014