28 – Evaluating energy savings from Energy Efficiency Obligation schemes in the residential sector using deemed savings
This guide can be applied to evaluate the savings due to EEOs (Energy Efficiency Obligation schemes) in the residential sector using the deemed savings method. It includes guidance and explanations specific to this combination of types of policy measure, sector and method, as well as links to general guidance and explanations that can also apply to this combination.
1. USE OF THE GUIDE – AUDIENCE, OBJECTIVES AND FOCUS
The primary audience for this guide is public authorities, obligated parties and other stakeholders involved in EEOs, as well as evaluators looking for guidance on the evaluation process of energy savings in the scope of this guide.
Although the application of the guide will generally concern the national level, account will be taken of issues at EU level when relevant (e.g. the specific format of saving figures for the EED and more particularly its article 7).
This guide is not about the preceding step in the evaluation process, the choice of the method. About this previous step in the evaluation process, see the general guidance about integrating evaluation into the policy cycle. However, after presenting the capabilities and limitations of the guide at hand, the user will be offered alternatives for the method within this guide (see section 6).
The objective of this guide is to provide:
- Information on the scope of the guide that enables the user to decide whether this guide is suited to his/her needs, and whether complementary or additional method(s) could be needed or useful (section 2);
- Guidance about specifying the evaluation objectives and requirements (section 3);
- Guidance about key methodological choices to calculate energy savings (section 4);
- Guidance about the inputs (data requirements) and outputs of the method (energy savings metrics) (section 5);
- Possible alternative methods (with pros and cons) (section 6)
- Background about evaluation results other than energy savings (section 7);
- Relevant examples, case studies and/or good practices (section 8);
- Relevant references for further reading (section 9).
The guide is intended for assessing realised (ex-post) energy savings. However, account is taken of earlier (ex-ante) evaluations of expected savings, if available (see section 4).
The focus of the guide is on impact evaluation, i.e. determining the energy savings, but not on how this has been reached through a step by step process with intermediate results (process evaluation).
Readers looking for the basic and general principles of energy efficiency evaluation may find the following link about general guidance useful.
2. SCOPE OF THE GUIDE – POLICY, SECTOR and METHOD
2.1 About Energy Efficiency Obligation schemes (EEOs)
EEOs are a type of market-based instruments (general category used in the EPATEE Toolbox and Knowledge Base, as well as in the MURE database). Market-based instruments are policies that set targets in terms of outcomes (e.g. energy savings) to be delivered by market actors, without prescribing the delivery mechanisms and types of actions to achieve these targets (IEA, 2017).
EEOs are regulatory mechanisms setting energy savings targets, which can be expressed in energy or CO2 savings that must be achieved by obligated parties (either energy suppliers or energy distributors). EEOs include rules about what types of actions can be eligible, how energy savings shall be calculated, how obligated parties shall demonstrate their role in stimulating actions (materiality issue), if third parties can be involved in the scheme, if approved energy savings can be traded (cf. white certificates schemes), whether penalties in case of non-achievement should be implemented, etc. (Bertoldi et al. 2015).
The main specificity of market-based instruments (here EEOs) is that, within the framework set by the policy, the market actors are free to choose their strategy (flexibility of delivery mechanisms). The underlying assumption is that this approach helps to minimize or optimize the costs of energy savings. In practice, the cost-efficiency of an EEO scheme will depend, among other factors, on the efficiency of its administration, and particularly the way that the energy savings are monitored and verified (Broc, 2017b).
EEOs can also have other goals than least-cost savings, such as alleviating energy poverty or favouring the market penetration of innovative actions, or any actions favoured by authorities for ad hoc reasons.
In terms of evaluation, one specificity of EEOs is that most of the activities of the schemes are implemented by market actors. This can create difficulties in accessing or collecting data, particularly about costs (Rosenow and Bayer, 2017). Public authorities can enforce requirements about what data shall be provided or stored by the obligated parties. A key design issue is thus to find the right balance between these requirements and minimizing the administration costs (for the obligated parties as well as for the public authorities making the controls and verifications).
For more details about EEOs, see for example (ENSPOL 2015 ; RAP 2012).
Further references about evaluations of EEOs can be found in the Knowledge Base, by selecting “market-based instruments” in the search filter “Type of policy instruments”.
2.2 Evaluation for a combination of policy measures
Depending on the national context, there can be overlaps between the EEO and other policy measures. The EEO can include rules to avoid certain overlaps, for example by specifying that actions receiving a public subsidy cannot be eligible to the EEO (e.g. in Austria or Italy), or that only actions with a performance higher than minimum requirements set in current regulations are eligible (rule used for most EEOs).
At the opposite, the EEO can allow actions that received a support or incentive from other policy measures. For example, when the financial barriers are supposed to be too high to be overcome by incentives provided by market actors only (e.g. for renovation works in dwellings, as in France and Ireland).
When an EEO is combined with other policy measures, it is usually difficult to disentangle the relative effects of each policy measure. This goes beyond the scope of this guidance.
Therefore in this guidance, it is assumed that the way to attribute the energy savings to the overlapping policy measures has already been decided. See also “Double counting” in the section on Gross to Net savings.
2.3 Evaluation when combined with energy taxes
The calculated savings effect for EEOs will overlap with that of carbon and/or energy tax(es). Separating the effects of the EEOs and carbon/energy taxes goes beyond the scope of this guidance.
It should be noted that EEOs can have an impact on energy prices, depending on the cost recovery mechanisms, i.e. how the obligated parties can recover the costs they have incurred to meet their energy efficiency targets. When analysing the interactions between EEOs and carbon/energy taxes, a first issue is therefore to assess the impact of the EEOs on energy prices (Giraudet and Finon, 2015). However, in most cases, this impact remains small compared to the share that energy taxes represent in energy prices. Nevertheless, this impact might be high compared to the margin of the energy suppliers, which is important when the supplier is selling forward energy at fixed price and shall anticipate the future impact of EEOs on the energy price.
2.4 About the focus on residential sector
EEOs can be transversal (i.e. actions are eligible in several or all sectors, like for most EEOs in Europe), or restricted to a particular sector (e.g. residential sector, like in UK). This guidance is focused on actions done in the residential sector.
Most EEOs make the difference between action types that can be standardised and thus evaluated with deemed savings (case dealt with in this guidance), and action types that are more specific and require case-by-case energy savings calculations (case dealt with in Specific Guidance 30 ).
Actions done in the residential sector are usually easier to standardise, hence the focus of this guidance. Nevertheless, most of this guidance can also be applied to standardised actions defined for other sectors.
Information on (sub) sectors defined in the Toolbox can be found in the EPATEE terminology, chapter 2, p.17.
2.5 About deemed savings
The Energy Efficiency Directive (2018(2002), Annex V (1.a)) defines “deemed savings” as “results of previous independently monitored energy improvements in similar installations”. They are also often called ex-ante or stipulated savings.
In practice, deemed savings are the result of standardised calculations where most of the parameters are defined beforehand, based on reference values (e.g. national statistics on the building stock) or results from previous studies on the same type of action. This gives a ratio of energy savings per unit of action. The units can be for example the number of actions implemented, the areas of insulation materials installed, etc.
Deemed savings are thus unitary savings. The number of actions (or related units) needs to be obtained by another method (see next section).
Deemed savings are associated to standardised types of actions. This usually includes detailed specifications of the action types to ensure that deemed savings are used to count savings from actions that can be considered similar.
A set of deemed savings can also be defined for the same action type, to take into account variations due to parameters that can easily be monitored. For example, different values for deemed savings of insulation of walls can be defined according to climate zones, building types (individual houses / multifamily buildings), etc. In that case, the obligated parties will have to monitor data about these parameters, in addition to the number of actions.
As the main objective of using deemed savings is to keep monitoring and evaluation requirements as simple as possible, the number of data to monitor is usually limited to a few parameters that can be documented easily (e.g. building type) or is needed to be registered anyway (e.g. location of the building).
However, this does not mean that the calculation made to define the deemed savings is simplistic. Depending on the data available from general references and other studies, and on the means and time available, deemed savings can be defined from simple experts’ estimates up to sophisticated calculations including statistical analysis (for example about the building stock or previous market trends).
Due to the way they are defined, deemed savings do not reflect the energy savings that are achieved for a given situation, but an average result for a typical situation or a large population of similar actions.
In most cases, the various standardised action types and related deemed savings will form a catalogue that market actors can use to have visibility for defining their strategies. For an analysis of catalogues used in some of the European countries, see for example (Labanca and Bertoldi, 2016).
In the US, these catalogues are named Technical Reference Manuals (TRMs). For an analysis of the experience of several states with TRMs, see for example (Del Balso and Grabner, 2013).
Information about the various evaluation methods can be found here, table 1 and 2. This source also covers the combination of the method at hand with other methods, which will be dealt with below.
2.6 Complementary methods to determine total savings
Complementary methods are methods that are required, in addition to the primary selected method (here deemed savings), to calculate total energy savings.
Deemed savings are meant to calculate unitary savings. In case of actions in the residential sector, the unit can be a participant (household), an equipment (e.g. appliance), a dwelling, a m² of area insulated, etc.
These unitary savings then need to be multiplied by the number of actions or participants in order to have the calculated total savings. The number of actions or participants can be obtained in various ways. See this link , table 2 and 3.
In the case of EEOs, the number of actions is usually directly available from the monitoring system, as the obligated parties need to report about their achievements, and particularly the number of actions installed.
This guidance does not deal with monitoring systems. For more details about the links between monitoring and evaluation, see the corresponding EPATEE case study: (Maric et al., 2018).
2.7 Additional methods to increase reliability of the results
An additional method can be applied on top of using deemed savings to improve the reliability of the evaluation results and/or the cost-effectiveness of the evaluation approach.
Deemed savings are the result of a standardised assessment, based on data available from general references (e.g. national statistics) or studies about actions previously implemented. Therefore, they are not reflecting the possible particularities of the actions reported by the obligated parties.
This means that deemed savings do not necessarily reflect the actual energy savings (i.e. as could be experienced by a given participant). Unless further analysis is done, the related uncertainty is unknown.
Depending on the evaluation objectives, other methods can be used to investigate the reliability of the deemed savings (e.g. verifying the relevance of the data or assumptions used) or assess actual energy savings.
Type of method | Short description | Objective |
Survey of participants | Surveys can be used to collect data complementary to the ones collected by the monitoring system (e.g. about characteristics of the households, buildings, etc.)
For more details, see e.g. (Baumgartner, 2017) |
To verify if the average values and assumption used in the deemed savings are representative of the participants or actions reported |
Direct measurement | Direct measurement can be done on a specific parameter (e.g. sensors to monitor duration of use), or on the energy consumed by a given equipment or process (e.g. sub-metering).
For more details, see e.g. (Mort, 2017) |
To verify if the average values and assumption used in the deemed savings reflect actual conditions (e.g. energy performance, duration of use, indoor temperature) |
Billing analysis | Calculation of energy savings from metered data of energy bills.
For more details, see Specific Guidance 29 |
To assess actual energy savings (as can be experienced by the end-users) |
Engineering estimates | Calculation of energy savings, based on engineering model or formula, and using data specific to the cases evaluated (e.g. using of Energy Performance Certificates)
For more details, see Specific Guidance 30 |
To test the robustness of the deemed savings |
As the number of actions monitored for EEOs is usually very large, this could be costly to apply these additional methods to the whole actions reported by obligated parties. Therefore, a common practice is to apply these methods on samples of actions or participants. As sampling can create bias and uncertainties in the results, a particular attention should be paid to limit sampling bias and obtain samples as representative as possible.
Combining different methods can be a cost-effective approach to increase the reliability of the savings figures and to get a better understanding of the impacts of the actions and the EEO.
For possible combinations with an additional method see also chapter 6 in the document here.
3. EVALUATION OBJECTIVES and REQUIREMENTS
3.1 Meeting evaluation goals and ambition
Typical objectives of using deemed savings can be:
- Getting savings estimates on an almost on-going basis (no time-lag);
- Keeping the monitoring and evaluation system simple, and minimizing related running cost;
- Providing visibility to market actors.
At the opposite, deemed savings are not the most appropriate method in case the primary evaluation objective is to assess actual energy savings.
The table shows whether this guide can be used to report on general evaluation goals or criteria. See also (Broc et al., 2009).
General types of evaluation goals or criteria | Level of ambition | Remarks |
Calculation of realized energy savings from saving actions | Low to medium | Depending on the quality of the deemed savings.
In any case, complementing deemed savings with a method to verify actual energy savings is recommended for this objective (see above in section 2: Additional methods to increase reliability of the results) Deemed savings should not be used to estimate savings for only one or a small number of actions. Deemed savings should be used to estimate savings for a large number of actions. |
Calculation of energy savings attributed to the EEO | Low to medium | Additionality criteria or default adjustment factors can be used to make that deemed savings correspond to additional savings.
It is recommended for this objective to complement deemed savings with further ex-post analysis (see below in section 4: Calculating Gross and Net energy savings) |
Cost-effectiveness of saving action (for end-users) | Low | Deemed savings do not necessarily reflect energy savings as experienced by a given end-user. They represent average values. So they should only be used to estimate cost-effectiveness indicators on average.
When the objective is to get more detailed and accurate results about cost-effectiveness from the end-users’ perspective, it is recommended to consider further ex-post analysis and using other methods. |
Cost-effectiveness of the EEO (from a society’s perspective) | Low to medium | Depending on the quality of the deemed savings. In Any case, further ex-post analysis will usually be needed to collect cost data.
(see below in section 7: Calculating cost-effectiveness) Whenever possible, it is also recommended to complement deemed savings with ex-post verifications of savings (see above in section 7: Additional methods to increase reliability of the results) |
CO2-emission reduction from saving actions | Low to medium | The basis will be the estimated energy savings (see comment above about possible limitations).
As deemed savings are defined per action type, this will generally make it easy to apply standard emission factors to calculate CO2 savings from energy savings. (see below in section 7: Calculating avoided CO2 emissions) |
CO2-emission reduction attributed to the policy measure(s) | Low to medium | The basis will be the estimated energy savings (see comment above about possible limitations).
As deemed savings are defined per action type, this will generally make it easy to apply standard emission factors to calculate CO2 savings from energy savings. (see below in section 7: Calculating avoided CO2 emissions) |
3.2 Reporting expectations
The rules of the EEO include the definition of the unit used to count the results to be reported by the obligated parties. This unit includes several criteria for which various options are possible.
Criteria | Common options | Remarks |
Nature of the objective | Energy savings
CO2 savings Bill savings |
Choice mostly depending on the primary objective of the EEO |
Duration for which the results are counted | Annual (or first-year)
Lifetime cumulated Cumulative over the obligation period |
Choice depending on the EEO objectives and on the diversity of actions possible.
A “lifetime cumulated” unit can be chosen to value long-lifetime actions, possibly applying a discount factor. Defining savings lifetimes requires assumptions that can add uncertainties to the deemed savings. For more details see Energy savings over time in section 5. |
Energy basis (if nature = energy savings) | Primary energy savings
Final energy savings |
It is a political choice, depending of the intention of the objectives of the public authorities, and national current practices. The present two biggest EEOS in Europe are for one (French) based on final energy savings, and for the other (Italian) on primary energy savings. It shall be mentioned that to include cogeneration in the scope of the actions, primary energy savings shall be taken into account.
EED article 7 requires to report to the European Commission final energy savings. However it allows countries to count primary energy savings at national level. |
Energy unit (if nature = energy savings) | Multiples of Joule (e.g. PJ), toe (e.g. ktoe) respectively kWh (e.g. TWh) | Choice usually depending on the energy unit commonly used in the country (e.g. for the national energy balance). |
Evaluation perspective | Gross / Additional / Net | For more explanations, see in section 4 Calculating Gross and Net energy savings. |
Deemed savings will then be defined in the unit chosen for the EEO.
It can also be needed to express the results in another unit than the one of the EEO. For example, to compare results with other policies (benchmarking), or to report results in other context (e.g. for the article 7 of the Energy Efficiency Directive).
It is therefore important to keep the documentation of the deemed savings, as well as the distribution of the total energy savings according to the different types of standardised actions. These details will indeed be needed to convert the results from the unit of the EEO to the other unit.
Guidance and examples of template to document deemed savings can be found in the standard ISO 50046:2019.
3.3 Time frame for evaluation
The period under evaluation usually corresponds to the latest period of the EEO scheme, and more specifically to the period(s) for which the target(s) under review has been set.
One of the main advantages of deemed savings is that it enables to assess energy savings as soon as the data about the numbers (or other units) of actions per action type are available from the monitoring system. This is one of the reasons why deemed savings are often chosen when the objective is to evaluate target achievement.
Usually, a periodical (often annual) review of the energy savings results is made to take into account possible cancellations of actions (e.g. due to non-compliance detected through controls), and more generally to make verifications. It can also be done to convert the energy savings results into another energy savings unit when needed for external reporting purposes (e.g. for the annual reports in the context of the Energy Efficiency Directive).
It should be noted however that the definition of a set (or catalogue) of deemed savings requires time, as well as updates. The definition of deemed savings indeed usually includes a consultation process, that can be done for example through an open public consultation (see e.g. Ofgem, 2018) or technical working groups (see e.g. Broc et al., 2012). The time and resources needed to develop (or update) a catalogue of deemed savings depend on the number and diversity of the action types to be included in the catalogue, data availability, level of consensus (or disagreement) about the data and calculation formula, etc.
In practice, deemed savings are included as part of a catalogue specifying standardised actions. The specifications of the standardised actions usually include other criteria. For example, quality requirements, the nature of the proofs to be gathered by the obligated party to report the actions, etc.
The development of a catalogue of deemed savings is thus to be considered when there is enough visibility about the existence of the EEO scheme. The obligation periods defined in the Energy Efficiency Directive (currently 2014-2020, then 2021-2030) help to provide a legislative visibility.
The update of deemed savings can be done on a regular or case-by-case basis. It is usually made to take into account changes in the context that can affect either the definition of the baseline or of the energy efficiency action. For example, update of the regulations setting minimum energy performance requirements, changes in market average, availability of a new product or solution with higher energy performance.
The use of deemed savings can be complemented by other methods to verify their reliability, and possibly update them (see in section 2 Additional methods to increase reliability of the results). These additional methods have different timeframes. It is thus recommended to consider how the combination of evaluation activities should be planned, especially to ensure the feasibility of the corresponding data collection and to optimize the use of resources (time and budget) – see also the EPATEE guidance about how to plan and prepare evaluations).
3.4 Expertise needed for chosen method
The definition of deemed savings requires expertise about the following:
- Sectoral expertise about the energy efficiency actions eligible to the EEO scheme: this is needed to know what action types can be defined in a standardised way (and how), and to select the essential parameters to describe the action types and their energy saving effect.
- Sectoral expertise about data or trends in energy consumption and energy efficiency markets: this is needed to define the baseline.
A very important point for the definition of deemed savings is also the availability of studies and data to provide a sound basis for the savings calculations. For more details about data issues, see section 5.
This guidance considers the use of deemed savings in the residential sector. Depending on the action types eligible to the EEO scheme, the following sectoral expertise can be needed:
- about residential buildings, including building components (building envelope) and HVAC (Heating, Ventilation and Air Conditioning) systems;
- about residential appliances and lighting.
Most of this guidance can also be useful when considering the use of deemed savings in other sectors. In this case, other sectoral expertise can be needed depending on the action types to be covered. When an EEO scheme covers several sectors, the definition of deemed savings can thus be organised through sectoral working groups, or thanks to the support of technical institutes known for their expertise in the corresponding sectors.
4. KEY METHODOLOGICAL CHOICES FOR CALCULATION OF ENERGY SAVINGS
General principles of calculating savings using different methods can be found in (Broc et al., 2009) and in (Eichhammer et al., 2008).
This section deals with key methodological choices to be considered when calculating energy savings: consistency between ex-ante and ex-post evaluation, baseline, normalization and adjustment factors. These choices are important to document when reporting energy savings, to ensure the transparency of the results.
For more guidance about documentation, see for example ISO 50046:2019.
4.1 Matching method with earlier ex-ante evaluation
From the viewpoint of methodological consistency and data availability using the same method in the ex-ante evaluation and in the ex-post evaluation is recommended.
In the case of an EEO scheme, the ex-ante evaluation can take the form of an assessment of the energy savings potential within the scope of the scheme, complemented with assumptions about rates of achievement of this potential. This basis will then be taken into account when setting the target(s) for the EEO scheme. The ex-ante evaluation can also be an impact assessment, especially when such assessment is required due to the legislative process (e.g. when each period and target of the EEO scheme needs to be voted by the Parliament). In this later case, the impact assessment can include several scenarios to compare impacts according to distinct alternatives (e.g. in terms of levels of targets).
Other background elements usually enter into the process of defining the target(s): umbrella objectives (e.g. due to a national Energy Law, to the national targets transposed from the EU Energy Efficiency Directive), economic context (e.g. trends in energy prices), interactions with other policy measures, consultation(s) with stakeholders.
These other elements will mostly be related with the assumptions about rates of achievements. Therefore, the consistency between ex-ante and ex-post evaluations is mostly to be ensured between the assessment of savings potential (or impact assessment) and the ex-post evaluation based on deemed savings.
The same deemed savings can be used for both, the ex-ante and the ex-post evaluation. Respectively to set the target and to verify its achievement. When it is not the case, it is important to use the same principles when defining the baseline(s).
As discussed in section 2 (see Additional methods to increase reliability of the results), additional method(s) can be used to verify ex-post the energy savings. In that case, the results from these ex-post verifications can be used to improve the deemed savings for future ex-ante or ex-post evaluations. When doing so, the representativeness of the results from ex-post verifications should be analysed to see if these results can be used for updating the deemed savings.
It is important here to distinguish ex-post verifications done as part of controls, and ex-post verifications done as part of an evaluation.
Controls are commonly decided on a risk-based approach, in order to focus the means of control where the risks are higher or more critical. Therefore, results from controls are not meant to be representative.
Evaluation methodologies are usually designed to look at the whole scheme, taking into account representativeness, risks of sample bias, etc. Depending on the evaluation objectives, evaluation results can thus be expected to be representative.
For a general discussion about possible combinations of methods applied ex-ante and ex-post, see chapter 7 in this document.
4.2 Calculation baselines
Gross energy savings are defined in general as the difference between the situation including the implementation of energy saving actions and a reference situation without the saving actions. When using deemed savings, this reference situation cannot be defined specifically for each participant. Therefore the reference situation is commonly defined in this case as the stock average (for example, average energy consumption of existing dwellings, average efficiency of boilers in the stock of dwellings).
When evaluating the energy savings from an EEO scheme, the baseline is also usually defined to represent what the situation would have been in the absence of the EEO scheme. This is in particular linked to the concept of additionality, as defined in the amended Energy Efficiency Directive (EU2018 (2002)):
“To determine the savings that can be claimed as additional, Member States shall have regard to how energy use and demand would evolve in the absence of the policy measure in question by taking into account at least the following factors: energy consumption trends, changes in consumer behaviour, technological progress and changes caused by other measures implemented at Union and national level” (Energy Efficiency Directive 2018(2002), Annex V(2) point (a)).
Therefore, this guidance considers here the case where the evaluation objective is to define deemed savings as additional savings.
When using deemed savings, additionality has to be taken into account in the calculation assumptions, as deemed savings are defined before the actions are implemented (to give visibility to stakeholders, see section 3 Meeting evaluation goals and ambition). Two main approaches are thus possible:
- Take into account additionality criteria in the definition of the baseline (option presented below).
- Apply adjustment factors to gross energy savings (option presented later in this section, see Adjustment factors)
Additionality criteria are usually to take into account market trends or effects from other policy measures (particularly regulations setting minimum energy performance requirements). Typical examples of baselines used to calculate deemed savings as additional savings are:
- market average: using statistics on market average is a common way to reflect market trends in the baseline. This option can for example be used for appliances. The baseline is thus defined taking into account the market shares per energy class, enabling to define a market average energy consumption.
- minimum efficiency standards: using as baseline the minimum energy performance requirements set in current regulations is a common way to ensure that the deemed savings are additional to these regulations (thereby avoiding double counting).
It shall be noted that when reporting energy savings under the EED article 7, the Annex V of the EED requires that the savings calculations shall take into account the minimum efficiency standards or requirements set in other EU laws or regulations (e.g. minimum standards set in EcoDesign regulations, EU emission performance standards for new passenger cars and new light commercial vehicles).
In some cases, the baseline for additional savings can be the same as for gross savings, i.e. equivalent to a before/after comparison. When defining deemed savings, this case usually means using the average consumption in the stock as explained above. This option is usually chosen:
- when it is considers that the current rate of action is very low (e.g. insulation of solid walls), thus assuming that all corresponding actions can be considered additional;
- when it is considered that the actions have been done earlier than would have happened in the absence of the EEO scheme (e.g. early replacement of appliances).
The case of current low rate of actions is for example considered for the EED article 7 when dealing with actions about renovating the building envelope of existing dwellings. The average energy consumption in the stock of dwellings (usually defined per main types of dwellings) can then be used as baseline for these actions. At the opposite, EED article 7 implicitly assumes that boilers are mostly replaced at their end of lifetime. Therefore, the baseline to use for boilers under EED article 7 is at least the minimum efficiency as set in the EcoDesign regulations.
In case actions can be demonstrated to be early replacements, if the deemed savings cumulate savings over several years, the baseline needs to reflect two sub-periods in the energy savings calculation, for example as represented in the figure below.
A simplified approach to take into account “early replacement” can be to calculate the baseline as a weighted average of the stock and market average (or stock and minimum energy performance requirements). The weighting factors are then defined to reflect the respective length of the two sub-periods. These durations can for example be defined based on previous surveys or market analysis.
For a general discussion about calculation baselines, see the link to this document.
4.3 Normalization factors
The calculation of deemed savings is usually made based on normalized conditions of use. For action types related to space heating, this means that the calculation is made taking into account normalized weather conditions (e.g. reference Heating Degree Days as defined in building regulations).
Deemed savings can be the result of a simplified engineering calculations, such as the ones used for Energy Performance Certificates. In that case, the normalized weather conditions are an input data of the calculation (even most commonly directly integrated in the calculation model). When using this approach, another input of the calculation will be normalized assumptions about heating behaviours (e.g. indoor temperature per type of rooms, night and absence mode). This means that the deemed savings will be based on conventional energy consumption.
In that case, it is recommended to consider if it would be relevant to apply factors to take into account prebound and rebound effects:
- prebound effect: lower baseline energy consumption compared to the estimated conventional energy consumption. This can occur for example in very inefficient dwellings, for which a normalized use of space heating would not be affordable for the occupants. This can also occur when the statistics about the building stock do not take into account the fact that a share of the dwellings can have been renovated since their construction.
- rebound effect: higher energy consumption than the estimated conventional energy consumption for the situation after implementing the energy saving action. This can occur for example when the energy efficiency improvement makes affordable for the occupants to increase the indoor temperature, heat more rooms, or heat for longer periods.
Factors to take into account prebound effect can for example be derived from comparisons between conventional energy consumption from Energy Performance Certificates and metered energy consumption. Prebound effect can also be taken into account in the calibration of the calculation model used to define the deemed savings.
Factors to take into account rebound effect can for example be derived from previous surveys, measurement campaigns or the literature.
For more details about prebound and rebound effects, see for example the topical case study about the comparison between estimated and measured energy savings (Sipma et al., 2019), or the webinar #4 of EPATEE.
About the energy consumption for the situation with the energy saving action, another factor might need to be taken into account: performance gap.
Performance gaps correspond to cases where the observed energy performance of the energy saving action installed is lower than the expected energy performance. For example about insulation actions, because the insulation materials have not been installed properly (e.g. leaving thermal bridges) or the insulation materials have lower performance than claimed. About condensing boilers, an example can be when settings and conditions of use (e.g. types of radiators and related temperature for hot water) do not enable the boiler to condense and recover the heat from steam water in flue gas).
Enforcing quality requirements can help to minimize the risks of performance gaps.
Factors to take into account performance gaps can for example be derived from previous studies on samples of actions.
An example of use of correction factors to take into account rebound effect and performance gaps can be found in the case study about the UK Supplier Obligation (Rosenow, 2017) (see in this case study the explanations about the “in-use factor”). See also (BRE, 2018).
Deemed savings can also be based on statistics of metered energy consumption. In that case, time series long enough (e.g. 10 years) are used to ensure that the data reflect usual weather conditions. When only short time series are available, then the weather conditions of the years included in the time series can be compared with normalized weather conditions (e.g. from building regulations). When needed, weather corrections can then be applied (e.g. using a weather factors based on Heating Degree Days, or making regression analysis).
If the deemed savings are based on statistics of metered energy consumption, then possible prebound or rebound effects or performance gaps are directly included in the metered data (no need for further corrections).
4.4 Adjustment factors
Adjustment factors define which part of the calculated energy savings can be attributed to a policy measure or meets the definition of savings specified in the evaluation objectives or reporting requirements.
For a general introduction about adjustment factors, see table 1 here.
This guidance considers the case where the evaluation objective is to define deemed savings as additional savings (for a discussion about this, see below Calculating Gross and net savings)
When using deemed savings, additionality has to be taken into account in the calculation assumptions, as deemed savings are defined before the actions are implemented (to give visibility to stakeholders, see section 3 Meeting evaluation goals and ambition). Two main approaches are thus possible:
- Take into account additionality criteria in the definition of the baseline (option presented above in Calculation baselines).
- Apply adjustment factors to gross energy savings (option presented here below).
Adjustment factors can concern free rider effect and spill-over/multiplier effect. For related definitions, see the EPATEE terminology. Free-rider and spill-over effects can also be encompassed in a single additionality factor.
In any case, when using deemed savings, these effects can only be taken into account based on previous studies or surveys used to define corresponding adjustment factors. For an example about this, see the case study about Danish EEO scheme (Broc, 2017a).
It should be noted that in most of the available experience (especially in Europe), adjustment factors have been defined to take into account free-rider effect only. Spill-over effects have indeed proven to be more difficult to assess quantitatively (see also the experience from the Danish EEO scheme).
In case of other policy measures target the same saving action types as the EEO scheme (either partial or full overlap), the evaluation of the energy savings should also consider how double counting is avoided or corrected. Methods to tackle double counting or interaction between policy measures go beyond the scope of this guidance.
4.5 Calculating Gross and Net energy savings
Gross energy savings are energy savings calculated from the point of view of the final consumers, i.e. independently of whether the participants to the policy measure would have acted the same or differently in the absence of the policy measure.
Usually the baseline energy consumption used to calculate gross energy savings is the energy consumption before the energy saving actions were installed or implemented. When using deemed savings, data specific to each participant is often not available (to keep the monitoring simple). Therefore, the baseline energy consumption for deemed savings can be defined as the average energy consumption in the stock of building or equipment.
The calculation of gross energy savings can include the use of normalization factors to ensure that the baseline energy consumption and the energy consumption with the energy saving action are comparable. For the residential sector, this mainly concerns weather conditions (see Normalization factors above).
Depending on the data and approach used to calculate gross savings, it can be relevant to use also correction factors for prebound and rebound effects, as well as performance gaps (see Normalization factors above). These effects indeed affect energy savings from the point of view of the final consumers.
Net energy savings are calculated from the point of view of the public authority, policymaker or other stakeholder that provides any type of support or incentive to promote the energy saving actions. Therefore, this calculation takes into account effects related to the causality or attribution of the actions or energy savings to the policy measure or interventions of the stakeholders.
When evaluating an EEO scheme, it is common in Europe to speak about additional savings instead of net savings, due to the terminology of the EU Energy Efficiency Directive. More generally, the term “additional savings” is also used because the objective of the EEO schemes is usually to achieve energy savings either additional to a business-as-usual scenario, or additional to the effects of other policy measures (particularly regulations).
When using deemed savings, additionality has to be taken into account in the calculation assumptions, as deemed savings are defined before the actions are implemented (to give visibility to stakeholders, see section 3 Meeting evaluation goals and ambition). Two main approaches are thus possible:
- Take into account additionality criteria in the definition of the baseline (option presented above in Calculation baselines).
- Apply adjustment factors to gross energy savings (option also presented in Adjustment factors).
Additional methods can then be used to assess net or additional savings ex-post (e.g. at the end of an obligation period). Explanations and guidance about methods for this type of ex-post evaluation can be found in the topical case study about the evaluation of net energy savings (Voswinkel et al., 2018).
Additional or net savings should also be corrected for double counting, in case of possible overlap between the EEO scheme and other policy measures. The overlap in the calculated savings should be analysed at the level of the overall policy portfolio or sector. For addressing double counting see (Vreuls, 2005) or (Broc et al., 2009).
See also sections 8 about concrete examples and 9 about further reading.
5. INPUT AND OUTPUT
5.1 Main data requirements and data sources and collection technics
When defining deemed savings, a prerequisite is that reviews, studies, national statistics or any other sources of reliable data about the energy savings impact of the considered energy saving actions are available. These data sources provide the basis needed to elaborate deemed savings. The sounder the basis, the better the quality of the deemed savings.
Data requirements specified in the two tables below correspond to the calculation of gross energy savings, when using the baseline option [before/after], i.e. a baseline representing the situation before energy saving actions are implemented. The case about using baseline options to calculate additional energy savings is discussed later on (see Data issues when evaluating net energy savings).
The first table deals with the case where deemed savings are the result of simplified engineering calculations that estimate “before” and “after” energy consumption through intermediate parameters. This approach is commonly used for actions on building envelope or HVAC (Heating, Ventilation and Air Conditioning) systems.
Calculation subject | Data requirements | Possible data sources and collection technics |
Definition of segments or building types to represent the dwellings stock | Analysis of the main different segments or building types that need to be distinguished when defining the standardised actions and deemed savings | National statistics about the dwellings stock (e.g. national housing survey, national database of Energy Performance Certificates).
History of main construction periods (taking into account main trends in construction practices, and successive building regulations) For more details, see for example: |
Energy performance of the building envelope (heat transfer coefficient: U-values in W/m².K; or thermal resistance: R-values in m².K/W) | Average values representative of the different segments / building types. | National statistics about the dwellings stock (e.g. national housing survey, national database of Energy Performance Certificates).
Minimum energy performance requirements set in the successive building regulations. |
Definition of main types to represent the stock of heating systems | Analysis of the shares of the dwelling stocks per type of energy used for space heating.
Analysis of the main type(s) of heating systems per type of energy. |
National statistics about the dwellings stock (e.g. national housing survey, national database of Energy Performance Certificates).
National statistics on energy consumption in the residential sector (e.g. number of customers per energy supplier) |
Efficiency of heating systems (per main types of heating systems) | Average values representative of the main types of heating systems (per energy type, and if possible per building type). | National statistics about the dwellings stock (e.g. national housing survey, national database of Energy Performance Certificates).
Dedicated surveys or stock modelling. |
Normalization factor for weather condition | Normalized Heating Degree Days (possibly differentiated per climate zone, when needed) | Heating Degree Days specified in building regulations.
Data from the national meteorological office or weather service. |
Energy performance of the actions installed (U-values or R-values for insulation materials, efficiency for heating systems) | Possibility to define deemed savings for different levels of energy performance achieved | Minimum energy performance requirements set for the EEO scheme (eligibility criteria per action type).
Average energy performance per class of performance (when deemed savings defined per class of performance). |
Correction factors (e.g. to take into account prebound and rebound effects, performance gaps) | To be considered if the calculation model could not be calibrated based on statistics of metered energy consumption. | Available studies on these effects.
Data from the literature. Conservative assumptions. |
The second table deals with the case where deemed savings are directly based on data of energy consumption (without the use of intermediate parameters). This approach is commonly used for actions on appliances.
Calculation subject | Data requirements | Possible data sources and collection technics |
Baseline energy consumption | Average energy consumption representative of the stock | Previous studies (e.g. measurement campaigns, stock modelling)
National statistics of energy consumption per end-use |
Energy consumption with the energy saving action | Possibility to define deemed savings for different levels of energy performance or energy class | Minimum energy performance requirements set for the EEO scheme (eligibility criteria per action type).
Energy consumption per class of performance* (when deemed savings defined per class of performance). |
Factor for performance gaps | Can be needed when the data of energy consumption is based on laboratory tests | Previous studies (e.g. measurement campaigns)
Literature |
Normalization factor for weather condition (for actions related to space heating) | Can be needed when the energy consumption is not based on metered or measured data, or that the average consumption is defined with a short time series | Heating Degree Days specified in building regulations.
Data from the national meteorological office or weather service. |
*: an energy class corresponds to a range of energy consumption. In practice, the distribution of energy consumption within an energy class can be asymmetric (for example, most of the products can tend to have energy consumption close to the maximum in the range). Therefore, the average of the range is not necessarily representative. In case no data is available, a conservative assumption can be made to use the maximum energy consumption of each range to represent the different energy classes.
Deemed savings are defined for a unit of action (see in section 2 About deemed savings). Therefore, a complementary source of data is needed to obtain the number of actions (or other units of actions): see in section 2 Complementary methods to determine total savings.
Data issues when evaluating net energy savings
When evaluating energy savings from EEO schemes with deemed savings, it is more common in Europe to speak of additional energy savings than net energy savings (see in section 4: Calculating Gross and Net energy savings).
When using deemed savings, additionality has to be taken into account in the calculation assumptions, as deemed savings are defined before the actions are implemented (to give visibility to stakeholders, see section 3 Meeting evaluation goals and ambition). Two main approaches are thus possible:
- Take into account additionality criteria in the definition of the baseline (option presented above in Calculation baselines).
- Apply adjustment factors to gross energy savings (option also presented in Adjustment factors).
The first option will lead to use a baseline different from the “before/after” comparison (for more details, see in section 4 Calculation baselines). This means the following data requirements:
- for the baseline option “market average”: data about trends or recent market shares per energy class (or similar categories reflecting energy performance levels) to calculate the market average for energy consumption or energy performance characteristics (in case energy consumption is calculated with intermediate parameters).
- for the baseline option “minimum efficiency standards”: data about the current minimum energy performance requirements set in national or European regulations, per action type (and information about the scheduled updates of these regulations).
It is then useful to clarify the planning for updates or revisions of the baseline values, to provide stakeholders with visibility.
The baseline option for the second option is “before/after” comparison (see above). The data needed for the deemed savings to correspond to net or additional savings is then data to define the adjustment factors. Such data can be obtained from previous studies (e.g. previous surveys or market analysis) or the literature. As far as possible, it is recommended to use data from previous studies on the same policy measure.
Literature indeed shows that values for adjustment factors can vary significantly from one policy measure to another, one country from another, etc.
Complementary ex-post evaluations can be used to investigate and update or revise the adjustment factors. This is for example discussed in the Specific Guidance 8 about billing analysis.
For more details about the evaluation of net energy savings, see the dedicated topical case study (Voswinkel et al., 2018).
5.2 Energy savings in final terms or in primary terms
Energy savings can be expressed in final terms or in primary terms See definitions about primary and final energy in the EPATEE terminology.
Usually deemed savings are first calculated in final terms. One advantage of choosing to calculate in final energy is that it enables to compare or define deemed savings (or estimates of energy consumption) with statistics of metered energy consumption (mostly energy bills or laboratory tests).
Deemed savings can also be calculated in primary terms (as in the Italian white certificates scheme), provided that deemed savings are calculated for each energy carrier apart, and primary factors are available to convert the savings in final terms to savings in primary terms.
Deemed savings can also be calculated as monetary savings, i.e. savings on energy bills (see in section 3 Reporting expectations). This requires defining average energy prices per energy carrier, as well as a scenario of energy prices if the savings are calculated in cumulative terms, either over the obligation period or over the action lifetime. In this case, a discount rate can also be applied to the calculation (see below).
For consistency, the metrics should be the same for setting the target(s) and counting the savings.
The energy savings results can then be expressed in other metrics for other purposes (e.g. reporting in the context of the EED article 7), provided that the data needed to convert from one metric to the other is available. It is thus important to identify the needs to express the results in different metrics, so that data used in the energy savings calculations are documented enough to enable future conversions.
5.3 Energy savings over time
Saving actions installed in a year lead to savings over a number of consecutive years, depending on the savings lifetime. E.g. a more efficient boiler can save gas over its lifetime of about 15 years, insulation over up to 60 years and more efficient computers up to 5 years. For savings from behavioural changes due to a media campaign the life time might be not much longer than that of the campaign. Energy savings can be calculated in different metrics in terms of time reference, for example: year-to-year, annual, cumulated annual, cumulative. See the definitions in the EPATEE terminology.
The EEO scheme can count first-year savings only up to lifetime-cumulated savings (i.e. savings over the lifetime of the energy saving action).
For consistency, the metrics should be the same for setting the target(s) and counting the savings.
The energy savings results can then be expressed in other metrics for other purposes (e.g. reporting in the context of the EED article 7), provided that the data needed to convert from one metric to the other is available. It is thus important to identify the needs to express the results in different metrics, so that data used in the energy savings calculations are documented enough to enable future conversions.
If only first-year savings are counted and needed for reporting, then no further data is needed, apart from monitoring when the actions are installed (information needed anyway for the monitoring of the EEO scheme).
In other cases, the data about when the actions are installed will need to be complemented with data about the estimated lifetime of the savings, or the periods over which savings can be counted (crediting durations in the context of the EEO scheme).
The rules of the EEO scheme can indeed use different approaches, for example:
- Define standard lifetimes per action time (or general categories of actions) and cumulate directly the savings over these lifetimes (e.g. case of the UK EEO scheme)
- Define standard crediting durations, per general categories of action, and then credit savings each year over the corresponding number of years (e.g. case of the Italian white certificates scheme)
- Define special factors applied to first-year savings to give a premium to long-lifetime actions (factor > 1) and decrease the savings credits given to short-lifetime actions (factor < 1) (e.g. case of the Danish EEO scheme)
Whatever the approach used, this will also require that the monitoring system keeps track of the action types or categories. Especially if there is a need to convert the results in other metrics for other reporting purposes.
Examples of lifetime values can be found in the following sources:
- CWA 15693:2007. Saving lifetimes of Energy Efficiency Improvement Measures in bottom-up calculations. CEN Workshop Agreement, April 2007.
- EN 15459:2017. Energy performance of buildings — Economic evaluation procedure for energy systems in buildings. CEN standard, June 2017. (See annex D).
- Ecodesign Impact Accounting – Status Report 2017. Prepared by VHK for the European Commission December 2017. (see annex A, pp.73-76).
On top of cumulating savings over savings lifetime or crediting durations, the rules of the EEO scheme can also include the application of a discount factor. This discount factor can be used:
- For economic reasons: for example when the lifetime-cumulated energy savings are credited at once when the action is installed, and that then the energy savings credits can be traded as a commodity.
- For technical reasons: for example, to take into account that energy savings can decrease over time (e.g. for behavioural actions).
When using an economic discount factor, it is usually the same value for all action types. Because once the energy savings are credited, they can be traded independently of the action types that produced these energy savings. The discount factor used for the EEO scheme is often defined from discount factors commonly used by economic agents, for example discount factors used by public authorities.
When using technical discount factors, they usually need to be differentiated per action type, as the changes in energy savings over time depend on the action type. Overall, there is limited evidence about decrease (or increase) of energy savings over time (for more details about this issue, see for example Hoffman et al., 2015).
A general default discount factor (i.e. uniform for all action types) can also be decided to reflect the risks related to the investment in the energy saving action, as perceived by the final customers or investors.
6. ALTERNATIVE FOR CHOSEN METHOD
6.1 Alternatives to deemed savings
Deemed savings are often chosen when the objective is to provide stakeholders with visibility and to evaluate results from the monitoring system, without delay (see in section 3 on Meeting evaluation goals and ambition).
Deemed savings are appropriate to evaluate savings from action types that can be described in a standardised way. They are then a cost-effective way to assess savings from large numbers of similar actions (for each standardised action type). They are less relevant for energy saving projects that would be very specific, thereby requiring case-by-case calculations.
This is why EEO schemes often make use of different types of calculation methods, depending on the type of energy saving action or project. A common approach is to use deemed savings for the residential sector, and some “standardisable” action types (e.g. lighting) in the other sectors for which engineering calculations are used in case of large or specific energy saving projects.
Direct measurements can also be required, especially for large projects in the industry (or large commercial buildings) (see e.g. the option of Monitoring and Measurement Plans in the Italian white certificates scheme).
In parallel, billing analysis (or other types of econometric analysis) is a common alternative when the evaluation objective is more specifically to verify the energy savings actually achieved or to assess the cost-effectiveness or efficiency of the scheme.
The table below presents the pros and cons of these methods commonly used for evaluating energy savings from EEO scheme (see also in section 2 on Additional methods to increase reliability of the results).
Type of method | Pros | Cons |
Deemed savings | · Provide visibility to stakeholders
· No delay in getting results from the monitoring system · Low running cost (once the catalogue is defined) · Calculations directly related to the energy efficiency improvements due to the energy saving actions |
· Use limited to action types that can be described in a standardised way
· Do not reflect the energy savings achieved for a given situation, but an average result for a population of actions · Can require significant preliminary efforts (if many action types to be included in the catalogue) · Quality depending on the data available to define deemed savings · Possible gaps between deemed savings and actual savings (see section 4) · Additional method needed to evaluate ex-post the additionality of the savings (see section 4) |
Engineering calculations
(see also the related Specific Guidance 30) |
· Can be used for almost all action types
· Can enable to automatize energy savings calculations (through standardised formula for simple cases) · Can reflect the energy savings achieved for a given situation (specific calculations) · Limited delay in getting the results (calculations can be done before the actions are installed) |
· Require to collect data for each case (so can be costly if data collected only for this purpose and for large numbers of actions / projects)
· Possible gaps between engineering estimates and measured savings (see the corresponding topical case study (Sipma et al., 2019)) · Additional method needed to evaluate ex-post the additionality of the savings (see section 4) |
Type of method | Pros | Cons |
Direct measurements | · Provide data about actual energy consumption (for the baseline and/or for the situation with energy saving actions) or about actual values for key parameters (e.g. power, duration of use)
· Can be used to assess performance gaps |
· Can be costly if measurements only done for this purpose and for large numbers of actions
· If sampling is used, attention should be paid to avoid sampling bias (if data are to be extrapolated) · Additional method needed to evaluate ex-post the additionality of the savings (see section 4) · Delay in installing the actions (if used to verify the baseline, then time needed to make the measurements, unless data are already available) · Delay in getting the results (if used to verify the situation with energy saving actions, then time needed to make measurements after the actions are installed + time to analyse the data) |
Billing analysis
(see also the related Specific Guidance 8) |
· Provide data about actual energy consumption / energy savings
· Can be used to evaluate ex-post net or additional savings (if a control or comparison group can be found) |
· Can only be used for ex-post evaluation
· Frequent difficulties to collect billing data (unless data collection carefully planned and prepared in advance, e.g. collecting participants’ approval when actions are installed) · Difficulties to get representative samples (cf. sampling bias + data losses along the evaluation process) · Delays in getting the result (at least one year to get the consumption after installing actions + time to process and analyse data) · Difficulties to find relevant control or comparison groups (when assessing net or additional savings) |
Econometric method (top-down approach)
(see also the related Specific Guidance 29) |
· Based on data of actual energy consumption (at macro level)
· Can capture rebound effects (including indirect rebound effects) · Can capture market transformation effects (if market data are available) |
· Requires long time series that can be difficult to build in a consistent way for all variables of interest
· Results can be statistically significant only if the impact of the EEOS is large enough compared to the statistical noise (i.e. yearly random variations in energy consumption of each sector or sub-sector) · Due to the above, disaggregated data (per sub-sector) are most often required, and not always easy to collect |
7. ADDITIONAL EVALUATION RESULTS
7.1 Calculating avoided CO2 emissions
Depending on the priority objectives of the EEO scheme, the deemed savings can sometimes be expressed in CO2 savings (i.e. avoided CO2 emissions) (e.g. case of the UK EEO scheme from 2008 to 2018). In practice, deemed savings are first calculated in terms of energy savings. Then avoided CO2 emissions can be evaluated from the energy savings by applying emission factors. Four key aspects are to be taken into account when choosing the emission factor(s):
- Emission factors vary according to the energy type, so the deemed savings need to be defined per energy type.
- Emission factors for a given type of energy can vary over time (especially for electricity).
- Emission factors can take into account:
- Direct emission factors: that take into account the emissions generated when producing the energy used;
- Lifecycle emission factors: that take into account all the emissions generated from the extraction of the energy resources up to the dismantling of the energy plant.
Due to the differences that the choice of emission factor(s) can induce, it is important to document what emission factor(s) has (have) been used. Emission factors used for the EEO scheme can for example be based on official national emission factors used for the national inventory of emissions of greenhouse gases. The conversion of electricity savings into CO2 savings is however a special case, depending on the national mix for electricity production. Several choices are indeed possible, for example:
- Average emission factor, calculated from the total annual emissions from electricity production (possibly taking into account national imports and exports) divided by the annual amount of electricity consumed: this is a simple approach, but that might not reflect the fact that end-uses can have different times of use and thus correspond to different load profiles (while the emission factor for electricity can vary significantly between base load and peak load).
- Emission factors per type of end-use (also called marginal emission factors): this requires more sophisticated calculations (e.g. by decomposing the national load curves per type of end-use) that will be meant to use emission factors reflecting the differences in time of use (e.g. daily, seasonally).
The choice between the two options above will depend on the national electricity mix (cf. emission factor varying significantly with time of production or not) and the type of end-uses covered by the EEO scheme (and especially if savings related to electric heating can represent a significant share of the expected savings).
If the deemed savings cumulate savings over time, it can also be needed to define a scenario about the evolution of the national electricity mix over the period of calculation (e.g. taking into account the objectives of shares of electricity produced from renewable energy sources).
The avoided emission of other greenhouse gasses due to energy savings are not taken into account here, as these emissions (and more specifically their reductions) are generally negligible compared to CO2 for actions in the residential sector.
When needed, IPCC (Intergovernmental Panel on Climate Change) provides a detailed database of peer-reviewed emission factors.
7.2 Calculating cost-effectiveness
Cost-effectiveness is the ratio between costs to achieve energy savings and the amount of savings and possibly other benefits.
A distinction can be made according to the point of view adopted to assess cost-effectiveness:
- Cost-effectiveness for the end-user or participant (e.g. payback time)
- Cost-effectiveness from the obligated parties’ point of view (e.g. least cost of target achievement)
- Cost-effectiveness for society at large (e.g. social net present value)
- Cost-effectiveness from the point of view of the public authority (e.g. comparing different types of policy measures)
For more details about the different perspectives, see for example (Breitschopf et al., 2018).
In the case of an EEO scheme, the calculation of cost-effectiveness requires to collect other data on top of the ones used to calculate energy savings, as summarized in the table below. The costs and benefits listed in this table apply when considering the point of view of categories (i.e. all participants together, all obligated parties together, etc.). When making a cost-benefit analysis from an individual point of view (i.e. one participant, one obligated party), the selection of costs and benefits can be different according to the own view and practice of this participant or obligated party.
Point of view | Costs | Benefits |
Participants | · Part of the investments paid by the participants | · Financial aids received from obligated parties (or other intermediaries)
· Gross energy savings |
Obligated parties | · Costs to achieve their targets*
· Losses in revenues (due to the additional energy savings) |
· Costs recovered on network tariffs (if energy distributors) or energy prices (if energy suppliers)
· Costs of energy production (or distribution or purchase) avoided due to the additional energy savings |
Public authorities | · Administration costs
· Losses in tax revenues (due to additional energy savings) |
· Increases in tax revenues (due to additional investments made in energy efficiency actions) |
Society | · Part of the investments paid by the participants (for additional actions only)
· Costs of the obligated parties · Administration costs for the public authorities |
· Additional energy savings |
*: when possible, these costs can be disaggregated in sub-categories, especially to differentiate administration costs (e.g. costs of reporting to the public authority), communication & marketing costs (e.g. communication campaigns) and costs related to the technical or financial support provided to final customers (e.g. energy audits, grants). It can also be useful to identify costs related to quality processes and monitoring.
NOTE: the table above does not deal with non-energy impacts. Depending on the context and objectives of the EEO scheme, non-energy benefits can be larger than the benefits from energy savings. When assessing the cost-effectiveness of an EEO scheme from the society’s point of view, it is therefore recommended to consider if it is relevant to include non-energy impacts in the scope of analysis.
Experience has shown that the fact that EEO scheme involves private actors makes it often difficult to collect cost data, and particularly homogeneous cost data. When obligated parties report cost data, it is indeed common that they use different sub-categories of costs, or only their “total” costs (with a different scope of “total” costs from one obligated party to the other).
When possible, it can therefore be useful to consult with obligated parties to define clear categories of costs.
When the obligated parties are energy distributors, they usually have to report their total costs to the energy regulatory authority, so that they are allowed to recover their costs on network tariffs. This provides information about total costs for obligated parties. But it does not mean that costs per sub-categories will be available.
When the obligated parties are energy suppliers, there is usually no direct obligation for them to report cost data. This can be asked by the public authorities on a voluntary basis. Or it can be required by including a specific provision in the law enforcing the scheme (if enforced by law). However, there could be limitations due to other legal provisions about confidentiality of strategic data for private actors.
Depending on the indicator(s) used to assess cost-effectiveness, it can be needed to use discount factors (e.g. when the indicator is Net Present Values). In that case, it is important to document the use of discount factors, and if possible to make a sensitivity analysis (testing several values or ranges of discount factors). As this can affect significantly the results.
Likewise, the calculations of cost-effectiveness indicators will usually require to consider scenarios of energy prices over given periods. The assumptions about trends in energy prices should be documented. Whenever possible, it is recommended to make a sensitivity analysis (testing several scenarios of energy prices).
For more discussions about cost-benefit analysis of EEO schemes in Europe, see for example (Rosenow and Bayer 2017).
7.3 Calculating other Co-benefits
Co-benefits from saving energy can be for example:
- Extra employment
- Reduction of energy poverty
- Other emission reductions (NOx, SO2, fine particles, etc.)
- Better indoor climate
- Reduced dependency on (insecure) energy import
It should be noted that the impacts from EEO schemes on each of these aspects are usually positive, but can also be negative (especially on energy poverty, see below). Therefore, it is in general more appropriate to speak about non-energy impacts.
For a general background about non-energy impacts, see the corresponding EPATEE general guidance.
For the case of EEO scheme in the residential sector, a special attention is frequently paid to the impacts on energy poverty. This is indeed now required by the amended Energy Efficiency Directive (EU2018 (2002)).
Indeed for most EEO schemes, the costs incurred by the obligated parties will be recovered, partly or fully, either on network tariffs (if energy distributors) or energy prices (if energy suppliers). So at the end, EEO schemes can have an impact on energy prices, thereby increasing the risks of energy poverty.
However, the impact of an EEO scheme on energy prices should be compared to existing levels of taxations on energy prices, as well as to current trends in wholesale energy prices. If the impact of the EEO scheme is small compared to these, therefore it also means that its possible negative impact on energy poverty is also limited.
At the opposite, some EEO schemes include specific objectives or provisions to tackle energy poverty. For example, this is the priority objective and single scope of the new period of the UK EEO scheme (since December 2018).
In case of a transversal EEO scheme (covering several sectors), the evaluation objectives can also include assessing the possible cross-sectoral effects in terms of cross-subsidizing. Such effects can for example happen when it is more cost-effective for obligated parties to achieve their targets in a given sector, whereas costs will be recovered on energy prices in all sectors.
8. CONCRETE EXAMPLES
3 EPATEE case studies deal with the evaluation of EEO schemes. Each of this case includes the use of deemed savings (mostly combined with other methods):
- Evaluation of the Danish EEO scheme:
Broc, J.S. (2017a). Energy Companies’ Energy-Saving Efforts in Denmark. Case study prepared by IEECP for the EPATEE project, funded by the European Union’s Horizon 2020 programme. - Evaluation of the Italian white certificates scheme:
Di Santo, D. & Biele, E. (2017). The Italian white certificates scheme. Case study prepared by FIRE for the EPATEE project, funded by the European Union’s Horizon 2020 programme. - Evaluation of the UK Energy Supplier Obligation:
Rosenow, J. (2017). Supplier Obligations in Great Britain. Case study prepared by IEECP for the EPATEE project, funded by the European Union’s Horizon 2020 programme.
All EPATEE case studies are available at: https://www.epatee-toolbox.eu/knowledge-base/
For a discussion about the use of deemed savings in the US, see for example:
- Del Balso, R.J, & Grabner, K. (2013). But I Thought a State-wide TRM Would Solve Everything? Proceedings of IEPEC 2013
- Baumgartner, R. (2017). Chapter 12: Survey Design and Implementation for Estimating Gross Savings Cross-Cutting Protocol. The Uniform Methods Project: Methods for Determining Energy-Efficiency Savings for Specific Measures. Prepared for NREL (National Renewable Energy Laboratory), September 2017. https://www.energy.gov/eere/about-us/ump-protocols
- Breitschopf, B., Schlomann, B., & Voswinkel, F. (2018). Identifying current knowledge, suggestions and conclusions from the literature. Report of task 3.1 of the EPATEE project.
- Broc, J.S., Bourges, B., Adnot, J., Thomas, S. & Vreuls, H. (2009). The development process for harmonized bottom-up evaluation methods of energy savings. Deliverable 4 of the EMEEES project.
- Eichhammer, W., Boonekamp, P., Labanca, N., Schlomann, B., & Thomas, S. (2008). Distinction of energy efficiency improvement measures by type of appropriate evaluation method. Report of the EMEEES project.
- Hoffman, I., Schiller, S., Todd, A., Billingsley, M., Goldman, C., & Schwartz, L., (2015). Energy Savings Lifetimes and Persistence: Practices, Issues and Data. Technical Brief, Lawrence Berkeley National Laboratory, May 2015.
- ISO 50046:2019. General methods for predicting energy savings. March 2019.
- Maric, L., Thenius, G., Gynther, L., & C. Guermont (2018). Links between M&V tools (data collection) and evaluation (complementary analysis). Topical case study of the EPATEE project, funded by the European Union’s Horizon 2020 programme.
- Mort, D. (2017). Chapter 9: Metering CrossCutting Protocol. The Uniform Methods Project: Determining Energy Efficiency Savings for Specific Measures. Prepared for NREL (National Renewable Energy Laboratory), September 2017. https://www.energy.gov/eere/about-us/ump-protocols
- Sipma, J., Broc, J.S. & Skema, R. (2019). Comparing estimated versus measured energy savings. Topical case study of the EPATEE project, funded by the European Union’s Horizon 2020 programme.
- Voswinkel, F., Broc, J.S., Breitschopf, B. & Schlomann, B. (2018). Evaluating net energy savings. Topical case study of the EPATEE project, funded by the European Union’s Horizon 2020 programme.
- Vreuls, H. (2005). Evaluating energy efficiency policy measures & DSM programmes: Volume I – Evaluation guidebook. Report prepared for the sub-task 9 of task 1 of the IEA DSM agreement.
About EEO schemes
- Bertoldi, P., Castellazzi, L., Oikonomou, V., Fawcett, T., Spyridaki, N. A., Renders, N., & Moorkens, I. (2015). How is article 7 of the Energy Efficiency Directive being implemented? An analysis of national energy efficiency obligation schemes. Proceedings of the ECEEE 2015 Summer Study, paper 2-380-15, 455-466.
- Broc, J.S. (2017b). Snapshot of Energy Efficiency Obligations schemes in Europe: 2017 update. Report for ATEE and the Fourth European Workshop Meeting of the White Certificates Club, 30 June 2017.
- ENSPOL (2015). Report on existing and planned EEOSs in the EU (Part I: Evaluation of existing schemes ; Part II: Description of planned schemes). Deliverable 2.1.1 of the ENSPOL project, March 2015. http://enspol.eu/results
- IEA (2017). Market-based Instruments for Energy Efficiency – Policy Choice and Design. International Energy Agency’s Insights Series 2017.
- RAP (2012). Best practices in designing and implementing energy efficiency obligation schemes. Report for the Task XXII of the IEA-DSM (International Energy Agency Demand Side Management) Programme.
Practical examples related to using deemed savings for the evaluation of savings from EEO schemes
- BRE (2018). ECO3 Deemed Scores Methodology. Report of BRE (Building Research Establishment Ltd) prepared for Ofgem, July 2018.
- Broc, J., Melo, C.A., & Jannuzzi, G.D.M., (2012). Detailed comparison of Brazilian and French obligation schemes to promote energy efficiency, Proceedings of the 2012 International Energy Program Evaluation Conference, 12-14 June 2012, Rome, Italy.
- Del Balso, R.J, & Grabner, K. (2013). But I Thought a State-wide TRM Would Solve Everything? Proceedings of IEPEC 2013.
- Labanca, N., & Bertoldi, P. (2016). Energy Savings Calculation Methods under Article 7 of the Energy Efficiency Directive. JRC Report no EUR 27663 EN for DG Energy, January 2016.
-