30 – Evaluating energy savings from Energy Efficiency Obligation schemes in the industry sector using engineering estimates

The primary audience for this guide is public authorities, obligated parties and other stakeholders involved in EEOs, as well as evaluators looking for guidance on the evaluation process of energy savings in the scope of this guide.

Although the application of the guide will generally concern the national level, account will be taken of issues at EU level when relevant (e.g. the specific format of saving figures for the EED and more particularly its article 7).

This guide is not about the preceding step in the evaluation process, the choice of the method, but to present the case of one particular situation (combination of policy measure, sector and method). About the previous step in the evaluation process, see the general guidance about integrating evaluation into the policy cycle. However, after presenting the capabilities and limitations of the guide at hand, the user will be offered alternatives for the method within this guide (see section 6).

The objective of this guide is to provide:

• Information on the scope of the guide that enables the user to decide whether this guide is suited to his/her needs, and whether complementary or additional method(s) could be needed or useful (section 2);
• Guidance about specifying the evaluation objectives and requirements (section 3);
• Guidance about key methodological choices to calculate energy savings (section 4);
• Guidance about the inputs (data requirements) and outputs of the method (energy savings metrics) (section 5);
• Possible alternative methods (with pros and cons) (section 6)
• Background about evaluation results other than energy savings (section 7);
• Relevant examples, case studies and/or good practices (section 8);
• Relevant references for further reading (section 9).

This guide is intended for assessing realised (ex-post) energy savings. However, account is taken of earlier (ex-ante) evaluations of expected savings, if available (see section 4).

The focus of the guide is on impact evaluation, i.e. determining the energy savings, but not on how this has been reached through a step by step process with intermediate results (process evaluation).

Readers looking for the basic and general principles of energy efficiency evaluation may find the following link about general guidance useful.

2.1 About Energy Efficiency Obligation schemes (EEOs)

EEOs are a type of market-based instruments (general category used in the EPATEE Toolbox and Knowledge Base, as well as in the MURE database). Market-based instruments are policies that set targets in terms of outcomes (e.g. energy savings) to be delivered by market actors, without prescribing the delivery mechanisms and types of actions to achieve these targets (IEA, 2017).

EEOs are regulatory mechanisms setting energy efficiency targets (e.g., energy or CO2 savings) that must be achieved by obligated parties (either energy suppliers or energy distributors). EEOs include rules about what types of actions can be eligible, how energy savings shall be calculated, how obligated parties shall demonstrate their role in stimulating actions (materiality issue), if third parties can be involved in the scheme, if approved energy savings can be traded (cf. white certificates schemes), penalties in case of non-achievement, etc. (Bertoldi et al. 2015).

The main specificity of market-based instruments (here EEOs) is that, within the framework set by the policy, the market actors are free to choose their strategy (flexibility of delivery mechanisms). The underlying assumption is that this approach helps to minimize or optimize the costs of energy savings. In practice, the cost-efficiency of an EEO scheme will depend, among other factors, on the efficiency of its administration, and particularly the way that the energy savings are monitored and verified (Broc 2017).

EEOs can also have other goals than least-cost savings, such as alleviating energy poverty or favouring the market penetration of innovative actions. These goals can influence the evaluation objectives.

In terms of evaluation, one specificity of EEOs is that most of the activities of the schemes are implemented by market actors. This can create difficulties in accessing or collecting data, particularly about costs (Rosenow and Bayer, 2017). Public authorities can enforce requirements about what data shall be provided or stored by the obligated parties. A key design issue is thus to find the right balance between these requirements and minimizing the administration costs (for the obligated parties, as well as for the public authorities making the controls and verifications).

For more details about EEOs, see for example (ENSPOL 2015 ; RAP 2012).

Further references about evaluations of EEOs can be found in the Knowledge Base, by selecting “market-based instruments” in the search filter “Type of policy instruments”.

2.2 Evaluation for a combination of policy measures

Depending on the national context, there can be overlaps between the EEO and other policy measures. The EEO can include rules to avoid certain overlaps, for example by specifying that actions receiving a public subsidy cannot be eligible to the EEO (e.g. in Austria or Italy), or that only actions with a performance higher than minimum requirements set in current regulations are eligible (rule used for most EEOs).

At the opposite, the EEO can allow actions that received a support or incentive from other policy measures. For example, when the financial barriers are supposed to be too high to be overcome by incentives provided by market actors only (e.g. for renovation works in dwellings, as in France and Ireland).

In practice, when dealing with actions in the industry sector, most of the EEOs do not allow to count energy savings from actions receiving public subsidies. However, there can be overlaps with other types of policy measures, especially informative measures such as mandatory energy audits for non-SMEs (as required by EED article 8).

When an EEO is combined with other policy measures, it is usually difficult to disentangle the relative effects of each policy measure. This goes beyond the scope of this guidance.

Therefore in this guidance, it is assumed that the way to attribute the energy savings to the overlapping policy measures has already been decided. See also “Double counting” in the section on Gross to Net savings.

2.3 Evaluation when combined with energy taxes

The calculated savings effect for EEOs will overlap with that of carbon and/or energy tax(es). Separating the effects of the EEOs and carbon/energy taxes goes beyond the scope of this guidance.
It should be noted that EEOs can have an impact on energy prices, depending on the cost recovery mechanisms, i.e. how the obligated parties can recover the costs they have incurred to meet their energy efficiency targets. When analysing the interactions between EEOs and carbon/energy taxes, a first issue is therefore to assess the impact of the EEOs on energy prices (Giraudet and Finon 2015). However, in most cases, this impact remains small compared to the share that energy taxes represent in energy prices.

2.4 About the focus on industry sector

EEOs can be transversal (i.e. actions are eligible in several or all sectors, like for most EEOs in Europe), or restricted to a particular sector (e.g. residential sector, like in UK). This guidance is focused on actions done in the industry sector, where a large share of energy savings (often the largest) has been achieved so far in many of the European EEOs (see Broc 2017 for details per country).

Most EEOs make the difference between action types that can be standardised and thus evaluated with deemed savings (case dealt with in Specific Guidance 28 ; also see this link) and action types that are more specific and require case-by-case energy savings calculations (the case dealt with in this guidance).

Actions done in the industry sector can be difficult to standardise, as they can be tailored to very specific industrial processes or industry sub-sectors. They can also deal with action types that are applicable in most industries (e.g. heat recovery systems, variable speed drives). However, the implementation of the same action type can vary significantly from one sub-sector to the other, depending on the size of the site, etc. Which can make it impossible to define average values that would be representative of the whole actions done. Hence the focus of this guidance on the combination of the industry sector and the method of engineering estimates (that enables specific energy savings calculations).

Most of this guidance can also be applied to specific actions implemented in other sectors, especially energy savings projects in large commercial buildings.

Information on (sub) sectors defined in the Toolbox can be found in the EPATEE terminology, chapter 2, p.17.

2.5 About engineering estimates

The revised Energy Efficiency Directive (2018/2002), Annex V(1.a)) defines the results of engineering estimates as “scaled savings”, also mentioning that “those estimates are carried out on the basis of nationally established methodologies and benchmarks by qualified or accredited experts that are independent of the obligated, participating or entrusted parties involved”.

In practice, engineering estimates correspond to energy savings calculations or modelling that take into account values specific to the energy saving actions or projects implemented.

When using engineering estimates in the frame of an EEO scheme, one of the main challenges is to ensure that these specific calculations are done in a consistent way by the various stakeholders who can report savings, meeting minimum quality requirements (e.g. about the documentation of the calculations) and complying with the calculation principles set for the EEO scheme (e.g. for defining the baseline). Two main approaches are commonly used to achieve this harmonisation when dealing with actions in the industry sector:

• Standardised calculation formulas that define what parameters need to be taken into account, and how they shall be estimated.
• A general calculation methodology that defines the rules to define the baseline, what data needs to be documented (about the baseline situation and the situation after implementing the action), what explanations need to be provided, and possibly what data needs to be measured (+ minimum measurement periods).

The first approach (standardized formulas) can for example be used for common action types (e.g. installation of variable speed drives). This requires the public authority to define formulas per type of action. They can be prepared by the national energy agency or technical institutes. They can also be submitted by professional organizations or other stakeholders, and then reviewed and validated by the public authority.

The second approach (general methodology) can for example be used for complex or specific projects (e.g. improving specific industrial processes). The methodology is usually related to the implementation of an energy audit or alike (e.g. feasibility study). Existing qualification schemes for energy auditors or ESCO’s, templates for audit reports, etc. can thus be used as part of the requirements for the EEO.

In any case, a good practice is to organize a consultation to get feedback from stakeholders involved in the EEO scheme, before the final adoption of the formulas or methodology. This increases their legitimacy and the further use of the corresponding action types and methodology (increased awareness of the opportunities, and visibility about the outputs).

Approach	Advantages	Drawbacks
Standardised formulas	Higher harmonisation of the calculations Can enable automatized verifications of the calculations Higher awareness / visibility of the corresponding action types	Can restrict the types of actions eligible to the EEO Require to define a set of formulas (per action type)
General methodology	Higher flexibility / larger scope (can be used for any type of energy saving action or project) Can be part of a process to identify energy savings opportunities (audits) Can build on the experience of previous audit programmes (or alike)	Lower harmonisation of the calculations Require more work to verify the calculations Can require to develop a qualification scheme for energy auditors or ESCO’s

The size and complexity of energy saving actions or projects evaluated with engineering estimates can vary widely. The requirements about the data, calculation formula (or model) and their documentation can be adapted according to criteria to take into account this diversity. For example, measurements can be required for projects with expected energy savings above a given threshold.

Another key criteria when setting the requirements for the calculation method is the balance between minimizing administration costs and ensuring the reliability of the engineering estimates (and that they can be verified).

Depending on the complexity of the action or project, the engineering estimates can be calculated with simple formulas (with a limited number of parameters) up to sophisticated modelling (requiring larger sets of data). When a model is used, the requirements should deal with the stage of calibration (i.e. testing the model on a known case to see if it needs to be adjusted).

As engineering estimates are calculations specific to each action or project, they can have multiple purposes, beyond reporting energy savings for the EEO scheme. When done before implementing the action (ex-ante), they can for example be part of a feasibility study or defining an investment plan. When done or verified after implementing the action (ex-post), they can be part of an energy management or monitoring system.

Information about the various evaluation methods covered by the EPATEE toolbox can be found here, table 1 and 2. This source also covers the combination of the method at hand with other methods, which will be dealt with below.

2.6 Complementary methods to determine total savings

Complementary methods are methods that are required, in addition to the primary selected method (here engineering estimates), to calculate total energy savings.

Scaled savings are calculated at the level of a given action, project or participant (unitary savings). The unitary savings then need to be summed over the whole actions counted for the EEO scheme, to obtain the total energy savings.

As the unitary savings are specific to each action, project or participant, the most common approach is to use a monitoring system that will gather the data of the unitary savings. Indeed, the obligated parties (or the other stakeholders that can claim energy savings for the EEO) have usually to report this data so that the corresponding energy savings can be counted for the achievement of their obligation.

This guidance does not deal with monitoring systems. For more details about the links between monitoring and evaluation, see the corresponding EPATEE topical case study.

2.7 Additional methods to increase reliability of the results

An additional method can be applied on top of using engineering estimates to improve the reliability of the evaluation results and/or the cost-effectiveness of the evaluation approach.

Engineering estimates are usually based on calculation formulas or models that estimate conventional energy consumption. This means that the calculations are made with conventional assumptions about the conditions of use of the energy using systems (e.g. weather conditions, volume of production). The performance or efficiency of the energy using systems can also be based on standard values (e.g. manufacturers’ data) that do not necessarily reflect their actual performance or efficiency.

Depending on the data and assumptions used for the calculations, scaled savings do not necessarily reflect actual energy savings (i.e. as could be calculated from measured energy consumption). The related uncertainties are due to the assumptions made in the calculations. These uncertainties can for example be reduced by calibrating the model used with data of measured or metered energy consumption.

Moreover, errors or uncertainties can also arise along the calculation process (e.g. uncertainties due to measurements of particular parameters such as load, pressure or flows; errors in conversions from one unit to another). Part of these errors or uncertainties can be corrected or reduced through ex-post verifications (e.g. automatized plausibility checks, on-site inspections).

Depending on the evaluation objectives, other methods can be used to investigate the reliability of the scaled savings (e.g. tracking errors, verifying the relevance of the data or assumptions used), assess actual energy savings or investigate additionality / causality (see in section 4 Calculating Gross and Net energy savings).

Type of method	Short description	Objective
Survey of participants	Surveys can be used to collect data complementary to the ones collected by the monitoring system, especially to investigate additionality (see for example the case study about the Danish EEO scheme).	Investigate issues of particular interest (e.g. additionality, market trends)
Direct measurement	Direct measurement can be done on a specific parameter (e.g. sensors to monitor duration of use), or on the energy consumed by a given equipment or process (e.g. sub-metering). For more details, see e.g. (Mort 2017)	To verify if the assumptions used in the engineering estimates reflect actual conditions (e.g. energy performance, duration of use, load levels)
Billing analysis	Calculation of energy savings from metered data of energy bills. For more details, see (CAG Consultants et al. 2015) and Specific Guidance 29	To assess actual energy savings (as can be experienced by the end-users). To assess net energy savings (when using econometric methods, e.g. comparing participants and non-participants). In practice, it is difficult to use this method when the EEO scheme has been in place for many years (more challenging to find non-participants, and more risks of bias in comparing participants and non-participants). It can also be difficult to find comparable non-participants for specific industries with few (or no) similar sites or industrial processes. Moreover, billing analysis is usually not relevant to use for cases where the expected energy savings are too small (e.g. less than 10% of the total consumption metered for the bills).
Ex-post verifications	Verifications of the data used and calculations made (as reported by the obligated parties), for example through automatized plausibility checks, benchmarking or on-site inspections.	To check the data reported, remove possible errors and reduce uncertainties. Ex-post verifications can be part of the monitoring system of the EEO scheme. Further verifications can also be done as part of an ex-post evaluation. See for example the case study about the Danish EEO scheme.

As the number of actions or projects monitored for EEOs can be very large, this could be costly to apply these additional methods to the whole actions reported by obligated parties. Therefore, a common practice is to apply these methods on samples of actions or participants. As sampling can create bias and uncertainties in the results, a particular attention should be paid to limit sampling bias and obtain samples as representative as possible.

It should be noted that the sampling approach for evaluating the total savings from the EEO scheme can be different from the sampling approach for the M&V system of the EEO scheme:

When evaluating the total savings, sampling is aiming at ensuring representativeness so that the results can be extrapolated from the sample(s) to the whole actions counted for the EEO scheme.

For the M&V system, sampling is usually defined with a combination of random and risk-based approaches, to give the signal that any energy saving action or project can be subject to verification (random approach) and to check in priority the actions or projects representing higher risks of errors or frauds (risk-based approach).

Combining different methods can be a cost-effective approach to increase the reliability of the savings figures and to get a better understanding of the impacts of the actions and the EEO.

For possible combinations with an additional method see also chapter 6 in the document here.

3.1 Meeting evaluation goals and ambition
Typical objectives of using engineering estimates can be:
• Getting savings estimates along the implementation of the actions (no or limited time-lag);
• Taking into account the diversity of actions and context of implementation;
• Getting savings estimates that reflect the specificities of each action and technical energy efficiency improvements;
• Providing visibility to market actors about the rules to calculate energy savings (possible to estimate savings before the action is implemented, e.g. as part of feasibility studies or business plans).
At the opposite, engineering estimates are not the most appropriate method in case the primary evaluation objective is to assess actual energy savings (as experienced by the end-users) or to take into account possible changes in users’ behaviours.
The table shows whether this guide can be used to report on general evaluation goals or criteria. See also (Broc et al., 2009).

General types of evaluation goals or criteria	Level of ambition	Remarks
Energy savings calculations	Medium	Depending on the quality of the engineering estimates, and especially the factors taken into account in the calculations. In any case, complementing engineering estimates with a method to verify actual energy savings is recommended for this objective (see above in section 2: Additional methods to increase reliability of the results)
Calculation of energy savings attributed to the EEO	Low to medium	Additionality criteria (e.g. about defining the baseline) or default adjustment factors can be included in the calculations to make that engineering estimates correspond to additional savings. It is recommended for this objective to complement engineering estimates with further ex-post analysis (see below in section 4: Calculating Gross and Net energy savings)
Cost-effectiveness of saving action (for end-users)	Medium to high	Engineering estimates can reflect the characteristics of the actions and conditions of implementation specific to each case. When the calculations can be calibrated or benchmarked, they can offer a reliable estimate for an investment plan. As the calculations have to be done for each case, cost data and assumptions about savings lifetime are usually available to or made by the company or consultancy making the calculations.
Cost-effectiveness of the EEO (from a society’s perspective)	Medium	Depending on the quality of the engineering estimates (see previous line). Further ex-post study might be needed to collect cost data, unless the EEO rules required stakeholders to report cost data together with energy savings data (e.g. as part of audit reports). Depending on the reporting requirements of the EEO scheme, further assumptions about savings lifetime might be needed. (see below in section 7: Calculating cost-effectiveness) Whenever possible, it is also recommended to complement engineering estimates with ex-post verifications of savings (see above in section 2: Additional methods to increase reliability of the results)
CO2-emission reduction from saving actions	Medium	The basis will be the estimated energy savings. As engineering estimates are calculated specifically for each action or project, this will generally make it easy to apply standard emission factors to calculate CO2 savings from energy savings. For example, the EEO rules can include CO2 savings as part as the reporting requirements, or can require energy savings to be reported per energy type (facilitating the calculation of CO2 savings by the monitoring body). (see below in section 7: Calculating avoided CO2 emissions)
CO2-emission reduction attributed to the policy measure(s)	Low to medium	The basis will be the estimated energy savings (see previous line). Additionality criteria (e.g. about defining the baseline) or default adjustment factors can be included in the calculations to make that engineering estimates correspond to additional savings. (see below in section 4: Calculating Gross and Net energy savings ; and in section 7: Calculating avoided CO2 emissions)

3.2 Reporting expectations

The rules of the EEO include the definition of the unit used to count the results to be reported by the obligated parties. This unit includes several criteria for which various options are possible.

Criteria	Common options	Remarks
Nature of the objective	Energy savings CO2 savings Bill savings	Choice mostly depending on the primary objective of the EEO
Duration for which the results are counted	Annual (or first-year) Lifetime cumulated Cumulative over the obligation period (see section 5)	Choice depending on the EEO objectives. A “lifetime cumulated” unit can be chosen to value long-lifetime actions. But this can require to estimate energy savings over long durations, which can increase uncertainties.
Energy basis (when primary objective = energy savings)	Primary energy savings Final energy savings	Choice depending on the objectives and scope of the EEO. For example, primary energy savings can be chosen if the scope includes actions for cogeneration (CHP). Final energy savings can be chosen if the focus is on actions on energy end-uses. The energy savings to be reported in the context of the EED article 7 are final energy savings. However the EEO scheme can count primary energy savings for national purposes, and then convert the results in final energy savings when reporting for EED article 7 (e.g. case of the Italian white certificates scheme).
Energy unit (when primary objective = energy savings)	PJ / ktoe / TWh / …	Choice usually depending on the energy unit commonly used in the country (e.g. for the national energy balance), or on the objectives or scope of the EEO. For example, ktoe can be used if the priority is to save fossil fuels. TWh can be used if the scope of the EEO is restricted to electricity savings.
Evaluation perspective	Gross / Additional / Net	For more explanations, see in section 4 Calculating Gross and Net energy savings.

Scaled savings from engineering estimates will then be calculated in the unit chosen for the EEO.

It can also be needed to express the results in another unit than the one of the EEO. For example, to compare results with other policies (benchmarking), or to report results in other context (e.g. for the article 7 of the Energy Efficiency Directive).It is therefore important to keep the documentation of the engineering estimates (e.g. through standard template for audit reports). These details can indeed be needed to convert the results from the unit of the EEO to the other unit.

Guidance and examples of template to document deemed savings can be found in the standard ISO 50046:2019.

3.3 Time frame for evaluation

The period under evaluation usually corresponds to the latest period of the EEO scheme, and more specifically to the period(s) for which the target(s) under review has been set.

One of the advantages of engineering estimates is that it enables to assess energy savings along the implementation of the action (and possibly before the actions are implemented). Two steps can be distinguished:

1. Calculations made by the stakeholders reporting energy savings to the public authority or monitoring body of the EEO scheme: these are often to be done before the actions are implemented (e.g. as part of a feasibility study or energy audit), possibly with an update once the actions are in place.
2. Verification and validation of the calculations by the public authority or monitoring body: depending on the complexity of the actions or projects (and on the use of standardised formulas or general calculation methodology), the validation can be partly or fully automatized (i.e. with no or limited time lag) or can require further analysis (i.e. with some time lag). In the latter case, the time lag can also increase with the flow of projects to validate.

Usually, a periodical (often annual) review of the energy savings results is made to take into account possible cancellations of energy savings (e.g. due to non-compliance detected through controls), and more generally to make verifications. It can also be done to convert the energy savings results into another energy savings unit when needed for external reporting purposes (e.g. for the annual reports in the context of the Energy Efficiency Directive).

In case the rules of the EEO scheme include standardised calculation formulas, time is needed to prepare and validate them. Further updates might also be needed, for example based on feedback from stakeholders or verifications. The time and resources needed to develop (or update) a catalogue of calculation formulas depend on the number and diversity of the action types to be included in the catalogue, previous experience (e.g. through voluntary agreements or audit programmes), level of consensus (or disagreement) about calculation formula, etc.

The development of a catalogue of calculation formulas is thus to be considered when there is enough visibility about the existence of the EEO scheme. The obligation periods defined in the EED Art. 7 (currently 2014-2020, then 2021-2030) help to provide a legislative visibility.

In case the rules of the EEO scheme include a general calculation methodology, this will usually need less time than for standardised formulas to be prepared. But this can require an iterative process to find the appropriate level of details in the requirements and guidelines set in the methodology.

The validation process of the formulas or calculation methodology can include a consultation of the stakeholders, either through an open public consultation or within technical working groups. The time needed for the consultation can be set by law. In this case, this should be taken into account to avoid postponing the start or revision of the EEO scheme.

The use of engineering estimates can be complemented by other methods to verify and improve their reliability (see in section 2 Additional methods to increase reliability of the results). These additional methods have different timeframes. It is thus recommended to consider how the combination of evaluation activities should be planned, especially to ensure the feasibility of the corresponding data collection and to optimize the use of resources (time and budget). (Guidance and examples of template to document deemed savings can be found in the standard ISO 50046:2019.

3.4 Expertise needed for chosen method

The calculation of engineering estimates requires expertise about the following:

• Sectoral (or even more specific) expertise about the energy efficiency actions eligible to the EEO scheme: this is needed to select the essential parameters to describe the action types and their energy saving effect (either for defining standardised calculation formulas, or for performing calculations based on a general calculation methodology).
• Sectoral (or even more specific) expertise about data or trends in energy consumption and energy efficiency markets: this can be needed to define the baseline, especially if the EEO rules require calculating additional energy savings.
• Expertise with energy modelling (in case models are used for the calculations), and especially with calibrating models.

This guidance considers the use of engineering estimates in the industry sector. Depending on the action types eligible to the EEO scheme, the following expertise specific to action types can for example be needed:

• about heat, steam and cooling processes (including about heat recovery systems);
• about electric motors;
• about compressed air systems

In addition to these cross-cutting action types, energy saving opportunities in the industry sector can correspond to actions specific to various types of industrial processes that can require specific expertise about their optimisation.

Most of this guidance can also be useful when considering the use of engineering estimates in other sectors (e.g. for large commercial buildings or for the agriculture sector). In this case, other sectoral expertise can be needed depending on the action types to be covered.

Guidance and examples of template to document deemed savings can be found in the standard ISO 50046:2019. This section deals with key methodological choices to be considered when calculating energy savings: consistency between ex-ante and ex-post evaluation, baseline, normalization and adjustment factors. These choices are important to document when reporting energy savings, to ensure the transparency of the results.

For more guidance about documentation, see for example ISO 50046:2019.

4.1 Matching method with earlier ex-ante evaluation

From the viewpoint of methodological consistency and data availability using the same method in the ex-ante evaluation and in the ex-post evaluation is recommended.

In the case of an EEO scheme, the ex-ante evaluation can take the form of an assessment of the energy savings potential within the scope of the scheme, complemented with assumptions about rates of achievement of this potential. This basis will then be taken into account when setting the target(s) for the EEO scheme. The ex-ante evaluation can also be an impact assessment, especially when such assessment is required due to the legislative process (e.g. when each period and target of the EEO scheme needs to be voted by the Parliament). In this later case, the impact assessment can include several scenarios to compare impacts according to distinct alternatives (e.g. in terms of levels of targets).

Other background elements usually enter into the process of defining the target(s): umbrella objectives (e.g. due to a national Energy Law, to the national targets transposed from the EU Energy Efficiency Directive), economic context (e.g. trends in energy prices), interactions with other policy measures, consultation(s) with stakeholders. These other elements will mostly be related with the assumptions about rates of achievements. Therefore, the consistency between ex-ante and ex-post evaluations is mostly to be ensured between the assessment of savings potential (or impact assessment) and the ex-post evaluation based on engineering estimates (for the case considered in this guidance). The same calculation methodology can be used for both, the ex-ante and the ex-post evaluation. Respectively to set the target and to verify its achievement. When it is not the case, it is important to use the same principles when defining the baseline(s).

As discussed in section 2 (see Additional methods to increase reliability of the results), additional method(s) can be used to verify energy savings ex-post. In that case, the results from these ex-post verifications can be used to improve the engineering estimates for future ex-ante or ex-post evaluations. For a general discussion about possible combinations of methods applied ex-ante and ex-post, see chapter 7 in this document.

It is important here to distinguish ex-post verifications done as part of controls, and ex-post verifications done as part of an evaluation. Controls are commonly decided on a risk-based approach, in order to focus the means of control where the risks are higher or more critical. Therefore, results from controls are not meant to be representative.

Evaluation methodologies are usually designed to look at the whole scheme, taking into account representativeness, risks of sample bias, etc. Depending on the evaluation objectives, evaluation results can thus be expected to be representative.

4.2 Calculation baselines

Gross energy savings are defined in general as the difference between the situation including the implementation of energy saving actions and a reference situation without the saving actions. This reference situation can be defined using various calculation baselines. For a general discussion about calculation baselines, see the link to this document.

When evaluating the energy savings from an EEO scheme, the baseline is also usually defined to represent what the situation would have been in the absence of the EEO scheme. This is in particular linked to the concept of additionality, as defined in the amended EED (EU 2018/2002)):

“To determine the savings that can be claimed as additional, Member States shall have regard to how energy use and demand would evolve in the absence of the policy measure in question by taking into account at least the following factors: energy consumption trends, changes in consumer behavior, technological progress and changes caused by other measures implemented at Union and national level” (EED 2018/2002), Annex V (2) point (a)).

Therefore, this guidance considers here for the EED Art. 7 purposes the case where the evaluation objective is to calculate additional savings.

When using engineering estimates, additionality has to be taken into account in the calculation assumptions, as engineering estimates are usually first calculated before the actions are implemented (to give visibility to stakeholders, see section 3 Meeting evaluation goals and ambition). Two main approaches are thus possible:

1. Take into account additionality criteria in the definition of the baseline (option presented below).
2. Apply adjustment factors to gross energy savings (option presented later in this section, see Adjustment factors)

Additionality criteria are usually to take into account market trends or effects from other policy measures (particularly regulations setting minimum energy performance requirements). Typical examples of baselines used to calculate additional savings are:

• market average: using statistics on market average is a common way to reflect market trends in the baseline. This option can for example be used for cross-cutting technologies. One possible difficulty to use this option is when market data are not available. In this case, one alternative is to use as market average the characteristics of the technology deemed to be dominant on the market. Moreover, this option might not be applicable for actions very specific to particular industrial processes (for which there is no real market to compare with).
• minimum efficiency standards: using as baseline the minimum energy performance requirements set in current regulations is a common way to ensure that energy savings are additional to these regulations (thereby avoiding double counting).
• energy efficiency indicators from benchmarking: when no market average can be defined or no standard is applicable, an alternative can be to look for benchmarking data about energy efficiency indicators (e.g. kWh per ton of product).

In some cases, the baseline for additional savings can be the same as for gross savings, i.e. equivalent to a before/after comparison. When using engineering estimates, the rules of the EEO scheme can require in that case to calibrate the baseline energy consumption with metered data. This option can for example be allowed when it is considered that the current rate of action is very low (e.g. actions with long payback time), thus assuming that all corresponding actions can be considered additional.

4.3 Normalization factors

The calculation of engineering estimates is usually made based on normalized conditions of use (of the energy systems). When dealing with actions in industry, the main conditions to normalize are often the volumes and types of production.

In practice, volumes and types of production can change rapidly and significantly. The rules about the calculation methodology therefore needs to specify what conditions should be taken as basis for the energy savings calculations. This is also needed to provide stakeholders with visibility about the energy savings that can be counted for the EEO.

One option can for example be to require to calculate the normalized conditions as an average over previous years (e.g. 3 to 5 years).

A complementary option can be to ask for verifications of the conditions sometime after the implementation of the actions. This can be used to obtain results closer to actual energy savings (as experienced by the end users). However this creates uncertainties for the stakeholders about the energy savings they can count for the EEO.

Depending on the type of action and related previous experience or study, it might also be relevant to consider the risks of performance gap. Performance gaps correspond to cases where the observed energy performance of the energy saving action installed is lower than the expected energy performance. For example about motors, this can happen when the actual load curve is different from the load curve assumed in the calculations (and that motor efficiency varies with load). More generally, it can for example be due to defaults in the technologies or inappropriate conditions of use.

Enforcing quality requirements can help to minimize the risks of performance gaps. Likewise, if the action is installed in a site where an energy management system is in place, this system should help detect defaults or suboptimal conditions of use. Thereby limiting the risks of performance gaps.

Factors to take into account performance gaps can for example be derived from previous studies on samples of actions.

In case, the engineering estimates are based on measured data (e.g. metered load or flow data), the calculation methodology should include requirements about the representativeness of the metering or measurement period. If so, no further normalization is needed (nor factor about performance gap, depending on the type of data that is measured or metered).

4.4 Adjustment factors

Adjustment factors define which part of the calculated energy savings can be attributed to a policy measure or meets the definition of savings specified in the evaluation objectives or reporting requirements. For a general introduction about adjustment factors, see table 1 here.

This guidance considers the case where the evaluation objective is to calculate additional savings (for a discussion about this, see below Calculating Gross and net savings)

When using engineering estimates, additionality has to be taken into account in the calculation assumptions, as engineering estimates are usually first calculated before the actions are implemented (to give visibility to stakeholders, see section 3 Meeting evaluation goals and ambition). Two main approaches are thus possible:

1. Take into account additionality criteria in the definition of the baseline (option presented above in Calculation baselines).
2. Apply adjustment factors to gross energy savings (option presented here below)

Adjustment factors can concern free rider effect and spill-over/multiplier effect. For related definitions, see the EPATEE terminology. Free-rider and spill-over effects can also be encompassed in a single additionality factor.

In any case, when using deemed savings, these effects can only be taken into account based on previous studies or surveys used to define corresponding adjustment factors. For an example about this, see the case study about Danish EEO scheme (Broc, 2017a).

It should be noted that in most of the available experience (especially in Europe), adjustment factors have been defined to take into account free-rider effect only. Spill-over effects have indeed proven to be more difficult to assess quantitatively (see also the experience from the Danish EEO scheme).

In case of other policy measures target the same saving action types as the EEO scheme (either partial or full overlap), the evaluation of the energy savings should also consider how double counting is avoided or corrected. Methods to tackle double counting or interaction between policy measures go beyond the scope of this guidance.

4.5 Calculating Gross and Net energy savings

Gross energy savings are energy savings calculated from the point of view of the final consumers, i.e. independently of whether the participants to the policy measure would have acted the same or differently in the absence of the policy measure.

Usually the baseline energy consumption used to calculate gross energy savings is the energy consumption before the energy saving actions were installed or implemented.

The calculation of gross energy savings can include the use of normalization factors to ensure that the baseline energy consumption and the energy consumption with the energy saving action are comparable. For the industry sector, key normalization factors are often related to the volumes and types of production (see Normalization factors above).

Depending on the type of action and data used to calculate gross savings, it can also be relevant to use correction factors for performance gaps (see Normalization factors above). These effects indeed affect energy savings from the point of view of the final consumers.

Net energy savings are calculated from the point of view of the public authority, policymaker or other stakeholder that provides any type of support or incentive to promote the energy saving actions. Therefore, this calculation takes into account effects related to the causality or attribution of the actions or energy savings to the policy measure or interventions of the stakeholders.

When evaluating an EEO scheme, it is common in Europe to speak about additional savings instead of net savings, due to the terminology of the EU Energy Efficiency Directive. More generally, the term “additional savings” is also used because the objective of the EEO schemes is usually to achieve energy savings either additional to a business-as-usual scenario, or additional to the effects of other policy measures (particularly regulations).

When using engineering estimates, additionality has to be taken into account in the calculation assumptions, as engineering estimates are usually first calculated before the actions are implemented (to give visibility to stakeholders, see section 3 Meeting evaluation goals and ambition). Two main approaches are thus possible:

1. Take into account additionality criteria in the definition of the baseline (option presented above in Calculation baselines).
2. Apply adjustment factors to gross energy savings (option presented above in Adjustment factors).

Additional methods can then be used to assess net or additional savings ex-post (e.g. at the end of an obligation period). Explanations and guidance about methods for this type of ex-post evaluation can be found in the topical case study about the evaluation of net energy savings (Voswinkel et al., 2018).

Additional or net savings should also be corrected for double counting, in case of possible overlap between the EEO scheme and other policy measures. The overlap in the calculated savings should be analysed at the level of the overall policy portfolio or sector. For addressing double counting see (Vreuls, 2005) or (Broc et al., 2009).

See also sections 8 about concrete examples and 9 about further reading.

5.1 Main data requirements and data sources and collection technics

General data requirements specified in the table below correspond to the calculation of gross energy savings, when using the baseline option [before/after], i.e. a baseline representing the situation before energy saving actions are implemented. The case about using baseline options to calculate additional energy savings is discussed later on (see Data issues when evaluating net energy savings).

The table below deals with the case where engineering estimates are the result of calculations that estimate “before” and “after” energy consumption through intermediate parameters.

Calculation subject	Data requirements	Possible data sources and collection technics
Parameters used to calculate the energy consumption “before”	Selecting data representative of usual conditions (e.g. averaged over several years)	Data can be collected or estimated as part of an energy audit or through the implementation of an energy management system
Parameters used to calculate the energy consumption “after”	Justifying the energy efficiency improvement brought by the action	Data can for example be taken from manufacturers’ documentation, or adapted from results of measurements previously made on similar actions.
Assumptions on savings lifetime (and possibly on evolution of savings over time)	Can be required depending on the EEO rules Needed to assess the cost-effectiveness of the action	Can be based on energy auditors’ expertise or benchmarking Can be updated afterwards (for example if monitored with an energy management system)
Normalization factor for volumes and types of production (when relevant)	Can be part of the explanations required for the reporting	To be defined as part of the calculations (specific to the case evaluated). Can be based for example on conditions of previous years or on planned production levels for coming years.
Correction factor for performance gap (when relevant)	Can be replaced by quality requirements (or alike, e.g. about energy management system)	Data from previous studies or verification on similar actions. Data from the literature. Conservative assumptions.

One common approach to enable normalization is to estimate unitary energy consumption (e.g. in kWh per ton of product). However, it should be noted that energy consumption is not necessarily linearly proportional to production. For example, part of the consumption can be fixed, another part depending on the production. The corresponding formula can either be based on the engineering analysis of the process (when the influencing variables and their relations to energy consumption are well established by physics or other scientific laws), or based on statistical analysis of previous periods (e.g. for complex processes).

Complementary information

Depending on the rules of the scheme, complementary information or data can be required, for example:

• Explanations about the selection of the action implemented;
• Cost data (e.g. to calculate payback time, see also in section 7 Calculating cost-effectiveness)

Data issues when evaluating net energy savings

When evaluating energy savings from EEO schemes with engineering estimates, it is more common in Europe to speak of additional energy savings than net energy savings (see in section 4: Calculating Gross and Net energy savings).

When using engineering estimates, additionality has to be taken into account in the calculation assumptions, as engineering estimates are usually first calculated before the actions are implemented (to give visibility to stakeholders, see section 3 Meeting evaluation goals and ambition). Two main approaches are thus possible:

1. Take into account additionality criteria in the definition of the baseline (option presented in section 4, see Calculation baselines).
2. Apply adjustment factors to gross energy savings (option presented in section 4, see Adjustment factors).

The first option will lead to use a baseline different from the “before/after” comparison (for more details, see in section 4 Calculation baselines). This means the following data requirements:

• for the baseline option “market average”: data about trends or recent market shares per energy class (or similar categories reflecting energy performance levels) to calculate the market average for energy consumption or energy performance characteristics (in case energy consumption is calculated with intermediate parameters), or alternatively the energy consumption or performance of the dominant technology on the market.
• for the baseline option “minimum efficiency standards”: data about the current minimum energy performance requirements set in national or European regulations.
• for the baseline option “energy efficiency indicators from benchmarking”: data from sectoral statistics or sectoral studies on energy efficiency (or alike).

When assessing net or additional savings with adjustment factors, the baseline option is “before/after” comparison (see above). The data needed for the engineering estimates to correspond to net or additional savings is then data to define the adjustment factors. Such data can be obtained from previous studies (e.g. previous surveys or market analysis) or the literature. As far as possible, it is recommended to use data from previous studies on the same policy measure and for the same or similar (sub-) sector.

Literature indeed shows that values for adjustment factors can vary significantly from one policy measure to another, one country from another, one sector from another, etc.

Complementary ex-post evaluations can be used to investigate and update or revise the adjustment factors.

For more details about the evaluation of net energy savings, see the dedicated topical case study (Voswinkel et al., 2018).

5.2 Energy savings in final terms or in primary terms

Energy savings can be expressed in final terms or in primary terms See definitions about primary and final energy in the EPATEE terminology.

Usually engineering estimates (scaled savings) are first calculated in final terms. One advantage of choosing to calculate in final energy is that it enables to compare scaled savings with statistics of metered energy consumption (mostly energy bills, sub-metering or laboratory tests).

Scaled savings can also be calculated in primary terms (as in the Italian white certificates scheme), provided that energy savings are calculated for each energy carrier apart, and primary factors are available to convert the savings in final terms into savings in primary terms.

Scaled savings can also be calculated as monetary savings, i.e. savings on energy bills (see in section 3 Reporting expectations). This requires defining average energy prices per energy carrier, as well as a scenario of energy prices if the savings are calculated in cumulative terms, either over the obligation period or over the action lifetime. In this case, a discount rate can also be applied to the calculation (see Energy savings over time below).

For consistency, the metrics should be the same for setting the target(s) and counting the savings.

The energy savings results can then be expressed in other metrics for other purposes (e.g. reporting in the context of the EED article 7), provided that the data needed to convert from one metric to the other is available. It is thus important to identify the needs to express the results in different metrics, so that data used in the energy savings calculations are documented enough to enable future conversions.

5.3 Energy savings over time

Saving actions installed in a year lead to savings over a number of consecutive years, depending on the savings lifetime. E.g. a more efficient industrial boiler can save gas over its lifetime of about 20 years. Energy savings can be calculated in different metrics in terms of time reference, for example: year-to-year, annual, cumulated annual, cumulative. See the definitions in the EPATEE terminology.

The EEO scheme can count from first-year savings only, up to lifetime-cumulated savings (i.e. savings over the lifetime of the energy saving action).

For consistency, the metrics should be the same for setting the target(s) and counting the savings.

The energy savings results can then be expressed in other metrics for other purposes (e.g. reporting in the context of the EED article 7), provided that the data needed to convert from one metric to the other is available. It is thus important to identify the needs to express the results in different metrics, so that data used in the energy savings calculations are documented enough to enable future conversions.

If only first-year savings are counted and needed for reporting, then no further data is needed, apart from monitoring when the actions are installed (information needed anyway for the monitoring of the EEO scheme). For example, first-year savings of new actions correspond to the new annual energy savings as defined for EED article 7.

In other cases, the data about when the actions are installed will need to be complemented with data about the estimated lifetime of the savings, or the periods over which savings can be counted (crediting durations in the context of the EEO scheme).

When dealing with engineering estimates, the rules of the EEO scheme can indeed use different approaches, for example:

• Define standard lifetimes per action type (or general categories of actions), when using standardised calculation formula, or require justifications about the assumption on savings lifetime. Then energy savings are cumulated over this saving lifetime (e.g. case of the French white certificates scheme).
• Define standard crediting durations, per general categories of action, and then credit savings each year over the corresponding number of years (e.g. case of the Italian white certificates scheme).
• Define special factors applied to first-year savings to give a premium to long-lifetime actions (factor > 1) and decrease the savings credits given to short-lifetime actions (factor < 1) (e.g. case of the Danish EEO scheme).

Depending on the approach used, the monitoring system might need to keep track of the action types or categories. Especially if there is a need to convert the results in other metrics for other reporting purposes.

Examples of lifetime values can be found in the following sources:

• CWA 15693:2007. Saving lifetimes of Energy Efficiency Improvement Measures in bottom-up calculations. CEN Workshop Agreement, April 2007.
• EN 15459:2017. Energy performance of buildings — Economic evaluation procedure for energy systems in buildings. CEN standard, June 2017. (See annex D).
• Ecodesign Impact Accounting – Status Report 2017. Prepared by VHK for the European Commission December 2017. (See annex A, pp.73-76).
• When dealing with actions in the industry sector, the calculation methodology can take into account that the savings lifetime can be shorter than the technical lifetime of the action due to business cycles or unexpected changes in activity (e.g. closure or relocation).

On top of cumulating savings over savings lifetime or crediting durations, the rules of the EEO scheme can also include the application of a discount factor. This discount factor can be used:

• For economic reasons: for example when the lifetime-cumulated energy savings are credited at once when the action is installed, and that the energy savings credits can be traded as a commodity (e.g. case of the French white certificates scheme)
• For technical reasons: for example, to take into account that energy savings can decrease over time (e.g. for behavioural actions).

When using an economic discount factor, it is usually the same value for all action types. Because once the energy savings are credited, they can be traded independently of the action types that produced these energy savings. The discount factor used for the EEO scheme is often defined from discount factors commonly used by economic agents, for example discount factors used by public authorities.

When using technical discount factors, they usually need to be differentiated per action type, as the changes in energy savings over time depend on the action type. Overall, there is limited evidence about decrease (or increase) of energy savings over time (for more details about this issue, see for example Hoffman et al. 2015).

A general default discount factor (i.e. uniform for all action types) can also be decided to reflect the risks related to the investment in the energy saving action, as perceived by the final customers or investors.

6.1 Alternatives to engineering estimates

Engineering estimates are often chosen when the objective is to evaluate energy savings from actions or projects that can be specific to their context and conditions of implementation, while providing stakeholders with visibility about the calculation rules and enabling to inform directly the monitoring system (see in section 3 Meeting evaluation goals and ambition).

Engineering estimates are appropriate to evaluate savings from specific actions or cross-cutting actions whose characteristics can vary significantly from one project to the other. They are then a cost-effective when case-by-case calculations are needed, or when a feasibility study or energy audit would be done anyway (e.g. to prepare an investment plan). When the engineering estimates are based on standardised calculation formulas, they can also be used to process data in a partial or fully automatized way, thereby minimizing administration costs (especially when the data needed for the formula can be easily collected and reported by the stakeholders).

However, in case the action types can be standardised (e.g. standardised lighting actions) and the number of actions can be very large, deemed savings can be a more cost-effective evaluation approach (especially if the savings per action or project remain small). See the corresponding Specific Guidance 28 .

This is why EEO schemes often make use of different types of calculation methods, depending on the type of energy saving action or project.

Direct measurements can also be required, especially for large projects in the industry (or large commercial buildings) (see e.g. the option of Monitoring and Measurement Plans in the Italian white certificates scheme).

In parallel, billing analysis (or other types of econometric analysis) is a common alternative when the evaluation objective is more specifically to verify the energy savings actually achieved or to assess the cost-effectiveness or efficiency of the scheme. See the corresponding Specific Guidance 29 .

The table below presents the pros and cons of these methods commonly used for evaluating energy savings from EEO scheme (see also in section 2 Additional methods to increase reliability of the results).

Type of method		Pros	Cons
Engineering estimates	Can be used for almost all action types Can enable to automatize energy savings calculations (through standardised formula for simple cases) Can reflect the energy savings achieved for a given situation (specific calculations) Limited delay in getting the results (calculations can be done before the actions are installed)		Require collecting data for each case (so can be costly if data collected only for this purpose and for large numbers of actions / projects) Possible gaps between engineering estimates and measured savings (see the corresponding topical case study) Additional method needed to evaluate ex-post the additionality of the savings (see section 4)
Deemed savings	Provide visibility to stakeholders No delay in getting results from the monitoring system Low running cost (once the catalogue is defined) Calculations directly related to the energy efficiency improvements due to the energy saving actions		Use limited to action types that can be described in a standardised way Do not reflect the energy savings achieved for a given situation, but an average result for a population of actions Can require significant preliminary efforts (if many action types to be included in the catalogue) Quality depending on the data available to define deemed savings Possible gaps between deemed savings and actual savings (see section 4) Additional method needed to evaluate ex-post the additionality of the savings (see section 4)
Direct measurements	Provide data about actual energy consumption (for the baseline and/or for the situation with energy saving actions) or about actual values for key parameters (e.g. power, duration of use) Can be used to assess performance gaps		Can be costly if measurements only done for this purpose and for large numbers of actions If sampling is used, attention should be paid to avoid sampling bias (if data are to be extrapolated) Additional method needed to evaluate ex-post the additionality of the savings (see section 4) Delay in installing the actions (if used to verify the baseline, then time needed to make the measurements, unless data are already available) Delay in getting the results (if used to verify the situation with energy saving actions, then time needed to make measurements after the actions are installed + time to analyse the data)
Billing analysis (see also the related Specific Guidance 29) and this article	Provide data about actual energy consumption / energy savings Can be used to evaluate ex-post net or additional savings (if a control or comparison group can be found)		Can only be used for ex-post evaluation Limited to cases where the expected energy savings are large enough (e.g. higher than 10% of the energy bills) to be distinguished from changes due to other factors. Frequent difficulties to collect billing data (unless data collection carefully planned and prepared in advance, e.g. collecting participants’ approval when actions are installed) Difficulties to get representative samples (cf. sampling bias + data losses along the evaluation process) Delays in getting the result (at least one year to get the consumption after installing actions + time to process and analyse data) Difficulties to find relevant control or comparison groups (when assessing net or additional savings)

7.1 Calculating avoided CO2 emissions

Depending on the priority objectives of the EEO scheme, scaled savings from engineering estimates can sometimes be expressed in CO2 savings (i.e. avoided CO2 emissions).

In practice, scaled savings are first calculated in terms of energy savings. Then avoided CO2 emissions can be evaluated from the energy savings by applying emission factors. Four key aspects are to be taken into account when choosing the emission factor(s):

1. Emission factors vary according to the energy type, so the deemed savings need to be defined per energy type.
2. Emission factors for a given type of energy can vary over time (especially for electricity).
3. Emission factors can take into account:
   1. Direct emission factors: that take into account the emissions generated when producing the energy used;
   2. Lifecycle emission factors: that take into account all the emissions generated from the extraction of the energy resources up to the dismantling of the energy plant.

Due to the differences that the choice of emission factor(s) can induce, it is important to document what emission factor(s) has (have) been used.

Emission factors used for the EEO scheme can for example be based on official national emission factors used for the national inventory of emissions of greenhouse gases.

The conversion of electricity savings into CO2 savings is however a special case, depending on the national mix for electricity production. Several choices are indeed possible, for example:

• Average emission factor, calculated from the total annual emissions from electricity production (possibly taking into account national imports and exports) divided by the annual amount of electricity consumed: this is a simple approach, but that might not reflect the fact that end-uses can have different times of use and thus correspond to different load profiles (while the emission factor for electricity can vary significantly between base load and peak load).
• Emission factors per type of end-use: this requires more sophisticated calculations (e.g. by decomposing the national load curves per type of end-use) that will be meant to use emission factors reflecting the differences in time of use (e.g. daily, seasonally).
• Specific emission factor: in case the industrial site has its own electricity generation capacities (e.g. CHP units). In that case, the definition of the emission factor needs to be explained by the stakeholders reporting the project.

The choice between the three options above can also depend on the national electricity mix (cf. emission factor varying significantly with time of production or not) and the type of end-uses covered by the EEO scheme.

If the engineering estimates cumulate energy savings over time, it can also be needed to define a scenario about the evolution of the national electricity mix over the period of calculation (e.g. taking into account the objectives of shares of electricity produced from renewable energy sources).

The avoided emission of other greenhouse gases due to energy savings are not addressed in this guidance. In most cases, these emissions (and more specifically their reductions) are generally negligible compared to CO2. However, depending on the industrial processes, other greenhouses gases can be relevant to consider.

When needed, IPCC (Intergovernmental Panel on Climate Change) provides a detailed database of peer-reviewed emission factors.

7.2 Calculating cost-effectiveness

Cost-effectiveness is the ratio between costs to achieve energy savings and the amount of savings and possibly other benefits.

A distinction can be made according to the point of view adopted to assess cost-effectiveness:

• Cost-effectiveness for the end-user or participant (e.g. payback time, net present value, internal rate of return)
• Cost-effectiveness from the obligated parties’ point of view (e.g. least cost of target achievement)
• Cost-effectiveness for society at large (e.g. social net present value)
• Cost-effectiveness from the point of view of the public authority (e.g. comparing different types of policy measures)

For more details about the different perspectives, see for example (Breitschopf et al., 2018).

In the case of an EEO scheme, the calculation of cost-effectiveness requires to collect other data on top of the ones used to calculate energy savings, as summarized in the table below.

Point of view	Costs	Benefits
Participants	Part of the investments paid by the participants	Financial aids received from obligated parties (or other intermediaries) Gross energy savings
Obligated parties	Costs to achieve their targets* Losses in revenues (due to decrease in sales related to the additional energy savings)	Costs recovered on network tariffs (if energy distributors) or energy prices (if energy suppliers) Costs of energy production (or distribution or purchase) avoided due to the additional energy savings
Public authorities	Administration costs Losses in tax revenues (due to additional energy savings)	Increases in tax revenues (due to additional investments made in energy efficiency actions)
Society	Part of the investments paid by the participants (for additional actions only) Costs of the obligated parties Administration costs for the public authorities	Additional energy savings

*: when possible, these costs can be disaggregated in sub-categories, especially to differentiate administration costs (e.g. costs of reporting to the public authority), communication & marketing costs (e.g. communication campaigns) and costs related to the technical or financial support provided to final customers (e.g. energy audits, grants). It can also be useful to identify costs related to quality processes and monitoring.

NOTE: the table above does not deal with non-energy impacts. Depending on the context and objectives of the EEO scheme, non-energy benefits can be larger than the benefits from energy savings. When assessing the cost-effectiveness of an EEO scheme from the society’s point of view, it is therefore recommended to consider if it is relevant to include non-energy impacts in the scope of analysis.

Experience has shown that the fact that an EEO scheme involves private actors, makes it often difficult to collect cost data, and particularly homogeneous cost data. When obligated parties report cost data, it is indeed common that they use different sub-categories of costs, or only their “total” costs (with a different scope of “total” costs from one obligated party to the other).

When possible, it can therefore be useful to consult with obligated parties to define clear categories of costs.

When the obligated parties are energy distributors, they usually have to report their total costs to the energy regulatory authority, so that they are allowed to recover their costs on network tariffs. This provides information about total costs for obligated parties. But it does not mean that costs per sub-categories will be available.

When the obligated parties are energy suppliers, there is usually no direct obligation for them to report cost data. This can be asked by the public authorities on a voluntary basis. Or it can be required by including a specific provision in the law enforcing the scheme (if enforced by law). However there could be limitations due to other legal provisions about confidentiality of strategic data for private actors.

Depending on the indicator(s) used to assess cost-effectiveness, it can be needed to use discount factors (e.g. when the indicator is Net Present Values). In that case, it is important to document the use of discount factors, and if possible to make a sensitivity analysis (testing several values or ranges of discount factors). As this can affect significantly the results.

Likewise, the calculations of cost-effectiveness indicators will usually require to consider scenarios of energy prices over given periods. The assumptions about trends in energy prices should be documented. Whenever possible, it is recommended to make a sensitivity analysis (testing several scenarios of energy prices).

For more discussions about cost-benefit analysis of EEO schemes in Europe, see for example (Rosenow and Bayer 2017).

7.3 Calculating other Co-benefits

Co-benefits (also often named multiple benefits) from saving energy can be for example:

• Extra employment
• Increased competitiveness of companies
• Other emission reductions (NOx, SO2, fine particles, etc.)
• Reduced dependency on (insecure) energy import

It should be noted that the impacts from EEO schemes on each of these aspects are usually positive, but can also be negative. Therefore, it is in general more appropriate to speak about non-energy impacts.

For a general background about non-energy impacts, see the corresponding EPATEE general guidance.

For the case of EEO scheme in the industry sector, a special attention is frequently paid to the impacts on competitiveness. For more information about multiple impacts for companies, see for example the M-Benefits project: https://www.mbenefits.eu/

In case of a transversal EEO scheme (covering several sectors), the evaluation objectives can also include assessing the possible cross-sectoral effects in terms of cross-subsidizing. Such effects can for example happen when it is more cost-effective for obligated parties to achieve their targets in a given sector, whereas costs will be recovered on energy prices in all sectors.

Three EPATEE case studies deal with the evaluation of EEO schemes. Two of these cases include the use of engineering estimates (mostly combined with other methods):

• Evaluation of the Danish EEO scheme:
  https://www.epatee-toolbox.eu/wp-content/uploads/2018/10/epatee_case_study_denmark_eeo_scheme_ok.pdf
• Evaluation of the Italian white certificates scheme:
  https://www.epatee-toolbox.eu/wp-content/uploads/2018/10/epatee_case_study_italy_white_certificates_ok.pdf

Examples of standardized calculation formulas can be found:

• From the Austrian EEO scheme (website of the monitoring body, in German):
  https://www.monitoringstelle.at/index.php?id=589#c1347
• From the Irish EEO scheme (calculation tools and examples, in English):
  https://www.seai.ie/resources/tools/
• From the Italian white certificates scheme (see the documents related to standardised projects “PS”, in Italian):
  https://www.gse.it/servizi-per-te/efficienza-energetica/certificati-bianchi/documenti

Examples of general calculation methodology can be found:

• From the Danish EEO scheme (see chapter 10 of the scheme agreement, English version):
  https://ens.dk/sites/ens.dk/files/Energibesparelser/energispareaftale_161216mbilag_6_eng.pdf
• From the French white certificates scheme (English version):
  https://www.ademe.fr/sites/default/files/assets/documents/87411_7736-cee-op-specifiques-2013-gb.pdf
  (updated version, available in French only: https://www.ademe.fr/guide-technique-certificats-deconomies-denergies-operations-specifiques-installations-fixes )

• General guidance on evaluations
  Hoffman, I., Schiller, S., Todd, A., Billingsley, M., Goldman, C., Schwartz, L., 2015. Energy Savings Lifetimes and Persistence: Practices, Issues and Data. Technical Brief, Lawrence Berkeley National Laboratory, May 2015. https://emp.lbl.gov/publications/energy-savings-lifetimes-and
  ISO 50046:2019. General methods for predicting energy savings. March 2019. https://www.iso.org/standard/67790.html
  Mort, D. (2017). Chapter 9: Metering CrossCutting Protocol. The Uniform Methods Project: Determining Energy Efficiency Savings for Specific Measures. Prepared for NREL (National Renewable Energy Laboratory), September 2017. https://www.nrel.gov/docs/fy17osti/68565.pdf
• About EEO schemes
  Bertoldi, P., Castellazzi, L., Oikonomou, V., Fawcett, T., Spyridaki, N. A., Renders, N., & Moorkens, I. (2015). How is article 7 of the Energy Efficiency Directive being implemented? An analysis of national energy efficiency obligation schemes. Proceedings of the ECEEE 2015 Summer Study, paper 2-380-15, 455-466. https://www.eceee.org/library/conference_proceedings/eceee_Summer_Studies
  Broc, 2017. Snapshot of Energy Efficiency Obligations schemes in Europe: 2017 update. Report for ATEE and the Fourth European Workshop Meeting of the White Certificates Club, 30 June 2017. http://atee.fr/sites/default/files/part_6_2017_snapshot_of_eeos_in_europe.pdf
  ENSPOL (2015). Report on existing and planned EEOSs in the EU (Part I: Evaluation of existing schemes ; Part II: Description of planned schemes). Deliverable 2.1.1 of the ENSPOL project, March 2015. http://enspol.eu/results
  IEA (2017). Market-based Instruments for Energy Efficiency – Policy Choice and Design. International Energy Agency’s Insights Series 2017. https://webstore.iea.org/insights-series-2017-market-based-instruments-for-energy-efficiency
  RAP (2012). Best practices in designing and implementing energy efficiency obligation schemes. Report for the Task XXII of the IEA-DSM (International Energy Agency Demand Side Management) Programme https://www.raponline.org/knowledge-center/best-practices-in-designing-and-implementing-energy-efficiency-obligation-schemes/
• References related to other aspects of evaluating EEO schemes
  Giraudet, L. G., & Finon, D. (2015). European experiences with white certificate obligations: A critical review of existing evaluations. Economics of Energy & Environmental Policy, 4(1), 113-130. https://hal.archives-ouvertes.fr/hal-01016110/document
  Rosenow, J. & Bayer, E. (2017). Costs and benefits of energy efficiency obligations: a review of European programmes. Proceedings of the ECEEE 2017 Summer Study. Paper 2-011-17, 249-259. https://www.eceee.org/library/conference_proceedings/eceee_Summer_Studies

Example of using econometric methods for billing analysis (not applied to an EEO scheme, but to another type of policy measure covering industrial and commercial companies):

• CAG Consultants, Carbon Trust, Databuild, Imperial College Business School, 2015. CRC Energy Efficiency Scheme Evaluation. Final report for the DECC, July 2015. https://www.gov.uk/government/publications/evaluation-of-the-crc-energy-efficiency-scheme