14 – Evaluating Financial incentives on energy efficiency for Household appliances using the Deemed savings method

The primary audience for this guide concerns energy efficiency programme designers, implementers or supervisors, and evaluators looking for guidance on the evaluation process of energy savings in the scope of this guide.

Although the application of this guide will generally concern the national level, account will be taken of issues at EU level when relevant (e.g. the specific format of saving figures for the EED).

This guide is not about the preceding step in the evaluation process, the choice of the method. About this previous step in the evaluation process, see the guidance provided here. However, after presenting the capabilities and limitations of the guide at hand, the user will be offered alternatives for the method within this guide (see section 6).

The objective of this guide is to provide:

• Information on the scope of the guide that enables the user to decide whether this guide is suited to his/her needs, and whether complementary or additional method(s) could be needed or useful (section 2);
• Guidance about specifying the evaluation objectives and requirements (section 3);
• Guidance about key methodological choices to calculate energy savings (section 4);
• Guidance about the inputs (data requirements) and outputs of the method (energy savings metrics) (section 5);
• Possible alternative methods (with pros and cons) (section 6)
• Background about evaluation results other than energy savings (section 7);
• Relevant examples, case studies and/or good practices (section 8);
• Relevant references for further reading (section 9).

The guide is intended for assessing realised (ex-post) energy savings. However, account is taken of earlier (ex-ante) evaluations of expected savings, if available (see section 4).

The focus of the guide is on impact evaluation, i.e. determining the energy savings, but not on how this has been reached through a step by step process with intermediate results (process evaluation).

Readers looking for the basic and general principles of energy efficiency evaluation may find the following link useful.

2.1 About financial incentives

More information and examples on the different subtypes residing under the main type Grants & Subsidies (GS) can be found here and here. The focus of this guide is restricted to household appliances.

Financial incentives are restricted to subsidies for efficient electrical appliances that should make it more attractive to buy more efficient products. An important element in the subsidy schemes is the target group, which normally consists of all consumers or of specific segments of consumer groups based on income criteria (as for instance subsidies for low income households or households on energy poverty).

The subsidies are often combined with efficiency standards in the form of energy labels (sometimes indicated as Minimum Energy Performance Standards – MEPS). MEPS for appliances can be considered specific standards as they are set specifically for a type (or even sub-type) of appliance (e.g. refrigerator, dishwasher, boiler). The evaluation should focus on separating the effect of the subsidy from the autonomous development: part of the increased share of more efficient appliances will happen due to other developments than the subsidy, for example the growing availability of more efficient appliances.

Information on incentives like subsidies can be found here.

2.2 Evaluation for a combination of policy measure types

In practice, subsidies are most often implemented jointly with energy labelling schemes (type of policy: legislative/information, such as theEnergy Labelling regulations) to help the replacement of the most inefficient appliances in the stock. When subsidies are combined with other policy measures types it is assumed that the overall savings are mainly resulting from the policy measure at hand. Likewise, when subsidies for low income households (or energy poverty groups) are combined with other social policies, overlaps can take place. However, the evaluation concerns the combined savings effect of both policy measures. The guide is not capable of attributing part of the (overall) calculated savings to each of the policy measures (see also Double counting in the section on Gross to Net savings). A practical approach for combined policy measure types is to perform the evaluation for the set of policy measures as such, or to attribute the savings to the policy measure deemed to have the strongest effect.

Overlaps (and complementarities) can also occur with other types of policy measures, such as:

• Minimum Energy Performance Standards (MEPS)
• Taxes on electricity prices (see below)
• Programmes for energy advice for households
• Other voluntary labelling schemes

2.3 Evaluation when combined with energy taxes

The calculated savings effect of subsidies can overlap with that of the energy tax (in practice mostly electricity tax). Increasing the energy prices makes appliances with higher energy efficiency more attractive as their lifecycle cost decreases (compared to less efficient appliances). Both instruments (subsidies and energy taxes) thus act in the same direction: encouraging customers to buy more efficient appliances. Hence the overlap when evaluating the respective effects. Special care is required when dealing with energy taxes and subsidies for low income households.

The guide is not capable of attributing part of the (overall) calculated savings to either subsidies or the energy tax. For dealing with this overlap see section on Calculating Gross and Net savings.

One practical way to avoid overlap in the energy savings counted for subsidies and energy taxes can thus be to consider that the savings from buying more efficient appliances are attributed to subsidies. Whereas energy savings from an efficient use of the appliances are attributed to energy taxes (that can for example counteract possible rebound effects). In practice, this can be done by calculating first the savings from subsidies, and then deduct them from the total savings calculated for the energy taxes.

2.4 About household appliances

Household appliances are electrical appliances that are not fixed to the building, and not meant as the standard heating option. The appliances covered are washing machines, driers, fridges, TV and radio, ICT etc.

The list of household appliances and devices (with sub-types when relevant), eligible for subsidies, are normally the ones covered by EcoDesign regulations is (as of May 2019): Lighting; Heaters (Local space heaters, Space and water heaters, Solid fuel boilers, Air heating and cooling products); Fridges and freezers; Vacuum cleaners; Washing machines; Tumble driers; Air conditioners and comfort fans; Ventilation units; Air heating and cooling products; Televisions; Set-top boxes; Cooking appliances; Dishwashers; Pumps (Water pumps, Circulators); Computers; Game consoles; Off mode, standby and networked standby. For more details and updates about the list of energy-using products covered by the EcoDesign and Energy Labelling regulations, see here.

Information on (sub)sectors defined in the Toolbox can be found here, chapter 2, p.17

2.5 Evaluation for cross-sector saving actions

This guide is also applicable to evaluate cross-sector savings, e.g. more efficient PCs in offices, provided that the data needed are available for all relevant sectors.

2.6 About deemed savings

The Energy Efficiency Directive (2018(2002), Annex V(1.a)) defines “deemed savings” as “results of previous independently monitored energy improvements in similar installations”. They are also often called ex-ante or stipulated savings.

Deemed savings, in the case of subsidies on electrical appliances, are an estimate of the energy saved for one appliance type. To determine the total savings due to the subsidy scheme, the deemed savings are multiplied by the total amount of efficient appliances (see Complementary methods), and also by the fraction of efficient appliances bought because of the subsidy scheme (see chapter 4).
Information about the various evaluation methods can be found here, table 1 and 2. This source also covers the combination of the method at hand with other methods, which will be dealt with below.

2.7 Complementary methods to determine total savings

Complementary methods are methods that are required, in addition to the primary selected method, to calculate energy savings.

The method at hand is meant to calculate unitary savings for one appliance. These unitary savings should be multiplied by the number of saving actions or participants in order to get the calculated total savings. The number of actions, i.e. more efficient appliances purchased, can be obtained in various ways. See this link, table 2 and 3. The number of actions (in this case efficient appliances bought) is sometimes directly available from the monitoring of policy measures, such as in case of a subsidy scheme.

For more information about methods to calculate unitary savings, see this link, table 2.

Complementarities are found in the diffusion indicators method (Specific Guidance 7 ), which concerns the determination of the number of actions or participants (more specifically here the number of appliances).

2.8 Additional methods to increase reliability of the results

An additional method can be applied on top of deemed savings to improve the reliability of the evaluation results and/or the cost-effectiveness of the evaluation approach.

As deemed savings are estimates based on expert judgement, laboratory research or other sources, their reliability is unknown. It depends on the quality of the research and whether the data and assumptions used to calculate the deemed savings represent the actions and their conditions of implementation well. Measurement of savings (direct measurement) for a sample group of the saving actions can be an additional method to check the deemed savings. For more guidance about sampling issues and methods, see for example (Khawaja et al 2017). While direct measurement will usually have a high accuracy to determine the electricity consumption of the appliances measured or tested, it can have limitations or uncertainties in estimating the total savings. For example, due to difficulties in defining a relevant baseline situation or in estimating representative results as mentioned above. The methods “stock modelling” or “diffusion indicators” can then be relevant additional methods to provide more reliable estimates about the baseline situation and the number of actions.

Surveys can also be used as an additional method to investigate factors influencing the energy consumption of appliances, such as user behaviours (see for example Cabrera et al. 2015).

Combining methods can increase the reliability of the savings figures and is more cost-effective than measurement for all dwellings.

For possible combinations with an additional method see chapter 6 in this link.

3.1 Meeting evaluation goals and ambition

The table shows whether this guide can be used to report on general evaluation goals or criteria. See also this document.

General types of evaluation goals or criteria	Level of ambition	Remarks
Calculation of realized energy savings from saving actions	medium	Depending on the complementary method used to estimate unitary energy consumption or savings, and the normalization factors taken into account: Low in case of diffusion indicators or standardized energy consumption from energy label without any normalization factor.
Calculation of energy savings attributed to the policy measure(s)	low	Hard to estimate due to  free riders
Cost-effectiveness of saving action (for end-users)	medium
Cost-effectiveness of policy (government spending)	low	Due to  free riders
CO2-emission reduction from saving actions	medium	CO2 per kWh known
CO2-emission reduction attributed to the policy measure(s)	low	Due to free riders

The evaluation objectives depend mostly on the needs and priorities of policy makers (e.g. public authorities in charge of designing, revising subsidy schemes). Experience subsidy schemes shows that two of the most frequent key evaluation questions are: 1) how well the policies are implemented and received (process evaluation); and 2) what have the policies achieved (impact evaluation) (Corry Smith et al. 2016). This guidance is focused on impact evaluation, and more specifically the evaluation of energy savings.

For more information on verification of actual energy savings and attribution/baseline/corrections, see section 4, and for cost-effectiveness and emission reduction see section 7.

3.2 Reporting expectations

The following methods will make it possible to report (net) savings of subsidies for household appliances. In practice, the deemed savings method is usually associated with a very low effort (cf. costs related to the direct measurement).

• Gross and Net savings
• Yearly or cumulative savings
• Market transformation due to subsidy scheme (only supply of more efficient appliances).

3.3 Time frame for evaluation

The length of the period under evaluation is dependent on the active period of the policy measure, the need to monitor developments before the implementation of savings actions (in case of methods based on before/after saving actions), and the time needed to present (reliable enough) results or impacts that fit into the decision making process. In some cases, the periodicity of evaluation can be set by law.

The planning of evaluation activities concerns regular monitoring of energy consumption and factors that define consumption, intermediate check of (ex-ante) estimated (unitary) savings through measuring or surveys, intermediate evaluations to improve the policy implementation and the final evaluation and reporting.

The use of deemed savings can be complemented by other methods to verify their reliability, and possibly update them (see Additional methods to increase reliability of the results in section 2). These additional methods have different timeframes. It is thus recommended to consider how the combination of evaluation activities should be planned. Especially to ensure the feasibility of the corresponding data collection and to optimize the use of resources (time and budget). See also planning of evaluation in the link here.

3.4 Expertise needed for chosen method

Knowledge about purchasing behaviour is needed to set up and evaluate subsidy schemes for appliances, especially the size of the free rider effect. Substantial knowledge of the policy background for energy efficiency in appliances (e.g. about the Ecodesign and energy labelling regulations) is a prerequisite to analyse the impacts of MEPS and possible interactions with other policy measures, in order to unveil the additional effect of subsidies.

3.5 Boundaries for the evaluation

The scope of subsidy schemes for electrical appliances is national and can be restricted to specific types of appliances. Appliances (innovative) that do not fall under the deemed savings technology lists will need to be estimated separately.

This section deals with key methodological choices to be considered when calculating energy savings: consistency between ex-ante and ex-post evaluation, baseline, normalization and adjustment factors. These choices are important to document when reporting energy savings, to ensure the transparency of the results.

General principles of calculating realized savings using different methods can be found here and here.

The deemed savings method estimates the energy saved per type of appliance depending on the energy label and the hours of use per appliance type. To estimate the effect of the policy, it also needs to take into account which part of the sales of more efficient appliances is due to the policy. Depending on the baseline option chosen, adjustment factors might be needed to assess net or additional savings.

4.1 Matching method with earlier ex-ante evaluation

From the viewpoint of methodological consistency and data availability using the same method in the ex-ante evaluation and in this guide on ex-post evaluation might be an obvious choice. For the guide at hand the ex-ante evaluation method deemed savings is indeed available.

The ex-post method deemed savings can be combined with a different method for the ex-ante evaluation, such as engineering estimate or stock modelling, depending on the evaluation objectives, timeframe and data available for the situation after implementing the actions. For possible combinations of methods applied ex-ante and ex-post, see chapter 7 in this document.

In practice, ex-ante and ex-post evaluations are applied consecutively. The ex-post evaluation builds on an ex-ante evaluation that makes use of data coming from previous ex-post evaluation or studies (e.g. about previous periods of the same policy measure, or about the same types of energy saving actions as the ones promoted by the new policy measure). These previous ex-post studies could have used another type of method as well.

When different methods are used for the ex-ante and ex-post evaluations, comparisons should be done with caution. Especially, differences in the baselines, data used and factors taken into account should be analysed.

4.2 Calculation baselines

Energy savings are defined in general as the difference between the actual situation and a reference situation without the saving actions (and without the policy measures that influence these saving actions, taking into account also the free-rider effects). In case of saving actions the reference situation can be defined using various calculation baselines: Before/after, With/without, Trend, Target/control group and Minimum efficiency standards, see further here.

Total savings are found by multiplying the unitary savings per action by the number of actions. The chosen baseline can define the total savings through the unitary savings, but also through the number of actions.

The applicable baseline in this case is “With/without” (subsidies for efficient appliances). The definition of a baseline scenario often requires stock modelling or similar analysis, taking into account the existing stock of appliances, replacement rates or patterns, and trends in market shares (per energy class).

The results of the baseline scenario used to calculate the energy savings (to understand in essence which part of the purchase due to subsidy is autonomous) will usually be:

• Either directly a time series of average unitary energy consumption per type of appliance for the appliances sold each year included in the time series.
• Or a time series with the estimated market shares per energy class for each year included in the time series.

Baseline or business-as-usual scenarios are commonly used for ex-ante evaluations (for example for impact assessments). Using the same baseline scenario for the ex-post evaluation help ensuring consistency. However, the baseline scenario can also be adapted for the ex-post evaluation, for example to take into account unforeseen changes in the context (e.g. economic crisis) or improvements in data about appliances’ energy consumption.

4.3 Normalization factors

The calculation with baseline With/without provides a change in energy consumption that should be corrected for influences on energy consumption other than the saving actions. When evaluating energy savings from household appliances, these so-called normalization factors can be weather (with effect on consumption, depending on the type of appliance), the rebound effect and other changes in energy behaviours related to the appliances under evaluation. All normalization factors affect total savings through the unitary savings.

For the guide at hand the normalization factors can concern the rebound effect, see table 1 here.

The rebound effect, the extra energy consumption of more intense use of the appliance or buying a larger appliance, is due to lower running costs of the appliance. When evaluating policies related to appliances, a type of rebound effects other than the intensity of use might need to be considered: rebound effects related to the size or number of appliances. For example, the fact that the new appliance consumes less energy can make that households will buy larger or more sophisticated appliances, or will have more appliances of the same type (for example, keeping the old refrigerator as secondary refrigerator). These effects can be challenging to assess, as they can be due to other reasons than the MEPS, for example due to an increase of the household income, or to the marketing of the manufacturers and retailers encouraging households to buy larger appliances. The evaluation of these market transformation effects or trends goes beyond the scope of this guidance.

For a more general discussion about normalization factors see this link on normalization and adjustment factors, gross and net savings, etc. and in this link or here.

4.4 Adjustment factors

Adjustment factors define which part of the calculated energy savings can be attributed to a policy measure or meets the definition of savings specified in the evaluation objectives or reporting requirements (see next section on “Calculating Gross and Net savings).

Adjustment factors can concern the Free rider effect, the Spill-over/multiplier effect, Additionality and Non-compliance (see also this link, table 1).

Additionality and non-compliance are connected to unitary savings, while free riders and spill-over work through the number of actions.

Double counting might be relevant in case of another policy focusing on the same saving actions as evaluated with the specific guidance at hand. Double counting can only be accounted for at a higher level than individual specific guidances; see Distinction of energy efficiency improvement measures by type of appropriate evaluation method, here and here.

See also link to note “Saving calculation methods and their application in the Epatee Toolbox”.

The adjustment factors to consider in priority are for the free rider effect, spill-over and non-compliance. For free riders a distinction must be made between saving actions due to the policy measure and actions which would have been taken anyway.

The deemed savings method does not provide this information directly, thus other ways must be found, such as a survey among participants to the policy measure about their motivation, or application of Randomized Controlled Trial (RCT) or Quasi-experimental design (see further in this topical case study).

In order to correct for non-compliance due to inappropriate implementation of saving actions, data should be available of sample-wise checks on the implementation.

4.5 Calculating Gross and Net energy savings

Gross savings concern the calculated savings from saving actions using a chosen baseline and normalization factors. Net savings concern the savings attributed to the subsidies.

When calculating savings a distinction must be made for unitary savings times the number of actions; see here.

Gross unitary savings can be calculated using the deemed savings and correcting for the normalization factor (here the rebound effect). The number of actions is the number of subsidized appliances.

Net unitary savings can be calculated from gross unitary savings by applying the adjustment factor non-compliance (see previous section). The net number of actions follows from the gross number, corrected for the adjustment factors free riders and spill-over. See here and here.

The savings should be corrected for the Double counting effect, i.e. the overlap between the savings due to subsidies and savings due to other policy measures. The overlap in the calculated savings of both policy measures cannot be processed at the level of a guide but must be corrected at the level of savings due to overall policy portfolios. For addressing double counting see here or here.

See also chapter 8 on Concrete examples.

5.1 Main data requirements, sources and collection techniques

Data requirements specified in the table below correspond to the calculation of energy savings, when using the baseline option With/without.

Calculation subject	Data requirements	Possible data sources and collection techniques
Unitary Energy consumption	Energy consumption of appliances per label class Long enough measurement periods (for more details about measurement requirements, see for example Mort 2017) Representativeness of the samples used for the measurements (for more details about sampling issues, see for example Khawaja et al. 2017)	Data from manufacturers Data from countries that use Deemed Savings for such appliances
Normalization factors affecting energy consumption	Rebound effect Identifying the data or indicators that can be used to apply normalization factors (e.g. for weather conditions) Representativeness of the data collected	Survey on behaviour when using more efficient appliances Literature research about rebound effects for specific technologies
Adjustment factors –          Free riders –          Non-compliance	Fraction of free riders; degree of non-compliance	Free riders: surveys; non-compliance: measurements
Primary energy factors applied (for conversion from final to primary savings)	Primary energy factor for electricity	National statistics
Number of actions	Number of appliances sold with subsidy Market (and stock, when relevant) data at least disaggregated per energy class	Data from the subsidy scheme Sales data, stock modelling, market surveys or analysis
Operating hours	Hours of usage per appliance	Surveys

Data issues when evaluating net energy savings

The main good practice to ensure the feasibility and reliability of the evaluation of net energy savings is to think about the method to be used when designing (or revising) the subsidy scheme.

Experience indeed shows that unless the data collection has been planned in advance, it will be very costly, time-consuming or even impossible to collect the data required to apply most of the methods that can be used to evaluate net energy savings. Which makes that in practice, using surveys will remain the only option possible (or considered feasible).

The main challenges when using surveys are:

• to achieve a high answer rate, in order to limit sampling bias;
• to use question phrasing that can limit the risk of bias in the answers.

For more details about the evaluation of net energy savings, see this topical case study.

When dealing with subsidies, one approach to evaluate net savings can be to use as baseline a scenario that is assumed to represent how the market would have evolved in the absence of the subsidies (e.g. by extrapolating previous market trends). This usually requires detailed stock modelling and market analysis, which can be challenging. Especially if disaggregated data (at least per energy class) about appliances in the stock and sales on the market are not easily available. Disaggregated sales data are often costly to acquire.

The classical data sources for sales or market data (mostly based on market surveys) have a time lag of one year or more. New data collection techniques based on data available from the web (e.g. manufacturers’ and retailers’ websites, web-market places) can have much shorter delays, even close to real-time monitoring. For more details, see for example (Enervee 2014; Bennich et al. 2017).

With the number of efficient appliances sold and their energy saved compared to the baseline with minimum efficiency standards, the gross savings can be calculated. To get to the net savings, that is savings that can be attributed to the subsidy, data on free-ridership are needed.

Data issues with the additional method

For possible other methods with different data demands see the section on alternatives for the chosen method.

5.2 Energy savings in final terms or in primary terms

Energy savings can be expressed in final terms or in primary terms. See definitions about primary and final energy here.

The deemed savings method can calculate savings in final energy terms. It can also calculate savings in primary terms provided that savings at end-users are calculated for each energy carrier apart, and primary factors are available to convert the savings in final terms to savings in primary terms.

When dealing with energy savings from appliances, energy savings will usually be calculated separately for each type of appliances, therefore easily providing energy savings per type of energy carrier.

Most of the households appliances use electricity. If the energy savings calculated over the lifetime of the appliances (see below), then it is important to clarify if the primary factor used for electricity in the calculations takes into account the likely changes in the electricity mix over the period (e.g. due to the increase in the share of renewable energy sources), or if the current primary factor is used for all years.

5.3 Energy savings over time

Implemented saving actions in a year lead to savings over a number of consecutive years. E.g. a more efficient boiler can save gas over its lifetime of about 15 years and more efficient computers up to 5 years. Energy savings can be calculated in different metrics in terms of time reference, for example: year-to-year, annual, cumulated annual, cumulative. See the definitions here.

The calculated yearly savings concern the savings of all new saving actions in that year. In this approach only data for the savings in the chosen year are needed.

Adding up the yearly savings over a period, provided that earlier saving actions are still delivering savings, leads to cumulative savings. For the cumulative savings data are needed for the whole period.

Another cumulative approach, to be applied for the EED, is to multiply the (new) savings in a year with the number of years up to a target year and sum this result with that for all other years up to the target year. This cumulative approach stimulates early saving actions, as these count more times to the target than later actions. However, it should be noted that energy savings from appliances bought due to subsidy which fall under the Ecodesign regulations are not eligible to EED article 7. Only savings above the minimum energy performance levels set in Ecodesign regulations can be eligible (hence appliances that are bought due to a subsidy that are above the Ecodesign).

Finally, savings from a saving action can be discounted and summed up over the lifetime of the action See link here.

The deemed savings method can provide yearly savings of new saving actions in that year. It can also provide cumulative savings provided that data are available over a period.

Cumulative savings according to the Energy Efficiency Directive can be provided when the lifetimes of savings actions are known, and when correcting for the existing EU standard of Ecodesign, as only the additional savings may be counted.

6.1 Alternatives for the chosen method

Often other savings calculation methods can be applied as well, although they will all have pros and cons regarding various aspects dealt with in preceding sections.

The table below presents the pros and cons of the chosen method for evaluating subsidies for household appliances, and for commonly used alternative methods for the same combination of policy measure and sector.

Type of method	Pros	Cons
Deemed savings	Relatively easy, less corrections needed	Not very precise with regard to unitary savings (which old versus which new?) and with regard to the number of more efficient appliances
Stock modelling	Gives a complete picture of the composition of the appliance stock with regard to efficiency in any year	Needs more data
Direct Measurement	High accuracy for the data on energy consumption Data that reflects actual use of the appliances (if on-site measurements; not for laboratory tests) Possibility to identify non-compliance (especially for laboratory tests)	Expensive and time-consuming Difficult to obtain data that are representative of the whole market or stock of appliances. Complementary methods needed to assess the number of actions (and market shares at least disaggregated per energy class)

Stock modelling

The stock model shows the composition of the stock of appliances, with respect to characteristics such as energy efficiency, over time. Each year a number of appliance owners (participants) replace their old appliances with certain electricity consumption for new appliances belonging to different efficiency classes. A set of (deemed) unitary savings figures is defined for the different old-to-new replacements. The choice for an appliance from a certain efficiency class will depend on e.g. a financial incentive, such as a subsidy.

For past years, the total number and energy label class of appliances are based on observed yearly data about new appliances and discarded appliances (per type). For future years, the changes in the appliance stock are based on scenario data, such as population data, incomes, household composition and purchasing behaviour.

To evaluate the effect of appliance subsidies, a reference situation is constructed in which the subsidy measures are not present. The result is a change in the number of appliances with different efficiency levels, which translates into numbers for all old-to-new replacements. Combined with the set of deemed savings per action, this results in the total ex-ante savings.

In the ex-ante evaluation of subsidies, another stock is constructed including the effects of the subsidy. The result is a larger change in the number of more efficient appliances, and thus more saving actions and/or larger deemed savings per action. Comparing this to the reference savings leads to the total ex-ante savings.

In the ex-post evaluation of the purchase subsidies, the actual stock is based on observed data. Comparison with the reference situation provides the actual number of saving actions. Combined with deemed savings per action this results in the actual number of actions and the total ex-post savings.

7.1 Calculating avoided CO2 emissions

Avoided CO2 emissions can be evaluated from the energy savings by applying emission factors. Four key aspects are to be taken into account when choosing the emission factor(s):

1. Emission factors vary according to the energy type, so the data about energy savings need to be available per energy type.
2. Emission factors for a given type of energy can vary over time (especially for electricity).
3. Emission factors can take into account:
   1. Direct emission factors: that take into account the emissions generated when producing the energy used;
   2. Lifecycle emission factors: that take into account all the emissions generated from the extraction of the energy resources up to the dismantling of the energy plant.

Due to the differences that the choice of emission factor(s) can induce, it is important to document what emission factor(s) has(have) been used.

It is important to document what emission factor is used for electricity, as there can be different ways to calculate this emission factor. For example, averaging the emissions over the whole annual electricity production, or taking into account that the electricity mix can be different according to the time of the day or the season, due to differences in the load curve and availability of capacities per type of electricity source. This latter approach can be used to define emission factors specific to each end-use, calculating the emission factor based on the load curve of the end-use.

The reduction in CO2-emissions can be calculated from the savings with an emission factor for electricity that takes into account the different inputs of power production. The actual factor to be applied can vary, depending on saving action(s) and sector, year of implementation, policy considerations and etcetera, see for example here.

The avoided emission of other greenhouse gases due to energy savings are not taken into account here, as these emissions (and more specifically their reductions) are generally negligible compared to CO2 (apart from policy measures targeting the agriculture sector).

IPCC (Intergovernmental Panel on Climate Change) provides a detailed database of peer-reviewed emission factors.

7.2 Calculating cost-effectiveness

Cost-effectiveness is the ratio between costs to achieve energy savings and the amount of savings and possibly other benefits.

A distinction can be made according to the point of view adopted to assess cost-effectiveness:

• Cost-effectiveness for the end-user or participant
• Cost-effectiveness for society at large
• Cost-effectiveness for the party that takes responsibility for saving targets (government or actor with an Energy Efficiency Obligation)

See further example (Breitschopf et al., 2018).

The calculation of cost-effectiveness for end-users demands, apart from the savings, data on:

• investments made, subsidies on investments, interest rates, lifetimes of the saving actions per type of appliance
• energy prices (including taxes) and discount factors per type of end-user.

In the case of electric appliances, the purchase price after subsidy can be compared with the expected avoided costs of electricity.

For cost-effectiveness from a societal viewpoint no account is taken of subsidies and taxes, energy prices concern world market price, and a lower value of the discount factors is valid. The subsidies in this case can be compared with the avoided costs of imported fuels for electricity generation.

7.3 Calculating other co-benefits

Possible co-benefits from saving energy concern:

• Extra employment
• Reduction of energy poverty
• Other emission reductions (NOx, SO2, fine particles, etc.)
• Better indoor climate
• Reduced dependency on (insecure) energy import
• Social inclusion of energy poor groups

The following co-benefits are regarded as most relevant and/or feasible to calculate in conjunction to this guide (see table)

The calculation of the co-benefits is explained in the references.

Type of co-benefits	Why it can be relevant (and for whom)	References where more details can be found
Reduced energy poverty	Relevant for poorer households	Literature review on social welfare, energy poverty and health effects (COMBI project) Energy Efficiency for low income households http://www.europarl.europa.eu/RegData/etudes/STUD/2016/595339/IPOL_STU(2016)595339_EN.pdf

7.4 Other aspects of importance

No other aspects need to be addressed.

A study by VHK describes the effect on residential energy consumption as a result of the introduction of energy labels related to the Ecodesign directive. It is thanks to more efficient appliances for cleaning, cooling and lighting that electricity consumption went down despite an increase in ownership levels. See Further reading for the reference.

About the deemed savings method:

• Bennich, P., Stengard, L. , Christensen, S.F., Hartikainen, T., Mogensen, K. & Fjordbak Larsen, T. (2017). Using webcrawler techniques for improved market surveillance – new possibilities for compliance and energy policy. Proceedings of the ECEEE 2017 Summer Study, 1633-1642. https://www.eceee.org/library/conference_proceedings/eceee_Summer_Studies/2017/7-appliances-products-lighting-and-ict/using-webcrawler-techniques-for-improved-market-surveillance-new-possibilities-for-compliance-and-energy-policy/
• About surveys and measurement campaigns:
  Cabrera, D., Bertholet, J.L. & Lachal, B. (2015). Survey of behaviour usage of refrigerators, light bulbs and stand-by in households. Proceedings of EEDAL (Energy Efficiency in Domestic Appliances and Lighting) 2015, 756-770. https://e3p.jrc.ec.europa.eu/publications/proceedings-8th-international-conference-energy-efficiency-domestic-appliances-and
• Enervee (2014). Recent and Historical Product Energy Efficiency and Life-cycle Cost Improvement in Swedish Appliance Markets. January 2014. https://superefficient.org/publications/recent-and-historical-product-energy-efficiency-and-life-cycle-cost-improvement-in-swedish-appliance-markets
• Guidance about sampling issues and methods:
  Khawaja, S., Rushton, J., Keeling, J. (2017). Chapter 11: Sample Design Cross-Cutting Protocol. The Uniform Methods Project: Determining Energy Efficiency Savings for Specific Measures. Prepared for NREL (National Renewable Energy Laboratory), September 2017. https://www.nrel.gov/docs/fy17osti/68567.pdf
• About measurement requirements: Mort, D. (2017). Chapter 9: Metering Cross Cutting Protocol. The Uniform Methods Project: Determining Energy Efficiency Savings for Specific Measures. Prepared for NREL (National Renewable Energy Laboratory), September 2017. https://www.nrel.gov/docs/fy17osti/68565.pdf

Evaluation questions:

• Corry Smith, J. (2016). Ex-Post Impact Evaluations of Appliance Standards and Labelling Programmes: A Global Review of Best Practices and Lessons Learned. Proceedings of IEPPEC 2016. https://storage.googleapis.com/clasp-siteattachments/IEPPEC-Paper_Ex-post-impact-evaluations_Final.pdf

Relevant case studies:

• VHK 2008: Elektrische apparatuur in Nederlandse huishoudens (Electric appliances in Dutch households (Dutch only))