07 – Evaluating energy savings from mandatory energy labelling for household appliances using diffusion indicators
This guide can be applied to evaluate the savings due to mandatory energy labelling for household appliances using diffusion indicators. It includes guidance and explanations specific to this combination of types of policy measure, sector and method. As well as links to general guidance and explanations that can also apply to this combination.
1. USE OF THE GUIDE – AUDIENCE, OBJECTIVES AND FOCUS
The primary audience for this guide is energy efficiency programme designers, implementers or supervisors, and evaluators looking for guidance on the evaluation process of energy savings in the scope of this guide.
Although the application of the guide will generally concern the national level, account will be taken of issues at EU level when relevant (e.g. issues related to the European policy framework for appliances or to the eligibility of savings to EED article 7).
This guide is not about the preceding step in the evaluation process, the choice of the method. About this previous step in the evaluation process, see the guidance provided here. However, after presenting the capabilities and limitations of the guide at hand, the user will be offered alternatives for the method within this guide (see section 6).
The objective of this guide is to provide:
- Information on the scope of the guide that enables the user to decide whether this guide is suited to his/her needs, and whether complementary or additional method(s) could be needed or useful (section 2);
- Guidance about specifying the evaluation objectives and requirements (section 3);
- Guidance about key methodological choices to calculate energy savings (section 4);
- Guidance about the inputs (data requirements) and outputs of the method (energy savings metrics) (section 5);
- Possible alternative methods (with pros and cons) (section 6)
- Background about evaluation results other than energy savings (section 7);
- Relevant examples, case studies and/or good practices (section 8);
- Relevant references for further reading (section 9).
The guide is intended for assessing realised (ex-post) energy savings. However, account is taken of earlier (ex-ante) evaluations of expected savings, if available (see section 4).
The focus of the guide is on impact evaluation, i.e. determining the energy savings, but not on how this has been reached through a step by step process with intermediate results (process evaluation).
Readers looking for the basic and general principles of energy efficiency evaluation may find the following link useful.
2. SCOPE OF THE GUIDE – POLICY, SECTOR and METHOD
2.1 About mandatory energy labelling
The Ecodesign Directive (2009/125/EC) and the Energy Labelling Regulation (2017/1369/EU, earlier Energy Labelling Directive 2010/30/EU), set requirements in terms of minimum energy performance and energy labelling for energy-related products. The Energy Labelling regulation has already witnessed a few transformations, with a label reform underway for 2021 onwards; the labels will be rescaled to a simpler A through G, by eliminating the A+ signs.
For more details about the European policy framework for energy-related products, see the dedicated pages on the European Commission’s website.
The implementation of the Ecodesign and energy labelling regulations was expected to decrease EU energy use by 10%, achieving around 100 Mtoe of electricity savings and 66 Mtoe of direct fuel savings in 2020 (Kemna, 2014). But this target can only be achieved if the implementation is truly effective.
One of such successful market surveillance schemes which was put in place to ensure an effective enforcement of is called Nordsyn. For more details, see (Blomqvist & Fjordbak Larsen, 2015) and the corresponding EPATEE case studies (Dragovic and Broc 2018).
Another example can be found in the verification testing done by the Singapore National Environment Agency (NEA, 2015).
International reviews of energy labelling schemes for appliances can for example be found in (Hirayama et al., 2008; IEA 4E 2016). More guidance about labelling schemes can also be found in the guidebook coordinated by Wiel & McMahon (2005).
The focus of this guide is restricted to mandatory energy labelling for household appliances. Mandatory energy labelling is a sub-type of the general type of policy measure “legislative informative”.
More information and examples on the different subtypes residing under the main type (legislative / normative measures) can be found here and here.
More detailed information on the evaluation of Mandatory Energy Labelling can be found here and in resources cited in section 9 of this guidance (e.g. Corry Smith et al. 2016; Vine 2005).
2.2 Evaluation for a combination of policy measures
Mandatory energy labelling and Minimum Energy Performance Standards (MEPS) are often implemented concurrently, as is the case in the EU. The effects of Energy Labelling regulations are thus closely tied to the Ecodesign directive and minimum energy performance requirements.
Usually the same agencies and bodies deal with assessing the effects of both the energy labelling and minimum energy performance requirements, assessing the overall effect. Thus it can be very hard to disentangle the effects of each of the above policy measures. That’s why in practice, ex-post evaluations assess the impacts of energy labelling and MEPS as a whole, without attributing savings to either labelling or MEPS (Corry Smith et al. 2016).
Providing guidance about methods to evaluate interactions from several policy measures are beyond the scope of this guidance, however, it is important to consider what overlaps can occur. A possible approach to separate the effects of energy labelling and MEPS is to use econometric methods, taking advantages of changes in the regulations over time. See for example (Bjerregaard and Møller, 2019) (see also Adjustment factors in section 4 of this guidance).
Overlap can also occur with other types of policy measures, such as:
- Subsidy scheme for early replacements of inefficient appliances (see the specific guidance #14 about the case of subsidy schemes for household appliances) ;
- Subsidy scheme for replacements of appliances for low income households (as part of policy to tackle energy poverty)
- Taxes on electricity prices (see below)
- Programmes for energy advice for households
- Other voluntary labelling schemes
For more details about the policies implemented by Member States for energy efficiency in appliances, see for example the MURE database or (Tholen et al. 2017).
2.3 Evaluation when combined with energy taxes
The calculated savings effect for mandatory Energy Labelling of household appliances will overlap with that of the energy taxes (particularly with electricity taxes). Increasing the energy prices makes appliances with higher energy efficiency more attractive as their lifecycle cost decreases (compared to less efficient appliances). Both instruments (energy labelling and energy taxes) thus act in the same direction: encouraging customers to buy more efficient appliances. Hence an overlap when evaluating the respective effects.
However in practice, with the progressive reinforcements of the MEPS, the incremental difference in energy consumption between remaining energy classes becomes lower. This means that the incentive to save is much lower, as due to the Ecodesign and other industry standards, the market is already forced to get rid of the most inefficient products. On the contrary, the prices of appliances from one end to the other of the energy scale is becoming more condensed, as the standards are pushing out the least efficient devices, so the price difference amongst high- and low-efficient devices has also diminished.
Bjerregaard and Møller (2019) have also pointed out that it happens that newer and more efficient appliances are discounted when first put on the market, so the first spur of buying more of the most efficient appliances might be also due to this incentive. However, such distortions are not large enough to justify the trend of buying more efficient appliances due to increased transparency through energy labelling.
One practical way to avoid overlap in the energy savings counted for energy labelling and energy taxes can thus be to consider that the savings from buying more efficient appliances are attributed to energy labelling. Whereas energy savings from an efficient use of the appliances are attributed to energy taxes (that can for example counteract possible rebound effects). In practice, this can be done by calculating first the savings from energy labelling, and then deduct them from the total savings calculated for the energy taxes.
2.4 About household appliances
Information on (sub)sectors defined in the Toolbox can be found here, chapter 2, p.17
The subsector considered here only concerns appliances in households and explicitly excludes buildings and HVAC (Heating, Ventilation and Air Conditioning) equipment. As of May 2019, mandatory energy labelling in European Union is applicable to 5 product groups of household appliances: 1) dishwashers, 2) washing machines and tumble driers, 3) refrigerating appliances, including wine storage fridges, 4) lamps, 5) electronic displays including televisions, 6) cooking appliances.
For more details, see the European product database for energy labelling. For more details and updates about the list of energy-using products covered by the EcoDesign and Energy Labelling regulations, see here.
HVAC equipment are also covered by mandatory energy labelling: Space and water heaters; Ventilation units; Air conditioners. As well as some non-residential appliances (e.g. professional refrigerators). These types of equipment are nevertheless out of the scope of this guidance. For example, they are not taken into account when considering the normalization factors. Most of this guidance is however relevant to these other cases.
2.5 Evaluation for cross-sector saving actions
This guide is also applicable to evaluate cross-sector savings, provided that the data needed are available for all relevant sectors. Specifically, the same approach can be used for evaluating savings for mandatory labelling for equipment in the services sector and, to a certain extent, for the industry sector, when dealing with standardized devices. The difficulty would be to find a wide enough set of data to test whether the transparent labelling affects customer choices, and to what extent. Further analysis about normalization factors might also be needed.
2.6 About the method [diffusion indicators]
Diffusion indicators are indicators of the share of specific equipment or practice in the market, combined with deemed estimates of energy savings per equipment or practice. In the case of mandatory energy labelling for household appliances, the indicators will usually be the market shares per energy class for each category of appliances (see for example Harrington 2017; Michel et al. 2017).
Evaluating the impacts of energy labelling can for example be done by assessing the effects of energy labelling on monthly sales, in the months/stages when the labelling scheme is being introduced or revised. This requires to analyse time series with econometric methods. An example is to use a Cointegrated Vector-Auto-Regressive (CVAR) model, as done by Bjerregaard and Møller (2019) on market data over 2005-2017 in Denmark.
Another approach can be to compare the sales per energy class of a baseline scenario and a “policy” scenario. See for example (Fischer et al. 2017) about the case of Germany.
Whatever the approach used, it is important to have market data on sales and choose for how long to monitor sales and which behaviour to attribute to the labels and which not.
After the effects on market shares have been assessed, then the energy savings are usually calculated from the data about standardized energy consumption per energy class as defined by the energy labelling scheme (see Complementary methods below). The energy consumption that is portrayed on the label is achieved in testing conditions, which might not always reflect actual conditions of use of the appliances. Further analysis might thus be needed to get more reliable results (see Additional methods below).
Likewise, the evaluation of the effects of the labelling scheme can go beyond the analysis of sales data. Other data and further analysis can be needed to investigate the effects on purchasing behaviours of consumers or on business strategies of manufacturers and retailers (see Additional methods below).
Information about the various evaluation methods can be found here, table 1 and 2. This source also covers the combination of the method at hand with other methods, which will be dealt with below.
2.7 Complementary methods to determine total savings
The diffusion indicators method concerns the determination of the number of actions or participants (more specifically here the number of appliances). In order to provide the total savings, the unitary savings per appliance should also be calculated. For more information about methods to calculate unitary savings, see the link here, table 2 and 3 for more information.
The easiest and cheapest way to assess unitary energy savings is to use the standardized energy consumption, i.e. the energy consumption from standardized laboratory tests used to label the appliances. The approach used then mostly depends on the level of disaggregation of the data available:
- Data aggregated per energy class: average energy consumption is defined for each energy class.
- Data can be collected for each model of appliance (or most of them).
In both cases, the unitary energy savings will correspond to deemed savings, as they are estimated from reference data (and not from data specific to the appliances as bought and used by the consumers). The least disaggregated approach includes higher uncertainties, as in that case the average energy consumption per energy class can rarely be based on actual sales data to weight the various energy consumption levels included in the same energy class. A default assumption is often to take the higher limit of the range of energy consumption for the energy class. This avoids to overestimate the unitary energy savings. However this very likely leads to underestimation.
Even when more disaggregated data can be used (data per model of appliance), the use of standardized energy consumption includes uncertainties related to the differences between the conditions used to test the appliances (and determine the standardized energy consumption) and the actual conditions of use by the households. Another source of uncertainties can be due to non-compliance between the energy consumption stated by manufacturers and energy consumption found in compliance verifications.
When aiming at more reliable results, evaluation can therefore make use of direct measurement methods, such as:
- On-site measurement campaigns (when the objective is to investigate energy consumption in actual conditions of use) (see for example Dupret & Zimmerman 2017).
- Independent laboratory tests (when the objective is to investigate the compliance of the appliances vs. the information stated by manufacturers) (see for example Blomqvist & Fjordbak Larsen 2015; Dragovic & Broc 2018; Lock et al. 2015; NEA 2015; Spiliotopoulos et al. 2017).
For more details, see Specific Guidance 4 about using direct measurements.
Direct measurement can be costly, and therefore is usually done on samples. Synergies with other purposes than ex-post evaluation can be found to optimize the use of direct measurement. For example with market surveillance activities. More generally, data from direct measurement can be used for (Vine 2005):
- Calibrating the energy consumption data used in stock modelling.
- Verifying compliance of appliances sold on the market.
- Defining correction factors to be applied to the results from standardised tests.
- Providing energy advice to end users (based on the analysis of factors influencing the actual energy consumption of appliances).
2.8 Additional methods to increase reliability of the results
Stock modelling can be used in addition to diffusion indicators, to improve market analysis. Stock modelling can for example enable to track renewal rates and renewal patterns. This can be used to improve assumptions about actual lifetime of use of the appliances or to make disaggregated analysis (e.g. per categories of consumers). For more details, please refer to Specific Guidance 5 about using stock modelling.
Another approach to assess the effects of energy labelling on market shares per energy class is to survey customers about their purchasing behaviours. This can provide qualitative findings about how energy labelling is taken into account by customers in their decisions (which cannot be found from econometric analysis). Examples of such studies can be found in (Stadelmann and Schubert, 2018) about Switzerland, or in (Fries et al. 2017) about a large survey covering eight European countries. A meta study can be found in (Rohling and Schubert, 2013).
Likewise, surveys of manufacturers and retailers can provide complementary findings about market transformation effects of energy labelling, looking at changes in manufacturers’ and retailers’ strategies. For more details, see for example the section 9.4.3 in (Vine 2005).
These types of surveys are most often used to provide qualitative findings and analyse how to improve the labelling schemes. Surveys can also be used to obtain data for assessing quantitative impacts on market shares and ultimately energy savings. This requires conducting the same survey repeatedly over time, and to survey larger samples than for qualitative analysis. It can be challenging to obtain representative samples, and answers to surveys can include several biases (such as social desirability bias) making that the answers do not correspond to the actual purchases or changes in market strategies (Vine 2005). For more details about sampling, see for example (Khawaja et. al., 2017). And about survey methods, see for example (Baumgartner 2017).
Therefore in practice, quantitative analysis of sales data and survey of customers are complementary to evaluate the effects of energy labelling schemes, and more specifically to (Vine 2005):
- Determine the influence of energy labelling on purchase decisions;
- Track sales-weighted efficiency trends (changes in diffusion indicators);
- Assess energy savings.
For possible combinations with an additional method see chapter 6 here.
3. EVALUATION OBJECTIVES and REQUIREMENTS
3.1 Meeting evaluation goals and ambition
The table shows whether this guide can be used to report on general evaluation goals or criteria. See also in this report.
General types of evaluation goals or criteria |
Level of ambition | Remarks |
Calculation of realized energy savings from saving actions | Low to high | Depending on the complementary method used to estimate unitary energy consumption or savings, and the normalization factors taken into account:
Low in case of deemed savings or standardized energy consumption from energy label without any normalization factor. High in case of representative direct measurements with normalization factors (see Specific Guidance 4 about using direct measurement) |
Calculation of energy savings attributed to the policy measure(s) | Medium | Depending on the method used for unitary energy savings (see above) and on the method use to assess the effects of the labelling scheme on diffusion indicators.
Stock modelling can be used as additional method to improve the reliability of the results. Savings usually evaluated for the policy package (energy labelling + MEPS). Difficult to separate the energy savings from energy labelling. |
Cost-effectiveness of saving action (for end-users) | Low | Diffusion indicators are meant to monitor or evaluate effects at the policy level, not at end-users’ level. A complementary method is needed to assess unitary savings (see first line of this table and section 2). |
Cost-effectiveness of policy (government spending) | Medium | Same comment as for energy savings attributed to the policy measure(s).
Government spending can include administration as well as information costs (see section 7). |
CO2-emission reduction from saving actions | Low to high | Depending on the complementary method used to estimate unitary energy consumption or savings (see above). Energy savings are usually obtained per energy carrier, enabling to calculate CO2 savings (see section 7). |
CO2-emission reduction attributed to the policy measure(s) | Medium | Same comment as for energy savings attributed to the policy measure(s).
Energy savings are usually obtained per energy carrier, enabling to calculate CO2 savings (see section 7). |
The evaluation objectives depend mostly on the needs and priorities of policy makers (e.g. public authorities in charge of designing, revising or enforcing the labelling schemes). Experience about evaluating mandatory energy labelling schemes shows that two of the most frequent key evaluation questions are: 1) how well the policies are implemented and received (process evaluation) ; and 2) what have the policies achieved (impact evaluation) (Corry Smith et al. 2016). This guidance is focused on impact evaluation, and more specifically the evaluation of energy savings. For general guidance about other types of evaluation, see this link.
When using diffusion indicators to evaluate energy savings from labelling schemes, the evaluation of energy savings is closely related to the evaluation of market transformation effects. Detailed discussions about market transformation effects go beyond the scope of this guidance. More guidance about these issues can be found in the references included in section 9 (see for example Corry Smith et al. 2016; Vine 2005).
For more information on verification of actual energy savings and attribution/baseline/corrections, see section 4, and for cost-effectiveness and emission reduction see section 7.
3.2 Reporting expectations
The method will make it possible to report (net) savings of energy labelling schemes for household appliances. As explained in section 2 of this guidance, in practice, energy savings will be evaluated for the policy package including energy labelling and minimum energy performance standards (MEPS).
The unit used to express the savings will usually depend on the unit used to define the energy labels for appliances, i.e. in most cases final energy savings. This is because energy labelling is firstly meant to inform end-users, i.e. about final energy consumption.
Other units can be used to answer other evaluation objectives:
- Primary energy savings: for example used when the evaluation objective is to assess the contribution of energy labelling schemes to national targets in terms of primary energy consumption or savings, or to assess the contribution to reduction of energy imports or security of supply. This metric requires data about primary energy factors (see section 5).
- Lifecycle energy savings: for example used when the energy consumption used to produce and recycle the appliances can be significant compared to the energy consumption of the using phase. This metric requires data from LCA (Life Cycle Analysis), which goes beyond the scope of this guidance.
- CO2 savings: often used when the energy labelling scheme also includes a focus on climate change impacts. This metric requires data about emission factors (see section 7).
- Lifecycle CO2 savings: same approach as for lifecycle energy savings.
3.3 Time frame for evaluation
The length of the period under evaluation depends on the active period of the policy measure, the need to monitor developments and more specifically to obtain sales or market data, and the time needed to present (reliable enough) results or impacts that fit into the decision making process. In some cases, the periodicity of evaluation can be set by law.
When evaluating labelling schemes with diffusion indicators, the planning of evaluation activities mostly deals with the needs in data collection (sales or market data; unitary energy consumption per label or model of appliance), feedback for policy making and reporting. (See also planning of evaluation in the link here).
Experience shows the importance to plan the evaluation and anticipate data needs from the onset of the scheme (Corry Smith et al. 2016; Vine 2005). Long and detailed enough time series of sales data will be needed to assess the effects of the labelling scheme on the diffusion indicators (e.g. market shares per energy class). These data are usually needed as well for the monitoring of the scheme or as part of market surveillance. It is therefore important to ensure that the data collected for other purposes meet the requirements of the evaluation method chosen, so that they can then be used for the ex-post evaluation.
A regular monitoring of the diffusion indicators can also be used to detect when further analysis (i.e. ex-post evaluation) is needed. For example, if the trends are not evolving as expected in the impact assessment, or if the pace of energy efficiency improvements is slowing down.
The classical data sources for sales or market data (mostly based on market surveys) have a time lag of one year or more. New data collection techniques based on data available from the web (e.g. manufacturers’ and retailers’ websites, web-market places) can have much shorter delays, even close to real-time monitoring. For more details, see for example (Enervee 2014; Bennich et al. 2017).
When relevant, evaluation planning should also consider if additional methods will be needed, such as surveys of consumers, measurement campaigns or laboratory tests.
3.4 Boundaries for the evaluation
In the European context, energy labelling is usually evaluated either at European level (see for example: Ecofys et al. 2014; Geismar 2015) or at national level (see for example: Bjerregaard and Møller, 2019; Fischer et al. 2017).
The energy labelling regulations are adopted at the EU level, and then implemented by each Member State. The effects in Member States can vary according to various national factors, such as purchasing power of households, energy prices, overall energy or environmental awareness of households, accompanying measures (e.g. information campaign).
For example, in their study about Denmark Bjerregaard and Møller (2019) conclude that the positive results observed might be due to the fact that Denmark was already a front-runner in energy efficiency and people were aware of the benefits of buying more efficient products. On the other side, the same fact could have meant that the econometric model portrayed lower results than hold true, as consumers were aware about the labels even before new labelling campaign was introduced.
Likewise, Mills and Schleich (2010) have pointed that socioeconomic characteristics other than energy/monetary saving can be important drivers for the choice of purchasing an appliance with high energy efficiency (Mills and Schleich, 2010).
Thus, it is important to know the context of the markets where the labelling scheme is being evaluated.
4. KEY METHODOLOGICAL CHOICES FOR CALCULATION OF ENERGY SAVINGS
General principles of calculating realized savings using different methods can be found here and here
This section deals with key methodological choices to be considered when calculating energy savings: consistency between ex-ante and ex-post evaluation, baseline, normalization and adjustment factors. These choices are important to document when reporting energy savings, to ensure the transparency of the results.
One of the most important and challenging methodological choices is to define the baseline against which the actual situation (based on observed data) will be compared. When dealing with diffusion indicators for appliances, this is about defining a baseline scenario or using econometric methods to analyse changes in market shares per energy class.
Depending on the method used to assess the unitary energy consumption or savings (i.e. energy consumption per appliance), normalization factors might be needed to ensure that the results are representative of the whole market, and when relevant, representative of actual conditions of use.
Depending on the baseline option chosen, adjustment factors might be needed to assess net or additional savings.
4.1 Matching method with earlier ex-ante evaluation
From the viewpoint of methodological consistency and data availability, using the same method in the ex-ante evaluation and in the ex-post evaluation might be an obvious choice. However, for ex-ante evaluation only a few methods are usually considered, namely deemed savings, engineering estimate and stock modelling.
A different method than the one(s) used for the ex-ante evaluation can be applied for the ex-post evaluation, depending on the evaluation objectives, timeframe and data available for the situation after implementing the actions. For possible combinations of methods applied ex-ante and ex-post, see section 7, calculation approaches here.
In the case of energy labelling schemes, ex ante evaluations are often based on stock modelling or similar modelling tools to estimate impacts on forecasted sales, especially in terms of market shares per energy class. These ex-ante evaluations can for example be done as part of an impact assessment, before the adoption or revision of a labelling scheme, looking at the potential impacts on consumers, manufacturers, overall economy, energy consumption, etc. (Corry Smith et al. 2016).
When such an ex-ante evaluation is available, it can be used as an input for the ex-post evaluation by taking the business-as-usual (or without labelling scheme) scenario defined in the ex-ante evaluation as the baseline scenario for the ex-post evaluation. The consistency can then be ensured by using the same indicators and types of data sources when assessing ex-post the diffusion indicators as were used ex-ante in the stock modelling.
Ex-post evaluations based on actual sales data are particularly useful to evaluate the actual effectiveness of the labelling schemes in transforming the market. Ex-post analysis can test the key assumptions made in the ex-ante evaluations. This can help to improve the labelling schemes, as well as the modelling tools in view of future ex-ante evaluations.
4.2 Calculation baselines
Energy savings are defined in general as the difference between the actual situation and a reference situation without the saving actions (and without the policy measures that influence these saving actions, here the mandatory labelling scheme). The reference situation can be defined using various calculation baselines. see further in this internal EPATEE communication regarding an assessment of energy savings in the references of the EPATEE Knowledge Base here.
The choice of the baseline is a major methodological choice that should be made according to the evaluation objectives. In this guidance, it is considered that the evaluation objective is to assess the energy savings from the point of view of the public authorities in charge of the labelling scheme.
The main objective of energy labelling scheme is usually to encourage consumers to buy more efficient appliances than the ones they would have bought in the absence of the scheme. The labelling scheme does not provide directly an incentive for early replacement of appliances. It can thus be assumed that the appliances sold in this context correspond either to old appliances that would have been replaced anyway, or to new appliances that would have been bought anyway.
When using diffusion indicators, energy savings from policy makers’ point of view can for example be calculated in this case through a with/without comparison. The “with” situation is the one based on actual sales data. The “without” situation is a business-as-usual or baseline scenario, i.e. a scenario assumed to represent how the market would have evolved in the absence of the labelling scheme, and that can for example be built through stock modelling or similar analysis, taking into account the existing stock of appliances, replacement rates or patterns, and recent trends in market shares (per energy class) (trend analysis, sub-category of with/without comparison). An example of ex-post evaluation using a business-as-usual scenario can be found in (Fischer et al. 2017).
Baseline or business-as-usual scenarios are commonly used for ex-ante evaluations (for example for impact assessments). Using the same baseline scenario for the ex-post evaluation helps ensuring consistency. However, the baseline scenario can also be adapted for the ex-post evaluation, for example to take into account unforeseen changes in the context (e.g. economic crisis) or improvements in data about appliances’ energy consumption.
Several baseline scenarios can also be used to test the robustness of the results or perform sensitivity analysis about key assumptions (Corry Smith, 2016).
As it can be difficult to build reliable baseline scenarios, another approach to assess the energy savings from policy makers’ point of view can be to use econometric methods that will make statistical analysis of periods or regions without the labelling scheme or before scheme updates (“without” situation), and periods of regions with the new labelling scheme (“with” situation). An example of ex-post evaluation using an econometric method can be found in (Bjerregaard & Møller 2019).
This guidance considers the policy makers’ point of view, and therefore as main baseline option the with/without comparison. Depending on the evaluation objectives and point of view, other types of baseline can be used:
- “before” situation: in that case, the baseline is the energy consumption of the appliances that are replaced (or depending on the data available, the average energy consumption of appliances in the stock). This baseline can for example be used to estimate the energy savings as observed by the end-users, or to show the full energy efficiency improvements, independently of their causes. This baseline is however not appropriate to estimate the impact of labelling schemes, as labelling schemes of new appliances are not meant to change the renewal rates of appliances. So it can be assumed that end-users would have replaced their appliances anyway. In case the labelling scheme would apply to appliances in use (as now the case in Germany for old boilers), it could be relevant to consider the “before” situation as the baseline option.
- minimum energy performance standards: in that case, the baseline is the maximum standardised energy consumption allowed on the market. This baseline can for example be used as a default baseline when evaluating additional energy savings of the labelling scheme compared to the MEPS. However, further analysis and adjustment factors would be needed, as appliances with efficiency higher than the MEPS would very likely have been bought even in the absence of the labelling scheme.
The baseline option of a control group is rarely possible when evaluating labelling schemes. Labelling schemes are usually implemented at national, or even supra-national (e.g. European Union), level. Therefore using a control group would usually mean to make a cross-country comparison, which can raise issues to ensure that the evolutions in different countries can be compared.
About baseline issues, see further here and in the evaluation guidebook of the IEA (including also case studies).
4.3 Normalization factors
In most cases, the unitary energy savings are not calculated from data measured in situ, but from standardised laboratory tests (see Complementary methods in section 2 of this guidance). As the tests are performed in standardised conditions, there is usually no need for further normalization for external factors that can influence energy consumption (e.g. weather conditions). However normalization factors might be needed when there is evidence that the standardized conditions used for the tests do not completely reflect the actual conditions of use by households. This is for example the case if studies have shown a systematic difference in energy consumption between energy consumption as stated in the energy label of the appliances and energy consumption as measured in-situ.
Reasons for such differences can include rebound effects in terms of intensity of use (e.g. more frequent use of dishwasher, use of cycles at higher temperature for washing machines). As energy labelling is an informative measure, it is also possible that its effects go in the opposite way, leading to a more efficient use of the appliances.
Correction factors to improve standardized energy consumption can be determined from measurement or monitoring campaigns. As this type of campaigns can be costly (especially when aiming at representative results), an alternative is to define default values from a literature review.
Investigating the difference between standardized and actual energy consumption can be done for other purpose than evaluating energy savings, thereby creating synergies to fund measurement campaigns. Such investigations can for example be used to improve the test procedures used to determine standardized energy consumption (thereby providing consumers with more reliable information), or to identify good and bad practices in terms of using behaviours, which can then be used to provide households with energy advice.
In case the unitary energy savings are calculated from in-situ measurements, normalization factors might also need to be taken into account, especially if the sample used for the measurements cannot be considered representative of the whole households in the country. These factors will deal with the using behaviours (e.g. number and types of cycles used for washing machines), and possibly with weather conditions (when relevant for the type of appliance evaluated). For more details, see Specific Guidance 4 about using direct measurement.
When evaluating policies related to appliances, types of rebound effects other than the intensity of use might need to be considered: rebound effects related to the size or number of appliances. For example, the fact that the new appliance consumes less energy can make that households will buy larger or more sophisticated appliances, or will have more appliances of the same type (for example, keeping the old refrigerator as secondary refrigerator). These effects can be challenging to assess, as they can be due to other reasons than the energy labelling scheme. For example due to an increase of the household income, or to the marketing of the manufacturers and retailers encouraging households to buy larger appliances. The evaluation of these market transformation effects or trends goes beyond the scope of this guidance.
For a more general discussion about normalization factors see link to internal project note on normalization and adjustment factors, gross and net savings, etc. and this document.
4.4 Adjustment factors
Adjustment factors define which part of the calculated energy savings can be attributed to a policy measure or meets the definition of savings specified in the evaluation objectives or reporting requirements (see next section on “Calculating Gross and net savings).
When using the diffusion indicators method, adjustment factors can be autonomous savings (or technological progress) or price-induced energy efficiency progress. See also this link, table 1.
If the baseline option chosen is the “with/without” comparison, then these factors are usually directly taken into account in the definition of the baseline.
When this is not the case, autonomous savings and price-induced energy efficiency progress can be assessed with surveys of consumers, similarly to the assessment of free-rider effects for bottom-up methods.
When dealing with mandatory energy labelling, and especially when using data from standardized energy consumption, a major adjustment factor to take into account is non-compliance: the possible differences between standardized energy consumption as stated by the manufacturers and the standardized energy consumption from independent laboratory tests.
A frequent difficulty to take into account non-compliance is that the main source of data about non-compliance is usually the verifications or controls done as part of market surveillance activities. These controls are often done with a risk-based approach: the samples controlled are selected according to the assumed risks of non-compliance (for example, taking into account results from previous controls or from the literature). Whereas an adjustment factor for non-compliance would need to be based on data representative of the whole market (i.e. defined with a sampling strategy aiming at representativeness).
As the objectives of energy labelling schemes usually include to promote market transformation towards more energy efficient appliances and higher market shares for the most energy efficient appliances, it can also be relevant to assess spill-over effects related to market transformation effects. For example the fact that energy labelling schemes can stimulate R&D efforts by manufacturers, or changes in manufacturers’ or retailers’ marketing in favour of the most energy efficient appliances.
Assessing this type of effects usually requires taking a long term perspective, and goes beyond the scope of this guidance.
In case another policy can overlap with the energy labelling scheme, the adjustment factor Double counting might be relevant. As discussed in section 2, the most common overlap is with MEPS (minimum energy performance standards).
A possible approach to separate the effects of energy labelling and MEPS is to use econometric methods, taking advantages of changes in the regulations over time. See for example (Bjerregaard and Møller, 2019). This study analysed monthly sales data over 2005-2017. In line with the respective policy objectives, the results confirmed that energy labelling helps promote higher-efficiency appliances, while MEPS help to clear the market of products with the lowest-efficiency.
In practice, the most common approach is to evaluate energy savings for the whole policy package targeting household appliances (energy labelling, MEPS and possibly other policy measure). Then complementary evaluations or studies are done to investigate the respective effects of the different policy measures to obtain market transformation. For example by using theory-based evaluations (for more details about this type of evaluation, see general guidance in this link).
4.5 Calculating Gross and Net energy savings
Gross savings concern the calculated savings from saving actions using a chosen baseline and normalization factors. Net savings concern the savings attributed to the policy measures (e.g. here the labelling scheme). Net savings can be evaluated either directly (when using a control or comparison group, or a business-as-usual scenario as baseline) or from gross savings by applying further adjustment (or gross-to-net) factors.
Gross unitary savings can be calculated using another baseline option: the before/after comparison. As it is difficult and costly to get data about the appliances replaced for all the appliances sold, this is usually estimated either from a survey on a sample of consumers, or taking as reference the average energy consumption of the appliances in the stock. A correction for relevant normalization factors (weather, rebound effect and other behaviour-related factors) is then applied.
The gross number of actions can be determined from the analysis of sales data, taking into account all the appliances sold during the period under evaluation.
Total gross savings are then equal to gross unitary savings times gross number of actions. These savings represent the impact of the technological progress due changes in the market and natural renewal rates of the appliances, independently of their causes (i.e. not necessarily due to the policy measures).
In this guidance, the baseline option considered is a with/without comparison. When the without situation is based on a baseline scenario, the calculations using this baseline directly provide additional energy savings, i.e. energy savings additional vs. the baseline scenario. Depending on the way the business-as-usual scenario has been defined, these additional savings can be considered net savings.
Net unitary savings can thus be determined by using as baseline option the with/without comparison, the without situation being a baseline scenario taking into account previous market trends.
Net total savings are then obtained as the difference between the baseline scenario and the data observed in terms of market shares per energy class (changes in the diffusion indicators). These results are then combined with energy consumption per energy class (or per model of appliance, depending in the level of disaggregation in the data available) to complete the calculation of the savings. When relevant, adjustment factors about non-compliance and double counting can also be included in the calculation of the net savings.
Another approach to calculate net energy savings is to use econometric methods, when the types of data and related time series available enable to separate statistically the effects of the policy from other factors. See for example (Bjerregaard and Møller, 2019).
5. INPUT AND OUTPUT
5.1 Main data requirements and data sources and collection techniques
Data requirements specified in the table below correspond to the calculation of energy savings, when using the method of diffusion indicators and the baseline option of “with/without” comparison.
Calculation subject | Data requirements | Possible data sources and collection technics |
Standardised energy consumption | Energy consumption per energy class (ranges or average)
Or energy consumption per model sold on the market |
Product registration and certification database
Manufacturers’ and retailers’ websites |
Number of actions | Time series of sales data for appliances, at least disaggregated per energy class | National statistics, market surveys, data about shipments and imports, market research firms (e.g. GfK, Euromonitor) |
Complementary data about energy consumption (see Specific Guidance 4 about using direct measurement) | Measured data about energy consumption (or about key parameters influencing actual energy consumption of appliances, e.g. duration of use) | Laboratory tests, measurement campaigns |
Complementary data about appliances | For example: product attributes or sub-categories, prices | Product registration and certification database
Manufacturers’ and retailers’ websites |
In recent years, new opportunities offered by gathering data from web-market places or other web sources have been investigated as cheaper and faster data collection techniques compared to classical market surveys (or market research firms). An example of this new method is using web-crawler techniques. For more details, see for example (Bennich et al. 2017; Enervee 2014).
Data issues when evaluating net energy savings
The main good practice to ensure the feasibility and reliability of the evaluation of net energy savings is to think about the method to be used when designing (or revising) the energy savings from mandatory energy labelling for household appliances.
Experience indeed shows that unless the data collection has been planned in advance, it will be very costly, time-consuming or even impossible to collect the data required to apply most of the methods that can be used to evaluate net energy savings. Which makes that in practice, using surveys will remain the only option possible (or considered feasible).
The main challenges when using surveys are:
- to achieve a high answer rate, in order to limit sampling bias;
- to use question phrasing that can limit the risk of bias in the answers.
When evaluating net savings from energy labelling scheme with diffusion indicators, the main challenge is to assess the effects on diffusion indicators that would not have happened in the absence of the labelling scheme. This is usually done by defining a baseline or business-as-usual scenario (see section 4).
Further analysis can then be made to investigate more qualitatively the effects of the labelling scheme, especially through surveys. Data taken into account in these surveys include for example energy awareness prior to the introduction of energy labels, awareness about the energy labels, economic wealth of the buyers, living conditions. For examples of questions for this type of survey, see for example the section 9.4.2 in (Vine 2005). Examples of studies using this type of surveys can be found in (Stadelmann and Schubert, 2018) about Switzerland, or in (Fries et al. 2017) about a large survey covering eight European countries. A meta study of surveys about energy labelling can be found in (Rohling and Schubert, 2013).
For more details about the evaluation of net energy savings, see the topical case study.
For possible other methods with different data demands see the section on alternatives for the chosen method.
5.2 Energy savings in final terms or in primary terms
Energy savings can be expressed in final terms or in primary terms. See definitions about primary and final energy here.
Final energy savings are based on a reduction of energy consumption at the end-user premises and savings for all energy carriers are added up. For primary energy savings account is taken of the conversion losses when providing the energy carriers to end-users, e.g. for electricity typically 2-3 times the amount delivered to the end-user is used as input in power production. Therefore, saving one unit of electricity saves about 2-3 unit of fuel in power production (depending on the state of technologies concerned. The energy savings in primary energy terms provide savings that represent the reduction in primary energy consumption (before conversion in energy carriers for end-users), and before possible transport and distribution of energy that can generate further energy losses).
Energy savings of exchanging household appliances can be calculated in final terms or in primary terms provided that savings at end-users are calculated for each energy carrier apart, and primary factors are available to convert the savings in final terms to savings in primary terms.
When dealing with energy savings from appliances, energy savings will usually be calculated separately for each type of appliances, therefore easily providing energy savings per type of energy carrier.
Most of the appliances covered by energy labelling use electricity. If the energy savings are calculated over the lifetime of the appliances (see below), then it is important to clarify if the primary factor used for electricity in the calculations takes into account the likely changes in the electricity mix over the period (e.g. due to the increase in the share of Renewable Energy Sources), or if the current primary factor is used for all years.
5.3 Energy savings over time
Implemented saving actions in a year lead to savings over a number of consecutive years. E.g. a more efficient boiler can save gas over its lifetime of about 15 years, and more efficient computers up to 5 years. Energy savings can be calculated in different metrics in terms of time reference, for example: year-to-year, annual, cumulated annual, cumulative. See the the definitions here.
The calculated yearly savings concern the savings of all new saving actions in that year. In this approach only data for the savings in the chosen year are needed.
Adding up the yearly savings over a period, provided that earlier saving actions are still delivering savings, leads to cumulative savings. For the cumulative savings data are needed for the whole period.
Energy savings from a saving action can also be discounted and summed up over the lifetime of the action; see here.
Another cumulative approach, to be applied for the EED, is to multiply the (new) savings in a year with the number of years up to a target year and sum this result with that for all other years up to the target year. This cumulative approach stimulates early saving actions, as these count more times to the target than later actions. However, it should be noted that energy savings from the implementation of the EU energy labelling regulations are not eligible to EED article 7.
Implementing energy labelling for household appliances can provide yearly savings of new saving actions in that year. It can also provide cumulative savings provided that data are available over a period, for the amount of time that the appliance is expected to be used. The average life expectancy for household appliances
Finally, the guide can provide discounted cumulative savings when discount factors have been defined for yearly savings over time. In the case of appliances, the technical discounting might be used, to take into account that the performance of the saving action can decrease over time. The few studies available about evolution of energy performance of appliances over time have shown that energy performance of more energy efficient appliances tend to maintain better over time than less energy efficient appliances. See for example (Hoffman et al. 2015; Proctor Engineering Group 1996).
6. ALTERNATIVE FOR CHOSEN METHOD
6.1 Alternatives for the chosen method
As described in earlier sections, the only options for evaluating the success and impact of energy labels on consumer choices is to either track the changes in market shares once the labelling is introduced. The table below presents the pros and cons of the method for evaluating the effect of energy labels on household appliances, and for commonly used alternative methods for the same combination of policy measure and sector.
Type of method | Pros | Cons |
Diffusion indicators (method considered in this guidance) | Detailed analysis about the number of actions, possibly enabling an evaluation of market transformation effects | Complementary method needed to estimate the energy consumption per type of appliances (usually deemed savings, with the limitations mentioned below) |
Stock modelling (see Specific Guidance 5) | Detailed analysis about the number of actions, possibly enabling an evaluation of market transformation effects
Possibility to calibrate the model with national statistics about energy consumption of household appliances |
Complementary method needed to estimate the energy consumption per type of appliances |
Deemed savings (see Specific Guidance 14) | Simplest method for rather detailed estimate
Very simple and cost-effective Can be used for ex-ante evaluation, and with small, or even no, time lag for ex-post evaluation. |
Can include various sources of uncertainties (e.g. differences between standardised and actual energy consumption)
Complementary method required to assess the number of actions (cf. market shares at least disaggregated per energy class) |
Direct Measurement (see Specific Guidance 4) | High accuracy for the data on energy consumption
Data that reflects actual use of the appliances (if on-site measurements; not for laboratory tests) Possibility to identify non-compliance (especially for laboratory tests) |
Possibly expensive and time-consuming
Difficult to obtain data that are representative of the whole market or stock of appliances. Complementary methods needed to assess the number of actions (cf. market shares at least disaggregated per energy class) |
Consumer surveys (see Additional method in section 2 of this Guidance) | Provide a better understanding of the effects of the labelling scheme, and thereby useful feedback to improve the scheme
|
Difficulty to obtain representative samples and quantitative results that can be used to calculate energy savings (unless large and repeated surveys can be done) |
7. ADDITIONAL EVALUATION RESULTS
7.1 Calculating avoided CO2 emissions
Avoided CO2 emissions can be evaluated from the energy savings by applying emission factors. Four key aspects are to be taken into account when choosing the emission factor(s):
- Emission factors vary according to the energy type, so the data about energy savings need to be available per energy type.
- Emission factors for a given type of energy can vary over time (especially for electricity).
- Emission factors can take into account:
- Direct emission factors: that take into account the emissions generated when producing the energy used;
- Lifecycle emission factors: that take into account all the emissions generated from the extraction of the energy resources up to the dismantling of the energy plant.
Due to the differences that the choice of emission factor(s) can induce, it is important to document what emission factor(s) has(have) been used.
The reduction in CO2-emissions can only be calculated when savings are calculated per relevant energy carrier and a specific emission factor is available for each energy carrier.
Most of the appliances covered by MEPS use electricity. It is important to document what emission factor is used for electricity, as there can be different ways to calculate this emission factor. For example, averaging the emissions over the whole annual electricity production, or taking into account that the electricity mix can be different according to the time of the day or the season, due to differences in the load curve and availability of capacities per type of electricity source. This latter approach can be used to define emission factor specific to each end-use, calculating the emission factor based on the load curve of the end-use.
The avoided emission of other greenhouse gasses due to energy savings are not taken into account here, as these emissions (and more specifically their reductions) are generally negligible compared to CO2 (apart from policy measures targeting the agriculture sector).
IPCC (Intergovernmental Panel on Climate Change) provides a detailed database of peer-reviewed emission factors.
7.2 Calculating cost-effectiveness
Cost-effectiveness is the ratio between costs to achieve energy savings and the amount of savings achieved and possibly other benefits.
A distinction can be made according to the point of view adopted to assess cost-effectiveness:
- Cost-effectiveness for the end-user or participant (e.g. payback time)
- Cost-effectiveness from the manufactures’ or retailers’ point of view (e.g. taking into account extra R&D investments, changes in turnover or mark-up)
- Cost-effectiveness for society at large (e.g. social net present value)
- Cost-effectiveness from the point of view of the public authority (e.g. comparing different types of policy measures)
For more details about the different perspectives, see for example the report on the Knowledge Base, identifying current knowledge, suggestions and conclusions from existing literature, available via this link.
Diffusion indicators are mostly used to evaluate the effects of public policies, so this guidance considers cost-effectiveness assessment from the viewpoint of the society and the public authority. The respective categories of costs and benefits to take into account when assessing the cost of the energy saved are presented in the table below. It should be noted that some elements can either be costs or benefits depending on the changes in prices of appliances (see discussion below the table).
Point of view | Costs | Benefits |
Public authorities | · Administration costs
· Costs of accompanying measures (e.g. information campaign) (when relevant) · Losses in tax revenues related to energy taxes (due to additional energy savings) |
· Additional or net energy savings |
· Changes in tax revenues related to VAT on appliances | ||
Society | · Administration costs for the public authorities
· Costs of accompanying measures (e.g. information campaign) (when relevant) |
· Additional or net energy savings |
· Marginal investments in new appliances (compared to the business-as-usual scenario) |
NOTE: the table above does not deal with non-energy impacts. Depending on the context and objectives of the energy labelling scheme, non-energy benefits can be important to include in the cost-benefit analysis (e.g. reductions in GHG emissions).
When assessing the cost-effectiveness of a labelling scheme, several challenges can arise:
- The scope of administration costs (and possibly of accompanying measures) can be difficult to define, and corresponding data can be difficult to collect or establish (e.g. part of the administration cost is related to time spent by civil servants; market surveillance activities can be common to other policy measures).
- Prices of appliances can be difficult to monitor in a systematic way, and it can be challenging to define a business-as-usual scenario including assumptions about trends in costs of appliances.
Usually, cost-benefit analyses of energy efficiency policies assume that the more energy efficient options will be more expensive than the less energy efficient ones in terms of investment costs. Therefore, they include as costs the additional investment costs. When dealing with appliances, this is not always the case. Sometimes, more efficient appliances can have the same or even small investment costs than less efficient appliances (for example, when the more efficient appliances become dominant on the market). That is why in the table above, the parameters related to the prices of appliances can either be costs or benefits.
The table above considers the calculation of the net cost of the energy savings. Depending on the indicator(s) used to assess cost-effectiveness, it can be needed to use discount factors (e.g. when the indicator is Net Present Values). In that case, it is important to document the use of discount factors, and if possible to make a sensitivity analysis (testing several values or ranges of discount factors). As this can affect significantly the results.
Likewise, the calculations of cost-effectiveness indicators will usually require to consider scenarios of energy prices over given periods. The assumptions about trends in energy prices should be documented. Whenever possible, it is recommended to make a sensitivity analysis (testing several scenarios of energy prices).
In practice, as it is difficult to separate the effects of energy labelling and MEPS (minimum energy performance standards), the cost-effectiveness analysis will be made for the whole policy package (energy labelling + MEPS). In that case, the business-as-usual scenario should be adapted accordingly and the administration costs of the MEPS should be taken into account.
7.3 Calculating other co-benefits
Co-benefits or non-energy impacts of energy efficiency policies can include for example:
- Extra employment
- Reduction of energy poverty
- Other emission reductions (NOx, SO2, fine particles, etc.)
- Reduced dependency on (insecure) energy import
In practice, co-benefits and non-energy impacts have more rarely been subject to evaluation when dealing with energy labelling schemes, compared to MEPS. And like for other energy efficiency policies, non-energy impacts have been more often evaluated as part of impact assessments (ex-ante evaluations) than in ex-post evaluations. The following examples were found in the review of ex-post evaluations of standards and labelling programmes done by Cory Smith et al. (2016).
Type of non-energy impacts | Why it can be relevant (and for whom) | Case(s) mentioned by Cory Smith et al. (2016) |
R&D and technology developments | Investments in R&D | Europe (ex-ante ; Ecodesign Impact Accounting report, see Kemna & Wierda 2015) |
Industrial competitiveness | Important criteria for policy makers, and to engage private stakeholders | Europe (ex-ante ; Ecodesign Impact Accounting report, see Kemna & Wierda 2015) |
Job creation | Important criteria for policy makers | Europe (ex-ante ; Ecodesign Impact Accounting report, see Kemna & Wierda 2015) |
Environmental impacts (water use, noise, air pollution) | Europe (ex-ante ; Ecodesign Impact Accounting report, see Kemna & Wierda 2015)
US (ex-ante and ex-post evaluation of the US standards for residential and commercial appliances, see Meyers et al. 2008). |
Providing support for the evaluation of non-energy impacts goes beyond the scope of this guidance. Examples of such evaluations can be found in the references mentioned above.
8. CONCRETE EXAMPLES
EPATEE case study dealing with household appliances:
- Dragovic, M., Broc, J.S. (2018). Nordsyn (Market Surveillance of Ecodesign and Energy Labelling) and the Effect Project in the Nordic Countries. Case study prepared by IEECP for the EPATEE project, funded by the European Union’s Horizon 2020 programme. https://epatee.eu/system/tdf/epatee_case_study_nordic_countries_nordsyn_and_the_effect_project_market_surveillance_ok_0.pdf?file=1&type=node&id=81
Examples of evaluations of energy labelling schemes for household appliances:
- Bjerregaard, C., & Møller, N. F. (2019). The impact of EU’s energy labelling policy: An econometric analysis of increased transparency in the market for cold appliances in Denmark. Energy Policy, 128, 891-899. https://doi.org/10.1016/j.enpol.2019.01.057
- Fischer, C., Yifaat, B., Blepp, M., Hesse, T., Rüdenauer, I., Stratmann, B, Sutter, J. & Weiss, U. (2017). Living up to the expectations? Monitoring the effects of ecodesign and energy labelling in Germany. Proceedings of the ECEEE 2017 Summer Study, 1621-1632. https://www.eceee.org/library/conference_proceedings/eceee_Summer_Studies/2017/7-appliances-products-lighting-and-ict/living-up-to-the-expectations-monitoring-the-effects-of-ecodesign-and-energy-labeling-in-germany/
- Geismar, P. (2015). EU Energy Label on Vacuum Cleaners: evaluation and market changes. Proceedings of EEDAL 2015, 196-206. https://e3p.jrc.ec.europa.eu/publications/proceedings-8th-international-conference-energy-efficiency-domestic-appliances-and
- Harrington, L. (2017). Tracking the energy efficiency of whitegoods in Australia. Proceedings of EEDAL 2017, 551-564.
- Lane, K., Harrington, L., & Ryan, P. (2007). Evaluating the impact of energy labelling and MEPS–a retrospective look at the case of refrigerators in the UK and Australia. European Council for Energy-Efficient Economy (Paris): Proceedings of the ECEEE 2007 Summer Study, 743-751. https://www.eceee.org/library/conference_proceedings/eceee_Summer_Studies/2007/Panel_4/4.292/
- Michel, A., Bush, E. & Attali, S. (2017). How has the European White Goods market changed in the past 10 years? – Analysis based on sales data reveals constant improvements, contradictory trends, and big successes for a new technology. Proceedings of EEDAL 2017, 565-579.
9. FURTHER READING
General guidance on evaluations:
- Baumgartner, R. (2017). Chapter 12: Survey Design and Implementation for Estimating Gross Savings Cross-Cutting Protocol. national laboratory of the U.S. Department of Energy Office of Energy Efficiency & Renewable Energy Operated by the Alliance for Sustainable Energy, LLC. https://www.nrel.gov/docs/fy17osti/68568.pdf
- Khawaja, S., Rushton, J., Keeling, J. (2017). Chapter 11: Sample Design Cross-Cutting Protocol. The Uniform Methods Project: Determining Energy Efficiency Savings for Specific Measures. Prepared for NREL (National Renewable Energy Laboratory), September 2017. https://www.nrel.gov/docs/fy17osti/68567.pdf
About mandatory energy labelling:
- Ecofys, Waide SE, SoWatt, Öko-Institut, SEVEn & University of Coimbra (2014). Evaluation of the Energy Labelling Directive and specific aspects of the Ecodesign Directive. Final technical report for the European Commission, June 2014. https://ec.europa.eu/energy/en/studies/evaluation-energy-labelling-directive-and-specific-aspects-ecodesign-directive
- European Commission (2019). Clearer and simple energy labels will help consumers save money and contribute to the Energy Union’s objectives. https://ec.europa.eu/info/news/clearer-and-simple-energy-labels-will-help-consumers-save-money-and-contribute-energy-unions-objectives-2019-mar-11_en
- Hirayama, S., Nakagami, H., Murakoshi, C., Nakamura, M., Mizutani, S. & Okuda, S. (2008). International Comparison of Energy Efficiency Standard and Labels: Development Process and Implementation Phase. Proceedings of the 2008 ACEEE Summer Study on Energy Efficiency in Buildings, 8-110:8-122. https://aceee.org/files/proceedings/2008/data/papers/8_194.pdf
- IEA 4E (2016). Achievements of appliance energy efficiency standards and labelling programs: A global assessment in 2016. Report of the IEA Technology Collaboration Programme on Energy Efficient End-Use Equipment (IEA 4E), August 2016. https://www.iea-4e.org/document/387/achievements-of-appliance-energy-efficiency-standards-and-labelling-programs-a-global-assessment-in-2016
- Kemna, R. (2014). Ecodesign impact accounting – Part 1. Report by VHK for the European Commission. 30 May 2014. https://ec.europa.eu/energy/sites/ener/files/documents/2014_06_ecodesign_impact_accounting_part1.pdf
- Tholen, L., Adisorn, T. & Götz, T. (2017). It’s nothing to write about? How EU-Member States address appliances in their National Energy Efficiency Action Plans. Proceedings of EEDAL (Energy Efficiency in Domestic Appliances and Lighting) 2017, 171-182. https://e3p.jrc.ec.europa.eu/events/eedal17
- Waide, P., Watson, R., Attali, S., Eide, A., Krivosik, J. & Schiellerup, P. (2013). The new energy label: assessing consumer comprehension and effectiveness as a market transformation tool. Proceedings of the ECEEE 2013 Summer Study, . https://www.eceee.org/library/conference_proceedings/eceee_Summer_Studies/2013/6-appliances-product-policy-and-ict/the-new-energy-label-assessing-consumer-comprehension-and-effectiveness-as-a-market-transformation-tool/
- Wiel, S., & McMahon, J.E. (2005). Energy-Efficiency Labels and Standards: A Guidebook for Appliances, Equipment, and Lighting (No. LBNL-45387-2nd-Edition; LBNL-45387). Ernest Orlando Lawrence Berkeley National Laboratory, Berkeley, CA (US). https://www.osti.gov/servlets/purl/877316-HHBvs5/
About diffusion indicators for monitoring energy efficiency in households appliances:
- Harrington, L. (2017). Tracking the energy efficiency of whitegoods in Australia. Proceedings of EEDAL 2017, 551-564.
- Michel, A., Bush, E. & Attali, S. (2017). How has the European White Goods market changed in the past 10 years? – Analysis based on sales data reveals constant improvements, contradictory trends, and big successes for a new technology. Proceedings of EEDAL 2017, 565-579.
- ODYSSEE database of energy efficiency indicators: http://www.indicators.odyssee-mure.eu/energy-efficiency-database.html
About methodologies to evaluate impacts from standards and labelling schemes for households appliances:
- Corry Smith, J. (2016). Ex-Post Impact Evaluations of Appliance Standards and Labelling Programmes: A Global Review of Best Practices and Lessons Learned. Proceedings of IEPPEC 2016. https://energy-evaluation.org/wp-content/uploads/2019/06/2016-paper-corry-smith.pdf
- De Melo, C. A., & de Martino Jannuzzi, G. (2010). Energy efficiency standards for refrigerators in Brazil: A methodology for impact evaluation. Energy Policy, 38(11), 6545-6550. https://doi.org/10.1016/j.enpol.2010.07.032
- Vine, E., Choi, J.Y., du Pont, P. & Waide, P. (2005). Chapter 9: Evaluating the impact of energy –efficiency labelling and standard-setting programs. In: Wiel, S., & McMahon, J. E. (2005). Energy-Efficiency Labels and Standards: A Guidebook for Appliances, Equipment, and Lighting (No. LBNL-45387-2nd-Edition; LBNL-45387). https://www.osti.gov/biblio/877316-HHBvs5/
- Vine, E., du Pont, P., & Waide, P. (2001). Evaluating the impact of appliance efficiency labelling programs and standards: process, impact, and market transformation evaluations. Energy, 26(11), 1041-1059. https://doi.org/10.1016/S0360-5442(01)00053-6
About new data collection techniques:
- Bennich, P., Stengard, L. , Christensen, S.F., Hartikainen, T., Mogensen, K. & Fjordbak Larsen, T. (2017). Using webcrawler techniques for improved market surveillance – new possibilities for compliance and energy policy. Proceedings of the ECEEE 2017 Summer Study, 1633-1642. https://www.eceee.org/library/conference_proceedings/eceee_Summer_Studies/2017/7-appliances-products-lighting-and-ict/using-webcrawler-techniques-for-improved-market-surveillance-8211-new-possibilities-for-compliance-and-energy-policy/
- Enervee (2014). Recent and Historical Product Energy Efficiency and Life-cycle Cost Improvement in Swedish Appliance Markets. January 2014. https://superefficient.org/publications/recent-and-historical-product-energy-efficiency-and-life-cycle-cost-improvement-in-swedish-appliance-markets
About evolution of energy performance over time:
- Hoffman, I., Schiller, S., Todd, A., Billingsley, M., Goldman, C., & Schwartz, L. (2015). Energy Savings Lifetimes and Persistence: Practices, Issues and Data. Technical Brief, Lawrence Berkeley National Laboratory, May 2015. https://emp.lbl.gov/publications/energy-savings-lifetimes-and
- Proctor Engineering Group (1996). Statewide Measure Performance Study for the California Statewide Persistence Subcommittee CADMAC. Final Report. An Assessment of Relative Technical Degradation Rates. https://www.proctoreng.com/energy-efficiency/archive.html
Examples about distinguishing effects from energy labelling and other policy measures:
- Bjerregaard, C., & Møller, N. F. (2019). The impact of EU’s energy labelling policy: An econometric analysis of increased transparency in the market for cold appliances in Denmark. Energy Policy, 128, 891-899. https://doi.org/10.1016/j.enpol.2019.01.057
- Sulyma, I. & Tiedemann, K.H. (2015). Evaluating the Combined Impact of Financial incentives, Tax Incentives, Information and Minimum Energy Performance Standards in Transforming a Residential Appliance Market. Proceedings of EEDAL (Energy Efficiency in Domestic Appliances and Lighting) 2015, 1186-1196. https://e3p.jrc.ec.europa.eu/publications/proceedings-8th-international-conference-energy-efficiency-domestic-appliances-and
About issues related to compliance:
- Blomqvist, L. & Fjordbak Larsen, T. (2015). Effect project – estimating the benefits of Nordic market surveillance cooperation on ecodesign and energy labelling. Proceedings of the ECEEE 2015 Summer Study, paper 8-266-15, 1837-1848. https://www.eceee.org/library/conference_proceedings/eceee_Summer_Studies/2015/8-monitoring-and-evaluation-building-confidence-and-enhancing-practices/effect-project-8211-estimating-the-benefits-of-nordic-market-surveillance-cooperation-on-ecodesign-and-energy-labelling/
- Lock, T., Muir, S., Faninger, T., Tinetti, B. & Krivosik, J. (2015). What’s on? Compliance of Televisions with Energy Labelling and Ecodesign Regulations. Proceedings of EEDAL (Energy Efficiency in Domestic Appliances and Lighting) 2015, 247-258. https://e3p.jrc.ec.europa.eu/publications/proceedings-8th-international-conference-energy-efficiency-domestic-appliances-and
- NEA, 2015. Results of Verification Testing of Registrable Goods Under the Mandatory Energy Labelling Scheme. Report of Singapore’s National Environment Agency, January 2015. https://www.nea.gov.sg/docs/default-source/our-services/report-on-vt-results-(updated).pdf
Examples of using surveys to evaluate the effects of labelling schemes (additional method):
- Fries, B., Braungardt, S., Kreuzer, M. & Wohlfahrt, K. (2017). EU product policy and consumer purchase decisions – empirical evidence from eight EU member states. Proceedings of the ECEEE 2017 Summer Study, 1593-1602. https://www.eceee.org/library/conference_proceedings/eceee_Summer_Studies/2017/7-appliances-products-lighting-and-ict/eu-product-policy-and-consumer-purchase-decisions-8211-empirical-evidence-from-eight-eu-member-states/
- Mills, B. & Schleich, J. (2010). What’s driving energy efficient appliance label awareness and purchase propensity? Energy Policy, 38 (2), 814–825. https://www.econstor.eu/bitstream/10419/28512/1/605186650.pdf
- Rohling, M. & Schubert, R. (2013). Energy Labels for Household Appliances and Their Disclosure Format: A Literature Review. Institute for Environmental Decisions (IED) Working Paper 21, ETH Zurich. http://www.econ.ethz.ch/content/dam/ethz/special-interest/gess/economics-dam/documents/Projekts/EnergyLabelReview.pdf
- Stadelmann, M., & Schubert, R. (2018). How do different designs of energy labels influence purchases of household appliances? A field study in Switzerland. Ecological economics, 144, 112–123. https://doi.org/10.1016/j.ecolecon.2017.07.031
Examples of evaluations or studies assessing non-energy impacts (related to MEPS, not to energy labelling):
- Kemna, R. & Wierda, L. (2015). Ecodesign Impacts Accounting. Report of VHK prepared for the European Commission. https://ec.europa.eu/energy/sites/ener/files/documents/Ecodesign%20Impacts%20Accounting%20%20-%20final%2020151217.pdf
- Meyers, S, McMahon, J.E. & Atkinson, B.A. (2008). Realized and Projected Impacts of U.S. Energy Efficiency Standards for Residential and Commercial Appliances. Report of the Lawrence Berkeley National Laboratory, March 2008. https://eta.lbl.gov/publications/realized-projected-impacts-us-energy
Examples of studies about measurement campaigns and laboratory tests (additional method):
- Dupret, M. & Zimmerman, J.P. (2017). Electricity consumption of cold appliances, washing machines, dish washers, tumble driers and air conditioners – On-site monitoring campaign in 100 households. Analysis of the evolution of the consumption over the last 20 years. Proceedings of the ECEEE 2017 Summer Study, 1501-1509. https://www.eceee.org/library/conference_proceedings/eceee_Summer_Studies/2017/7-appliances-products-lighting-and-ict/electricity-consumption-of-cold-appliances-washing-machines-dish-washers-tumble-driers-and-air-conditioners-on-site-monitoring-campaign-in-100-households-analysis-of-the-evolution-of-the-consumption-over-the-last-20-years/
- Spiliotopoulos, C., Stamminger, R. & Siderius, H.P (2017). Bringing the home in the lab: consumer relevant testing for household electrical products. Proceedings of the ECEEE 2017 Summer Study, 1527-1540. https://www.eceee.org/library/conference_proceedings/eceee_Summer_Studies/2017/7-appliances-products-lighting-and-ict/bringing-the-home-in-the-lab-consumer-relevant-testing-for-household-electrical-products/