04 – Evaluating Minimum Energy Performance Standards measures on energy efficiency in Household appliances using direct measurements

The primary audience for this guide concerns energy efficiency programme designers, implementers or supervisors, and evaluators looking for guidance on the evaluation process of energy savings for energy efficiency policies.

Although the application of the tool will generally concern the national level, account will be taken of issues at EU level when relevant (e.g. issues related to the implementation of the EcoDesign Directive or to the eligibility of savings to EED article 7)¹.

This tool is not about the preceding step in the evaluation process, i.e. the choice of the method. About this previous step in the evaluation process, see the guidance provided here.
However, after presenting the capabilities and limitations of the tool at hand, the user will be offered alternatives for the method within this tool (see section 6).

The objective of this guide is to provide:

• Information on the scope of the tool that enables the user to decide whether this tool is suited to his/her needs, and whether complementary or additional method(s) could be needed or useful (section 2);
• Guidance about specifying the evaluation objectives and requirements (section 3);
• Guidance about key methodological choices to calculate energy savings (section 4);
• Guidance about the inputs (data requirements) and outputs of the method (energy savings metrics) (section 5);
• Possible alternative methods (with pros and cons) (section 6)
• Background about evaluation results other than energy savings (section 7);
• Relevant examples, case studies and/or good practices (section 8);
• Relevant references for further reading (section 9).

The tool is intended for assessing realised (ex-post) energy savings. However, account is taken of earlier (ex-ante) evaluations of expected savings, if available (see section 4).

The focus of the tool is on impact evaluation, i.e. determining the energy savings, but not on how this has been reached through a step by step process with intermediate results (process evaluation).

Readers looking for the basic and general principles of energy efficiency evaluation may find the following link useful.

¹ The implementation of the EcoDesign requirements cannot be eligible to EED article 7 due to the additionality rules defined in EED Annex V.

2.1 About legislative/normative measures

Legislative/normative measures for household appliances in practice come down to efficiency standards (often indicated as Minimum Energy Performance Standards – MEPS). MEPS for appliances can be considered specific standards as they are set specifically for a type (or even sub-type) of appliance (e.g. refrigerator, dishwasher, boiler). This distinction is sometimes made compared to system-wide standards that can apply to buildings (e.g. maximum gas consumption for new dwellings), encompassing several components (e.g., building envelope and heating systems).

MEPS are usually defined in connection to energy labels or classes. The implementation, and regular updates, of the MEPS can then be used to remove progressively from the market the appliances with the least efficient labels or classes.

In the European Union, the framework for MEPS for appliances is set by the Ecodesign Directive (2009/125/EC) in connection with the Energy Labelling Regulation (2017/1369/EU, earlier Energy Labelling Directive 2010/30/EU). For more details about their background and implementation, see here.

More information and examples on the different subtypes residing under the main type (legislative / normative measures) can be found in the Odyssee-Mure database and in the Knowledge Base of EPATEE.

More detailed information on the evaluation of legislative / normative measures can be found here

2.2 Evaluation for a combination of policy measure types

This specific guidance tool addresses legislative/normative measures, and more specifically MEPS (Minimum Energy Performance Standards). But sometimes a combination of policy measure types is implemented. In practice, MEPS are indeed most often implemented jointly with energy labelling schemes (type of policy: legislative/information). This is the case for the European legislative framework (joint implementation of the EcoDesign and Energy Labelling regulations). Moreover, complementary policy measures can be implemented at national level, such as a subsidy scheme to help the replacement of the most inefficient appliances in the stock.

More details about the case of energy labelling can be found in Specific Guidance 7 . Likewise, more details about the case of subsidy scheme for appliances can be found in Specific Guidance 14 .

With two or more policy measures stimulating the same saving actions, it is difficult to ascribe part of the overall savings to each policy measure type. A practical approach for combined policy measure types is to perform the evaluation for the set of policy measures as such, or to attribute the savings to the policy measure deemed to have the strongest effect. When legislative/normative measures are combined with other policy measure types it is assumed that the overall savings are mainly resulting from the policy measure at hand. However, the evaluation concerns the combined savings effect of both policy measures.

This guidance is not capable of attributing part of the (overall) calculated savings to each of the policy measures (see also double counting in the section 4 on Gross to Net savings).

For an example of evaluation aiming at distinguishing the respective effects of MEPS and energy labelling, see Bjerregaard and Møller, 2019.

2.3 Evaluation when combined with energy taxes

The calculated savings effect for MEPS for household appliances will overlap with that of the energy tax (in practice, mostly electricity tax). This guidance is not capable of attributing part of the (overall) calculated savings to either the MEPS or the energy tax. For dealing with this overlap, see section 4 on Gross to Net savings.

In principle, the effects of MEPS and energy tax are complementary: MEPS will remove from the market the least efficient appliances, whereas the energy tax will make the most efficient appliances more financially attractive (as higher energy prices will increase the differences in lifecycle cost). Therefore one practical way to allocate savings to both policy measures can be to calculate first the savings from MEPS, and then deduct them from the savings calculated for the energy tax.

2.4 About appliances in households

Information on (sub)sectors defined in the Toolbox can be found here, chapter 2, p.17

The subsector considered here only concerns appliances in households and explicitly excludes buildings.

The list of household appliances and devices (with sub-types when relevant) covered by EcoDesign regulations is (as of May 2019): Lighting; Heaters (Local space heaters, Space and water heaters, Solid fuel boilers, Air heating and cooling products); Fridges and freezers; Vacuum cleaners; Washing machines; Tumble driers; Air conditioners and comfort fans; Ventilation units; Air heating and cooling products; Televisions; Set-top boxes; Cooking appliances; Dishwashers; Pumps (Water pumps, Circulators); Computers; Game consoles; Off mode, standby and networked standby. However, this specific guidance tool does not apply to heating, ventilation and air conditioning technologies. For these technologies please refer to the guidance tool on buildings.

For more details and updates about the list of energy-using products covered by the EcoDesign and Energy Labelling regulations, see the link here.

Most of this guidance can also be applicable to MEPS for energy-using products in other sectors. Some of the energy-using products are indeed cross-cutting technologies (e.g. pumps).

Moreover the list of non-residential energy-using products covered by the EcoDesign and Energy Labelling regulations is (as of May 2019): Professional refrigerators; Industrial fans; Transformers and converters (Power transformers, External power supplies); Servers; Imaging Equipment; Electric motors; Tyres.

2.5 About the direct measurement method

When evaluating energy savings related to household appliances, the direct measurement means that meters or other energy consumption measurement devices are used to measure separately the electricity consumption of the appliance under investigation. This cannot be done by using data from electricity bills, as analysing bills data would require other methods to separate electricity consumption of the appliance under investigation from other electricity consumption in the dwelling (example of method: load disaggregation using smart meter data with very short time steps).

In practice, there can be two main types of direct measurement: on-site measurements and laboratory tests.

On-site measurements will usually be made with metering plugs installed at the outlet of the appliance to be measured. Non-intrusive measurement methods can also be used (e.g. high frequency metering at electricity meter), but they will require further analysis (e.g. load disaggregation algorithms).

Laboratory tests are often part of market surveillance activities. They use standardised cycles of use of the appliances, assumed to represent a normalized use. The electricity consumption is then recorded from measurement devices, along the test of the appliance.

Sub-type of method	Advantages	Limitations
On-site measurement with metering plugs or other devices installed at the appliance outlet	Reflect actual behaviours of the users High measurement accuracy (if reliable measurement device)	Large samples needed to get representative results (not influenced by outliers) Participation to measurement campaigns might be difficult to obtain (higher risks of self-selection bias) and plugs can be removed by end-users
On-site measurement with non-intrusive method	Reflect actual behaviours of the users (if reliable disaggregation method) Approval from end-users might be easier to get (smaller risks of self-selection bias)	Large samples needed to get representative results (not influenced by outliers) Uncertainties due to the method used to disaggregate the various end-uses included in the electricity consumption
Laboratory test	Can use small samples + test conditions well defined (cf. standardised cycles of use) High measurement accuracy (often certified)	Might not reflect actual behaviours of the users

For more details about measurement methods, see for example (Mort, 2017).

This guidance considers the case where direct measurement is used to determine the electricity consumption of both situations: the baseline situation and the situation observed with the MEPS. In case a different method is used to estimate the electricity consumption of the appliance for the baseline situation, it should be analysed if the use of a different method for both situations can create bias in the savings calculation (see Normalization factors in section 4).

Measurement in the case of energy performance standards for household appliances comes down to measuring the energy consumption of appliances with comparable functionality but with different performance standards.

Direct measurement provides data to calculate unitary energy savings while one unit is usually one appliance. See below Complementary methods to determine total savings (i.e. for the whole appliances counted for the MEPS).

Likewise, direct measurement usually implies significant costs (related to the measurement campaigns or laboratory tests). Therefore it is most often used on samples, as it would usually be impossible to make direct measurements on the whole appliances sold on the market or on the whole appliances of the housing stock. Additional methods related to sampling are thus needed to ensure the reliability of the savings estimates (see Additional methods at the end of this section).

Information about the various evaluation methods can be found here, table 1 and 2. This source also covers the combination of the method at hand with other methods, which will be dealt with below.

2.6 Complementary methods to determine total savings

Complementary methods are methods that are required, in this case in addition to direct measurement, to calculate energy savings.

Direct measurement is meant to calculate unitary savings for one appliance. These unitary savings should be multiplied by the number of actions (i.e. appliances bought) in order to get the calculated total savings. The number of actions or appliances can be obtained in various ways. See the link here, table 2 and 3 for more information.

When evaluating energy savings related to household appliances, direct measurement will usually be used to verify the electricity consumption per energy class or label. A complementary method will then be used to assess the market shares per energy class. The market shares without and with the implementation of the MEPS will finally be combined with the values of electricity consumption per energy class or label to obtain the total energy savings.

The complementary method can thus be a method to evaluate the market transformation effects of the MEPS.

A simplified approach can be to count as number of actions for the MEPS the number of appliances sold that have an energy efficiency corresponding to the MEPS (i.e. the minimum efficiency allowed on the market). The baseline situation can then be taken as the average energy consumption of the appliances sold over a similar period before the MEPS (or its update) entered into force, and that have an energy class or label with the same or lower efficiency than the MEPS.

For example, if the revised MEPS enforce that the minimum efficiency allowed is class C, removing from the market appliances of classes D and E previously allowed, then the number of actions will be the sales of appliances of class C in the year once the revised MEPS is enforced. And the baseline situation will be the average energy consumption of the sales of appliances with classes E to C in the year before the revised MEPS entered into force.

This approach can be simple to implement. However it can include several biases or disregard side effects, for example:

• Effects of the MEPS on the second-hand market (possibly reducing the effects of the MEPS)
• Effects of the MEPS on R&D for more efficient appliances (possibly increasing the effects of the MEPS)

For more discussions about the evaluation of market transformation effects, see Specific Guidance 7 about mandatory energy labelling for household appliances .

2.7 Additional methods to increase reliability of the results

An additional method can be applied on top of measurement to improve the reliability of the evaluation results and/or the cost-effectiveness of the evaluation approach.

As discussed about direct measurement methods, a first type of additional methods will be sampling methods to ensure that the estimates from the direct measurements can be considered representative of the whole appliances market or stock. For more guidance about sampling issues and methods, see for example (Khawaja et al 2017).

While direct measurement will usually have a high accuracy to determine the electricity consumption of the appliances measured or tested, it can have limitations or uncertainties in estimating the total savings, for example due to difficulties in defining a relevant baseline situation or in estimating representative results as mentioned above. The methods “stock modelling” or “diffusion indicators” can then be relevant additional methods to provide more reliable estimates about the baseline situation and the number of actions.

Surveys can also be used as an additional method to investigate factors influencing the energy consumption of appliances, such as user behaviours (see for example, Cabrera et al. 2015).

For possible combinations with an additional method see chapter 6 here.

3.1 Meeting evaluation goals and ambition

The table shows whether this tool can be used to report on general evaluation goals or criteria. See also this document.

General types of evaluation goals or criteria	Level of ambition	Remarks
Calculation of realized energy savings from saving actions	Medium to High	On-site measurements provide more reliable results than laboratory tests (see About the direct measurement method, in section 2)
Calculation of energy savings attributed to the policy measure(s)	Medium to High	Importance of the choice of the baseline Reliability will mostly depend on the complementary method used to assess total savings (see section 2)
Cost-effectiveness of saving action (for end-users)	Medium to High	On-site measurements are very reliable and energy cost savings can be determined from end-user energy prices. Differences in prices of appliances can be found from market surveys The main uncertainty source will usually be the lifetime taken into account (and related assumptions about savings evolution over time)
Cost-effectiveness of policy (government spending)	Medium	There is no government spending at the saving action level for legislative policies, but the government administrative cost (for defining the standards, and above all for enforcing them, e.g. market surveillance) can be compared with the combined savings result. Data about administrative costs might be difficult to find (see section 7)
CO2-emission reduction from saving actions	Medium to High	Results are usually monitored per type of appliance, which makes it easy to have energy savings per type of energy, then applying the corresponding emission factor Possible difficulties related to the emission factor for electricity (see section 7)
CO2-emission reduction attributed to the policy measure(s)	Medium to High	See comments above about the need for complementary method(s) (see section 2) and possible difficulties with the emission factor for electricity (see section 7)

When evaluating minimum energy performance standards (MEPS), the evaluation objectives can include investigating if the corresponding requirements are met in practice (cf. compliance issue), or if the requirements are defined in a relevant way and how they could be improved.

These issues are essential for the (cost-)effectiveness of the MEPS. Direct measurement and related additional methods can contribute to these evaluation objectives, for example:

• Laboratory tests are usual components of market surveillance activities to verify compliance of appliance available on the market (see for example Blomqvist & Fjordbak Larsen 2015; Lock et al. 2015).
• Detailed market analysis can help to identify defaults in the regulations (see for example Granda & Mauer 2017).
• Measurement campaigns and complementary surveys can help to understand the influence of behaviours and external factors (e.g. outdoor temperature) on energy consumption of appliances. These findings can then be used to improve savings estimates, as well as the way standards are defined (see for example Cabrera et al. 2015; Harrington 2017; Ruggieri et al. 2015; Stankovic et al. 2015).

For more information on verification of actual energy savings and attribution/baseline/corrections, see section 4, and for cost-effectiveness and emission reduction see section 7.

3.2 Reporting expectations

The method will make it possible to report (net) savings of minimum energy performance standards (MEPS) for household appliances.

In practice, direct measurement is usually associated with a very high effort (cf. costs related to the measurement campaigns or laboratory tests). Therefore it is most often used on samples, as it would usually be impossible to make direct measurements on the whole appliances sold on the market or on the whole appliances of the housing stock (see Additional methods in section 2 about sampling issues).

The unit used to express the savings will usually depend on the unit used to set the MEPS for appliances, i.e. in most cases final energy savings. This is because MEPS are usually based on energy labelling. Energy labelling is meant to inform end-users, i.e. about final energy consumption.

Other units can be used to answer other evaluation objectives:

• Primary energy savings: for example used when the evaluation objective is to assess the contribution of MEPS to national targets in terms of primary energy consumption or savings, or to assess the contribution to reduction of energy imports or security of supply. This metric requires data about primary energy factors (see section 5).
• Lifecycle energy savings: for example used when the energy consumption used to produce and recycle the appliances can be significant compared to the energy consumption of the using phase. This metric requires data from LCA (Life Cycle Analysis), which goes beyond the scope of this guidance.
• CO2 savings: often used when the MEPS include a focus on climate change impacts. This metric requires data about emission factors (see section 7).
• Lifecycle CO2 savings: same approach as for lifecycle energy savings.

3.3 Time frame for evaluation

The length of the period under evaluation is dependent on the active period of the policy measure, the need to monitor developments before the implementation of the MEPS or their revision (in case of methods based on before/after comparison to assess changes in market shares), and the time needed to present (reliable enough) results or impacts that fit into the decision making process. In some cases, the periodicity of evaluation can be set by law.

More specifically to MEPS, the evaluation will usually be planned so that results can be taken into account when preparing the revision or update of the MEPS. For example, to investigate if the way the requirements are specified should be improved or if additional provisions would be needed to ensure a better compliance. Direct measurement methods can also be used along the implementation of the MEPS, as part of market surveillance activities (for example to verify compliance of the products available on the market).

The planning of evaluation activities concerns regular monitoring of energy consumption and factors that define consumption, intermediate check of (ex-ante) estimated (unitary) savings through measuring or surveys, intermediate evaluations to improve the policy implementation and the final evaluation and reporting. Whereas laboratory tests can be done at any time once appliances are available on the market, on-site measurements can only be performed after the installation of the appliances in the dwellings. Laboratory tests can also be made on appliances collected by recycling companies (for example to investigate the energy performance of appliances after some years of use). This approach might require planning tests over several years (See also planning of evaluation in the link here.

To ensure the feasibility and optimise the costs of laboratory tests or measurement campaigns, it is strongly recommended to plan the evaluation in advance. The early phase of the evaluation process should in particular be used to define sampling strategy (see for example Khawaja et al 2017).

When using laboratory tests, it should be noted that the sampling strategy will usually not be the same if the primary objective is to verify compliance or to evaluate energy savings. For verifying compliance, the strategy is often a risk-based approach to target appliances with the highest risks of non-compliance, whereas for evaluating savings, the strategy needs to aim at obtaining a representative sample.

When using on-site measurements, the preparatory phase of the evaluation will be essential to define a strategy that can increase participation rate and limit self-selection bias (e.g. because households that are willing to participate in measurement campaigns are more likely to have a higher energy awareness than others).

3.4 Expertise needed for chosen method

A good knowledge of the policy background for energy efficiency in appliances (e.g. about the Ecodesign and energy labelling regulations) is a prerequisite to analyse the impacts of MEPS and possible interactions with other policy measures.

The use of direct measurement requires expertise in measurement devices and related uncertainties (see for example Mort 2017; Stamminger & Spiliotopoulos 2017).

Depending on the evaluation objectives, the following expertise can also be needed:

• About sampling methods (see for example Khawaja et al 2017).
• About survey methods (see for example Baumgartner 2017).
• About market data analysis and market transformation.

This section deals with key methodological choices to be considered when calculating energy savings: consistency between ex-ante and ex-post evaluation, baseline, normalization and adjustment factors. These choices are important to document when reporting energy savings, to ensure the transparency of the results.

The direct measurement method is applied using either laboratory tests or in-situ measurements of consumption of appliances directly in dwellings. It directly compares appliances regarding their energy efficiency.

General principles of calculating realized savings using different methods can be found here and here.

4.1 Matching method with earlier ex-ante evaluation

From the viewpoint of methodological consistency and data availability using the same method for the ex-ante evaluation and for the ex-post evaluation might be an obvious choice. However, for ex-ante evaluation only a few methods are usually considered, namely deemed savings, engineering estimate and stock modelling.

By essence, direct measurements on actions resulting from the MEPS (i.e. appliances sold after the MEPS entered into force) cannot be ex-ante. However, as MEPS aim at removing the least efficient appliances from the market, the appliances newly removed from the market due to the MEPS can be found in the market (and in the stock) before the MEPS enter into force. And usually likewise for appliances that have an efficiency equal to the MEPS. Therefore an ex-ante evaluation of MEPS can make use of data from direct measurements (either laboratory tests or measurement campaigns). However, the effects of MEPS on the number of appliances sold and shares per energy class cannot be based on observed market data (for the ex-ante evaluation). A different method than the one(s) used for the ex-ante evaluation can thus be applied for the ex-post evaluation, depending on the evaluation objectives, timeframe and data available for the situation after implementing the MEPS. For possible combinations of methods applied ex-ante and ex-post, chapter 7, calculation approaches in this document.

If the measurement method does not provide an acceptable combination with the earlier applied ex-ante method, it might be useful to select another method (see examples of alternatives in section 6).

In practice, ex-ante and ex-post evaluations are applied consecutively. The ex-post evaluation builds on an ex-ante evaluation that makes use of data coming from previous ex-post evaluation or studies (e.g. about previous periods of the MEPS, or about the same types of appliances as the ones targeted by the MEPS). These previous ex-post studies could have used another type of method as well.

When different methods are used for the ex-ante and ex-post evaluations, comparisons should be done with caution. Especially, differences in the baselines, data used and factors taken into account should be analysed. For example, if energy savings (or energy consumption) from direct measurements is compared with deemed savings (or standardised energy consumption from energy labels), differences can come from the fact that standardised energy consumption from energy labels do not necessarily reflect the actual use of appliances (cf. end-users’ behaviours vs. normalised cycles of use). These differences can also come from the fact that the data from direct measurements might have been obtained on samples that are not representative of the whole market or stock.

4.2 Calculation baselines

Energy savings are defined in general as the difference between the actual situation and a reference situation without the saving actions (and without the policy measures that influence these saving actions, here the MEPS). In case of saving actions the reference situation can be defined using various calculation baselines: Before/after, With/without, Trend, Target/control group and Minimum efficiency standards; see further here.

The main objective of MEPS is usually to remove the least efficient appliances from the market. As it does not provide directly an incentive for early replacement of appliances, it can be assumed that the appliances sold under the MEPS correspond either to old appliances that would have been replaced anyway, or to new appliances that would have been bought anyway.

Energy savings in this case are calculated by applying the with/without method: analysis of changes in energy consumption with/without the MEPS, where the “without” situation acts as the baseline for the “with” situation. Results can be corrected for adjustment factors, providing Net savings. The with/without baseline can also be applied more generally for evaluation of policy measures, especially to evaluate energy savings additional to a predefined scenario (e.g., business-as-usual scenario or a scenario including other policy measures). In this case, this predefined scenario will be used to specify the “without” situation.

Examples of baseline or business-as-usual scenario used to assess the impact of the Ecodesign regulations can be found in the Ecodesign Impact Accounting report done for the European Commission (see Wierda & Kemna 2019) for an overall scenario, or in the respective impact assessments made for the regulation for each type of appliance (see list of impact assessments in this document.

The definition of a baseline scenario often requires stock modelling or similar analysis, taking into account the existing stock of appliances, replacement rates or patterns, and trends in market shares (per energy class).

The results of the baseline scenario used to calculate the energy savings will usually be:

• Either directly a time series of average unitary energy consumption per type of appliance for the appliances sold each year included in the time series.
• Or a time series with the estimated market shares per energy class for each year included in the time series.

Baseline or business-as-usual scenarios are commonly used for ex-ante evaluations (for example for impact assessments of new MEPS or MEPS’ revision). Using the same baseline scenario for the ex-post evaluation helps ensuring consistency. However, the baseline scenario can also be adapted for the ex-post evaluation, for example to take into account unforeseen changes in the context (e.g. economic crisis) or improvements in data about appliances’ energy consumption.

Several baseline scenarios can also be used to test the robustness of the results or perform sensitivity analysis about key assumptions (Corry Smith, 2016).

This guidance considers as main baseline option the with/without comparison, possibly using a baseline scenario. However, depending on the objectives defined for the MEPS, the other policy measures that can interact with the MEPS and the evaluation objectives, other types of baseline can be used:

• “before” situation: in that case, the baseline is the energy consumption of the appliances that are replaced. This baseline can for example be used to estimate the energy savings as observed by the end-users, or to show the full energy efficiency improvements, independently of their causes. As explained above, this baseline is not appropriate to estimate the impact of MEPS, as MEPS are not meant to change the renewal rates of appliances. So it can be assumed that end-users would have replaced their appliances anyway.
• previous minimum energy performance standards: in that case, the baseline is the maximum energy consumption level as allowed in the MEPS previously in force. This baseline can for example be used when evaluating revised or updated MEPS, to show the incremental improvements compared to the previous MEPS.

The baseline option of a control group is rarely possible when evaluating MEPS. MEPS are usually implemented at national, or even supra-national (e.g. European Union), level. Therefore using a control group would usually mean to make a cross-country comparison which can raise issues to ensure that the evolutions in different countries can be compared.

See also here.

4.3 Normalization factors

The calculation with a with/without baseline provides a change in energy consumption that should be corrected for influences on energy consumption other than the saving actions. When evaluating energy savings from household appliances, these so-called normalization factors can for example be weather conditions (with effect on consumption, depending on the type of appliance), the rebound effect and other changes in energy using behaviours related to the appliances under evaluation.

For a general discussion about normalization factors, see table 1 in this document.

On-site measurements provide data about energy consumption in actual conditions of use. They therefore reflect the actual behaviours of end-users, including possible rebound effects in terms of intensity of use. The objective of using on-site measurements is indeed to investigate actual energy consumption, and possibly the influences of the factors that can affect energy consumption of appliances. Usually, the only normalization that needs to be applied to the measured data will be about weather conditions (when relevant for the appliances under investigation), for example with reference to degree-days.

For the behaviour-related factors, the issue will not be to normalize the data measured, but to make measurements on a sample that can be considered representative. Complementary surveys can also be needed to identify possible outliers (e.g. changes in the number of persons in the dwelling; persons newly retired or unemployed).

At the opposite, laboratory tests provide data for normalized cycles and conditions of use. In that case, no further normalization needs to be applied about external factors such as weather conditions. However, normalization factors might be needed about behaviour-related factors, when complementary studies show differences in the conditions considered for the tests and the actual conditions observed in measurement campaigns or surveys.

When evaluating policies related to appliances, a type of rebound effects other than the intensity of use might need to be considered: rebound effects related to the size or number of appliances. For example, the fact that the new appliance consumes less energy can make that households will buy larger or more sophisticated appliances, or will have more appliances of the same type (for example, keeping the old refrigerator as secondary refrigerator). These effects can be challenging to assess, as they can be due to other reasons than the MEPS. For example due to an increase of the household income, or to the marketing of the manufacturers and retailers encouraging households to buy larger appliances. The evaluation of these market transformation effects or trends goes beyond the scope of this guidance.

See also here.

4.4 Adjustment factors

Adjustment factors define which part of the calculated energy savings can be attributed to a policy measure or meets the definition of savings specified in the evaluation objectives or reporting requirements (see next section on “Calculating Gross and net savings).

Adjustment factors can concern the Free rider effect, the Spill-over/multiplier effect, Additionality and Non-compliance, see here.

Additionality and non-compliance are connected to unitary savings, while free riders and spill-over can have an effect on both, unitary savings and the number of actions (see next section on “Calculating Gross and net savings).

Double counting might be relevant in case of another policy focusing on the same saving actions as evaluated with the specific guidance tool at hand. Double counting can only be accounted for at a higher level than individual specific guidance tools; see Distinction of energy efficiency improvement measures by type of appropriate evaluation method, here and here.

For the case of MEPS, the adjustment factors free rider is not relevant because it is a regulation applying to the whole market (no choice for the buyers to use or not the MEPS). A kind of free-rider effect could be that part of the households would have bought the same appliance in the absence of the MEPS. However, this is assumed to be taken into account in the baseline scenario (representing the “without” situation). Likewise for additionality. So no further adjustment is needed about these factors. MEPS are usually not meant to influence the renewal rate of appliances. But MEPS can have other types of spill-over effects, for example on R&D and availability of more efficient appliances on the market. Manufacturers might indeed be willing to anticipate further reinforcements of MEPS. The evaluation of this type of long-term market transformation effects goes beyond the scope of this guidance.

When dealing with MEPS, the major issue in terms of adjustment factors is usually related to non-compliance. Non-compliance can be for example appliances that do not meet the MEPS but are still available on the market, or appliances with energy performance lower than stated by the manufacturers. Direct measurement, and especially laboratory tests, is one of the common methods to investigate non-compliance of appliances. Examples of evaluations or studies tackling the compliance issue can be found in Blomqvist & Fjordbak Larsen 2015; Granda & Mauer 2017; Lock et al. 2015.

Further information can be found in chapter 8 of this document.

4.5 Calculating Gross and Net energy savings

Gross savings concern the calculated savings from saving actions using a chosen baseline and normalization factors. Net savings concern the savings attributed to the policy measure, here the MEPS.

When calculating the savings a distinction must be made for most specific guides to the unitary savings and number of actions; see here.

In this guidance, the baseline option considered is a with/without comparison, where the without situation is based on a baseline scenario. The calculations using this baseline directly provide additional energy savings, i.e. energy savings additional vs. the baseline scenario.

Gross unitary savings can be calculated using another baseline option: the before/after comparison. A correction for relevant normalization factors (weather, rebound effect and other behaviour-related factors) is then applied.
The number of actions can be determined with the complementary method stock modelling or other analysis of sales data (see section 2).

MEPS are usually not meant to change the number of appliances sold, but to remove the least efficient appliances from the market. A key challenge when evaluating MEPS is to assess the share of the sales that has been impacted by the MEPS. It is usually related to the share of the sales corresponding to the least efficient appliances allowed. Assuming that without the MEPS, households might have bought appliances with lower energy performance than set in the MEPS.

The gross number of actions can be taken as the sales of appliances with the minimum energy class allowed by the MEPS. Total gross savings are then equal to gross unitary savings times gross number of actions.

Net unitary savings can be determined by using as baseline option the with/without comparison, the without situation being a baseline scenario taking into account previous market trends.

The baseline scenario can also be used to assess the net number of actions, by estimating what the sales of appliances would have been in the absence of the MEPS.

Net total savings are then obtained as the difference between the baseline scenario and the data observed in terms of market shares per energy class. Then the direct measurements provide measured energy consumption per energy class to complete the calculation of the savings. When relevant, adjustment factors about non-compliance can also be included in the calculation of the net savings, for example by adjusting the data of the observed scenario.

Net total savings are equal to net unitary savings times number of actions or unitary savings times net number of actions; see here and here.

The savings should be corrected for the Double counting effect, i.e. the overlap between the savings due to MEPS and savings due to other policy measures (e.g. energy labelling). The overlap in the calculated savings of both policy measures cannot be processed at the level of a Specific Guidance but must be corrected at the level of savings due to overall policy portfolios. For addressing double counting see earlier links above on DSM programmes.

Examples of evaluations or studies dealing with double counting or distinguishing the effects of MEPS and other policy measures can be found in (Bjerregaard & Møller 2019; Sulyma & Tiedemann 2015).

5.1 Main data requirements, sources and collection techniques

Data requirements specified in the table below correspond to the calculation of energy savings, when using the baseline option with/without.

Calculation subject	Data requirements	Possible data sources and collection techniques
Unitary energy consumption	Uncertainties due to the measurement devices Long enough measurement periods (for more details about measurement requirements, see for example Mort 2017) Representativeness of the samples used for the measurements (for more details about sampling issues, see for example Khawaja et al. 2017) Delineation of energy consumption of equipment affected by the savings action and energy consumption of other systemic components	Measurement campaigns, laboratory tests, surveys including sub-metering self-reported by households Recommended to define a measurement protocol to ensure the quality and consistency of the measurements (when not already available, as can be the case for standardised laboratory tests)
Normalization factors affecting energy consumption	Information on preferences in energy usage and energy-using behaviours Identifying the data or indicators that can be used to apply normalization factors (e.g. for weather conditions) Representativeness of the data collected	Literature research about rebound effects for specific technologies Complementary surveys (e.g. about energy-using behaviours) Complementary measurements or monitoring (e.g. about weather conditions, duration of use).
Adjustment factors	Data about non-compliance Representativeness of the verifications or controls made	Laboratory tests, market surveys, market surveillance activities
Primary energy factors applied (for conversion from final to primary savings)	Energy source saved (Employed technology)	National or international publications like EED Guidance notes.
Number of actions	Defining a baseline scenario Market (and stock, when relevant) data at least disaggregated per energy class	Sales data, stock modelling, market surveys or analysis, customer surveys, market transformation evaluation

Data issues when evaluating net energy savings

The main good practice to ensure the feasibility and reliability of the evaluation of net energy savings is to think about the method to be used when designing (or revising) the MEPS.

Experience indeed shows that unless the data collection has been planned in advance, it will be very costly, time-consuming or even impossible to collect the data required to apply most of the methods that can be used to evaluate net energy savings. Which makes that in practice using surveys will remain the only option possible (or considered feasible).

The main challenges when using surveys are:

• to achieve a high answer rate, in order to limit sampling bias
• to use question phrasing that can limit the risk of bias in the answers

For more details about the evaluation of net energy savings, see this topical case study

As discussed in section 4, when dealing with MEPS, one approach to evaluate net savings can be to use as baseline a scenario that is assumed to represent how the market would have evolved in the absence of the MEPS (e.g. by extrapolating previous market trends). This usually requires detailed stock modelling and market analysis, which can be challenging. Especially if disaggregated data (at least per energy class) about appliances in the stock and sales on the market are not easily available. Disaggregated sales data are often costly to acquire.

For possible other methods with different data demands, see section 6 on alternatives for the chosen method.

5.2 Energy savings in final terms or in primary terms

Energy savings can be expressed in final terms or in primary terms. See definitions about primary and final energy here.

Final energy savings are based on a reduction of energy consumption at the end-user premises and savings for all energy carriers are added up. For primary energy savings account is taken of the conversion losses when providing the energy carriers to end-users. E.g. for electricity 2-3 times the amount delivered to the end-user is used as input in power production. Therefore, saving one unit of electricity saves 2-3 unit of fuel in power production. The energy savings in primary energy terms provides savings that represent the reduction in primary energy consumption (before conversion in energy carriers for end-users), and before possible transport and distribution of energy that can generate further energy losses.

Direct measurements can calculate savings in final terms. It can also calculate savings in primary terms provided that savings at end-users are calculated for each energy carrier apart, and primary factors are available to convert the savings in final terms to savings in primary terms. When dealing with energy savings from appliances, energy savings will usually be calculated separately for each type of appliances, therefore easily providing energy savings per type of energy carrier.

Most of the appliances covered by MEPS use electricity. If the energy savings calculated over the lifetime of the appliances (see below), then it is important to clarify if the primary factor used for electricity in the calculations takes into account the likely changes in the electricity mix over the period (e.g. due to the increase in the share of Renewable Energy Sources), or if the current primary factor is used for all years.

5.3 Energy savings over time

Implemented saving actions in a year lead to savings over a number of consecutive years. E.g. a more efficient boiler can save gas over its lifetime of about 15 years, more efficient computers up to 5 years. Energy savings can be calculated in different metrics in terms of time reference, for example: year-to-year, annual, cumulated annual, cumulative. See the definitions here.

The calculated yearly savings concern the savings of all new saving actions in that year. In this approach only data for the savings in the chosen year are needed.

Adding up the yearly savings over a period, provided that earlier saving actions are still delivering savings, leads to cumulative savings. For the cumulative savings data are needed for the whole period.

Finally, savings from a saving action can be discounted and summed up over the lifetime of the action See link here.

Another cumulative approach, to be applied for the EED, is to multiply the (new) savings in a year with the number of years up to a target year and sum this result with that for all other years up to the target year. This cumulative approach stimulates early saving actions, as these count more times to the target than later actions. However, it should be noted that energy savings from the implementation of the Ecodesign regulations are not eligible to EED article 7. Only savings above the minimum energy performance levels set in Ecodesign regulations can be eligible.

Direct measurement can provide yearly savings of new saving actions in that year. It can also provide cumulative savings provided that data are available over a period, and data (or assumptions) are available about the lifetime of the appliances.

Finally, the method can provide discounted cumulative cashflow savings when discount factors (e.g. economic discounting or technical performance discounting) have been defined for yearly savings over time.

6.1 Alternatives for the chosen method

Often other savings calculation methods can be applied as well, although they will all have pros and cons regarding various aspects dealt with in preceding sections.

The table below presents the pros and cons of the method for evaluating normative/legislative measures in appliances in households, and for commonly used alternative methods for the same combination of policy measure and sector.

Type of method	Pros	Cons
Direct Measurement (method at hand)	High accuracy for the data on energy consumption Data that reflects actual use of the appliances (if on-site measurements; not for laboratory tests) Possibility to identify non-compliance (especially for laboratory tests)	Expensive and time-consuming Difficult to obtain data that are representative of the whole market or stock of appliances. Complementary methods needed to assess the number of actions (and market shares at least disaggregated per energy class) Analysis only after physical implementation of savings action
Engineering Estimates	Detailed estimate Relatively simple and cost-effective Calculation already after planning phase	Real savings can deviate from estimate Complementary methods needed to assess the number of actions (and market shares at least disaggregated per energy class) Participants may deviate from the planned action in actual implementation, leading to savings different from the ones calculated in the engineering estimate (not relevant as far as legislative measure is not used in combination with other measures)
Deemed savings	Simplest method for rather detailed estimate Very simple and cost-effective Calculation already available after planning phase	Imprecise May neglect systemic influences on savings Participants may deviate from the planned action in actual implementation leading to savings different from the ones calculated in the deemed savings (see above)
Stock modelling or diffusion indicators	Detailed analysis about the number of actions, possibly enabling an evaluation of market transformation effects	Complementary method needed to estimate the energy consumption per type of appliances (usually deemed savings, with the limitations mentioned above)

7.1 Calculating avoided CO2 emissions

Avoided CO2 emissions can be evaluated from the energy savings by applying emission factors. Four key aspects are to be taken into account when choosing the emission factor(s):

1. Emission factors vary according to the energy type, so the data about energy savings need to be available per energy type.
2. Emission factors for a given type of energy can vary over time (especially for electricity).
3. Emission factors can take into account:
   1. Direct emission factors: that take into account the emissions generated when producing the energy used;
   2. Lifecycle emission factors: which take into account all the emissions generated from the extraction of the energy resources up to the dismantling of the energy plant.

Due to the differences that the choice of emission factor(s) can induce, it is important to document what emission factor(s) has(have) been used.

The reduction in CO2-emissions can only be calculated when savings are calculated per relevant energy carrier and a specific emission factor is available for each energy carrier.

Most of the appliances covered by MEPS use electricity. It is important to document what emission factor is used for electricity, as there can be different ways to calculate this emission factor. For example, averaging the emissions over the whole annual electricity production, or taking into account that the electricity mix can be different according to the time of the day or the season, due to differences in the load curve and availability of capacities per type of electricity source. This latter approach can be used to define emission factor specific to each end-use, calculating the emission factor based on the load curve of the end-use.

The avoided emission of other greenhouse gases due to energy savings are not taken into account here, as these emissions (and more specifically their reductions) are generally negligible compared to CO2 (apart from policy measures targeting the agriculture sector).

IPCC (Intergovernmental Panel on Climate Change) provides a detailed database of peer-reviewed emission factors.

7.2 Calculating other co-benefits

Possible co-benefits from saving energy concern:

• Extra employment
• Reduction of energy poverty
• Other emission reductions (NOx, SO2, fine particles, etc.)
• Better indoor climate
• Reduced dependency on (insecure) energy import

The following co-benefits are regarded as most relevant and/or feasible to calculate in conjunction to this tool (see table).

The calculation of the co-benefits is explained in the references.

Type of co-benefits	Why it can be relevant (and for whom)	References where more details can be found
Employment	Investment in energy efficiency especially in the building sector generates local employment	https://combi-project.eu/

7.3 Other aspects of importance

For the following aspect an evaluation can be executed (see table).

Type of aspects	Why it is evaluated	References where more details can be found
Indoor air quality	It can worsen if the appliances emit too much (e.g. heat, CO2, nitrogen oxides, dust)	Evaluation of emissions of appliances (Wade, J. & Eyre, N. (2015)] in this link

Relevant EPATEE case studies:

• Dragovic, M., Broc, J.S., 2018. Nordsyn (Market Surveillance of Ecodesign and Energy Labelling) and the Effect Project in the Nordic countries. Case study prepared by IEECP for the EPATEE project, funded by the European Union’s Horizon 2020 programme.
• EPATEE Case Study [Denmark] Energy Companies’ Energy Savings Efforts
• EPATEE Case Study [Germany] Energy Efficiency Fund

One comprehensive study that among others treats household appliances has been performed by Joanne Wade and Nick Eyre (2015) for the UK Energy Research Centre. See in particular the Section 4.3 appliance market transformation activities of:
Wade, J. & Eyre, N. (2015): Energy Efficiency Evaluations: The evidence for real energy savings from energy efficiency programmes in the household sector. UKERC Technology & Policy Assessment Function (REF UKERC/RR/TPA/2015/001). http://www.ukerc.ac.uk/programmes/technology-and-policy-assessment/energy-efficiency-evaluation.html

Examples of evaluation of MEPS (possibly together with other policy measures) for energy efficiency in household appliances:

• Bjerregaard, C., & Møller, N. F. (2019). The impact of EU’s energy labelling policy: An econometric analysis of increased transparency in the market for cold appliances in Denmark. Energy Policy, 128, 891-899. https://doi.org/10.1016/j.enpol.2019.01.057
• Boardman, B. (2004). Achieving energy efficiency through product policy: the UK experience. Environmental Science & Policy, 7(3), 165-176. https://doi.org/10.1016/j.envsci.2004.03.002
• Lane, K., Harrington, L., & Ryan, P. (2007). Evaluating the impact of energy labelling and MEPS–a retrospective look at the case of refrigerators in the UK and Australia. European Council for Energy-Efficient Economy (Paris): Proceedings of the ECEEE 2007 Summer Study, 743-751. https://www.eceee.org/library/conference_proceedings/eceee_Summer_Studies/2007/Panel_4/4.292/
• Meier, A. K. (1997). Observed energy savings from appliance efficiency standards. Energy and Buildings, 26(1), 111-117. https://doi.org/10.1016/S0378-7788(96)01021-3
• Schiellerup, P. (2002). An examination of the effectiveness of the EU minimum standard on cold appliances: the British case. Energy Policy, 30(4), 327-332. https://doi.org/10.1016/S0301-4215(01)00099-4
• Vine, E., du Pont, P., & Waide, P. (2001). Evaluating the impact of appliance efficiency labelling programs and standards: process, impact, and market transformation evaluations. Energy, 26(11), 1041-1059. https://doi.org/10.1016/S0360-5442(01)00053-6

General guidance on evaluations:

• Baumgartner, R. (2017). Chapter 12: Survey Design and Implementation for Estimating Gross Savings Cross-Cutting Protocol. The Uniform Methods Project: Methods for Determining Energy-Efficiency Savings for Specific Measures. Prepared for NREL (National Renewable Energy Laboratory), September 2017. http://www.nrel.gov/docs/fy17osti/68568.pdf
• Eichhammer et al., 2008. Distinction of energy efficiency improvement measures by type of appropriate evaluation method. Final Report on Evaluation and Monitoring for the EU Directive on Energy End-Use Efficiency and Energy Services. EMEEES Project report. https://www.epatee-lib.eu/media/docs/EMEEES_WP3_Report_Final.pdf
• Hoffman, I., Schiller, S., Todd, A., Billingsley, M., Goldman, C., Schwartz, L., 2015. Energy Savings Lifetimes and Persistence: Practices, Issues and Data. Technical Brief, Lawrence Berkeley National Laboratory, May 2015. https://emp.lbl.gov/publications/energy-savings-lifetimes-and
• Khawaja, S., Rushton, J., Keeling, J. (2017). Chapter 11: Sample Design Cross-Cutting Protocol. The Uniform Methods Project: Determining Energy Efficiency Savings for Specific Measures. Prepared for NREL (National Renewable Energy Laboratory), September 2017. https://www.nrel.gov/docs/fy17osti/68567.pdf
• Mort, D. (2017). Chapter 9: Metering Crosscutting Protocol. The Uniform Methods Project: Determining Energy Efficiency Savings for Specific Measures. Prepared for NREL (National Renewable Energy Laboratory), September 2017. https://www.nrel.gov/docs/fy17osti/68565.pdf

About MEPS and policy packages for household appliances

• CSES & Oxford Research (2012). Evaluation of the Ecodesign Directive (2009/125/EC) – Final Report. Prepared for the European Commission, March 2012. https://www.eceee.org/static/media/uploads/site-2/ecodesign/products/ecodesign-directive-evaluation-functioning/ecodesign-evaluation-report-part1-en.pdf
• EPRS (2017). The Ecodesign Directive (2009/125/EC) – European Implementation Assessment. Report of the EPRS (European Parliamentary Research Service), November 2017. http://www.europarl.europa.eu/RegData/etudes/STUD/2017/611015/EPRS_STU(2017)611015_EN.pdf
• Nadel, S. (2002). Appliance and equipment efficiency standards. Annual Review of Energy and the Environment, 27(1), 159-192. https://stuff.mit.edu/afs/athena/dept/cron/Backup/project/urban-sustainability/Old%20files%20from%20summer%202009/Ingrid/Urban%20Sustainability%20Initiative.Data/PDF/annurev.energy.27.122001-2942156038/annurev.energy.27.122001.pdf
• Stamminger, R., Spiliotopoulos, C. (2017). Why is knowledge on measurement uncertainty so important in setting policies on energy efficiency? Proceedings of EEDAL (Energy Efficiency in Domestic Appliances and Lighting) 2017, 1094-1103.see https://ec.europa.eu/jrc/en/publication/eur-scientific-and-technical-research-reports/energy-efficiency-domestic-appliances-and-lighting-proceedings-4th-international-conference
• Tholen, L., Adisorn, T. & Götz, T. (2017). It’s nothing to write about? How EU-Member States address appliances in their National Energy Efficiency Action Plans. Proceedings of EEDAL (Energy Efficiency in Domestic Appliances and Lighting) 2017, 171-182. https://e3p.jrc.ec.europa.eu/events/eedal17
• Wiel, S., & McMahon, J. E. (2005). Energy-Efficiency Labels and Standards: A Guidebook for Appliances, Equipment, and Lighting (No. LBNL-45387-2nd-Edition; LBNL-45387). Ernest Orlando Lawrence Berkeley National Laboratory, Berkeley, CA (US). https://www.osti.gov/servlets/purl/877316-HHBvs5/

Umbrella report on the background studies for the EcoDesign regulations:

• Wierda, L. & Kemna, R. (2019). Ecodesign Impact Accounting – Status report 2018. Prepared by VHK for the European Commission, revised version January 2019. https://ec.europa.eu/energy/en/studies/ecodesign-impact-accounting-0

About methodologies to evaluate impacts from standards and labelling schemes for household appliances:

• Corry Smith, J. (2016). Ex-Post Impact Evaluations of Appliance Standards and Labelling Programmes: A Global Review of Best Practices and Lessons Learned. Proceedings of IEPPEC 2016. http://www.ieppec.org/wp-content/uploads/2016/05/Paper-Corry-Smith-1.pdf
• De Melo, C. A., & de Martino Jannuzzi, G. (2010). Energy efficiency standards for refrigerators in Brazil: A methodology for impact evaluation. Energy Policy, 38(11), 6545-6550. https://doi.org/10.1016/j.enpol.2010.07.032

Examples of measurement campaigns and other studies related to normalization factors:

• Cabrera, D., Bertholet, J.L. & Lachal, B. (2015). Survey of behaviour usage of refrigerators, light bulbs and stand-by in households. Proceedings of EEDAL (Energy Efficiency in Domestic Appliances and Lighting) 2015, 756-770. https://e3p.jrc.ec.europa.eu/publications/proceedings-8th-international-conference-energy-efficiency-domestic-appliances-and
• Harrington, L. (2017). Evaluating energy savings from replacement of old refrigerators. Proceedings of EEDAL (Energy Efficiency in Domestic Appliances and Lighting) 2017, 41-50. https://e3p.jrc.ec.europa.eu/events/eedal17
• REMODECE, Intelligent Energy Europe project: https://remodece.isr.uc.pt/
• Ruggieri, G. Zangheri, P., Pistochini, P., Bau, M., Villani, M.G., Fumagalli, S. & Tarantini V. (2015). Monitoring the energy consumption of fridge-freezers and washing machines of a sample of households in Northern Italy. Proceedings of EEDAL (Energy Efficiency in Domestic Appliances and Lighting) 2015, 174-188. https://e3p.jrc.ec.europa.eu/publications/proceedings-8th-international-conference-energy-efficiency-domestic-appliances-and
• Stankovic, L., Wilson, C., Liao, J., Stankovic, V., Hauxwell-Baldwin, R., Murray, D. & Coleman, M. (2015). Understanding domestic appliance use through their linkages to common activities. Proceedings of EEDAL (Energy Efficiency in Domestic Appliances and Lighting) 2015, 786-799. https://e3p.jrc.ec.europa.eu/publications/proceedings-8th-international-conference-energy-efficiency-domestic-appliances-and

Example about distinguishing effects from MEPS and energy labelling:

• Bjerregaard, C., & Møller, N. F. (2019). The impact of EU’s energy labelling policy: An econometric analysis of increased transparency in the market for cold appliances in Denmark. Energy Policy, 128, 891-899. https://doi.org/10.1016/j.enpol.2019.01.057
• Sulyma, I. & Tiedemann, K.H. (2015). Evaluating the Combined Impact of Financial incentives, Tax Incentives, Information and Minimum Energy Performance Standards in Transforming a Residential Appliance Market. Proceedings of EEDAL (Energy Efficiency in Domestic Appliances and Lighting) 2015, 1186-1196. https://e3p.jrc.ec.europa.eu/publications/proceedings-8th-international-conference-energy-efficiency-domestic-appliances-and

About issues related to compliance and pitfalls of MEPS

• Blomqvist, L. & Fjordbak Larsen, T. (2015). Effect project – estimating the benefits of Nordic market surveillance cooperation on eco-design and energy labelling. Proceedings of the ECEEE 2015 Summer Study, paper 8-266-15, 1837-1848. https://www.eceee.org/library/conference_proceedings/eceee_Summer_Studies/2015/8-monitoring-and-evaluation-building-confidence-and-enhancing-practices/effect-project-8211-estimating-the-benefits-of-nordic-market-surveillance-cooperation-on-ecodesign-and-energy-labelling/
• Granda, C. & Mauer, J. (2017). Impacts of a Loophole in US Fluorescent Lamp Efficiency Standards. Proceedings of EEDAL (Energy Efficiency in Domestic Appliances and Lighting) 2017, 30-40. https://e3p.jrc.ec.europa.eu/events/eedal17
• Lock, T., Muir, S., Faninger, T., Tinetti, B. & Krivosik, J. (2015). What’s on? Compliance of Televisions with Energy Labelling and Ecodesign Regulations. Proceedings of EEDAL (Energy Efficiency in Domestic Appliances and Lighting) 2015, 247-258. https://e3p.jrc.ec.europa.eu/publications/proceedings-8th-international-conference-energy-efficiency-domestic-appliances-and

About issues related to laboratory testing

• Fuchs, G., Roux, A., Siderius, H.P., Spiliotopoulos, C. & Stamminger, R. (2017). A methodology to assess consumer relevant aspects for product testing. Proceedings of EEDAL (Energy Efficiency in Domestic Appliances and Lighting) 2017, 1073-1081. https://e3p.jrc.ec.europa.eu/events/eedal17