18 – Evaluating Financial incentives on energy efficiency in the agricultural sector using energy intensity indicators

The primary audience for this guide concerns energy efficiency programme designers, implementers or supervisors, and evaluators looking for guidance on the evaluation process of energy savings in the scope of this guide.

Although the application of the guide will generally concern the (sub)national level, account will be taken of issues at EU level when relevant (e.g. the specific format of saving figures for the EED).

This guide is not about the preceding step in the evaluation process, the choice of the method. About this previous step in the evaluation process, see the guidance provided here. However, after presenting the capabilities and limitations of the guide at hand, the user will be offered alternatives for the method within this guide (see section 6).

However, after presenting the capabilities and limitations of the guide at hand, the user will be offered alternatives for the method within this guide (see section 6).

The objective of this guide is to provide:

• Information on the scope of the guide that enables the user to decide whether this guide is suited to his/her needs, and whether complementary or additional method(s) could be needed or useful (section 2);
• Guidance about specifying the evaluation objectives and requirements (section 3);
• Guidance about key methodological choices to calculate energy savings (section 4);
• Guidance about the inputs (data requirements) and outputs of the method (energy savings metrics) (section 5);
• Possible alternative methods (with pros and cons) (section 6)
• Background about evaluation results other than energy savings (section 7);
• Relevant examples, case studies and/or good practices (section 8);
• Relevant references for further reading (section 9).

The guide is intended for assessing realised (ex-post) energy savings. However, account is taken of earlier (ex-ante) evaluations of expected savings, if available (see section 4).

The focus of the guide is on impact evaluation, i.e. determining the energy savings, but not on how this has been reached through a step by step process with intermediate results (process evaluation).

Readers looking for the basic and general principles of energy efficiency evaluation may find the following link useful.

2.1 About financial incentives

More information and examples on the different subtypes residing under the main type Grants & Subsidies (GS) can be found here and here.

Financial incentive policy is restricted to grants of subsidies concerning specific energy using systems which should make it more attractive to invest in the most efficient version.

More detailed information on the evaluation of financial incentives can be found here.

2.2 Evaluation for a combination of policy measure types

When the subsidy scheme is combined with other policy measures types (e.g. a voluntary agreement on savings) it is assumed that the overall savings are mainly resulting from the policy measure at hand. However, in this case the evaluation could concern the combined savings effect of additional policy measures.

The guide may not be capable of attributing part of the (overall) calculated savings to each of the policy measures (see also Double counting in the section on Gross to Net savings).

2.3 Evaluation when combined with energy taxes

The calculated savings effect for the subsidy scheme can overlap with that of the energy tax. The guide is not capable of attributing part of the (overall) calculated savings to either the policy measures at hand or the energy tax. For dealing with this overlap, see paragraph 4.5 on Calculating Gross and Net savings.

2.4 About the agricultural sector

Information on (sub)sectors defined in the Toolbox can be found here, chapter 2, p.17 .

The agricultural industry includes enterprises engaged in growing crops, raising fish and animals, and logging wood. It encompasses, among others, farms, dairies, hatcheries, greenhouses and ranches. Agriculture uses energy directly as fuel or electricity to operate machinery and equipment, to heat or cool buildings, and for lighting on the farm any indirect energy use in e.g. fertilizer industry or chemicals produced for the agricultural sector (for example fertilisers) are outside the scope of this advanced guide.

2.5 Evaluation for cross-sector saving actions

Cross-sector saving actions concern specific energy using systems, like electric drive of pumps that can be present in other sectors as well. This specific guide does not focus on these energy using systems. Therefore, this specific guide is not applicable for calculating cross-sector savings.

2.6 About the indicator method

Information about the various evaluation methods can be found here , table 1 and 2. This source also covers the combination of the method at hand with other methods, which will be dealt with below.

Energy efficiency indicators describe the ratio between energy consumption and the output/production/performance for which energy is used. If the yearly calculated ratio decreases this can be seen as an energy efficiency improvement.

A distinction can be made as to scale, where the unit consumption indicator concerns the equipment level and the energy-intensity indicator the (sub) sector level. This specific guide concerns the (sub)sector level of agriculture, where energy consumption can be related to, for example, the amount of crop, milk, meat, etc. However, getting reliable data for such consumption could be challenging. Please see also chapter 2.6 in Advanced Guidance #27.

A lower energy intensity can be due to other factors than energy efficiency, e.g. a shift in production to less energy-intensive products. In order to avoid these unintended effects, the intensity method should be applied at the lowest possible aggregation level with one uniform product type. Data availability and quality will depend on the level of disaggregation. One could start by collecting data available from national statistics bureau.

2.7 Complementary methods to determine total savings

Complementary methods are methods that are required, in addition to the primary selected method, to calculate energy savings. The method at hand provides directly the savings, instead of combining unitary savings (per action) with the number of actions. Hence no complementary method is needed. For further information see this link, table 2.

2.8 Additional methods to increase reliability of the results

An additional method can be applied on top of the indicator method to improve the reliability and accurateness of the evaluation results and/or the cost-effectiveness of the evaluation approach.

The method of energy indicator is a method that is based on statistical data on (for example) energy use per unit produced. In order to verify the energy indictor for a particular product or service the method may be combined by an additional method like measurement (direct measurement or billing analysis) or engineering method (for this method in combination with a voluntary agreement, please see the corresponding guide).The combination of methods can increase the reliability of the savings figures in a cost-effective way.

For possible combinations with an additional method see chapter 6 in this link.

3.1 Meeting evaluation goals and ambition

The table shows whether this guide can be used to report on general evaluation goals or criteria. See also this document.

General types of evaluation goals or criteria	Level of ambition	Remarks
Calculation of realized energy savings from saving actions	Low	For reasonably uniform savings actions
Calculation of energy savings attributed to the policy measure(s)	Low	Policy measure should be well documented
Cost-effectiveness of saving action (for end-users)	Fair	Required investment can be compared with savings in final energy use
Cost-effectiveness of policy (government spending)	Fair	Due to free rider effect
CO2-emission reduction from saving actions	Low	Can be integrated in engineering method
CO2-emission reduction attributed to the policy measure(s)	Low	Due to free rider effect

For more information on verification of actual energy savings and attribution/baseline/corrections, see section 4 of this guide and for cost-effectiveness and emission reduction see section 7 of this guide.

3.2 Reporting expectations

• Gross and Net savings, these are the savings that are calculated by the method of unit consumption (gross) or the adjusted savings attributed to the policy measure. See further in section 4 under the heading of Calculating Gross and Net savings.
• Yearly or cumulative savings, these energy savings can be ranked as cumulative over a specific period or on a yearly basis.

3.3 Time frame for evaluation

Relevant information regarding the time frame of the evaluation can include: the period for which the evaluated policy measure is active, the planning of the different activities necessary for the evaluation (the timely start of monitoring the “before” development, frequency and timing of data gathering, final monitoring after end of policy, etc.) and estimating roughly the time needed to make an evaluation depending on the type of method used.

The length of the period under evaluation is dependent on the active period of the policy measure, the need to monitor developments before the implementation of savings actions (in case of methods based on before/after saving actions), and the time needed to present (reliable enough) results or impacts that fit into the decision making process. In some cases, the periodicity of evaluation can be set by law.

The planning of evaluation activities concerns regular monitoring of energy consumption and factors that define consumption, intermediate check of (ex-ante) estimated (unitary) savings through measuring or surveys, intermediate evaluations to improve the policy implementation and the final evaluation and reporting (see also: section “Evaluation planning and preparation” in the link here.

3.4 Expertise needed for chosen method

Expertise required for the chosen method of indicators includes knowledge on access to energy statistical data per agricultural sector within a country. The expertise required has to be addressed in the overall evaluation protocol of this guide, at the start of the evaluation process.

3.5 Boundaries for the evaluation

Typical boundaries for this evaluation concern the defined subsector.

General principles of calculating realized savings using different methods can be found here and here.

This section deals with key methodological choices to be considered when calculating energy savings: consistency between ex-ante and ex-post evaluation, baseline, normalization and adjustment factors. These choices are important to document when reporting energy savings, to ensure the transparency of the results.

The method of indicator is strongly based on statistical data available for geographic regions, specific sectors and/or applications.

The Odyssey Efficiency Indicator is a good reference for efficiency indicators that should be well defined in a program regarding the application of energy efficiency indicators.

4.1 Matching method with earlier ex-ante evaluation

From the viewpoint of methodological consistency and data availability using the same method in the ex-ante evaluation and in this specific guide on ex-post evaluation might be an obvious choice. However, for ex-ante evaluation only a few methods are usually considered, namely deemed savings, engineering estimate and stock modeling.

A different method than the one(s) used for the ex-ante evaluation can be applied for the ex-post evaluation, depending on the evaluation objectives, timeframe and data available for the situation after implementing the actions. For possible combinations of methods applied ex-ante and ex-post, see chapter 7 in this document.

If the indicator method does not provide an acceptable combination with the earlier applied ex-ante method, it might be useful to select another method (see examples of alternatives in chapter 6).

In practice, ex-ante and ex-post evaluations are applied consecutively. The ex-post evaluation builds on an ex-post evaluation that makes use of data coming from previous ex-post evaluation or studies (e.g. about previous periods of the same policy measure, or about the same types of energy saving actions as the ones promoted by the new policy measure). These previous ex-post studies could have used another type of method as well.

4.2 Calculation baselines

Energy savings are defined in general as the difference between the actual situation and a reference situation without the saving actions (and without the policy measures that influence these saving actions). In case of saving actions the reference situation can be defined using various calculation baselines: Before/after, With/without, Trend, Target/control group and Minimum efficiency standards, see further here.

For the calculation of energy savings the baseline options concern Before/After, With/Without and Target/Control group. The Before/After calculation can be applied for existing energy using systems (e.g. dwellings). For new energy using systems, without a “before” situation, the With/Without calculation can be applied

For the “financial incentives – agriculture sector – energy intensity indicators” combination the baseline options Before/After or Trend can be used. With the Before/After option, the energy-intensity after introduction of the policy is compared with that before, that acts as the baseline. The difference between the two is used to calculate the energy savings. When there is already a gradual improvement in the energy-intensity before the introduction of the policy, the Trend option can be applied as baseline. The (steeper) decreasing trend for the intensity indicator after introduction of the policy is compared with the trend before that acts as baseline.

See also this link.

4.3 Normalization factors

The calculation with the Before/After baseline considered in the previous section provides a change in energy consumption that should be corrected for influences on energy consumption other than the saving actions. These so-called normalization factors can be weather (with effect on consumption), the rebound effect and changes in energy using activities, such as production (agriculture products), occupation rate (buildings) or car/truck/fork lift usage (transport).

For this specific guide on energy intensity in agriculture, the normalization factors concern weather and changes in production. Energy consumption can be corrected for differences in production volume and composition for the baseline situation and the situation after the saving action. Especially for horticulture, weather might be an important factor to correct energy consumption.

4.4 Adjustment factors

Adjustment factors define which part of the calculated energy savings can be attributed to a policy measure or meets the definition of savings specified in the evaluation objectives or reporting requirements (see next section on “Calculating Gross and net savings”).

Adjustment factors can concern the Free rider effect, the Spill-over/multiplier effect, Additionality and Non-compliance (see also this link, table 1).

With this specific guide, the savings are not calculated from unitary savings and the number of saving actions. Therefore, no adjustment for free riders is needed (see next section on “Calculating Gross and net savings).

Non-compliance will lead to lower savings than expected and will automatically affect the value of the intensity indicator “after”. Therefore, no correction is needed for this adjustment factor [see Table 1 in this link].

Double counting might be relevant in case of another policy focusing on the same saving actions as evaluated with the specific guide at hand. Double counting can only be accounted for at a higher level than individual specific guides.

More information related to distinction of energy efficiency improvement measures by type of appropriate evaluation method can be found in this link and here and here.

In case of top-down methods, such as the energy intensity indicators, special adjustment factors can also concern autonomous savings (or technological progress) and price-induced energy efficiency progress. However, this specific guide does not offer a way to correct for this adjustment factors.

4.5 Calculating Gross and Net energy savings

Gross savings concern the calculated savings from saving actions using a chosen baseline and normalization factors. Net savings concern the savings attributed to policy measures or to a stakeholder (e.g. an energy company with an obligation to realise savings at their customers). Net energy savings can be evaluated either directly (when using a control or comparison group) or from gross savings by applying further adjustment (or gross-to-net) factors.

When calculating savings a distinction must be made for unitary savings times the number of actions; see here.

The gross total savings are calculated using the Before/After or Trend baseline and correcting for relevant normalization factors (see section on normalisation factors above).

Net total savings can be determined from gross unitary savings applying the relevant adjustment factors (see section on adjustment factors). See here and here.

Net savings will incorporate the effect of non-compliance due to inappropriate implementation of saving actions.

The savings should be corrected for the Double counting effect, i.e. the overlap between the savings due to Financial incentives and savings due to other policy measures. The overlap in the calculated savings of both policy measures cannot be processed at the level of a specific guidance but must be corrected at the level of savings due to overall policy portfolios. . For addressing double counting see here or here.

For additional information, see section 8 on Concrete examples.

5.1 Main data requirements, sources and collection techniques

Data requirements specified in the table below correspond to the calculation of energy savings, when using the baseline option Before/After.

Calculation subject	Data requirements	Possible data sources and collection technics
Energy consumption	Fuel and electricity consumption at the level of subsectors with uniform products (contingent on reliable data)	Energy consumption statistics
Factors affecting energy consumption	Data on production volume (uniform products)	Production statistics or branch data
Normalization factors weather and production changes	Actual and historical data on outdoor temperature to correct heating and cooling energy use	Weather statistics
Primary energy factors applied (for conversion from final to primary savings)[	Conversion factor for electricity	Statistics on input and output of power production

Data issues when evaluating net energy savings

The main good practice to ensure the feasibility and reliability of the evaluation of net energy savings is to think about the method to be used when designing (or revising) the Financial incentive.

For more details about the evaluation of net energy savings, see this topical case study.

5.2 Energy savings in final terms or in primary terms

Energy savings can be expressed in final terms or in primary terms. See definitions about primary and final energy here.

Final energy savings are based on a reduction of energy consumption at the end-user premises and savings for all energy carriers are added up. For primary energy savings account is taken of the conversion losses when providing the energy carriers to end-users. E.g. for electricity 2-3 times the amount delivered to the end-user is used as input in power production. Therefore, saving one unit of electricity saves 2-3 unit of fuel in power production. The energy savings in primary energy terms provides savings that represent the reduction in primary energy consumption (before conversion in energy carriers for end-users), and before possible transport and distribution of energy that can generate further energy losses).

The indicator method can calculate savings in final terms. It can also calculate savings in primary terms provided that savings at end-users are calculated for each energy carrier apart, and primary factors are available to convert the savings in final terms to savings in primary terms.

5.3 Energy savings over time

Implemented saving actions in a year lead to savings over a number of consecutive years. E.g. a more efficient boiler can save gas over its lifetime of about 15 to 25 years, insulation over up to 60 years and more efficient computers up to 5 years. For savings from behavioral changes due to a media campaign the life time might be not much longer than that of the campaign. Energy savings can be calculated in different metrics in terms of time reference, for example: year-to-year, annual, cumulated annual, cumulative.

See the definitions here.

The energy intensity indicator method calculates a ratio for a number of years that encompasses all saving effects of all saving actions, whether due to the policy evaluated or other influences. Therefore, no results on individual saving actions, with their introduction year and lifetime, are available. Thus this specific guide cannot deliver cumulative or discounted savings.

See also this link.

6.1 Alternatives for the chosen method

Often other savings calculation methods can be applied as well, although they will all have pros and cons regarding various aspects dealt with in preceding sections.

The table below presents the pros and cons of the engineering estimate method as alternative for evaluating a subsidy based voluntary agreement in agriculture. The focus of the engineering estimate method is on individual saving actions, such as more efficient greenhouses (horticulture), boilers or air condition units (stables).

Type of method	Pros	Cons
Method at hand: Energy intensity indicator	Relatively cheap and simple method using aggregated statistical data	Should be applied at the level uniform production, for which energy data might not be available. Delivers total savings instead of the savings due to policy
Alternative method: engineering estimates	Far more transparent analysis of the effect of policy on more efficient energy using systems. Better view on adjustment factors and possibility to apply different formats for resulting savings	Extensive surveys needed on implemented numbers per type of energy using system

7.1 Calculating avoided CO2 emissions

Avoided CO2 emissions can be evaluated from the energy savings by applying emission factors. Four key aspects are to be taken into account when choosing the emission factor(s):

1. Emission factors vary according to the energy type, so the data about energy savings need to be available per energy type.
2. Emission factors for a given type of energy can vary over time (especially for electricity).
3. Emission factors can take into account:
   1. Direct emission factors: that take into account the emissions generated when producing the energy used;
   2. Lifecycle emission factors: that take into account all the emissions generated from the extraction of the energy resources up to the dismantling of the energy plant.

Due to the differences that the choice of emission factor(s) can induce, it is important to document what emission factor(s) has(have) been used.

If the savings concern one fuel only (e.g. for gas boilers):

The reduction in CO2-emissions can be calculated from the savings with an emission factor for the fuel at stake.

If the savings concern electricity only (e.g. appliances):

The reduction in CO2-emissions can be calculated from the savings with an emission factor for electricity that takes into account the different inputs of power production. The actual factor to be applied can vary, depending on saving action(s) and sector, year of implementation, policy considerations, etc., see for example here.

If the savings concern different energy carriers:

The reduction in CO2-emissions can only be calculated when savings are calculated per relevant energy carrier and a specific emission factor is available for each energy carrier

The avoided emission of other greenhouse gases due to energy savings are not taken into account here, as these emissions (and more specifically their reductions) are generally negligible compared to CO2 (apart from policy measures targeting the agriculture sector).

IPCC (Intergovernmental Panel on Climate Change) provides a detailed database of peer-reviewed emission factors.

7.2 Calculating cost-effectiveness

Cost-effectiveness is the ratio between costs to achieve energy savings and the amount of savings and possibly other benefits.

A distinction can be made according to the point of view adopted to assess cost-effectiveness:

• Cost-effectiveness for the end-user or participant
• Cost-effectiveness for society at large
• Cost-effectiveness for the party that takes responsibility for saving targets (government or actor with an Energy Efficiency Obligation)]

See further here, report on Evaluation into Practice: lessons learnt from 23 evaluations of energy efficiency policies, section 13.1.

The calculation of cost-effectiveness demands, next to the savings, data on investments made, subsidies on investments, interest rates, lifetimes of the saving actions, energy prices (including taxes) per type of end-user and discount factors per type of end-user. Because this specific guide concerns aggregated savings, without data on specific saving actions, it does not offer the possibility to present cost-effectiveness results.

7.3 Calculating other co-benefits

Possible co-benefits from saving energy concern:

• Extra employment
• Reduction of energy poverty
• Other emission reductions (NOx, SO2, fine particles, etc.)
• Better indoor climate
• Reduced dependency on (insecure) energy import

Due to the aggregated nature of the results this specific guide cannot calculate other co-benefits except reduced dependency on imports due to the calculated total savings for the agriculture subsectors.

In 2008, the Dutch agro sectors signed a Public-Private Partnership with the government entitled ‘Clean and Efficient Agro Sectors’, also referred to as the ‘Agro Covenant’, setting out pledges to make the sectors more sustainable by 2020. Targets include reducing CO2 emissions in 2020 by at least 3.5 Mt and other greenhouse gases like methane and nitrous oxide by 4.0 to 6.0 Mt (in CO2-equivalents). In addition, 200 petajoules of renewable energy from biomass must be produced per year, as well as 3.5 billion kilowatt-hours per year of wind energy.

In order to achieve the targets and ambitions in the Dutch Agro covenant, greenhouse sector and the government are working together in the energy transition programme ‘Greenhouse as Energy Source’ (KaE). The aim of KaE is that from 2020 climate neutral and economically profitable cultivation will be possible in new greenhouses. Translated into an energy indicator, this means that in net terms (purchase minus sales) primary fuel will no longer be required in new greenhouses.

In order to achieve this aim, transition paths are being developed. These focus on the reduction of the demand for energy, the more efficient production of energy, the production of sustainable energy and the sale of energy. There is also attention for the optimum use of light and other production factors which can improve energy efficiency. The greenhouse sector also buys more efficiently produced and sustainable energy, but this is not part of the transition paths.

In this context, an Energy Monitor has been developed. It quantifies and analyses the developments and backgrounds of energy efficiency, CO2 emissions and the share of sustainability. Furthermore, it monitors the energy input, the energy output and the electricity balance (purchasing, production, consumption and sales) of the greenhouse sector, as well as part of the KaE transition paths.

The energy efficiency is a relative indicator, defined as the primary fuel consumption per unit produced. So it is taken into account with the size of the horticultural production for which the fuel is used, what a measure for the sustainability of production. The share of renewable energy used is also a relative indicator and is expressed as a percentage of the total energy consumption of greenhouse horticulture. The CO2 emission is expressed in Mton per year and is determined according to the IPCC method and covers the entire greenhouse horticulture sector. Being distinguished based on the total CO2 emissions of the sector and the CO2 emissions for cultivation (exclusive sale of electricity).

For further details, see this report.

N. v.d. Velden (LEI), Energiemonitor van de Nederlandse Glastuinbouw 2009 (Agricultural monitor) (in Dutch with English summary), available at http://edepot.wur.nl/15538

General guidance on evaluations

• Eichhammer et al., 2008. Distinction of energy efficiency improvement measures by type of appropriate evaluation method. Final Report on Evaluation and Monitoring for the EU Directive on Energy End-Use Efficiency and Energy Services. EMEEES Project report.

General guidance on energy efficiency indicators

• Energy Efficiency Indicators: report on “Best practice and potential use in developing country policy making “.