20 – Evaluating impacts of Fiscal measures on energy efficiency in Transport using Stock modelling

The primary audience for this guide concerns energy efficiency programme designers, implementers or supervisors, and evaluators looking for guidance on the evaluation process of energy savings in the scope of this specific guide.

Although the application of the Specific guide will generally concern the (sub)national level, account will be taken of issues at EU level when relevant (e.g. the specific format of saving figures for the EED).

This guide is not about the preceding step in the evaluation process, the choice of the method. About this previous step in the evaluation process, see the guidance provided here. However, after presenting the capabilities and limitations of the guide at hand, the user will be offered alternatives for the method within this guide (see section 6).

The objective of the guide is to provide:

• Information enabling the user to decide whether this specific guide is suited to his/her needs, and whether complementary or additional method(s) are needed or useful (section 2);
• Guidance about specifying the evaluation objectives and requirements (section 3);
• Guidance about key methodological choices to calculate energy savings (section 4);
• Guidance about the inputs (data requirements) and outputs of the method (energy savings metrics) (section 5);
• Possible alternative methods (with pros and cons) (section 6)
• Background about evaluation results other than energy savings (section 7);
• Relevant examples, case studies and/or good practices (section 8);
• Relevant references for further reading (section 9).

The specific guide is intended for assessing realised (ex-post) energy savings. However, account is taken of earlier (ex-ante) evaluations of expected savings, if available (see section 4).

The focus of the specific guide is on impact evaluation, i.e. determining the energy savings, but not on how this has been reached through a step by step process with intermediate results (process evaluation).

Readers looking for the basic and general principles of energy efficiency evaluation may find the following link useful.

2.1 About fiscal measures

Fiscal measures in transport encompass taxes on energy consumption and on greenhouse gas & pollutants emissions (CO2, NOx, particulate matter, CO etc.), tax exemptions and tax reductions, special depreciation and other measures. Because EPATEE is about evaluating energy efficiency, the focus is on energy and greenhouse gas related taxes (as energy efficiency standards for cars are often formulated as CO2 emission standards). Taxes can be a higher purchase or road tax on inefficient vehicles and tax reductions for efficient ones. Polluting emission reductions are considered here as an additional benefit of energy efficiency policies. More information and examples on the different subtypes residing under the main type Fiscal/Tariffs (FT) can be found here and here.

To evaluate energy consumption or GHG emissions in relation to taxes on vehicle purchase, a reference situation in which the taxation measures are not present needs to be known. This situation can be constructed by using a vehicle stock model using purchasing behaviour and population data on incomes and household composition.

More detailed information on the evaluation of fiscal measures can be found here and here.

2.2 Evaluation for a combination of policy measure types

When the tax on vehicles purchases is combined with other policy measures types it is assumed that the overall savings are mainly resulting from the policy measure at hand. However, the evaluation in this case concerns the combined savings effect of both policy measures.

For attributing part of the (overall) calculated savings to each of the policy measures see section on Concrete examples.

2.3 Evaluation when combined with energy taxes

The calculated savings effect of the purchase tax (reduction) may overlap with that of a generic energy tax. The specific guide is not capable of attributing part of the (overall) calculated savings to either the policy measures at hand or the energy tax. For dealing with this overlap see section on Calculating Gross and Net savings.

2.4 About the transport sector

The major part of the energy consumption of the transport sector is due to passenger and freight road transport. The focus in this guide is on passenger road transport, as most policies until recently were targeted at passenger cars. Information on (sub)sectors defined in the Toolbox can be found here, chapter 2, p.17 .

A distinction needs to be made between the existing vehicle fleet and new cars. The purchase tax in many, but not all member states, involves only newly sold cars.

2.5 Evaluation for cross-sector saving actions

Cross-sector saving actions can be applied in different sectors. However, more efficient cars are confined to the transport sector. Thus no cross-sector saving actions are applicable.

2.6 About stock modelling

Information about the various evaluation methods can be found in this link, table 1 and 2, table 1 and 2. This source also covers the combination of the method at hand with other methods, which will be dealt with below.

To evaluate the effect of energy or emission based vehicle purchase taxes, a reference situation in which the taxation measures are not present needs to be known. This reference situation can be constructed by using a vehicle stock model. The model uses purchasing behaviour and data on population, incomes and household composition. The result is a gradual change in the stock of vehicles due to the yearly disposal of old cars and new cars of various types that are purchased. The stock model method allows to assess the (extra) number of saving actions (the number of more efficient cars), but not the unitary savings (actual energy saved by each more efficient car).

2.7 Complementary methods to determine total savings

Complementary methods are methods that are required, in addition to the primary selected method, to calculate the total energy savings. The method at hand concerns the determination of the number of participants that replace their old car with a certain fuel consumption with a new car belonging to a different fuel efficiency class or having better energy efficiency. In order to provide the total savings, the unitary savings per action (new more efficient car of the participant) should also be calculated, i.e. the set of differences in fuel use between various old types and various new types. A possible method to determine the unitary savings is deemed savings.

For more information about methods to calculate unitary savings, see this link, table 2.

2.8 Additional methods to increase reliability of the results

An additional method can be applied to stock modelling to improve the reliability of the evaluation results and/or the cost-effectiveness of the evaluation approach. Given a sound monitoring of the purchase of new cars, the stock modelling method will provide reliable data on the number of participants that replace their old car by a new one which is more fuel efficient. If deemed savings has been chosen as the primary method to calculate unitary savings, the reliability can be increased by regularly checking the deemed savings with measurement on sample basis.

For possible combinations with an additional method see chapter 6 in this link.

3.1 Meeting evaluation goals and ambition

The table shows whether this guide can be used to report on general evaluation goals or criteria. See also this document.

General types of evaluation goals or criteria	Level of ambition	Remarks
Calculation of realized energy savings from saving actions	high	If deemed savings checked regularly
Calculation of energy savings attributed to the policy measure(s)	medium	Other factors play a role (e.g. petrol price)
Cost-effectiveness of saving action (for end-users)	high	Can be calculated from change in tax level and difference in fuel use
Cost-effectiveness of policy (government spending)	high	Cost-effectiveness depends on the ratio between the revenues of the tax and the costs of tax deductions for the government
Greenhouse gas and pollutants emissions reduction from saving actions	high	Easy, based on fuel use and fuel types
Greenhouse gas and pollutants emission reduction attributed to the policy measure(s)	medium	See policy savings

For more information on verification of actual energy savings and attribution/baseline/corrections, see section 4, and for cost-effectiveness and emission reduction see section 7.

3.2 Reporting expectations

Possible reporting options:

• Gross and Net savings
• Yearly and cumulative (discounted) savings
• Rebound and free rider effects
• Market transformation due to purchase tax (only supply of more efficient cars).

3.3 Time frame for evaluation

The length of the period under evaluation is dependent on the active period of the policy measure, the need to monitor developments before the implementation of savings actions (in case of methods based on before/after saving actions), and the time needed to present (reliable enough) results or impacts that fit into the decision making process. In some cases, the periodicity of evaluation can be set by law.

The planning of evaluation activities concerns regular monitoring of energy consumption and factors that define consumption, intermediate check of (ex-ante) estimated (unitary) savings through measuring or surveys, intermediate evaluations to improve the policy implementation and the final evaluation and reporting. See also this link.

3.4 Expertise needed for chosen method

To be able to evaluate the effects of fiscal measures for passenger vehicles, one needs to have good insight in the past and current car market, the behaviour of cars purchasers according to their income and in all other external factors that can have an impact on energy consumption and on greenhouse gas emissions (driving costs). With this knowledge, a reliable reference situation can be constructed.

3.5 Boundaries for the evaluation

As vehicle purchase taxes and road and fuel taxes in general are set at the national level, the geographical area for evaluating fiscal measures for transport is limited to individual countries.

This section deals with key methodological choices to be considered when calculating energy savings: consistency between ex-ante and ex-post evaluation, baseline, normalization and adjustment factors. These choices are important to document when reporting energy savings, and to ensure the transparency of the results.

General principles of calculating realized savings using different methods can be found here and here.

The stock modelling method uses knowledge about the composition of the vehicle stock over time, vehicle purchasing behaviour, and purchasers characteristics (income, number of persons per household…), to construct a reference situation (without changes in purchasing tax). The difference between the energy consumption of the actual stock and the stock before the implementation of the measure (excluding the external effects) provides the fuel savings due to the purchase tax.

4.1 Matching method with earlier ex-ante evaluation

In the case of stock modelling, it is possible to also apply the method for ex-ante evaluation, provided that scenario data are available on all factors that define the purchase decisions that change the numbers of more efficient cars.

For the ex-ante evaluation of unitary savings only a few methods are usually considered, namely deemed savings and engineering estimates. From these options deemed savings is in line with the chosen method in the ex-post evaluation (see section on complementary methods).

A different method from the one(s) used for the ex-ante evaluation can be applied for the ex-post evaluation, depending on the evaluation objectives, timeframe and data available for the situation after implementing the actions. The ex-ante calculation of unitary savings using the deemed savings method, for instance, can be combined with the ex-post calculation of unitary savings using measurement (see section on additional methods).

For possible combinations of methods applied ex-ante and ex-post, see chapter 7, calculation approaches in in this (internal EPATEE) document.

4.2 Calculation baselines

Energy savings are defined in general as the difference between the actual situation and a reference situation without the saving actions (and without the policy measures that influence these saving actions). In case of saving actions, the reference situation can be defined using various calculation baselines: Before/after, With/without, Trend, Target/control group and minimum efficiency standards (for further information see here).

Total savings are found by multiplying the unitary savings per action by the number of actions. The chosen baseline can define total savings through the unitary savings, but also through the number of actions.

The applicable baseline in the case of stock modelling is ‘with/without’ (the purchase tax). Another possible baseline option is a minimum efficiency standard, but it is assumed that the purchase tax influences the above-standard fuel use. See this document for further information.

4.3 Normalization factors

The calculation using the ‘With/without’ baselines provides a change in energy consumption that should be corrected for influences on energy consumption other than the saving actions. These so-called normalization factors can be weather (with effect on consumption), the rebound effect and changes in energy using activities, such as eco-driving, the rate of car use or the fuel quality.

When stock modelling is used to determine the number of saving actions, in combination with deemed savings to determine the unitary savings per action, the normalization factors can concern the (direct) rebound effect, see table 1 in this link.

The rebound effect concerns extra energy consumption due to behavioural change (driving more km and faster) with lower fuel costs as a result of increased fuel efficiency. Therefore, less energy is saved than expected due to a longer distance travelled. The calculated change in energy consumption can be corrected for the rebound effect to get the gross savings.

Energy consumption is not corrected for weather, nor for changes in activity because driving more km is already covered in the rebound effect.

4.4 Adjustment factors

Adjustment factors define which part of the calculated energy savings can be attributed to a policy measure or meets the definition of savings specified in the evaluation objectives or reporting requirements (see next section on “Calculating Gross and Net savings).

Adjustment factors can concern the free rider effect, the spill-over/multiplier effect, additionality and/or non-compliance, see table 1 in this link. Additionality and non-compliance are connected to unitary savings, while free riders and spill-over work through the number of actions.

If another policy is focusing on the same saving actions as the one evaluated with this specific guide at hand, the adjustment factor for removing double counting should be implemented. If the other policy is not covered in the specific guide at hand, double counting can only be accounted for at a higher level than that of individual guides (see section on Calculating Gross and Net savings in this link and also here and here.

Adjustment factors to be considered are the free rider and spill-over effects and non-compliance. The free rider effect (car buyers that profit from the tax reduction but would have bought the same efficient car anyway) decreases the number of savings actions attributable to purchase tax (reduction). The spill-over/multiplier effect, e.g. a larger (and cheaper) supply of efficient cars due to the purchase tax, has the opposite effect of free rider effects. The non-compliance effect, meaning the efficiency of the car is worse than claimed by sellers due to not complying with the efficiency class, leads to lower unitary savings than expected according to the deemed savings method. For checking non-compliancy, analysis of sample data should be performed to compare effective savings to the “theoretical” ones assuming compliancy with the efficiency class claimed by sellers.

See this link.

To determine the impact of the free rider effect, a distinction must be made between saving actions due to the policy measure and actions which would have been taken anyway. The stock modelling method does not provide this information directly, so other ways must be found, such as a survey among participants of the policy measure about their motivation, or application of Randomized Controlled Trial (RCT) or Quasi-experimental design (see further in topical case study).

4.5 Calculating Gross and Net energy savings

Gross savings concern the calculated savings from saving actions using a chosen baseline and normalisation factors. Net savings concern the savings attributed to policy measures or to a stakeholder (e.g. an energy company with an obligation to realise savings at their customers).

When calculating the savings a distinction must be made for most guides to the unitary savings and number of actions, see this link.

The gross unitary savings can be calculated using the (complementary) method deemed savings and correcting for the normalization factors (here the rebound effect). The gross number of participants/actions is equal to the number of more efficient new cars purchased.

Net unitary savings can be calculated from gross unitary savings by applying the adjustment factor non-compliance (see previous section). The net number of participants/actions follows from correcting the gross number for the adjustment factors free riders and spill-over. See this link and this link.

The savings can be corrected for the double counting effect, i.e. the overlap between the savings due to vehicle taxes and savings due to other policy measures. The overlap in the calculated savings of both policy measures cannot be processed at the level of a specific guide but must be corrected at the level of savings due to overall policy portfolios. For addressing double counting link and this link.

Also see the section on concrete examples.

5.1 Main data requirements, sources and collection techniques

Data requirements specified in the table below correspond to the calculation of energy savings, when using the baseline option with/without.

Calculation subject	Data requirements	Possible data sources and collection techniques
Energy consumption	Per saving action: kilometres driven and fuel efficiency per vehicle type, driving manners (ecodriving, strengthening of speed limit regulation etc.)	Kilometres driven: surveys; fuel efficiency per car: effective fuel consumption measurements
Normalization factors: rebound effect or opposite	Increase in km driven compared to baseline Evolution of persons per vehicle	Surveys
Adjustment factors – free riders – non-compliance	Free riders: number of efficient cars bought independent of tax measure; non-compliance with performances claimed by sellers (energy class)	Free riders: surveys; non-compliance: effective fuel consumption and emissions measurements
Primary energy factors for conversion to primary savings	Conversion losses for producing transport fuels	Energy statistics
Number of actions	Changes in number of cars bought per efficiency group	Future: stock modelling; historic: stock data and newly sold car data from trade organisations
Purchasing behaviour	Data on family composition, income distribution	Statistics

Data issues when evaluating net energy savings

The main good practice to ensure the feasibility and reliability of the evaluation of net energy savings is to think about the method to be used when designing (or revising) the fiscal measure.

Experience indeed shows that unless the data collection has been planned in advance, it will be very costly, time-consuming or even impossible to collect the data required to apply most of the methods that can be used to evaluate net energy savings.

The main challenges when using surveys on the rebound and free rider effects are:

• to achieve a high and reliable answer rate on a representative sample in order to limit sampling bias;
• to ask questions limiting the risk of bias in the answers.

For more details about the evaluation of net energy savings, see the topical case study.

To determine the gross savings, first the number of actions is needed. For this, sales data per car type and knowledge about the effect of sales price changes on purchasing behaviour need to be known. This can be corrected for the rebound effect. For the energy saved per action, a deemed energy savings number can be used. To determine net savings, the free rider and non-compliance effects need to be applied.

Data issues with the additional method

For other methods related to different data demands see the section on alternatives for the chosen method.

5.2 Energy savings in final terms or in primary terms

Energy savings can be expressed in final terms or in primary terms. Energy savings can be expressed in final terms or in primary terms. See definitions about primary and final energy here.

Stock modelling combined with a method to determine energy saved per car can calculate savings in final terms. It can also calculate savings in primary terms provided that savings at end-users are calculated for each energy carrier separately, and primary factors are available to convert the savings in final terms to savings in primary terms.

5.3 Energy savings over time

Implemented saving actions in a year lead to savings over a number of consecutive years. A more efficient car can save fuel over its lifetime of about 10 years.

Energy savings can be calculated in different metrics in terms of time reference, for example: year-to-year, annual, cumulated annual, cumulative. See the definitions here.

The calculated yearly savings concern the savings of all new saving actions in that year. In this approach only data for the savings in the chosen year are needed. Adding up the yearly savings over a period, provided that earlier saving actions are still delivering savings, leads to cumulative savings. For the cumulative savings, data are needed for the whole period under considered. Another cumulative approach, as applied for the EU energy efficiency directive, is to multiply the (new) savings for a given year with the number of years up to a target year. This cumulative approach stimulates early saving actions, as these count more times towards the target than later actions.

Finally, savings from a saving action can be discounted and summed up over the lifetime of the action See link here.

Stock modelling can provide yearly savings of new saving actions in that year. It can also provide cumulative savings provided that data are available over a period.

Cumulative savings according to the Energy Efficiency Directive can be provided when the lifetimes of savings actions are known. Finally, the specific guide can provide discounted cumulative savings when discount factors have been defined for yearly savings over time. A relevant discount factor for efficient cars is that the energy efficiency of cars is improving anyway, so the baseline will improve over time.

6.1 Alternatives for the chosen method

The table below presents the pros and cons of the chosen method for evaluating fiscal measures in transport, and pros and cons for commonly used alternative methods for the same combination of policy measure and sector.

Type of method	Pros	Cons
Stock modelling	Clear and precise	Sizable data collection effort needed. Deemed savings to be corrected for free riders
Regression analysis	Part of total savings can be attributed to the fiscal measure (in the ideal case)	Complicated, difficult to make sure all factors are included

When using regression analysis, the (change in) a dependent variable, here energy consumption, is assumed to be a function of a number of explaining variables. These variables may be the level of energy using activities, energy prices or any variable influencing energy consumption. Time series should be available for all explaining and dependent variables.

In the regression formula, a factor is attached to each explaining variable. The energy consumption is calculated accordingly and compared to the observed energy consumption. The value of the factors is determined, using the least square algorithm, such that the calculated yearly energy consumption fits best to the observed energy consumption. The values found for each factor show the relative influence of each explaining variable on the (change in) energy consumption.

7.1 Calculating avoided CO2 emissions

Avoided CO2 emissions can be evaluated from the energy savings by applying emission factors. Four key aspects need to be taken into account when choosing the emission factor(s):

1. Emission factors vary according to the energy type, so the data about energy savings need to be available per energy type.
2. Emission factors for a given type of energy can vary over time (especially for electricity).
3. Emission factors can take into account:
   1. Direct emission factors: that take into account the emissions generated when producing the energy used;
   2. Lifecycle emission factors: that take into account all the emissions generated from the extraction of the energy resources up to the dismantling of the energy plant.

Due to the differences that the choice of emission factor(s) can induce, it is important to document what emission factor(s) has(have) been used.

The reduction in CO2 emissions can only be calculated when savings are calculated per relevant energy carrier and a specific emission factor is available for each energy carrier. In the case of electric vehicles, the reduction in CO2 emissions can be calculated from the savings with an emission factor for electricity that takes into account the different inputs of power production. The actual factor to be applied can vary, depending on saving action(s) and sector, year of implementation, policy considerations, etcetera, see here.

The avoided emission of other greenhouse gases due to energy savings are not taken into account here, as these emissions (and more specifically their reductions) are generally negligible compared to CO2 (apart from policy measures targeting the agriculture sector).

IPCC (Intergovernmental Panel on Climate Change) provides a detailed database of peer-reviewed emission factors.

7.2 Calculating cost effectiveness

Cost-effectiveness is the ratio between costs to achieve energy savings and the amount of savings and possibly other benefits.

A distinction can be made according to the point of view adopted to assess cost-effectiveness:

• Cost-effectiveness for the end-user or participant
• Cost-effectiveness for society at large
• Cost-effectiveness for the party that takes responsibility for saving targets (government or actor with an Energy Efficiency Obligation)

See further here, report on Evaluation into Practice: lessons learnt from 23 evaluations of energy efficiency policies, section 13.1.

The calculation of cost-effectiveness for end-users demands, apart from the savings, requires data on:

• investments made, interest rates, lifetimes of the saving actions per efficiency class
• energy prices (including taxes) per type of car and discount factors per type of end-user.

Here it involves the price of the vehicle, possibly interest rates for a loan, expected fuel consumption and estimated fuel prices.

Besides looking at costs from the end user perspective, one can use the societal costs perspective. This perspective involves costs for the country as a whole. For cost-effectiveness from a societal viewpoint, no account is taken of subsidies and taxes (as these stay within the country), energy prices concern world market prices, and a lower value for the discount factors is applicable. The societal costs can be negative if the change in vehicle tax results in lower fuel consumption and thus in lower fuel imports.

7.3 Calculating other co-benefits

Possible co-benefits from energy savings and CO2 emission reduction concern:

• Extra employment
• Reduction of energy poverty
• Other emission reductions (NOx, SO2, particulate matter, etc.). Emission factors specific to the road transport are available in the HBEFA database (Handbook of Emission Factors for Road Transport): http://www.hbefa.net
• Reduced dependency on (insecure) energy import

The co-benefit regarded as most relevant and/or feasible to calculate in conjunction to this specific guide is shown in the table below.
The methods to use for calculation of the co-benefits are detailed at the given web links.

Type of co-benefits	Why it can be relevant (and for whom)	References where more details can be found
Lower emissions of polluting substances	Polluting substances are detrimental for health	http://www.odyssee-mure.eu/data-tools/multiple-benefits-energy-efficiency.html https://combi-project.eu/

In the example case of ‘greening’ the purchase tax of passenger cars in the Netherlands, the effect of three policy measures was studied. The first was a bonus/malus system based on vehicle energy labels, the second a ‘guzzle tax’ for very inefficient cars, and the third was changing the vehicle purchase tax by relating it to vehicle CO2 emissions instead of to the catalogue prices of cars. The greening of the purchase tax appeared to be effective: new private cars sold between 2010 and 2012 had a 4 to 5% lower CO2 emission than without the policy. Private buyers appear to respond quite strongly to the purchase price of a car, and less so to road tax an fuel prices, which means that changing the purchase tax is an effective policy for influencing car choice. A factor that helped the policy to succeed was that a wider range of efficient cars came to the market around the same time. The 4 to 5% reduction in CO2 emissions was based on test results. In practice, the emissions were only 2.5 to 3% lower.

See the case study on “Greening of the purchase tax for passenger cars” in the reference section for further reading.

The case study “Voluntary agreement for freight companies” gives a concrete example on the evaluation of GHG and pollutants emissions in the transport sector in compliancy with European standard EN 16258.

See the case study on “Voluntary agreement for freight companies” .

About stock modelling

• Energy efficiency in the German residential sector – A bottom-up building stock model based analysis in the context of energy-political target. McKenna et al, Building and Environment journal, January 2013

About fiscal measures

• Evaluation of Fiscal Measures in the National Policies and Methodologies to Implement Article 7 of the Energy Efficiency Directive, Final report, Europe-Economics, October 2016

Relevant case studies

• Vergroening van de aanschafbelasting voor personenauto’s – Effecten op de verkoop van zuinige auto’s en de CO2-uitstoot (‘Greening’ of the purchase tax for passenger cars – effects on sales of efficient cars and the CO2 emissions (report in Dutch only) .