by Bruce Rowse

Operating since 2009, the Victorian Energy Efficiency Target (VEET) scheme promotes the uptake of technologies that use energy more efficiently.

There is, however, substantial uncertainty as to the extent of greenhouse gas emissions (GHG) savings achieved by the scheme. For example, it’s been asserted that standby power controllers (SPCs) will save Victorian’s over $1 billion in electricity costs – but there is insufficient evidence to assert this claim. The absence of a robust measurement and verification (M&V) approach to validate energy savings has undermined the effectiveness of the VEET policy.

The VEET scheme and other energy efficiency policy are discussed in my upcoming book Carbon Policy – How robust measurement and verification can improve policy effectiveness.

The CarbonetiX submission on the recent public consultation on the VEET scheme below is just one of the examples presented in Carbon Policy – How robust measurement and verification can improve policy effectiveness.

CarbonetiX response to the June 2013 issues paper (VEET / Energy Saver Incentive)

CarbonetiX thanks the DPI for the opportunity to respond to the Regulatory Impact Statement (RIS).

As stated on the RIS webpage, the purpose of the RIS process is to ensure that:

• regulation is only implemented when there is a justified need;
• only the most efficient forms of regulation are adopted; and
• there is an adequate level of public consultation in the development of regulatory measures.

The Victorian government is acknowledged for its commitment to reduce carbon emissions. The Energy Saver Incentive (ESI) scheme was introduced primarily to reduce greenhouse gas (GHG) emissions, and the schemes targets are based on the amount of GHG the scheme saves.

The focus of our comments, therefore, is on the effectiveness of the ESI in reducing GHG emissions, and we address the following key issues:

-          What is policy effectiveness?

-          Has the ESI scheme been effective?

-          How can the effectiveness of the ESI scheme be improved?

What is policy effectiveness?

The effectiveness of a policy is dependent on the environmental benefit with respect to the economic costs and benefits of the policy.

I consider the cost of a carbon policy the total cost of the policy to society, which includes government investment in developing and administrating the policy (and in some cases, actually financing the policy) and the costs imposed by the policy either directly or indirectly on households and business. The ESI policy has sought to leverage household and business investment.

The financial benefits of a carbon policy are the benefits to society, such as reduced power bills. The key environmental benefit is the reduction in GHG emissions, although there may be other environmental benefits also of high importance, such as reduced air pollution because of a reduction in the combustion of fossil fuels.  The environmental benefit is why the ESI exists.

Can the transaction costs arising from a policy be considered a benefit? For example, under the ESI, electricity retailers have passed on their costs, probably with a mark-up, to consumers. These costs actually then reduce the financial benefits to end users. In the ESI scheme the power bill savings from reduced energy consumption may be partially, wholly or more than offset by the increase in electricity tariffs. Can any profit made by the electricity retailer be considered a benefit? It could be considered an economic benefit (i.e. to shareholders of the retailer), but this economic benefit comes at the cost of others (energy consumers). It adds nothing to the environmental benefit. While transaction costs are unavoidable, the larger the transaction costs for a given amount of carbon saved, the greater the cost of abatement.

Therefore, policy effectiveness, in my view, comes back to the total cost, including the transaction costs, per tonne of carbon saved. The lower the total cost per tonne of GHG emissions saved, the more effective the policy. With all other factors considered equal, this also translates into greater real economic benefit.

In this submission policy effectiveness is evaluated based on the total cost per tonne of GHG emission saved.

Why is policy effectiveness important?

Policy effectiveness is important for a number of reasons.

First, carbon policy fundamentally aims to reduce carbon emissions, and the less this costs, the greater the amount of carbon that can be saved within a given investment.

Second, an ineffective policy does not generate the spin-off effects of an effective policy. For example, a policy that does not effectively reduce the energy used to heat and cool a home does not improve comfort as much as it otherwise would have, and in addition, does not create national competitive advantage in being able to export skills, services, and products that effectively reduce home heating and cooling requirements.

Third, ineffective policy means carbon abatement is not an investment that can provide a financial return on investment, but rather an expense. Many carbon abatement technologies now exist that can, if applied efficiently, provide a reasonable financial return on investment. However, ineffective policy can result in poor or no financial return on investment.

Fourth, ineffective policy reduces the political capital that is so important to reducing carbon emissions. Ineffective policy damages credibility.

Has the ESI scheme been effective?

Unfortunately the effectiveness of the ESI scheme cannot be determined, as the scheme has not incorporated sufficient measurement and verification (M&V) to determine its effectiveness.  This failure could mean that the scheme has been far less beneficial than was as modelled in the scheme design.

The dominant energy saving measure implemented under the ESI scheme (also referred to as the VEET scheme in this document) has been the installation of Standby Power Controllers (SPCs). SPCs account for roughly half of the certificates generated in the system’s lifetime (based on the public register of certificates at the VEET website www.veet.vic.gov.au). Therefore the bulk of this discussion is centred around SPCs and the M&V of savings achieved by SPCs.

SPCs are like a form of power board, with a “master” and “slave”, or in the case of audiovisual (AV) equipment, an infrared detector that determines whether the remote control has been operated. When the master is turned off, power is automatically disconnected from the slave. Some AV SPCs also automatically turn off devices if activity from the remote controller (infrared) has not been detected for a certain period of time, which again saves power. SPCs are clearly a clever invention, but how much power do they save?

SPC savings

SPCs have dominated the VEET scheme since their introduction in mid-2011. In 2012, 78% of certificates came from the installation of SPCs.[i] This is because the deemed savings from SPCs are so high that they can be installed at no charge in residential buildings. Other measures, such as commercial lighting upgrades, which are popular in other Australian schemes that do not include SPCs,[ii] have had minimal uptake – it is hard to compete with something that is free!

The deeming methodology used for SPCs is based on two assumptions:

1. It is assumed that the device will be in use for 10 years.
2. The deemed savings are accurately determined by field trial.

With over 1 million of these devices now installed in Victoria, 9 million tonnes of GHG emissions is deemed to have been saved, with each SPC deemed to be in use for 10 years.[iii] To put this in context, 9 million tonnes is around 14% of the annual emissions arising from the use of electricity in Victoria.[iv] This represents fully half of the certificates generated in the lifetime of the VEET scheme since its commencement in January 2009 to date.

However, the question arises, “Is 9 million tonnes a realistic reflection of the actual deemed GHG emissions savings from SPCs?” In the absences of robust M&V this cannot be determined.

I would argue that there is a considerable body of evidence that the actual savings achieved by SPCs is lower than the deemed values, and  that it is not possible to determine the savings achieved by SPCs with any reasonable degree of certainty.

This is because, first, consumers are disconnecting and removing their SPCs. An article in The Melbourne Age reported that out of 1,000 households surveyed by the Essential Services Commission (the program administrator), 16% of SPCs were removed after installation.[v] This survey took place just a year after SPCs became eligible for use in the VEET scheme. A disconnection rate of 16% in the first year, for a product deemed to be saving energy for ten years, is not insignificant.

Second, the deemed values appear to be too high to be realistic. The same survey quoted by The Melbourne Age showed that 26% of devices did not have the right number or type of devices plugged in to qualify under the scheme’s rules, and thus would not likely achieve the deemed savings.

The deemed savings were determined by a combination of laboratory tests and field trial. The field trial methodology stipulated a minimum sample size of 10 AV and 10 IT devices in 20 households. The testing methodology required a baseline period (i.e. “before”) of 2 weeks of power logging, with 1 week of logging “after”.[vi] The methodology and sample sizes used, however, are clearly inadequate to give a high degree of statistical confidence in the savings achieved, especially over a ten year period.[vii] Through field trials, the most popular AV controllers in use in Victoria have been deemed to generate either 3 or 4 certificates and IT controllers 2 certificates.

Under the VEET scheme regulations, up to 4 SPCs can be installed per household.

One of the SPC manufacturers[viii] represents their product as follows:

-          With an average of two AV switches and 0.5 IT switches installed per home;

-          750,000 devices installed;

-          Savings of 2800 GWh (deemed electricity savings over the next 10 years).

By reverse engineering these numbers, the average savings per SPC (assuming a ratio of 2 AV switches to 0.5 IT switches as stated by the manufacturer) is thus 373 kWh/year, or just over 1 kWh/day, or 43 watts over the full 8760 hours in a year, for 10 years.

Based on the manufacturer’s statement of 2.5 SPCs installed per household, the deemed daily energy savings equate to just over 2.5 kWh, or more than 15% of likely average household electricity consumption.[ix]

Clearly, the deemed savings are achieved not just by standby savings, as 43 W represents a very high standby load that does not occur in reality.

Assuming that 4 devices are plugged into each SPC (the maximum possible under the scheme rules), this represents a standby load of nearly 11 W per appliance, which is reported to be the upper limit of standby loads.[x] In my experience of measuring standby loads while undertaking energy audits over the last 11 years, 10 W could be considered very high, and standby loads have been dropping during that period. The Third Survey of Residential Standby Power Consumption of Australian Homes – 2010[xi] found that the average standby load of home entertainment equipment was just 4.7 W, and for computer and peripheral equipment, it was determined to be 4.9 W, but the survey noted that “newer individual products on average are using less standby power”.

There has been a conscious effort by manufacturers to reduce standby loads, driven by regulation and the International Energy Agency’s 1999 One Watt Initiative,[xii] which aims to reduce standby loads to less than 1 W. In Australia, the One Watt Plan will be regulated from 2013.[xiii]

During the 10-year deemed life of an SPC, if a home retains the SPC but upgrades its AV devices (television, games console, computer, speakers, etc.), and these AV devices only have standby loads of 1 W or less, the savings achieved will decline significantly.

The savings deemed to be achieved from the SPCs also come from the energy saved by the SPC turning off appliances after a certain period of inactivity, such as televisions, and not just standby savings. and, in reality, based on the average standby figures from the Third Survey of Residential Standby Power Consumption of Australian Homes – 2010, standby savings would account for less than half the savings achieved by an SPC.

However, this turning off of unused devices has been a key driver for consumers to disconnect their SPCs – they can’t stand their televisions being switched off mid-program.

In addition, just as standby loads are dropping, so are the loads used by appliances such as televisions and gaming consoles. For example, with the advent of light emitting diode (LED) backlighting, screen power consumption (whether television or computer) is also dropping.

Based on high consumer-disconnection rates, the high proportion of SPCs with the number of appliances connected not as deemed and deeming factors that do not appear to adequately take into account the savings driven by ever more stringent appliance energy-efficiency standards, it appears to me that the million or so SPCs installed in Victoria will not save the carbon emissions over the 10 years they are deemed to. A failure to robustly measure actual savings to a high level of certainty over a sufficiently long period and adjust the deeming factors has led to this situation.

This appears to have distorted the VEET scheme’s market, depressed the certificate price, and meant that measures which may be more likely to achieve more certain GHG emissions savings have not been implemented.

The SPC manufacturer brochure referred to earlier boldly claims that by the end of 2012, $1 billion will be the total targeted deemed savings over 10 years from planned installations during 2012.

But the billion dollar question of “Will SPCs save a billion dollars?” can’t be answered, as savings haven’t been measured with much certainty. This is a serious and fundamental flaw in the design of the VEET scheme.

Accurate measurement of costs

A second key weakness of the VEET scheme, which applies to all measures implemented by the program, is the failure to measure the cost transfer through to all energy consumers. As such, the overall cost benefit of the VEET scheme to electricity consumers is unknown.

The financial return of the VEET program to energy users is represented by the following equation:

From a carbon perspective, the effectiveness of the program can be represented as follows:

However, the total cost is unknown, as there is no obligation for retailers to report the amount that they actually charge their energy consumers.

Retailers have passed on the cost of the program to electricity users through increases in electricity charges. These charges are not itemised on electricity bills where consumption is low (i.e. residential, small business), they only appear as line items for large energy users consuming more than 160 MWh of electricity in a year.

Retailers could choose not to pass on the cost of the certificates they are obliged to purchase to consumers, but there is no evidence that this is the case, and in fact, it appears as though the cost passed onto consumers is actually well in excess of the cost incurred in creating the certificates. For example, in 2012, my estimate is that the cost levied on electricity users was around $190 million.[xiv]On the basis of the average certificate price of approximately $22 for 2012,[xv] and regulator administration charges of $1 per certificate, my estimate is that for the 5.4 million certificates retailers were obliged to surrender in 2012, they together with the certificate brokers made around $60 million over and above what they paid for the certificates.

The effectiveness of the VEET scheme with respect to SPCs cannot be determined.

With no measurement in place to actually record benefit or total cost, the financial return and GHG emissions savings cannot be calculated with certainty, and the effectiveness of the policy supporting SPCs cannot be accurately determined.

The failure to accurately measure and verify the effectiveness of key elements of the VEET program, particularly SPCs, which have dominated the scheme, means that inefficiencies in the program have not been authoritatively identified. This has in turn made it much harder for policy makers to tune the scheme to improve its effectiveness and efficiency.

With SPCs dominating the scheme, it is improbable that the VEET scheme is resulting in the expected GHG emissions savings.

One could then argue that robust M&V is too expensive. But is it?

The cost of failing to incorporate robust M&V in the use of SPCs

Deeming methodologies, where there is low certainty of both short- and long-term outcomes, must incorporate robust M&V and thus require longer trial periods based on sound statistical principle to provide greater certainty of savings.

Let’s now consider the possible public loss of a failure to incorporate sufficiently robust M&V in the roll out of SPCs. For argument’s sake, let’s say that by undertaking robust M&V the deeming factor was determined to be of one-third the actual deeming factor now used.[1] Let’s also assume that the rest of the VEET program is robust, and the savings achieved from other measures in the scheme are genuine.

First, being a market-based scheme, any unfair advantage the SPCs may be enjoying would not have occurred.

Second, and depending on whether the SPC providers were still able to provide and install the devices for free, the number of SPCs rolled out would possibly be much lower than it actually is.

Third, and based solely on the 2012 roll-out of SPCs, GHG emissions savings would be around 4 million tonnes higher.[xvi]

Fourth, and based solely on the 2012 roll-out of SPCs, savings to Victorian electricity consumers would be around $1 billion higher.[xvii]

In the absence of robust M&V, I cannot assert that these figures are accurate, but this does illustrate the point that the failure to engage in robust M&V may have been extremely expensive to Victorians. The benefit of incorporating robust M&V into the program, by providing much greater certainty of savings and costs, far outweighs the cost of incorporating such M&V[xviii].

How can ESI policy effectiveness be improved?

As has been argued above the effectiveness of the VEET scheme cannot be determined, however there is a considerable body of evidence that indicates, with respect to SPCs, that the policy has been much less effective than modelled.

The VEET scheme should incorporate much stronger M&V in order to improve policy effectiveness.

Its not the purpose of this submission to provide detailed guidance on incorporating M&V into policy, which is covered elsewhere[xix].  However in general terms the ESI white certificate scheme should adopt the following principles for each measure that is included in the scheme.

M&V should be incorporated into policy development and implementation as follows:

1. Decide the intent of the policy;
2. Develop an M&V plan for the policy. This plan needs to include the following:

(a) A description of policy intent;

(b) A statement as to what level of certainty with respect to the carbon abatement/GHG emissions savings is required;

(c) Depending on the policy type, for each type of measure to be implemented, the plan should show how savings will be measured and verified to meet the certainty requirements (noting that this must address the long-term certainty of savings);

(d) A description regarding how total costs, including transaction costs, will be captured, in order to enable determination of policy effectiveness;

(e) For each type of measure, the plan should show how the results of M&V will be used to improve the policy and the frequency with which M&V and policy adjustment will occur;

(f) The plan should provide details of the budget that will be allocated to M&V not only for undertaking M&V reports but also for developing M&V plans for each measure, and for the iterative process of improving policy outcomes based on M&V reports.

1. Develop a plan for policy administration that ensures that the M&V function is adequately resourced;
2. Define the penalties that will be levied for falsification of information and how the measures implemented as a result of the policy will be audited;
3. Produce M&V reports in accordance with the M&V plan(s);
4. Adjust policy (including abandoning the policy or individual measures as appropriate) following each M&V report;
5. Assess and report on policy effectiveness.

Policy must use M&V to incorporate a feedback loop and incorporate a process by which the policy can be changed, and in fact, the expectation is that it will be changed and improved. This process should be incorporated from the outset and very clearly communicated as being a core part of the policy.

If there is high certainty that the savings from an energy-savings measure will be long lasting, then savings measured over a short timeframe, in accordance with the International Performance Measurement and Verification Protocol (IPMVP), can be extrapolated forward into the future. If there is less certainty of long-term savings, then M&V must be ongoing or periodic or else be undertaken in a way that allows prediction of future performance with a high degree of confidence.

Policy design must therefore take into account the certainty of long-term savings too, as this will determine the need for longer term M&V and the amount of time that may be needed to adequately trial a new technology.

The critical flaw in the design of the ESI scheme to date is that it relies on policy by modelling rather than policy by measurement.

The policy approach appears to have been as follows:

(a)    Formulate the policy;

(b)   Develop an economic model;

(c)    Contract a consulting company to develop the policy detail and to model detailed cost–benefit analysis;

(d)   Implement the policy;

(e)   Review it once, or perhaps twice.

The critical flaw in such policy making is that it assumes that the models developed and the detailed cost–benefit analysis are correct. But often, the certainty around the modelling accuracy is low, and in fact, in a world where technological change is happening exponentially, it is extremely hard if not impossible to model with great confidence. The modelling should be considered as a hypothesis; rather, it is treated as the truth.

In other words, the modeller takes an “educated” bet on the outcome, but there is no guarantee of certainty.

Policy by modelling is a linear approach with few feedback loops to gauge real policy impact or effectiveness, whereas policy with strong M&V is iterative with an emphasis on maximising real benefits.

Policy by modelling is similar to taking a bet at the races.

Policy with robust M&V is like putting your money on term deposit.

Modelled savings versus actual savings – and why modelling should be treated as a hypothesis

As an energy auditor, I have a great deal of experience in estimating or modelling the likely savings that will arise out of an investment in energy efficiency. I have audited over a thousand buildings and produced hundreds of energy audit reports.

An energy audit is essentially a business case for investment in energy efficiency. It lists a range of measures, and for each measure shows the estimated cost, annual savings, payback and annual GHG savings. It could be considered as a form of micro-modelling from a policy perspective.

It is extremely hard to model with a great deal of accuracy. Experience helps. In the following, I list some of the difficult learning experiences that I have been through as an energy auditor. What I’ve gleaned from these learning experiences has no doubt helped me improve my accuracy, but nonetheless it is still extremely difficult to estimate energy savings from many energy-efficiency measures with a high degree of confidence:

-          On one occasion, where I under-estimated the cost of a lighting controls upgrade in a school by 40%, I ended up doing a lot of work for free and paying contractors out of my own pocket to get the work complete.

-          On another, I overestimated savings from a lighting upgrade by 50%.

-          While in another instance, I guaranteed savings of 7% in a tender bid from installing a voltage optimisation unit. Fortunately, I lost that one. The company that won the job and put in technology similar to what I would have installed, only achieved a 5% saving. Phew! I was out by a factor of 40% in my estimate!

-          On another occasion, the advice I approved resulted in a $350,000 investment in a cogeneration unit. The expected annual cost savings were $27,000 but the actual cost savings were $0. The carbon savings were closer to the estimate. But clearly, this was not cost-effective carbon abatement.

And I am not the only energy auditor who can get it wrong. A study by Texas A&M when evaluating the work of pre-qualified energy auditors 5 years after projects had been implemented found that measured cost savings on average, across 24 projects, were 25.1% lower than estimated. In some cases, savings were as little as 5.5% of what was estimated![xx]

If energy auditors, who understand the details of technology and undertake site-specific investigations, can be out by 25% or more on individual projects, how accurate can policy modelling be? Modelling is usually based on a number of assumptions and possibly a small number of case studies, and the results are then assumed to apply to a large number of buildings. It is generally far less rigorous than an energy audit – an energy audit can account for the diversity in an individual building, but modelling needs to somehow account for the diversity across a whole range of buildings.

Policy modelling should only be treated as a hypothesis. Robust M&V is needed to determine the effectiveness of carbon policy with a high degree of certainty.

By failing to incorporate robust M&V into the ESI scheme its effectiveness cannot be determined. This should be rectified.

Bruce Rowse

Director

CarbonetiX Pty Ltd.