I’m fresh off of the road attending the US Department of Energy’s National Weatherization Training Conference in Indianapolis last week. This was my first trip to the Circle City, and I was pleasantly surprised. Clean, great architecture, nice bars, including an authentic 1897 German Rathskeller beer hall (I went there twice in three days).

I was also surprised by the information presented at one of the conference sessions. Michael Blasnik of Blasnik & Associates in Boston facilitated a session called “How Not to Save Energy.” I was excited about the session because I had heard lots of colleagues talk about Blasnik’s research.

Blasnik first explained his basic approach to energy efficiency data analysis: It’s easy to lie with statistics, but it’s a lot easier without statistics. So he reviews raw data, and lots of it. He analyzes information from as many homes as possible to get a realistic representation of performance without skews for occupant behavior. In the weatherization business, the savings-to-investment ratio (SIR) is a hugely important bit of datum. This is an indicator without units: the savings generated divided by the retrofit investment. If the SIR of an improvement is 1 or greater based on a 10-year payback period, it’s good. If the SIR is less than 1, then in some states weatherization assistance funds can be used for the percentage of the investment equivalent to the SIR ratio. For example, if a particular upgrade has a SIR of 2, then for every dollar invested in the upgrade, $2 is saved. If an upgrade has a SIR of .6, then DOE weatherization funds can pay for 60% of the strategy as long as the remainder of the funds are coming from somewhere else.

Blasnik discussed a few trends in residential energy efficiency that make sense. Homes that use a lot of energy have the potential for the most savings. Big energy users equals big energy savers. Low energy users equals low energy savers. This relationship follows around a 15% trendline. To put it another way, average homeowners can save around 15% on energy bills with some simple weatherization techniques, no matter the size of the home and the energy usage. This goes for natural gas and electric baseloads.

There are inaccuracies in what really saves energy and why homes that are weatherized aren’t saving as much as anticipated. Many times, occupant behavior gets the blame when it’s really that the wrong measures were taken to upfit the home. The data that Blasnik has seen over the years does not indicate that occupants altering thermostats is a significant reason behind lower-than-expected savings. The blame game also points the finger at people removing CFLs and low-flow showerheads, but Blasnik’s data shows that this only happens 25% of the time. Some folks blame lower savings on poor workmanship, but Blasnik again shows this to be false. Studies indicate that weatherization crews actually do a pretty good job with upgrades such as insulation installation.

So what is the real culprit? According to Blasnik, the reasons are twofold: The algorithms that are used to calculate the savings are just bad, and this leads to some of the wrong strategies being employed to save energy. The algorithms, Blasnik argues, are outdated and underrepresent equipment efficiencies. He says that these calculations ignore basic physics, assume a high heating balance point, and underestimate heat regain from basements and attics.

Bottom line, the algorithms used to calculate savings only measure the easily measured and implies that more sophisticated analysis is necessary to get to the root of the issue: What does/does not save real energy and real money?