The Nest Learning Thermostat has been on the market for nearly four years now. One of the biggest things the Nest folks use as a selling point is energy savings. “Programs itself. Then pays for itself.” That’s the first thing you see when you go to the Nest homepage. But what do the data say? Three independent studies plus a white paper from Nest provide some answers. (GBA first reported on these three studies in a February 2015 news story.)

The first study was done on 185 Oregon homes that used heat pumps, and the study was conducted over one heating season (2013-14), with the report released in October 2014. Two nearly identical studies run by two different gas and electric utilities in Indiana looked at both heating and cooling in a total of 700 homes. The results were nearly identical, too. Finally, Nest published a white paper in which they analyzed the energy use of 735 Nest owners with gas furnaces and 624 Nest owners with electric cooling. (See links to all studies at end of article.)

Let’s take a look.

A study in Oregon

This one was focused on heating with heat pumps. The goal was to find a way to reduce the amount of electric resistance heat that typically is installed as a supplemental heat source in heat pumps. The Energy Trust of Oregon, which launched the research project, was looking for an alternative to their “advanced heat pump controls measure.” As I showed in my recent article on Michael Blasnik’s presentation on big data from Nest, there’s a lot of opportunity for savings in heat pump supplemental heating.

The Energy Trust installed the Nest thermostats in 185 homes, and 174 of those made it all the way through the study with the thermostat installed and working. The researchers looked at energy use in the heating season before and the heating season after installation of the Nest. They also interviewed the participants at the end of the study period.

Here what I think are two of the biggest results they found when they looked at the final data:

• Homes averaged 12% savings of electricity used for heating after installing the Nest. (These numbers have been adjusted for weather.)
• More than 60% of the participants found their homes “somewhat more comfortable” or “much more comfortable” while using the Nest thermostat.

Then there were some interesting things that fell out when they analyzed the data in other ways. They looked at all kinds of demographics — age, education level, housing type, geographic location, and much more — and here are a few that struck me as noteworthy.

• Manufactured homes had the highest savings, almost twice as much as the overall average.
• In homes where the Nest replaced a programmable thermostat, the savings were about twice as high as in homes where it replaced a non-programmable thermostat. The uncertainties were high enough, though, that the report says those numbers only “suggest” that the Nest does better when replacing a programmable thermostat.
• The homes that used the most energy saved the most energy (1,785 kWh per year), whereas the group with the lowest incomes had the highest percent savings: 11.1% (1,654 kWh).

Although the results are encouraging, the Oregon report urges some caution:

We have some reservations about the reliability of the results from the subgroup analyses. Each subgroup comparison began with a relatively small sample of pilot homes and cut it into even smaller pieces to analyze. With such small samples to work with and so many comparisons that the team was interested in, there may have been random fluctuations in the data that resulted in observing spurious differences.

In other words, they feel more confident about the numbers based on all the participants, less so about the ones based on things like those who live in manufactured housing, those who used to have a different programmable thermostat, and those who floss religiously on Sunday, Tuesday, Thursday, and Saturday every single week. Also, the data cover only one heating season, so running it for multiple years would provide a better picture of what’s happening.

Still, one of the biggest takeaways here is that this study found good savings with the Nest thermostat connected to heat pumps. In their white paper, Nest discussed the difficulties of using setbacks with heat pumps and included a U.S. Department of Energy warning: “Programmable thermostats are generally not recommended for heat pumps.” That’s because many of them tend to cause the electric resistance heat to run more as the heat pump attempts to recover from setbacks.

Two studies in Indiana

These two projects were done by two different gas and electric utilities in Indiana. The Cadmus Group conducted both studies and wrote the reports. Both studies looked at gas use during heating season and electricity use during cooling season. One of the studies had 400 participating homes, the other 300.

The Oregon project was a pre- and post-installation study with the same group of homes. The Indiana studies were different. They compared the energy use in homes with a Nest thermostat to homes with a standard programmable thermostat (the Honeywell TH211) by looking at each relative to homes with non-programmable thermostats as the baseline.

Here are the main results from both studies:

• For heating, the Nest showed ~13% energy savings compared to the baseline homes.
• For heating, the programmable thermostat showed 8% and 5% savings compared to the baseline.
• For cooling, the Nest showed 16% and 14% energy savings compared to the baseline homes.
• For cooling, the programmable thermostat showed 15% and 13% savings compared to the baseline.

The Nest beat the baseline homes (non-programmable thermostat) by double digits for both heating and cooling. It also beat the standard programmable thermostat for heating. For cooling, the Nest and the programmable thermostat came out about the same.

A white paper from Nest

This white paper is a good read even without the energy savings information because the paper discusses a lot of the issues around trying to determine whether or not a given measure saves energy and how you would go about determining the amount of savings. (Read the Background and Methodology sections.)

The Nest energy savings analysis is different from both the Oregon and the Indiana studies. The researchers used “pooled” data from a control group of non-Nest users and compared that data to data from the Nest users. The first chart below shows percent savings for the control group. Notice that the center of the data is at 0% savings. In contrast to the control group, the data from the Nest users are shifted to the right of the 0% savings line, as you can see in the second chart.

The actual numbers they found for savings of natural gas for heating and electricity for cooling are:

• 9.6% for heating (as a percentage of heating, not total energy use)
• 17.5% for cooling (as a percentage of cooling, not total energy use)

The heating number is lower than the other studies, but the cooling number is about the same.

The Nest white paper discusses potential biases and the issues surrounding using data from the MyEnergy customers. In the end, though, they conclude that there wasn’t any large bias in the results.

Yes, the Nest does save energy

The results here provide a lot of evidence that the Nest Learning Thermostat does indeed save energy. It saves on heating energy with both gas furnaces and heat pumps, and it saves on cooling with air conditioners and heat pumps. All four of the reports go into the details of their methodology, telling how they chose the participants, how they collected and analyzed the data, and more. (Use the links below to download the papers.)

One important thing to point out here is that the Oregon and Indiana studies used carefully chosen participants, and they installed the Nest thermostats for them. One question that comes up with Nest data sometimes is, what is the effect of having self-selected gadget-geeks or energy savers on the savings data? In those three studies, that’s a moot point. It seems that the average Nest owner will save energy even when they’re not the kind of people who would go out and buy one on their own.

Of course, not everyone buys a Nest because it saves energy. Some people get them just so they can adjust the thermostat remotely from the app on their phone. Maybe it’s even because it’s flossing day, and they have to get up early.

The Four Reports

Oregon Study (pdf)

Indiana Study – NIPSCo (pdf)

Indiana Study – Vectren

Nest white paper (pdf)

Allison Bailes of Decatur, Georgia, is a speaker, writer, energy consultant, RESNET-certified trainer, and the author of the Energy Vanguard Blog. Check out his in-depth course, Mastering Building Science at Heatspring Learning Institute, and follow him on Twitter at @EnergyVanguard.