How we test smart thermostats
Find out how we test smart thermostats and what it takes for one to be made a Best Buy
Our tests go further than anyone else to find the best and worst smart thermostats.
Read on to find out more about what we look for in our tests. Or head straight to our smart thermostat reviews to see how each model fared.
Video: what makes a Best Buy smart thermostat
What makes our smart thermostat tests different
Other smart thermostat reviews you'll see online are very often just by one tester who has spent some time using it, and often they might receive products free of charge from manufacturers to review.
Which? smart thermostat tests go above and beyond this. We don't accept free samples; we buy every single thermostat we review. This is so you know our opinion is honest and unbiased.
Our lab test scores are based on measurements and ratings by our expert lab testers, ensuring there is no unconscious bias. When we say a thermostat is a Best Buy, we have the data to back it up.
Finally, we also carry out extensive privacy and security tests on our smart thermostats. If there are any vulnerabilities, we'll find them and let you know – something other reviews won't.
Only smart thermostats that perform well in all our tests will get a Best Buy recommendation. A Don't Buy is a product to avoid as it may not keep your data secure, not work as expected or be a pain to use.
- All smart thermostats with a score of 70% or more are Best Buys
- All those with a score of 45% or less are Don't Buys
How to buy the best smart thermostat - our guide to the features to look out for and one is right for you
How the Which? score is calculated
All our tests are used to create an overall score for each smart thermostat.
We survey consumers and Which? members regularly to understand what's important to you when buying a smart thermostat.
Based on this, we refine our evaluation so the overall score is representative of what consumers value.
Our total test score ignores price and is based on:
- 50% ease of use
- 40% thermostat function
- 10% features.
Our key smart thermostat testing criteria
Is it easy to use?
- Ease of installation Do you need a professional to install your smart thermostat or whether you can do it yourself. We assess how straightforward the process is and how useful the instructions are
- Ease of connecting to wi-fi Devices that require a really long or complicated setup will get a lower score
- Ease of adjusting settings and programming schedules How easy it is to find and adjust your thermostats temperature (both on the device and in the app), as well as how easy is it to set up a heating schedule and create multiple heating zones.
How quick and accurate does the thermostat function?
The thermostat should be precise and fast responding, so your room stays at the temperature you expect.
We put smart thermostats in a temperature-controlled chamber, measuring how accurately they record the temperature, and how long it takes for them to trigger the relay after the temperature drops below the selected threshold.
What features does it offer?
Some smart thermostats come with a range of features that make life easier, such as:
- Child lock to stop your children from changing the settings
- Frost protection to stop your pipes freezing
- GPS tracking to know when you've left the house to lower/turn off your heating, or preheat your house ready for when you return
- Weather responsive This raises or lowers your heating to counteract an unusually cold day or sudden heatwave.
Our reviews run through all the features of each smart thermostat so you can make sure you buy one that meets your needs.
Visit our guide to Nest vs other smart thermostats to see how top brands features stack up against each other.
Are there any security issues with the thermostat?
Privacy is vitally important with a smart thermostat. Data in the wrong hands could show when you’re likely to be out, leaving your home vulnerable to a robbery.
Our thorough tests make sure that each smart thermostat meets the standards we expect for preventing potential data breaches.
Firmware
Firmware is the software programmed into the smart thermostat. It needs to be updated or maintained regularly or else it could more vulnerable to hackers. We check that the thermostat and app can receive updates, and whether the app automatically checks for and performs the latest available updates.
Password policy
We check whether the thermostat and app force you to use passwords that aren't easily cracked. Our guide explains more about setting a secure password.
Permissions
We check what personal data is needed when you set up an account with the app and what permissions are requested (such as location, access to camera and calendar). We check to see whether it's really necessary to collect each piece of data and investigate the reasoning behind each app permission.
Encryption
During daily use, data is being sent back and forth from your app and smart thermostat. If the information is converted into secret code, it means your data is encrypted and safe-guarded against any snoopers. We check that your personal data is encrypted and where this data is being sent to. If we spot any personal data being sent unencrypted, we will alert both you and the manufacturer of the product, and investigate further.
Man-in-the-middle attacks
Hackers may try to intercept data passing to or from your app or thermostat, which is of particular concerning if any is unencrypted personal data. We run tests on every thermostat and valve to make sure they aren't susceptible to this kind of attack. If they are, we investigate further.
Decommissioning
You may decide to change to a different brand of thermostat, or move house and leave the device behind. You therefore, need to be able to delete your account and reset the device. We check whether you can do all of this easily – as you should be able to.
Our testing checks how easy it is to reset the thermostat to its factory settings and how easy it is to reset your account. We also check whether a future user of your smart thermostat can see or access any of your data.
New forms of cyberthreats
The methods hackers use are constantly changing, so we'll update our tests to counter new threats. There’s no such thing as 'unbeatable' security when it comes to connected devices, but we check for the most common attack vectors and alert manufacturers to any issues we find. The smart thermostats that we recommend had no issues that concerned us at time of testing.
View smart thermostat reviews
