In a recent study published in JAMA Cardiology, researchers from the Cleveland Clinic tested the accuracy of heart rate monitoring in several popular wearables including the Apple Watch, Fitbit Charge, Mio Alpha, and Basis Peak.

Given the lack of oversight for many connected health devices and wearable health devices, there’s been growing interest from many academic medical centers to independently test these devices. For example, we previously went through a study testing the accuracy of Fitbit’s step tracking.

Heart rate monitoring is built into many wearables and fitness trackers. Because their intended use is general wellness, as opposed to management of any specific health condition, they don’t generally fall under regulatory oversight. However, we’ve certainly seen many reports of use for management of health conditions, like a recent anecdotal story about using the Fitbit’s heart rate tracker to help manage atrial fibrillation. And there some health conditions, like atrial fibrillation, where simple heart rate tracking could be useful – if its accurate.

In this study, researchers asked health subjects to wear these devices and get on a treadmill. They collected heart rate based on an EKG and based on these devices at different stages of exercise. They then looked at the correlation of the heart rate on the wearables with the EKG-measured heart rate as well as patterns of deviation.

The Apple Watch and Mio Fuse had the most accurate heart rate tracking, demonstrating the best concordance with EKG-measured heart rate (0.91 for both). Interesting, there was no overlap in the 95% confidence intervals for these devices vs. the Fitbit and Basis devices, which concordance coefficients of 0.84 and 0.83 respectively.

Contrary to what we might expect, they also found that the most variability and largest errors in heart rate seemed to be clustered in the mid-range of heart rates, roughly from 100 – 160 bpm. That said, its worth looking at the actual plots in the manuscript because the error pattern seems a bit different for each device though that isn’t really fleshed out in the discussion. Again, Fitbit and Basis fell short compared to Apple Watch and Mio Fuse; the former had a wider spread on heart rates, particularly in that mid-range, than the latter.

It is a small study (50 patients) and only included healthy patients, so I do worry that the errors may be worse in people with stiffer arteries, some micro or macrovascular disease, and so on. It would have also been nice to see some repeated measures in the same patients to get a sense of whether this is a problem of accuracy or precision across the range of heart rates.

Overall, it does serve as a useful reminder of the limitations of consumer health wearables that are worth considering if you’re going to incorporate them into recommendations for patients.

Reference: Wang et al. JAMA Cardiology. 2016. Link