A new study shows how ordinary phone unlocks could eventually support passive heart-rate tracking, using facial video and deep learning to make cardiovascular monitoring more accessible while raising important questions about accuracy, privacy, and real-world clinical use.
Study: Passive heart-rate monitoring during smartphone use in everyday life. Image Credit: Have a nice day Photo / Shutterstock
A smartphone could soon automatically track heart health while people use it normally, without requiring a smartwatch, fitness tracker, or a deliberate heart-rate check. In a large study published in the journal Nature, researchers created a Passive Heart-Rate Monitoring (PHRM) system that uses the phone’s front camera to detect blood flow changes in a person’s face.
The system then analyzes these changes using deep-learning algorithms to estimate an individual’s heart rate (HR) and resting heart rate (RHR). Validated on more than 160,000 videos, the system performed well in laboratory settings and everyday situations across skin tones, addressing limitations of previous systems. Since smartphones are already widely used, this technology could make heart health tracking more accessible for many people, especially those who cannot afford smart wearable devices.
RHR refers to the number of times the heart beats per minute while the body is at rest. This is an important measure of cardiovascular health. Changes in RHR may indicate an increased long-term risk of heart disease. Today, measuring RHR over time typically requires a fitness band or smartwatch. However, not everyone regularly uses these devices. Smartphones, on the other hand, are widely used worldwide. If phones could measure RHR, heart tracking could become easier, more convenient, and affordable for people across different socioeconomic groups.
Scientists have previously used remote photoplethysmography (rPPG) technology to measure RHR using smartphone cameras. These studies, however, included only a small number of participants and tested the technology in controlled conditions. The system was also less accurate for people with darker skin tones. As a result, it remains unclear whether this technology can reliably measure RHR during routine smartphone use across diverse races and ethnic populations worldwide.
About the Study
In the present study, researchers developed and validated the PHRM technology using data from several laboratory investigations and real-world studies conducted between 2020 and 2024. The system records short, eight-second videos of a person’s face when they unlock their phone. AI algorithms then analyze these videos to estimate the person’s HR and RHR.
The team first trained the system using 192,353 video recordings obtained from 485 people to recognize HR patterns. They then tested the system using another set of 162,546 recordings collected from 211 different individuals of different ages, sexes, body sizes, and skin tones. This diverse participant pool helped researchers assess whether the technology could perform across a wide range of users. The researchers intentionally included many people with darker skin tones to address previous limitations in measurement accuracy.
Researchers assessed skin tone using spectrocolorimeter measurements and Fitzpatrick skin-type classifications in laboratory studies, while participants in the free-living study self-reported skin tone using the Monk Skin Tone (MST) scale.
The AI algorithms verified the quality of the video inputs. They then used confidence-based gating to reject low-quality measurements, such as recordings affected by poor signal quality, inadequate lighting, or excessive movement. The system subsequently combined multiple measurements obtained throughout the day to calculate a person’s RHR.
Lastly, the team compared the smartphone’s measurements with ECG recordings for HR and wearable-derived RHR estimates to evaluate the accuracy of the new PHRM system. They also examined whether the phone-based RHR correlated with established indicators of cardiovascular health.
Results
The PHRM system accurately measured HR in both controlled laboratory tests and during regular day-to-day use. It also produced accurate estimates of RHR. When researchers compared the smartphone measurements with those obtained from reference ECG recordings, they observed similar results. For valid HR measurements, mean absolute percentage errors remained below the 10 percent industry threshold across all three skin-tone groups.
The PHRM system met industry accuracy standards for consumer heart rate monitors. Compared with wearable HR trackers, PHRM achieved a daily mean absolute error (MAE) below the prespecified target of five beats per minute (bpm) overall. The system also outperformed 15 models using rPPG technology.
However, the system did not generate a usable measurement from every video, and the valid video-level measurement rate was lower in the darkest skin-tone group in free-living conditions.
For RHR, participant-level accuracy in the darkest skin-tone group did not initially meet the 5 bpm target, but performance improved from the third day onward as the system’s filtering algorithm converged.
A key finding was that higher smartphone-derived RHR was associated with known cardiovascular risk markers, including higher body mass index and lower cardiorespiratory fitness. The smartphone measurements therefore supported the physiological validity of the approach, rather than showing that the system can diagnose heart disease.
The smartphone-based measurements were also more consistent from day to day than conventional one-time RHR checks. If further validated, this approach could enable smartphones to monitor changes in cardiovascular health during everyday use.
Conclusions
The findings suggest that simply using smartphones throughout the day could eventually help individuals track their heart health. Phones could obtain multiple readings as people unlocked their phones at different times and assess changes in their heart rate patterns without wearing specific trackers or fitness devices. Further efforts could optimize performance by reducing battery usage and testing in people with different heart conditions.
The researchers also emphasized that any real-world use would require explicit informed consent, strong privacy protections, and secure on-device processing because the system involves passive facial video capture.
By publicly releasing the AI model and study dataset, researchers hope to advance privacy-conscious heart rate-monitoring technologies that can broaden access to cardiovascular monitoring through the smartphones people already use every day.
The study was conducted by Google Research and University of Washington researchers, funded by Alphabet or an Alphabet subsidiary, and the authors reported Alphabet employment, possible stock ownership, and a related patent application.
