AI models analyzing audio from AirPods could determine a user's heart rate
Apple's latest research reveals how models made for speech recognition can determine your heart rate by processing phonocardiograms, which could be used in AirPods.
Apple could add heart rate detection to future AirPods, building upon their existing health-related features.
On Thursday, a week after the company revealed its Humanoid Policy study that deals with robotics, Apple published new research that explores health applications. Specifically, heart rate estimation.
In its latest study, titled "Foundation Model Hidden Representations for Heart Rate Estimation from Auscultation," the company's researchers explain how foundation models can be used to determine heart rates from audio recordings, even though they were created with a different purpose in mind.
Apple tested six popular foundation models, including Whisper, which is used for audio transcription, in an effort to determine whether they could be used beyond their intended purpose. An in-house CLAP (Contrastive Language-Audio Pretraining) model was also tested.
Rather than recorded speech, the foundation models were made to process phonocardiogram recordings, or heart sounds, totaling around 20 hours.
The recordings in question belong to the publicly available CirCor DigiScope Phonocardiogram dataset, and varied in length from 5.1 to 64.5 seconds.
Apple performed tests with around 20 hours of phonocardiogram audio files. Image Credit: Apple
A human annotator annotated cardiac murmurs in the phonocardiogram audio files, and the files were split into five-second snippets that shift every second. The process yielded a total of 23,381 heart sound snippets, intended for conversion into heart rate measurements.
The company's study explains that its in-house CLAP model did better than existing models in this task. "We observed that the representations from the audio encoder of the in-house CLAP model achieved the lowest mean absolute error (MAE) across various data splits, outperforming the baseline model trained with standard acoustic features."
Unlike Whisper, wav2vec2, and wavLM, which the company also tested in the same way, its in-house CLAP model was trained with audio data that encompasses more than just recorded speech. The varied training data "likely enhances its ability to capture non-speech features relevant to heart sounds, contributing to its improved effectiveness," says the study.
Apple's in-house CLAP model outperformed other foundation models. Image Credit: Apple
Apple's researchers say that foundation models created for speech processing "can be effectively adapted for auscultation and vital sign estimation, offering a robust and efficient alternative to some conventional methods." The company's results also suggest that larger foundation models didn't always perform better at determining a heart rate.
According to the study, additional fine-tuning of foundation models may lead to better heart rate estimates. The company's researchers see potential applications in the "pathological analysis of cardiorespiratory sounds, which may facilitate more accurate detection of abnormalities such as arrhythmias and murmurs."
As for use in devices, Apple may incorporate this technology into future AirPods models, given the company's interest in health-related features. The AirPods Pro 2, for instance, supports a set of Hearing Health features and offers Active Noise Cancellation (ANC) through built-in microphones.
A rumor from December 2024 suggested that Apple was looking to implement heart and temperature sensors, among other things. It's possible that the company will use the AirPods' existing microphones alongside new sensors for more accurate heart rate determination.
Read on AppleInsider
Comments