Overview: Modern smartphones include an array of general purpose sensors such as accelerometers, gyroscopes, cameras and microphones. The computing power of modern smartphones enables running advanced AI models to process health-relevant sensor data and extract computational biomarkers on-device. High-speed Internet access provided by 4G and 5G communications enables smartphone apps to interact seamlessly with the most resource-intensive, cloud-based AI models such as large language models and other forms of large-scale analytic and generative AI. The ubiquity, sensing capabilities and mobile computing power of modern smartphones make them ideally suited to deploying applications that combine sensing and AI models to continuously monitor, assess and support multiple aspects of physical and cognitive health.
MassAITC Pilot Project Highlights: MassAITC has funded multiple projects that leverage mobile sensing and computing based technologies. These funded technologies are tackling diverse problems spanning accessible at- home hearing assessments, low-cost blood pressure monitoring, memory assistants, and early detection of cognitive impairment via vocal biomarkers and other passive measures obtained via smartphones. ForesightCares—developing a mobile app to revolutionize mobility assessments to reduce fall risk and enable better care for neurological disorders—received follow-on funding via a National Science Foundation Phase II SBIR award. Sonde Health has expanded their strategic partnership with Qualcomm Technologies, Inc. bringing their best in class vocal biomarker platform to new classes of devices—expanding beyond smartphones to smart glasses and other AR enabled devices.
More information on funded pilots in this area is listed below, along with additional resources including MassAITC webinars touching on this topic area:
Low-Cost mHealth Technology for Objectively Assessing Hearing Loss at Home
Wenyao Xu, Auspex Medix. Wei Sun, University at Buffalo. This project is to investigate an AI-powered, smartphone-based hearing screening tool that uses non-volitional pupillary responses to objectively assess hearing functions and loss.
AI-Driven Early Detection of AD Risk Using Speech Features
Marziye Eshghi, MGH Institute of Health Professions. This project will leverage AI-driven analysis of remotely collected speech data to detect early signs of Alzheimer’s disease (AD) by linking speech acoustic and kinematic features to AD molecular pathologies.
Leveraging Digital Cognitive Rhythms to Detect ADRD Risk in Family Caregivers
Raeanne C Moore, UCSD, Yeonsu Song, UCLA. This project will develop and pilot machine learning algorithms to passively monitor cognitive fluctuations among family caregivers of persons living with dementia by analyzing their smartphone typing patterns and speech.
AI-Based Video App for At-Home Monitoring of Motor Functions in PD Patients
Hamed Tabkhi, Mona Azarbayjani, ForesightCares Inc. Sanjay Iyer, Memory & Movement Charlotte. The pilot project focused on developing and validating an AI-based visual assessment (AVA) app for at-home monitoring of motor function in older adults with Parkinson’s Disease (PD).
A Downloadable Oscillometric BP Monitor for All Smartphones with No Attachments
Edward Jay Wang, Billion Labs Inc. This project aims to establish accessible early screening of hypertension by democratizing BP monitoring. We aim to achieve this by converting the billions of smartphones into oscillometric BP monitors without hardware add-ons.
An Objective Assessment Tool for Evaluating Functioning in Older Adults
Ehsan Adeli, Victor W. Henderson, Stanford University. The proposed project aims to design a mobile app that not only instructs and records individuals performing Short Physical Performance Battery (SPPB) tests but also uses these data for predictive analysis to monitor and quantify the risk of cognitive impairment over time, utilizing video data analyzed for motor-cognitive relationships.
Intelligent Cognitive Assistant for the Individuals with AD/ADRD for Handling Word-finding Difficulty
Archna Bhatia, Institute for Human & Machine Cognition, George Sperling, UCI. This project aims to develop an Intelligent Cognitive Assistant that provides real-time word retrieval support as well as personalized training to enhance word retrieval for individuals with mild cognitive impairment and Alzheimer’s Disease and Related Dementias.
Utilizing the Druid Impairment App to Assess and Enhance Senior Adult’s Driving Performance
Micheal Milburn and William DeJong, Impairment Science, Inc. Anuj Pradhan and Shannon Roberts, UMass Amherst. The Druid app uses multiple divided attention tasks to assess cognitive-motor behaviors related to driving. This project aims to validate the Druid app for use by older adults aged 64-85 years.
Smartphone blood pressure monitoring for healthy aging
Edward Jay Wang, UCSD. The pilot project focused on developing and validating BPClip, a low-cost, smartphone-based blood pressure monitoring device.
Testing a vocal biomarker platform for remote detection and monitoring of cognitive impairment in the home environment
Erik Larsen, Sonde Health. Brad Dickerson, Massachusetts General Hospital.
Bonnie Wong, Massachusetts General Hospital. The pilot project, conducted by researchers from Massachusetts General Hospital and Sonde Health, explored the use of a vocal biomarker platform to detect and monitor cognitive impairment in older adults.
MassAITC Webinars on MobiIe Sensing and Computing
Webinar – Advances in Cardiovascular Health Monitoring to Support Healthy Aging at Home, Anna Barnacka and Edward Wang
Overview: This webinar comprises two presentations from Anna Barnacka of MindMics and Edward Wang of Billion Labs. They each describe the work from their MassAITC a2 Pilot Award on addressing heart health monitoring. Anna Barnacka delves into a patented in-ear IH technology that enables heart health monitoring through TWS earbuds and discusses future plans to make this technology accessible to all. Edward Wang describes the VibroBP Smartphone app, the first app of its kind to use AHA-recommended oscillometry methods to ascertain BP and that does not require per-individual calibration. Abstracts: About the Speakers:
Webinar – Novel Technological Approaches for Detection of Cognitive and Functional Impairment: Drs. Larsen, Stamps, and Milburn
Abstract: This webinar explored cutting-edge technologies aimed at improving early detection and monitoring of cognitive and functional impairments in older adults. Dr. Kate Papp (Mass General Brigham) opened the session by highlighting the challenges of traditional clinical assessments—lengthy, labor-intensive, and inaccessible to many—and the promise of scalable, remote, and ecologically valid digital tools to address the growing needs of an aging population. Three MassAITC pilot awardees presented innovative approaches: A panel discussion with Dr. Rhoda Au (Boston University) addressed barriers to widespread adoption, including data privacy concerns, user acceptability, and integration into clinical workflows. Presenters emphasized the importance of validating these technologies in real-world environments to ensure accuracy, usability, and patient trust. About the Speakers: