Computer engineering scientist Jarek Nabrzyski and his team are developing Voice4PIMD, a personalized artificial intelligence system designed to interpret nonverbal communication for individuals with profound intellectual and multiple disabilities. By utilizing cameras and sensors, the technology monitors eye gaze, facial expressions, heart rate, and body movements. Unlike generic language models, this system requires caregivers to train the software on an individual’s unique behavioral baseline, ensuring that a specific gesture or sound is translated accurately according to the user's personal history.
In section Releases
Decoding silence: Notre Dame AI gives voice to the nonverbal
For Ed Fennell, every subtle shift in his granddaughter Haley’s expression carries a world of meaning. After two decades of decoding her nonverbal signals, he faces a daunting reality: who will understand her once he is gone? Researchers at the University of Notre Dame are now building an AI solution to bridge that gap.
The project aims to provide long-term continuity of care, a challenge often faced by families as primary caregivers change over time. Currently, the team is conducting pilot testing with families through LOGAN Community Resources in South Bend, Indiana, while simultaneously evaluating the system's potential efficacy for individuals living with autism or dementia. By focusing on this historically underserved population, Nabrzyski hopes to demonstrate that human-centered AI can provide dignity and agency to those who have long been excluded from traditional communication research.
Comments (0)
No comments yet. Be the first!