Smartwatch app alerts hearing-impaired users to common home-related sounds

Smartwatches offer people a private method for getting notifications about their surroundings -- such as a phone call, health alerts or an upcoming package delivery. Now National Science Foundation-funded researchers have developed SoundWatch, a smartwatch app for deaf and hard-of-hearing people who want to be aware of nearby sounds. When the smartwatch picks up a sound the user is interested in -- examples include a siren, a microwave beeping or a bird chirping -- SoundWatch will identify it and send the user a friendly buzz along with information about the sound.

The University of Washington team recently presented these findings at the ACM (Association for Computing Machinery) conference on computing and accessibility.

"This technology provides people with a way to experience sounds that require an action -- such as getting food from the microwave when it beeps," said lead author Dhruv Jain, a UW researcher in the Paul G. Allen School of Computer Science & Engineering.

Jain was born hard of hearing.

"I use the watch prototype to notice birds chirping and waterfall sounds when I am hiking," he said. "It makes me feel present in nature. My hope is that other deaf and hard-of-hearing people who are interested in sounds will also find SoundWatch helpful."

The team started this project by designing a system called HomeSound for deaf and hard-of-hearing people who wanted to know what was going on around their homes.

"I used to sleep through the fire alarm," Jain said.

The researchers tested HomeSound in the Seattle-area homes of six deaf or hard-of-hearing participants for three weeks. Participants were instructed to go about their lives as normal and complete weekly surveys.

Based on feedback, a prototype used machine learning to classify sounds in real time. The researchers created a dataset of more than 31 hours of 19 common home-related sounds, such as a dog bark or a cat meow, a baby crying or a knock on the door.

Read more: https://www.nsf.gov/discoveries/disc_summ.jsp?cntn_id=301616&org=NSF&fro...

See a video: https://www.youtube.com/watch?v=1UK_Gb1yeUs