Researchers at the University of Washington have developed "semantic hearing" technology that blends noise cancellation with customizable sound filtering.
The experimental technology uses deep-learning algorithms on a connected smartphone to filter ambient sounds picked up by the headphones' mics.
- The system can block most sounds but allows the wearer to hear select sounds like birds, alarms, or sirens.
- It can also do the opposite: block select sounds while letting all others through.
- There are 20 allowable/blockable sounds so far.
Listen to a demonstration
Semantic hearing, the future of intelligent hearables.
Why it matters
This "semantic hearing" system marks a significant upgrade to noise-blocking technology.
- Using the system would allow music lovers or distraction-prone workers to maintain situational awareness by picking out meaningful sounds.
How it works
- The prototype headphones connect to a smartphone app applying deep neural networks to process ambient sounds in real-time.
- Unlike current noise canceling techniques, these algorithms differentiate specific noises to filter.
- The system processes sounds in real-time at less than a 1/100th of a second to sync what the user is hearing with what they are seeing. This requires more computing power than current noise-canceling headphone electronics can provide, so a connected smartphone is necessary.
- As sounds from different directions reach people's ears at different times, the system preserves these time delays so users can still perceive sounds in their environment accurately.
Semantic-hearing-enabled noise-canceling headphones put to the test at the University of Washington.
“The challenge is that the sounds headphone wearers hear need to sync with their visual senses. You can’t be hearing someone’s voice two seconds after they talk to you. This means the neural algorithms must process sounds in under a hundredth of a second.” —lead author Shyam Gollakota, Paul G. Allen School of Computer Science & Engineering
The takeaway
Semantic hearing could enable personalized soundscapes where you determine what sounds reach your ears.
- Imagine filtering out crying babies on flights but still hearing the flight attendants' voices.
