AI-powered ‘hearing glasses’ could help filter out background noise in real time

Artificial intelligence is being used to create “hearing glasses” which use lip reading technology to help people with hearing loss.

The project, led by Heriot-Watt University and involving researchers from the University of Edinburgh, Napier University and the University of Stirling, aims to help people with hearing loss by filtering out background noise in real-time, even in loud environments.

The technology combines lip-reading technology, artificial intelligence and cloud computing, and uses a small camera built into glasses to track the speaker’s lip movements while a smartphone app uses 5G to send both audio and visual data to a powerful cloud server.

AI then isolates the speaker’s voice from surrounding noise and sends the cleaned-up sound back to the listener’s hearing aid or headphones almost instantly.

More than 1.2 million adults in the UK have hearing loss severe enough to make ordinary conversation difficult, according to the Royal National Institute for Deaf People, and the impact of hearing aids can be limited in noisy places.

Researchers hope to have a working version of the glasses by 2026.

They are speaking to hearing aid manufacturers about future partnerships and hope to reduce costs to make the devices more widely available.

Scientists have collected noise samples, from washing machines to traffic, to improve the system’s training.

Project leader Professor Mathini Sellathurai, of Heriot-Watt University, said: “We’re not trying to reinvent hearing aids. We’re trying to give them superpowers.

“You simply point the camera or look at the person you want to hear.

“Even if two people are talking at once, the AI uses visual cues to extract the voice of the person you’re looking at.”

This approach, known as audio-visual speech enhancement, takes advantage of the close link between lip movements and speech.

Some noise-cancelling technologies already exist, but struggle with overlapping voices or complex background sounds — something this system aims to overcome.

By shifting the heavy processing work to cloud servers — some as far away as Stockholm — the researchers can apply powerful deep-learning algorithms without overloading the small, wearable device.

The technology is still in the prototype stage but researchers have tested the technology with people who use hearing aids and said early results are promising.

Prof Sellathurai said: “There’s a slight delay, since the sound travels to Sweden and back, but with 5G, it’s fast enough to feel instant.

“One of the most exciting parts is how general the technology could be.

“It’s aimed to support people who use hearing aids and who have severe visual impairments, but it could help anyone working in noisy places, from oil rigs to hospital wards.

“There are only a few big companies that make hearing aids and they have limited support in noisy environments.

“We want to break that barrier and help more people, especially children and older adults, access affordable, AI-driven hearing support.”

The project is funded by the Engineering and Physical Sciences Research Council (EPSRC).