App Helps the Blind "See" With Their Ears

New technology allows people to perceive color and shape through music.

It's not a magic act but a demonstration of a new app that enables the visually impaired to hear information usually perceived through sight. The woman is wearing headphones and a miniature camera attached to a pair of glasses, which are connected to a laptop on the table. A series of musical cues—which combine into a pleasant tune—let her know the color, shape, and location of the fruit.

The app, called EyeMusic SSD (for sensory substitution device), uses a computer algorithm to construct a "soundscape" that conveys visual information through musical notes. With training to extract meaning from patterns of notes, people who have been blind since birth can learn to read letters and numbers, tell what they are looking

DON'T MISS THE REST OF THIS STORY!
Create a free account to continue and get unlimited access to hundreds of Nat Geo articles, plus newsletters.

Create your free account to continue reading

No credit card required. Unlimited access to free content.
Or get a Premium Subscription to access the best of Nat Geo - just $19
SUBSCRIBE

Read This Next

'World’s worst shipwreck' was bloodier than we thought
World’s first ultrasounds of wild manta rays reveal a troubling truth
Titanic was found during secret Cold War Navy mission

Go Further

Subscriber Exclusive Content

Why are people so dang obsessed with Mars?

How viruses shape our world

The era of greyhound racing in the U.S. is coming to an end

See how people have imagined life on Mars through history

See how NASA’s new Mars rover will explore the red planet

Why are people so dang obsessed with Mars?

How viruses shape our world

The era of greyhound racing in the U.S. is coming to an end

See how people have imagined life on Mars through history

See how NASA’s new Mars rover will explore the red planet

Why are people so dang obsessed with Mars?

How viruses shape our world

The era of greyhound racing in the U.S. is coming to an end

See how people have imagined life on Mars through history

See how NASA’s new Mars rover will explore the red planet