App Helps the Blind "See" With Their Ears
New technology allows people to perceive color and shape through music.
It's not a magic act but a demonstration of a new app that enables the visually impaired to hear information usually perceived through sight. The woman is wearing headphones and a miniature camera attached to a pair of glasses, which are connected to a laptop on the table. A series of musical cues—which combine into a pleasant tune—let her know the color, shape, and location of the fruit.
The app, called EyeMusic SSD (for sensory substitution device), uses a computer algorithm to construct a "soundscape" that conveys visual information through musical notes. With training to extract meaning from patterns of notes, people who have been blind since birth can learn to read letters and numbers, tell what they are looking