Sensory substitution

 

Sensory substitution devices (SSDs) are non-invasive sensory aids that provide visual information to the blind via their existing senses. For example, a visual-to-auditory SSD translates images into sounds, retaining the visual shape information. In my PhD work with Amir Amedi, we developed a unique program for teaching blind people how to use the vOICe (an SSD developed by Peter Meijer) to learn to see information captured by a miniature webcam that they carried on their sunglasses.

 

 

 Recently covered in the New Yorker

The video from the sunglasses was carried to a laptop, which used the vOICe to translate the images into sounds, conveyed through earphones (A newer SSD, EyeMusic, was later further developed in Amir Amedi’s lab to also convey color and use only one earphone). Following ~70 hours of training, the fully and congenitally blind users of the SSDs could already walk in busy corridors while avoiding obstacles, identify and locate everyday objects, notice people in their surroundings and identify their facial expression (see video). Furthermore, an assessment of their visual acuity using an optometrists’ Snellen test suggested they can perceive fine details even beyond the World Health Organization blindness criterion.

The detailed description of how our training was conducted can be found below (originally published in the supplementary material of my 2012 Neuron study).

Using the SSD also provided me with the opportunity to study how the blind brain processes visual information – without visual experience but with a targeted rehabilitation in adulthood – see below.