Background: Sensory substitution

Sensory substitution and augmentation

Sensory augmentation systems such as eyeglasses and hearing aids enhance the existing capability of a functional human sensory system. Sensory substitution is the use of one human sense to receive information normally received by another sense. Braille and speech synthesizers are examples of systems that substitute touch and hearing, respectively, for information that is normally visual (printed or displayed text). For a review of sensory augmentation and substitution, see:

• K. A. Kaczmarek, "Sensory augmentation and substitution," in CRC Handbook of Biomedical Engineering, J. D. Bronzino, Ed. Boca Raton, FL: CRC Press, 1995, pp. 2100-2109.
• Peter Meijer's website - the Voice

Vision substitution

Reported attempts to present spatial visual information via electrotactile stimulation date from at least the early part of last century (Collins, 1985, Collins et al., 1970, Machts, 1920). All of these systems operate by using some kind of electronic camera or matrix of light sensors to control the stimulation intensity on a spatially-corresponding matrix of electrodes on the surface of the skin. The user perceives tactile shapes on the skin having the same shape as the visual image recorded by the camera. Blind and blindfolded users are able to identify simple objects in a high-contrast environment, and have reported visual concepts such as distal attribution (i.e., perceptually localizing the target object out in front of the camera, rather than on the tactile display proper), looming, and perspective (Bach-y-Rita, 1971), and also optic flow phenomena (Unitech Research Inc., 1993). The latter device is commercially available by ForeThought Development, LLC as a laboratory instrument.

Auditory substitution

Auditory prostheses using electrotactile stimulation typically use the stimulation intensity on each electrode in a linear (Saunders et al., 1981, Weisenberger et al., 1989) or two-dimensional (Sparks et al., 1979) matrix to represent the sound intensity in a particular frequency range of sounds recorded by a microphone. The instrumentation therefore performs a frequency analysis, similarly to the ear’s cochlea. Results have generally included increased awareness of sounds and improved lipreading ability. At least two devices (made by Tacticon Corporation and Sevrain-Tech, Inc.) were commercially available in the 1970s–1980s.

Tactile feedback

Touch perception is an integral part of motor control in the hand; the insensate hand can manipulate objects only clumsily. Despite a lack of appropriate tactile sensors (which are now becoming available because of robotics research), attempts have been made to transmit pressure patterns on the insensate hand to sensible cutaneous regions. For example, a leprosy patient with insensate hands in one study (Collins et al., 1974) reported perception of surface textures sensed by a special instrumented glove and presented electrotactilly to the forehead. We have duplicated this result in our laboratory. Touch substitution for the insensate foot has also been reported (Wertsch et al., 1988), in an attempt to reduce incidence of pressure sores.

Virtual environments

Computer-generated simulations of the physical world, which are being explored to aid surgery (Taylor et al., 1995) as well as a wealth of industrial applications (Burdea et al., 1994), gain realism when they can be navigated visually and tactually by use of kinesthetic user input devices (e.g. data gloves and hand position trackers). Force feedback hand controllers enhance both realism and task performance in virtual environments; truly spatial tactile feedback has been slower to emerge, mostly because of the lack of compact tactile feedback devices (Burdea, 1996). Electrotactile stimulation is ideally positioned to fill this need.

Literature references

• Bach-y-Rita, P. (1971).
• Burdea, G. (1996). Force and Touch Feedback for Virtual Reality. Wiley, New York.
• Burdea, G. and Coiffet, P. (1994). Virtual Reality Technology. Wiley, New York.
• Collins, C. C. (1985). On mobility aids for the blind. Electronic Spatial Sensing for the Blind (Warren, D. H. and Strelow, E. R., Eds.), pp. 35-64. Matinus Nijhoff, Dordrecht, The Netherlands.
• Collins, C. C. and Madey, J. M. J. (1974). Tactile sensory replacement. Proc. San Diego Biomed. Symp. pp. 15-26.
• Collins, C. C. and Saunders, F. A. (1970). Pictorial display by direct electrical stimulation of the skin. J. Biomed. Sys. 1, 3-16.
• Machts, L. (1920). Device for converting light effects into effects perceptible by blind persons, German patent 326283.
• Saunders, F. A., Hill, W. A. and Franklin, B. (1981). A wearable tactile sensory aid for profoundly deaf children. J. Med. Sys. 5, 265-270.
• Sparks, D. W., Ardell, L. A., Bourgeois, M., Wiedmer, B. and Kuhl, P. K. (1979). Investigating the MESA (Multipoint Electrotactile Speech Aid): The transmission of connected discourse. J. Acoust. Soc. Am. 65, 810-815.
• Taylor, R. H., Lavalée, S., Burdea, G. C. and Mösges, R. (Eds.) (1995). Computer-Integrated Surgery. MIT Press, Cambridge.
• Unitech Research Inc. (1993). SBIR Phase I Final Report: A new electrotactile prosthesis for the blind. R43-EY09499, August, Unitech Research, Inc.
• Weisenberger, J. M., Broadstone, S. M. and Saunders, F. A. (1989). Evaluation of two multichannel tactile aids for the hearing impaired. J. Acoust. Soc. Am. 86, 1764-1175.
• Wertsch, J. J., Bach-y-Rita, P., Price, M. B., Harris, J. and Loftsgaarden, J. (1988). Development of a sensory substitution system for the insensate foot. J. Rehab. Res. Dev. 25(1), 269-270.