Seeing the World through Sound

Seeing the World through Sound

Sensory Substitution Devices (SSDs) convert visual information into another sensory channel (e.g. sound) to improve the everyday functioning of blind and visually impaired persons (BVIP). However, the range of possible functions and options for translating vision into sound is largely open-ended. To provide constraints on the design of this technology, we interviewed ten BVIPs who were briefly trained in the use of three novel devices that, collectively, showcase a large range of design permutations. The SSDs include the ‘Depth-vOICe,’ ‘Synaestheatre’ and ‘Creole’ that offer high spatial, temporal, and colour resolutions respectively via a variety of sound outputs (electronic tones, instruments, vocals). The participants identified a range of practical concerns in relation to the devices (e.g. curb detection, recognition, mental effort) but also highlighted experiential aspects. This included both curiosity about the visual world (e.g. understanding shades of colour, the shape of cars, seeing the night sky) and the desire for the substituting sound to be responsive to movement of the device and aesthetically engaging. This project was supported by the RM Phillips Endownment Fund.

 

[VIDEO]

 

 

Date

24 April 2017

Tags

Project

About the SCHI Lab

The SCHI Lab research lies in the area of Human-Computer Interaction (HCI), an area in which research on multisensory experiences makes a difference on how we design and interact with technology in the future. The interdisciplinary team explores tactile, gustatory, and olfactory experiences as novel interaction modalities.

Contact

Sussex Computer Human Interaction Lab

Creative Technology Research Group

School of Engineering and Informatics

University of Sussex Chichester, 1

BN1 9QJ Brighton, UK

Phone: +44 (0)1273 877837

Mail: m.obrist [at] sussex.ac.uk

© 2017 SCHI Lab. All Rights Reserved. Designed By JoomShaper