The goal of the GT Accessible Aquarium Project is to make dynamic exhibits such as those at museums, science centers, zoos and aquaria more engaging and accessible for visitors with vision impairments by providing real-time interpretations of the exhibits using innovative tracking, music, narrations, and adaptive sonification. It is a truly interdisciplinary collaboration between GT researchers Bruce Walker, Tucker Balch, Gil Weinberg, Carrie Bruce, Jon Sanford, and Aaron Bobick, bringing together the fields of Psychology, Computing, Music, and Assistive Technology. See the Georgia Tech Press Release, and Project Overview Videos.
Zoos and aquaria are in the business of educating and entertaining the visiting public. However, as the number of people with disabilities living in the community has grown, and as public environments have become more accessible to them, such informal learning environments (ILEs) are faced with accommodating an increasingly diverse visitor population with varying physical and sensory needs. This is even more challenging for ILEs with dynamic exhibits, where the movements, changes, and interactions are extremely difficult to describe to individuals who lack vision. The behavior of fish and other sea life in an aquarium, the play of monkeys or lion cubs in a zoo habitat, and the cosmic dance of the planets in a science center's model of the solar system are all examples of dynamic exhibits that have until now been completely inaccessible to visitors with vision impairments.
In this project, we are developing cutting edge bio-tracking and behavior analysis techniques that can provide input for sophisticated, informative, and compelling multimedia auditory displays, music, and sonifications (sounds used to convey information about some kind of data to a listener). We are focusing first on the aquarium domain, since almost every exhibit in an aquarium is dynamic. Further, the aquatic nature of these exhibits provides unique and interesting challenges to the use of typical sensing technologies and techniques for tracking. The principles and techniques we develop will be immediately applicable to zoos, museums, and other ILEs with dynamic exhibits, leading to a dramatic increase in the opportunities for people with vision impairments to experience these types of exhibits. The developments in tracking and multimedia display will also have broader applicability in a range of fields, including tracking people for many purposes.
Dynamic exhibits, such as those in aquaria, zoos, science museums, etc., are exciting to a visitor largely because of the active, changing natire of the exhibit. However, a visually impaired or blind visitor generally has a very difficult time knowing what is in the exhibit, and, more importantly, what is happening. Are the big fish chasing the littel fish? Are the monkeys playing or sleeping? Are all the critters in one corner, or are they running around the entire exhibit?
The aim of this project is to track the movements of the objects (fish, monekeys, planets) in a dynamic exhibit, and then convey to the visitor some sense of what is happening, via sound (and possibly other senses such as touch).
The Accessible Aquarium Project is a truly interdisciplinary collaboration:
PI: Bruce N. Walker, Associate Professor in the School of Psychology and School of Interactive Computing
Tucker Balch, Associate Professor in the School of Interactive Computing
Aaron F. Bobick, Professor and Chair of the School of Interactive Computing
Carrie M. Bruce, Research Scientist in the Center for Assistive Technology and Environmental Access (CATEA) and PhD student in the School of Interactive Computing
Gil Weinberg, Associate Professor in the School of Music
Psychology: Myounghoon "Philart" Jeon
Computing: Carrie M. Bruce, Jinhan Lee
Music: Ryan Nikolaidis, Sriram Viswanathan, Mark Godfrey, Jonathan Kim, Jason Orlosky
HCI: Mary Frances Jones, Stephen Garrett, Anandi Pendse, Michael Pate
To give a general idea of what we are trying to do, and a sort of chronological story of the project, the following videos show different fish tanks, and have different audio associated with the fish movement. Note that even though the project has already come a long way, these are still just the beginning, and we have lots of different things to try, before we actually implement this in a real setting.
Publications Relating to the Research
(See the Publications page for all Sonification Lab publications.)
Bruce, C, M., & Walker, B. N. (2009). Developing Effective Real-time Audio Interpretation to Enhance Accessibility of Dynamic Zoo and Aquaria Exhibits. Proceedings of the Association for the Advancement of Assistive Technology in Europe Conference (AAATE 2009), Florence, Italy (31 August - 02 September). pp. TBD. <PDF>
Pendse, A., Pate, M., & Walker, B. N. (2008). The Accessible Aquarium: Identifying and evaluating salient creature features for sonification. Proceedings of the Tenth International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS2008), Halifax, Canada (13-15 October, 2008). pp. 297-298. DOI: 10.1145/1414471.1414546 <PDF>
Walker, B. N., Kim, J., & Pendse, A. (2007). Musical soundscapes for an accessible aquarium: Bringing dynamic exhibits to the visually impaired. Proceedings of the International Computer Music Conference (ICMC 2007), Copenhagen, Denmark (27-30 August). <PDF>
Walker, B. N., Godfrey M. T., Orlosky, J. E., Bruce, C., & Sanford, J. (2006). Aquarium sonification: Soundscapes for accessible dynamic informal learning environments. Proceedings of the International Conference on Auditory Display (ICAD 2006), London, England (20-24 June). pp. 238-241. <PDF>
Georgia Tech Press Release, and Project Overview Videos (Nov, 2008)
Associated Press Video, via Atlanta Journal Constitution, ajc.com (Dec 17, 2008)
Associated Press Story, via Atlanta Journal Constitution, ajc.com (Dec 17, 2008)
Associated Press Story, via FoxNews.com [PDF] (Dec 19, 2008)
Bruce Walker discusses the project on The Takeaway, a BBC/NY Times radio show [Download mp3] (Dec 24, 2008)
Carrie Bruce discusses CATEA and the Accessible Aquarium Project on Eye on Vision, from WYPL-FM 89.3 in Memphis, TN [Download mp3] (Jan 17, 2009)
Bruce Walker discusses the Accessible Aquarium Project on Eye on Vision, from WYPL-FM 89.3 in Memphis, TN [PART 1: Download mp3] [PART 2: Download mp3] (Feb 7, 2009)
Atlanta Journal Constitution Story, ajc.com [PDF] (Feb 22, 2009)
This research has been supported, in part, by grants from the US Department of Education through the National Institute on Disability and Rehabilitation Research (NIDRR) via the Wireless RERC, and the National Science Foundation (NSF). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the funding agencies or sponsors.