School of Psychology - Georgia Institute of Technology
tick marks, or labels
will generally earn you an 'F' in highschool math class. After all, it is
just a squiggle on a page without the added context that those things
provide. Auditory graphs require the same elements, so we are studying how
best to create them, introduce them into an auditory graph, and examine
how people learn to use them for better (auditory) graph comprehension.
How can
we determine, in advance, what these differences will make (if any) in the
perception and comprehension of auditory graphs and sonification? What are
the characteristics of the listener that predict performance? We are studying
a range of factors, including perception, cognition, and listening experience.
One major factor we are considering is whether a listener is sighted or
blind, and if blind, at what age blindness occurred. 
so few people have experience with them. As a result, there is a great need
to understand how to train listeners to use and interpret auditory displays.
We are looking at training types, including simple exposure, part-task training,
whole-task training, and others, to determine what works best, and under
what circumstances.
to help educate and entertain you about the 2024 Total Solar Eclipse that is passing over North America on April 8, 2024.
There are two accessible maps (one "push", one "pull" style) that convey the path of totality, and the regions of partial totality. Since they are multimodal, they are designed to be accessible even to screen reader users.
There is an immersive VR experience in which you can view the eclipse from the location of Rochester NY, wander around the town as the light levels fade, animal behaviors change, and environmental sounds evolve...and learn about the eclipse in the VR environment. The VR experience is available as either an immersive APK for Quest headsets, or as an iPhone app that you can install from the App Store.
We have a YouTub video that allows you to preview the VR experience.
We also have the stand-alone soundtrack available for your enjoyment, either as a WAV file or in the form of a YouTube video.
and sonification of the 2017 Solar Eclipse, to help persons with vision loss (and anyone, really!) experience the eclipse. The soundtrack is composed of three sound layers: the pre-composed base track, which conveys the temporal dynamics; additional sounds that represent the "false dusk" and "false dawn"; and a layer of musical sounds that are generated live, in realtime, based on meteorological data.
zoos and aquaria are faced with
educating and entertaining an increasingly diverse visitor population with
varying physical and sensory needs. There are very few guidelines to help
these facilities develop non-visual exhibit information, especially for
dynamic exhibits. In an effort to make such informal learning environments
(ILEs) more accessible to visually impaired visitors, the Georgia Tech Accessible
Aquarium Project is studying auditory display and sonification methods for
use in exhibit interpretation.
application of sound in the human-computer
interface is the auditory menu. Generally, menu items are spoken via text-to-speech
(TTS), and the user simply navigates to the menu item of interest, and presses
a button to enter that menu or execute the menu command. We are studying
advances to this paradigm, including new interaction techniques, in desktop interfaces, mobile devices, in-vehicle interfaces, among others. Enhancement techniques include, for example, "spearcons", "spindex", and auditory scrollbars. Check out our
Auditory Menus Tech Report, or our many other publications and software.
that use auditory displays, covering the ears of the
user is not acceptable (e.g., pedestrians amongst traffic; firefighters
who need to communicate). We are studying the use of bone conduction technology
as a method of transmitting auditory information to a listener, without
covering the ears. Our research ranges from basic psychophysics (minimum
hearing thresholds, frequency response, etc.), all the way up to 3D audio
via bonephones (yes, if can be done!).
who have perceptual, cognitive, attention, decision-making, and emotion regulation challenges, often the result of traumatic brain injury (TBI). Carefully designed software solutions, running on hardware that is integrated into the vehicle's data streams, are proving to be very effective in helping these individuals be better, safer, more indepedent drivers. We are leveraging our experience in multimodal user interfaces of all types, and utilizing the many facilities available to us, including the GT School of Psychology's Driving Research Facility.
that can efficiently and effectively evaluate the salient aspects of an auditory user interface, or the audio aspects of a multimodal user interface. Can be used in conjunction with SUS, UMUX, UMUX-Lite, NASA TLX, etc.
and sonification of the 2017 Solar Eclipse, to help persons with vision loss (and anyone, really!) experience the eclipse. The soundtrack is composed of three sound layers: the pre-composed base track, which conveys the temporal dynamics; additional sounds that represent the "false dusk" and "false dawn"; and a layer of musical sounds that are generated live, in realtime, based on meteorological data.
wayfinding and navigation system to help those who cannot see (temporarily
or permanently) to navigate and learn about their environment. It uses a combination of spatialized speech and non-speech sounds to lead a user along a path to a destination, while presenting sounds indicating the location of environmental features such as benches, shops, buildings, rooms, emergency exits, etc. It is useful as an assistive technology for people with functional vision loss, and also as a tool for firefighters, first responders, or other tactical personnel.
ONLINE software package for creating data sonifications
and both visual and auditory graphs. Built in collaboration with Highcharts, makers of popular and powerful online charting tools...and as a replacement for the aging (and no longer supported) Sonification Sandbox.
Includes the ability to create or import data (inlcuding from Google Sheets), map data to sound
parameters in multiple flexible ways, add contextual sounds like click tracks
and notifications, and save the resulting sound, chart, or data files. Written in JavaScript, leveraging web audio and other online tools.
software package for creating data sonifications
and auditory graphs. Includes the ability to import data, map data to sound
parameters in multiple flexible ways, add contextual sounds like click tracks
and notifications, and save the resulting sound file. Written in cross-platform
Java/JavaSound. (No longer under development, and no longer supported. See the online replacement Sonification Studio.)
to display specific data values, like
the exact price of a stock or the precise temperature. Standalone application,
or plugin for the Sonification Sandbox. Written in cross-platform Java/JavaSound.
such as birdcalls, insect songs, rain,
and thunder to create an immersive soundscape to sonify continuous
data such as the stock market index. This approach leads to a display that
can be easily distinguished from the background when necessary, but can
also be allowed to fade out of attention, and not be tiring or "intrusive"
when not desired.
3D spatialized audio in order to study
source location, type, and timing for neural events. The multichannel EEG data are turned into multichannel audio, via direct audification, or via a transformation into a sonification. 
is software which allows a designer to prototype a spatialized audio environment, and contains much of the functionality that a full-fledged design environment should have.
MAD Monkey is written in MATLAB. Any computer with MATLAB installed should be able to run the application.
in the practice of medicine,
we have begun to study both the individual display components (both auditory
and visual),a nd how the technology is deployed. The MedTech Project is
a collaborative effort to examine how the physical arrangement of technology
(e.g., a desktop computer in a doctor's examination room) affects the quality,
or perceived quality, of patient care. 
can be informed by the results
of eye tacking studies, in addition to the more traditional reaction time
and accuracy studies. We are applying eye tracking techniques to examine
interaction with, for example, dialog boxes and other widgets.