<< back
Development of sonification design theorgy: Metaphors, mappings,
holistic sound design, and data-specific sonification
A major element in the practice of science and engineering, and in the
education and training of researchers in those fields, is the analysis
and interpretation of data. Although most data exploration tools are
exclusively visual in nature, data presentation and exploration systems
could benefit greatly from the addition of sonification capacities.
Auditory representation of data, or sonification, promises gains in the
representation of temporal and high-dimensional data, in data
monitoring tasks where the eyes are busy, in tasks requiring
recognition of patterns in a data set, in high-stress or critical
conditions where cross-modal correlations would be of value, and where
the display users are students or researchers with visual disabilities.
However, very little work has been done to determine the best ways to
map data dimensions, such as temperature or pressure, onto auditory
display dimensions, such as pitch or tempo. Further, little work has
been done on the question of the metaphorical associations that sounds
create, and how to utilize these connotations to design effective,
compelling, and pleasant sonifications which are easy to learn and
intuitive to use.
The proposed research seeks to discover the optimal data-to-display
mappings for use in scientific sonification and investigate whether
these optimal mappings vary within and/or across fields of application.
To accomplish this, we propose to develop a cross-platform research
environment that will incorporate sophisticated sound synthesis and
data collection capabilities. The sound synthesis engine will allow for
the precise control of many auditory dimensions, so that we can explore
a range of promising new ways to map data onto sound.
Experiment 1 will ask undergraduate participants to compare pairs of
sounds and indicate which pair member is better for representing
various data types. For example, a participant would hear two sounds
that differ only in pitch, and be asked which best represents something
hotter. This is a very direct approach to determining population
stereotypes and listener preferences in data-to-sound mappings. Over
14,000 data points will allow us to discern response patterns that
indicate population stereotypes for this group of listeners.
Experiment 2 will extend Experiment 1 to listeners who are researchers
in the fields of Chemistry, Physics, Biology, Economics, and
Psychology, to examine how mappings differ across various scientific
areas. This will be critical for designing effective sonifications that
take into account the specific needs of scientists in different fields.
Experiment 3 will measure performance (reaction time and accuracy) on a
number of tasks that use data sonification, in order to validate the
population-specific mapping preferences that emerge from Experiments 1
and 2, It is critical to determine how the stated preferences of
listeners compares to their actual performance using those preferences.
Sonification design guidelines should be based on the results of task
performance, as well as on the population stereotypes that listeners
report directly. Experiment 3A will involve a statistical graph
interpretation task, representative of tasks currently performed in
laboratories and classrooms using visual graphs. Experiment 3B will
involve a data-mining and exploration task, requiring listeners to
detect patterns in multivariate data sets representative of those that
prove unyielding to visual analysis.
In order to validate the results and conclusions derived from
Experiments 1-3, and to continue to explore the practical issues
associated with sonification, we will develop a cross-platform
sonification software tool for use by researchers and educators. The
sonification application will include two novel and important features
Ñ a content-analysis wizard and enhanced auditory design Ñ to create
optimal displays depending on the characteristics of the user, the
domain of study, and the type of data to be explored. The tools will be
suitable for widespread deployment and integration into existing
scientific and education settings. Tool use will be monitored and
analyzed, through usability testing and usage questionnaires, to inform
subsequent refinements in both the theory and tool design.
Dissemination of the theory, design guidelines, and our research
paradigm will bootstrap further research efforts in this field. Use of
the sonification tools will answer a call from the auditory display
researcher community for better theory-based tools, provide researchers
with a new and powerful data-analysis tool, and provide
visually-disabled scientists and students immediate inroads into the
world of science that is still largely dominated by visuocentric
techniques and technologies.
<< back
