Data Sonification: Give Data a Voice

By Kirsten Giesbrecht

Title: Using Data Sonification to Overcome Science Literacy, Numeracy, and Visualization Barriers in Science Communication

Author(s) and Year: Nik Sawe, Chris Chafe, Jeffrey Treviño, 2020

Journal: Frontiers in Communication

​TL;DR: Data sonification translates data into sound, increasing inclusivity in science communication and reaching audiences such as the visually impaired and people who are not comfortable reading graphs. Data sonification can also be used as a data exploration tool for scientists. I also outline tools from the paper to consider if you’d like to try translating your data into sounds. Why I chose this paper: I’ve never heard of data sonification and was immediately intrigued when I listened to an example and learned how it could reduce science communication barriers.

Listen to this composition by Sawe and authors. Although it sounds like a somewhat abstract orchestral piece, each note actually corresponds to a tree from Stanford ecologists’ data set of thousands of trees across the Alaskan coast. Every aspect of the music – from the pitch and length of a note to the instruments playing, even the timepoint in the piece where each note is played – corresponds to a data variable of a tree represented. Listen to the piece again. How do the instruments differ from the beginning to the end? You may notice the piano, representing the Alaskan yellow cedar, dominates initially, but the flute, playing the western hemlock, steadily builds to eventually overshadow the piano by the end. This reflects the Alaskan yellow cedar dying out due to climate change and the western hemlock flourishing as you travel south. We grasp this ecological enormity without glancing at a single graph.

What is Data Sonification?

This translation of data into sound is data sonification. Sawe and authors argue that data sonification isn’t just a flashy complement to showcase data, but a tool to break down barriers and reach a wider audience to create more inclusive science communication.

Why Sonify Data?

Data sonification can:

  1. Reach the visually impaired. The visually impaired community typically doesn’t have alternatives to data visualizations presented in spaces such as zoos and museums. Over half of US museums have a quarter or fewer of their exhibits available for the visually impaired, leading many in the community to forego museums entirely. Data sonification can be used in these informal learning environments, as well as in schools, in conjunction with visualizations to increase accessibility to visually impaired students.
  2. Provide an alternative method for people or students with low science literacy to understand data. Socioeconomic factors can aggravate disparities in science literacy and understanding for Americans. Over a quarter of Americans have low scientific knowledge and may disengage if faced with complicated datasets. Additionally, as data become more complex, some propose students learn to interpret large multivariate datasets early in their education. Data sonification can expose Americans with low science knowledge and students to highly complex data without requiring a mathematical representation.
  3. Enhance learning. Listening to music can improve performance on cognitive tasks. Additionally, information presented both visually and through sound increases the working memory of the learners.
  4. Help scientists understand patterns in data. Datasets are growing larger, increasingly complex, more interdisciplinary, and often higher in dimensionality. Representing these huge, complicated data sets is posing new challenges to both scientists and science communicators. Data sonification can express multiple dimensions simultaneously, show potential relationships between variables, and allow scientists new ways of exploring the data. This can enable researchers to discover patterns that are not as obvious in visual representations, such as these researchers who used data sonification to explore the migration patterns of elephant seals.

How to Sonify Data:

If you have a large, high-dimensional dataset that you want to sonify you can get started by downloading this software and using sound libraries to capture nature sounds and more. Here are 4 principles to consider if you’d like to try data sonification:

  1. Variable Coordination: Depending on the number of variables you wish to represent, you may need to simplify or sample the data while maintaining data integrity. Additionally, audiences are most likely to notice changes in the pitches played and the types of instruments they hear, so Sawe and authors suggest reserving pitch and instrument mapping for the top two variables in the dataset you’d like to emphasize.
  2. Timing: Natural data sets for sonification are data sets tracking variables temporally, brainwaves over time; or spatially, trees along the Alaskan coastline. In these cases, the time point in the sonification corresponds to a specific timepoint or location in the data. You also need to consider the length of time to present the data: too long of a piece and you’ll lose your audience to boredom, but a frenetic tempo may cause the audience to miss important patterns. Alternatively, you could create a self-directed pacing sonification.
  3. Abstraction: You may want to consider how musical you want to make the data sonification. The tree data sonification example you listened to earlier is an abstract musical mapping, but sonar and Geiger counters are also forms of data sonification that involve less abstraction. Keep in mind that as mappings grow more abstract, interpreting the science can grow more difficult for the audience.
  4. Aesthetics: If you are constructing a more musical data sonification, you may consider acoustic versus synthesized sounds and genre of music. The tree data sonification example was played using Western orchestral instruments in minor chords to elicit feelings of deep loss, however, Sawe and authors recognize that their data sonification could have alternatively been played with different instruments in a major key to prompt feelings of joy for the rise of western hemlock growth. Sawe and authors warn that science communicators must carefully select how they present the data sonification, due to music’s ability to evoke powerful emotions, to avoid manipulating listeners. Additionally, depending on the subject of the data, you could select symbolic sounds, such as the sound of striking a match to express environmental data or direct sounds such as recorded animal calls.

The Impact

Instead of grappling with graphs to understand the latest climate change news, imagine hearing the data. Data sonification can break down obstacles in science communication to reach the visually impaired community and individuals reluctant to engage with science through typical graphical representations due to inequities in science literacy. It can provide alternative modes for museum-goers to explore the numbers by adding an expressive and artistic layer to the data. Additionally, data sonification can be used as an investigative tool by scientists for uncovering patterns in data. Not to mention it can be absolutely mesmerizing.

Edited by Kay McCallum and Stephanie Deppe

Cover image credit: Happy Woman Listening Music on White Headphone by SoundOn, CC By 0. and Pexels license