Data visualization is a trendy means to convey large datasets into digestible, visual representations. Entire communities are built around rendering big data artistically. While useful and appealing, an over-reliance on data visualization can limit diverse perspectives on the data and exclude people, particularly those with visual impairments.
Enter data sonification, a new tool that renders datasets as sound. As the number of data professionals grows, sonification offers a new way to think about data, providing new ways to parse, think, and share the insights of large datasets. Simultaneously, the technique has huge profound implications for science communication and accessibility.
What is data sonification?
In short, data sonification is a way of representing data via sound waves. Whereas data visualization leverages location or color to convey meaning, sonification uses pitch, volume, and tempo. But there’s an art to it.
“It means representing data in a way that sounds pleasing, interesting, or illuminating,” said Jeff Cooke, a professor of astronomy at the Swinburne University of Technology and one of the pioneers in the realm of sonification. “We try to exploit the properties of sound to understand data in multiple dimensions, to pick out data in noisy environments, and to accelerate understanding.”
Cooke uses sonification techniques in his research on fast transients, incredibly fast phenomena in space that elapse anywhere from hours to as fast as a millisecond.
“These transients help us learn about extreme physics we cannot create on Earth, how and where the elements were created, the expansion rate of the universe, and more,” he said. “The idea for this program is to coordinate telescopes of every type all around the planet and in space, to simultaneously gather a large and varied amount of data.
“But we have to move fast,” he added.
Cooke and his team leverage the cocktail party effect—the phenomenon which explains the human brain’s knack for picking out faint sounds in noisy environments—and other sonification techniques. By training their ears on the sonified datasets of fast transients, Cooke and his globally distributed team can identify these events to trigger other telescopes for more detailed information needed to understand them before they fade away forever.
“Humans can hear faster than they see, which may sound counterintuitive but is true. If you train your ear, you can make decisions based on sound input faster,” he said.
When you combine all these effects, sonification promises to be a powerful tool; one that can represent the unknowable nature of space and turn foreign environments into human landscapes.
Science communication and accessibility
Sonification isn’t just a tool for data scientists and researchers. Much like its visual cousin, data sonification has a vital role to play in science communication.
Photographs from space captured the public’s imagination, and now NASA is leveraging sonification to represent humankind’s first foray into interstellar space. Presented as a flute solo at South by Southwest EDU, the composition gives character to NASA’s Voyager 1 and its galactic environs and helps assign human meaning to an extraterrestrial environment.
Beyond disseminating scientific discovery, though, sonification also offers new opportunities in the world of accessibility. Particularly for blind and visually impaired people, sonification can open new pathways into research and STEM fields.
“How many charts and graphs did you look at throughout your education? How many have you seen today? We take that massively for granted,” said Kate Meredith, director of education at Geneva Lake Astrophysics and STEAM Inc.
When thinking about the long career path of an academic, there’s an incredible amount of time spent staring at charts. What is second nature for many can be a major hurdle for others. Sonification offers more pathways for historically underrepresented groups in science and research.
“From a people empowerment perspective, I believe sonification has the avenue to engage students who would be otherwise excluded in STEM fields at a very early age,” said Meredith. “Creating an equitable playing field enhances equity in the field and creates better scientists overall.”
That ethos drives Meredith, Cooke, and many of their colleagues. Since the world of sonification is still rather small, many efforts are coordinated through the “Sonification World Chat.” There, a nascent community forges connections that provide expertise, talent, and funding for sonification projects around the world.
“The distributed, grassroots nature of sonification is part of what makes it so wonderful,” Meredith said. “There are so many disciplines involved: data, coding, artists, educators … it’s a powerful tool for access, equity, and global engagement.
“There’s no one person to solve this; it’s best solved by the collective,” she said.
Turn up the sound, amplify the message
The benefits of sonification are obvious and numerous, but significant challenges remain in a wide-scale adoption of this data science technique. As with many research tools, funding is an omnipresent concern.
“My opinion is that it has to help the sighted person to get funding,” said Cooke, a stance which Meredith agrees with.
“The perception is that it is a niche technology, which applies to both securing funding and development,” Meredith said. “Sonification can be a significant tool for science communication for all people, but to get to that stage we need to break through that perception.”
As it stands, the development of sonification tools is bespoke, with software and techniques being developed for specific use cases. Cooke and his colleagues from the Sonification World Chat are working to centralize some of these tools, with the aim to distribute and possibly commercialize the result. Meredith’s vision is even grander.
“Think about the graph tool in Excel,” she said. “In the future, that’s the kind of thing we need in the field: effortless, ubiquitous sonification.”
Artwork by Rachel Garcera
