Many people use data sets to form a line on a graph. But Mark Ballora, a professor of music technology, maps data to auditory characteristics — such as pitches and loudness — and creates music in the process.
This technique, which is called "sonification," is Ballora’s specialty. He’s used it on a variety of natural phenomena such as the aurora borealis and tropical storms.
“When I was doing solar winds — the things that create the aurora borealis — I wanted it to sound shimmery,” Ballora said. “But then I wanted that to change according to the characteristics of the data, so that what you're hearing has a shimmery sound that evolves in a way that describes the behavior of the data set.”
The process begins with a spreadsheet file that he reads into a synthesis program, which saves the data set as an array of numbers. After transposing the numbers, he said it’s a matter of choosing an instrument that “seems suitable for the data set” on an intuitive level.
Ballora said most researchers likely won’t be integrating sonification into their methods anytime soon, but he envisions it being used for science outreach and educational applications for kids in grade school.
“It could get them to think that science is interesting because it sounds cool, not just because it looks cool,” he said.
Ballora has been sonifying data for many years, but his “big break” came when he collaborated with Mickey Hart of the Grateful Dead. Hart, a musicologist himself, asked Ballora to sonify data for the film “Rhythms of the Universe,” which drove Ballora to work more like a sound designer than a sound informer.
“That just kicked me into another level of productivity, and it led me to other collaborations with people here at Penn State,” Ballora said.
Recently, he’s been working with Jenni Evans, a meteorology professor at Penn State, and sonifying data sets describing tropical storms.
One of the major characteristics of these storms is air pressure levels, for which he used an oscillator to create a variety of “windy, swirly” sounds.
“That’s the part that appeals to the overgrown kid who likes playing with synthesizers — I can design cool sounds,” Ballora said.
Evans said she has enjoyed imagining the best kinds of sounds to use to communicate the information on different aspects of hurricanes, such as rain and wind.
Earlier this year, Ballora received seed grants from the National Academies Keck Futures Initiative for two of his next projects.
For one of these projects, he will collaborate with a marine biologist and cellist named Heather Spence to create a music performance piece featuring Ballora’s electronic-based sonification of underwater data and recordings.
“One of the great things about working with Mark is that he’s been very inclusive,” Jeffrey Rimland, assistant teaching professor of information sciences and technology, said. “So he’s not just interested in the technology, but how the technology can apply to people, and how he can help people with it [and] bridge gaps between disciplines.”
State College and Seattle-based electronic band “knowhere” released its first single on Labor Day.