Sonification Journals

Air quality data for two cities, Vancouver in the left channel, Edmonton in the right, is sonified using a mix of FM and Granular synthesis. This exerpt includes hourly data from January and February 2014.

Data Streams: Carbon Monoxide = low carrier frequency Ozone = medlow carrier frequency Sulphur Dioxide = medhigh carrier frequency Nitrogen Dioxide = high carrier frequency Particulate Matter 2.5 micrometers = granular 'clicks' (this is the deadliest)

Here is a more aesthetically driven sonification/soundscape composition I’ve been working on. 2014 Weather data for the Gastown area (precipitation, mean temperature, wind), controls the effects (granulation, filter, pan) of the single recording provided at the beginning of the piece. 

What results is a composition with interesting properties. The sonification partially transforms the sound of the steam clock into trickling water. That is, trickling water can certainly be heard througout the piece, but the iconic sound of the clock is still recognizable. This design appears to transform the sound while maintaining its relationship to its source. Hence the twin designation of soundscape composition. 

BC Hydro electricity transmission data to and from Alberta over a period of 24 hours (from 12am) on January 1st 2007.

rising shepard tones represent a net gain of power

falling shepard tones represent a net loss of power

Initial impressions reveal that there are issues with the design of the Shepard tone mapping. Mainly the fact that the Shepard tone is meant to be an auditory illusion, and its masking qualities, even when reduced, compound the perceptual work needed for comprehension. Also, the mapping of power consumption onto speed is psychoacoustically harder to differentiate than pitch would be. It’s true that “the degree of rhythmic regularity of an event’s time structure affects how and how effectively we attend to it” (Jones, 2004 p.76). By tying the rate of modulation to the onset times of individual Shepard sequences, I have created both a temporal and a frequency masking scheme. The ear naturally corrects for this by partially adjusting its perception of phase - it hears a coherent and continuously changing rhythm rather than discrete data points.

This second sonification scheme was deliberately designed not to rely on any visual conceptual models, with the idea that it is more accessible by a broader audience including those with no culturally established norms of visual perception (i.e. the blind). Instead, the sonification takes on a more embodied approach (Antle, Corness & Droumeva, 2009), where BC Hydro represents the subject of affect rather than the object of analysis. This means that, from the embodied perspective of BC Hydro, power is either flowing into you or from you, absent of any notion of a particular direction of travel.

The binary dimension is, in this case, mapped to frequency. As power flows into you, you ‘power up’, and as it flows out of you, you ‘power down’. This is expressed through a Shepard-Risset tone, which conveniently never ends and is therefore more suited to represent states of being rather than discrete values.

It is worth noting that ‘perfect’ Shepard tones were not implemented. ‘Perfect’ refers to the ability for a Shepard tone to maintain the illusion of never ending. In short, this is done by designing the amplitude curve of the super-imposed octaves in such a way as to mask the repetition of their rising or falling sequences. In this sonification, a simple exponential amplitude curve was used instead of Risset’s original formula. This is meant to make the perception of discrete data points clearer and easier to follow.

BC Hydro electricity transmission data to and from Alberta over a period of 24 hours (from 12am) on January 1st 2007.

rising shepard tones represent a net gain of power

falling shepard tones represent a net loss of power

There are essentially two dimensions of data that require sonification. The first is some sort of binary mapping which represents the direction of travel. In other words, is it negative or positive; is BC Hydro losing or gaining power? The second dimension refers to how much power is being consumed or lost in either direction.

My first solution to mapping the binary data dimension onto an audible parameter is based on the conceptual model of the image provided. We can imagine electricity travelling from left to right when the value is negative, and from right to left when the value is positive. Electricity to and from BC Hydro is naturally suited, in this case, to the panning parameter of an audio signal. As electricity is being sent to Alberta, we can hear what is meant to be the ‘charge’ travelling from the left to right speaker and vice versa.

The hypothesis is that listener comprehension of a particular sonification improves when they can anchor their understanding of the information to a familiar conceptual model (Landy, 2007). Because of the sighted listener’s familiarity with 2-dimensional maps, the binary dimension of negative power and positive power can be easily mapped and understood as travelling direction from left to right on a coordinate plane.

The second dimension, the amount of power transfer, is mapped to frequency. From a designers’ perspective, this mapping is the easiest to implement, and presumably it is also the most intuitive form of mapping to listen to. A higher frequency corresponds to more power being transferred in either direction. 

BC Hydro electricity transmission data to and from Alberta over a period of 24 hours (from 12am) on January 1st 2007.

direction of travel is mapped to the panning parameter

amount of power sent in either direction is mapped to the frequency parameter

The dataset under examination is called Hourly Telemetered Tielines and refers to the actual flow of electricity being either sent or received between BC Hydro and Alberta over a 24 hour span on January 1st, 2007.

There are 24 individual data points beginning at 1 in the morning and measured in MWh (megawatt hours):

-348

-400

-404

-454

-446

-353

-339

-263

-362

-356

-287

-308

-302

-275

-135

67

314

158

259

122

192

148

-48

-293

Negative values represent a loss of BC power; positive values represent a gain in power from Alberta.