#movement sonification

Collection of examples on sonification of body movement. By the use of computer-vision, body-tracking techniques or electronic sensors the body’s movements are captured and used to generate/control sound. Check related works under the tag “Movement Sonification”.

[pic. GøL4M, Frédéric Deslias // OBLIQUE THEOREM, Jung In Jung + Skye Reynolds, Tamsyn Russell + Chih Peng Lucas Kao // PRÉSENCE,  Rudolfo Quintas]

POLYTOPYA

"The second in a series of pieces focusing on creating connections between real body parts, totally synthetic sounds and imagined body parts through artificial neural networks. In this respect, a series of interactive audiovisual bodily extensions are attached to the bodies of the dancers to generate a shared world where both entities (real and virtual) establish their creative conditions reciprocally."

By Pablo Palacio and Muriel Romero, 2014. More info and full credits here.

SOMAPHONY

An interactive audio-visual composition, based on hand gestures tracking and biofeedback sensors.  

By Antony Rayzhekov, performed by Katharina Kölle, 2015.

OBLIQUE THEOREM

“Sound and dance collaboration using interactive computer technology. The sound design corresponds with an improvised physical score during which the dancers animate and sculpt the sound by manipulating the wires on the hacked game controllers (Gametrak) which are interconnected and fixed on the floor in a triangle formation. Data relating to position and length of the wires is programmed into MAX MSP.”

Sound by Jung In Jung; Choreography and Performance: Skye Reynolds, Tamsyn Russell; film: Chih Peng Lucas Kao, 2015. More info here.

GøL4M

“GøL4M is thus both a laboratory choregraphic work on the amplification and the mediatization of the recorded movement, and conversely, the execution of that movement in a field of external forces experienced by the dancer. GøL4M is a mutant form which brings together the digital body and the real, physical, organic being. Here all the elements harmonize progressively to form something indissociable and tragic …”

Directed by Frédéric Deslias, 2014. More info and full credits here.

PRÉSENCE

Interactive sound sculpture that uses the movement of visitors to generate sound.

New collection where movement is used to generate and manipulate visual shapes and forms. Check more under the tag Motion Sculptures.

[Pic: AEOLUS, Ersin Han Ersin // MOMENTUM, Schnellebuntebilder and Kling Klang Klong // SYNTAX ERROR, Marvin Bratke and Alex Grasser]

DANCE PARTICLES Explorations with real-time animated meshes of dancers.

By Andreas Muller, 2015.

AEOLUS “Aeolus is an artistic investigation of human motion and invisible natural forces, “wind” in particular. As a real time piece Aeolus enables us to visualise the wind flow for the time being and the pre-recorded choreographically designed human movement. Aeolus visualises imaginary relationships between movement and space, past and the time being, force and the flow.”

By Ersin Han Ersin, 2014. More info here.

MY NAME IS NOBODY Interactive multimedia dance performance

By Bartek Szlachcic (Visuals/sound spatialization), Dawid Lorenc (choreography), Kostas Georgakopoulos (project direction), Frano Durović (sound composition), 2014. More info here.

MOMENTUM “Interactive framework to create synaesthetic experiences of sounds and visuals generated from body movement.In their first collaborative work “MOMENTUM“ the user and his surroundings transforms into a fluid creature in an ever-transforming system of particles. A virtual reality, in which movement becomes sound and music and matter seems to dissolve.”

[Behind the screens]

By Schnellebuntebilder and kling klang klong, 2014. More info here.

SYNTAX ERROR

“Three-dimensional, digital metaphor of a real life process influenced by the precision, rich detail, fine mechanism and energy. The digital tectonics mostly capture the fragility of a dancer's movement and therefore show the beauty of human inaccuracy in the syntax of a programmed dance sequence.

The digital sculpture is a representation of individual human interpretation and implies the attributes that contrast human behavior from mechanical perfection. “

[pic. Clara Rockmore performing in 1932 (top left),Lydia Kavina performing in 1978 (top right), Terpsitone's diagram] The instrument had a metallic conductive platform where the dancer would move. Vertical movements would change pitch and volume is controlled by the distance from the back of the platform. More information here and here. For more deep knowledge related to the Terpsitone, the original Theremin and its variations and another experimental sound instruments from the early 20th century in Russia check the  book : Sound in Z - Experiments in Sound and Electronic Music by Andrei Smirnov. SEINE HOHLE FORM [2000]

"It is a fragment from the prose poem "Gesichter" by Rainer Maria Rilke, roughly translating to "its hollow form." As a starting point for this interactive work, it serves as an emblem for the interesting challenge of creating a musical work that only exists when a dancer moves. Using real-time synthesis and video tracking technology, the choreography is affected by the live generation of sound through sensors and computer systems, and the music is in turn shaped by these movements." Music & Interactive Programming: Butch Rovan, Video Tracking System: Frieder Weiss, Choreography: Robert Wechsler, Dance: Robert Wechsler / Laura Warren More info here. AUDFIT [2014] "AUDFIT is a dance costume that precisely reads the movement and position of the dancer's body using motion detectors and accelerometers designed by Krystian Klimowski. Data that is thus provided is converted in real time into sounds. The audience listens to the music created from the dancers movement through the headphones (Silent Disco headphones sets), with a choice of three CB radio channels, each expressing the sound in a completely different manner."

  "AUDFIT project the music is generated by the movement - even the tiniest move of the body releases a sound sequence and electronic tunes. The body of the dancer is mapped by 9 strategic points that orientate its position in space. In AUDFIT the movement creates the sound, which is integral with the action, dance becomes an instrument, a programmer of audio-visual perception formed in the skull of the recipient." By Strangeloop. Marta Romaszkan - dance/movement, Krystian Klimowski - physical computing, sensor programming, Patryk Lichota - conception and sound programming. OMINOUS [2014] "Similarly to a mime, I model malleable sonic matter produced by my body. By using the wearable biotech instrument “Xth Sense”, the bioacoustic sound of muscle contractions is amplified, digitally processed, and played back through eight loudspeakers and subwoofers. The natural sound of my muscles and its virtual counterpart blend together into an unstable sonic object."

"This oscillates between a state of high density and one of violent release. As the listeners imagine the object’s shape by following the gesture, the sonic stimuli induce a perceptual coupling. The listeners see through sound the sculpture which their sight cannot perceive." By Marco Donnarumma. More info here. EXPERIMENTS IN CHOREOGRAPHY AND 3D-SOUND [2013]