Immersive Studio

Please Watch with sound on! 

Pre workshop test of my immersive room. 

First participants using the space

Motion to sound test with internal webcam 

Creating the East of England’s first immersive, reactive, and programmable  space for the community, is not for the faint of heart. In an old warehouse that is now an amazing studio space in Peterborough, I’m trying to realise dreams...

This journey started after experiencing some truly amazing interactive projection work at Bournemouth’s festival of light and a wonderful visit to the work of Luke Woodbury https://www.dotlib.org/about, especially at the Three Ways School in Bath, and their amazing immersive classroom or sensory studio. 

After the experience of participating and being expertly shown the possibilities of this technology, I was keen to learn more. Luke, generously offered to help with advice and signposting to correct equipment and software links! So, I had no excuses now...

This started a passion for new approaches to technology, space / environments, reactions and interactions. The integration of software and hardware, plus the way we can control and influence the interactions to explore our social boundaries... This is currently an extremely interesting topic for me, as I’m designing the space within the backdrop of a pandemic!

How it works:

A programmable, immersive, interactive studio uses software to help the artist or creator tell the room what they what to happen, they can add the sounds, lights and / or projections to create an environment or explore a situation... it can be a bit like VR crossed with a sensory room, that can all be programmed (For Star Trek fans it’s basically the holodeck... eventually!)

For it to be responsive and interactive it requires feedback from the user. Basically the user does something and the rooms sound, colour or image responds to that something. The feedback device can be a camera, light detector, infrared detector or tracking camera. It could be a switch, floor plate, conductive switch, contact card / receiver (think contactless payment without the payment, something else happens) it could be temperature sensors or distance/proximity sensors the list goes on... basically any device or sensor that a robot needs to navigate the world can be adapted and used, plus other devices like the wonderful makey makey and ardunio.  

https://makeymakey.com/       https://www.arduino.cc/

I now have some ideas and some of the technologies knowledge plus the space. It’s now a matter of building, programming, hacking, and testing!

Where to Start?

Luke advises to start simple and then build on that simplicity to more complexity, it sounds logical and manageable, so I decided to start with getting the lights to react to movement. I’ve worked with light before at nightclubs, creating moods manually with DMX light controllers, but welcomed the challenge to learn more.

Then I would add sound. I’m fairly confident working with sound or audio as that’s my major (to coin an Americanism). The software I’m using to control the space is Max for live. This is part of an audio workstation application that I’m very familiar with called Ableton Live.

If I can get these two elements working together I can build upon this as I go. Eventually to 360o projection and 360o ambisonic sound!!!

I was able to apply for some funding from the Cambridge technology fund to help with capital costs involved of equipment purchases (lights, controller, webcam, cables, etc.) also for R&D time at events, plus some testing via workshops to gage effectiveness and response to the idea and inform the application / approach going forward.

In October 2019 I started to research the lighting opinions available... there are a bewildering array of different lights with extremely variable costs! It took longer than expected to find the lights that would fill the space and be on budget.

I thought that a hardware controller for the lights would be best, plus it could be used to control the lights when not using the software. I did contact Luke just to make sure that was the correct choice. He said that a better choice would be the Enttec DMX usb pro mk2 as there was already a little program that had been made to access and control the lights via the software and it was free to use. So I took his advice, as programming a new device to interface with max/map was probably a step too far for me, at this early stage in the development.

The first experiment needed one more piece of technology - an external webcam will enable an image to be used as a data source. I had managed to get a little experiment working with the built-in webcam see video 2

Again there are lots of webcams on the market so very expensive! As I had heard I might need to manually hack the hardware I was reluctant to spend a lot! I did try to use a security camera but the type of video output data was not compatible I’m sure this could work with a little more research or extra hardware video graphics cards etc... maybe.

I chose a Microsoft webcam as it looked simple and received ok reviews.. I hooked it up to the computer via usb and opened the software found out how to select the new webcam and managed to get it up and running quite quickly see video 3

I thought I could now try the idea of turning on lights and hearing the music by moving in our large space with the external webcam able to cover an area by placing it through our internal office window 

I set all the parameters in the software and placed a water effect light in the main view of the webcam, it soft undulating light was used to get the threshold value working - basically, this light was being used to set up the software so that other lights were just reacting to the movement from the water effect. Therefore, the further movements of people would trigger the lights and sound. see video 1

After a local event producer / crypto artist Lee Mason saw an example of the work he commissioned a bespoke installation for his future night club called Flux, this was the perfect opportunity to test the ideas on the general public. I was able to assigning lights to enhance the aesthetic of the room, and provide a bespoke reactive experience via ‘The Dancing Dresser’  Video clip here. In the clip is the amazing Rosie Summers creating Art in VR, and you will see occasional glimpses of the Dresser lighting up. The lights are reacting to movement in the room, from other lights or people’s movements. 

Now was the time to provide a full workshop to showcase the space for some Young people from the City Collage Peterborough. 

Results - Even a limited review of 5 young people with Special educational needs and disability has shown me that it's an enjoyable space to play with, dance, and connect with others.. 

Work in progress...