This week I've completed (ish) work on the first iteration of the live version of the Tower of Babel software (babelLive v1), culminating in a real short presentation/performance as part of the ECA degree show.
In its current state babelLive generates only midi, controlling software instruments in Logic, rather than generating any sound itself. At the moment this is for speed, allowing me to focus on the semantic analysis portion of the patch with results that immediately sound fleshed out.
Four separate 'semantic loopers', each assigned to a different instrumental voice. These loopers scroll through the sentences they're sent, analysis each individual word and outputting its valence, dominance and arousal to control midi note pitch, velocity and length. Rhythms are generated by the word length being multiplied by the millisecond value of the tempo. The current 'stop' command '()*' can be applied to any of these loopers by prefixing it with one of the heading tags, or applied to all by sending it on its own.
Markdown (ish) control. I've chosen to borrow bits of syntax from the markdown language, as it was the closest tool I could find to plain text. These markdown tags are currently mapped as follows:
Headers (#,##,###,####) - looper, and therefore instrument, selection. the loopers are labelled sequentially from left to right, with the heading number deliniating the looper number.
Italics ('*') - records and repurposes keystrokes to generate rhythms. This felt like a really fun addition as the rhythm of the keystrokes is such a present element of the physical performance.
Parameter Control - as well as just influencing pitch and rhythm, it would be remiss of me to not provide support for deeper influence over the sound. The two instances of that in the current iteration are control of send level from the piano track to the spectral drone maker in logic, increasing the level each time the individual word valence drops below a certain value, and increasing the chances of the buffer repeat plugin firing with every stroke of the back key, so that the users mistakes build up through the piece and become part of it.
Midi Out - at the minute there is no built in sound generation in the software, and while this is something I hope to implement in future versions it does give it a greater deal of flexibility at this stage for trialing sounds and aesthetics. This also gives the software the ability to control external hardware, functionality that cannot be rated highly enough.
The most obvious bug at the moment is that the keystroke tracking is pretty unreliable, sometimes only firing every other time or not looping the phrase at all.
On board sound generation - in its first form I think this will take the form of VST/AU support, easily done through Max's objects designed for this support. Following this, I hope to build a set of custom sound generators for the software.
Deeper Control - The most obvious way in which the project will progress is much deeper semantic analysis, controlling a wider range of instruments, of parameter control, and perhaps of some more interesting visual elements. I'm currently dedicating my time to exploring the context of the work, and finding a narrative to situate the project within. There is so much interesting discourse regarding the relationship between music and language, as well as the relationship between live coding and an audience, and therefore there is incredibly fertile ground for planting this project in.
This version of the software is available, although still a little messy, for poking at my github: https://github.com/twistandsht/babelLive