#neural synesthesia

Neural Synesthesia

This experiment by Xander Steenbrugge is the kind of thing I expect to see more of as we get deeper into the neural art age. There a motivation behind the exploration of the latest space, in this case translating audio into visuals.

(Note that there are some annoying bright flashes when the high-pitched drums get going.)

As Xander explains it, the way this works is that he picked a bunch of points in latent space and put them in an order that had something of a narrative arc to it, and then ran a custom-written Python program to interpolate between them and do beat modulation.

I’d be interested if someone can invent a way to find a narrative arc programmatically, or maybe combine it with things like Spleeter for more informed audio processing. 

https://youtu.be/85l961MmY8Y

An AI-created music video where procedural transitions between multiple paintings is timed to music.