#word2vec

"Experimental Creative Writing with the Vectorized Word" by Allison Parrish

I’ve been a bit busy (more on that in future posts) but I had to stop and watch this talk by Allison Parrish about using word vectors for creative writing.

It’s a good introduction to both word vectors and some things you can do with them. Colors! Distributional hypothesis! Visualizing word vectors, which is a really neat way to learn about them!

The overall theme of the talk is about treating the written language as smooth spaces rather than discrete words, so you can do things like apply a blur to a paragraph. The analogy of a theremin for text is both a striking image and a call for a different kind of relationship with text, drawing on ideas from Amiri Baraka.

It’s especially worth watching for the poetry reading. The audible performance highlights how sound and the emotion are an essential part of language, yet tend to have less attention paid to it in some parts of the generative space.

Thanks for posting about my translation of Genesis into words starting with A! If you’d like to play with the word2vec-derived tech behind it, I have written a program that makes rhyming word pairs on any topic or combination of topics, and put it up on the web. Here’s the URL to try it out, if you want: http://73.172.60.168:8080/rhyme

Removing bias from word vectors

I think it’s important to remember that algorithms are not neutral, objective truths. This is especially true when they’re trained on unfiltered public data. So this writeup about removing bias from the ConceptNet Numberbach word vector dataset is compelling on both a practical and theoretical level.

For the practical, they have word vector data that has measurably less gender bias embedded in its gender analogies. For the theoretical, they discuss some methods that can be applied to other kinds of machine learning, and link to more research.

ALPHA (A TRANSLATION OF GENESIS 1)

Here’s a very clever use of word2vec.

Using the word2vec data, Douglas Summers Stay made a list of adjective-noun pairs, indexed by the sum of the two words’ vectors, and used that to translate general words into phrases that started with the desired letter. He then used this as a tool to help with his poetry composition, translating the first chapter of Genesis into words that all started with the letter A.

The resulting phrases remind me of kennings, the Old Norse poetic technique of using compound expressions to obliquely refer to  a person or concept. (The Skaldic Project has a database of actual historical examples.)

I’m fond of kennings, parallelism, sestinas, and villanelles: poetic forms that introduce extra structure and constraints on top of the natural sound-rhythms of the words and use repetition to juxtapose imagery. They’re especially useful for procedural generation. These regular forms are things that the machine can more easily detect and compose, even if it isn’t quite sure about the meanings.

The other significant thing to note here is that this was done with a tool, an AI-assistant. Poets have been using tools like this at least since the invention of the rhyming dictionary, but generative methods offer much more flexible ways to explore unexpected connections.

Poetry is, in some sense, a way to get beyond the surface meaning of words and find the hidden patterns. The meaning we didn’t anticipate, the alien way of understanding what it is to be human. And AI, at its best, is about giving us access to alien ways of thinking.