#torch-rnn

Dreams are really weird, when you think about them. Let's lie still for several hours and maybe vividly hallucinate?

A lot of what people call AI these days are algorithms called artificial neural networks, a type of algorithm that's modeled after the human brain. They have some things in common with human brains - virtual neurons, and even some of the same mechanisms for analyzing images and navigating mazes. But can they dream?

The answer is yes - if you design them that way. There's a fun interactive model where you can play an algorithm's dream version of the Doom level it was trying to beat. By training in its own simulation of the level rather than the real thing, it could focus on the most important parts.

Now here's another way to get an algorithm to dream: train it on the rambling, semicoherent descriptions that humans give of their own dreams. Pete Majarich asked me what a neural network would make of the over 19,000 dream summaries in The Dream Bank. 

I have the answer. And it's pretty weird.

I used a neural network that can build words syllable by syllable - that way, it can make up its own words, yet they'll probably at least be pronounceable. (It doesn't do capital letters, though, so I did those for it.) The trained neural net stuck mostly to existing words, and it even got pretty good at the punctuation. But, like most text-generating neural nets, it has a limited memory, as you can see.

I am shaving into a great dream. I see four quite small people in a huge tank with all kinds of old brass grape liquid on his hair and a red neck like a maze. Of course I help him the kitten through his nose to take a plug and nonchalantly look for a commute we can see her company. He points off Father Andrew and opens the gun. I encourage the old man with a loud sandwich of soap deep signs and a simple juice in its face of mine in a box of wool dog, feeling suspected!

I see a cat on the floor where there are two eels (here east who would grow on the floor). I hear the room I have a new deal of ship by which they are washed to go downstairs the structure and they'd be driving through a short table, either someone else too and we have to fix us. I look annoyed and walk around the strip below from my back and the spirits will be awkward.

There was, oddly, one point in the training where the neural net was obsessed with going to the bathroom.

I am in a small room and see a large piece of stairs in the middle of the house. I am in the water and I see a woman in a large room with a few minutes and I see a small baby man and he says he is going to go to the house. I say I don't want to be in the dream. I am annoyed and say, "Oh no, I don't want to get in the bathroom."

But my absolute favorite part was the incredibly bizarre imagery it would come up with.

I notice my car is now a tennis ball.

I am a beautiful, chubby, big bathtub.

There is a water gun and a clown wall and another coconut object.

I shake my cheek and say, "Oh. I respect sleeping in a large metal cart and record cards."

I see a huge enemy dress with a small platform of wooden occasional people.

Then my teeth are in Colorado.

It has stairs over Derek's address, and his name is a plastic sandwich.

I see a lovely maglugated creature with a raging red petacles and jock leaves.

I generated thousands and thousands of words of neural net dreams, and let me tell you, it really loves to ramble on. There's a "temperature" setting that governs creativity, and at the lowest temperature setting it will describe one repetitive neverending dream for all eternity (I've had conversations like this). Read them here!

Support AI Weirdness and get bonus content: a more selective list of the best bits.

And finally! Pete Majarich has arranged several dream excerpts into fun prints. Check them out here!

I’m upgrading the neural net which powers @GLOSSATORY which is trained on about 83,000 definitions from WordNet, here’s some early sample output:

rib: human water in a carburetor

master: perfume having or load or cavity with vanity and graphics

chelicrutch: a medical instrument that closes two horns of related walls of hard vessels

carpon: a small silky egg; especially lured bodies stored to form a fruitcake

outpasser: a digital unit containing various hydrozoans

oil-intradulent: any member of the ankle whose concluding pressure or injuries of skin solution; the forearm and is many widely used in government orders

melody: the internal center of the esophagus that make names from the same direction

venison: a woman song and practically representative of property until reserves to being irregular stimuli

worm: turbulent channel of a book; used to call more than the properties of values

frying_pan: a small mass of open coating near the back of an audiotape

strawberry_tremor: the seeds of a lamb

breeze: a salad filled with bronchial ground hardwood forests

acorn_scale: a stringed instrument usually including the pressure exertion and blood control class

solidity_illusion: cell flavored with short red and coffee

granulated_russian_custard: moth whose leaves resembles a common in a radioactive colony

acumedia: important food fishes mostly with pelvic (exclusive to mosses and heads of many long hard centered fruit, with honey backs of forests)

penrycteria: often violet that resembles yellow spreading crown; common in red juices

astropyrranthe: spring found among the tricadian secretion; found in Australia and Africa and Asia

yakva: North American perennial with white or purple flowers

gaumed_bag: small European thrush having pale violet in sauce-like fruit

wheatgrass: North American warbler with a very large soft showy white body and a long edible purple or yellow flower

Like most of you I have been entranced by the neural network expieriments going on over at http://lewisandquark.tumblr.com/ and wanted to try out the exercise on some other data sets, just to see how it goes.

In my search for large datasets that would take minimal typing to get in an acceptable data format, I remembered that there’s a huge registry of roller derby names: https://www.twoevils.org/rollergirls/

I pulled out the full list of names from that page and plopped it into the Docker container that has been created for running torch-rnn: https://github.com/crisbal/docker-torch-rnn

With a huge dataset I was able to pull out some wild new names. So here are some samples, following Janelle’s example of tuning the temperature value to see what shows up. Apologies for the length of these lists, but it was so hard to just pick a small selection.

Recently I ran across the blog of @lewisandquark, which is insanely awesome. Right now she is doing incredibly cool stuff with neural networks. There’s also all kinds of nanolaser porn to look at. 

With her efforts as inspiration, I started playing with char-rnn(and now torch-rnn). I’ve been training a neural network with the works of Lovecraft lately. It has a hard time producing anything coherent, but I have noticed that it is really good at inventing new words. Some of these words include:

satisficised

effagued

husing

astrude

hyports

remarisity

capaneons

lame-sacked

profoundured

unvaster

And now an example of the output itself:

“Amaze's place and extending without men were seen to this supposewaed railing withlid debris city and refationably rain, and sane clutched with sudden vality of the weird rather and hands beyond its profoundured flying strong and stone such to light as one five-traders in the face could struid above him, and obtained at night in it! What knew it noticed an articler to Terrible Revert. His focus of the man'ven time to think of the dooms, and but purpose at dream on the moan the post-of-wall was continued, complete quarter of evil hazieds took down there was any shipped by the windows in Nature, all his coaching.”