#nanogenmo

I think National Novel Generation Month is more relevant than ever, as I write about here, because the most important thing about a generative model is having a strong concept and conveying that in a context the reader can experience.

During the weekend, I managed to sort out this year's NaNoGenMo works. Here is my top of interesting projects:

1. Banalified Moby-Dick — used generic BERT to determine the most unexpected words in the text and replace them with more expected ones according to the model. The overall perplexity of the text decreases, while the text degrades interestingly. There are examples of results with Moby Dick, Hamlet, and a Portrait of the Artist as a Young Man.

2. This Comic Does Not Exist — the author collected a large dataset of horror comics, cut them into frames and fine-tuned StyleGAN2. Then he generated a lot of new frames and assembled them into a 55-sheet pdf-book. It turned out very creepy.

3. Simple Dialogues — simple dialogues in which the utterance of each sound is automatically detailed to a detailed description of the movement of the lips, tongue, ligaments, and the entire speech apparatus.

Not in the top, but attracted some attention:

4. Activity logs — automatic generation of work activity reports ("...Then, I supervise monitoring report for Org 1 for 15 minutes...."), once I already wrote a similar story.

Earlier this month, I tried training an algorithm called a neural network to generate the first line of a novel.

It didn’t go so well. A neural network learns by example, looking at a database of things (paint color names, craft beer names, halloween costumes) and trying to figure out how to imitate it. The problem was, I didn’t have many example first sentences to give the neural network, and supplementing with winners from a worst opening sentence contest didn’t help matters. An example:

Stop! I caused the Narguuse man who was new on Alabama, the screaming constipated eggs.

So, I asked my readers for help. I asked people to enter the first line of any novel or short story they had handy, even their own. And folks, you have made me and the neural network so very happy.

Total # of entries: 11135

Here were the most frequently-entered lines:

It was a nice day. (27) Shadow had done three years in prison. (21) In a hole in the ground there lived a hobbit. (17) The primroses were over. (17) The sky above the port was the color of television, tuned to a dead channel. (16) We slept in what had once been the gymnasium. (16) The body lay naked and facedown, a deathly gray, spatters of blood staining the snow around it. (15) It is a truth universally acknowledged, that a single man in possession of a good fortune, must be in want of a wife. (14) All children, except one, grow up. (14) It was a pleasure to burn. (13) Far out in the uncharted backwaters of the unfashionable end of the Western Spiral arm of the Galaxy lies a small unregarded yellow sun. (13)

And the most frequently-entered authors:

Terry Pratchett (268) Neil Gaiman (99) Douglas Adams (87) Arthur Conan Doyle (81) Diana Wynne Jones (76) Isaac Asimov (61) William Gibson (55) Lois McMaster Bujold (54) Philip K. Dick (53) Stephen King (51) Jack Vance (51) Jane Austen (50) Charissa Johnson (50)

(Note that misspellings and variations aren’t counted; Ursula Le Guin and J. R. R. Tolkien would have easily made the list if their spellings were more standardized.)

This crowdsourced database was quite wide-ranging indeed: from Star Wars stories to Chuck Tingle to a surprising amount of My Little Pony fan fiction. The latest version of the database is here. With my original dataset added, I ended up with 10096 unique lines (except for typos). 

I used the same neural network as last time, an open-source neural net that (somewhat unusually) uses syllables as its building blocks.

The first results showed, if not promise, then at least evidence of the high number of My Little Pony stories in the dataset.

And town go the resions with the the as to the vicered the whatboat, fluenza the of the his mix.ing the think'

What or the the not of un the the in you conple a tonight fire on the of of the at mouning and she trouror" the fash no the the which they to, an of the the all stood ponyville at es that ev the.

The neural net soon made some improvement, and once in a while would produce a grammatically correct sentence as long as it was very short.

It was an hour of the night.

The waves - there, and a class that glass of the wall of a dull place.

Something 17514.

I know they are from the mountain.

Her emlage were playing the three of the fible of the Sinnia Ously of St, in the hole is his life in a moist king.

Aliens were reared by the steam of a drought of a good ears, the young wall of the water, but she was going upon a over and had been stolen to chapter the Irish of the ratred of the building star is in Coats-and-La is a flash of dewy and big enough.

It learned eventually how to begin a book by talking about the weather - although not always successfully.

There was the snow of a perfect custom sea and early gusts from the back of the bay.

The sky was dead.

The night was very a moonless story in a mountain-10 years.

The night is like a wounded carpets from the Crumzon.

The first day of the world was born in the year 1985, in an old side of the world, and the air of the old sky of lemon and waves and berries.

And it developed a curious fondness for a line by Jacqueline Carey about not being a cuckoo’s child. Everything, for some reason, was a cuckoo’s child.

I am a cuckoo’s child of the monster.

I am not a simple person, I am the truth, and I am a cuckoo's child and the first thing you've never be blamed to be born.

"I am a cuckoo's man," said the man who had found the tapes when he had been in the door.

Many lines, especially the shortest ones, almost made sense.

The morning room came to the deep camp.

"I have no question,” I said, as I had been ringing from Inniwhite.

The farmer was born on one side.

"You are even much!"

It was a good day that had once been any thing.

It is a man trick.

The night was over.

There began with the dead end of the wind.

The telephone was coming.

With Mr. Bilbo had always been so much procision

The sky has gone.

Here was a grey one.

The sun was coming.

"She's no acterity," said the hoarse man.

There was a very high slacks for our house 2g19.

The first thing you know is not a good idea.

I am not a king.

I was surfing for my table.

There was just a man who was able to be sick.

It was god.

This is the worst thing, in an old old man of baker and bay.

A noise is a good recruit.

And some were actually rather intriguing. I might read these books.

The silence was unlike a place.

"I am forced to write to my neighbors about the beast."

Her mother was packing by the black anthill.

The sun was probably for his wife.

I am a story that was not a truth.

"I am not the door!"

I don't know what is a combined life.

This is the story of a certain man who had invented a young man.

The sky was at the door.

Alice is a story of interest.

The question was enjoying himself.

This is a story of a man in the morning.

The world was born to say that I was lost.

I saw the last of a man, who was dead.

The old man was the first of us of the beginning to the sky.

Longer sentences, though? Still a problem. Grammar is hard.

"Bleeeck, “You are clearly out of my uncle Christmas Eve, I am a cuckoo's advice and at the day that I can tell you to be a man," but he had no children to remember to the boars of the ancient girl (or Claudius the Idiot."

The year of the island is discovered the Missouri of the galaxy like a teenage lying and always discovered the year of her own class-writing bed and implored the creation of his head, and the constant final ones in the back of the high water of the stock of the dark.

All the light of the smallest man’s body in the ocean in an old angle of a giant mountain and exclaimed that the sky was the gunslinger caught over the pale of the great kitchen floor.

I knew how felt my father being to our interested to the baseing and so walter along her hours, and the holy summer of the world with the sea of the m and the exvitions of the light of elephant novice, and the top of the phenomwhere, and the witch of the world was firmer and slid and an invisible company of the year and the ancient head of the square, the song of the day of the interest note, a large zzzzzz for a very mind, and a wizard of chess.

Want to see what the raw neural network output looked like? This project is my entry for NaNoGenMo (National Novel Generating Month) which means that I generated 140,000 words’ worth of first lines, also available at GitHub.  Unfortunately, due to a prank in the input data that I didn’t catch till after I trained the neural network, 37,000 of them are the word “sand”.

I’m not bugged too much that computers are better than me at playing chess. Or at playing checkers. Or even at suggesting nearby pizza with clear driving directions (people stopped asking me long ago, anyway *sigh*). But now computers are also cranking out novels for National Novel Writing Month, too?! Ah, beans!

Since 2013, coders have been celebrating NaNoGenMo, or National Novel Generation Month. Just like NaNoWriMo, the goal is to have at the end of the month an original 50k word novel. Unlike NaNoWriMo, the novel is actually “written” by a computer program.

Sure, the generated novels tend to be just two notches above complete gibberish. But it’s a fun, self-inflicted coding exercise, and even the failures can be hoots to read.

NaNoGenMo 2020

It’s December first, and you know what that means--time to look at the 2020 National Novel Generation Month results!

There’s lots of fascinating things from this year, more than I’ve had time to dig into yet. I’m looking forward to it.

If you want to beat me to it and check them out for yourself, the submissions are in the Issues tab on the GitHub repository:

Generated Poetry: X except its Y

Poetry generated using X except its Y on permutations of the lyrics from the first 4 Nine Inch Nails albums.

It was part of enkiv2′s National Poetry Generation Month work for 2017. It uses word2vec to combine a source text with a trained style. Quite effective.

Though I think that in the most striking imagery it’s a bit prone to plagiarism, as a side effect of how it rotates through the combinations. As you can see, some of the lines are the exact words in the original lyrics.

Plagiarism, in the generative sense, is what happens when an algorithm trained on input data, such as a Markov chain, outputs verbatim from the source data.

Markov chains with too little data or too high an order are particularly prone to this problem. But other algorithms are as well; it’s particularly tricky with many kinds of machine learning, and it’s one reason why it is important to keep the training and validation sets separate from the testing set.

These X except its Y results aren’t quite the same as generative plagiarism in that sense. Indeed, the effect of combining the two texts is part of the point, I think. But the concept comes up a lot and it’s worth critiquing results with it in mind.

Does the reoccurrence of a familiar line sufficiently counterbalance the way it shows the limitations of the generation? For me, I think it tends to highlight how the original uses repetition to create a resonance in its structure that the generated poetry is unaware of. But part of generative poetry’s draw is exactly how it can take the familiar and recontextualize it.

How do the dissonantly different word choices change the effect of the lyrics?