Aubry

I’ve done several experiments with a text-generating neural network called GPT-2. Trained at great expense by OpenAI (to the tune of tens of thousands of dollars worth of computing power), GPT-2 learned to imitate all kinds of text from the internet. I’ve interacted with the basic model, discovering its abilities to generate fan fiction, British snacks, or tea. I’ve also used a tool called gpt-2-simple that Max Woolf developed to make it easy to finetune GPT-2 on more specialized datasets - I’ve tried it on datasets like recipes or crochet. One of my favorite applications of GPT-2 and other text-generating neural nets is Dungeons and Dragons spells, creatures, character names, and character bios.

Recently Max published a tutorial on how to use GPT-2 to make new apps that are more complicated than just printing out large reams of text. To my delight, people have used them to made D&D games. First, there was Nick Walton’s AI Dungeon, in which a finetuned GPT-2 generates a story and three very strange action options at each step.

My latest obsession is GPT-2 Adventure, designed by a cognitive neuroscience student named Nathan, which lets you type in your own actions. And folks, this game is SO WEIRD and SO FUN.

YOU FLOAT IN SPACE, LIKE A BALLOON, AND AS YOU DROP THE CUBE, THE X, IT VANISHES IN A CLOUD OF GREASY BLACK SMOKE

>>

When I first launch the game, this is what I’m greeted with, a simple text prompt reminiscent of nostalgic dungeon crawling games. It sets the scene. It waits for me to describe my next move.

Even getting to this point is like a blast from the past. The game runs in a Google Colab notebook, a format where you’re looking at cells of code, with cryptic comments, vaguely alarming warning messages, and all. It’s not a sleek web app, but feels more akin to playing a game on an old DOS machine, or maybe on a graphing calculator. There are only two clicks to begin playing, but there’s a good two-minute wait while the game loads.

And it’s my turn to type something.

>> look up

It takes several seconds for the game to respond, as if I’m playing it on a very ancient computer. The computer is in fact a state-of-the-art NVIDIA GPU, likely a K80 or a T4. The T4 could eat my laptop for lunch. I’m just playing with a very very large neural net. Finally it answers.

YOU’RE IN LABYRINTH OF TWISTY LITTLE PASSAGES, ALL ALIKE

Almost immediately, the context seems to have changed. I’m not floating at all. I’m in a maze. I guess I’ll walk somewhere.

>> walk north

THE TROLL STEPS OUT FROM BENEATH THE BRIDGE AND BLOCKS YOUR WAY

YOU ARE ON THE SOUTH SIDE OF THE CHASM

A NOD IS GIVEN TO THE INFINITE WONDER THAT IS URBZIG

A SOLID RAINBOW SPANS THE CHASM

And suddenly things are happening. There is a troll. There is a chasm. There is a rainbow. There is… Urbzig? I play it safe.

>> praise urbzig

URBZIG IS THE PATRON SAINT OF ZORK

HE IS THE GODFATHER OF ZORK, THE PLACE TO GET NEWS OF EPIC PROPORTIONS

URBZIG IS A STRONGMAN,遊 ALL

If you’ve played classic dungeon crawling games like Zork, you may recognize some of these names and phrases. Checking Nathan’s original post, I see that the training data is indeed drawn from these games. But because it’s based on GPT-2, the training data also in some sense also includes much of the internet. This gives it a lot of latent flexibility to deal with situations that were never in the classic games.

>> ask Harry Potter to try walking across the rainbow

ASK HARRY POTTER TO TRY WALKING ACROSS THE RAINBOW HE CAN’T THE RAINBOW IS EXTREMELY WIDE ABOVE YOU IS A LARGE DOME UP AROUND ITS NECK IS A STRONGG PORE [UNFORTUNATELY, THE DIRECTION OF TRAVEL IS A LOT MORE INTERESTING THAN YOU THINK ][YOU ARE IN A LARGE ROOM, WITH A PASSAGE TO THE EAST, A PASSAGE TO THE WEST, AND A WALL OF BROKEN ROCK TO THE SOUTH

An interesting effect of its human-written training data is that, like Visual Chatbot, it isn’t prepared for humans that ask questions about objects that aren’t there. If I refer to Harry Potter, then Harry Potter must be present. If I type something about a giraffe, then it assumes the giraffe must be present.

>> ask the giraffe which direction to go next

THE GIRAFFE IS ALREADY OUTFITTED WITH A VISOR WHICH IS CLEARLY DEFINED

Although it’s not clear that this AI actually understands what a giraffe is.

>> look at the giraffes

THE GAZEBO HAS DISAPPEARED

>> put giraffe in backpack

DONE

Navigating this world is like navigating in a dream. My location suddenly changes without warning. I thought I was by a chasm with a rainbow bridge. Now apparently I’m in a building. Dreamlike, it is also really hard to read text.

>> look around

YOU ARE IN A LARGE ROOM, WITH A PASSAGE TO THE SOUTH, A PASSAGE TO THE EAST, AND A WALL OF BROKEN ROCK TO THE NORTH A SIGN POSTED ABOVE THE NORTH WALL READS, “STOP! PAY TROLL!” A SMALL WINDOW CAN BE SEEN IN EITHER WALL

>> read sign

THE SIGN IS VERY SIMPLE

>> yes but read sign

WE NOW HAVE ALL FIFTEEN TREASURES IT IS TIME TO GO BACK INTO THE CAVE FOR THE END GAME

>> that’s awesome. but i still want to know what the sign says. read sign.

YOU ARE IN A GAZEBO ROOM WHICH HAS DOORS ONLY TO THE EAST AND SOUTH

Another thing that’s surprisingly difficult to do is check inventory.

>> check inventory

DONE

>> check inventory

THERE IS AN INTERESTING LOOT HERE

>> what is the loot

WHAT IS THE LOOT?

>> check inventory

YOU ARE IN THE BAG CASE

Very occasionally I will succeed in opening a bag. It will usually be weird.

>> open bag

OPENING THE BAG REVEALS A TEA BAG

>> make tea

AY! A LOVELY SURPRISE FOR THE SURPRISE

More so than any other game I’ve played, I’m unsure of the rules here. Some weird things unexpectedly work.

IF YOU DON’T GET KILLED, YOU ARE TOAST

>> Eat the toast.

EAT THE TOAST IT TASTES LIKE A STEAK A STEAK IS PERFECTLY FINE

Other seemingly obvious things don’t work.

THERE IS A PATH TO THE NORTH, A PATH TO THE SOUTH, OR A STEEP PATHWAY CLIMBING SOUTHWEST ALONG THE EDGE OF A CHASM

>> go north

I DON’T KNOW HOW TO DO THAT

>> go south

I DON’T KNOW HOW TO DO THAT

>> go southwest

YOU WILL BE DESCENDING INTO A VALLEY THERE IS A NARROW BEACH TO THE NORTH AND A PATH LEADING INTO A CANYON TO THE SOUTHEAST YOU ARE ON THE CLIFF OVERLOOKING THE FAR SIDE OF THE ROOM A PATH LEADS SOUTH ALONG THE EDGE OF A CHASM, AND A NARROW PATH LEADING INTO A LOZENGE ROOM

Sometimes I’m wandering endlessly from room to room, from cave to cave, encountering the same chasm in slightly different variations, the same dried-up lake. Other times I’m embroiled in seemingly unescapable fights, usually with trolls. Often reality shifts entirely around me and I end up in a maze of twisty little passages, all alike, or in the hall of the mountain king. Everything seems weirdly familiar - even the room descriptions are echoes of classic dungeon crawling games. It is as if I’m dreaming about playing them again.

My actions also mold reality. I learn quickly to always check for diamonds, and never check for traps. If I find myself in trouble, I can always do something confusing and provoke a context change. Writing “In a large bowl, combine butter and sugars” is usually enough to teleport myself out of whatever situation I’m in. I can always ask a giraffe for advice, although the giraffe usually isn’t very helpful.

Without a particular mission to fulfill, the caves and chasms begin to blend into one another. I give myself missions, and that helps. Once I decide my goal is to put together a picnic (incessant “check for food” commands eventually yielded a sack with a lunch and a clove of garlic, belonging to a horse who agreed to join me on a picnic). Another time I demand a mission and am given some php code; i decide my mission is therefore to build a website, but am astounded when a unicorn steals my laptop. This is not the only time other content intrudes on the game’s fantasy landscape - once the game tries to tell me about finding cash online, and another time I am sadly informed that there is no wi-fi. I encounter dinosaurs and troikas, none of which were in the original training data. Occasionally things get creepy as hell.

>> check for loot

CHECK FOR LOOT HERE CHECK FOR GOLD HERE CHECK FOR TORCH HERE CHECK FOR BONE HERE DO CHEST COMPRESSIONS AGAIN DO CHEST ROTATE DO CRESCENT MOON APPEAR IN THE ROOM’S CENTER

>> do back away slowly

DO BACK AWAY SLOWLY DO NOT OVERDO IT THE CHEST WILL EXPLODE AND YOU WILL DIE OKAY, SO YOU SEE ME

You can play the game too! Just follow the colab notebook link in Nathan’s blog post. Tips: 1) It may be more fun if you set yourself an arbitrary goal to reach. 2) If you find things getting repetitive, try adjusting the temperature in the “Play the game” cell to 0.6 or 1.0.

To read the fulltext of a couple of my adventures (much too long to include here), enter your email and I’ll send them to you!

You can order my book You Look Like a Thing and I Love You! It’s out November 5 2019.

[Make Caows and Shapcho - MeganAnn]

[Pitsilised Koekirjad Cushion Sampler Poncho - Maeve]

[Lacy 2047 - michaela112358]

I use algorithms called neural networks to write humor. What’s fun about neural networks is they learn by example - give them a bunch of some sort of data, and they’ll try to figure out rules that let them imitate it. They power corporate finances, recognize faces, translate text, and more. I, however, like to give them silly datasets. I’ve trained neural networks to generate new paint colors, new Halloween costumes, and new candy heart messages. When the problem is tough, the results are mixed (there was that one candy heart that just said HOLE).

One of the toughest problems I’ve ever tried? Knitting patterns.

I knew almost nothing about knitting when @[email protected] sent me the suggestion one day. She sent me to the Ravelry knitting site, and to its adults-only, often-indecorous LSG forum, who as you will see are amazing people. (When asked how I should describe them, one wrote “don’t forget the glitter and swearing!”)

And so, we embarked upon Operation Hilarious Knitting Disaster.

The knitters helped me crowdsource a dataset of 500 knitting patterns, ranging from hats to squids to unmentionables. JC Briar exported another 4728 patterns from the site stitch-maps.com. 

I gave the knitting patterns to a couple of neural networks that I collectively named “SkyKnit”. Then, not knowing if they had produced anything remotely knittable, I started posting the patterns. Here’s an early example.

MrsNoddyNoddy wrote, “it’s difficult to explain why 6395, 71, 70, 77 is so asthma-inducingly funny.” (It seems that a 6000-plus stitch count is, as GloriaHanlon put it, “optimism”). 

As training progressed, and as I tried some higher-performance models, SkyKnit improved. Here’s a later example.

Even at its best, SkyKnit had problems. It would sometimes repeat rows, or leave them out entirely. It could count rows fairly reliably up to about 22, but after that would start haphazardly guessing random largish numbers. SkyKnit also had trouble counting stitches, and would confidently declare at the end of certain lines that it contained 12 stitches when it was nothing of the sort.

But the knitters began knitting them. This possibly marks one of the few times in history when a computer generated code to be executed by humans.

[Mystery lace - datasock]

[Reverss Shawl - citikas]

[Frost - Odonata]

The knitters didn’t follow SkyKnit’s directions exactly, as it turns out. For most of its patterns, doing them exactly as written would result in the pattern immediately unraveling (due to many dropped stitches), or turning into long whiplike tentacles (due to lots of leftover stitches). Or, to make the row counts match up with one another, they would have had to keep repeating the pattern until they’d reached a multiple of each row count - sometimes this was possible after a few repeats, while other times they would have had to make the pattern tens of thousands of stitches long. And other times, missing rows made the directions just plain impossible. 

So, the knitters just started fixing SkyKnit’s patterns.

Knitters are very good at debugging patterns, as it turns out. Not only are there a lot of knitters who are coders, but debugging is such a regular part of knitting that the complicated math becomes second nature. Notation is not always consistent, some patterns need to be adjusted for size, and some simply have mistakes. The knitters were used to taking these problems in stride. When working with one of SkyKnit’s patterns, GloriaHanlon wrote, “I’m trying not to fudge too much, basically working on the principle that the pattern was written by an elderly relative who doesn’t speak much English.”

Each pattern required a different debugging approach, and sometimes knitters would each produce their own very different-looking versions. Here are three versions of “Paw Not Pointed 2 Stitch 2″.

[Top - ActualJellyfish; Middle - LadyAurian; Bottom (sock version) - ShoelessJane]

Once, knitter MeganAnn came across a stitch that didn’t even exist (something SkyKnit called ’pbk’). So she had to improvise. “I googled it and went with the first definition I got, which was ‘place bead and knit’.” The resulting pattern is “Ribbed Rib Rib” below (note bead).

[Ribbed Rib Rib - MeganAnn]

Even debugged, the patterns were weird. Like, really, really nonhumanly weird.

“I love how organic it comes out,“ wrote Vastra. SylviaTX agreed, loving “the organic seeming randomness. Like bubbles on water or something,” 

SkyKnit’s patterns were also a pain. Michaela112358 called Row 15 of Mystery Lace (above) “a bit of a head melter”, commenting that it “lacked the rhythm that you tend to get with a normal pattern”. Maeve_ish wrote that Shetland Bird Pat “made my brain hurt so I went to bed.” ShoelessJane asked, “Okay, now who here has read Snow Crash?”

[Winder Socks (2 versions) - TotesMyName]

“I was laughing a few days ago because I was trying to math a Skyknit pattern and my brain…froze. Like, no longer could number at all. I stared blankly at my scribbles and at the screen wondering what had happened til somehow I rebooted. Yup, Skyknit crashed my brain.” - Rayn63

[Paw chain 2 - HMSChicago]

On the pattern SkyKnit called “Cherry and Acorns Twisted To”:

“Couple notes on the knitting experience, which while funny wasn’t terribly pleasurable: Because there’s no rhythm or symmetry to the pattern, I felt I was white-knuckling it through each line, really having to concentrate. There are also some stitch combinations that aren’t very comfortable to execute physically, YO, SSK in particular.

That said, I’m nearly tempted to add a bit of random AI lace to a project, perhaps as cuffs on a sweater or a short-row lace panel in part of a scarf, like Sylvia McFadden does in many of her shawl designs. As another person in the thread said, it would add a touch of spider-on-LSD.” -SarahScully

[cherry and acorns twisted to - Sarah Scully]

BridgetJ’s comments on “Butnet Scarf”:

“Four repeats in to this oddball, daintily alien-looking 8-row lace pattern, and I have, improbably, begun to internalize it and get in to a rhythm like every other lace pattern.

I still have a lingering suspicion that I’m knitting a pattern that could someday communicate to an AI that I want to play a game of Global Thermonuclear War, but I suppose at least I’ll have a scarf at the end of it?” -BridgetJ

[butnet scarf - BridgetJ]

There was also this beauty of a pattern, that SkyKnit called “Tiny Baby Whale Soto”. GloriaHanlon managed somehow to knit it and described it as “a bona fide eldritch horror. Think Slenderman meets Cthulu and you wouldn’t be far wrong.”

[Tiny Baby Whale Soto - GloriaHanlon]

Other than being a bit afraid of Tiny Baby Whale Soto, the knitters seem happy to do the bidding of SkyKnit, brain melts and all.

“I cast on for a lovely MKAL with a designer I totally trust and became immediately suspicious because the pattern made sense. All rows increase in an orderly manner. There are no “huh?” moments. There are no maths at all…it has all been done for me. I thought I would be happy, yo. Instead, I am kind of missing the brain scrambling and I keep looking for pigs and tentacles. Go figure.” - Rayn63

Check out the rest of the SkyKnit-generated patterns, and the glorious rainbow of weird test-knits at SkyKnit: The Collection and InfiKnit. 

There’s also a great article in the Atlantic that talks a bit more about the debugging. 

If you feel so inspired (and don’t mind the kind-hearted yet vigorous swearing), join the conversation on the LSG Ravelry SkyKnit thread - many of SkyKnit’s creations have not yet been test-knit at all, and others transform with every new knitter’s interpretation. Compare notes, commiserate, and do SkyKnit’s inscrutable bidding!

Heck yeah there is bonus material this week. Have some neural net-generated knitting & crochet titles. Some of them are mixed with metal band names for added creepiness. Enter your email here to get more like these: