#gancats

Recently I had my first-ever experiment with training a neural net to generate images, when I trained StyleGAN2 to generate screenshots from The Great British Bakeoff. Although recognizable as the baking show, it was a highly distorted, nightmarish version - it would have helped had the neural net been able to keep track of how many faces humans have.

The problem was that the task was too hard. StyleGAN2 has good success when it only has to do human faces from the front, but when its task is broader, like generating internet cats, it struggles hilariously. So I decided to give it a more restricted cat dataset: just one cat. My cat. In more or less the same pose.

After a couple of data-collection attempts that were sabotaged by Char’s tendency to put her nose in the camera, I managed to get a full two minutes of video of her sitting still, staring intently at something invisible in the corner of the kitchen. Probably ghosts. It doesn’t matter. The point is, I got the video.

I extracted 3,934 frames from the video and used runwayml to train StyleGAN2 on them for 3000 iterations. Training took about 5 hours, and by the end, it was producing this:

Now, they aren’t perfect, but they are much much better than the Bakeoff images. The cat spent the most time looking off to the right, so the neural net learned that pose the best.

But my word, to GET to this point the model had to travel through terrible places. You see, I started with a version of StyleGAN2 that had been pretrained on human faces, which saved a lot of time since the AI already knew how to do edges and textures and hair. But it means that the process of switching gears from generating humans to generating cats yielded a stage that looked like…. this.

This is one of the most horrible things I have ever seen, and I saw CATS (2019) in the theater.

Here’s a video of the full training process.

What is particularly alarming, perhaps, is the PATH it took from human to cat. Rather than transforming human eyes, nose, and mouth into the feline equivalents, it erases all the human features and then forms the cat features out of featureless fur. Here’s a snapshot of the training process in which you might be able to see how the cat/people have four eyeballs, or no eyeballs. (I adjusted the contrast to make these more visible, as these steps got really dark for some reason, as if to shield human eyes from what was happening).

What is great/horrifying about this is that if you have a computer, an internet connection, and a two-minute video of something, you can do this too. Got a cat? A bunch of Easter eggs? A lemon? It’s now seriously easy to generate your own horrible abominations. The simpler and more consistent your subject, the more recognizable your results will be.

Subscribers get bonus content: more amazing pictures that wouldn't fit in this post.

How to make an neural network draw your pets

Has science gone too far? Inspired by Janelle Shane (aka @lewisandquark​), I’ve taught a neural network how to draw my cat Mulder. The results are ... delightfully off-kilter.

Here’s what the process started with:

Here’s the ALARMING midpoint:

And here, after 3,000 iterations, is the rather impressive result!

The process is actually easy enough a non-computer-geek like me could figure it out — there’s no coding involved, nor do you need a ton of computing power. All you need is a video of your pet (or whatever you want to see AI try to recreate), and two pieces of free software: VLC Media Player and RunwayML.

DETAILED TUTORIAL BELOW THE CUT... If you use it, please reblog this post with your results -- I’d love to see ‘em! Note: I’m a Windows user; the process for Mac might be slightly different at some steps.

A new paper from NVIDIA recently made waves with its photorealistic human portraits. Called StyleGAN, the algorithm had a new training dataset pulled from Flickr, with a wider range of ages and skin tones than in other portrait datasets. (If your pictures are on Flickr with the right license, your picture might have been used to train StyleGAN). Thanks to that big dataset, a new method of generating images, and a staggering amount of computer power, its human faces are indeed impressive. But, to prove that their method doesn’t just work for human faces, they also generated bedrooms, and cars... and cats.

The cats are so much fun.

In many ways, generating cats is more of a challenge than generating human faces, since cats can be in so many different poses. It has an easier time generating texture than figuring out where all the legs and tails go.

I’ve noticed that for some reason, whenever it generates kittens, one is normal and the rest are haunted. There must be something difficult about pictures containing multiple subjects.

Its humans are even worse, proving that the difference between it and the version that was impressively good at generating human faces was all in the training data.

Speaking of training data, StyleGAN’s training data was something called LSUN Cat. Evidently, LSUN Cat’s images were sourced from the internet, because when it generated some cats, it added meme text to them, assuming that white blocky lettering is just part of what “cat” is.

It also attempted to generate cats with the Shutterstock watermark, although actually spelling “Shutterstock” proved tricky for it.

Another delightful thing about its internet-derived dataset: quite a lot of the cats look an awful lot like Grumpy Cat. She’s only one cat, but she had a profound impact on what StyleGAN thinks cats look like.

There are far more amazing cats than would fit in this blog post. In this bonus material, I’ve collected a few more of my favorites, including several with meme text, perfect for your internet communication needs. Support AI Weirdness and get bonus content!