Thereās a story I tell in my book because itās a great illustration of how AI gets the wrong idea about what problem weāre asking it to solve:
Researchers at the University of Tuebingen trained a neural net to recognize images, and then had it point out which parts of the images were the most important for its decision. When they asked it to highlight the most important pixels for the category ātenchā (a kind of fish), this is what it highlighted:
Human fingers against a green background!
Why was it looking for human fingers when it was supposed to be looking for a fish? It turns out that most of the tench pictures the neural net had seen were of people holding the fish as a trophy. It doesnāt have any context for what a tench actually is, so it assumes the fingers are part of the fish.
The image-generating neural net in ArtBreeder (called BigGAN) was also trained on the same dataset, called ImageNet, and when you ask it to generate tenches, this is what it does:
The humans are much more distinct than the fish, and Iām fascinated by the highly exaggerated human fingers.
There are other categories in ImageNet that have similar problems. Hereās āmicrophoneā.
Itās figured out about dramatic stage lighting and human forms, but many of its images donāt contain anything that remotely resembles a microphone. In so many of its training pictures the microphone is a tiny part of the image, easy to overlook. There are similar problems with small instruments like āfluteā and āoboeā.
In other cases, there might be evidence of pictures being mislabeled. In these generated images of āfootball helmetā, some of them are clearly of people NOT wearing helmets, and a few even look suspiciously like baseball helmets.
ImageNet is a really messy dataset. It has a category for agama, but none for giraffe. Rather than horse as a category, it has sorrel (a specific color of horse). āBicycle built for twoā is a category, but not skateboard.
A huge reason for ImageNetās messiness is that it was automatically scraped from images on the internet. The images were supposed to have been filtered by the crowdsourced workers who labeled them, but plenty of weirdness slipped through. And horribleness - many images and labels that definitely shouldnāt have appeared in a general-purpose research dataset, and images that looked like they had gotten there without the consent of the people pictured. After several years of widespread use by the AI community, the ImageNet team has reportedly been removing some of that content. Other problematic datasets - like those scraped from online images without permission, or from surveillance footage - have been removed recently. (Others, like Clearview AIās, are still in use.)
This week Vinay Prabhu and Abeba Birhane pointed out major problems with another dataset, 80 Million Tiny Images, which scraped images and automatically assigned tags to them with the help of another neural net trained on internet text. The internet text, you may be shocked to hear, had some pretty offensive stuff in it. MIT CSAIL removed that dataset permanently rather than manually filter all 80 million images.
This is not just a problem with bad data, but with a system where major research groups can release datasets with such huge issues with offensive language and lack of consent. As tech ethicist Shannon Vallor put it, āFor any institution that does machine learning today, āwe didnāt knowā isnāt an excuse, itās a confessionā. Like the algorithm that upscaled Obama into a white man, ImageNet is the product of a machine learning community where thereās a huge lack of diversity. (Did you notice that most of the generated humans in this blog post are white? If you didnāt notice, that might be because so much of Western culture treats white as default).
It takes a lot of work to create a better dataset - and to be more aware of which datasets should never be created. But itās work worth doing.
Subscribers get bonus content: a gallery from a few of my favorite BigGAN image categories.
My book on AI is out, and, you can now get it any of these several ways! Amazon - Barnes & Noble - Indiebound - Tattered Cover - Powellāsļ»æ - Boulder Bookstore
