One of the biggest problems with the idea of ethics around generative AI, particularly with artwork, is how it was trained. (1) Artists of assert â publicly and in lawsuits â that OpenAI and other firms keep violating copyright to steal their work to use for training data. The firms typically claim, while admitting they used to do that, they behaving more ethically now and respecting copyright.
I believe those claims are false, and that the efforts of visual artists to combat this are showing results in the generated output. Added to the shallow and repetitive nature of genAI images and the increasing public dislike of genAI images⌠well, it's not looking good for them. (You may cheer or boo as you wish.)
Nightshade, and later, Glaze were developed in order to "poison" the training data. The alterations those programs make are subtle to the human eye, but are apparently quite noticeable to AI training data. It's meant to make the training data unusable for the models without appreciably disturbing the human experience.
But the alterations are visible, and have a very specific "look" to them. At high intensities, the process is pretty noticeable:
You can still see the same patterns, although they're much less obvious, at the low intensity setting as well. As above, the treated image is on the right. And just like above, it combines small dots and natural variation of texture and color into small blotches of patterns and lines.
While subtle, those are not compression artifacts; these are the same resolution being displayed side by side and saved as one image. (I tried making them into video or an animated GIF; the video encoding algorithm made it harder to see, not easier.)
But does it actually work?
Well, take a look at these images from OpenAI's "Images" that made me go "huh."
Because the way those "freckles" are blotches, the shadows in the woman's cheek, and the pattern of the beard, seem awfully familiar.
I'll admit, I've only generated a handful of images using their tool, and this does not show up in every image. It did, however, show up in the images generated with this style.
That's exactly what you'd expect to see from images generated from poisoned training data.
This is Super Not Good for the big AI companies, even if I'm wrong about what caused these artifacts in the images.
If I'm right, then not only does this demonstrate that Nightshade, Glaze, and similar technologies are having an impact, but it also implies that the big AI companies are still doing exactly the same copyright violations as before. It also implies that these companies aren't paying any attention to what they pull into their training data, which not only leads to poisoned images, but also makes them far more likely to suffer from model collapse. That's a degree of lassez-faire that crosses the line into irresponsibility.
But it's even worse if I'm wrong.
Because if what I'm seeing here is not the effect of Nightshade and Glaze⌠then that just means their product has significantly degraded than the already questionable quality that it had previously.
And that is perhaps worse.
Post-script: I was having trouble finding an authentic image on Pixabay that fit the theme of this post (although I eventually did, the featured image is by amurca from Pixabay).
So just to test things one more time, I asked OpenAI's "Images" to create an image showing a robot copying off a human artist.
First, I did see the effect again, here with the same kinds of linear and blocky patterns in the shirt where there should be shadows and texture:
Secondly, it added this as a poster in the image:
I don't know whether to be more amused or pissed off.
(1) As with all of my AI-related posts, I'm focusing on a specific aspect rather than trying to cover all the pros and cons of the technology. It is fair to say that I am an AI/ML realist about the technology, but am generally negative about the big companies and their late-stage capitalist profit extraction tactics.
germansingergirl
romaincrisis
origamiaround
turnnoffyourmind
jbbartram-illu