AI Detecting AI
I keep seeing people use free AI detection websites as evidence in AI witch hunts.
These websites are often scams. They use AI to detect AI. They rarely disclose their methods. They are typically only reliable on obvious material and are no better than a coin flip on content that is more challenging (which is usually when AI detection is needed most).
You'll even note that these websites do not give you a strict yes or no. They give you a percentage of confidence. Yes, it is true that human forensic analysis often has a confidence rating instead of an absolute certainty. But in this case, this gives these websites cover for when they create false positives. They don't tell you their actual success rate of detecting AI, so if you get a bad result, they can just say the tool was only 80% confident.
Even laboratory-grade forensic tools struggle to get above 70-80% accuracy, and staying ahead of AI improvements to maintain that accuracy is a constant struggle.
The best forensic analysis of AI media always has a human component. They use detection software in combination with human experts. There is no reliable, fully automated process. Especially now that people are combining AI with traditional techniques. They might generate an AI image and then use Photoshop to add noise, resize it, change the color and contrast. 5 minutes of work can remove the classic signs of AI that many of these detectors look for.
If you are suspicious of something and you want to use one of these tools as an initial indicator that you should seek out more evidence, I feel that is their only real use case. Like, it should be step one of a multistep process of investigation and it should not be your only evidence. In fact, you should not include it as evidence when making an accusation.
Look for creation dates. Look for provenance. Look at the artist's other work. Look for process videos.
But if you use the result of an AI detector to confidently say something is AI, you could very well hurt an artist using a dubious tool that consistently spits out false positives.
Or you could abstain from these high-effort, individual witch hunts altogether and focus your AI advocacy on the larger issues and the people in charge of these companies.
I'm feeling a bit psychic now.
I'm guessing Google has one of the most advanced AI detection tools in the world. But this process cannot be 100% automated or else it will continue to hurt artists.
Google has long refused to hire actual humans to assist their automated processes. If a small YouTube channel needs help, they can rarely reach an actual human being. You need millions of subscribers for the privilege of not dealing with some sort of bot.
And they are smart enough to realize that AI slop hurts their platform, but they throw AI at the AI slop, and continue the tradition of not adding human labor to the mix. It is bonkers that they don't even inform people they have been flagged as AI. You just have to figure it out through deductive reasoning.


















