ohhh man. apparently when you copy+paste stuff directly from claude into ao3 it includes code. which anyone can find. people have found it in big, popular fics in multiple fandoms π¬. also heated rivalry; some of which the authors fervently denied using AI. link to thread.
link to their complete findings for those without twitter
Writing this up for data literacy purposes. There's a philosophical conversation to be bad about AI but this post is not that, this is just explaining what the Claude fragment ai checker can say and what it can't.
I didn't write the checker and have no involvement with that.
What the tool linked above can say:
This fic includes content that was directly copied from Claude into the ao3 post box
What it can't say:
How much of the fic is ai
What the ai was used for
Reiterating that it only finds Claude fragments so that means that the checker:
Doesn't know if another ai was used
Doesn't know if Claude was used and the copy from Claude feature was used
Doesn't know if content comes from Claude but was first pasted into another document first
Here's some scenarios where ONLY ONE Claude fragment would be detected:
Person goes into Claude, asks it to generate 20k words of a college au meet cute fic. Person pastes the text directly into ao3. One ping
Person asks Claude to generate text that would make their pre-written text look like a tweet, copies that into the ao3 box. One pingΒ
Person uses Claude as a spellchecker and copies the output into the ao3 box. One pingΒ
Scenario where Claude ai checker would ping more than once:
Person pastes a paragraph into Claude and asks it to rewrite it more smoothly, copies that into ao3, then does that again
Frequency of code fragment does not equal quantity of ai usage.
Here's a scenario where the Claude fragment ai checker would not detect ai even when that person was using Claude:
Person asks Claude to write their entire fic but then copies it into a Google doc before pasting it into ao3
More does not mean more. This is a binary check of βhas fragment or does not have fragment.βΒ
and GOING FORWARD, because this is such a simple and distinct marker, I expect that no future Claude AI fic will include this marker. It is trivial to remove it. So this is not a preventative future checker. What this IS good for is to read flagged fic and hone a personal sense of what an AI fic reads like.
Today I was also turned onto this fic which can help articulate what makes an output feel like AI, if someone is looking for another training tool.
