let’s raise a little hell

Not at all! There’s a lot of nuance in this issue and it’s very much ever-evolving. I take a lot of things into account when making those determinations, and it’s taken me over 2 years of grading student work and researching AI technology to feel confident in my evaluations.

The biggest thing I look at is references. I primarily GSA for upper-level psychology classes where each student is expected to write a 10 page paper with at least 10 empirical, peer-reviewed sources. The very first thing I do when grading these papers is go through every single source and make sure they exist (AI often hallucinates references that look real but are not) and they are at least tangentially related to the topic at hand. I would say around 90% of the students in my department who have been caught using AI were caught because of hallucinated sources.

Another big thing is vocabulary. AI tends to really hook onto certain words and phrases: just to name a few, I see “foster(ed)”, “enhance”, “highlight”, “crucial”, “leverage”, and “testament” a lot in AI writing. This isn’t to say that using these words will automatically get you flagged for AI use; it’s when they’re overused and clearly redundant. Still, though, we can’t really prove AI use in these cases unless the student confesses.

The last thing I really look at is argument depth. My students know that I like to see a lot of critical thinking in their arguments; I want to see comparing and contrasting, theoretical frameworks, and (most importantly) a clear conclusion where they make a case for their opinion. AI has a tendency to make very surface-level arguments when prompted to discuss controversial topics, and the systems often come to a very safe conclusion that typically just gives “both sides have their strengths and weaknesses” vibes.

All of this said, we have very limited ways to actually prove AI use and bring about disciplinary action. The main way we are able to do this is through hallucinated sources, since that is an objective telltale sign that cannot be mistaken for a student’s personal writing style. We can also look at the HTML when assignments are submitted directly into Canvas, but this doesn’t really work for document submissions. So honestly, even if we are 99% sure AI was used based on our assessments, most of the time there’s nothing we can do beyond asking the student if they used AI (which, of course, most students will say no, and we have to just accept that).