How accurate were readers?
Share of judgements that were correct. Guessing is 50%, marked by the dark line. Bars run from guessing to the result, so anything left of the line is worse than a coin flip.
~ means I derived the figure from the paper's own breakdown (for example the mean of the AI-caught and student-cleared rates in a 50:50 set). Group sizes range from 3 to 295 readers; the smallest panels (Kurt, Revell, Yeadon) are 3–7 people.
Catching AI versus accusing students
Each dot is a group of readers. Up is more AI texts caught; right is more real student texts wrongly called AI. The line is pure guessing.
Frequent LLM users sit alone at the top left. Most teacher and marker groups catch fewer AI texts than they wrongly accuse students of, or only a little more. Non-expert readers in Russell et al. were at the guessing line.
Confidence ran ahead of accuracy
Stated confidence against how often the reader was actually right.
Fleckenstein values are for the AI-written texts only. Kurt et al. had three teachers who had taught the course, with 60 summaries.
Feedback and instructions helped a little
Accuracy without and with feedback or guidance.
Berger et al. compared two groups of non-humanities university teachers (a preprint). Teachers rarely get feedback on who really wrote a text.
Why this may not match what you see when marking
My reading of the evidence, not established findings.
You judge your own students
These studies are cold judgements. Only Kurt et al. let teachers use what they knew about the course and students, and that was three teachers. A baseline of a student's earlier writing has not been tested with teachers. One algorithm that compares against it (Oliveira et al. 2025) is promising.
Default output is easier than prompted output
Fleckenstein's AI essays were prompted to be weak or medium. When ChatGPT was asked to make student-like grammar mistakes, instructors scored 54% (Saarna 2024). Frequent LLM users (Russell et al.) know the default vocabulary and structure.
You hear about the ones you catch
In the exam study, markers flagged 4 of 63 AI scripts and had no way to learn about the other 59. Without feedback, confidence stays high at low accuracy.
Catching and clearing differ
Experienced teachers cleared 73% of real student essays but caught 38% of AI ones. "It's usually obvious" fits that pattern: the AI you spot is obvious, and the rest passes as student work.
Exposure matters more than teaching years
Frequent LLM users reached 93% while non-experts hit chance. Linguistics experts (Casal & Kessler) got 38.9%, and self-reported ChatGPT experience did not help in Waltzer's studies.
The target is moving
Persona prompts, style imitation and humanizers weaken the cues. I found no human-detection study on the newest models.
Gaps
- No experiment gives teachers a student's earlier writing and tests whether that helps.
- No experiment with primary-school teachers, and none with teachers who know the pupils.
- No pooled estimate for teachers; the one review aimed at this question (Han 2025) I could only read in abstract.
- No larger replication of the Russell et al. result, and no controlled study of teacher training with feedback.
- No data on how many proven misconduct cases came from human suspicion rather than software.
All studies
Evidence: Full text read, Abstract or page read, or Summary only (seen in a search or news summary; check before quoting).
| Study | Readers | Material and task | Result | Evidence |
|---|