Are AI Detectors Accurate? What Writers Should Know Before Trusting a Score
AI detectors produce a probability, not proof. Here is what OpenAI, Turnitin, the FTC and peer-reviewed research say about their accuracy and false positives and what to do if your writing gets flagged.
No. AI detectors are not accurate enough to prove who wrote a piece of text, and the companies that build them say so in their own documentation. A detector score is a statistical guess about how predictable your writing looks, not evidence that a person used ChatGPT, Claude or Gemini. Human writing gets flagged as AI, lightly edited AI writing slips through, and results change from one tool to the next.
That matters if you write for clients, publish under your own name or edit other people's work. This month the argument has spilled into the mainstream: CNN ran a feature on the "anti-AI writing police" on October 21, 2026, and Turnitin says it updated its English detection model in October 2026. Below is what the primary evidence actually shows, why real writing gets flagged, and what to do if a client or editor questions your work because of a score.
The Short Answer
- Detectors estimate, they do not verify. They score how closely text matches patterns typical of language models. There is no database lookup that confirms a draft came from an AI tool.
- False positives are documented. OpenAI's own classifier labeled human writing as AI-written 9% of the time before OpenAI withdrew it for low accuracy.
- Some writers are flagged far more often. A Stanford-led study found seven popular detectors misclassified non-native English essays as AI-generated at an average rate of 61%.
- Vendors warn against using scores alone. Turnitin says its AI percentage "should not be used as the sole basis for action."
- Accuracy marketing has been challenged. The FTC finalized an order in 2025 against a company that advertised its AI content detector as 98% accurate.
How AI Detectors Work
Most text detectors are classifiers. They are trained on large samples of human-written and machine-written text and learn which statistical features separate the two. A common signal is predictability: language models tend to choose likely next words, so text where every word is the "expected" one looks more machine-like to a detector. Variation in sentence length and structure, sometimes called burstiness, is another signal.
Turnitin describes its approach plainly in its AI writing detection FAQ: a model assigns a score to each sentence, sentences above a threshold are labeled AI, and the document-level percentage is the share of sentences above that threshold. It does not display scores below 20%. Other tools differ in the details, but the output is the same kind of thing everywhere: a probability dressed up as a number.
That is the core problem. A probability can be useful when you are sorting thousands of documents. It is a weak basis for accusing one writer about one article.
What the Evidence Says
OpenAI withdrew its own detector
In January 2023, OpenAI released a classifier to flag AI-written text. On its challenge set it correctly identified only 26% of AI-written text as "likely AI-written," and it wrongly labeled human-written text as AI-written 9% of the time. OpenAI warned that it was very unreliable on texts under 1,000 characters and should not be used as a primary decision-making tool. On July 20, 2023, OpenAI removed it, citing its low rate of accuracy. The company behind ChatGPT could not build a detector it was willing to keep online.
Non-native English writers are flagged far more often
Researchers led by Weixin Liang and James Zou at Stanford tested seven widely used detectors on 91 TOEFL essays written by non-native English speakers and 88 essays by US eighth graders. The detectors were near-perfect on the US essays but misclassified the TOEFL essays as AI-generated at an average false positive rate of 61.22%. Eighty-nine of the 91 TOEFL essays were flagged by at least one detector. The likely reason: simpler, more predictable vocabulary looks "AI-like" to tools that reward unpredictability. The study, published in the journal Patterns in 2023, cautions against using detectors in evaluative or educational settings.
Paraphrasing defeats them, and watermarks can be spoofed
A University of Maryland team (Sadasivan and colleagues) stress-tested several kinds of detectors, including watermark-based ones. Running AI text through a paraphrasing model, repeatedly if needed, significantly reduced detection rates with only slight loss of quality in many cases. They also showed that watermarks can be spoofed so that human writing is misclassified as AI output. In other words, the people most motivated to hide AI use are the easiest to miss, while honest writers carry the false-positive risk.
The FTC acted on an inflated accuracy claim
In April 2025, the Federal Trade Commission alleged that Workado, LLC advertised its AI Content Detector as "98 percent" accurate, while independent testing put its accuracy on general-purpose content at 53%. The FTC also said the model had been trained or fine-tuned to classify academic content, despite marketing aimed at everyday users. The director of the FTC's Bureau of Consumer Protection said the product "did no better than a coin toss." The order, made final in August 2025, bars the company from making accuracy claims without competent and reliable evidence. If you see a detector promising near-perfect accuracy, ask what it was tested on.
Even the biggest vendor says not to rely on the score alone
Turnitin, the detector many US schools and colleges use, states that it "does not make a determination of misconduct" and that its AI percentage "should not be used as the sole basis for action or a definitive grading measure." Turnitin updated its English model in October 2026 to improve accuracy where text switches between human and likely AI-generated writing, which is a reminder that these systems keep changing, and that last year's score may not match this year's.
Why Real Writing Gets Flagged
From the evidence above and the vendors' own stated limitations, a few kinds of human writing are at higher risk of a false flag:
- Short texts. Product descriptions, social captions and email copy give a detector little to work with. OpenAI's classifier was "very unreliable" under 1,000 characters.
- Predictable formats. How-to steps, listicles, definitions, legal boilerplate and SEO templates follow conventions, and conventional text looks statistically "likely."
- Plain, simple English. Writers who use a smaller vocabulary, on purpose or because English is a second language, were flagged far more in the Stanford study.
- Heavily edited copy. Grammar and style tools smooth out some of the irregularities detectors associate with human writing.
- Mixed drafts. A piece where AI helped with an outline and a person wrote the rest can produce patchy sentence-level results that are hard to interpret.
None of these mean the writing is low quality. Clear, conventional prose is often exactly what a client asked for.
What a Detector Score Can and Cannot Tell You
| A score can | A score cannot |
|---|---|
| Suggest which passages to read more closely | Prove that a specific person used an AI tool |
| Help triage a large batch of submissions | Tell you how much AI was used, or for what |
| Prompt a conversation about process | Replace drafts, notes and version history as evidence |
| Flag text that may need a closer fact-check | Tell you whether the content is accurate or useful |
If a Client or Editor Flags Your Work
Stay factual and make it easy for them to see how the piece was made:
- Ask which tool and which passages. Request the score, the tool name and the highlighted sentences. Different detectors often disagree, and a bare percentage is hard to respond to.
- Share your process evidence. Google Docs and Microsoft Word both keep version history that shows a draft growing over time. Outlines, research notes, interview recordings and earlier drafts are stronger evidence than any score.
- Point to the vendors' own guidance. OpenAI, Turnitin and the research above all say a detector result is not proof on its own. Linking to those pages keeps the conversation calm.
- Offer a walkthrough. A short call where you explain your sources and choices usually settles it faster than a long email thread.
- Agree an AI policy up front next time. Spell out in your contract or brief whether AI tools may be used for research, outlining or editing, and how that will be disclosed.
Avoid "fixing" a flagged piece by running it through a paraphrasing or humanizer tool. It proves nothing, it can introduce errors, and it often makes the writing worse. If you want warmer, more specific prose, edit it yourself; our guide to humanizing AI-assisted content covers the techniques that actually improve a draft.
If You Are the One Checking
Marketing teams and editors who review freelance or AI-assisted work get more from a process than from a percentage:
- Use scores as triage, never as a verdict. A high score means "read this closely," not "reject this writer."
- Check the things that can actually hurt you. Wrong facts, invented quotes and made-up statistics are the real risk in AI-assisted copy. A fact-check pass catches them; a detector does not.
- Judge the page the way Google does. Google's guidance focuses on whether content is helpful and reliable, not on how it was produced. Our AI content and SEO guide explains what that means in practice.
- Write your disclosure rules down. If you publish reviews or endorsements, US disclosure rules apply whoever wrote the draft; see our plain-English summary of the FTC rules for AI-written reviews and affiliate content.
What About Watermarks?
Watermarking is the more promising route because it works at the source. Google's SynthID adjusts the probability scores of word choices as text is generated, leaving a pattern that SynthID can later detect, and Google says it now watermarks text generated by the Gemini app and web experience. The limits are practical: a watermark only identifies text from the system that added it, text from other tools carries no mark, and the Maryland research shows paraphrasing can weaken watermarks too. They may help platforms label content in future, but they will not settle whether a freelancer wrote a particular blog post.
The Bottom Line
Treat an AI detector like a smoke alarm that goes off when you make toast: worth noticing, not worth calling the fire brigade over by itself. If you write, keep your drafts and version history so your process speaks for you. If you edit or hire writers, judge accuracy, originality and usefulness directly, and talk to the person before you trust a number. For the ownership side of AI-assisted work, see what the U.S. Copyright Office says about copyrighting AI-assisted writing.
Sources (checked October 28, 2026): OpenAI, New AI classifier for indicating AI-written text (with July 20, 2023 update); Liang et al., GPT detectors are biased against non-native English writers (Patterns, 2023); Sadasivan et al., Can AI-Generated Text be Reliably Detected?; Federal Trade Commission, FTC Order Requires Workado to Back Up Artificial Intelligence Detection Claims and FTC Approves Final Order against Workado, LLC; Turnitin, AI writing detection capabilities FAQ; Google DeepMind, SynthID; CNN, "The anti-AI writing police are everywhere. What are they looking for?" (October 21, 2026).
Frequently asked questions
Can an AI detector prove that someone used ChatGPT?
No. Detectors estimate how closely text matches patterns typical of language models; they do not confirm where a draft came from. OpenAI withdrew its own classifier in 2023 for low accuracy, and Turnitin says its AI percentage should not be used as the sole basis for action.
How often do AI detectors flag human writing?
It depends on the tool and the text. OpenAI's classifier labeled human-written text as AI-written 9% of the time on its test set, and a Stanford-led study found seven detectors misclassified non-native English essays as AI-generated at an average rate of about 61%. Short and formulaic texts are especially prone to false flags.
Which AI detector is the most accurate?
There is no independent, standard benchmark that settles this, and accuracy figures are usually self-reported by vendors. In 2025 the FTC finalized an order against a company that advertised its detector as 98% accurate when independent testing showed 53% on general-purpose content. Treat any accuracy claim as something to verify, not a guarantee.
What should I do if a client says my writing is AI-generated?
Ask which tool flagged it and which passages, then share process evidence such as Google Docs or Word version history, outlines and research notes. Point to the vendors' own guidance that scores are not proof, offer a short walkthrough call, and agree an AI-use policy in writing for future projects.