AI Detectors: How They Work & Which Ones Actually Work

AI Detector: How It Works and Which Ones Actually Catch AI Text

AI detectors use machine learning models to analyze writing patterns — primarily perplexity (word predictability) and burstiness (sentence variation) — and produce a probability score that text was AI-generated. Accuracy varies wildly: independent studies show detection rates from 68% to 99%, with false positive rates ranging from 0.05% to 68.6%. No detector is reliable enough to serve as sole evidence in academic or high-stakes decisions.

Quick Facts

How AI Detectors Actually Work: The Core Mechanisms

AI detectors do not verify authorship. They estimate the probability that a piece of text resembles writing produced by a large language model. They work by analyzing statistical patterns, not by matching text against a database or checking for watermarks (with some newer exceptions).

Read moreIs AI Taking Your Job? Data Says No (Mostly)

The technology rests on two foundational metrics: perplexity and burstiness.

Perplexity: How Predictable Are the Words?

Perplexity measures how surprising each word is, given the words that came before it. AI models are trained to select the most statistically likely next word. This produces text with low perplexity — it flows predictably, one word logically following the next.

Read moreAI and Office Jobs: What Research Actually Shows

Human writers are messier. They choose unexpected words, change direction mid-sentence, and occasionally use phrases that don’t follow the most probable path. This produces higher perplexity.

Example: “The hiker went out into the…”

• Low perplexity (AI-like): “woods.”

• High perplexity (human-like): “drainage ditch behind a strip mall.”

An AI picks “woods” every time. A human might pick something stranger.

Burstiness: How Much Do Sentences Vary?

Burstiness measures variation in sentence length and complexity. Human writing naturally mixes short and long sentences, simple and complex structures. AI-generated text tends to maintain a steady, even rhythm.

Low burstiness — uniform sentence structure — strongly indicates AI involvement to a detector. High burstiness — wide variation — suggests human authorship.

The Classifier Model

These metrics are fed into a machine learning model trained on large datasets of both human-written and AI-generated text. The classifier learns to distinguish between the two based on these and other linguistic features. It then produces a probability score — typically a percentage — indicating how likely it is that the text was AI-generated.

Critical point: The detector does not read for meaning. It does not fact-check. It does not verify authorship. It provides a statistical guess with confidence attached.

AI Detectors vs. Plagiarism Checkers: What’s the Difference?

AI detectors and plagiarism checkers are fundamentally different tools that are often confused.

Why this matters: Original human writing can be flagged as AI-generated because it happens to follow predictable patterns. Conversely, AI-generated text that has been heavily edited or paraphrased may evade detection entirely. Neither tool proves what the other is designed to prove.

Comparison Table: AI Detector Accuracy Across Major Tools

Which AI Detectors Actually Work? Independent Test Results

The University of Florida Study (May 2026)

The most rigorous independent evaluation to date was presented at the 2026 IEEE Symposium on Security and Privacy. Researchers at the University of Florida tested five commercial AI detectors using approximately 6,000 research papers submitted to top-tier security conferences before ChatGPT existed. They then had LLMs create clones of those papers and ran both sets through the detectors.

Results:

• False positive rates: 0.05% to 68.6%

• False negative rates: 0.3% to 99.6%

• After a “lexical complexity attack” (rewriting with more complex vocabulary), two of the five detectors were rendered largely useless

Patrick Traynor, the professor who led the study, said: “We really can’t use them to adjudicate these decisions. People’s careers are on the line here”.

The International Journal for Educational Integrity Study (June 2026)

This peer-reviewed study compared four popular detectors — GPTZero, Pangram, Copyleaks, and Turnitin — on four types of academic papers: fully human-written, fully AI-written, hybrid (human with AI-inserted passages), and humanized AI text.

Results:

• Pangram consistently performed better than the other tools, achieving high accuracy across all text types

• The other tools significantly underestimated GenAI content, particularly for texts generated with the most advanced models

• All tools correctly identified fully human texts

• False positives were rare across all tools, suggesting improvement compared to earlier studies

Key conclusion: “Detection tools can provide useful initial flags, [but] they should not be used as sole evidence in high-stakes decision-making but should be implemented in a broader evaluation strategy”.

The Epoch AI Benchmark (July 2026)

Epoch AI tested three major detectors — Pangram, GPTZero, and Originality.ai — against frontier LLMs including Claude Opus 4.8, GPT-5.5, and Gemini 3.1 Pro. They also tested style imitation, where LLMs were given five real writings from an author and asked to mimic their style.

Results:

• Pangram missed 48% of Gemini-generated academic text

• Originality.ai missed 39% of GPT-5.5-generated academic content

• GPTZero missed 11% of style-imitated texts; Originality.ai missed 18%

The Chicago Booth Benchmark (2025-2026)

University of Chicago researchers Brian Jabarian and Alex Imas tested leading detectors against 1,992 passages across six genres and four frontier models. Pangram was the only detector to meet a stringent false-positive cap (FPR ≤ 0.005) while still catching AI text, with near-zero error rates that held even against “humanizer” tools like StealthGPT.

GPTZero lost much of its detection ability against humanized text (FNR ~50%+). The open-source RoBERTa baseline performed at or near random (AUROC ≈ 0.5) and is unsuitable for high-stakes use.

The False Positive Problem: Who Gets Hurt Most

Non-Native English Speakers

The most consistent finding across independent research is that AI detectors systematically flag non-native English writers at higher rates.

A Stanford University study found that several widely used AI detectors consistently misclassified non-native English writing samples as AI-generated. When seven widely used detectors were tested on 91 essays written for TOEFL (the standard English-proficiency exam) by people who do not speak English as their first language, they wrongly flagged an average of 61% of that genuine human writing as AI-generated.

Why this happens: Many detectors work by measuring how predictable writing is. Varied, surprising phrasing looks human; plain and repetitive phrasing looks machine-made. Someone writing carefully in a second language tends to use a narrower range of words and sentence patterns. The very habit of the diligent multilingual student is the habit the tool reads as a robot.

Neurodivergent Writers

Multiple studies document elevated false-positive rates for neurodivergent students, including those with autism, ADHD, and dyslexia, whose writing may be unusually repetitive, highly structured, or unconventional.

Turnitin, which claims a <1% false positive rate, has faced notable faculty backlash. Vanderbilt University disabled the tool after multiple cases of misclassification involving bilingual and neurodivergent students.

The Scale Problem

When Turnitin launched its detector, it reported a false-positive rate of about 1%. Vanderbilt University did the arithmetic on its own figures: it had run approximately 75,000 papers through Turnitin the previous year, so 1% implied about 750 pieces of real student work wrongly branded as AI.

A number that sounds trivial in a vendor’s brochure becomes hundreds of individuals the moment it meets a real cohort, and each of them is then asked to prove they did their own work.

Universities Are Disabling Detectors

At least a dozen universities, including Northwestern, Georgetown, and NYU, have disabled Turnitin’s AI detection feature outright. Yale’s teaching center now says detection scores can’t be used as evidence in integrity complaints. Johns Hopkins has downgraded AI detection to advisory use only. The University of Waterloo disabled Turnitin’s AI detector after internal testing reportedly flagged entirely human-written work as AI-generated.

The University of the Free State in South Africa announced it would discontinue AI detection tools across all faculties from July 2026, with Turnitin’s AI detection function no longer available to staff or students.

AI Detection in Practice: Real-World Applications

Academic Integrity

AI detectors are most commonly used in educational settings to check student submissions. But the reliability problems are most acute here, where false accusations can have serious consequences for students’ academic records and futures.

Best practice: Detection scores should be one signal among many. They should never be the sole basis for misconduct accusations. Institutions should focus on assessment design that makes AI misuse less relevant, rather than relying on detection technology.

Content and Publishing

Publishers and content agencies use AI detectors to screen freelance submissions and verify originality. Originality.ai is positioned for this market, offering detection plus plagiarism checking in one scan.

Limitation: A 3.8% false positive rate means nearly 1 in 26 human-written articles could be wrongly flagged as AI-generated.

Hiring and Recruitment

Hiring managers use AI detectors to screen cover letters and writing samples. This raises serious ethical concerns: false positives could disqualify qualified candidates, and the bias against non-native English speakers means international applicants face higher scrutiny.

Social Media and Platform Integrity

Trust-and-safety teams at large platforms use AI detectors to identify bot content and AI-generated misinformation. The challenge is scale: at millions of posts per day, even a 1% false positive rate affects tens of thousands of legitimate posts.

The Arms Race: Detection vs. Evasion

Humanizers and Paraphrasers

An entire market has emerged for tools designed to evade AI detection. “Humanizers” like StealthGPT rewrite AI-generated text to increase perplexity and burstiness, making it harder for detectors to flag.

In the Chicago Booth benchmark, GPTZero lost much of its detection ability against humanized text (FNR ~50%+). Pangram held up better, maintaining near-zero error rates.

Lexical Complexity Attacks

The University of Florida study demonstrated that asking an LLM to rewrite its output using more complex vocabulary could render some detectors largely useless. Two of the five detectors tested performed well initially but failed after this simple modification.

Watermarking: A Different Approach

Rather than detecting AI text after the fact, some companies are embedding invisible watermarks directly into generated text.

Anthropic announced in August 2026 that new Claude models would embed invisible watermarks in all generated text, everywhere Claude is offered. The watermark detection mechanism would be available as a Watermark Detection API.

Google has implemented SynthID Text, which embeds watermarks in Gemini-generated text.

Limitation: Watermarking only works if the AI provider cooperates. Open-source models can’t be watermarked, and watermarks can potentially be removed through paraphrasing or editing.

Comparison Table: Fear vs. Reality of AI Detection

What Experts and Researchers Actually Say

On the Limits of Detection

Patrick Traynor, University of Florida: “We really can’t use them to adjudicate these decisions. People’s careers are on the line here”.

Traynor added: “For as many studies as we see claiming that a certain percentage of academic work is AI-generated, we actually don’t have tools to measure any of that”.

On the Bias Problem

The HEPI fairness analysis (August 2026): “The honest reading is not that detection never works but that there is no universal accuracy figure: the false-positive rate swings widely with the tool, the sample and the settings, and it is worst precisely on the writing of students who learned English later”.

On Watermarking

Anthropic’s August 2026 announcement: New Claude models will embed invisible watermarks in all generated text, with a Watermark Detection API available for verification.

On the Arms Race

From the Techlusive analysis: “AI models are improving fast. Newer models don’t sound as repetitive or predictable as before, which makes detection harder than it used to be”.

On Human Judgment

The International Journal for Educational Integrity: Detection tools “should not be used as sole evidence in high-stakes decision-making but should be implemented in a broader evaluation strategy”.

Decision Tree: Should You Trust an AI Detector?

Question 1: What is the consequence of a false positive?

• Low stakes (curiosity, self-check): A detector result is fine as a rough signal.

• High stakes (academic misconduct, employment decision): Go to Question 2.

Question 2: Is the writing from a population known to face detection bias?

• Yes (non-native English speaker, neurodivergent writer): Do not rely on detector results. Seek human review.

• No: Go to Question 3.

Question 3: Is the detector independently validated?

• Yes (tested by peer-reviewed research): Use results as one signal among many.

• No (only vendor claims): Treat results with extreme caution.

Question 4: Has the text been paraphrased or humanized?

• Yes: Detection is significantly less reliable. GPTZero’s FNR on humanized text is ~50%+.

• No: Detection is more reliable but still not definitive.

Question 5: Are you using this as sole evidence?

• Yes: Don’t. Experts universally agree this is inappropriate.

• No: Document your reasoning and keep the detector result as one data point among many.

How Individuals Can Protect Themselves

If You’re a Student

• Keep drafts and version history. Google Docs and Word track revision history automatically.

• Use multiple detectors. If one flags your work, check with others.

• Know your rights. Many universities have policies against using detection scores as sole evidence.

• Write in your natural style. Don’t try to “sound like AI” or “sound human” — just write normally.

If You’re an Educator

• Do not use detection scores as sole evidence. This is the single most important rule.

• Understand the bias problem. Non-native English speakers and neurodivergent students face higher false-positive rates.

• Redesign assessments. Focus on process, drafts, and critical thinking rather than final products alone.

• Advocate for policy. Support institutional policies that limit detector use to advisory signals.

If You’re a Non-Native English Speaker

• Be aware of the bias. Your writing is more likely to be flagged.

• Keep documentation. Save drafts, notes, and research materials.

• Request human review. If flagged, ask for evaluation by a human who understands second-language writing patterns.

If You’re a Content Creator or Freelancer

• Run your work through detectors before submitting. Know what they’ll show.

• Keep records of your process. Notes, outlines, and drafts demonstrate human authorship.

• Consider tools that don’t trigger false positives. Some writing styles naturally produce higher perplexity.

If You’re an Employer or Hiring Manager

• Don’t use AI detectors for screening. The false positive rate is too high and the bias against non-native speakers is documented.

• Focus on interviews and work samples. These are more reliable indicators of ability than a detector score.

Common Questions

How do AI detectors work?

AI detectors use machine learning models trained on both human-written and AI-generated text. They analyze writing patterns — primarily perplexity (word predictability) and burstiness (sentence variation) — and produce a probability score. They do not verify authorship, check against databases, or read for meaning.

Are AI detectors accurate?

Accuracy varies widely. Independent studies show detection rates from 68% to 99%, with false positive rates ranging from 0.05% to 68.6%. The University of Florida study concluded that commercial AI detectors are “poorly suited for deployment in academic or high-stakes contexts.”

Which AI detector is most accurate?

Pangram consistently performs best in independent testing, with near-zero false positives and high detection rates even against humanized text. GPTZero performs well on raw AI text but loses detection ability against humanized or paraphrased content. Turnitin’s accuracy drops significantly with paraphrased text.

What is perplexity in AI detection?

Perplexity measures how predictable a sequence of words is. AI models choose the most statistically likely next word, producing text with low perplexity. Human writing includes more variation in word choice, producing higher perplexity. Low perplexity is a signal to detectors that text may be AI-generated.

What is burstiness in AI detection?

Burstiness measures variation in sentence length and complexity. Human writing naturally mixes short and long sentences, simple and complex structures. AI-generated text tends to maintain a steady, even rhythm. Low burstiness — uniform structure — strongly indicates AI involvement.

Why do AI detectors produce false positives?

False positives occur because detectors analyze statistical patterns, not actual authorship. Writing that happens to follow predictable patterns — common in formal, academic, or structured writing — can be flagged as AI-generated even when written by humans. Non-native English speakers and neurodivergent writers face higher false-positive rates because their writing may be more repetitive or structured.

Are AI detectors biased against non-native English speakers?

Yes. A Stanford study found that several widely used AI detectors consistently misclassified non-native English writing as AI-generated. When tested on TOEFL essays by non-native speakers, detectors wrongly flagged an average of 61% of genuine human writing as AI-generated.

Should universities use AI detectors?

Experts recommend against using detection scores as sole evidence in misconduct cases. Many universities — including Northwestern, Georgetown, NYU, Yale, Johns Hopkins, and the University of Waterloo — have disabled or restricted Turnitin’s AI detection feature due to unreliability and bias concerns.

What is the difference between AI detection and plagiarism checking?

AI detectors analyze writing patterns to estimate whether text resembles AI-generated writing. Plagiarism checkers compare text against existing sources to find matches. They work completely differently and detect different things. Neither proves authorship.

Do AI watermarks work?

Watermarking embeds invisible markers in AI-generated text. Anthropic and Google have implemented watermarking for their models. However, watermarking only works if the AI provider cooperates — open-source models can’t be watermarked, and watermarks can potentially be removed through paraphrasing.

Can AI detectors detect GPT-5 and Claude?

Detection accuracy drops 4 to 7 percentage points on GPT-5 versus GPT-4o for most detectors. Detection of Claude output runs 2 to 5 points lower than GPT. Newer models produce more human-like writing patterns, making detection harder.

What should I do if I’m falsely accused of using AI?

Keep documentation of your writing process — drafts, notes, research materials, and revision history. Request human review of your work. Provide evidence of your writing process. Many institutions have policies against using detection scores as sole evidence.

What is the best way to prove I wrote something myself?

Maintain version history through your writing software. Keep notes, outlines, and research materials. Use multiple detectors to show inconsistent results. Provide context about your writing process. Human review is more reliable than any detector score.

Will AI detectors ever be reliable enough for high-stakes decisions?

Experts are skeptical. The arms race between detection and evasion is ongoing, and newer AI models are increasingly difficult to detect. The fundamental problem — that detectors measure statistical patterns rather than authorship — is unlikely to be solved. Process-based assessment and human judgment are more reliable approaches.

Key Takeaways

• AI detectors measure statistical patterns — perplexity and burstiness — not actual authorship. They produce probability scores, not proof.

• Accuracy varies wildly. Independent studies show detection rates from 68% to 99% and false positive rates from 0.05% to 68.6%.

• The University of Florida study concluded that commercial AI detectors are “poorly suited for deployment in academic or high-stakes contexts.”

• Non-native English speakers and neurodivergent writers face higher false-positive rates. Stanford research found detectors flagged 61% of genuine TOEFL essays as AI-generated.

• Pangram consistently performs best in independent testing, with near-zero false positives even against humanized text. GPTZero performs well on raw AI text but fails against humanized content.

• At least a dozen universities have disabled Turnitin’s AI detector, including Northwestern, Georgetown, NYU, Yale, Johns Hopkins, and the University of Waterloo.

• No detector is reliable enough to serve as sole evidence in academic misconduct or employment decisions. Experts universally recommend using detection scores as one signal among many.

• Watermarking is an alternative approach — Anthropic and Google embed invisible markers in AI-generated text — but it only works for cooperative providers and can be removed.

• The arms race continues. Newer AI models are harder to detect, and humanizer tools can evade most detectors. Detection accuracy drops 4-7 percentage points on GPT-5 versus GPT-4o.

• The best protection is documentation. Keep drafts, notes, and version history. If falsely accused, request human review and provide evidence of your writing process.

Official & Trusted Resources

• University of Florida — “Watching the Detectors: Researchers Probe Efficacy and Danger of AI Detection Tools” (2026 IEEE Symposium on Security and Privacy): The most rigorous independent evaluation of commercial AI detectors, testing five tools on 6,000 research papers. Found false positive rates up to 68.6% and false negative rates up to 99.6%.

• International Journal for Educational Integrity — “Who wrote this? Evaluating the reliability of AI detection tools in higher education” (June 2026): Peer-reviewed comparison of GPTZero, Pangram, Copyleaks, and Turnitin on four types of academic papers. Pangram performed best; all tools correctly identified fully human texts.

• Stanford University — Research on AI Detector Bias Against Non-Native English Speakers: Found that detectors consistently misclassify ESL writing as AI-generated. Widely cited in subsequent research.

• University of Chicago — Brian Jabarian and Alex Imas, BFI Working Paper No. 2025-116: Tested leading detectors against 1,992 passages across six genres and four frontier models. Pangram was the only detector to meet a stringent false-positive cap while catching AI text.

• Epoch AI — AI Detector False Negatives Benchmark (July 2026): Tested Pangram, GPTZero, and Originality.ai against frontier LLMs including Claude Opus 4.8, GPT-5.5, and Gemini 3.1 Pro. Found significant missed detection on style-imitated text.

• HEPI — “AI detectors and the fairness gap: read the evidence before you trust the score” (August 2026): Analysis of detection bias and the consequences of false positives for non-native English speakers.

• Anthropic — Invisible Watermarking Announcement (August 2026): New Claude models embed invisible watermarks in generated text, with a Watermark Detection API available for verification.

• GPTZero — GPTZero 4o Technical Report (September 2026): Vendor-published benchmarks showing 0% FPR and >99% AI recall on four AI families.

• Pangram Labs — Third-Party Evaluations (August 2026): Independent testing showing 97.5% detection on fully AI-generated text and zero false positives on ESL essays.

• MIT Technology Review, Reuters, Associated Press, BBC News: Ongoing independent journalism covering AI detection and academic integrity.