The AI watermark will flag the wrong writers

  |   Kevin Meyer

My grammar can be atrocious, and I know exactly why. I grew up in a Spanish-speaking country and still speak it close to natively, which left a permanent groove in my English prose. Spanish tolerates long sentences with clauses stacked in an order that lands backwards for an English reader, and decades later I still build sentences that way whenever I write fast. Linguists call it syntactic transfer. My early writing here is a monument to it, and I would rather you not go looking.

So I run drafts through Claude for a cleanup pass. It untangles the inverted clauses and tells me when I have buried the subject four commas deep. Then I make every change by hand in my own editor because I want to know what changed.

That habit acquired a new complication on August 2. Article 50 of the EU AI Act took effect, requiring providers of generative AI to mark synthetic audio, image, video, and text in a machine-readable format. Anthropic responded by marking Claude's output with an embedded text watermark plus signed metadata on generated files. The marking applies to every model launched on or after that date, worldwide, whether or not you are anywhere near Europe.

Ben Thompson wrote a sharp piece on this at Stratechery, and I agree with nearly all of it while landing somewhere slightly different on why it bothers me.

What the mark actually says

Anthropic's own documentation is admirably candid. A detected mark signals that content may have been processed by Claude. It establishes nothing about who wrote the underlying ideas, since people routinely use these models to proofread or reformat work that is entirely their own. Absence of a mark establishes nothing either, because heavy editing, paraphrasing, format conversion, or simple brevity will all wipe the signal out.

The result has poor sensitivity and poor specificity. In manufacturing terms we have deployed an inspection gage whose measurement error swamps the tolerance we actually care about, and we are about to read its output as pass/fail. Anyone who has developed a gage R&R study knows how that story ends.

Who actually pays

Marking happens at the model-serving layer, which means anyone running open weights on their own hardware produces unmarked text by default. Models never placed on the EU market sit outside enforcement reach entirely. None of this touches the thousands of fine-tunes floating around Hugging Face, or the person spinning up a voice clone for a fraud call.

The mark is also forgeable. Researchers have demonstrated that collecting enough watermarked output lets an attacker infer which tokens carry the signal, which enables both stripping the mark and stamping it onto text a model never wrote. A public detector, the very thing that would make any of this useful to a reader, doubles as the oracle that makes forgery easier.

There is a design trade underneath: a watermark tough enough to survive paraphrasing is by construction insensitive to editing, including edits that reverse the meaning of the sentence it is riding in. So we have a control system with the dominant source of variation sitting outside the loop.

The part that should worry writers

In 2023, Stanford researchers ran 7 commercial AI detectors against essays from native and non-native English writers. More than half of the TOEFL essays written by non-native speakers came back flagged as AI-generated. Essays from American 8th graders scored nearly perfect. The likely mechanism is that those detectors were scoring how predictable the word choices were, and someone writing in a second language works from a narrower vocabulary.

I read that study with more personal interest than most people would. The fix that worked is the part I keep chewing on: running those human-written essays back through ChatGPT with instructions to use more sophisticated language got them reclassified… as human.

Watermarking works differently from perplexity-based detection, and I want to be careful about conflating the two technologies. What transfers is the institutional behavior. Schools and employers adopted an unreliable signal and used it punitively anyway, and the cost landed hardest on the people least positioned to argue back. Nothing in the new marking regime dampens that appetite. It supplies a fresh signal, thinner than it looks, to an audience already primed to treat any hit as a verdict.

Grammar is not authorship

Some writers I respect regard these models as plagiarism engines, given how they were trained. That argument is real and deserves its own post. It has almost nothing to do with what happens when I ask a model to tell me that four paragraphs in a row start with the same word.

We settled this question for other tools decades ago. Nobody appends a disclosure when a spreadsheet computes an IRR, and no one has ever argued that spell-check makes Microsoft the author. The engineer who runs a finite element analysis still owns the design, including the parts the software talked him out of.

Underneath the whole regime sits an assumption that human-origin content is trustworthy and machine-origin content is suspect. A person can lie, or be confidently sloppy on deadline. A model with a careful prompt and honest sourcing can produce something more measured than either of us manages on a bad morning. I have written before about how much of what unsettles us in these models turns out to be our own reflection. Provenance and accuracy are separate questions, and building infrastructure that answers the first while everyone reads it as an answer to the second seems likely to make the information environment worse.

Meanwhile I will keep applying my corrections by hand, which now looks less like stubbornness and more like a workaround. That is the tell. When a measurement system starts inducing rework in a process that was working fine, the measurement is the thing to fix.

So a question for the writers reading this. If a tool untangled a sentence you had built the way another language taught you to build sentences, and taking its advice carried some risk of a label you could never remove, would you still take the advice?

Tags
AI
Share