UNSPUN, Unit 7: AI Can Be Fluent and Wrong
A Jewish Onliner media literacy series in 8 parts: How manipulation works, how narratives are built, and how to see through it.
Jewish Onliner is an independent publication. If you find our work valuable, please consider becoming a paid subscriber. You can also subscribe to us on WhatsApp and Telegram. Follow us on YouTube, TikTok, Instagram, Discord, and X.
In 2023, a New York attorney submitted a federal court brief citing six legal precedents supporting his client's case. The brief was competent. It cited specific cases by name, quoted them accurately, and reasoned through them the way lawyers reason. It "worked" — right up until opposing counsel could not find any of the six cases in any legal database.
ChatGPT had invented all of them. The lawyer had trusted the output because it read like something a lawyer would write. It read that way because AI is very good at producing text that reads like something a competent writer would write. What it is not good at, what it fundamentally cannot do, is know whether any of it is true.
That gap, between how a claim reads and whether it is true, is where AI-era manipulation lives.
Generative AI produces text that reads like it was written by an expert: perfect grammar, authoritative tone, logical structure. None of that means it is true. These models do not know what is true; they predict which words are likely to come next. Sometimes those predictions match reality and sometimes they do not — and when they do not, the model writes the falsehood with exactly the same confidence and polish. Fluency has become cheaper than accuracy, and because most people read clarity as credibility, the fluent falsehood often outperforms honest uncertainty.
Why Fluency Stopped Being A Signal
For most of history, fluent writing was a rough proxy for expertise. Not a perfect one, but strong enough that most readers used it as a shortcut. If someone could produce a polished paragraph; good grammar, confident tone, coherent structure, they had usually spent time with the material.
That shortcut just broke. Polished paragraphs are now free, in unlimited quantity, on any topic, indifferent to whether they are correct. The Mata brief was not an anomaly; it was an early, unusually well-documented instance of a pattern that now runs at scale. AI-generated summaries, “explainers,” social-media threads, and even citations circulate constantly, and most of them are never checked because they don’t look like the kind of thing that would need checking. They look fine. That is exactly the problem.
When Manipulation Scales
The individual hallucinated citation is a small problem. The scale is the real one. What made the Mata case newsworthy — that AI produced a professional-looking output with no relationship to reality — is now happening millions of times a day, everywhere, on every topic.
AI has made it essentially free to produce fluent commentary, plausible-sounding “analysis,” polished summaries of events, and confident answers to questions the model cannot actually answer. Some of this is honest error. Someone asks a chatbot a question, gets a fluent answer, treats it as fact, and passes it on. Some of it is deliberate: AI-written blog posts, “explainers,” social threads, and product reviews engineered to look organic and rank well in search. And a growing share of it is invisible — quietly folded into articles, emails, and reports by people who used AI to draft and never disclosed it.
The old ecosystem of information was slow enough that most of what circulated had at least been touched by a human who cared whether it was correct. That is no longer true. A meaningful share of what you read on any given day was produced by a system indifferent to the truth of what it was saying and increasingly, downstream systems are being trained on that output, which means the errors compound.
What To Do
You cannot check everything. But you can adjust your defaults.
Check citations. If a claim references a study, an article, or a case, confirm it exists before you trust it. The Mata attorneys did not, and the cost was public sanction. Verify against the source directly rather than trusting a link, a screenshot, or a summary.
Treat AI-fluent tone as a reason to slow down, not a reason to trust. If something is unusually polished, confident, and free of hedging, that is not evidence, it is style. Ask what the claim rests on, and whether the writer (human or otherwise) had any way of actually knowing what they are asserting.
And when someone says “ChatGPT said…” or cites an AI output as a source, treat it the way you would treat “I heard somewhere that…” — interesting, unverified, please continue.