LLM: 13,000 Phrases Reveal AI Writing

Graphite has scrutinized the writing of several language models and identified thousands of formulations that are significantly more frequent than those used by human authors. Opus 5.5 and Astra each display their quirks, while the hunt for em dashes progresses. The labs promise more natural styles, but the markers do not disappear.

Persistent Indicators Despite Promises of More Natural Style

Graphite observes that the total number of writing indicators remains generally stable over time. Greg Druck notes that if the most well-known markers are removed, others emerge, and each model version develops its own specificities. He expresses doubts about the labs' ability to completely eliminate these constructions, citing the size of the models and the limited number of tests that can be conducted, which allows certain elements to slip through. This contrasts with announcements from Anthropic, which presents Opus 5.5 as more natural and reports that early users found its writing clearer and easier to follow, as well as with statements from OpenAI regarding GPT-6 Sol and Luna promising more clarity, less jargon, and fewer awkward phrasings.

A Comparative Protocol on 10,000 Pre-ChatGPT Articles

A corpus of 10,000 articles published before the arrival of ChatGPT was used to create a human control group. AI models were tasked with rewriting these contents from summaries to limit bias related to the source texts. The researchers then compared the frequency of words, expressions, and sentence construction patterns. Graphite derives a mapping of each model's lexical preferences while highlighting the persistent use of marked oppositions. In total, 13,000 phrasings appear at least twice as often in texts generated by LLMs than in human writings.

Opus 5.5 Favors "Reliable" and Repeats "It Matters"

In texts generated by Opus 5.5, the term "reliable" appears 23 times more frequently than in human samples. While the structure "it's not X, it's Y" is now avoided, the model uses variants like "it's more than an X, it's a Y." Two quirks dominate: "it matters," used 116 times more than by humans, and the construction "why X matters," 92 times more frequent. Greg Druck indicates that Claude models tend to align more closely with human word distribution, while GPT models diverge from it.

Astra Focuses on "Corrective Framing" and Language Cautions

OpenAI's Astra is distinguished by its use of the phrase "another dimension" and by mitigated formulations such as "may provide" or "may offer." Its main marker is a corrective framing, presenting a subject as "not just X" or as an alternative "rather than relying on X," phrasings that are more than 100 times more frequent than in human writing. Meanwhile, the labs have reacted to the past abuse of em dashes: Opus 5.5 uses them 99% less than Opus 5, Astra 88% less than human texts, and Gemini 3.1 Pro has nearly eliminated them altogether. Some early indicators, including these dashes and the use of the word "dive," have thus disappeared.