Detecting AI in Text Games: Many Alerts, Little Help

Low-cost checks aimed at spotting movie reviews generated by language models have massively marked texts as suspicious and caused a drop in the accuracy of a sentiment classifier. Meanwhile, recent scientific work estimates that a notable portion of AI conference evaluations has been substantially altered by tools like ChatGPT, and describes a risk of "collapse" when retraining models on synthetic text.

Many False Positives and Decreased Accuracy After Filtering

Applied to the unused archive for training, the combined score flagged 444 out of 594 reviews as potentially written by an AI and 158 out of 200 test reviews. These markings remain alerts, not proof of AI authorship. At the setting chosen to intercept 80% of synthetic texts, the incorporation density also classified 47% of human references as suspicious. Once filtered with the average of the three checks, the performance of the sentiment classifier dropped from 67.5% to 57.5% on a separate batch of 200 reviews. In the audit, 444 reviews were above the threshold and 150 below. The 1,394 reviews were rated together to allow for comparisons of similarity and density, with thresholds recalibrated on this volume. The threshold of 0.548 set for the training set of 600 reviews was not reused, as the scores were resized on the complete set.

Three Technical Checks Aggregated into an Alert Score

The method assigns three scores to each text. Perplexity, measured with GPT-2, assesses the predictability of words, and a low value is merely a weak signal. Near-duplication similarity relies on all-MiniLM-L6-v2 embeddings to find the closest match in meaning or form. Incorporation density looks at the average proximity to the five nearest neighbors. The three values are normalized between 0 and 1, with perplexity inverted, and then averaged into a combined score presented as an alert indicator rather than a probability. Comparisons are made by varying a threshold, measuring recall and false positive rates, with a setting aiming to capture at least 80% of generated texts while limiting reports among 200 human references.

A Protocol on 600 Texts Mixing 200 References and 400 Synthetic Ones

The initial collection consists of 1,000 reviews covering 10 films, manually annotated as positive or negative. For training, 200 reviews from films such as A Beautiful Mind, Amadeus, Avatar, or Clash of the Titans served as human references, balanced in sentiment. A separate test batch of 200 reviews, also balanced and drawn from other films like Les Misérables, Star Wars Episode I: The Phantom Menace, The Expendables I, The Godfather, and The Matrix Revolutions, was kept aside to avoid any leakage related to the films. Two models, gpt-6-astra and gpt-5-nano, generated 400 additional reviews, with 100 positive and 100 negative per model, based on prompts asking to invent a film and a short review, without examples or targeted imitation. The final training set comprises 600 lines, with 200 sources treated as human and 400 synthetic, and the results are limited to these models, prompts, data, and experimental conditions.

IMDb Origin Data Released in 2019 Under CC BY 4.0 License

The dataset 1000 Movie Reviews for Reputation Generation, uploaded by Abdessamad Benlahbib on Mendeley Data on March 9, 2019, describes reviews extracted from IMDb. Published before the public launch of ChatGPT in November 2022, the collection was used as human reference, with the caveat that the identity of the authors is not verified text by text and that this qualification relies on the source and date. The Creative Commons Attribution 4.0 International license allows for reuse and adaptation with attribution and link, and the use of a subset does not imply any endorsement from IMDb.

Expanded Audit of the Archive and Threshold Calibration

Before targeted testing, the entire set of 1,000 entries was screened to identify texts with a generated appearance, and then three checks were applied to 600 reviews including 400 synthetically controlled texts. Each check allowed for counting detections on the synthetics and reports on human references, measuring the impact on a sentiment classifier. During data loading, six duplicates were removed, leaving 994 distinct reviews, of which 200 were used for training and 200 for testing, with the remaining 594 kept for the audit.

In Scientific Evaluation, Signs of AI-Modified Writings

A statistical method presented at the 2024 International Conference on Machine Learning estimates the portion of text substantially written or rewritten by a language model. Applied to evaluations submitted to four major AI conferences after the launch of ChatGPT, it places this portion between 6.5% and 16.9%. These evaluations, written by researchers, carry technical judgments that have concrete effects. This estimate remains confined to conferences and does not inform the frequency in other contexts. Meanwhile, a study published in 2024 in Nature describes a mode of failure related to repeated retraining on generated text, with a loss of rare examples and tightening of outputs.