Brief IA
›

Pangram: AI Detection Used for Online Reporting

Models & LLM·Tom Levy·

Pangram: AI Detection Used for Online Reporting

Pangram: AI Detection Used for Online Reporting
Key Takeaways
1Pangram has hired and then dismissed a journalist tasked with exposing the use of AI on social media
2Pangram's scores only measure the involvement of AI, without indicating the effort or intention of the author
3Researchers are having their articles rejected based on these scores, even for legitimate uses like translation
4The company is now aiming for the detection of slight modifications and a role of authority in originality verification
Why it matters : Pangram's practice illustrates how AI detection can lead to public accusations and consequences for various uses, without the tool distinguishing between intent or human contribution.

Pangram initially hired and then dismissed a "watchdog" tasked with exposing the use of AI online, but its CEO, Max Spero, continues to publicly assign scores. These scores only indicate the involvement of AI, without explaining how it was used. From assisted translation to rewriting, the tool is employed for accusations that also affect researchers.

Academic Rejections Based on Detection Percentages

Individuals claiming to be the AI police rely on scores to assert that a passage comes from a model and does not reflect the author's thoughts. In the academic world, articles are rejected based on such percentages. Many of the affected manuscripts come from researchers who write in a second language or who are more proficient in science than in prose and who use AI to clarify the expression of their results. Used this way, detection penalizes editorial improvement. A high score can apply to both a text rich in research and original arguments and a text generated quickly.

What Pangram's Scores Indicate (and Don't Indicate)

A detection score only indicates the likelihood that AI was involved. It does not allow for identifying whether the entire text was produced from a prompt, whether AI played a supportive role in the thinking process, whether it contributed to the formulation of ideas, or whether the author merely used a tool to improve the language or perform a translation. To differentiate these situations, the writing process would need to be analyzed, which Pangram does not do. In a concrete example, an English version of an article translated from German with the help of AI, then edited manually and via AI, was rated "28% AI," with the last paragraphs flagged, even though each section had been produced and edited similarly. The term "Pangram shaming," however, associates any involvement of AI with a lack of effort or care on the part of the author.

Le brief IA que lisent les pros

Le brief IA que les pros lisent chaque soir

Les 7 actus IA du jour, décryptées en 5 min. Gratuit.

Inclus dès l'inscription : notre sélection des meilleurs guides & comparatifs IA.

Choisissez votre rythme

Gratuit · Pas de spam · Désabonnement en 1 clic

From "Hunter" to Asserted Public Responsibility

Pangram hired a journalist tasked with publicly shaming AI users. Rod Breslau, who had offered his services earlier in the year due to his fatigue with generated content, advised the company on its strategy while tracking suspected accounts. He advocates that executives and CEOs should write their own publications, and that if they do not, mockery is justified. Pangram subsequently severed ties with him. Nevertheless, CEO Max Spero continued to publish scores and denounce individuals based on this, believing it was necessary to hold them accountable when they attempted to conceal AI involvement.

A Shift Towards Detecting Minor Modifications and Economic Bet

The separation from the "hunter" marked a change in direction. Pangram now starts from the idea that the use of AI will become more accepted and wants to detect even minor interventions, while positioning itself as an authority on originality checks in publishing, education, and other sectors, according to Max Spero. However, the company's business model depends on potential clients associating the use of AI with a lack of integrity or effort: if this stigma weakens, the incentive to pay for detection diminishes.

A Literary Precedent: Dumas, Maquet, and the Value of Work

Alexandre Dumas relied on Auguste Maquet for drafts that he then polished, including for The Three Musketeers. The value came from the initial concept and the final revision work. Transposed to the present day, AI detection would have pointed out the assistant's contribution and overlooked what underpins the quality of the work.

Brief IA — L'actualité IA en français

L'essentiel de l'actualité de l'intelligence artificielle, décrypté et expliqué chaque jour.