ChatGPT and the Rise of Fake Citations in Biomedicine

An Explosion of Fictitious References in Biomedical Research

A recent study conducted by researchers at Columbia University, in collaboration with other institutions, has highlighted an alarming increase in fictitious references in biomedical articles. Since 2023, the number of these references has multiplied by more than twelve times. Researchers point to advanced language models, such as ChatGPT, as likely culprits behind this concerning phenomenon.

These fictitious references, although false, appear credible and pose a major risk when they appear in review articles influencing clinical guidelines. To combat this issue, researchers advocate for the establishment of automated reference checks before publication, as well as retroactive screening of already published articles. Some platforms, such as Arxiv, have already begun to impose sanctions for AI-related errors.

The Scope of the Problem Revealed by a Massive Audit

A comprehensive audit of 2.5 million biomedical articles revealed that invented references have become a systemic problem in peer-reviewed research. Since 2023, the rate of these references has increased more than twelvefold. This study, published in The Lancet by a team led by Maxim Topaz, analyzed 2.47 million articles from the open archive PubMed Central, covering the period from January 2023 to February 2026.

Among the 97.1 million verified references, 4,046 were identified as fabricated, spread across 2,810 articles. A reference is considered fabricated if its title cannot be found in any of the four major databases: PubMed, Crossref, OpenAlex, and Google Scholar.

A Worrying Trend Since 2023

The chronological evolution of this phenomenon is revealing. In 2023, the rate of fabricated references was stable, with about four fictitious references for every 10,000 articles. However, starting from mid-2024, this rate skyrocketed, reaching 51.3 per 10,000 by the end of 2025, and 56.9 per 10,000 at the beginning of 2026. This increase coincides with the massive adoption of language models like ChatGPT, which began in late 2022. The authors of the study note that the publication process typically takes between 100 and 200 days, which explains why the effects of AI only manifested from mid-2024.

Researchers do not rule out other contributing factors, such as increased activity from paper mills or changes in indexing practices. However, the rapid rise in hallucinated references starting in the summer of 2024 remains concerning.

A Major Detection Challenge

The main challenge lies in detecting these fictitious references. They are difficult to identify because they conform to the expected format, cite real researchers, and include plausible publication years. For example, in a urology article, 18 of the 30 verified references were fabricated, while being closely related to the subject matter.

Researchers also discovered signs of coordinated activity by paper mills. Two authors were involved in eleven articles from the same surgical journal, with a total of 15 fabricated references on topics such as CRISPR diagnostics and the gut microbiome.

The Urgency of an Appropriate Scientific Response

During the audit, 98.4% of the affected articles had received no response from their editors. Review articles were particularly impacted, exhibiting a fabrication rate 57% higher than other types of articles. This is especially concerning as these reviews often serve as the basis for clinical guidelines. If a guideline relies on an article containing partially fabricated sources, the entire chain of evidence behind treatment decisions is compromised.

Although the scientific community has begun to respond, the reaction remains uneven. Arxiv has tightened its sanctions for unverified outputs from language models in manuscripts, threatening offending authors with a one-year ban. An analysis of accepted articles at NeurIPS 2025 has already shown that even the top AI conferences cannot reliably detect fabricated citations. A possible countermeasure is CiteAudit, an open-source system for automated citation verification, although it also highlights how ineffective commercial language models are at detecting their own reference issues.

Researchers recommend four steps to improve the situation:

• Implement automated reference checks before peer review.
• Include integrity metadata in article datasets.
• Conduct retroactive screening of already published articles.
• Create a dedicated "fabricated references" category in research integrity databases.

For their own study, the authors used Claude for code development and grammar checking.