ArXiv Bans Authors Misusing AI for One Year

ArXiv, a widely used open-access research platform for disseminating preliminary work, is intensifying its efforts to counter the reckless use of large language models (LLMs) in scientific papers. Although articles published on ArXiv are not subject to peer review, the site has become a major tool for disseminating research in fields such as computer science and mathematics, and it also serves as a source of data on current scientific trends.

To combat the rise of low-quality AI-generated articles, ArXiv had already implemented measures such as requiring new authors to obtain the support of an established author. After being hosted by Cornell University for over two decades, the organization has become an independent nonprofit association. This change is expected to allow it to raise more funds to address the quality issues of AI-generated content.

In its latest initiative, Thomas Dietterich, president of the computer science section of ArXiv, announced that if a submission contains irrefutable evidence that the authors did not verify the results generated by the LLMs, it means that nothing in the article can be trusted. Such evidence includes "hallucinated references" and comments generated by the LLM, Dietterich clarified. If such evidence is found, the authors will face a one-year ban from submitting to ArXiv.

After this ban period, authors will need to ensure that their subsequent submissions to ArXiv are first accepted by a reputable peer-reviewed venue. It is important to note that this is not a total ban on the use of LLMs, but rather a requirement for authors to take full responsibility for the content, regardless of how it is generated.

Thus, if researchers copy and paste inappropriate language, plagiarized content, biased information, errors, mistakes, incorrect references, or misleading content directly from an LLM, they are held accountable. Dietterich explained to 404 Media that this rule will be enforced from the first infraction, but moderators must report the issue and section chairs must confirm the evidence before imposing the sanction. Authors will also have the option to appeal the decision.

Recent peer-reviewed research has revealed that fabricated citations are on the rise in biomedical research, likely due to LLMs. However, it is fair to note that scientists are not the only ones caught red-handed using AI-generated invented citations.