Math: OpenAI Ordered to Prove Data Usage

Andreas Thom questions the ethics and transparency of OpenAI, suspecting that the company has exploited user interactions for scientific results. Beyond his case, researchers fear a race for publications fueled by rumors and industrial power, while OpenAI claims major advancements.

Researchers Fear More Secrecy and a Race for Results

Many researchers express concern that the mere existence of rumors about significant advancements is enough to trigger competition with a tech group that has substantial resources. Andreas Thom's remarks come amid a growing climate of unease towards OpenAI within mathematical circles. The company announced that it has solved one of the Millennium Prize problems, an achievement presented as extraordinary that, if verified, would be difficult to contest. OpenAI explained that it decided to tackle this problem after hearing online rumors about the progress of other researchers. When contacted, the company did not immediately respond to a request for comment.

Thom Demands Proof on Data Usage and Denounces a Perceived Fallacious Response

Andreas Thom emailed Sébastien Bubeck and Mark Sellke to inquire whether his interactions with ChatGPT had been incorporated into the training or reasoning of OpenAI's models. He claims he only received a response regarding direct access to his conversations, without clarification on their potential inclusion in training datasets. According to him, no qualification, explanation, or proof was provided, which he considers dishonest. Thom believes that researchers cannot reverse-engineer OpenAI's training pipeline and that only the company holds the relevant data. He argues that in the event of a denial, it is up to OpenAI to disclose the necessary datasets and specify the parameters and terms of data usage. He also considers that a categorical response from Mark Sellke was unjustifiably broad and materially misleading, and, in hindsight, evidently dishonest. Thom believes it would be ethically indefensible for non-public research provided by users to be used to improve models that are then used to outpace them without consent, appropriate disclosure, or credit. He argues that OpenAI's distinction between specific data and de-identified data does not address the core issue, as de-identification does not remove the intellectual content of a mathematical idea.

What OpenAI Claims About Navier-Stokes and User Data

OpenAI asserts that neither its researchers nor its agents had access to the work of other teams before their public publication, and that no specific user data was used to solve the Navier-Stokes equations, which pertain to fluid motion. However, the company clarifies that it cannot rule out the possibility that de-identified data from the use of its products may have contributed to improving its models. This position, publicly noted and discussed in its exchanges with Tristan Buckmaster, raises questions about the origin of the highlighted results.

Group Theory at the Heart of Attribution Criticism

Last month, OpenAI announced ten results, including one on non-Sofic groups, an area of expertise for Andreas Thom. The company acknowledged that this result heavily relied on previous work by Thom and Gábor Kun. Following the announcement, sharp criticism emerged within the mathematical community regarding the lack of acknowledgment of these authors' recent contributions, and OpenAI discreetly modified its article. Andreas Thom was surprised to see an extensive use of methods that, in his view, were neither the most intuitive nor the most promising at the time they were implemented. Non-Sofic groups refer to infinite mathematical objects that cannot be approximated by finite objects.

The Buckmaster Precedent and the Shadow of Codex

The public questioning by Tristan Buckmaster, who used OpenAI's Codex in his work, led Andreas Thom to reflect on his own interactions with ChatGPT. Thom subsequently expressed his concerns publicly. Buckmaster was personally collaborating with Levent Alpöge, a researcher at Anthropic. Among his contacts at OpenAI, Thom notably wrote to Mark Sellke, a statistician at Harvard.