OpenAI Releases Perplexity Measures to Regulate AI

OpenAI has made its internal perplexity measures public, showcasing a series that traces its progress since GPT-3.5. This quality criterion for predictions is proposed as a benchmark for comparing models and establishing common safety requirements. However, the published values remain tied to the company's proprietary evaluation data.
The Limits of Laboratory Comparisons
Developers do not train their models on the same data mixes and do not evaluate them on identical corpora. The figures revealed by OpenAI come from its own primary evaluation set: they do not, therefore, constitute a common comparison across laboratories. Nevertheless, the company believes that a commitment to transparency would allow for the development of transferable comparison methods between base models.
The measure also presents another limitation: it does not directly correspond to any practical task. According to OpenAI, chain reasoning can significantly enhance capabilities without altering the perplexity achieved by the model. Therefore, the advancements made after pre-training are not all reflected by this indicator.
Its minimum remains undetermined as well. A loss of 0 would imply predicting the data in a completely deterministic manner, a possibility dismissed by OpenAI, while their intrinsic entropy remains unknown. As this floor is approached, the company believes that small additional decreases could correspond to greater intelligence gains, despite an apparent slowdown of the indicator.
OpenAI Links Its Scores to Observed Capabilities
Perplexity evaluates the average quality of predicting the next token: the lower it is, the better the prediction. OpenAI indicates that it has been tracking this criterion for years on data retained for evaluation. The company correlates its evolution with several capabilities that have emerged over generations, without claiming that their precise thresholds were predictable.
According to OpenAI, GPT-2 achieved an unprecedented performance on older corpora; the company cautiously presents it as the first model capable of composing coherent paragraphs. GPT-3 could then learn a task from the provided context, based on a few examples. For GPT-3.5, the foundation of the initial ChatGPT and capable of fluid conversations, OpenAI reports a log-perplexity of approximately 0.99 on its current primary corpus.
GPT-4 was reported by the company to be around 0.91. Its ability to leverage chain reasoning has encouraged targeted work on this capability, leading to the development of modern reasoning models. Below 0.8, OpenAI reports capabilities it describes as transformative in cybersecurity as well as mathematics, while noting that their emergence at these values had not been anticipated.
The company claims to have been presenting open mathematical problems to its models for over a year to evaluate them. With a model around 0.76, it says it has achieved significant mathematical results. Successive solutions circulated in the same Slack thread, surprising team members who had worked in mathematics or related disciplines.
Le brief IA que les pros lisent chaque soir
Les 7 actus IA du jour, décryptées en 5 min. Gratuit.
Inclus dès l'inscription : notre sélection des meilleurs guides & comparatifs IA.
Choisissez votre rythme
Gratuit · Pas de spam · Désabonnement en 1 clic
Pre-training Serves as a Benchmark for Progress
This focus on prediction is based on a particular description of AI development. OpenAI presents the training of a compressor, also described as a predictor, on a vast corpus as the primary source of the capabilities exploited by modern methods. In this simplified scheme, a large computer is used for pre-training on unstructured data; instructions, fine-tuning, and reinforcement learning then bring out latent abilities.
Algorithmic advancements can blur this separation. Nevertheless, OpenAI distinguishes the formation of capabilities that are difficult to anticipate, previously referred to as "alien mentality," from subsequent work aimed at specific behaviors. The company also presents cross-entropy loss as a measure equivalent to perplexity.
Task evaluations, on the other hand, change with the models. OpenAI considers most current tests to be of little relevance for GPT-2 or GPT-3, while older tests were already largely saturated by the time of GPT-4. Monitoring pre-training seems likely to remain informative for a longer period: the company emphasizes that many teams have long been working to optimize perplexity and associates it with regular gains in general intelligence.
Research on Pre-training Could Accelerate
OpenAI reports an increasing involvement of AI in its own development. Discussions often revolve around assisting human researchers or replacing them at each stage, up to complete recursive self-improvement: a fully automated process, without human supervision. The maintenance of human control is precisely invoked to rule out this outcome.
However, an acceleration could occur before this general automation, solely through the contribution of models to algorithmic research on pre-training. OpenAI envisions a breakthrough in deep learning comparable to what it reports in mathematics. The company acknowledges that this field relies more on experimentation, while reminding that simple ideas can remain overlooked for years.
A surge of fundamental advancements is thus anticipated as early as 2027, though not guaranteed. OpenAI also expects significant contributions from AI to alignment and oversight. However, the company believes that these safety advancements may not be rapid enough to allow for a complete acceleration of pre-training.
Common Limits Related to Oversight Progress
The proposal put forth by OpenAI is to make perplexity a shared benchmark for general intelligence, regardless of the degree of automation in development, and to use it to compare developers as well as establish safety standards. The company acknowledges that its measures inform its competitors about its performance. Nevertheless, it invokes the urgency of common requirements to justify this transparency and desires more of it in the industry.
By extending recent trends, OpenAI envisions other transformative capabilities towards a log-perplexity of 0.68-0.7, without being able to specify which ones. These abilities may not be covered by current evaluations. The company also anticipates a reduced effectiveness of dominant alignment audits and a better recognition, by the models, of situations where they are trained or evaluated.
The proposal includes common rules for evaluation, transparency, and examination of safety guarantees from the first use of the relevant models. Limits based on perplexity and other measures could only be crossed after concrete progress in understanding and monitoring their behavior. The development of a successor to the devices monitoring chain reasoning is among the conceivable conditions. An initial industrial limit could then be followed by other thresholds governing the continuation of development.
Brief IA — L'actualité IA en français
L'essentiel de l'actualité de l'intelligence artificielle, décrypté et expliqué chaque jour.