833 Tests on 6 AI Models Challenge Strict Application

The "strict application" mechanisms proposed by some providers are not always sufficient to ensure the reliability of AI in production. Tests conducted on 6 models, totaling 833 tests under various constraints, indicate that this approach often fails and can even degrade results. The major causes of failure relate to the models' capabilities and the complexity of the formats, while common evaluation practices overlook the most costly errors.

Measurement: evaluators can agree while being wrong

The way tests are designed can lead to erroneous conclusions about actual performance. Evaluators can converge in their judgments while still being mistaken. The results presented come with clearly stated limitations, and data as well as an open-source method are also mentioned.

833 tests on 6 models: strict application often fails

A total of 833 tests conducted on 6 models, under different constraint parameters, evaluate the "strict application" proposed by providers. In most cases, this approach does not work and can, in certain configurations, worsen results. Small errors can be enough to compromise a system, a risk heightened when multiple models are chained together. The observed failures are attributed to the models' capabilities and the complexity of the expected format, while common checks like verifying a "well-formed" output miss the most damaging issues.

Four recommendations for AI teams in production

It is advisable to select models first before adjusting the infrastructure. Forced illegal responses should be avoided. Teams are encouraged to concretely verify what providers are implementing. Finally, it is important to integrate the risks of cumulative failures at each stage of a model chain.