LLMs: Open Models Could Reduce Their Lag by 70 to 90%

Open-weight language models have reportedly closed 70 to 90% of the gap with leading proprietary offerings in standard evaluations. This convergence does not imply that the offerings are interchangeable: privacy, processed volumes, adaptability, and operational means remain crucial factors in choosing a deployment.
Data Sometimes Requires Internal Hosting
A legal or contractual obligation to retain data in a specific environment excludes services that do not comply with this constraint. Self-hosting an open model or using a private cloud can then become deployment options, even before comparing performance and pricing.
Health records and clinical notes, particularly those subject to HIPAA in the United States, are among the categories to consider. Personal financial information, transactions, accounts, and trade secrets covered by a confidentiality agreement also fall into this category. Government and defense sectors may impose sovereignty or classification requirements.
For European personal data, the processing conditions offered by a provider may not meet the legal requirements of the GDPR. The French company Mistral, for instance, offers Mistral Small 4, which is presented as suitable for deployments compliant with this regulation. Conversely, common commercial content, public information, or anonymized datasets may be acceptable with a third-party provider.
Volume Changes the Total Cost
For fewer than 1 million tokens per month, proprietary APIs would be less expensive once infrastructure and engineering costs are factored in. Self-hosting requires GPUs, operational work, and additional security measures. APIs would also be favorable for rapid experimentation; between 1 and 10 million tokens per month, they would maintain operational simplicity, but their costs would increase significantly.
Together AI, Groq, or Replicate provide access to hosted open models without requiring the client to manage the infrastructure. For Qwen 3 or DeepSeek V3, average prices would range from $0.20 to $0.60 per million tokens, compared to $1.25 to $3.00 per million incoming tokens for mid-tier proprietary offerings such as GPT-4o, Claude Sonnet, or Gemini Flash. The reported gap would be 4 to 10 times. More broadly, proprietary rates would range from $0.05 to $30 per million incoming tokens, with outputs billed two to four times higher.
Beyond 10 million tokens per month, an open model on dedicated GPUs would become economically more attractive. For over 200,000 monthly queries, the estimated savings from enterprise self-hosting would reach 70 to 90% compared to equivalent proprietary APIs. The break-even point would be around 500,000 to 2 million daily tokens, depending on the compared offerings.
API prices have already dropped by about 80% since early 2023, driven by efficiency gains and open competition. However, the most advanced proprietary models would still carry a price premium of 2 to 5 times compared to equivalent hosted open models. To evaluate self-hosting, the relevant comparison also includes these managed open services: Llama 4 is not a direct substitute for Claude Opus or GPT-5 Pro.
Le brief IA que les pros lisent chaque soir
Les 7 actus IA du jour, décryptées en 5 min. Gratuit.
Inclus dès l'inscription : notre sélection des meilleurs guides & comparatifs IA.
Choisissez votre rythme
Gratuit · Pas de spam · Désabonnement en 1 clic
Tasks Still Distinguish Capabilities
The best open models could match or even surpass their proprietary competitors in programming, structured extraction, and mathematics. For about 80% of applications in data science, a well-chosen open model would provide functionally indistinguishable results from those of a proprietary API: classification, summarization, extraction, generation augmented by internal document search, or development assistance. The remaining 20% would include nuanced long reasoning, complex agent chaining, and applications with high reliability requirements, for which leading proprietary models would retain a measurable advantage.
Claude Opus-class systems would maintain an edge for complex multi-step reasoning, particularly in GPQA Diamond. Differences would also persist in the consistency of adherence to instructions at scale and for low-resource languages. These distinctions justify an evaluation based on tasks intended for production rather than a choice based solely on an overall ranking.
Among open offerings, Meta's Llama 4 is presented as the most influential family; Scout features a context of 10 million tokens, touted as a first for an open model. DeepSeek V3 and its successors would be competitive against the GPT-4o class in programming and reasoning; R1, designed for mathematics and logic, would be particularly suited for complex inferences. Alibaba's Qwen 3.5 ranks highly in several open benchmarks, with its 235B-A22B expert mixture variant competing with the best proprietary models in most standard evaluations. Its multilingual capabilities, particularly in Chinese and Arabic, along with the performance reported relative to their size for Mistral Small variants, provide additional points of comparison.
The proprietary offerings for 2026 include GPT-5 from OpenAI, Claude from Anthropic, and Gemini from Google. Claude offers a context of 200K tokens, suitable for long document analysis. The GPT-5 environment includes development kits, integrations, structured response formats, and API Assistants. Gemini benefits from close ties with Google Cloud and Vertex AI.
Licenses and Model Adaptation
Both open and proprietary offerings are already being used in production, but they do not provide the same level of control. For a language model, the label "open source" generally refers to publicly accessible weights, without guaranteeing access to data, code, or training methods. Some licenses designated as such do not meet the definition of the Open Source Initiative. A proprietary API does not provide the weights; its operator manages infrastructure, updates, and capacity scaling.
Commercial permissions also differ. The Meta Community License for Llama allows commercial use but includes restrictions. DeepSeek uses MIT, described as the most permissive license among major open families. Qwen 3.5, Mistral 7B, and Mistral Small are offered under Apache 2.0.
Additional training on specialized data could yield results that a simple formulation of general instructions cannot achieve, whether it involves clinical vocabulary, financial instruments, contracts, or internal jargon. Open models allow for successive trials on proprietary data and ownership of the obtained weights. The adapted model can then operate without ongoing dependency on an API or payment for its use. This degree of customization, operational means, and compliance obligations may weigh more heavily than overall scores.
Some proprietary providers also allow for this additional training, but do not offer the same level of in-depth control.
Brief IA — L'actualité IA en français
L'essentiel de l'actualité de l'intelligence artificielle, décrypté et expliqué chaque jour.