SGLang Unites NVIDIA, AMD, Google, and Meta in a Consortium

LMSYS Org brings together major players in chips and infrastructure around the SGLang Consortium, announced at the SGLang Summit in San Francisco. The structure defines a common framework for open-source inference, at a time when this phase of the AI cycle is taking an increasing share of computing expenditure.

A dedicated summit gathers over 100 speakers in San Francisco

The SGLang Summit, the first conference dedicated to SGLang and its open-source community, took place on November 12 and 13 at the Fort Mason Center for Arts & Culture in San Francisco. Over 100 speakers from AI hardware, models, infrastructure, and applications participated, with sessions covering reinforcement learning, multimodal models, and hardware platforms. SGLang, already used in some of the largest inference deployments in the world and deployed on hundreds of thousands of GPUs, serves as a common foundation for these discussions.

A shared working framework and targeted technical projects

The consortium provides its members with a permanent forum to engage with SGLang maintainers and transform common priorities into engineering work. The topics presented include support for advanced models, reasoning development, multimodal load management, distributed inference, and hardware optimization. Each founding member will appoint a technical representative and will attend two annual roadmap meetings, with the opportunity to contribute expertise, feedback on deployments, testing resources, or shared developments. Working groups may convene between these meetings, and public updates will inform about progress and advancements. SGLang maintainers retain control over technical choices and code integration, and the entire project remains accessible to the community.

Founders from NVIDIA to Google, and strong positions

The SGLang Consortium includes founding members NVIDIA, AMD, Intel, Google, Meta, Cloudflare, Modal, and RadixArk. NVIDIA, through Ian Buck, expresses its commitment to advancing open inference software, believing that better software enhances performance derived from its computations, that open models foster new applications and businesses, and that reduced inference costs make AI more accessible. Ying Sheng highlights that leveraging SGLang at a massive scale allows for better forecasting of inference-related needs and that, in the face of obstacles to creating new data centers, improving efficiency through software will be essential for SGLang users and the entire ecosystem. On the industry side, RadixArk, a company founded by Ying Sheng and Banghua Zhu, brings together key creators and maintainers of SGLang and Miles, offering managed infrastructure and tools for inference and training. LMSYS Org, the nonprofit organization behind SGLang, also leads open and scalable projects such as FastChat, Vicuna, and Chatbot Arena, with resources available at lmsys.org and sglang.io.

Inference gains traction and SGLang integrates into PyTorch

According to Deloitte, the share of inference in the computing power dedicated to AI is expected to reach about two-thirds by 2026, while it represents about half in 2025. Thus, the inference engine has a direct impact on the speed and cost of processes carried out on existing infrastructures. SGLang, integrated into the PyTorch ecosystem, operates at the scale of hundreds of thousands of GPUs and powers large production deployments at several major players. It is in this context that the creation of the consortium was announced during the opening keynote of the first SGLang Summit.