Kubernetes Gains Momentum with AI Agents

The rise of agents transforms a task into waves of inference calls, shifting the effort from training to usage. Kubernetes, designed for distributed workloads, is coming back to the forefront, according to Jonathan Bryce. The CNCF highlights projects like llm-d, Agent Substrate, and OpenShell, while over 300 companies are already contributing to the security aspect.

Agent Security and Governance: OpenShell Unites Over 300 Companies

OpenShell, governed neutrally by the CNCF after a contribution from Nvidia Corp., has surpassed 300 contributing companies in just a few months, a milestone that would have previously taken two years. Security and control remain a challenge, as agents can produce different decisions for the same task. Discussions around securing AI agents were part of the topics addressed with the CNCF, alongside hardware optimization and the rising demands for inference.

The Explosion of Inference Calls and Tools to Better Utilize Hardware

Jonathan Bryce describes scenarios where a coding agent generates 10 sub-agents, each making thousands of inference calls for hours. The CNCF is also introducing a new category dedicated to inference and AI agents at KubeCon. As the demand for inference drives the construction of data centers and raises local concerns about resource consumption, initiatives like the llm-d inference engine and Agent Substrate, designed for compact and long-term hosting, aim to further optimize the use of existing hardware, according to Jonathan Bryce.

Use Cases Shifting Towards Inference, Creating a Favorable Environment for Kubernetes

Investments are shifting from training models to their usage and the execution of agents, while the open-source community develops tools to handle the influx of inferences. Kubernetes was designed for dynamic and distributed workloads and has served as a reliable foundation for rapidly growing AI companies, according to Jonathan Bryce, who also emphasizes massive investment and infrastructure expansion with the rise of AI. He recalls that early chatbots triggered only a single inference call per message. Beyond performance on benchmarks in mathematics and software engineering, a model only adds value if it can be served at high performance. The rise of agentic AI systems is thus presented as providing a new lease on life for Kubernetes.

Framework for Exchange and Priority on Open Collaboration

Jonathan Bryce detailed these observations during an interview with Rebecca Knight at KubeCon + CloudNativeCon NA, broadcast live by theCUBE, the studio of SiliconANGLE Media. He mentions that he is constantly thinking about reducing friction in collaboration and how to promote open solutions that could become common infrastructure building blocks.