Gimlet Labs Raises $80 Million to Revolutionize AI Inference

Gimlet Labs, a startup founded by Zain Asgar, an assistant professor at Stanford and a recognized entrepreneur, recently raised $80 million in a Series A funding round. This round, led by Menlo Ventures, aims to tackle the persistent problem of AI inference bottlenecks through an innovative and elegant approach.

The company has developed a "multi-silicon inference cloud," which it claims is the first and only of its kind. This software allows for the simultaneous execution of AI workloads across different types of hardware, including traditional CPUs, AI-optimized GPUs, and high-memory systems. "We essentially run on any available hardware," Asgar told TechCrunch, emphasizing the flexibility and efficiency of their solution.

Tim Tully, lead investor at Menlo, explained in a blog post that each step of AI inference requires specific hardware: inference is tied to computation, decoding to memory, and tool calls to networking. No processor can yet do it all, but with the emergence of new hardware and the repurposing of aging GPUs, the multi-silicon fleet is ready to be harnessed. Gimlet Labs provides the necessary software layer to orchestrate these processes smoothly.

According to McKinsey, if current trends continue, data center spending could reach nearly $7 trillion by 2030. Asgar pointed out that current applications only utilize 15 to 30% of the already deployed hardware, representing a potential waste of hundreds of billions of dollars. "Our goal was essentially to figure out how to make AI workloads 10 times more efficient than they are today," he stated.

The co-founders of Gimlet Labs, Michelle Nguyen, Omid Azizi, and Natalie Serrino, developed orchestration software that slices AI workloads to distribute them efficiently across various hardware. Gimlet Labs claims that its technology can reliably accelerate AI inference by 3 to 10 times for the same cost and power, thereby optimizing hardware resource utilization.

The startup has already established strategic partnerships with chip manufacturers such as NVIDIA, AMD, Intel, ARM, Cerebras, and d-Matrix. Gimlet's product is available as software or via an API to its own Gimlet Cloud, primarily targeting large AI model labs and data centers.

Since its public launch in October, Gimlet Labs has generated eight-figure revenues, totaling at least $10 million, and has seen its customer base more than double in four months. While the names of its major clients remain confidential, the startup counts among them a significant model manufacturer and a very large cloud computing company.

The founders of Gimlet Labs previously worked together at Pixie, a startup that created an open-source observability tool for Kubernetes and was acquired by New Relic in 2020. With total funding of $92 million, Gimlet Labs also benefits from the support of angel investors such as Bill Coughran from Sequoia, Stanford professor Nick McKeown, former VMware CEO Raghu Raghuram, and Intel CEO Lip-Bu Tan. The company currently employs 30 people.

Other investors include Factory, which led the seed funding, as well as Eclipse Ventures, Prosperity7, and Triatomic. After a chance meeting between Asgar and Tully about a year ago, angel investments poured in, and the funding round was quickly oversubscribed.