Safeworld raises $12M to validate AI robot safety

The start-up Safeworld aims to test the safety of robots powered by generative AI through dense simulations of human scenarios. It has announced that it has raised over $12 million from investors including a16z Speedrun and Shine Capital, while testing its approach with Gritt Robotics. The company, which originated from the Safe AI lab at Carnegie Mellon, has not yet decided whether its product will take the form of a software platform or a service offering.

An Open Business Model and Ongoing Industrial Testing

Safeworld has not determined whether its product will be a platform for external users or a service offering. The company indicates that it is operating in a still-early context for the use of generative AI in robotics. A first partnership is underway with Gritt Robotics to develop safety simulations. Gritt Robotics, whose CTO is Vishal Dugar, designs AI for robots that assist workers in installing photovoltaic panels on large solar farms and ultimately aims for more complex construction tasks. These robots operate in close proximity to human workers, and avoiding any impact from the robotic arm is a priority. Ensuring this safety requires anticipating numerous concrete scenarios.

Empirical Validation and Coverage of Varied Human Behaviors

For Vishal Dugar, formally proving through mathematical methods that a system is safe is very difficult; evaluation must be done empirically. The simulations must cover multiple postures and dynamics such as kneeling, standing, tripping, falling, crouching, or running. They must also incorporate the diversity of appearances of the exposed individuals, including clothing, size, shape, height, and skin color.

The Safeworld Method: Digital Environments and Thousands of Scenarios

Safeworld digitally reproduces usage environments, for example in engines like Genesis or MuJoCo. The robot being evaluated is simulated with its actual control software. The company then executes thousands of scenarios where human models encounter the robot, a step complicated by the unpredictability of human behaviors. Among the tested cases is the fall of a person, common in simulation campaigns. Concrete situations guide these trials, such as a blind spot in a factory, with questions of speed, stopping distance, and detecting a person carrying boxes.

Why a Trusted Third Party is Sought by Industry Players

While internal tools already exist among manufacturers, the founders of Safeworld believe that third-party validation will be sought, including for sharing safety cases among competitors. Ding Zhao emphasizes the difficulty of edge cases and notes that the stakes are high for large-scale deployments among users who are not familiar with robots. From the investors' side, Jonathan Lai of a16z Speedrun considers it necessary to define a safety standard now and warns that the arrival of robots in households without these safeguards could lead to collisions and avoidable incidents.

Financial Resources and a Team Rooted at Carnegie Mellon

Safeworld has announced over $12 million raised in a round led by Shine Capital and a16z Speedrun. Box Group, the endowment of Carnegie Mellon University, Innovation Endeavors, and SV Angel are also participating. The company was founded by Ding Zhao, Kyle Wong, and Simo Rachidi. Zhao, who leads the Safe AI lab at Carnegie Mellon, describes the safety of generative AI systems as a problem of managing probabilistic risk and building trust. He asserts that Safeworld will likely be the first profitable company in this field, estimating that any deployment will require compensating the company for safety management.

The Technical Context: The Rise of Generative AI and Its Unpredictability

Robot control is increasingly relying on generative AI models, whose unpredictability contrasts with traditional algorithms. According to Ding Zhao, evaluating these systems in unstructured environments, with site-specific safety requirements, is more complex than assessing autonomous vehicles. Safeworld positions itself on evaluating these controllers through simulations that incorporate realistic human models.