Physical Intelligence and π0.7: Robots for Unprecedented Tasks

Physical Intelligence Introduces the π0.7 Model

The start-up Physical Intelligence recently made waves by introducing π0.7, an artificial intelligence model that enables robots to perform tasks they have never learned before. This model can guide a robot through an unknown task using plain language instructions. During laboratory trials, a robot was able to carry out certain manipulations without having been specifically trained for them, thanks to precise guidance.

Continuous Improvement Through Rephrasing

One of the remarkable features of this technology is that the robot can improve its success rate on a given task simply by rephrasing the instructions. This occurs without any modification to the model or the addition of new data. Through repetitions, the robot manages to stabilize the coordination of its movements, thus demonstrating an adaptive learning capability.

A Unique System and Diverse Data

Founded in 2024, Physical Intelligence has published work on a unique system that can be used across multiple types of robots. The training data for the model includes recordings from various machines, videos of human behaviors, and autonomous executions. The model is designed to process textual instructions, visual cues, and speed parameters, giving it great flexibility.

Experimentation in a Test Kitchen

In an experiment conducted in a test kitchen, a robot was placed in front of an air fryer. The training data contained a few isolated recordings, such as a robot closing a device or placing an object in a container. No sequence corresponded to a complete cooking process. Nevertheless, the robot successfully opened the compartment, grabbed a sweet potato, and attempted to insert it into the device. The action was interrupted without precise instructions, but the engineers then detailed each step of the manipulation, allowing the robot to resume execution based on this sequence of instructions.

Reconstructed Actions from Separate Experiences

In another trial, the same task was phrased in several ways. The first version resulted in a series of failures due to vague instructions. In contrast, a second version, detailing each step separately—from positioning the object to the final action—achieved a complete result. According to Ashwin Balakrishna, a researcher involved in the tests, this was a gradual adjustment of the instructions that improved the success rate without modifying the model.

Skill Transfer Between Robots

The training data included sequences showing isolated gestures with no direct link to the final task. Sometimes, the closure of a device or the placement of an object in a container was observed, but no recording showed the steps of a complete cooking process with an air fryer. Despite this, during trials, the robot linked these gestures in a new environment, opening the device, moving a food item, and adjusting its position after several guided attempts.

To consolidate these results, the researchers conducted the same test with a different industrial robot than those used during training. This robot, having a distinct structure and dimensions, successfully executed a folding of laundry after several adjustments to the instructions. However, no common method was used to verify these performances independently, with each team employing its own testing conditions and success criteria.

Towards Future Applications

It is crucial to note that these results currently represent only laboratory tests. No announcement has been made regarding the operational deployment of this technology, whether in the industrial sector or for everyday use.