Runway Releases Praxis-1, a Video Model for Controlling Robots

Runway releases Praxis-1 as open source, a video model designed to execute robotic movements from images and textual instructions. Internal benchmarks place this World Action Model at the top of its category on RoboLab-120, while detailing its limitations, data requirements, and hardware prerequisites.

Known Limitations, Explicit Steps, and Still Internal Checks

Runway indicates that Praxis-1 sometimes confuses objects that look similar and does not yet reliably plan complex action sequences autonomously. The model also struggles with vague instructions. For long tasks, it is necessary to explicitly formulate each step. The published results come from tests conducted by Runway and have not yet been independently verified. The company believes that the performance is promising based on its internal evaluations.

Open Source Access and Required Hardware Resources

Praxis-1 is offered with weights and code under the Apache 2.0 license, allowing for commercial use. The model can be downloaded via Hugging Face after registration and acceptance of access conditions, and the code is available on GitHub. Runway announces pre-adapted versions for different robots, in addition to the base model. The latter requires a GPU with 80 GB of memory. According to Runway's estimates, a compressed version should work on an RTX 5090 with 32 GB of VRAM, but maximum memory usage has not yet been measured according to GitHub documentation.

Benchmarks: 46.2% on RoboLab-120 and 88 Successes on Real Arm

On Nvidia's RoboLab-120 ranking, which evaluates 120 simulated tasks of sorting, grasping, and stacking, Praxis-1 achieves a success rate of 46.2%. FLUX 3 Action, built on a similar approach, scores 42.9%, placing Praxis-1 at the top of the World Action Models category and fifth overall, behind two AI agents and two robotic models of different architectures. After an adjustment on DROID, a test on a real Franka arm, with ten tasks and ten attempts per task, resulted in 88 successes for Praxis-1 compared to 80 for FLUX 3 Action. Runway also evaluated the transfer to a mobile platform with a previously unseen gripper arm, stemming from Google's work on RT-1. After about 95 hours of adjustment, the model achieves an average success rate of 80% in simulation, across four types of tasks including grasping cans, opening and closing drawers, and placing an apple in a drawer. In the same benchmark, RT-1 scores 52.4%.

Training Data: General Video, DROID, and Ongoing Robot Needs

Praxis-1 relies on a general video model refined with videos of human actions and data from robots. This approach could reduce dependence on strictly robotic datasets but still requires dedicated corpora. To adapt the model to a Franka arm, approximately 277 hours from the public DROID dataset were used, featuring scenes of grasping and moving objects. For a dual-arm robot, training involved between 1,400 and 2,600 hours of data. Runway indicates that it has already integrated DROID data and other systems at earlier stages. It is not established that the approach requires a total of less robotic data than models without general video pre-training.

Operation, Positioning, and Testing Partners

The model processes camera images, textual instructions, and positional information to predict scene evolution and calculate appropriate robot movements, adjusting its control as new camera frames arrive. Runway describes Praxis-1 as a World Action Model and mentions trials conducted since September with Standard Bots, Noble Machines, and Ultra on various platforms, including a humanoid robot, an industrial arm, and a dual-arm warehouse robot. According to Runway, the model can be adapted to new systems through additional training. Introduced to partners at the end of September, Praxis-1 is now accompanied by a detailed technical report. Runway, known for its generative video models like Gen-4.5, positions this system for rapid movement execution and envisions that it could eventually operate alongside generalist models tasked with breaking down complex tasks.