Open X-Embodiment Dataset Explained: What’s Inside, What RT-X Showed and Its Limits

Most robot AI models you read about in 2026, from open models like Octo and OpenVLA to Google DeepMind’s own RT-X models, were trained at least partly on the same pool of data: the Open X-Embodiment dataset. Released by Google DeepMind and partner labs in October 2023, it collected robot demonstrations from dozens of research groups into one shared format, so a single model could learn from many different robots at once.

This guide explains what is in the dataset, how its data is stored, what the RT-X experiments showed, how to download it, what the license says and where its limits are. It’s a research dataset, not a product, and most of what it contains is recorded lab work with robot arms.

What is the Open X-Embodiment dataset?

“Embodiment” is the robotics word for a robot’s physical body: its arm, gripper, cameras and the way it is controlled. Before Open X-Embodiment, most robot learning datasets came from a single kind of robot in a single lab, and a model trained on one rarely helped another. The project, abbreviated OXE, asked whether pooling data from many different robots could produce one “generalist” policy that works better across all of them.

To test that, Google DeepMind and its partners gathered existing datasets that labs had already recorded and converted them into a common format. According to the project site, the result pools “60 existing robot datasets from 34 robotic research labs around the world,” with more than 1 million real robot trajectories. The paper describes 22 robot embodiments, 21 institutions, 527 skills and 160,266 tasks. Google DeepMind’s announcement on October 3, 2023 used rounder numbers: more than 20 institutions, 33 academic labs, more than 500 skills and 150,000 tasks.

The counts differ because the sources count different things (institutions vs labs) and because the collection kept growing after the first release. The arXiv paper has been revised nine times; the latest version is dated May 2025.

What is inside: robots, tasks and format

The robots range “from single robot arms to bi-manual robots and quadrupeds,” the project site says, but the bulk of the data is robot arms doing tabletop manipulation. The paper’s breakdown shows:

• Franka arms appear in the most datasets and the most distinct scenes.
• xArm and the Google Robot (the mobile manipulator used for RT-1 and RT-2) contribute the most trajectories.
• Pick-and-place skills dominate, with a long tail of others such as wiping and assembling.

Every dataset is stored as a sequence of episodes in the RLDS format, which saves data as serialized TFRecord files. RLDS can hold different action spaces and sensor setups side by side, “such as differing numbers of RGB cameras, depth cameras and point clouds,” according to the paper. That matters because the robots don’t agree on much. Examples in the paper include robots controlled three, five and ten times a second, some sending position changes to the gripper and others sending velocities.

The data itself is mostly recorded demonstrations. The paper describes the field’s earlier datasets as covering grasping, pushing and teleoperated demonstrations, in which a person drives the robot. OXE repackages that kind of lab data; it is not footage of robots working autonomously in homes or factories.

What RT-1-X and RT-2-X showed

To check whether the pooled data helps, the team trained two existing Google model designs on it. RT-1-X is based on RT-1, a 35-million-parameter robot transformer. RT-2-X is based on RT-2, a vision-language-action model built on a large vision-language model. Both output a seven-number action for the gripper: x, y, z, roll, pitch, yaw and how open it is. For training they used a mixture from nine manipulators, not all 22 robot types.

The results were mixed in a useful way:

• Small datasets improved. Partner labs tested RT-1-X on their own robots and tasks. Google DeepMind’s chart shows mean success rising from 41% for each lab’s original method to 63% for RT-1-X, which is the “50% higher” headline. RT-1-X won on four of five tasks: kitchen manipulation (63% vs 43%), cable routing (56% vs 24%), door opening (80% vs 53%) and task-agnostic play (72% vs 33%). It lost on the Berkeley UR5 task (45% vs 53%).
• Big datasets needed a bigger model. On the data-rich Bridge and RT-1 domains, RT-1-X did not beat the single-robot baseline, which the authors describe as underfitting. The 55-billion-parameter RT-2-X did beat it.
• Skills moved between robots. On tasks that existed only in another robot’s data, RT-2-X scored 75.8% against 27.3% for RT-2. Removing the Bridge dataset cut RT-2-X to 42.8%, which suggests the skills really came from that other robot’s data.
• Design choices mattered. A 5-billion-parameter RT-2-X scored 44.4% on those tasks with a short image history and 14.5% without one. Trained from scratch rather than from a web-pretrained model, it scored 0%.

On unseen objects, backgrounds and environments, RT-2 and RT-2-X performed roughly the same (62% and 61%). Our guide to RT-2 and the robotics transformer covers the base model in more detail.

How Octo, OpenVLA and others use it

The dataset’s biggest impact came after the RT-X paper, as other teams used it to train open models:

• Octo, an open-source generalist policy released in 2024, was trained on 800,000 trajectories from Open X-Embodiment. Its authors tested it on nine robot platforms and say it can be fine-tuned to new setups in a few hours on standard consumer GPUs.
• OpenVLA, a 7-billion-parameter open vision-language-action model, was trained on 970,000 robot episodes from the dataset.
• OXE-AugE, a December 2025 paper, takes a different route. Its authors note that the top four robot types make up more than 85% of OXE’s real data, so they built a software pipeline that augments 16 OXE datasets with nine different robot arm and gripper combinations. The result has more than 4.4 million trajectories. They report that fine-tuning OpenVLA and π0 on it improved success by 24 to 45% on unseen robot and gripper pairings in four real-world tasks. Those are the authors’ results, not an independent benchmark.

The OXE-AugE authors also describe how the collection grew: version 1.1 has about 2.4 million trajectories, but only about 1.4 million are real robot manipulation or mobile manipulation. The rest is simulation, navigation, locomotion, human or visual question-answering data. That’s worth knowing before you quote a single “size” for the dataset.

How to download Open X-Embodiment and what the license allows

The official home is the google-deepmind/open_x_embodiment repository on GitHub. It links to:

• A dataset spreadsheet listing every included dataset, its metadata and the citation for each one.
• A Colab notebook that shows how to visualize episodes and build training batches.
• Downloads through TensorFlow Datasets (tfds.load) or directly from the public gs://gdm-robotics-open-x-embodiment storage bucket with gsutil.
• An RT-1-X checkpoint in TensorFlow and JAX, with an inference example. The README says the model takes one RGB image from a workspace camera plus a text instruction, and doesn’t use wrist cameras or depth.

If you work in PyTorch, Hugging Face’s LeRobot team also hosts an “Open X-Embodiment” collection with the datasets converted to the LeRobot format.

On licensing, the repository says its software is under the Apache 2.0 license and “all other materials” are under Creative Commons Attribution 4.0 (CC BY 4.0). It also says it is “not an official Google product.” The README asks users to cite the Open X-Embodiment paper and also the individual datasets they use, since each one came from a different lab. If you plan to build a commercial product on the data, read the entry for each dataset you use rather than assuming one rule covers everything.

Limits of the Open X-Embodiment dataset

• It is unbalanced. A handful of robot types and a lot of pick-and-place dominate the data.
• It is mostly arms on tables. There are some mobile manipulators, bimanual robots and quadrupeds, but little that looks like a humanoid in a home or factory.
• The original paper set its own limits. The authors wrote that their experiments “do not consider robots with very different sensing and actuation modalities” and “do not study generalization to new robots.”
• It is getting older. The original collection pooled datasets recorded up to 2023, and the released RT-1-X checkpoint uses a single workspace camera with no wrist or depth input.
• Lab results, not deployments. The success rates above come from research evaluations on lab tasks. They don’t tell you how a robot would do on a real job.

Newer datasets aim to fill some of these gaps with longer tasks, richer labels or humanoid data, which is why OXE is now usually one part of a training mix rather than the whole thing. For where it sits among today’s models, see our explainer on Physical Intelligence’s π0.

Frequently asked questions

What is the Open X-Embodiment dataset?

It’s an open collection of more than 1 million real robot trajectories from 60 existing datasets and 22 robot types, converted into one format. Google DeepMind released it with academic partners in October 2023 to train models that work across many robots.

How big is Open X-Embodiment?

The original release has more than 1 million real robot trajectories. The OXE-AugE authors report that version 1.1 has about 2.4 million trajectories in total, of which about 1.4 million are real robot manipulation or mobile manipulation data.

Is Open X-Embodiment free to use?

Yes, it can be downloaded for free. The GitHub repository puts its software under Apache 2.0 and other materials under CC BY 4.0, and asks users to cite the paper and each dataset they use.

What models were trained on Open X-Embodiment?

Google DeepMind’s RT-1-X and RT-2-X, plus open models such as Octo (800,000 trajectories) and OpenVLA (970,000 episodes). Many later robot models use it as one part of a larger training mix.

What is RT-X?

RT-X is the name for the RT-1-X and RT-2-X models trained on Open X-Embodiment. RT-2-X was about three times as successful as RT-2 on skills that appeared only in other robots’ data.

Sources

• Open X-Embodiment Collaboration, “Open X-Embodiment: Robotic Learning Datasets and RT-X Models” (arXiv 2310.08864, v1 October 13, 2023; v9 May 14, 2025): arxiv.org
• Open X-Embodiment project site: robotics-transformer-x.github.io
• Google DeepMind, open_x_embodiment repository (README, dataset spreadsheet, Colab, license): github.com
• Google DeepMind, “Scaling up learning across many different robot types” (October 3, 2023): deepmind.google
• Octo Model Team, “Octo: An Open-Source Generalist Robot Policy” (arXiv 2405.12213): arxiv.org
• Kim et al., “OpenVLA: An Open-Source Vision-Language-Action Model” (arXiv 2406.09246): arxiv.org
• Ji et al., “OXE-AugE: A Large-Scale Robot Augmentation of OXE for Scaling Cross-Embodiment Policy Learning” (arXiv 2512.13100, December 2025): arxiv.org
• LeRobot, “Open X-Embodiment” collection on Hugging Face: huggingface.co
• Google DeepMind, “RT-X Animation” (official YouTube channel, October 2023): youtube.com

All sources accessed November 1, 2026.

Last updated: November 1, 2026. To report an error, see our corrections page. Articles are drafted with AI assistance and reviewed and edited by an editor; see our editorial policy.