What Is a Vision-Language-Action (VLA) Model? A Plain-English Guide

Vision-language-action models let robots turn what they see and what they are told into movement. A sourced timeline from RT-2 (2023) to Gemini Robotics 2 (2026), plus limits.
Orange robot arm with a silver three-fingered gripper on a Care-O-bot service robot, reaching for a small white canister with a green cap on a table Orange robot arm with a silver three-fingered gripper on a Care-O-bot service robot, reaching for a small white canister with a green cap on a table
A Care-O-bot service robot reaching for an object on a table with its three-fingered gripper, the camera-to-gripper manipulation that VLA models target. The photo dates from 2010, long before VLA models, and is for illustration only. Photo: Jiuguang Wang / Wikimedia Commons, CC BY-SA 2.0 (cropped).

In 2023, Google DeepMind got a robot to cope with objects and situations it had never been trained on, by writing the robot’s arm movements as if they were words. The model, RT-2, roughly doubled how well the robot handled unseen situations compared with its predecessor, according to its authors. It also gave the field a new term: the vision-language-action model, or VLA.

How do robots understand commands? Many of today’s general-purpose robots use a vision-language-action (VLA) model. The language part interprets an instruction such as “put the cup in the sink,” the vision part reads the robot’s camera images, and the action part outputs motor commands or joint targets, a short sequence at a time. In many recent designs a slower system (about 7–10 times per second) decides what to do and a faster one (about 120–200 times per second) turns that into smooth motion.

In short: a vision-language-action (VLA) model is an AI system that takes camera images and a language instruction, then outputs robot actions. Open weights exist for some models (OpenVLA, NVIDIA GR00T); others, including Gemini Robotics 2’s VLA, stay limited to early-access partners. The sections below walk through how that works, the main published systems, and what still fails.

Advertisement

A VLA is an AI model that takes camera images and a written or spoken instruction as input, and outputs the actions a robot should take. The vision part reads images from the robot’s cameras. The language part understands an instruction such as “put the cup in the sink.” The action part outputs motor commands or joint targets, usually a short sequence at a time. The appeal is that one model can handle many tasks and objects, instead of needing a separate program for each.

Since RT-2, VLAs from NVIDIA, Physical Intelligence, Figure and Google DeepMind have become the main approach to teaching robots general tasks. They’re still limited by data, speed and dexterity. Here’s how they work, where the training data comes from, and how to read the claims.

How a VLA turns pixels into motion

Most VLAs start from a vision-language model (VLM), an AI trained on images and text from the web. Robot data is then added so the model learns to output actions. RT-2 did this by writing actions as text tokens that the language model could generate, and training on web data and robot data together. Later models added dedicated components for actions.

Many recent designs split the work in two, which the field often calls a dual-system design:

Part Job Speed (as reported at release)
System 2 (slow) Understands the scene and the instruction; decides what to do About 7–10 times per second
System 1 (fast) Turns that decision into smooth motor control About 120–200 times per second

The speed gap matters because a large model is too slow to balance and move limbs on its own. Separately, humanoids typically use a lower-level whole-body controller for balance and walking. Boston Dynamics says its whole-body controllers are trained with reinforcement learning in simulation, and a VLA sits above that layer rather than replacing it. See our explainer on reinforcement learning in robots.

From RT-2 to Gemini Robotics 2 in three years

Model Who When Key published details
RT-2 Google DeepMind July 2023 Coined “VLA.” Actions written as text tokens; trained on web and robot data. Models of 5B and 55B parameters ran at roughly 5 Hz and 1–3 Hz. Roughly doubled generalization to unseen situations versus RT-1, per the authors. Could not learn motions beyond those in robot data.
OpenVLA Stanford, Berkeley and others June 2024 Open 7B-parameter model trained on about 970,000 robot demonstrations. Authors report 16.5 percentage points higher success than the 55B RT-2-X across 29 tasks. About 6 Hz on one high-end GPU; can be fine-tuned on consumer GPUs.
π0 Physical Intelligence October 2024 3.3B parameters: a 3B language model plus a 300M-parameter “action expert” using flow matching. Outputs action chunks of 50 steps at up to 50 Hz. Pretrained on more than 10,000 hours of robot data.
Helix Figure February 2025 Two-part system for a humanoid upper body: a 7B model at 7–9 Hz and an 80M-parameter controller at 200 Hz controlling 35 degrees of freedom. Trained on about 500 hours of teleoperated data, per Figure. Runs on embedded onboard GPUs.
GR00T N1 NVIDIA March 2025 Open foundation model for humanoids with a slow VLM (about 10 Hz) and a fast diffusion-transformer controller (about 120 Hz). The 2B-class version has 2.2B parameters. Tested on the GR-1 humanoid.
Gemini Robotics 2 Google DeepMind July 2026 Three models: a VLA for whole-humanoid control, an embodied-reasoning model (ER 2) available through Google AI Studio, and an on-device model that DeepMind says adapts to new two-arm robots with fewer than 200 examples. VLA models are available to early-access partners. Demonstrated on Apptronik Apollo 2.

Parameter counts and speeds are as published at each release and may have changed. Benchmark comparisons are the developers’ own results, not independent tests.

Video: Google DeepMind. “Gemini Robotics 2 brings whole body intelligence to robots” from the official Google DeepMind YouTube channel (watch on YouTube).

The bottleneck is data

VLAs need examples of robots doing tasks. Common sources are:

  • Teleoperation: a person controls the robot while it records. Figure’s Helix used about 500 hours of this.
  • Shared datasets: OpenVLA used roughly 970,000 demonstrations from the Open X-Embodiment collection of many labs.
  • Simulation: useful for control and motion practice, harder for realistic contact.
  • Human video: abundant, but it does not show the robot’s own forces and joints.

The amount and quality of this data is often cited by developers as a main bottleneck, and several companies say they are building robot fleets partly to collect it. We cover the data bottleneck in humanoid robot training data.

What VLAs still can’t do well

  • Tasks are still narrow and short. NVIDIA’s GR00T N1 paper describes short-horizon tabletop tasks as a limitation.
  • Dexterity. DeepMind itself says multi-finger dexterous manipulation “remains challenging” and that movement speed needs work.
  • Cost and latency. RT-2’s largest model ran at 1–3 Hz; newer designs address this by splitting fast and slow parts.
  • Demos versus reliability. Success rates in papers are often below 100 percent. OpenVLA’s authors report typical success below 90 percent on many tasks.
  • Safety. DeepMind released a safety benchmark, ASIMOV-Agentic, with Gemini Robotics 2; independent evaluation is still developing.

Four questions for any VLA demo

  1. Is the task shown autonomously, or with a remote operator?
  2. Was it tested in the training environment or a new one?
  3. How many trials, and what was the success rate?
  4. Is the model available (open weights, API, early access) or only shown in a video?

For how these models sit inside broader robot AI stacks, see what a robot foundation model is and NVIDIA Isaac GR00T explained. For Gemini Robotics 2’s three-model split, see Gemini Robotics 2 explained. For the body those models control, see our humanoid robot explainer.

The open question is whether the data problem gets solved by brute force or by cleverness. Several companies say they’re building robot fleets partly to collect training data. The next generation of VLA papers, and whether DeepMind opens Gemini Robotics 2 beyond early-access partners, will show which approach is winning.

Frequently asked questions

Is a VLA the same as a large language model?

No. An LLM works with text. A VLA also takes images and produces robot actions, and is usually built on a vision-language model with an added action output.

Do VLAs run on the robot or in the cloud?

Both exist. Figure says Helix runs on onboard embedded GPUs, and DeepMind offers an on-device Gemini Robotics model. Larger reasoning models may run elsewhere.

Are VLA models open source?

Some are. OpenVLA and NVIDIA’s GR00T N1 have been released openly; others, such as Gemini Robotics 2, are available through early access or limited programs.

Related reading

Sources

  • Brohan et al., “RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control,” arXiv:2307.15818: arxiv.org/abs/2307.15818
  • Kim et al., “OpenVLA: An Open-Source Vision-Language-Action Model,” arXiv:2406.09246: arxiv.org/abs/2406.09246
  • Black et al., “π0: A Vision-Language-Action Flow Model for General Robot Control,” arXiv:2410.24164: arxiv.org/abs/2410.24164
  • NVIDIA, “GR00T N1: An Open Foundation Model for Generalist Humanoid Robots,” arXiv:2503.14734: arxiv.org/abs/2503.14734
  • Figure, “Helix: A Vision-Language-Action Model for Generalist Humanoid Control” (Feb 20, 2025): figure.ai/news/helix
  • Google DeepMind, “Gemini Robotics 2 brings whole-body intelligence to robots” (July 30, 2026): deepmind.google
  • Boston Dynamics, “Robot Hands for Modern AI and Real Work” (whole-body control trained with RL in simulation): bostondynamics.com

Last updated: October 8, 2026. To report an error, see our corrections page. See our editorial policy.

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Advertisement