What is a vision-language-action (VLA) model?

A VLA model is a robotics foundation model that ingests camera frames plus a natural-language instruction and emits robot actions end-to-end. It extends the vision-language-model recipe by adding an action head, tokenizing motor commands the same way language tokens are handled.

OpenVLA is an open-source 7B-parameter vision-language-action model trained on the Open X-Embodiment dataset. It became the reference baseline for VLA research because the weights, code, and finetuning recipes are public.

Why do compact VLAs like SmalVLA and TinyVLA exist?

Full-size VLAs are too slow and too large to run on robot-grade hardware at 30 FPS. Compact variants trade parameter count and vocabulary breadth for latency so a policy can actually close a control loop on the edge.

How does geometric priming compare to VLA?

A VLA tokenizes pixels and language, then predicts actions. A geometry-primed model like MoreSight extracts 3D state — position, orientation, velocity — from depth and feeds that compact state to a focused action model. The result is realtime control with far less data and no sim-to-real gap.

Contact

Guide · VLA

Vision-Language-Action
Models.

VLAs are the dominant recipe for robotics foundation models: take a vision-language model, bolt on an action head, and predict motor tokens from pixels and prompts. This guide covers the family, what OpenVLA established, why compact variants exist, and where a geometry-primed model wins instead.

01What a VLA Actually Is

A vision-language-action model is a single network trained to map (image, instruction) pairs to robot actions. The vision-and-language half comes from a pretrained VLM; the action half tokenizes motor commands — joint deltas or end-effector poses — into the same vocabulary so the model predicts them autoregressively.

The bet is the LLM playbook for control: pretrain once on large-scale robot data, then adapt with a small task set. When it works, one model handles many embodiments and open-ended instructions. When it doesn't, it's usually because pixels are a lossy substrate for realtime physics.

02The Model Family

Model	Params	Notes
RT-2	55B	Google DeepMind; co-fine-tunes a VLM on robot trajectories.
OpenVLA	7B	Open-source reference; trained on Open X-Embodiment.
π0 / π0-FAST	~3B	Physical Intelligence; flow-matching action head.
SmalVLA	~0.5B	Compact VLA aimed at edge inference.
TinyVLA	<1B	Distilled VLA for latency-sensitive control.
OpenVLA-OFT	7B	Optimized fine-tuning recipe; higher success on new tasks.

03Why OpenVLA Became the Baseline

OpenVLA (Stanford, 2024) is a 7B model trained on the Open X-Embodiment dataset — the largest open pool of teleoperation trajectories. The reason it dominates papers isn't raw performance; it's that the weights, tokenizer, and finetuning recipe are public. Follow-ups like OpenVLA-OFT show that the finetuning protocol matters more than parameter count once the substrate is fixed.

For teams evaluating VLAs, OpenVLA is the honest starting line. For teams that need 30 FPS control, it's also where the pixel-token cost becomes obvious.

04Why Compact VLAs Exist

SmalVLA, TinyVLA, and their cousins exist because a 7B model can't close a control loop on robot-grade hardware. Distilling to sub-billion parameters trades open-vocabulary breadth for latency. It's the same pattern LLMs went through — but robotics can't wait for a slower response the way a chat window can.

The lesson: parameter count is the wrong axis. What matters is whether the representation lets a policy act at the rate the physics demand.

05VLA vs. Geometry-Primed

Dimension	VLA	Geometry-primed
Input	RGB frames + language tokens	Depth-rich 3D data
Output	Tokenized action deltas	6DoF pose + velocity state
Typical rate	3–10 Hz on server GPU	30 FPS on edge
Data appetite	Millions of teleop episodes	Minutes to hours per skill
Sim-to-real	Sensitive to shaders and textures	State is identical in sim and real
Strength	Open-vocabulary intent binding	Realtime, precise manipulation

MoreSight™ inverts the VLA pipeline: GINN extracts 3D state from depth, and a compact action model consumes state instead of pixels. The two approaches aren't mutually exclusive — a VLA can name intent while a geometry-primed model closes the loop.

06When to Reach for Which

Use a VLA when — you need broad instruction following, semantic reasoning, or a research baseline on Open X-Embodiment.
Use a compact VLA when — you want a VLA-shaped policy but latency matters — expect to trade vocabulary breadth for FPS.
Use a geometry-primed foundation when — the task is precise manipulation, closed-loop control, or anything that has to survive lighting, texture, and shader changes on real hardware.
Combine them when — language must bind to physical state — 'grasp the red mug, tilted 15°'. The VLA supplies intent; the geometric model supplies the state to act on.

07Frequently Asked

Are VLAs the same as robotics foundation models?

VLA is an architecture; RFM is a role. Most RFMs today are VLAs, but a geometry-primed foundation is also an RFM — it just uses depth and 3D state instead of pixel tokens.

Do compact VLAs really run at 30 FPS?

Some approach it on dedicated inference hardware. Most still run in the single-digit-Hz range once you include perception, action tokenization, and post-processing. If 30 FPS on-robot is the requirement, plan for a state-based policy.

Why can't a VLA just be trained on depth?

It can — but tokenizing depth pixels inherits the same quadratic-attention bill as RGB. The gain comes from representing the scene as compact geometric state, not from swapping one image channel for another.

08Further Reading

Pair this with Robotics Foundation Models for the broader landscape, or Spatial AI for Robotics for the geometry-first case in depth. Or return to the MoreAI overview.

←BACK TO GUIDES

Vision-Language-ActionModels.