01What Spatial Intelligence Means
Spatial intelligence is the ability to reason about a scene the way a physicist would: as a set of objects with positions, orientations, velocities, and physical relationships to each other. It is a specific stance about how a machine should model the world — not a bag of features layered on top of a vision-language model.
In academic settings the term is often used loosely to describe any 3D perception task. For robotics, we mean something tighter: the model outputs realtime spatial state — position, orientation, and velocity of the relevant objects in a scene — and a downstream policy acts on that state.
02Why Pixels Are the Bottleneck
Language models tokenize text because language is discrete. Physical intelligence (PI) models copied that recipe and tokenized every pixel of every frame. It is the wrong analogy. Language is a lossy encoding of thought; the physical world is already structured by geometry and physics, and those structures are cheap to compute compared to attending over every pixel.
Pixel-token models pay for that mismatch three times over:
- Training cost. Attention over pixel tokens scales quadratically. Most of the compute goes to relearning what a rigid body is.
- Inference latency. Realtime spatial state — the state a controller actually needs — becomes impossible to compute on-edge at 30 FPS.
- Sim-to-real gap. Pixel distributions differ between simulator and the real world. Policies trained on pixels overfit to lighting, shaders, and sensor noise.
03Geometric Priming
A model is primed when it begins training already guided by rules of relevant geometry and physics. It does not have to discover that objects have surfaces, that surfaces have normals, or that rigid bodies conserve momentum. Those invariances come from a network — in our stack, a Geometric Invariance Neural Network (GINN) — trained on 3D data with depth.
Concretely, geometric priming lets a robot policy start from a world that already has structure and spend its GPU cycles solely on the skills required to master a task. The pretraining bill is paid once, at the model level, not per deployment.
04The Pipeline: Prime, Train, Execute
Spatial-first robotics stacks tend to fall into three stages. They are worth naming because papers frequently collapse them and hide the tradeoffs.
Prime with geometry
A spatial model is pretrained on 3D data. It learns to output geometric features — surfaces, edges, poses — that are invariant to lighting and viewpoint. This is the priming stage.
Train the skill
A focused action model is trained on top of the primed features. Because it does not have to learn the world, it can be small, task-specific, and cheap to fine-tune on new hardware.
Execute with mastery
At inference, the pipeline emits realtime spatial state and the action model consumes it. Because the representation is geometric, the sim-to-real gap collapses toward zero and control runs on edge hardware.
05What Changes for Practitioners
If you are building a policy today, adopting a spatial-first approach usually means three concrete shifts:
- Represent state, not frames. Feed the policy explicit object poses and velocities, not raw camera streams.
- Move perception upstream. Treat the spatial model as pretraining infrastructure. Fine-tune the small policy, not the perception stack.
- Budget for the edge. Because state is compact, control loops can close on a device instead of round-tripping through a datacenter.
06Frequently Asked
Is spatial AI the same as computer vision?
No. Classical computer vision is upstream of spatial intelligence — it recovers pixels, features, or depth. Spatial AI is the layer above: it produces a structured, physical description of the scene a policy can act on.
Do I still need a large language model?
For task specification and semantic reasoning, yes — but it can be a small, focused model. Language directs intent; spatial intelligence supplies the world.
How much 3D data do I need?
The pretraining is done at the model provider level. As a practitioner you consume the primed features and add task demonstrations on top — typically orders of magnitude less data than a pixel-token policy of the same competence.
07Further Reading
For a product-level view of how these ideas ship in MoreSight and GINN, return to the MORE, Inc. overview. The Tokenize Features and GINN sections show the same pipeline described here, applied to precision manipulation.
