Daily AI Paper

Robotic foundation models are getting stronger, but choosing among them is still surprisingly expensive. The most trustworthy evaluation is usually a closed-loop rollout: let the policy act, see whether it succeeds, and repeat across many tasks. That works, but it is slow, noisy, and tightly tied to the benchmark and robot setup. So the practical question is: before we spend all that time on real or simulated rollouts, can we estimate which model is likely to work best from offline data alone? This paper argues that the answer is yes, and it does so with a new idea called Sensorimotor Alignment, or S-squared. The core intuition is simple. Existing offline scores for robot models often look at the wrong thing. Vision-language models may be scored by question answering ability, and video models may be scored by pixel-level quality. But a robot policy does not just need to perceive or generate realistic-looking content. It needs to map what it sees and reads into the motion it should execute. Two scenes can look very different but require the same action, and two scenes can look similar but require different trajectories. So the representation has to organize experience in a way that reflects action, not just appearance. That idea connects to the Platonic Representation Hypothesis, or PRH. PRH says that as models improve, different modalities like vision and language begin to approximate the same underlying structure of reality in their own representation spaces. This paper extends that thought to robotics. The authors ask whether physical motion is another kind of projection of the world, and whether stronger embodied models should have sensory representations that line up more closely with the geometry of the motions they produce. In other words, if a model truly understands the embodied task, then episodes that are close in sensory space should also be close in motor space. To test this, the paper defines a training-free offline metric called Sensorimotor Alignment. The setup uses datasets that pair sensory inputs with real robot motions, specifically DROID and BridgeDataV2. For each episode, they sample an image frame and pair it with the task instruction. That image-text pair is encoded by the model’s vision and text encoders to produce a sensory embedding. Separately, they take the end-effector trajectory after that frame — positions, orientations, and gripper state over the next several steps — and treat that as the motor representation. The key question is whether these two spaces organize episodes similarly. To measure that, they compare neighborhoods rather than trying to directly regress one representation onto the other. For each episode, they find its k nearest neighbors in sensory space using cosine distance, and its k nearest neighbors in motor space using Dynamic Time Warping, which is useful because trajectories can unfold at different speeds. Then they measure how much the two neighbor sets overlap. If episodes that are close in sensory space are also close in motor space, the overlap is high, and sensorimotor alignment is strong. The reported version uses a centered, normalized neighborhood-overlap score with ten neighbors. That is the metric. The important question is whether it actually predicts anything useful. The paper tests that across several embodied model families and in both simulation and the real world. The headline result is that higher S-squared tends to go with higher downstream task success. First, they evaluate Vision-Language-Action models in simulation. Across 12 VLAs, S-squared correlates strongly with success on SimplerEnv, with a Pearson correlation of 0.717. That is a substantial positive relationship, and the paper says it is robust across different reference datasets, model families, scales, and training setups. So the metric is not just tracking one particular architecture or one particular benchmark; it appears to capture something more general about embodied competence. Next, they ask a more interesting question: can you compute the metric before robot fine-tuning, using only a raw vision-language model backbone, and still predict how good the downstream VLA will be after fine-tuning? The answer here is also yes. For 8 raw VLM backbones from VLM4VLA, S-squared on the backbone itself correlates with the success rate of the VLAs later fine-tuned from those backbones, with correlation 0.681. That matters because it turns S-squared into a selection tool. You could use it to rank candidate backbones before paying the cost of robot training. They then test world action models, which are based on video generators rather than VLMs. On 6 WAMs evaluated with the same ALOHA trajectories, S-squared again correlates strongly with downstream success on RoboTwin 2.0, with correlation 0.718. And they also test real-robot policies using a unified setting from MolmoB0T: five policies, forty tasks, and public checkpoints scored on the same DROID reference data. There, S-squared correlates with physical-robot success at 0.831. That is especially notable because real-robot evaluation is the hardest and noisiest setting, yet the offline metric still tracks performance well. The paper also compares S-squared to more conventional offline metrics, and this is where the contribution becomes especially clear. For VLM backbones, aggregate VQA scores show almost no relationship with downstream VLA success, with correlation -0.154. That is consistent with prior work showing that language or question-answering ability does not necessarily tell you whether a model will become a good robot policy. S-squared, by contrast, is strongly positive. For WAMs, S-squared is also compared with standard video quality metrics like FID, SSIM, and PSNR. Those metrics measure how realistic or similar predicted video frames are, but they do not directly reflect whether the model’s latent space is organized around the motions the robot needs to take. S-squared does better there as well. So what is actually being shown? Not that S-squared magically solves robot evaluation, but that it captures a different and more relevant property than passive perception or pixel fidelity. It seems to measure whether the model’s internal sensory space is already shaped by the structure of action. That makes it a plausible proxy for embodiment quality. The paper also uses S-squared to explain some previously observed behaviors at the representation level. One example is that fine-tuned VLAs often seem to ignore language and act mostly on vision. The authors find that task success correlates with the S-squared of the vision features, not the text features, which helps explain why visual structure may dominate once the model is adapted for action. More broadly, the metric gives a representation-level account of phenomena that were previously known only from closed-loop outcomes. There are, of course, limitations to keep in mind. The metric depends on having a reference dataset with paired sensory and motion data, and it is computed on neighborhoods in a particular evaluation space, so the result can depend on the choice of dataset, distance metric, and sampling protocol. The paper does check robustness to design choices, but it is still an offline proxy, not a replacement for real rollout evaluation. Correlation is also not causation: a high S-squared score does not prove that alignment causes success, only that they move together in the tested settings. And the paper’s strongest claims are empirical, so the metric should still be validated carefully when moving to new robots, new tasks, or different action distributions. Still, the main takeaway is compelling. The paper extends the Platonic Representation Hypothesis into embodied AI and shows that sensorimotor alignment can be measured directly. In practical terms, S-squared offers a cheap way to rank models before expensive closed-loop testing. In scientific terms, it suggests that successful robot policies are not just good at seeing or generating; their sensory representations are organized in a way that mirrors the geometry of physical motion. That gives us a more principled way to understand why some embodied models succeed, and a useful proxy for finding them sooner.