Daily AI Paper

Mobile manipulation is one of those robotics problems that sounds straightforward until you try to do it in the real world. A robot has to move its base through the environment, coordinate an arm, sometimes a torso or head, and do all of that while perceiving a messy, changing human-centric setting. That makes it a key step toward general-purpose robots, but also a very hard control problem. The challenge is not just perception or planning in isolation. It is the coupling between modules: if the base approaches a table a little off, the arm may suddenly be too far away, at the wrong angle, or forced into an awkward trajectory. In a tightly coupled system like this, small errors compound quickly. This paper is about how to fine-tune an already pre-trained mobile manipulation policy directly on a real robot, using only limited online interaction and sparse human guidance. That matters because pre-trained policies often work well in the narrow conditions they saw during training, but real deployment is full of variation: object positions shift, the robot starts from slightly different poses, and the world is less tidy than a benchmark. So some kind of post-training adaptation is usually needed. The difficulty is that real-world fine-tuning has to be sample-efficient, safe, and stable. You cannot collect huge amounts of robot data the way you might in simulation. And in mobile manipulation, the action space is high-dimensional and the modules are interdependent, which makes ordinary fine-tuning methods much less reliable. The authors place their work in the landscape between imitation learning and reinforcement learning. Supervised fine-tuning from demonstrations is stable, but it is limited by the amount and quality of expert data, and it tends not to explore beyond what was demonstrated. Reinforcement learning can explore and improve beyond the dataset, but in this setting it is difficult to apply directly, especially when the base policy is a modern generative model whose likelihood is hard to compute. That is one reason residual reinforcement learning is attractive: instead of replacing the existing policy, you freeze it and learn a correction on top. This gives you a practical way to adapt a strong prior without having to retrain everything from scratch. HiRe-MoMa, short for Hierarchical Residual Mobile Manipulation, builds on that idea but adds two important pieces tailored to the structure of mobile manipulation. First, it uses a policy-agnostic residual: the system learns additive corrections on top of a frozen base policy, so it can work across different kinds of pre-trained policies. The paper specifically says this makes the approach compatible with architectures such as Diffusion Policy and ACT. Second, it introduces hierarchical decoupling. Instead of treating the whole robot as one monolithic action vector, it separates the residual into a mobile-base part and a manipulator-arm part. The key idea is that the arm should be conditioned on the base’s correction, because physically the base sets up the arm’s reach. That unidirectional conditioning mirrors the robot’s actual hierarchy and is meant to reduce gradient interference between modules. There is also a third component that addresses sample efficiency in the real world: expert-guided value alignment. The authors use sparse human interventions not as full demonstrations, but as a signal to reshape the critic’s value landscape. In other words, when a human steps in during a difficult or risky moment, that intervention is treated as evidence about what behavior should be valued more highly. This is a clever middle ground between pure reinforcement learning and full imitation learning. The robot still explores autonomously, but the sparse expert corrections bias learning toward safer, more successful behavior. The paper emphasizes that this uses under 20% human coverage of the collected data. The overall training setup is an asynchronous actor-learner system running on a single real robot. An onboard rollout worker executes the current policy, while a central learner on a workstation updates the residual policy and critic. That matters because real-world robot learning needs to keep the robot operating while learning proceeds, and the asynchronous setup helps make the process practical. To understand the method more concretely, it helps to think about the residual policy as a small correction module sitting on top of a strong prior. The base policy proposes an action. The residual policy then adds a bounded correction, with the correction norm constrained so it does not drift too far from the base trajectory. The learning is off-policy and value-based, using temporal-difference updates and a conservative target derived from a critic ensemble. The paper also uses entropy regularization to keep exploration from collapsing too early, while still staying local to the base policy’s behavior. That local correction idea is important: rather than trying to rediscover the entire task from scratch, the system is learning how to nudge a mostly competent policy in the right direction. The hierarchical decoupling is the part that most directly addresses mobile manipulation. High-dimensional RL often struggles when all modules are optimized together, because the gradients for base and arm can interfere with one another. Here, the residual action is split into mobile and manipulator components. The mobile correction is treated as upstream context for the arm policy and its critic. So the arm’s refinement is conditioned on where the base is trying to go. This is a structural prior, but one grounded in the robot’s physical reality rather than in a hand-crafted task-specific controller. The authors suggest this design can extend naturally to deeper hierarchies as tasks become more complex. The expert-guided value alignment component uses a shared-control style setup. Human experts intervene during critical moments, and those interventions are stored in a dedicated buffer. During training, the system samples from this buffer in a balanced way so that the critic learns not just from task outcomes, but also from the expert’s corrective preferences. The paper frames this as reshaping the value surface, which is a useful way to think about it: instead of only rewarding final success, the learning process is nudged toward trajectories that a human would consider more promising or safer. The experiments are real-world, not simulated, and they are run on the TIAGo++ mobile manipulation platform. The paper says they evaluate three tasks of increasing complexity, though the extracted text does not list the task names. The important point is that these are genuine on-robot trials, and the method is tested across two different base policy architectures. That is a strong design choice because it checks whether the method is actually policy-agnostic rather than tuned to one specific pretrained model. The paper also emphasizes that the online interaction budget is small: roughly 25 to 40 minutes, with about 30 minutes highlighted in the abstract. That is a very limited amount of real robot time, which makes sample efficiency a central part of the claim. The reported results are that HiRe-MoMa consistently improves task performance over the frozen base policies, and it does so regardless of whether the base policy is Diffusion Policy or ACT. The authors present this as evidence that the method is robust to the choice of base architecture. They also claim that the expert-guided value alignment helps improve sample efficiency and accelerate convergence, especially when the human coverage is sparse. Taken together, the results suggest that the combination of residual adaptation, hierarchical decomposition, and sparse expert guidance is more effective than trying to fine-tune a whole mobile manipulation policy in a flat way. What do these results actually show? First, they show that real-world fine-tuning of mobile manipulation is feasible with a carefully structured residual approach. Second, they show that respecting the robot’s hierarchy matters: separating base and arm is not just an implementation detail, but a way to make optimization more stable. Third, they show that sparse human interventions can be used more strategically than simply as demonstration data. Instead of replacing RL, the human signal shapes where the critic thinks good behavior lies. At the same time, there are important limitations to keep in mind. The paper is still operating in a relatively controlled real-world setup on a specific platform, TIAGo++. That is valuable, but it is not the same as proving broad generalization across many robot embodiments, task families, or sensing setups. The method also assumes access to a reasonably competent frozen base policy to start from; it is a fine-tuning framework, not a solution for learning everything from scratch. And while the expert-guided component uses sparse interventions efficiently, it still depends on human availability and on the quality of those interventions. Another thing to watch is that the method is designed around the physical hierarchy of mobile manipulation. That is a strength, but also means its benefits may be strongest where the task structure is similarly hierarchical. Extending it to even more complex systems, or to settings where module dependencies are less clear, may require additional design choices. Overall, the paper’s main contribution is a practical recipe for real-world mobile manipulation fine-tuning: freeze a strong base policy, learn bounded residual corrections, split those corrections according to the robot’s hierarchy, and use sparse human interventions to guide the critic. The broader message is that in real robot learning, structure matters. If you want stable adaptation from limited data, it is not enough to throw reinforcement learning at the whole system. You need to respect how the robot is built, how its modules interact, and how little data you can actually afford to collect.