Daily AI Paper

Today’s pick is TTPO, or Test-Time Policy Optimization. The problem it tackles is a big one for reasoning models: how do you keep improving at test time when you don’t have ground-truth answers to train on? Existing methods often depend on labels, or they use majority vote as a pseudo-teacher, but that can go badly wrong if the vote is incorrect. TTPO’s key insight is that disagreement is still useful signal. If a rollout disagrees with the pseudo-label, it is usually wrong, so the method can learn from both sides at once: it distills the agreeing answers and penalizes the disagreeing ones, while focusing updates only on the tokens that matter most. That makes the self-training process much more robust. The result is a label-free approach that matches supervised methods on hard reasoning benchmarks and boosts performance substantially, even on smaller models. It matters because it points toward cheaper, more autonomous model improvement after deployment.