Daily AI Paper

Today’s pick is TTPO, or Test-Time Policy Optimization. The problem it tackles is simple to state but hard to solve: can a language model keep improving at test time, without access to ground-truth answers? That matters because many powerful post-training methods depend on labeled data, which is expensive or unavailable in real deployment settings. TTPO starts from a noisy substitute for labels, using majority vote among multiple rollouts. The key insight is that even when the vote is wrong, answers that disagree with it are usually wrong too, so disagreement is still useful training signal. TTPO combines two updates: it distills the rollouts that agree with the pseudo-label, and it applies reinforcement-style penalties to confident disagreements. It also selects tokens more carefully, so it focuses learning where the model is still uncertain. The result is a label-free method that matches label-supervised systems on hard reasoning benchmarks and can significantly boost small models at inference time.