PaperGym: Rubric-Centered Evolution for Research-Plan Generation
Yuhan Wang, Zhengxi Lu, Yuchen Yan, Kaitao Song, Wenqi Zhang, Weiming Lu, et al.
arXiv:2608.31119v1Research planning is one of the hardest jobs we ask AI to do, because unlike math problems or code bugs, there is no single correct answer to optimize against. This paper tackles that missing piece by turning scientific papers into training environments for research-plan generation. The key idea is simple but powerful: use the paper’s goal and background to synthesize the question, and use the method and experiments to build a rubric that acts like a critic. That rubric is then used twice, first to guide a self-teaching step and then as the reward signal in reinforcement learning. The authors also reduce a common failure mode called criterion leakage, where the model can win by paraphrasing the source instead of reasoning about it. Across several Qwen models, PaperGym improves performance on multiple benchmarks and even helps a much smaller model beat a far larger competitor on ResearchQA. If it holds up, this could make AI systems much better at proposing real research ideas instead of just sounding plausible.
Also spotted that day
Previous daily papers