Daily AI Paper

This paper is about a very specific but increasingly important problem: can a language model not just help write machine learning code, but actually run the whole engineering loop on its own and get better at it over time? The authors frame this as AI for AI, or AI4AI, which is the idea that models can help build, tune, and improve other AI systems. That matters because a lot of real machine learning work is not one-shot code generation. It is long-horizon experimentation: you try an idea, run it, inspect the result, revise the code, try again, and keep going until time runs out. That long loop is exactly where current systems struggle. Many existing agentic machine learning systems still rely on a predefined workflow or an external search strategy to decide what happens next. In those systems, the model may write code or debug a step, but the overall experiment orchestration is often handled by hand-designed control logic. More recent systems go further and let the model maintain a research state and choose what to do next, but training a model for that kind of sustained optimization is hard. The challenge is not just generating a good answer. It is learning how to make a sequence of decisions under uncertainty, where each experiment result reflects several earlier choices. The paper identifies three core difficulties. First, you need an environment that preserves the state of the experiment while still letting the model choose the next action. Second, it is hard to collect good supervision from long trajectories, because these runs contain a mix of useful exploration, partial success, and sometimes late-stage degradation. Third, because each experiment is expensive and only reveals feedback after several decisions, reinforcement learning has to use intermediate signals before the full run is over, and it has to deal with policy mismatch when data are collected asynchronously. To address this, the authors propose a post-training recipe for autonomous machine learning engineering and use it to train MLExpert, a 35 billion parameter mixture-of-experts agentic model. The overall system has three pieces: a way to build executable machine learning tasks, a long-horizon interaction harness so the model can direct the experiment itself, and two training stages called hierarchical agentic supervised fine-tuning and Windowed Interaction Reinforcement Learning, or WIRL. A useful way to understand the setup is to think about machine learning engineering as an optimization problem over full pipelines. A task includes a description, data, and an evaluation metric. The agent can inspect the public data, write and revise code, run experiments, and maintain a candidate submission. The final answer is judged on hidden test labels. So the model is not solving a single coding prompt. It is trying to discover a working pipeline, improve it, and choose what to submit. The authors build what they call the Kaggle-MLE Factory to create these tasks from historical Kaggle competitions. That is important because Kaggle competitions are a rich source of real-world machine learning problems, but they are messy: different data formats, different submission rules, different evaluation protocols. The Factory turns them into standardized executable tasks with a public workspace, a hidden test split, and a unified evaluation interface. A multi-agent pipeline does the construction work: one agent filters competitions, another extracts the structure of the data and task, another prepares the split and evaluator, and validation plus repair steps check that the task actually runs correctly. The result is a set of standardized tasks that can be used for training and evaluation. The interaction harness is the other key infrastructure piece. It is designed to let the model manage a long experiment loop in a ReAct-style pattern, but without forcing a fixed workflow. Experimental states are stored in solution nodes, each with code, execution logs, and metadata. These nodes form a tree of revisions, so the model can revisit earlier branches, compare alternatives, backtrack, or change direction. The harness provides tools for analysis, code revision, execution, history retrieval, and submission management. It also supports summarizing older context when the interaction gets long, and it can delegate deeper analysis to a sub-agent with its own context. There is even a mechanism to invalidate nodes if the model suspects overfitting or data leakage, which is a practical safeguard in this kind of iterative optimization. On the training side, the first stage is hierarchical agentic SFT. The idea is to teach the model the general skills it needs before focusing on MLE-specific optimization. So the authors first use trajectories from diverse agentic tasks to learn tool use and code revision. Then they use teacher trajectories from machine learning and related scientific tasks to improve algorithm design, experimental analysis, and iterative optimization. They also include trajectories that cover task initialization, long-horizon improvement, and analysis sub-agent behavior. A particularly interesting part of the SFT process is how they choose supervision from long trajectories. Not every step in a long experiment is equally valuable. A run may start well, hit a strong solution, and then get worse because the model keeps tinkering. So the authors use execution results to identify the first best-performing submission, measure how much performance drops afterward, and decide whether to keep the prefix up to that anchor or use the last submission if later degradation is small. In other words, they try to keep the useful exploration and remove the degraded tail. Training is then applied only to the newly generated reasoning, text, and tool-call tokens, while the inherited context and tool feedback are used as conditioning, not as targets. The second stage is WIRL, which is their reinforcement learning method for long-horizon decision making. The key idea is to treat the sequence of decisions between two completed experiment results as a window. Instead of waiting until the very end of a long trajectory, the method assigns the same outcome-based reward to the decisions inside that window. This lets the model learn from intermediate experimental feedback, which is exactly what long MLE runs produce. The method also supports asynchronous collection and policy correction, so the model can keep updating even while trajectories are still being gathered. That matters because in long-running experimentation, you do not want to wait for every entire run to finish before learning from it. The paper evaluates MLExpert on MLE-Bench Lite, and also on NatureBench Lite, plus some transfer tasks involving agent scaffold optimization and algorithm optimization. The headline result on MLE-Bench Lite is strong: the average medal rate of the base model rises from 37.9 percent to 72.7 percent. The authors say this is comparable to larger frontier models such as Kimi-K3 and GLM-5.3. So the main takeaway is that training for autonomous long-horizon optimization really does change the behavior of the model in a meaningful way. What does that result actually show? It suggests that the model is not just better at writing isolated code snippets. It is better at the whole loop of analysis, revision, execution, and submission selection. Because the benchmark is built around iterative machine learning tasks, a higher medal rate indicates that the model is making better decisions across the experiment lifecycle. The paper also reports that the learned optimization capability transfers beyond the original benchmark: MLExpert improves performance on NatureBench Lite and on agent scaffold and algorithm optimization tasks. That is important because it hints that the model is learning a more general optimization skill, not merely memorizing one benchmark format. At a higher level, the paper’s contribution is not just the final model, but the training recipe and environment around it. The authors are making the case that if you want a model to become a generalist agent for autonomous machine learning engineering, you need three things together: executable tasks that look like real engineering work, an interaction environment that lets the model control the experiment loop, and learning methods that can extract signal from long, messy trajectories. There are also some clear limitations and things to watch next. First, the work is still centered on benchmarked machine learning engineering tasks, especially those derived from Kaggle-style competitions. That is a realistic and valuable setting, but it is still a subset of the broader world of machine learning practice. Second, the system depends on a carefully built interaction harness and task factory, which means the quality of the environment matters a lot. Third, long-horizon optimization is inherently expensive: it involves many executions, many intermediate states, and complex state management. Finally, although the paper shows transfer to other optimization tasks, the scope of that transfer is still limited relative to the full ambition of general autonomous AI engineering. So the big picture is this: MLExpert is a step toward models that can do more than assist with machine learning engineering. It is trained to manage the engineering loop itself, using execution feedback to guide multi-turn decisions over long horizons. The paper’s main message is that if we want agentic models that improve AI systems autonomously, we need to train them not just on answers, but on the process of experimentation, revision, and selection that produces those answers.