Daily AI Paper

Today’s pick is MLExpert, a new agentic model for autonomous machine learning engineering. The problem it tackles is surprisingly hard: if you want an AI system to improve other AI systems, it has to run long experiment loops, remember what happened in earlier trials, and learn from noisy feedback that arrives only after many decisions. MLExpert is trained for exactly that kind of workflow. The authors build a task factory for executable machine learning challenges, then use hierarchical supervised fine-tuning to teach general tool use and coding skills, followed by a reinforcement learning method that groups the decisions between experiment results into windows and rewards the whole window based on outcome. This lets the model learn from long-horizon cause and effect instead of isolated steps. On benchmark tasks, MLExpert roughly doubles the base model’s medal rate and reaches performance comparable to much larger frontier systems. That matters because it points toward AI agents that can conduct real research and engineering loops, not just suggest code snippets. In the method section, the authors make an important distinction between ordinary code generation and agentic machine learning engineering. In a normal coding benchmark, the model gets a prompt and produces an answer. Here, by contrast, the model is placed inside an environment where it has to manage a whole optimization process. It sees the task description, the public data, and whatever history it has retained from earlier attempts, then decides whether to inspect the data, rewrite the pipeline, rerun experiments, or change direction entirely. That distinction is central, because in machine learning engineering the best solution is rarely produced in one shot. It usually emerges through repeated cycles of analysis, implementation, execution, and correction. The paper’s formulation captures this by treating the trajectory as a sequence of observations and actions, with the final answer chosen only after the agent has accumulated enough evidence from the interaction history and the workspace state. A useful detail here is that the experimental state is not trapped inside the language model’s context window. Instead, it lives in an external workspace, which stores code, execution logs, and candidate solutions. The model then accesses that state through tools. That design matters because long-horizon machine learning work quickly exceeds the memory limits of a single prompt, and because the model needs to revisit earlier experiments, compare branches, and preserve a submission candidate while exploring alternatives. The interaction harness is built around that reality. It is not a fixed workflow that tells the model exactly what to do next. Rather, it is a ReAct-style loop that keeps the branching decisions inside the model itself. The agent can analyze, revise code, execute a candidate, fetch history, invalidate a suspicious node, or update the current answer. In other words, the system is designed to support research-like behavior: hypothesis, test, inspect, revise. The Kaggle-MLE Factory is another key piece of the puzzle. Instead of relying on synthetic toy tasks, the authors convert historical Kaggle competitions into executable machine learning engineering problems. That is a smart choice because Kaggle tasks already contain the kinds of variation that make real MLE difficult: different data modalities, different objectives, different file formats, and different evaluation protocols. But those competitions are not immediately ready for agent training, so the factory uses a multi-agent pipeline to discover, analyze, generate, validate, and repair tasks. One agent filters competitions by status, scale, and submission format. Another extracts the structure of the task, including fields and modality characteristics. Another prepares the data, constructs a held-out test split with hidden labels from the original labeled data, and implements the evaluator. Then the task is actually executed and checked for data integrity, submission format, and evaluation behavior. That final validation step is important because agent training is only as good as the tasks being run. If the environment is brittle, inconsistent, or leaky, the model can learn the wrong lessons. The training strategy is also layered quite carefully. The first stage of hierarchical agentic supervised fine-tuning is meant to transfer generic tool-use and code-revision ability from broader agentic data. That makes sense because a model that cannot reliably call tools, edit code, or follow multi-step instruction chains will struggle before it ever reaches the domain-specific MLE part. The second stage then shifts toward machine learning engineering itself, using teacher trajectories from the MLE domain and from related scientific tasks. This is where the model starts to learn how to reason about algorithms, interpret experimental results, and iterate toward better performance. The paper emphasizes that the training data are not just successful final solutions. They also include the prefixes of trajectories that begin with task understanding and lead into the first complete solution, as well as longer optimization trajectories that show how an agent improves a pipeline over time. That gives the model examples of both initial construction and sustained refinement. What makes the supervised stage especially interesting is the outcome-guided selection process. Long trajectories are messy. They often contain genuine exploration, but they can also include mistaken turns, wasted experiments, or degradation after the model has already found a strong candidate. Rather than using every step indiscriminately, the authors inspect execution results and keep the parts of the trajectory that are actually informative. If a trajectory passes a quality threshold, they identify the first best-performing submission and then look at what happens afterward. If performance later falls off too much, they keep a prefix anchored around that best point and cut away the degraded tail. If not, they can keep the last submission. This is a practical way to preserve useful reasoning while avoiding training on long stretches of drift or failure. It also reflects a broader lesson in agent training: more data is not always better if the data includes many low-quality decisions after the model has already lost its way. The reinforcement learning stage, WIRL, is where the paper directly addresses the challenge of delayed feedback. In long MLE runs, the quality of a result is only visible after a whole chain of decisions has already been made. If you try to assign credit one step at a time, the signal becomes noisy and ambiguous, because any given result may depend on several earlier edits, analyses, or execution choices. WIRL handles this by grouping the decisions between two successive experiment results into an execution window. The entire window then receives a shared reward based on the outcome. That is a neat compromise between fully sparse end-of-trajectory reward and unrealistic per-token supervision. It gives the model a way to connect a cluster of decisions to a concrete experimental result, which is closer to how real engineering feedback works. If a model rewrites preprocessing, changes the validation split handling, and adjusts the training loop before the next run, the improvement or failure should be attributed to that bundle of actions rather than to one token in isolation. The asynchronous part of the method is equally important. In a real online setting, trajectories keep unfolding while the policy is being updated. That creates policy mismatch: the model that collected the data is not exactly the model being trained. The paper addresses this with asynchronous collection and policy correction, allowing updates before the full trajectory is finished. This is especially relevant for long-horizon experimentation because waiting for an entire run to end before learning from any of it would be too slow and would waste a lot of potentially useful feedback. By correcting for policy changes during collection, the method can keep learning from fresh experience without completely breaking the link between the collected behavior and the current policy. The reported results suggest that this combination works. On MLE-Bench Lite, MLExpert raises the average medal rate of the base model from 37.9 percent to 72.7 percent. That is a substantial jump, and the authors note that it brings the system into the range of much larger frontier models such as Kimi-K3 and GLM-5.3. What is notable here is not just the absolute score, but the implication that a 35B Mixture-of-Experts agent, when trained with the right long-horizon supervision and reward structure, can compete with systems that are much larger. In a field where scale often dominates the conversation, that points to the value of better training design, better interaction environments, and better credit assignment. The paper also claims generalization beyond the benchmark it was trained around. MLExpert improves on NatureBench Lite, which suggests that the learned optimization behavior is not limited to Kaggle-style competitions. It also transfers to agent scaffold optimization and algorithm optimization tasks. That is an important broader signal, because it means the model is not merely memorizing a narrow benchmark template. Instead, it appears to have learned a more general capability for iterative problem solving, tool-mediated experimentation, and improvement under feedback. In practical terms, that kind of capability could be useful anywhere an AI system needs to tune pipelines, compare candidate solutions, or manage repeated trials under a budget. There is also a bigger research context here. For years, a lot of work on LLM agents has focused on short-horizon reasoning, tool use, and code generation. Those are necessary ingredients, but they are not sufficient for autonomous machine learning engineering. Real MLE involves planning under uncertainty, choosing when to explore versus exploit, and handling feedback that is delayed, partial, and sometimes misleading. It also involves knowing when to preserve a promising branch and when to abandon a dead end. This paper is trying to move agents into that more realistic regime. In that sense, it sits at the intersection of agentic coding, automated experimentation, and self-improving AI systems. That is why the interaction harness matters so much. By keeping an experiment tree with parent-linked revisions, the system gives the agent a structured memory of its own search process. The model can revisit earlier nodes, compare branches, and decide whether a line of attack is still worth pursuing. This resembles how human machine learning engineers work when they keep notebooks, experiment logs, and versioned code repositories. The difference is that here the process is being learned by the model itself, rather than scripted entirely by the environment. The ability to invalidate a node is also a nice practical safeguard. When validation looks too good to be true, or when leakage and overfitting are suspected, the agent can mark that branch as unreliable instead of being trapped by a misleading score. From a systems perspective, the paper is also addressing a very real bottleneck in agent training: context length. Long trajectories do not fit neatly into a single prompt, and even when they do, the important parts may be buried under a lot of irrelevant chatter. The authors handle this by summarizing older messages while retaining recent interactions, and by allowing a separate analysis sub-agent to do deeper inspection with its own context. That division of labor is interesting because it mirrors how a human team might work: one part of the system maintains the active optimization loop, while another part performs focused analysis and returns a compact summary. This can help the main model stay oriented without losing the ability to inspect data or diagnose failures. The broader implication is that autonomous MLE may be one of the most promising testbeds for AI4AI and recursive improvement research. It is concrete enough to evaluate, because tasks have executable code and measurable scores, but complex enough to require genuine long-horizon reasoning. Success in this setting suggests that models can learn not just to answer questions, but to run experiments, interpret outcomes, and improve their own solutions over time. That does not mean recursive self-improvement is solved, of course. There are still major challenges around robustness, safety, evaluation leakage, and the reliability of open-ended agent behavior. But a system like MLExpert shows that post-training methods can make a meaningful difference in how well models manage sustained optimization. It is also worth noting that the contribution here is not just the final model, but the training recipe and infrastructure. The Kaggle-MLE Factory, the interaction harness, the hierarchical SFT scheme, and WIRL together form a pipeline for studying long-horizon agentic learning. That makes the work valuable beyond the reported benchmark numbers, because other researchers can build on the same kinds of task construction and credit assignment ideas. In the long run, systems like this could be used to automate parts of model development, assist with hyperparameter search, generate stronger baselines for scientific competitions, or help teams iterate on data pipelines and evaluation strategies faster than before. So the headline is not simply that MLExpert is a stronger benchmark agent. It is that the paper takes a meaningful step toward training models that can participate in the full experimental rhythm of machine learning engineering. The model is taught to keep state, manage evidence, learn from intermediate results, and improve across a sequence of costly experiments. That is exactly the kind of capability you would want if AI systems are going to help build better AI systems in a sustained, autonomous way.