Daily AI Paper

A model is only half of an AI system. The other half is the inference strategy: the prompt, the reasoning procedure, the sequence of model calls, and the way those calls are stitched together into an answer. That second half matters enormously, because it determines how a model is actually used in practice. Yet in most systems, it is still handcrafted. A person designs a strategy, tests it, tweaks it, and then moves on. The design does not accumulate. It does not improve from one task to the next. This paper is about changing that. The core idea is to treat reasoning strategies the way we already treat programs: as things that can be written, reused, composed, evaluated, and evolved. That sounds natural, but it requires a representation that is both flexible and reusable. If the strategy format is too rigid, you can only express a few fixed patterns. If it is just a blob of code, you can express anything, but the result becomes isolated and hard to build on later. The paper proposes a middle ground called the Strategy Blueprint. A Strategy Blueprint is an executable Python program that captures the structure of a reasoning procedure without locking in the task-specific prompts. Every blueprint exposes the same interface, so any blueprint can be used as a component inside another. In other words, blueprints are not just one-off scripts; they are reusable building blocks. The paper gives simple examples like a loop that generates several candidate answers, reviews them, and then majority-votes. More generally, a blueprint can choose which model to call, in what order, and at what “tier” of effort. The system allows three tiers of models—fast, balanced, and complex—and each can run in a non-thinking or thinking mode. That matters because not every step in a reasoning workflow needs the most expensive model. Some steps can be cheap, while others can be more powerful, and the strategy can route between them internally. On top of this representation sits Strategist, the meta-agent. Strategist does not solve the user’s task directly. Instead, it designs the strategy that will solve the task. Given a task and a small development set, it runs a write-try-adjust loop at scale. It retrieves relevant blueprints from a shared library, proposes candidate strategies, evaluates them on real samples, reads the traces and scores, revises the strategies, and repeats. The important twist is that every candidate goes back into the library, so the library grows over time. This makes the process cumulative rather than resettable. The library itself is a central part of the contribution. It stores blueprints, along with measurement cards and experience notes describing when they work and when they do not. Search over the library is hybrid: it uses both TF-IDF and embedding similarity, and it searches not just the blueprint structure but also the measured behavior and prior analyses. The library also returns “adjacent inspiration,” meaning blueprints from other domains or reasoning styles that may spark a useful new design. There are a few rules to keep the library healthy: a shared interface, no forward calls, no cycles, and no duplicates. Because every new blueprint can wrap or extend an existing one, the library compounds. Early blueprints are simple; later ones build on those and become deeper and more specialized. The search-and-evolve loop has four stages. First, retrieve relevant blueprints and inspiration from the library. Second, propose candidates. Each iteration includes three kinds of candidates: a brand-new blueprint, a composed blueprint that extends an existing one, and a reuse candidate that directly applies an existing blueprint with new prompts. This balance is intentional: it keeps the system exploring while also exploiting what already works. Third, evaluate the candidates. Here the paper is careful about separating levels. At the inner, agent level, each candidate runs on actual samples and the system observes the full reasoning trace, execution status, and scores. At the outer, meta level, Strategist uses those observations to improve the next round of proposals. The data is split into training, validation, hidden, and test sets. The training split exposes full traces; validation hides the traces and shows only aggregated scores; hidden data is used to choose the winner across iterations; and test data is held out entirely until the end. Fourth, accumulate. Strategist writes an analysis of what each candidate structurally did, then adds the underlying blueprints to the library with their measurements and notes. That is the mechanism that makes later tasks benefit from earlier ones. The experiments are broad. The paper evaluates on 40 benchmarks spanning seven categories, including math, code, commonsense, multi-hop QA, knowledge, logic, and recommendation. Thirty benchmarks are used for the main results, and ten are held out for one-shot retrieval tests. Each benchmark gets training, validation, and hidden splits of up to 100 samples each, plus a fixed 200-sample test set. The task models are three Qwen3 sizes—0.6B, 1.7B, and 4B—each in thinking and non-thinking mode, giving six execution configurations. Strategist itself is a separate frozen GPT-6 model used only to write strategies. It never answers the benchmark questions directly. The baseline comparison is quite strong: 18 baselines, all using the same task models and no internet access. These span direct prompting and more complex multi-agent workflows, and each is run across all six model configurations. For each benchmark, the paper defines a baseline cell, BL, as the most accurate cell on the training split, then scores it on the test set. That mirrors a realistic deployment choice: pick the best-looking method on a small development set, then apply it to unseen data. The paper measures both accuracy and weighted cost. Cost is token usage weighted by model tier, reflecting the fact that larger models are more expensive to run. The main result is that Strategist is not just competitive, but often better. Across the 30 main benchmarks, it wins on 24. In the remaining cases, it either ties or trades off accuracy and cost. On average, it matches the strongest baseline’s accuracy while reducing weighted cost by 39 percent. That is the headline: same or better quality, substantially lower inference cost. The paper also notes that some wins come from fully new strategies designed by Strategist, while others come from simpler baseline-style scripts discovered during search that happen to match the baseline’s accuracy at lower cost. So the system is not merely inventing exotic workflows; it is also finding leaner ones. The benchmark-level details help explain what this means. On tasks like SQuAD v2, TruthfulQA, 2WikiMQA, Amazon-Rec, SAT-Math, and the LSAT datasets, Strategist improves accuracy while also cutting cost substantially. On tasks where the models are already near their ceiling, such as MMLU, GSM8K, ASDiv, SciQ, and MMLU-STEM-Hard, it mostly preserves accuracy and focuses on reducing cost, sometimes by very large margins. It is not Pareto-improving everywhere. On MedQA and AIME 2024, it is cheaper but less accurate than the baseline. On HotpotQA and COPA, it matches accuracy but uses slightly more cost. So the system is powerful, but not magical. A particularly interesting test is one-shot library retrieval. After the 30 main benchmark runs, the library has grown to 1,865 blueprints. The authors then ask whether a future task can skip the full search process. On 10 held-out benchmarks, Strategist retrieves from the final strategies of the main searches, instantiates the top five blueprints, evaluates them on a small development set, and runs the chosen one without any additional search. Even in this reduced setting, it stays within two points of the baseline on all 10 benchmarks and exceeds it on 8, with better average accuracy overall. That suggests the library is doing real work: it is not just storing artifacts, it is providing transferable reasoning structure for new tasks. The paper also tests cross-model transfer. The same discovered strategies are run, without changing the blueprint code or prompts, on GPT-5 and Gemini-2.5 model families that Strategist never saw during search. The strategies still perform well: they win on most of the transferred benchmarks, and the aggregate cost in USD drops as well. This is important because it suggests the blueprints are not overfit to one executor. They capture task structure, not just quirks of a particular model. There are a few limitations to keep in mind. First, Strategist still relies on a development set and on expensive search-time evaluation, so the search process itself is not free. Second, the system’s success depends on the quality of the library and the retrieval mechanism; if the library does not contain useful building blocks, the search starts from a weaker position. Third, the paper’s results are broad but still benchmark-based, so real-world deployment could surface additional complexity, especially for tasks with shifting distributions or very long-horizon interactions. And while cross-model transfer is promising, the paper only tests a limited set of model families. Still, the main takeaway is clear. This work moves optimization from the model alone to the whole agent system. Instead of treating reasoning strategy as a one-off artifact, it turns it into something that can evolve cumulatively across tasks. In that sense, Strategist is trying to make the rest of the AI system learn the way models already do: by building on what came before.