DR Pixel: Page-Native Deep Research with Vision-Language Models
Tiansheng Hu, Yichuan Wang, James V. Talwar, Nicholas Tomlin, Chen Zhao, Sewon Min
arXiv:2611.03097v1Deep research agents are designed to do something that sounds simple but is actually very hard: given a complex question, they search through a large body of evidence, read what matters, and keep reasoning until they can answer reliably. That works reasonably well on the open web, but scientific papers and other technical documents create a different challenge. The important evidence is not just in running text. It is in tables, figures, charts, and equations, often all on the same page. Yet most existing systems still flatten PDFs into plain text first, then search and reason only over that extracted text. The problem is that this pipeline throws away structure and introduces OCR and parsing errors. If a result is only visible in a figure, or if a table is misread, the agent may never recover the truth downstream. This paper asks a direct question: what if deep research were page-native? Instead of converting a paper into text, could an agent retrieve rendered page images directly and reason over those pages with a vision-language model? That would mean the page image itself is the retrieval unit, and the model sees the same visual evidence a human would see. The authors argue this is especially natural for scientific literature, where layout and visual elements carry meaning, and they build an end-to-end system around that idea called DR Pixel. To study this properly, they first point out that existing benchmarks are not quite enough. Some deep research benchmarks are text-only, so they do not test visual document understanding. Other multimodal document QA benchmarks do involve papers or reports, but the retrieval problem is often shallow: one search step is enough to find the answer. That is not really the same as deep research, where the agent may need many rounds of search and reasoning. So they introduce a new benchmark, DRPixelBench, built specifically for this setting. The benchmark is constructed from scientific literature, drawing from arXiv and open-access conference proceedings, and merged with papers used in three existing datasets so that everything shares one corpus of 99,055 papers. The questions are generated from abstracts, but the process is carefully designed so the question is not just a paraphrase of the abstract. First, the model extracts atomic facts from the full paper, including facts that come from tables or figures, and enriches the abstract with those facts. Then it writes a question from that enriched abstract alone, so the question reflects curiosity about the paper’s contribution rather than direct access to the evidence. After that, the model locates the supporting evidence in the paper and produces a short answer. Each question is labeled as either Text, meaning it can be answered from the running text, or Visual, meaning it requires a table, figure, or equation. The final dataset contains 500 QA pairs, and each pair is filtered and checked to make sure the question is plausible, the evidence is sufficient, the visual element is truly necessary when needed, and the answer is not something that could just be guessed from general knowledge. Human experts then review and rewrite the questions for naturalness. Now to the core method. DR Pixel is framed as a ReAct-style agent, which means it alternates between internal reasoning and external actions. At each turn, it produces a thought and then either a search query or a final answer. If it searches, the retrieval tool returns relevant evidence from the corpus; if it answers, the trajectory ends. The key difference from standard document agents is what the corpus contains and what the model sees. In a conventional pipeline, the corpus is parsed text, and retrieval returns text chunks. In DR Pixel, the corpus is 1.06 million rendered page images, and retrieval returns page images directly. Those images stay visual all the way through the trajectory; they are never converted back into plain text for reasoning. The agent can therefore use layout, tables, figures, and equations as they appear on the page. The authors evaluate this page-native setup against OCR-based baselines and find something striking: even without task-specific training, page-native deep research works better. Across GPT-5.4, Gemini-3.5-Flash, and Qwen3.6-27B, the page-native version improves accuracy by about 5.1 to 8.4 points over OCR-based pipelines, while also using 20 to 30 percent fewer retrieval calls. That matters because it suggests the visual pipeline is not just more faithful to the source material; it is also more efficient. In other words, once the system can directly inspect the page image, it may need less back-and-forth searching to find the answer. The paper also emphasizes that this is not just a retrieval story. The open-weight model, Qwen3.6-27B, still trails the closed models in the untrained setting, which motivates training. But training agents that make many tool calls is tricky. You can collect demonstration trajectories, but those trajectories are noisy: some steps are essential, while others are redundant, unhelpful, or even failed attempts. If you simply imitate every step equally, supervised fine-tuning can waste capacity on bad behavior. To address that, the authors introduce counterfactual credit reweighting. The idea is simple in spirit: estimate how much each step actually contributed to the final answer. For each reasoning or action step, they remove it from the trajectory and see how much the answer quality changes. If removing a step hurts the final answer, that step gets more weight during training. If it does not matter much, or if it was unhelpful, it gets less weight. This gives a finer-grained form of credit assignment while keeping the overall simplicity of supervised fine-tuning, rather than switching to a more complex reinforcement learning setup. With this training method, the gains are substantial. On DRPixelBench, training improves retrieval recall by about 15 points and final accuracy by more than 10 points. For Qwen3.6-27B specifically, the paper reports a 12-point improvement over the untrained model, which is about a 30 percent relative gain, and the trained model closes much of the gap to GPT-5.4. The improvements also transfer beyond the new benchmark: on four existing benchmarks, the trained DR Pixel outperforms the untrained baseline, and on three additional multimodal scientific literature benchmarks it remains competitive, even surpassing GPT-5.4 in retrieval on two of them. One especially notable result is on ViDoRe-V3, an out-of-distribution benchmark. There, the trained DR Pixel comes within 1.8 points of an oracle that is given the gold pages directly. That is a strong signal that the page-native approach generalizes beyond the exact benchmark it was trained on, and that the system is learning something useful about how to search and reason over visual documents rather than just memorizing one dataset. So what do these results actually show? First, they provide evidence that the conventional OCR-then-retrieve pipeline is not just a convenience choice; it can be a real bottleneck. When the answer depends on visual structure, flattening the document can lose information that matters. Second, they show that modern vision-language models are already strong enough to serve as the reasoning engine for page-native deep research, even before specialized training. Third, they demonstrate that careful credit assignment over agent trajectories can make a big difference, because not every step in a demonstration deserves equal imitation. There are still limitations to keep in mind. The benchmark is focused on scientific papers, so the conclusions are strongest in that domain, where pages are structured and multimodal evidence is common. The approach also relies on a large rendered-page corpus and on visual retrieval infrastructure, which may be more expensive or more complex than standard text indexing. And although the paper shows strong gains, the open-weight model still lags behind the best closed models in some settings, so there is room for better visual reasoning and better agent training. More broadly, page-native deep research is promising, but it will need to prove itself across other document types, longer contexts, and even messier real-world scans. Still, the main message is clear. If the evidence lives on the page, then treating the page as an image is not a workaround; it may be the better representation. DR Pixel shows that for complex document research, directly retrieving and reasoning over rendered pages can be both more effective and more efficient than forcing everything through OCR first.
Also spotted that day
Previous daily papers