Daily AI Paper

Daily AI Paper

2026-11-03

Archived
2026-11-03

DR Pixel: Page-Native Deep Research with Vision-Language Models

Tiansheng Hu, Yichuan Wang, James V. Talwar, Nicholas Tomlin, Chen Zhao, Sewon Min

arXiv:2611.03097v1

Deep research agents are designed to do something that sounds simple but is actually very hard: given a complex question, they search through a large body of evidence, read what matters, and keep reasoning until they can answer reliably. That works reasonably well on the open web, but scientific papers and other technical documents create a different challenge. The important evidence is not just in running text. It is in tables, figures, charts, and equations, often all on the same page. Yet most existing systems still flatten PDFs into plain text first, then search and reason only over that extracted text. The problem is that this pipeline throws away structure and introduces OCR and parsing errors. If a result is only visible in a figure, or if a table is misread, the agent may never recover the truth downstream. This paper asks a direct question: what if deep research were page-native? Instead of converting a paper into text, could an agent retrieve rendered page images directly and reason over those pages with a vision-language model? That would mean the page image itself is the retrieval unit, and the model sees the same visual evidence a human would see. The authors argue this is especially natural for scientific literature, where layout and visual elements carry meaning, and they build an end-to-end system around that idea called DR Pixel. To study this properly, they first point out that existing benchmarks are not quite enough. Some deep research benchmarks are text-only, so they do not test visual document understanding. Other multimodal document QA benchmarks do involve papers or reports, but the retrieval problem is often shallow: one search step is enough to find the answer. That is not really the same as deep research, where the agent may need many rounds of search and reasoning. So they introduce a new benchmark, DRPixelBench, built specifically for this setting. The benchmark is constructed from scientific literature, drawing from arXiv and open-access conference proceedings, and merged with papers used in three existing datasets so that everything shares one corpus of 99,055 papers. The questions are generated from abstracts, but the process is carefully designed so the question is not just a paraphrase of the abstract. First, the model extracts atomic facts from the full paper, including facts that come from tables or figures, and enriches the abstract with those facts. Then it writes a question from that enriched abstract alone, so the question reflects curiosity about the paper’s contribution rather than direct access to the evidence. After that, the model locates the supporting evidence in the paper and produces a short answer. Each question is labeled as either Text, meaning it can be answered from the running text, or Visual, meaning it requires a table, figure, or equation. The final dataset contains 500 QA pairs, and each pair is filtered and checked to make sure the question is plausible, the evidence is sufficient, the visual element is truly necessary when needed, and the answer is not something that could just be guessed from general knowledge. Human experts then review and rewrite the questions for naturalness. Now to the core method. DR Pixel is framed as a ReAct-style agent, which means it alternates between internal reasoning and external actions. At each turn, it produces a thought and then either a search query or a final answer. If it searches, the retrieval tool returns relevant evidence from the corpus; if it answers, the trajectory ends. The key difference from standard document agents is what the corpus contains and what the model sees. In a conventional pipeline, the corpus is parsed text, and retrieval returns text chunks. In DR Pixel, the corpus is 1.06 million rendered page images, and retrieval returns page images directly. Those images stay visual all the way through the trajectory; they are never converted back into plain text for reasoning. The agent can therefore use layout, tables, figures, and equations as they appear on the page. The authors evaluate this page-native setup against OCR-based baselines and find something striking: even without task-specific training, page-native deep research works better. Across GPT-5.4, Gemini-3.5-Flash, and Qwen3.6-27B, the page-native version improves accuracy by about 5.1 to 8.4 points over OCR-based pipelines, while also using 20 to 30 percent fewer retrieval calls. That matters because it suggests the visual pipeline is not just more faithful to the source material; it is also more efficient. In other words, once the system can directly inspect the page image, it may need less back-and-forth searching to find the answer. The paper also emphasizes that this is not just a retrieval story. The open-weight model, Qwen3.6-27B, still trails the closed models in the untrained setting, which motivates training. But training agents that make many tool calls is tricky. You can collect demonstration trajectories, but those trajectories are noisy: some steps are essential, while others are redundant, unhelpful, or even failed attempts. If you simply imitate every step equally, supervised fine-tuning can waste capacity on bad behavior. To address that, the authors introduce counterfactual credit reweighting. The idea is simple in spirit: estimate how much each step actually contributed to the final answer. For each reasoning or action step, they remove it from the trajectory and see how much the answer quality changes. If removing a step hurts the final answer, that step gets more weight during training. If it does not matter much, or if it was unhelpful, it gets less weight. This gives a finer-grained form of credit assignment while keeping the overall simplicity of supervised fine-tuning, rather than switching to a more complex reinforcement learning setup. With this training method, the gains are substantial. On DRPixelBench, training improves retrieval recall by about 15 points and final accuracy by more than 10 points. For Qwen3.6-27B specifically, the paper reports a 12-point improvement over the untrained model, which is about a 30 percent relative gain, and the trained model closes much of the gap to GPT-5.4. The improvements also transfer beyond the new benchmark: on four existing benchmarks, the trained DR Pixel outperforms the untrained baseline, and on three additional multimodal scientific literature benchmarks it remains competitive, even surpassing GPT-5.4 in retrieval on two of them. One especially notable result is on ViDoRe-V3, an out-of-distribution benchmark. There, the trained DR Pixel comes within 1.8 points of an oracle that is given the gold pages directly. That is a strong signal that the page-native approach generalizes beyond the exact benchmark it was trained on, and that the system is learning something useful about how to search and reason over visual documents rather than just memorizing one dataset. So what do these results actually show? First, they provide evidence that the conventional OCR-then-retrieve pipeline is not just a convenience choice; it can be a real bottleneck. When the answer depends on visual structure, flattening the document can lose information that matters. Second, they show that modern vision-language models are already strong enough to serve as the reasoning engine for page-native deep research, even before specialized training. Third, they demonstrate that careful credit assignment over agent trajectories can make a big difference, because not every step in a demonstration deserves equal imitation. There are still limitations to keep in mind. The benchmark is focused on scientific papers, so the conclusions are strongest in that domain, where pages are structured and multimodal evidence is common. The approach also relies on a large rendered-page corpus and on visual retrieval infrastructure, which may be more expensive or more complex than standard text indexing. And although the paper shows strong gains, the open-weight model still lags behind the best closed models in some settings, so there is room for better visual reasoning and better agent training. More broadly, page-native deep research is promising, but it will need to prove itself across other document types, longer contexts, and even messier real-world scans. Still, the main message is clear. If the evidence lives on the page, then treating the page as an image is not a workaround; it may be the better representation. DR Pixel shows that for complex document research, directly retrieving and reasoning over rendered pages can be both more effective and more efficient than forcing everything through OCR first.

Previous daily papers

2026-11-07Scientist World: Grow a Research World for AI-Scientists2026-11-07Understanding AI from the Inside: The State of Mechanistic Interpretability for AI Safety2026-11-06User Embeddings are Superpositions of Interpretable Behavioral Modes2026-11-05FrontierPhysics: Evaluating Agents for End-to-End Physics Research2026-11-05Reinforcement Learning Enables Scaling Generalist Robot Policy Improvement2026-11-05Perceptive Humanoid Motion from Simulation2026-11-04From Autonomous Agents to Cognitive Symbionts: Toward AI-Augmented Human Intelligence2026-11-04DreamX: Video Diffusion Model is a Universal Latent Action Learner2026-11-04PyRUA-DataFarm: LLM Agent as a Cost-Efficient Teleoperator for Real-Ready Robot Data2026-11-03Pixel: A Co-Designed Full-Stack Robot System for Learning General-Purpose Mobile Manipulation2026-11-03Strategist: Evolving Reasoning with Agents2026-11-02ACCT: Asymmetric Cross-LLM Context Transfer2026-11-02Latent Reasoning in Continuous Space for Unified Multimodal Models2026-11-02Subquadratic Attention with a Hierarchical Tree-Structured Indexer2026-11-01Testimony from Timelines: Evidence-Grounded Significance Testing of Temporal Clinical Hypotheses2026-11-01ShallowDefend: Harnessing Shallow Safety Alignment for Multi-Turn Defender2026-10-31Difficulty Calibration Across Time for Membership Inference in Diffusion Models2026-10-30world2vec: Mechanistic world models through latent dynamics learning2026-10-30Elucidating the Design Space of LM Data Attribution2026-10-29Pushing Dexterity and Safety for In-Hand Object Pose Reaching That Works Remarkably Well2026-10-29Can LLMs Uncover Hidden Physical Laws through Free Exploration?2026-10-29Scaling Laws for Tool-Use Agents via Controllable Environment Synthesis2026-10-28Multi-State Linear Attention with Generalized Second-Order Routing2026-10-28Heads of Security: Learned Attention Gates Improve Prompt Injection Robustness2026-10-27Pretraining Knows What, Simulation Teaches How: Compositional Generalization in a Sim-Trained Robot Agent2026-10-27Retriever-Free Retrieval: Navigating Documents with Decision Models2026-10-27Reva: Real-Time Vulnerability Detection through Contextual Probing of Large Language Models2026-10-26Learning Positive Linear Combinations of ReLUs in Fully Polynomial Time2026-10-25Reliable AI-Assisted Decisions via Selective Auditing2026-10-25A Fix for Softmax and Cross-Entropy2026-10-25Sensorimotor Alignment: A Platonic Proxy for Embodied Policy Success2026-10-22LongZero: Enhancing Long-Context Reasoning in LLMs from Zero Data2026-10-22The End of Consensus Research?2026-10-22Asymmetric On-Policy Distillation Learns In-Context Exploration2026-10-21UniWALM: Unified World Action Language Model for Embodied Robot Control2026-10-21Joints, Camera, Action! Co-Generating 3D Motion and Video for Arbitrary Rigs2026-10-21Evolving Agent Interfaces2026-10-20OpenVLHarness: Real World Visual Agentic Tool-Use with Multimodal Memory2026-10-20Same Simulation, Different Evidence: Where Retrieved Tool Output Costs LLM Agents Accuracy2026-10-20Text-to-Image Generative Modeling Without Language Pretraining2026-10-19DiscoveryRSI: Recursively Learning to Discover Valuable Hypotheses2026-10-19Learning without relearning physical symmetry: an SO(3)-equivariant convolutional network (CNN) on the sphere2026-10-19Self-Building Labs2026-10-18T2-z: Test-Time Optimization of Environment Latents for Robotic World Models2026-10-18FDE-Gym: The Last Mile of AI Delivery2026-10-18MLExpert: Towards a Generalist Agentic Model for Autonomous Machine Learning Engineering2026-10-17Discovering state-of-the-art quantum circuits with large language models2026-10-15Your Transformer Is Secretly Elastic2026-10-15Hierarchical Residual Policy Learning for Real-World Mobile Manipulation with Sparse Human Guidance2026-10-15Latent Compaction with Test-Time Training2026-10-14Tev: Scaling Long-Horizon Agentic Environments with Dense Progress Rewards2026-10-14The Last Remote Sensing Model Built by Humans? Casting Any Earth Observation Task as Recursive Self-Improvement2026-10-14Hybrid Reasoning Agents2026-10-13AI Models Can Provably Hide Arbitrary Capabilities2026-10-13RATED: Reliability-Aware AI Agent Tool Retrieval via Execution Dynamics2026-10-13Designing Protein Structures from Natural Language2026-10-12RSI-Anything: Unlocking Recursive Self-Improvement with World Models2026-10-11A Balanced Data Diet: Addressing the Exploration Bottleneck in Mega-Scale RL for Robot Control2026-10-11FAITH: Feasibility-Aware Safety-Filtered RL for High-Dimensional Systems