Daily AI Paper

Every day, our AI scans the latest arXiv submissions, picks the most notable one, and narrates a video introduction — automatically.

Latest
2026-09-06

SWE-Gate: Passing Functional Tests Is Not Enough for Software Engineering Agents

Xin He, Yanlin Wang, Mingwei Liu, Jiachi Chen, Hongyu Zhang, Guanbin Li

arXiv:2609.04167v1

Today’s pick is SWE-Gate, a new benchmark for software engineering agents that asks a very practical question: is passing the tests really enough? In real repositories, a patch often has to satisfy more than functional correctness. Review comments can impose extra constraints about style, structure, safety, or how the code should be changed. SWE-Gate builds those review constraints directly into the evaluation, using real pull request feedback from open-source Python projects. The benchmark contains repair tasks with separate tests for functionality and for constraint compliance, so an agent can’t just brute-force its way to a green checkmark. The authors show a striking gap: many patches that pass the functional tests still fail the review requirements. That matters because coding agents are increasingly being judged, and deployed, on benchmarks that may overestimate their real usefulness. SWE-Gate pushes evaluation closer to how software is actually developed, where correctness and acceptability both matter.

Also spotted that day

Compile by Training: Turning Natural-Language Specifications into Local Neural FunctionsClean Engineering, Unstable Measurement: A Preregistered Reliability Failure of Black-Box LLM Observers on Shared EndpointsESPO: Error-Structured Prompt Optimization via Diagnose, Diversify, and StabilizeLegibility is Not Interpretability: Comparing Judged and Actual Importance in Chain-Of-Thought ReasoningOne Editor, Many Edits: A Unified Training-Free Framework for Diverse Video EditingRobust PAC Learning of Concurrent Stochastic GamesSeeing Before Synthesizing: VLM-Guided Transition Event Discovery for Weakly-Supervised Dense Video CaptioningKnowledge Acquisition During Pre-training? Large Language Models Learn Better With Auxiliary ViewsA Computationally Feasible Framework for Causal Probabilistic ExplanationLast Translation BenchmarkRethinking On-Policy Distillation of Large Language Models II: One Training ExampleA Case Study on Emergent Cheating and Whistleblowing in Autonomous Research SwarmsPara-Pipe: Exploiting Hierarchical Operator Parallelism of ML Computational Graphs on SoCsFrom Deceptive Outputs to Deceptive Mechanisms: A Causal Framework for Language-Model Deception Research

Previous daily papers

2026-09-05From Deceptive Outputs to Deceptive Mechanisms: A Causal Framework for Language-Model Deception Research2026-09-05Last Translation Benchmark2026-09-04Rethinking On-Policy Distillation of Large Language Models II: One Training Example2026-09-03A Common Measure of Communication for Speech Brain-Computer Interfaces2026-09-03Discriminative World Models for Web Agents2026-09-03Post-Training Language Models for Gold-Medal Performance in Coding Competitions2026-09-02Mechanism Design for Alignment and Control2026-09-02Beyond Scores: Understanding LLM-as-a-Judge Mechanisms in Summarization Evaluation2026-09-02The Rise of Verbal Reinforcement Learning2026-09-01Stress-Testing Efficient Responsible-AI Evaluation: When Compute Savings Change Benchmark Conclusions2026-09-01Aspire: Can Models Self-Evolve from Vague Goals?2026-09-01PaperGym: Rubric-Centered Evolution for Research-Plan Generation2026-08-31DARTS: Decoder-Aware Representation Tuning via Surgery for Model Merging2026-08-31Video Generative Models as Geometry Learner2026-08-31When Robots Mishear Us: Mapping the Safety Risks of Voice-Controlled Embodied AI2026-08-30CritICL: Inference-Time Weak-to-Strong Generalization from Small Language Model Failure Modes2026-08-30Boosting LLM Exploration via Weak-Model Guidance in RLVR2026-08-30CLAP: Cross-Embodiment Video World Models are Zero-Shot Physical Simulators2026-08-30TTPO: Test-Time Policy Optimization2026-08-29TTPO: Test-Time Policy Optimization2026-08-28TTPO: Test-Time Policy Optimization2026-08-27Planetary Prediction Engine: Autonomous Geospatial Prediction via Intelligent Data Selection and Foundation Model Embeddings2026-08-26BrowserForge: Scaling Web Episode via Parallel Browser Sandboxes2026-08-25ConvergeFlow: Language Flow with Provable Convergence to Token Embeddings2026-08-24VIALS: A Benchmark for Visual Interpretation of Artifacts in the Life Sciences2026-08-23Inducing Task Models from Computer-Use Traces2026-08-22Inducing Task Models from Computer-Use Traces2026-08-21Inducing Task Models from Computer-Use Traces2026-08-20SPADE: Self-Play in Adaptive Synthetic Executable Environments2026-08-19From Corpora to Co-Evolving Capabilities: Capability-Centric Data Design for Generalist Image Generation2026-08-18Towards Computational Provenance: Carrying Causal-State Evidence in Generated Text2026-08-17Universal Thermodynamic Interatomic Potentials for Crystalline Materials2026-08-16OmniScientist: An Omni-Modal Omni-Discipline AI Scientist