Daily AI Paper

Modern software has a timing problem. The later a vulnerability is discovered, the more expensive it becomes to fix. That is why the ideal detector is not one that works eventually, or after a long reasoning loop, but one that can screen each new function as it is written. That matters even more now that coding agents can produce code faster than developers can manually inspect it. If the review step cannot keep up, vulnerabilities can slip into the codebase before anyone has a chance to react. This paper, Reva, is trying to solve that exact bottleneck. The authors start from a simple observation: the most accurate LLM-based vulnerability detectors today do not just look at a function in isolation. They behave more like a security analyst. They gather surrounding context, consult static analysis, inspect call relationships, and reason over the evidence before making a judgment. Systems like VulTriage, VulAgent, and VulTrial do better than simpler detectors because they use this richer process. But that accuracy comes at a cost: they require multiple model calls and lots of generated text, so a single verdict can take minutes. That is far too slow for real-time screening. At the other end of the spectrum are faster approaches: fine-tuned classifiers, or prompting an LLM directly for a yes-or-no vulnerability verdict. These are fast enough, but the paper argues they leave too much accuracy on the table. They often judge a function without enough context, and even when you give an LLM more information, its written verdict can still be inconsistent or unfaithful to what it internally “knows.” So the core question is: can we keep most of the useful security context that agentic systems use, but compress the whole decision into a single forward pass? That is the idea behind Reva, short for Real-time Vulnerability Analysis. Instead of letting the model reason step by step and then reading its answer, Reva gives the model a carefully assembled prompt and then reads its internal hidden states with a small trained probe. In other words, the model processes the function and its context once, silently, and the detector extracts a risk score from the model’s internal representation rather than from generated text. To understand why this is interesting, it helps to know the landscape. Vulnerability benchmarks often pair a vulnerable function with a benign version of the same function. That is useful because the detector must learn the difference between insecure and secure code, not just memorize broad project patterns. But it also creates a trap: if a detector relies on superficial features that both versions share, it can look good while failing to truly identify the vulnerability. The authors are careful about this and evaluate functions independently, then check whether each pair is labeled correctly. Reva’s pipeline has three main ingredients. First, it builds a security context for each function. The prompt includes the function itself, plus a threat model for the project and the module containing the function. This threat model is built once per project using LLM agents over the whole software and summarizes things like the project’s purpose, attacker-controlled inputs, trust boundaries, and the kinds of sinks and guards relevant to that module. Reva also adds lightweight static-analysis findings from Semgrep, along with call-site information, helper signatures, and file-level context such as includes, types, macros, and sibling functions extracted with tree-sitter. If that repository context is unavailable, the prompt still includes the function and its Semgrep findings. The median prompt is fairly compact by modern LLM standards, at about 3,840 tokens, with 601 tokens of code. Second, Reva does not ask the LLM to generate an explanation or verdict. It runs the model in a single forward pass and stops after the deepest layer that will be probed. No output token is generated. The authors probe four intermediate layers, chosen by cross-validation, because prior probing work often finds that mid-to-late layers are the most informative. They then keep only the hidden states corresponding to the code tokens of the function. Third, a small attention-based probe turns those hidden states into a risk score. This probe is not huge; it has about 0.57 million parameters. It is trained on labeled functions from PrimeVul, and its loss combines ordinary classification with a ranking objective that encourages each vulnerable function to score above its benign counterpart. Importantly, those vulnerable-benign pairs are used only during training. At inference time, each function is scored on its own. The experiments are built around PrimeVul, a dataset of vulnerable and benign C and C++ functions spanning 651 projects and 121 vulnerability types. The held-out test set contains 435 pairs, and the authors also use training and validation splits for design choices and cross-validation. They compare Reva against VulTriage and VulAgent, running all methods on the same open-weight LLMs: Muse-Glimmer-30B, Qwen3.8-27B, and GLM-5.3-Flash. For GLM-5.3-Flash, the agentic baselines use an FP8 version of the model, while Reva uses an exact BF16 conversion. The main metric is function-level accuracy, and they also track pair-level behavior such as whether both versions of a pair are correctly labeled, both are flagged, both are cleared, or the labels are reversed. The main result is that Reva closes most of the gap between speed and accuracy. On GLM-5.3-Flash, Reva reaches 60.0% accuracy on the test set, compared with 59.2% for VulTriage and 52.8% for VulAgent on the same model. The confidence interval around Reva versus VulTriage includes zero, so the two are essentially tied on this benchmark, but the important point is that Reva matches the state of the art while being dramatically faster. On the other two LLMs, Reva is clearly better: it is about 5 to 7 points more accurate than both agentic detectors, and those differences are statistically meaningful. The pair-level results are also informative. Reva has a lower false positive rate than VulTriage on all three LLMs, and it produces fewer reversed pairs. On two of the three models, it also labels more pairs entirely correctly. That suggests Reva is not just learning some artifact shared by the vulnerable and benign versions of each function. It is separating the two versions more reliably, which is what you want if the detector is actually capturing vulnerability evidence rather than surface similarity. The speed difference is enormous. Reva takes about 0.88 to 1.33 seconds per function end to end, while the agentic baselines take from 108 seconds to more than 5,600 seconds on the same LLMs. The paper notes that these are measured under parallel load, so they are not perfect single-request latency numbers, but the token counts tell the same story: Reva processes roughly 3,100 to 3,400 tokens and generates none, while the agentic systems process several to many times more tokens, much of it generated text. In cost terms, screening 1,000 functions on a rented H200 GPU costs Reva under a few dollars, while the agentic systems are an order of magnitude or more expensive at public API rates. The ablation study is especially revealing. If you keep the same prompt and ask the LLM directly for a yes-or-no verdict, accuracy collapses to chance. That direct-verdict approach remains well below the probe on the test set. The static-analysis tools alone are also around chance. So the hidden representations inside the LLM are carrying useful information that is not exposed by the model’s own verbal answer and is not contained in the static-analysis findings by themselves. This is one of the paper’s strongest claims: the model internally encodes much of the evidence that agentic detectors spend many calls trying to assemble. Within the probe itself, attention pooling matters. Replacing it with a simple average over token states drops accuracy by several points, implying that vulnerability evidence is concentrated in a few important tokens rather than spread evenly across the whole function. Using only one layer, removing normalization, or dropping the ranking objective all hurt a little. Removing repository context or keeping only the function also lowers accuracy, though less dramatically on this benchmark. The authors interpret that carefully: PrimeVul functions are relatively short, so the function itself carries most of the signal here. They expect broader context to matter more for interprocedural vulnerabilities. They also test for leakage and selection effects, which are common concerns in probing work. When they ensure that no project appears in both training and evaluation folds, accuracy drops only modestly, suggesting the probe generalizes across projects. Choosing probe layers only on training data, rather than on all development data, barely changes results. But there is still a notable gap between cross-validation and the held-out test set, which the authors attribute to the chronological split in PrimeVul and the resulting shift toward newer code. So what should we take away? Reva shows that you do not necessarily need a long agentic reasoning loop to get strong vulnerability detection. If you can provide the right security context up front, the LLM’s internal states already contain a lot of the relevant signal. A small probe can extract that signal quickly and cheaply, making near-real-time screening plausible. The limitations are important, though. Reva depends on access to the model’s hidden states, so it cannot be used as a black-box API-only tool. Its current evaluation is on function-level detection, and the authors themselves note that context may matter more for harder vulnerabilities that span multiple functions or files. The benchmark also shows a test-set drop relative to cross-validation, which suggests that data shift remains a real issue. And while Reva is much faster than agentic systems, the paper still envisions a broader pipeline where likely vulnerabilities are escalated to more capable agents for deeper proof construction. In short, Reva is a strong argument that the intelligence inside an LLM can be more useful than the text it produces. By probing those internal representations in a single pass, the system gets much closer to the accuracy of agentic vulnerability detectors while operating at real-time speed.