Daily AI Trading Paper

Statistical arbitrage has always had a hidden-state problem. You observe prices, returns, and factor residuals, but what you really want to know is the state of the spread right now: is it converging back to fair value fast enough to make a trade worth paying for, or is it drifting away and likely to keep doing so? That matters because the trading decision is not just about predicting the next move. It is about deciding whether a temporary dislocation is real enough, and strong enough, to overcome transaction costs over your holding horizon. The standard statistical arbitrage pipeline, going back to classic pairs and residual trading approaches, is simple and elegant: regress returns on factors, accumulate the residuals into a spread, fit a mean-reversion model on a rolling window, and trade when the fitted spread looks extreme. The problem is that this pipeline usually turns a noisy estimate into a hard decision and then forgets how uncertain that estimate was. In a large cross-section, that becomes even more dangerous, because you are ranking many candidate spreads and selecting the most attractive ones. That selection step amplifies optimism: the ones that look best are often the ones with the most favorable estimation error. So the strategy can end up mistaking a lucky estimate for a genuine opportunity. This paper is trying to fix that exact issue. It keeps the residual-spread framework, but instead of outputting only a point estimate for the latent state of the spread, it outputs a calibrated posterior distribution. In other words, it asks not just “what state do I think this spread is in?” but “how sure am I?” That uncertainty is then used directly in the trading decision. The broader context is useful here. Finance has a long history of factor models, from static characteristic factors like Fama-French, to dynamic latent factor models, to more recent deep-learning systems that learn factors end to end from characteristics. Those newer systems improve how the residuals are built, but in many cases they still trade on a point estimate. Separately, outside finance, there has been a major wave of work on tabular foundation models and amortized inference: train a neural network on synthetic draws from a prior, and then use one forward pass to approximate a posterior on new data. This paper brings that idea into statistical arbitrage. The authors call their estimator the Neural Filter Estimator, or NFE. The key idea is to train it on simulated paths of a spread process whose parameters are chosen to resemble real residuals. Then, once trained, it can take a look-back window for any spread in a cross-section and return the posterior over the latent state in a single pass. So instead of fitting a separate filter for each asset or spread, you amortize that inference across the whole universe. The state-space setup is intentionally tied to the trading problem. The spread is the cumulative sum of residual returns, and the latent state describes whether the spread is currently in a converging regime or a diverging regime, along with how fast it mean-reverts. The paper defines regimes relative to the trading horizon: a regime is not just a statistical label, but a state that changes what a trade can expect to earn over the next H days. If the spread is likely to reverse enough within the holding period, that is one regime; if not, it is the other. That is a very trader-centric way to define state, and it matters because a regime only matters if it changes the action. For the underlying dynamics, the paper uses an Ornstein-Uhlenbeck-style mean-reverting process, but with latent elements: the mean-reversion speed, the level, and the innovation law are all treated carefully rather than assumed away. The innovation distribution is not just taken from a textbook Gaussian. Instead, candidate innovation laws are tested against real residuals, and only those that survive that comparison are used for simulation. That is important because the quality of the posterior depends on the realism of the simulator. If the simulator is wrong, the posterior can be beautifully calibrated to the wrong world. The training procedure is simulation-based inference. The model sees many simulated windows where the latent state is known, learns the map from observed history to posterior over state, and then is applied to real data without further fitting. In effect, the expensive work is done once in the simulator, and then inference on live spreads becomes a fast forward pass. The empirical design has three main parts. First, they check whether the simulator actually resembles the real residuals. They do this with a classifier that tries to distinguish simulated windows from real ones. The Gaussian noise version of the classical model is rejected. The only candidate that passes on both cross-sections is the one with time-varying volatility and asymmetric shocks; fat tails alone are not enough. So the simulator is not just a convenience — it is validated against the data. Second, they test how well the latent regime can be identified at all. On simulated spreads, the true regime is known, so every method can be compared with an ideal classifier that has access to the true model, level, and volatility. Here the result is striking: every practical estimator, including the NFE, lands in one narrow band of accuracy that is well below the ideal. That means the main bottleneck is not the neural filter. It is the information content of the window itself. The spread level has to be inferred from the same window as the reversion dynamics, and that creates a fundamental estimation limit. Better inference cannot fully overcome that. Only more information can. Third, they ask what the posterior changes in actual trading. They test the methods on Korean equities, net of costs, under fixed trading rules. The conventional point estimate turns out to be overly optimistic: it forecasts about a thousand basis points more per holding period than it realizes. That is a huge gap, and it is exactly the kind of overconfidence you would expect when uncertainty is ignored and the best-looking spreads are selected from a large universe. The amortized methods reduce that forecast-realization gap by roughly a factor of two to eight. So the posterior does not magically create more alpha, but it does make the strategy’s own expectations much more honest. That distinction is central to the paper’s message. Under the fixed trading rules they test, posterior-based sizing does not add meaningful alpha over the point estimate. It does not deliver a dramatic lift in realized return by itself. But it does something arguably more important for risk control and model governance: it calibrates the strategy’s beliefs. The point estimate systematically overstates what the trade will earn. The posterior corrects that overconfidence. In a live setting, that can matter a lot, because a strategy that knows it is uncertain can size down, skip borderline trades, or at least avoid pretending that weak signals are strong. There is also an interesting comparison to the end-to-end factor-learning literature. When the same learned factors are used in an Attention Factor Model, sizing positions from the posterior matches the Sharpe ratio of those factors under the base rule, within the noise of the sample, and does so without taking on more risk. So the posterior is not merely a diagnostic afterthought. It can be plugged into a modern learned-factor pipeline and preserve performance while improving calibration. So what should we take away? First, the paper argues that in statistical arbitrage, uncertainty is not a nuisance term to be ignored. It is part of the signal. Second, the main value of amortized filtering here is not higher raw accuracy in identifying the hidden state, because that seems to be fundamentally limited by the data window. The value is better calibrated probabilities. Third, on real equities, the conventional point estimate appears to overpromise substantially, while the posterior-based methods bring those expectations much closer to reality. The limitations are just as important. The method depends on the simulator, so if the simulated process misses an important feature of the real market, the posterior will inherit that misspecification. The paper does a good job testing the simulator against residual data, but that is still only an approximation. The empirical results are also specific to the tested universe and holding period, so one should not assume the same calibration or trading impact will automatically transfer to other markets, horizons, or cost structures. And because the posterior does not materially increase alpha in their setup, its main benefit is better uncertainty accounting rather than a guaranteed boost in returns. Still, that may be exactly the point. In statistical arbitrage, the hard part is often not finding a complicated signal. It is knowing how much to trust the signal you already have. This paper’s contribution is to make that trust explicit, probabilistic, and tradeable.