Daily AI Trading Paper

Today’s paper is about a deceptively simple question with big implications for AI trading: when a foundation model is trained directly on limit order book message streams, does it just predict the next event well, or does it actually build an internal model of the market? That distinction matters a lot. In market microstructure, the short-horizon behavior of prices is driven by a flood of tiny events: submissions, cancellations, and executions arriving at nanosecond speed. If a model can only fit historical patterns externally, it may still look strong in backtests while remaining fragile when regimes shift. But if it learns a reusable internal representation of supply, demand, and liquidity dynamics, then it becomes more interesting both scientifically and practically. It may generalize better, be more interpretable, and be easier to trust in unusual conditions. The authors frame this in the same spirit as mechanistic interpretability work in games and language. Sequence models trained only on moves in Othello or chess have been shown to develop latent board representations, even though they are never explicitly told the rules or shown the board state. Here the question is whether a model trained only on market messages can similarly recover an internal representation of the hidden market state, even though the full order book is never given as input. To study this, they use an 8.2 million parameter Mamba-3 state-space model. This is a small but efficient recurrent foundation model, with 13 SSM layers and a 256-dimensional residual stream. It is trained autoregressively on tokenized limit order book message streams from eight large-cap US equities: GOOG, AAPL, NVDA, AMZN, META, TSLA, MSFT, and AMD. The data cover 2022 through 2025, and the model sees windows of 500 messages at a time. Importantly, the model is trained only on the message stream, not on the reconstructed book itself. That setup is important because it forces the model to infer latent market structure from the event sequence alone. The paper also notes that despite the model’s small size, it performs competitively on LOB-Bench, and with faster inference than larger alternatives. The core interpretability question is where the market representation lives. The authors compare two internal locations: the residual stream and the recurrent state. They probe both for several market quantities at a fixed boundary near the end of each window. The main target is order-flow imbalance, or OFI, which is a classic microstructure signal capturing net buying versus selling pressure over a lookback window. They also probe traded volume, realized volatility, touch width, and book depth. A key technical detail is that Mamba-3’s recurrence can be written in an equivalent attention-like form. That lets the authors decompose the recurrent state exactly into contributions from individual messages. In plain terms, they can ask not only what is stored in the state, but which past messages wrote that information there. That is a powerful way to move beyond black-box probing. So what do they find? First, the market representation is in the recurrent state, not the residual stream. OFI decodes from the state with an R-squared of about 0.59, which is far above a random-weights control that sits near zero. Traded volume and realized volatility are also decodable. This means the state is carrying a meaningful summary of the market’s recent flow and dynamics, not just arbitrary hidden activations. Second, this representation generalizes. It is not just memorizing ticker-specific quirks. The same kind of signal appears on stocks that were not seen during training, which suggests the model has learned something closer to a general microstructure representation than a lookup table of individual names. Third, the information is not spread uniformly across the whole history. The authors find that a small set of learned long-memory heads carry the flow signal, with effective half-lives on the order of 28 to 66 messages. So the model is not simply reacting to the last event; it is integrating information over a meaningful short-horizon memory window. They also test whether this latent representation is actually used by the model, not just present in a probe. That is where causal steering comes in. The idea is to shift the internal state along a direction corresponding to a feature, such as OFI, and then see whether the model’s next-message beliefs move accordingly. They build steering directions from the difference between states in high-feature and low-feature windows, then add varying doses of that direction into the state. The result is that the model’s beliefs shift monotonically with dose. For OFI, pushing the state toward buy-side pressure changes the model’s next-message expectations in the direction you would want if the state really encodes market flow. They compare this against matched random and shuffled controls, which strengthens the case that this is not just a generic perturbation effect. There is an important nuance here, though. The authors are careful not to overclaim that the model is predicting future prices in a strong trading sense. A linear readout of the frozen state predicts the next 25 messages’ mid-price move with only about R-squared 0.04, roughly on par with a readout of the visible book. So the state clearly contains information about current market conditions, but its direct forward predictive power is limited. In other words, the model appears to understand the present more than it forecasts the future. That limitation is actually informative. It suggests the foundation model is not a magical alpha machine. Instead, it seems to be learning a latent world model of the market’s microstructure: a compressed representation of order flow, liquidity, and volatility that helps it generate realistic message sequences. That is a different claim from being a profitable predictor of returns. The broader significance is that this is, to the authors’ knowledge, the first mechanistic interpretability study of a foundation model trained natively on real limit order book data. Prior work in finance has mostly looked at external performance, sensitivity analyses, or post-hoc attribution. Those methods can tell you which inputs mattered for a prediction, but not how the model internally organizes the market. This paper moves one level deeper. There are still caveats. The study is on a relatively small model and a limited set of large-cap US equities. The steering tests are compelling, but they are not the same as proving the model would behave safely or profitably in live trading. The model’s forward prediction of price movement is weak, so its internal market representation should not be confused with strong alpha generation. And because this is a mechanistic study, the main contribution is understanding, not a trading strategy. Still, the paper’s message is important. Autoregressive foundation models on limit order book data do not just generate plausible event streams; at least in this setting, they appear to build an internal, generalizable representation of market flow, and that representation can be probed and even causally steered. For researchers in AI trading, that opens a path toward models that are not only useful, but inspectable.