Tauric Research
Perspectives

Reinforcement learning for reasoning

  • Intelligence

Reinforcement learning sharpens reasoning; its reward and data set the limit.

Reinforcement learning is behind several recent advances in model reasoning. OpenAI’s o1 models are “trained with large-scale reinforcement learning to reason using chain of thought” (OpenAI, 2024 (external site)), and DeepSeek reports that reasoning ability can be incentivized through reinforcement learning without human-written reasoning examples (DeepSeek-AI, 2025 (external site)). Markets keep score every day, which makes it natural to ask whether they can teach models to reason about investments.

State of the field

A model learns whatever the reward actually pays for. Training a text-to-SQL model, researchers at Bridgewater’s AIA Labs found in a pilot run that 32.8% of positive rewards went to queries that were not fully equivalent to the correct one, so that nearly one time in three the reward reinforced the wrong query. Mislabeled training data reverses the signal, and they found that “algorithmic tweaks cannot compensate”; cleaning up the data, with expert verification, was crucial to making the training work (Zhu et al., 2026 (external site)).

Market rewards bring their own problems. Returns are noisy, so a good decision can lose money and a bad one can win. Man Group researchers, testing an agentic research workflow, warn that multiple testing “demands rigorous statistical standards and appropriate scepticism to avoid spurious findings”, and that “a system that only produced positive results would be useless” (Luk et al., 2026 (external site)). Trading-R1 (external site) (Xiao et al., 2025) rewards a model for structured investment theses that cite their evidence, alongside volatility-adjusted decisions, rather than for returns alone.

Principles

A reward should favor reasoning that cites its evidence and survives scrutiny, and the labels behind it deserve expert attention. A backtest is where an investigation starts, and a result is only as credible as the care taken to keep the future out of both the training data and the test.

Our approach

Trading-R1, our reasoning model for trading, is trained with reinforcement learning. We aim to train models that improve by comparing their reasoning with what markets actually did, evaluate them point-in-time by construction, and publish our test methods so results can be checked.

References