Tauric Research
Perspectives

Evaluation and decision support

  • Application

How to evaluate research systems, and why professionals keep the decision.

Research earns its keep when it changes a decision. That raises the bar for evaluation: a persuasive report and a conclusion that deserves to be acted on can look alike, and only measurement tells them apart.

State of the field

Verification has become the hard part. At Jane Street, Ron Minsky observes that models can seem to do more work than they really do “because this verification piece is as hard as it ever was. In fact, maybe harder”, and the team now writes evaluations for the model graders it uses (Minsky and Crepezzi, 2026 (external site)).

Models are often used to grade other models. Strong judges agree with human judges more than 80% of the time, but they also favor longer answers, certain positions, and their own output (Zheng et al., 2023 (external site)). Anthropic recommends deterministic graders where possible, model graders where necessary, and human graders “judiciously for additional validation”, and, because model outputs vary between runs, running multiple trials (Anthropic, 2026 (external site)).

Firms that use these systems in investing stay cautious. Man Group holds AI-generated signals to the same thresholds as human research and writes that “we can’t leave it unsupervised just yet” (Fang and Moore, 2025 (external site)). The NIST AI Risk Management Framework (external site) (NIST, 2023) gives organizations a voluntary structure for that kind of oversight.

Principles

Decisions belong to the professionals who act on them. A research system should hand them a clear conclusion with the reasoning, the evidence, the risks, and the open questions laid out beside it, so they can disagree with it for good reasons. Nothing we publish is investment advice; our tools are built to support judgment.

Our approach

We aim to measure research quality against outcomes and expert review, and to keep a short, documented path from a finished analysis to a considered action, with the decision left to professionals.

References