Results

What we measured, and what it showed.

The same questions, answered by Claude alone and by Claude with Day Theory. Every gain is checked with an exact paired test. Where a gain is too small to prove, we say so.

tests with a proven gain in right answersIncidents, filings and three data domains tests more accurate at a lower cost per right answerMeasured Claude usage, same questions gains too small to prove, shown anywayMarked “not proven” wherever they appear
Methods

How every number on this site is measured.

Plain rules, the same for every test.

Same questions, both ways
Every test asks the same questions of Claude alone and of Claude with Day Theory, so each question is its own comparison. Each question is run twice in each arm.
The bar
More right answers, under a 1 in 20 chance of luck, and no group of questions down more than 10 points. The bar is set before the test.
Exact odds
The chance a gain is luck is exact, counted from every way the runs could be relabelled. No approximations.
Held-out and development sets
Held-out questions were not used while the knowledge was written. Development sets are labelled wherever they appear: held out from the knowledge, but run repeatedly during development.
Scored strictly
An answer counts only if it is right. Every score is a floor.
Cost
Claude usage per right answer: measured usage divided by right answers, without Day Theory’s own price.
Models
Claude Sonnet 5.5 for software incidents, filings and documentation. Claude Opus 5.5 for Telecom, Bank and Market.

Numbers. Run counts and exact tests come from the site’s own data and statistics files, loaded unchanged: software incidents (133 held-out incidents on three systems), financial filings (150 questions, development set) and documentation (124 held-out questions on 13 public databases, with half or all of the documentation), each run twice with Claude Sonnet 5.5. The per-system rows and the 8-database row come from the numbers guide; Telecom, Bank and Market cost rows from the coordinator’s run record of 13 November; the warehouse runs from the graph gallery’s run data, loaded unchanged. Telecom ran an earlier version on Opus 5.5 and is a development set.

Self-check. Running…

See how many wrong answers it removes on your own questions.

Bring questions your team has already answered. You’ll see the measured gain on your data, with the same proof shown here, before you deploy anything.