Tools from the research.
The decision tools our papers produced, runnable in your browser where the math allows and downloadable as the Python we used. Your data stays on your machine. For everyday calculators — API cost, tokens, eval sample sizes, neural data laws — see the free tools.
Model evaluation
Verifiers, judge panels and emergence claims.
- verifier-budgetIn-browser · Python CLI
The modelled Best-of-N sampling budget N*(ε) and the selection rule (argmax vs. consensus) from a verifier's measured exploitability, with an ε estimator for a labelled validation set. A model: real judges did not collapse by N = 32 in the lab's pre-registered test (findings 8.13).
- verifier-panelIn-browser · Python CLI
How much an LLM-judge panel is really worth: error correlation, effective panel size, the majority-vote ceiling — from two numbers or straight from a critics × items miss matrix — and a tail check before trusting a Φ-based ε.
- selection-reliabilityIn-browser · Python CLI
Did the agent forget a skill or pick the wrong one? Selection error ε with confidence intervals, a forgetting vs. mis-selection verdict from counts or a decision log, and the largest library a router can face before accuracy drops below target.
- exploitability-auditIn-browser · Python CLI
Is your verifier or reward model exploitable? Measured ε, the consensus threshold ε* = p/(1−p) and the empirical Best-of-N peak N* from a log of scored candidates, broken out by pool and by problems the verifier saw in training.
- selector-tuneIn-browser · Python CLI
Fix the selector, not the skills: per-candidate score standardisation, a deferral margin that meets a target ε and the smallest verifier shortlist, fitted on half a decision log and judged on the other half.
- emergence-auditPython harness
Four owned controls — stateless generator, non-solution generator, prior swap at fixed fit, family distance — run against your own training script for any "X emerges from a network" claim.
Experiment design
Whether a pre-registered prediction could have been measured at all.
Hardware feasibility
Back-of-envelope gates for photonic and analog compute proposals.
