Reasoning Models & Reinforcement Learning

From human-preference training to reinforcement learning on verifiable rewards: how reasoning models are trained, and what the evidence says that training actually buys.

3 lessons • 28 min read

1. 01

   From RLHF to reinforcement learning with verifiable rewards

   Where does the reward come from when a language model is trained by reinforcement learning? Between 2017 and 2025 the answer moved from "a model of human preferences" to "a program that checks the answer", and that shift explains both the rise of reasoning models and a new family of failure modes.

   9 min read
2. 02

   Reasoning models: training models to think before answering

   A reasoning model is a language model trained to spend a variable, often large, number of tokens working through a problem before it commits to an answer. This lesson covers how that became a training objective instead of a prompting trick, what the 2024–2026 systems show, and why the visible "thinking" is evidence about the computation but not a transcript of it.

   10 min read
3. 03

   What reasoning training does — and doesn't — buy

   When reinforcement learning with verifiable rewards makes a model better at maths or code, has it learned to solve problems it could not solve before, or has it learned to pick, more reliably, solutions it could already sample? As of November 2026 this is one of the most actively disputed empirical questions in post-training. The answer decides whether RL can scale capability by itself or mainly sharpens what pre-training put there.

   9 min read