Probability as the arithmetic of uncertainty

Every modern AI system that outputs a prediction, from a spam filter to a large language model, is in the business of assigning numbers to uncertain outcomes. Probability is the small set of rules those numbers must obey to stay consistent, and nearly everything else in machine learning is built on top of that arithmetic.

Outcomes, events and three axioms

Start with a sample space $\Omega$: the set of everything that could happen. Roll a die and $\Omega = \{1,2,3,4,5,6\}$. Ask a language model for the next token and $\Omega$ is its vocabulary, perhaps 100,000 strings. An event is a subset of $\Omega$: "the roll is even" is $\{2,4,6\}$; "the next token is a digit" is the set of digit tokens.

A probability $P$ assigns each event a number. In 1933 Andrey Kolmogorov showed that the whole theory could be derived from three axioms:^[1]

1. $P(A) \ge 0$ for every event $A$.
2. $P(\Omega) = 1$: something happens.
3. If $A$ and $B$ cannot both happen (they are disjoint), $P(A \cup B) = P(A) + P(B)$ (and likewise for countably many disjoint events).

Everything else is a consequence. The complement rule, $P(\text{not } A) = 1 - P(A)$, follows because $A$ and its complement are disjoint and together make up $\Omega$. For events that can overlap, inclusion–exclusion corrects the double count:

Why should degrees of belief, rather than long-run frequencies, obey these same rules? Richard Cox argued in 1946 that any system for reasoning about plausibility that satisfies two consistency requirements (the plausibility of "not A" is determined by that of A, and the plausibility of "A and B" is determined by that of B and of A given B) must, up to rescaling, be isomorphic to probability theory.^[2] This is the formal version of Laplace's description of probability as common sense reduced to calculation.^[3] It means the rules are not just a convention for dice: they apply to a model's belief about the next word, or your belief that a benchmark result will replicate.

Conditioning: updating the denominator

The conditional probability of $A$ given that $B$ has happened is

Conditioning shrinks the sample space to $B$ and renormalises. If a fair die shows an even number, the probability it is a 6 rises from $1/6$ to $\frac{1/6}{1/2} = 1/3$.

Rearranged, the definition becomes the product rule: $P(A \cap B) = P(B)\,P(A \mid B)$. Applied repeatedly to a sequence of outcomes $x_1, \dots, x_T$, it gives the chain rule of probability:

This identity is exact, with no modelling assumptions, and it is the blueprint for every autoregressive language model. Rather than modelling the astronomically large joint distribution over whole texts, a model learns the conditional distribution of each next token given what came before, and the chain rule stitches the pieces back together. Bengio and colleagues' 2003 neural language model was framed in exactly these terms: learn a function that outputs $P(x_t \mid \text{context})$ and multiply.^[4] The model's final layer turns arbitrary scores $z_i$ into a valid distribution with the softmax, $p_i = e^{z_i} / \sum_j e^{z_j}$, which is non-negative and sums to one by construction, so axioms 1 and 2 hold automatically. See Next-token prediction for where this leads.

Why we work in logarithms

The chain rule multiplies many numbers below one. If a model gives each of 100 tokens probability 0.9, the whole sequence has probability $0.9^{100} \approx 2.7 \times 10^{-5}$; for a 2,000-token document with typical per-token probabilities, the product underflows standard floating-point formats. Taking logarithms turns the product into a sum,

which is numerically stable and is the quantity training actually optimises (see Maximum likelihood).

Independence, and what it buys

Events $A$ and $B$ are independent when $P(A \cap B) = P(A)\,P(B)$, equivalently $P(A \mid B) = P(A)$: learning $B$ tells you nothing about $A$. Independence is what lets small probabilities compound into large ones.

Worked example: coverage from repeated sampling

Suppose a model solves a hard problem on a single attempt with probability $p = 0.1$. Draw $N$ independent attempts. What is the probability that at least one is correct?

"At least one success" is awkward to count directly, but its complement, "every attempt fails", is a single product:

Brown and colleagues measured exactly this quantity, which they call coverage, for language models given many attempts, and found it kept rising across four orders of magnitude of samples.^[5] The arithmetic explains why: the failure probability decays geometrically.

Two warnings follow directly from the axioms.

Coverage is not accuracy. At least one of 64 attempts being right does not mean you can tell which one. Picking the winner requires a verifier, and the verifier's errors compound with $N$ as well. The NeuroGenesys paper Thurstone bounds Best-of-N works through a reference case ($p = 0.10$) in which coverage at $N = 64$ is 0.9988 while selected accuracy peaks near 0.70 and then falls as $N$ grows further.^[6] The course on test-time compute develops this in detail.

Independence is an assumption, not a default. Samples from one model share its blind spots, and judges that read the same context share its errors. The lab's note A critic that shares the generator's prompt shares its errors measured a same-context critic that missed 12 of 12 planted errors that fresh critics all caught.^[7] When events are positively correlated, $1-(1-p)^N$ overstates what repetition buys; in the extreme it buys nothing.

Marginalisation

The last basic tool is the law of total probability. If events $B_1, \dots, B_k$ partition $\Omega$ (exactly one of them happens), then

This is how you compute a probability you cannot observe directly by averaging over hidden cases. If 70% of a model's traffic is easy queries answered correctly 95% of the time and 30% is hard queries answered correctly 60% of the time, overall accuracy is $0.7 \times 0.95 + 0.3 \times 0.6 = 0.845$. A reported average hides that split, which is why a change in the query mix can move "accuracy" with no change in the model. Combined with the product rule, total probability gives Bayes' rule, the subject of the next lesson.

Key takeaways

• Probability is defined by three axioms (non-negativity, normalisation, additivity for disjoint events); complements, inclusion–exclusion and everything else follow.
• Conditioning restricts the sample space and renormalises. Chained together, conditionals give the exact factorisation $P(x_{1:T}) = \prod_t P(x_t \mid x_{<t})$ that autoregressive language models implement.
• Products of many probabilities underflow; work with summed log-probabilities.
• Independence makes failure probabilities multiply, which is why repeated sampling raises coverage as $1-(1-p)^N$. Correlation erodes that gain, and coverage is not the same as being able to pick the right answer.

Sources