NeuroGenesys All reference videosAcademy • Reference video • 30:38Open on YouTube
Solving Wordle using information theory
3Blue1Brown
Why watch
Entropy as expected information gain, applied to a game.
Featured in
Watch next
32:203Blue1BrownReinventing Entropy | Compression is Intelligence Part 1
Derives entropy from scratch, framed around prediction and compression.
Watch
33:513Blue1BrownBut what is cross-entropy? | Compression is Intelligence Part 2
The loss every language model minimizes, and its link to compression.
Watch
5:13Adian LiusieIntuitively Understanding the KL Divergence
A five-minute intuition for KL as the gap between cross-entropy and entropy.
Watch
Series16 videosJakob FoersterInformation Theory, Pattern Recognition, and Neural Networks
David MacKay's Cambridge lectures, the companion to his free textbook.
Open series
Hosted by YouTube and played through its privacy-enhanced embed. The creator is not affiliated with NeuroGenesys; a video is a way in, not a source.
