NeuroGenesys All reference videosAcademy • Reference video • 5:13Open on YouTube
Intuitively Understanding the KL Divergence
Adian Liusie
Why watch
A five-minute intuition for KL as the gap between cross-entropy and entropy.
Featured in
Watch next
32:203Blue1BrownReinventing Entropy | Compression is Intelligence Part 1
Derives entropy from scratch, framed around prediction and compression.
Watch
33:513Blue1BrownBut what is cross-entropy? | Compression is Intelligence Part 2
The loss every language model minimizes, and its link to compression.
Watch
30:383Blue1BrownSolving Wordle using information theory
Entropy as expected information gain, applied to a game.
Watch
Series16 videosJakob FoersterInformation Theory, Pattern Recognition, and Neural Networks
David MacKay's Cambridge lectures, the companion to his free textbook.
Open series
Hosted by YouTube and played through its privacy-enhanced embed. The creator is not affiliated with NeuroGenesys; a video is a way in, not a source.
