NeuroGenesys All reference videosAcademy • Reference video • 32:20Open on YouTube
Reinventing Entropy | Compression is Intelligence Part 1
3Blue1Brown
Why watch
Derives entropy from scratch, framed around prediction and compression.
Featured in
Watch next
33:513Blue1BrownBut what is cross-entropy? | Compression is Intelligence Part 2
The loss every language model minimizes, and its link to compression.
Watch
5:13Adian LiusieIntuitively Understanding the KL Divergence
A five-minute intuition for KL as the gap between cross-entropy and entropy.
Watch
30:383Blue1BrownSolving Wordle using information theory
Entropy as expected information gain, applied to a game.
Watch
Series16 videosJakob FoersterInformation Theory, Pattern Recognition, and Neural Networks
David MacKay's Cambridge lectures, the companion to his free textbook.
Open series
Hosted by YouTube and played through its privacy-enhanced embed. The creator is not affiliated with NeuroGenesys; a video is a way in, not a source.
