NeuroGenesys All reference videosAcademy • Video series • 16 videosOpen on YouTube
Information Theory, Pattern Recognition, and Neural Networks
Jakob Foerster
Why watch
David MacKay's Cambridge lectures, the companion to his free textbook.
This is a full series. The player starts at the first episode; use its playlist menu, or open it on YouTube, to pick an episode.
Featured in
Watch next
32:203Blue1BrownReinventing Entropy | Compression is Intelligence Part 1
Derives entropy from scratch, framed around prediction and compression.
Watch
33:513Blue1BrownBut what is cross-entropy? | Compression is Intelligence Part 2
The loss every language model minimizes, and its link to compression.
Watch
5:13Adian LiusieIntuitively Understanding the KL Divergence
A five-minute intuition for KL as the gap between cross-entropy and entropy.
Watch
30:383Blue1BrownSolving Wordle using information theory
Entropy as expected information gain, applied to a game.
Watch
Hosted by YouTube and played through its privacy-enhanced embed. The creator is not affiliated with NeuroGenesys; a video is a way in, not a source.
