How Does Deep Learning Work?

How Does Deep Learning Work?

Reading Time: 1 minute

How Does Deep Learning Work?

A new theory seeks to explain the success of Deep Learning by highlighting generalization as a kind of algorithmic ‘forgetting.’

As an example, some photos of dogs might have houses in the background, while others don’t. As a network cycles through these training photos, it might “forget” the correlation between houses and dogs in some photos as other photos counteract it. It’s this forgetting of specifics, Tishby and Shwartz-Ziv argue, that enables the system to form general concepts. Indeed, their experiments revealed that deep neural networks ramp up their generalization performance during the compression phase, becoming better at labeling test data.


  1. Gideon Rosenblatt thank you for all these posts!! Enjoy your weekend, as I catch up on reading from you! Cheers

  2. You too, MicheleElys MER!

  3. It is the sum of the probabilities of myriad associations anyway. Selective remembering and forgetting are two sides of the same coin. It is a good perspective to have. This is perhaps a way of naming the elephant in the room.

  4. You’re the best Gideon Rosenblatt​! 👏

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Sign up here for the latest articles. You can opt out at any time.

Subscribe by email:

Or subscribe by RSS:

%d bloggers like this: