Go offline with the Player FM app!
Episode 08: Giancarlo Kerg, Mila, on approaching deep learning from mathematical foundations
Manage episode 289279639 series 2906499
Giancarlo Kerg (Google Scholar) is a PhD student at Mila, supervised by Yoshua Bengio and Guillaume Lajoie. He is working on out-of-distribution generalization and modularity in memory-augmented neural networks.
Highlights from our conversation:
🧮 Pure math foundations as an approach to progress and structural understanding in deep learning research
🧠How a formal proof on the way self-attention mitigates gradient vanishing when capturing long-term dependencies in RNNs led to a relevancy screening mechanism resembling human memory consolidation
🎯 Out-of-distribution generalization through modularity and inductive biases
37 episodes
Manage episode 289279639 series 2906499
Giancarlo Kerg (Google Scholar) is a PhD student at Mila, supervised by Yoshua Bengio and Guillaume Lajoie. He is working on out-of-distribution generalization and modularity in memory-augmented neural networks.
Highlights from our conversation:
🧮 Pure math foundations as an approach to progress and structural understanding in deep learning research
🧠How a formal proof on the way self-attention mitigates gradient vanishing when capturing long-term dependencies in RNNs led to a relevancy screening mechanism resembling human memory consolidation
🎯 Out-of-distribution generalization through modularity and inductive biases
37 episodes
All episodes
×Welcome to Player FM!
Player FM is scanning the web for high-quality podcasts for you to enjoy right now. It's the best podcast app and works on Android, iPhone, and the web. Signup to sync subscriptions across devices.