Android Backstage, a podcast by and for Android developers. Hosted by developers from the Android engineering team, this show covers topics of interest to Android programmers, with in-depth discussions and interviews with engineers on the Android team at Google. Subscribe to Android Developers YouTube → https://goo.gle/AndroidDevs
…
continue reading
Content provided by TWIML and Sam Charrington. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by TWIML and Sam Charrington or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://player.fm/legal.
Similar to The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)
The Data Skeptic Podcast features interviews and discussion of topics related to data science, statistics, machine learning, artificial intelligence and the like, all from the perspective of applying critical thinking and the scientific method to evaluate the veracity of claims and efficacy of approaches.
…
continue reading
BBC Radio 5 live’s award winning gaming podcast, discussing the world of video games and games culture.
…
continue reading
Flash Forward is a show about possible (and not so possible) future scenarios. What would the warranty on a sex robot look like? How would diplomacy work if we couldn’t lie? Could there ever be a fecal transplant black market? (Complicated, it wouldn’t, and yes, respectively, in case you’re curious.) Hosted and produced by award winning science journalist Rose Eveleth, each episode combines audio drama and journalism to go deep on potential tomorrows, and uncovers what those futures might re ...
…
continue reading
The internet is broken—but it doesn’t have to be. If you’re concerned about how surveillance, online advertising, and automated content moderation are hurting us online and offline, the Electronic Frontier Foundation’s How to Fix the Internet podcast offers a better way forward. EFF has been defending your rights online for over thirty years and is behind many of the biggest digital rights protections since the invention of the internet. Through curious conversations with some of the leading ...
…
continue reading
Show notes are at https://stevelitchfield.com/sshow/chat.html
…
continue reading
Hanselminutes is Fresh Air for Developers. A weekly commute-time podcast that promotes fresh technology and fresh voices. Talk and Tech for Developers, Life-long Learners, and Technologists.
…
continue reading
The 21st Century began with the rise of the Internet and social media. The next decade will mark the rise of the Intelligent Machines. AI will inhabit all our devices from cars and appliances to smart phones and robots. The Intelligent Machines podcast explores the most exciting revolution humanity has ever seen, filled with promise and peril. More than ever we need to understand what these new devices will bring to our lives and how to make best use of them as the 21st century unfolds. On t ...
…
continue reading
Tech Life discovers and explains the ways technology is changing our lives, wherever we are in the world. We meet the people with bright ideas for rethinking the way we work, learn and play, and get hands-on with the products they dream up. We hold tech giants to account for their huge power to affect our lives, and ask who wins, and who loses, in the technology transformation. Tech Life is your guide to a future being made, and remade, at lightning speed in front of our eyes.
…
continue reading
Why do only 10% of companies succeed with AI? In this series by MIT SMR and BCG, we talk to the leaders who've achieved big wins with AI in their companies and learn how they did it. Hear what gets experts from companies like NASA, Github, and others excited to do their jobs every day and what they consider the keys to their success.
…
continue reading
Player FM - Podcast App
Go offline with the Player FM app!
Go offline with the Player FM app!
))
Exploring the Biology of LLMs with Circuit Tracing with Emmanuel Ameisen - #727
Manage episode 477036739 series 2355587
Content provided by TWIML and Sam Charrington. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by TWIML and Sam Charrington or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://podcastplayer.com/legal.
In this episode, Emmanuel Ameisen, a research engineer at Anthropic, returns to discuss two recent papers: "Circuit Tracing: Revealing Language Model Computational Graphs" and "On the Biology of a Large Language Model." Emmanuel explains how his team developed mechanistic interpretability methods to understand the internal workings of Claude by replacing dense neural network components with sparse, interpretable alternatives. The conversation explores several fascinating discoveries about large language models, including how they plan ahead when writing poetry (selecting the rhyming word "rabbit" before crafting the sentence leading to it), perform mathematical calculations using unique algorithms, and process concepts across multiple languages using shared neural representations. Emmanuel details how the team can intervene in model behavior by manipulating specific neural pathways, revealing how concepts are distributed throughout the network's MLPs and attention mechanisms. The discussion highlights both capabilities and limitations of LLMs, showing how hallucinations occur through separate recognition and recall circuits, and demonstrates why chain-of-thought explanations aren't always faithful representations of the model's actual reasoning. This research ultimately supports Anthropic's safety strategy by providing a deeper understanding of how these AI systems actually work.
The complete show notes for this episode can be found at https://twimlai.com/go/727.
747 episodes
Exploring the Biology of LLMs with Circuit Tracing with Emmanuel Ameisen - #727
The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)
Share
Manage episode 477036739 series 2355587
Content provided by TWIML and Sam Charrington. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by TWIML and Sam Charrington or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://podcastplayer.com/legal.
In this episode, Emmanuel Ameisen, a research engineer at Anthropic, returns to discuss two recent papers: "Circuit Tracing: Revealing Language Model Computational Graphs" and "On the Biology of a Large Language Model." Emmanuel explains how his team developed mechanistic interpretability methods to understand the internal workings of Claude by replacing dense neural network components with sparse, interpretable alternatives. The conversation explores several fascinating discoveries about large language models, including how they plan ahead when writing poetry (selecting the rhyming word "rabbit" before crafting the sentence leading to it), perform mathematical calculations using unique algorithms, and process concepts across multiple languages using shared neural representations. Emmanuel details how the team can intervene in model behavior by manipulating specific neural pathways, revealing how concepts are distributed throughout the network's MLPs and attention mechanisms. The discussion highlights both capabilities and limitations of LLMs, showing how hallucinations occur through separate recognition and recall circuits, and demonstrates why chain-of-thought explanations aren't always faithful representations of the model's actual reasoning. This research ultimately supports Anthropic's safety strategy by providing a deeper understanding of how these AI systems actually work.
The complete show notes for this episode can be found at https://twimlai.com/go/727.
747 episodes
Alle Folgen
×
Welcome to Player FM!
Player FM is scanning the web for high-quality podcasts for you to enjoy right now. It's the best podcast app and works on Android, iPhone, and the web. Signup to sync subscriptions across devices.
Similar to The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)
Android Backstage, a podcast by and for Android developers. Hosted by developers from the Android engineering team, this show covers topics of interest to Android programmers, with in-depth discussions and interviews with engineers on the Android team at Google. Subscribe to Android Developers YouTube → https://goo.gle/AndroidDevs
…
continue reading
The Data Skeptic Podcast features interviews and discussion of topics related to data science, statistics, machine learning, artificial intelligence and the like, all from the perspective of applying critical thinking and the scientific method to evaluate the veracity of claims and efficacy of approaches.
…
continue reading
BBC Radio 5 live’s award winning gaming podcast, discussing the world of video games and games culture.
…
continue reading
Flash Forward is a show about possible (and not so possible) future scenarios. What would the warranty on a sex robot look like? How would diplomacy work if we couldn’t lie? Could there ever be a fecal transplant black market? (Complicated, it wouldn’t, and yes, respectively, in case you’re curious.) Hosted and produced by award winning science journalist Rose Eveleth, each episode combines audio drama and journalism to go deep on potential tomorrows, and uncovers what those futures might re ...
…
continue reading
The internet is broken—but it doesn’t have to be. If you’re concerned about how surveillance, online advertising, and automated content moderation are hurting us online and offline, the Electronic Frontier Foundation’s How to Fix the Internet podcast offers a better way forward. EFF has been defending your rights online for over thirty years and is behind many of the biggest digital rights protections since the invention of the internet. Through curious conversations with some of the leading ...
…
continue reading
Show notes are at https://stevelitchfield.com/sshow/chat.html
…
continue reading
Hanselminutes is Fresh Air for Developers. A weekly commute-time podcast that promotes fresh technology and fresh voices. Talk and Tech for Developers, Life-long Learners, and Technologists.
…
continue reading
The 21st Century began with the rise of the Internet and social media. The next decade will mark the rise of the Intelligent Machines. AI will inhabit all our devices from cars and appliances to smart phones and robots. The Intelligent Machines podcast explores the most exciting revolution humanity has ever seen, filled with promise and peril. More than ever we need to understand what these new devices will bring to our lives and how to make best use of them as the 21st century unfolds. On t ...
…
continue reading
Tech Life discovers and explains the ways technology is changing our lives, wherever we are in the world. We meet the people with bright ideas for rethinking the way we work, learn and play, and get hands-on with the products they dream up. We hold tech giants to account for their huge power to affect our lives, and ask who wins, and who loses, in the technology transformation. Tech Life is your guide to a future being made, and remade, at lightning speed in front of our eyes.
…
continue reading
Why do only 10% of companies succeed with AI? In this series by MIT SMR and BCG, we talk to the leaders who've achieved big wins with AI in their companies and learn how they did it. Hear what gets experts from companies like NASA, Github, and others excited to do their jobs every day and what they consider the keys to their success.
…
continue reading
Player FM - Podcast App
Go offline with the Player FM app!
Go offline with the Player FM app!
