Search a title or topic

Over 20 million podcasts, powered by 

Player FM logo
Artwork

Content provided by PocketPod. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by PocketPod or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://podcastplayer.com/legal.
Player FM - Podcast App
Go offline with the Player FM app!

Reasoning Models are Bad at Thinking, Benchmarking LLMs for Medical and Physical World Understanding

10:32
 
Share
 

Manage episode 464518449 series 3568650
Content provided by PocketPod. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by PocketPod or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://podcastplayer.com/legal.

Today we explore how artificial intelligence may be rushing to conclusions instead of thinking deeply, as researchers discover that language models often jump between thoughts too quickly to solve complex problems. Scientists are developing new techniques to make AI pause and ponder, while a challenging new medical exam reveals just how far these systems still need to go to match human doctors' careful reasoning. These stories raise important questions about balancing AI's speed with the methodical thinking needed for critical tasks in healthcare and beyond. Links to all the papers we discussed: Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs, Streaming DiLoCo with overlapping communication: Towards a Distributed Free Lunch, MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understanding, PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding, WILDCHAT-50M: A Deep Dive Into the Role of Synthetic Data in Post-Training, Large Language Models Think Too Fast To Explore Effectively

  continue reading

145 episodes

Artwork
iconShare
 
Manage episode 464518449 series 3568650
Content provided by PocketPod. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by PocketPod or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://podcastplayer.com/legal.

Today we explore how artificial intelligence may be rushing to conclusions instead of thinking deeply, as researchers discover that language models often jump between thoughts too quickly to solve complex problems. Scientists are developing new techniques to make AI pause and ponder, while a challenging new medical exam reveals just how far these systems still need to go to match human doctors' careful reasoning. These stories raise important questions about balancing AI's speed with the methodical thinking needed for critical tasks in healthcare and beyond. Links to all the papers we discussed: Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs, Streaming DiLoCo with overlapping communication: Towards a Distributed Free Lunch, MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understanding, PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding, WILDCHAT-50M: A Deep Dive Into the Role of Synthetic Data in Post-Training, Large Language Models Think Too Fast To Explore Effectively

  continue reading

145 episodes

सभी एपिसोड

×
 
Loading …

Welcome to Player FM!

Player FM is scanning the web for high-quality podcasts for you to enjoy right now. It's the best podcast app and works on Android, iPhone, and the web. Signup to sync subscriptions across devices.

 

Listen to this show while you explore
Play