Fixing AI's Suicide Problem TechFirst With John Koetsier podcast

Content provided by John Koetsier. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by John Koetsier or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://podcastplayer.com/legal.

TechFirst with John Koetsier »
Fixing AI's suicide problem

4d ago 16:38

MP3•Episode home

Is AI empathy a life-or-death issue? Almost a million people ask ChatGPT for mental health advice DAILY ... so yes, it kind of is.

Rosebud co-founder Sean Dadashi joins TechFirst to reveal new research on whether today’s largest AI models can recognize signs of self-harm ... and which ones fail. We dig into the Adam Raine case, talk about how Dadashi evaluated 22 leading LLMs, and explore the future of mental-health-aware AI.

We also talk about why Dadashi was interested in this in the first place, and his own journey with mental health.

00:00 — Intro: Is AI empathy a life-or-death matter?

00:41 — Meet Sean Dadashi, co-founder of Rosebud

01:03 — Why study AI empathy and crisis detection?

01:32 — The Adam Raine case and what it revealed

02:01 — Why crisis-prevention benchmarks for AI don’t exist

02:48 — How Rosebud designed the study across 22 LLMs

03:17 — No public self-harm response benchmarks: why that’s a problem

03:46 — Building test scenarios based on past research and real cases

04:33 — Examples of prompts used in the study

04:54 — Direct vs indirect self-harm cues and why AIs miss them

05:26 — The bridge example: AI’s failure to detect subtext

06:14 — Did any models perform well?

06:33 — All 22 models failed at least once

06:47 — Lower-performing models: GPT-40, Grok

07:02 — Higher-performing models: GPT-5, Gemini

07:31 — Breaking news: Gemini 3 preview gets the first perfect score

08:12 — Did the benchmark influence model training?

08:30 — The need for more complex, multi-turn testing

08:47 — Partnering with foundation model companies on safety

09:21 — Why this is such a hard problem to solve

10:34 — The scale: over a million people talk to ChatGPT weekly about self-harm

11:10 — What AI should do: detect subtext, encourage help, avoid sycophancy

11:42 — Sycophancy in LLMs and why it’s dangerous

12:17 — The potential good: AI can help people who can’t access therapy

13:06 — Could Rosebud spin this work into a full-time safety project?

13:48 — Why the benchmark will be open-source

14:27 — The need for a third-party “Better Business Bureau” for LLM safety

14:53 — Sean’s personal story of suicidal ideation at 16

15:55 — How tech can harm — and help — young, vulnerable people

16:32 — The importance of giving people time, space, and hope

17:39 — Final reflections: listening to the voice of hope

18:14 — Closing

345 episodes

#News #Tech News #John Koetsier