We already have many of the climate solutions we need. But scaling them is hard. The Green Blueprint is a show about the people who are architecting the clean economy. Every other week, host Lara Pierpoint profiles the founders, investors, and organizational leaders who are solving complex challenges in the quest to build climate technologies fast.
…
continue reading
Content provided by Peter Yang. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Peter Yang or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://podcastplayer.com/legal.
Similar to Behind the Craft
Thanks for visiting The Cell Phone Junkie! I will be taking the time each week to discuss my favorite topic, cell phones. Any feedback is appreciated and welcome. You can email me at: questions (AT) thecellphonejunkie (DOT) com or call: 206-203-3734 Thanks and welcome!
…
continue reading
Show notes are at https://stevelitchfield.com/sshow/chat.html
…
continue reading
The director’s commentary track for Daring Fireball. Long digressions on Apple, technology, design, movies, and more.
…
continue reading
WSJ’s Bold Names brings you conversations with the leaders of the bold-named companies featured in the pages of The Wall Street Journal. Hosts Tim Higgins and Christopher Mims speak to CEOs and business leaders in interviews that challenge conventional wisdom and take you inside the decisions being made in the C-suite and beyond.
…
continue reading
A podcast for developers interested in building great software products. Every episode, Adam Wathan is joined by a guest to talk about everything from product design and user experience to unit testing and system administration.
…
continue reading
Download This Show is your weekly guide to the world of media, culture, and technology. From social media to gadgets, streaming services to privacy issues. Each week Rae Johnston and guests take a fun, deep dive into how technology is reshaping our lives.
…
continue reading
The power of Data is undeniable. And unharnessed - it’s nothing but chaos. Making data your ally. Using it to lead with confidence and clarity. Host Jess Carter is solving problems in real-time to reveal what’s possible. Helping communities and people thrive. This is Data Driven Leadership, a show brought to you by Resultant.
…
continue reading
We help founders make something people want.
…
continue reading
Android Backstage, a podcast by and for Android developers. Hosted by developers from the Android engineering team, this show covers topics of interest to Android programmers, with in-depth discussions and interviews with engineers on the Android team at Google. Subscribe to Android Developers YouTube → https://goo.gle/AndroidDevs
…
continue reading
Player FM - Podcast App
Go offline with the Player FM app!
Go offline with the Player FM app!
))
Complete Beginner's Course on AI Evaluations: Step by Step (2025) | Aman Khan
Manage episode 502161037 series 3621237
Content provided by Peter Yang. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Peter Yang or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://podcastplayer.com/legal.
Today, I want to share a new episode with Aman Khan.The best way to learn about AI evaluations is to watch 2 PMs build them live from scratch. In our new episode, Aman and I walk through creating evals for an AI customer support agent — from labeling a golden dataset to aligning LLM judges. This is the complete beginners AI eval course you've been waiting for.Aman and I talked about:
(00:00) What are AI evals and how to get good at them
(02:52) The 4 types of AI evaluations everyone should know
(06:08) Live demo: Building evals for a customer support agent
(10:29) Using Anthropic's console to generate great prompts
(15:13) Creating the evaluation criteria
(17:40) Adding human labels to the golden dataset
(31:05) Scaling evals with LLM-judge prompts
(38:21) How to align LLM judges with human judgmentGet the takeaways: https://creatoreconomy.so/p/complete-beginner-course-on-ai-evaluations-aman-khanWhere to find Aman:
X: https://www.linkedin.com/in/amanberkeley/
Website: https://arize.com/📌 Subscribe to this channel – more interviews coming soon!
71 episodes
Share
Manage episode 502161037 series 3621237
Content provided by Peter Yang. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Peter Yang or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://podcastplayer.com/legal.
Today, I want to share a new episode with Aman Khan.The best way to learn about AI evaluations is to watch 2 PMs build them live from scratch. In our new episode, Aman and I walk through creating evals for an AI customer support agent — from labeling a golden dataset to aligning LLM judges. This is the complete beginners AI eval course you've been waiting for.Aman and I talked about:
(00:00) What are AI evals and how to get good at them
(02:52) The 4 types of AI evaluations everyone should know
(06:08) Live demo: Building evals for a customer support agent
(10:29) Using Anthropic's console to generate great prompts
(15:13) Creating the evaluation criteria
(17:40) Adding human labels to the golden dataset
(31:05) Scaling evals with LLM-judge prompts
(38:21) How to align LLM judges with human judgmentGet the takeaways: https://creatoreconomy.so/p/complete-beginner-course-on-ai-evaluations-aman-khanWhere to find Aman:
X: https://www.linkedin.com/in/amanberkeley/
Website: https://arize.com/📌 Subscribe to this channel – more interviews coming soon!
71 episodes
All episodes
×
Welcome to Player FM!
Player FM is scanning the web for high-quality podcasts for you to enjoy right now. It's the best podcast app and works on Android, iPhone, and the web. Signup to sync subscriptions across devices.
Similar to Behind the Craft
We already have many of the climate solutions we need. But scaling them is hard. The Green Blueprint is a show about the people who are architecting the clean economy. Every other week, host Lara Pierpoint profiles the founders, investors, and organizational leaders who are solving complex challenges in the quest to build climate technologies fast.
…
continue reading
Thanks for visiting The Cell Phone Junkie! I will be taking the time each week to discuss my favorite topic, cell phones. Any feedback is appreciated and welcome. You can email me at: questions (AT) thecellphonejunkie (DOT) com or call: 206-203-3734 Thanks and welcome!
…
continue reading
Show notes are at https://stevelitchfield.com/sshow/chat.html
…
continue reading
The director’s commentary track for Daring Fireball. Long digressions on Apple, technology, design, movies, and more.
…
continue reading
WSJ’s Bold Names brings you conversations with the leaders of the bold-named companies featured in the pages of The Wall Street Journal. Hosts Tim Higgins and Christopher Mims speak to CEOs and business leaders in interviews that challenge conventional wisdom and take you inside the decisions being made in the C-suite and beyond.
…
continue reading
A podcast for developers interested in building great software products. Every episode, Adam Wathan is joined by a guest to talk about everything from product design and user experience to unit testing and system administration.
…
continue reading
Download This Show is your weekly guide to the world of media, culture, and technology. From social media to gadgets, streaming services to privacy issues. Each week Rae Johnston and guests take a fun, deep dive into how technology is reshaping our lives.
…
continue reading
The power of Data is undeniable. And unharnessed - it’s nothing but chaos. Making data your ally. Using it to lead with confidence and clarity. Host Jess Carter is solving problems in real-time to reveal what’s possible. Helping communities and people thrive. This is Data Driven Leadership, a show brought to you by Resultant.
…
continue reading
We help founders make something people want.
…
continue reading
Android Backstage, a podcast by and for Android developers. Hosted by developers from the Android engineering team, this show covers topics of interest to Android programmers, with in-depth discussions and interviews with engineers on the Android team at Google. Subscribe to Android Developers YouTube → https://goo.gle/AndroidDevs
…
continue reading
Player FM - Podcast App
Go offline with the Player FM app!
Go offline with the Player FM app!
