Go offline with the Player FM app!
The Challenge of AI Model Evaluations with Ankur Goyal
Manage episode 487918240 series 1418208
Evaluations are critical for assessing the quality, performance, and effectiveness of software during development. Common evaluation methods include code reviews and automated testing, and can help identify bugs, ensure compliance with requirements, and measure software reliability.
However, evaluating LLMs presents unique challenges due to their complexity, versatility, and potential for unpredictable behavior.
Ankur Goyal is the CEO and Founder of Braintrust Data, which provides an end-to-end platform for AI application development, and has a focus on making LLM development robust and iterative. Ankur previously founded Impira which was acquired by Figma, and he later ran the AI team at Figma. Ankur joins the show to talk about Braintrust and the unique challenges of developing evaluations in a non-deterministic context.
Sponsorship inquiries: [email protected]
The post The Challenge of AI Model Evaluations with Ankur Goyal appeared first on Software Engineering Daily.
875 episodes
Manage episode 487918240 series 1418208
Evaluations are critical for assessing the quality, performance, and effectiveness of software during development. Common evaluation methods include code reviews and automated testing, and can help identify bugs, ensure compliance with requirements, and measure software reliability.
However, evaluating LLMs presents unique challenges due to their complexity, versatility, and potential for unpredictable behavior.
Ankur Goyal is the CEO and Founder of Braintrust Data, which provides an end-to-end platform for AI application development, and has a focus on making LLM development robust and iterative. Ankur previously founded Impira which was acquired by Figma, and he later ran the AI team at Figma. Ankur joins the show to talk about Braintrust and the unique challenges of developing evaluations in a non-deterministic context.
Sponsorship inquiries: [email protected]
The post The Challenge of AI Model Evaluations with Ankur Goyal appeared first on Software Engineering Daily.
875 episodes
All episodes
×Welcome to Player FM!
Player FM is scanning the web for high-quality podcasts for you to enjoy right now. It's the best podcast app and works on Android, iPhone, and the web. Signup to sync subscriptions across devices.