Guests

• David Eason, Principal Product Manager at Trainline
• Billie Bradley, Product Manager, Travel Assistant at Trainline
• Matt Farrelly, Head of AI and Machine Learning at Trainline

Key Takeaways

• AI assistants need both scalable reasoning and deep domain context to be useful.
• Tool design and guardrails are as critical as prompt design in agent systems.
• LLM-as-judge evals make it possible to measure open-ended systems without massive labeling costs.
• Even legacy companies can move fast when they embrace experimentation and tight PM–engineering collaboration.

Chapters: 00:00 Introduction and Team Introductions 00:51 Overview of Trainline's Mission and History 02:30 AI Integration in Trainline's Services 05:08 Challenges and Solutions in AI Implementation 06:52 Building and Iterating the AI Travel Assistant 14:58 User Experience and Guardrails 22:26 Technical Challenges and Solutions 34:29 The Challenge for Product Managers in AI 34:55 Billy's Background in AI 35:42 The Rapid Evolution of AI Technology 37:14 Managing Information Overload 37:58 Collaboration Between Product Managers and Engineers 38:42 Trainline's Approach to Machine Learning 39:36 Scaling Up: From 450 to 700,000 Pages 40:21 Challenges in Data Retrieval and Processing 45:55 Evaluating AI Assistants 48:22 The Role of LLM as Judges 50:19 User Context Simulation for Real-Time Evaluation 01:06:56 Future Directions for Trainline's AI Assistant