Six Hard Lessons From Building With AI Agents
Manage episode 498218458 series 3667014
In this episode of the Vernon Richard show, the hosts discuss their experiences with AI tools and agents, focusing on the challenges and lessons learned from using these technologies in coding and software engineering. They explore best practices for utilizing AI effectively, the importance of context in interactions with AI, and the future of AI agents in the workplace. The conversation highlights the balance between leveraging AI for efficiency while maintaining control and understanding of the underlying processes.
Links to stuff we mentioned during the pod:
- 09:16 - The LinkedIn post talking about Replit messing with someone's production code 😳
- And the link to the thread of person who went through it
- The tool in question, Replit
- 13:01 - Rich's LinkedIn post with his tips
- 14:21 - GitHub Copilot
- 18:09 - VS Code
- 29:01 - Folks at different ends of the "AI Enthusiasm Spectrum"
- On the enthusiastic end
- Jason Arbon is on the positive side and is always creating something interesting like...
- On the unenthusiastic end
- Keith Klain has created a reading list to help get us up to speed...
- Keith's reading AI reading list
- You can see his full resources list here
- Maaike Brinkhof has a bunch of thought-provoking posts on the topic...
- Keith Klain has created a reading list to help get us up to speed...
- On the enthusiastic end
- 34:44 - Want to know what "conflabulation" means? Listen to Martin explain it on the Ghost in th code podcast (that's not a typo!)
- 37:24 - What is Context Engineering? Perplexity has answers!
- 46:38 - The legendary Lt. Geordi La Forge from Star Trek: The Next Generation.
- 51:48 - After recording, the very cool Paul Coles published his article The Subtle Art of Herding Cats: Why AI Agents Ignore Your Rules (Part 1 of 4, explaining the topic of Context Engineering. It’s brilliant!
- 59:04 - The promises of technology over the years...
- 60:50 - The always insightful Meredith Whittaker of Signal fame, where is the president and services on its board of directors, explains the privacy and security concerns with agentic technology.
Watch the clip, then go back and watch the whole thing!
00:00 - Intro
01:17 - Welcome
01:30 - TANGENT BEGINS... All kinds of egregious waffling follows. Skip to the actual content at 08:34
01:31 - Rich VS Tree Stump
01:57 - What on earth did Rich need the pulley for?
02:26 - Vern's nerdy confession and pulley confusion
02:52 - Does Rich live next door to Tony Stark?!
03:22 - What to do when you need a steel RSJ
03:35 - We admit defeat. 03:36 - Welcome to Rich's Garden Adventures Podcast!
07:25 - What has Vern been up to?
08:34 - We attempt to segue into the episode at last!
08:35 - TANGENT ENDS...
08:51 - Rich’s POC: using agents to help build AI tools
09:45 - The Replit disaster: vibe coding meets deleted production data 11:12 - Sociopathic assistants and the case for AI gaslighting 11:55 - Vernon wants his team experimenting with AI tools
12:50 - Rich explains the context for his latest AI adventures
13:18 - Rich’s bench project and “putting the engineering hat on”
15:22 - Setting up the stack and staying in control
16:53 - A familiar story: things were going fine until they weren’t
17:00 - Ask vs Edit vs Agent mode in Copilot explained
19:06 - The innocent linting error that spiralled out of control
21:16 - Stuck in a loop: “I didn’t know what it was doing, but I let it keep going”
22:11 - The fateful click: “I’m going to reset the DB”
23:10 - The aftermath: no data, no damage… but very nearly
23:33 - Security wake-up call: agents are acting as you
24:39 - You can’t fix what you don’t know it broke
25:52 - Can you interrupt an agent mid-task?
27:14 - When agents get “are you sure?” moments
28:15 - Tea breaks as a dev strategy: outsourcing work to agents
29:24 - Jason Aborn vs Keith & Maaike: where Rich sits on the AI enthusiasm spectrum
30:41 - Tip1. The first of Rich’s 6 agent tips: commit after every interaction
32:12 - Why trusting the “keep all” button is risky
34:01 - Writing your own commits vs letting the agent do it
35:26 - When agents lose the plot: reset instead of fixing
36:55 - “You’re insane now, GPT. I’m giving you a break.”
37:54 - Tip 2: Make the task as small as possible
39:59 - The middle ground between 'ask' and full agent delegation
41:12 - Tip 3: Ask the agent to break the task down for you
43:36 - The order matters: why you shouldn’t start with the form UI
44:33 - Vernon compares it to shell command pipelines
45:09 - It can now open browsers and run Playwright tests (!)
46:23 - Star Trek and the rise of the engineer-agent hybrid
47:57 - Tips 4–6: Test often, review the code, use other models
49:39 - Pattern drift and the importance of prompt templates
50:51 - Vernon’s nemesis: m dashes, emojis, and being ignored by GPT
51:48 - Context engineering vs prompt engineering
52:43 - When codebases get too big for agents to cope
53:40 - Why agents sometimes act dumber than your IDE
54:32 - The danger of outsourcing good practices to AI
54:48 - Spoilers: Rich’s upcoming keynote at TestIt
55:01 - Agents don’t ask why — they just keep going
56:42 - Goals vs loops: when failure isn’t part of the plan
58:32 - The question of efficiency: is training agents worth it?
59:47 - Rich’s take: we’ll buy agents like we buy SaaS
61:08...
31 episodes