The Future Of Product Management In The Age Of AI: Lessons From A Five Leader Panel Fireside Product Management podcast

Content provided by Tom Leung. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Tom Leung or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://podcastplayer.com/legal.

Fireside Product Management »
The Future of Product Management in the Age of AI: Lessons From a Five Leader Panel

2d ago 1:23:15

MP3•Episode home

Every few years, the world of product management goes through a phase shift. When I started at Microsoft in the early 2000s, we shipped Office in boxes. Product cycles were long, engineering was expensive, and user research moved at the speed of snail mail. Fast forward a decade and the cloud era reset the speed at which we build, measure, and learn. Then mobile reshaped everything we thought we knew about attention, engagement, and distribution.

Now we are standing at the edge of another shift. Not a small shift, but a tectonic one. Artificial intelligence is rewriting the rules of product creation, product discovery, product expectations, and product careers.

To help make sense of this moment, I hosted a panel of world class product leaders on the Fireside PM podcast:

• Rami Abu-Zahra, Amazon product leader across Kindle, Books, and Prime Video• Todd Beaupre, Product Director at YouTube leading Home and Recommendations• Joe Corkery, CEO and cofounder of Jaide Health • Tom Leung (me), Partner at Palo Alto Foundry• Lauren Nagel, VP Product at Mezmo• David Nydegger, Chief Product Officer at OvivaThese are leaders running massive consumer platforms, high stakes health tech, and fast moving developer tools. The conversation was rich, honest, and filled with specific examples.

This post summarizes the discussion, adds my own reflections, and offers a practical guide for early and mid career PMs who want to stay relevant in a world where AI is redefining what great product management looks like.

Table of Contents

* What AI Cannot Do and Why PM Judgment Still Matters

* The New AI Literacy: What PMs Must Know by 2026

* Why Building AI Products Speeds Up Some Cycles and Slows Down Others

* Whether the PM, Eng, UX Trifecta Still Stands

* The Biggest Risks AI Introduces Into Product Development

* Actionable Advice for Early and Mid Career PMs

* My Takeaways and What Really Matters Going Forward

* Closing Thoughts and Coaching Practice

1. What AI Cannot Do and Why PM Judgment Still Matters

We opened the panel with a foundational question. As AI becomes more capable every quarter, what is left for humans to do. Where do PMs still add irreplaceable value. It is the question every PM secretly wonders.

Todd put it simply: “At the end of the day, you have to make some judgment calls. We are not going to turn that over anytime soon.”

This theme came up again and again. AI is phenomenal at synthesizing, drafting, exploring, and narrowing. But it does not have conviction. It does not have lived experience. It does not feel user pain. It does not carry responsibility.

Joe from Jaide Health captured it perfectly when he said: “AI cannot feel the pain your users have. It can help meet their goals, but it will not get you that deep understanding.”

There is still no replacement for sitting with a frustrated healthcare customer who cannot get their clinical data into your system, or a creator on YouTube who feels the algorithm is punishing their art, or a devops engineer staring at an RCA output that feels 20 percent off.

Every PM knows this feeling: the moment when all signals point one way, but your gut tells you the data is incomplete or misleading. This is the craft that AI does not have.

Why judgment becomes even more important in an AI world

David, who runs product at a regulated health company, said something incredibly important: “Knowing what great looks like becomes more essential, not less. The PM's that thrive in AI are the ones with great product sense.”

This is counterintuitive for many. But when the operational work becomes automated, the differentiation shifts toward taste, intuition, sequencing, and prioritization.

Lauren asked the million dollar question. “How are we going to train junior PMs if AI is doing the legwork. Who teaches them how to think.”

This is a profound point. If AI closes the gap between junior and senior PMs in execution tasks, the difference will emerge almost entirely in judgment. Knowing how to probe user problems. Knowing when a feature is good enough. Knowing which tradeoffs matter. Knowing which flaw is fatal and which is cosmetic.

AI is incredible at writing a PRD. AI is terrible at knowing whether the PRD is any good.

Which means the future PM becomes more strategic, more intuitive, more customer obsessed, and more willing to make thoughtful bets under uncertainty.

2. The New AI Literacy: What PMs Must Know by 2026

I asked the panel what AI literacy actually means for PMs. Not the hype. Not the buzzwords. The real work.

Instead of giving gimmicky answers, the discussion converged on a clear set of skills that PMs must master.

Skill 1: Understanding context engineering

David laid this out clearly: “Knowing what LMS are good at and what they are not good at, and knowing how to give them the right context, has become a foundational PM skill.”

Most PMs think prompt engineering is about clever phrasing. In reality, the future is about context engineering. Feeding models the right data. Choosing the right constraints. Deciding what to ignore. Curating inputs that shape outputs in reliable ways.

Context engineering is to AI product development what Figma was to collaborative design. If you cannot do it, you are not going to be effective.

Skill 2: Evals, evals, evals

Rami said something that resonated with the entire panel: “Last year was all about prompts. This year is all about evals.”

He is right.

• How do you build a golden dataset.• How do you evaluate accuracy.• How do you detect drift.• How do you measure hallucination rates.• How do you combine UX evals with model evals.• How do you decide what good looks like.• How do you define safe versus unsafe boundaries.

AI evaluation is now a core PM responsibility. Not exclusively. But PMs must understand what engineers are testing for, what failure modes exist, and how to design test sets that reflect the real world.

Lauren said her PMs write evals side by side with engineering. That is where the world is going.

Skill 3: Knowing when to trust AI output and when to override it

Todd noted: “It is one thing to get an answer that sounds good. It is another thing to know if it is actually good.”

This is the heart of the role. AI can produce strategic recommendations that look polished, structured, and wise. But the real question is whether they are grounded in reality, aligned with your constraints, and consistent with your product vision.

A PM without the ability to tell real insight from confident nonsense will be replaced by someone who can.

Skill 4: Understanding the physics of model changes

This one surprised many people, but it was a recurring point.

Rami noted: “When you upgrade a model, the outputs can be totally different. The evals start failing. The experience shifts.”

PMs must understand:

• Models get deprecated• Models drift• Model updates can break well tuned prompts• API pricing has real COGS implications• Latency varies• Context windows vary• Some tasks need agents, some need RAG, some need a small finetuned model

This is product work now. The PM of 2026 must know these constraints as well as a PM of the cloud era understood database limits or API rate limits.

Skill 5: How to construct AI powered prototypes in hours, not weeks

It now takes one afternoon to build something meaningful. Zero code required. Prompt, test, refine. Whether you use Replit, Cursor, Vercel, or sandboxed agents, the speed is shocking.

But this makes taste and problem selection even more important. The future PM must be able to quickly validate whether a concept is worth building beyond the demo stage.

3. Why Building AI Products Speeds Up Some Cycles and Slows Down Others

This part of the conversation was fascinating because people expected AI to accelerate everything. The panel had a very different view.

Fast: Prototyping and concept validation

Lauren described how her teams can build working versions of an AI powered Root Cause Analysis feature in days, test it with customers, and get directional feedback immediately.

“You can think bigger because the cost of trying things is much lower,” she said.

For founders, early PMs, and anyone validating hypotheses, this is liberating. You can test ten ideas in a week. That used to take a quarter.

Slow: Productionizing AI features

The surprising part is that shipping the V1 of an AI feature is slower than most expect.

Joe noted: “You can get prototypes instantly. But turning that into a real product that works reliably is still hard.”

Why. Because:

• You need evals.• You need monitoring.• You need guardrails.• You need safety reviews.• You need deterministic parts of the workflow.• You need to manage COGS.• You need to design fallbacks.• You need to handle unpredictable inputs.• You need to think about hallucination risk.• You need new UI surfaces for non deterministic outputs.

Lauren said bluntly: “Vibe coding is fast. Moving that vibe code to production is still a four month process.”

This should be printed on a poster in every AI startup office.

Very Slow: Iterating on AI powered features

Another counterintuitive point. Many teams ship a great V1 but struggle to improve it significantly afterward.

David said their nutrition AI feature launched well but: “We struggled really hard to make it better. Each iteration was easy to try but difficult to improve in a meaningful way.”

Why is iteration so difficult.

Because model improvements may not translate directly into UX improvements. Users need consistency. Drift creates churn. Small changes in context or prompts can cause large changes in behavior.

Teams are learning a hard truth: AI powered features do not behave like typical deterministic product flows. They require new iteration muscles that most orgs do not yet have.

4. The PM, Eng, UX Trifecta in the AI Era

I asked whether the classic PM, Eng, UX triad is still the right model. The audience was expecting disagreement. The panel was surprisingly aligned.

The trifecta is not going anywhere

Rami put it simply: “We still need experts in all three domains to raise the bar.”

Joe added: “AI makes it possible for PMs to do more technical work. But it does not replace engineering. Same for design.”

AI blurs the edges of the roles, but it does not collapse them. In fact, each role becomes more valuable because the work becomes more abstract.

• PMs focus on judgment, sequencing, evaluation, and customer centric problem framing• Engineers focus on agents, systems, architecture, guardrails, latency, and reliability• Designers focus on dynamic UX, non deterministic UX patterns, and new affordances for AI outputs

What does change

AI makes the PM-Eng relationship more intense. The backbone of AI features is a combination of model orchestration, evaluation, prompting, and context curation. PMs must be tighter than ever with engineering to design these systems.

David noted that his teams focus more on individual talents. Some PMs are great at context engineering. Some designers excel at polishing AI generated layouts. Some engineers are brilliant at prompt chaining. AI reveals strengths quickly.

The trifecta remains. The skill distribution within it evolves.

5. The Biggest Risks AI Introduces Into Product Development

When we asked what scares PMs most about AI, the conversation became blunt and honest.

Risk 1: Loss of user trust

Lauren warned: “If people keep shipping low quality AI features, user trust in AI erodes. And then your good AI product suffers from the skepticism.”

This is very real. Many early AI features across industries are low quality, gimmicky, or unreliable. Users quickly learn to distrust these experiences.

Which means PMs must resist the pressure to ship before the feature is ready.

Risk 2: Skill atrophy

Todd shared a story that hit home for many PMs. “Junior folks just want to plug in the prompt and take whatever the AI gives them. That is a recipe for having no job later.”

PMs who outsource their thinking to AI will lose their judgment. Judgment cannot be regained easily.

This is the silent career killer.

Risk 3: Safety hazards in sensitive domains

David was direct: “If we have one unsafe output, we have to shut the feature off. We cannot afford even small mistakes.”

In healthcare, finance, education, and legal industries, the tolerance for error is near zero. AI must be monitored relentlessly. Human in the loop systems are mandatory. The cycles are slower but the stakes are higher.

Risk 4: The high bar for AI compared to humans

Joe said something I have thought about for years: “AI is held to a much higher standard than human decision making. Humans make mistakes constantly, but we forgive them. AI makes one mistake and it is unacceptable.”

This slows adoption in certain industries and creates unrealistic expectations.

Risk 5: Model deprecation and instability

Rami described a real problem AI PMs face: “Models get deprecated faster than they get replaced. The next model is not always GA. Outputs change. Prompts break.”

This creates product instability that PMs must anticipate and design around.

Risk 6: Differentiation becomes hard

I shared this perspective because I see so many early stage startups struggle with it.

If your whole product is a wrapper around an LLM, competitors will copy you in a week. The real differentiation will not come from using AI. It will come from how deeply you understand the customer, how you integrate AI with proprietary data, and how you create durable workflows.

6. Actionable Advice for Early and Mid Career PMs

This was one of my favorite parts of the panel because the advice was humble, practical, and immediately useful.

A. Develop deep user empathy. This will become your biggest differentiator.

Lauren said it clearly: “Maintain your empathy. Understand the pain your user really has.”

AI makes execution cheap. It makes insight valuable.

If you can articulate user pain precisely.If you can differentiate surface friction from underlying need.If you can see around corners.If you can prototype solutions and test them in hours.If you can connect dots between what AI can do and what users need.

You will thrive.

Tactical steps:

• Sit in on customer support calls every week.• Watch 10 user sessions for every feature you own.• Talk to customers until patterns emerge.• Ask “why” five times in every conversation.• Maintain a user pain log and update it constantly.

B. Become great at context engineering

This will matter as much as SQL mattered ten years ago.

Action steps:

• Practice writing prompts with structured context blocks.• Build a library of prompts that work for your product.• Study how adding, removing, or reordering context changes output.• Learn RAG patterns.• Learn when structured data beats embeddings.• Learn when smaller local models outperform big ones.

C. Learn eval frameworks

This is non negotiable.

You need to know:

• Precision vs recall tradeoffs• How to build golden datasets• How to design scenario based evals for UX• How to test for hallucination• How to monitor drift• How to set quality thresholds• How to build dashboards that reflect real world input distributions

You do not need to write the code.You do need to define the eval strategy.

D. Strengthen your product sense

You cannot outsource product taste.

Todd said it best: “Imagine asking AI to generate 20 percent growth for you. It will not tell you what great looks like.”

To strengthen your product sense:

• Review the best products weekly.• Take screenshots of great UX patterns.• Map user flows from apps you admire.• Break products down into primitives.• Ask yourself why a product decision works.• Predict what great would look like before you design it.

The PMs who thrive will be the ones who can recognize magic when they see it.

E. Stay curious

Rami’s closing advice was simple and perfect: “Stay curious. Keep learning. It never gets old.”

AI changes monthly. The PM who is excited by new ideas will outperform the PM who clings to old patterns.

Practical habits:

• Read one AI research paper summary each week.• Follow evaluation and model updates from major vendors.• Build at least one small AI prototype a month.• Join AI PM communities.• Teach juniors what you learn. Nothing accelerates mastery faster.

F. Embrace velocity and side projects

Todd said that some of his biggest career breakthroughs came from solving problems on the side.

This is more true now than ever.

If you have an idea, you can build an MVP over a weekend. If it solves a real problem, someone will notice.

G. Stay close to engineering

Not because you need to code, but because AI features require tighter PM engineering collaboration.

Learn enough to be dangerous:

• How embeddings work• How vector stores behave• What latency tradeoffs exist• How agents chain tasks• How model versioning works• How context limits shape UX• Why some prompts blow up API costs

If you can speak this language, you will earn trust and accelerate cycles.

H. Understand the business deeply

Joe’s advice was timeless: “Know who pays you and how much they pay. Solve real problems and know the business model.”

PMs who understand unit economics, COGS, pricing, and funnel dynamics will stand out.

7. Tom’s Takeaways and What Really Matters Going Forward

I ended the recording by sharing what I personally believe after moderating this discussion and working closely with a variety of AI teams over the past 2 years.

Judgment becomes the most valuable PM skill

As AI gets better at analysis, synthesis, and execution, your value shifts to:

• Choosing the right problem• Sequencing decisions• Making 55 45 calls• Understanding user pain• Making tradeoffs• Deciding when good is good enough• Defining success• Communicating vision• Influencing the org

Agents can write specs.LLMs can produce strategies.But only humans can choose the right one and commit.

Learning speed becomes a competitive advantage

I said this on the panel and I believe it more every month.

Because of AI, you now have:

• Infinite coaches• Infinite mentors• Infinite experts• Infinite documentation• Infinite learning loops

A PM who learns slowly will not survive the next decade.

Curiosity, empathy, and velocity will separate great from good

Many panelists said versions of this. The common pattern was:

• Understand users deeply• Combine multiple tools creatively• Move quickly• Learn constantly

The future rewards generalists with taste, speed, and emotional intelligence.

Differentiation requires going beyond wrapper apps

This is one of my biggest concerns for early stage founders. If your entire product is a wrapper around a model, you are vulnerable.

Durable value will come from:

• Proprietary data• Proprietary workflows• Deep domain insight• Organizational trust• Distribution advantage• Safety and reliability• Integration with existing systems

AI is a component, not a moat.

8. Closing Thoughts

Hosting this panel made me more optimistic about the future of product management. Not because AI will not change the job. It already has. But because the fundamental craft remains alive.

Product management has always been about understanding people, making decisions with incomplete information, telling compelling stories, and guiding teams through ambiguity and being right often.

AI accelerates the craft. It amplifies the best PMs and exposes the weak ones. It rewards curiosity, empathy, velocity, and judgment.

If you want tailored support on your PM career, leadership journey, or executive path, I offer 1 on 1 career, executive, and product coaching at tomleungcoaching.com.

OK team. Let’s ship greatness.

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit firesidepm.substack.com

111 episodes

#Tech #MBA #Business #Tom Leung #Productdevelopment #Productmanagement #Productivity