The Vertical AI Advantage: Lessons from Building GenAI Products for Lawyers

Fireside Product Management

Content provided by Tom Leung. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Tom Leung or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://podcastplayer.com/legal.

5M ago 38:09

MP3•Episode home

Earlier on the Fireside PM podcast, I sat down with Carl Wu, a veteran product leader who built and launched an AI-first product from scratch—targeting one of the most conservative and risk-averse professions out there: immigration law.

Carl's story isn't just a case study in GenAI product development. It's a case study in how technical intuition, product fundamentals, and real-world empathy for users come together when you're building for high-stakes use cases.

If you're building (or planning to build) AI-native products—especially in a vertical domain—this one's for you.

From Code to Customer: Carl's Unusual Arc

Carl started his career as an engineer at Microsoft before transitioning into product. He later built search engines at Tencent and led teams building video and ML-powered systems at startups.

His technical fluency isn't just a badge of honor; it's the lens through which he approaches product thinking. "The biggest mental switch," he said, "was thinking less about system optimization and more about user optimization. But having that technical foundation helped me build credibility and intuition."

That background came in handy when he joined a legal-tech startup as their founding AI PM, tasked with turning foundational models into real customer value.

AI is Powerful. PM Fundamentals Still Matter More.

Carl didn’t come in trying to train the biggest model or chase the buzziest trends. His first question was simple: What’s the most painful, expensive problem we can solve with this tech?

That led to a set of vertical AI theses:

* Focus on domains where language is the product

* Prioritize workflows with high structure and high stakes

* Use the LLM for synthesis, drafting, and structured transformation

Legal fit perfectly. Immigration law, in particular, had everything he wanted: repeatable document types, expensive expert time, and huge amounts of unstructured data ripe for automation.

Carl explained:

"We were working in immigration law, and saw that some law firms were outsourcing their drafting to journalists because the petitions were so complex. That was the lightbulb. If someone is paying a human writer to stitch together legal arguments, an LLM might be able to help."

That insight narrowed the use case to a single visa type—one that law firms actively avoided because of the overhead.

Actionable Advice: Find the Burning Problem

Too many PMs start with the model and go hunting for a problem. Carl did the reverse:

* Pick a high-value domain

* Talk to users (lawyers)

* Observe workflows

* Identify pain so acute that firms were outsourcing or avoiding it

Takeaway for PMs: Your GenAI MVP shouldn't be an experiment. It should be a wedge into a critical workflow where users already know they need help.

Taking the Technology Risk So the User Doesn’t Have To

Carl had a tough call to make: Should they require users to fill out guided prompts and forms, or should they lean fully into autonomous generation from source docs?

He chose the latter, betting that removing all user friction—even at the cost of increased technical risk—would pay off.

"I decided that in a 0-to-1 product, especially one this disruptive, we should optimize for user experience and absorb complexity on the system side."

The result? Documents that used to take lawyers six months to draft could now be generated and reviewed in 48 hours.

Prompt Engineering Is a System, Not a Skill

One of the most eye-opening parts of our conversation was how Carl talked about prompt systems. Not as static prompts. Not as clever tokens. But as a full-stack orchestration layer that included:

* Smart retrieval from unstructured documents

* Chained prompts and intermediate reasoning steps

* Evaluation systems to assess output quality

"It’s not just writing a good prompt," Carl said. "You need a full evaluation stack. In our case, that included using GPT-4.5 as an evaluator model to score drafts generated by cheaper, faster models."

For example:

* Drafts were scored on legal logic, writing style, and argument rigor

* Outputs were linked back to citations and source documents to reduce hallucinations

* Users could rate and comment on individual sections to create a feedback loop

Pro tip for PMs: Build your evaluation stack early. Hallucinations are product-killers in high-trust domains. Don’t rely on vibes.

Integration and Compliance Are Features, Not Afterthoughts

One of the hardest parts of going from demo to deployment was integration with legacy systems—and gaining trust from clients concerned about privacy and compliance.

"Clients are asking new questions now. Who trained your model? Where is the data stored? How do we know our documents aren’t being used to retrain the model?"

This is where Carl's vertical AI strategy paid off. By focusing on a niche domain, the team could:

* Build tight integrations with specific case management tools

* Offer clear guarantees around data residency and model usage

* Design workflows that mirrored existing processes, not replaced them

What Carl Would Do Differently

Despite the success, Carl reflected on one thing he might have underinvested in:

"In hindsight, I think we could’ve done more on the user experience layer. Not just the data outputs, but how those outputs are presented, edited, and refined by the user. UX is perception. And perception is reality."

He pointed to Midjourney as an example:

* Many models can generate images

* But Midjourney added affordances like zoom, re-prompt, and edit

* That made the tool feel alive, adaptable, and human-friendly

Takeaway: Don’t treat UI as a wrapper. It’s a co-pilot.

What PMs Get Wrong About AI

We wrapped up the conversation with one of my favorite questions: What do most PMs get wrong about AI?

"PMs overestimate what AI can do and underestimate the importance of the core use case. Just because it feels magical doesn’t mean you can skip the fundamentals."

In other words:

* Don’t get blinded by novelty

* Solve a real, valuable problem

* Make it work before you make it scale

Final Thoughts: It’s a Golden Age for Scrappy Builders

Carl ended our conversation with a quiet bombshell:

"Five years ago, people would assume you'd need a 30-person team to build this. Today, a handful of builders can launch vertical AI startups serving million-dollar use cases."

That stuck with me.

We’re not just witnessing the rise of foundational models. We’re seeing the birth of a new generation of product teams—tiny, focused, fast-moving, and capable of punching way above their weight.

If you're early in your PM journey, and you want to be part of this shift:

* Learn the fundamentals (value, user pain, workflows)

* Embrace ambiguity (AI is still unpredictable)

* Be technical enough to evaluate what's feasible

* Be empathetic enough to know what matters

And if you want help accelerating your journey, I offer 1:1 coaching at tomleungcoaching.com and product consulting services at paloaltofoundry.com.

Let’s get back to work.

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit firesidepm.substack.com

106 episodes

#Tech #MBA #Business #Tom Leung #Productdevelopment #Productmanagement #Productivity

The Vertical AI Advantage: Lessons from Building GenAI Products for Lawyers

Fireside Product Management

14 subscribers

published 5M ago