Search a title or topic

Over 20 million podcasts, powered by 

Player FM logo
Artwork

Content provided by Leah Tharin. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Leah Tharin or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://podcastplayer.com/legal.
Player FM - Podcast App
Go offline with the Player FM app!

107: Andres Glusman: Why Your A/B Tests Are Lying (and How to Fix Them)

49:12
 
Share
 

Manage episode 483929587 series 3409156
Content provided by Leah Tharin. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Leah Tharin or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://podcastplayer.com/legal.

Andres Glusman, experimentation expert and former Meetup product leader, unpacks why 89% of experiments fail - and why that’s a good thing. From redesign pitfalls to balancing data with intuition, he shares hard-earned lessons on turning failure into explosive growth.

It’s not about running more tests, but smarter ones.

Timestamps & Segments

00:01:02 – 00:10:00
The Experimentation Conundrum
Andres breaks down why most experiments fail, the "VC mindset" for testing, and why a 65% loss rate is better than industry standards.

00:10:05 – 00:20:00
Low Traffic, Big Swings
Strategies for companies with limited traffic: why timid tweaks waste time and how to bundle changes for measurable impact.

00:20:05 – 00:30:00
Redesigns: When to Rip the Band-Aid
Why most redesigns backfire, how to test without tanking metrics, and the political minefield of shipping "VP-approved" flops.

00:30:05 – 00:40:00
Storing (and Ignoring) Experiment Data
The lifecycle of experiment insights, why social proof is overrated, and how to avoid repeating past mistakes (hint: spreadsheets).

00:40:05 – 00:49:00
Hot Takes & Must-Reads
Andres’ controversial read on stale A/B testing dogma + book recs to overhaul your product playbook.

Hot Takes
🔥 “89% failure rate? You’re doing it right.” – Andres
🔥 “If stats aren’t significant after 4 months, ship it anyway—your samples are probably garbage.” – Leah

Send us a text

Leah on Linkedin / Twitter / Youtube

  continue reading

107 episodes

Artwork
iconShare
 
Manage episode 483929587 series 3409156
Content provided by Leah Tharin. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Leah Tharin or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://podcastplayer.com/legal.

Andres Glusman, experimentation expert and former Meetup product leader, unpacks why 89% of experiments fail - and why that’s a good thing. From redesign pitfalls to balancing data with intuition, he shares hard-earned lessons on turning failure into explosive growth.

It’s not about running more tests, but smarter ones.

Timestamps & Segments

00:01:02 – 00:10:00
The Experimentation Conundrum
Andres breaks down why most experiments fail, the "VC mindset" for testing, and why a 65% loss rate is better than industry standards.

00:10:05 – 00:20:00
Low Traffic, Big Swings
Strategies for companies with limited traffic: why timid tweaks waste time and how to bundle changes for measurable impact.

00:20:05 – 00:30:00
Redesigns: When to Rip the Band-Aid
Why most redesigns backfire, how to test without tanking metrics, and the political minefield of shipping "VP-approved" flops.

00:30:05 – 00:40:00
Storing (and Ignoring) Experiment Data
The lifecycle of experiment insights, why social proof is overrated, and how to avoid repeating past mistakes (hint: spreadsheets).

00:40:05 – 00:49:00
Hot Takes & Must-Reads
Andres’ controversial read on stale A/B testing dogma + book recs to overhaul your product playbook.

Hot Takes
🔥 “89% failure rate? You’re doing it right.” – Andres
🔥 “If stats aren’t significant after 4 months, ship it anyway—your samples are probably garbage.” – Leah

Send us a text

Leah on Linkedin / Twitter / Youtube

  continue reading

107 episodes

Alle episoder

×
 
Loading …

Welcome to Player FM!

Player FM is scanning the web for high-quality podcasts for you to enjoy right now. It's the best podcast app and works on Android, iPhone, and the web. Signup to sync subscriptions across devices.

 

Listen to this show while you explore
Play