Best Data Pipeline Architecture Podcasts (2025)

1
Scaling AI in Engineering with Peter Bell | Ep. 7 27:16

Play Pause

3d ago27:16

27:16

Listen: https://confluent.buzzsprout.com | Today, Adi Polak talks to her guest, Peter Bell (gather.dev), about his career in software engineering leadership, CTO community building, and AI-driven development. Peter’s first job: electronics lab technician at their school (alongside shifts at Tesco). His challenge/theme: working at scale with AI adop…

1
Bridging the AI–Data Gap: Collect, Curate, Serve 50:40

4d ago50:40

50:40

Summary In this episode of the Data Engineering Podcast Omri Lifshitz (CTO) and Ido Bronstein (CEO) of Upriver talk about the growing gap between AI's demand for high-quality data and organizations' current data practices. They discuss why AI accelerates both the supply and demand sides of data, highlighting that the bottleneck lies in the "middle …

1
AI, IDEs, Copilot & Critical Thinking 53:20

7d ago53:20

53:20

Share Episode Microsoft's John Papa, Partner General Manager of Developer Relations for all things dev and code joins the show to talk developer relations...from his Mac. He reveals his small part in the birth of VS Code (back when its codename was Ticino) after he spent a year trying a new editor every month. The conversation dives deep into "Agen…

1
How Kafka Expert Robin Moffat Tackles Open Source Problems | Ep. 6 24:50

10d ago24:50

24:50

Today, Viktor Gamov talks to his colleague Robin Moffat (Confluent) about his career in data engineering. His first job: paperboy. His challenge: working at a retailer with Oracle materialized views as well as teaching others how to productively approach Kafka’s internal systems. Blog posts mentioned in the podcast: ► Oracle Materialized Views trou…

1
Beyond the Perimeter: Practical Patterns for Fine‑Grained Data Access 1:05:00

11d ago1:05:00

1:05:00

Summary In this episode of the Data Engineering Podcast Matt Topper, president of UberEther, talks about the complex challenge of identity, credentials, and access control in modern data platforms. With the shift to composable ecosystems, integration burdens have exploded, fracturing governance and auditability across warehouses, lakes, files, vect…

1
Episode 2: AWS Data Store Mastery 14:16

13d ago14:16

14:16

Where should you put your data? We tackle Domain 2 (26% of the DEA-C01 exam) by comparing Redshift, DynamoDB, and RDS. Learn how to design optimal schemas, use the AWS Glue Data Catalog, and implement S3 Lifecycle Policies to manage data lifespan and control costs.By James

1
Episode 4: The Data Fortress: Securing and Governing Data for the DEA-C01 12:20

13d ago12:20

12:20

Lock down your data platform! This is the final domain, Domain 4 (18% of the DEA-C01 exam). We cover essential security best practices: using IAM and Lake Formation for access control, enforcing encryption with KMS (at rest and in transit), and securing network access via VPC and Security Groups.By James

1
Episode 3: The Pipeline Pit Crew: Monitoring, Troubleshooting, and Optimizing Your AWS Data 12:36

13d ago12:36

12:36

Keep your data pipelines running smoothly! This episode covers Domain 3 (22% of the DEA-C01 exam). We dive into setting up alarms with CloudWatch, troubleshooting stuck jobs with Glue Logs, optimizing performance and cost in Redshift, and ensuring data quality with AWS Glue DataBrew.By James

1
Episode 1: Mastering the AWS Data Assembly Line 18:05

14d ago18:05

18:05

This is the essential guide to Domain 1: Data Ingestion and Transformation—the biggest section (34%) of the AWS Certified Data Engineer - Associate (DEA-C01) exam! We break down the core components of a successful data pipeline. Learn to compare Batch vs. Streaming with services like Kinesis and DMS, master ETL/ELT using AWS Glue and EMR, and orche…

1
Mix, Master & Mesh 50:59

15d ago50:59

50:59

In this genre-blending episode of Data Powered Innovation Jam, hosts Ron Tolido, Robert Engels, and Arne Rossman welcome Stephen Brobst, CTO of Ab Initio and former CTO of Terradata, for a deep dive into the art of mixing data, AI, and music. From punk rock roots and stage-diving legends to the reinvention of enterprise data platforms, Stephen shar…

1
Building Parquet into Apache Pinot ft. Neha Pawar | Ep. 5 26:07

17d ago26:07

26:07

Today, Tim Berglund talks to Neha Pawar (StarTree) about her career in real-time analytics and open source database engineering. Her first job: a year-long internship at NVIDIA. Her challenge: leading the technical effort to add native Parquet support into Apache Pinot. SEASON 2 Hosted by Tim Berglund, Adi Polak and Viktor Gamov Produced and Edited…

1
Solving incidents with one-time ephemeral runbooks 49:59

18d ago49:59

49:59

Share Episode ⸺ Episode Sponsor: Attribute - https://dev0ps.fyi/attribute In the wake of one of the worst AWS incidents in history, we're joined by Lawrence Jones, Founding Engineer at Incident.io. The conversation focuses on the challenges of managing incidents in highly regulated environments like FinTech, where the penalties for downtime are har…

1
The True Costs of Legacy Systems: Technical Debt, Risk, and Exit Strategies 1:04:16

19d ago1:04:16

1:04:16

Summary In this episode Kate Shaw, Senior Product Manager for Data and SLIM at SnapLogic, talks about the hidden and compounding costs of maintaining legacy systems—and practical strategies for modernization. She unpacks how “legacy” is less about age and more about when a system becomes a risk: blocking innovation, consuming excess IT time, and cr…

1
The Fix That Secured 1000s of Credit Cards ft. Brian Sletten | Ep. 4 29:37

24d ago29:37

29:37

In this episode, Tim talks to Brian Sletten (Bosatsu Consulting) about his career in software development. His first job: working at a small communications company that built network matrix switch interfaces. His challenge/theme: overhauling credit card storage and security at a major hospitality company. SEASON 2 Hosted by Tim Berglund, Adi Polak …

1
Context Engineering as a Discipline: Building Governed AI Analytics 51:58

26d ago51:58

51:58

Summary In this episode of the Data Engineering Podcast, host Tobias Macey welcomes back Nick Schrock, CTO and founder of Dagster Labs, to discuss Compass - a Slack-native, agentic analytics system designed to keep data teams connected with business stakeholders. Nick shares his journey from initial skepticism to embracing agentic AI as model and a…

1
Supply Chain Reaction 45:11

30d ago45:11

45:11

Welcome to the latest episode of the Data Powered Innovation Jam, where data meets disco and AI grooves with funk. After a long summer break, our hosts return with fresh stories, musical nostalgia, and cutting-edge insights into the world of supply chain superintelligence. In this vibrant and eclectic episode, we’re joined by Guillaume Waline, Seni…

1
How Viktor Gamov Stays Curious as Tech Rapidly Evolves | Ep. 3 30:11

1M ago30:11

30:11

Adi Polak interviews her co-host, Viktor Gamov, about his career’s evolution from distributed systems to streaming technology. Viktor’s first job: apple picking. His challenge/theme: staying curious and non-judgmental in the ever-changing landscape of tech. SEASON 2 Hosted by Tim Berglund, Adi Polak and Viktor Gamov Produced and Edited by Noelle Ga…

1
The Data Model That Captures Your Business: Metric Trees Explained 1:01:05

1M ago1:01:05

1:01:05

Summary In this episode of the Data Engineering Podcast Vijay Subramanian, founder and CEO of Trace, talks about metric trees - a new approach to data modeling that directly captures a company's business model. Vijay shares insights from his decade-long experience building data practices at Rent the Runway and explains how the modern data stack has…

1
The IT Dictionary: Post-Mortems, Cargo Cults, and Dropped Databases 29:34

1M ago29:34

29:34

Share Episode ⸺ Episode Sponsor: Attribute - https://dev0ps.fyi/attribute We're joined by 20 year industry veteran and DevOps advocate, Adam Korga, celebrating the release of his book IT Dictionary. In this episode we quickly get down to the inspiration behind postmortems as we review some cornerstone cases both in software and in general technolog…

1
How Tim Berglund Found His Calling | Ep. 2 30:36

1M ago30:36

30:36

Viktor Gamov interviews his co-host, Tim Berglund, about his career in the world of streaming data. Tim’s first job: Burger King broiler steamer. His challenge/theme: pivoting from working in hardware and firmware to finding his calling in enterprise software and developer relations. SEASON 2 Hosted by Tim Berglund, Adi Polak and Viktor Gamov Produ…

1
From GPUs-as-a-Service to Workloads-as-a-Service: Flex AI’s Path to High-Utilization AI Infra 56:31

1M ago56:31

56:31

Summary In this crossover episode of the AI Engineering Podcast, host Tobias Macey interviews Brijesh Tripathi, CEO of Flex AI, about revolutionizing AI engineering by removing DevOps burdens through "workload as a service". Brijesh shares his expertise from leading AI/HPC architecture at Intel and deploying supercomputers like Aurora, highlighting…

1
Vector Databases Explained: From E-commerce Search to Molecule Research 55:29

1M ago55:29

55:29

Share Episode ⸺ Episode Sponsor: Attribute - https://dev0ps.fyi/attribute Jenna Pederson, Staff Developer Relations at Pinecone, joins us to close the loop on Vector Databases. Demystifies how they power semantic search, their role in RAG, and also unexpected applications. Jenna takes us beyond the buzzword bingo, explaining how vector databases ar…

1
Building Real-time Systems for Apple, Nike & more ft. Adi Polak | Ep. 1 32:53

2M ago32:53

32:53

The Confluent Developer Podcast is here! For this first episode, Tim Berglund talks to his co-host, Adi Polak (Confluent), about her career in distributed data systems. Her first job: neighborhood dogwalker. Her challenge/theme: early Hadoop, working at Akamai on data optimization and real-time threat detection for huge global customers like Apple,…

1
From RAG to Relational: How Agentic Patterns Are Reshaping Data Architecture 52:58

2M ago52:58

52:58

Summary In this episode of the AI Engineering Podcast Mark Brooker, VP and Distinguished Engineer at AWS, talks about how agentic workflows are transforming database usage and infrastructure design. He discusses the evolving role of data in AI systems, from traditional models to more modern approaches like vectors, RAG, and relational databases. Ma…

1
The Unspoken Challenges of Deploying to Customer Clouds 52:41

2M ago52:41

52:41

Share Episode This episode we are joined by Andrew Moreland, co-founder of Chalk. Andrew explains how their company’s core business model is to deploy their software directly into their customers’ cloud environments. This decision was driven by the need to handle highly sensitive data, like PII and financial records, that customers don't want to ha…

1
Duck Lake: Simplifying the Lakehouse Ecosystem 1:10:41

2M ago1:10:41

1:10:41

Summary In this episode of the Data Engineering Podcast Hannes Mühleisen and Mark Raasveldt, the creators of DuckDB, share their work on Duck Lake, a new entrant in the open lakehouse ecosystem. They discuss how Duck Lake, is focused on simplicity, flexibility, and offers a unified catalog and table format compared to other lakehouse formats like I…

1
How to build in Observability at Petabyte Scale 45:31

2M ago45:31

45:31

Share Episode We welcome guest Ang Li and dive into the immense challenge of observability at scale, where some customers are generating petabytes of data per day. Ang explains that instead of building a database from scratch—a decision he says went "against all the instincts" of a founding engineer—Observe chose to build its platform on top of Sno…

1
We're back! Welcome to the Confluent Developer Podcast. 1:20

2M ago1:20

1:20

Weekly episodes launching Sept. 22! | Hi, I'm Tim Berglund. It's been about four years since I've been podcasting at Confluent, and "Streaming Audio" has been on hiatus for a little more than two, but I've got great news: we are back! We're back with a new name, a new format, and new hosts. Welcome to the Confluent Developer Podcast, where we talk …

1
Aligning Business and Data: The Essential Role of Data Modeling 1:06:51

2M ago1:06:51

1:06:51

Summary In this episode of the Data Engineering Podcast Serge Gershkovich, head of product at SQL DBM, talks about the socio-technical aspects of data modeling. Serge shares his background in data modeling and highlights its importance as a collaborative process between business stakeholders and data teams. He debunks common misconceptions that dat…

1
From Academia to Industry: Bridging Data Engineering Challenges 50:54

2M ago50:54

50:54

Summary In this episode of the Data Engineering Podcast Professor Paul Groth, from the University of Amsterdam, talks about his research on knowledge graphs and data engineering. Paul shares his background in AI and data management, discussing the evolution of data provenance and lineage, as well as the challenges of data integration. He explores t…

1
The Open-Source Product Leader Challenge: Navigating Community, Code, and Collaboration Chaos 59:26

2M ago59:26

59:26

In a special solo flight, Warren welcomes Meagan Cojocar, General Manager at Pulumi and a self-proclaimed graduate of “PM school” at AWS. They dive into what it’s like to own an entire product line and why giving up that startup hustle for the big leagues sometimes means you miss the direct signal from your users. The conversation goes deep on the …

1
High Performance And Low Overhead Graphs With KuzuDB 1:01:29

3M ago1:01:29

1:01:29

Summary In this episode of the Data Engineering Podcast Prashanth Rao, an AI engineer at KuzuDB, talks about their embeddable graph database. Prashanth explains how KuzuDB addresses performance shortcomings in existing solutions through columnar storage and novel join algorithms. He discusses the usability and scalability of KuzuDB, emphasizing its…

1
Bridging Data and Decision-Making: AI's Role in Modern Analytics 1:10:44

3M ago1:10:44

1:10:44

Summary In this episode of the Data Engineering Podcast Lucas Thelosen and Drew Gilson from Gravity talk about their development of Orion, an autonomous data analyst that bridges the gap between data availability and business decision-making. Lucas and Drew share their backgrounds in data analytics and how their experiences have shaped their approa…

1
From Bits to Tables: The Evolution of S3 Storage 50:08

3M ago50:08

50:08

Summary In this episode of the Data Engineering Podcast Andy Warfield talks about the innovative functionalities of S3 Tables and Vectors and their integration into modern data stacks. Andy shares his journey through the tech industry and his role at Amazon, where he collaborates to enhance storage capabilities, discussing the evolution of S3 from …

1
FinOps: Holding engineering teams accountable for spend 55:07

3M ago55:07

55:07

In this episode of Adventures in DevOps, we dive into the world of FinOps, a concept that aims to apply the DevOps mindset to financial accountability. Yasmin Rajabi, Chief Strategy Officer at CloudBolt, joins us to demystify, as we acknowledge the critical challenge of bringing together financial accountability and engineering teams who often are …

1
Revolutionizing Python Notebooks with Marimo 51:56

3M ago51:56

51:56

Summary In this episode of the Data Engineering Podcast Akshay Agrawal from Marimo discusses the innovative new Python notebook environment, which offers a reactive execution model, full Python integration, and built-in UI elements to enhance the interactive computing experience. He discusses the challenges of traditional Jupyter notebooks, such as…

1
Warehouse Native Incremental Data Processing With Dynamic Tables And Delayed View Semantics 55:07

4M ago55:07

55:07

Summary In this episode of the Data Engineering Podcast Dan Sotolongo from Snowflake talks about the complexities of incremental data processing in warehouse environments. Dan discusses the challenges of handling continuously evolving datasets and the importance of incremental data processing for optimized resource use and reduced latency. He expla…

1
The Auth Showdown: Single tenant versus Multitenant Architectures 53:24

4M ago53:24

53:24

Get ready for a lively debate on this episode of Adventures in DevOps. We're joined by Brian Pontarelli, founder of FusionAuth and CleanSpeak. Warren and Brian face off by diving into the controversial topic of multitenant versus single-tenant architecture. Expert co-host Aimee Knight joins to moderate the discussion. Ever wondered how someone beco…

1
Streamlining Data Pipelines with MCP Servers and Vector Engines 52:04

4M ago52:04

52:04

Summary In this episode of the Data Engineering Podcast Kacper Łukawski from Qdrant about integrating MCP servers with vector databases to process unstructured data. Kacper shares his experience in data engineering, from building big data pipelines in the automotive industry to leveraging large language models (LLMs) for transforming unstructured d…

1
Foundational Data Engineering At Two Sigma 55:05

4M ago55:05

55:05

Summary In this episode of the Data Engineering Podcast Effie Baram, a leader in foundational data engineering at Two Sigma, talks about the complexities and innovations in data engineering within the finance sector. She discusses the critical role of data at Two Sigma, balancing data quality with delivery speed, and the socio-technical challenges …

1
The Hour of (data) Power 47:40

4M ago47:40

47:40

Ready for your Hour of (Data) Power with some Radioactive and Electric Feel thrown in? Ok, so hang tight, get your coffee or lemonade (depending on how your summer looks like!), as we bring you a double whammy from your newest Data-powered Innovation Jam to celebrate the anniversary launch of the 10th edition of the Data Powered Innovation Review. …

1
Enabling Agents In The Enterprise With A Platform Approach 54:18

4M ago54:18

54:18

Summary In this episode of the Data Engineering Podcast Arun Joseph talks about developing and implementing agent platforms to empower businesses with agentic capabilities. From leading AI engineering at Deutsche Telekom to his current entrepreneurial venture focused on multi-agent systems, Arun shares insights on building agentic systems at an org…