))
Welcome to The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI— the podcast where we keep you up to date with insights and ideas propelling the Airflow community forward. Join us each week, as we explore the current state, future and potential of Airflow with leading thinkers in the community, and discover how best to leverage this workflow management system to meet the ever-evolving needs of data engineering and AI ecosystems. Podcast Webpage: https://www.astronomer.io ...
…
continue reading
1
Modernizing Legacy Data Systems With Airflow at Procter & Gamble with Adonis Castillo Cordero
22:13
22:13
Play later
Play later
Lists
Like
Liked
22:13
Legacy architecture and AI workloads pose unique challenges at scale, especially in a global enterprise with complex data systems. In this episode, we explore strategies to proactively monitor and optimize pipelines while minimizing downstream failures. Adonis Castillo Cordero, Senior Automation Manager at Procter & Gamble, joins us to share action…
…
continue reading
1
Building an End-to-End Data Observability System at Netflix with Joseph Machado
38:54
38:54
Play later
Play later
Lists
Like
Liked
38:54
Building reliable data pipelines starts with maintaining strong data quality standards and creating efficient systems for auditing, publishing and monitoring. In this episode, we explore the real-world patterns and best practices for ensuring data pipelines stay accurate, scalable and trustworthy. Joseph Machado, Senior Data Engineer at Netflix, jo…
…
continue reading
1
Why Developer Experience Shapes Data Pipeline Standards at Next Insurance with Snir Israeli
30:28
30:28
Play later
Play later
Lists
Like
Liked
30:28
Creating consistency across data pipelines is critical for scaling engineering teams and ensuring long-term maintainability. In this episode, Snir Israeli, Senior Data Engineer at Next Insurance, shares how enforcing coding standards and investing in developer experience transformed their approach to data engineering. He explains how implementing a…
…
continue reading
1
Data Quality and Observability at Tekmetric with Ipsa Trivedi
22:49
22:49
Play later
Play later
Lists
Like
Liked
22:49
Airflow’s adaptability is driving Tekmetric’s ability to unify complex data workflows, deliver accurate insights and support both internal operations and customer-facing services — all within a rapidly growing startup environment. In this episode, Ipsa Trivedi, Lead Data Engineer at Tekmetric, shares how her team is standardizing pipelines while su…
…
continue reading
1
Introducing Apache Airflow® 3 with Vikram Koka and Jed Cunningham
27:28
27:28
Play later
Play later
Lists
Like
Liked
27:28
The Airflow 3.0 release marks a significant leap forward in modern data orchestration, introducing architectural upgrades that improve scalability, flexibility and long-term maintainability. In this episode, we welcome Vikram Koka, Chief Strategy Officer at Astronomer, and Jed Cunningham, Principal Software Engineer at Astronomer, to discuss the ar…
…
continue reading
1
Airflow in Action: Powering Instacart's Complex Ecosystem
25:14
25:14
Play later
Play later
Lists
Like
Liked
25:14
The evolution of data orchestration at Instacart highlights the journey from fragmented systems to robust, standardized infrastructure. This transformation has enabled scalability, reliability and democratization of tools for diverse user personas. In this episode, we’re joined by Anant Agarwal, Software Engineer at Instacart, who shares insights i…
…
continue reading
1
From ETL to Airflow: Transforming Data Engineering at Deloitte Digital with Raviteja Tholupunoori
27:42
27:42
Play later
Play later
Lists
Like
Liked
27:42
Data orchestration at scale presents unique challenges, especially when aiming for flexibility and efficiency across cloud environments. Choosing the right tools and frameworks can make all the difference. In this episode, Raviteja Tholupunoori, Senior Engineer at Deloitte Digital, joins us to explore how Airflow enhances orchestration, scalability…
…
continue reading
1
A Deep Dive Into the 2025 State of Airflow Survey Results with Tamara Fingerlin of Astronomer
23:26
23:26
Play later
Play later
Lists
Like
Liked
23:26
The 2025 State of Airflow report sheds light on how global users are adopting, evolving and innovating with Apache Airflow. With over 5,000 responses from 116 countries, the survey reveals critical insights into Airflows’ role in business operations, new use cases and what’s ahead for the community. In this episode, Tamara Fingerlin, Developer Advo…
…
continue reading
1
Airflow’s Role in the Rise of DataOps with Andy Byron
26:15
26:15
Play later
Play later
Lists
Like
Liked
26:15
The orchestration layer is evolving into a critical component of the modern data stack. Understanding its role in DataOps is key to optimizing workflows, improving reliability and reducing complexity. In this episode, Andy Byron, CEO at Astronomer, discusses the rapid growth of Apache Airflow, the increasing importance of orchestration and how Astr…
…
continue reading
1
The Software Risk That Affects Everyone and How To Address It with Michael Winser and Jarek Potiuk
28:27
28:27
Play later
Play later
Lists
Like
Liked
28:27
The security of open-source software is a growing concern, especially as dependencies and regulations become more complex, making it essential to understand how to manage software supply chains effectively. In this episode, we sit down with Michael Winser, Co-Founder at Alpha-Omega and Security Strategy Ambassador at Eclipse Foundation, and Jarek P…
…
continue reading
1
Building Scalable ML Infrastructure at Outerbounds with Savin Goyal
36:46
36:46
Play later
Play later
Lists
Like
Liked
36:46
Machine learning is changing fast, and companies need better tools to handle AI workloads. The right infrastructure helps data scientists focus on solving problems instead of managing complex systems. In this episode, we talk with Savin Goyal, Co-Founder and CTO at Outerbounds, about building ML infrastructure, how orchestration makes workflows eas…
…
continue reading
1
Customizing Airflow for Complex Data Environments at Stripe with Nick Bilozerov and Sharadh Krishnamurthy
27:40
27:40
Play later
Play later
Lists
Like
Liked
27:40
Keeping data pipelines reliable at scale requires more than just the right tools — it demands constant innovation. In this episode, Nick Bilozerov, Senior Data Engineer at Stripe, and Sharadh Krishnamurthy, Engineering Manager at Stripe, discuss how Stripe customizes Airflow for its needs, the evolution of its data orchestration framework and the t…
…
continue reading
1
Harnessing Airflow for Data-Driven Policy Research at CSET with Jennifer Melot
17:54
17:54
Play later
Play later
Lists
Like
Liked
17:54
Turning complex datasets into meaningful analysis requires robust data infrastructure and seamless orchestration. In this episode, we’re joined by Jennifer Melot, Technical Lead at the Center for Security and Emerging Technology (CSET) at Georgetown University, to explore how Airflow powers data-driven insights in technology policy research. Jennif…
…
continue reading
1
Leveraging Airflow To Build Scalable and Reliable Data Platforms at 99acres.com with Samyak Jain
25:08
25:08
Play later
Play later
Lists
Like
Liked
25:08
Data orchestration is evolving rapidly, with dynamic workflows becoming the cornerstone of modern data engineering. In this episode, we are joined by Samyak Jain, Senior Software Engineer - Big Data at 99acres.com. Samyak shares insights from his journey with Apache Airflow, exploring how his team built a self-service platform that enables non-tech…
…
continue reading
1
Hybrid Testing Solutions for Autonomous Driving at Bosch with Jens Scheffler and Christian Schilling
33:45
33:45
Play later
Play later
Lists
Like
Liked
33:45
Testing autonomous vehicles demands precision, scalability and powerful orchestration tools — enter Apache Airflow, a key component of Bosch’s cutting-edge testing framework. In this episode, we sit down with Jens Scheffler, Test Execution Cluster Technical Architect, and Christian Schilling, Product Owner Open Loop Testing Automated Driving, both …
…
continue reading
1
Overcoming Airflow Scaling Challenges at Monzo Bank with Jonathan Rainer
43:39
43:39
Play later
Play later
Lists
Like
Liked
43:39
Scaling a data orchestration platform to manage thousands of tasks daily demands innovative solutions and strategic problem-solving. In this episode, we explore the complexities of scaling Airflow and the challenges of orchestrating thousands of tasks in dynamic data environments. Jonathan Rainer, Former Platform Engineer at Monzo Bank, joins us to…
…
continue reading
1
Orchestrating Analytics and AI Workflows at Telia with Arjun Anandkumar
26:00
26:00
Play later
Play later
Lists
Like
Liked
26:00
The future of data engineering lies in seamless orchestration and automation. In this episode, Arjun Anandkumar, Data Engineer at Telia, shares how his team uses Airflow to drive analytics and AI workflows. He highlights the challenges of scaling data platforms and how adopting best practices can simplify complex processes for teams across the orga…
…
continue reading
1
The Role of Airflow in Finance Transformation at Etraveli Group with Mihir Samant
21:19
21:19
Play later
Play later
Lists
Like
Liked
21:19
Transforming bottlenecked finance processes into streamlined, automated systems requires the right tools and a forward-thinking approach. In this episode, Mihir Samant, Senior Data Analyst at Etraveli Group, joins us to share how his team leverages Airflow to revolutionize finance automation. With extensive experience in data workflows and a passio…
…
continue reading
1
Inside Ford’s Data Transformation: Advanced Orchestration Strategies with Vasantha Kosuri-Marshall
38:54
38:54
Play later
Play later
Lists
Like
Liked
38:54
Data engineering is entering a new era, where orchestration and automation are redefining how large-scale projects operate. This episode features Vasantha Kosuri-Marshall, Data and ML Ops Engineer at Ford Motor Company. Vasantha shares her expertise in managing complex data pipelines. She takes us through Ford's transition to cloud platforms, the a…
…
continue reading
1
Powering Finance With Advanced Data Solutions at Ramp with Ryan Delgado
24:35
24:35
Play later
Play later
Lists
Like
Liked
24:35
Data is the backbone of every modern business, but unlocking its full potential requires the right tools and strategies. In this episode, Ryan Delgado, Director of Engineering at Ramp, joins us to explore how innovative data platforms can transform business operations and fuel growth. He shares insights on integrating Apache Airflow, optimizing dat…
…
continue reading
1
Exploring the Power of Airflow 3 at Astronomer with Amogh Desai
30:24
30:24
Play later
Play later
Lists
Like
Liked
30:24
What does it take to go from fixing a broken link to becoming a committer for one of the world’s leading open-source projects? Amogh Desai, Senior Software Engineer at Astronomer, takes us through his journey with Apache Airflow. From small contributions to building meaningful connections in the open-source community, Amogh’s story provides actiona…
…
continue reading
1
Using Airflow To Power Machine Learning Pipelines at Optimove with Vasyl Vasyuta
24:11
24:11
Play later
Play later
Lists
Like
Liked
24:11
Data orchestration and machine learning are shaping how organizations handle massive datasets and drive customer-focused strategies. Tools like Apache Airflow are central to this transformation. In this episode, Vasyl Vasyuta, R&D Team Leader at Optimove, joins us to discuss how his team leverages Airflow to optimize data processing, orchestrate ma…
…
continue reading
1
Maximizing Business Impact Through Data at GlossGenius with Katie Bauer
25:49
25:49
Play later
Play later
Lists
Like
Liked
25:49
Bridging the gap between data teams and business priorities is essential for maximizing impact and building value-driven workflows. Katie Bauer, Senior Director of Data at GlossGenius, joins us to share her principles for creating effective, aligned data teams. In this episode, Katie draws from her experience at GlossGenius, Reddit and Twitter to h…
…
continue reading
1
Optimizing Large-Scale Deployments at LinkedIn with Rahul Gade
27:47
27:47
Play later
Play later
Lists
Like
Liked
27:47
Scaling deployments for a billion users demands innovation, precision and resilience. In this episode, we dive into how LinkedIn optimizes its continuous deployment process using Apache Airflow. Rahul Gade, Staff Software Engineer at LinkedIn, shares his insights on building scalable systems and democratizing deployments for over 10,000 engineers. …
…
continue reading
1
How Uber Manages 1 Million Daily Tasks Using Airflow, with Shobhit Shah and Sumit Maheshwari
28:44
28:44
Play later
Play later
Lists
Like
Liked
28:44
When data orchestration reaches Uber’s scale, innovation becomes a necessity, not a luxury. In this episode, we discuss the innovations behind Uber’s unique Airflow setup. With our guests Shobhit Shah and Sumit Maheshwari, both Staff Software Engineers at Uber, we explore how their team manages one of the largest data workflow systems in the world.…
…
continue reading
