Best Apache Kafka® Podcasts (2025)

1
Making Apache Kafka Diskless (with Filip Yonov & Josep Prat) 1:29:29

8d ago1:29:29

1:29:29

How do you retrofit a clustered data-processing system to use cheap commodity storage? That’s the big question in this episode as we look at one of the many attempts to build a version of Kafka that uses object storage services like S3 as its main disk, sacrificing a little latency for cheap, infinitely-scalable disks. There are several companies t…

1
Anna: A KVS For Any Scale 19:01

16d ago19:01

19:01

This research paper introduces Anna, a key-value store (KVS) designed for scalable performance across diverse computing environments, from single multi-core machines to globally distributed cloud deployments. Anna achieves high performance and adaptability through a partitioned, multi-master architecture utilizing wait-free execution and coordinati…

1
Java's Cutting Edge Comeback (with Josh Long) 1:24:29

21d ago1:24:29

1:24:29

Java’s has been evolving faster than any 30 year old language has a right to do, and there’s probably no-one more pleased about it than my guest this week - Josh Long. He’s a Java & Kotlin programming, a JVM enthusiast in general, and an advocate for Spring, and he has chapters full of news about what’s been happening in Javaland over the past few …

1
Conflict-free Replicated Data Types 26:17

23d ago26:17

26:17

This academic paper introduces Conflict-free Replicated Data Types (CRDTs), which are abstract data types designed for distributed systems where data is replicated across multiple locations. CRDTs allow any replica to be modified without needing immediate coordination with other replicas, ensuring high availability and low latency. The core concept…

1
CAP Twelve Years Later: How the "Rules" Have Changed 29:47

1M ago29:47

29:47

This content from InfoQ provides insights for software architects and developers through various formats like newsletters, articles, and conference information. It highlights topics in architecture, AI, data engineering, culture, methods, and DevOps. Featured pieces discuss Slack's cellular architecture, data stream processing patterns, cultivating…

1
The State & Future of Apache Kafka (with Anatoly Zelenin) 1:12:22

1M ago1:12:22

1:12:22

I’m joined this week by one of the authors of Apache Kafka In Action, to take a look at the state of Kafka, event systems & stream-processing technology. It’s an approach (and a whole market) that’s had at least a decade to mature, so how has it done? What does Kafka offer to developers and businesses, and which parts do they actually care about? W…

1
Raft versus Paxos: An Understandable Consensus Algorithm 33:14

1M ago33:14

33:14

Raft, a consensus algorithm designed for managing a replicated log in distributed systems. It aims to be more understandable than Paxos, a widely used but complex alternative, while achieving equivalent efficiency and safety. Raft separates key consensus elements like leader election, log replication, and safety, using techniques such as problem de…

1
AI-in-BI: The Roadmap for Superset and Beyond 1:00:36

1M ago1:00:36

1:00:36

We'll be exploring how the Apache Superset project (and Preset) have been making moves toward AI/LLM features, what SIPs are open on this topic, and what we consider to be the path for AI in BI as a generalized practice, as an extension to this blog post.

1
Neo4j Architecture: Graph Database Internals, Performance, and Optimization 17:42

1M ago17:42

17:42

This compilation of resources offers a comprehensive examination of Neo4j's graph database architecture. It explains how Neo4j differs from relational and document-oriented databases through its native graph storage. The materials describe how nodes, relationships, and properties are stored and indexed for efficient traversal and query processing. …

1
DataFusion - The Database Building Toolkit (with Andrew Lamb) 1:32:10

2M ago1:32:10

1:32:10

Building a database is a serious undertaking. There are just so many parts that you have to implement before you even get to a decent prototype, and so many hours of work before you could begin working on the ideas that would make your database unique. Apache DataFusion is a project that hopes to change all that, but building an extensible, composa…

1
Sentry: Error Monitoring at Scale - Design Principles Analysis 15:48

2M ago15:48

15:48

Sentry is a large-scale, open-source error monitoring platform designed for modern distributed systems. It prioritizes actionable insights by focusing on exceptions and crashes, enriching errors with contextual data, and using features such as breadcrumbs and error grouping. Sentry's architecture employs modular and decoupled components like Relay …

1
Istio Service Mesh: Architecture, Security, and Traffic Management 33:58

2M ago33:58

33:58

These excerpts offer a detailed look at Istio's service mesh architecture, a critical component for managing microservices in cloud-native environments. The architecture is divided into a control plane and data plane, emphasizing security through automated mTLS and traffic management with advanced load balancing techniques. Observability is achieve…

1
Jupyter's Architecture Unpacked (with Afshin Darian & Sylvain Corlay) 1:29:11

2M ago1:29:11

1:29:11

Jupyter’s become an incredibly popular programming and data science tool, but how does it actually work? How have they built an interactive language execution engine? And if we understand the architecture, what else could it be used for? Joining me to look inside the Jupyter toolbox are Afshin Darian and Sylvain Corlay, two of Jupyters long-standin…

1
CockroachDB: SQL for Global Scale Design Principles 14:33

2M ago14:33

14:33

CockroachDB is a distributed SQL database designed for global scalability and resilience. The database achieves this through a unique architecture built on a monolithic key-value store, Raft-based replication, and hybrid logical clocks. Transaction management is optimized for global workloads using a non-blocking commit protocol and multi-region ca…

1
AI in BI: the Path to Full Self-Driving Analytics 30:51

2M ago30:51

30:51

As a corollary to Max's new blog post, "AI in BI: The Path to Full Self Drive", this episode is a bit of.... an experiment. It explores how artificial intelligence is revolutionizing business intelligence (BI), from accelerating data analysis and surfacing key insights to enabling predictive models and automating decision-making. AI is changing how…

1
Snowflake: Revolutionizing Cloud Data Warehousing and Analytics 17:21

2M ago17:21

17:21

Snowflake, a cloud-native data warehouse, revolutionizes modern analytics through its unique architecture and capabilities. The platform separates compute and storage layers, enabling independent scaling and optimized performance. Its three-layer design encompasses cloud services, a compute layer using virtual warehouses, and a storage layer levera…

1
Nix, The Build-Everything Language (with Julian Arni) 1:20:36

3M ago1:20:36

1:20:36

Ever since we invented makefiles, the programming world has been wrestling with the problem of building software stacks reliably. This week we’re going to look at one of the most ambitious solutions available - Nix. Nix tries to do everything from invoking your compiler to installing your language, and even providing your operating system. But how …

1
Kubernetes: Container Orchestration, Architecture, and Evolution 25:56

3M ago25:56

25:56

This collection of excerpts comprehensively examines Kubernetes, the leading container orchestration platform. It traces the historical evolution of container orchestration and highlights Kubernetes' architectural foundations, including its control plane and node components. Scalability mechanisms like horizontal pod autoscaling and cell-based arch…

1
Elasticsearch: Architecture, Applications, and Emerging Trends 18:13

3M ago18:13

18:13

This compilation of excerpts thoroughly examines Elasticsearch, focusing on its architecture, applications, and future trends. The core architecture and its integration within the Elastic Stack are highlighted, emphasizing scalability and real-time analytics. Various specialized applications are discussed, including maritime data storage, academic …

1
Graphite: Image Editing as a Syntax Tree (with Keavon Chambers & Dennis Kobert) 1:17:32

3M ago1:17:32

1:17:32

Graphite is a new image editor with an interesting architecture - it’s a classic UI-driven app, an image-manipulation language, and a library of programmable graphics primitives that any Rust coder could use, extend or add to. The result is something that you can use like Photoshop or Inkscape, or make use of in batch pipelines, a bit like ImageMag…

1
Ray: A Distributed Framework for Emerging AI Applications 19:40

3M ago19:40

19:40

This research paper introduces Ray, a distributed framework designed for emerging AI applications, particularly those involving reinforcement learning. It addresses the limitations of existing systems in handling the complex demands of these applications, which require continuous interaction with the environment. Ray unifies task-parallel and actor…

1
Zanzibar: Google's Global Authorization System 27:21

3M ago27:21

27:21

This paper details Zanzibar, Google's globally distributed authorization system, designed to manage access control lists (ACLs) at a massive scale. Zanzibar uses a flexible data model and configuration language to handle diverse access control policies for numerous Google services, achieving high availability and low latency. The system maintains e…

1
Google Mesa: A Geo-Replicated, Near Real-Time Data Warehouse 15:02

4M ago15:02

15:02

**Mesa** is a highly scalable, geo-replicated data warehousing system developed at Google to handle petabytes of data related to its advertising business. **Designed for near real-time data ingestion and querying**, it processes millions of updates per second and serves billions of queries daily. **Key features include strong consistency, high avai…

1
Malcolm Matalka: Co-Founder at Terrateam 50:38

4M ago50:38

50:38

Interview With Malcolm Matalka: Co-Founder, Terrateam Ben Rometsch chats with Malcolm Matalka, co-founder of Terrateam, to discuss the company’s inception and its eventual shift to open source. Together, they discuss how his frustrations with the click ops approach prompted Malcolm to create his own team to disrupt the system and pave the way for m…

1
ReScript: A Better Typed JavaScript? (with Gabriel Nordeborn) 1:32:53

4M ago1:32:53

1:32:53

ReScript is a strongly-typed programming language that compiles to JavaScript, and that puts it squarely in competition with TypeScript. So why would a JavaScript developer choose to learn it next? What does it offer that makes it a tempting proposition? And how are the ReScript developers making life easier for anyone who wants to make the switch?…

1
Time, Clocks, and the Ordering of Events in a Distributed System 13:50

4M ago13:50

13:50

This paper, "Time, Clocks, and the Ordering of Events in a Distributed System," explores the challenges of defining and managing time in distributed systems. It introduces the concept of a "happened before" relation to partially order events and presents an algorithm for creating a consistent total ordering using logical clocks. The paper then exte…

1
ZooKeeper: Wait-Free Coordination for Internet-Scale Systems 26:38

4M ago26:38

26:38

This paper details the design and implementation of ZooKeeper, a high-performance coordination service for large-scale distributed systems. ZooKeeper provides a simple, wait-free API enabling developers to build various coordination primitives, such as locks and group membership, without server-side modifications. It achieves high throughput throug…

1
Sam Stepanyan, OWASP London Chapter Leader & OWASP Global Board Member 53:21

4M ago53:21

53:21

With almost every transaction being done through the internet, your personal information and finances must be protected at all costs from hackers and scammers. Therefore, cybersecurity is important now more than ever. Leading the charge in keeping the digital world safe and secure is OWASP, a non-profit organization working mainly on software secur…

1
A universal query engine in Rust (with Predrag Gruevski) 1:15:31

4M ago1:15:31

1:15:31

Trustfall is a library based on a simple question - what happens if we can query absolutely anything? If you could join REST APIs and databases with filesystems and dockerfiles? It’s possible in theory because those are all just datasources. Predrag Gruevski is trying to make it easy by building a universal query engine, with pluggable datasources,…

1
TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems 17:02

4M ago17:02

17:02

This paper details TensorFlow, a large-scale machine learning system developed by Google. TensorFlow uses dataflow graphs to represent computation and manages state across diverse hardware, including CPUs, GPUs, and TPUs. It offers a flexible programming model, allowing developers to experiment with novel optimizations and training algorithms beyon…

1
Firestore: A Serverless NoSQL Database 27:46

5M ago27:46

27:46

This paper details Google Firestore, a NoSQL serverless database built on Spanner. It highlights Firestore's ease of use, scalability, real-time query capabilities, and support for disconnected operations. The architecture, which enables multi-tenancy and efficient handling of large datasets, is explained. Performance benchmarks and practical lesso…

1
Raspberry Pi Hardware & A Lisp Brain (with Dimitris Kyriakoudis) 1:34:43

5M ago1:34:43

1:34:43

Dimitris Kyriakoudis is a researcher, programmer and musician who's combining all three talents to build dedicated music hardware. Specifically a device called the µseq, which reads Lisp programs and uses them to drive synthesizers to make music. In this episode we go through the full platform that he's building, from soldering resistors to an RPi …

1
Apache Flink: Stream and Batch Processing in a Single Engine 18:12

5M ago18:12

18:12

This research paper details Apache Flink, an open-source system unifying stream and batch data processing. Flink uses a dataflow model to handle various data processing needs, including real-time analytics and batch jobs, within a single engine. The paper explores Flink's architecture, APIs (including DataStream and DataSet APIs), and fault-toleran…

1
Chris Villa, Co-Founder of Puck & Measured 50:01

5M ago50:01

50:01

Interview With Chris Villa: Co-Founder, Puck & Measured Puck, an open-source project, has exploded in popularity, becoming a major focus for Chris Villa and his team. But how do you balance a runaway open-source hit with the demands of running a consultancy? Join Ben Rometsch as he sits down with Chris Villa, co-founder of both Puck and Measured, t…

1
Software Systems Aren't Just Software (with Diana Montalion) 1:50:14

5M ago1:50:14

1:50:14

If you want to build really large software systems well, you have to stop thinking of them as just software systems. Beyond a certain size, everything your software touches becomes part of the wider system. You’re part of the system, your users are part of the system, and every other employee & department & priority eventually forms part of that sy…

1
Kafka: A Distributed Messaging System for Log Processing 16:52

5M ago16:52

16:52

This paper introduces Kafka, a novel distributed messaging system designed for high-throughput log processing. Kafka addresses limitations in existing messaging systems and log aggregators by offering a scalable, efficient architecture with a simple API. Key features include a pull-based consumption model, efficient storage and data transfer mechan…

1
Building Fyrox: A Rust Game Engine (with Dmitry Stepanov) 1:43:45

5M ago1:43:45

1:43:45

To kick off 2025 we’re looking at Fyrox a game engine built in Rust, largely by one person - Dmitry Stepanov. For an individual project, it’s covered an incredible amount of ground, covering the rendering and animation features you’d expect from a game engine, with some features that might surprise you - like Rust scripting support with hot-reloadi…

1
LinkedIn: Using Set Cover to Optimize a Large-Scale Low Latency Distributed Graph 12:59

5M ago12:59

12:59

This research paper details LinkedIn's solution for optimizing low-latency graph computations within their large-scale distributed graph system. To improve performance, they implemented a modified greedy set cover algorithm to minimize the number of machines needed for processing second-degree connection queries. This optimization significantly red…

1
James McLeod: Open Source Program Lead at NatWest Group 56:39

5M ago56:39

56:39

Joining this episode is James McLeod, Open Source Program Lead at NatWest Group. He looks back on his career journey, from his role at the Linux Foundation, his efforts in scaling the FINOS project, to the creation of London.js. Reflecting on the many lessons throughout his professional life, James stressed the importance of proper data documentati…

1
Monolith: A Real-Time Recommendation System 20:25

5M ago20:25

20:25

This research paper details Monolith, a real-time recommendation system developed by Bytedance. Monolith addresses challenges in building scalable recommendation systems, such as sparse and dynamic data, and concept drift, by employing a collisionless embedding table and an online training architecture. Key innovations include a Cuckoo HashMap for …

1
Meta FlexiRaft: Flexible Quorums for Raft Consensus 24:27

6M ago24:27

24:27

This research paper details FlexiRaft, a modified Raft consensus algorithm designed for Meta's petabyte-scale MySQL deployments. The core improvement is the introduction of flexible quorums, allowing configurable trade-offs between latency, throughput, and fault tolerance. Two quorum modes are presented: static and dynamic. The paper explores the a…

1
Testing TVs At Scale With Elixir (with Dave Lucia) 1:16:05

6M ago1:16:05

1:16:05

Integration testing is always a tricky thing, fraught with problems setting up the right environment and attempting to control the system’s state. That’s particularly true when you’re dealing with a mix of software and hardware, and even worse when you don’t have control of what the hardware can do. This week I’m joined by Dave Lucia of TVLab’s, wh…

1
Moritz Klack & Christopher Möller: Co-Founders, xyflow 34:41

6M ago34:41

34:41

This episode features Moritz Klack and Christopher Möller, Co-Founders of xyflow. They look back on their career journey leading to their current company, from developing webkid, creating Datablocks, and launching React Flow. They discuss their transition from agency work to open source, their pricing strategies, and the importance of community fee…

1
Spanner: Google’s Globally Distributed Database 13:28

6M ago13:28

13:28

This research paper details Spanner, Google's globally-distributed database system. Spanner achieves strong consistency across its geographically dispersed data centers using a novel TrueTime API that accounts for clock uncertainty. The system features automatic sharding, failover, and a semi-relational data model, addressing limitations of previou…

1
Programming As An Expressive Instrument (with Sam Aaron) 1:50:01

6M ago1:50:01

1:50:01

Sam Aaron is the creator of Sonic Pi, one of the most unusual software platforms you’ll encounter. It’s a live-coding playground for making music. A tool that lets you write code that defines sounds and musical phrases, and build up a hole program that plays anything from a short bleep to a whole nightclub set. And Sam’s creator has been using it l…

1
Meta Minesweeper: Scalable Statistical Root Cause Analysis on App Telemetry 17:54

6M ago17:54

17:54

This research paper introduces Minesweeper, a novel technique for automated root cause analysis (RCA) of software bugs at scale. Leveraging telemetry data, Minesweeper efficiently identifies statistically significant patterns in user app traces that correlate with bugs, even in the absence of detailed debugging information. The method uses sequenti…

1
Elm & The Future of Open Source (with Evan Czaplicki) 50:24

7M ago50:24

50:24

Evan Czaplicki—the creator of the Elm programming language —joins me to discuss the state and future of Elm, the friendly, type-safe functional programming language. On many fronts Elm has been a huge success: it’s been popular with new and seasoned programmers alike; it’s helped push several language ideas into the mainstream; it’s been a key part…

1
Cassandra- A Decentralized Structured Storage System 14:27

7M ago14:27

14:27

This paper details Cassandra, a decentralized structured storage system designed for managing massive amounts of structured data across numerous commodity servers. High availability and scalability are key features, achieved through techniques like consistent hashing for data partitioning and replication strategies across multiple data centers to h…

1
Programmers, ADHD, And How To Manage Them Both (with Chris Ferdinandi) 1:39:14

7M ago1:39:14

1:39:14

This week we’re going to look at the most essential piece of firmware in a programmer’s toolkit - the brain. I’m joined by Chris Ferdinandi to explore what it’s like to be a programmer with ADHD. It’s an unusual topic for the channel, but the more I spoke to him, the more I wanted to know what coding is like when your brain is wired differently, ho…

1
FoundationDB: A Distributed Unbundled Transactional Key Value Store 21:02

7M ago21:02

21:02

The provided text is an excerpt from a research paper on FoundationDB, an open-source, distributed transactional key-value store. The paper details FoundationDB's design principles, architecture, and key features, including its unbundled architecture, strict serializability through a combination of optimistic concurrency control (OCC) and multi-ver…

Podcasts Worth a Listen

Apache Kafka® Podcasts

Podcasts Worth a Listen