Go offline with the Player FM app!
InfluxDB: The Evolution of a Time Series Database (with Paul Dix)
Manage episode 497401087 series 3476072
What happens when you need to rewrite a database three times? Paul Dix knows firsthand—and he's brutally honest about every mistake along the way.
As CTO and co-founder of InfluxData, Paul has navigated the treacherous waters of building a time series database that can handle billions of rows a day. But this isn't your typical founder success story. This is one of the most candid conversations you'll ever hear about what really happens when you're building infrastructure software: how the problems evolve faster than your solutions, how business reality collides with technical vision, and why the challenges you think you need to solve often aren't the ones that matter.
Paul walks through InfluxDB's evolution from Go to Rust with unflinching honesty about what went wrong. The custom storage engines (TSM trees) that seemed brilliant until they hit production workloads. The rewrite that solved infinite cardinality and analytics queries—problems customers were asking for—while accidentally breaking the fast time series queries they actually needed. The usage-based pricing model that customers hated, the multi-tenant architecture that created quality-of-service nightmares, and the period where they were literally losing money on every customer.
This is a masterclass in how startup problems compound and evolve. Start with a time series database, realize you need better compression, build custom storage engines, discover customers want SQL, rewrite everything in Rust, find out Parquet integration doesn't work like advertised, learn that what people ask for isn't what they use, and end up maintaining three different database versions simultaneously while figuring out pricing models that don't bankrupt you.
Paul's transparency is remarkable—from technical decisions to business pivots to the personal challenge of staying hands-on as a founder. If you want to understand what building infrastructure software actually looks like, beyond the polished conference talks and success stories, this conversation is essential.
--
Support Developer Voices on Patreon: https://patreon.com/DeveloperVoices
Support Developer Voices on YouTube: https://www.youtube.com/@developervoices/join
InfluxData: https://www.influxdata.com/
InfluxDB: https://www.influxdata.com/products/influxdb/
DataFusion: https://datafusion.apache.org/
DataFusion Episode: https://www.youtube.com/watch?v=8QNNCr8WfDM
Apache Arrow: https://arrow.apache.org/
Apache Parquet: https://parquet.apache.org/
BoltDB: https://github.com/boltdb/bolt
LevelDB: https://github.com/google/leveldb
RocksDB: https://rocksdb.org/
Gorilla: A Fast, Scalable, In-Memory Time Series Database (Facebook paper): https://www.vldb.org/pvldb/vol8/p1816-teller.pdf
Paul on LinkedIn: https://www.linkedin.com/in/pauldix/
Kris on Bluesky: https://bsky.app/profile/krisajenkins.bsky.social
Kris on Mastodon: http://mastodon.social/@krisajenkins
Kris on LinkedIn: https://www.linkedin.com/in/krisjenkins/
94 episodes
Manage episode 497401087 series 3476072
What happens when you need to rewrite a database three times? Paul Dix knows firsthand—and he's brutally honest about every mistake along the way.
As CTO and co-founder of InfluxData, Paul has navigated the treacherous waters of building a time series database that can handle billions of rows a day. But this isn't your typical founder success story. This is one of the most candid conversations you'll ever hear about what really happens when you're building infrastructure software: how the problems evolve faster than your solutions, how business reality collides with technical vision, and why the challenges you think you need to solve often aren't the ones that matter.
Paul walks through InfluxDB's evolution from Go to Rust with unflinching honesty about what went wrong. The custom storage engines (TSM trees) that seemed brilliant until they hit production workloads. The rewrite that solved infinite cardinality and analytics queries—problems customers were asking for—while accidentally breaking the fast time series queries they actually needed. The usage-based pricing model that customers hated, the multi-tenant architecture that created quality-of-service nightmares, and the period where they were literally losing money on every customer.
This is a masterclass in how startup problems compound and evolve. Start with a time series database, realize you need better compression, build custom storage engines, discover customers want SQL, rewrite everything in Rust, find out Parquet integration doesn't work like advertised, learn that what people ask for isn't what they use, and end up maintaining three different database versions simultaneously while figuring out pricing models that don't bankrupt you.
Paul's transparency is remarkable—from technical decisions to business pivots to the personal challenge of staying hands-on as a founder. If you want to understand what building infrastructure software actually looks like, beyond the polished conference talks and success stories, this conversation is essential.
--
Support Developer Voices on Patreon: https://patreon.com/DeveloperVoices
Support Developer Voices on YouTube: https://www.youtube.com/@developervoices/join
InfluxData: https://www.influxdata.com/
InfluxDB: https://www.influxdata.com/products/influxdb/
DataFusion: https://datafusion.apache.org/
DataFusion Episode: https://www.youtube.com/watch?v=8QNNCr8WfDM
Apache Arrow: https://arrow.apache.org/
Apache Parquet: https://parquet.apache.org/
BoltDB: https://github.com/boltdb/bolt
LevelDB: https://github.com/google/leveldb
RocksDB: https://rocksdb.org/
Gorilla: A Fast, Scalable, In-Memory Time Series Database (Facebook paper): https://www.vldb.org/pvldb/vol8/p1816-teller.pdf
Paul on LinkedIn: https://www.linkedin.com/in/pauldix/
Kris on Bluesky: https://bsky.app/profile/krisajenkins.bsky.social
Kris on Mastodon: http://mastodon.social/@krisajenkins
Kris on LinkedIn: https://www.linkedin.com/in/krisjenkins/
94 episodes
All episodes
×Welcome to Player FM!
Player FM is scanning the web for high-quality podcasts for you to enjoy right now. It's the best podcast app and works on Android, iPhone, and the web. Signup to sync subscriptions across devices.