TiDB Under the Hood: A Source Tour of the MySQL-Compatible Distributed SQL Layer
Every ML platform eventually hits the same wall: the feature store, the event log, or the experiment tracker outgrows a single MySQL instance, and the team is forced to choose between sharding by hand and rewriting the app for some...
SQLite Under the Hood: A Source Tour of the World's Most Deployed Database
SQLite is almost certainly running within arm’s reach of you right now. It is the database inside every phone, every major browser, most operating systems, and an enormous share of embedded devices. It has no server process, no network protocol,...
RocksDB Under the Hood: A Source Tour of the LSM Storage Engine Behind Modern...
RocksDB is the storage engine you have probably used today without noticing. It sits underneath Flink’s checkpointed operator state, Kafka Streams’ local state stores, TiKV (the transactional key-value layer that powers TiDB), and a growing family of feature stores that...
PostgreSQL Under the Hood: A Source Tour of the Engine Powering Your AI Data ...
Ask around any serious ML team and you will hear the same answer to the question “where does your data actually live?”: PostgreSQL. The vector search boom did not replace it — it extended it, with pgvector and friends turning...
Neo4j Under the Hood: A Source Tour of the Graph Database Engine
Every large language model application eventually hits the same wall: text alone cannot answer “how is entity A related to entity B through three intermediaries?” Vector search gives you semantic neighbors, but relationships — who funded whom, which user reviewed...
Inside MySQL: A Source-Level Tour of InnoDB, the Optimizer, and Replication
MySQL is the quiet backbone of the internet. Written in C and C++, maintained by Oracle in the open on GitHub, it has spent three decades being the default transactional database behind forums, e-commerce carts, payment ledgers, and SaaS dashboards....
LevelDB Under the Hood: A Source Tour of the LSM Engine That Started It All
Yesterday we walked the RocksDB source tree — the heavily forked engine that powers Flink state stores, Kafka Streams, TiKV, and the local segments of countless vector databases. But RocksDB did not appear out of nothing. It is a fork...
Dolt Under the Hood: A Source Tour of Git for Data
Every serious ML effort eventually collides with the same question: which exact rows did this model train on? Training data is not static. Labels get corrected, schemas grow columns, bad batches are rolled back, and feature-engineering experiments fork the dataset...
CockroachDB Under the Hood: A Source Tour of the Distributed SQL Engine Built...
If you are serving AI features to users in several regions at once, you eventually hit the same wall: a single-node Postgres works beautifully until you need writes to survive a datacenter failure, or until feature data, embeddings metadata, and...
BadgerDB Under the Hood: A Source Tour of the Pure-Go LSM Key-Value Store
Most storage engines we have toured in this series descend from the same C++ lineage: RocksDB and LevelDB pack keys and values into the same LSM tree and pay for it with compaction write amplification. BadgerDB, maintained by the Dgraph/Hypermode...
Velox: Inside Meta's C++ Vectorized Execution Engine Behind Presto and Spark
Most engineers meet Velox indirectly. If you have run a query on Presto’s C++ worker (Prestissimo) or accelerated Spark through Apache Gluten, Velox is the library that actually executed your plan. Created at Meta and now developed with partners including...
Inside Apache Spark: How One Line of SQL Becomes a Cluster Job
The best description of this project is the one its authors wrote: the README's first line calls Apache Spark "a unified analytics engine for large-scale data processing." That phrasing undersells what the repository actually holds. Behind that sentence sits a...
Presto Under the Hood: A Source Tour of the Distributed SQL Engine That Queri...
Presto describes itself in one line in its own README: “Presto is a distributed SQL query engine for big data.” That modest sentence hides one of the most influential architectures in data infrastructure. Born at Facebook to let engineers query...
Apache Paimon: Inside the Streaming Lakehouse's LSM-Tree Table Format
Modern ML systems are hungry for fresh tabular data. Feature tables churn all day: a user’s click counts change, an item’s embedding ages, a fraud score drifts. Classically you had to choose between a streaming log like Kafka, which is...
Apache Iceberg Source Tour: The Table Format That Puts ACID on Object Storage
On a data lake, a table is just a folder of files — until you need transactions. The moment two jobs write the same directory, or a query starts while a compaction job finishes, plain folders stop answering “what exactly...
Ibis: One Python DataFrame API That Compiles to Every Query Engine You Ship
Every serious ML and LLM workflow hits the same wall eventually. You prototype a feature workflow on pandas or a local engine, it works beautifully on ten gigabytes, and then someone says “great — now run it over the full...
Apache Gluten: Pushing Spark SQL Down to Native Engines With Substrait
Every serious machine learning project eventually hits the same wall: the model is only as fast as the data preparation that feeds it. Feature engineering at petabyte scale is overwhelmingly SQL-shaped work — filters, joins, aggregations, window functions — and...
Apache Flink Under the Hood: A Source Tour of True Streaming, Event Time, and...
Most machine-learning systems treat streaming data as plumbing: events arrive in Kafka, a nightly batch job in Spark rebuilds aggregates, and the model serving tier reads whatever is freshest in Redis. Apache Flink attacks that gap differently. It is a...
DataFusion: The Rust and Arrow Query Engine Powering Embeddable Analytics
Modern AI and ML work is, more often than we admit, query-shaped. Before a model sees a tensor, somebody filters events, joins features, aggregates sessions, and dedupes rows out of a lake of Parquet files. Enter Apache DataFusion: an extensible...
cuDF: Inside NVIDIA's GPU DataFrame Engine That Speaks Fluent pandas
Every serious ML workflow eventually hits the same wall: the model trains on the GPU, but the data prep before it runs on the CPU. Feature engineering, joins, group-by aggregations, CSV and Parquet parsing — all of it grinds through...
Zipkin Under the Hood: A Source Tour of the Tracing System That Started It All
Every AI stack you operate today — an API gateway routing to a model server, an agent orchestrating tool calls, a RAG chain fanning out to an embedding service and a vector database — creates the same debugging problem Zipkin...
Vector: The Rust Router That Moves Your LLM's Logs, Metrics, and Traces
Ask anyone running a large language model service what their telemetry situation looks like and watch them wince. A single serving cluster can emit millions of structured log lines per day — prompts, completions, tool calls, guardrail verdicts — plus...
Grafana Tempo Under the Hood: Kafka, Block Builders and a TraceQL Engine That...
An AI application in 2026 is not one program. It is a chain: a router picks a model, a retriever fetches context from a vector store, an inference server runs the prompt, tool calls fan out to external APIs, and...
Telegraf Under the Hood: The Plugin-Driven Go Agent That Moves Your Metrics
Training and serving AI models is a telemetry problem as much as a compute problem. A training run is only as trustworthy as the numbers you can see while it happens: GPU utilization and memory pressure from every node, host...