PyTorch/XLA: Inside the Bridge from Eager PyTorch to TPU Hardware
Most deep learning engineers meet PyTorch as an eager framework: every op fires a kernel the moment you call it. On a TPU, that model of execution does not work. TPUs are fed by a compiler — XLA, the Accelerated...
TFLite Micro: How Neural Network Inference Fits in 256 KB Without malloc
Every inference runtime we have toured in this series — TensorFlow, ONNX Runtime, llama.cpp — assumes a fat host: gigabytes of RAM, an allocator that can hand out memory at will, maybe a GPU on the side. The repo under...
TensorRT Source Tour: From ONNX Graph to Serialized CUDA Engine
Most deep-learning frameworks you meet day to day are executors: they take a graph, schedule its ops, and hope the kernels underneath are fast. TensorRT is a different animal. It is a compiler: it ingests a trained network, picks concrete...
TensorRT Model Optimizer: A Source Tour of NVIDIA's Model Compression Toolkit
Most inference stacks spend their lives downstream of a model. NVIDIA’s Model Optimizer — ModelOpt for short — lives upstream, where the model itself is made smaller. The repository, NVIDIA/TensorRT-Model-Optimizer on GitHub, is the source behind the nvidia-modelopt PyPI package:...
KerasHub Deep Dive: The Pretrained-Model Hub That Makes Keras 3 Feel Like a M...
Most deep learning work in 2026 starts from a checkpoint, not from a blank model. You pick an architecture, download pretrained weights, attach a small head or a prompt, and start training. KerasHub is the keras-team’s answer to that workflow:...
Flax Source Tour: Inside Linen, NNX, and the Functional Heart of JAX
Flax is Google’s neural-network library built directly on top of JAX, and it occupies a peculiar position in the deep-learning ecosystem: it is the closest thing the functional-programming world has to a household framework. Where PyTorch gives you a mutable...
FastChat Source Tour: The Controller-Worker Mesh Behind Chatbot Arena
Before Chatbot Arena became the leaderboard everyone cites in model release tweets, there had to be a machine that could host seventy models at once, shuffle anonymous battles to millions of visitors, and record every vote. That machine is FastChat,...
DeepSpeed-MII: A Source Tour of Microsoft's Low-Latency Inference Library
Most inference libraries make you choose between a research project and a one-liner. Microsoft’s DeepSpeed-MII, the model-serving layer of the DeepSpeed ecosystem, bets that you can have both: a four-line script to try a model, and a load-balanced, multi-replica gRPC...
coremltools: How Apple Compiles PyTorch Models Onto the Neural Engine
If you have ever trained a network in PyTorch and then wondered how it ends up running inside an iOS app with a 40% smaller footprint and zero Python interpreter in sight, the answer is almost always coremltools. Apple’s open-source...
BERT: A Source Tour of Google's Original Transformer Encoder
When Google published “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding” in October 2018, the paper landed together with a reference implementation — the google-research/bert repository. That repo is the artifact this tour reads. It is written for TensorFlow...
XNNPACK Source Tour: The Microkernel Library Powering Mobile Inference
If you have ever run a neural network on a phone, chances are the actual arithmetic was done by XNNPACK. It is the C library that sits underneath TensorFlow Lite, ExecuTorch, and the mobile builds of other runtimes, taking care...
torchvision Under the Hood: transforms v2 and the C++ Ops That Power Detection
Most people treat torchvision as a convenience layer: a Resize here, a pretrained ResNet there. But if you train or deploy detection models, you are standing on a much more interesting machine. The torchvision.transforms.v2 package rebuilt augmentation around a single...
torchaudio Source Tour: TorchCodec I/O, Custom Autograd, and the C++/CUDA Ker...
Most people meet torchaudio as a one-liner: torchaudio.load("clip.wav") returns a tensor and a sample rate, and that is the end of the story. Under the hood, though, the pytorch/audio repository is a fascinating case study in how a PyTorch domain...
Inside TensorFlow 2: Eager Execution, tf.function Tracing, and the Grappler M...
TensorFlow has a strange reputation problem. Ten years ago it was the framework everyone complained about: graphs, sessions, feed dictionaries, a learning curve shaped like a wall. TensorFlow 2 deleted almost all of that. There is no tf.Session anymore, no...
Optimum: How HuggingFace Routes Transformers Models to Every Accelerator
If you have ever typed pip install optimum[onnxruntime] and watched a PyTorch checkpoint turn into an ONNX Runtime model with one class swap, you have met Optimum at its best. Optimum is HuggingFace’s optimization toolbox that sits beside Transformers as...
open_clip: A Source Tour of the Maintained Open CLIP Implementation
When OpenAI released CLIP in 2021 it changed how the community thinks about vision-language pretraining: two towers, a temperature-scaled contrastive objective, and zero-shot classification from prompt templates. The original repository — which we walked through in the earlier tour of...
NVIDIA NeMo: A Source Tour of the Typed-Module Architecture Behind Speech and...
NVIDIA’s NeMo (Neural Modules) is one of those frameworks that quietly backs a large fraction of the open speech models you have heard of — Parakeet, FastConformer, Canary, Magpie — while also carrying NVIDIA’s newer speech-aware language models. Unlike a...
MaxText: Inside Google's JAX Blueprint for TPU-Scale LLM Training and Serving
Most LLM frameworks you meet on this blog start from PyTorch and bend toward accelerators. MaxText starts from the opposite shore: it is a pure Python / JAX reference implementation built first for Google Cloud TPUs and GPUs, and it...
gemma.cpp: Inside Google's Tiny C++ Inference Engine for Gemma Models
Most production LLM serving stacks are enormous: graph compilers, paged KV caches, continuous batching, speculative decoding. Google’s gemma.cpp walks in from the opposite direction. It is a lightweight, standalone C++ inference engine built for exactly one job — running the...
OpenAI CLIP Source Tour: Inside the Dual Encoder That Learned to Match Images...
When OpenAI released CLIP (Contrastive Language-Image Pre-training) in January 2021, the accompanying paper made a claim that sounded almost careless: train on 400 million (image, text) pairs scraped from the web, and the resulting model can name things it was...
TT-Metal Source Tour: Two Stacks on One RISC-V Accelerator
Most inference frameworks we have toured on this blog, like vLLM for serving, llama.cpp for local chat, or PyTorch itself as the universal substrate, sit on top of somebody else’s hardware abstraction. Tenstorrent’s tt-metal is the opposite kind of project:...
Tokenizers Source Tour: The Rust Engine Behind Every HF Tokenizer
Every request that reaches a large language model is translated twice by the same unglamorous layer: from text into token ids on the way in, and from ids back to text on the way out. When you load a model...
Safetensors Source Tour: The Zero-Copy Weight Container
Every large model you download today is, on disk, mostly just a bag of numbers. The interesting engineering question is what wraps around that bag. For years the answer in the PyTorch world was torch.save, which wraps Python pickle —...
PyTorch Source Tour: One Op Call from Python to the Caching Allocator
Almost every framework we have toured on this blog sits on the same substrate. vLLM [(/vllm-2/)] spends its effort on KV-cache management and batch scheduling, but its tensor math is PyTorch. TorchServe [(/torchserve/)] and ExecuTorch [(/executorch/)] exist to package and...