LLM Fundamentals
How large language models work, from tokens and embeddings to attention, the Transformer and training, built around “Attention Is All You Need”.
Series
Learn LLMs in order
Start with the fundamentals and the math behind them, train a small model yourself, move on to how models are served fast and how they run on GPUs, then design real systems and agents with them. Each series reads in order, and every article also stands on its own.
The Math Behind LLMs
The math inside a language model, built up from high-school algebra: vectors, probability, gradients, backpropagation, attention, RoPE and the equations behind training and alignment.
Build an LLM From Scratch
Train a small GPT-style model on one laptop GPU, step by step: tokenizer, data pipeline, model, training loop, scaling laws, evaluation, then fine-tuning, DPO and RL.
Inside the Model: Interpretability
Look inside real language models on one laptop GPU: the logit lens, probes, induction heads, activation patching, superposition, sparse autoencoders, circuits and steering, measured.
LLM Inference & Optimization
What happens when a model generates text, and the tricks that make it fast: KV caching, batching, PagedAttention, quantization, speculative decoding and more.
LLMs on the GPU: Kernels and Hardware
How a GPU runs a language model, down to the kernels: CUDA, the memory hierarchy, tensor cores, Triton, attention and matmul kernels, and multi-GPU communication, measured on real hardware.
LLM System Design
Twenty-five deep dives on building production LLM systems, from retrieval and evaluation to serving, security and operations.
Agentic AI: How AI Agents Work
How AI agents work, measured on the Claude API: the agent loop, tool calling, planning, context engineering, MCP, multi-agent systems, coding and computer-use agents, evals, reliability and cost.