Tagged: Training

Articles

• Fundamentals · Part 1How LLMs Work

  How a Large Language Model Works: The Big Picture

  What really happens when ChatGPT or Llama answers you: tokens, next-token prediction, the Transformer, and why one 2017 paper changed everything.

• Fundamentals · Part 3How LLMs Work

  Neural Networks: Weights, Gradient Descent and Backpropagation

  How a neural network learns: neurons as weighted sums, loss, gradient descent worked by hand, and backpropagation explained without calculus.

• Fundamentals · Part 6How LLMs Work

  Multi-Head Attention and the Transformer Block

  How a Transformer block works: many attention heads, the MLP, residual connections and layer norm, with shapes and a full GPT-2 parameter count.

• Fundamentals · Part 8How LLMs Work

  Reading "Attention Is All You Need", the 2017 Transformer Paper

  A guided reading of the 2017 Transformer paper: encoder, decoder and cross-attention, how it was trained, what it scored, and which choices survived.

• Fundamentals · Part 12How LLMs Work

  Pretraining: Data, Scaling Laws and Training at Scale

  Where pretraining text comes from and how it's filtered, what scaling laws say about model size versus data, and how a run spreads over 16,000 GPUs.

• Fundamentals · Part 13How LLMs Work

  From Base Model to Assistant: SFT, RLHF and DPO

  How a next-token predictor becomes a helpful assistant: supervised fine-tuning, reward models, RLHF with PPO, and DPO, worked with real numbers.

• Fundamentals · Part 14How LLMs Work

  Reasoning Models: Chain of Thought, RL and Test-Time Compute

  How o1 and DeepSeek-R1 learned to think before answering: chain of thought, RL with checkable rewards, GRPO, and what thinking tokens cost.

• Math · Part 5Math for LLMs

  Softmax and Log-Sum-Exp: From Scores to Probabilities, Safely

  Why softmax uses the exponential, how one subtraction stops it overflowing, what log-sum-exp measures, and the online softmax inside FlashAttention.

• Math · Part 6Math for LLMs

  Entropy, Cross-Entropy and Perplexity

  What an LLM's loss number means: surprise, entropy, cross-entropy, perplexity, bits per byte and KL divergence, and why predicting text is compressing it.

• Math · Part 7Math for LLMs

  Derivatives, Gradients and the Chain Rule

  The derivative built from the slope of a tiny secant, gradients as the uphill direction, the chain rule and Jacobians, checked with NumPy and PyTorch.

• Math · Part 8Math for LLMs

  Backpropagation by Hand: From the Chain Rule to Autograd

  The chain rule as an algorithm: computational graphs, backward passes for matmul, ReLU, GELU and softmax, activation memory, and a 40-line autograd.

• Math · Part 9Math for LLMs

  Optimizers: From Gradient Descent to AdamW

  Why gradient descent zig-zags, and how momentum, Adam's bias correction, AdamW's weight decay, warm-up, cosine decay, clipping and Muon fix it.

• Math · Part 10Math for LLMs

  Keeping Signals Stable: Initialization, Normalization and Residuals

  Why signals explode or vanish in deep networks, and the math of the fixes: Xavier and He init, LayerNorm and RMSNorm, residuals, pre-norm and μP.

• Math · Part 14Math for LLMs

  Floating Point and Quantization: The Math of Fewer Bits

  How computers store decimals, why 256 + 1 = 256 in BF16, why training keeps FP32 copies, and the error math of scales, groups, GPTQ and AWQ on real weights.

• Math · Part 15Math for LLMs

  Counting: Parameters, FLOPs, Memory and Scaling Laws

  Exact parameter counts from a config file, 6ND derived with what it leaves out, KV cache bytes, log-log fits, the Chinchilla optimum and MoE routing math.

• Math · Part 16Math for LLMs

  The Math of Alignment: Policy Gradients, PPO, DPO and GRPO

  The equations behind RLHF and reasoning RL: the log-derivative trick, baselines, PPO's clip, Bradley-Terry, the KL-regularized optimum, DPO and GRPO.

• From Scratch · Part 1Training LLMs

  Build an LLM From Scratch: What a Laptop Can Train

  Measured training speed on an 8 GB laptop GPU, 6ND budget math, and the model this series builds: 88M parameters, 1.76B tokens, about 17 hours.

• From Scratch · Part 2Training LLMs

  Training a BPE Tokenizer: 40 Lines of Python to 32K Tokens

  Build byte-level BPE from scratch, train the series' 32,768-token tokenizer on FineWeb-Edu, and measure what vocabulary size and training data change.

• From Scratch · Part 3Training LLMs

  The Pretraining Data Pipeline: 2.5 Billion Tokens on a Laptop

  Download FineWeb-Edu, check its filters, remove near-duplicates with MinHash, then tokenize, shuffle and pack the series' training data into token shards.

• From Scratch · Part 4Training LLMs

  Build an LLM From Scratch: Writing the Model

  The series' 88M-parameter Llama-style model in 179 lines of PyTorch, checked against the formula, Hugging Face's Llama, a first loss and an overfit test.

• From Scratch · Part 5Training LLMs

  Build an LLM From Scratch: The Training Loop

  train.py for the series' model: AdamW, warm-up and cosine decay, clipping, BF16, gradient accumulation, exact resume, and a first real loss curve.

• From Scratch · Part 6Training LLMs

  Build an LLM From Scratch: Making Training Fast on One GPU

  From 5,800 to 30,000 tokens per second on a laptop GPU: TF32, BF16, FlashAttention, torch.compile, fused AdamW and batch size, each measured on its own.

• From Scratch · Part 7Training LLMs

  Build an LLM From Scratch: The Pretraining Run

  16.8 hours, 1.76 billion tokens, 0.83 kWh: the 88M model's run on a laptop GPU, read through its loss curves, spikes, held-out loss and samples.

• From Scratch · Part 8Training LLMs

  Build an LLM From Scratch: Scaling Laws on a Laptop

  Five model sizes from 11M to 47M parameters in 12 hours on a laptop GPU, a fitted power law, and a direct test of Chinchilla's 20 tokens per parameter.

• From Scratch · Part 9Training LLMs

  Build an LLM From Scratch: Evaluating a Base Model

  Perplexity, bits per byte and zero-shot multiple choice for the 88M model against GPT-2, Pythia and SmolLM2, with a harness checked against lm-eval.

• From Scratch · Part 10Training LLMs

  Build an LLM From Scratch: Supervised Fine-Tuning

  Turning the 88M base model into a chat model in 15 minutes: a chat template on reserved tokens, masked loss, 26,000 conversations, and what it cost.

• From Scratch · Part 11Training LLMs

  Build an LLM From Scratch: Preference Tuning with DPO

  DPO on the 88M chat model: UltraFeedback pairs made its replies longer; its own replies, ranked by a reward model, taught it to stop. Same loss.

• From Scratch · Part 12Training LLMs

  Build an LLM From Scratch: RL with Verifiable Rewards (GRPO)

  GRPO on the 88M model: a checker for addition, 8 replies per problem, reward curves, pass@k before and after, and what RL couldn't teach it.

• From Scratch · Part 13Training LLMs

  Build an LLM From Scratch: Distillation and Synthetic Data

  A 15M student learns from our 88M model's probabilities, and the 88M model learns from replies an open 1.5B model wrote: costs, filters, results.

• From Scratch · Part 14Training LLMs

  Build an LLM From Scratch: Fitting Bigger Models in Memory

  Where 8 GB goes when training the 88M model, predicted and measured; activation checkpointing, 8-bit AdamW and offload timed; ZeRO and FSDP explained.

• From Scratch · Part 15Training LLMs

  Build an LLM From Scratch: From Our Model to Frontier Models

  Our 88M run next to Llama 3, DeepSeek-V3, OLMo 2 and SmolLM, the model exported to Hugging Face and llama.cpp, and the whole series reviewed.

• Interpretability · Part 10Interpretability

  What Training Builds Inside a Model, Snapshot by Snapshot

  Word counts, then word pairs, then induction heads in 300 steps: 49 snapshots of the From Scratch model show what forms when, and what set the timing.

• Agents · Part 14Agents & Tool Use

  Training Models to Be Agents: Trajectories, Environments, Rewards

  How models learn to act: fine-tuning on agent trajectories, sandboxes by the million, rewards a program can check, multi-turn RL and reward hacking.