Tagged: Neural networks

Articles

• Fundamentals · Part 3How LLMs Work

  Neural Networks: Weights, Gradient Descent and Backpropagation

  How a neural network learns: neurons as weighted sums, loss, gradient descent worked by hand, and backpropagation explained without calculus.

• Math · Part 3Math for LLMs

  Matrices as Transformations, and Why Neural Networks Need a Bend

  What a matrix does to space, why stacked linear layers collapse into one, how ReLU and SiLU fix that, and how batching changes a matmul's cost.

• Math · Part 8Math for LLMs

  Backpropagation by Hand: From the Chain Rule to Autograd

  The chain rule as an algorithm: computational graphs, backward passes for matmul, ReLU, GELU and softmax, activation memory, and a 40-line autograd.