Tagged: Self attention

Articles

• Fundamentals · Part 5How LLMs Work

  Attention Explained: Queries, Keys and Values

  How attention lets each token pull in context: queries, keys and values worked by hand, why we divide by √d, the causal mask, and the length-squared cost.

• Fundamentals · Part 6How LLMs Work

  Multi-Head Attention and the Transformer Block

  How a Transformer block works: many attention heads, the MLP, residual connections and layer norm, with shapes and a full GPT-2 parameter count.