Multimodal · Part 1Images, Speech & Video
How AI Sees, Hears and Draws: Everything Becomes Tokens
Images, audio and video all become sequences of vectors before a transformer sees them: a measured token ledger across eight open models and Claude.
Tagged: Diffusion
5 articles.
Articles
Multimodal · Part 5Images, Speech & Video
Diffusion From Scratch: Noise, Scores and a Laptop DDPM
The DDPM forward process, noise schedules, the noise-prediction loss and the score view, with a spiral and a Fashion-MNIST U-Net trained and measured.
Multimodal · Part 6Images, Speech & Video
Latent Diffusion: How Stable Diffusion Turns Text into Pictures
Stable Diffusion 1.5 taken apart and timed on a laptop GPU: the VAE, the U-Net, CLIP text, classifier-free guidance, negative prompts and samplers, measured.
Multimodal · Part 7Images, Speech & Video
Diffusion Transformers and Flow Matching: Straight Paths, Measured
How flow matching learns straight paths from noise to images, why reflow makes one step work, and what a DiT changes, measured on a laptop GPU.
Multimodal · Part 12Images, Speech & Video
Video Generation: Spacetime Patches and Attention, Measured
How video models squeeze clips into spacetime patches, why attention cost explodes with clip length, and what temporal layers buy, measured on a laptop GPU.