This is called Speculative Decoding

Gaurav Sen explains speculative decoding, the technique Google published in 2022 to make large language models answer 2 to 3 times faster with the same output: a small model drafts several tokens, the large model verifies the draft's prefixes in parallel, and the method now serves production traffic at Google and DeepSeek.

Tutorial · AI Inference · Large Language Models · Machine Learning

Do we really need bigger AI weather forecasting models to improve resolution?

Emma Scharfmann shows that instead of building a bigger AI weather model you can train a small downscaler on top of an existing global one: trained in about an hour, it sharpens AIFS-ENS temperature forecasts over France from ~28 km to 5.5 km and cuts station-measured error by 14% RMSE and 21% CRPS over the first two days.

Tutorial · Machine Learning · AI · Science · Open Source

Handwired: turn your regular JavaScript code into a neural net

Cheng Lou's Handwired transforms ordinary JavaScript control flow into exact, fully differentiable neural nets that run in the browser without a GPU—including demos like a mini desktop, trainable gallery, and parallel JSON search.

Tutorial · Machine Learning · Programming · JavaScript · Tutorials

Welcome Muse Spark Open: open-weight frontier multimodal mixture-of-experts model from Meta

Hugging Face's guide to Meta's Muse Spark Open, an 816B-parameter (42B active) open-weight multimodal mixture-of-experts model with a 1M-token context: architecture, perception and audio encoders, and how to run it with Transformers, llama.cpp, SGLang, vLLM and MLX, deploy it, fine-tune it with TRL, plus demos.

Tutorial · AI · LLMs · Open Source · Machine Learning

You could have invented transformers from memory

Alex Wang builds intuition for linear maps, MLPs and self-attention as different kinds of memory stores, using the interference-to-signal ratio to estimate each one's capacity. With 64-dimensional keys, capacity grows from about 64 memories for a linear layer to about 2,700 for an MLP and over a million for self-attention.

Tutorial · AI · Machine Learning · Transformers · Mathematics

Running RAG with a (small) language model in the browser

Bart de Goede extends his static blog's in-browser semantic search into full retrieval-augmented generation by running small language models client-side with transformers.js on WebGPU or WASM, and evaluates how faithful and useful answers from Qwen3, Gemma and other small models are compared with larger ones.

Tutorial · LLMs · Web Development · Machine Learning · Tutorials

Build your own decision model

Nish Tahir explains "system one" decision models like TypeSafe AI's Jev, which answer in a single forward pass by masking the vocabulary to a fixed set of options, then walks through fine-tuning a small model on multiple-choice data and calibrating its confidence with temperature scaling, with a companion GitHub repo.

Tutorial · AI · Machine Learning · LLMs

Why Is Speculative Decoding Fast?

Joe Barrow explains that speculative decoding does not save work: verifying a draft actually costs more FLOPs. It is fast because single-token decoding is memory bound, so drafted tokens use idle GPU compute at small batch sizes. That is why it can waste compute at high throughput, and why dynamic per-sequence speculation depth helps.

Tutorial · Machine Learning · LLM Inference · GPUs

Functional ultrasound imaging from scratch

Functional ultrasound imaging (fUSI) is a brain imaging modality that is just starting to be demonstrated in human studies. I’m sure many will be familiar with structural ultrasound imaging, which forms static images using ultrasound, as in a sonogram1. Functional ultrasound instead captures movies of brain tissue very rapidly, tracking subtle changes caused by the blood flow that follows neural activity.

Tutorial · Science · Machine Learning · Research · Tutorials

Build an LLM Tokenizer and Attention from Scratch in TypeScript

SitePoint tutorial that implements BPE tokenization, cosine similarity vector search, and scaled dot-product attention in TypeScript—exposing LLM primitives as ordinary readable code.

Tutorial · LLMs · Tutorials · Programming · Machine Learning

Turn GLM-5.3-Flash into a Jev-like System One model

Johannes Hötter walks through turning GLM-5.3-Flash into a fast Jev-like “System One” decision model—typed options with probabilities in a single forward pass, matching Jev’s accuracy and speed.

Tutorial · AI · LLMs · Machine Learning · Open Source

Teaching a World Model to Play Pokémon

Training a JEPA-style LeWorldModel on Pokémon Red screenshots and button presses, then using CEM latent planning to select a starter—why latent collapse, SIGReg, and rollout fine-tuning mattered, and 52/100 plans succeeding after tuning.

Tutorial · AI · Machine Learning · Research · Programming

Mixture of Experts (MoE) for Backend Engineers

A detailed visual guide to token routing, expert batching, weighted combination, and the memory and communication tradeoffs of MoE serving. Suppose a model has dozens of feed-forward subnetworks, but each token uses only two of them. The arithmetic per token can stay modest while the total weight set grows. Now place that model on eight GPUs. If every GPU stores all experts, memory can become the limit; if experts are split across GPUs, token activations must travel to whichever GPU owns their…

Tutorial · AI · LLMs · Machine Learning · Engineering

V-JEPA: Learning Video Representations by Feature Prediction

A hands-on walkthrough of Meta’s V-JEPA: video tubelets, feature prediction, pretrained encoder features, temporal tests, and a small action-classification experiment.

Tutorial · Machine Learning · AI · Research · Tutorial

Heretic tutorial: automatic censorship removal for language models

A hands-on tutorial for Heretic, an open-source tool that automatically removes refusal/censorship behaviors from language models—setup, workflow, and what to watch for.

Tutorial · AI · Open Source · LLMs · Machine Learning

Transformer Explainer: LLM Transformer Model Visually Explained

Georgia Tech’s Polo Club walks through GPT-2’s Transformer stack—embeddings, multi-head attention, MLP, sampling—with an interactive in-browser model for learning how next-token prediction works.

Tutorial · AI · LLMs · Machine Learning · Education

Multimodal Agents: From Perception to Action

Illustrated notes from Berkeley’s LLM Agents lecture 7: OSWorld outcome checks, AgentTrek trajectories, TACO tools, and Aguvis grounding—why reading a screen is not the same as finishing the task.

Tutorial · AI Agents · Machine Learning · Research

In Search of PCA's Lost Variance

A geometric derivation of why PCA reconstruction mean-squared error equals the sum of discarded eigenvalues, using orthogonal projections and a clear 2D example to make the lost-variance identity intuitive.

Tutorial · Mathematics · Machine Learning · Education

Training Search Agents with GRPO

Hands-on introduction to reinforcement learning by training a search agent with group-relative policy optimization (GRPO), with open rollouts, code, and reward-design lessons for LLM search.

Tutorial · AI · LLMs · Machine Learning · Research

Use 20x less peak RAM with dp_knapsack_sliding_hirschberg, a new activation memory budget solver for PyTorch

Jędrzej Maczan explains PyTorch's activation-checkpointing memory planner as a 0/1 knapsack problem and introduces dp_knapsack_sliding_hirschberg, a sliding-window DP with Hirschberg-style reconstruction that cuts peak RAM during memory planning by about 20x.

Tutorial · Machine Learning · Algorithms · Performance · Open Source