Visualization

LLM Attention VisualizationA visualization of the attention mechanism in LLMs.

Isham Faizal built a browser-based tool that shows which past tokens a language model draws on when generating each new token. The implementation uses a custom generation loop with Transformers.js and a modified ONNX model to expose internal attention values, combined with pre-generated prompts to avoid multi-hundred-megabyte download waits.

Blog post · LLMs · Machine Learning · Visualization · Transformers