In-Browser Attention Visualiser
Attention Lens runs the 124 million parameter GPT-2 model inside your browser, captures the attention matrices from every layer and every head, and renders them as interactive heatmaps, token to token connection views, and a full 12 by 12 model overview. Nothing leaves your device.
Input
Attention Visualisation
Six views share the same underlying tensor, switch with arrow keys inside the tablist.
Heads View
Attention heatmap
What am I seeing?
Each cell shows the attention weight from a query token (row) onto a key token (column). Rows sum to 1 after softmax with the causal mask. The top axis is the query token, the left axis is the key token. A strong diagonal means the head reads nearby tokens; a first-column stripe is the beginning-of-sequence sink. Use the Mono or Pattern toggles below if the colour ramp is hard to read.
Token to token
Incoming attribution
Layers View
Attention rollout
What is rollout?
Attention rollout stacks the head-averaged attention for every layer, mixes in the residual stream as an identity matrix, and multiplies them together so the resulting row for a query token approximates how much of each earlier token contributed through the whole network.
Model Overview
Compare two heads
Pane A
Pane B
Head statistics
Entropy and sparsity per head
Top 5 peakiest
Top 5 most diffuse
Induction heads detected
Head ablation
Head ablation approximation
What does this do?
We zero one attention head, re-weight the last-layer value projection, and compare the resulting next-token top five against the unablated baseline. This is an approximation. We do not rerun the full forward pass client-side; we apply the head mask to the exported attention tensor and recompute the final value-mixed residual. Use this as a rough signal, not a causal trace.
Δ next-token probability, top 5
Positive bar means the ablation increased the probability of that candidate token, negative means it decreased it.
Selection Controls
Ranges respect arrow keys, Home and End, plus Page Up and Page Down.
Case studies (precomputed attention)
Performance
| Run | Tokenise | Forward | Extract | Render | Total |
|---|
About and credits
Attributions and references
- GPT-2 architecture, Radford et al., 2019.
- Transformers.js, Xenova, 2023.
- WebGPU specification, W3C, 2024.
- ONNX Runtime Web, Microsoft, 2022.
- Attention is all you need, Vaswani et al., 2017.
- In-context Learning and Induction Heads, Olsson et al., 2022.
- Attention Rollout, Abnar and Zuidema, 2020.