Lab
Inside LLMs, Visualized
A hands-on collection of interactive aids for building intuition about context and tokens.
Share
↓
Understand why attention dilutes (Part 1), then see the practical traps that follow (Part 2). The tokenizer is a standalone tool you can jump into anytime.
Part 1 · Cause (Mechanism)
Why attention disperses as tokens grow
Transformer attention computes token-to-token relations at N² cost. See how attention dilutes as the token count grows.
Attention · N² Scaling
Part 2 · Consequence
Lost in the Middle: position-based accuracy curve
Find out why an LLM forgets the instructions you gave at the start as a conversation grows, explained through the accuracy-by-position curve.
Position Bias · Smart Zone
Tool · Tokenizer
Tiktokenizer: how a tokenizer splits text into tokens
Type text and watch how each model's tokenizer draws token boundaries in real time. An interactive tool spanning token counts, IDs, and embeddings.
Tokenization · Interactive