Lab
A Lab for Seeing How Complex Technology Works
Some technologies never quite click through text alone. The Lab unpacks the core principles behind LLMs with interactive visualizations and provides hands-on tools like the tokenizer. We keep adding content so anyone can grasp technologies that feel difficult, from AI to blockchain, by experimenting with them directly.
Understand why attention dilutes (Part 1), then see the practical traps that follow (Part 2). The tokenizer is a standalone tool you can jump into anytime.
Part 1 · Cause (Mechanism)
Why attention disperses as tokens grow
Transformer attention computes token-to-token relations at N² cost. See how attention dilutes as the token count grows.
Part 2 · Consequence
Lost in the Middle: position-based accuracy curve
Find out why an LLM forgets the instructions you gave at the start as a conversation grows, explained through the accuracy-by-position curve.
Tool · Tokenizer
Tiktokenizer: how a tokenizer splits text into tokens
Type text and watch how each model's tokenizer draws token boundaries in real time. An interactive tool spanning token counts, IDs, and embeddings.