Inside LLMs, Visualized
Tool · Tokenizer
Tiktokenizer: how a tokenizer splits text into tokens
Type text and watch how each model's tokenizer draws token boundaries in real time. An interactive tool spanning token counts, IDs, and embeddings.
Tool · Tokenizer
Type text and watch how each model's tokenizer draws token boundaries in real time. An interactive tool spanning token counts, IDs, and embeddings.