Part 2 · Consequence
Lost in the Middle: position-based accuracy curve
Find out why an LLM forgets the instructions you gave at the start as a conversation grows, explained through the accuracy-by-position curve.
Note: this visualization's interface is in Korean.
Read the full notes as text
“Lost in the Middle” describes a consistent failure mode of LLMs: information at the start and end of the context is retrieved well, while information in the middle loses 30%+ accuracy. The same U-shaped pattern appears across every major LLM (source: Chroma Research · Liu et al. 2023, “Lost in the Middle”).
| Zone | Position | Accuracy | Notes |
|---|---|---|---|
| Start | 0–20% | ~90% | System prompt and early instructions. Remembered clearly |
| Middle | 40–60% | ~55% | 30%+ drop. Mid-conversation content and middle document sections are easily missed |
| End | 80–100% | ~88% | Recent messages and the final question. The model’s focus area |
Q1. What exactly does the X axis mean?
The X axis is not “percent of the whole context window” but the relative position within the tokens you actually provided. If you fed 1,000 tokens, those 1,000 tokens span 0–100%; if you fed 800k, those 800k span 0–100%.
- Capacity (context window): the maximum tokens a model accepts at once, e.g. 200k or 1M. The size of the bookshelf.
- Position (token order): where a token sits within the input you provided, which decides how much attention it gets. The spot where a book is shelved.
Key point: a bigger bookshelf means you can shelve more, not that the model reads every shelf equally well. Attention weights still concentrate on the front and back of the input.
Q2. When does the curve show up strongly?
The U-shaped curve is the profile of a full (dense) window. With only 1,000 tokens in a 1M window, everything sits effectively “at the front” and mid-context loss barely occurs. Fill 800k–1M and the “middle” stretches long, blurring information rapidly.
The density rule: growing the window to 1M does not remove the U pattern; it widens the middle. Simple retrieval still works, but multi-fact reasoning collapses in the middle.
Q3. Smart Zone / Dumb Zone: the absolute-token view
Instead of density (%), you can reason in absolute tokens(source: AI Hero). The region the model handles “sharply” is the Smart Zone; everything after it, where attention thins and information sinks, is the Dumb Zone.
- Smart Zone (~80,000–100,000 tokens): nearly constant regardless of window size. On a 200k window that is roughly the first 40% used “smartly”.
- Dumb Zone (everything after): what actually grows when windows grow. The information is in there, but the model does not perceive it clearly. The heart of “fits ≠ seen”.
The 1M trap: going 200k → 1M multiplies the window by five, yet the Smart Zone barely moves. Filling ~100k of a 1M model (about 10% occupancy) already reaches the edge of the Smart Zone. You did not get “a roomy 1M”; you got a 1M whose extra space is Dumb Zone.
Practical conversion: on a 200k window the Smart Zone is ≈ 40%; on a 1M window it is ≈ 8–10%. This is where intuition fails. At 10% of a 1M window you feel like “90% is still free”, but that 10% is already ~100k tokens, the tail end of the Smart Zone. The larger the window, the more absolute tokens the same occupancy holds, so a window that looks empty in percent may already be past the limit of clear vision.
Recent changes: on Opus 4.8 you can enable 1M context with the [1m] suffix (e.g. claude-opus-4-8[1m]), and Claude Fable 5 (claude-fable-5, released June 2026) ships with 1M context by default. Still, “available (or on by default) ≠ should always be filled”. Even at 1M the Smart Zone stays at ~80k–100k; what grew is the Dumb Zone.
Reality check: fits ≠ should be fed
In Vibe Coding, Gene Kim and Steve Yegge estimate that approximately 80% of Git repositories worldwide fit within a 500,000-token context window. With tools like GitIngest, Repomix, and files-to-prompt you really can concatenate a whole repository into one text and feed it in. But the warning the authors attach right after the statistic is the conclusion of this page: “just because you can doesn’t mean you always should.” Filling 500K blows past the Smart Zone (~80k–100k) from Q3 by a factor of five, and most of the repository sinks into the Dumb Zone.
Source: Gene Kim & Steve Yegge, Vibe Coding (IT Revolution, 2025), p. 113, Chapter 10 “Managing Your Cutting Board: AI Context and Conversations”. The figure is the authors’ own analysis (“based on our analysis”), with a footnote on p. 113 linking to the ChatGPT conversation behind it.
Q4. So how full should the window get?
Stability varies with input density; 100k in a 1M window is 10% and very safe. Different models give slightly different answers to the same question, so it helps to compare three views side by side.
Gemini’s view: the density rule, general Q&A and document work
| Occupancy | State | Recommended strategy |
|---|---|---|
| 0–20% | Safe | Peak focus. Place important instructions and logic freely. |
| 20–60% | Caution | Middle content starts to dilute. Put the essentials at the front and back. |
| 60–90% | Risk | Mid-context loss kicks in. Prune stale logs and old conversation. |
| 90%+ | Limit | Sharp degradation. Reset the session with /clear or feed only key chunks via RAG. |
Gemini’s rule of thumb: judgment is most stable at 30–40% density. Not “full means smart” but “empty space makes it smart”.
Claude’s view: agent and coding work (multi-hop reasoning, accumulating tool results)
| Occupancy | State | Recommended strategy |
|---|---|---|
| 0–30% | Safe | Multi-file edits and long-range references all stable. Complex plans fine. |
| 30–50% | Fair | Single tasks OK. In multi-step reasoning, early instructions start to blur. |
| 50–75% | Caution | Tool results and file reads pile into the middle. Re-read stale files for accuracy. |
| 75%+ | Risk | Near auto-compact. /clear, or move key decisions to CLAUDE.md/plan and open a fresh session. |
Claude’s rule of thumb: the safe band for agentic work is narrower than for plain Q&A. Tool results accumulate in the middle, so a “packed 30%” is far blurrier than a “roomy 30%”.
Codex’s view: large codebases, refactoring, tool-call loops
Codex slices the context by what occupies it. The same occupancy carries different risk depending on its composition, and code work leans on multi-hop reasoning more than plain Q&A, so the thresholds sit lower.
① Code-work stability by occupancy
| Occupancy | State | Recommendation |
|---|---|---|
| 0–25% | Best | All complex work allowed: large refactors, new module design. |
| 25–45% | Stable | Single features and bug fixes stable. Split multi-file refactors. |
| 45–70% | Warning | Early plan/CLAUDE.md instructions drift into the middle. Re-confirm them. |
| 70%+ | Compact | Auto-compact imminent. Moving to a fresh session is almost always faster. |
② Recommended ceilings per context component
| Ceiling | Component | Why |
|---|---|---|
| ~5% | System · CLAUDE.md · plan | Kept “at the front” for stable attention. Bloat here taxes the real work. |
| ~25% | Read files · code snippets | Refactor targets must be seen precisely. Past the limit, re-read only relevant chunks. |
| ~15% | Tool results (ls/grep/test logs) | The fastest-growing item and the main filler of the middle. Summarize or discard. |
| ~10% | Conversation history | Old user turns may sink safely. Extract key decisions into the plan, then forget. |
Codex sees the sum staying within ~55% as the “sustainable working area”, leaving the rest as thinking room for the model.
③ Session reset and compaction triggers
| Trigger | Signal | Response |
|---|---|---|
| Weak | Reading the same file 3+ times | Write a summary of that file into the plan. |
| Medium | Tool results alone exceed 30% | Keep only the essence of logs/grep output. Narrow the next call. |
| Strong | Second retry on the same bug | Likely mid-context loss. /clear and resume with conclusions only. |
| Critical | Model ignores early instructions | Already pushed to the U’s center. New session immediately. |
Codex’s rule of thumb: what fills the context matters more than the occupancy number. 30% of tool results is far riskier than 30% of code; both fill the middle, but their value density differs.
Field rule (Modern Web Labs + some engineers): independent of the three views above, in practice we hold the line at never exceeding 20%. The more important the task, the more deliberately we keep the context empty; when the 20% threshold nears, we move state into a plan or memo and /clear. The key is not clearing blindly but writing the current state into a HANDOFF.md file first: the goal, work done so far, next steps, and the key files and decisions (just tell the agent “summarize our progress into HANDOFF.md”). The next session resumes from that one file without the full history. You are storing context in files, not the model’s window. Slack is accuracy.
Cost angle: the old beta charged a long-context premium (2× input, 1.5× output) beyond 200K, but when 1M context went GA in March 2026 the premium was retired. Today the entire 1M range bills at standard rates: a 900k-token request costs the same per token as a 9k one. Even so, “the window grew, so feed it everything” still loses. The multiplier is gone, but cost scales linearly with tokens, and above all context rot degrades quality. Work within 200K by default and switch 1M on deliberately for jobs that must see cross-file dependencies at once, like architecture reviews and large refactors.
Q5. I tuned my workflow for 200K. How should it change at 1M?
A frequent question since 1M context opened up to Claude Code Max users. The short answer: not “I can feed 5× more” but “the workflow itself needs redesigning”. A bigger window does not make context management less important; it makes finer-grained management possible.
① Compaction: granularity, not timing
At 200K the norm was a preemptive /compact around 70%. The same observation holds at 1M: manage actively near 75% (reports say pushing to 90% raises output volume but lowers quality). What changes is the move from whole-session summaries to partial summaries.
Practical tip: selectively pruning old tool results, or rewinding to a specific point with Esc+Esc (rewind), is where partial compaction pays off at 1M. Rewind shines when low-value output like grep/ls dumps or long logs have flooded the context: rewinding to just before that call sweeps the waste out in one move.
② “Load everything” vs “explore on demand”
- Full loading wins (architecture reviews, large refactors): when cross-file dependencies must be visible at once, e.g. redrawing domain boundaries or splitting modules. Turn 1M on deliberately.
- On-demand wins (single-file bugs, feature work): for everyday tasks, reading via search tools as needed stays more efficient and flexible. Anthropic’s official guidance agrees.
Caution: having 1M does not justify shoving in the whole codebase. It sinks into the Dumb Zone unseen, and even at standard rates cost still grows linearly with tokens.
③ Subagents: from “saving” to “separation of concerns”
At 200K, subagents were primarily a context-saving device. At 1M their role shifts to parallelism and separation of concerns: the main session holds the architecture and orchestrates, subagent A investigates the auth module, subagent B analyzes test coverage, each returns a summary report, and the main session synthesizes and implements. Even with room to spare, subagents keep exploration for different concerns from polluting the main context.
④ CLAUDE.md and external state files, redefined
- CLAUDE.md: loaded every session, so keep it lean. It should point at “which file to read when”.
- TODO.md / SPEC.md: task state kept in external files (with Plan Mode) survives compaction.
- HANDOFF.md: a handover file written before
/clearor ending a session: goal, progress, next steps, key files. The next session resumes from this file alone (see the field rule above). - Custom instructions: writing “preserve the modified-file list and test commands during compaction” into CLAUDE.md keeps the critical context alive.
⑤ Cost and switching guardrails
| Situation | Recommended mode | Why |
|---|---|---|
| Everyday feature work and bug fixes | Standard 200K + on-demand exploration | Most work finishes inside the Smart Zone. Wins on cost and quality. |
| Domain-model refactoring, module boundary redesign | Switch to opus-4-8[1m] → record results in SPEC.md → execute in a fresh session | Use 1M for the planning phase only; execution returns to standard mode. |
| Code review, security audits, cross-cutting analysis | Load whole modules at 1M | Patterns only show when seen at once. High value justifies the token spend. |
| After the task completes | /compact or /clear → resume from SPEC.md | Do not leave 1M “always on”. Next session returns to standard. |
One-line summary: 1M is a tool you expand into strategically, not a mode you leave on. Run 200K economically by default and enable 1M deliberately for high-value work.
What this means in practice
The U-shaped accuracy curve appears in every LLM, which means placement itself affects accuracy when you design prompts.
- Put long documents at the top (start) and the question at the very end → up to 30% better performanceper Anthropic’s official Long Context Tips.
- Do not sandwich critical context in the middle. “It’s in there” does not mean the model will see it.
- As a long conversation grows, early instructions drift into the middle and start being ignored. Reset with
/clear.
Key takeaway
“Context isn’t just a length problem; it’s a position problem.” Put what matters at the start and the end.