Coding / AI Engineering / Context Engineering / billion-context
CHAPTER 02 / 11 · CONTEXT ENGINEERING
The long-context problem
A growing history, a finite workspace
Tool outputs and new turns enlarge the working set. Window capacity and a token budget constrain different things: how much fits on one request and how much repeated processing costs.
INTERACTIVE STUDY / 02
Local teaching modelCONTEXT TRAJECTORY
A window is a limit.
Work is a running total.
Compare the same synthetic sequence of requests. A fold drops resident input; it does not erase work already processed.
Fits inside the chosen window
Sum of every plotted request input
Distinct from repeated input processing
Synthetic accounting model: +2K new text per turn, a fold every fourth turn, +300 summary tokens per cycle. The chart continues hypothetical growth past overflow to show the limit; a real request must fit. No cost or quality estimate.
THE NOTEBOOK
Concise learning notes · source-grounded mechanismsThere is a conflict between:
- the user's token / monetary budget,
- the model's limited context window,
- and the continuously growing amount of information produced during a long-running agent session.
As the session grows:
Eventually, this can increase cost or exceed the model's context-window limit.
CHECK YOUR INTUITION
What does a context window limit constrain?
Choose an answer to reveal the reasoning.
FOLLOW THE SOURCE
Reviewed source: billion-context 0.1.180 / 03d27f9. The notes are Lawson’s learning interpretation of the reviewed repository.