Context is everything
A model has a fixed amount of context. When that space gets crowded, you may see it forget earlier details, contradict itself or bring back code you already removed.
A model can only hold a fixed amount at once. Your files, your chat, its own replies and your rules file all compete for that same space.
When that room fills, the oldest material falls out. Not the least important. The oldest.
This single fact explains almost every mystifying agent behaviour a beginner meets.
It re-adds code you told it to delete, because the instruction to delete it is gone. It contradicts a decision from an hour ago, because the decision is gone. It invents a function signature, because the file it read at the start is gone and it is filling the gap from memory of similar code.
Load the window below until it overflows. The panel underneath does not just say the model is confused; it shows the specific kind of failure each missing item can cause.
What the model can still see
2.9k / 200k tokensCosts are realistic orders of magnitude, not measurements. The proportions are the point.
Everything fits. The model can see all of it, and will behave consistently. Keep adding: the interesting part is what happens when it does not fit.
Two habits help a lot here. Start a fresh session when you move to a new task instead of dragging a four-hour conversation along with you. If a session starts behaving oddly, restate the few facts it needs or start a clean session.
You should now be able to
- Explain what a context window is and what falls out of it first
- Predict which failure you will get when it overflows
- Keep a long session useful instead of watching it degrade
Loading…