Context engineering: giving an agent what it needs, not everything

Prompt engineering crafts the wording of a single instruction — how you phrase the ask so the model responds well. Context engineering is the broader discipline of deciding the entire payload the model sees at each step: the instructions, the retrieved knowledge, the tool results, and the memory, including what to include and, just as importantly, what to leave out. For a one-shot prompt the two nearly coincide, but for an agent that loops many times they diverge sharply, because no amount of polishing a single instruction rescues a context full of irrelevant documents, stale tool output or hostile injected text. The model attends to everything in the window as if it were equally intended, so what you assemble around the instruction often matters more than the instruction itself. The practical shift is from optimising one string to engineering a per-step assembly process: choosing, ordering, tagging and pruning what enters the window. Prompt engineering is a skill within context engineering, not a replacement for it, and for agents the assembly is where most of the leverage lives.

Because the model has finite attention and you pay for every token, so adding context has real costs that quickly outweigh the marginal benefit. The first cost is diluted attention: padding the window with marginally relevant material buries the few facts the model actually needs, and quality drops even though you added information. The second is money and latency: every token is paid on every iteration of the loop, so a bloated context is a recurring tax, not a one-time expense, and it compounds across a long run. The third is security: every extra document or tool output you include is more surface for injected or poisoned content to ride along into the model's view, and the more untrusted material is present, the more chances a hostile instruction has to be obeyed. Relevant, minimal context beats large context on all three dimensions at once. The instinct to include everything just in case feels safe but is usually the wrong trade — the discipline is to include what the current step needs and deliberately drop the rest.

Treat every step as a fresh assembly problem rather than carrying an ever-growing transcript forward. Include four things and nothing more: the task instructions, kept clear and stable; only the retrieved knowledge that is actually relevant to the current step, not the whole store, which is where memory architecture and retrieval quality earn their keep; the tool results the current step needs, validated as untrusted data; and the minimum memory required, tagged by trust level. Then drop everything else. The two habits that make this work are re-assembling per step, so the context reflects what the agent needs now rather than everything it has ever seen, and tagging provenance so the system knows which parts are trusted instructions and which are untrusted content. Good assembly is selective by default and additive only with reason: you start from the minimum that could do the job and add a piece only when its absence causes a concrete failure. Done well, the agent sees a tight, relevant, provenance-aware context at each step, which is what simultaneously improves quality, lowers cost and shrinks the attack surface.

No — a larger context window changes the budget, not the discipline, and in some ways it makes context engineering more important, not less. It is tempting to read a million-token window as permission to stop curating and just pour everything in, but the same three problems persist at larger scale. Attention is still finite: models exhibit context rot, where relevant facts buried in a vast context are attended to less reliably than the same facts in a tight one, so quality can fall even as you add more. Cost still scales with tokens on every iteration, so a bloated context is a recurring tax that a bigger window makes easier to run up. And the attack surface still grows with every extra document you include, giving injected or poisoned content more room to ride along. A bigger window is genuinely useful — it raises the ceiling on what you can include when you need to — but the goal remains relevance, not volume. Use the extra room to include the right things more fully, not to abandon the decision about what belongs.