What Is Agentic AI Context Compaction? Decide What Goes

Agentic AI context compaction is the step in an agent’s runtime loop where the harness takes a conversation that is nearing the context window limit, has the model write a summary of it, and restarts the loop with that summary in place of the original history. Everything before the summary is gone from the model’s view. The agent keeps working, and what it remembers is whatever the summary kept. Compaction is a deletion with a narrative attached, and that narrative is written by the same model whose work it is describing.

You Cannot Keep Everything. You Can Decide What Goes.

That is the post. The rest is how the mechanism works, where it sits in the six layers, what the evidence says about what survives, and the one design decision that separates a compaction you can defend from one you cannot.

What compaction actually does

Anthropic’s compaction documentation describes the server-side version plainly. When input tokens reach a trigger, 150,000 by default and never below 50,000, the API pauses, has the model write a summary of the transcript, wraps it in a compaction block, and continues the response from the summary. On the next request, everything before that block is ignored. A long task can compact several times, and each new block replaces everything before it.

Two details in the docs matter more than the mechanism. The summary is written by the same model that is doing the work, and there is no option to use a different one. And the summarization prompt is a vendor default that you can replace but not extend: the instructions parameter, when you supply it, replaces the whole default prompt rather than adding to it. So the question of what survives is decided by a prompt, and by default that prompt is not yours.

There is a third detail that only shows up if you read to the bottom. On the newest models the agent’s earlier thinking is not carried into the summary. The summary is all the model has of the reasoning it did before the cut. Whatever it decided and did not write down is gone.

Anthropic’s engineering post on effective context engineering explains why anyone would accept that trade. As a context window fills, the model gets worse at using what is in it. Chroma’s context rot report measured the effect across 18 models: recall degrades as input length grows, and degrades unevenly, so the fact you need most may be the one that fades first. Compaction exists because a smaller, curated context outperforms a larger, polluted one, well before the hard limit is reached. The docs say it directly: context is a finite resource with diminishing returns.

What survives and what does not

Anthropic’s own cookbook run is the most useful number I have. A research agent reading eight documents climbed to 335,279 tokens across five turns, 96 percent of it file contents. With compaction set to trigger at 180,000, it fired once at turn four and replaced the prior turns with a summary of about 2,800 tokens. Peak context fell to 169,164. The cookbook then probes what the agent can still answer. Headline facts central to the task survive. Specifics that were incidental at the time of the summary tend not to.

Claude Code’s behavior is the same pattern at the desk. Its compaction prompt is tuned to preserve architectural decisions, unresolved bugs and implementation details while dropping redundant tool output, and it carries the five most recently touched files forward. The engineering post is candid that over-aggressive compaction loses subtle context whose importance only becomes apparent later, and its advice to teams building their own is to tune the prompt on real agent traces, maximize recall first, then trim for precision.

The complaints from people running Claude Code for long sessions are the field data. File paths, error codes, and the reason a decision was made are the recurring casualties. That is why the PreCompact and PostCompact hooks exist: one fires before the transcript is summarized so state can be written to disk, and one fires after with the summary in hand so it can be logged or restored. The community built a save-before-forgetting layer because the default forgets things engineers need. A regulated program cannot rely on a community layer. It has to build its own and call it a control.

Where it sits in the six layers

In the six-layer architecture, compaction is a runtime-layer component. It lives inside the harness, the code around the model that runs the loop, and it is a performance mechanism first: it keeps the agent working past the window and keeps the working context sharp. That is where most teams leave it, as a setting that makes long tasks possible.

The governance problem is one layer down and one layer over. Compaction rewrites the agent’s working memory, and the intent layer’s file, the intent.md that says what the agent was asked to do and who approved it, is only as durable as its position in the context. If intent lives in the message history, compaction can summarize it into a sentence, and the sentence can drift. If intent is pinned outside the compactable history, in the system prompt with its own cache breakpoint, which is exactly what the docs recommend for cache efficiency, the agent re-reads the original after every cut. That is the same file, in two places, with two very different governance outcomes.

What breaks

The first failure is the drifting constraint. The intent said do not touch the pricing table. Three compactions later the summary says the agent was asked to update the catalog. Nothing in the loop is wrong. The instruction simply got paraphrased into something looser, and the agent is now acting on the paraphrase.

The second is the silent summary. The docs note that when tools are defined, the model occasionally calls a tool during the summarization step instead of writing the summary, and the compaction block comes back empty. The fix is one line in the instructions. The failure, if nobody checks for it, is an agent that continues with no memory of the task at all.

The third is the missing receipt. Compaction is a second sampling step and it is billed. Usage comes back as an iterations array, and if your cost tracking reads the top-level token counts it will under-report every compacted request. You will see the cost per completed task drift upward on long tasks and not be able to explain it.

What transfers to regulated origination

In my work in regulated loan origination the agent that works a loan file for an hour will compact, and every compaction is a rewrite of the working record. The design I use treats the compaction summary as an artifact, not a byproduct. Three rules.

The constraints do not live in the history. The intent file, the condition list, the approval, and the list of things the agent may not do sit in the system prompt with their own cache breakpoint, so they are re-read verbatim after every cut. What the agent may summarize is what it did, never what it was told.

The summarizer is a governed prompt. I replace the default instructions with one that names what must be preserved: every document referenced, every value calculated, every open condition, and an explicit line that the summary must not call tools. That prompt is versioned in the repository and changes to it go through the same review as a change to a hook.

Every summary is logged with the loop it replaced. The PostCompact hook, or its equivalent in a custom harness, writes the summary and the token counts from the iterations array to the trace, keyed to the loan file. When an examiner asks why the agent did what it did after the third compaction, the answer is the summary it was working from, and I can produce it.

When your agent last compacted, who wrote the summary prompt, and where is the summary stored?

Bottom line

Compaction is not optional for long-running agents, and it is not a cache. It is the moment the harness decides what the agent will forget, using a prompt you probably did not write, producing a record you probably did not keep. Pin the intent outside the history. Own the summarization prompt. Log every summary. That turns a performance setting into something you can show a reviewer. The runtime layer and the harness get a fuller treatment in Agentic AI, and the case for making every one of these mechanisms auditable before autonomy is granted is in Earned Autonomy.

Instructions in, results out was IT. Intent in, outcomes out is agentic AI, and an intent the agent has summarized into something else is no longer the intent you approved.

What did your agent forget last week, and would you know?

Go deeper

Agentic AI Architect: control design for enterprise agents.
Agentic AI Case Studies: deployment evidence, one teardown at a time.
Agentic AI P&L: cost, payback, and risk in a CFO’s voice.
Agentic AI Governance: who owns the harness.
Substack first access: agenticaiarch.substack.com/subscribe

Dr. Harish Kotadia, Ph.D., is an Enterprise AI Architect with 20+ years of IT consulting experience serving Fortune 100 clients, specializing in agentic AI systems built on Anthropic Claude, AWS Bedrock, and Google Vertex AI.

Disclaimer: This blog post is based on publicly available academic publications, vendor documentation, open standards, and news items from reputed media sources linked above. This post is intended for educational purposes, to help the enterprise agentic AI community build a shared vocabulary from public, authoritative sources.

Views and opinions expressed here are my own and do not represent those of any employer or client, past or present. The analysis presented is my independent interpretation of the published sources linked above and does not constitute legal, financial, or consulting advice of any kind.