The conversation that breaks permanently

Replaying the whole transcript works until it doesn't. Then every later turn fails with "context length exceeded" — and it keeps failing, because the message that tipped it over is still in the history.

That permanence is what makes this different from an ordinary error. A rate limit clears. A provider outage ends. A conversation that has outgrown the model's window is broken in a way that gets worse with every attempt, and the user's options are to start again and lose everything, or to keep pressing send.

It is also the failure people ship without ever seeing, because it does not happen in testing. Your test conversations are six turns long.

Three strategies, one of which is a decision you should make on purpose

Configured per project, not per request:

Under trim, what goes first is configurable, and the default is worth understanding: tool_output_first guts old tool outputs before dropping whole messages. A command's output can be produced again. The reasoning around it cannot.

Four things that always hold

These matter more than the strategy choice, because they are what make the whole thing safe to leave on.

Below the budget, nothing changes. Every strategy replays the transcript unaltered. This only ever activates on a conversation that would otherwise break.

The task survives. The first user message and the most recent turns are never evicted.

The transcript stays valid. Eviction is followed by a repair pass, so trimming can never leave the agent with tool calls that nothing answered — which would fail every later turn on that session, permanently, which is the bug this whole system exists to prevent.

Nothing is deleted. Summarized messages keep their content. GET /chat/sessions/{id}/messages still returns the real conversation. What was compacted is what the model sees, not what happened.

One caution on that second point, because it defeats the obvious test: a marker planted in turn 1 is exactly what survives while the bulk around it is truncated. Reading it back does not prove the history was kept. Use tokens_after and budget as the evidence.

Two fields, because there are two questions

strategy is what the project asked for. resolved_by is what actually fitted the window. They are usually the same, and the interesting case is when they are not:

{ "phase": "done", "strategy": "summarize", "resolved_by": "trim",
  "fallback": "summarizer_returned_nothing", "compacted": 0, "dropped": 3 }

That is a summarize policy that could not run, so the window was fitted by trimming instead. The turn still worked. It worked on a cut view of a conversation you had asked to have summarized, which is worth knowing and used to be invisible.

fallback names which way it gave up: summarizer_timed_out, summarizer_returned_nothing, nothing_to_summarize, nothing_evictable. The first two are worth telling apart — one points at your infrastructure, the other at the model's output.

The number that explains a slow first token

summarizer_timed_out names the failure and not the reason. From outside, a ceiling too tight, a model too slow, and a summarizer that never finishes look identical. So the report carries all three numbers:

{ "summarizer_timeout_s": 45, "summarizer_ms": 45002, "summary_model": "hukuk-db" }

summary_model matters more than it looks. Unless you set one, summarizing a 96k-token transcript is done by your chat model — and a reasoning model that spends its output budget thinking before it emits anything is close to a worst case for this job.

"Raise the ceiling" and "point summaryModel at something fast" are different fixes, and those fields are what tell you which you need. They are reported on a successful compaction too, which is the useful part: summarizer_ms close to summarizer_timeout_s is the warning that the next slightly longer conversation will fall back to a trim.

Summarizing runs before the turn's first token, so every second of it is a second of blank screen. Streaming clients get phase: "compacting" ahead of the model call — render "compacting conversation…" rather than nothing.

And register onContext when you open the stream, not after the turn starts. Fitting happens before the model is called, so the context frame is the first thing a turn emits — earlier than the frame carrying the session id.

Three counts, because "we dropped three messages" is not "we summarised the earlier part"

They were one number once. They are not the same sentence to show a user, and they are not the same thing to debug.

Images cost about 1600 tokens each

Which means a multimodal conversation reaches the budget far sooner than its text suggests. Media is priced accordingly rather than as the few characters a placeholder occupies.

Under summarize, the media in the evicted span is attached to the summarization call itself — it is the last thing that will ever look at those images, so the model is asked to write down what each one actually shows before it goes. Under trim media is dropped, and a marker is left either way so the agent knows a picture was there.

Something the turn produced itself

Fitting happens before the turn runs. But a turn can produce something too big for the window while it runs: a sandbox command that printed a whole build log, an MCP tool that returned a repository.

Those are fitted too, at the step boundary before the model reads them, and reported with phase: "step". The tool output carries a marker saying it was shortened, so the agent answers knowing its view was cut rather than confidently from half a file.

A "step" report never summarizes, whatever the project's strategy. A model round trip per step of a tool loop is not a trade worth making.

Until this existed, the two halves disagreed in a way nobody could have predicted from the settings. A client tool's oversized result comes back on a new request, so it was fitted and the turn survived. An MCP or sandbox tool's ran inside the turn, so it went to the provider whole and the turn died with the provider's own context error. Same size, same project, same policy, opposite outcomes.

If you turn on one thing

Archive to memory. Whatever leaves the window is embedded into the project's vector store, and the agent gets a search_history tool scoped to that one conversation — so it can go back for a detail instead of asking the user to repeat something they already said.

That is the difference between a long conversation degrading and a long conversation forgetting.