The cancelled request that got carried out a minute later

A user typed "close every berth for July", thought better of it, hit stop. A minute later they asked what time it was. They got both.

Nothing was broken. Every component did what it was designed to do, and the composition of those correct behaviours produced an action the user had explicitly stopped.

Here is the chain. Cancel stops generation — that part worked. The user's message stays in the conversation, because the user typed it and a transcript that silently drops what someone said is worse than useless for support, audit or the user's own memory. Then the next turn replays the transcript. And a user message that nothing has answered reads to a model, entirely reasonably, as work still outstanding.

So the model picked it up. It was not confused. It was doing the obvious thing with the evidence it had.

What cancel actually has to guarantee

It took us a while to write down the guarantee properly, because the first version was too weak: "cancel stops generation." That is a statement about compute. What a user means by stop is a statement about consequences.

The guarantee now has two halves:

1. Generation stops.
2. The request it stopped is not carried out later.

The second half is the one that needed building. The message stays visible in sessions/{id}/messages — the user typed it — but it is marked unanswered at the start of the next turn with extra.turn_abandoned: true, and the model never sees it.

The same rule covers every other way a turn can end without an answer: the provider hung up, the server was replaced mid-turn, a stream failed terminally. Nothing was answered, so nothing is resumed. Send it again if you still want it.

That is a deliberately conservative default. The alternative — replaying anything unanswered — is right about 95% of the time and catastrophic the other 5%, and the 5% is concentrated exactly on the messages a user cancelled, which are disproportionately the ones they cancelled because the action was consequential.

The thing cancel cannot do

cancel() cannot stop a blocking chat.send().

There is no resumable run to cancel, and closing the HTTP request does not stop generation either — the work continues server-side, spending money, with nobody to hand it to. So /chat/stream/{run_id}/cancel answers 409 rather than reporting a turn stopped that is in fact still running and still billing.

Which gives a rule that is easy to state and worth following: use chat.stream() for anything a user may want to stop. Even if you only ever read the final done. A stream has a run id, can be re-attached, can be cancelled, and can be resumed after a dropped connection. A blocking call has none of those, and a blocking call that times out leaves the work done and paid for with no way to collect the answer.

While we are on things a turn can do to itself

One conversation runs one turn at a time. Starting a second while the first is generating answers 409, naming the run in flight:

409 a turn is already running on this conversation (run 9f3c…). Starting a second one
    would interleave the two in the transcript, and nothing stored says which answer
    belongs to which question.

That refusal is almost always what the second caller actually wanted. A retry, a second tab, a phone resuming while the desktop submits — attach to the run and you get the answer already being written.

Starting a genuine second turn is the bad outcome, and it is bad permanently rather than momentarily. The stored conversation stops alternating (user, user, assistant, assistant), nothing pairs an answer with its question, and every later turn replays a transcript some providers refuse outright. One racing request, and the conversation is broken for good.

The mirror-image bug is worth naming too, because we had it: a turn that failed at the provider — a rotated key, a rate limit — used to report the failure and then leave its conversation refusing every later turn, while the endpoint that reports what is running said that same conversation was idle. Two doors, opposite answers, and a conversation nobody could use again.

A turn that ended does not wedge its conversation, however it ended. If /run says idle, posting will not be refused. Those two answers come from the same place now, which is the only way to keep them agreeing.

The general shape

Three bugs in this post, and they are the same bug: a piece of state that two parts of the system described differently.

A cancelled message that the transcript called "said" and the model called "outstanding". A turn that the error path called "failed" and the run endpoint called "idle". A run that expired and answered 404, in a system where 404 also means "you may not see this".

The fix is never a smarter component. It is deciding which one owns the fact, and making everything else ask it.