"Completed" did not mean it worked

A scheduled agent run asked for permission to archive a record. Nobody was there. The turn stopped. The task reported status: "completed", run_count: 1, last_error: null — which is byte-for-byte what a run that did the work reports.

That is the kind of bug I find genuinely embarrassing, because it is not a crash and it is not a race condition. It is a status field that answered a question nobody asked. "Did the turn run?" Yes. "Did the work get done?" Different question, and for a year the API only had a word for the first one.

Where unattended work actually breaks

The demo of a scheduled agent is easy. cron: "0 9 * * MON", a prompt, a callback URL. It fires. It works. You ship it.

Production is where you learn that a turn is not a function call. A turn can pause. And everything an agent can pause on — a question for the user, a permission request, a client-side tool that needs the browser to answer — assumes there is a browser, and at 9am on a Monday inside a cron job there is not.

So a task told to "summarise this week's filings and archive the ones we've already reported" runs, summarises, reaches the archive step, calls the approval tool, and waits. Forever, or until the run times out. Nothing broke. The model behaved correctly. The work did not happen.

We split it into three outcomes instead of two:

last_finish_reason uses the same vocabulary the turn.stopped webhook does: needs_input, max_tool_iterations, no_answer, guardrail, subagents_incomplete. One word your code can branch on, and it is set on recurring tasks too — where the status stays scheduled and this is the only record of how the last fire went.

That last part matters more than the one-shot case. A recurring task in this state does not fail loudly once. It stalls again on every fire, quietly, weekly, and the only symptom is a digest email that stopped arriving three weeks ago and nobody has mentioned yet.

Which tools an unattended run may reach

Having split the status, we had to decide what to do about the cause.

ask_user is withheld from unattended runs. There is nobody to ask; a tool whose only outcome is a hang is worse than no tool, and the agent handles its absence fine — it writes down what it could not determine and finishes.

Approvals are not withheld, and that was the harder call. Continuing without a permission is the one thing worse than stopping. If your prompt tells a scheduled task to delete records and the design of the system quietly grants that permission because nobody is around to withhold it, then "requires approval" means "requires approval when convenient", which is not a security property.

So the rule is: an unattended run that needs a person is a run that will not finish, and the fix belongs in the prompt. Design the scheduled work so it does not need a permission — summarise and flag rather than summarise and act — or handle the pause out-of-band, from the turn.stopped delivery, where an actual human can answer it.

You do not have to go looking for these. GET /readiness carries a tasks-waiting step whenever any task's last run ended needs_input, naming the task, when it stopped, and the session holding the question it asked. It reads off that one finish reason rather than off a list of things that can pause, so an approval, a client tool, a question and a browser hand-off all raise it. And it is deliberately not marked essential — the project answers questions perfectly well, it is the scheduled work that stopped, and failing your deploy gate over that would be wrong.

Then: the same event arriving twice

Scheduling answers "every Monday". The other half of unattended work is "when a ticket arrives", which is a webhook trigger — a URL you hand to another system, where every POST starts a turn.

Two things about that surface took us longer to get right than the scheduler did.

It answers immediately. A webhook sender retries a slow response and gives up on a long one. An agent turn is tens of seconds. So the run is started durably and you get {"accepted": true} — never the agent's reply. The reply lands in the session and fires your turn-stopped webhook like any other unattended run.

A retry is not a second run. Runs are keyed per event, so the same delivery arriving twice starts the agent once, and you get duplicate: true in the response. Still a 200, because a sender retrying an event we already took wants to hear that it landed — a 4xx would only make it retry harder.

The event is identified by your X-Event-Id or Idempotency-Key header if you send one, and otherwise by a hash of the body. Which brings me to the thing I would put on a poster if I could:

If your events do not identify themselves, identify them.

Two failures live here and they point in opposite directions. The first: many senders re-serialise their JSON, and a re-ordered object is a different body even though it is the same event — so one event arrives as two. Annoying, visible, fixable.

The second is the expensive one. A bare {"event":"ping"}. An empty payload. A "something changed, come and look" webhook with no detail. Two people submitting the identical form. To a body hash those are one event, and deduplication here lasts 24 hours — so a heartbeat with a constant body runs once a day and its sender is cheerfully told, every time, that it landed. accepted: true, duplicate: true. Nothing wrong anywhere. Nothing running either.

None of those payloads carry a natural id, which is exactly why nobody sending them thinks to add a header. A counter, a timestamp, a uuid in X-Event-Id is enough.

Refusals you should not retry

One more, because it wastes real time. A cron, timezone or interval_seconds the scheduler will not take is answered 400, with its own diagnosis:

Invalid schedule spec: Minute is not in range [0-59]
Invalid schedule spec: CronString does not have 5-7 fields
Invalid schedule spec: unknown time zone Mars/Olympus
Invalid schedule spec: interval is too small

Do not retry any of those. No number of attempts makes 99 3 * * * a valid cron. A 5xx from the same endpoint means the scheduler itself was unreachable, which is worth retrying, and telling those two apart is the entire reason the status code is different.

Cron accepts five to seven fields, so second- and year-precision expressions work. timezone is an IANA name and blank means UTC. Once a recurring task is accepted, next_run_at tells you when it actually fires next — and it is never null on a task reporting scheduled, so that field is your answer to "is this schedule alive".

The general version

Every one of these is the same mistake wearing a different hat: a system reporting on itself in a vocabulary that cannot express the interesting failure. completed for a run that stopped. 200 for an event that started nothing. ready for a document with no chunks in it.

The fix is never cleverness. It is one more word in the enum, and then being willing to go back and change a field a customer is already reading.

Oberik is free while we launch and you bring your own provider keys. oberik.com