The blocked topic that caught the most on-domain question in the corpus
A bookkeeping assistant. Blocked topic: "personal tax advice", which is a perfectly reasonable thing not to want your product doing. It caught "What is the mileage rate for 2026?" — a company reimbursement rate, and about the most on-domain question that corpus could be asked.
Nothing malfunctioned. A blocked topic is matched by a model, so it generalises, and "personal tax advice" generalises straight through the middle of a bookkeeping product.
Guardrails are off by default, and that is a position
A new project runs none of this. The agent still refuses obvious injection on its own judgement — guardrails are the checks you want enforced rather than left to it.
The difference matters. A model's own judgement is a habit with a good success rate. A guardrail is a check a client cannot switch off, configured per project, applied server-side. You want the second thing for anything you would have to explain afterwards.
await oberik.guardrails.set({
enabled: true,
injection: true, // deterministic, costs nothing
blockedTopics: ["politics", "legal advice"],
groundedness: true, // one model call on the turns it runs
onViolation: "flag", // start here
});Start on flag. I mean it.
onViolation decides what a verdict costs. On flag, a violation is annotated in guard_flags and the answer still goes out. On block, the answer is replaced with a refusal the user cannot get past.
The semantic checks — groundedness, blockedTopics, moderation — are model judgements. They are wrong sometimes. On block, every wrong judgement is a refused answer to a question your corpus could have answered, and the user has no way to tell that anything was blocked, let alone rephrase.
So run on flag against your real corpus first. guard_flags is present on every turn, streamed or not, including the blocked ones — which is what lets you measure your own false-positive rate rather than guessing at it. Watch it for a week on traffic you know is good, then decide.
injection and pii are deterministic and carry none of this risk. They can go straight to block.
The other lesson from the mileage-rate case: name the topic narrowly. "personal income tax filing" refuses much less of your own product than "personal tax advice". And the flag names which of your topics matched, so you can tell which string is catching real questions.
The refusal that is a dead end
By default a refused user sees "I can't help with that request." That is all. They cannot tell anything was blocked, which of their words did it, or whether rephrasing would help.
await oberik.guardrails.set({ explainRefusals: true });
// "I can't help with that request. (asks for personal tax filing advice)"Off by default on purpose, and the tradeoff is real rather than theoretical: the sentence that helps a confused user also helps somebody probing the filter iterate against it. If your product is internal, or your users are your customers' employees, turn it on. If it is a public surface, think about it.
A groundedness block starts from its own sentence — "I don't have enough information in the available data to answer that reliably" — because "I can't help with that request" would be untrue. The agent can help. It just cannot support this particular answer.
Two stages, and only one of them fires per check
Input, before the agent runs: injection heuristics, blocked topics, PII. Output, before the answer is returned: groundedness, moderation, PII.
stage on the guardrail event says which fired, and it is the only thing that does — guard_flags on the turn is a flat union of both.
| when it fires | carries | |
|---|---|---|
stage: "input" | before the first token | flags, and reason on a refusal. No content: what was redacted is the user's own message, not ours to hand back |
stage: "output" | after the answer, before done | flags and the corrected content |
That second row is the one to implement. While streaming, tokens have already reached the user before the output check completes — so the event carries the corrected content and you have to reconcile:
onGuardrail: (stage, flags, content) => {
if (content) replaceAnswer(content);
log(stage, flags);
}Handle it, or a blocked answer stays visible in your UI, which is the exact opposite of what you configured.
PII is the only check with both sides. Injection and blocked topics are input-only; groundedness and moderation are output-only. So an input-only PII policy produces exactly one event, labelled input, before anything is generated — which is the point at which telling the user "we removed a phone number from your message" is still useful to them.
Two shapes, and the 200 that applied nothing
get() returns the policy nested: {enabled, input: {injection, blockedTopics, pii}, output: {…}}. set() takes a flat patch. Note that input.pii reads back and inputPii writes.
Feeding set() the object get() returned used to return 200, echo the unchanged policy, and apply nothing. A project ran a whole round of testing believing a blocked topic was in force while nothing was enforcing it — which is worse than no guardrail, because you stop watching.
It is a 400 now, naming the flat fields to send instead. Anything omitted is left as it was; anything unrecognised is refused rather than dropped.
What the record keeps
guard_flags comes back on the turn, and the same list is stored on the assistant row (extra.guard_flags, with extra.guardrail: "blocked" when the answer was replaced) and written to the audit trail.
So a caller who did not keep the flags at the time can still find out afterwards. That used to be impossible for an output guard: finish_reason: "guardrail" said that one fired and nothing said which — which makes a support question unanswerable a week later, when it is asked.
Small detail, same principle as everywhere: if a decision affects what a user saw, the record has to say which decision it was.