Documents

Upload a file and it becomes searchable for the users you allow. Parsing is layout-aware, so tables and headings survive, and each PDF page is checked for a text layer — so only the pages that actually need OCR are sent anywhere.

Upload

const doc = await ai.documents.uploadAndWait(file, {
  filename: "q3-report.pdf",
  contentType: "application/pdf",
  tags: ["finance", "q3"],
  visibility: "shared",              // set the ACL here, not afterwards
  visibility_scope: "acme:finance",
});

doc.status;       // "ready" — searchable now ("failed" carries `error`)
doc.chunk_count;
doc.visibility;   // reads back what you set

Set visibility at upload. A document becomes retrievable the moment ingestion finishes, so patching it afterwards leaves a window where more people could read it than you intended.

Ingestion is asynchronous, so a just-uploaded document isn't retrievable yet. uploadAndWait polls until it is. The pieces are also available separately:

const doc = await ai.documents.uploadSimple(file, { filename, tags });   // small files
await ai.documents.waitReady(doc.id, { timeoutMs: 120_000 });

For large files use the resumable presigned upload, which goes straight to storage — the bytes never transit your backend, and an interrupted upload resumes per part:

await ai.documents.upload(file, {
  filename: "10gb-export.csv",
  visibility: "private",
  onProgress: (sent, total) => setPct(sent / total),
});

A corpus you curate, that users only read

The most common shape: you upload the documents, your end-users ask questions of them and never upload anything themselves. Three settings:

1. Capabilities → Document access → Read only. Minted tokens then never carry documents:write, so a user cannot upload however your backend mints them — it's a project-level guarantee, not something every mint call has to remember.
2. Upload with your project key, which stores them tenant-visible by default — readable by every end-user of the project. Pass visibility to narrow that.
3. Restrict the tools (allowed_tools: ["rag_search"], and don't grant web_search) if answers must come from your corpus and nowhere else.

// your backend, with the project key — a different client from the one your app uses
import { createProjectClient } from "@oberik/sdk";

const oberik = createProjectClient({ projectId, projectKey: process.env.OBERIK_PROJECT_KEY! });
await oberik.documents.upload(file, { filename, tags: ["handbook"] });          // tenant-wide

await oberik.documents.upload(salaries, {                                       // narrower
  filename: "salary-bands.xlsx",
  visibility: "groups",
  acl_roles: ["finance"],
});

visibility takes the same values here as it does on an end-user upload, and the table below is the whole vocabulary. A project key authenticates as the subject service, so private means that principal plus admins and self means that principal alone — both are coherent, and both are narrower than tenant.

What a project key cannot do is upload as one of your users: the owner recorded is service, not them. A document that belongs to a particular end-user goes through the data-plane client with that user's token (ai.documents.uploadAndWait), where the subject is the owner.

Note this interacts with memory: a project-wide corpus is the one case where a shared agent wiki is safe.

Visibility

Who may retrieve a document is set per document and enforced on every search.

visibility	Retrievable by
self	only the exact uploader — not even admins
private	the owner, admins, and tokens scoped above the owner
shared	anyone under visibility_scope (e.g. a team path)
groups	anyone whose token carries a matching acl_group
tenant	everyone in the project

acl_roles / acl_groups are matched against the token's roles / groups.

self is also unlistable, so an operator's totals exclude it

The row above is about retrieval, and the same rule applies to listing: a self document does not appear in documents.list for anybody but its uploader, and documents.get answers 404. So a project key's own count of the corpus is short by however many there are.

That number is available without weakening the guarantee. with_total answers hidden alongside total for an admin or a project key — how many documents exist that the caller may not list:

const page = await project.documents.list({ with_total: true });
page.total;   // what you can see
page.hidden;  // how many you cannot, and nothing else about them

A count is not a read: it names no filename, no owner and no text. It is null for an end-user token, because to a reader the same number would say how many private documents other people have.

await ai.documents.update(doc.id, {
  visibility: "shared",
  visibility_scope: "acme:finance",
  tags: ["finance", "q3", "reviewed"],
});

Retrieval applies the same rules, so a user cannot get an answer grounded in a document they aren't allowed to read — the model never sees it in the first place.

How a file is read

You don't choose this per file, and you shouldn't have to. Each file is read the way that file can be read — and for a PDF, that decision is made per page:

File	How it is read
.pdf page with a text layer	Extracted locally, about 150ms, no cost. Reports text-layer.
.pdf page without one	Sent to your OCR engine, one page at a time.
A mixed .pdf	Both. A 200-page report with two scanned exhibits costs two pages of OCR, not 200.
.png, .jpg, .tiff, …	Straight to OCR. There is no text layer in a bitmap to look for.
.docx, .pptx, .xlsx, .odt, …	Converted to PDF to ask the same question. If no page needs OCR the original is parsed natively, which keeps structure a round trip through PDF would flatten. If a page does, the converted PDF goes through the ladder above — which is what makes a .docx of photographed pages readable.
.txt, .md, .csv, .html, …	Parsed directly. There is no OCR question for a text file.

Who does the OCR

One setting, under LLM & limits → Document processing:

// The PROJECT client — this is your setting, not your end-user's.
await oberik.documentProcessor.set({ ocr: "auto" });                        // the default
await oberik.documentProcessor.set({ ocr: "model", ocrModel: "gpt-4o" });   // a specific one
await oberik.documentProcessor.set({ ocr: "local" });                       // never leaves us

• auto uses your project's default model when it can be handed a page, and local OCR when it cannot. Register a vision model and good OCR is the default.
• model pins a specific vision/OCR model, called with your own key. Saving a model whose registered inputs don't include an image or a file is refused here rather than failing on every later upload.
• local never sends a page to a model. Lower quality on hard scans, and a guarantee.

Local OCR reports itself as local-ocr in read_with — so a project on auto can tell at a glance whether a document went to a model or was read on our machines.

ocrComplexPages additionally sends pages that do have a text layer but a flattening layout — tables, multi-column — to the model, at one call per such page. Off by default, and ignored unless the resolved engine is a model: local OCR reads a table worse than local extraction does.

What it tells you afterwards

Every document reports which engine actually read it, which is the answer to both "why did this come back empty" and "why did ingesting these cost what it did":

const doc = await ai.documents.get(id);
doc.read_with;       // ["text-layer", "gpt-4o"] — local for most pages, the model for scans
doc.pdf_type;        // "text_based" | "scanned" | "image_based" | "mixed" | null
doc.ocr_page_count;  // 2 — what OCR was billed for

read_with names what happened, not what we run:

text-layer	the PDF's own text, extracted here. Free.
document-parser	a non-PDF format parsed natively. Free.
local-ocr	OCR on our machines. No model call, no cost.
anything else	a model name — the one you registered, and the line on your bill.

On auto this is the only place you can see whether your scans went to a model or to local OCR. A file nothing could read comes back failed with a reason, never ready with chunk_count: 0.

A failed document reports these three as well, and they mean the same thing there: a page sent to OCR that came back empty is still a page you were billed for. They are read off the same record the error sentence is written from, so the two always agree.

Every field on a document

documents.get and each row of documents.list carry all of these. They are listed together because a field that is on the wire and named nowhere can only be found by printing an object and then guessed at — which is not a way to learn an API.

field	what it is
id	the document. Stable across update and reingest; a re-upload is a different document.
filename	the name it is stored under. A label, not a key — see the duplicate note below.
content_type	what the upload declared, or what was sniffed from the bytes. Null when neither could say.
tags	how a curated corpus is scoped: retrieve({tags}) and chat({tags}) read this.
owner_subject	the end-user who uploaded it. A project-key upload is owned by the project.
visibility · visibility_scope · acl_roles · acl_groups	who may retrieve it — see Visibility.
status	pending → processing → ready | failed. Only ready is searchable.
chunk_count	how many pieces are in the index. Never 0 on a ready document.
error	the sentence to show a person, when it failed.
failure	the same failure as one word for your code to branch on — usage_cap, rate_limit, unsupported_type, unreadable, embedding, vector_store, storage, unknown.
embedding_model	which model's vectors this document is in. A document embedded by an older model lives in its own collection; this is how you tell which ones a reingest still owes.
pipeline_version	the reader and embedder that produced the current vectors, as one string. Same use: it tells you what is stale after you change the project's settings.
content_hash	SHA-256 of the bytes as uploaded. This is what you check to find a duplicate before uploading one, rather than reading duplicate_of afterwards. Null until ingestion has read the file.
duplicate_of	the oldest document with the same content_hash, or null — see the note below.
read_with · pdf_type · ocr_page_count	how it was read and what OCR cost — the table above.
created_at	when the row was made, which is when the upload was accepted rather than when it finished indexing.

Retrieval

Usually you don't call retrieval: chat does it and returns citations. Call it directly when you want the chunks themselves — to build your own UI, or feed another system:

const chunks = await ai.documents.retrieve({
  query: "refund window",
  tags: ["policies"],
  top_n: 5,
});
// [{ text, score, document_id, chunk_index, metadata: { filename, page, … } }]

top_k is how many candidates are fetched; top_n is how many survive reranking — and when no rerank model is configured, how many of the vector hits come back.

Reranking is a second, more expensive pass that reorders what recall found, and it only happens when the project has a rerank model set (LLM & limits → Rerank model, or retrieval set --rerank-model). Setting one now checks it can actually rerank and refuses it if it cannot, because a rerank that fails degrades to vector order on purpose — losing the whole query because a cross-encoder is down would be worse — which means a model that can never work would otherwise look exactly like one that works.

Note that the model has to be registered on one of the project's providers, not just named here, and it has to be reachable through a provider that can rerank: a model on an OpenRouter key cannot, whatever the model is.

And it is re-checked afterwards. A credential that worked when the model was set can stop working later, and the same silence applies — so GET /readiness carries a rerank-model step that asks the model to rerank two documents and reports what it said. It is not essential: a dead reranker costs a project the precision half of retrieval and nothing else, so canAnswer stays true and a deploy gated on it still passes. ready goes false, and the step names both the reason and how to stop the setting claiming something that is not happening.

Managing documents

await ai.documents.list({ tag: "finance", status_filter: "ready" });
await ai.documents.get(id);
await ai.documents.chunks(id);        // exactly what was indexed
await ai.documents.reingest(id);      // re-parse / re-embed
await ai.documents.downloadUrl(id);   // signed URL for the original file
await ai.documents.download(id);      // resumable ranged download -> Blob
await ai.documents.delete(id);        // removes the file and its vectors

chunks is the tool to reach for when an answer looks wrong: it shows what the model actually had to work with. reingest is what you call after changing the project's document processor or embedding model.

Deleting from the two clients is deliberately not the same

The end-user client above deletes without being asked. The project client does not:

await ai.documents.delete(id);                          // end-user token — goes
await project.documents.delete(id, { confirm: true });  // project key — required

That is not an inconsistency to be tidied up. An end-user removing their own upload is an ordinary action, and the place to confirm it is your product's UI, where the person is; a project key is an operator credential reaching across every end-user in the workspace, usually from a script that nobody is watching, and the confirmation is the only thing standing between a stray loop and a corpus (OBE-270).

So the gate is on the operator route, not on the document. confirm is a required argument rather than an optional one, which means a script that has not thought about it fails to compile instead of failing at run time against a corpus it was halfway through.

Uploading the same file twice

Nothing is refused. A filename is a label — documents have ids — and re-uploading a revised file under its stable name is a normal, useful thing to do, so a duplicate name is a 201 like any other upload.

What you get instead is duplicate_of on the new document: the id of the oldest document in the project carrying the same content hash, or null. It fires on identical bytes whatever they are called, including a re-upload under the identical name, and both copies stay retrievable — which is worth knowing, because retrieval will then return the same passage twice and that quietly doubles its weight in an answer.

const doc = await ai.documents.get(id);
if (doc.duplicate_of) {
  // already in the corpus under `doc.duplicate_of` — delete one of them
}

Read it off get or list, not off the upload response. It is resolved when you ask, against the corpus as it stands, so an upload that has not finished indexing yet has no hash to match on and reports null. That is also why a whole folder uploaded at once resolves correctly: every copy links as soon as the rows have hashes, with nothing to sweep and nothing to backfill. Delete the original later and the copies stop pointing at it.

The link is always to something strictly older, so two uploads racing cannot come back naming each other.

list returns a page, the same envelope chat.sessions.list does — a corpus is the one collection that only grows, and a bare array asserts completeness by omission:

let offset: number | null = 0;
const all = [];
while (offset !== null) {
  const page = await ai.documents.list({ limit: 200, offset });
  all.push(...page.items);
  offset = page.next_offset;   // null on the last page
}

limit defaults to 100 and is capped at 500; a value outside that is a 422 naming the bound rather than a 200 that quietly means something else. Page with next_offset rather than adding your own limit to your own offset — the cap can make that arithmetic skip rows. has_more is free (the server reads one spare row); with_total: true adds a total and costs a second query, so it is for a summary rather than for paging.

Retagging a document you uploaded

Tags scope a curated corpus — retrieve({tags}) and chat({tags}) both key on them — so getting one wrong is ordinary, and fixing it should not cost the document's identity:

await oberik.documents.update(id, { tags: ["policies", "2026"] });   // project client
await oberik.documents.reingest(id);

tags replaces the list; fields you do not pass are left alone. The id does not change, which is the point: deleting and re-uploading gives you a new id and silently breaks every citation, stored document_ids scope and audit record that pointed at the old one.

This is on the project client, not the end-user one. An end-user token is refused a document it does not own, deliberately — your users must not be able to rewrite the corpus you curate. Uploads made with a project key are recorded against the subject service, which is why the project client is the surface that can edit them.

An upload needs an embedding model. Without one there is nothing to index with, so the upload is refused with a 409 naming the setting rather than accepted and failed minutes later — GET /readiness lists it as embedding-model, and registering an embedding model with your provider sets it (and measures its dimension) for you. A project on a deployment whose shared embedder works is not affected, and one on a deployment that supplies no models at all is told exactly that — "this deployment provides no default embedding model" — rather than being handed a credential error about somebody else's key.

And so does searching. rag_search is not bound at all without one, so the agent is told it cannot search this project's documents and says so — rather than being handed the embedder's own error mid-turn, which reads as retryable and had it trying the same call three times. It is the same fact as the paragraph above and the same fact as memory: one setting, and everything that turns text into vectors waits on it. Nothing is silently degraded — a project in that state answers from what it is given in the conversation, and tells the user why it cannot look further.