Mothership
Knowledge
Apps' documents are extracted and indexed privately, searched within the documents chosen, and answered with citations. Source permissions are checked again on every read.
A document's life
- Upload and queue — the app uploads a file and queues ingestion (queued by default since SDK 1.1). The gateway records the job and answers at once.
- Extract and index — the Knowledge worker extracts text (PDF, Office, OCR'd images, text) and indexes it, with vectors when an embedding model is set.
- Search and cite — the app searches chosen collections and documents; results carry citable excerpts.
- Delete — deleting a document removes its searchable content and source file together.
ts
const file = await ctx.files.upload(selectedFile);
const queued = await ctx.knowledge.ingest({
collectionId: "review-documents", fileId: file.fileId,
title: file.name, scope: "user_private",
}, { idempotencyKey: `ingest:${file.fileId}` });
await ctx.jobs.wait(queued.jobId);
const results = await ctx.knowledge.search({
collections: ["review-documents"], documentIds: [queued.documentId],
query: "What requires approval?", topK: 6,
});


Sharing
| Scope | Who can see it |
|---|---|
user_private (default) |
The uploader |
space_shared |
Members of the team space the host selected (never an ID the app sends) |
org_shared |
Members of the same company, app and environment |
The source file stays private even when its extracted text is shared. Every list, search, citation and download checks the collection declaration and current membership again.
How it searches, and its limits
- PostgreSQL: a GIN full-text index shortlists candidates; with embeddings configured, a bounded set is re-ranked by meaning. No unbounded vector scan, no ANN vector database.
- SQLite: a bounded lexical search, for development.
- Results report the mode used and whether they were truncated. After changing the embedding provider or model, re-index before old vectors join semantic ranking.
- Documents are processed in a separate process with CPU, memory, page, pixel, output-size and time limits: by default 25 MiB in, 25 pages, 120 seconds. Decrypt PDFs before uploading.
Workers
make dev (development) and the production Compose workers service run the Knowledge worker. On their own:
bash
.venv/bin/python backend/manage.py knowledge_worker
.venv/bin/python backend/manage.py document_worker # PDF and OCR conversion
document search · RAG · retrieval · embeddings · semantic search