@promptev/context-engine 0.0.2 → 0.0.4

package/README.md

@@ -180,6 +180,87 @@ const router = createHonoRouter(engine, { auth: myAuth, principals: myPrincipals
 
 `auth` and `principals` are required. Identical routes exist for Express and Fastify.
 
+## The knowledge tool
+
+One tool with actions:
+
+| Action | What it is for |
+|---|---|
+| `discover` | what documents there are and what is inside each one — sheets and columns, sections and pages, or JSON keys — plus the extracted fields by document type, a census of the corpus, and what each action can do. Call it first |
+| `search` | passages by meaning or keywords |
+| `get_doc` / `get_docs` | one whole document by id, or several at once |
+| `get_chunks` | walk one long document in order, a piece at a time |
+| `list` | browse the documents without searching |
+| `query_meta` | filter documents by their structured fields |
+| `compute` | a figure derived from the spreadsheets, over every row |
+| `map_reduce` | the same question asked of every document in scope |
+| `get_neighbors` / `traverse` | what is one step, or a few steps, from a named thing |
+| `find_related` | connections of a kind across the corpus |
+| `community_summary` | the themes the corpus groups into |
+
+One description carries the decision rules once and there is one name for a
+model to route through. An action this deployment cannot service is advertised
+as off rather than hidden, and called anyway it answers `{ success: false,
+error }` the model can relay. A truncated answer carries the `next_page` call
+that reads the rest.
+
+`discover` answers with the SCHEMA, not an inventory, so one call is enough to
+write one correct `compute` or `query_meta` call instead of three exploratory
+ones. Every listed document carries `document_type`, `mode`, and a `structure`
+whose shape follows the type — `sheets` (name, column headers, row count) for
+a workbook, `sections` and `last_page` for a document with headings, `keys`
+for JSON, and `chunks` as the floor for anything else. Beside them,
+`fields_by_type` lists the extracted structured field names with each field's
+data type and how many in-scope documents carry it, and `document_types` is a
+census of the corpus. Both are keyed by `(kind, type)`: `kind` comes from the
+mime and is always known, while `document_type` is free text an LLM wrote and
+exists only where `extractStructured` was enabled — so a PDF invoice and a
+spreadsheet of invoice rows stay separate rather than merging under one label.
+
+Lists inside a `structure` are capped, with the remainder reported as
+`more_columns` / `more_sections` / `more_keys` / `more_sheets`, because
+`discover` is the first call of a conversation and an unbounded list lands in
+the model's context before it has asked anything. **The cap is for the call
+that did not name its documents:** `discover` and `list` take `document_ids`,
+and a `discover` scoped to specific documents answers their structure whole
+and narrows the field grouping and the census to them too. While the remainder
+is small enough to be worth fetching, the truncated structure carries that
+call already filled in; past that it says what to do instead, because a sheet
+with hundreds of columns is one to compute over rather than read back.
+
+Omit `mode` on a search and it is worked out from the documents in scope — the
+graph leg is added when the deployment has a graph and something in scope was
+ingested that way, and it runs beside the other legs rather than ahead of
+them. Naming `"hybrid"` or `"graph"` forces one.
+
+```ts
+import { UNSCOPED } from "@promptev/context-engine";
+
+const answer = await engine.searchKnowledgeBase({
+  action: "search",
+  query: "annual leave",
+  principals: caller,
+  scope: ["hr"],
+});
+```
+
+`createMcpApp` serves that same function, so an application driving its own
+agent loop needs no MCP and cannot get a different answer.
+`knowledgeToolDefinition()` exports the tool as data — name, description and
+input schema as JSON Schema — so a host never hand-writes it. `createMcpApp`
+also takes `redaction` (applied per call), and `compute` / `mapReduce`
+callables that REPLACE the built-in ones, which is where a host applies its own
+rules for permission, billing and approval.
+
+`scope` is REQUIRED on both: the ceiling of source ids (or a `Scope` with
+document ids) the tool may ever reach. It is host-supplied and resolved per
+call like `principals`, and never a tool argument — a model can name any
+source id, and a tool that believed it would let one caller read another's
+documents. Map your own word onto it: a project, a matter, a customer. A
+caller narrows within the ceiling and never past it; an id outside is dropped
+silently; a request left with nothing in scope returns nothing. A host with
+one shared corpus writes `scope: UNSCOPED` on purpose.
+
 ## Tools
 
 Register a governed tool — HTTP, SQL, MCP, or a plain function — and every call