@promptev/context-engine 0.0.0

package/src/skills/context-engine/SKILL.md

@@ -0,0 +1,160 @@
+---
+name: context-engine
+description: Use when writing code against the @promptev/context-engine library — retrieval, ACLs/principals, ingestion, redaction, governed tools, or mounting its HTTP/MCP surfaces. Covers the access-control semantics that cause silent data leaks when guessed.
+---
+
+# context-engine (JavaScript / TypeScript)
+
+Governed retrieval over your own Postgres. Access control is enforced inside
+the SQL predicate of every retrieval leg, before ranking — not applied to
+results afterwards.
+
+Import `@promptev/context-engine`. Full guide: https://promptev.ai/documentation/context-engine/
+
+## The one that leaks data
+
+`principals` is a **trichotomy**, and the dangerous value is the one that
+looks like an absence:
+
+| value | meaning |
+|---|---|
+| `principals=TRUSTED` | Trusted caller — ACL filtering is **DISABLED**. Returns everything. Import it: `import { TRUSTED } from "@promptev/context-engine"`. |
+| `principals=[]` | Anonymous — matches only documents with no ACL. |
+| `principals=["group:hr", ...]` | Matches unrestricted documents plus any whose ACL overlaps the list. |
+| `principals=null` | **DEPRECATED.** Same as `TRUSTED` today, warns, and raises in 1.0. |
+
+**Never pass `null` because a request had no authenticated user.** That is the
+opposite of what it means and it returns the entire corpus. Unauthenticated
+means `[]`.
+
+```ts
+// WRONG — leaks every document to an anonymous caller
+const principals = user ? user.groups : null;
+
+// RIGHT
+const principals = user ? user.groups : [];
+```
+
+The reason `null` is being retired: it is falsy, so it is what every accessor
+degrades to — `user.groups || null`, a missing dict key, `user ? user.groups : null`.
+The unauthenticated path silently became the maximum-privilege path. `TRUSTED`
+cannot be produced by a null-coalesce; reaching it requires typing it.
+
+Use `TRUSTED` for genuinely trusted in-process callers — a backfill script, an
+admin job — never for anything derived from a request.
+
+```ts
+import { TRUSTED } from "@promptev/context-engine";
+
+await engine.search(q, { principals: TRUSTED });   // explicit: skip ACL
+await engine.search(q, { principals: [] });        // anonymous
+await engine.search(q, { principals: user.groups }); // scoped
+```
+
+## Rules that prevent the other silent failures
+
+- **Never read `principals` from a request body or query string.** On remote
+  surfaces they are injection-only: derive them server-side from the
+  authenticated session. A caller who can name their own principals has no
+  access control.
+- **`auth` and `principals` are REQUIRED** on `createHonoRouter`,
+  `createExpressRouter`, `createFastifyPlugin` and `createMcpApp`. There is
+  no default. If you hit a `TypeError`, supply real ones — do **not** silence
+  it with `principals: () => null`, which disables access control on a public
+  port. Anonymous is `() => []`. Trusted is `() => TRUSTED`.
+- **`acl=null` on `updateDocument` UNRESTRICTS the document.** "Leave it
+  alone" is expressed by omitting the argument (the `UNSET` sentinel), because
+  `null` already means unrestricted and cannot also mean absent. Same for
+  `metaData`, which merges rather than replaces.
+- **Requires pgvector 0.8 or newer.** Below that, ACL-filtered vector search
+  silently loses recall — queries return fewer rows than match, or none, with
+  no error. The library cannot compensate.
+- **`config.enableCodeExecution` is `false` on purpose.** `engine.compute()`
+  executes generated code. Turn it on only deliberately.
+- Filing a document under an ACL the caller does not hold returns **403**, on
+  ingest and on PATCH alike.
+
+## Search
+
+```ts
+const result = await engine.search("how much annual leave do I get", {
+  principals: ["group:hr"],     // from the session, never the request body
+  sourceIds: ["hr-handbook"],   // optional scope
+  topK: 8,
+});
+for (const hit of result.hits) {
+  console.log(hit.chunkText, hit.meta);
+}
+```
+
+Hybrid full-text + trigram + vector, fused with RRF. `mode: "graph"` switches
+to the graph leg when the graph subsystem is configured.
+
+## Ingest
+
+```ts
+const report = await engine.ingest({
+  content: fs.readFileSync("handbook.pdf"),
+  filename: "handbook.pdf",
+  sourceId: "hr-handbook",
+  acl: ["group:hr"],            // omit for an unrestricted document
+});
+```
+
+Accepts `file`, `content` + `filename`, or `text`. Re-ingesting identical
+content is deduplicated by hash; ACL changes on re-ingest are applied.
+
+## Mounting the HTTP surface
+
+Framework adapters live on **subpath exports**, not the main package entry
+(so `import "@promptev/context-engine"` never pulls Hono/Express/Fastify):
+
+```ts
+import { createHonoRouter } from "@promptev/context-engine/hono";
+import { TRUSTED } from "@promptev/context-engine";
+
+app.route(
+  "/context",
+  createHonoRouter(engine, {
+    auth: async (c) => { /* raise / return 401 on anonymous */ },
+    principals: (c) => c.get("user")?.groups ?? [],
+  }),
+);
+```
+
+`createExpressRouter` (`@promptev/context-engine/express`),
+`createFastifyPlugin` (`@promptev/context-engine/fastify`) and
+`createMcpApp` (`@promptev/context-engine/mcp`) take the same two
+arguments. MCP has no request object: `principals` is a zero-argument
+callable resolved fresh on every tool call. The CLI `context-engine mcp`
+binds `127.0.0.1` and answers as anonymous (`principals=[]`).
+
+Not on a supported framework? Call `engine.search()` / `engine.ingest()`
+from your own handler — the routers are thin wrappers over the same core.
+
+## Lifecycle
+
+Construction is lazy; close the engine to tear down only what was built. A
+process-lifetime singleton can skip it, but anything that builds engines
+repeatedly (worker per job, test per case) must close them or it accumulates
+connection pools.
+
+## Redaction
+
+Rules with `applyAt: "ingest"` run **before chunks are embedded**, so masked
+text is what reaches the embedding provider. `applyAt: "output"` masks on the
+way out and leaves stored text intact. Use `unless: [...]` to let named
+principals see through a rule.
+
+Presidio NER is an HTTP adapter (`@promptev/context-engine/presidio`): set
+`CE_PRESIDIO_URL` or `PRESIDIO_URL` to a Presidio Analyzer endpoint.
+
+## Don't
+
+- Don't add a `WHERE` clause of your own on top of a retrieval call to filter
+  by tenant — pass `principals`/`sourceIds` and let it go into the indexed
+  predicate. Filtering after the fact is what loses recall.
+- Don't cache a `SearchResult` across users. Results are ACL-scoped to the
+  caller who requested them.
+- Don't expect `engine.search()` to enforce anything about *writes*; ingest
+  and PATCH authorisation is separate and already handled.