@promptev/context-engine 0.0.0 → 0.0.2

package/README.md

@@ -9,9 +9,15 @@
   as a TypeScript library, on your own Postgres.</b>
 </p>
 
-> TypeScript client in this repo (Python lives at the root). Same Postgres
-> schema — a corpus migrated or ingested by either client is searchable by
-> the other. Apache-2.0.
+<p align="center">
+  <a href="https://www.npmjs.com/package/@promptev/context-engine"><img src="https://img.shields.io/npm/v/@promptev/context-engine?label=npm&color=CB3837" alt="npm"></a>
+  <a href="https://pypi.org/project/promptev-context-engine/"><img src="https://img.shields.io/pypi/v/promptev-context-engine?label=PyPI%20(python)&color=3775A9" alt="PyPI"></a>
+  <a href="https://github.com/promptev/context-engine/actions/workflows/js.yml"><img src="https://github.com/promptev/context-engine/actions/workflows/js.yml/badge.svg" alt="CI"></a>
+  <img src="https://img.shields.io/badge/node-22%2B-339933" alt="Node 22+">
+  <img src="https://img.shields.io/badge/types-included-3178C6" alt="TypeScript types included">
+  <a href="../LICENSE.md"><img src="https://img.shields.io/badge/license-Apache--2.0-blue" alt="Apache-2.0"></a>
+</p>
+
 
 It is a **library, not a server**. BYO database, embedding provider, and LLM.
 `npm install`, point it at Postgres you already run, call `engine.search()`.
@@ -22,16 +28,52 @@ optional graph). PII redaction runs **before** a hosted reranker sees hits and
 
 📖 **Docs:** https://promptev.ai/documentation/context-engine/ · **[access control](../docs/access-control.md)**
 
+## Why you'd choose this
+
+**1. Access control that survives the vector index.** Filtering an approximate
+index with a `WHERE` clause is a *post-filter* — the index picks candidates
+first, your ACL discards them second, and the query returns short. We measured
+it on 200k public documents at production embedding width, and because real
+permissions are topically clustered rather than random, the loss is about
+twice what a conventional benchmark reports: **recall 0.473 against 0.980 at
+identical visibility, a quarter of queries empty, no error raised.** Tuning
+does not fully close it, so this library decides between an exact and an
+approximate scan per query instead. [Method](../benchmarks/acl_recall/).
+
+**2. Redaction that runs in the right place.** Not more detectors — better
+*placement*. Hits are masked **before** a hosted reranker sees them and
+**before** the audit row is written, so the masked value is the only one that
+leaves the process.
+
+**3. It's a library, so the data never moves.** No server, no corpus handed
+over. It only ever calls the Postgres, embedding endpoint, and LLM you
+configure.
+
+**4. Governed tool execution in the same place as retrieval.** ACL-scoped
+tools, encrypted credentials, a fail-closed approval gate, and an audit row
+per call — see [Tools](#tools) below.
+
+**Prove it on your own data** — one read-only command, offline, no embedding
+provider called:
+
+```bash
+npx context-engine check-acl-exposure --database-url postgresql://...
+```
+
+It reports, for the ACL values that actually occur in your corpus, what a bare
+scoped query returns versus what this library returns — both scored against an
+exact oracle computed inside Postgres.
+
 ## Install
 
 ```bash
 npm install @promptev/context-engine pg
 ```
 
-Needs **Node.js 20+** and **Postgres** with `vector`, `pg_trgm`, and `unaccent`
+Needs **Node.js 22+** and **Postgres** with `vector`, `pg_trgm`, and `unaccent`
 (created by `context-engine migrate` if the role has `CREATE EXTENSION`).
 
-`pg` is a peer dependency (LGPL) so a bare install pulls no copyleft.
+`pg` is a peer dependency, and it — like everything else here — is MIT/BSD/Apache, so a bare install pulls **no copyleft at all**.
 
 Optional peers — install only what you use:
 
@@ -65,6 +107,13 @@ const engine = new ContextEngine(config);
 
 Or from `CE_`-prefixed env vars (`CE_EMBEDDING__PROVIDER`, nested with `__`).
 
+`storage` holds the deployment knobs: `annExactThreshold` (the vector leg's
+exact/approximate crossover) and the `pg.Pool` settings — `poolMax` (10),
+`poolIdleTimeoutMs` (10000) and `poolConnectionTimeoutMs` (30000). Reachable
+as env vars too, e.g. `CE_STORAGE__POOL_MAX`. (`pg` has no pre-ping, so unlike
+the Python client there is nothing to turn on: a connection the database has
+dropped fails once and is replaced.)
+
 ## Provision
 
 ```bash
@@ -85,6 +134,7 @@ await engine.ingest({
 
 const result = await engine.search("how much annual leave do I get", {
   sourceIds: ["hr-handbook"],
+  documentIds: null, // optionally pin to specific documents WITHIN the sources
   principals: ["group:hr"], // TRUSTED = skip ACL; [] = anonymous
   topK: 10,
 });
@@ -97,6 +147,29 @@ trusted caller and returns the entire corpus. Use `[]` or `TRUSTED` explicitly.
 npx context-engine install-skill
 ```
 
+## Compute over tables
+
+`engine.compute()` turns the in-scope CSV/XLSX documents into row tables and has
+the LLM write code against them. `computeOverFrames()` is the same path for
+tables you already hold — an uploaded workbook, a connector's sheet, a query
+result — with no document to point at:
+
+```ts
+import { computeOverFrames } from "@promptev/context-engine";
+
+const out = await computeOverFrames(
+  { expenses: rows },             // { sheetName: Array<Record<string, unknown>> }
+  "total the amount column",
+  { config: engine.config },
+);
+out.result;
+```
+
+Both are **off by default** (`enableCodeExecution: true` turns them on — only
+behind real OS-level isolation) and both apply `config.redaction` before the LLM
+sees anything: every string cell *and* every column header is masked, and the
+returned object — generated code included — is swept on the way out.
+
 ## Serve
 
 ```ts
@@ -107,6 +180,44 @@ const router = createHonoRouter(engine, { auth: myAuth, principals: myPrincipals
 
 `auth` and `principals` are required. Identical routes exist for Express and Fastify.
 
+## Tools
+
+Register a governed tool — HTTP, SQL, MCP, or a plain function — and every call
+is ACL-scoped, credential-encrypted, optionally approval-gated, and audited.
+
+```ts
+import { ToolConfig, resolveApproval } from "@promptev/context-engine";
+
+await engine.registerTool(new ToolConfig({
+  name: "get_weather",
+  kind: "http",
+  description: "Fetch the current weather for a city",
+  config: { method: "GET", url: "https://api.example.com/weather" },
+  acl: ["group:ops"],
+  requiresApproval: true, // pauses for a human before it runs
+}));
+
+const out = await engine.executeTool("http_get_weather", { city: "Lahore" }, {
+  principals: ["group:ops"],
+  approvalScope: "run-42", // opaque, chosen by you — a run id is the usual shape
+});
+// -> { approval_required: { approval_id: "...", expires_at: "...", ... } }  (not executed yet)
+
+await resolveApproval(engine, approvalId, "approved", "boss");
+// the same call, same approvalScope, runs for real — exactly once
+```
+
+`approvalScope` is the claim scope: an approved record is consumed only by the
+same call in the same scope (and, when the record carries principals, by a
+caller who overlaps them). Within a scope, identical pending requests share one
+record. Resolve it server-side — on the adapters and the MCP gateway it is an
+`approvalScope` resolver beside `principals`, never a request field. Omitting it
+still works this release, unscoped and with a `DeprecationWarning`; the next
+release refuses a gated call with no scope.
+
+A caller may only register or relabel a tool under principals it holds — filing
+one under another group's ACL is a 403.
+
 ## License
 
 Apache-2.0. See `LICENSE.md` and `NOTICE`.