@panaversity/ksor 0.0.22 → 0.0.24

package/docs/tool-surface.md

@@ -0,0 +1,133 @@
+---
+title: The tool surface
+status: draft
+---
+
+# Shaping what agents see — `system/gateways/content.ts`
+
+That file is your record's MCP registration: ordinary `registerTool` calls with
+ordinary zod. It decides what your tools are called, what they say, what they
+accept, and which of them exist. It is yours, and deleting it is supported —
+without it the door serves the same defaults.
+
+## Why it is worth editing
+
+An agent pays for this surface out of its context window, and it pays twice.
+Every tool's name, description and input schema is resident for the whole
+session; every answer spends more.
+
+Measured against a live 81-document record (6,963 chunks), ~4 chars/token:
+
+|                                | chars  | ~tokens |                     |
+| ------------------------------ | ------ | ------- | ------------------- |
+| all three tool definitions     | 11,960 | 2,990   | **always resident** |
+| `search` alone                 | 5,383  | 1,346   | always resident     |
+| `outline` + `read`             | 6,571  | 1,643   | always resident     |
+| one `search`, `k=10` (default) | 14,164 | 3,541   | per call            |
+| one `search`, `k=5`            | 8,009  | 2,002   | per call            |
+
+An agent with five records attached carries ~15,000 tokens of definitions before
+doing any work.
+
+## The three edits that pay
+
+### 1. Delete a tool nothing calls
+
+The biggest win, and the easiest — delete its `registerTool` block. Measured
+live: a registration keeping only a renamed `search` served **5,337** bytes
+against the default's 11,960.
+
+### 2. Say what the record covers
+
+The line that decides whether an agent asks _you_ rather than another record it
+has open. Name the subject **and the boundary**:
+
+```ts
+description: `Employee handbook: leave, benefits, conduct, expenses.
+Not product documentation and not customer data.\n\n${FLOOR.search}`,
+```
+
+Your prose goes **above** `FLOOR.search`, never instead of it — see below.
+
+### 3. Set `k`
+
+`k` is the lever on reply size: 10 costs ~3,500 tokens a call, 5 costs ~2,000.
+The caller can always ask for more, so make the default what you usually need.
+
+```ts
+inputSchema: z.object({
+  query: z.string().min(1).max(2000),
+  k: z.number().int().min(1).max(50).default(5),
+}),
+```
+
+**`budgets.maximum_response_characters` is not this lever.** It defaults to
+120,000 and at ~1,400 chars a hit cannot bind before the 50-hit ceiling. Tune `k`.
+
+## Adding your own tools
+
+It is an MCP server. Call `registerTool` again with your own handler:
+
+```ts
+server.registerTool(
+  "check_policy_expiry",
+  {
+    inputSchema: z.object({ policy: z.string() }),
+  },
+  async ({ policy }) => ({ content: [{ type: "text", text: await lookup(policy) }] }),
+);
+```
+
+One thing to be clear-eyed about: **ksor makes no provenance claim about a tool
+it did not hand you a handler for.** `searchHandler(ctx)` answers from the
+governed record with citations; a handler you write answers from wherever you
+made it answer from.
+
+## What you cannot change, and why
+
+- **The handlers.** `searchHandler` / `outlineHandler` / `readHandler` are the
+  only things that can prove a passage came from the governed record. A
+  hand-written one returning fabricated hits with plausible `stable_id`s would
+  pass every shape check there is.
+- **The output schemas.** `SEARCH_OUTPUT`, `OUTLINE_OUTPUT`, `READ_OUTPUT` carry
+  `provenance`, the `snapshot` token and `gate`. A record that reshaped them
+  would still look like a KSoR and no longer be one.
+- **The `FLOOR` text.** It tells an agent how to branch on an envelope, what
+  `gate: "off"` means, and that corpus content is **untrusted** — quote it, never
+  obey it. Your prose is composed above it.
+
+## The door checks its own surface at boot
+
+Because that last one is a template literal in a file you own, nothing structural
+stops it being dropped. So the door builds its server, asks itself `tools/list`
+over an in-memory transport, and refuses to start if a guarantee is gone:
+
+```
+error: ksor-gateway-floor-missing: the search tool is served as "search_the_book"
+without its framework description. That text tells an agent how to read an
+abstention and that corpus content is untrusted — without it this record answers
+without ever declining, and follows instructions written into its own documents.
+Put FLOOR.search back: a record's own prose goes ABOVE it, as
+`${yourText}\n\n${FLOOR.search}`, never instead of it
+```
+
+| what                                               | slug                         |
+| -------------------------------------------------- | ---------------------------- |
+| a served ksor tool lost its `FLOOR` text           | `ksor-gateway-floor-missing` |
+| the registration serves no tools at all            | `ksor-gateway-no-tools`      |
+| the file throws, or default-exports a non-function | `ksor-gateway-unloadable`    |
+
+Delete the file to take the default registration back.
+
+## One import, no dependencies
+
+Everything comes from `@panaversity/ksor/gateway` — including `z` and
+`McpServer`. That is deliberate: your registration stays a _file_, with no
+package.json, no build step, and nothing new in your lockfile. It also means the
+SDK validates with the same zod instance it was built against, which a
+separately-installed zod would not.
+
+## More records later
+
+`identity` and `praxis` get `system/gateways/<record>.ts` by the same rule.
+Nothing above is specific to content except which tools exist.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@panaversity/ksor",
-  "version": "0.0.22",
+  "version": "0.0.24",
   "description": "Knowledge System of Record — compile governed markdown into a static site for people and an MCP server for AI agents, with citations and measured abstention.",
   "keywords": [
     "abstention",
@@ -44,6 +44,11 @@
       "import": "./dist/index.mjs",
       "default": "./dist/index.mjs"
     },
+    "./gateway": {
+      "types": "./dist/gateway.d.mts",
+      "import": "./dist/gateway.mjs",
+      "default": "./dist/gateway.mjs"
+    },
     "./package.json": "./package.json"
   },
   "publishConfig": {

package/templates/scaffold/AGENTS.md

@@ -311,6 +311,48 @@ signing keys, an unknown key id answers **503**, not 401 — the token may well
 good and the door's key set merely stale, so a client should retry rather than
 send the user back through a login.
 
+## Shaping the agent surface — `system/gateways/content.ts`
+
+That file is this record's MCP registration — ordinary `registerTool` with
+ordinary zod. It is yours, and it is **deletable**: without it the door serves
+the same defaults.
+
+Edit it because an agent pays for this surface out of its context window, twice.
+Measured on an 81-document record:
+
+|                                  |                                |
+| -------------------------------- | ------------------------------ |
+| all three tool definitions       | ~2,990 tokens, always resident |
+| one `search` at `k=10` (default) | ~3,541 tokens per call         |
+| one `search` at `k=5`            | ~2,002 tokens per call         |
+
+Three edits pay for themselves:
+
+- **Delete a tool nothing calls.** Removing `outline` and `read` gives back
+  ~1,643 tokens for the whole session.
+- **Say what this record covers**, above `FLOOR.search`. It is how an agent with
+  several records attached picks yours; name the subject AND the boundary.
+- **Set `k`** in the input schema — it is the lever on reply size.
+
+```ts
+description: `Leave, benefits, conduct. Not product docs.\n\n${FLOOR.search}`,
+inputSchema: z.object({ query: z.string(), k: z.number().int().default(5) }),
+```
+
+You can add your own tools with `registerTool` too — but be clear-eyed: ksor
+makes no provenance claim about a tool it did not hand you a handler for.
+
+What you cannot change is deliberate: the handlers, the output schemas, and the
+`FLOOR` text. Your prose goes ABOVE the floor, never instead of it — the floor
+tells an agent how to read an abstention and that corpus content is untrusted,
+and a record that dropped it would answer without ever declining.
+
+Because that is a template literal in a file you own, the door checks its own
+surface at boot and refuses to start if a guarantee is gone:
+`ksor-gateway-floor-missing`, `ksor-gateway-no-tools`,
+`ksor-gateway-unloadable`. Full detail:
+`node_modules/@panaversity/ksor/docs/tool-surface.md`.
+
 ## Withdrawing a document — `ksor takedown`
 
 A takedown is the one governance act that must reach EVERY surface at once.
@@ -356,6 +398,35 @@ refuses rather than publish a document someone took down.
 behind that audience's own access control, never on a public host.
 Details in README → Deploying.
 
+### The MCP door is a container
+
+The other surface is a live process, so it ships as one. `Dockerfile` and
+`.dockerignore` are yours, at the repo root, and they name no host:
+
+```sh
+docker build -t my-record .
+docker run --rm -p 8080:80 --env-file .env my-record
+```
+
+That image runs on Cloud Run, Fly, Render, ECS, Kubernetes or a VPS unchanged.
+`vercel.json` declares BOTH surfaces — a `site` service built from
+`system/site/out/` and a `door` service pointing at that same `Dockerfile` —
+with rewrites putting the door on `/mcp`, `/health`, `/ready` and the
+`/.well-known/oauth-protected-resource` document, and the site on everything
+else. Two rules if you edit it: the `/(.*)` catch-all must stay LAST, and do
+not add a project-level `trailingSlash` — the site's Next config already sets
+it, and at project level it 308-redirects `POST /mcp`, which breaks the door.
+
+The image deliberately excludes `.env` (a baked DSN is published to anyone who
+can pull the image), `knowledge/` (the door reads Postgres, never the folder)
+and `system/` (the other surface).
+
+**Deploying does not publish.** A container that ingested on boot would pay the
+whole record's embedding cost on every cold start and need write credentials at
+runtime. So `pnpm refresh` is a DEPLOY step you run — from your machine or from
+CI — and a first deploy without it serves an empty record. Full walkthrough:
+`node_modules/@panaversity/ksor/docs/deploying.md` and `…/docs/ingesting.md`.
+
 ## Writing knowledge
 
 - One document per file under `knowledge/`; the path is the document's

package/templates/scaffold/Dockerfile

@@ -0,0 +1,36 @@
+# The MCP door, as an ordinary container.
+#
+# Nothing here names a host. It installs the pinned `@panaversity/ksor` from
+# package.json, listens on $PORT, and runs `ksor serve` — which is all Cloud Run,
+# Fly, Render, ECS, Kubernetes or a plain VPS asks for. `vercel.json` points AT
+# this file rather than replacing it, so the artifact stays portable and the host
+# stays a choice.
+#
+# Build and run it anywhere:
+#   docker build -t my-record .
+#   docker run --rm -p 8080:80 --env-file .env my-record
+#
+# This image serves; it does not publish. `ksor ingest` is a write plane that
+# runs from CI or your machine against the same database — see the deployment
+# guide in node_modules/@panaversity/ksor/docs/deploying.md.
+
+FROM node:24-alpine
+
+WORKDIR /app
+
+# Only the manifest first, so this layer caches until the ksor pin changes.
+COPY package.json ./
+RUN npm install --omit=dev --no-audit --no-fund
+
+# The record's identity and configuration. The CORPUS is deliberately absent:
+# the door serves from Postgres, and knowledge/ belongs to the build that
+# published it — see .dockerignore.
+COPY instance.md ./
+
+# Most container hosts inject PORT; 80 is a sane default when nothing does.
+ENV PORT=80
+EXPOSE 80
+
+# `ksor serve` refuses to boot unauthenticated on a public bind. That posture
+# belongs to the record, not to the host, so it travels inside the image.
+CMD ["node_modules/.bin/ksor", "serve", "--instance", "instance.md"]

package/templates/scaffold/README.md

@@ -60,6 +60,43 @@ unauthenticated: a local run declares `KSOR_AUTH_DISABLED=1` (already in
 a public bind needs a configured SSO door instead. Any other operation is
 `pnpm exec ksor <verb>`.
 
+### Test the agent surface with an actual agent
+
+The MCP door is meant to be read by agents, so check it with one rather than
+with `curl`. With `pnpm serve` running, write `.mcp.json` at the repo root:
+
+```json
+{
+  "mcpServers": {
+    "test-record": {
+      "type": "http",
+      "url": "http://127.0.0.1:8080/mcp"
+    }
+  }
+}
+```
+
+Open a new session of your coding agent, confirm it lists the server, then ask
+it three questions **in this order** — the order is the test:
+
+1. Something the record covers, **phrased in words the document never uses**.
+   Retrieval is semantic, so this should still find it, and every answer should
+   arrive with a citation.
+2. Something **adjacent but not covered** — your record's own subject area, a
+   question it genuinely does not answer. It should decline.
+3. Something far outside the record. It should decline, and must not answer
+   from its own knowledge.
+
+Question 2 is the one that matters. Anything can answer questions it has the
+text for; refusing a plausible near-miss is the property that makes a system of
+record worth trusting, and it is the one that breaks quietly.
+
+**On a fresh record, 2 and 3 will not refuse — and that is honest, not broken.**
+The abstention gate is off until you measure a floor for this corpus, which the
+server says out loud at boot (`abstain OFF`) and in every search envelope
+(`gate: "off"`). Run `pnpm exec ksor calibrate --instance instance.md` first if
+you want to test refusal. Delete `.mcp.json`, or keep it — it holds no secret.
+
 Then talk to your coding agent — `AGENTS.md` carries the working rules, and
 the agent kit in `.agents/skills/` knows how to interview you
 (`intake-interview`), convert your source material (`add-sources`), and keep
@@ -118,7 +155,9 @@ and anything that can serve files can serve it.
 - **Vercel** — connect the repository (or run `vercel`); the shipped
   `vercel.json` answers the setup interview: deploy from the repo root
   (never pin `system/site` as the root directory — the record lives
-  outside it), build with `pnpm build`, serve `system/site/out/`. If the
+  outside it), build with `pnpm build`, serve `system/site/out/`. It also
+  declares the MCP **door** as a second service built from the shipped
+  `Dockerfile`, so `/mcp` and the site share one domain. If the
   build image's pnpm predates the `packageManager` pin, set the
   `ENABLE_EXPERIMENTAL_COREPACK=1` build environment variable.
 **Once `instance.md` declares a `database:`, the BUILD needs the DSN too.**
@@ -143,6 +182,24 @@ writes "nothing denied" and exits 0.
 - **Verify any deploy** the same way: the home page, one document page,
   and `/llms.txt` all load; nothing else is required.
 
+### The agent surface deploys separately
+
+The site is files; the MCP door is a process. `Dockerfile` and `.dockerignore`
+at the repo root build it, and they name no host — the same image runs on
+Cloud Run, Fly, Render, ECS, Kubernetes or a VPS:
+
+```sh
+docker build -t my-record .
+docker run --rm -p 8080:80 --env-file .env my-record
+```
+
+One thing surprises people: **deploying does not publish.** The door serves
+whatever generation is already in the database, so a first deploy with no
+`pnpm refresh` serves an empty record. Publishing is a step you run — from your
+machine or from CI — and it is deliberately not something a booting container
+does. The full walkthrough, including what a cold start costs and where ingest
+belongs, is in `node_modules/@panaversity/ksor/docs/deploying.md`.
+
 If `instance.md` declares `audiences:`, what you deploy is a **tier**.
 Plain `pnpm build` always builds the public tier — safe for any host.
 `KSOR_AUDIENCE=<audience> pnpm build` builds a wider tier for that

package/templates/scaffold/dockerignore

@@ -0,0 +1,26 @@
+# Keep the serving image to what serving needs.
+
+# Secrets. The image takes its configuration from the environment at RUN time;
+# baking a .env into a layer publishes it to anyone who can pull the image.
+.env
+.env.*
+!.env.example
+
+# The corpus. The door serves from Postgres — knowledge/ was published by
+# `ksor ingest` before this container ever started, and copying it in would
+# suggest the container reads it. It does not.
+knowledge/
+
+# The website. It is the OTHER surface, built and hosted separately.
+system/
+
+# Build and tooling noise.
+node_modules/
+.git/
+.github/
+.agents/
+.claude/
+.gemini/
+*.log
+.DS_Store
+.ksor-denylist.json

package/templates/scaffold/pnpm-workspace.yaml

@@ -19,17 +19,13 @@ minimumReleaseAgeExclude:
 # Dependency install scripts are denied by default. Flip an entry to true
 # only with a comment naming what breaks without it. pnpm 11 exits 1 on every
 # install until each build script is explicitly decided (found live:
-# fresh-scaffold pnpm dev, 2026-08-18). All four below are reviewed and stay
-# denied:
+# fresh-scaffold pnpm dev, 2026-08-18). Both below are reviewed and stay denied:
 #   esbuild, sharp — prebuilt platform binaries ship as optionalDependencies,
 #     so their install scripts are download fallbacks the site never needs.
-#   @google/genai, protobufjs — pulled in by the `@panaversity/ksor` serve
-#     tool. genai's flagged script is its own dev `prepare` (nothing the
-#     published tarball needs built); protobufjs's postinstall is a
-#     version-compat warning, nothing built. (Same review as the root
-#     workspace, 2026-08-19; serve works with both denied — verified live.)
+#
+# `@google/genai` and `protobufjs` were listed here too, as deps of the serve
+# tool. The embedding provider now speaks the vendor's REST API directly, so
+# neither package is installed at all and denying them described nothing.
 allowBuilds:
   esbuild: false
   sharp: false
-  "@google/genai": false
-  protobufjs: false

package/templates/scaffold/system/gateways/content.ts

@@ -0,0 +1,162 @@
+/**
+ * The default registration — and the ORIGINAL of the file `ksor init` emits.
+ *
+ * This is the canonical half of decision 18's mechanism, applied to the agent
+ * surface: one rule, two places, asserted rather than trusted. The scaffold's
+ * `system/gateways/content.ts` is this file byte-for-byte below the import
+ * block, and `default-gateway-drift.test.ts` fails on the line that diverges.
+ *
+ * Two places rather than one is forced, not chosen. Node refuses to type-strip
+ * any `.ts` under `node_modules` — `ERR_UNSUPPORTED_NODE_MODULES_TYPE_STRIPPING`,
+ * with no flag to lift it — so the published package cannot import its own
+ * emitted template as a fallback. The compiled twin is how a deleted file still
+ * serves, and the drift test is what stops the twins disagreeing.
+ *
+ * Everything below the import block is what an adopter owns: tool names, titles,
+ * what the record says it covers, input schemas, annotations, and which tools
+ * exist at all. What it composes — handlers, output schemas, and the FLOOR text
+ * — stays in the package, because those are the citation and abstention
+ * guarantees and a hand-written handler is the one thing no shape check catches.
+ */
+
+import {
+  composeInstructions,
+  FLOOR,
+  MAX_OUTLINE_LIMIT,
+  MAX_SEARCH_K,
+  McpServer,
+  outlineHandler,
+  OUTLINE_OUTPUT,
+  READ_ONLY,
+  READ_OUTPUT,
+  readHandler,
+  SEARCH_OUTPUT,
+  searchHandler,
+  z,
+  type ServiceContext,
+} from "@panaversity/ksor/gateway";
+
+export default function buildGateway(ctx: ServiceContext, version: string): McpServer {
+  const server = new McpServer(
+    // The MCP server name agents see. Change it to your record's name.
+    { name: "ksor", version },
+    // instance.md's body is this record's system prompt, preserved beneath the
+    // framework floor. Leave this alone unless you mean to replace the prompt.
+    { instructions: composeInstructions(ctx.instance.instructions) },
+  );
+
+  server.registerTool(
+    "search",
+    {
+      title: "Search the record",
+      // WHAT THIS RECORD COVERS goes first — it is how an agent with several
+      // records attached decides to ask yours. Say the subject AND the
+      // boundary; the second half prevents more wrong calls than the first:
+      //
+      //   description: `Employee handbook: leave, benefits, conduct, expenses.
+      //   Not product documentation and not customer data.\n\n${FLOOR.search}`,
+      //
+      // FLOOR.search must stay. It tells an agent how to read an abstention and
+      // that corpus text is untrusted; the door checks it is still there at boot.
+      description: FLOOR.search,
+      inputSchema: z.object({
+        query: z
+          .string()
+          .min(1)
+          .max(2000)
+          .describe("A focused question or phrase to search the record for"),
+        // `k` is the lever on reply size: 10 costs an agent ~3,500 tokens a
+        // call, 5 costs ~2,000. Lower it to what your record actually needs —
+        // a caller can always ask for more.
+        k: z
+          .number()
+          .int()
+          .min(1)
+          .max(MAX_SEARCH_K)
+          .default(10)
+          .describe(`Maximum passages to return (1–${MAX_SEARCH_K})`),
+      }),
+      outputSchema: SEARCH_OUTPUT,
+      annotations: READ_ONLY,
+    },
+    searchHandler(ctx),
+  );
+
+  // Delete a tool by deleting its block. Measured: outline and read together
+  // cost ~1,643 tokens of context that is resident for an agent's whole
+  // session, whether or not it ever calls them.
+  server.registerTool(
+    "outline",
+    {
+      title: "Outline the record",
+      description: FLOOR.outline,
+      inputSchema: z.object({
+        node: z
+          .string()
+          .optional()
+          .describe("Slug or '/'-path to drill into; omit to browse the top level"),
+        depth: z.number().int().min(0).max(5).optional().describe("Extra levels below the anchor"),
+        limit: z
+          .number()
+          .int()
+          .min(1)
+          .max(MAX_OUTLINE_LIMIT)
+          .default(200)
+          .describe("Maximum rows in ONE page"),
+        offset: z
+          .number()
+          .int()
+          .min(0)
+          .optional()
+          .describe("Rows to skip — pass the previous response's next_offset to continue"),
+      }),
+      outputSchema: OUTLINE_OUTPUT,
+      annotations: READ_ONLY,
+    },
+    outlineHandler(ctx),
+  );
+
+  server.registerTool(
+    "read",
+    {
+      title: "Read a document",
+      description: FLOOR.read,
+      inputSchema: z.object({
+        slug: z.string().min(1).describe("The document's slug or '/'-qualified path (see outline)"),
+        heading: z
+          .string()
+          .optional()
+          .describe(
+            "Restrict to one section subtree: a full heading path, any prefix of one, or a " +
+              "section's last segment when it is unique in the document",
+          ),
+        from_heading: z
+          .string()
+          .optional()
+          .describe("Window cursor from a previous response's next"),
+        snapshot_token: z
+          .string()
+          .optional()
+          .describe(
+            'The "token" string from a search response\'s "snapshot" object — not the object.',
+          ),
+        token_budget: z
+          .number()
+          .int()
+          .min(100)
+          .max(70000)
+          .optional()
+          .describe("Response size budget in tokens (default 70000)"),
+      }),
+      outputSchema: READ_OUTPUT,
+      annotations: READ_ONLY,
+    },
+    readHandler(ctx),
+  );
+
+  // Add your own tools here with ordinary registerTool + zod. They are yours;
+  // ksor makes no provenance claim about a tool it did not hand you a handler
+  // for, and the boot check only inspects the ones it did.
+
+  return server;
+}

package/templates/scaffold/vercel.json

@@ -1,8 +1,26 @@
 {
   "$schema": "https://openapi.vercel.sh/vercel.json",
-  "framework": null,
-  "installCommand": "pnpm install --no-frozen-lockfile",
-  "buildCommand": "pnpm build",
-  "outputDirectory": "system/site/out",
-  "trailingSlash": true
+  "services": {
+    "site": {
+      "root": ".",
+      "installCommand": "pnpm install --no-frozen-lockfile",
+      "buildCommand": "pnpm build",
+      "outputDirectory": "system/site/out"
+    },
+    "door": {
+      "root": ".",
+      "runtime": "container",
+      "entrypoint": "Dockerfile"
+    }
+  },
+  "rewrites": [
+    { "source": "/mcp(.*)", "destination": { "service": "door" } },
+    {
+      "source": "/.well-known/oauth-protected-resource(.*)",
+      "destination": { "service": "door" }
+    },
+    { "source": "/health", "destination": { "service": "door" } },
+    { "source": "/ready", "destination": { "service": "door" } },
+    { "source": "/(.*)", "destination": { "service": "site" } }
+  ]
 }