@panaversity/ksor 0.0.3 → 0.0.5

package/templates/scaffold/.claude/skills/intake-interview/SKILL.md

@@ -2,15 +2,15 @@
 name: intake-interview
 description: The first conversation with the owner of this Knowledge System of Record — six questions that define what it is authoritative for and who may read it, then write instance.md together. Use when the owner asks to set up, configure, or "get started with" this project, when instance.md still contains its scaffold placeholder text, or when the scope of the corpus is unclear.
 metadata:
-  version: "1.2.0"
+  version: "1.3.0"
 ---
 
 # Intake interview
 
 `instance.md` is the identity of this Knowledge System of Record, and its
-prose will one day be the agent surface's system prompt. Do not draft it from
-guesses — interview the owner, one question at a time, and write down what
-they actually say.
+prose IS the agent surface's system prompt (`ksor serve` wires it into the MCP
+server's instructions). Do not draft it from guesses — interview the owner, one
+question at a time, and write down what they actually say.
 
 ## The six questions
 
@@ -49,10 +49,15 @@ act on:
   record's **display title**, the human name every page will lead with
   ("Acme Operations Handbook", not the slug) — then the authority sentence,
   boundary, audience, and strictness — plain prose, written for a reader
-  who must act on it. Leave the frontmatter keys alone, with one exception:
-  an audience model from question 6 is written there as `audiences:` (a
-  list) and `default_visibility:`, and `pnpm check` will hold the record to
-  it from that moment on.
+  who must act on it. Leave the identity frontmatter keys alone; two things
+  are written there when they apply: an audience model from question 6, as
+  `audiences:` (a list) and `default_visibility:` (`pnpm check` holds the
+  record to it from that moment on); and — only when the owner stands up the
+  served MCP rung — the `database:`/`embedding:`/`retrieval:` blocks (see
+  `AGENTS.md` → "Serving to agents"; that is a later climb, not part of this
+  interview). The strictness answer from question 5 is the intent behind the
+  `retrieval.vector_floor` on that climb, measured by `ksor calibrate` — capture
+  it in the prose now so it is ready.
 - Restart `pnpm dev` afterwards so the site picks the new title up, and
   show the owner their name on the page.
 - Offer to capture the source list from question 4 as the first real

package/templates/scaffold/AGENTS.md

@@ -5,22 +5,25 @@ here; every coding agent reads this file first.
 
 ## The two worlds
 
-| Path          | What it is                                                                                                                                                                                                                                                                                                                                  |
-| ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `knowledge/`  | **the record** — governed markdown, the owner's world, the product                                                                                                                                                                                                                                                                          |
-| `system/`     | **the system** — all code that serves the record                                                                                                                                                                                                                                                                                            |
-| `instance.md` | what this SoR is authoritative for; its prose is the future agent surface's system prompt. Its `name:` is the machine identity (llms.txt, future citations) and its body `# H1` is the DISPLAY TITLE every page leads with — both read when the server or build STARTS, so restart `pnpm dev` after changing either (found live 2026-08-18) |
+| Path          | What it is                                                                                                                                                                                                                                                                                                                                                                                     |
+| ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `knowledge/`  | **the record** — governed markdown, the owner's world, the product                                                                                                                                                                                                                                                                                                                             |
+| `system/`     | **the system** — all code that serves the record                                                                                                                                                                                                                                                                                                                                               |
+| `instance.md` | what this SoR is authoritative for; its prose IS the agent surface's system prompt (`ksor serve` wires the body into the MCP server's instructions). Its `name:` is the machine identity (llms.txt, citations) and its body `# H1` is the DISPLAY TITLE every page leads with — both read when the server or build STARTS, so restart `pnpm dev` after changing either (found live 2026-08-18) |
 
 The record survives the system: `knowledge/` must stay readable and complete
 even if `system/` is deleted. Dependency flows one way — the system reads the
 record; the record never references the system.
 
 `instance.md` carries a closed key set — `format`, `name`, `ksor`, `site`,
-and the optional pair `audiences` + `default_visibility` (the record's reader
+the optional pair `audiences` + `default_visibility` (the record's reader
 audiences, ordered least- to most-restricted with `public` first, and the one
-a document takes when it names none — declared together or not at all) — and
-everything that matters about it is the prose below that frontmatter;
-`pnpm check` names any other key rather than ignoring it.
+a document takes when it names none — declared together or not at all), and
+the four serve-config blocks `database` / `embedding` / `retrieval` / `budgets`
+(present only once you climb to the served MCP rung — see "Serving to agents"
+below; a `pnpm dev`-only project declares none). Everything else that matters
+about the instance is the prose below the frontmatter; `pnpm check` names any
+other key rather than ignoring it.
 
 ## Critical rules
 
@@ -37,12 +40,141 @@ everything that matters about it is the prose below that frontmatter;
 ## Commands (run at the repo root)
 
 ```sh
-pnpm install     # once, after cloning or scaffolding
+pnpm install     # once, after cloning or scaffolding (also fetches the pinned `ksor` tool)
 pnpm dev         # the site, hot-reloading, at http://localhost:3000
 pnpm build       # static site into system/site/out/
 pnpm check       # the format checker — run before handing off any knowledge change
 ```
 
+## Serving to agents — the MCP rung (needs Postgres + a provider key)
+
+`ksor serve` runs an MCP server over the record so agents get cited retrieval
+with honest abstention. It is the climbed rung — not required for `pnpm dev`.
+Stand it up in this order (each step's errors explain how to fix themselves):
+
+1. **Configure `instance.md`.** One block is required — the name of the
+   environment variable holding your DSN (never the DSN itself):
+
+   ```yaml
+   database:
+     dsn_env: KSOR_DB_URL
+   ```
+
+   That is enough. `embedding:` is optional and already defaults to
+   `provider: gemini`, `model: gemini-embedding-001`, `dim: 1536`; write it out
+   only to pin the space explicitly or to change it — and note that model and
+   dim are the PERSISTED identity of the embedding space, so changing either
+   later means re-embedding the whole corpus. Keep `dim` at or below 2000: the
+   pgvector HNSW index refuses more, and `gemini-embedding-001` can emit 3072.
+
+   Leave `retrieval:` out for now — the gate is off and the server says so.
+   Turning it on is step 4, AFTER the record is serving.
+
+2. **Copy `.env.example` to `.env`** and fill it in — `ksor` reads it
+   automatically, so nothing needs exporting, and `.env` is already gitignored.
+   A real environment variable still wins over the file, so CI and production
+   overrides behave normally.
+
+   ```sh
+   cp .env.example .env
+   ```
+
+   Three values matter:
+
+   - `KSOR_DB_URL` — the Postgres store named by `instance.md`'s `dsn_env`. It
+     needs the pgvector extension: `CREATE EXTENSION vector;`
+   - `GEMINI_API_KEY` — the embedding provider key.
+   - `KSOR_AUTH_DISABLED=1` — **required for a local run.** `ksor serve`
+     refuses to boot unauthenticated without it, deliberately, so a server is
+     never left open by accident. It binds loopback, where auth off is the
+     intended dev shape. A PUBLIC deployment configures the SSO door instead —
+     see the comments in `.env.example` and "Serving safely" below.
+
+3. **Bring it up — one command:**
+
+   ```sh
+   pnpm up      # schema → grant → ingest → serve
+   ```
+
+   Every step is re-runnable, so this is also how you **refresh after editing
+   `knowledge/`**: an applied schema reports "already applied", an existing
+   grant reports "already granted", and ingest builds a fresh generation.
+
+   **`pnpm up` ingests every time — but it re-EMBEDS nothing that has not
+   changed.** Chunks carry forward by content hash, so a rerun on an untouched
+   corpus makes zero provider calls (`embedded 0, carried N` in the output).
+   What it does spend is a generation: each run creates and activates a new one,
+   and they accumulate. So:
+
+   | You want to                      | Run                                        |
+   | -------------------------------- | ------------------------------------------ |
+   | set up, or refresh after an edit | `pnpm up`                                  |
+   | just restart the server          | `pnpm serve` — no new generation           |
+   | reap superseded generations      | `pnpm exec ksor gc --instance instance.md` |
+
+   Run the steps individually (`pnpm schema`, `pnpm grant`, `pnpm ingest`)
+   when the acts belong to different people — a DBA holding the credentials
+   that authorize ingest, for instance.
+
+4. **Turn the abstention gate on — deliberately, once it serves.** This is the
+   step that makes "not in this corpus" a real answer, and it is measured, never
+   guessed:
+
+   ```sh
+   pnpm exec ksor calibrate --instance instance.md
+   ```
+
+   It prints a recommended `vector_floor` for THIS corpus in THIS embedding
+   space. Paste the number in and restart:
+
+   ```yaml
+   retrieval:
+     vector_floor: 0.55 # measured by ksor calibrate on <date>
+   ```
+
+   Never copy a floor from another corpus — recalibrate, and record the
+   measurement beside the number. Writing `vector_floor: uncalibrated` declares
+   the intent to gate WITHOUT a measurement, and every serve refuses until a
+   number replaces it; that is the fail-closed posture, not a starting point.
+
+```sh
+pnpm schema      # apply the DDL (once)
+pnpm grant       # authorize ingest for this corpus (once)
+pnpm ingest      # embed knowledge/ into a generation and activate it
+pnpm serve       # run the MCP server; any other verb: pnpm exec ksor <verb>
+```
+
+### How serving updates work (the generation model)
+
+Each `ksor ingest` builds a **fresh generation** (invisible until activated) and
+carries every unchanged embedding forward from the last complete generation —
+so **re-ingest is safe and cheap**; only changed or failed chunks re-embed.
+`--flip` swaps the active pointer, guarded by a catastrophic-shrink check
+(`KSOR_MAX_SHRINK`, default `0.15` — a flip that drops more than 15% of nodes
+refuses; override with `KSOR_ALLOW_SHRINK=1` when the shrink is intended). The
+previous generation stays as a rollback target; `pnpm exec ksor gc` collects
+abandoned ones.
+
+### Serving safely (fail-closed posture)
+
+`pnpm serve` binds **loopback with auth off** — safe for local use. A **public**
+bind refuses to boot unless auth is configured (`KSOR_SSO_URL` +
+`KSOR_MCP_RESOURCE_URL` + `KSOR_JWT_ALLOWED_AUDIENCES`, making it an OAuth
+Resource Server) OR you deliberately set `KSOR_ALLOW_PUBLIC_UNAUTHENTICATED=1`.
+Never let a dropped auth variable silently ship an open door. On a non-loopback
+bind, set `KSOR_ALLOWED_HOSTS` / `KSOR_ALLOWED_ORIGINS`; on more than one
+replica, set a shared `KSOR_SNAPSHOT_KEYS` (unset ⇒ a per-process key, so a
+search token minted by one replica fails on another).
+
+Two things worth being deliberate about:
+
+- **`KSOR_ALLOW_PUBLIC_UNAUTHENTICATED=1` serves your whole record to anyone
+  who can reach the port.** It exists for deployments fronted by your own
+  gateway or network policy. If nothing else is in front, do not set it.
+- **Set `KSOR_SSO_ISSUER` when your SSO stamps a stable `iss`.** Audience is
+  always enforced against `KSOR_JWT_ALLOWED_AUDIENCES`; naming the issuer adds
+  one more check for the cost of one variable.
+
 ## Publishing
 
 `pnpm build` emits a fully static site (`system/site/out/`) deployable to

package/templates/scaffold/README.md

@@ -18,7 +18,39 @@ pnpm dev        # browse the knowledge at http://localhost:3000
 ```
 
 No pnpm? Run `npm install -g pnpm` — or `corepack enable pnpm` on Node
-versions that bundle corepack.
+versions that bundle corepack. The first `pnpm install` also fetches the
+`ksor` tool (pinned in `package.json`) and writes it into your lockfile —
+commit the updated lockfile.
+
+### Serving to agents
+
+The record's other surface is an MCP server for AI agents — the same
+knowledge, cited, with honest abstention. It is the climbed rung: it needs a
+Postgres store (with pgvector) and an embedding provider key, so it is not
+part of `pnpm dev`. The ordered path is:
+
+```sh
+cp .env.example .env    # fill in KSOR_DB_URL, GEMINI_API_KEY, KSOR_AUTH_DISABLED=1
+pnpm up                 # schema → grant → ingest → serve
+```
+
+`ksor` reads `.env` automatically — nothing to export. `KSOR_AUTH_DISABLED=1`
+is required for a local run: serve refuses to boot unauthenticated on purpose,
+so a server is never open by accident.
+
+Add one block to `instance.md` first — `database: { dsn_env: KSOR_DB_URL }`,
+the NAME of the variable, never the DSN. That is the whole required config:
+`embedding:` already defaults to Gemini at 1536 dimensions, and leaving
+`retrieval:` out starts you with the abstention gate off and honest about it
+(turn it on afterwards with `ksor calibrate`, once the record is serving).
+
+`pnpm up` is re-runnable — it is also how you refresh after editing
+`knowledge/`. It re-ingests each time but re-embeds only what changed, so an
+untouched corpus costs no provider calls; to simply restart the server without
+building a new generation, run `pnpm serve` on its own. `AGENTS.md` → "Serving to agents" is the
+full runbook; your coding agent reads it first. `pnpm serve` binds loopback
+with auth off for local use; a public bind fails closed unless auth is
+configured. Any other operation is `pnpm exec ksor <verb>`.
 
 Then talk to your coding agent — `AGENTS.md` carries the working rules, and
 the agent kit in `.agents/skills/` knows how to interview you
@@ -34,7 +66,7 @@ different coding agent's way of finding the same working contract.
 | -------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
 | `knowledge/`                     | **the record** — your governed markdown. The product; everything else serves it.                                                                                                                                                 |
 | `system/`                        | the code that serves the record: the site today, more as you need it.                                                                                                                                                            |
-| `instance.md`                    | what this record is authoritative for; its `name:` is the identity every surface publishes (read at server/build start — restart `pnpm dev` after renaming). When the agent surface ships, this prose becomes its system prompt. |
+| `instance.md`                    | what this record is authoritative for; its `name:` is the identity every surface publishes (read at server/build start — restart `pnpm dev` after renaming). This prose IS the agent surface's system prompt — `ksor serve` wires it into the MCP server's instructions. |
 | `AGENTS.md`                      | the working contract every coding agent reads first — the rules for writing knowledge here.                                                                                                                                      |
 | `CLAUDE.md`                      | one line, pointing at `AGENTS.md`. Claude Code looks for this filename, not that one.                                                                                                                                            |
 | `.agents/skills/`                | the agent kit: `intake-interview` (define the record with you), `add-sources` (turn source material into governed documents), `format-checker` (the rules, as a program).                                                        |
@@ -42,8 +74,9 @@ different coding agent's way of finding the same working contract.
 | `.gemini/settings.json`          | points Gemini CLI at `AGENTS.md`; Gemini does not read that filename on its own.                                                                                                                                                 |
 | `.github/workflows/validate.yml` | your CI: runs the same checker on every pull request and push to main.                                                                                                                                                           |
 | `.gitattributes`                 | markdown is checked out byte-stable on every platform, so the same commit hashes the same everywhere.                                                                                                                            |
-| `.gitignore`                     | keeps build output, `node_modules/`, and `.env*` out of the record's history.                                                                                                                                                    |
-| `package.json`                   | the `pnpm dev` / `pnpm build` / `pnpm check` commands, and the pnpm version this project pins.                                                                                                                                   |
+| `.env.example`                   | the variables the served rung needs; copy to `.env` (gitignored) and fill in. |
+| `.gitignore`                     | keeps build output, `node_modules/`, and `.env` out of the record's history.                                                                                                                                                    |
+| `package.json`                   | the `pnpm dev` / `pnpm build` / `pnpm check` commands and the served rung's `pnpm up` (schema → grant → ingest → serve, or run them separately), the pinned `@panaversity/ksor` tool, and the pnpm version this project pins.                                                |
 | `pnpm-workspace.yaml`            | where the workspace looks for code (`system/site`, plus reserved `system/gateways/*` and `system/packages/*`), and the supply-chain policy for installs.                                                                         |
 | `pnpm-lock.yaml`                 | the exact dependency versions — the reason two machines build the same site.                                                                                                                                                     |
 

package/templates/scaffold/env.example

@@ -0,0 +1,23 @@
+# Copy to .env — ksor reads it automatically, and .env is gitignored.
+# A real environment variable always wins over this file, so CI and production
+# keep their own values.
+
+# The Postgres store, named by instance.md's database.dsn_env.
+# Needs the pgvector extension: CREATE EXTENSION vector;
+KSOR_DB_URL=postgresql://user:password@host:5432/dbname
+
+# The embedding provider key. instance.md defaults to gemini-embedding-001.
+GEMINI_API_KEY=
+
+# Local development posture. `ksor serve` REFUSES to boot unauthenticated
+# without this — deliberately, so a server is never open by accident. It binds
+# loopback, where auth off is the intended dev shape.
+#
+# For a PUBLIC deployment, delete this line and configure the SSO door instead:
+#   KSOR_SSO_URL=https://your-sso.example.com
+#   KSOR_MCP_RESOURCE_URL=https://your-host.example.com/mcp
+#   KSOR_JWT_ALLOWED_AUDIENCES=https://your-host.example.com/mcp
+# Serving a public bind with auth off additionally requires
+# KSOR_ALLOW_PUBLIC_UNAUTHENTICATED=1, which serves your whole record to anyone
+# who can reach the port.
+KSOR_AUTH_DISABLED=1

package/templates/scaffold/gitignore

@@ -12,5 +12,6 @@ system/site/.staged-knowledge/
 
 # secrets never enter the record — system/ is their future home (serve)
 .env*
+!.env.example
 
 .DS_Store

package/templates/scaffold/instance.md

@@ -18,8 +18,9 @@ the single most important sentence in the project._
 
 Everything below this frontmatter is the identity of this instance: what the
 corpus covers, who it serves, and how strictly it should decline questions it
-does not cover. When the agent surface ships, this prose becomes its system
-prompt — write it for a reader who must act on it.
+does not cover. This prose IS the agent surface's system prompt — `ksor serve`
+wires it into the MCP server's instructions — so write it for a reader who must
+act on it.
 
 Ask your coding agent to run the **intake interview** (it knows how — see
 `.agents/skills/intake-interview/`), answer its questions, and let it write

package/templates/scaffold/package.json

@@ -6,7 +6,15 @@
   "scripts": {
     "dev": "pnpm -C system/site dev",
     "build": "pnpm -C system/site build",
-    "check": "node .agents/skills/format-checker/check.mjs"
+    "check": "node .agents/skills/format-checker/check.mjs",
+    "up": "pnpm schema && pnpm grant && pnpm ingest && pnpm serve",
+    "schema": "ksor schema --instance instance.md --apply",
+    "grant": "ksor grant --instance instance.md",
+    "ingest": "ksor ingest --instance instance.md --knowledge knowledge --flip",
+    "serve": "ksor serve"
+  },
+  "dependencies": {
+    "@panaversity/ksor": "KSOR-STAMP-VERSION"
   },
   "engines": {
     "node": ">=24"

package/templates/scaffold/pnpm-workspace.yaml

@@ -8,12 +8,28 @@ packages:
 # `pnpm install` here never picks up a day-zero compromised release.
 minimumReleaseAge: 2880
 
+# The one exception is the tool that scaffolded this project: `ksor init` pins
+# `@panaversity/ksor` to the EXACT version the adopter deliberately ran, so the
+# quarantine (a defense against unknown third-party releases) must not block the
+# first install for 48 hours after every ksor release. Remove this line only if
+# you unpin the dependency.
+minimumReleaseAgeExclude:
+  - "@panaversity/ksor"
+
 # Dependency install scripts are denied by default. Flip an entry to true
-# only with a comment naming what breaks without it. esbuild and sharp are
-# reviewed and stay denied: both ship prebuilt platform binaries as
-# optionalDependencies, so their install scripts are download fallbacks the
-# site never needs — but pnpm 11 exits 1 on every install/dev until each is
-# explicitly decided. (found live: fresh-scaffold pnpm dev, 2026-08-18)
+# only with a comment naming what breaks without it. pnpm 11 exits 1 on every
+# install until each build script is explicitly decided (found live:
+# fresh-scaffold pnpm dev, 2026-08-18). All four below are reviewed and stay
+# denied:
+#   esbuild, sharp — prebuilt platform binaries ship as optionalDependencies,
+#     so their install scripts are download fallbacks the site never needs.
+#   @google/genai, protobufjs — pulled in by the `@panaversity/ksor` serve
+#     tool. genai's flagged script is its own dev `prepare` (nothing the
+#     published tarball needs built); protobufjs's postinstall is a
+#     version-compat warning, nothing built. (Same review as the root
+#     workspace, 2026-08-19; serve works with both denied — verified live.)
 allowBuilds:
   esbuild: false
   sharp: false
+  "@google/genai": false
+  protobufjs: false

package/templates/scaffold/vercel.json

@@ -1,7 +1,7 @@
 {
   "$schema": "https://openapi.vercel.sh/vercel.json",
   "framework": null,
-  "installCommand": "pnpm install --frozen-lockfile",
+  "installCommand": "pnpm install --no-frozen-lockfile",
   "buildCommand": "pnpm build",
   "outputDirectory": "system/site/out",
   "trailingSlash": true