@panaversity/ksor 0.0.3 → 0.0.4

package/templates/scaffold/.claude/skills/intake-interview/SKILL.md

@@ -2,15 +2,15 @@
 name: intake-interview
 description: The first conversation with the owner of this Knowledge System of Record — six questions that define what it is authoritative for and who may read it, then write instance.md together. Use when the owner asks to set up, configure, or "get started with" this project, when instance.md still contains its scaffold placeholder text, or when the scope of the corpus is unclear.
 metadata:
-  version: "1.2.0"
+  version: "1.3.0"
 ---
 
 # Intake interview
 
 `instance.md` is the identity of this Knowledge System of Record, and its
-prose will one day be the agent surface's system prompt. Do not draft it from
-guesses — interview the owner, one question at a time, and write down what
-they actually say.
+prose IS the agent surface's system prompt (`ksor serve` wires it into the MCP
+server's instructions). Do not draft it from guesses — interview the owner, one
+question at a time, and write down what they actually say.
 
 ## The six questions
 
@@ -49,10 +49,15 @@ act on:
   record's **display title**, the human name every page will lead with
   ("Acme Operations Handbook", not the slug) — then the authority sentence,
   boundary, audience, and strictness — plain prose, written for a reader
-  who must act on it. Leave the frontmatter keys alone, with one exception:
-  an audience model from question 6 is written there as `audiences:` (a
-  list) and `default_visibility:`, and `pnpm check` will hold the record to
-  it from that moment on.
+  who must act on it. Leave the identity frontmatter keys alone; two things
+  are written there when they apply: an audience model from question 6, as
+  `audiences:` (a list) and `default_visibility:` (`pnpm check` holds the
+  record to it from that moment on); and — only when the owner stands up the
+  served MCP rung — the `database:`/`embedding:`/`retrieval:` blocks (see
+  `AGENTS.md` → "Serving to agents"; that is a later climb, not part of this
+  interview). The strictness answer from question 5 is the intent behind the
+  `retrieval.vector_floor` on that climb, measured by `ksor calibrate` — capture
+  it in the prose now so it is ready.
 - Restart `pnpm dev` afterwards so the site picks the new title up, and
   show the owner their name on the page.
 - Offer to capture the source list from question 4 as the first real

package/templates/scaffold/AGENTS.md

@@ -5,22 +5,25 @@ here; every coding agent reads this file first.
 
 ## The two worlds
 
-| Path          | What it is                                                                                                                                                                                                                                                                                                                                  |
-| ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `knowledge/`  | **the record** — governed markdown, the owner's world, the product                                                                                                                                                                                                                                                                          |
-| `system/`     | **the system** — all code that serves the record                                                                                                                                                                                                                                                                                            |
-| `instance.md` | what this SoR is authoritative for; its prose is the future agent surface's system prompt. Its `name:` is the machine identity (llms.txt, future citations) and its body `# H1` is the DISPLAY TITLE every page leads with — both read when the server or build STARTS, so restart `pnpm dev` after changing either (found live 2026-08-18) |
+| Path          | What it is                                                                                                                                                                                                                                                                                                                                                                                     |
+| ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `knowledge/`  | **the record** — governed markdown, the owner's world, the product                                                                                                                                                                                                                                                                                                                             |
+| `system/`     | **the system** — all code that serves the record                                                                                                                                                                                                                                                                                                                                               |
+| `instance.md` | what this SoR is authoritative for; its prose IS the agent surface's system prompt (`ksor serve` wires the body into the MCP server's instructions). Its `name:` is the machine identity (llms.txt, citations) and its body `# H1` is the DISPLAY TITLE every page leads with — both read when the server or build STARTS, so restart `pnpm dev` after changing either (found live 2026-08-18) |
 
 The record survives the system: `knowledge/` must stay readable and complete
 even if `system/` is deleted. Dependency flows one way — the system reads the
 record; the record never references the system.
 
 `instance.md` carries a closed key set — `format`, `name`, `ksor`, `site`,
-and the optional pair `audiences` + `default_visibility` (the record's reader
+the optional pair `audiences` + `default_visibility` (the record's reader
 audiences, ordered least- to most-restricted with `public` first, and the one
-a document takes when it names none — declared together or not at all) — and
-everything that matters about it is the prose below that frontmatter;
-`pnpm check` names any other key rather than ignoring it.
+a document takes when it names none — declared together or not at all), and
+the four serve-config blocks `database` / `embedding` / `retrieval` / `budgets`
+(present only once you climb to the served MCP rung — see "Serving to agents"
+below; a `pnpm dev`-only project declares none). Everything else that matters
+about the instance is the prose below the frontmatter; `pnpm check` names any
+other key rather than ignoring it.
 
 ## Critical rules
 
@@ -37,12 +40,103 @@ everything that matters about it is the prose below that frontmatter;
 ## Commands (run at the repo root)
 
 ```sh
-pnpm install     # once, after cloning or scaffolding
+pnpm install     # once, after cloning or scaffolding (also fetches the pinned `ksor` tool)
 pnpm dev         # the site, hot-reloading, at http://localhost:3000
 pnpm build       # static site into system/site/out/
 pnpm check       # the format checker — run before handing off any knowledge change
 ```
 
+## Serving to agents — the MCP rung (needs Postgres + a provider key)
+
+`ksor serve` runs an MCP server over the record so agents get cited retrieval
+with honest abstention. It is the climbed rung — not required for `pnpm dev`.
+Stand it up in this order (each step's errors explain how to fix themselves):
+
+1. **Configure `instance.md`.** Add the serve blocks to the frontmatter
+   (`pnpm check` accepts them; the kernel validates their values):
+
+   ```yaml
+   database:
+     dsn_env: KSOR_DB_URL # the NAME of the env var holding the DSN — never the DSN itself
+   embedding:
+     provider: gemini # default; the seam, not the vendor, is the contract
+     model: gemini-embedding-001
+     dim: 1536 # ≤ 2000 for the pgvector HNSW index
+   retrieval:
+     vector_floor: uncalibrated # see step 6; `uncalibrated` REFUSES every serve until you paste a number
+   ```
+
+2. **Provision Postgres** with the `vector` extension (`CREATE EXTENSION vector`),
+   e.g. a Neon database. Export the DSN under the name `dsn_env` chose, plus the
+   provider key:
+
+   ```sh
+   export KSOR_DB_URL='postgresql://…'   # the var instance.md names
+   export GEMINI_API_KEY='…'             # the embedding provider key
+   ```
+
+3. **Apply the schema:** `pnpm schema` (creates tables, indexes, and the
+   ingest role).
+
+4. **Authorize ingest:** `pnpm grant` — writes the one row row-level security
+   requires before any write to this corpus is allowed. Idempotent, and
+   `pnpm exec ksor grant --instance instance.md --revoke` withdraws it.
+
+   This is a separate, named act on purpose: applying the schema and
+   authorizing writes are different decisions, and a schema step that granted
+   itself write access would make the tool its own authorizer. Apply the schema
+   and ingest as the SAME Postgres login (the ingest role is granted to whoever
+   applied the DDL).
+
+5. **Ingest:** `pnpm ingest` — embeds `knowledge/` into a fresh generation and
+   activates it (`--flip`). Safe to re-run (see the generation model below).
+
+6. **Calibrate the abstention floor** (only if `vector_floor: uncalibrated`):
+   `pnpm exec ksor calibrate --instance instance.md` prints a recommended
+   `vector_floor` measurement; paste the number into `instance.md`'s `retrieval:`
+   block and re-run. A corpus that declares no `retrieval:` block serves with the
+   gate OFF (honest: it will not refuse out-of-corpus questions).
+
+7. **Serve:** `pnpm serve`.
+
+```sh
+pnpm schema      # apply the DDL (once)
+pnpm grant       # authorize ingest for this corpus (once)
+pnpm ingest      # embed knowledge/ into a generation and activate it
+pnpm serve       # run the MCP server; any other verb: pnpm exec ksor <verb>
+```
+
+### How serving updates work (the generation model)
+
+Each `ksor ingest` builds a **fresh generation** (invisible until activated) and
+carries every unchanged embedding forward from the last complete generation —
+so **re-ingest is safe and cheap**; only changed or failed chunks re-embed.
+`--flip` swaps the active pointer, guarded by a catastrophic-shrink check
+(`KSOR_MAX_SHRINK`, default `0.15` — a flip that drops more than 15% of nodes
+refuses; override with `KSOR_ALLOW_SHRINK=1` when the shrink is intended). The
+previous generation stays as a rollback target; `pnpm exec ksor gc` collects
+abandoned ones.
+
+### Serving safely (fail-closed posture)
+
+`pnpm serve` binds **loopback with auth off** — safe for local use. A **public**
+bind refuses to boot unless auth is configured (`KSOR_SSO_URL` +
+`KSOR_MCP_RESOURCE_URL` + `KSOR_JWT_ALLOWED_AUDIENCES`, making it an OAuth
+Resource Server) OR you deliberately set `KSOR_ALLOW_PUBLIC_UNAUTHENTICATED=1`.
+Never let a dropped auth variable silently ship an open door. On a non-loopback
+bind, set `KSOR_ALLOWED_HOSTS` / `KSOR_ALLOWED_ORIGINS`; on more than one
+replica, set a shared `KSOR_SNAPSHOT_KEYS` (unset ⇒ a per-process key, so a
+search token minted by one replica fails on another).
+
+Two things worth being deliberate about:
+
+- **`KSOR_ALLOW_PUBLIC_UNAUTHENTICATED=1` serves your whole record to anyone
+  who can reach the port.** It exists for deployments fronted by your own
+  gateway or network policy. If nothing else is in front, do not set it.
+- **Set `KSOR_SSO_ISSUER` when your SSO stamps a stable `iss`.** Audience is
+  always enforced against `KSOR_JWT_ALLOWED_AUDIENCES`; naming the issuer adds
+  one more check for the cost of one variable.
+
 ## Publishing
 
 `pnpm build` emits a fully static site (`system/site/out/`) deployable to

package/templates/scaffold/README.md

@@ -18,7 +18,32 @@ pnpm dev        # browse the knowledge at http://localhost:3000
 ```
 
 No pnpm? Run `npm install -g pnpm` — or `corepack enable pnpm` on Node
-versions that bundle corepack.
+versions that bundle corepack. The first `pnpm install` also fetches the
+`ksor` tool (pinned in `package.json`) and writes it into your lockfile —
+commit the updated lockfile.
+
+### Serving to agents
+
+The record's other surface is an MCP server for AI agents — the same
+knowledge, cited, with honest abstention. It is the climbed rung: it needs a
+Postgres store (with pgvector) and an embedding provider key, so it is not
+part of `pnpm dev`. The ordered path is:
+
+```sh
+export KSOR_DB_URL='postgresql://…'   # the DSN var your instance.md names
+export GEMINI_API_KEY='…'             # the embedding provider key
+pnpm schema     # apply the database schema (once)
+pnpm grant      # authorize ingest for this corpus (once)
+pnpm ingest     # embed knowledge/ into a generation and activate it
+pnpm serve      # run the MCP server over the record
+```
+
+One setup step comes before this — adding the `database:`/`embedding:` blocks
+to `instance.md` — and there is more to know about the generation model and the
+fail-closed security posture. `AGENTS.md` → "Serving to agents" is the
+full runbook; your coding agent reads it first. `pnpm serve` binds loopback
+with auth off for local use; a public bind fails closed unless auth is
+configured. Any other operation is `pnpm exec ksor <verb>`.
 
 Then talk to your coding agent — `AGENTS.md` carries the working rules, and
 the agent kit in `.agents/skills/` knows how to interview you
@@ -34,7 +59,7 @@ different coding agent's way of finding the same working contract.
 | -------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
 | `knowledge/`                     | **the record** — your governed markdown. The product; everything else serves it.                                                                                                                                                 |
 | `system/`                        | the code that serves the record: the site today, more as you need it.                                                                                                                                                            |
-| `instance.md`                    | what this record is authoritative for; its `name:` is the identity every surface publishes (read at server/build start — restart `pnpm dev` after renaming). When the agent surface ships, this prose becomes its system prompt. |
+| `instance.md`                    | what this record is authoritative for; its `name:` is the identity every surface publishes (read at server/build start — restart `pnpm dev` after renaming). This prose IS the agent surface's system prompt — `ksor serve` wires it into the MCP server's instructions. |
 | `AGENTS.md`                      | the working contract every coding agent reads first — the rules for writing knowledge here.                                                                                                                                      |
 | `CLAUDE.md`                      | one line, pointing at `AGENTS.md`. Claude Code looks for this filename, not that one.                                                                                                                                            |
 | `.agents/skills/`                | the agent kit: `intake-interview` (define the record with you), `add-sources` (turn source material into governed documents), `format-checker` (the rules, as a program).                                                        |
@@ -43,7 +68,7 @@ different coding agent's way of finding the same working contract.
 | `.github/workflows/validate.yml` | your CI: runs the same checker on every pull request and push to main.                                                                                                                                                           |
 | `.gitattributes`                 | markdown is checked out byte-stable on every platform, so the same commit hashes the same everywhere.                                                                                                                            |
 | `.gitignore`                     | keeps build output, `node_modules/`, and `.env*` out of the record's history.                                                                                                                                                    |
-| `package.json`                   | the `pnpm dev` / `pnpm build` / `pnpm check` commands, and the pnpm version this project pins.                                                                                                                                   |
+| `package.json`                   | the `pnpm dev` / `pnpm build` / `pnpm check` / `pnpm schema` / `pnpm grant` / `pnpm ingest` / `pnpm serve` commands, the pinned `@panaversity/ksor` tool, and the pnpm version this project pins.                                                |
 | `pnpm-workspace.yaml`            | where the workspace looks for code (`system/site`, plus reserved `system/gateways/*` and `system/packages/*`), and the supply-chain policy for installs.                                                                         |
 | `pnpm-lock.yaml`                 | the exact dependency versions — the reason two machines build the same site.                                                                                                                                                     |
 

package/templates/scaffold/instance.md

@@ -18,8 +18,9 @@ the single most important sentence in the project._
 
 Everything below this frontmatter is the identity of this instance: what the
 corpus covers, who it serves, and how strictly it should decline questions it
-does not cover. When the agent surface ships, this prose becomes its system
-prompt — write it for a reader who must act on it.
+does not cover. This prose IS the agent surface's system prompt — `ksor serve`
+wires it into the MCP server's instructions — so write it for a reader who must
+act on it.
 
 Ask your coding agent to run the **intake interview** (it knows how — see
 `.agents/skills/intake-interview/`), answer its questions, and let it write

package/templates/scaffold/package.json

@@ -6,7 +6,14 @@
   "scripts": {
     "dev": "pnpm -C system/site dev",
     "build": "pnpm -C system/site build",
-    "check": "node .agents/skills/format-checker/check.mjs"
+    "check": "node .agents/skills/format-checker/check.mjs",
+    "schema": "ksor schema --instance instance.md --apply",
+    "grant": "ksor grant --instance instance.md",
+    "ingest": "ksor ingest --instance instance.md --knowledge knowledge --flip",
+    "serve": "ksor serve"
+  },
+  "dependencies": {
+    "@panaversity/ksor": "KSOR-STAMP-VERSION"
   },
   "engines": {
     "node": ">=24"

package/templates/scaffold/pnpm-workspace.yaml

@@ -8,12 +8,28 @@ packages:
 # `pnpm install` here never picks up a day-zero compromised release.
 minimumReleaseAge: 2880
 
+# The one exception is the tool that scaffolded this project: `ksor init` pins
+# `@panaversity/ksor` to the EXACT version the adopter deliberately ran, so the
+# quarantine (a defense against unknown third-party releases) must not block the
+# first install for 48 hours after every ksor release. Remove this line only if
+# you unpin the dependency.
+minimumReleaseAgeExclude:
+  - "@panaversity/ksor"
+
 # Dependency install scripts are denied by default. Flip an entry to true
-# only with a comment naming what breaks without it. esbuild and sharp are
-# reviewed and stay denied: both ship prebuilt platform binaries as
-# optionalDependencies, so their install scripts are download fallbacks the
-# site never needs — but pnpm 11 exits 1 on every install/dev until each is
-# explicitly decided. (found live: fresh-scaffold pnpm dev, 2026-08-18)
+# only with a comment naming what breaks without it. pnpm 11 exits 1 on every
+# install until each build script is explicitly decided (found live:
+# fresh-scaffold pnpm dev, 2026-08-18). All four below are reviewed and stay
+# denied:
+#   esbuild, sharp — prebuilt platform binaries ship as optionalDependencies,
+#     so their install scripts are download fallbacks the site never needs.
+#   @google/genai, protobufjs — pulled in by the `@panaversity/ksor` serve
+#     tool. genai's flagged script is its own dev `prepare` (nothing the
+#     published tarball needs built); protobufjs's postinstall is a
+#     version-compat warning, nothing built. (Same review as the root
+#     workspace, 2026-08-19; serve works with both denied — verified live.)
 allowBuilds:
   esbuild: false
   sharp: false
+  "@google/genai": false
+  protobufjs: false