@panaversity/ksor 0.0.11 → 0.0.12

package/CHANGELOG.md

@@ -1,5 +1,112 @@
 # @panaversity/ksor
 
+## 0.0.12
+
+### Patch Changes
+
+- 36e4a4c: The scaffold documents what a CLIENT has to do to reach a public MCP door
+
+  `ksor serve` implements the OAuth Resource Server handshake — an
+  unauthenticated request gets a 401 carrying
+  `WWW-Authenticate: Bearer resource_metadata="…"`, and that document names the
+  record's resource identifier and its authorization server, so a client discovers
+  where to authenticate instead of being told. None of it was written down
+  anywhere an adopter or their agent reads. The operator half was documented (the
+  three environment variables); the half their agents actually execute was not.
+
+  The scaffold's `AGENTS.md` now walks the three steps, and names the failure that
+  goes wrong quietly: a token minted for a different audience is a perfectly valid
+  token, and this door rejects it, so `aud` against `KSOR_JWT_ALLOWED_AUDIENCES` is
+  the first thing to compare when a client authenticates fine and still gets 401.
+  It also records the two behaviours a client author has to know and could not have
+  guessed — RS256 only, no opaque-token introspection, and an unknown key id
+  answering 503 rather than 401, because during a key rotation the token is
+  probably good and retrying beats sending the user back through a login.
+
+  This closes one of the three items named in issue #26; the worked provider
+  recipes and the introspection/rotation policy remain open there.
+
+- 125970c: Every 401 from the MCP door carries its `WWW-Authenticate` challenge, not just the first
+
+  Only the missing-token branch emitted `WWW-Authenticate: Bearer
+resource_metadata="…"`. A token that failed verification — expired, wrong
+  audience, no subject, bad signature — came back as a bare 401. That is the most
+  common 401 a real client will ever see, because tokens expire mid-conversation,
+  and it left the client with no pointer back to the resource-metadata document:
+  it could not re-discover the authorization server it had just been talking to.
+  Only a caller that had never sent a token was told where to go.
+
+  The MCP authorization spec requires `WWW-Authenticate` on a 401 without
+  qualification. Every 401 now carries it, with RFC 6750's `error="invalid_token"`
+  so a client refreshes rather than retrying the dead token.
+
+  A **503** stays deliberately unchallenged: an unreachable key set is our outage,
+  not the token's fault, and challenging there would send a user whose token is
+  perfectly good back through a login over a key-fetch failure.
+
+  Found by adversarially checking the release that documented this door. The
+  adversarial auth suite missed it by asserting the STATUS of each rejection and
+  never the header — it now sweeps every 401-producing token and asserts the
+  challenge on each, with the 503 as the negative control.
+
+- 0a94e31: A 503 refusal no longer puts the database host and user on the wire
+
+  When the deferred boot checks fail, `/mcp` refuses with the thrown error's
+  message in full under `data.detail`. For the three authored failures that is the
+  point — a too-old schema, a governance violation and a text-search mismatch each
+  carry a multi-line remedy the operator has to act on. But the catch treated every
+  error alike, and `pg` writes the host, its resolved address, the port and the
+  database user into its connection and authentication failures. Those went out
+  verbatim to any caller who could reach the door.
+
+  What may leave is now decided in one place and by TYPE, not by inspecting
+  message text: a class we wrote is a class whose words we control. A driver error
+  is refused with its class named and its text withheld, and the caller is told
+  which kind of failure it is — infrastructure, not their request.
+
+  The full text still reaches the operator, deliberately: the refusal says the
+  reason is in the server's logs, and the deferred-boot line recorded only the
+  error's NAME, so before this the real message existed nowhere. That is also why
+  the boot checks are not sanitised at their source — reducing a driver error to a
+  class name early would destroy the one copy anyone can act on.
+
+  **The test that covered this was holding it in place.** It asserted that
+  `http.ts` contains the literal string `data: { detail: message }` — so the leak
+  was pinned by an assertion with reasoning attached. Grepping source is the right
+  instrument for "does this check run before dispatch", because position is a
+  property of source, and the wrong one for "what does the response contain".
+  Response contents are now asserted against real bodies, including a `pg`-shaped
+  connection failure whose host, address, port and user must all be absent.
+
+  Verified live: a gateway pointed at an unreachable database answers
+  `the content store is unavailable (Error)` with no host, port, user or database
+  name anywhere in the body, while the server log carries
+  `connect ECONNREFUSED 127.0.0.1:59999` in full.
+
+- 1dd6211: The dimension ceiling says which shape it applies to, instead of blaming pgvector
+
+  `ksor schema` refuses an embedding dimension above 2000 with
+  "(pgvector vector + HNSW ceiling)". The refusal is right and the reason was
+  wrong: pgvector indexes a `vector` to 2000, but a **`halfvec` to 4000**, via an
+  expression index on the cast — verified live against a real database, where
+  `hnsw ((embedding::halfvec(3072)) halfvec_cosine_ops)` plans an Index Scan.
+
+  The old wording read as pgvector's own limit, so an adopter whose model emits
+  more than 2000 dimensions could conclude it was unusable here, over a wall that
+  is not one. The message now names the shape the ceiling belongs to — this schema
+  declares `VECTOR(dim)` columns and indexes one directly — and the constant
+  carries why raising it is a decision rather than an edit: every query site would
+  have to use the same cast as the index or fall silently back to a sequential
+  scan, and the halfvec arm's float16 rounding lands on the score the abstention
+  gate reads.
+
+  The same claim is corrected in the scaffold's `AGENTS.md`, which gains the reason
+  `dim: 1536` is the shipped default: `gemini-embedding-001` emits 3072 and ksor
+  asks it for 1536, which per Google's published MTEB table costs nothing
+  measurable — 1536 scores 68.17 against 2048's 68.16.
+
+  The 2000 refusal is unchanged. Issue #49 records the decision it now points at.
+
 ## 0.0.11
 
 ### Patch Changes

package/dist/cli.mjs

@@ -16,7 +16,7 @@ import { bodyLimit } from "hono/body-limit";
 import { execFileSync, spawnSync } from "node:child_process";
 import { parseArgs } from "node:util";
 import { readFile, readdir, stat } from "node:fs/promises";
-//#region ../content-gateway/dist/main-3DN4fOiE.mjs
+//#region ../content-gateway/dist/main-CqOtG-q1.mjs
 /**
 * A connection could not be ESTABLISHED in time — retryable.
 *
@@ -448,6 +448,8 @@ const CHUNK_POLICY$1 = "heading-aware-1500-content-only-v5";
 * The Markdown BODY below the frontmatter is the authored agent-surface
 * instructions — byte-preserved, stripped only at the edges.
 */
+/** Mirrors `schema.ts`'s ceiling, so a bad `dim:` is refused when instance.md is
+*  PARSED rather than when the DDL is rendered. The why lives there. */
 const EMBED_DIM_MAX$1$1 = 2e3;
 const SUPPORTED_FORMATS$1$1 = [1];
 var InstanceParseError$1 = class extends Error {
@@ -541,7 +543,7 @@ const groupSchemas$1 = {
 	embedding: z.object({
 		provider: z.string().min(1).default("gemini"),
 		model: z.string().min(1).default(EMBED_MODEL$1),
-		dim: z.coerce.number().int().min(1).max(EMBED_DIM_MAX$1$1).default(EMBED_DIM$1)
+		dim: z.coerce.number().int().min(1).max(EMBED_DIM_MAX$1$1, { error: `at most ${EMBED_DIM_MAX$1$1}: this schema declares VECTOR columns and indexes one directly, and pgvector's HNSW takes a vector to ${EMBED_DIM_MAX$1$1}` }).default(EMBED_DIM$1)
 	}),
 	retrieval: z.object({
 		/**
@@ -3723,6 +3725,54 @@ async function compose(instancePath, version) {
 	};
 }
 /**
+* The body of a deferred-boot refusal, and the one decision inside it: whose
+* message may go on the wire.
+*
+* Three AUTHORED failures carry a remedy written for the operator —
+* `SchemaVersionError`, `GovernanceGateError`, `TextSearchConfigMismatch`. Their
+* whole multi-line message is the point; a caller that receives only the first
+* line has been told a problem exists and not how to end it.
+*
+* Everything else reaching this catch is infrastructure — most of it straight
+* from `pg`, whose connection and authentication failures name the host, the
+* resolved address, the port and the database user. Those went out verbatim
+* under `data.detail`, because the catch treated every error alike.
+*
+* So the split is by TYPE, not by message inspection: a class we wrote is a
+* class whose text we control. A driver error is refused with its class named
+* and its text withheld — the operator finds the real message in the server's
+* own logs, which is where an infrastructure fault belongs.
+*/
+/**
+* Did WE write this error's text?
+*
+* Named classes only. Matching on message prose would put the decision back
+* inside the strings it is meant to police, and a reworded driver error would
+* quietly re-open the leak.
+*/
+function isAuthored(error) {
+	return error instanceof SchemaVersionError || error instanceof GovernanceGateError$1 || error instanceof TextSearchConfigMismatch;
+}
+function refusalBody(error) {
+	if (isAuthored(error)) return {
+		jsonrpc: "2.0",
+		error: {
+			code: -32001,
+			message: `this record cannot be served: ${error.message.split("\n")[0] ?? ""}`,
+			data: { detail: error.message }
+		},
+		id: null
+	};
+	return {
+		jsonrpc: "2.0",
+		error: {
+			code: -32001,
+			message: `this record cannot be served: the content store is unavailable (${error instanceof Error ? error.name : "Error"}). The reason is in this server's logs; it is withheld here because a driver error names the database host and user.`
+		},
+		id: null
+	};
+}
+/**
 * The MCP door: the SDK v2 HTTP entry (Request → Response, stateless)
 * behind Hono, serving the 2026-07-28 revision with 2025-era clients still
 * answered through the stateless fallback. Modern exchanges are buffered JSON;
@@ -3886,16 +3936,8 @@ async function runHttp(composition) {
 		if (verifyBoot !== null) try {
 			await verifyBoot();
 		} catch (error) {
-			const message = error instanceof Error ? error.message : String(error);
-			return new Response(JSON.stringify({
-				jsonrpc: "2.0",
-				error: {
-					code: -32001,
-					message: `this record cannot be served: ${message.split("\n")[0]}`,
-					data: { detail: message }
-				},
-				id: null
-			}), {
+			console.error(`refusing requests — boot checks failing: ${error instanceof Error ? error.stack ?? error.message : String(error)}`);
+			return new Response(JSON.stringify(refusalBody(error)), {
 				status: 503,
 				headers: { "content-type": "application/json" }
 			});
@@ -3939,7 +3981,7 @@ async function runHttp(composition) {
 				identity = await auth.verify(token);
 			} catch (error) {
 				const transient = error instanceof TokenVerifyError && error.transient;
-				return c.json({ error: transient ? "token verification temporarily unavailable" : "invalid token" }, transient ? 503 : 401);
+				return c.json({ error: transient ? "token verification temporarily unavailable" : "invalid token" }, transient ? 503 : 401, transient ? {} : { "www-authenticate": `Bearer error="invalid_token", resource_metadata="${resourceMetadataUrl}"` });
 			}
 			bearer = token;
 		}
@@ -4389,7 +4431,7 @@ async function withPgRetry(op, options = {}) {
 	throw lastError;
 }
 //#endregion
-//#region ../content/dist/commands-ulU-h9ei.mjs
+//#region ../content/dist/commands-Cysnkk_R.mjs
 /**
 * EVAL-LOCKED constants, quarried verbatim from the oracle
 * (sor-agentfactory @ b554f91, config.py) — changing any of these is a
@@ -4425,6 +4467,8 @@ const HARD_MAX_CHARS = 4e3;
 * The Markdown BODY below the frontmatter is the authored agent-surface
 * instructions — byte-preserved, stripped only at the edges.
 */
+/** Mirrors `schema.ts`'s ceiling, so a bad `dim:` is refused when instance.md is
+*  PARSED rather than when the DDL is rendered. The why lives there. */
 const EMBED_DIM_MAX$1 = 2e3;
 const SUPPORTED_FORMATS$1 = [1];
 var InstanceParseError = class extends Error {
@@ -4518,7 +4562,7 @@ const groupSchemas = {
 	embedding: z.object({
 		provider: z.string().min(1).default("gemini"),
 		model: z.string().min(1).default(EMBED_MODEL),
-		dim: z.coerce.number().int().min(1).max(EMBED_DIM_MAX$1).default(EMBED_DIM)
+		dim: z.coerce.number().int().min(1).max(EMBED_DIM_MAX$1, { error: `at most ${EMBED_DIM_MAX$1}: this schema declares VECTOR columns and indexes one directly, and pgvector's HNSW takes a vector to ${EMBED_DIM_MAX$1}` }).default(EMBED_DIM)
 	}),
 	retrieval: z.object({
 		/**
@@ -4942,7 +4986,29 @@ async function runIngest(pool, tenantId, op) {
 * fresh DDL is rendered from the instance that will fill it, never
 * hand-edited.
 */
-/** pgvector vector + HNSW ceiling. */
+/**
+* The largest embedding dimension this schema will render.
+*
+* It is the ceiling for the shape we USE, not pgvector's ceiling: `schema.sql`
+* declares two `VECTOR(dim)` columns and indexes one of them directly, and
+* pgvector's HNSW and IVFFlat take a `vector` to 2000. They take a `halfvec` to
+* **4000**, reachable by indexing an expression — `hnsw ((embedding::halfvec(N))
+* halfvec_cosine_ops)` — which we do not do, so 2000 binds here.
+*
+* Said precisely because the old wording ("pgvector vector + HNSW ceiling")
+* read as pgvector's own limit and sent a reader off to change providers over a
+* wall that is not one (verified live against a real database, 2026-08-21:
+* a halfvec(3072) expression index plans an Index Scan).
+*
+* Raising it is a decision, not a constant: every query site would have to use
+* the same cast as the index or fall silently back to a sequential scan, and
+* the halfvec arm's float16 rounding lands on the score the abstention gate
+* reads. Recorded in issue #49, along with the evidence for staying at 1536 —
+* Google's published MTEB table runs 128..2048 and is FLAT at the top of that
+* range (1536 scores 68.17, 2048 scores 68.16), so there is no gradient to
+* climb toward the ceiling. It carries no 3072 row, so the cost of the
+* truncation itself is unpublished; do not infer one.
+*/
 const EMBED_DIM_MAX = 2e3;
 /** The schema version schema.sql declares — parsed from the DDL so code and
 * the applied database share ONE source (a drift test pins the coupling). */
@@ -4989,7 +5055,7 @@ function verifyTemplate(text, dim) {
 }
 /** The pure core: render the given template text at the given dimension. */
 function renderSchemaText(text, dim, textSearchConfig = SHIPPED_TS_CONFIG) {
-	if (!Number.isInteger(dim) || dim < 1 || dim > 2e3) throw new Error(`dim must be an integer in 1..${EMBED_DIM_MAX} (pgvector vector + HNSW ceiling), got ${JSON.stringify(dim)}`);
+	if (!Number.isInteger(dim) || dim < 1 || dim > 2e3) throw new Error(`dim must be an integer in 1..${EMBED_DIM_MAX} — this schema indexes a vector column directly, and pgvector's HNSW takes a vector to 2000 — got ${JSON.stringify(dim)}`);
 	verifyTemplate(text, EMBED_DIM);
 	const withTs = renderTsConfig(text, textSearchConfig);
 	if (dim === 1536) return withTs;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@panaversity/ksor",
-  "version": "0.0.11",
+  "version": "0.0.12",
   "description": "Knowledge System of Record — compile governed markdown into a static site for people and an MCP server for AI agents, with citations and measured abstention.",
   "keywords": [
     "abstention",

package/templates/scaffold/AGENTS.md

@@ -65,8 +65,13 @@ Stand it up in this order (each step's errors explain how to fix themselves):
    `provider: gemini`, `model: gemini-embedding-001`, `dim: 1536`; write it out
    only to pin the space explicitly or to change it — and note that model and
    dim are the PERSISTED identity of the embedding space, so changing either
-   later means re-embedding the whole corpus. Keep `dim` at or below 2000: the
-   pgvector HNSW index refuses more, and `gemini-embedding-001` can emit 3072.
+   later means re-embedding the whole corpus. Keep `dim` at or below 2000 — the
+   schema indexes a `vector` column directly and pgvector's HNSW takes a
+   `vector` to 2000. `gemini-embedding-001` emits 3072 by default, so ksor asks
+   it for 1536. Google's published MTEB table runs 128–2048 and is flat at the
+   top of it — 1536 scores 68.17 against 2048's 68.16 — so there is no gradient
+   to climb toward the ceiling; going the other way, 768 costs 0.18 if you want
+   the storage back.
 
    Leave `retrieval:` out for now — the gate is off and the server says so.
    Turning it on is step 4, AFTER the record is serving.
@@ -227,7 +232,7 @@ bind, set `KSOR_ALLOWED_HOSTS` / `KSOR_ALLOWED_ORIGINS`; on more than one
 replica, set a shared `KSOR_SNAPSHOT_KEYS` (unset ⇒ a per-process key, so a
 search token minted by one replica fails on another).
 
-Two things worth being deliberate about:
+Three things worth being deliberate about:
 
 - **`KSOR_ALLOW_PUBLIC_UNAUTHENTICATED=1` serves your whole record to anyone
   who can reach the port.** It exists for deployments fronted by your own
@@ -244,6 +249,62 @@ Two things worth being deliberate about:
   `KSOR_JWKS_URL` only to override that, or when your SSO publishes no metadata
   at all.
 
+### What a CLIENT has to do
+
+Once the SSO door is configured (the three variables above), the server is an
+OAuth **Resource Server**, which means a client is not told the authorization
+server — it discovers it. Nothing here needs configuring beyond those variables;
+this is what your agents will experience, and what to check when one cannot
+connect. With `KSOR_AUTH_DISABLED=1` — the local default `.env.example` ships —
+none of it applies: there is no challenge and the metadata document answers 404,
+because there is no authorization server to point at.
+
+1. The client calls `POST /mcp` with no token and gets **401** carrying
+
+   ```
+   WWW-Authenticate: Bearer resource_metadata="https://<your-host>/.well-known/oauth-protected-resource/mcp"
+   ```
+
+   That header is the whole handshake: it names a DOCUMENT, not the resource.
+
+2. The client fetches that document and finds the record's resource identifier
+   and its authorization server:
+
+   ```json
+   {
+     "resource": "https://<your-host>/mcp",
+     "authorization_servers": ["https://your-sso.example.com"]
+   }
+   ```
+
+   Those two values are `KSOR_MCP_RESOURCE_URL` and `KSOR_SSO_URL`.
+
+3. The client gets a token from that authorization server, asking for THIS
+   record as the resource (RFC 8707: `resource=https://<your-host>/mcp`), and
+   sends it as `Authorization: Bearer <token>`.
+
+Every 401 carries that same header, not just the first one — including the one
+your clients will hit most often, a token that expired mid-conversation. It
+arrives as `Bearer error="invalid_token", resource_metadata="…"`, so a client
+knows to refresh rather than to retry the dead token. A **503** is deliberately
+_not_ challenged: an unreachable key set is our outage, not your token's fault,
+and telling a client to re-authenticate over it would send a perfectly good user
+back through a login.
+
+The one thing that goes wrong here goes wrong quietly: a token minted for a
+different audience is a perfectly valid token, and this door rejects it. The
+`aud` claim must match one of `KSOR_JWT_ALLOWED_AUDIENCES` — normally the same
+value as `KSOR_MCP_RESOURCE_URL` — because a bearer accepted for any audience is
+a bearer stolen from one service and replayed against this one. If a client
+authenticates fine and still gets 401, compare its token's `aud` against that
+list before looking anywhere else.
+
+Tokens must be signed **RS256**; nothing else is accepted, and opaque tokens are
+not supported (there is no introspection call). When your SSO rotates its
+signing keys, an unknown key id answers **503**, not 401 — the token may well be
+good and the door's key set merely stale, so a client should retry rather than
+send the user back through a login.
+
 ## Withdrawing a document — `ksor takedown`
 
 A takedown is the one governance act that must reach EVERY surface at once.