@panaversity/ksor 0.0.18 → 0.0.20

package/CHANGELOG.md

@@ -1,5 +1,88 @@
 # @panaversity/ksor
 
+## 0.0.20
+
+### Patch Changes
+
+- f25e963: A token from another authorization server is refused, not reported as an outage
+
+  An unknown key id raises the same error whether the cause is key-rotation lag or
+  a token minted by an entirely different authorization server. Both were treated
+  as transient, so a client presenting a credential that can never work got `503
+service unavailable` — and retried it, forever, while the misconfiguration read
+  as an outage in every dashboard.
+
+  Reproduced across two real servers: a door configured for Ory Hydra, presented
+  with a genuine Keycloak token, answered 503. It now answers 401, before it
+  fetches a key at all.
+
+  The check runs only when `KSOR_SSO_ISSUER` is set, because only then has the
+  operator stated what the issuer should be. It reads the issuer from an unverified
+  payload, which is sound for exactly one purpose — refusing. A token that passes
+  it still has its signature verified in full, so a lie there buys nothing.
+
+  Genuine rotation lag is still transient, still uncached, and a valid bearer is
+  still re-admitted the instant the key set catches up.
+
+  **New: `docs/authorization.md`**, shipped in the package — two worked recipes for
+  putting a record behind an authorization server, both executed against real
+  servers rather than written from their documentation, plus what an agent does to
+  obtain a token and what each refusal means.
+
+## 0.0.19
+
+### Patch Changes
+
+- aa4bdce: Record why the vector index is unused, and what fixing it would cost
+
+  Diagnosis only — no serving behaviour changes. Answers are unaffected, and were
+  already correct: the query plans a sequential scan, and a sequential scan is
+  EXACT. What grows with the corpus is the work, not the error.
+
+  The cause recorded until now — a window function, then joins and predicates
+  Postgres cannot estimate — was incomplete. Testing each clause on its own shows
+  a cost mispricing underneath: a full sequential pass over 20,000 chunks,
+  including 20,000 1536-dimension distance computations, is priced at 1904 for
+  work that takes ~130 ms, while the HNSW scan's startup cost alone is 2137.
+
+  A restructured arm reaches 36 ms against 648 ms — but only with `ef_search` at
+  pgvector's default, which is the setting where the index missed the true nearest
+  neighbour for 1 query in 100 on a bed with real cluster structure, dropping the
+  top-1 similarity by 0.99. Against this record's ~0.01 abstention separation,
+  that flips an abstention: the corpus holds the answer and the door says it does
+  not. The speed and the approximation cannot be separated, so taking them is an
+  owner decision rather than a tuning change.
+
+  Both the current plan and the fix path are now pinned by tests, so neither can
+  drift unnoticed.
+
+- b9f3d00: A citation pin no longer outlives a restriction
+
+  A snapshot token pins a generation so a citation keeps resolving to the same
+  bytes. It was also deciding the _audience_ question — evaluating `visibility` on
+  the pinned row — so a document restricted after the token was issued kept reading
+  in full for the token's life, to a caller the record had just closed it to.
+
+  Three routes refused it and one served it, on the same surface, in the same
+  second: `outline` omitted it, `search` filtered it, an unpinned `read` refused it,
+  and `read` with a pre-flip token returned the whole document.
+
+  The generation pointers are why the obvious guard missed it. A flip sets
+  `rollback_generation` to the generation just superseded, so a pre-flip pin is
+  exactly the rollback pointer — servable by design, and the check that narrows a
+  pin to {active, rollback} passed it.
+
+  **Governance is now read from the record as it stands.** A pin still decides
+  which generation's content is served; it no longer decides whether the caller may
+  have it. A document the record no longer contains cannot be resurrected by one
+  either. Unpinned reads are unaffected — with nothing pinned, the two generations
+  are the same one and the check is an identity.
+
+  The cost is deliberate: a citation can stop resolving within the token's 30
+  minutes when the record restricts what it points at. That is what "the record
+  changed" should look like. The alternative is a window in which a withdrawal is
+  not a withdrawal.
+
 ## 0.0.18
 
 ### Patch Changes

package/dist/cli.mjs

@@ -15,7 +15,7 @@ import { bodyLimit } from "hono/body-limit";
 import { execFileSync, spawnSync } from "node:child_process";
 import { parseArgs } from "node:util";
 import { readFile, readdir, stat } from "node:fs/promises";
-//#region ../content-gateway/dist/main-Deg5kd9y.mjs
+//#region ../content-gateway/dist/main-9UsxyoZa.mjs
 /**
 * A connection could not be ESTABLISHED in time — retryable.
 *
@@ -2053,6 +2053,28 @@ g AS (
         (SELECT active_generation FROM corpora
           WHERE tenant_id = $1 AND corpus_id = $2)
     ) AS gen
+),
+-- The generation the record is on RIGHT NOW, which decides governance even when
+-- content is served from a pinned one.
+--
+-- A snapshot pin exists so a citation keeps resolving to the same bytes. It used
+-- to decide the audience question too, by evaluating visibility on the pinned
+-- row — so a document restricted after the token was issued kept reading in full
+-- for the token's life, while outline, search and an unpinned read all
+-- refused it in the same second. servableGenerations could not catch it: a
+-- flip sets rollback_generation to the generation just superseded, so a pre-flip
+-- pin IS the rollback pointer and is servable by design (issue #87).
+--
+-- Pins yield. A citation may stop resolving within the token's life, which is
+-- what "the record changed" should look like — the alternative is a window in
+-- which a withdrawal is not a withdrawal, and decision 19 says a surface that
+-- refuses must refuse everywhere, which includes its own fourth route.
+--
+-- When nothing is pinned this is the SAME generation as g, so the join is an
+-- identity and no unpinned read changes behaviour.
+live AS (
+    SELECT active_generation AS gen FROM corpora
+     WHERE tenant_id = $1 AND corpus_id = $2
 )`;
 /** Candidates by LEAF slug ($4), each with its full root path (for suffix disambiguation). */
 const NODE_BY_SLUG_SQL = `
@@ -2082,7 +2104,12 @@ SELECT n.node_id, n.slug, n.title, n.stable_id, n.path, n.generation, n.permalin
 FROM tree n
 JOIN content_nodes self ON self.node_id = n.node_id AND self.tenant_id = $1
                        AND self.generation = n.generation
-WHERE n.slug = $4 AND ${DENY$1} AND ${audienceAllowed$1("self")}
+-- INNER join, so a document the record no longer contains cannot be
+-- resurrected by a pin either: no live row, no read.
+JOIN live ON TRUE
+JOIN content_nodes now ON now.tenant_id = $1 AND now.generation = live.gen
+                      AND now.stable_id = self.stable_id
+WHERE n.slug = $4 AND ${DENY$1} AND ${audienceAllowed$1("now")}
 ORDER BY n.path`;
 const ALIAS_SQL = `
 WITH ${GEN}
@@ -2099,8 +2126,11 @@ const NODE_BY_STABLE_ID_SQL = `
 WITH RECURSIVE ${GEN}, ${DENIED_CTE$1}
 SELECT n.node_id, n.slug, n.title, n.stable_id, n.stable_id::text AS path, n.generation, n.permalink
 FROM content_nodes n JOIN g ON n.generation = g.gen
+JOIN live ON TRUE
+JOIN content_nodes now ON now.tenant_id = $1 AND now.generation = live.gen
+                      AND now.stable_id = n.stable_id
 WHERE n.tenant_id = $1 AND n.stable_id = $4 AND n.status = 'published' AND ${DENY$1}
-  AND ${AUDIENCE_ALLOWED$1}`;
+  AND ${audienceAllowed$1("now")}`;
 const DOCUMENT_CHUNKS_SQL = `
 WITH ${GEN}
 SELECT c.ordinal, COALESCE(c.heading_path_text, ''), c.content
@@ -3478,6 +3508,27 @@ const POS_TTL_S = 60;
 function isBadToken(err) {
 	return err instanceof errors.JWTExpired || err instanceof errors.JWTClaimValidationFailed || err instanceof errors.JWTInvalid || err instanceof errors.JWSInvalid || err instanceof errors.JWSSignatureVerificationFailed || err instanceof errors.JOSEAlgNotAllowed || err instanceof errors.JOSENotSupported;
 }
+/**
+* The `iss` a token CLAIMS, read without verifying anything.
+*
+* Sound for exactly one purpose: REFUSING. A token that passes this check still
+* has its signature verified in full, so an attacker gains nothing by lying here
+* — the worst they achieve is being refused for a different reason. It must
+* never be used to admit anything.
+*
+* Returns null when the payload is not readable JSON, which sends the token down
+* the ordinary path rather than inventing a verdict about it.
+*/
+function claimedIssuer(token) {
+	try {
+		const payload = token.split(".")[1];
+		if (payload === void 0 || payload === "") return null;
+		const iss = JSON.parse(Buffer.from(payload, "base64url").toString("utf8")).iss;
+		return typeof iss === "string" && iss !== "" ? iss : null;
+	} catch {
+		return null;
+	}
+}
 function describeError(err) {
 	return err instanceof Error ? `${err.name}: ${err.message}` : String(err);
 }
@@ -3541,6 +3592,13 @@ function createVerify(config, deps, jwksOf) {
 		if (hit !== void 0 && hit.until > now()) return hit.identity;
 		let claims = null;
 		if (token.split(".").length === 3) {
+			if (config.issuer !== null) {
+				const claimed = claimedIssuer(token);
+				if (claimed !== null && claimed !== config.issuer) {
+					reject(key);
+					throw new TokenVerifyError(`token iss ${JSON.stringify(claimed)} is not this record's authorization server ${JSON.stringify(config.issuer)}`, { transient: false });
+				}
+			}
 			try {
 				claims = await verifyJwt(token);
 			} catch (err) {

package/docs/authorization.md

@@ -0,0 +1,197 @@
+---
+title: Authorization
+status: draft
+---
+
+# Putting the record behind an authorization server
+
+`ksor serve` refuses to boot unauthenticated on a public bind. That is the whole
+posture, and it means the last step of a deployment is standing up an
+authorization server and pointing the door at it.
+
+This page is two worked recipes, both executed against real servers rather than
+written from their documentation, plus what an agent does to obtain a token. The
+mechanism is standard OAuth 2.0 — nothing here is specific to either product, and
+that is the point: two different implementations are shown because a single one
+proves nothing about neutrality.
+
+## What the door needs
+
+Three variables, and one more you should set even though it is optional:
+
+| variable                     | what it is                                                                    | where the value comes from                                                                          |
+| ---------------------------- | ----------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- |
+| `KSOR_SSO_URL`               | your authorization server's base URL                                          | the AS itself; for OIDC it is the issuer, the URL whose `/.well-known/openid-configuration` answers |
+| `KSOR_MCP_RESOURCE_URL`      | **this record's** canonical URL — the identifier a token must be audienced at | you choose it; it is the public URL agents reach the door on                                        |
+| `KSOR_JWT_ALLOWED_AUDIENCES` | which audiences are accepted, comma-separated                                 | normally exactly `KSOR_MCP_RESOURCE_URL`                                                            |
+| `KSOR_SSO_ISSUER`            | the issuer to enforce                                                         | the `issuer` field of the AS's discovery document                                                   |
+
+`KSOR_MCP_RESOURCE_URL` is **not** a place the door fetches anything from. It is
+the name of this resource, in the RFC 8707 sense: a token minted for a different
+resource is refused even when the signature is perfect and the issuer is right.
+That is what stops a token issued for some other service being replayed at your
+record.
+
+The door finds the signing keys by discovery, in this order, and says at boot
+which one it used:
+
+```
+1. KSOR_JWKS_URL                        you stated the URI outright
+2. /.well-known/oauth-authorization-server   RFC 8414 metadata
+3. /.well-known/openid-configuration         OIDC discovery
+4. <KSOR_SSO_URL>/api/auth/jwks              a vendor default, reported as a GUESS
+```
+
+**Set `KSOR_SSO_ISSUER`.** Without it, a token from a _different_ authorization
+server produces an unknown key id, which is indistinguishable from key-rotation
+lag — so the door answers `503`, the client retries a credential that can never
+work, and a misconfiguration reads as an outage. With the issuer declared, that
+same token is refused `401` before any key is fetched.
+
+## Recipe: Keycloak
+
+Run it:
+
+```sh
+docker run -d --name kc -p 8180:8080 \
+  -e KC_BOOTSTRAP_ADMIN_USERNAME=admin -e KC_BOOTSTRAP_ADMIN_PASSWORD=admin \
+  quay.io/keycloak/keycloak:26.0 start-dev
+```
+
+Get an admin token, then create a client for the agent. `client_credentials` is
+the machine-to-machine shape — an agent is not a person and has no browser:
+
+```sh
+KC=http://127.0.0.1:8180
+ADM=$(curl -s -X POST "$KC/realms/master/protocol/openid-connect/token" \
+  -d client_id=admin-cli -d username=admin -d password=admin -d grant_type=password \
+  | jq -r .access_token)
+
+curl -s -X POST "$KC/admin/realms/master/clients" -H "Authorization: Bearer $ADM" \
+  -H 'Content-Type: application/json' -d '{
+    "clientId":"ksor-agent","protocol":"openid-connect","publicClient":false,
+    "serviceAccountsEnabled":true,"standardFlowEnabled":false,"secret":"agent-secret"}'
+```
+
+Keycloak does not put your resource in the token's `aud` on its own. Add an
+audience mapper — this is the step people miss, and its absence looks exactly
+like a rejected token:
+
+```sh
+CID=$(curl -s "$KC/admin/realms/master/clients?clientId=ksor-agent" \
+  -H "Authorization: Bearer $ADM" | jq -r '.[0].id')
+
+curl -s -X POST "$KC/admin/realms/master/clients/$CID/protocol-mappers/models" \
+  -H "Authorization: Bearer $ADM" -H 'Content-Type: application/json' -d '{
+    "name":"ksor-resource-audience","protocol":"openid-connect",
+    "protocolMapper":"oidc-audience-mapper",
+    "config":{"included.custom.audience":"https://records.example.com/mcp",
+              "access.token.claim":"true"}}'
+```
+
+Point the door at it:
+
+```sh
+export KSOR_SSO_URL=http://127.0.0.1:8180/realms/master
+export KSOR_SSO_ISSUER=http://127.0.0.1:8180/realms/master
+export KSOR_MCP_RESOURCE_URL=https://records.example.com/mcp
+export KSOR_JWT_ALLOWED_AUDIENCES=https://records.example.com/mcp
+ksor serve --instance instance.md
+```
+
+The boot block tells you whether discovery worked:
+
+```
+auth     bearer tokens, verified against the record's authorization server
+keys     openid-configuration — http://127.0.0.1:8180/realms/master/protocol/openid-connect/certs
+```
+
+If that line says `guess` instead of naming a discovery document, the AS did not
+publish metadata where the door looked, and you should set `KSOR_JWKS_URL`
+yourself rather than rely on the vendor default.
+
+## Recipe: Ory Hydra
+
+A different implementation, and a different way of asking for the audience —
+which is the neutrality proof. Hydra takes the RFC 8707 `audience` parameter on
+the token request, so no mapper is involved:
+
+```sh
+docker run -d --name hydra -p 4444:4444 -p 4445:4445 \
+  -e DSN=memory -e URLS_SELF_ISSUER=http://127.0.0.1:4444 \
+  -e SECRETS_SYSTEM=change-me-0000000000000000000000 \
+  -e STRATEGIES_ACCESS_TOKEN=jwt \
+  oryd/hydra:v2.2.0 serve all --dev
+
+curl -s -X POST http://127.0.0.1:4445/admin/clients -H 'Content-Type: application/json' -d '{
+  "client_id":"ksor-agent","client_secret":"agent-secret",
+  "grant_types":["client_credentials"],"token_endpoint_auth_method":"client_secret_post",
+  "audience":["https://records.example.com/mcp"],"access_token_strategy":"jwt"}'
+```
+
+Only the two SSO variables change:
+
+```sh
+export KSOR_SSO_URL=http://127.0.0.1:4444
+export KSOR_SSO_ISSUER=http://127.0.0.1:4444
+```
+
+Hydra publishes its keys at `/.well-known/jwks.json` rather than Keycloak's
+`/protocol/openid-connect/certs`. Nothing in ksor knows that; discovery reads it
+from the metadata document, which is why the door works against both unmodified.
+
+## What an agent does
+
+Ask the token endpoint for a token audienced at the record, then send it as an
+ordinary bearer:
+
+```sh
+# Hydra — the audience is a request parameter
+curl -s -X POST http://127.0.0.1:4444/oauth2/token \
+  -d grant_type=client_credentials -d client_id=ksor-agent -d client_secret=agent-secret \
+  -d audience=https://records.example.com/mcp
+
+# Keycloak — the audience comes from the mapper, so the request is plain
+curl -s -X POST "$KC/realms/master/protocol/openid-connect/token" \
+  -d client_id=ksor-agent -d client_secret=agent-secret -d grant_type=client_credentials
+```
+
+With the MCP TypeScript SDK:
+
+```ts
+const transport = new StreamableHTTPClientTransport(new URL("https://records.example.com/mcp"), {
+  requestInit: { headers: { authorization: `Bearer ${accessToken}` } },
+});
+```
+
+A client that does not know where to authenticate can find out: an unauthorized
+request answers `401` with a pointer to this record's metadata, per RFC 9728.
+
+```
+www-authenticate: Bearer resource_metadata="https://records.example.com/.well-known/oauth-protected-resource/mcp"
+```
+
+## What the door refuses, and how it says so
+
+| what you send                                                    | answer                                                                      |
+| ---------------------------------------------------------------- | --------------------------------------------------------------------------- |
+| no token                                                         | `401`, with the `resource_metadata` pointer                                 |
+| a malformed or unsigned token                                    | `401`, `error="invalid_token"`                                              |
+| a token for a **different resource**                             | `401` — the audience binding, and the reason it exists                      |
+| a token from a **different issuer** (with `KSOR_SSO_ISSUER` set) | `401`, before any key is fetched                                            |
+| a token from a different issuer (issuer NOT set)                 | `503` — indistinguishable from rotation lag, which is why you should set it |
+| an expired token                                                 | `401`                                                                       |
+| a valid token                                                    | the record answers, and the answer carries its citations                    |
+
+A genuine key-rotation lag stays a `503` on purpose: it is transient, retrying is
+the right response, and the refusal is never cached — a valid bearer is
+re-admitted the instant the key set catches up.
+
+## Before a public bind
+
+- Auth configured as above, **or** `KSOR_ALLOW_PUBLIC_UNAUTHENTICATED=1` set
+  deliberately — the door will not come up on a public address without one of
+  them, and the second is a decision, not a default.
+- `KSOR_ALLOWED_HOSTS` set to the host you serve on.
+- `KSOR_SNAPSHOT_KEYS` shared across every replica. Unset means a key per
+  process, so a citation minted by one replica fails on another.

package/docs/index.md

@@ -32,6 +32,10 @@ instead of their training memory. The corpus grows with each implemented verb.
   export the manifest the site build reads), `ksor calibrate` (measure the
   abstention floor) and `ksor gc` (reap retired generations). Only `ksor dev` and `ksor build` remain designed, not
   implemented: each prints an honest notice and exits `2`.
+  - **[authorization.md](./authorization.md)** — putting the record behind an
+    authorization server, with worked recipes for two of them, executed rather
+    than written. `ksor serve` refuses to boot unauthenticated on a public bind,
+    so this is the last step of a deployment, not an optional hardening pass.
 - Exit codes are a contract: `1` refused (first stderr line is a stable
   slug such as `error: bad-name`, followed by a remedy), `2` designed but
   not implemented, `3` the environment cannot run ksor

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@panaversity/ksor",
-  "version": "0.0.18",
+  "version": "0.0.20",
   "description": "Knowledge System of Record — compile governed markdown into a static site for people and an MCP server for AI agents, with citations and measured abstention.",
   "keywords": [
     "abstention",
@@ -66,8 +66,8 @@
     "tsdown": "0.22.14",
     "typescript": "7.0.2",
     "vitest": "^4.1.10",
-    "@panaversity/ksor-content": "0.0.0",
-    "@panaversity/ksor-content-gateway": "0.0.0"
+    "@panaversity/ksor-content-gateway": "0.0.0",
+    "@panaversity/ksor-content": "0.0.0"
   },
   "engines": {
     "node": ">=24"