@panaversity/ksor 0.0.40 → 0.0.41

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (103) hide show
  1. package/CHANGELOG.md +869 -0
  2. package/README.md +11 -7
  3. package/dist/checker/check-main.mjs +14049 -0
  4. package/dist/cli.mjs +11528 -5213
  5. package/dist/gateway-api-CF4ED9_g-BQusM_dK.mjs +10895 -0
  6. package/dist/gateway.d.mts +52 -13
  7. package/dist/gateway.mjs +2 -2
  8. package/dist/index.d.mts +1 -1
  9. package/dist/index.mjs +1 -1
  10. package/dist/{src-pl4aOpVs.mjs → src-dqpI-p1a.mjs} +1 -0
  11. package/docs/authorization.md +8 -6
  12. package/docs/deploying.md +36 -25
  13. package/docs/index.md +26 -13
  14. package/docs/ingesting.md +70 -22
  15. package/docs/tool-surface.md +69 -16
  16. package/package.json +4 -3
  17. package/schema/migrations/2.4-2.5__okf-profile.sql +114 -0
  18. package/schema/schema.sql +77 -14
  19. package/templates/scaffold/.agents/skills/add-sources/SKILL.md +63 -18
  20. package/templates/scaffold/.agents/skills/format-checker/SKILL.md +42 -33
  21. package/templates/scaffold/.agents/skills/format-checker/check.mjs +13827 -1314
  22. package/templates/scaffold/.agents/skills/intake-interview/SKILL.md +65 -27
  23. package/templates/scaffold/.agents/skills/make-slides/SKILL.md +7 -5
  24. package/templates/scaffold/.agents/skills/make-summary/SKILL.md +13 -6
  25. package/templates/scaffold/.claude/skills/add-sources/SKILL.md +63 -18
  26. package/templates/scaffold/.claude/skills/format-checker/SKILL.md +42 -33
  27. package/templates/scaffold/.claude/skills/format-checker/check.mjs +13827 -1314
  28. package/templates/scaffold/.claude/skills/intake-interview/SKILL.md +65 -27
  29. package/templates/scaffold/.claude/skills/make-slides/SKILL.md +7 -5
  30. package/templates/scaffold/.claude/skills/make-summary/SKILL.md +13 -6
  31. package/templates/scaffold/.github/workflows/validate.yml +9 -1
  32. package/templates/scaffold/.ksor/governance.yaml +17 -0
  33. package/templates/scaffold/AGENTS.md +234 -113
  34. package/templates/scaffold/Dockerfile +5 -1
  35. package/templates/scaffold/README.md +160 -42
  36. package/templates/scaffold/env.example +37 -6
  37. package/templates/scaffold/gitignore +13 -8
  38. package/templates/scaffold/instance.md +21 -17
  39. package/templates/scaffold/knowledge/governance-ladder.md +6 -2
  40. package/templates/scaffold/knowledge/index.md +9 -0
  41. package/templates/scaffold/knowledge/surfaces/for-agents.md +7 -6
  42. package/templates/scaffold/knowledge/surfaces/for-people.md +7 -6
  43. package/templates/scaffold/knowledge/surfaces/index.md +4 -20
  44. package/templates/scaffold/knowledge/surfaces/overview.md +25 -0
  45. package/templates/scaffold/knowledge/what-is-a-ksor.md +6 -5
  46. package/templates/scaffold/knowledge/what-is-a-ksor.summary.md +4 -0
  47. package/templates/scaffold/package.json +3 -4
  48. package/templates/scaffold/pnpm-lock.yaml +3 -0
  49. package/templates/scaffold/system/gateways/content.ts +13 -0
  50. package/templates/scaffold/system/site/app/(home)/page.tsx +2 -2
  51. package/templates/scaffold/system/site/app/.well-known/mcp/server.json/route.ts +10 -0
  52. package/templates/scaffold/system/site/app/docs/[[...slug]]/page.tsx +126 -91
  53. package/templates/scaffold/system/site/app/global.css +13 -5
  54. package/templates/scaffold/system/site/app/layout.tsx +8 -3
  55. package/templates/scaffold/system/site/app/llms-full.txt/route.ts +13 -7
  56. package/templates/scaffold/system/site/app/llms.txt/route.ts +12 -7
  57. package/templates/scaffold/system/site/app/md/[[...slug]]/route.ts +26 -25
  58. package/templates/scaffold/system/site/components/footer-mark.tsx +3 -2
  59. package/templates/scaffold/system/site/components/governance.tsx +205 -87
  60. package/templates/scaffold/system/site/components/record-index.tsx +5 -5
  61. package/templates/scaffold/system/site/components/record-stack.tsx +10 -9
  62. package/templates/scaffold/system/site/components/sidebar-status.tsx +19 -18
  63. package/templates/scaffold/system/site/lib/attachment-rule.ts +6 -1
  64. package/templates/scaffold/system/site/lib/attachments.ts +0 -28
  65. package/templates/scaffold/system/site/lib/audience-rule.ts +15 -21
  66. package/templates/scaffold/system/site/lib/audience.ts +42 -146
  67. package/templates/scaffold/system/site/lib/embed-rule.ts +9 -0
  68. package/templates/scaffold/system/site/lib/governance.ts +339 -225
  69. package/templates/scaffold/system/site/lib/index-routes.ts +125 -0
  70. package/templates/scaffold/system/site/lib/lifecycle-rule.ts +52 -0
  71. package/templates/scaffold/system/site/lib/lock.ts +282 -0
  72. package/templates/scaffold/system/site/lib/order-rule.ts +37 -0
  73. package/templates/scaffold/system/site/lib/record-href.ts +68 -0
  74. package/templates/scaffold/system/site/lib/record-link.tsx +26 -0
  75. package/templates/scaffold/system/site/lib/rules-version.ts +11 -0
  76. package/templates/scaffold/system/site/lib/shared.ts +67 -104
  77. package/templates/scaffold/system/site/lib/sim-rule.ts +49 -0
  78. package/templates/scaffold/system/site/lib/source.ts +256 -186
  79. package/templates/scaffold/system/site/lib/stage-knowledge.ts +566 -492
  80. package/templates/scaffold/system/site/lib/stage-manifest.ts +128 -0
  81. package/templates/scaffold/system/site/package.json +1 -0
  82. package/templates/scaffold/system/site/record/actor.ts +23 -0
  83. package/templates/scaffold/system/site/record/check.ts +571 -0
  84. package/templates/scaffold/system/site/record/citations.ts +312 -0
  85. package/templates/scaffold/system/site/record/frontmatter.ts +134 -0
  86. package/templates/scaffold/system/site/record/git-ledger.ts +171 -0
  87. package/templates/scaffold/system/site/record/hygiene.ts +320 -0
  88. package/templates/scaffold/system/site/record/index-file.ts +150 -0
  89. package/templates/scaffold/system/site/record/index.ts +103 -0
  90. package/templates/scaffold/system/site/record/instance.ts +257 -0
  91. package/templates/scaffold/system/site/record/instant.ts +43 -0
  92. package/templates/scaffold/system/site/record/ledger.ts +694 -0
  93. package/templates/scaffold/system/site/record/load.ts +129 -0
  94. package/templates/scaffold/system/site/record/lock.ts +306 -0
  95. package/templates/scaffold/system/site/record/near-miss.ts +37 -0
  96. package/templates/scaffold/system/site/record/policy.ts +414 -0
  97. package/templates/scaffold/system/site/record/profile.ts +535 -0
  98. package/templates/scaffold/system/site/record/refusal.ts +106 -0
  99. package/templates/scaffold/system/site/record/yaml-file.ts +103 -0
  100. package/templates/scaffold/system/site/source.config.ts +77 -22
  101. package/dist/gateway-api-CmIthmJS-IUA9qS-T.mjs +0 -3225
  102. package/templates/scaffold/system/site/lib/denial-rule.ts +0 -220
  103. package/templates/scaffold/system/site/lib/page-order.ts +0 -93
@@ -1,7 +1,7 @@
1
1
  import { CallToolResult, McpServer as McpServer$1, StandardSchemaWithJSON } from "@modelcontextprotocol/server";
2
2
  import { z as z$1 } from "zod";
3
3
  import pg from "pg";
4
- //#region ../content-gateway/dist/gateway-api-D9COH1tR.d.mts
4
+ //#region ../content-gateway/dist/gateway-api-Da-9ssbU.d.mts
5
5
  //#region src/instructions.d.ts
6
6
  /**
7
7
  * Has the owner said what this record is FOR yet?
@@ -16,6 +16,10 @@ import pg from "pg";
16
16
  declare function recordIsUndescribed(authored: string): boolean;
17
17
  declare function composeInstructions(authored: string): string;
18
18
  //#endregion
19
+ //#region ../content/dist/index-CQcB_oVG.d.mts
20
+ declare const TRUST_TIERS: readonly ["unverified", "machine-confirmed", "human-reviewed"];
21
+ type TrustTier = (typeof TRUST_TIERS)[number];
22
+ //#endregion
19
23
  //#region ../content/dist/index.d.mts
20
24
  //#endregion
21
25
  //#region src/lib/abstain.d.ts
@@ -45,26 +49,40 @@ interface AbstainConfig {
45
49
  * never set one by intuition, recalibrate per corpus.
46
50
  */
47
51
  readonly keywordFloor: number | null;
52
+ /**
53
+ * The digest of the retrieval predicate {@link vectorFloor} was MEASURED
54
+ * under (`GATE_PREDICATE_DIGEST`), or null on a record that declares none.
55
+ *
56
+ * A number alone cannot say which candidate set it separated, so a floor
57
+ * carried across a predicate change reads as calibrated while gating a set
58
+ * it was never measured on. The door compares this at boot and refuses on a
59
+ * mismatch — the declared-but-uncalibrated state, not `gate: off`.
60
+ */
61
+ readonly floorDigest: string | null;
48
62
  }
49
63
  interface ContentInstance {
50
- /** The corpus identity — instance.md's `name:` (the path/name IS the identity). */
64
+ /** The corpus identity — instance.md's `name:` (the one sanctioned identity key). */
51
65
  readonly name: string;
52
66
  readonly corpusId: string;
53
67
  readonly tenantId: string;
68
+ /** The display title (the root index's heading) and the one-sentence description that seeds `llms.txt` and `server.json`. */
69
+ readonly title: string;
70
+ readonly description: string;
71
+ /** The upgrade stamp `ksor init` wrote; null on a hand-written instance. */
72
+ readonly toolchain: {
73
+ readonly requires: string;
74
+ readonly scaffolded: string;
75
+ } | null;
54
76
  /** The NAME of the env var holding the DSN; the composition root resolves it. */
55
77
  readonly dsnEnv: string;
56
78
  readonly abstain: AbstainConfig;
57
79
  readonly maximumResponseCharacters: number;
58
- /** The authored agent-surface instructions (the body, edge-trimmed). */
80
+ /** The MCP server's instructions (the body, edge-trimmed). */
59
81
  readonly instructions: string;
60
82
  /** Transport name (registry key, never persisted). */
61
83
  readonly embeddingProvider: string;
62
- /** The record's reader audiences, least- to most-restricted; empty = no model. */
63
84
  /** The Postgres text-search configuration the keyword arm stems with. */
64
85
  readonly textSearchConfig: string;
65
- readonly audiences: readonly string[];
66
- /** The tier a document takes when it declares none; null = none declared. */
67
- readonly defaultVisibility: string | null;
68
86
  /** model + dim are the persisted IDENTITY of the embedding space. */
69
87
  readonly embeddingModel: string;
70
88
  readonly embeddingDim: number;
@@ -86,12 +104,23 @@ interface ServiceContext {
86
104
  /** The verified caller, or null → audited as "anonymous". */
87
105
  readonly actor?: () => string | null;
88
106
  /**
89
- * The audience tier this door serves. null = the record's least-privileged
90
- * tier, which is the safe default: a door that cannot establish who is asking
91
- * must not hand out the restricted half of the record. Ignored entirely when
92
- * the instance declares no `audiences:` model.
107
+ * The viewer list this door serves (record spec §2.4) validated against the
108
+ * ingested policy's registry at boot (`validateViewer`), always including
109
+ * `public`. Absent = `[public]`, the safe default: a door that cannot
110
+ * establish who is asking must not hand out the restricted half of the record.
93
111
  */
94
- readonly audience?: string | null;
112
+ readonly viewer?: readonly string[];
113
+ /**
114
+ * The lowest trust tier this door will answer from (record spec §2.3):
115
+ * 0 unverified, 1 machine-confirmed, 2 human-reviewed. Absent = 0, which
116
+ * admits every tier — the honest default, since `verified` is never required
117
+ * and a record with no verifications is a legitimate level-0 state.
118
+ *
119
+ * Bound as an ARM predicate, never applied to the hits afterwards: a floor
120
+ * enforced after ranking has already let a lower-tier passage decide what
121
+ * the answer was.
122
+ */
123
+ readonly minTrustTier?: TrustTier | number;
95
124
  }
96
125
  /**
97
126
  * The largest outline a caller may ASK for. The tool schema and the service
@@ -123,6 +152,16 @@ declare const READ_OUTPUT: StandardSchemaWithJSON;
123
152
  interface SearchArgs {
124
153
  readonly query: string;
125
154
  readonly k: number;
155
+ /**
156
+ * The lowest trust tier the caller will accept an answer from.
157
+ *
158
+ * OPTIONAL on the handler, not on the wire, and that is the point: the
159
+ * registration file is adopter-owned code (decision 23), so a record
160
+ * scaffolded before this parameter existed keeps working — the handler
161
+ * supplies `unverified`, which is what it always had. The boot inspection
162
+ * NOTICES the absence and never refuses it.
163
+ */
164
+ readonly min_trust_tier?: TrustTier | undefined;
126
165
  }
127
166
  declare function searchHandler(ctx: ServiceContext): (args: SearchArgs) => Promise<CallToolResult>;
128
167
  interface OutlineArgs {
@@ -141,4 +180,4 @@ interface ReadArgs {
141
180
  }
142
181
  declare function readHandler(ctx: ServiceContext): (args: ReadArgs) => Promise<CallToolResult>;
143
182
  //#endregion
144
- export { FLOOR, MAX_OUTLINE_LIMIT, MAX_SEARCH_K, McpServer$1 as McpServer, OUTLINE_OUTPUT, type OutlineArgs, READ_ONLY, READ_OUTPUT, type ReadArgs, SEARCH_OUTPUT, type SearchArgs, type ServiceContext, composeInstructions, outlineHandler, readHandler, recordIsUndescribed, searchHandler, z$1 as z };
183
+ export { FLOOR, MAX_OUTLINE_LIMIT, MAX_SEARCH_K, McpServer$1 as McpServer, OUTLINE_OUTPUT, type OutlineArgs, READ_ONLY, READ_OUTPUT, type ReadArgs, SEARCH_OUTPUT, type SearchArgs, type ServiceContext, TRUST_TIERS, type TrustTier, composeInstructions, outlineHandler, readHandler, recordIsUndescribed, searchHandler, z$1 as z };
package/dist/gateway.mjs CHANGED
@@ -1,2 +1,2 @@
1
- import { L as z$1, O as readHandler, a as MAX_OUTLINE_LIMIT, d as READ_OUTPUT, f as SEARCH_OUTPUT, j as searchHandler, k as recordIsUndescribed, l as OUTLINE_OUTPUT, o as MAX_SEARCH_K, r as FLOOR, s as McpServer$1, u as READ_ONLY, w as outlineHandler, y as composeInstructions } from "./gateway-api-CmIthmJS-IUA9qS-T.mjs";
2
- export { FLOOR, MAX_OUTLINE_LIMIT, MAX_SEARCH_K, McpServer$1 as McpServer, OUTLINE_OUTPUT, READ_ONLY, READ_OUTPUT, SEARCH_OUTPUT, composeInstructions, outlineHandler, readHandler, recordIsUndescribed, searchHandler, z$1 as z };
1
+ import { D as outlineHandler, I as searchHandler, N as readHandler, P as recordIsUndescribed, W as z$1, c as McpServer$1, d as READ_ONLY, f as READ_OUTPUT, h as TRUST_TIERS, i as FLOOR, o as MAX_OUTLINE_LIMIT, p as SEARCH_OUTPUT, s as MAX_SEARCH_K, u as OUTLINE_OUTPUT, x as composeInstructions } from "./gateway-api-CF4ED9_g-BQusM_dK.mjs";
2
+ export { FLOOR, MAX_OUTLINE_LIMIT, MAX_SEARCH_K, McpServer$1 as McpServer, OUTLINE_OUTPUT, READ_ONLY, READ_OUTPUT, SEARCH_OUTPUT, TRUST_TIERS, composeInstructions, outlineHandler, readHandler, recordIsUndescribed, searchHandler, z$1 as z };
package/dist/index.d.mts CHANGED
@@ -21,7 +21,7 @@ declare const exitCodes: {
21
21
  type ExitCode = (typeof exitCodes)[keyof typeof exitCodes];
22
22
  /** The CLI vocabulary. Lifecycle verbs plus the corpus operations the bundled
23
23
  * kernel provides (one binary — decision 12 publish revision). */
24
- declare const verbs: readonly ["init", "dev", "build", "serve", "ingest", "schema", "grant", "takedown", "calibrate", "gc"];
24
+ declare const verbs: readonly ["init", "dev", "build", "migrate", "serve", "ingest", "schema", "grant", "takedown", "calibrate", "gc"];
25
25
  type Verb = (typeof verbs)[number];
26
26
  interface ResolvedCommand {
27
27
  /** The first non-flag token, or null when only flags (or nothing) appear. */
package/dist/index.mjs CHANGED
@@ -1,2 +1,2 @@
1
- import { n as resolveCommand, r as verbs, t as exitCodes } from "./src-pl4aOpVs.mjs";
1
+ import { n as resolveCommand, r as verbs, t as exitCodes } from "./src-dqpI-p1a.mjs";
2
2
  export { exitCodes, resolveCommand, verbs };
@@ -24,6 +24,7 @@ const verbs = [
24
24
  "init",
25
25
  "dev",
26
26
  "build",
27
+ "migrate",
27
28
  "serve",
28
29
  "ingest",
29
30
  "schema",
@@ -12,7 +12,7 @@ authorization server and pointing the door at it.
12
12
  This page is four worked recipes, all executed against real servers rather than
13
13
  written from their documentation, plus what an agent does to obtain a token. The
14
14
  mechanism is standard OAuth 2.0 — nothing here is specific to any one product,
15
- and that is the point: three different implementations are shown because a
15
+ and that is the point: four different implementations are shown because a
16
16
  single one proves nothing about neutrality. Two are self-hosted (one `docker
17
17
  run` each, no account), one is a hosted commercial provider with a free tier,
18
18
  and one is an organization's own SSO — which is the case that matters most,
@@ -41,9 +41,11 @@ per-request case needs a decision first.
41
41
  **It is one gate, not per-user rules.** The door checks that a token was signed
42
42
  by the issuer you named and audienced at this record. It reads no scopes, no
43
43
  roles, no groups. **Any caller holding a valid token gets the whole record**, to
44
- the extent the audience tier allows. If different readers must see different
45
- documents, that is the record's `audiences:` / `visibility:` model, and it is a
46
- different mechanism from this page see the scaffold's AGENTS.md.
44
+ the extent its audience list allows. If different readers must see different
45
+ documents, that is the record's audience model the registry in
46
+ `.ksor/governance.yaml`, each concept's `ksor.audience` list, and the viewer
47
+ list the door is configured for — and it is a different mechanism from this
48
+ page. See the scaffold's AGENTS.md.
47
49
 
48
50
  So: this page answers _"can a stranger read my record over MCP?"_ It does not
49
51
  answer _"can Alice read what Bob can."_
@@ -67,7 +69,7 @@ works, not while you are trying to make it work.
67
69
  to anyone who can reach the port — and it wins over everything below. A scaffold
68
70
  ships with `KSOR_AUTH=disabled-local` in its `.env.example`, so a deployment
69
71
  that copied that file has it set and will stay unauthenticated no matter how
70
- carefully you configure the four variables above. Configuring the SSO door is
72
+ carefully you configure the variables above. Configuring the SSO door is
71
73
  what turns auth **on**; removing `KSOR_AUTH` is what stops it being off.
72
74
 
73
75
  Every variable here is read from the environment of the `ksor serve` process —
@@ -190,7 +192,7 @@ opaque by default, no OAuth token endpoint taking a custom audience, no metadata
190
192
  document. That is a coherent design; it is simply a different one, and adapting
191
193
  it means writing the verification layer ksor already is.
192
194
 
193
- The three recipes below all pass. If yours does too, they will read as the same
195
+ The four recipes below all pass. If yours does too, they will read as the same
194
196
  recipe with different button names — because underneath they are.
195
197
 
196
198
  ## Recipe: Keycloak
package/docs/deploying.md CHANGED
@@ -82,16 +82,21 @@ The emitted `Dockerfile` names no host. It installs the pinned
82
82
 
83
83
  ```sh
84
84
  docker build -t my-record .
85
- docker run --rm -p 8080:80 --env-file .env my-record
85
+ docker run --rm -p 8080:80 --env-file .env \
86
+ -e KSOR_AUTH=disabled-public my-record
86
87
  ```
87
88
 
88
89
  Two things about that command, both of which bite:
89
90
 
90
91
  - **The image listens on 80**, which is why the mapping is `8080:80`. If your
91
92
  `.env` sets `PORT`, change the right-hand side to match.
92
- - **`.env` must contain `KSOR_AUTH=disabled-public`, even locally.** A container
93
- gets `$PORT` and therefore binds `0.0.0.0` — a public bind — and
94
- `disabled-local` refuses there by design. Your laptop is not the exception.
93
+ - **`KSOR_AUTH=disabled-public` is required, even locally.** A container gets
94
+ `$PORT` and therefore binds `0.0.0.0` — a public bind — and the
95
+ `disabled-local` a scaffolded `.env` carries refuses there by design. Your
96
+ laptop is not the exception. Pass it with `-e` rather than editing `.env`, so
97
+ a plain `ksor serve` outside the container keeps its loopback posture; `-e`
98
+ overrides `--env-file`. A real deployment sets it — or the SSO variables — in
99
+ the host's environment, since `.dockerignore` keeps `.env` out of the image.
95
100
 
96
101
  That runs on Cloud Run, Fly, Render, ECS, Kubernetes, or a VPS with no changes.
97
102
  `vercel.json` **points at this same file** rather than replacing it, which is
@@ -234,22 +239,20 @@ rebinding is worth an attacker's effort only when it reaches something they coul
234
239
  not reach directly. With `KSOR_AUTH=disabled-public` the record is already served
235
240
  to anyone who types the URL, so there is nothing for rebinding to steal.
236
241
 
237
- ### The site build needs the DSN too
242
+ ### The site build runs `ksor build` first
238
243
 
239
- Once `instance.md` declares a `database:` block, `pnpm build` runs
240
- `pnpm export-denylist` first it asks the database what has been withdrawn and
241
- writes `.ksor-denylist.json`. Without `KSOR_DB_URL` the build **refuses**:
244
+ `pnpm build` is `ksor build` followed by the site build. `ksor build` needs no
245
+ database: it regenerates every `index.md`, runs the record checker, and writes
246
+ `build.lock.json` commit it which every machine artefact stamps. A refusal
247
+ stops the build before a byte is written; `--strict` also refuses an
248
+ uncommitted input. Takedowns reach the site through the committed ledger
249
+ (`.ksor/takedowns.yaml`), which is a file in the repository — so the site build
250
+ needs no `KSOR_DB_URL` at all.
242
251
 
243
- ```
244
- KSOR_DB_URL is unset, and instance.md declares a database
245
- why: a takedown lives in that database. Without it this build cannot tell
246
- 'nothing is denied' from 'nobody asked'
247
- ```
248
-
249
- That refusal is the design working. A takedown reaches the door instantly (it is
250
- a row) and reaches the site at its next build (it reads a file), so a site built
251
- without the DSN would keep publishing what the door already refuses. Set
252
- `KSOR_DB_URL` on the site build as well as on the door.
252
+ That is the design working. A takedown reaches the door instantly (it is a row)
253
+ and reaches the site at its next build (it reads the ledger), so the act that
254
+ withdraws a document is the same merged commit on both surfaces. Merge the
255
+ ledger entry, rebuild, redeploy.
253
256
 
254
257
  ## Keeping people out of the site
255
258
 
@@ -282,12 +285,20 @@ nothing.
282
285
 
283
286
  ### "Some documents are restricted, most are not"
284
287
 
285
- **Build per audience.** `KSOR_AUDIENCE=<tier> pnpm build` stages only what that
286
- tier may see, so restricted documents are **never written into the artifact** —
287
- enforcement by absence, which is the only kind a static host can honour. Publish
288
- the public artifact openly and the wider one behind the gate above.
288
+ **Build per audience.** `KSOR_AUDIENCE=public,internal pnpm build` stages only
289
+ what that viewer may see, so the concepts it may not are **never written into
290
+ the artifact** — enforcement by absence, which is the only kind a static host
291
+ can honour. Publish the public artifact openly and the wider one behind the
292
+ gate above.
293
+
294
+ The value is a comma list of registered audiences and it must include `public`
295
+ — a bare `KSOR_AUDIENCE=internal` is refused (`ksor-viewer-omits-public`),
296
+ because every reader of a restricted build is also a reader of the open one and
297
+ a build for the restricted audience alone would silently drop every public
298
+ concept. A concept is staged when its `ksor.audience` list overlaps the
299
+ viewer's; there is no ordering between audiences to be narrower or wider than.
289
300
 
290
- Plain `pnpm build` is always the public tier, so the safe thing is the default.
301
+ Plain `pnpm build` is `[public]`, so the safe thing is the default.
291
302
 
292
303
  ### "Different readers see different documents, decided per request"
293
304
 
@@ -428,8 +439,8 @@ connections** — the pool minimum is 0 and an unused connection closes after 10
428
439
  first request after a suspend both wakes the database and retries the connect
429
440
  rather than failing.
430
441
 
431
- Measured against a live deployment on Vercel, Neon behind it, an 81-document
432
- record of 6,963 chunks:
442
+ Measured 2026-08-23 against a live deployment on Vercel, Neon behind it, an
443
+ 81-document record of 6,963 chunks, and not re-measured since:
433
444
 
434
445
  | | |
435
446
  | -------------------------------------- | --------- |
package/docs/index.md CHANGED
@@ -31,10 +31,14 @@ instead of their training memory. The corpus grows with each implemented verb.
31
31
  needing Postgres and a provider key — alongside the write plane that keeps
32
32
  the record current: `ksor schema` (provision or migrate the database),
33
33
  `ksor grant` (authorize a tenant for ingest), `ksor ingest` (build and publish
34
- a generation), `ksor takedown` (withdraw a document from EVERY surface, and
35
- export the manifest the site build reads), `ksor calibrate` (measure the
36
- abstention floor) and `ksor gc` (reap retired generations). Only `ksor dev` and `ksor build` remain designed, not
37
- implemented: each prints an honest notice and exits `2`.
34
+ a generation), `ksor takedown` (withdraw a document from EVERY
35
+ surface the committed ledger first, the denylist row second, so a record
36
+ with no database can withdraw one), `ksor calibrate` (measure the
37
+ abstention floor) and `ksor gc` (reap retired generations). `ksor build`
38
+ (check the record, generate its indexes, write `build.lock.json`) and
39
+ `ksor migrate` (rewrite a pre-profile record into the KSoR Profile — a diff
40
+ first, `--write` to apply) need no database. Only `ksor dev` remains designed, not implemented: it prints an
41
+ honest notice and exits `2`.
38
42
  - **[tool-surface.md](./tool-surface.md)** — shaping what agents see.
39
43
  `system/gateways/content.ts` is emitted, adopter-owned and deletable; it
40
44
  decides tool names, what the record says it covers, and how much of the
@@ -62,15 +66,24 @@ instead of their training memory. The corpus grows with each implemented verb.
62
66
  ## For the agent operating a scaffolded project
63
67
 
64
68
  Read the scaffold's own `AGENTS.md` first — it is the working contract.
65
- Knowledge lives in `knowledge/` and never inside the site; frontmatter uses
66
- a closed key set (`title` + `status` required); `pnpm check` explains any
67
- violation and how to fix it. Reading order is the governed `order:`
68
- frontmatter key never `meta.json` or `sidebar_position` and it drives
69
- every surface: the sidebar, `llms.txt`, and the MCP `outline` tool. If the
70
- instance declares an `audiences:` model, documents may carry a
71
- `visibility:` key and per-audience builds (`KSOR_AUDIENCE=<tier> pnpm
72
- build`) stage only what that tier may see publication, not authorship:
73
- anyone who can clone reads everything. The site shell
69
+ Knowledge lives in `knowledge/` and never inside the site. A document is an
70
+ OKF concept in the KSoR Profile: `type`, `title`, `description`, `status`
71
+ (`draft | stable | deprecated`) and `ksor.audience` are required, and a
72
+ `stable` one additionally carries `generated` and a `ksor.approval` by an
73
+ actor the Governance Policy authorises. Keys the profile does not know are
74
+ PRESERVED rather than refused; the pre-profile ones (`visibility`, `owner`,
75
+ `provenance`, `effective`, `superseded`, `sor_id`) are refused by name, each
76
+ naming its replacement. `pnpm check` explains any violation and how to fix it.
77
+ Reading order is the governed `order:` frontmatter key — never `meta.json` or
78
+ `sidebar_position` — and it drives every surface: the sidebar, `llms.txt`, and
79
+ the MCP `outline` tool.
80
+
81
+ Audience is a LIST, matched by overlap. `.ksor/governance.yaml` registers the
82
+ audiences; each concept lists the ones it is for; a build names its viewer as a
83
+ comma list that must include `public`
84
+ (`KSOR_AUDIENCE=public,internal pnpm build`) and stages only the concepts whose
85
+ list overlaps it. It is publication, not authorship: anyone who can clone reads
86
+ everything. The site shell
74
87
  at `system/site/` is replaceable behind a five-clause surface contract — the
75
88
  shell is a slot, and the contract (render the record, `llms.txt`, per-page md
76
89
  artifacts, browser smoke, no authored content) is what a replacement must meet.
package/docs/ingesting.md CHANGED
@@ -20,12 +20,12 @@ Ingest reads your markdown, sends each new chunk to an embedding provider, and
20
20
  writes the result to Postgres. So four things must be true, and none of them is
21
21
  created for you.
22
22
 
23
- | | what | how |
24
- | -------------------- | ------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- |
25
- | **The corpus** | `knowledge/` at your repo root — CommonMark `.md`, one document per file, `title` and `status` in frontmatter | `pnpm check` validates it and explains any violation |
26
- | **The database** | Postgres with **pgvector** — `CREATE EXTENSION vector;` | any managed host; the DDL below needs a role that can create tables |
27
- | **The provider key** | `GEMINI_API_KEY` — the default embedding provider is `gemini-embedding-001` | [aistudio.google.com](https://aistudio.google.com/apikey); the free tier covers a first corpus |
28
- | **The DSN** | `KSOR_DB_URL`, named by `instance.md`'s `database.dsn_env` | uncomment the `database:` block in `instance.md` first |
23
+ | | what | how |
24
+ | -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
25
+ | **The corpus** | `knowledge/` at your repo root — CommonMark `.md`, one document per file, in the KSoR Profile of OKF: `type`, `title`, `description`, `status` and `ksor.audience` in frontmatter | `pnpm check` validates it and explains any violation; `ksor build` must have written a current `build.lock.json` before ingest will run |
26
+ | **The database** | Postgres with **pgvector** — `CREATE EXTENSION vector;` | any managed host; the DDL below needs a role that can create tables |
27
+ | **The provider key** | `GEMINI_API_KEY` — the default embedding provider is `gemini-embedding-001` | [aistudio.google.com](https://aistudio.google.com/apikey); the free tier covers a first corpus |
28
+ | **The DSN** | `KSOR_DB_URL`, named by `instance.md`'s `database.dsn_env` | uncomment the `database:` block in `instance.md` first |
29
29
 
30
30
  Both variables go in `.env` beside `instance.md` — `ksor` reads it automatically,
31
31
  and `.env` is gitignored. Every command below is run **from your repository
@@ -55,6 +55,24 @@ serves immediately; the website is a static build and only changes when you
55
55
  rebuild and redeploy it. Ingest alone leaves the human surface showing the old
56
56
  content, which reads as a half-failed ingest and is not one.
57
57
 
58
+ ## When ingest says the lock is stale
59
+
60
+ `ksor ingest` refuses a record whose `build.lock.json` does not match the tree
61
+ (`ksor-lock-stale`), or that has none at all (`ksor-lock-missing`). **The fix is
62
+ always `ksor build` in that directory — never an edit to the lock.** The lock is
63
+ what says which bytes were checked; editing it to agree with the tree asserts a
64
+ check that never ran, which is the one thing it exists to prevent.
65
+
66
+ Freshness covers **seven** sets, not just your documents: `instance.md`,
67
+ `.ksor/governance.yaml`, `.ksor/takedowns.yaml`, the concepts, the companions,
68
+ the assets, and the generated `index.md` files. So a refusal can name a file you
69
+ would not think of as content — the policy, the takedown ledger, a diagram you
70
+ replaced, or an index you have never opened. That breadth is deliberate: a
71
+ freshness claim that cannot see the ledger is not a freshness claim. The
72
+ website's lock was fixed for exactly this once already, after deleting a
73
+ denial's four lines republished the document with the committed lock still
74
+ validating; ingest had the same hole, found by review before anyone hit it.
75
+
58
76
  ## What a generation is
59
77
 
60
78
  Each ingest builds a **fresh generation** — invisible until activated — and
@@ -106,17 +124,19 @@ An empty `nodes` array means nothing was published — the ingest did not run, o
106
124
  ran without `--flip`. Then search for a phrase you know is in the record and
107
125
  check the hits carry `provenance.stable_id` and `generation`.
108
126
 
109
- **`provenance.stable_id` is also how you name a document to `takedown`.** For
110
- most documents it is `knowledge/<path-without-.md>`; a search result is the
111
- reliable way to read one off rather than guessing.
127
+ **`provenance.stable_id` is also how you name a document to `takedown`.** It is
128
+ `knowledge/<path-without-.md>` always, since path is identity and nothing
129
+ overrides it — and a search result is still the reliable way to read one off
130
+ rather than typing it from memory.
112
131
 
113
132
  ## Where ingest runs — not on the host
114
133
 
115
134
  Ingest is a long job. It embeds every new chunk through the provider, and that
116
135
  is bounded by the provider's throughput rather than by anything ksor does.
117
136
 
118
- **Measured:** an 81-document book — 6,963 chunks — took **about 50 minutes**
119
- against a remote Postgres on a first, cold ingest with nothing to carry forward.
137
+ **Measured 2026-08-23:** an 81-document book — 6,963 chunks — took **about 50
138
+ minutes** against a remote Postgres on a first, cold ingest with nothing to
139
+ carry forward. Not re-measured since.
120
140
 
121
141
  Compare that with the request timeouts of the platforms people reach for first:
122
142
  a serverless function caps out in the region of 300–800 seconds depending on
@@ -148,7 +168,8 @@ If your provider offers both a **pooled** and a **direct** endpoint (Neon's
148
168
  `-pooler` host, or port 6432), use whichever it gives you. ksor detects which and
149
169
  says so at boot, but the line is informational — it classifies, it never
150
170
  transforms, and the hazard it descends from cannot arise here. The 6,963-chunk
151
- ingest measured above ran through a pooled endpoint without incident. Reach for
171
+ ingest measured above (2026-08-23) ran through a pooled endpoint without
172
+ incident. Reach for
152
173
  the direct endpoint only if you actually hit pooler connection limits.
153
174
 
154
175
  ## Turning the abstention gate on
@@ -161,14 +182,24 @@ measure until the corpus is in there.
161
182
  pnpm exec ksor calibrate --instance instance.md
162
183
  ```
163
184
 
164
- It prints a recommended `vector_floor`. Paste it into **`instance.md`** with the
165
- date you measured it, then restart `ksor serve` the floor is read at boot:
185
+ It ends with a block to paste into **`instance.md`**'s frontmatter, exactly as
186
+ printed the floor, the measurement recorded beside it as a comment, and
187
+ `floor_digest`, the digest of the retrieval predicate the floor was measured
188
+ through. Paste it, then restart `ksor serve`: the floor is read at boot.
166
189
 
167
190
  ```yaml
168
191
  retrieval:
169
- vector_floor: 0.55 # measured by ksor calibrate on 2026-08-23
192
+ vector_floor: 0.55 # calibrated 2026-08-23 on generation 3, model gemini-embedding-001/d1536, door: synthesized
193
+ floor_digest: 8bfb07d0e6f5
170
194
  ```
171
195
 
196
+ If the file already declares `retrieval:`, merge the keys into the block it has
197
+ rather than adding a second one — a duplicate key is refused.
198
+
199
+ A measurement that does not separate in-corpus from out-of-corpus prints **no
200
+ floor at all**, and a `vector_floor: uncalibrated` block instead. That block is
201
+ the fail-closed state: every serve refuses until a real number replaces it.
202
+
172
203
  Until you do, `/health` reports the gate as `OFF (no floor declared — will not
173
204
  refuse out-of-corpus questions)` and every search envelope carries
174
205
  `gate: "off"`. That is honest absence, and an agent reading the envelope knows
@@ -186,11 +217,13 @@ correctly stays uncalibrated.
186
217
 
187
218
  ## Withdrawing a document
188
219
 
189
- A takedown is a row, not a file, so it reaches the door immediately:
220
+ A takedown is a committed ledger entry FIRST and a database row second, written
221
+ in one act — so it reaches the door immediately and the site at its next build,
222
+ and a record with no database can still withdraw a document:
190
223
 
191
224
  ```sh
192
225
  pnpm exec ksor takedown --instance instance.md <stable-id> \
193
- --reason "legal request 2026-08" --actor "j.smith"
226
+ --reason "legal request 2026-08" --actor human:j.smith
194
227
  ```
195
228
 
196
229
  `--actor` is required, and there is no default. A name taken from the
@@ -198,8 +231,23 @@ environment reads like a person and is whatever the shell happened to be
198
231
  (`runner` under CI, `root` in a container) — worse than no name at all in the
199
232
  one row that exists to record who did this.
200
233
 
201
- **The site stops at its next build.** It reads `.ksor-denylist.json`, which
202
- `pnpm build` refreshes via `pnpm export-denylist`. So after a takedown, rebuild
203
- and redeploy the site, or the human surface keeps publishing what the agent
204
- surface already refuses. See [deploying.md](./deploying.md) for why that build
205
- needs `KSOR_DB_URL`.
234
+ Two things the actor must satisfy, both refused before any database is touched.
235
+ It must be a well-formed actor — `human:<handle>` or `process:<id>`, never a
236
+ bare name (`ksor-actor-form`) and never a `team:` (a team cannot perform an
237
+ act). And `takedown_authorities` in `.ksor/governance.yaml` must name it
238
+ (`ksor-takedown-unauthorised`); the same check runs over every entry in the
239
+ ledger at `pnpm check`, `ksor build` and ingest, so a line appended by hand in a
240
+ pull request is refused exactly as the verb would refuse it.
241
+
242
+ Lifting a takedown is `--revoke <entry-id>` — the id of the LEDGER ENTRY, not
243
+ the stable id. The denial that created it prints the id, `ksor takedown
244
+ --ledger` lists it, and it is written in `.ksor/takedowns.yaml`; none of the
245
+ three needs a database, because the ledger is a file in the repository. The
246
+ ledger is append-only: a revocation is a new entry, never a deleted line, and a
247
+ build whose ledger shrank against its own git history is refused.
248
+
249
+ **The site stops at its next build.** It reads the committed ledger
250
+ (`.ksor/takedowns.yaml`), so after a takedown, merge the entry, rebuild and
251
+ redeploy the site, or the human surface keeps publishing what the agent
252
+ surface already refuses. The site needs no database for this: the
253
+ ledger is a file in the repository, and the site build reads it.