@panaversity/ksor 0.0.40 → 0.0.42

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (103) hide show
  1. package/CHANGELOG.md +932 -0
  2. package/README.md +11 -7
  3. package/dist/checker/check-main.mjs +14049 -0
  4. package/dist/cli.mjs +11529 -5214
  5. package/dist/gateway-api-CF4ED9_g-BQusM_dK.mjs +10895 -0
  6. package/dist/gateway.d.mts +52 -13
  7. package/dist/gateway.mjs +2 -2
  8. package/dist/index.d.mts +1 -1
  9. package/dist/index.mjs +1 -1
  10. package/dist/{src-pl4aOpVs.mjs → src-dqpI-p1a.mjs} +1 -0
  11. package/docs/authorization.md +8 -6
  12. package/docs/deploying.md +54 -23
  13. package/docs/index.md +26 -13
  14. package/docs/ingesting.md +70 -22
  15. package/docs/tool-surface.md +69 -16
  16. package/package.json +4 -3
  17. package/schema/migrations/2.4-2.5__okf-profile.sql +114 -0
  18. package/schema/schema.sql +77 -14
  19. package/templates/scaffold/.agents/skills/add-sources/SKILL.md +63 -18
  20. package/templates/scaffold/.agents/skills/format-checker/SKILL.md +42 -33
  21. package/templates/scaffold/.agents/skills/format-checker/check.mjs +13827 -1314
  22. package/templates/scaffold/.agents/skills/intake-interview/SKILL.md +65 -27
  23. package/templates/scaffold/.agents/skills/make-slides/SKILL.md +7 -5
  24. package/templates/scaffold/.agents/skills/make-summary/SKILL.md +13 -6
  25. package/templates/scaffold/.claude/skills/add-sources/SKILL.md +63 -18
  26. package/templates/scaffold/.claude/skills/format-checker/SKILL.md +42 -33
  27. package/templates/scaffold/.claude/skills/format-checker/check.mjs +13827 -1314
  28. package/templates/scaffold/.claude/skills/intake-interview/SKILL.md +65 -27
  29. package/templates/scaffold/.claude/skills/make-slides/SKILL.md +7 -5
  30. package/templates/scaffold/.claude/skills/make-summary/SKILL.md +13 -6
  31. package/templates/scaffold/.github/workflows/validate.yml +9 -1
  32. package/templates/scaffold/.ksor/governance.yaml +17 -0
  33. package/templates/scaffold/AGENTS.md +237 -116
  34. package/templates/scaffold/Dockerfile +5 -1
  35. package/templates/scaffold/README.md +160 -42
  36. package/templates/scaffold/env.example +40 -8
  37. package/templates/scaffold/gitignore +13 -8
  38. package/templates/scaffold/instance.md +30 -23
  39. package/templates/scaffold/knowledge/governance-ladder.md +6 -2
  40. package/templates/scaffold/knowledge/index.md +9 -0
  41. package/templates/scaffold/knowledge/surfaces/for-agents.md +7 -6
  42. package/templates/scaffold/knowledge/surfaces/for-people.md +7 -6
  43. package/templates/scaffold/knowledge/surfaces/index.md +4 -20
  44. package/templates/scaffold/knowledge/surfaces/overview.md +25 -0
  45. package/templates/scaffold/knowledge/what-is-a-ksor.md +6 -5
  46. package/templates/scaffold/knowledge/what-is-a-ksor.summary.md +4 -0
  47. package/templates/scaffold/package.json +3 -4
  48. package/templates/scaffold/pnpm-lock.yaml +3 -0
  49. package/templates/scaffold/system/gateways/content.ts +13 -0
  50. package/templates/scaffold/system/site/app/(home)/page.tsx +2 -2
  51. package/templates/scaffold/system/site/app/.well-known/mcp/server.json/route.ts +10 -0
  52. package/templates/scaffold/system/site/app/docs/[[...slug]]/page.tsx +126 -91
  53. package/templates/scaffold/system/site/app/global.css +13 -5
  54. package/templates/scaffold/system/site/app/layout.tsx +8 -3
  55. package/templates/scaffold/system/site/app/llms-full.txt/route.ts +13 -7
  56. package/templates/scaffold/system/site/app/llms.txt/route.ts +12 -7
  57. package/templates/scaffold/system/site/app/md/[[...slug]]/route.ts +26 -25
  58. package/templates/scaffold/system/site/components/footer-mark.tsx +3 -2
  59. package/templates/scaffold/system/site/components/governance.tsx +205 -87
  60. package/templates/scaffold/system/site/components/record-index.tsx +5 -5
  61. package/templates/scaffold/system/site/components/record-stack.tsx +10 -9
  62. package/templates/scaffold/system/site/components/sidebar-status.tsx +19 -18
  63. package/templates/scaffold/system/site/lib/attachment-rule.ts +6 -1
  64. package/templates/scaffold/system/site/lib/attachments.ts +0 -28
  65. package/templates/scaffold/system/site/lib/audience-rule.ts +15 -21
  66. package/templates/scaffold/system/site/lib/audience.ts +42 -146
  67. package/templates/scaffold/system/site/lib/embed-rule.ts +9 -0
  68. package/templates/scaffold/system/site/lib/governance.ts +339 -225
  69. package/templates/scaffold/system/site/lib/index-routes.ts +125 -0
  70. package/templates/scaffold/system/site/lib/lifecycle-rule.ts +52 -0
  71. package/templates/scaffold/system/site/lib/lock.ts +282 -0
  72. package/templates/scaffold/system/site/lib/order-rule.ts +37 -0
  73. package/templates/scaffold/system/site/lib/record-href.ts +68 -0
  74. package/templates/scaffold/system/site/lib/record-link.tsx +26 -0
  75. package/templates/scaffold/system/site/lib/rules-version.ts +11 -0
  76. package/templates/scaffold/system/site/lib/shared.ts +67 -104
  77. package/templates/scaffold/system/site/lib/sim-rule.ts +49 -0
  78. package/templates/scaffold/system/site/lib/source.ts +256 -186
  79. package/templates/scaffold/system/site/lib/stage-knowledge.ts +566 -492
  80. package/templates/scaffold/system/site/lib/stage-manifest.ts +128 -0
  81. package/templates/scaffold/system/site/package.json +1 -0
  82. package/templates/scaffold/system/site/record/actor.ts +23 -0
  83. package/templates/scaffold/system/site/record/check.ts +571 -0
  84. package/templates/scaffold/system/site/record/citations.ts +312 -0
  85. package/templates/scaffold/system/site/record/frontmatter.ts +134 -0
  86. package/templates/scaffold/system/site/record/git-ledger.ts +171 -0
  87. package/templates/scaffold/system/site/record/hygiene.ts +320 -0
  88. package/templates/scaffold/system/site/record/index-file.ts +150 -0
  89. package/templates/scaffold/system/site/record/index.ts +103 -0
  90. package/templates/scaffold/system/site/record/instance.ts +257 -0
  91. package/templates/scaffold/system/site/record/instant.ts +43 -0
  92. package/templates/scaffold/system/site/record/ledger.ts +694 -0
  93. package/templates/scaffold/system/site/record/load.ts +129 -0
  94. package/templates/scaffold/system/site/record/lock.ts +306 -0
  95. package/templates/scaffold/system/site/record/near-miss.ts +37 -0
  96. package/templates/scaffold/system/site/record/policy.ts +414 -0
  97. package/templates/scaffold/system/site/record/profile.ts +535 -0
  98. package/templates/scaffold/system/site/record/refusal.ts +106 -0
  99. package/templates/scaffold/system/site/record/yaml-file.ts +103 -0
  100. package/templates/scaffold/system/site/source.config.ts +77 -22
  101. package/dist/gateway-api-CmIthmJS-IUA9qS-T.mjs +0 -3225
  102. package/templates/scaffold/system/site/lib/denial-rule.ts +0 -220
  103. package/templates/scaffold/system/site/lib/page-order.ts +0 -93
@@ -16,26 +16,57 @@ An agent pays for this surface out of its context window, and it pays twice.
16
16
  Every tool's name, description and input schema is resident for the whole
17
17
  session; every answer spends more.
18
18
 
19
- Measured against a live 81-document record (6,963 chunks), ~4 chars/token:
19
+ **Definitions**, re-measured 2026-08-25 from the served `tools/list` of the
20
+ default registration (~4 chars/token). These depend only on the code, so the
21
+ numbers are exact for every record:
20
22
 
21
23
  | | chars | ~tokens | |
22
24
  | ------------------------------ | ------ | ------- | ------------------- |
23
- | all three tool definitions | 11,960 | 2,990 | **always resident** |
24
- | `search` alone | 5,383 | 1,346 | always resident |
25
- | `outline` + `read` | 6,571 | 1,643 | always resident |
26
- | one `search`, `k=10` (default) | 14,164 | 3,541 | per call |
27
- | one `search`, `k=5` | 8,009 | 2,002 | per call |
28
-
29
- An agent with five records attached carries ~15,000 tokens of definitions before
25
+ | all three, as transmitted | 16,734 | 4,184 | **always resident** |
26
+ | `search` alone | 7,932 | 1,983 | always resident |
27
+ | `outline` alone | 3,332 | 833 | always resident |
28
+ | `read` alone | 5,466 | 1,367 | always resident |
29
+ | `outline` + `read`, if deleted | 8,798 | 2,200 | the saving below |
30
+
31
+ **Two measurements, so read the first row apart from the rest.** Each tool's
32
+ row is the JSON of that tool's own object; the first row is the JSON of the
33
+ whole `tools` array, which carries four characters no tool's row does — its
34
+ two brackets and the two separators between three tools. So the three tools
35
+ sum to **16,730** and the array is **16,734**. Deleting a tool saves that
36
+ tool's own row, not a share of the total.
37
+
38
+ They grew: `search` was 5,383 chars before the trust floor and the governance
39
+ each hit now carries, and `read` 3,396 before it carried the same governance
40
+ block beside the frontmatter. That is the price of an agent being able to tell
41
+ a reviewed document from an unreviewed one, and it is charged once per session.
42
+ The last 520 of them are the price of that signal being HONEST: `trust_tier` is
43
+ derived from reviews a document declares about itself, which no authority list
44
+ gates, and both floors now say so rather than letting `human-reviewed` read as a
45
+ check the record performed.
46
+
47
+ **Replies** depend on your record's passages, so these are the 2026-08-23
48
+ measurement against a live 81-document record (6,963 chunks), plus the
49
+ governance block every hit — and every `read` reply — now carries: measured
50
+ exactly at **262 chars** for a document with a verification and an approval,
51
+ **133** where a level-0 record has neither. They were NOT re-measured against
52
+ that record:
53
+
54
+ | | ~chars | ~tokens | |
55
+ | ------------------------------ | ------ | ------- | -------- |
56
+ | one `search`, `k=10` (default) | 16,784 | 4,196 | per call |
57
+ | one `search`, `k=5` | 9,319 | 2,330 | per call |
58
+ | one `search`, `k=3` | 4,939 | 1,235 | per call |
59
+
60
+ An agent with five records attached carries ~20,000 tokens of definitions before
30
61
  doing any work.
31
62
 
32
63
  ## The three edits that pay
33
64
 
34
65
  ### 1. Delete a tool nothing calls
35
66
 
36
- The biggest win, and the easiest — delete its `registerTool` block. Measured
37
- live: a registration keeping only a renamed `search` served **5,337** bytes
38
- against the default's 11,960.
67
+ The biggest win, and the easiest — delete its `registerTool` block. Dropping
68
+ `outline` and `read` takes **8,798 chars (~2,200 tokens)** off every session,
69
+ whether or not the agent would ever have called them.
39
70
 
40
71
  ### 2. Say what the record covers
41
72
 
@@ -51,18 +82,37 @@ Your prose goes **above** `FLOOR.search`, never instead of it — see below.
51
82
 
52
83
  ### 3. Set `k`
53
84
 
54
- `k` is the lever on reply size: 10 costs ~3,500 tokens a call, 5 costs ~2,000.
85
+ `k` is the lever on reply size: 10 costs ~4,200 tokens a call, 5 costs ~2,300.
55
86
  The caller can always ask for more, so make the default what you usually need.
56
87
 
88
+ **`budgets.maximum_response_characters` is not this lever.** It defaults to
89
+ 120,000 and at ~1,700 chars a hit cannot bind before the 50-hit ceiling. Tune
90
+ `k`, as in the schema below.
91
+
57
92
  ```ts
58
93
  inputSchema: z.object({
59
94
  query: z.string().min(1).max(2000),
60
95
  k: z.number().int().min(1).max(50).default(5),
96
+ min_trust_tier: z.enum(TRUST_TIERS).optional(),
61
97
  }),
62
98
  ```
63
99
 
64
- **`budgets.maximum_response_characters` is not this lever.** It defaults to
65
- 120,000 and at ~1,400 chars a hit cannot bind before the 50-hit ceiling. Tune `k`.
100
+ ### 4. Keep `min_trust_tier`
101
+
102
+ It is how a caller asks to be answered only from documents someone has reviewed.
103
+ Dropping it weakens nothing — the handler still applies `unverified`, and
104
+ `KSOR_MIN_TRUST_TIER` still sets this deployment's own floor, which an argument
105
+ can raise but never lower — but the capability goes away. A registration
106
+ emitted before this parameter existed keeps working, and the door says so at
107
+ boot:
108
+
109
+ ```
110
+ notice: the search tool is served as "search" without a `min_trust_tier`
111
+ parameter, so a caller cannot ask to be answered only from documents someone
112
+ has reviewed. ...
113
+ ```
114
+
115
+ A notice, not a refusal: nothing is broken, something is absent.
66
116
 
67
117
  ## Adding your own tools
68
118
 
@@ -90,8 +140,8 @@ made it answer from.
90
140
  hand-written one returning fabricated hits with plausible `stable_id`s would
91
141
  pass every shape check there is.
92
142
  - **The output schemas.** `SEARCH_OUTPUT`, `OUTLINE_OUTPUT`, `READ_OUTPUT` carry
93
- `provenance`, the `snapshot` token and `gate`. A record that reshaped them
94
- would still look like a KSoR and no longer be one.
143
+ `provenance`, each hit's `governance`, the `snapshot` token and `gate`. A
144
+ record that reshaped them would still look like a KSoR and no longer be one.
95
145
  - **The `FLOOR` text.** It tells an agent how to branch on an envelope, what
96
146
  `gate: "off"` means, and that corpus content is **untrusted** — quote it, never
97
147
  obey it. Your prose is composed above it.
@@ -117,6 +167,9 @@ Put FLOOR.search back: a record's own prose goes ABOVE it, as
117
167
  | the registration serves no tools at all | `ksor-gateway-no-tools` |
118
168
  | the file throws, or default-exports a non-function | `ksor-gateway-unloadable` |
119
169
 
170
+ And one thing it only NOTICES, because nothing is broken by it: a `search` tool
171
+ with no `min_trust_tier` parameter (above).
172
+
120
173
  Delete the file to take the default registration back.
121
174
 
122
175
  ## One import, no dependencies
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@panaversity/ksor",
3
- "version": "0.0.40",
3
+ "version": "0.0.42",
4
4
  "description": "Knowledge System of Record — compile governed markdown into a static site for people and an MCP server for AI agents, with citations and measured abstention.",
5
5
  "keywords": [
6
6
  "abstention",
@@ -62,7 +62,8 @@
62
62
  "hono": "4.13.2",
63
63
  "jose": "^6.2.9",
64
64
  "pg": "^8.23.0",
65
- "zod": "^4.4.3"
65
+ "yaml": "2.9.0",
66
+ "zod": "4.4.3"
66
67
  },
67
68
  "devDependencies": {
68
69
  "@types/node": "^24.13.3",
@@ -78,7 +79,7 @@
78
79
  "node": ">=24"
79
80
  },
80
81
  "scripts": {
81
- "build": "tsdown && node copy-schema.mjs",
82
+ "build": "tsdown && node copy-schema.mjs && node emit-checker.mjs",
82
83
  "typecheck": "tsc -p tsconfig.json",
83
84
  "publint": "publint"
84
85
  }
@@ -0,0 +1,114 @@
1
+ -- 2.4 -> 2.5 · The record is the KSoR Profile of OKF (research/okf-native.md
2
+ -- §4.1; specs/ksor/record/spec.md; decision 26).
3
+ --
4
+ -- Three things change shape, and each is mapped rather than dropped:
5
+ --
6
+ -- audience The ranked, single-valued `visibility` becomes a LIST with
7
+ -- overlap semantics (record spec §2.4). A carried row keeps
8
+ -- exactly the tier it declared — ARRAY[visibility] — which
9
+ -- under the old model meant "this tier and above" and under
10
+ -- the new one means "this identifier only". That narrowing is
11
+ -- deliberate: `ksor migrate` widens the FILE to every tier at or
12
+ -- above, and a re-ingest carries the widened list here. Until
13
+ -- then a pre-2.5 generation refuses to serve (GOVERNANCE_SINCE),
14
+ -- so no viewer is answered from a half-mapped row.
15
+ -- doc_status approved -> stable, review -> draft, superseded -> deprecated;
16
+ -- anything else the author wrote is not a profile status and
17
+ -- becomes NULL (the checker refuses it on the next ingest). The
18
+ -- CHECK then closes the set, because the lifecycle predicate
19
+ -- keys on it.
20
+ -- trust_tier 0 unverified · 1 machine-confirmed · 2 human-reviewed. A
21
+ -- carried row has no `verified` list, so it is 0: the honest
22
+ -- state of a stable, approved, unverified concept (plan §2.13).
23
+ --
24
+ -- The run row gains the build it published (`build_id`), the policy it was
25
+ -- checked against (registry + authorities, and its digest) and the ledger's id
26
+ -- set; the denylist row gains the ledger entry that wrote it and, nullable,
27
+ -- the one that revoked it — the `DENIED` seam denies only `revoked_at IS NULL`.
28
+ --
29
+ -- compatible_from: 2.5
30
+
31
+ ALTER TABLE content_nodes
32
+ ADD COLUMN IF NOT EXISTS audience TEXT[],
33
+ ADD COLUMN IF NOT EXISTS sources JSONB,
34
+ ADD COLUMN IF NOT EXISTS verified JSONB,
35
+ ADD COLUMN IF NOT EXISTS generated JSONB,
36
+ ADD COLUMN IF NOT EXISTS approval JSONB,
37
+ ADD COLUMN IF NOT EXISTS deprecated JSONB,
38
+ ADD COLUMN IF NOT EXISTS effective_from TIMESTAMPTZ,
39
+ ADD COLUMN IF NOT EXISTS stale_after TIMESTAMPTZ,
40
+ ADD COLUMN IF NOT EXISTS trust_tier SMALLINT;
41
+
42
+ UPDATE content_nodes
43
+ SET audience = ARRAY[visibility]
44
+ WHERE visibility IS NOT NULL AND visibility <> '' AND audience IS NULL;
45
+
46
+ UPDATE content_nodes
47
+ SET doc_status = CASE doc_status
48
+ WHEN 'approved' THEN 'stable'
49
+ WHEN 'review' THEN 'draft'
50
+ WHEN 'superseded' THEN 'deprecated'
51
+ WHEN 'draft' THEN 'draft'
52
+ WHEN 'stable' THEN 'stable'
53
+ WHEN 'deprecated' THEN 'deprecated'
54
+ ELSE NULL
55
+ END
56
+ WHERE doc_status IS NOT NULL;
57
+
58
+ UPDATE content_nodes SET trust_tier = 0 WHERE kind = 'document' AND trust_tier IS NULL;
59
+
60
+ ALTER TABLE content_nodes DROP COLUMN IF EXISTS visibility;
61
+ ALTER TABLE content_nodes
62
+ ADD CONSTRAINT nodes_doc_status_profile
63
+ CHECK (doc_status IS NULL OR doc_status IN ('draft','stable','deprecated')),
64
+ ADD CONSTRAINT nodes_trust_tier_range
65
+ CHECK (trust_tier IS NULL OR trust_tier BETWEEN 0 AND 2);
66
+ CREATE INDEX IF NOT EXISTS idx_nodes_audience ON content_nodes USING gin (audience);
67
+
68
+ COMMENT ON COLUMN content_nodes.audience IS
69
+ 'ksor.audience, a list; a section carries the union of its descendants'' lists so `audience && viewer` admits it iff a descendant is visible. NULL on a pre-2.5 row that declared no visibility.';
70
+ COMMENT ON COLUMN content_nodes.trust_tier IS
71
+ '0 unverified, 1 machine-confirmed, 2 human-reviewed — derived from `verified` at ingest, never authored.';
72
+
73
+ ALTER TABLE ingestion_runs
74
+ ADD COLUMN IF NOT EXISTS build_id TEXT,
75
+ ADD COLUMN IF NOT EXISTS policy JSONB,
76
+ ADD COLUMN IF NOT EXISTS policy_sha256 TEXT,
77
+ ADD COLUMN IF NOT EXISTS ledger_ids TEXT[];
78
+
79
+ COMMENT ON COLUMN ingestion_runs.build_id IS
80
+ 'build.lock.json''s build_id — the publication this generation was ingested from; NULL on a pre-2.5 run.';
81
+ COMMENT ON COLUMN ingestion_runs.policy IS
82
+ 'The Governance Policy the generation was checked against: {audiences, approval_authorities, takedown_authorities, ownership}. The door binds to this row, never to the file.';
83
+ COMMENT ON COLUMN ingestion_runs.ledger_ids IS
84
+ 'Every id in .ksor/takedowns.yaml at ingest, in file order — the baseline ksor-ledger-shrank compares the next ingest against.';
85
+
86
+ ALTER TABLE takedown_denylist
87
+ ADD COLUMN IF NOT EXISTS ledger_id TEXT,
88
+ ADD COLUMN IF NOT EXISTS actor TEXT,
89
+ ADD COLUMN IF NOT EXISTS applied_at TIMESTAMPTZ,
90
+ ADD COLUMN IF NOT EXISTS revoked_ledger_id TEXT,
91
+ ADD COLUMN IF NOT EXISTS revoked_at TIMESTAMPTZ,
92
+ -- A carried row predates the ledger entirely, so `present` is the only
93
+ -- honest default: nothing in the repository yet says the file was deleted.
94
+ -- The first ingest folds the ledger and writes what it actually says.
95
+ ADD COLUMN IF NOT EXISTS expected TEXT NOT NULL DEFAULT 'present';
96
+ ALTER TABLE takedown_denylist
97
+ ADD CONSTRAINT takedown_expected_values CHECK (expected IN ('present','removed'));
98
+
99
+ COMMENT ON COLUMN takedown_denylist.ledger_id IS
100
+ 'The .ksor/takedowns.yaml entry that wrote this row. NULL = written before the ledger existed; the boot gate refuses it (ksor-takedown-unledgered) until an ingest attaches one by stable_id.';
101
+ COMMENT ON COLUMN takedown_denylist.expected IS
102
+ 'What the ledger expects of the FILE, as the latest amendment left it: present, or removed for a document withdrawn and then deliberately deleted. The orphan check skips `removed`; the DENIED seam never reads it, so a removed document stays denied.';
103
+ COMMENT ON COLUMN takedown_denylist.revoked_at IS
104
+ 'Set by a revocation entry; the DENIED seam denies only rows where this is NULL. A re-denial clears it — the ledger holds the history, the row holds the state.';
105
+
106
+ -- The author's own frontmatter bytes, so `read` can return them intact rather
107
+ -- than re-serialising the parsed columns into a document the record does not
108
+ -- contain. Additive and nullable: a carried row simply has none, and such a
109
+ -- generation is refused at boot anyway (GOVERNANCE_SINCE).
110
+ ALTER TABLE sources
111
+ ADD COLUMN IF NOT EXISTS frontmatter TEXT;
112
+
113
+ COMMENT ON COLUMN sources.frontmatter IS
114
+ 'The file''s frontmatter block, byte-exact as authored (comments and unknown keys included). Served verbatim by `read`; never re-serialised.';
package/schema/schema.sql CHANGED
@@ -49,6 +49,15 @@ CREATE TABLE ingestion_runs (
49
49
  -- record with an audience model must refuse to serve such a generation
50
50
  -- rather than quietly publish restricted documents (2.4).
51
51
  schema_version TEXT,
52
+ -- 2.5: the publication this generation was ingested from (build.lock.json's
53
+ -- build_id), the Governance Policy it was checked against — registry and
54
+ -- authorities as a row, with its digest, so the door binds to the row and
55
+ -- the served container never needs the file — and the ledger's id set, the
56
+ -- baseline the next ingest's ksor-ledger-shrank compares against.
57
+ build_id TEXT,
58
+ policy JSONB,
59
+ policy_sha256 TEXT,
60
+ ledger_ids TEXT[],
52
61
  UNIQUE (tenant_id, corpus_id, generation)
53
62
  );
54
63
 
@@ -71,11 +80,33 @@ CREATE TABLE content_nodes (
71
80
  -- surface reads one source instead of re-deriving it from markdown. `status`
72
81
  -- above is the SERVING state of the row; `doc_status` is what the document says.
73
82
  corpus_id TEXT, -- which record this node belongs to
74
- visibility TEXT, -- audience tier; NULL = instance default_visibility
75
- doc_status TEXT, -- draft / approved / superseded, as authored
83
+ -- The KSoR Profile of OKF (2.5, record spec §2): `ksor.audience` as a LIST
84
+ -- with overlap semantics a section carries the union of its descendants'
85
+ -- lists, so `audience && viewer` admits it iff a descendant is visible;
86
+ -- the authored lifecycle status, closed to the profile's set; the trust
87
+ -- vocabulary as JSONB; effectivity and staleness as instants; and the trust
88
+ -- tier derived from `verified` at ingest (0 unverified, 1 machine-confirmed,
89
+ -- 2 human-reviewed). `provenance` (2.2) stays for a carried row; a 2.5
90
+ -- ingest writes `sources`.
91
+ audience TEXT[],
92
+ -- Named, not auto-named: the 2.4 -> 2.5 migration adds this same CHECK by
93
+ -- name, and an unnamed inline constraint is auto-named after the column, so a
94
+ -- migrated database and a fresh one carried the SAME rule under two different
95
+ -- names — one schema nobody could diff (schema-parity.db.test.ts).
96
+ doc_status TEXT CONSTRAINT nodes_doc_status_profile
97
+ CHECK (doc_status IS NULL OR doc_status IN ('draft','stable','deprecated')),
76
98
  owner TEXT,
77
- provenance JSONB, -- where the claims come from, as authored
78
- superseded_by TEXT, -- stable_id of the replacement
99
+ provenance JSONB, -- pre-profile `provenance:`, carried rows only
100
+ superseded_by TEXT, -- ksor.superseded_by, as a stable_id
101
+ sources JSONB,
102
+ verified JSONB,
103
+ generated JSONB,
104
+ approval JSONB,
105
+ deprecated JSONB,
106
+ effective_from TIMESTAMPTZ,
107
+ stale_after TIMESTAMPTZ,
108
+ trust_tier SMALLINT CONSTRAINT nodes_trust_tier_range
109
+ CHECK (trust_tier IS NULL OR trust_tier BETWEEN 0 AND 2),
79
110
  created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
80
111
  updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
81
112
  CONSTRAINT nodes_stable_uniq UNIQUE (tenant_id, generation, stable_id),
@@ -88,12 +119,10 @@ CREATE TABLE content_nodes (
88
119
  CREATE INDEX idx_nodes_gen ON content_nodes (tenant_id, generation, kind);
89
120
  CREATE INDEX idx_nodes_parent ON content_nodes (parent_id);
90
121
  CREATE INDEX idx_nodes_keywords ON content_nodes USING gin (keywords);
91
- -- NO index on visibility. The serving predicate is
92
- -- `coalesce(n.visibility, <runtime GUC>) = ANY(...)`, and a plain btree cannot
93
- -- serve a coalesce over a value that is only known per transaction — the index
94
- -- would be built and maintained and never read, which is exactly the defect
95
- -- the HNSW arm was just fixed for. The audience filter rides the
96
- -- (tenant_id, generation, kind) index that every serving arm already uses.
122
+ -- The overlap predicate (`audience && viewer`) is an array-overlap, which a GIN
123
+ -- serves and a btree cannot; the ranked predicate it replaces rode the
124
+ -- (tenant_id, generation, kind) index through a coalesce no index could read.
125
+ CREATE INDEX idx_nodes_audience ON content_nodes USING gin (audience);
97
126
  CREATE UNIQUE INDEX nodes_root_slug_uniq ON content_nodes (tenant_id, generation, slug) WHERE parent_id IS NULL;
98
127
 
99
128
  CREATE TABLE slug_aliases (
@@ -117,6 +146,12 @@ CREATE TABLE sources (
117
146
  embedding_model TEXT NOT NULL,
118
147
  chunk_policy TEXT NOT NULL,
119
148
  source_commit TEXT,
149
+ -- 2.5: the file's frontmatter block, byte-exact as the author wrote it —
150
+ -- comments and unknown keys included (OKF §11). `read` serves it back
151
+ -- verbatim; a re-serialisation from the parsed columns would be a
152
+ -- DIFFERENT document wearing the record's name. NULL for a source with no
153
+ -- frontmatter, and for a pre-2.5 carried row.
154
+ frontmatter TEXT,
120
155
  seeded_at TIMESTAMPTZ NOT NULL DEFAULT now(),
121
156
  updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
122
157
  PRIMARY KEY (tenant_id, generation, source_id),
@@ -187,6 +222,31 @@ CREATE TABLE takedown_denylist (
187
222
  scope TEXT NOT NULL DEFAULT 'node' CHECK (scope IN ('node','subtree')),
188
223
  reason TEXT NOT NULL,
189
224
  created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
225
+ -- 2.5: the row is the STATE, the ledger (.ksor/takedowns.yaml) is the
226
+ -- history. `ledger_id` names the entry that wrote the row (NULL = written
227
+ -- before the ledger existed — the boot gate refuses it until an ingest
228
+ -- attaches one by stable_id); a revocation sets `revoked_ledger_id` /
229
+ -- `revoked_at`, and the DENIED seam denies only `revoked_at IS NULL`; a
230
+ -- re-denial clears them. Rows are never deleted.
231
+ ledger_id TEXT,
232
+ actor TEXT,
233
+ applied_at TIMESTAMPTZ,
234
+ revoked_ledger_id TEXT,
235
+ revoked_at TIMESTAMPTZ,
236
+ -- What the ledger expects of the FILE, as the latest amendment left it
237
+ -- (record spec §5). `present` is the ordinary case. `removed` says the
238
+ -- document was withdrawn and then deliberately deleted — a state the boot
239
+ -- gate must not read as an orphaned denial, because the honest answer to
240
+ -- "is there still a node with this id?" is no, forever. Without this column
241
+ -- every such denial refused `ksor ingest` and `ksor serve` permanently, and
242
+ -- the remedy the refusal printed (`--removed`) moves no row, so the only
243
+ -- escape was to un-withdraw the document (review 2026-08-25).
244
+ --
245
+ -- It does NOT weaken the denial: `removed` rows stay in force and the
246
+ -- DENIED seam never reads this column. It records what happened to the
247
+ -- FILE, not whether the withdrawal still stands.
248
+ expected TEXT NOT NULL DEFAULT 'present'
249
+ CONSTRAINT takedown_expected_values CHECK (expected IN ('present','removed')),
190
250
  PRIMARY KEY (tenant_id, corpus_id, stable_id)
191
251
  );
192
252
 
@@ -227,10 +287,13 @@ CREATE TABLE schema_meta (
227
287
  -- 2.4 stamps each generation with the schema it was built against, so a
228
288
  -- generation predating the governance columns can be REFUSED rather than served
229
289
  -- at default_visibility.
230
- -- Both are additive and nullable, so a 2.0 reader still reads a 2.2 database —
231
- -- compatible_from stays 2.0. Existing databases move forward through
232
- -- schema/migrations/; schema.sql provisions a FRESH one at the current version.
233
- INSERT INTO schema_meta (schema_version, compatible_from) VALUES ('2.4', '2.0');
290
+ -- 2.5 puts the KSoR Profile on the node row (audience list, closed status set,
291
+ -- trust vocabulary, effectivity, trust tier), the policy and the lock on the
292
+ -- run, and the ledger on the denylist row; it DROPS `visibility`, so a 2.4
293
+ -- reader's predicate no longer resolves — compatible_from moves to 2.5.
294
+ -- Existing databases move forward through schema/migrations/; schema.sql
295
+ -- provisions a FRESH one at the current version.
296
+ INSERT INTO schema_meta (schema_version, compatible_from) VALUES ('2.5', '2.5');
234
297
 
235
298
  CREATE OR REPLACE FUNCTION touch_updated_at() RETURNS trigger AS $$
236
299
  BEGIN NEW.updated_at = now(); RETURN NEW; END; $$ LANGUAGE plpgsql;
@@ -2,7 +2,7 @@
2
2
  name: add-sources
3
3
  description: Turn source material — documents, pages, pasted text, notes — into governed knowledge in knowledge/. Use when the owner shares material to add, says "add this to the knowledge base", or asks how to get existing content in. Not for editing the site.
4
4
  metadata:
5
- version: "1.2.0"
5
+ version: "1.3.0"
6
6
  ---
7
7
 
8
8
  # Add sources
@@ -16,31 +16,76 @@ The rules that make it _governed_ rather than merely stored:
16
16
  hyphens, a folder per natural grouping. Plain CommonMark `.md` — if the
17
17
  source is rich (tables, images), tables become markdown tables and images
18
18
  land _beside the document_ with relative links.
19
- - A folder's front page is `<folder>/index.md`; reading order is the
20
- `order:` frontmatter key (ordered documents first, ascending; the rest
21
- follow alphabetically) never `meta.json` or `sidebar_position`.
22
- - Frontmatter: `title` and `status: draft` always; add `owner` (who stands
23
- behind this content) and `provenance` (a list naming the actual sources —
24
- file names, systems, people, dates) whenever the owner can tell you.
25
- Precision matters: "Finance policy manual §4.2, 2025 edition" governs;
26
- "internal docs" does not.
27
- - When `instance.md` declares `audiences:`, ask the owner which audience the
28
- new material belongs to and write it as `visibility:` never guess that
29
- restricted material is public.
19
+ - A folder's `index.md` is GENERATED by `ksor build` and committed — never
20
+ author one. A folder's own introduction is a named document inside it,
21
+ such as `overview.md`. Reading order is the `order:` frontmatter key
22
+ (ordered documents first, ascending; the rest follow alphabetically)
23
+ never `meta.json` or `sidebar_position`.
24
+ - Frontmatter is the KSoR Profile of OKF. Always: `type: Document`, `title`,
25
+ `description` (one sentence), `status: draft`, and `ksor.audience` (a
26
+ list). Reach for a reserved `type` — `Policy`, `Procedure`, `Control`,
27
+ `Standard`, `Definition`, `Decision Record`, `Example`,
28
+ `Attested Computation` when the knowledge really is one; those additionally require `sources` and
29
+ `ksor.owner`, which is the point of them.
30
+
31
+ ```yaml
32
+ ---
33
+ type: Policy
34
+ title: Purchase approval
35
+ description: Who may approve a purchase, at which thresholds.
36
+ status: draft
37
+ sources:
38
+ - id: fin-2025
39
+ title: Finance policy manual §4.2, 2025 edition
40
+ resource: https://intranet.example.com/finance/manual.pdf
41
+ ksor:
42
+ audience: [public]
43
+ owner: team:finance
44
+ ---
45
+
46
+ A purchase above 10,000 needs a director's signature. [^fin-2025]
47
+
48
+ [^fin-2025]: Finance policy manual §4.2, 2025 edition.
49
+ ```
50
+
51
+ Every source needs a `resource` — a URL where one exists, otherwise the
52
+ scope descriptor that names it ("Finance policy manual §4.2, 2025
53
+ edition"). Precision matters: that governs; "internal docs" does not. A
54
+ claim cites ONE of them by footnote, and the label must match a
55
+ `sources[].id` or the checker refuses it.
56
+
57
+ - `status: draft` always, and never anything else from here: `stable` needs
58
+ `ksor.approval` by an actor `.ksor/governance.yaml` names, and recording
59
+ an approval nobody gave is the one thing this skill must never do. Ask the
60
+ owner; if they approve, THEY are the approver and you write down what they
61
+ said.
62
+ - When `.ksor/governance.yaml` registers audiences, ask the owner which of
63
+ them may read the new material and list every one in `ksor.audience` —
64
+ never guess that restricted material is public, and never write an
65
+ identifier the registry does not declare (the checker refuses it, because
66
+ a typo reads as a restriction).
30
67
 
31
68
  ## Fidelity rules
32
69
 
33
70
  - **Copy load-bearing values exactly** — numbers, thresholds, dates, names.
34
71
  Never round, never paraphrase a figure.
35
- - **Two disagreeing sources stay two statements**, each with its provenance
36
- — never smooth a conflict into one invented truth; flag it to the owner.
72
+ - **Two disagreeing sources stay two statements**, each with its own
73
+ footnote — never smooth a conflict into one invented truth; flag it to the
74
+ owner.
37
75
  - **Do not fill gaps from general knowledge.** If the source doesn't cover
38
76
  something, the record doesn't either — that boundary is the product.
39
- - A document replacing an older one: mark the old one `status: superseded`
40
- with `superseded_by:` pointing at the new never delete it.
77
+ - A document replacing an older one: mark the old one `status: deprecated`
78
+ with `ksor.deprecated: { by, at }` (a takedown authority the policy names, or
79
+ the owner an `ownership:` rule resolves — never the document's own
80
+ `ksor.owner`; ask, never guess) and `ksor.superseded_by:` naming the
81
+ successor by id (`policies/refunds-v2`, no `./` and no `.md`) — never
82
+ delete it. The successor must exist, be `stable`, and be readable by every
83
+ reader of the deprecated one.
41
84
 
42
85
  ## Finish every batch
43
86
 
44
87
  Run `pnpm check` and fix what it reports (its errors explain themselves),
45
- then show the owner the rendered result (`pnpm dev`) the site is the
46
- review surface: you write, they check.
88
+ then `ksor build` to regenerate every folder's `index.md` and write
89
+ `build.lock.json`, and commit both with the documents. Then show the owner
90
+ the rendered result (`pnpm dev`) — the site is the review surface: you write,
91
+ they check.
@@ -2,45 +2,54 @@
2
2
  name: format-checker
3
3
  description: The record's format rules as a runnable check — frontmatter, filenames, links, structure. Use before handing off any change to knowledge/, when a check fails and you need to fix it, or when unsure whether a document is well-formed. Run with `pnpm check` (or node .agents/skills/format-checker/check.mjs).
4
4
  metadata:
5
- version: "1.2.0"
5
+ version: "2.1.0"
6
6
  ---
7
7
 
8
8
  # Format checker
9
9
 
10
- `pnpm check` runs `check.mjs` — dependency-free Node, owned by this repo. It
11
- enforces what AGENTS.md states in prose:
10
+ `pnpm check` runs `check.mjs` — a self-contained Node program that needs no
11
+ install. It is **generated** by ksor from the same rule set `ksor build` and
12
+ `ksor ingest` run, so the three can never disagree about what a well-formed
13
+ record is. Do not edit it: `ksor init` writes it and `ksor migrate` rewrites
14
+ both copies of it when you upgrade the tool, so an edit is overwritten rather
15
+ than kept. It is read-only in the other direction too — it reports, and never
16
+ rewrites a file.
12
17
 
13
- - `knowledge/` holds CommonMark `.md` and images only no `.mdx`, no
14
- `meta.json`, no other file types and it is never empty; a record with no
15
- documents stands behind nothing.
16
- - Every document carries `title` + `status` (level-0 requirement); the full
17
- allowed key set is closed; a `superseded` document names a `superseded_by`
18
- that resolves to a document that exists.
19
- - Filenames are portable identities: lowercase, Windows-safe, no spaces, no
20
- case-collisions, no `foo.md` + `foo/index.md` pairs, no parentheses.
21
- (`.DS_Store` and friends are skipped, never reported.)
22
- - Relative links resolve and never leave `knowledge/` inline,
23
- `<angle-bracketed>`, and reference-style (`[text][label]` with its
24
- `[label]: target` definition) alike. Links inside code spans and fenced
25
- blocks are code, not links, and are ignored.
26
- - `instance.md` exists, is `format: 1`, and carries only the keys the format
27
- defines an unknown key is named, never ignored.
28
- - The audience model, when there is one: `audiences:` is ordered least- to
29
- most-restricted with `public` first, no duplicates, and never without
30
- `default_visibility:`; a document's `visibility:` names one of the declared
31
- audiences; and no link or `superseded_by:` points from a wider audience at a
32
- narrower one the leak no single build can catch, since the build that
33
- publishes the pointer has already dropped its target. A record that declares
34
- no `audiences:` is checked exactly as it was before the key existed.
18
+ If its refusals contradict this document, the checker is older than the record:
19
+ upgrade `@panaversity/ksor` and re-run `ksor migrate`, and never "fix" the
20
+ record by undoing what the migration wrote.
21
+
22
+ What it holds the record to (the full contract is ksor's record spec):
23
+
24
+ - Every document in `knowledge/` is a concept in the KSoR Profile of OKF:
25
+ frontmatter is real YAML carrying `type`, `title`, `description`, `status`
26
+ (`draft | stable | deprecated`) and `ksor.audience` (a list; `public` or
27
+ audiences registered in `.ksor/governance.yaml`). A `stable` concept carries
28
+ `generated` and an approval by an authorised actor; a `deprecated` one says
29
+ who deprecated it and usually names its successor; a reserved type (`Policy`,
30
+ `Procedure`, …) names `sources` and `ksor.owner`. Every timestamp is an
31
+ instant with an offset; every footnote label matches a `sources[].id`.
32
+ - `.ksor/governance.yaml` exists and names approval and takedown authorities;
33
+ `.ksor/takedowns.yaml` is append-only, every entry by a takedown authority,
34
+ and never names a concept that no longer exists.
35
+ - `index.md` files are generated by `ksor build`, never authored a stale or
36
+ missing one is refused (`ksor-index-stale`; run `ksor build`). `log.md` and
37
+ `README.md` are reserved names.
38
+ - Filenames are portable identities: lowercase ascii, no spaces, no
39
+ case-collisions, no `x.md` beside `x/`, no parentheses or leading
40
+ underscore. `knowledge/` holds `.md`, companions (`<doc>.summary.md`,
41
+ `<doc>.{flashcards,quiz,slides}.yaml`) and images only.
42
+ - Links resolve inside `knowledge/` — inline, `<angle-bracketed>` and
43
+ reference-style alike; code is never a link — and never reach a document
44
+ that not every reader of the source may read (the widening rule).
45
+ - `instance.md` is `format: 2` with the closed key set the profile defines.
35
46
  - `CLAUDE.md` stays a one-line pointer; `.agents/skills/` and
36
- `.claude/skills/` hold the same files byte for byte **in both directions**
37
- (a file only one tree carries is a rule nobody reviewed); the site contains
38
- no content files.
47
+ `.claude/skills/` hold the same files byte for byte in both directions; the
48
+ site contains no content files.
39
49
 
40
- Every failure prints what is wrong, why the rule exists, and the fix — obey
41
- the printed fix literally; if it doesn't resolve the problem, that is a bug
42
- worth reporting to ksor.
50
+ Every failure prints where, the rule's slug (`problem: ksor-…`), why the rule
51
+ exists, and the fix — obey the printed fix literally; if it doesn't resolve
52
+ the problem, that is a bug worth reporting to ksor.
43
53
 
44
54
  When you edit any skill under `.agents/skills/`, re-copy it to
45
- `.claude/skills/` — the checker holds the two trees identical, and it now
46
- notices a file added on either side.
55
+ `.claude/skills/` — the checker holds the two trees identical.