@panaversity/ksor 0.0.39 → 0.0.41
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +965 -0
- package/README.md +29 -9
- package/dist/checker/check-main.mjs +14049 -0
- package/dist/cli.mjs +11528 -5213
- package/dist/gateway-api-CF4ED9_g-BQusM_dK.mjs +10895 -0
- package/dist/gateway.d.mts +52 -13
- package/dist/gateway.mjs +2 -2
- package/dist/index.d.mts +1 -1
- package/dist/index.mjs +1 -1
- package/dist/{src-pl4aOpVs.mjs → src-dqpI-p1a.mjs} +1 -0
- package/docs/authorization.md +8 -6
- package/docs/deploying.md +36 -25
- package/docs/index.md +26 -13
- package/docs/ingesting.md +70 -22
- package/docs/tool-surface.md +69 -16
- package/package.json +4 -3
- package/schema/migrations/2.4-2.5__okf-profile.sql +114 -0
- package/schema/schema.sql +77 -14
- package/templates/scaffold/.agents/skills/add-sources/SKILL.md +63 -18
- package/templates/scaffold/.agents/skills/format-checker/SKILL.md +42 -33
- package/templates/scaffold/.agents/skills/format-checker/check.mjs +13827 -1314
- package/templates/scaffold/.agents/skills/intake-interview/SKILL.md +65 -27
- package/templates/scaffold/.agents/skills/make-slides/SKILL.md +7 -5
- package/templates/scaffold/.agents/skills/make-summary/SKILL.md +13 -6
- package/templates/scaffold/.claude/skills/add-sources/SKILL.md +63 -18
- package/templates/scaffold/.claude/skills/format-checker/SKILL.md +42 -33
- package/templates/scaffold/.claude/skills/format-checker/check.mjs +13827 -1314
- package/templates/scaffold/.claude/skills/intake-interview/SKILL.md +65 -27
- package/templates/scaffold/.claude/skills/make-slides/SKILL.md +7 -5
- package/templates/scaffold/.claude/skills/make-summary/SKILL.md +13 -6
- package/templates/scaffold/.github/workflows/validate.yml +9 -1
- package/templates/scaffold/.ksor/governance.yaml +17 -0
- package/templates/scaffold/AGENTS.md +293 -119
- package/templates/scaffold/Dockerfile +5 -1
- package/templates/scaffold/README.md +160 -42
- package/templates/scaffold/env.example +37 -6
- package/templates/scaffold/gitignore +14 -6
- package/templates/scaffold/instance.md +21 -17
- package/templates/scaffold/knowledge/governance-ladder.md +11 -2
- package/templates/scaffold/knowledge/index.md +9 -0
- package/templates/scaffold/knowledge/surfaces/for-agents.md +13 -6
- package/templates/scaffold/knowledge/surfaces/for-people.md +7 -6
- package/templates/scaffold/knowledge/surfaces/index.md +4 -20
- package/templates/scaffold/knowledge/surfaces/overview.md +25 -0
- package/templates/scaffold/knowledge/what-is-a-ksor.md +6 -5
- package/templates/scaffold/knowledge/what-is-a-ksor.summary.md +4 -0
- package/templates/scaffold/package.json +3 -4
- package/templates/scaffold/pnpm-lock.yaml +3 -0
- package/templates/scaffold/system/gateways/content.ts +13 -0
- package/templates/scaffold/system/site/app/(home)/page.tsx +2 -2
- package/templates/scaffold/system/site/app/.well-known/mcp/server.json/route.ts +10 -0
- package/templates/scaffold/system/site/app/docs/[[...slug]]/page.tsx +134 -91
- package/templates/scaffold/system/site/app/global.css +198 -6
- package/templates/scaffold/system/site/app/layout.tsx +8 -3
- package/templates/scaffold/system/site/app/llms-full.txt/route.ts +13 -7
- package/templates/scaffold/system/site/app/llms.txt/route.ts +12 -7
- package/templates/scaffold/system/site/app/md/[[...slug]]/route.ts +26 -25
- package/templates/scaffold/system/site/components/code-block.tsx +87 -0
- package/templates/scaffold/system/site/components/embed.tsx +276 -0
- package/templates/scaffold/system/site/components/footer-mark.tsx +3 -2
- package/templates/scaffold/system/site/components/governance.tsx +219 -87
- package/templates/scaffold/system/site/components/mdx.tsx +10 -2
- package/templates/scaffold/system/site/components/record-index.tsx +5 -5
- package/templates/scaffold/system/site/components/record-stack.tsx +10 -9
- package/templates/scaffold/system/site/components/record-views.tsx +6 -17
- package/templates/scaffold/system/site/components/sidebar-status.tsx +19 -18
- package/templates/scaffold/system/site/lib/alert-rule.ts +214 -0
- package/templates/scaffold/system/site/lib/attachment-rule.ts +6 -1
- package/templates/scaffold/system/site/lib/attachments.ts +0 -28
- package/templates/scaffold/system/site/lib/audience-rule.ts +15 -21
- package/templates/scaffold/system/site/lib/audience.ts +42 -146
- package/templates/scaffold/system/site/lib/embed-rule.ts +255 -0
- package/templates/scaffold/system/site/lib/governance.ts +339 -225
- package/templates/scaffold/system/site/lib/index-routes.ts +125 -0
- package/templates/scaffold/system/site/lib/lifecycle-rule.ts +52 -0
- package/templates/scaffold/system/site/lib/lock.ts +282 -0
- package/templates/scaffold/system/site/lib/order-rule.ts +37 -0
- package/templates/scaffold/system/site/lib/record-href.ts +68 -0
- package/templates/scaffold/system/site/lib/record-link.tsx +26 -0
- package/templates/scaffold/system/site/lib/rules-version.ts +11 -0
- package/templates/scaffold/system/site/lib/shared.ts +67 -104
- package/templates/scaffold/system/site/lib/sim-rule.ts +49 -0
- package/templates/scaffold/system/site/lib/source.ts +256 -186
- package/templates/scaffold/system/site/lib/stage-knowledge.ts +638 -467
- package/templates/scaffold/system/site/lib/stage-manifest.ts +128 -0
- package/templates/scaffold/system/site/package.json +1 -0
- package/templates/scaffold/system/site/record/actor.ts +23 -0
- package/templates/scaffold/system/site/record/check.ts +571 -0
- package/templates/scaffold/system/site/record/citations.ts +312 -0
- package/templates/scaffold/system/site/record/frontmatter.ts +134 -0
- package/templates/scaffold/system/site/record/git-ledger.ts +171 -0
- package/templates/scaffold/system/site/record/hygiene.ts +320 -0
- package/templates/scaffold/system/site/record/index-file.ts +150 -0
- package/templates/scaffold/system/site/record/index.ts +103 -0
- package/templates/scaffold/system/site/record/instance.ts +257 -0
- package/templates/scaffold/system/site/record/instant.ts +43 -0
- package/templates/scaffold/system/site/record/ledger.ts +694 -0
- package/templates/scaffold/system/site/record/load.ts +129 -0
- package/templates/scaffold/system/site/record/lock.ts +306 -0
- package/templates/scaffold/system/site/record/near-miss.ts +37 -0
- package/templates/scaffold/system/site/record/policy.ts +414 -0
- package/templates/scaffold/system/site/record/profile.ts +535 -0
- package/templates/scaffold/system/site/record/refusal.ts +106 -0
- package/templates/scaffold/system/site/record/yaml-file.ts +103 -0
- package/templates/scaffold/system/site/source.config.ts +109 -23
- package/dist/gateway-api-CmIthmJS-IUA9qS-T.mjs +0 -3225
- package/templates/scaffold/system/site/lib/denial-rule.ts +0 -220
- package/templates/scaffold/system/site/lib/page-order.ts +0 -93
package/dist/gateway.d.mts
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
import { CallToolResult, McpServer as McpServer$1, StandardSchemaWithJSON } from "@modelcontextprotocol/server";
|
|
2
2
|
import { z as z$1 } from "zod";
|
|
3
3
|
import pg from "pg";
|
|
4
|
-
//#region ../content-gateway/dist/gateway-api-
|
|
4
|
+
//#region ../content-gateway/dist/gateway-api-Da-9ssbU.d.mts
|
|
5
5
|
//#region src/instructions.d.ts
|
|
6
6
|
/**
|
|
7
7
|
* Has the owner said what this record is FOR yet?
|
|
@@ -16,6 +16,10 @@ import pg from "pg";
|
|
|
16
16
|
declare function recordIsUndescribed(authored: string): boolean;
|
|
17
17
|
declare function composeInstructions(authored: string): string;
|
|
18
18
|
//#endregion
|
|
19
|
+
//#region ../content/dist/index-CQcB_oVG.d.mts
|
|
20
|
+
declare const TRUST_TIERS: readonly ["unverified", "machine-confirmed", "human-reviewed"];
|
|
21
|
+
type TrustTier = (typeof TRUST_TIERS)[number];
|
|
22
|
+
//#endregion
|
|
19
23
|
//#region ../content/dist/index.d.mts
|
|
20
24
|
//#endregion
|
|
21
25
|
//#region src/lib/abstain.d.ts
|
|
@@ -45,26 +49,40 @@ interface AbstainConfig {
|
|
|
45
49
|
* never set one by intuition, recalibrate per corpus.
|
|
46
50
|
*/
|
|
47
51
|
readonly keywordFloor: number | null;
|
|
52
|
+
/**
|
|
53
|
+
* The digest of the retrieval predicate {@link vectorFloor} was MEASURED
|
|
54
|
+
* under (`GATE_PREDICATE_DIGEST`), or null on a record that declares none.
|
|
55
|
+
*
|
|
56
|
+
* A number alone cannot say which candidate set it separated, so a floor
|
|
57
|
+
* carried across a predicate change reads as calibrated while gating a set
|
|
58
|
+
* it was never measured on. The door compares this at boot and refuses on a
|
|
59
|
+
* mismatch — the declared-but-uncalibrated state, not `gate: off`.
|
|
60
|
+
*/
|
|
61
|
+
readonly floorDigest: string | null;
|
|
48
62
|
}
|
|
49
63
|
interface ContentInstance {
|
|
50
|
-
/** The corpus identity — instance.md's `name:` (the
|
|
64
|
+
/** The corpus identity — instance.md's `name:` (the one sanctioned identity key). */
|
|
51
65
|
readonly name: string;
|
|
52
66
|
readonly corpusId: string;
|
|
53
67
|
readonly tenantId: string;
|
|
68
|
+
/** The display title (the root index's heading) and the one-sentence description that seeds `llms.txt` and `server.json`. */
|
|
69
|
+
readonly title: string;
|
|
70
|
+
readonly description: string;
|
|
71
|
+
/** The upgrade stamp `ksor init` wrote; null on a hand-written instance. */
|
|
72
|
+
readonly toolchain: {
|
|
73
|
+
readonly requires: string;
|
|
74
|
+
readonly scaffolded: string;
|
|
75
|
+
} | null;
|
|
54
76
|
/** The NAME of the env var holding the DSN; the composition root resolves it. */
|
|
55
77
|
readonly dsnEnv: string;
|
|
56
78
|
readonly abstain: AbstainConfig;
|
|
57
79
|
readonly maximumResponseCharacters: number;
|
|
58
|
-
/** The
|
|
80
|
+
/** The MCP server's instructions (the body, edge-trimmed). */
|
|
59
81
|
readonly instructions: string;
|
|
60
82
|
/** Transport name (registry key, never persisted). */
|
|
61
83
|
readonly embeddingProvider: string;
|
|
62
|
-
/** The record's reader audiences, least- to most-restricted; empty = no model. */
|
|
63
84
|
/** The Postgres text-search configuration the keyword arm stems with. */
|
|
64
85
|
readonly textSearchConfig: string;
|
|
65
|
-
readonly audiences: readonly string[];
|
|
66
|
-
/** The tier a document takes when it declares none; null = none declared. */
|
|
67
|
-
readonly defaultVisibility: string | null;
|
|
68
86
|
/** model + dim are the persisted IDENTITY of the embedding space. */
|
|
69
87
|
readonly embeddingModel: string;
|
|
70
88
|
readonly embeddingDim: number;
|
|
@@ -86,12 +104,23 @@ interface ServiceContext {
|
|
|
86
104
|
/** The verified caller, or null → audited as "anonymous". */
|
|
87
105
|
readonly actor?: () => string | null;
|
|
88
106
|
/**
|
|
89
|
-
* The
|
|
90
|
-
*
|
|
91
|
-
*
|
|
92
|
-
* the
|
|
107
|
+
* The viewer list this door serves (record spec §2.4) — validated against the
|
|
108
|
+
* ingested policy's registry at boot (`validateViewer`), always including
|
|
109
|
+
* `public`. Absent = `[public]`, the safe default: a door that cannot
|
|
110
|
+
* establish who is asking must not hand out the restricted half of the record.
|
|
93
111
|
*/
|
|
94
|
-
readonly
|
|
112
|
+
readonly viewer?: readonly string[];
|
|
113
|
+
/**
|
|
114
|
+
* The lowest trust tier this door will answer from (record spec §2.3):
|
|
115
|
+
* 0 unverified, 1 machine-confirmed, 2 human-reviewed. Absent = 0, which
|
|
116
|
+
* admits every tier — the honest default, since `verified` is never required
|
|
117
|
+
* and a record with no verifications is a legitimate level-0 state.
|
|
118
|
+
*
|
|
119
|
+
* Bound as an ARM predicate, never applied to the hits afterwards: a floor
|
|
120
|
+
* enforced after ranking has already let a lower-tier passage decide what
|
|
121
|
+
* the answer was.
|
|
122
|
+
*/
|
|
123
|
+
readonly minTrustTier?: TrustTier | number;
|
|
95
124
|
}
|
|
96
125
|
/**
|
|
97
126
|
* The largest outline a caller may ASK for. The tool schema and the service
|
|
@@ -123,6 +152,16 @@ declare const READ_OUTPUT: StandardSchemaWithJSON;
|
|
|
123
152
|
interface SearchArgs {
|
|
124
153
|
readonly query: string;
|
|
125
154
|
readonly k: number;
|
|
155
|
+
/**
|
|
156
|
+
* The lowest trust tier the caller will accept an answer from.
|
|
157
|
+
*
|
|
158
|
+
* OPTIONAL on the handler, not on the wire, and that is the point: the
|
|
159
|
+
* registration file is adopter-owned code (decision 23), so a record
|
|
160
|
+
* scaffolded before this parameter existed keeps working — the handler
|
|
161
|
+
* supplies `unverified`, which is what it always had. The boot inspection
|
|
162
|
+
* NOTICES the absence and never refuses it.
|
|
163
|
+
*/
|
|
164
|
+
readonly min_trust_tier?: TrustTier | undefined;
|
|
126
165
|
}
|
|
127
166
|
declare function searchHandler(ctx: ServiceContext): (args: SearchArgs) => Promise<CallToolResult>;
|
|
128
167
|
interface OutlineArgs {
|
|
@@ -141,4 +180,4 @@ interface ReadArgs {
|
|
|
141
180
|
}
|
|
142
181
|
declare function readHandler(ctx: ServiceContext): (args: ReadArgs) => Promise<CallToolResult>;
|
|
143
182
|
//#endregion
|
|
144
|
-
export { FLOOR, MAX_OUTLINE_LIMIT, MAX_SEARCH_K, McpServer$1 as McpServer, OUTLINE_OUTPUT, type OutlineArgs, READ_ONLY, READ_OUTPUT, type ReadArgs, SEARCH_OUTPUT, type SearchArgs, type ServiceContext, composeInstructions, outlineHandler, readHandler, recordIsUndescribed, searchHandler, z$1 as z };
|
|
183
|
+
export { FLOOR, MAX_OUTLINE_LIMIT, MAX_SEARCH_K, McpServer$1 as McpServer, OUTLINE_OUTPUT, type OutlineArgs, READ_ONLY, READ_OUTPUT, type ReadArgs, SEARCH_OUTPUT, type SearchArgs, type ServiceContext, TRUST_TIERS, type TrustTier, composeInstructions, outlineHandler, readHandler, recordIsUndescribed, searchHandler, z$1 as z };
|
package/dist/gateway.mjs
CHANGED
|
@@ -1,2 +1,2 @@
|
|
|
1
|
-
import {
|
|
2
|
-
export { FLOOR, MAX_OUTLINE_LIMIT, MAX_SEARCH_K, McpServer$1 as McpServer, OUTLINE_OUTPUT, READ_ONLY, READ_OUTPUT, SEARCH_OUTPUT, composeInstructions, outlineHandler, readHandler, recordIsUndescribed, searchHandler, z$1 as z };
|
|
1
|
+
import { D as outlineHandler, I as searchHandler, N as readHandler, P as recordIsUndescribed, W as z$1, c as McpServer$1, d as READ_ONLY, f as READ_OUTPUT, h as TRUST_TIERS, i as FLOOR, o as MAX_OUTLINE_LIMIT, p as SEARCH_OUTPUT, s as MAX_SEARCH_K, u as OUTLINE_OUTPUT, x as composeInstructions } from "./gateway-api-CF4ED9_g-BQusM_dK.mjs";
|
|
2
|
+
export { FLOOR, MAX_OUTLINE_LIMIT, MAX_SEARCH_K, McpServer$1 as McpServer, OUTLINE_OUTPUT, READ_ONLY, READ_OUTPUT, SEARCH_OUTPUT, TRUST_TIERS, composeInstructions, outlineHandler, readHandler, recordIsUndescribed, searchHandler, z$1 as z };
|
package/dist/index.d.mts
CHANGED
|
@@ -21,7 +21,7 @@ declare const exitCodes: {
|
|
|
21
21
|
type ExitCode = (typeof exitCodes)[keyof typeof exitCodes];
|
|
22
22
|
/** The CLI vocabulary. Lifecycle verbs plus the corpus operations the bundled
|
|
23
23
|
* kernel provides (one binary — decision 12 publish revision). */
|
|
24
|
-
declare const verbs: readonly ["init", "dev", "build", "serve", "ingest", "schema", "grant", "takedown", "calibrate", "gc"];
|
|
24
|
+
declare const verbs: readonly ["init", "dev", "build", "migrate", "serve", "ingest", "schema", "grant", "takedown", "calibrate", "gc"];
|
|
25
25
|
type Verb = (typeof verbs)[number];
|
|
26
26
|
interface ResolvedCommand {
|
|
27
27
|
/** The first non-flag token, or null when only flags (or nothing) appear. */
|
package/dist/index.mjs
CHANGED
|
@@ -1,2 +1,2 @@
|
|
|
1
|
-
import { n as resolveCommand, r as verbs, t as exitCodes } from "./src-
|
|
1
|
+
import { n as resolveCommand, r as verbs, t as exitCodes } from "./src-dqpI-p1a.mjs";
|
|
2
2
|
export { exitCodes, resolveCommand, verbs };
|
package/docs/authorization.md
CHANGED
|
@@ -12,7 +12,7 @@ authorization server and pointing the door at it.
|
|
|
12
12
|
This page is four worked recipes, all executed against real servers rather than
|
|
13
13
|
written from their documentation, plus what an agent does to obtain a token. The
|
|
14
14
|
mechanism is standard OAuth 2.0 — nothing here is specific to any one product,
|
|
15
|
-
and that is the point:
|
|
15
|
+
and that is the point: four different implementations are shown because a
|
|
16
16
|
single one proves nothing about neutrality. Two are self-hosted (one `docker
|
|
17
17
|
run` each, no account), one is a hosted commercial provider with a free tier,
|
|
18
18
|
and one is an organization's own SSO — which is the case that matters most,
|
|
@@ -41,9 +41,11 @@ per-request case needs a decision first.
|
|
|
41
41
|
**It is one gate, not per-user rules.** The door checks that a token was signed
|
|
42
42
|
by the issuer you named and audienced at this record. It reads no scopes, no
|
|
43
43
|
roles, no groups. **Any caller holding a valid token gets the whole record**, to
|
|
44
|
-
the extent
|
|
45
|
-
documents, that is the record's
|
|
46
|
-
|
|
44
|
+
the extent its audience list allows. If different readers must see different
|
|
45
|
+
documents, that is the record's audience model — the registry in
|
|
46
|
+
`.ksor/governance.yaml`, each concept's `ksor.audience` list, and the viewer
|
|
47
|
+
list the door is configured for — and it is a different mechanism from this
|
|
48
|
+
page. See the scaffold's AGENTS.md.
|
|
47
49
|
|
|
48
50
|
So: this page answers _"can a stranger read my record over MCP?"_ It does not
|
|
49
51
|
answer _"can Alice read what Bob can."_
|
|
@@ -67,7 +69,7 @@ works, not while you are trying to make it work.
|
|
|
67
69
|
to anyone who can reach the port — and it wins over everything below. A scaffold
|
|
68
70
|
ships with `KSOR_AUTH=disabled-local` in its `.env.example`, so a deployment
|
|
69
71
|
that copied that file has it set and will stay unauthenticated no matter how
|
|
70
|
-
carefully you configure the
|
|
72
|
+
carefully you configure the variables above. Configuring the SSO door is
|
|
71
73
|
what turns auth **on**; removing `KSOR_AUTH` is what stops it being off.
|
|
72
74
|
|
|
73
75
|
Every variable here is read from the environment of the `ksor serve` process —
|
|
@@ -190,7 +192,7 @@ opaque by default, no OAuth token endpoint taking a custom audience, no metadata
|
|
|
190
192
|
document. That is a coherent design; it is simply a different one, and adapting
|
|
191
193
|
it means writing the verification layer ksor already is.
|
|
192
194
|
|
|
193
|
-
The
|
|
195
|
+
The four recipes below all pass. If yours does too, they will read as the same
|
|
194
196
|
recipe with different button names — because underneath they are.
|
|
195
197
|
|
|
196
198
|
## Recipe: Keycloak
|
package/docs/deploying.md
CHANGED
|
@@ -82,16 +82,21 @@ The emitted `Dockerfile` names no host. It installs the pinned
|
|
|
82
82
|
|
|
83
83
|
```sh
|
|
84
84
|
docker build -t my-record .
|
|
85
|
-
docker run --rm -p 8080:80 --env-file .env
|
|
85
|
+
docker run --rm -p 8080:80 --env-file .env \
|
|
86
|
+
-e KSOR_AUTH=disabled-public my-record
|
|
86
87
|
```
|
|
87
88
|
|
|
88
89
|
Two things about that command, both of which bite:
|
|
89
90
|
|
|
90
91
|
- **The image listens on 80**, which is why the mapping is `8080:80`. If your
|
|
91
92
|
`.env` sets `PORT`, change the right-hand side to match.
|
|
92
|
-
-
|
|
93
|
-
|
|
94
|
-
`disabled-local` refuses there by design. Your
|
|
93
|
+
- **`KSOR_AUTH=disabled-public` is required, even locally.** A container gets
|
|
94
|
+
`$PORT` and therefore binds `0.0.0.0` — a public bind — and the
|
|
95
|
+
`disabled-local` a scaffolded `.env` carries refuses there by design. Your
|
|
96
|
+
laptop is not the exception. Pass it with `-e` rather than editing `.env`, so
|
|
97
|
+
a plain `ksor serve` outside the container keeps its loopback posture; `-e`
|
|
98
|
+
overrides `--env-file`. A real deployment sets it — or the SSO variables — in
|
|
99
|
+
the host's environment, since `.dockerignore` keeps `.env` out of the image.
|
|
95
100
|
|
|
96
101
|
That runs on Cloud Run, Fly, Render, ECS, Kubernetes, or a VPS with no changes.
|
|
97
102
|
`vercel.json` **points at this same file** rather than replacing it, which is
|
|
@@ -234,22 +239,20 @@ rebinding is worth an attacker's effort only when it reaches something they coul
|
|
|
234
239
|
not reach directly. With `KSOR_AUTH=disabled-public` the record is already served
|
|
235
240
|
to anyone who types the URL, so there is nothing for rebinding to steal.
|
|
236
241
|
|
|
237
|
-
### The site build
|
|
242
|
+
### The site build runs `ksor build` first
|
|
238
243
|
|
|
239
|
-
|
|
240
|
-
|
|
241
|
-
|
|
244
|
+
`pnpm build` is `ksor build` followed by the site build. `ksor build` needs no
|
|
245
|
+
database: it regenerates every `index.md`, runs the record checker, and writes
|
|
246
|
+
`build.lock.json` — commit it — which every machine artefact stamps. A refusal
|
|
247
|
+
stops the build before a byte is written; `--strict` also refuses an
|
|
248
|
+
uncommitted input. Takedowns reach the site through the committed ledger
|
|
249
|
+
(`.ksor/takedowns.yaml`), which is a file in the repository — so the site build
|
|
250
|
+
needs no `KSOR_DB_URL` at all.
|
|
242
251
|
|
|
243
|
-
|
|
244
|
-
|
|
245
|
-
|
|
246
|
-
|
|
247
|
-
```
|
|
248
|
-
|
|
249
|
-
That refusal is the design working. A takedown reaches the door instantly (it is
|
|
250
|
-
a row) and reaches the site at its next build (it reads a file), so a site built
|
|
251
|
-
without the DSN would keep publishing what the door already refuses. Set
|
|
252
|
-
`KSOR_DB_URL` on the site build as well as on the door.
|
|
252
|
+
That is the design working. A takedown reaches the door instantly (it is a row)
|
|
253
|
+
and reaches the site at its next build (it reads the ledger), so the act that
|
|
254
|
+
withdraws a document is the same merged commit on both surfaces. Merge the
|
|
255
|
+
ledger entry, rebuild, redeploy.
|
|
253
256
|
|
|
254
257
|
## Keeping people out of the site
|
|
255
258
|
|
|
@@ -282,12 +285,20 @@ nothing.
|
|
|
282
285
|
|
|
283
286
|
### "Some documents are restricted, most are not"
|
|
284
287
|
|
|
285
|
-
**Build per audience.** `KSOR_AUDIENCE
|
|
286
|
-
|
|
287
|
-
enforcement by absence, which is the only kind a static host
|
|
288
|
-
the public artifact openly and the wider one behind the
|
|
288
|
+
**Build per audience.** `KSOR_AUDIENCE=public,internal pnpm build` stages only
|
|
289
|
+
what that viewer may see, so the concepts it may not are **never written into
|
|
290
|
+
the artifact** — enforcement by absence, which is the only kind a static host
|
|
291
|
+
can honour. Publish the public artifact openly and the wider one behind the
|
|
292
|
+
gate above.
|
|
293
|
+
|
|
294
|
+
The value is a comma list of registered audiences and it must include `public`
|
|
295
|
+
— a bare `KSOR_AUDIENCE=internal` is refused (`ksor-viewer-omits-public`),
|
|
296
|
+
because every reader of a restricted build is also a reader of the open one and
|
|
297
|
+
a build for the restricted audience alone would silently drop every public
|
|
298
|
+
concept. A concept is staged when its `ksor.audience` list overlaps the
|
|
299
|
+
viewer's; there is no ordering between audiences to be narrower or wider than.
|
|
289
300
|
|
|
290
|
-
Plain `pnpm build` is
|
|
301
|
+
Plain `pnpm build` is `[public]`, so the safe thing is the default.
|
|
291
302
|
|
|
292
303
|
### "Different readers see different documents, decided per request"
|
|
293
304
|
|
|
@@ -428,8 +439,8 @@ connections** — the pool minimum is 0 and an unused connection closes after 10
|
|
|
428
439
|
first request after a suspend both wakes the database and retries the connect
|
|
429
440
|
rather than failing.
|
|
430
441
|
|
|
431
|
-
Measured against a live deployment on Vercel, Neon behind it, an
|
|
432
|
-
record of 6,963 chunks:
|
|
442
|
+
Measured 2026-08-23 against a live deployment on Vercel, Neon behind it, an
|
|
443
|
+
81-document record of 6,963 chunks, and not re-measured since:
|
|
433
444
|
|
|
434
445
|
| | |
|
|
435
446
|
| -------------------------------------- | --------- |
|
package/docs/index.md
CHANGED
|
@@ -31,10 +31,14 @@ instead of their training memory. The corpus grows with each implemented verb.
|
|
|
31
31
|
needing Postgres and a provider key — alongside the write plane that keeps
|
|
32
32
|
the record current: `ksor schema` (provision or migrate the database),
|
|
33
33
|
`ksor grant` (authorize a tenant for ingest), `ksor ingest` (build and publish
|
|
34
|
-
a generation), `ksor takedown` (withdraw a document from EVERY
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
34
|
+
a generation), `ksor takedown` (withdraw a document from EVERY
|
|
35
|
+
surface — the committed ledger first, the denylist row second, so a record
|
|
36
|
+
with no database can withdraw one), `ksor calibrate` (measure the
|
|
37
|
+
abstention floor) and `ksor gc` (reap retired generations). `ksor build`
|
|
38
|
+
(check the record, generate its indexes, write `build.lock.json`) and
|
|
39
|
+
`ksor migrate` (rewrite a pre-profile record into the KSoR Profile — a diff
|
|
40
|
+
first, `--write` to apply) need no database. Only `ksor dev` remains designed, not implemented: it prints an
|
|
41
|
+
honest notice and exits `2`.
|
|
38
42
|
- **[tool-surface.md](./tool-surface.md)** — shaping what agents see.
|
|
39
43
|
`system/gateways/content.ts` is emitted, adopter-owned and deletable; it
|
|
40
44
|
decides tool names, what the record says it covers, and how much of the
|
|
@@ -62,15 +66,24 @@ instead of their training memory. The corpus grows with each implemented verb.
|
|
|
62
66
|
## For the agent operating a scaffolded project
|
|
63
67
|
|
|
64
68
|
Read the scaffold's own `AGENTS.md` first — it is the working contract.
|
|
65
|
-
Knowledge lives in `knowledge/` and never inside the site
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
`
|
|
72
|
-
|
|
73
|
-
|
|
69
|
+
Knowledge lives in `knowledge/` and never inside the site. A document is an
|
|
70
|
+
OKF concept in the KSoR Profile: `type`, `title`, `description`, `status`
|
|
71
|
+
(`draft | stable | deprecated`) and `ksor.audience` are required, and a
|
|
72
|
+
`stable` one additionally carries `generated` and a `ksor.approval` by an
|
|
73
|
+
actor the Governance Policy authorises. Keys the profile does not know are
|
|
74
|
+
PRESERVED rather than refused; the pre-profile ones (`visibility`, `owner`,
|
|
75
|
+
`provenance`, `effective`, `superseded`, `sor_id`) are refused by name, each
|
|
76
|
+
naming its replacement. `pnpm check` explains any violation and how to fix it.
|
|
77
|
+
Reading order is the governed `order:` frontmatter key — never `meta.json` or
|
|
78
|
+
`sidebar_position` — and it drives every surface: the sidebar, `llms.txt`, and
|
|
79
|
+
the MCP `outline` tool.
|
|
80
|
+
|
|
81
|
+
Audience is a LIST, matched by overlap. `.ksor/governance.yaml` registers the
|
|
82
|
+
audiences; each concept lists the ones it is for; a build names its viewer as a
|
|
83
|
+
comma list that must include `public`
|
|
84
|
+
(`KSOR_AUDIENCE=public,internal pnpm build`) and stages only the concepts whose
|
|
85
|
+
list overlaps it. It is publication, not authorship: anyone who can clone reads
|
|
86
|
+
everything. The site shell
|
|
74
87
|
at `system/site/` is replaceable behind a five-clause surface contract — the
|
|
75
88
|
shell is a slot, and the contract (render the record, `llms.txt`, per-page md
|
|
76
89
|
artifacts, browser smoke, no authored content) is what a replacement must meet.
|
package/docs/ingesting.md
CHANGED
|
@@ -20,12 +20,12 @@ Ingest reads your markdown, sends each new chunk to an embedding provider, and
|
|
|
20
20
|
writes the result to Postgres. So four things must be true, and none of them is
|
|
21
21
|
created for you.
|
|
22
22
|
|
|
23
|
-
| | what
|
|
24
|
-
| -------------------- |
|
|
25
|
-
| **The corpus** | `knowledge/` at your repo root — CommonMark `.md`, one document per file, `title` and `
|
|
26
|
-
| **The database** | Postgres with **pgvector** — `CREATE EXTENSION vector;`
|
|
27
|
-
| **The provider key** | `GEMINI_API_KEY` — the default embedding provider is `gemini-embedding-001`
|
|
28
|
-
| **The DSN** | `KSOR_DB_URL`, named by `instance.md`'s `database.dsn_env`
|
|
23
|
+
| | what | how |
|
|
24
|
+
| -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
|
|
25
|
+
| **The corpus** | `knowledge/` at your repo root — CommonMark `.md`, one document per file, in the KSoR Profile of OKF: `type`, `title`, `description`, `status` and `ksor.audience` in frontmatter | `pnpm check` validates it and explains any violation; `ksor build` must have written a current `build.lock.json` before ingest will run |
|
|
26
|
+
| **The database** | Postgres with **pgvector** — `CREATE EXTENSION vector;` | any managed host; the DDL below needs a role that can create tables |
|
|
27
|
+
| **The provider key** | `GEMINI_API_KEY` — the default embedding provider is `gemini-embedding-001` | [aistudio.google.com](https://aistudio.google.com/apikey); the free tier covers a first corpus |
|
|
28
|
+
| **The DSN** | `KSOR_DB_URL`, named by `instance.md`'s `database.dsn_env` | uncomment the `database:` block in `instance.md` first |
|
|
29
29
|
|
|
30
30
|
Both variables go in `.env` beside `instance.md` — `ksor` reads it automatically,
|
|
31
31
|
and `.env` is gitignored. Every command below is run **from your repository
|
|
@@ -55,6 +55,24 @@ serves immediately; the website is a static build and only changes when you
|
|
|
55
55
|
rebuild and redeploy it. Ingest alone leaves the human surface showing the old
|
|
56
56
|
content, which reads as a half-failed ingest and is not one.
|
|
57
57
|
|
|
58
|
+
## When ingest says the lock is stale
|
|
59
|
+
|
|
60
|
+
`ksor ingest` refuses a record whose `build.lock.json` does not match the tree
|
|
61
|
+
(`ksor-lock-stale`), or that has none at all (`ksor-lock-missing`). **The fix is
|
|
62
|
+
always `ksor build` in that directory — never an edit to the lock.** The lock is
|
|
63
|
+
what says which bytes were checked; editing it to agree with the tree asserts a
|
|
64
|
+
check that never ran, which is the one thing it exists to prevent.
|
|
65
|
+
|
|
66
|
+
Freshness covers **seven** sets, not just your documents: `instance.md`,
|
|
67
|
+
`.ksor/governance.yaml`, `.ksor/takedowns.yaml`, the concepts, the companions,
|
|
68
|
+
the assets, and the generated `index.md` files. So a refusal can name a file you
|
|
69
|
+
would not think of as content — the policy, the takedown ledger, a diagram you
|
|
70
|
+
replaced, or an index you have never opened. That breadth is deliberate: a
|
|
71
|
+
freshness claim that cannot see the ledger is not a freshness claim. The
|
|
72
|
+
website's lock was fixed for exactly this once already, after deleting a
|
|
73
|
+
denial's four lines republished the document with the committed lock still
|
|
74
|
+
validating; ingest had the same hole, found by review before anyone hit it.
|
|
75
|
+
|
|
58
76
|
## What a generation is
|
|
59
77
|
|
|
60
78
|
Each ingest builds a **fresh generation** — invisible until activated — and
|
|
@@ -106,17 +124,19 @@ An empty `nodes` array means nothing was published — the ingest did not run, o
|
|
|
106
124
|
ran without `--flip`. Then search for a phrase you know is in the record and
|
|
107
125
|
check the hits carry `provenance.stable_id` and `generation`.
|
|
108
126
|
|
|
109
|
-
**`provenance.stable_id` is also how you name a document to `takedown`.**
|
|
110
|
-
|
|
111
|
-
reliable way to read one off
|
|
127
|
+
**`provenance.stable_id` is also how you name a document to `takedown`.** It is
|
|
128
|
+
`knowledge/<path-without-.md>` — always, since path is identity and nothing
|
|
129
|
+
overrides it — and a search result is still the reliable way to read one off
|
|
130
|
+
rather than typing it from memory.
|
|
112
131
|
|
|
113
132
|
## Where ingest runs — not on the host
|
|
114
133
|
|
|
115
134
|
Ingest is a long job. It embeds every new chunk through the provider, and that
|
|
116
135
|
is bounded by the provider's throughput rather than by anything ksor does.
|
|
117
136
|
|
|
118
|
-
**Measured:** an 81-document book — 6,963 chunks — took **about 50
|
|
119
|
-
against a remote Postgres on a first, cold ingest with nothing to
|
|
137
|
+
**Measured 2026-08-23:** an 81-document book — 6,963 chunks — took **about 50
|
|
138
|
+
minutes** against a remote Postgres on a first, cold ingest with nothing to
|
|
139
|
+
carry forward. Not re-measured since.
|
|
120
140
|
|
|
121
141
|
Compare that with the request timeouts of the platforms people reach for first:
|
|
122
142
|
a serverless function caps out in the region of 300–800 seconds depending on
|
|
@@ -148,7 +168,8 @@ If your provider offers both a **pooled** and a **direct** endpoint (Neon's
|
|
|
148
168
|
`-pooler` host, or port 6432), use whichever it gives you. ksor detects which and
|
|
149
169
|
says so at boot, but the line is informational — it classifies, it never
|
|
150
170
|
transforms, and the hazard it descends from cannot arise here. The 6,963-chunk
|
|
151
|
-
ingest measured above ran through a pooled endpoint without
|
|
171
|
+
ingest measured above (2026-08-23) ran through a pooled endpoint without
|
|
172
|
+
incident. Reach for
|
|
152
173
|
the direct endpoint only if you actually hit pooler connection limits.
|
|
153
174
|
|
|
154
175
|
## Turning the abstention gate on
|
|
@@ -161,14 +182,24 @@ measure until the corpus is in there.
|
|
|
161
182
|
pnpm exec ksor calibrate --instance instance.md
|
|
162
183
|
```
|
|
163
184
|
|
|
164
|
-
It
|
|
165
|
-
|
|
185
|
+
It ends with a block to paste into **`instance.md`**'s frontmatter, exactly as
|
|
186
|
+
printed — the floor, the measurement recorded beside it as a comment, and
|
|
187
|
+
`floor_digest`, the digest of the retrieval predicate the floor was measured
|
|
188
|
+
through. Paste it, then restart `ksor serve`: the floor is read at boot.
|
|
166
189
|
|
|
167
190
|
```yaml
|
|
168
191
|
retrieval:
|
|
169
|
-
vector_floor: 0.55 #
|
|
192
|
+
vector_floor: 0.55 # calibrated 2026-08-23 on generation 3, model gemini-embedding-001/d1536, door: synthesized
|
|
193
|
+
floor_digest: 8bfb07d0e6f5
|
|
170
194
|
```
|
|
171
195
|
|
|
196
|
+
If the file already declares `retrieval:`, merge the keys into the block it has
|
|
197
|
+
rather than adding a second one — a duplicate key is refused.
|
|
198
|
+
|
|
199
|
+
A measurement that does not separate in-corpus from out-of-corpus prints **no
|
|
200
|
+
floor at all**, and a `vector_floor: uncalibrated` block instead. That block is
|
|
201
|
+
the fail-closed state: every serve refuses until a real number replaces it.
|
|
202
|
+
|
|
172
203
|
Until you do, `/health` reports the gate as `OFF (no floor declared — will not
|
|
173
204
|
refuse out-of-corpus questions)` and every search envelope carries
|
|
174
205
|
`gate: "off"`. That is honest absence, and an agent reading the envelope knows
|
|
@@ -186,11 +217,13 @@ correctly stays uncalibrated.
|
|
|
186
217
|
|
|
187
218
|
## Withdrawing a document
|
|
188
219
|
|
|
189
|
-
A takedown is a
|
|
220
|
+
A takedown is a committed ledger entry FIRST and a database row second, written
|
|
221
|
+
in one act — so it reaches the door immediately and the site at its next build,
|
|
222
|
+
and a record with no database can still withdraw a document:
|
|
190
223
|
|
|
191
224
|
```sh
|
|
192
225
|
pnpm exec ksor takedown --instance instance.md <stable-id> \
|
|
193
|
-
--reason "legal request 2026-08" --actor
|
|
226
|
+
--reason "legal request 2026-08" --actor human:j.smith
|
|
194
227
|
```
|
|
195
228
|
|
|
196
229
|
`--actor` is required, and there is no default. A name taken from the
|
|
@@ -198,8 +231,23 @@ environment reads like a person and is whatever the shell happened to be
|
|
|
198
231
|
(`runner` under CI, `root` in a container) — worse than no name at all in the
|
|
199
232
|
one row that exists to record who did this.
|
|
200
233
|
|
|
201
|
-
|
|
202
|
-
|
|
203
|
-
|
|
204
|
-
|
|
205
|
-
|
|
234
|
+
Two things the actor must satisfy, both refused before any database is touched.
|
|
235
|
+
It must be a well-formed actor — `human:<handle>` or `process:<id>`, never a
|
|
236
|
+
bare name (`ksor-actor-form`) and never a `team:` (a team cannot perform an
|
|
237
|
+
act). And `takedown_authorities` in `.ksor/governance.yaml` must name it
|
|
238
|
+
(`ksor-takedown-unauthorised`); the same check runs over every entry in the
|
|
239
|
+
ledger at `pnpm check`, `ksor build` and ingest, so a line appended by hand in a
|
|
240
|
+
pull request is refused exactly as the verb would refuse it.
|
|
241
|
+
|
|
242
|
+
Lifting a takedown is `--revoke <entry-id>` — the id of the LEDGER ENTRY, not
|
|
243
|
+
the stable id. The denial that created it prints the id, `ksor takedown
|
|
244
|
+
--ledger` lists it, and it is written in `.ksor/takedowns.yaml`; none of the
|
|
245
|
+
three needs a database, because the ledger is a file in the repository. The
|
|
246
|
+
ledger is append-only: a revocation is a new entry, never a deleted line, and a
|
|
247
|
+
build whose ledger shrank against its own git history is refused.
|
|
248
|
+
|
|
249
|
+
**The site stops at its next build.** It reads the committed ledger
|
|
250
|
+
(`.ksor/takedowns.yaml`), so after a takedown, merge the entry, rebuild and
|
|
251
|
+
redeploy the site, or the human surface keeps publishing what the agent
|
|
252
|
+
surface already refuses. The site needs no database for this: the
|
|
253
|
+
ledger is a file in the repository, and the site build reads it.
|