@panaversity/ksor 0.0.10 → 0.0.12
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +218 -0
- package/dist/cli.mjs +220 -48
- package/docs/index.md +3 -2
- package/package.json +1 -1
- package/templates/scaffold/AGENTS.md +82 -12
- package/templates/scaffold/instance.md +7 -0
- package/templates/scaffold/system/site/app/.well-known/mcp/server.json/route.ts +5 -2
- package/templates/scaffold/system/site/lib/shared.ts +49 -0
- package/templates/scaffold/system/site/lib/stage-knowledge.ts +19 -2
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,223 @@
|
|
|
1
1
|
# @panaversity/ksor
|
|
2
2
|
|
|
3
|
+
## 0.0.12
|
|
4
|
+
|
|
5
|
+
### Patch Changes
|
|
6
|
+
|
|
7
|
+
- 36e4a4c: The scaffold documents what a CLIENT has to do to reach a public MCP door
|
|
8
|
+
|
|
9
|
+
`ksor serve` implements the OAuth Resource Server handshake — an
|
|
10
|
+
unauthenticated request gets a 401 carrying
|
|
11
|
+
`WWW-Authenticate: Bearer resource_metadata="…"`, and that document names the
|
|
12
|
+
record's resource identifier and its authorization server, so a client discovers
|
|
13
|
+
where to authenticate instead of being told. None of it was written down
|
|
14
|
+
anywhere an adopter or their agent reads. The operator half was documented (the
|
|
15
|
+
three environment variables); the half their agents actually execute was not.
|
|
16
|
+
|
|
17
|
+
The scaffold's `AGENTS.md` now walks the three steps, and names the failure that
|
|
18
|
+
goes wrong quietly: a token minted for a different audience is a perfectly valid
|
|
19
|
+
token, and this door rejects it, so `aud` against `KSOR_JWT_ALLOWED_AUDIENCES` is
|
|
20
|
+
the first thing to compare when a client authenticates fine and still gets 401.
|
|
21
|
+
It also records the two behaviours a client author has to know and could not have
|
|
22
|
+
guessed — RS256 only, no opaque-token introspection, and an unknown key id
|
|
23
|
+
answering 503 rather than 401, because during a key rotation the token is
|
|
24
|
+
probably good and retrying beats sending the user back through a login.
|
|
25
|
+
|
|
26
|
+
This closes one of the three items named in issue #26; the worked provider
|
|
27
|
+
recipes and the introspection/rotation policy remain open there.
|
|
28
|
+
|
|
29
|
+
- 125970c: Every 401 from the MCP door carries its `WWW-Authenticate` challenge, not just the first
|
|
30
|
+
|
|
31
|
+
Only the missing-token branch emitted `WWW-Authenticate: Bearer
|
|
32
|
+
resource_metadata="…"`. A token that failed verification — expired, wrong
|
|
33
|
+
audience, no subject, bad signature — came back as a bare 401. That is the most
|
|
34
|
+
common 401 a real client will ever see, because tokens expire mid-conversation,
|
|
35
|
+
and it left the client with no pointer back to the resource-metadata document:
|
|
36
|
+
it could not re-discover the authorization server it had just been talking to.
|
|
37
|
+
Only a caller that had never sent a token was told where to go.
|
|
38
|
+
|
|
39
|
+
The MCP authorization spec requires `WWW-Authenticate` on a 401 without
|
|
40
|
+
qualification. Every 401 now carries it, with RFC 6750's `error="invalid_token"`
|
|
41
|
+
so a client refreshes rather than retrying the dead token.
|
|
42
|
+
|
|
43
|
+
A **503** stays deliberately unchallenged: an unreachable key set is our outage,
|
|
44
|
+
not the token's fault, and challenging there would send a user whose token is
|
|
45
|
+
perfectly good back through a login over a key-fetch failure.
|
|
46
|
+
|
|
47
|
+
Found by adversarially checking the release that documented this door. The
|
|
48
|
+
adversarial auth suite missed it by asserting the STATUS of each rejection and
|
|
49
|
+
never the header — it now sweeps every 401-producing token and asserts the
|
|
50
|
+
challenge on each, with the 503 as the negative control.
|
|
51
|
+
|
|
52
|
+
- 0a94e31: A 503 refusal no longer puts the database host and user on the wire
|
|
53
|
+
|
|
54
|
+
When the deferred boot checks fail, `/mcp` refuses with the thrown error's
|
|
55
|
+
message in full under `data.detail`. For the three authored failures that is the
|
|
56
|
+
point — a too-old schema, a governance violation and a text-search mismatch each
|
|
57
|
+
carry a multi-line remedy the operator has to act on. But the catch treated every
|
|
58
|
+
error alike, and `pg` writes the host, its resolved address, the port and the
|
|
59
|
+
database user into its connection and authentication failures. Those went out
|
|
60
|
+
verbatim to any caller who could reach the door.
|
|
61
|
+
|
|
62
|
+
What may leave is now decided in one place and by TYPE, not by inspecting
|
|
63
|
+
message text: a class we wrote is a class whose words we control. A driver error
|
|
64
|
+
is refused with its class named and its text withheld, and the caller is told
|
|
65
|
+
which kind of failure it is — infrastructure, not their request.
|
|
66
|
+
|
|
67
|
+
The full text still reaches the operator, deliberately: the refusal says the
|
|
68
|
+
reason is in the server's logs, and the deferred-boot line recorded only the
|
|
69
|
+
error's NAME, so before this the real message existed nowhere. That is also why
|
|
70
|
+
the boot checks are not sanitised at their source — reducing a driver error to a
|
|
71
|
+
class name early would destroy the one copy anyone can act on.
|
|
72
|
+
|
|
73
|
+
**The test that covered this was holding it in place.** It asserted that
|
|
74
|
+
`http.ts` contains the literal string `data: { detail: message }` — so the leak
|
|
75
|
+
was pinned by an assertion with reasoning attached. Grepping source is the right
|
|
76
|
+
instrument for "does this check run before dispatch", because position is a
|
|
77
|
+
property of source, and the wrong one for "what does the response contain".
|
|
78
|
+
Response contents are now asserted against real bodies, including a `pg`-shaped
|
|
79
|
+
connection failure whose host, address, port and user must all be absent.
|
|
80
|
+
|
|
81
|
+
Verified live: a gateway pointed at an unreachable database answers
|
|
82
|
+
`the content store is unavailable (Error)` with no host, port, user or database
|
|
83
|
+
name anywhere in the body, while the server log carries
|
|
84
|
+
`connect ECONNREFUSED 127.0.0.1:59999` in full.
|
|
85
|
+
|
|
86
|
+
- 1dd6211: The dimension ceiling says which shape it applies to, instead of blaming pgvector
|
|
87
|
+
|
|
88
|
+
`ksor schema` refuses an embedding dimension above 2000 with
|
|
89
|
+
"(pgvector vector + HNSW ceiling)". The refusal is right and the reason was
|
|
90
|
+
wrong: pgvector indexes a `vector` to 2000, but a **`halfvec` to 4000**, via an
|
|
91
|
+
expression index on the cast — verified live against a real database, where
|
|
92
|
+
`hnsw ((embedding::halfvec(3072)) halfvec_cosine_ops)` plans an Index Scan.
|
|
93
|
+
|
|
94
|
+
The old wording read as pgvector's own limit, so an adopter whose model emits
|
|
95
|
+
more than 2000 dimensions could conclude it was unusable here, over a wall that
|
|
96
|
+
is not one. The message now names the shape the ceiling belongs to — this schema
|
|
97
|
+
declares `VECTOR(dim)` columns and indexes one directly — and the constant
|
|
98
|
+
carries why raising it is a decision rather than an edit: every query site would
|
|
99
|
+
have to use the same cast as the index or fall silently back to a sequential
|
|
100
|
+
scan, and the halfvec arm's float16 rounding lands on the score the abstention
|
|
101
|
+
gate reads.
|
|
102
|
+
|
|
103
|
+
The same claim is corrected in the scaffold's `AGENTS.md`, which gains the reason
|
|
104
|
+
`dim: 1536` is the shipped default: `gemini-embedding-001` emits 3072 and ksor
|
|
105
|
+
asks it for 1536, which per Google's published MTEB table costs nothing
|
|
106
|
+
measurable — 1536 scores 68.17 against 2048's 68.16.
|
|
107
|
+
|
|
108
|
+
The 2000 refusal is unchanged. Issue #49 records the decision it now points at.
|
|
109
|
+
|
|
110
|
+
## 0.0.11
|
|
111
|
+
|
|
112
|
+
### Patch Changes
|
|
113
|
+
|
|
114
|
+
- 0a0dd27: A record describes itself on the surface agents discover it through
|
|
115
|
+
|
|
116
|
+
`/.well-known/mcp/server.json` carried one hard-coded sentence — "The <name>
|
|
117
|
+
Knowledge System of Record: governed markdown served with citations and honest
|
|
118
|
+
abstention." — byte-identical in every ksor record ever scaffolded. An agent
|
|
119
|
+
choosing between records in a registry learned nothing from any of them, which
|
|
120
|
+
is the opposite of what that document exists for.
|
|
121
|
+
|
|
122
|
+
The description now comes from the record's own prose: its display title and the
|
|
123
|
+
first real sentence of `instance.md`, which is what the intake interview writes.
|
|
124
|
+
A record whose owner has not described it yet SAYS so rather than borrowing a
|
|
125
|
+
confident sentence it has not earned — the same answer the MCP door already
|
|
126
|
+
gives an agent that connects, so the two surfaces do not disagree about whether
|
|
127
|
+
this record knows what it is.
|
|
128
|
+
|
|
129
|
+
The scaffold's opening paragraphs are authoring guidance, not scope, so the
|
|
130
|
+
template is detected across the whole body rather than paragraph by paragraph:
|
|
131
|
+
publishing instructions-to-the-author as a description would be worse than
|
|
132
|
+
admitting there is none.
|
|
133
|
+
|
|
134
|
+
- 0fe759d: Three defects found by auditing 0.0.10 against a live record
|
|
135
|
+
|
|
136
|
+
**A repeated `sslmode` was read the wrong end.** `pinnedTlsDsn` took the FIRST
|
|
137
|
+
value of a repeated parameter; `pg` takes the LAST. So on
|
|
138
|
+
`?sslmode=require&sslmode=disable` — whose effective mode is `disable` — the pin
|
|
139
|
+
saw a weak mode, collapsed the duplicates into one `verify-full`, turned TLS on,
|
|
140
|
+
and printed "TLS verified" at an operator whose DSN ended in `disable`. The
|
|
141
|
+
direction was safe; silently overruling an explicit opt-out and then misreporting
|
|
142
|
+
it was not. All three TLS functions now read the mode the driver will use.
|
|
143
|
+
|
|
144
|
+
The same sweep asserted the larger worry the pin creates — that re-serializing a
|
|
145
|
+
connection string could alter a credential. Seventeen DSNs with the passwords
|
|
146
|
+
people actually paste (raw `@`, spaces, `%`, `+`, brackets, non-ASCII,
|
|
147
|
+
percent-encoded separators) are now checked field by field against `pg`'s own
|
|
148
|
+
resolved view: everything the driver derives is byte-identical, and so is the
|
|
149
|
+
TLS decision.
|
|
150
|
+
|
|
151
|
+
**The outline's `position` disclosed documents an audience may not see.** It was
|
|
152
|
+
the rank in the whole record, so a public caller received 1, 3, 4 — a gap exactly
|
|
153
|
+
where an internal sibling sat, telling them something exists and roughly where.
|
|
154
|
+
The same row's `child_count` was already computed over visible children only, so
|
|
155
|
+
one response object disagreed with itself. `position` is now the rank among the
|
|
156
|
+
siblings the caller can see, computed as a window over the filtered set so it
|
|
157
|
+
stays correct across pages and at every depth, and both it and `depth` say what
|
|
158
|
+
they are in the tool schema.
|
|
159
|
+
|
|
160
|
+
**`ksor serve` now says when the record has no identity yet.** The MCP door
|
|
161
|
+
already refused to pass an unedited `instance.md` to agents as instructions —
|
|
162
|
+
it substitutes a plain statement that the scope is unstated — but the operator
|
|
163
|
+
starting the server was told nothing, so a record serving with no declared
|
|
164
|
+
identity looked exactly like one that had been described. It is a boot line now,
|
|
165
|
+
beside the abstention posture: both answer "how much should I trust this".
|
|
166
|
+
|
|
167
|
+
- f5cd885: The bearer door's key line joins the boot block instead of interrupting it
|
|
168
|
+
|
|
169
|
+
In bearer mode the line naming where the signing keys were discovered printed
|
|
170
|
+
before the aligned posture block and in a different shape, so it read as a stray
|
|
171
|
+
log line rather than as part of what the server was telling you about itself. It
|
|
172
|
+
is a `keys` row in the block now, under `auth`, resolved at boot exactly as
|
|
173
|
+
before.
|
|
174
|
+
|
|
175
|
+
- 5f30b5f: The site build no longer fails when two evaluations of the record staging overlap
|
|
176
|
+
|
|
177
|
+
The scaffold stages a per-audience copy of the record before the site build
|
|
178
|
+
reads it, removing the previous stage first. `rmSync(..., { force: true })`
|
|
179
|
+
suppresses ENOENT but retries nothing: Node retries EBUSY / EMFILE / ENFILE /
|
|
180
|
+
ENOTEMPTY / EPERM only when `maxRetries` is set, and it defaults to zero. The
|
|
181
|
+
bundler evaluates the source config more than once when it wants it in more than
|
|
182
|
+
one place, so one run could remove the stage while another was still copying
|
|
183
|
+
into it — surfacing as `ENOTEMPTY` and failing the entire site build (seen once
|
|
184
|
+
in CI, 2026-08-21).
|
|
185
|
+
|
|
186
|
+
The removal now asks for those retries. Losing that race is safe: the stage is a
|
|
187
|
+
deterministic function of the record and the denylist, so redoing it produces
|
|
188
|
+
the same bytes.
|
|
189
|
+
|
|
190
|
+
Three claims in the scaffold's `AGENTS.md` that recent releases made false are
|
|
191
|
+
also corrected: `--actor` no longer "defaults to the operating user" (it is
|
|
192
|
+
required, and there is no default by design); the signing keys are discovered
|
|
193
|
+
from the SSO's own metadata rather than fetched from Better Auth's path; and the
|
|
194
|
+
`order:` key now drives the MCP `outline` tool alongside the sidebar and
|
|
195
|
+
`llms.txt`, which is what "one order drives every surface" was always supposed
|
|
196
|
+
to mean.
|
|
197
|
+
|
|
198
|
+
- 4a1c154: The shrink guard guards `ksor ingest --flip` again — it had stopped
|
|
199
|
+
|
|
200
|
+
`.env.example` documents `KSOR_MAX_SHRINK` as "a corpus that shrinks by more
|
|
201
|
+
than this FRACTION refuses to flip". In 0.0.10 it did not. Deleting eight of ten
|
|
202
|
+
documents and running `ksor ingest --flip` published the two that were left,
|
|
203
|
+
silently, exit 0.
|
|
204
|
+
|
|
205
|
+
The cause was the fix that stopped a refused ingest from publishing. That moved
|
|
206
|
+
the flip out of `buildGeneration` and into the command, so the governance gate
|
|
207
|
+
could run against the new generation BEFORE it became the active one — and the
|
|
208
|
+
shrink check, which lived inside the build's flip branch, was stepped straight
|
|
209
|
+
over. The library test that covers the guard stayed green throughout, because it
|
|
210
|
+
drives `buildGeneration` directly with `flip: true`, which is no longer the path
|
|
211
|
+
the CLI takes.
|
|
212
|
+
|
|
213
|
+
There is now one answer to "may this generation be activated" — `flipRefusal` —
|
|
214
|
+
and both flip paths ask it, in the same transaction as the flip itself. The new
|
|
215
|
+
test drives the command rather than the library, so a guard that only one of two
|
|
216
|
+
paths performs fails the tier that proves it.
|
|
217
|
+
|
|
218
|
+
Verified against a live record: a 10 → 2 node build now names all eight removed
|
|
219
|
+
documents, refuses with exit 1, and leaves the previous generation serving.
|
|
220
|
+
|
|
3
221
|
## 0.0.10
|
|
4
222
|
|
|
5
223
|
### Patch Changes
|
package/dist/cli.mjs
CHANGED
|
@@ -16,7 +16,7 @@ import { bodyLimit } from "hono/body-limit";
|
|
|
16
16
|
import { execFileSync, spawnSync } from "node:child_process";
|
|
17
17
|
import { parseArgs } from "node:util";
|
|
18
18
|
import { readFile, readdir, stat } from "node:fs/promises";
|
|
19
|
-
//#region ../content-gateway/dist/main-
|
|
19
|
+
//#region ../content-gateway/dist/main-CqOtG-q1.mjs
|
|
20
20
|
/**
|
|
21
21
|
* A connection could not be ESTABLISHED in time — retryable.
|
|
22
22
|
*
|
|
@@ -93,6 +93,20 @@ function isLoopbackHost$1(hostname) {
|
|
|
93
93
|
return host === "" || host === "localhost" || host === "127.0.0.1" || host === "::1";
|
|
94
94
|
}
|
|
95
95
|
/**
|
|
96
|
+
* The sslmode the DRIVER will use, which is the LAST one written.
|
|
97
|
+
*
|
|
98
|
+
* `URLSearchParams.get` returns the FIRST value; `pg` takes the last. On
|
|
99
|
+
* `?sslmode=require&sslmode=disable` those disagree, and reading the first made
|
|
100
|
+
* the pin treat an explicitly disabled connection as a weak one — collapsing the
|
|
101
|
+
* duplicates into a single `verify-full`, turning TLS on, and printing "verified"
|
|
102
|
+
* at an operator whose DSN ended in `disable`. The direction was safe; silently
|
|
103
|
+
* overruling an explicit opt-out and then misreporting it is not (found by
|
|
104
|
+
* sweeping the driver's own parser, 2026-08-21).
|
|
105
|
+
*/
|
|
106
|
+
function effectiveSslMode$1(url) {
|
|
107
|
+
return (url.searchParams.getAll("sslmode").at(-1) ?? "").toLowerCase();
|
|
108
|
+
}
|
|
109
|
+
/**
|
|
96
110
|
* The DSN ksor actually connects with — the weak sslmode SPELLED OUT.
|
|
97
111
|
*
|
|
98
112
|
* pg 8 treats `sslmode=require|prefer|verify-ca` as aliases for `verify-full`,
|
|
@@ -116,8 +130,7 @@ function pinnedTlsDsn$1(dsn) {
|
|
|
116
130
|
return dsn;
|
|
117
131
|
}
|
|
118
132
|
if (isLoopbackHost$1(url.hostname)) return dsn;
|
|
119
|
-
|
|
120
|
-
if (!WEAK_SSLMODES$1.includes(mode)) return dsn;
|
|
133
|
+
if (!WEAK_SSLMODES$1.includes(effectiveSslMode$1(url))) return dsn;
|
|
121
134
|
url.searchParams.set("sslmode", "verify-full");
|
|
122
135
|
return url.toString();
|
|
123
136
|
}
|
|
@@ -133,7 +146,7 @@ function tlsPosture(dsn) {
|
|
|
133
146
|
return null;
|
|
134
147
|
}
|
|
135
148
|
if (isLoopbackHost$1(url.hostname)) return null;
|
|
136
|
-
const mode = (url
|
|
149
|
+
const mode = effectiveSslMode$1(url);
|
|
137
150
|
if (mode === "disable") return "TLS off (sslmode=disable)";
|
|
138
151
|
if (mode === "no-verify") return "TLS UNVERIFIED (sslmode=no-verify)";
|
|
139
152
|
if (WEAK_SSLMODES$1.includes(mode)) return `TLS verified (sslmode=${mode} pinned to verify-full)`;
|
|
@@ -189,7 +202,7 @@ function tlsOptionsFor$1(dsn) {
|
|
|
189
202
|
return;
|
|
190
203
|
}
|
|
191
204
|
if (isLoopbackHost$1(url.hostname)) return void 0;
|
|
192
|
-
const mode = (url
|
|
205
|
+
const mode = effectiveSslMode$1(url);
|
|
193
206
|
if (mode === "disable" || mode === "no-verify") return void 0;
|
|
194
207
|
return { rejectUnauthorized: true };
|
|
195
208
|
}
|
|
@@ -435,6 +448,8 @@ const CHUNK_POLICY$1 = "heading-aware-1500-content-only-v5";
|
|
|
435
448
|
* The Markdown BODY below the frontmatter is the authored agent-surface
|
|
436
449
|
* instructions — byte-preserved, stripped only at the edges.
|
|
437
450
|
*/
|
|
451
|
+
/** Mirrors `schema.ts`'s ceiling, so a bad `dim:` is refused when instance.md is
|
|
452
|
+
* PARSED rather than when the DDL is rendered. The why lives there. */
|
|
438
453
|
const EMBED_DIM_MAX$1$1 = 2e3;
|
|
439
454
|
const SUPPORTED_FORMATS$1$1 = [1];
|
|
440
455
|
var InstanceParseError$1 = class extends Error {
|
|
@@ -528,7 +543,7 @@ const groupSchemas$1 = {
|
|
|
528
543
|
embedding: z.object({
|
|
529
544
|
provider: z.string().min(1).default("gemini"),
|
|
530
545
|
model: z.string().min(1).default(EMBED_MODEL$1),
|
|
531
|
-
dim: z.coerce.number().int().min(1).max(EMBED_DIM_MAX$1$1).default(EMBED_DIM$1)
|
|
546
|
+
dim: z.coerce.number().int().min(1).max(EMBED_DIM_MAX$1$1, { error: `at most ${EMBED_DIM_MAX$1$1}: this schema declares VECTOR columns and indexes one directly, and pgvector's HNSW takes a vector to ${EMBED_DIM_MAX$1$1}` }).default(EMBED_DIM$1)
|
|
532
547
|
}),
|
|
533
548
|
retrieval: z.object({
|
|
534
549
|
/**
|
|
@@ -2058,7 +2073,22 @@ walk AS (
|
|
|
2058
2073
|
-- gates each row on its OWN visibility (round-9 review of PR 43).
|
|
2059
2074
|
WHERE n.tenant_id = $1 AND n.status = 'published' AND w.depth < $5
|
|
2060
2075
|
)
|
|
2061
|
-
|
|
2076
|
+
-- The rank among the siblings THIS CALLER CAN SEE, not the stored one.
|
|
2077
|
+
--
|
|
2078
|
+
-- content_nodes.position is the rank in the whole record, so a tier that
|
|
2079
|
+
-- cannot see a sibling saw a GAP where it sat -- 1, 3, 4 -- which discloses
|
|
2080
|
+
-- that a document exists and roughly where, to a caller the record refuses to
|
|
2081
|
+
-- show it to. The same row's child_count was already computed over visible
|
|
2082
|
+
-- children only, so one response object disagreed with itself about whether
|
|
2083
|
+
-- hidden siblings are disclosed (found live 2026-08-21).
|
|
2084
|
+
--
|
|
2085
|
+
-- Computed as a WINDOW over the filtered set: window functions run after WHERE
|
|
2086
|
+
-- and before LIMIT/OFFSET, so the rank is the true visible sibling rank on
|
|
2087
|
+
-- every page and at every depth. Doing it in JS would have to renumber a page
|
|
2088
|
+
-- at a time -- which is how this query already produced two paging defects.
|
|
2089
|
+
SELECT w.slug, w.kind, w.title, w.heading_path,
|
|
2090
|
+
row_number() OVER (PARTITION BY w.parent_id ORDER BY w.sort_key)::int AS position,
|
|
2091
|
+
w.depth,
|
|
2062
2092
|
(SELECT count(*) FROM content_nodes ch
|
|
2063
2093
|
WHERE ch.tenant_id = $1 AND ch.generation = w.generation
|
|
2064
2094
|
AND ch.parent_id = w.node_id AND ch.status = 'published'
|
|
@@ -2903,9 +2933,23 @@ const FRAMEWORK_INSTRUCTIONS = `You are answering from a Knowledge System of Rec
|
|
|
2903
2933
|
* replaced it with "has not yet been described" (review of PR #43).
|
|
2904
2934
|
*/
|
|
2905
2935
|
const TEMPLATE_MARKER = "_fill this in; it is";
|
|
2936
|
+
/**
|
|
2937
|
+
* Has the owner said what this record is FOR yet?
|
|
2938
|
+
*
|
|
2939
|
+
* The MCP door already answers honestly when they have not — it replaces the
|
|
2940
|
+
* template with a plain statement that the scope is unstated. But the operator
|
|
2941
|
+
* starting the server was told nothing, so a record serving with no declared
|
|
2942
|
+
* identity looked exactly like one that had been described. The boot report is
|
|
2943
|
+
* where that belongs, beside the abstention posture: both are answers to "how
|
|
2944
|
+
* much should I trust what this thing says".
|
|
2945
|
+
*/
|
|
2946
|
+
function recordIsUndescribed(authored) {
|
|
2947
|
+
const body = authored.trim();
|
|
2948
|
+
return body === "" || body.includes(TEMPLATE_MARKER);
|
|
2949
|
+
}
|
|
2906
2950
|
function composeInstructions(authored) {
|
|
2907
2951
|
const body = authored.trim();
|
|
2908
|
-
return
|
|
2952
|
+
return recordIsUndescribed(authored) ? `${FRAMEWORK_INSTRUCTIONS}
|
|
2909
2953
|
|
|
2910
2954
|
(This record has not yet been described by its owner — instance.md still carries the scaffold template. Treat its scope as unstated.)` : `${FRAMEWORK_INSTRUCTIONS}
|
|
2911
2955
|
|
|
@@ -2955,8 +2999,8 @@ const OUTLINE_OUTPUT = z.object({
|
|
|
2955
2999
|
kind: z.string(),
|
|
2956
3000
|
title: z.string(),
|
|
2957
3001
|
heading_path: z.string(),
|
|
2958
|
-
position: z.number().int(),
|
|
2959
|
-
depth: z.number().int(),
|
|
3002
|
+
position: z.number().int().describe("Rank among the siblings YOU can see, from 1. Rows already arrive in reading order, so this is for citing a place, not for sorting."),
|
|
3003
|
+
depth: z.number().int().describe("Levels below the record's root, so rows are self-locating."),
|
|
2960
3004
|
child_count: z.number().int(),
|
|
2961
3005
|
permalink: z.string().nullable().describe("The page a person can open, when the record publishes one; null otherwise."),
|
|
2962
3006
|
has_content: z.boolean()
|
|
@@ -3568,6 +3612,16 @@ function abstainPosture(floor) {
|
|
|
3568
3612
|
return `floor ${floor} — below it, this record abstains`;
|
|
3569
3613
|
}
|
|
3570
3614
|
/**
|
|
3615
|
+
* What the boot report says when instance.md is still the scaffold template.
|
|
3616
|
+
*
|
|
3617
|
+
* Not a scolding: a level-0 record is allowed to be undescribed and this is not
|
|
3618
|
+
* an error. It is stated because the instance.md body IS the agent surface's
|
|
3619
|
+
* system prompt, so leaving it unwritten is a decision with a runtime effect —
|
|
3620
|
+
* every agent is told this record's scope is unstated — and an operator should
|
|
3621
|
+
* learn that from the server rather than from an agent's answer.
|
|
3622
|
+
*/
|
|
3623
|
+
const UNDESCRIBED_RECORD = "instance.md is still the scaffold template — agents are told this record's scope is unstated; run the intake interview to describe it";
|
|
3624
|
+
/**
|
|
3571
3625
|
* Composition (oracle main.py's boot order, adapted): instance → DSN via
|
|
3572
3626
|
* the declared env NAME → provider → pool → space guard → service context.
|
|
3573
3627
|
* Auth is built by the door that needs it (http.ts) — BEFORE the pool
|
|
@@ -3671,6 +3725,54 @@ async function compose(instancePath, version) {
|
|
|
3671
3725
|
};
|
|
3672
3726
|
}
|
|
3673
3727
|
/**
|
|
3728
|
+
* The body of a deferred-boot refusal, and the one decision inside it: whose
|
|
3729
|
+
* message may go on the wire.
|
|
3730
|
+
*
|
|
3731
|
+
* Three AUTHORED failures carry a remedy written for the operator —
|
|
3732
|
+
* `SchemaVersionError`, `GovernanceGateError`, `TextSearchConfigMismatch`. Their
|
|
3733
|
+
* whole multi-line message is the point; a caller that receives only the first
|
|
3734
|
+
* line has been told a problem exists and not how to end it.
|
|
3735
|
+
*
|
|
3736
|
+
* Everything else reaching this catch is infrastructure — most of it straight
|
|
3737
|
+
* from `pg`, whose connection and authentication failures name the host, the
|
|
3738
|
+
* resolved address, the port and the database user. Those went out verbatim
|
|
3739
|
+
* under `data.detail`, because the catch treated every error alike.
|
|
3740
|
+
*
|
|
3741
|
+
* So the split is by TYPE, not by message inspection: a class we wrote is a
|
|
3742
|
+
* class whose text we control. A driver error is refused with its class named
|
|
3743
|
+
* and its text withheld — the operator finds the real message in the server's
|
|
3744
|
+
* own logs, which is where an infrastructure fault belongs.
|
|
3745
|
+
*/
|
|
3746
|
+
/**
|
|
3747
|
+
* Did WE write this error's text?
|
|
3748
|
+
*
|
|
3749
|
+
* Named classes only. Matching on message prose would put the decision back
|
|
3750
|
+
* inside the strings it is meant to police, and a reworded driver error would
|
|
3751
|
+
* quietly re-open the leak.
|
|
3752
|
+
*/
|
|
3753
|
+
function isAuthored(error) {
|
|
3754
|
+
return error instanceof SchemaVersionError || error instanceof GovernanceGateError$1 || error instanceof TextSearchConfigMismatch;
|
|
3755
|
+
}
|
|
3756
|
+
function refusalBody(error) {
|
|
3757
|
+
if (isAuthored(error)) return {
|
|
3758
|
+
jsonrpc: "2.0",
|
|
3759
|
+
error: {
|
|
3760
|
+
code: -32001,
|
|
3761
|
+
message: `this record cannot be served: ${error.message.split("\n")[0] ?? ""}`,
|
|
3762
|
+
data: { detail: error.message }
|
|
3763
|
+
},
|
|
3764
|
+
id: null
|
|
3765
|
+
};
|
|
3766
|
+
return {
|
|
3767
|
+
jsonrpc: "2.0",
|
|
3768
|
+
error: {
|
|
3769
|
+
code: -32001,
|
|
3770
|
+
message: `this record cannot be served: the content store is unavailable (${error instanceof Error ? error.name : "Error"}). The reason is in this server's logs; it is withheld here because a driver error names the database host and user.`
|
|
3771
|
+
},
|
|
3772
|
+
id: null
|
|
3773
|
+
};
|
|
3774
|
+
}
|
|
3775
|
+
/**
|
|
3674
3776
|
* The MCP door: the SDK v2 HTTP entry (Request → Response, stateless)
|
|
3675
3777
|
* behind Hono, serving the 2026-07-28 revision with 2025-era clients still
|
|
3676
3778
|
* answered through the stateless fallback. Modern exchanges are buffered JSON;
|
|
@@ -3736,10 +3838,11 @@ function resolveSecurity(bind) {
|
|
|
3736
3838
|
}
|
|
3737
3839
|
async function runHttp(composition) {
|
|
3738
3840
|
const auth = buildAuth(process.env);
|
|
3841
|
+
const keyLines = [];
|
|
3739
3842
|
if (auth.mode === "public") {
|
|
3740
3843
|
const keys = await auth.jwks();
|
|
3741
|
-
|
|
3742
|
-
if (keys.advisory !== null)
|
|
3844
|
+
keyLines.push(bootLine("keys", `${keys.source} — ${keys.url}`));
|
|
3845
|
+
if (keys.advisory !== null) keyLines.push(bootLine("", keys.advisory));
|
|
3743
3846
|
}
|
|
3744
3847
|
const resourceMetadataUrl = auth.mode === "public" ? new URL("/.well-known/oauth-protected-resource/mcp", auth.config.resourceUrl).toString() : "";
|
|
3745
3848
|
const bind = resolveBind(process.env);
|
|
@@ -3833,16 +3936,8 @@ async function runHttp(composition) {
|
|
|
3833
3936
|
if (verifyBoot !== null) try {
|
|
3834
3937
|
await verifyBoot();
|
|
3835
3938
|
} catch (error) {
|
|
3836
|
-
|
|
3837
|
-
return new Response(JSON.stringify({
|
|
3838
|
-
jsonrpc: "2.0",
|
|
3839
|
-
error: {
|
|
3840
|
-
code: -32001,
|
|
3841
|
-
message: `this record cannot be served: ${message.split("\n")[0]}`,
|
|
3842
|
-
data: { detail: message }
|
|
3843
|
-
},
|
|
3844
|
-
id: null
|
|
3845
|
-
}), {
|
|
3939
|
+
console.error(`refusing requests — boot checks failing: ${error instanceof Error ? error.stack ?? error.message : String(error)}`);
|
|
3940
|
+
return new Response(JSON.stringify(refusalBody(error)), {
|
|
3846
3941
|
status: 503,
|
|
3847
3942
|
headers: { "content-type": "application/json" }
|
|
3848
3943
|
});
|
|
@@ -3886,7 +3981,7 @@ async function runHttp(composition) {
|
|
|
3886
3981
|
identity = await auth.verify(token);
|
|
3887
3982
|
} catch (error) {
|
|
3888
3983
|
const transient = error instanceof TokenVerifyError && error.transient;
|
|
3889
|
-
return c.json({ error: transient ? "token verification temporarily unavailable" : "invalid token" }, transient ? 503 : 401);
|
|
3984
|
+
return c.json({ error: transient ? "token verification temporarily unavailable" : "invalid token" }, transient ? 503 : 401, transient ? {} : { "www-authenticate": `Bearer error="invalid_token", resource_metadata="${resourceMetadataUrl}"` });
|
|
3890
3985
|
}
|
|
3891
3986
|
bearer = token;
|
|
3892
3987
|
}
|
|
@@ -3925,7 +4020,9 @@ async function runHttp(composition) {
|
|
|
3925
4020
|
});
|
|
3926
4021
|
s.once("error", reject);
|
|
3927
4022
|
});
|
|
4023
|
+
if (recordIsUndescribed(instance.instructions)) console.error(bootLine("identity", UNDESCRIBED_RECORD));
|
|
3928
4024
|
console.error(bootLine("auth", authPosture(auth.mode, bind.host)));
|
|
4025
|
+
for (const line of keyLines) console.error(line);
|
|
3929
4026
|
console.error(bootLine("abstain", abstainPosture(instance.abstain.vectorFloor)));
|
|
3930
4027
|
console.error(bootLine("serving", `http://${bind.host}:${bind.port}/mcp`));
|
|
3931
4028
|
let draining = false;
|
|
@@ -4080,6 +4177,20 @@ function isLoopbackHost(hostname) {
|
|
|
4080
4177
|
return host === "" || host === "localhost" || host === "127.0.0.1" || host === "::1";
|
|
4081
4178
|
}
|
|
4082
4179
|
/**
|
|
4180
|
+
* The sslmode the DRIVER will use, which is the LAST one written.
|
|
4181
|
+
*
|
|
4182
|
+
* `URLSearchParams.get` returns the FIRST value; `pg` takes the last. On
|
|
4183
|
+
* `?sslmode=require&sslmode=disable` those disagree, and reading the first made
|
|
4184
|
+
* the pin treat an explicitly disabled connection as a weak one — collapsing the
|
|
4185
|
+
* duplicates into a single `verify-full`, turning TLS on, and printing "verified"
|
|
4186
|
+
* at an operator whose DSN ended in `disable`. The direction was safe; silently
|
|
4187
|
+
* overruling an explicit opt-out and then misreporting it is not (found by
|
|
4188
|
+
* sweeping the driver's own parser, 2026-08-21).
|
|
4189
|
+
*/
|
|
4190
|
+
function effectiveSslMode(url) {
|
|
4191
|
+
return (url.searchParams.getAll("sslmode").at(-1) ?? "").toLowerCase();
|
|
4192
|
+
}
|
|
4193
|
+
/**
|
|
4083
4194
|
* The DSN ksor actually connects with — the weak sslmode SPELLED OUT.
|
|
4084
4195
|
*
|
|
4085
4196
|
* pg 8 treats `sslmode=require|prefer|verify-ca` as aliases for `verify-full`,
|
|
@@ -4103,8 +4214,7 @@ function pinnedTlsDsn(dsn) {
|
|
|
4103
4214
|
return dsn;
|
|
4104
4215
|
}
|
|
4105
4216
|
if (isLoopbackHost(url.hostname)) return dsn;
|
|
4106
|
-
|
|
4107
|
-
if (!WEAK_SSLMODES.includes(mode)) return dsn;
|
|
4217
|
+
if (!WEAK_SSLMODES.includes(effectiveSslMode(url))) return dsn;
|
|
4108
4218
|
url.searchParams.set("sslmode", "verify-full");
|
|
4109
4219
|
return url.toString();
|
|
4110
4220
|
}
|
|
@@ -4158,7 +4268,7 @@ function tlsOptionsFor(dsn) {
|
|
|
4158
4268
|
return;
|
|
4159
4269
|
}
|
|
4160
4270
|
if (isLoopbackHost(url.hostname)) return void 0;
|
|
4161
|
-
const mode = (url
|
|
4271
|
+
const mode = effectiveSslMode(url);
|
|
4162
4272
|
if (mode === "disable" || mode === "no-verify") return void 0;
|
|
4163
4273
|
return { rejectUnauthorized: true };
|
|
4164
4274
|
}
|
|
@@ -4321,7 +4431,7 @@ async function withPgRetry(op, options = {}) {
|
|
|
4321
4431
|
throw lastError;
|
|
4322
4432
|
}
|
|
4323
4433
|
//#endregion
|
|
4324
|
-
//#region ../content/dist/commands-
|
|
4434
|
+
//#region ../content/dist/commands-Cysnkk_R.mjs
|
|
4325
4435
|
/**
|
|
4326
4436
|
* EVAL-LOCKED constants, quarried verbatim from the oracle
|
|
4327
4437
|
* (sor-agentfactory @ b554f91, config.py) — changing any of these is a
|
|
@@ -4357,6 +4467,8 @@ const HARD_MAX_CHARS = 4e3;
|
|
|
4357
4467
|
* The Markdown BODY below the frontmatter is the authored agent-surface
|
|
4358
4468
|
* instructions — byte-preserved, stripped only at the edges.
|
|
4359
4469
|
*/
|
|
4470
|
+
/** Mirrors `schema.ts`'s ceiling, so a bad `dim:` is refused when instance.md is
|
|
4471
|
+
* PARSED rather than when the DDL is rendered. The why lives there. */
|
|
4360
4472
|
const EMBED_DIM_MAX$1 = 2e3;
|
|
4361
4473
|
const SUPPORTED_FORMATS$1 = [1];
|
|
4362
4474
|
var InstanceParseError = class extends Error {
|
|
@@ -4450,7 +4562,7 @@ const groupSchemas = {
|
|
|
4450
4562
|
embedding: z.object({
|
|
4451
4563
|
provider: z.string().min(1).default("gemini"),
|
|
4452
4564
|
model: z.string().min(1).default(EMBED_MODEL),
|
|
4453
|
-
dim: z.coerce.number().int().min(1).max(EMBED_DIM_MAX$1).default(EMBED_DIM)
|
|
4565
|
+
dim: z.coerce.number().int().min(1).max(EMBED_DIM_MAX$1, { error: `at most ${EMBED_DIM_MAX$1}: this schema declares VECTOR columns and indexes one directly, and pgvector's HNSW takes a vector to ${EMBED_DIM_MAX$1}` }).default(EMBED_DIM)
|
|
4454
4566
|
}),
|
|
4455
4567
|
retrieval: z.object({
|
|
4456
4568
|
/**
|
|
@@ -4874,7 +4986,29 @@ async function runIngest(pool, tenantId, op) {
|
|
|
4874
4986
|
* fresh DDL is rendered from the instance that will fill it, never
|
|
4875
4987
|
* hand-edited.
|
|
4876
4988
|
*/
|
|
4877
|
-
/**
|
|
4989
|
+
/**
|
|
4990
|
+
* The largest embedding dimension this schema will render.
|
|
4991
|
+
*
|
|
4992
|
+
* It is the ceiling for the shape we USE, not pgvector's ceiling: `schema.sql`
|
|
4993
|
+
* declares two `VECTOR(dim)` columns and indexes one of them directly, and
|
|
4994
|
+
* pgvector's HNSW and IVFFlat take a `vector` to 2000. They take a `halfvec` to
|
|
4995
|
+
* **4000**, reachable by indexing an expression — `hnsw ((embedding::halfvec(N))
|
|
4996
|
+
* halfvec_cosine_ops)` — which we do not do, so 2000 binds here.
|
|
4997
|
+
*
|
|
4998
|
+
* Said precisely because the old wording ("pgvector vector + HNSW ceiling")
|
|
4999
|
+
* read as pgvector's own limit and sent a reader off to change providers over a
|
|
5000
|
+
* wall that is not one (verified live against a real database, 2026-08-21:
|
|
5001
|
+
* a halfvec(3072) expression index plans an Index Scan).
|
|
5002
|
+
*
|
|
5003
|
+
* Raising it is a decision, not a constant: every query site would have to use
|
|
5004
|
+
* the same cast as the index or fall silently back to a sequential scan, and
|
|
5005
|
+
* the halfvec arm's float16 rounding lands on the score the abstention gate
|
|
5006
|
+
* reads. Recorded in issue #49, along with the evidence for staying at 1536 —
|
|
5007
|
+
* Google's published MTEB table runs 128..2048 and is FLAT at the top of that
|
|
5008
|
+
* range (1536 scores 68.17, 2048 scores 68.16), so there is no gradient to
|
|
5009
|
+
* climb toward the ceiling. It carries no 3072 row, so the cost of the
|
|
5010
|
+
* truncation itself is unpublished; do not infer one.
|
|
5011
|
+
*/
|
|
4878
5012
|
const EMBED_DIM_MAX = 2e3;
|
|
4879
5013
|
/** The schema version schema.sql declares — parsed from the DDL so code and
|
|
4880
5014
|
* the applied database share ONE source (a drift test pins the coupling). */
|
|
@@ -4921,7 +5055,7 @@ function verifyTemplate(text, dim) {
|
|
|
4921
5055
|
}
|
|
4922
5056
|
/** The pure core: render the given template text at the given dimension. */
|
|
4923
5057
|
function renderSchemaText(text, dim, textSearchConfig = SHIPPED_TS_CONFIG) {
|
|
4924
|
-
if (!Number.isInteger(dim) || dim < 1 || dim > 2e3) throw new Error(`dim must be an integer in 1..${EMBED_DIM_MAX}
|
|
5058
|
+
if (!Number.isInteger(dim) || dim < 1 || dim > 2e3) throw new Error(`dim must be an integer in 1..${EMBED_DIM_MAX} — this schema indexes a vector column directly, and pgvector's HNSW takes a vector to 2000 — got ${JSON.stringify(dim)}`);
|
|
4925
5059
|
verifyTemplate(text, EMBED_DIM);
|
|
4926
5060
|
const withTs = renderTsConfig(text, textSearchConfig);
|
|
4927
5061
|
if (dim === 1536) return withTs;
|
|
@@ -8138,6 +8272,39 @@ async function sameCommit(c, tenantId, generation, sourceCommit) {
|
|
|
8138
8272
|
const stored = r.rows[0]?.source_commit ?? null;
|
|
8139
8273
|
return String(stored ?? "") === String(sourceCommit ?? "");
|
|
8140
8274
|
}
|
|
8275
|
+
/**
|
|
8276
|
+
* May this generation be ACTIVATED? Returns the refusal, or null.
|
|
8277
|
+
*
|
|
8278
|
+
* Extracted so there is exactly ONE answer to that question. It used to live
|
|
8279
|
+
* inside `buildGeneration`'s flip branch, which made it unreachable the moment
|
|
8280
|
+
* a caller flipped separately — and `ksor ingest --flip` does, deliberately: the
|
|
8281
|
+
* governance gate has to run against the new generation BEFORE it becomes the
|
|
8282
|
+
* active one. That change silently retired this guard on the CLI path, so a
|
|
8283
|
+
* record that lost 80% of its documents published without a word, while the
|
|
8284
|
+
* library test that covers the guard stayed green because it drives
|
|
8285
|
+
* `buildGeneration` directly (found live 2026-08-21, auditing 0.0.10).
|
|
8286
|
+
*
|
|
8287
|
+
* A pre-flip check that only one of two flip paths performs is not a guard.
|
|
8288
|
+
*/
|
|
8289
|
+
async function flipRefusal(client, options) {
|
|
8290
|
+
const { log } = options;
|
|
8291
|
+
const delta = await flipDelta(client, {
|
|
8292
|
+
tenantId: options.tenantId,
|
|
8293
|
+
corpusId: options.corpusId,
|
|
8294
|
+
newGeneration: options.newGeneration
|
|
8295
|
+
});
|
|
8296
|
+
const added = addedSlugs(delta);
|
|
8297
|
+
const removed = removedSlugs(delta);
|
|
8298
|
+
log(`pre-flip delta vs gen ${delta.priorGeneration}: ${delta.priorSlugs.size} -> ${delta.newSlugs.size} nodes (+${added.length} / -${removed.length})`);
|
|
8299
|
+
if (removed.length > 0) log(` removed: ${JSON.stringify(removed.slice(0, 20))}`);
|
|
8300
|
+
if (added.length > 0) log(` added: ${JSON.stringify(added.slice(0, 20))}`);
|
|
8301
|
+
const configuredShrink = envFloat("KSOR_MAX_SHRINK", .15, 0);
|
|
8302
|
+
const maxShrink = configuredShrink <= 1 ? configuredShrink : .15;
|
|
8303
|
+
if (configuredShrink > 1) log(`KSOR_MAX_SHRINK=${configuredShrink} is not a fraction in [0,1]; using ${maxShrink} (did you mean ${configuredShrink / 100}?)`);
|
|
8304
|
+
const allowed = options.force || process.env["KSOR_ALLOW_SHRINK"] === "1";
|
|
8305
|
+
if (!shrinkUnsafe(delta.priorSlugs.size, delta.newSlugs.size, maxShrink) || allowed) return null;
|
|
8306
|
+
return `REFUSING FLIP: corpus shrank ${pct(shrinkFraction(delta.priorSlugs.size, delta.newSlugs.size))} vs gen ${delta.priorGeneration} (> KSOR_MAX_SHRINK=${pct(maxShrink)}); ${removed.length} node(s) vanished. Generation ${options.newGeneration} is READY but NOT served — the old generation keeps serving. If the drop is intended, re-run with KSOR_ALLOW_SHRINK=1; otherwise fix the corpus and re-ingest.`;
|
|
8307
|
+
}
|
|
8141
8308
|
/** Thrown inside the build transaction to roll it back when nothing changed. */
|
|
8142
8309
|
var UnchangedCorpus = class extends Error {
|
|
8143
8310
|
activeGeneration;
|
|
@@ -8262,26 +8429,19 @@ async function buildGeneration(pool, instance, options) {
|
|
|
8262
8429
|
flipped: false,
|
|
8263
8430
|
refusal: null
|
|
8264
8431
|
};
|
|
8265
|
-
const
|
|
8432
|
+
const refusal = await flipRefusal(c, {
|
|
8266
8433
|
tenantId: tenant,
|
|
8267
8434
|
corpusId: instance.corpusId,
|
|
8268
|
-
newGeneration: generation
|
|
8435
|
+
newGeneration: generation,
|
|
8436
|
+
force: options.force === true,
|
|
8437
|
+
log
|
|
8269
8438
|
});
|
|
8270
|
-
|
|
8271
|
-
const removed = removedSlugs(delta);
|
|
8272
|
-
log(`pre-flip delta vs gen ${delta.priorGeneration}: ${delta.priorSlugs.size} -> ${delta.newSlugs.size} nodes (+${added.length} / -${removed.length})`);
|
|
8273
|
-
if (removed.length > 0) log(` removed: ${JSON.stringify(removed.slice(0, 20))}`);
|
|
8274
|
-
if (added.length > 0) log(` added: ${JSON.stringify(added.slice(0, 20))}`);
|
|
8275
|
-
const configuredShrink = envFloat("KSOR_MAX_SHRINK", .15, 0);
|
|
8276
|
-
const maxShrink = configuredShrink <= 1 ? configuredShrink : .15;
|
|
8277
|
-
if (configuredShrink > 1) log(`KSOR_MAX_SHRINK=${configuredShrink} is not a fraction in [0,1]; using ${maxShrink} (did you mean ${configuredShrink / 100}?)`);
|
|
8278
|
-
const allowed = options.force === true || process.env["KSOR_ALLOW_SHRINK"] === "1";
|
|
8279
|
-
if (shrinkUnsafe(delta.priorSlugs.size, delta.newSlugs.size, maxShrink) && !allowed) return {
|
|
8439
|
+
if (refusal !== null) return {
|
|
8280
8440
|
ready,
|
|
8281
8441
|
centroids,
|
|
8282
8442
|
health,
|
|
8283
8443
|
flipped: false,
|
|
8284
|
-
refusal
|
|
8444
|
+
refusal
|
|
8285
8445
|
};
|
|
8286
8446
|
await flip(c, {
|
|
8287
8447
|
tenantId: tenant,
|
|
@@ -8652,11 +8812,23 @@ async function ingestCommand(args) {
|
|
|
8652
8812
|
const governance = await withPool(dsn, (pool) => assertGovernanceServable(pool, instance, report.generation).then(() => null, (error) => error instanceof Error ? error.message : String(error)));
|
|
8653
8813
|
if (governance !== null) return fail$1(REFUSED, `generation ${report.generation} was built and NOT activated — no surface could serve it\n ${governance.split("\n").join("\n ")}\n note: generation ${report.generation} is left behind, un-activated; \`ksor gc\` reaps it once the grace window passes. The previously active generation still serves.`);
|
|
8654
8814
|
if (values.flip === true && !report.unchanged) {
|
|
8655
|
-
await withPool(dsn, (pool) => runIngest(pool, instance.tenantId, (client) =>
|
|
8656
|
-
|
|
8657
|
-
|
|
8658
|
-
|
|
8659
|
-
|
|
8815
|
+
const refusal = await withPool(dsn, (pool) => runIngest(pool, instance.tenantId, async (client) => {
|
|
8816
|
+
const stop = await flipRefusal(client, {
|
|
8817
|
+
tenantId: instance.tenantId,
|
|
8818
|
+
corpusId: instance.corpusId,
|
|
8819
|
+
newGeneration: report.generation,
|
|
8820
|
+
force: false,
|
|
8821
|
+
log: (line) => process.stdout.write(line + "\n")
|
|
8822
|
+
});
|
|
8823
|
+
if (stop !== null) return stop;
|
|
8824
|
+
await flip(client, {
|
|
8825
|
+
tenantId: instance.tenantId,
|
|
8826
|
+
corpusId: instance.corpusId,
|
|
8827
|
+
toGeneration: report.generation
|
|
8828
|
+
});
|
|
8829
|
+
return null;
|
|
8830
|
+
}));
|
|
8831
|
+
if (refusal !== null) return fail$1(REFUSED, refusal);
|
|
8660
8832
|
process.stdout.write(`FLIPPED active generation -> ${report.generation}\n`);
|
|
8661
8833
|
}
|
|
8662
8834
|
if (values.flip !== true) process.stdout.write("ready; flip withheld (pass --flip to activate)\n");
|
package/docs/index.md
CHANGED
|
@@ -45,8 +45,9 @@ instead of their training memory. The corpus grows with each implemented verb.
|
|
|
45
45
|
Read the scaffold's own `AGENTS.md` first — it is the working contract.
|
|
46
46
|
Knowledge lives in `knowledge/` and never inside the site; frontmatter uses
|
|
47
47
|
a closed key set (`title` + `status` required); `pnpm check` explains any
|
|
48
|
-
violation and how to fix it.
|
|
49
|
-
frontmatter key — never `meta.json` or `sidebar_position
|
|
48
|
+
violation and how to fix it. Reading order is the governed `order:`
|
|
49
|
+
frontmatter key — never `meta.json` or `sidebar_position` — and it drives
|
|
50
|
+
every surface: the sidebar, `llms.txt`, and the MCP `outline` tool. If the
|
|
50
51
|
instance declares an `audiences:` model, documents may carry a
|
|
51
52
|
`visibility:` key and per-audience builds (`KSOR_AUDIENCE=<tier> pnpm
|
|
52
53
|
build`) stage only what that tier may see — publication, not authorship:
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@panaversity/ksor",
|
|
3
|
-
"version": "0.0.
|
|
3
|
+
"version": "0.0.12",
|
|
4
4
|
"description": "Knowledge System of Record — compile governed markdown into a static site for people and an MCP server for AI agents, with citations and measured abstention.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"abstention",
|
|
@@ -65,8 +65,13 @@ Stand it up in this order (each step's errors explain how to fix themselves):
|
|
|
65
65
|
`provider: gemini`, `model: gemini-embedding-001`, `dim: 1536`; write it out
|
|
66
66
|
only to pin the space explicitly or to change it — and note that model and
|
|
67
67
|
dim are the PERSISTED identity of the embedding space, so changing either
|
|
68
|
-
later means re-embedding the whole corpus. Keep `dim` at or below 2000
|
|
69
|
-
|
|
68
|
+
later means re-embedding the whole corpus. Keep `dim` at or below 2000 — the
|
|
69
|
+
schema indexes a `vector` column directly and pgvector's HNSW takes a
|
|
70
|
+
`vector` to 2000. `gemini-embedding-001` emits 3072 by default, so ksor asks
|
|
71
|
+
it for 1536. Google's published MTEB table runs 128–2048 and is flat at the
|
|
72
|
+
top of it — 1536 scores 68.17 against 2048's 68.16 — so there is no gradient
|
|
73
|
+
to climb toward the ceiling; going the other way, 768 costs 0.18 if you want
|
|
74
|
+
the storage back.
|
|
70
75
|
|
|
71
76
|
Leave `retrieval:` out for now — the gate is off and the server says so.
|
|
72
77
|
Turning it on is step 4, AFTER the record is serving.
|
|
@@ -227,7 +232,7 @@ bind, set `KSOR_ALLOWED_HOSTS` / `KSOR_ALLOWED_ORIGINS`; on more than one
|
|
|
227
232
|
replica, set a shared `KSOR_SNAPSHOT_KEYS` (unset ⇒ a per-process key, so a
|
|
228
233
|
search token minted by one replica fails on another).
|
|
229
234
|
|
|
230
|
-
|
|
235
|
+
Three things worth being deliberate about:
|
|
231
236
|
|
|
232
237
|
- **`KSOR_ALLOW_PUBLIC_UNAUTHENTICATED=1` serves your whole record to anyone
|
|
233
238
|
who can reach the port.** It exists for deployments fronted by your own
|
|
@@ -235,11 +240,70 @@ Two things worth being deliberate about:
|
|
|
235
240
|
- **Set `KSOR_SSO_ISSUER` when your SSO stamps a stable `iss`.** Audience is
|
|
236
241
|
always enforced against `KSOR_JWT_ALLOWED_AUDIENCES`; naming the issuer adds
|
|
237
242
|
one more check for the cost of one variable.
|
|
238
|
-
- **
|
|
239
|
-
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
|
|
243
|
+
- **The signing keys are DISCOVERED; you rarely set `KSOR_JWKS_URL`.** The door
|
|
244
|
+
reads your SSO's own metadata document — RFC 8414
|
|
245
|
+
(`/.well-known/oauth-authorization-server`), then OpenID Discovery
|
|
246
|
+
(`/.well-known/openid-configuration`) — so Auth0, Okta, Entra, Keycloak,
|
|
247
|
+
Cognito, Google and Better Auth all work unmodified. The boot report's `keys`
|
|
248
|
+
line names which document answered and where the keys came from; set
|
|
249
|
+
`KSOR_JWKS_URL` only to override that, or when your SSO publishes no metadata
|
|
250
|
+
at all.
|
|
251
|
+
|
|
252
|
+
### What a CLIENT has to do
|
|
253
|
+
|
|
254
|
+
Once the SSO door is configured (the three variables above), the server is an
|
|
255
|
+
OAuth **Resource Server**, which means a client is not told the authorization
|
|
256
|
+
server — it discovers it. Nothing here needs configuring beyond those variables;
|
|
257
|
+
this is what your agents will experience, and what to check when one cannot
|
|
258
|
+
connect. With `KSOR_AUTH_DISABLED=1` — the local default `.env.example` ships —
|
|
259
|
+
none of it applies: there is no challenge and the metadata document answers 404,
|
|
260
|
+
because there is no authorization server to point at.
|
|
261
|
+
|
|
262
|
+
1. The client calls `POST /mcp` with no token and gets **401** carrying
|
|
263
|
+
|
|
264
|
+
```
|
|
265
|
+
WWW-Authenticate: Bearer resource_metadata="https://<your-host>/.well-known/oauth-protected-resource/mcp"
|
|
266
|
+
```
|
|
267
|
+
|
|
268
|
+
That header is the whole handshake: it names a DOCUMENT, not the resource.
|
|
269
|
+
|
|
270
|
+
2. The client fetches that document and finds the record's resource identifier
|
|
271
|
+
and its authorization server:
|
|
272
|
+
|
|
273
|
+
```json
|
|
274
|
+
{
|
|
275
|
+
"resource": "https://<your-host>/mcp",
|
|
276
|
+
"authorization_servers": ["https://your-sso.example.com"]
|
|
277
|
+
}
|
|
278
|
+
```
|
|
279
|
+
|
|
280
|
+
Those two values are `KSOR_MCP_RESOURCE_URL` and `KSOR_SSO_URL`.
|
|
281
|
+
|
|
282
|
+
3. The client gets a token from that authorization server, asking for THIS
|
|
283
|
+
record as the resource (RFC 8707: `resource=https://<your-host>/mcp`), and
|
|
284
|
+
sends it as `Authorization: Bearer <token>`.
|
|
285
|
+
|
|
286
|
+
Every 401 carries that same header, not just the first one — including the one
|
|
287
|
+
your clients will hit most often, a token that expired mid-conversation. It
|
|
288
|
+
arrives as `Bearer error="invalid_token", resource_metadata="…"`, so a client
|
|
289
|
+
knows to refresh rather than to retry the dead token. A **503** is deliberately
|
|
290
|
+
_not_ challenged: an unreachable key set is our outage, not your token's fault,
|
|
291
|
+
and telling a client to re-authenticate over it would send a perfectly good user
|
|
292
|
+
back through a login.
|
|
293
|
+
|
|
294
|
+
The one thing that goes wrong here goes wrong quietly: a token minted for a
|
|
295
|
+
different audience is a perfectly valid token, and this door rejects it. The
|
|
296
|
+
`aud` claim must match one of `KSOR_JWT_ALLOWED_AUDIENCES` — normally the same
|
|
297
|
+
value as `KSOR_MCP_RESOURCE_URL` — because a bearer accepted for any audience is
|
|
298
|
+
a bearer stolen from one service and replayed against this one. If a client
|
|
299
|
+
authenticates fine and still gets 401, compare its token's `aud` against that
|
|
300
|
+
list before looking anywhere else.
|
|
301
|
+
|
|
302
|
+
Tokens must be signed **RS256**; nothing else is accepted, and opaque tokens are
|
|
303
|
+
not supported (there is no introspection call). When your SSO rotates its
|
|
304
|
+
signing keys, an unknown key id answers **503**, not 401 — the token may well be
|
|
305
|
+
good and the door's key set merely stale, so a client should retry rather than
|
|
306
|
+
send the user back through a login.
|
|
243
307
|
|
|
244
308
|
## Withdrawing a document — `ksor takedown`
|
|
245
309
|
|
|
@@ -258,8 +322,12 @@ pnpm exec ksor takedown --instance instance.md --revoke <stable-id>
|
|
|
258
322
|
The stable id is what a search result reports as `provenance.stable_id` — for
|
|
259
323
|
most documents that is `knowledge/<path-without-.md>`. `--subtree` withdraws a
|
|
260
324
|
section and everything beneath it, including documents added later.
|
|
261
|
-
`--actor NAME` names who performed the act in the ledger
|
|
262
|
-
|
|
325
|
+
`--actor NAME` names who performed the act in the ledger, and a denial or a
|
|
326
|
+
revocation is REFUSED without it. There is no default: a name taken from the
|
|
327
|
+
environment reads like a person and is whatever the shell happened to be
|
|
328
|
+
(`runner` under CI, `root` in a container), which is worse than no name at all
|
|
329
|
+
in the one row that exists to record who did this. Read-only modes
|
|
330
|
+
(`--list`, `--ledger`, `--export`) need nothing.
|
|
263
331
|
|
|
264
332
|
**The MCP door stops serving it immediately. The SITE stops at its next
|
|
265
333
|
build** — the site reads a file, not the database, and `pnpm build` refreshes
|
|
@@ -329,8 +397,10 @@ Details in README → Deploying.
|
|
|
329
397
|
takes the position that page declares.
|
|
330
398
|
- Sidebar position is the governed `order:` key: documents that declare it come
|
|
331
399
|
first, ascending; the rest follow in name order.
|
|
332
|
-
- One order drives the sidebar, `llms.txt`,
|
|
333
|
-
link
|
|
400
|
+
- One order drives every surface — the sidebar, `llms.txt`, the home page's
|
|
401
|
+
first-document link, and the MCP `outline` tool an agent reads to decide what
|
|
402
|
+
to read first. Set it once and they agree. The door picks up a reorder at the
|
|
403
|
+
next `pnpm refresh`, which costs no embedding: only the ordering changed.
|
|
334
404
|
- Never `meta.json` or `sidebar_position`: the checker refuses framework files
|
|
335
405
|
in the record, which has to read the same without the site.
|
|
336
406
|
|
|
@@ -42,3 +42,10 @@ act on it.
|
|
|
42
42
|
Ask your coding agent to run the **intake interview** (it knows how — see
|
|
43
43
|
`.agents/skills/intake-interview/`), answer its questions, and let it write
|
|
44
44
|
this document with you.
|
|
45
|
+
|
|
46
|
+
Until you do, `ksor serve` says so — at boot, and to every agent that connects:
|
|
47
|
+
the MCP surface replaces this template with a plain statement that the record's
|
|
48
|
+
scope is unstated, rather than passing authoring guidance to a runtime agent as
|
|
49
|
+
if it were instructions. Nothing breaks, and the record still answers with
|
|
50
|
+
citations; it just cannot tell an agent what it is authoritative FOR, which is
|
|
51
|
+
the one thing that makes an answer worth trusting.
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import { appName, mcpEndpoint, mcpNamespace, recordVersion } from "@/lib/shared";
|
|
1
|
+
import { appName, mcpEndpoint, mcpNamespace, recordDescription, recordVersion } from "@/lib/shared";
|
|
2
2
|
|
|
3
3
|
/**
|
|
4
4
|
* `/.well-known/mcp/server.json` — how an agent DISCOVERS this record's MCP
|
|
@@ -33,7 +33,10 @@ export function GET(): Response {
|
|
|
33
33
|
{
|
|
34
34
|
$schema: SCHEMA,
|
|
35
35
|
name: `${mcpNamespace()}/${appName}`,
|
|
36
|
-
|
|
36
|
+
// The record's OWN account of itself — see recordDescription. A
|
|
37
|
+
// description identical in every ksor record cannot help an agent choose
|
|
38
|
+
// one, and a record with no scope yet says so instead of guessing.
|
|
39
|
+
description: recordDescription(),
|
|
37
40
|
version: recordVersion(),
|
|
38
41
|
// Absent until the owner declares where the server runs — an invented
|
|
39
42
|
// URL is worse than none, because an agent would try it and conclude the
|
|
@@ -59,6 +59,55 @@ function readInstanceTitle(): string {
|
|
|
59
59
|
|
|
60
60
|
export const appTitle: string = readInstanceTitle();
|
|
61
61
|
|
|
62
|
+
/**
|
|
63
|
+
* How this record describes ITSELF, in one line — what an agent reads in a
|
|
64
|
+
* registry listing to decide whether this record can answer its question.
|
|
65
|
+
*
|
|
66
|
+
* It comes from the record's own prose (instance.md's first real paragraph,
|
|
67
|
+
* which the intake interview writes) because the alternative is what shipped
|
|
68
|
+
* before: one hard-coded sentence, byte-identical in every ksor record ever
|
|
69
|
+
* scaffolded, telling a discovering agent nothing that distinguishes this record
|
|
70
|
+
* from any other. "Discoverability determines whether agents find you at all" is
|
|
71
|
+
* a product principle, and a description that cannot discriminate is not
|
|
72
|
+
* discoverability (found live 2026-08-21).
|
|
73
|
+
*
|
|
74
|
+
* An UNDESCRIBED record says so rather than borrowing a confident sentence it
|
|
75
|
+
* has not earned — the same answer the MCP door already gives an agent that
|
|
76
|
+
* connects, so the two surfaces do not disagree about whether this record knows
|
|
77
|
+
* what it is. The marker is the template's own unfilled placeholder, matched on
|
|
78
|
+
* the WHOLE body: a scaffold's first paragraphs are authoring guidance, and
|
|
79
|
+
* reading one of those as the record's scope is worse than admitting there is
|
|
80
|
+
* none.
|
|
81
|
+
*/
|
|
82
|
+
const TEMPLATE_MARKER = "_fill this in; it is";
|
|
83
|
+
|
|
84
|
+
function readInstanceScope(): string | null {
|
|
85
|
+
const text = readFileSync(findInstance(process.cwd()), "utf8");
|
|
86
|
+
const body = text.replace(/^\uFEFF?---\r?\n[\s\S]*?\r?\n---[ \t]*\r?\n?/, "");
|
|
87
|
+
if (body.includes(TEMPLATE_MARKER)) return null;
|
|
88
|
+
const afterHeading = body.replace(/^[\s\S]*?^#[ \t]+.+$/m, "");
|
|
89
|
+
for (const para of afterHeading.split(/\n[ \t]*\n/)) {
|
|
90
|
+
const one = para.trim().replace(/\s+/g, " ");
|
|
91
|
+
if (one === "" || one.startsWith("#") || one.startsWith("-") || one.startsWith(">")) continue;
|
|
92
|
+
const sentence = /^(.+?[.!?])(\s|$)/.exec(one)?.[1] ?? one;
|
|
93
|
+
return sentence.length > 300 ? `${sentence.slice(0, 297)}...` : sentence;
|
|
94
|
+
}
|
|
95
|
+
return null;
|
|
96
|
+
}
|
|
97
|
+
|
|
98
|
+
/** null until the owner has written one — never a guess. */
|
|
99
|
+
export const appScope: string | null = readInstanceScope();
|
|
100
|
+
|
|
101
|
+
/**
|
|
102
|
+
* The one-line description every discovery surface publishes. Built here so the
|
|
103
|
+
* registry document and anything else that needs one cannot drift apart.
|
|
104
|
+
*/
|
|
105
|
+
export function recordDescription(): string {
|
|
106
|
+
return appScope === null
|
|
107
|
+
? `${appTitle} — its owner has not yet described what this record covers.`
|
|
108
|
+
: `${appTitle} — ${appScope}`;
|
|
109
|
+
}
|
|
110
|
+
|
|
62
111
|
/**
|
|
63
112
|
* Where this record's MCP surface is published, if the owner has said.
|
|
64
113
|
*
|
|
@@ -343,13 +343,30 @@ function planStage(recordDir: string, denied: DenylistManifest): StagePlan {
|
|
|
343
343
|
return { files: [...documents, ...assets], documents: documents.length, total };
|
|
344
344
|
}
|
|
345
345
|
|
|
346
|
+
/**
|
|
347
|
+
* Remove the stage, asking for the retries this exact failure needs.
|
|
348
|
+
*
|
|
349
|
+
* `force: true` suppresses ENOENT; it does NOT retry anything. Node retries
|
|
350
|
+
* EBUSY / EMFILE / ENFILE / ENOTEMPTY / EPERM only when `maxRetries` is set,
|
|
351
|
+
* and it defaults to zero. The build evaluates `source.config.ts` more than
|
|
352
|
+
* once when the bundler wants it in more than one place, so two runs can
|
|
353
|
+
* overlap: one removing the stage while the other is still copying into it.
|
|
354
|
+
* That surfaced as `ENOTEMPTY` out of `rmSync` and failed the whole site build
|
|
355
|
+
* (CI, 2026-08-21) — a race that is safe to lose, because the stage is a
|
|
356
|
+
* deterministic function of the record and the denylist, so redoing it produces
|
|
357
|
+
* the same bytes.
|
|
358
|
+
*/
|
|
359
|
+
function removeStage(stageDir: string): void {
|
|
360
|
+
rmSync(stageDir, { recursive: true, force: true, maxRetries: 10, retryDelay: 50 });
|
|
361
|
+
}
|
|
362
|
+
|
|
346
363
|
/** Fill a clean stage with exactly the set this build may publish. */
|
|
347
364
|
function fillStage(recordDir: string, stageDir: string, denied: DenylistManifest): void {
|
|
348
365
|
// The old stage goes first, before any refusal can throw: a refused build
|
|
349
366
|
// that leaves the previous, more permissive stage on disk hands the next
|
|
350
367
|
// careless build a filtered copy nothing governs (review finding,
|
|
351
368
|
// 2026-08-19).
|
|
352
|
-
|
|
369
|
+
removeStage(stageDir);
|
|
353
370
|
const plan = planStage(recordDir, denied);
|
|
354
371
|
// An empty record is its own problem, reported by the page that renders it;
|
|
355
372
|
// an empty AUDIENCE is a misconfiguration that would otherwise surface as
|
|
@@ -468,7 +485,7 @@ export function knowledgeSourceDir(): string {
|
|
|
468
485
|
// A stage left behind by an earlier model would be a filtered copy of the
|
|
469
486
|
// record nothing governs any more — removed before the refusal below can
|
|
470
487
|
// throw, so a refused build never leaves one behind either.
|
|
471
|
-
|
|
488
|
+
removeStage(stageDir);
|
|
472
489
|
refuseVisibilityWithoutAudiences(recordDir);
|
|
473
490
|
return RECORD_DIR;
|
|
474
491
|
}
|