@panaversity/ksor 0.0.48 → 0.0.50

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,278 @@
1
1
  # @panaversity/ksor
2
2
 
3
+ ## 0.0.50
4
+
5
+ ### Patch Changes
6
+
7
+ - f27f947: The three embed tuning variables now take effect when set in `.env`.
8
+ `KSOR_EMBED_TIMEOUT_S`, `KSOR_QUERY_EMBED_TIMEOUT_S` and `KSOR_EMBED_CACHE_MAX`
9
+ were read once at module load — before the CLI applies `.env` in `main()` — so
10
+ a value set there was silently ignored and the default stood. An adopter who
11
+ set `KSOR_EMBED_CACHE_MAX` to fit a small runtime, for instance, still got the
12
+ ~250 MB default cache and could OOM in production with nothing pointing at why.
13
+ The reads now happen at use. Exported shell variables were unaffected and still
14
+ are.
15
+
16
+ ## 0.0.49
17
+
18
+ ### Patch Changes
19
+
20
+ - 5701679: **Three places where the record described a system that was never built.**
21
+ Nothing an adopter runs changes; what changes is whether the decision log can be
22
+ trusted without re-checking it against the code (issues #151 and #180).
23
+
24
+ Decision 13 said the door composes `secureHeaders` / `bodyLimit` middleware.
25
+ `bodyLimit` is real — `content-gateway/src/http.ts:26,522` — but `secureHeaders`
26
+ was never adopted: nothing imports it, and the door sets its own pair by hand
27
+ (HSTS and `x-content-type-options: nosniff`, "nothing else"). The code is right
28
+ and the entry was wrong, so the entry is corrected.
29
+
30
+ The same decision, and guard rule 5's why-comment, said `hono` and
31
+ `@hono/node-server` were "already the SDK's transitive deps, so zero new install
32
+ bytes". True of the 1.x monolith, false since v2 — `@modelcontextprotocol/server`
33
+ 2.0.0 depends on `zod` and `@modelcontextprotocol/core` and nothing else. The
34
+ reason that survives the upgrade is the one already recorded (the SDK's only HTTP
35
+ shape is Web-standard and hono needs no bridge to it); the weight is a cost paid
36
+ deliberately rather than an absence of cost.
37
+
38
+ The README stated OpenTelemetry in the present tense — "tells us what happened",
39
+ "records what the infrastructure did" — with no telemetry code in the tree. It is
40
+ future tense now, with the constraint the row's own wording already implies:
41
+ default auto-instrumentation captures `pg` statement text, and a trace backend is
42
+ a different security boundary from the MCP response.
43
+
44
+ SLSA/Sigstore, two rows above, needed the opposite correction. It is not future:
45
+ `release.yml` sets `id-token: write`, so every release attests the published
46
+ PACKAGE through npm provenance. What is unbuilt is signing a RECORD's own
47
+ `build.lock.json`. Both rows now say which half runs.
48
+
49
+ Also removed: a `publishConfig` block on `@panaversity/ksor-content-gateway`,
50
+ which is `private: true` and is bundled rather than published, so the block could
51
+ never apply.
52
+
53
+ - 7e72d35: **The embedding dimension ceiling is now held equal in both places it is
54
+ declared.**
55
+
56
+ `EMBED_DIM_MAX` is declared twice — in the instance parser, so a bad `dim:` is
57
+ refused when `instance.md` is READ, and in the DDL renderer, so it is refused
58
+ again before any schema is rendered. The split is deliberate and the comment
59
+ beside one calls it a mirror of the other. Nothing held them equal: before this,
60
+ the constant appeared in no test anywhere in the repository, so raising the
61
+ ceiling in one place alone would have left the parser and the renderer refusing
62
+ at different dimensions — one of the two would still have reddened an existing
63
+ wording assertion, and an instance-only edit would have passed everything.
64
+
65
+ The test asserts EQUALITY and never the number, so the ceiling can still move —
66
+ which is the point, because the decision that records why it sits at 2000 prices
67
+ raising it rather than forbidding it.
68
+
69
+ Also in the same area: the emitted `AGENTS.md` carried the benchmark figures
70
+ behind that default with no source and no date, shipped to every adopter. It
71
+ states the constraint an adopter acts on and points at the decision that holds
72
+ the numbers, so the measurement now lives beside the constant it constrains,
73
+ with its provenance, in one place.
74
+
75
+ - b7a7c5b: **Two audit rows were missing the fact that makes them auditable.**
76
+
77
+ `retrieval_log` exists so an operator can say what an act was allowed to see and
78
+ what it answered from. Two rows could not answer that:
79
+
80
+ - **`outline_served` recorded no generation.** Its two siblings,
81
+ `similarity_searched` and `content_served`, both pin one — so the single act
82
+ that hands an agent the SHAPE of the record was the one that could not be
83
+ joined to the publication it described. The projection carried no generation
84
+ to write, so this is a column rather than an extra query: `walk` already
85
+ selects it, and `OUTLINE_COLUMNS` moves 9 → 10 under the same width guard that
86
+ exists because a narrower fixture once hid a truncated projection.
87
+ - **An ANSWERED search recorded no `top_cosine`.** The abstained row has always
88
+ carried it, so the ledger held the deciding score only for queries the gate
89
+ REFUSED — precisely the half that cannot show a floor drifting as a record
90
+ grows. Both rows now record what the decision turned on, whichever way it
91
+ went.
92
+
93
+ An empty outline still records NULL, deliberately: it served no row from any
94
+ generation, which is the same reason `search_abstained` records NULL when
95
+ nothing matched.
96
+
97
+ **And `cleanCut` is deleted.** It was exported and documented as the tool the
98
+ search budget uses to trim an overflowing hit — in the present tense, for a
99
+ caller that has never existed. It came from the predecessor's grain-expansion
100
+ path, a feature ksor deliberately never carried, so it was a mechanism brought
101
+ across without the purpose it existed for.
102
+
103
+ - 50abe52: **`pnpm preview` no longer dies on a URL it cannot parse or a file it cannot
104
+ read.**
105
+
106
+ Two crashes of one shape, both in the emitted preview server. `pipe()` attaches
107
+ its error listener to the destination and never to the source, and
108
+ `decodeURIComponent` throws on a malformed escape — so either failure reached a
109
+ `node:http` request listener with nobody watching, which is an uncaught
110
+ exception. The process exited and left the adopter a dead port and a stack trace
111
+ instead of a page.
112
+
113
+ - **`http://localhost:3000/%`** ended the session. So did `/%zz` and any
114
+ truncated multi-byte escape — the first hostile URL a browser extension or a
115
+ scanner sends.
116
+ - **A file the export cannot read** did the same, with no attacker at all: a
117
+ mode-000 file anywhere under `out/` is a one-request kill, and so is the
118
+ ordinary loop of rebuilding in another pane while the preview runs, where the
119
+ export is torn down and a page re-requests an asset that has gone.
120
+
121
+ A request that cannot be parsed now resolves to nothing, which is what the 404
122
+ path is for. And a file that fails to open answers **500 with a reason**, not a
123
+ blank page: the response head is written on the read stream's `open` event
124
+ rather than before it, so a file that never opens can still be answered
125
+ honestly. (A file that vanished BEFORE the request was already a 404 — the
126
+ resolver stats every candidate — so this is the file that is there and will not
127
+ open.) Writing it first would have produced a complete, valid, EMPTY `200` —
128
+ which a browser renders as a blank page and `fetch().text()` reports as `""` —
129
+ and that is the same silent lie this change exists to stop telling, one layer
130
+ down. A failure PART WAY through, where the head is already out and no status
131
+ is left to send, destroys the connection instead of ending it cleanly, so the
132
+ client sees a truncated response because that is what happened. Either way the
133
+ reason goes to the console.
134
+
135
+ **Two more failures now explain themselves** instead of arriving as stack
136
+ traces: an occupied port names the collision — `dev` defaults to 3000 as well —
137
+ and a `PORT` that is not a port number is refused. That covers more than the
138
+ obvious case: `Number("")` and `Number(" ")` are `0`, so an unset `PORT=` in a
139
+ shell or a compose file used to bind an arbitrary port and print
140
+ `http://localhost:0`, exactly as `PORT=abc` printed `:NaN`.
141
+
142
+ **And the server binds where it says it binds.** `listen(PORT)` with no host
143
+ binds every interface while the log has always printed `localhost`, so the built
144
+ export was reachable from the whole network. It is loopback now, with
145
+ `KSOR_PREVIEW_HOST` as the way out for the cases where reaching it from
146
+ elsewhere is the point — a container published with `-p`, a cloud dev box, or
147
+ the built site on a phone. Set it on the command line: `preview` is plain `node`
148
+ and does not read `.env`.
149
+
150
+ Stated precisely, because a governance claim is the one thing to get exactly
151
+ right in both directions. A DEFAULT build carries no drafts at all (record spec
152
+ §2.5 admits them to no surface of a build), so what a default `out/` exposed is
153
+ the published record. The case that mattered is the one this first missed:
154
+ `KSOR_AUDIENCE=public,<audience> pnpm build`, whose output holds
155
+ audience-restricted documents and which the scaffold's AGENTS.md says "belongs
156
+ behind that audience's own access control, never on a public host" — that `out/`
157
+ was network-reachable from a preview. `KSOR_DRAFTS=show` is the other. `pnpm dev`,
158
+ where drafts live, is `next dev` and unchanged by this.
159
+
160
+ Alongside this, the containment check moved from once-per-request to
161
+ per-candidate. `resolve()` tries three filename shapes, and the third,
162
+ `` `${target}.html` ``, names a sibling of the export root whenever the target
163
+ IS the root. It is reachable only when the export has no `index.html`, so this
164
+ is defence in depth rather than a fixed leak — and it is the case the test now
165
+ builds an export without an index in order to reach, having previously asserted
166
+ the rule against a fixture that could not.
167
+
168
+ - 33b6080: **`ksor schema --apply` no longer loses a ROLE when two run at once.**
169
+
170
+ `schema.sql` creates three roles, and Postgres roles are CLUSTER-GLOBAL — so
171
+ `IF NOT EXISTS ... THEN CREATE ROLE` is check-then-act across every database on
172
+ the instance. Two concurrent applies both see the role absent and both create
173
+ it. Measured on Postgres 17.7 against an empty cluster: **six concurrent applies,
174
+ five failed.**
175
+
176
+ The SQLSTATE that surfaces is `unique_violation` (23505) on
177
+ `pg_authid_rolname_index`, **not** `duplicate_object` (42710) — catching only the
178
+ latter is the obvious fix and does not work. Both are caught now.
179
+
180
+ And each role is created in its **own** `DO` block. A `DO` block is a single
181
+ statement, so an exception anywhere in it rolls the whole block back: three
182
+ roles in one block meant a loser on the first never created the other two, and
183
+ the apply then granted against roles that did not exist.
184
+
185
+ The same check-then-act sat in the 2.2 → 2.3 migration, which `ksor schema
186
+ --apply` also reaches, and is fixed with it.
187
+
188
+ This is not only a test-tier problem, which is why it lives in the DDL: two
189
+ operators provisioning at once, or a deploy step racing a developer, hit it
190
+ identically.
191
+
192
+ **Scoped honestly:** what is fixed is role creation, which is the part that
193
+ raced across SEPARATE databases — the shape two concurrent runs actually have.
194
+ Two applies against the SAME database still race, on `CREATE EXTENSION` and then
195
+ on table creation; `applySchema`'s contract is a fresh database and that is
196
+ unchanged here.
197
+
198
+ - 988d749: **The emitted site builds with webpack, so a real record still deploys.**
199
+
200
+ The scaffold's site build was `next build`, which under the pinned Next 16.2.9
201
+ means Turbopack — and on Vercel's default build machine (4 cores, 8 GB) that
202
+ does not survive a record big enough to prerender a few hundred routes.
203
+ Measured on a real 205-document record, 435 routes: about seven minutes, then
204
+
205
+ ```
206
+ FATAL: An unexpected Turbopack error occurred:
207
+ Failed to write app endpoint /icon.png/route
208
+ - timeout while receiving message from process
209
+ - deadline has elapsed
210
+ ```
211
+
212
+ Nothing in that points at the build command, and it names `/icon.png/route`,
213
+ which is not the problem — the trace's own middle is the PostCSS step. The same
214
+ record compiled in 86s with `next build --webpack`, which is Next 16's
215
+ documented opt-out rather than a workaround: the v16 upgrade guide ships exactly
216
+ this `package.json` line for a project that needs webpack.
217
+
218
+ The scaffold now emits `next build --webpack`. `dev` is unchanged and still
219
+ Turbopack — the failure is production-only, where every route is prerendered at
220
+ once. An existing project takes the fix with `ksor migrate --write-site`, which
221
+ already offers `system/site/package.json`.
222
+
223
+ This also retires an intermittent CI failure: the conformance suites carried a
224
+ retry for `TurbopackInternalError: Input image not found`, a flake in
225
+ Turbopack's static-image metadata pipeline reading the scaffold's `app/icon.png`
226
+ mark. That pipeline is no longer on the production path, so the retry is gone
227
+ rather than left in place — a shim retrying quietly is what would stop the suite
228
+ reporting a real regression.
229
+
230
+ - 988d749: **A Vercel deployment that reports Ready and 404s everywhere now has a written
231
+ diagnosis, and the provenance hint stops blaming the reader.**
232
+
233
+ A deployment can report **Ready**, take the production alias, and serve
234
+ `404: NOT_FOUND` at every path, `llms.txt` included — after an install that ran,
235
+ a `ksor build` that ran, and every route prerendering. The only signal anywhere
236
+ is one build-log line, `WARNING! Build output contains no "functions" or
237
+ "static" directory`.
238
+
239
+ **The cause is not established, and the docs now say so rather than guessing.**
240
+ The Application Preset was the obvious suspect and is measured NOT to be it: two
241
+ live Git-linked projects, one preset `Services` and one preset `Other`, both
242
+ built the `services` block's `site` and `door` and both serve them (`/` 200,
243
+ `llms.txt` 200, `/mcp` 405 from the door). Naming a wrong cause in the deploy
244
+ guide would have sent every future reader to a field that is not the problem.
245
+
246
+ `deploying.md` and the scaffold README now name that failure, quote the warning
247
+ so it is searchable, and record two things verified live on a 205-document record
248
+ (issue #197): patching the project's own `outputDirectory` / `buildCommand` /
249
+ `installCommand` and taking a fresh Git-sourced deployment does **not** fix it —
250
+ `vercel.json` is what Vercel reads — and replacing the `services` block with the
251
+ classic top-level keys does. Two earlier sentences were corrected rather than
252
+ extended: the docs said a wrong preset meant "`/mcp` never exists", which asserted a
253
+ mechanism now measured false, and offered a site-only fallback as project
254
+ settings, which cannot override `vercel.json`.
255
+
256
+ **And `source: unspecified` now names both of its causes.** Only one is "you
257
+ never made a repository"; the other is a record that IS committed and pushed, on
258
+ a machine the `.git` directory never reached, because an upload-based deploy
259
+ excludes it — Vercel's CLI does. `git init` is still offered first, because it is
260
+ still right for the reader who has not made one; what is new is the second line,
261
+ for the reader who has, and who was previously being told to redo work they had
262
+ already done in the one message that governs provenance.
263
+
264
+ **A remedy also stops naming a flag the verb refuses.** `--source-commit` is an
265
+ `ingest` flag; `ksor build` rejects it as an unknown argument and exits 1. Two of
266
+ the five provenance notices offered it regardless of which verb was printing —
267
+ including the one this change is for, read by someone whose upload stripped
268
+ `.git` on the deploy path, for whom it would have turned a provenance warning
269
+ into a failed build. The flag is now offered only by the verb that accepts it,
270
+ and the same correction is applied to the `(dirty)` notice's `ksor build
271
+ --strict`, which had the defect latently. Both are asserted across every gap,
272
+ enumerated from the exported list rather than a copy of it.
273
+
274
+ The emitted `vercel.json` is unchanged.
275
+
3
276
  ## 0.0.48
4
277
 
5
278
  ### Patch Changes
package/README.md CHANGED
@@ -82,8 +82,10 @@ agent surface at all.
82
82
  The architecture: **one governed record** — Markdown in the KSoR Profile of
83
83
  the Open Knowledge Format (OKF) — behind **one governance boundary**,
84
84
  projected through open standards: MCP for agents, `llms.txt` for AI
85
- discovery, OAuth/OIDC for identity, SLSA/Sigstore for publication integrity,
86
- OpenTelemetry for observability.
85
+ discovery, and OAuth/OIDC for identity. Two more name which standard owns a
86
+ boundary rather than a surface that fully runs today: SLSA/Sigstore, which
87
+ attests this npm package through npm provenance but does not yet sign a
88
+ record's own `build.lock.json`, and OpenTelemetry, which emits nothing yet.
87
89
 
88
90
  Full concept, design goals, and project status:
89
91
  **<https://github.com/panaversity/ksor>**
package/dist/cli.mjs CHANGED
@@ -1,6 +1,6 @@
1
1
  #!/usr/bin/env node
2
2
  import { n as resolveCommand, r as verbs, t as exitCodes } from "./src-dqpI-p1a.mjs";
3
- import { A as parseViewer, B as tlsPosture, C as contentPoolMin, D as outlineHandler, E as keyRingFromEnv, F as runProbe$1, H as withPgRetry$1, I as searchHandler, L as servingPolicy$1, M as prewarmPool, N as readHandler, O as parseInstanceText$1, P as recordIsUndescribed, R as storedTextSearchConfig, S as contentPool$1, T as instancePathOf$1, U as withProbeDeadline$1, V as validateViewer, W as z$1, _ as assertGovernanceServable$1, a as GovernanceGateError$1, b as checkEmbeddingSpace$1, c as McpServer$1, d as READ_ONLY, f as READ_OUTPUT, g as TextSearchConfigMismatch, h as TRUST_TIERS$1, i as FLOOR, j as pooledEndpointFor, k as parseTrustFloor, l as MissingProviderKeyError$1, m as SchemaVersionError, n as ContentStoreError$1, o as MAX_OUTLINE_LIMIT, p as SEARCH_OUTPUT, r as EmbeddingSpaceMismatch$1, t as AudienceError$1, u as OUTLINE_OUTPUT, v as assertSchemaCompatible, w as embedQueryVlit, x as composeInstructions, y as buildShippedProvider$1, z as tallyHandlers } from "./gateway-api-CF4ED9_g-BQusM_dK.mjs";
3
+ import { A as parseViewer, B as tlsPosture, C as contentPoolMin, D as outlineHandler, E as keyRingFromEnv, F as runProbe$1, H as withPgRetry$1, I as searchHandler, L as servingPolicy$1, M as prewarmPool, N as readHandler, O as parseInstanceText$1, P as recordIsUndescribed, R as storedTextSearchConfig, S as contentPool$1, T as instancePathOf$1, U as withProbeDeadline$1, V as validateViewer, W as z$1, _ as assertGovernanceServable$1, a as GovernanceGateError$1, b as checkEmbeddingSpace$1, c as McpServer$1, d as READ_ONLY, f as READ_OUTPUT, g as TextSearchConfigMismatch, h as TRUST_TIERS$1, i as FLOOR, j as pooledEndpointFor, k as parseTrustFloor, l as MissingProviderKeyError$1, m as SchemaVersionError, n as ContentStoreError$1, o as MAX_OUTLINE_LIMIT, p as SEARCH_OUTPUT, r as EmbeddingSpaceMismatch$1, t as AudienceError$1, u as OUTLINE_OUTPUT, v as assertSchemaCompatible, w as embedQueryVlit, x as composeInstructions, y as buildShippedProvider$1, z as tallyHandlers } from "./gateway-api-D8HlLys2-Ca8OnwLn.mjs";
4
4
  import { appendFileSync, chmodSync, copyFileSync, existsSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, renameSync, rmSync, statSync, unlinkSync, writeFileSync } from "node:fs";
5
5
  import { fileURLToPath, pathToFileURL } from "node:url";
6
6
  import { InMemoryTransport, LATEST_PROTOCOL_VERSION, createMcpHandler } from "@modelcontextprotocol/server";
@@ -17,7 +17,7 @@ import { Document, YAMLParseError, isCollection, isMap, isPair, isSeq, parseAllD
17
17
  import { parseArgs } from "node:util";
18
18
  import { readFile } from "node:fs/promises";
19
19
  import { execFileSync, spawnSync } from "node:child_process";
20
- //#region ../content-gateway/dist/main-fqHB0gE_.mjs
20
+ //#region ../content-gateway/dist/main-DMCMO8ye.mjs
21
21
  /**
22
22
  * The default registration — and the ORIGINAL of the file `ksor init` emits.
23
23
  *
@@ -4709,7 +4709,7 @@ async function withPgRetry(op, options = {}) {
4709
4709
  throw lastError;
4710
4710
  }
4711
4711
  //#endregion
4712
- //#region ../content/dist/commands-AMitIS8m.mjs
4712
+ //#region ../content/dist/commands-BNxmBRjs.mjs
4713
4713
  /**
4714
4714
  * EVAL-LOCKED constants, quarried verbatim from the oracle
4715
4715
  * (sor-agentfactory @ b554f91, config.py) — changing any of these is a
@@ -5131,11 +5131,15 @@ async function runIngest(pool, tenantId, op) {
5131
5131
  * Raising it is a decision, not a constant: every query site would have to use
5132
5132
  * the same cast as the index or fall silently back to a sequential scan, and
5133
5133
  * the halfvec arm's float16 rounding lands on the score the abstention gate
5134
- * reads. Recorded in issue #49, along with the evidence for staying at 1536 —
5135
- * Google's published MTEB table runs 128..2048 and is FLAT at the top of that
5136
- * range (1536 scores 68.17, 2048 scores 68.16), so there is no gradient to
5137
- * climb toward the ceiling. It carries no 3072 row, so the cost of the
5138
- * truncation itself is unpublished; do not infer one.
5134
+ * reads. AGENTS.md decision 30 records the choice; the measurement behind it
5135
+ * lives HERE, beside the constant it constrains, and this is its only copy.
5136
+ *
5137
+ * Google's per-dimension table
5138
+ * (https://ai.google.dev/gemini-api/docs/embeddings, retrieved 2026-08-27)
5139
+ * runs 128..2048 and is FLAT at the top of that range: 1536 scores 68.17 MTEB
5140
+ * against 2048's 68.16. So there is no gradient to climb toward the ceiling
5141
+ * from where we sit. It carries no 3072 row, so the cost of the truncation
5142
+ * itself is unpublished; do not infer one.
5139
5143
  */
5140
5144
  const EMBED_DIM_MAX = 2e3;
5141
5145
  /** The schema version schema.sql declares — parsed from the DDL so code and
@@ -5783,11 +5787,12 @@ async function topOneScore(client, scope, queryVector) {
5783
5787
  })).rows[0]?.[0];
5784
5788
  return raw === void 0 || raw === null ? null : toNumber(raw, "score");
5785
5789
  }
5786
- const EMBED_TIMEOUT_S = envFloat("KSOR_EMBED_TIMEOUT_S", 60, 1);
5790
+ /** The per-request HTTP timeout for a document (ingest/batch) embed. */
5791
+ const EMBED_TIMEOUT_S = () => envFloat("KSOR_EMBED_TIMEOUT_S", 60, 1);
5787
5792
  /** Oracle env var: SOR_QUERY_EMBED_TIMEOUT_S. Note: query-embed.ts reads the
5788
5793
  * SAME variable with a different default (5.0) as its hard wall clock — two
5789
5794
  * deliberate reads, carried from the oracle (embedding.py:62 vs query_embed.py:44). */
5790
- const QUERY_EMBED_TIMEOUT_S = envFloat("KSOR_QUERY_EMBED_TIMEOUT_S", 10, 1);
5795
+ const QUERY_EMBED_TIMEOUT_S = () => envFloat("KSOR_QUERY_EMBED_TIMEOUT_S", 10, 1);
5791
5796
  /** The text we embed for a chunk: the readable hierarchical heading path, then the content. */
5792
5797
  function embedInput(title, headingPath, content) {
5793
5798
  const path = headingPath ? (title ? title + " > " : "") + headingPath.replaceAll("/", " > ").replaceAll("-", " ").trim() : title;
@@ -6209,8 +6214,8 @@ function buildShippedProvider(name, opts) {
6209
6214
  dim: opts.dim ?? 1536,
6210
6215
  documentTaskLabel: EMBED_TASK_DOCUMENT,
6211
6216
  queryTaskLabel: EMBED_TASK_QUERY,
6212
- documentTimeoutS: EMBED_TIMEOUT_S,
6213
- queryTimeoutS: QUERY_EMBED_TIMEOUT_S
6217
+ documentTimeoutS: EMBED_TIMEOUT_S(),
6218
+ queryTimeoutS: QUERY_EMBED_TIMEOUT_S()
6214
6219
  });
6215
6220
  }
6216
6221
  /** Written to yield ZERO ROWS, never an error, when the schema is absent. */
@@ -9806,15 +9811,25 @@ function provenanceGap(knowledgeDir) {
9806
9811
  /**
9807
9812
  * The remedy for each, because the reader's next command differs.
9808
9813
  *
9809
- * `subject` is the artefact whose provenance is missing the noun is the only
9810
- * thing that differs between the verbs, so it is the only thing parameterised.
9814
+ * `subject` is the artefact whose provenance is missing. It began as the noun
9815
+ * alone — "the only thing that differs between the verbs" and that was wrong:
9816
+ * the ESCAPE HATCH differs too. `--source-commit` is an `ingest` flag
9817
+ * (`commands.ts`); `ksor build`'s `parseArgs` refuses it as an unknown argument
9818
+ * and exits 1. So a remedy that named it unconditionally told a `build` reader
9819
+ * to run `ksor build --source-commit <sha>` and get `error: bad-args` — turning
9820
+ * a provenance WARNING into a failed build for anyone who followed it, in the
9821
+ * message product principle 4 asks to be documentation. Both are parameterised
9822
+ * now, and a verb that gains the flag renders the escape by declaring so here.
9811
9823
  */
9812
9824
  function provenanceNotice(gap, subject = "generation") {
9813
9825
  const why = `so this ${subject} cannot be traced back to a reviewed commit`;
9826
+ const offersSourceCommit = subject === "generation";
9814
9827
  switch (gap) {
9815
9828
  case "no-commit": return `source: unspecified — knowledge/ is in a git repository with no commits yet, ${why}.\n fix: commit the record (git add knowledge && git commit) and re-run`;
9816
- case "no-repo": return `source: unspecified — knowledge/ is not in a git repository, ${why}.\n fix: git init, commit the record, and re-run`;
9817
- case "no-git": return `source: unspecified — git is not installed, ${why}.\n fix: install git, or pass --source-commit <sha> if the record is versioned elsewhere`;
9829
+ case "no-repo": return `source: unspecified — knowledge/ is not in a git repository, ${why}.\n fix: git init, commit the record, and re-run
9830
+ if it IS committed: .git did not reach this machine an upload-based deploy
9831
+ excludes it (Vercel's CLI does). Deploy from the Git connection instead` + (offersSourceCommit ? ",\n or pass --source-commit <sha>" : "");
9832
+ case "no-git": return `source: unspecified — git is not installed, ${why}.\n fix: install git` + (offersSourceCommit ? ", or pass --source-commit <sha> if the record is versioned elsewhere" : "");
9818
9833
  case "no-input-commit": return `source: unspecified — no commit touches the record (knowledge/, instance.md, .ksor/governance.yaml, .ksor/takedowns.yaml), ${why}.\n fix: commit the record (git add knowledge instance.md .ksor && git commit) and re-run`;
9819
9834
  case "not-asked": return `source: unspecified — no knowledge directory was given, ${why}.`;
9820
9835
  }
@@ -9865,7 +9880,7 @@ function detectSourceCommit(knowledgeDir) {
9865
9880
  * this record that looks like provenance and is not.
9866
9881
  */
9867
9882
  function dirtyNotice(commit, subject = "build") {
9868
- return `source: ${commit} (dirty) — an input differs from that commit, so it does not contain the bytes this ${subject} published.\n fix: commit the inputs (git add -A && git commit) and re-run; \`ksor build --strict\` refuses this state instead of stamping it`;
9883
+ return `source: ${commit} (dirty) — an input differs from that commit, so it does not contain the bytes this ${subject} published.\n fix: commit the inputs (git add -A && git commit) and re-run${subject === "build" ? "; `ksor build --strict` refuses this state instead of stamping it" : ""}`;
9869
9884
  }
9870
9885
  /** One transaction (tenant GUC + ingest role): list collectables, then reap each. */
9871
9886
  async function runGc(pool, instance, options = {}) {
@@ -6,7 +6,7 @@ import { z, z as z$1 } from "zod";
6
6
  import path, { join } from "node:path";
7
7
  import { createHash, createHmac, randomBytes, timingSafeEqual } from "node:crypto";
8
8
  import pg from "pg";
9
- //#region ../content-gateway/dist/gateway-api-CF4ED9_g.mjs
9
+ //#region ../content-gateway/dist/gateway-api-D8HlLys2.mjs
10
10
  var __commonJSMin = (cb, mod) => () => (mod || (cb((mod = { exports: {} }).exports, mod), cb = null), mod.exports);
11
11
  var __require = /* #__PURE__ */ (() => createRequire(import.meta.url))();
12
12
  /**
@@ -8769,11 +8769,12 @@ function validate$1(ring, token, scope, nowMs = Date.now()) {
8769
8769
  reason: null
8770
8770
  };
8771
8771
  }
8772
- const EMBED_TIMEOUT_S = envFloat("KSOR_EMBED_TIMEOUT_S", 60, 1);
8772
+ /** The per-request HTTP timeout for a document (ingest/batch) embed. */
8773
+ const EMBED_TIMEOUT_S = () => envFloat("KSOR_EMBED_TIMEOUT_S", 60, 1);
8773
8774
  /** Oracle env var: SOR_QUERY_EMBED_TIMEOUT_S. Note: query-embed.ts reads the
8774
8775
  * SAME variable with a different default (5.0) as its hard wall clock — two
8775
8776
  * deliberate reads, carried from the oracle (embedding.py:62 vs query_embed.py:44). */
8776
- const QUERY_EMBED_TIMEOUT_S = envFloat("KSOR_QUERY_EMBED_TIMEOUT_S", 10, 1);
8777
+ const QUERY_EMBED_TIMEOUT_S = () => envFloat("KSOR_QUERY_EMBED_TIMEOUT_S", 10, 1);
8777
8778
  function l2Normalize(v) {
8778
8779
  const norm = Math.sqrt(v.reduce((acc, x) => acc + x * x, 0));
8779
8780
  return norm === 0 ? v : v.map((x) => x / norm);
@@ -9120,8 +9121,8 @@ function buildShippedProvider(name, opts) {
9120
9121
  dim: opts.dim ?? 1536,
9121
9122
  documentTaskLabel: EMBED_TASK_DOCUMENT,
9122
9123
  queryTaskLabel: EMBED_TASK_QUERY,
9123
- documentTimeoutS: EMBED_TIMEOUT_S,
9124
- queryTimeoutS: QUERY_EMBED_TIMEOUT_S
9124
+ documentTimeoutS: EMBED_TIMEOUT_S(),
9125
+ queryTimeoutS: QUERY_EMBED_TIMEOUT_S()
9125
9126
  });
9126
9127
  }
9127
9128
  /** Written to yield ZERO ROWS, never an error, when the schema is absent. */
@@ -9726,7 +9727,8 @@ SELECT w.slug, w.kind, w.title, w.heading_path,
9726
9727
  EXISTS (SELECT 1 FROM sources s
9727
9728
  WHERE s.tenant_id = $1 AND s.generation = w.generation
9728
9729
  AND s.node_id = w.node_id) AS has_content,
9729
- w.permalink
9730
+ w.permalink,
9731
+ w.generation
9730
9732
  FROM walk w
9731
9733
  JOIN content_nodes n ON n.node_id = w.node_id AND n.tenant_id = $1
9732
9734
  AND n.generation = w.generation
@@ -9875,7 +9877,7 @@ async function documentFrontmatter(client, scope, nodeId) {
9875
9877
  return value === void 0 || value === null ? null : String(value);
9876
9878
  }
9877
9879
  function outlineRows(result) {
9878
- if (result.fields.length !== 9) throw new TypeError(`outline projection drift: expected 9 columns, got ${result.fields.length} (${result.fields.map((f) => f.name).join(", ")})`);
9880
+ if (result.fields.length !== 10) throw new TypeError(`outline projection drift: expected 10 columns, got ${result.fields.length} (${result.fields.map((f) => f.name).join(", ")})`);
9879
9881
  return result.rows.map((row) => ({
9880
9882
  slug: String(row[0]),
9881
9883
  kind: String(row[1]),
@@ -9885,7 +9887,8 @@ function outlineRows(result) {
9885
9887
  depth: toNumber(row[5], "depth"),
9886
9888
  childCount: toNumber(row[6], "child_count"),
9887
9889
  hasContent: Boolean(row[7]),
9888
- permalink: row[8] === null ? null : String(row[8])
9890
+ permalink: row[8] === null ? null : String(row[8]),
9891
+ generation: toNumber(row[9], "generation")
9889
9892
  }));
9890
9893
  }
9891
9894
  /**
@@ -9928,6 +9931,13 @@ const OUTLINE_CEILING = 5002;
9928
9931
  * ROOT-ABSOLUTE depth + breadcrumb (the wire contract: rows are
9929
9932
  * self-locating; a leaf with no children returns an empty list — the
9930
9933
  * anchor itself is never echoed back).
9934
+ *
9935
+ * Returns the GENERATION it served from alongside the rows, because [] cannot
9936
+ * carry one. A drill-down onto a leaf resolves its anchor and pins that
9937
+ * generation, then returns no rows — and the tool description tells an agent to
9938
+ * drill in, so that is a routine call, not an edge case. Reading the generation
9939
+ * off `rows[0]` recorded NULL for every one of them. `content_served` sets the
9940
+ * precedent: pin what was RESOLVED, however little came back.
9931
9941
  */
9932
9942
  async function outline(client, scope, options = {}) {
9933
9943
  const root = options.root ?? null;
@@ -9963,7 +9973,10 @@ async function outline(client, scope, options = {}) {
9963
9973
  offset
9964
9974
  ]
9965
9975
  }));
9966
- if (root === null) return rows;
9976
+ if (root === null) return {
9977
+ rows,
9978
+ generation: pinned ?? rows[0]?.generation ?? null
9979
+ };
9967
9980
  const anchorRow = (await arrayQuery(client, {
9968
9981
  text: UP_WALK_SQL,
9969
9982
  values: [
@@ -9974,11 +9987,51 @@ async function outline(client, scope, options = {}) {
9974
9987
  ]
9975
9988
  })).rows[0];
9976
9989
  if (anchorRow === void 0) throw new Error(`no node with slug ${JSON.stringify(root)} — browse from the root with outline() (omit node=)`);
9977
- return rebaseOutlineRows(rows, String(anchorRow[0]), toNumber(anchorRow[1], "climbed"));
9990
+ return {
9991
+ rows: rebaseOutlineRows(rows, String(anchorRow[0]), toNumber(anchorRow[1], "climbed")),
9992
+ generation: pinned ?? rows[0]?.generation ?? null
9993
+ };
9978
9994
  }
9979
- let cacheMax = envInt("KSOR_EMBED_CACHE_MAX", 1e4, 1);
9980
- /** Oracle env var: SOR_QUERY_EMBED_TIMEOUT_S. */
9981
- const EMBED_WALL_TIMEOUT_S = envFloat("KSOR_QUERY_EMBED_TIMEOUT_S", 5, .1);
9995
+ /**
9996
+ * Query-embedding cache, converted from the oracle (sor-agentfactory @
9997
+ * b554f91, sor_content/lib/query_embed.py): L1 in-process LRU + SINGLE-FLIGHT
9998
+ * (concurrent identical misses share ONE paid embed), keyed with
9999
+ * model + task + dim so a model or dimension bump can never serve a stale
10000
+ * vector. Whitespace collapses; case does NOT fold (folding would change the
10001
+ * embedded text).
10002
+ *
10003
+ * A tiny CIRCUIT BREAKER guards the provider: after an embed failure, further
10004
+ * misses raise immediately for a short cooldown (cache hits still serve) —
10005
+ * during an outage every request degrades to keyword-only instantly instead
10006
+ * of each unique query paying its own failed attempt against a provider that
10007
+ * is already down. The breaker is keyed per SPACE (modelId, dim), like the
10008
+ * keys: a failing provider A must not degrade a healthy provider B.
10009
+ *
10010
+ * Conversion notes (decision 6):
10011
+ * - The oracle's optional Redis L2 (fail-open both directions, TTL
10012
+ * SOR_EMBED_CACHE_TTL, the `sor:emb:*` key scheme) is DROPPED — it was
10013
+ * multi-instance infrastructure; the L1 + single-flight carry a
10014
+ * single-process deployment. It returns, if ever, with real multi-instance
10015
+ * serving — nothing here forecloses it.
10016
+ * - The oracle's waiter-shield (`asyncio.shield`) and owner-cancel handling
10017
+ * protected the shared future from one caller's cancellation. JS promises
10018
+ * are not cancellable, so sharing the promise IS the whole mechanism: no
10019
+ * caller can cancel another, and there is no owner-cancelled path to map.
10020
+ * - `asyncio.wait_for` CANCELLED the embed on timeout; a JS promise cannot be
10021
+ * cancelled, so on timeout the losing call is abandoned (its settlement is
10022
+ * still observed, so it can never surface as an unhandled rejection) and
10023
+ * runs out its own HTTP timeout in the background.
10024
+ * - The breaker clock is Date.now() (ms) rather than a monotonic clock — the
10025
+ * 10 s cooldown is coarse, and fake-timer tests need the system clock.
10026
+ */
10027
+ /** Oracle env var: SOR_EMBED_CACHE_MAX. */
10028
+ let memoizedCacheMax;
10029
+ function currentCacheMax() {
10030
+ return memoizedCacheMax ??= envInt("KSOR_EMBED_CACHE_MAX", 1e4, 1);
10031
+ }
10032
+ /** Oracle env var: SOR_QUERY_EMBED_TIMEOUT_S. Read at use, not at module load
10033
+ * — see the cache-max note above. */
10034
+ const EMBED_WALL_TIMEOUT_S = () => envFloat("KSOR_QUERY_EMBED_TIMEOUT_S", 5, .1);
9982
10035
  const cache = /* @__PURE__ */ new Map();
9983
10036
  const inflight = /* @__PURE__ */ new Map();
9984
10037
  const breakerOpenUntilByMs = /* @__PURE__ */ new Map();
@@ -10022,10 +10075,11 @@ function breakerOpenUntil(provider) {
10022
10075
  return breakerOpenUntilByMs.get(spaceKey(provider)) ?? 0;
10023
10076
  }
10024
10077
  function withWallClock(work) {
10078
+ const wallTimeoutS = EMBED_WALL_TIMEOUT_S();
10025
10079
  return new Promise((resolve, reject) => {
10026
10080
  const timer = setTimeout(() => {
10027
- reject(new QueryEmbedTimeoutError(`query embed exceeded the ${EMBED_WALL_TIMEOUT_S}s wall clock — treated as a provider failure (degrade to keyword-only)`));
10028
- }, EMBED_WALL_TIMEOUT_S * 1e3);
10081
+ reject(new QueryEmbedTimeoutError(`query embed exceeded the ${wallTimeoutS}s wall clock — treated as a provider failure (degrade to keyword-only)`));
10082
+ }, wallTimeoutS * 1e3);
10029
10083
  work.then((value) => {
10030
10084
  clearTimeout(timer);
10031
10085
  resolve(value);
@@ -10046,7 +10100,7 @@ async function embedMiss(normalized, key, provider) {
10046
10100
  const literal = vlit(vec);
10047
10101
  cache.delete(key);
10048
10102
  cache.set(key, literal);
10049
- while (cache.size > cacheMax) {
10103
+ while (cache.size > currentCacheMax()) {
10050
10104
  const oldest = cache.keys().next().value;
10051
10105
  if (oldest === void 0) break;
10052
10106
  cache.delete(oldest);
@@ -10409,6 +10463,7 @@ async function search(ctx, query, k = 10) {
10409
10463
  query_chars: queryChars,
10410
10464
  k,
10411
10465
  k_effective: kb,
10466
+ top_cosine: topCosine,
10412
10467
  slugs: [...new Set(shaped.map((h) => h.slug))],
10413
10468
  truncated,
10414
10469
  degraded: degradedReason !== void 0
@@ -10579,7 +10634,7 @@ async function outlineDocuments(ctx, options = {}) {
10579
10634
  };
10580
10635
  const limit = Math.max(1, Math.min(options.limit ?? 200, MAX_OUTLINE_LIMIT));
10581
10636
  const offset = Math.max(0, options.offset ?? 0);
10582
- const rows = await runRead(ctx.pool, inst.tenantId, (client) => outline(client, scope, {
10637
+ const { rows, generation } = await runRead(ctx.pool, inst.tenantId, (client) => outline(client, scope, {
10583
10638
  root,
10584
10639
  depth,
10585
10640
  limit: limit + 1,
@@ -10593,6 +10648,7 @@ async function outlineDocuments(ctx, options = {}) {
10593
10648
  actor,
10594
10649
  action: "outline_served",
10595
10650
  instanceDigest: ctx.instanceDigest,
10651
+ ...generation === null ? {} : { generation },
10596
10652
  detail: {
10597
10653
  ...actScope(ctx),
10598
10654
  node: root,
package/dist/gateway.mjs CHANGED
@@ -1,2 +1,2 @@
1
- import { D as outlineHandler, I as searchHandler, N as readHandler, P as recordIsUndescribed, W as z$1, c as McpServer$1, d as READ_ONLY, f as READ_OUTPUT, h as TRUST_TIERS, i as FLOOR, o as MAX_OUTLINE_LIMIT, p as SEARCH_OUTPUT, s as MAX_SEARCH_K, u as OUTLINE_OUTPUT, x as composeInstructions } from "./gateway-api-CF4ED9_g-BQusM_dK.mjs";
1
+ import { D as outlineHandler, I as searchHandler, N as readHandler, P as recordIsUndescribed, W as z$1, c as McpServer$1, d as READ_ONLY, f as READ_OUTPUT, h as TRUST_TIERS, i as FLOOR, o as MAX_OUTLINE_LIMIT, p as SEARCH_OUTPUT, s as MAX_SEARCH_K, u as OUTLINE_OUTPUT, x as composeInstructions } from "./gateway-api-D8HlLys2-Ca8OnwLn.mjs";
2
2
  export { FLOOR, MAX_OUTLINE_LIMIT, MAX_SEARCH_K, McpServer$1 as McpServer, OUTLINE_OUTPUT, READ_ONLY, READ_OUTPUT, SEARCH_OUTPUT, TRUST_TIERS, composeInstructions, outlineHandler, readHandler, recordIsUndescribed, searchHandler, z$1 as z };
package/docs/deploying.md CHANGED
@@ -133,13 +133,88 @@ What the image deliberately does NOT contain (see `.dockerignore`):
133
133
  > every dashboard import until Vercel's detection changes — the layout that
134
134
  > triggers it, code under `system/`, is decision 8 and is not moving.
135
135
  >
136
- > Also confirm **Application Preset is `Services`**; with any other preset the
137
- > `services` key is ignored and `/mcp` never exists. Vercel Services is in Beta.
136
+ > ---
138
137
  >
139
- > **If it still argues, deploy the site alone** it needs no preset and no
140
- > services: build command `pnpm -C system/site build`, output directory
141
- > `system/site/out`. That is the stricter posture decision 29 describes, and
142
- > the door can be deployed separately.
138
+ > **The emitted `vercel.json` is verified working on the Git path.** Measured on
139
+ > two live Git-linked projects, 2026-08-27: both built the `services` block's
140
+ > `site` and `door`, and both serve `/` 200, `/llms.txt` 200, and `/mcp` 405,
141
+ > which is the door answering "Method Not Allowed" to a GET rather than a static
142
+ > 404, and is how you tell the door is routed at all.
143
+ >
144
+ > **It does not depend on the Application Preset**, which is the first thing
145
+ > everyone suspects and the reason to say so:
146
+ >
147
+ > | project's preset | `services` block built | serves |
148
+ > | ---------------- | ---------------------- | -------------------------------------- |
149
+ > | `Services` | `site` + `door` | `/` 200 · `/llms.txt` 200 · `/mcp` 405 |
150
+ > | `Other` | `site` + `door` | `/` 200 · `/llms.txt` 200 · `/mcp` 405 |
151
+ >
152
+ > **One failure has been seen that none of this explains.** On a 205-document
153
+ > record (2026-08-26, issue #197) the install ran, `ksor build` ran, every route
154
+ > prerendered — and Vercel collected nothing. The deployment reported **Ready**,
155
+ > took the production alias, and served `404: NOT_FOUND` at every path,
156
+ > `llms.txt` included. The only signal anywhere was one build-log line:
157
+ >
158
+ > ```
159
+ > WARNING! Build output contains no "functions" or "static" directory;
160
+ > the build may not have produced any deployable output.
161
+ > ```
162
+ >
163
+ > **Its cause is not established**, and that is written here rather than guessed
164
+ > at, because a wrong cause costs the reader the evening the right one would
165
+ > have saved. What is ruled out: the preset (above), and the project's own
166
+ > `outputDirectory` / `buildCommand` / `installCommand` — patching all three,
167
+ > confirming they read back, and taking a fresh Git-sourced production
168
+ > deployment (not a redeploy, which reuses the original settings snapshot)
169
+ > produced the same warning and the same 404.
170
+ >
171
+ > If you hit it, the one thing worth checking is the **Root Directory** above,
172
+ > because it is the one mechanism known to make a build read a `vercel.json`
173
+ > that is not there — though it normally fails LOUDLY, so it would be a
174
+ > different shape of the same cause rather than a match. Then please add what
175
+ > you saw to issue #197, with the deployment's `services` array from the API if
176
+ > you can: empty means the block genuinely was not read, populated moves the
177
+ > search elsewhere.
178
+ >
179
+ > **The fallback, if you need to ship before that is answered:** replace the
180
+ > `services` block with the classic top-level keys.
181
+ >
182
+ > ```json
183
+ > {
184
+ > "$schema": "https://openapi.vercel.sh/vercel.json",
185
+ > "installCommand": "pnpm install --no-frozen-lockfile",
186
+ > "buildCommand": "pnpm build",
187
+ > "outputDirectory": "system/site/out"
188
+ > }
189
+ > ```
190
+ >
191
+ > Verified live on the same repository and machine: root `200`, `llms.txt` with
192
+ > every entry, deep pages `200`, `source_commit` stamped.
193
+ >
194
+ > **It moves the door off your domain, and two values have to move with it.**
195
+ > The classic keys cannot express two services — which is the whole reason the
196
+ > emitted file uses `services` — so dropping the block also drops the rewrites
197
+ > for `/mcp`, `/health`, `/ready` and `/.well-known/oauth-protected-resource`.
198
+ > The door is then deployed separately from the same `Dockerfile`, on its own
199
+ > hostname, and `KSOR_MCP_RESOURCE_URL` plus the API Identifier registered with
200
+ > your SSO provider must both name that new origin, character for character —
201
+ > see [Authorization](./authorization.md), where a mismatch there is the failure
202
+ > that costs an afternoon.
203
+ >
204
+ > This is `buildCommand: "pnpm build"`, so it is still the DEFAULT posture of
205
+ > decision 29 — the host regenerates the lock on every deploy. The stricter one,
206
+ > where the shipped `build_id` is a reviewed one, is a different command and is
207
+ > [below](#the-site-build-runs-ksor-build-first).
208
+ >
209
+ > **Prefer the Git connection over `vercel deploy` while you work this out.** A
210
+ > CLI upload excludes `.git`, so `ksor build` cannot resolve a commit and every
211
+ > deploy publishes a record whose `build.lock.json` carries
212
+ > `"source_commit": null` — on a product whose claim is governed provenance.
213
+ > `build.lock.json` never spells it `unspecified`: that word is what the build
214
+ > prints, on **stdout** with the rest of its summary, so a step that inspects
215
+ > only stderr sees nothing at all. (`ksor ingest` is the other way round — a
216
+ > generation stores the literal string `unspecified` in `ingestion_runs`.) The
217
+ > Git path is the one that keeps the commit.
143
218
 
144
219
  The emitted `vercel.json` declares both services and routes between them:
145
220
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@panaversity/ksor",
3
- "version": "0.0.48",
3
+ "version": "0.0.50",
4
4
  "description": "Knowledge System of Record — compile governed markdown into a static site for people and an MCP server for AI agents, with citations and measured abstention.",
5
5
  "keywords": [
6
6
  "abstention",
@@ -32,11 +32,14 @@ CREATE POLICY takedown_write ON takedown_denylist FOR ALL TO sor_content_ingest
32
32
  AND g.tenant_id = takedown_denylist.tenant_id));
33
33
 
34
34
  -- ── retrieval_log: a role that can actually read the ledger ──────────────────
35
- DO $$
36
- BEGIN
37
- IF NOT EXISTS (SELECT 1 FROM pg_roles WHERE rolname = 'sor_content_auditor') THEN
38
- CREATE ROLE sor_content_auditor NOLOGIN;
39
- END IF;
35
+ -- Concurrency-tolerant for the same reason `schema.sql` is: roles are
36
+ -- CLUSTER-GLOBAL, so `IF NOT EXISTS` is check-then-act across every database on
37
+ -- the instance, and the loser raises `unique_violation` (23505) on
38
+ -- pg_authid_rolname_index not `duplicate_object` (42710). Two `ksor schema
39
+ -- --apply` runs reach this migration exactly as they reach the fresh DDL.
40
+ DO $$ BEGIN
41
+ CREATE ROLE sor_content_auditor NOLOGIN;
42
+ EXCEPTION WHEN duplicate_object OR unique_violation THEN NULL;
40
43
  END $$;
41
44
 
42
45
  GRANT USAGE ON SCHEMA public TO sor_content_auditor;
package/schema/schema.sql CHANGED
@@ -324,12 +324,36 @@ CREATE TABLE ingest_tenant_grants (
324
324
  PRIMARY KEY (role_name, tenant_id)
325
325
  );
326
326
 
327
+ -- Roles are CLUSTER-GLOBAL, so `IF NOT EXISTS` is check-then-act across every
328
+ -- database on the instance: two concurrent applies both see the role absent and
329
+ -- both create it. Measured on Postgres 17.7 — six concurrent runs against an
330
+ -- empty cluster, FIVE failed. Two `ksor schema --apply` runs, or two `pnpm
331
+ -- test:db` runs, are all it takes.
332
+ --
333
+ -- The raised SQLSTATE is `unique_violation` (23505) on pg_authid_rolname_index,
334
+ -- NOT `duplicate_object` (42710) — catching only the latter is the intuitive
335
+ -- fix and does not work. Both are caught, because which one surfaces depends on
336
+ -- where in the create the loser lands.
337
+ --
338
+ -- ONE BLOCK PER ROLE, deliberately: a `DO` block is a single statement, so an
339
+ -- exception anywhere in it rolls back the whole block. Three roles in one block
340
+ -- means a loser on the first role never creates the other two, and the apply
341
+ -- continues to GRANT against roles that do not exist.
342
+ DO $$ BEGIN
343
+ CREATE ROLE sor_content_runtime NOLOGIN;
344
+ EXCEPTION WHEN duplicate_object OR unique_violation THEN NULL;
345
+ END $$;
346
+
347
+ DO $$ BEGIN
348
+ CREATE ROLE sor_content_ingest NOLOGIN;
349
+ EXCEPTION WHEN duplicate_object OR unique_violation THEN NULL;
350
+ END $$;
351
+
352
+ -- The ledger's READER (2.3). Without it retrieval_log was write-only under
353
+ -- every credential ksor ships: FORCE RLS, an INSERT policy, and no way back in.
327
354
  DO $$ BEGIN
328
- IF NOT EXISTS (SELECT FROM pg_roles WHERE rolname = 'sor_content_runtime') THEN CREATE ROLE sor_content_runtime NOLOGIN; END IF;
329
- IF NOT EXISTS (SELECT FROM pg_roles WHERE rolname = 'sor_content_ingest') THEN CREATE ROLE sor_content_ingest NOLOGIN; END IF;
330
- -- The ledger's READER (2.3). Without it retrieval_log was write-only under
331
- -- every credential ksor ships: FORCE RLS, an INSERT policy, and no way back in.
332
- IF NOT EXISTS (SELECT FROM pg_roles WHERE rolname = 'sor_content_auditor') THEN CREATE ROLE sor_content_auditor NOLOGIN; END IF;
355
+ CREATE ROLE sor_content_auditor NOLOGIN;
356
+ EXCEPTION WHEN duplicate_object OR unique_violation THEN NULL;
333
357
  END $$;
334
358
 
335
359
  -- Explicit in-schema membership for the APPLYING role, so SET LOCAL ROLE works from day one
@@ -76,10 +76,10 @@ Stand it up in this order (each step's errors explain how to fix themselves):
76
76
  later means re-embedding the whole corpus. Keep `dim` at or below 2000 — the
77
77
  schema indexes a `vector` column directly and pgvector's HNSW takes a
78
78
  `vector` to 2000. `gemini-embedding-001` emits 3072 by default, so ksor asks
79
- it for 1536. Google's published MTEB table runs 128–2048 and is flat at the
80
- top of it 1536 scores 68.17 against 2048's 68.16 so there is no gradient
81
- to climb toward the ceiling; going the other way, 768 costs 0.18 if you want
82
- the storage back.
79
+ it for 1536, which the provider's own dimensionality table shows costs
80
+ nothing measurable against the ceiling going the other way trades a little
81
+ quality for storage. Whether to move is priced in ksor's decision 30, which
82
+ carries the figures and their source.
83
83
 
84
84
  Leave `retrieval:` out for now — the gate is off and the server says so.
85
85
  Turning it on is step 4, AFTER the record is serving.
@@ -134,6 +134,15 @@ export, so nothing serves it at runtime. `pnpm preview` is `node:http` and
134
134
  nothing else — no dependency, no network fetch — so it works offline and behind
135
135
  a firewall, like the build itself.
136
136
 
137
+ It binds loopback, so it is reachable from this machine only. To open the built
138
+ site from a container published with `-p`, a cloud dev box, or a phone on the
139
+ same wifi, name the address on the command line — `preview` is plain `node` and
140
+ does not read `.env`:
141
+
142
+ ```sh
143
+ KSOR_PREVIEW_HOST=0.0.0.0 pnpm preview
144
+ ```
145
+
137
146
  ---
138
147
 
139
148
  ## Serving to agents
@@ -274,8 +283,20 @@ declared`. The services ARE declared, in `vercel.json` at the repo root,
274
283
  3. **Set three environment variables** in Vercel: `KSOR_DB_URL`,
275
284
  `GEMINI_API_KEY`, and `KSOR_AUTH=disabled-public`.
276
285
 
277
- Two things catch people here, and both are the system being deliberate:
278
-
286
+ Three things catch people here. Two are the system being deliberate; the first
287
+ is not, and it is the one that fails without saying so:
288
+
289
+ - **A deployment can report Ready and serve nothing.** The build succeeds,
290
+ Vercel collects nothing, and the deployment takes your domain and answers
291
+ `404: NOT_FOUND` everywhere — with one build-log line as the only signal:
292
+ `WARNING! Build output contains no "functions" or "static" directory`. Seen
293
+ once, on a large record, and **the cause is not established**; it is *not* the
294
+ Application Preset, which was measured. The emitted `vercel.json` itself is
295
+ verified working on the Git path. If you hit this, the fallback is the
296
+ classic-keys form in `node_modules/@panaversity/ksor/docs/deploying.md` — read
297
+ it there rather than guessing, because it **moves the door off your domain**
298
+ and `KSOR_MCP_RESOURCE_URL` and your SSO API Identifier both have to move with
299
+ it.
279
300
  - **`disabled-local` will not deploy.** The container sets `$PORT`, so the door
280
301
  binds `0.0.0.0` — a PUBLIC bind — and refuses that value by design, saying so
281
302
  in as many words. `disabled-public` is you saying you know the door is
@@ -595,7 +616,7 @@ map rather than a substitute.
595
616
  | -------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- |
596
617
  | `pnpm check` refuses a document | `status: stable` without both `generated` and `ksor.approval`, or an approval earlier than the text it approves | add both keys; approval cannot precede what it approves |
597
618
  | `start` — missing script | there is none: the site is a static export, so nothing serves it at runtime | `pnpm preview`, or upload the folder |
598
- | `pnpm preview` exits `3` | there is no `system/site/out/` yet | run the build first |
619
+ | `pnpm preview` exits `3` | no `system/site/out/` yet, `PORT` is not a port number, the port is taken, or `KSOR_PREVIEW_HOST` cannot be bound — it says which | build first; or set a free `PORT` (`dev` uses 3000 too). `preview` binds loopback; set `KSOR_PREVIEW_HOST` to reach it from a container or another device |
599
620
  | `pnpm serve` refuses to boot | it will not run unauthenticated by accident | `KSOR_AUTH=disabled-local` in `.env` for a loopback run |
600
621
  | the deployed or containerised door refuses with `disabled-local` | it binds `0.0.0.0` — a public bind | `KSOR_AUTH=disabled-public` in the host environment, or configure the SSO variables |
601
622
  | the agent answers questions 2 and 3 instead of declining | no floor is measured, so the gate is off (`abstain OFF`, `gate: "off"`) — step 3's `calibrate` was skipped | `pnpm exec ksor calibrate --instance instance.md`, paste the block, restart |
@@ -4,7 +4,7 @@
4
4
  "private": true,
5
5
  "type": "module",
6
6
  "scripts": {
7
- "build": "next build",
7
+ "build": "next build --webpack",
8
8
  "dev": "next dev"
9
9
  },
10
10
  "dependencies": {
@@ -27,6 +27,15 @@ import process from "node:process";
27
27
 
28
28
  const ROOT = path.resolve(import.meta.dirname, "out");
29
29
  const PORT = Number(process.env.PORT ?? 3000);
30
+ if (!Number.isInteger(PORT) || PORT < 1 || PORT > 65535) {
31
+ // `Number("abc")` is NaN and `listen(NaN)` binds an arbitrary free port while
32
+ // the log prints `http://localhost:NaN` — a server you cannot find. So do
33
+ // `Number("")` and `Number(" ")`, which are 0: an unset `PORT=` in a shell or
34
+ // a compose file is the common way to reach this, and `listen(0)` has exactly
35
+ // the same ending. Hence 1, not 0.
36
+ console.error(`preview: PORT must be a port number, got ${JSON.stringify(process.env.PORT)}`);
37
+ process.exit(3);
38
+ }
30
39
 
31
40
  const TYPES = new Map([
32
41
  [".html", "text/html; charset=utf-8"],
@@ -52,13 +61,36 @@ try {
52
61
 
53
62
  /** The file a request resolves to, or null when it escapes the export. */
54
63
  function resolve(urlPath) {
55
- const decoded = decodeURIComponent(urlPath.split("?")[0]);
56
- // Contain every request inside the export: a `..` that resolves outside it
57
- // is refused rather than served, even in a preview.
64
+ let decoded;
65
+ try {
66
+ decoded = decodeURIComponent(urlPath.split("?")[0]);
67
+ } catch {
68
+ // `decodeURIComponent` THROWS on a malformed escape — `/%`, `/%zz`, a
69
+ // truncated multi-byte sequence. Thrown from a request listener that is an
70
+ // uncaught exception, and the whole preview server exits: one `curl
71
+ // http://localhost:3000/%` took it down mid-review, leaving the adopter
72
+ // with a dead port and a stack trace instead of a page. A request we
73
+ // cannot parse is a request that resolves to nothing.
74
+ return null;
75
+ }
76
+ // Contain every request inside the export: a `..` that resolves outside it —
77
+ // raw or percent-encoded — is refused rather than served, even in a preview.
78
+ // The check is LEXICAL, so it bounds paths and not the filesystem: a symlink
79
+ // INSIDE `out/` still leads wherever it points, because `statSync` follows it
80
+ // and these are strings. A Next export authors no symlinks, so that is a
81
+ // stated limit rather than a known hole; `realpathSync` on the winner is what
82
+ // would close it if an export ever carries one.
58
83
  const target = path.resolve(ROOT, `.${decoded}`);
59
84
  if (target !== ROOT && !target.startsWith(ROOT + path.sep)) return null;
60
85
 
61
86
  for (const candidate of [target, path.join(target, "index.html"), `${target}.html`]) {
87
+ // `${target}.html` is the one candidate that can sit outside the check
88
+ // above: whenever the target IS the root — `/`, `/.`, `/x/..` — it names
89
+ // the sibling `out.html`. Reachable only when the export has no
90
+ // `index.html` (candidate 2 wins otherwise), which is why the test builds
91
+ // an export without one. Containment is asserted per candidate rather than
92
+ // once, so the shapes we try can never outrun the rule they are tried under.
93
+ if (candidate !== ROOT && !candidate.startsWith(ROOT + path.sep)) continue;
62
94
  try {
63
95
  if (statSync(candidate).isFile()) return candidate;
64
96
  } catch {
@@ -68,14 +100,69 @@ function resolve(urlPath) {
68
100
  return null;
69
101
  }
70
102
 
71
- createServer((req, res) => {
103
+ /**
104
+ * Stream a file, and answer honestly when it cannot be read.
105
+ *
106
+ * `pipe()` attaches an 'error' listener to the DESTINATION, never to the
107
+ * source — so an error on the read stream has no listener and becomes an
108
+ * uncaught exception, which is how a malformed URL used to end this process.
109
+ * `statSync().isFile()` in `resolve()` does not make the later `open()` safe:
110
+ * the file can go between the two, and it does, in the ordinary loop this
111
+ * command exists for — the adopter leaves `preview` running and rebuilds in
112
+ * another pane, the export is torn down and rewritten, and an asset the open
113
+ * page re-requests is gone (`ENOENT`). A mode-000 file anywhere in the export
114
+ * is the same crash with no timing at all (`EACCES`, reproduced).
115
+ *
116
+ * The head is written on 'open', NOT before it. Writing it first meant an
117
+ * unreadable file answered `200` with an empty body and a valid terminating
118
+ * chunk — a complete, successful response carrying nothing, which a browser
119
+ * renders as a blank page and `fetch().text()` reports as `""`. That is the
120
+ * same silent lie this file exists to stop telling, moved one layer down.
121
+ * Once the head IS out there is no status left to send, so a failure part way
122
+ * through destroys the socket instead of ending it cleanly: a truncated
123
+ * response is what the client must see, because it is what happened.
124
+ */
125
+ function send(res, file, status, type) {
126
+ const stream = createReadStream(file);
127
+ let headSent = false;
128
+ stream.on("open", () => {
129
+ headSent = true;
130
+ res.writeHead(status, { "content-type": type });
131
+ stream.pipe(res);
132
+ });
133
+ stream.on("error", (error) => {
134
+ console.error(`preview: could not read ${path.relative(ROOT, file)} — ${error.message}`);
135
+ if (headSent) {
136
+ res.destroy();
137
+ return;
138
+ }
139
+ // ONE message, and it does not claim the file is still there. `resolve()`
140
+ // has already stat'd every candidate, so a file that vanished before the
141
+ // request is answered 404 by the 404 path and never reaches here; what does
142
+ // reach here is EACCES, EISDIR, EMFILE, or an ENOENT that landed in the
143
+ // window between that stat and this open. A split by errno was tried and
144
+ // removed: the window cannot be raced by a test, so it was a branch nothing
145
+ // could show working.
146
+ res.writeHead(500, { "content-type": "text/plain; charset=utf-8" });
147
+ res.end("500 — could not read that file; the preview log says why\n");
148
+ });
149
+ // A client that navigates away leaves the source with no consumer: `pipe()`
150
+ // unpipes on the destination's close but never destroys the readable, so
151
+ // 'end' never fires, `autoClose` never runs, and the fd leaks — one per
152
+ // cancelled image load.
153
+ res.on("close", () => stream.destroy());
154
+ }
155
+
156
+ const server = createServer((req, res) => {
72
157
  const file = resolve(req.url ?? "/");
73
158
  if (file === null) {
74
159
  const notFound = path.join(ROOT, "404.html");
75
160
  try {
76
- statSync(notFound);
77
- res.writeHead(404, { "content-type": "text/html; charset=utf-8" });
78
- createReadStream(notFound).pipe(res);
161
+ // `.isFile()`, the check `resolve()` already applies: `open()` succeeds on
162
+ // a DIRECTORY, so a directory named `404.html` used to write a head and
163
+ // then fail the first read.
164
+ if (!statSync(notFound).isFile()) throw new Error("not a file");
165
+ send(res, notFound, 404, "text/html; charset=utf-8");
79
166
  return;
80
167
  } catch {
81
168
  res.writeHead(404, { "content-type": "text/plain; charset=utf-8" });
@@ -83,11 +170,60 @@ createServer((req, res) => {
83
170
  return;
84
171
  }
85
172
  }
86
- res.writeHead(200, {
87
- "content-type": TYPES.get(path.extname(file)) ?? "application/octet-stream",
88
- });
89
- createReadStream(file).pipe(res);
90
- }).listen(PORT, () => {
91
- console.log(`preview: serving ${path.relative(process.cwd(), ROOT)} on http://localhost:${PORT}`);
173
+ send(res, file, 200, TYPES.get(path.extname(file)) ?? "application/octet-stream");
174
+ });
175
+
176
+ // Loopback by default, and NOT hardcoded. An omitted host binds every
177
+ // interface while the log below has always said `localhost`, so it now binds
178
+ // where it says it binds. But that breaks the cases where reaching it from
179
+ // elsewhere is the point — `docker run -p`, a cloud dev box addressed by IP,
180
+ // or opening the built site on a phone on the same wifi — and those fail as a
181
+ // refused connection, which reaches no process and so can document nothing.
182
+ //
183
+ // `KSOR_PREVIEW_HOST`, not `HOST`: the door already spells this
184
+ // `KSOR_MCP_HOST`, so the bare word would be a second name for one concept
185
+ // (working rule 8) — and tcsh and csh export `HOST` as the machine's hostname,
186
+ // which would make `preview` stop working on those shells for a variable
187
+ // nobody set on purpose.
188
+ //
189
+ // An EMPTY value binds every interface, because `listen(port, "")` takes
190
+ // Node's falsy-host branch — the same shape `PORT` is guarded against, the
191
+ // same as `PORT` above, and the same way to reach it (`KSOR_PREVIEW_HOST=` in a
192
+ // shell or a compose file). So blank falls back to loopback rather than
193
+ // silently widening what this file just narrowed.
194
+ const HOST = (process.env.KSOR_PREVIEW_HOST ?? "").trim() || "127.0.0.1";
195
+
196
+ let listening = false;
197
+ server.on("error", (error) => {
198
+ // Errors are documentation. Without this an occupied port is a raw
199
+ // `EADDRINUSE` stack trace — and the port most likely to be occupied is
200
+ // 3000, which `dev` also defaults to, so "I ran preview after dev" is the
201
+ // common case rather than an edge one.
202
+ if (!listening) {
203
+ if (error.code === "EADDRINUSE") {
204
+ console.error(`preview: port ${PORT} is already in use — set PORT to a free one.`);
205
+ console.error(" `dev` uses 3000 too, so stop it first or run `PORT=3001 preview`.");
206
+ } else if (error.code === "EADDRNOTAVAIL") {
207
+ console.error(`preview: nothing here can bind ${HOST} — check KSOR_PREVIEW_HOST.`);
208
+ } else if (error.code === "EACCES") {
209
+ console.error(`preview: not allowed to bind ${HOST}:${PORT}.`);
210
+ console.error(" usually a port below 1024 — try PORT=3000 or another high one.");
211
+ } else {
212
+ console.error(`preview: could not listen on ${HOST}:${PORT} — ${error.message}`);
213
+ }
214
+ process.exit(3);
215
+ }
216
+ // AFTER it is up, an error is an accept-path condition (EMFILE and friends),
217
+ // and exiting on it would be the very thing this file exists to stop: one
218
+ // transient ending the session.
219
+ // Still serving, because reaching here at all means `listening` fired and
220
+ // nothing closes this server — an accept-path error does not.
221
+ console.error(`preview: ${error.message} — still serving.`);
222
+ });
223
+
224
+ server.listen(PORT, HOST, () => {
225
+ listening = true;
226
+ const shown = HOST === "127.0.0.1" ? "localhost" : HOST;
227
+ console.log(`preview: serving ${path.relative(process.cwd(), ROOT)} on http://${shown}:${PORT}`);
92
228
  console.log(" this is the STATIC EXPORT — the same bytes a host would serve.");
93
229
  });