@panaversity/ksor 0.0.59 → 0.0.61

package/CHANGELOG.md

@@ -1,5 +1,126 @@
 # @panaversity/ksor
 
+## 0.0.61
+
+### Patch Changes
+
+- c8df2a0: `pnpm dev` now drops a document that is deleted or moved while it runs (#274).
+
+  The dev server keeps a staged copy of the record, and its watcher carried edits
+  and new documents into that copy but never removals. A document deleted during
+  a review went on answering 200 at its old url and stayed in the sidebar, and a
+  moved one was listed twice, once at each path, until the dev server restarted.
+  Nothing told the owner to restart.
+
+  The refresh now removes every staged file the record no longer holds, and every
+  folder that leaves empty, so the deleted or moved document answers 404 at its
+  old url and leaves the sidebar within about a second. A document whose audience
+  is edited so that the dev viewer may no longer read it leaves the same way.
+
+  Removals were held back because a 2026-08-18 measurement found that deleting a
+  staged file took the dev server down. On fumadocs-mdx 15.4.0 and Next 16.3.3
+  it recovers on its own, but the race behind it remains: Turbopack can compile
+  before fumadocs has regenerated the collection, so the dev log may show
+  `Module not found` for the removed file, and a request in that moment can
+  answer with a 500. Measured on a fresh scaffold, another page polled every 20ms
+  did so one to four times in 6 of 8 removals, then answered 200 again. Builds
+  are unaffected: they never run the watcher.
+
+  An existing project takes the fix with `ksor migrate --write-site`, which
+  reissues `system/site/lib/stage-knowledge.ts` with the rest of the site.
+
+- 7d2e28a: A new npm or bun scaffold builds its site again.
+
+  `mdast-util-to-markdown@2.1.3`, published 2026-09-27, broke the MDX stringifier
+  of `fumadocs-core@16.15.4`, the version the scaffold pins. Any page with bold or
+  italic text sent it into recursion without end, so `npm run build` and
+  `bun run build` failed with `RangeError: Maximum call stack size exceeded`
+  (fuma-nama/fumadocs#3604). The npm and bun scaffolds ship no lockfile, so a
+  fresh install picked up 2.1.3. Their root `package.json` now has
+  `"overrides": { "mdast-util-to-markdown": "2.1.2" }`, and their README explains
+  it. The pnpm scaffold has not changed, because its committed lockfile already
+  holds 2.1.2.
+
+  If you scaffolded with npm or bun and your build now fails this way, add the
+  same `overrides` entry to your root `package.json` and install again. Remove the
+  entry when you move `fumadocs-core` to 16.15.15 or later, which has the
+  upstream fix.
+
+## 0.0.60
+
+### Patch Changes
+
+- 2a7ef92: Five things a live walk of the published 0.0.59 found, fixed.
+
+  **`verify.mjs` reported ordinary markdown as an invented name.** Its name regex
+  let `\s+` cross a blank line, so a `## Meals` heading followed by a paragraph
+  opening `On travel…` was extracted as the name "Meals On" and reported as
+  changed-or-introduced. That fires on the first document an agent converts —
+  the check meant to make conversion trustworthy was crying wolf. Names are now
+  capitalised words on ONE line; a name the source never mentions is still caught.
+
+  **A scaffold followed verbatim published its first generation untraceably.**
+  `ksor init` runs `git init` and leaves zero commits, and its own epilogue went
+  install → dev → provision → refresh with no commit in between — so every
+  adopter's first publish said `source: unspecified` and skipped the R23
+  change-control check, which had no history to compare against. The epilogue now
+  says to commit before publishing, which is the whole fix.
+
+  **A plain build left a stale bundle tree unmentioned.** `ksor build` recomputes
+  every `bundles[].sha256` in the lock but only `--bundles` writes the directory,
+  so after one `--bundles` run and any ordinary build the lock claimed a digest
+  nothing on disk produced, while the tree that exists to be SENT somewhere aged
+  silently. A plain build now says which build the directory came from and how to
+  refresh it. Reported, never deleted: it is the adopter's output and may be
+  mid-handover.
+
+  **Tutorial 01's first build block was one line short.** R23 landed the day after
+  that walk, so the shipped block omitted `change-control: not checked` on a page
+  whose headline claim is that every output was pasted as it appeared. Re-captured
+  on 0.0.59, with a paragraph on what both honesty lines mean and when they go.
+
+  **The dev server's `/llms.txt` does not change after an approval.** Not a bug and
+  not fixable in the route: `output: "export"` requires a static route handler, so
+  `pnpm dev` computes it once per process while the document's page beside it
+  updates. Stated in the emitted README's troubleshooting table and in the tutorial
+  step whose own prompt is "Why isn't my refund policy in llms.txt?" — verified by
+  testing both alternatives, each of which breaks the export.
+
+- 6c47618: Measure WHICH skill a real agent reaches for, and record what the first sweeps
+  found.
+
+  A live walk of the published 0.0.59 reported that `add-sources` did not fire on
+  its headline prompt. The tempting repair is to reword the description until it
+  does, which is a guess. This is the instrument that replaces the guess: N runs
+  per phrase, in a fresh scaffold with all three skills present, graded on which
+  skill the agent actually invoked, across more than one model. Reported, never
+  gating — a model is stochastic and a threshold over a handful of runs flakes.
+
+  **The model is a column, because the answer depends on it.** Same phrase, same
+  scaffold, same harness: `claude-sonnet-5` fired `add-sources` 3/3 where
+  `claude-opus-5` fired nothing 0/2. The walk used the CLI default and the first
+  probe pinned Sonnet, which is why they disagreed — neither was wrong, and
+  neither alone measured the trigger.
+
+  **The finding is narrower than the walk suggested.** `add-sources` fires on four
+  of five phrases on both models, and both controls behave: a different skill wins
+  the intake phrase, and nothing fires on a question about the repo itself. It
+  misses exactly one shape on Opus — the owner pointing at a file already in the
+  repo and naming a destination.
+
+  **And the obvious repair does not work.** Naming that shape in the description
+  was tried and measured: unchanged at 0/3. So the cause is not the wording — an
+  instruction concrete enough to act on gets acted on, and no skill is consulted.
+  The clause was reverted rather than kept, because it is resident context in
+  every session and bought nothing a measurement can see. The negative result is
+  recorded beside the rows it explains.
+
+  Also fixed while building it: the CLI can emit raw control characters inside its
+  JSON, which `JSON.parse` rejects outright — the harness lost the whole transcript
+  to a stray byte. It now falls back to a scrubbed parse.
+
+  Test infrastructure only; nothing an adopter installs behaves differently.
+
 ## 0.0.59
 
 ### Patch Changes

package/dist/cli.mjs

@@ -12216,7 +12216,7 @@ function runBuild(args, cwd, io, options) {
 	io.out(`ksor build: ${lock.documents.length} document(s), ${admitted} admitted to a machine surface at ${lock.as_of}\n${provenanceLine({
 		...facts,
 		dirty: lock.dirty
-	}, root)}\n` + (change.notice === null ? "" : `  ${change.notice}\n`) + notice + `${pendingIndexes.map((w) => `  wrote ${w}\n`).join("")}${staleIndexes.map((r) => `  removed ${r} (its directory earns no index)\n`).join("")}` + (parsed.bundles ? bundlesReport(bundles, viewers) : "") + `  wrote build.lock.json — build_id ${lock.build_id}\n`);
+	}, root)}\n` + (change.notice === null ? "" : `  ${change.notice}\n`) + notice + `${pendingIndexes.map((w) => `  wrote ${w}\n`).join("")}${staleIndexes.map((r) => `  removed ${r} (its directory earns no index)\n`).join("")}` + (parsed.bundles ? bundlesReport(bundles, viewers) : staleBundlesNotice(root, lock.build_id)) + `  wrote build.lock.json — build_id ${lock.build_id}\n`);
 	return 0;
 }
 /**
@@ -12241,6 +12241,35 @@ function writeBundles(root, bundles, lockText) {
 	}
 	writeFileSync(path.join(out, LOCK_NAME), lockText);
 }
+/**
+* `.ksor/out/bundles/` from an EARLIER build, when this run did not write it.
+*
+* A plain `ksor build` recomputes every `bundles[].sha256` in the root lock —
+* they are a function of what `build_id` already hashes — but it does not
+* touch the directory, which only `--bundles` writes. So after one
+* `--bundles` run and any ordinary build, the root lock claims a digest that
+* nothing on disk produces, and the tree that exists to be SENT somewhere goes
+* on aging with nothing said (found by walking the published 0.0.59).
+*
+* Reported, never deleted and never refused: the directory is the adopter's
+* output, its co-located lock still describes its own bytes coherently, and a
+* build that removed it would destroy something an adopter may be mid-way
+* through handing over. Honest absence, never silent weakness.
+*/
+function staleBundlesNotice(root, buildId) {
+	const beside = path.join(root, BUNDLES_DIR, LOCK_NAME);
+	if (!existsSync(beside)) return "";
+	let wrote;
+	try {
+		wrote = JSON.parse(readFileSync(beside, "utf8")).build_id;
+	} catch {
+		wrote = "unreadable";
+	}
+	if (wrote === buildId) return "";
+	return `  bundles: ${BUNDLES_DIR}/ is from build ${wrote}, not this one — the lock records this build's digests, so what is on disk no longer matches them
+    fix: re-run with \`--bundles\` before sending them, or delete the directory
+`;
+}
 /** One line per bundle written, and a line per link it carries to a concept it excludes. */
 function bundlesReport(bundles, viewers) {
 	let text = "";
@@ -12308,6 +12337,20 @@ const SCRIPT_BODIES = {
 	}
 };
 /**
+* Versions of dependencies of dependencies that npm and bun must be told to
+* hold, written as `overrides`, which both read. pnpm needs none here: its
+* committed lockfile already holds each one.
+*
+* mdast-util-to-markdown 2.1.3 (2026-09-27) writes bold and italic only
+* through a handler's `attention`. The MDX stringifier of the fumadocs-core
+* 16.15.4 that the site pins wraps every handler without it, so the site
+* build recursed until the stack overflowed (issue #276,
+* fuma-nama/fumadocs#3604, fixed in fumadocs-core 16.15.15). Remove the entry,
+* and the README paragraph that explains it, in the change that moves the site
+* to that Fumadocs: init-manager.integration.test.ts fails until they are gone.
+*/
+const OVERRIDES = { "mdast-util-to-markdown": "2.1.2" };
+/**
 * Rewrite the scaffold's root package.json for the manager. Structured — a
 * JSON transform, never string surgery — because the manifest is the one
 * file where a half-applied spelling map would still parse and then lie.
@@ -12322,7 +12365,8 @@ function transformManifest(source, manager) {
 			...parsed.scripts,
 			...SCRIPT_BODIES[manager]
 		},
-		workspaces: [...WORKSPACE_GLOBS]
+		workspaces: [...WORKSPACE_GLOBS],
+		overrides: { ...OVERRIDES }
 	};
 	return `${JSON.stringify(out, null, 2)}\n`;
 }
@@ -14420,6 +14464,9 @@ function handoff(io, name, targetWasDot, manager) {
 	io.out(`${name} is ready — your knowledge, your repo, yours outright.\n
 Next (or just tell your coding agent to take it from here):
 ` + enter + `  ${install}\n  ${run("dev").padEnd(15)} # the site, live at http://localhost:3000\n
+Commit before you publish — a build traces to a reviewed commit:
+  git add -A && git commit -m 'Scaffold the record'
+
 Then, for the agent surface (needs Postgres and a provider key):
   ${run("provision").padEnd(15)} # once: copy .env.example to .env and set KSOR_DB_URL,\n                  #   then apply the schema
   ${run("refresh").padEnd(15)} # PUBLISH the record — ingest knowledge/ into a generation\n  ${run("serve").padEnd(15)} # the MCP server, over what you just published\n

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@panaversity/ksor",
-  "version": "0.0.59",
+  "version": "0.0.61",
   "description": "Knowledge System of Record — compile governed markdown into a static site for people and an MCP server for AI agents, with citations and measured abstention.",
   "keywords": [
     "abstention",

package/templates/scaffold/.agents/skills/add-sources/verify.mjs

@@ -38,7 +38,12 @@ const body = raw.replace(/^---\n[\s\S]*?\n---\n?/, "").replace(/\[\^[^\]]+\]:?/g
 
 const tokens = new Set();
 for (const m of body.matchAll(/\d[\d,.:/-]*\d|\d/g)) tokens.add(m[0]);
-for (const m of body.matchAll(/\b[A-Z][a-z]+(?:\s+[A-Z][a-z]+)+\b/g)) tokens.add(m[0]);
+// A name is capitalised words on ONE line. `\s+` here would cross a blank
+// line, so a `## Meals` heading followed by a paragraph opening `On travel…`
+// was extracted as the name "Meals On" and reported as invented — a false
+// positive on ordinary markdown, hit on the first document an agent writes
+// (found by walking the published 0.0.59).
+for (const m of body.matchAll(/\b[A-Z][a-z]+(?:[^\S\r\n]+[A-Z][a-z]+)+\b/g)) tokens.add(m[0]);
 
 const missing = [...tokens].filter((t) => !extraction.includes(fold(t))).sort();
 for (const t of missing) console.log(t);

package/templates/scaffold/.claude/skills/add-sources/verify.mjs

@@ -38,7 +38,12 @@ const body = raw.replace(/^---\n[\s\S]*?\n---\n?/, "").replace(/\[\^[^\]]+\]:?/g
 
 const tokens = new Set();
 for (const m of body.matchAll(/\d[\d,.:/-]*\d|\d/g)) tokens.add(m[0]);
-for (const m of body.matchAll(/\b[A-Z][a-z]+(?:\s+[A-Z][a-z]+)+\b/g)) tokens.add(m[0]);
+// A name is capitalised words on ONE line. `\s+` here would cross a blank
+// line, so a `## Meals` heading followed by a paragraph opening `On travel…`
+// was extracted as the name "Meals On" and reported as invented — a false
+// positive on ordinary markdown, hit on the first document an agent writes
+// (found by walking the published 0.0.59).
+for (const m of body.matchAll(/\b[A-Z][a-z]+(?:[^\S\r\n]+[A-Z][a-z]+)+\b/g)) tokens.add(m[0]);
 
 const missing = [...tokens].filter((t) => !extraction.includes(fold(t))).sort();
 for (const t of missing) console.log(t);

package/templates/scaffold/README.md

@@ -94,6 +94,15 @@ never picks up a day-zero compromised release. bun has no equivalent — its
 default refusal of dependency install scripts covers the OTHER half of that
 posture, and this sentence is the disclosure.
 
+<!-- /ksor:pm -->
+<!-- ksor:pm npm bun -->
+
+`package.json` also holds one dependency of a dependency back: `overrides`
+keeps `mdast-util-to-markdown` at 2.1.2. With the `fumadocs-core` this scaffold
+pins, version 2.1.3 makes the site build recurse until it runs out of stack
+([fumadocs#3604](https://github.com/fuma-nama/fumadocs/issues/3604)). Delete the
+override when you move `fumadocs-core` to 16.15.15 or later, which has the fix.
+
 <!-- /ksor:pm -->
 
 ---
@@ -724,6 +733,7 @@ map rather than a substitute.
 | the agent answers questions 2 and 3 instead of declining             | no floor is measured, so the gate is off (`abstain OFF`, `gate: "off"`) — step 3's `calibrate` was skipped  | `pnpm exec ksor calibrate --instance instance.md`, paste the block, restart             |
 | a deployed door serves an empty record                               | deploying does not publish — and a laptop DSN is unreachable from the host                                 | point both at one hosted Postgres, then `pnpm refresh`                                 |
 | the home page and `/llms.txt` are empty                              | every document is still a draft — correct, not broken                                                      | approve one and rebuild                                                                |
+| `/llms.txt` on the DEV server does not change after you approve            | `output: "export"` requires its route handler to be static, so `pnpm dev` computes it once per process — the document page beside it updates, that file does not | restart `pnpm dev`; the built site is always current, so this is a dev-server artefact only |
 | `ksor-record-empty`                                                  | every document was deleted — a record is never empty, so nothing was written                               | add one document of your own (or restore one from git) before deleting the last starter |
 | `ksor-approver-unauthorised`                                         | a document is approved by an actor `.ksor/governance.yaml` no longer names — usually `human:you` after the interview | re-attribute the approval to your handle, or restore the actor to `approval_authorities` |
 | `ksor-generated-stale` at `pnpm build` or `pnpm refresh`             | a `stable` document's body changed since a commit where it was stable, and `generated.at` was not advanced past that commit's (left alone, or moved backward) — the stamp dates the text, and `pnpm check` reads no document history, so it passed | set `generated.at` to an instant after the edit and re-approve (`ksor.approval.at` may not precede it) |

package/templates/scaffold/system/site/lib/stage-knowledge.ts

@@ -672,15 +672,15 @@ function fillStage(recordDir: string, stageDir: string, development: boolean): v
 }
 
 /**
- * Dev only: carry edits AND ARRIVALS into the stage, so `pnpm dev` shows the
- * record as the owner is writing it rather than as it stood when the server
- * started — the regenerated indexes included, so a retitled document is
+ * Dev only: carry edits, ARRIVALS and REMOVALS into the stage, so `pnpm dev`
+ * shows the record as the owner is writing it rather than as it stood when the
+ * server started — the regenerated indexes included, so a retitled document is
  * retitled in its folder's listing too.
  *
- * Adds and edits — never removals. The 2026-08-18 measurement this refused
- * adds on ("fumadocs' own watcher cannot see a dot-prefixed collection
- * directory") no longer holds: on fumadocs-mdx 15.3.0 a file written into
- * `.staged-knowledge` DOES regenerate the collection, twice-observed as
+ * Arrivals first. The 2026-08-18 measurement this refused adds on ("fumadocs'
+ * own watcher cannot see a dot-prefixed collection directory") no longer
+ * holds: on fumadocs-mdx 15.3.0 a file written into `.staged-knowledge` DOES
+ * regenerate the collection, twice-observed as
  * `[MDX] generated files` in the dev log. What actually kept a new document
  * off every surface was this function, which walked the STAGE and skipped
  * anything the stage did not already hold — so a plan entry with no file on
@@ -692,11 +692,23 @@ function fillStage(recordDir: string, stageDir: string, development: boolean): v
  * this way before the stage existed (0.0.40 serves an added document at 200),
  * so this is a regression repaired rather than a feature.
  *
- * REMOVALS still wait for the restart `pnpm dev` already needs for
- * instance.md: the same measurement found a deleted file leaves fumadocs'
- * generated imports pointing at something gone, which takes the dev server
- * down rather than showing a stale page. An arrival has no such failure mode
- * — nothing points at a file that has only just appeared.
+ * REMOVALS used to wait for a restart: the same 2026-08-18 measurement found a
+ * deleted file left fumadocs' generated imports pointing at something gone,
+ * which took the dev server down. On fumadocs-mdx 15.4.0 and Next 16.3.3 it
+ * does not stay down. Measured 2026-10-01 on a fresh scaffold: each document
+ * deleted or moved while `pnpm dev` ran regenerated the collection (`[MDX]
+ * generated files`), answered 404 at its old url within about a second and
+ * left the sidebar, and every other page answered 200 with no restart.
+ *
+ * What remains is a race, not an outage. Turbopack can compile
+ * `.source/server.ts` before fumadocs has rewritten it, so the log shows
+ * `Module not found` for the removed file and a request in that window answers
+ * 500: another page, polled every 20ms, did so one to four times in 6 of 8
+ * removals, then answered 200 again. Both watchers react to the same unlink,
+ * so nothing here can order them. Errors that clear themselves are still the
+ * better failure: before this, a deleted document went on serving at 200 and
+ * stayed in the sidebar, and a moved one was listed twice, until someone
+ * thought to restart (issue #274).
  */
 function refreshStage(recordDir: string, stageDir: string): void {
   // Under the lock like every other write here: a save landing while another
@@ -719,6 +731,9 @@ function refreshStage(recordDir: string, stageDir: string): void {
       mkdirSync(path.dirname(staged), { recursive: true });
       writeFileSync(staged, bytes);
     }
+    // Removals from the plan too: a staged file the plan no longer holds was
+    // deleted, moved, or withdrawn from this viewer, and is not the record.
+    pruneExcept(stageDir, new Set(plan.entries.map((e) => path.resolve(stageDir, e.rel))));
     writeManifest(stageDir, plan.manifest);
   });
 }
@@ -867,6 +882,14 @@ function publishSims(sourceDir: string): void {
  * nothing else writes it), so what is not published now does not belong.
  */
 function pruneSims(target: string, published: ReadonlySet<string>): void {
+  pruneExcept(target, published);
+}
+
+/**
+ * Every file under `target` whose resolved path `keep` does not hold, removed,
+ * and every directory that leaves empty. `target` itself always stays.
+ */
+function pruneExcept(target: string, keep: ReadonlySet<string>): void {
   const walk = (dir: string): boolean => {
     let entries;
     try {
@@ -882,7 +905,7 @@ function pruneSims(target: string, published: ReadonlySet<string>): void {
         else empty = false;
         continue;
       }
-      if (published.has(path.resolve(here))) {
+      if (keep.has(path.resolve(here))) {
         empty = false;
         continue;
       }