@panaversity/ksor 0.0.17 → 0.0.18

package/CHANGELOG.md

@@ -1,5 +1,47 @@
 # @panaversity/ksor
 
+## 0.0.18
+
+### Patch Changes
+
+- ea049fd: A takedown can no longer stop applying without saying so
+
+  Two ways a recorded withdrawal quietly stopped covering what it was recorded to
+  cover. Both were found by attacking the door before exposing it publicly, and
+  both were reproduced end to end against a real database.
+
+  **A denial matched nothing after the document moved.** `takedown_denylist`
+  records a `stable_id`, and the serving predicate matches those rows against the
+  documents in the generation being served — so an id that no longer exists denies
+  nothing. The default stable_id is derived from the file's path, which means an
+  ordinary rename or move of a withdrawn document was enough: search, read,
+  outline and the site all served it again, with no error anywhere. Adding an
+  `index.md` beside a withdrawn section did the same, by changing the section's id.
+
+  Serving now refuses in that state, and so does the ingest that would create it —
+  the same check at both ends, so a generation where a withdrawal has stopped
+  applying cannot be published _or_ served:
+
+  ```
+  2 takedown(s) match no document in generation 7: knowledge/legal/notice.md, …
+    why: … an id that no longer exists denies NOTHING — so a withdrawn document
+    that was renamed, moved, or had an index.md added beside it is served again
+    fix: point the denial at where the document lives now, or retire it
+    deliberately — never guess which one, because the tool cannot tell a rename
+    from a deletion
+  ```
+
+  Refusing rather than re-pointing automatically is the whole point: a tool that
+  guessed would eventually guess that a withdrawn document had been deleted when
+  it had been renamed.
+
+  **A withdrawn section did not cover its own directory.** When a section has no
+  `index.md` and its documents all live one level further down, it had no file to
+  name its own directory, so only the subdirectory was exported to the site. A
+  document written directly under the withdrawn section published to `/docs` and
+  `llms.txt` in the window before the next ingest. The section's directory is now
+  derived from its own identity, which for an index-less section is its path.
+
 ## 0.0.17
 
 ### Patch Changes

package/dist/cli.mjs

@@ -15,7 +15,7 @@ import { bodyLimit } from "hono/body-limit";
 import { execFileSync, spawnSync } from "node:child_process";
 import { parseArgs } from "node:util";
 import { readFile, readdir, stat } from "node:fs/promises";
-//#region ../content-gateway/dist/main-BtKmcm72.mjs
+//#region ../content-gateway/dist/main-Deg5kd9y.mjs
 /**
 * A connection could not be ESTABLISHED in time — retryable.
 *
@@ -1830,7 +1830,8 @@ async function assertGovernanceServable$1(pool, instance, targetGeneration) {
 		if (generation === 0) return {
 			generation,
 			builtAt: null,
-			restricted: 0
+			restricted: 0,
+			orphaned: []
 		};
 		return {
 			generation,
@@ -1839,13 +1840,27 @@ async function assertGovernanceServable$1(pool, instance, targetGeneration) {
 				instance.corpusId,
 				generation
 			])).rows[0]?.schema_version ?? null,
-			restricted: declaresModel ? 0 : Number((await client.query("SELECT count(*)::int AS n FROM content_nodes WHERE tenant_id = $1 AND generation = $2 AND visibility IS NOT NULL", [instance.tenantId, generation])).rows[0].n)
+			restricted: declaresModel ? 0 : Number((await client.query("SELECT count(*)::int AS n FROM content_nodes WHERE tenant_id = $1 AND generation = $2 AND visibility IS NOT NULL", [instance.tenantId, generation])).rows[0].n),
+			orphaned: (await client.query("SELECT d.stable_id FROM takedown_denylist d WHERE d.tenant_id = $1 AND d.corpus_id = $2   AND NOT EXISTS (SELECT 1 FROM content_nodes n                    WHERE n.tenant_id = d.tenant_id AND n.corpus_id = d.corpus_id                      AND n.generation = $3 AND n.stable_id = d.stable_id) ORDER BY d.stable_id", [
+				instance.tenantId,
+				instance.corpusId,
+				generation
+			])).rows.map((r) => r.stable_id)
 		};
 	});
 	if (state.generation === 0) return;
 	if (declaresModel && (state.builtAt === null || compareSchemaVersion$1(state.builtAt, "2.2") < 0)) throw new GovernanceGateError$1(`generation ${state.generation} was built against schema ${state.builtAt ?? "(before 2.4, which is when a generation started recording this)"}, older than 2.2 — the version that put visibility on the node row\n  why: instance.md declares an audience model, but the documents in this generation carry no visibility at all. Every one of them would be served at default_visibility — the WIDEST tier — including any document whose frontmatter restricts it
   fix: rebuild the record so its governance reaches the database:
     ksor ingest --instance instance.md --knowledge knowledge --flip`);
+	if (state.orphaned.length > 0) {
+		const named = state.orphaned.slice(0, 5).join(", ");
+		const more = state.orphaned.length - Math.min(5, state.orphaned.length);
+		throw new GovernanceGateError$1(`${state.orphaned.length} takedown(s) match no document in generation ${state.generation}: ${named}${more > 0 ? `, and ${more} more` : ""}\n  why: a denial is recorded against a stable_id, and the serving predicate matches it against the documents in this generation. An id that no longer exists denies NOTHING — so a withdrawn document that was renamed, moved, or had an index.md added beside it is served again by search, read, outline and the site, with no error anywhere. The denial is meant to be immune to reorganization; this is the state where it is not
+  fix: point the denial at where the document lives now, or retire it deliberately — never guess which one, because the tool cannot tell a rename from a deletion:
+    ksor takedown --instance instance.md --stable-id <the new id> --reason <why> --actor <who>
+    ksor takedown --instance instance.md --revoke <the old id> --actor <who>
+  (ksor takedown --list shows what is recorded)`);
+	}
 	if (!declaresModel && state.restricted > 0) throw new GovernanceGateError$1(`${state.restricted} document(s) in generation ${state.generation} declare visibility:, but instance.md declares no audiences:
   why: an author restricted those documents and nothing would enforce it — this door would serve them in full to every caller, and the frontmatter key saying otherwise would be the only trace. The site refuses to BUILD in this exact state (ksor-visibility-without-audiences); the door must not serve in it
   fix: declare the model in instance.md (audiences: least-restricted first, plus default_visibility:), or remove the visibility: keys and re-ingest`);
@@ -4501,7 +4516,7 @@ async function withPgRetry(op, options = {}) {
 	throw lastError;
 }
 //#endregion
-//#region ../content/dist/commands-CXqK2c2f.mjs
+//#region ../content/dist/commands-DcPJJlNb.mjs
 /**
 * EVAL-LOCKED constants, quarried verbatim from the oracle
 * (sor-agentfactory @ b554f91, config.py) — changing any of these is a
@@ -6425,7 +6440,8 @@ async function assertGovernanceServable(pool, instance, targetGeneration) {
 		if (generation === 0) return {
 			generation,
 			builtAt: null,
-			restricted: 0
+			restricted: 0,
+			orphaned: []
 		};
 		return {
 			generation,
@@ -6434,13 +6450,27 @@ async function assertGovernanceServable(pool, instance, targetGeneration) {
 				instance.corpusId,
 				generation
 			])).rows[0]?.schema_version ?? null,
-			restricted: declaresModel ? 0 : Number((await client.query("SELECT count(*)::int AS n FROM content_nodes WHERE tenant_id = $1 AND generation = $2 AND visibility IS NOT NULL", [instance.tenantId, generation])).rows[0].n)
+			restricted: declaresModel ? 0 : Number((await client.query("SELECT count(*)::int AS n FROM content_nodes WHERE tenant_id = $1 AND generation = $2 AND visibility IS NOT NULL", [instance.tenantId, generation])).rows[0].n),
+			orphaned: (await client.query("SELECT d.stable_id FROM takedown_denylist d WHERE d.tenant_id = $1 AND d.corpus_id = $2   AND NOT EXISTS (SELECT 1 FROM content_nodes n                    WHERE n.tenant_id = d.tenant_id AND n.corpus_id = d.corpus_id                      AND n.generation = $3 AND n.stable_id = d.stable_id) ORDER BY d.stable_id", [
+				instance.tenantId,
+				instance.corpusId,
+				generation
+			])).rows.map((r) => r.stable_id)
 		};
 	});
 	if (state.generation === 0) return;
 	if (declaresModel && (state.builtAt === null || compareSchemaVersion(state.builtAt, "2.2") < 0)) throw new GovernanceGateError(`generation ${state.generation} was built against schema ${state.builtAt ?? "(before 2.4, which is when a generation started recording this)"}, older than 2.2 — the version that put visibility on the node row\n  why: instance.md declares an audience model, but the documents in this generation carry no visibility at all. Every one of them would be served at default_visibility — the WIDEST tier — including any document whose frontmatter restricts it
   fix: rebuild the record so its governance reaches the database:
     ksor ingest --instance instance.md --knowledge knowledge --flip`);
+	if (state.orphaned.length > 0) {
+		const named = state.orphaned.slice(0, 5).join(", ");
+		const more = state.orphaned.length - Math.min(5, state.orphaned.length);
+		throw new GovernanceGateError(`${state.orphaned.length} takedown(s) match no document in generation ${state.generation}: ${named}${more > 0 ? `, and ${more} more` : ""}\n  why: a denial is recorded against a stable_id, and the serving predicate matches it against the documents in this generation. An id that no longer exists denies NOTHING — so a withdrawn document that was renamed, moved, or had an index.md added beside it is served again by search, read, outline and the site, with no error anywhere. The denial is meant to be immune to reorganization; this is the state where it is not
+  fix: point the denial at where the document lives now, or retire it deliberately — never guess which one, because the tool cannot tell a rename from a deletion:
+    ksor takedown --instance instance.md --stable-id <the new id> --reason <why> --actor <who>
+    ksor takedown --instance instance.md --revoke <the old id> --actor <who>
+  (ksor takedown --list shows what is recorded)`);
+	}
 	if (!declaresModel && state.restricted > 0) throw new GovernanceGateError(`${state.restricted} document(s) in generation ${state.generation} declare visibility:, but instance.md declares no audiences:
   why: an author restricted those documents and nothing would enforce it — this door would serve them in full to every caller, and the frontmatter key saying otherwise would be the only trace. The site refuses to BUILD in this exact state (ksor-visibility-without-audiences); the door must not serve in it
   fix: declare the model in instance.md (audiences: least-restricted first, plus default_visibility:), or remove the visibility: keys and re-ingest`);
@@ -6858,7 +6888,10 @@ async function deniedStableIds(pool, instance) {
 *
 * The seed's OWN file counts when the seed has children, and only then — see
 * the SQL comment: a container's index.md names its directory, a leaf's file
-* names its parent's.
+* names its parent's. An index-less container has no such file, so the one it
+* would have had is synthesized from its (path-derived) "#section" id — without
+* it, a section whose descendants all live one level down contributed only the
+* subdirectory and left its own level publishable (issue #86).
 */
 async function deniedSubtreeDirs(pool, instance) {
 	const paths = await runRead(pool, instance.tenantId, async (client) => {
@@ -6866,7 +6899,7 @@ async function deniedSubtreeDirs(pool, instance) {
          SELECT active_generation AS g FROM corpora WHERE tenant_id = $1 AND corpus_id = $2
        ),
        seed AS (
-         SELECT n.node_id
+         SELECT n.node_id, n.stable_id
            FROM takedown_denylist d
            JOIN content_nodes n ON n.tenant_id = d.tenant_id AND n.stable_id = d.stable_id
            JOIN gen ON n.generation = gen.g
@@ -6904,7 +6937,28 @@ async function deniedSubtreeDirs(pool, instance) {
                  WHERE NOT EXISTS (SELECT 1 FROM content_nodes kid
                                     JOIN gen ON kid.generation = gen.g
                                    WHERE kid.tenant_id = $1 AND kid.parent_id = s2.node_id)
-              )`, [instance.tenantId, instance.corpusId])).rows.map((r) => String(r.origin_path));
+              )
+        UNION
+        -- An INDEX-LESS container has no file at all, so the join above drops
+        -- it however many descendants it has, and a section whose files all
+        -- live one level down contributed only the SUBdirectory — leaving a
+        -- document written directly under the withdrawn section publishable
+        -- (issue #86). The round-10 "seed counts when it has children" rule was
+        -- right and could not fire here, because there was nothing to count.
+        --
+        -- So the index.md it WOULD have had is synthesized, and the container
+        -- then names its directory exactly as an index-bearing one does. This is
+        -- not the stable_id prefix matching decision 14 rejects: that fails
+        -- because a sor_id: override decouples an id from its path, and a
+        -- "#section" id is generated from the path with no frontmatter in
+        -- reach: there is no index file to carry an override
+        -- (adapters/plain-tree.ts:214-219).
+        SELECT substring(sd.stable_id from '^(.*)#section$') || '/index.md'
+          FROM seed sd
+         WHERE sd.stable_id LIKE '%#section'
+           AND EXISTS (SELECT 1 FROM content_nodes kid
+                        JOIN gen ON kid.generation = gen.g
+                       WHERE kid.tenant_id = $1 AND kid.parent_id = sd.node_id)`, [instance.tenantId, instance.corpusId])).rows.map((r) => String(r.origin_path));
 	});
 	const dirs = /* @__PURE__ */ new Set();
 	for (const raw of paths) {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@panaversity/ksor",
-  "version": "0.0.17",
+  "version": "0.0.18",
   "description": "Knowledge System of Record — compile governed markdown into a static site for people and an MCP server for AI agents, with citations and measured abstention.",
   "keywords": [
     "abstention",