@panaversity/ksor 0.0.13 → 0.0.15

package/CHANGELOG.md

@@ -1,5 +1,139 @@
 # @panaversity/ksor
 
+## 0.0.15
+
+### Patch Changes
+
+- 5b076e6: Short documents reach search again — navigation is a shape, not a length
+
+  A record could be fully ingested, report "embedded 16, failed 0", and still be
+  unable to answer questions it plainly contained. Sections were classified as
+  navigation by LENGTH — anything under 250 characters — and navigation is
+  excluded from every retrieval arm. On a handbook that inverts the intent,
+  because a handbook's most valuable statements are its shortest.
+
+  Walked on 0.0.14 with three ordinary policy statements — a refund window, an
+  escalation path, a badge rule, 200-300 characters each. Three of four chunks
+  were unsearchable, and:
+
+  > **Q.** "how long does a buyer have to send something back"
+  > **A.** the scaffold's placeholder page — against a record stating _thirty days_
+
+  The answer was in the corpus, correctly ingested, readable by slug, and
+  unreachable by search.
+
+  Navigation is now decided by shape: a section is navigation when link lines are
+  most of it, or when what remains after them is too short to answer anything —
+  the same floor the serving predicate already applies. Length is no longer
+  consulted, so a 180-character link list is navigation and a 51-character fact is
+  not, which is the ordering length had backwards.
+
+  Measured on an authored handbook gold set with real embeddings, paired: short
+  substantive facts went **0/9 to 9/9 at rank 1**, the long-prose control held at
+  **4/4**, and the link-list page was returned **0** times. That last number is the
+  one that matters — admitting everything would have improved the first two and
+  made the product worse.
+
+  **To pick this up, re-run `ksor ingest`.** Chunks are re-classified on every
+  build and unchanged content is not re-embedded, so the upgrade costs a build,
+  not an embedding bill. `CHUNK_POLICY` moves to v6 because it is persisted
+  provenance and the behaviour it labels has changed.
+
+- 5763e8b: Internal: a pool test that raced Postgres, and a comment that had it backwards
+
+  No adopter-visible behaviour changes.
+
+  `idle.db.test.ts` sampled `pg_stat_activity` immediately after a previous test's
+  `pool.end()`. Those are two different clocks — `end()` resolves when the client
+  socket closes, while the row disappears only once the server-side backend
+  actually exits — so the suite was order-coupled through the database and went
+  red in CI on a branch that changed nothing but a document. Each test now waits
+  for a quiet database before it starts, and states that it does.
+
+  The comment added in the previous release explaining the `env.example` guard fix
+  described the rename backwards: the TEMPLATE holds `env.example` and
+  `materialize.ts` maps it to `.env.example` on emit, not the other way round.
+
+## 0.0.14
+
+### Patch Changes
+
+- a0d98b0: Cut dead weight, and repair two guards that had quietly stopped guarding
+
+  A sweep across every package, with each candidate handed to a second reviewer
+  whose job was to prove it still alive. Net −154 lines. Nothing an adopter can
+  observe changes; two things that were supposed to fail no longer stay silent.
+
+  **The two repairs.** A guard asserting that no scaffolded document describes
+  serving as publishing — a claim this repo has had to correct four times — ran
+  `readFileSync` inside a `try` whose `catch` returned quietly, and one of its five
+  filenames was `.env.example` while the scaffold emits `env.example`. So the row
+  covering the file that actually carries the serving variables had never executed.
+  The name is fixed and a missing file now fails instead of passing. Separately,
+  two doc-blocks described a stdio transport in the present tense; there is no
+  stdio door in the product, and the suite claiming to drive one drives HTTP.
+
+  **The removals.** A 134-line live-walk script pinned to `@panaversity/ksor@0.0.4`
+  that nothing referenced. `AuthConfig.jwksUrl`, computed and stored but never read
+  — its live twin is `explicitJwksUrl`; the boot-time validation of
+  `KSOR_JWKS_URL` stays exactly where it was. An `allowedAudiences.length > 0 &&`
+  operand that no path can reach as false, and whose false side would have skipped
+  the audience allowlist entirely. A `PoolTimeoutError` message parameter no caller
+  passed, which was also the one input where two retry classifiers disagreed —
+  removing it closes that. Two `instanceof X || instanceof Error` disjuncts where
+  `X extends Error`, so the first could never decide anything. One unused icon
+  export in the workbench shell.
+
+  **Left alone deliberately.** `SearchScope.kinds` is genuinely dead, but removing
+  it renumbers positional parameters across three SQL statements, two of which
+  derive a shared CTE by string substitution, and the test that would catch a wrong
+  renumber is gated on a database. That is a change to make on its own, with the
+  gate watching — not alongside a release.
+
+- ce1595b: Ingest names the real reason it could not record a commit
+
+  Every first ingest of a freshly scaffolded project printed "knowledge/ is not in
+  a git repository". That is false: `ksor init` runs `git init`, so the repository
+  exists — it simply has no commit yet, and `rev-parse HEAD` fails with "unknown
+  revision" rather than because nothing is there. The reader was sent to `git
+init`, which they had already run, in the one message that decides whether an
+  answer can be traced back to a reviewed commit.
+
+  Three different states were collapsing into that one sentence, and each has a
+  different next command:
+
+  ```
+  knowledge/ is in a git repository with no commits yet …
+    fix: commit the record (git add knowledge && git commit) and re-run
+
+  knowledge/ is not in a git repository …
+    fix: git init, commit the record, and re-run
+
+  git is not installed …
+    fix: install git, or pass --source-commit <sha> if the record is versioned elsewhere
+  ```
+
+  Verified on a real scaffold: the fresh case prints the first, and committing the
+  record turns the next ingest's `source:` line into an actual SHA.
+
+- 474dedc: Internal: the env-contract drift test scans only the checkout's source
+
+  No adopter-visible behaviour changes. The test that guarantees every
+  adopter-settable environment variable is named in the scaffold's `env.example`
+  walked `packages/` with a `statSync` per entry, and descended into the fake npm
+  install another suite roots inside `packages/ksor`. That cost two ways: the
+  copied template sources were scanned twice, and an entry deleted between the
+  `readdir` and the `statSync` crashed the whole run — which is what took CI red
+  on run 32526491721, on an `llms.txt` being cleaned up concurrently.
+
+  The walk now takes each entry's type from the readdir snapshot itself, so a
+  vanishing entry cannot crash it, and it skips transient install trees, so its
+  input no longer depends on whether another suite is mid-run. The `REPO_ONLY`
+  exemption list was deleted as dead: it named seven variables that no scanned
+  file can contain, because the walk excludes test files in the first place. The
+  honesty check that is supposed to catch stale exemptions now covers every
+  exemption list, which is what its name always claimed.
+
 ## 0.0.13
 
 ### Patch Changes

package/dist/cli.mjs

@@ -16,7 +16,7 @@ import { bodyLimit } from "hono/body-limit";
 import { execFileSync, spawnSync } from "node:child_process";
 import { parseArgs } from "node:util";
 import { readFile, readdir, stat } from "node:fs/promises";
-//#region ../content-gateway/dist/main-B1G1VdMp.mjs
+//#region ../content-gateway/dist/main-wfyAlzsD.mjs
 /**
 * A connection could not be ESTABLISHED in time — retryable.
 *
@@ -42,8 +42,8 @@ var ConnectTimeoutError$1 = class extends Error {
 * a thundering herd aimed at the component already drowning.
 */
 var PoolTimeoutError$1 = class extends Error {
-	constructor(detail = "the configured checkout bound") {
-		super(`pool checkout timed out (${detail}) — the pool is saturated; shedding this request is the recovery path, retrying it is not`);
+	constructor() {
+		super("pool checkout timed out (the configured checkout bound) — the pool is saturated; shedding this request is the recovery path, retrying it is not");
 		this.name = "PoolTimeoutError";
 	}
 };
@@ -430,7 +430,7 @@ const EMBED_DIM$1 = 1536;
 const EMBED_TASK_DOCUMENT$1 = "RETRIEVAL_DOCUMENT";
 const EMBED_TASK_QUERY$1 = "RETRIEVAL_QUERY";
 /** bump ⇒ provenance (v5: CommonMark fences). All char limits count CODE POINTS (Python len parity). */
-const CHUNK_POLICY$1 = "heading-aware-1500-content-only-v5";
+const CHUNK_POLICY$1 = "heading-aware-1500-content-only-v6";
 /**
 * The kernel's view of `instance.md` (adapted from oracle SC/instance.py
 * under decision 11 — an adaptation, not a port: ksor has ONE instance file
@@ -3079,7 +3079,7 @@ function buildServer(ctx, version) {
 				structuredContent: result
 			};
 		} catch (error) {
-			if (error instanceof EmptyQueryError || error instanceof Error) return {
+			if (error instanceof Error) return {
 				content: [{
 					type: "text",
 					text: `Error: ${error.message}`
@@ -3188,7 +3188,7 @@ function toolError(error) {
 	return {
 		content: [{
 			type: "text",
-			text: `Error: ${error instanceof UnknownSlug || error instanceof Error ? error.message : String(error)}`
+			text: `Error: ${error instanceof Error ? error.message : String(error)}`
 		}],
 		isError: true
 	};
@@ -3366,12 +3366,10 @@ function configFromEnv(env) {
 	const allowedAudiences = (env.KSOR_JWT_ALLOWED_AUDIENCES ?? "").split(",").map((a) => a.trim()).filter((a) => a !== "");
 	const issuer = (env.KSOR_SSO_ISSUER ?? "").trim() || null;
 	const explicit = (env.KSOR_JWKS_URL ?? "").trim();
-	const jwksUrl = explicit || `${ssoUrl}/api/auth/jwks`;
-	assertHttpUrl("KSOR_JWKS_URL", jwksUrl, true);
+	assertHttpUrl("KSOR_JWKS_URL", explicit || `${ssoUrl}/api/auth/jwks`, true);
 	return {
 		ssoUrl,
 		resourceUrl,
-		jwksUrl,
 		explicitJwksUrl: explicit === "" ? null : explicit,
 		allowedAudiences,
 		issuer,
@@ -3492,7 +3490,7 @@ function createVerify(config, deps, jwksOf) {
 					cause: err
 				});
 			}
-			if (config.allowedAudiences.length > 0 && !audOk(claims.aud, config.allowedAudiences)) {
+			if (!audOk(claims.aud, config.allowedAudiences)) {
 				reject(key);
 				throw new TokenVerifyError(`token aud ${JSON.stringify(claims.aud ?? null)} not in allowlist ${JSON.stringify(config.allowedAudiences)}`, { transient: false });
 			}
@@ -3648,8 +3646,8 @@ const UNDESCRIBED_RECORD = "instance.md is still the scaffold template — agent
 * Composition (oracle main.py's boot order, adapted): instance → DSN via
 * the declared env NAME → provider → pool → space guard → service context.
 * Auth is built by the door that needs it (http.ts) — BEFORE the pool
-* serves anything; stdio is the local loopback-equivalent door and runs
-* with auth off by construction.
+* serves anything; a loopback bind is the local-equivalent door and is the
+* only posture that may run with auth explicitly disabled.
 */
 async function compose(instancePath, version) {
 	let instanceText;
@@ -4149,8 +4147,8 @@ var ConnectTimeoutError = class extends Error {
 * a thundering herd aimed at the component already drowning.
 */
 var PoolTimeoutError = class extends Error {
-	constructor(detail = "the configured checkout bound") {
-		super(`pool checkout timed out (${detail}) — the pool is saturated; shedding this request is the recovery path, retrying it is not`);
+	constructor() {
+		super("pool checkout timed out (the configured checkout bound) — the pool is saturated; shedding this request is the recovery path, retrying it is not");
 		this.name = "PoolTimeoutError";
 	}
 };
@@ -4454,7 +4452,7 @@ async function withPgRetry(op, options = {}) {
 	throw lastError;
 }
 //#endregion
-//#region ../content/dist/commands-LOfF_iLT.mjs
+//#region ../content/dist/commands-_8HvMWfD.mjs
 /**
 * EVAL-LOCKED constants, quarried verbatim from the oracle
 * (sor-agentfactory @ b554f91, config.py) — changing any of these is a
@@ -4469,7 +4467,7 @@ const EMBED_DIM = 1536;
 const EMBED_TASK_DOCUMENT = "RETRIEVAL_DOCUMENT";
 const EMBED_TASK_QUERY = "RETRIEVAL_QUERY";
 /** bump ⇒ provenance (v5: CommonMark fences). All char limits count CODE POINTS (Python len parity). */
-const CHUNK_POLICY = "heading-aware-1500-content-only-v5";
+const CHUNK_POLICY = "heading-aware-1500-content-only-v6";
 const MAX_CHARS = 1500;
 /** < Gemini's 2048-token embed input. */
 const HARD_MAX_CHARS = 4e3;
@@ -7884,11 +7882,48 @@ function cleanBody(rawBody) {
 function teachingBody(content) {
 	return pyStrip(pySplitLines(content, false).filter((ln) => !HEADING.test(ln)).join("\n"));
 }
+/**
+* A line that is navigation rather than prose: strip its list marker and what
+* remains is nothing but links.
+*
+* Deliberately narrow. A line with prose AROUND a link ("Claim within thirty
+* days; see the [expenses page](x) to file.") is prose, because the sentence is
+* the content and the link is incidental.
+*/
+const NAV_LINE = /^(?:[-*+]\s+|\d+[.)]\s+)?(?:\[[^\]]*\]\([^)]*\)|<https?:\/\/[^>]*>|https?:\/\/\S+)(?:[\s,;·|>—–-]*(?:\[[^\]]*\]\([^)]*\)|<https?:\/\/[^>]*>|https?:\/\/\S+))*[\s.,;:]*$/;
+/**
+* Is this segment NAVIGATION — a thing that points at content rather than
+* being content?
+*
+* The oracle answered this with length: under NAV_MAX_CHARS (250) meant nav.
+* On the curriculum corpus it was tuned against, that proxy holds — a short
+* segment there really is a link list. On a handbook it inverts, because a
+* handbook's most valuable statements are its shortest ("Six months, with a
+* written review at three and six"), and `nav` is excluded from search. Issue
+* #55, walked live on 0.0.14: three of four chunks in an ordinary policy
+* record were unsearchable, and a question the record plainly answered was
+* served the scaffold's placeholder instead.
+*
+* So the question is asked about SHAPE, which is what "navigation" always
+* meant. A segment is nav when link lines are most of it, or when what is left
+* after them is too little to answer anything (MIN_CONTENT_CHARS — the same
+* floor the serving predicate applies, so this never labels `prose` something
+* search would refuse to return anyway).
+*
+* Length is no longer consulted. A 180-character link list is nav and a
+* 51-character fact is prose, which is the ordering length got backwards.
+*/
+function isNavShaped(content) {
+	const lines = pySplitLines(teachingBody(content), false).map((ln) => pyStrip(ln)).filter((ln) => ln !== "");
+	if (lines.length === 0) return true;
+	if (lines.filter((ln) => NAV_LINE.test(ln)).length * 2 > lines.length) return true;
+	return cpLen(lines.filter((ln) => !NAV_LINE.test(ln)).join(" ")) < 24;
+}
 function classify(content, headingPath) {
 	if (JSX_ASSESS.test(content)) return "assessment";
 	const leaf = headingPath.length > 0 ? headingPath[headingPath.length - 1] : "";
 	if (JSX_EMBED.test(content) || content.includes("docs.google.com/presentation") || leaf.includes("Teaching Aid")) return "embed";
-	if (cpLen(teachingBody(content)) < 250) return "nav";
+	if (isNavShaped(content)) return "nav";
 	return "prose";
 }
 /** A segment DOMINATED by a line-leading widget (with < NAV_MAX_CHARS of
@@ -8008,7 +8043,7 @@ function chunkText(text, maxChars = MAX_CHARS) {
 			} else prefix += seg.text;
 			continue;
 		}
-		const segIsNav = cpLen(teachingBody(seg.text)) < 250;
+		const segIsNav = isNavShaped(seg.text);
 		const segMarker = segmentMarkerType(seg.text);
 		for (const piece of subsplit(seg.text, maxChars)) {
 			let sourceType;
@@ -8692,19 +8727,46 @@ function composeProvider(instance) {
 		return fail$1(REFUSED, `instance embedding.provider: ${exc instanceof Error ? exc.message : String(exc)}`);
 	}
 }
-/**
-* The commit the corpus was ingested from, resolved from git when the tree is
-* in a repository.
-*
-* `--source-commit` has always existed and the golden path never passed it, so
-* EVERY generation an adopter produced recorded the literal string
-* "unspecified" — product principle 6 requires a build to record the exact
-* corpus that produced it, and a placeholder records nothing (review
-* 2026-08-20). Resolved here rather than in the scaffold script so it is right
-* however the verb is invoked. A tree that is not a repository, or a git that
-* is not installed, still records the honest sentinel rather than failing an
-* ingest over provenance metadata.
-*/
+function provenanceGap(knowledgeDir) {
+	if (knowledgeDir === void 0) return "not-asked";
+	const run = (args) => {
+		try {
+			return {
+				ok: true,
+				out: execFileSync("git", [
+					"-C",
+					knowledgeDir,
+					...args
+				], {
+					encoding: "utf8",
+					stdio: [
+						"ignore",
+						"pipe",
+						"ignore"
+					]
+				}).trim()
+			};
+		} catch {
+			return {
+				ok: false,
+				out: ""
+			};
+		}
+	};
+	if (!run(["--version"]).ok && !run(["rev-parse", "--git-dir"]).ok) return "no-git";
+	if (!run(["rev-parse", "--git-dir"]).ok) return "no-repo";
+	return "no-commit";
+}
+/** The remedy for each, because the reader's next command differs. */
+function provenanceNotice(gap) {
+	const why = "so this generation cannot be traced back to a reviewed commit";
+	switch (gap) {
+		case "no-commit": return `source: unspecified — knowledge/ is in a git repository with no commits yet, ${why}.\n  fix: commit the record (git add knowledge && git commit) and re-run`;
+		case "no-repo": return `source: unspecified — knowledge/ is not in a git repository, ${why}.\n  fix: git init, commit the record, and re-run`;
+		case "no-git": return `source: unspecified — git is not installed, ${why}.\n  fix: install git, or pass --source-commit <sha> if the record is versioned elsewhere`;
+		case "not-asked": return `source: unspecified — no knowledge directory was given, ${why}.`;
+	}
+}
 function detectSourceCommit(knowledgeDir) {
 	if (knowledgeDir === void 0) return "unspecified";
 	try {
@@ -8890,7 +8952,7 @@ async function ingestCommand(args) {
 		process.stdout.write(`ingest: unchanged — generation ${report.generation} already serves this corpus\n`);
 		return 0;
 	}
-	process.stdout.write(sourceCommit === "unspecified" ? "source: unspecified — knowledge/ is not in a git repository, so this generation cannot be traced back to a reviewed commit\n" : `source: ${sourceCommit}\n`);
+	process.stdout.write(sourceCommit === "unspecified" ? provenanceNotice(provenanceGap(values.knowledge)) + "\n" : `source: ${sourceCommit}\n`);
 	process.stdout.write(`ingest: generation ${report.generation} — ${report.nodes} nodes, ${report.chunks} chunks; embedded ${report.embedded}, carried ${report.carried}, failed ${report.failed}\n`);
 	if (report.unsearchable > 0) {
 		const pct = Math.round(report.unsearchable / Math.max(report.chunks, 1) * 100);

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@panaversity/ksor",
-  "version": "0.0.13",
+  "version": "0.0.15",
   "description": "Knowledge System of Record — compile governed markdown into a static site for people and an MCP server for AI agents, with citations and measured abstention.",
   "keywords": [
     "abstention",