@nanobpm/urban 0.68.0 → 0.69.0

package/src/context/git/NOTES.md

@@ -0,0 +1,98 @@
+# `@nanobpm/urban/context/git` — S3 git substrate + PR-governance
+
+The git-as-system-of-record **write + governance** layer. Built on the merged
+S1 (binding), S2 (schema) and S6 (PII guard) slices — it composes them and does
+not re-implement clone/pull, the record schema, or the PII classifier/guard.
+
+## Write paths (`ContextWriter`)
+
+Construct a writer from a resolved substrate handle (`{ localPath, ref }` from
+`@nanobpm/urban/context/binding`):
+
+```ts
+import { ContextWriter } from "@nanobpm/urban/context/git";
+const writer = new ContextWriter(resolvedHandle); // no wiring needed
+```
+
+- `appendRecord(record)` — **append-via-commit**: validates the record against
+  the S2 schema (incl. the forgery-resistance invariants), runs the mandatory
+  PII guard, then commits it on a fresh write branch and merges it onto the base
+  branch. For `human` / `measured` / `instance` records. **`agent-retro` is
+  rejected here** — a hypothesis must go through governance.
+- `proposePrior(record)` — **PR-governance**: writes the record as a *proposed
+  prior* on a `context/proposal/<id>-<rand>` bot branch, left **unmerged**. This
+  is the git-only stand-in for a bot PR. Mandatory for `agent-retro`.
+- `ratify(proposal)` — **merges** the bot branch onto the base branch. Merge ==
+  ratification. Because the proposal branch may have been created or amended
+  outside `proposePrior`, the branch is untrusted, so before merging `ratify`:
+  (1) refuses any branch outside the `context/proposal/` namespace; (2) rejects any
+  proposal whose `baseBranch` differs from the writer's **own resolved base**, and
+  merges onto that resolved base (never the caller-supplied one), so a mutated
+  handle can't steer the merge off the line the writer is bound to; (3) bounds the
+  branch↔base diff to **exactly** the one proposed record file, so no extra file
+  (another record, or content outside the record layout) can ride onto the
+  authoritative line **unguarded**; (4) **re-reads, re-validates (S2 schema) and
+  re-runs the mandatory PII guard** against that record; and (5) asserts its
+  on-disk path is the record's canonical layout path. Only then is it merged — so
+  a PII-carrying, invalid, mislocated, smuggled, or misdirected record can never be
+  ratified. The ratifying merge's commit message is derived from the **guarded**
+  `record.id` (re-read and re-guarded above), never the caller-controlled
+  `proposalId`, so a mutated handle can't inject unguarded content (PII, newlines)
+  into the commit trail. Every commit/merge subject additionally passes `record.id`
+  through `commitLine` (collapses CR/LF and whitespace to a single line), so even a
+  guarded id can't split the subject to inject extra message lines or a forged
+  trailer — `id` is schema-validated only as a non-empty string.
+  It accepts the hypothesis onto the authoritative line; it never upgrades an
+  `agent-retro` record to `authoritative` (the S2 schema forbids that), so an
+  unratified — or even a ratified — prior can never present as a
+  measured/authoritative fact.
+- `isRatified(proposal)` — `true` iff the proposal's commit is an ancestor of the
+  writer's **own resolved base**. Like `ratify`, it rejects a proposal whose
+  `baseBranch` was mutated off that resolved base rather than trusting the
+  plain-object handle, so a retargeted handle can't report an unmerged hypothesis
+  as ratified.
+
+Concurrency: **within a single `ContextWriter`** operations are serialised on the
+shared working tree (one working tree) and each write still lands on its own
+uniquely-named branch, so interleaved calls never clobber each other and disjoint
+records merge cleanly. This is a **per-instance** guarantee: two `ContextWriter`
+instances (or separate processes) over the **same** working copy are uncoordinated
+and can corrupt each other — use a separate clone (or external locking) per
+concurrent writer. Each mutating op also restores the working tree to the resolved
+base in a `finally` (a force-checkout plus a layout-scoped clean), so a
+mid-operation failure never strands the shared tree on a transient write/proposal
+branch — with a half-written record — for the next serialised call to inherit.
+
+## PII enforcement BY CONSTRUCTION
+
+`PreCommitGuardRegistry` is **always** seeded with the mandatory S6
+`preCommitPiiGuard` as its first, non-removable member and has no removal API.
+The writer runs `assertAll` before **every** commit, so a PII-carrying write is
+rejected before it is committed — on the default code path, with no caller
+opt-in. `ratify` (a merge onto the authoritative line) is a write path too, so it
+re-runs `assertAll` against the proposal's content before merging. Additional
+guards can only be added (stricter), never subtracted.
+
+## Path / namespace partitioning (read by S4 & S6-CI)
+
+Records are persisted under a stable layout, exported as a small helper so S4
+retrieval and the S6 CI scan read the **same** scheme:
+
+```
+records/<scope>/<scopeRef-bucket>/<id>.json
+```
+
+- `LAYOUT_ROOT` (`"records"`), `RECORD_FILE_EXTENSION` (`".json"`),
+  `UNSCOPED_BUCKET` (`"_"`, for records with no `scopeRef`).
+- `scopeDir(scope)`, `recordDir(scope, scopeRef?)`, `recordRelativePath(record)`.
+- `sanitizeSegment(value)` — every path component is reduced to a
+  filesystem-safe segment that can never contain a separator or `..`
+  (path-traversal safe by construction).
+- `isRecordPath(relPath)` — selects record files while walking the substrate.
+
+## Seam for a future PII/mutable backend
+
+The write path is abstracted over the `WriteSubstrate` interface, not hard-wired
+to public git. `GitWriteSubstrate` is the default git-only MVP implementation; a
+future mutable/PII backend (with real erasure) can implement the same interface
+without the writer or its callers changing.

package/src/context/git/guard-registry.ts

@@ -0,0 +1,59 @@
+// Slice S3 (git/governance) — the mandatory pre-commit guard registry.
+//
+// This is the seam that makes PII enforcement NON-OPTIONAL and default-DENY BY
+// CONSTRUCTION on every write path. A registry is ALWAYS seeded with the S6
+// mandatory guard (`preCommitPiiGuard`) as its first, non-removable member:
+//
+//  - there is no method to remove or replace the default guard, and
+//  - a freshly-constructed registry (the default the writer uses when a caller
+//    supplies no wiring at all) already contains it,
+//
+// so a caller cannot silently bypass PII enforcement. Additional guards can only
+// be ADDED (making the policy stricter), never subtracted. The writer calls
+// {@link PreCommitGuardRegistry.assertAll} immediately before it stages/commits
+// any record, so a write carrying PII is rejected before it is ever committed.
+
+import type { PiiCandidate, PiiGuard } from "../pii/index.ts";
+import { preCommitPiiGuard } from "../pii/index.ts";
+
+/**
+ * An append-only registry of pre-commit guards. The mandatory S6 PII guard is
+ * pre-registered as the default and cannot be removed; callers may register
+ * ADDITIONAL guards, which run in addition to (never instead of) the default.
+ */
+export class PreCommitGuardRegistry {
+  // Index 0 is ALWAYS the mandatory S6 PII guard. There is no removal API, so it
+  // is impossible to end up with a registry that does not enforce it.
+  readonly #guards: PiiGuard[];
+
+  /**
+   * Build a registry. It is seeded with the mandatory {@link preCommitPiiGuard}
+   * first; any `additional` guards are appended after it. Even with no arguments
+   * the resulting registry enforces PII by construction.
+   */
+  constructor(additional: readonly PiiGuard[] = []) {
+    this.#guards = [preCommitPiiGuard, ...additional];
+  }
+
+  /** Register an ADDITIONAL guard. Cannot displace the mandatory default guard. */
+  register(guard: PiiGuard): void {
+    this.#guards.push(guard);
+  }
+
+  /** A snapshot of the registered guards, in run order (mandatory guard first). */
+  get guards(): readonly PiiGuard[] {
+    return [...this.#guards];
+  }
+
+  /**
+   * Run EVERY registered guard against `candidate`. The mandatory PII guard runs
+   * first. Each guard is default-DENY: the first guard to detect a violation
+   * throws (a {@link import("../pii/index.ts").PiiGuardError} for the PII guard),
+   * so a candidate carrying PII never reaches the commit that follows this call.
+   */
+  assertAll(candidate: PiiCandidate): void {
+    for (const guard of this.#guards) {
+      guard.assert(candidate);
+    }
+  }
+}

package/src/context/git/index.ts

@@ -1,2 +1,51 @@
-// Slice S3 (git/governance) — owner replaces this placeholder barrel.
-export {};
+// @nanobpm/urban/context/git — slice S3 (git substrate + PR-governance).
+//
+// The git-as-system-of-record WRITE + governance layer, built on top of a
+// resolved substrate handle (S1), the validated record schema (S2), and the
+// mandatory PII pre-commit guard (S6). It publishes three things downstream:
+//
+//  1. APPEND / GOVERNANCE — ContextWriter: append-via-commit for
+//     human/measured/instance facts, and PR-governance (propose → ratify) for
+//     agent-retro hypotheses, with the mandatory PII guard wired into EVERY write
+//     path so PII is rejected before any commit, by construction.
+//  2. PATH / NAMESPACE PARTITIONING — the layout helper (recordRelativePath,
+//     recordDir, scopeDir, isRecordPath, LAYOUT_ROOT). S4 retrieval and the S6 CI
+//     slice READ this exact layout, so it is exported as a stable seam.
+//  3. THE WRITE SUBSTRATE SEAM — WriteSubstrate / GitWriteSubstrate: the write
+//     path is abstracted over the substrate (not hard-wired to public git),
+//     leaving room for a future PII/mutable backend, and the
+//     PreCommitGuardRegistry that makes the PII guard non-bypassable.
+
+export {
+  LAYOUT_ROOT,
+  RECORD_FILE_EXTENSION,
+  UNSCOPED_BUCKET,
+  isRecordPath,
+  recordDir,
+  recordRelativePath,
+  sanitizeSegment,
+  scopeDir,
+} from "./layout.ts";
+
+export { PreCommitGuardRegistry } from "./guard-registry.ts";
+
+export {
+  type CommitAuthor,
+  GitWriteSubstrate,
+  SubstrateWriteError,
+  substratePath,
+  type WriteSubstrate,
+} from "./substrate.ts";
+
+export {
+  type AppendResult,
+  ContextWriter,
+  type ContextWriterOptions,
+  DEFAULT_BOT_AUTHOR,
+  DEFAULT_WRITE_AUTHOR,
+  GovernanceError,
+  type ProposalResult,
+  type RatifyResult,
+  type ResolvedSubstrateHandle,
+  serialiseRecord,
+} from "./writer.ts";

package/src/context/git/layout.test.ts

@@ -0,0 +1,112 @@
+// Slice S3 (git/governance) — unit tests for the path/namespace layout helper.
+
+import assert from "node:assert/strict";
+import { test } from "node:test";
+import type { MemoryRecord } from "../schema/index.ts";
+import {
+  LAYOUT_ROOT,
+  UNSCOPED_BUCKET,
+  isRecordPath,
+  recordDir,
+  recordRelativePath,
+  sanitizeSegment,
+  scopeDir,
+} from "./layout.ts";
+
+function record(overrides: Partial<MemoryRecord>): MemoryRecord {
+  const base: MemoryRecord = {
+    schemaVersion: 1,
+    id: "rec-1",
+    scope: "epic",
+    mode: "empirical",
+    provenance: "measured",
+    authority: "authoritative",
+    statement: "a measured fact",
+    createdAt: "2026-01-01T00:00:00Z",
+  };
+  return { ...base, ...overrides };
+}
+
+test("scopeDir / recordDir partition by scope and scopeRef", () => {
+  assert.equal(scopeDir("epic"), `${LAYOUT_ROOT}/epic`);
+  assert.equal(recordDir("epic", "issue-303"), `${LAYOUT_ROOT}/epic/issue-303`);
+  // No scopeRef → the shared bucket.
+  assert.equal(recordDir("corpus"), `${LAYOUT_ROOT}/corpus/${UNSCOPED_BUCKET}`);
+});
+
+test("recordRelativePath is deterministic and scoped", () => {
+  const r = record({ scope: "epic", scopeRef: "issue-303", id: "rec-42" });
+  assert.equal(recordRelativePath(r), `${LAYOUT_ROOT}/epic/issue-303/rec-42.json`);
+  // Same record → same path (idempotent write target).
+  assert.equal(recordRelativePath(r), recordRelativePath({ ...r }));
+});
+
+test("records without scopeRef land in the shared bucket", () => {
+  const r = record({ scope: "corpus", id: "global-1" });
+  assert.equal(recordRelativePath(r), `${LAYOUT_ROOT}/corpus/${UNSCOPED_BUCKET}/global-1.json`);
+});
+
+test("sanitizeSegment neutralises path traversal and separators", () => {
+  // A traversal attempt can never produce a `..` or a separator.
+  const nasty = sanitizeSegment("../../etc/passwd");
+  assert.ok(!nasty.includes("/"));
+  assert.ok(!nasty.includes(".."));
+  assert.notEqual(nasty, "");
+  // A clean value is preserved verbatim.
+  assert.equal(sanitizeSegment("rec-1"), "rec-1");
+});
+
+test("distinct lossy ids never collide onto one segment", () => {
+  const a = sanitizeSegment("a/b");
+  const b = sanitizeSegment("a\\b");
+  assert.notEqual(a, b);
+});
+
+test("a traversal id cannot escape the layout root", () => {
+  const r = record({ scope: "epic", scopeRef: "../../escape", id: "../../../root" });
+  const path = recordRelativePath(r);
+  assert.ok(path.startsWith(`${LAYOUT_ROOT}/epic/`));
+  assert.ok(!path.includes(".."));
+});
+
+test("isRecordPath selects record files under the layout root", () => {
+  assert.equal(isRecordPath("records/epic/issue-303/rec-42.json"), true);
+  assert.equal(isRecordPath("./records/corpus/_/global-1.json"), true);
+  assert.equal(isRecordPath("records\\epic\\x\\r.json"), true);
+  assert.equal(isRecordPath("records/epic/x/notes.md"), false);
+  assert.equal(isRecordPath("other/epic/x/r.json"), false);
+  assert.equal(isRecordPath("records/../secrets/r.json"), false);
+});
+
+test("sanitizeSegment eliminates internal '..' sequences", () => {
+  // An internal `..` (e.g. "a..b") must not survive: `isRecordPath` rejects any
+  // path containing "..", so a record whose id/scopeRef carried one would be
+  // invisible to retrieval/CI. It is disambiguated with a hash, never dropped.
+  const seg = sanitizeSegment("a..b");
+  assert.ok(!seg.includes(".."));
+  assert.notEqual(seg, sanitizeSegment("a.b"));
+});
+
+test("a sanitised scopeRef can never collide with UNSCOPED_BUCKET", () => {
+  // A raw scopeRef of exactly "_" must NOT map onto the shared unscoped bucket,
+  // otherwise a scoped record and an unscoped record share a directory.
+  const seg = sanitizeSegment(UNSCOPED_BUCKET);
+  assert.notEqual(seg, UNSCOPED_BUCKET);
+  assert.notEqual(
+    recordDir("epic", UNSCOPED_BUCKET),
+    recordDir("epic"),
+  );
+});
+
+test("every recordRelativePath output is a valid isRecordPath (no writer/scan drift)", () => {
+  // The writer's output and the retrieval/CI scanner must agree by construction:
+  // any path recordRelativePath emits must be selected by isRecordPath, for even
+  // adversarial ids/scopeRefs.
+  const adversarial = ["a..b", "..", ".", "_", "../../escape", "", "  ", "a/b\\c"];
+  for (const scopeRef of adversarial) {
+    for (const id of adversarial) {
+      const path = recordRelativePath(record({ scope: "epic", scopeRef, id }));
+      assert.equal(isRecordPath(path), true, `not a record path: ${path}`);
+    }
+  }
+});

package/src/context/git/layout.ts

@@ -0,0 +1,110 @@
+// Slice S3 (git/governance) — the on-disk path / namespace partitioning.
+//
+// Records are persisted into the substrate repo under a stable, deterministic
+// layout keyed by the record's scope (and, where present, its `scopeRef`). This
+// helper is the SINGLE SOURCE OF TRUTH for that layout: the write path (this
+// slice) writes to it, and BOTH S4 (retrieval) and the S6 CI slice READ it, so
+// it is exported as a small, stable surface rather than duplicated per-consumer.
+//
+// The scheme is:
+//
+//   records/<scope>/<scopeRef-bucket>/<id>.json
+//
+// where `<scope>` is a controlled-vocabulary ladder level (element … corpus),
+// `<scopeRef-bucket>` is the sanitised `scopeRef` (or the shared `_` bucket when
+// a record carries no `scopeRef`), and `<id>` is the sanitised record id. Every
+// path component is sanitised to a filesystem-safe segment that can never
+// contain a path separator or `..`, so an untrusted `id`/`scopeRef` can never
+// escape the layout root (path-traversal safe by construction).
+
+import { createHash } from "node:crypto";
+import type { MemoryRecord, MemoryScope } from "../schema/index.ts";
+
+/** The directory (relative to the substrate root) all records live under. */
+export const LAYOUT_ROOT = "records" as const;
+
+/** The on-disk extension every serialised record file carries. */
+export const RECORD_FILE_EXTENSION = ".json" as const;
+
+/**
+ * The bucket directory used for records that carry no `scopeRef` (e.g. a
+ * `corpus`-scoped record). `sanitizeSegment` never returns exactly `"_"` — an
+ * input that would sanitise to `"_"` is treated as reserved and gets a `_<hash>`
+ * form instead — so a sanitised `scopeRef` can never collide with this bucket.
+ */
+export const UNSCOPED_BUCKET = "_" as const;
+
+/**
+ * Reduce an untrusted string to a single filesystem-safe path segment.
+ *
+ * The result contains only `[A-Za-z0-9._-]`, never a path separator, never a
+ * `..` sequence, and is never `""`, `.`, `..`, or exactly `"_"` — so it can
+ * never traverse out of its parent directory nor collide with
+ * {@link UNSCOPED_BUCKET}. When sanitisation is LOSSY (the cleaned form differs
+ * from the input, or collapses to a reserved value) a short hash of the ORIGINAL
+ * value is appended, so two distinct raw values can never silently collide onto
+ * one segment while a clean value is preserved verbatim.
+ */
+export function sanitizeSegment(value: string): string {
+  const cleaned = value
+    .trim()
+    .replace(/[^A-Za-z0-9._-]+/g, "-")
+    .replace(/\.{2,}/g, ".")
+    .replace(/^[-.]+|[-.]+$/g, "")
+    .slice(0, 100);
+  const reserved =
+    cleaned === "" ||
+    cleaned === "." ||
+    cleaned === ".." ||
+    cleaned === UNSCOPED_BUCKET;
+  if (reserved) {
+    return `_${shortHash(value)}`;
+  }
+  // Preserve a clean, reversible value verbatim; disambiguate a lossy one so
+  // distinct inputs never map to the same segment.
+  return cleaned === value ? cleaned : `${cleaned}-${shortHash(value)}`;
+}
+
+function shortHash(value: string): string {
+  return createHash("sha256").update(value, "utf8").digest("hex").slice(0, 12);
+}
+
+/** The directory (relative to the substrate root) that holds a whole scope. */
+export function scopeDir(scope: MemoryScope): string {
+  return `${LAYOUT_ROOT}/${scope}`;
+}
+
+/**
+ * The directory a record with the given `scope`/`scopeRef` is partitioned into.
+ * Records sharing a `(scope, scopeRef)` namespace land in the same directory, so
+ * a retrieval slice can list one namespace cheaply.
+ */
+export function recordDir(scope: MemoryScope, scopeRef?: string): string {
+  const bucket = scopeRef === undefined ? UNSCOPED_BUCKET : sanitizeSegment(scopeRef);
+  return `${scopeDir(scope)}/${bucket}`;
+}
+
+/**
+ * The full path (relative to the substrate root, POSIX separators) a record is
+ * persisted at. Deterministic: the same record id/scope always maps to the same
+ * path, so re-appending an updated record with the same id overwrites its file
+ * in place while git retains the full append-only history.
+ */
+export function recordRelativePath(record: MemoryRecord): string {
+  return `${recordDir(record.scope, record.scopeRef)}/${sanitizeSegment(record.id)}${RECORD_FILE_EXTENSION}`;
+}
+
+/**
+ * `true` iff `relPath` looks like a record file under the layout root (used by
+ * S4 retrieval and the S6 CI scan to select record files while walking the
+ * substrate). Accepts POSIX or native separators and rejects anything that is
+ * not a `.json` file beneath `records/`.
+ */
+export function isRecordPath(relPath: string): boolean {
+  const normalised = relPath.replace(/\\/g, "/").replace(/^\.\//, "");
+  return (
+    normalised.startsWith(`${LAYOUT_ROOT}/`) &&
+    normalised.endsWith(RECORD_FILE_EXTENSION) &&
+    !normalised.includes("..")
+  );
+}