@codyswann/lisa 4.20.0 → 4.21.0

package/plugins/src/base/rules/reference/project-learnings.md

@@ -17,9 +17,10 @@ then recency — that fit the token/entry budget, plus how many were omitted).
 the bounded projection, not the full document. Do not duplicate numeric caps in
 rules or prompts; the exported contract is the source of truth.
 
-Each persisted entry has seven fields:
+Each persisted v2 entry has eight fields:
 
 - `id`
+- `fingerprint`
 - `rule`
 - `why`
 - `provenance`
@@ -27,6 +28,36 @@ Each persisted entry has seven fields:
 - `last_confirmed`
 - `confidence`
 
+`fingerprint` is the content-version token; `id` is the stable public identity.
+A new entry starts with `id = fingerprint`. Consolidation passes exact
+`{ id, fingerprint }` stamps copied from parsed entries. The writer checks the
+whole target set inside its lock: an exact set is replaced together while the
+deterministic primary id carries forward; any stale stamp removes nothing and
+the new fingerprint is appended safely. Fingerprints are unique, so an
+identical re-capture fails before mutation rather than gaining a sibling.
+
+## v1 compatibility and mutation-only migration
+
+`parseLearningsDocument` accepts both contract versions and reports
+`sourceVersion`, normalized `entries`, `canonicalSource`, and
+`sourceMaxTokens`. A v1 entry is normalized in memory with `fingerprint = id`,
+which records the version-token job its old id already performed. A read never
+rewrites the ledger. The next real writer mutation — append, exact stamped
+consolidation, confirmation, overflow drain, or merge — publishes one atomic v2
+document. Each source version is validated against its own derived budget; do
+not copy either numeric ceiling into instructions. Stable ids and fingerprints
+are bounded machine tokens, and the v2 budget is derived to absorb the maximum
+per-entry fingerprint expansion for every v1 document the compatibility parser
+accepts. That migration-closure invariant prevents an ordinary later mutation
+from discovering that a previously accepted legacy document cannot be
+published.
+
+The three-way merge understands mixed v1/v2 sides. When two branches both
+legitimately replace the same stable id from the same observed fingerprint, it
+keeps both successors deterministically: one retains the stable id and the
+other is keyed by its unique fingerprint. It never guesses when fingerprints
+collide.
+
 ## Who writes the ledger
 
 **Contract-mediated writers** — all going through the executable contract
@@ -34,6 +65,8 @@ Each persisted entry has seven fields:
 
 - The **learner agent** at capture time appends or consolidates new entries
   (`persistLearningEntry` / `persistConsolidatedLearning`) from task learnings.
+  Consolidations pass exact parsed `{ id, fingerprint }` stamps and report a
+  stale snapshot through `onStaleSupersede` instead of overwriting its winner.
   It is capture-only: it never appends to host rules, creates skills, or
   files upstream issues — promotion is the gardener's ticket-gated job.
 - **`lisa-debrief-apply`** routes accepted debrief findings in the three

package/plugins/src/base/skills/lisa-debrief-apply/SKILL.md

@@ -51,15 +51,16 @@ For each such Accepted row:
    ```
 
 2. **Consolidation check (mandatory before writing).** Parse the existing entries with `parseLearningsFile` and look for one related to the row (same failure class, overlapping topic, or near-duplicate wording). Then write through the contract:
-   - **Related entry found** → consolidate via `persistConsolidatedLearning(projectRoot, entry, { supersede: [<related ids>] })`, merging the old entry's still-true content into the new rule. Never append a near-duplicate sibling — a sibling is a bug.
+   - **Related entry found** → consolidate with the exact parsed versions: `persistConsolidatedLearning(projectRoot, entry, { supersede: [{ id: <related id>, fingerprint: <related fingerprint> }], onStaleSupersede: targets => report(targets) })`. Every stamp is checked together inside the writer lock. If any is stale, the writer removes none, safely appends the new fingerprint, and reports the mismatch. Never append a near-duplicate sibling by choice — a sibling is a bug; the stale append is the deliberate concurrent-writer preservation path.
    - **No related entry** → append via `persistLearningEntry(projectRoot, entry)`.
 
-3. **Entry mapping (seven fields).**
-   - `id` — the entry id returned by the contract writer (report it in the run summary).
+3. **Entry mapping (eight fields).**
+   - `fingerprint` — `debrief-` + the first 12 hex characters of `sha1(normalized_rule + "\n" + sorted_provenance.join("\n"))`, where the rule is lowercased with whitespace collapsed and provenance is sorted by codepoint. Compute it mechanically, never estimate it.
+   - `id` — initially `id = fingerprint`. On an exact stamped consolidation the writer carries forward the deterministic primary target id; report the returned id in the run summary.
    - `rule` / `why` — from the row's `Summary` and the category-specific guidance in the routing table above (≤240 chars, ≤2 lines).
    - `provenance` — **the triage-doc row's evidence links**. This is the same evidence link that doubles as the idempotency fingerprint below, so the row's provenance is what a later re-apply scans for.
    - `first_learned` = `last_confirmed` = today (ISO date; on consolidation keep the superseded entry's earliest `first_learned`).
-   - `confidence` = **`high`**. A human marking the row **Accept** is corroboration — an independent human judgement that the learning is real — so a debrief-accepted entry starts higher than the learner's single-occurrence auto-capture (which defaults to `low`). The writer re-asserts the entry and token budgets; an over-budget failure means consolidate harder or drop, never truncate by hand.
+   - `confidence` = **`high`**. A human marking the row **Accept** is corroboration — an independent human judgement that the learning is real — so a debrief-accepted entry starts higher than the learner's single-occurrence auto-capture (which defaults to `low`). A duplicate fingerprint fails before mutation. The writer re-asserts the entry and token budgets; an over-budget failure means consolidate harder or drop, never truncate by hand.
 
 **Machine-local memory is no longer a knowledge destination.** Auto-memory (`project_*.md`, `MEMORY.md`) remains available only for the assistant's *personal* collaboration notes — it is invisible to cloud runs and to teammates, so it can never hold shared project knowledge. **`AGENTS.md` is human-authored** agent operating instruction — it is the source of truth, and `CLAUDE.md` is only a one-line `@AGENTS.md` pointer — and the host-rules directory `.agents/rules/` is durable human-authored guidance; `apply` never writes to any of the three for these categories.
 

package/plugins/src/base/skills/lisa-persist-learning/SKILL.md

@@ -171,9 +171,9 @@ No learning content is ever committed without a PR — there is no other write p
    ```
 
 3. **Consolidation check (mandatory before writing).** Parse the existing entries (`parseLearningsFile` from `@codyswann/lisa/learnings`) and look for entries related to the new rule (same failure class, overlapping topic, or near-duplicate wording). Then write through the executable contract — **never hand-edit the markdown**:
-   - **Related entry found** → consolidate via `persistConsolidatedLearning(projectRoot, entry, { supersede: [<related ids>] })`, merging the old entry's still-true content into the new rule. Never append a near-duplicate sibling — a sibling is a bug that fails review.
+   - **Related entry found** → consolidate with the exact versions returned by the parser: `persistConsolidatedLearning(projectRoot, entry, { supersede: [{ id: <related id>, fingerprint: <related fingerprint> }], onStaleSupersede: targets => report(targets) })`. The writer checks every stamp together inside the lock. If any target is absent or its fingerprint changed, the whole supersede is stale: it removes nothing, safely appends the new fingerprint, and reports the mismatch. Never append a near-duplicate sibling by choice — a sibling is a bug that fails review; the writer's stale append is the deliberate race-safe preservation path.
    - **No related entry** → append via `persistLearningEntry(projectRoot, entry)` and state in the PR body why appending was correct.
-   - Entry mapping: `id` = the fingerprint; `rule`/`why`/`provenance` from the candidate; `first_learned` = `last_confirmed` = today (ISO date; on consolidation keep the superseded entry's earliest `first_learned`); `confidence` = the judge's `high`/`low`. The writer re-asserts the entry and token budgets — an over-budget failure means consolidate harder or drop, never truncate by hand.
+   - Entry mapping (eight fields): `fingerprint` = the deterministic Phase 0 fingerprint and initial `id = fingerprint`; `rule`/`why`/`provenance` from the candidate; `first_learned` = `last_confirmed` = today (ISO date; on consolidation keep the superseded entry's earliest `first_learned`); `confidence` = the judge's `high`/`low`. On an exact stamped consolidation the writer carries forward the deterministic primary target id while persisting the new fingerprint as the version token. A duplicate fingerprint fails before mutation. The writer re-asserts the entry and token budgets — an over-budget failure means consolidate harder or drop, never truncate by hand.
    - **On a budget-forced drop, signal saturation once (never silent).** When the writer's budget re-assertion cannot fit the entry even after consolidating harder — a durable capture has to be DROPPED for budget — the ledger is saturated, and that pressure must be visible to an operator instead of swallowed. Emit **exactly one** tracker signal via `lisa-tracker-write` (`issue_type: Task`; GitHub trackers carry the `type:Task` label), then continue — dropping the capture never blocks the build. Follow the same marker-dedupe discipline as every other comment here (match on the **marker, never the title**; **exactly one marker per body**; the eventual-consistency guard when the search index is stale), with **one deliberate difference: dedupe against OPEN signals only.** A *closed* saturation ticket means room was already reclaimed, so a fresh saturation is a new actionable event — searching closed tickets too (as the drop/upstream markers do) would permanently suppress every later saturation.
 
      The saturation fingerprint keys on the **ledger, not the candidate**, so the signal fires once per saturation episode — never once per capture:

package/scripts/verify-learner-frontmatter-built.mjs

@@ -12,7 +12,7 @@ import { parse as parseToml } from "smol-toml";
 import { discoverAndInstallAgents as installOpencodeAgents } from "../dist/opencode/agent-installer.js";
 
 const EXPECTED_DESCRIPTION =
-  "Post-implementation learning agent. Capture-only — collects task learnings, builds seven-field entries, and persists them to the machine-managed ledger through the executable contract with provenance. Never promotes: it creates no skills, appends no rules, files no upstream issues; promotion is exclusively the gardener's ticket-gated job.";
+  "Post-implementation learning agent. Capture-only — collects task learnings, builds eight-field entries, and persists them to the machine-managed ledger through the executable contract with provenance. Never promotes: it creates no skills, appends no rules, files no upstream issues; promotion is exclusively the gardener's ticket-gated job.";
 const executeFile = promisify(execFile);
 const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..");
 const destination = await mkdtemp(

package/scripts/verify-packed-learnings-contract.mjs

@@ -0,0 +1,268 @@
+#!/usr/bin/env node
+/** Empirical proof that the packed public learnings API preserves v2 semantics. */
+import assert from "node:assert/strict";
+import { execFile } from "node:child_process";
+import { mkdir, mkdtemp, readFile, rm } from "node:fs/promises";
+import path from "node:path";
+import { pathToFileURL } from "node:url";
+import { promisify } from "node:util";
+
+const executeFile = promisify(execFile);
+const root = path.resolve(import.meta.dirname, "..");
+const temporary = await mkdtemp(path.join(root, ".packed-learnings-proof-"));
+
+try {
+  const archive = path.join(temporary, "lisa-packed.tgz");
+  await executeFile(
+    "bun",
+    ["pm", "pack", "--ignore-scripts", "--filename", archive, "--quiet"],
+    { cwd: root }
+  );
+  await executeFile("/usr/bin/tar", ["-xzf", archive, "-C", temporary]);
+  const packedRoot = path.join(temporary, "package");
+  const learnings = await import(
+    pathToFileURL(path.join(packedRoot, "dist", "core", "learnings.js")).href
+  );
+
+  assert.equal(learnings.LEARNINGS_CONTRACT.version, 2);
+  assert.equal(learnings.LEARNINGS_CONTRACT.fields.length, 8);
+  assert.equal(learnings.MAX_STABLE_TOKEN_BYTES, 128);
+  assert.equal(learnings.LEARNINGS_CONTRACT.maxTokens, 14_900);
+
+  const packedVictim = "packed-victim";
+  const packedProvenance = "issue:#2015";
+  const packedDate = "2026-08-26";
+  const packedEntry = {
+    id: packedVictim,
+    fingerprint: "packed-victim-fingerprint",
+    rule: "Keep packed entry validation inert.",
+    why: "Consumers pass untrusted values through the public writer.",
+    provenance: [packedProvenance],
+    first_learned: packedDate,
+    last_confirmed: packedDate,
+    confidence: "high",
+  };
+  let entryGetterCalls = 0;
+  const accessorEntry = { ...packedEntry };
+  Object.defineProperty(accessorEntry, "why", {
+    get: () => {
+      entryGetterCalls += 1;
+      return "hostile";
+    },
+  });
+  assert.throws(
+    () => learnings.validateLearningEntry(accessorEntry),
+    /why.*accessor/i
+  );
+  assert.equal(entryGetterCalls, 0);
+  const hostileProvenance = [packedProvenance];
+  hostileProvenance.unexpected = true;
+  assert.throws(
+    () =>
+      learnings.validateLearningEntry({
+        ...packedEntry,
+        provenance: hostileProvenance,
+      }),
+    /provenance array/i
+  );
+
+  let stampGetterCalls = 0;
+  const accessorStamps = [];
+  accessorStamps.length = 1;
+  Object.defineProperty(accessorStamps, "0", {
+    get: () => {
+      stampGetterCalls += 1;
+      return { id: packedVictim, fingerprint: packedVictim };
+    },
+  });
+  assert.throws(
+    () => learnings.validateLearningEntryStamps(accessorStamps),
+    /supersede array/i
+  );
+  assert.equal(stampGetterCalls, 0);
+  Object.defineProperties(Object.prototype, {
+    id: { configurable: true, value: { value: packedVictim } },
+    fingerprint: {
+      configurable: true,
+      value: { value: packedVictim },
+    },
+  });
+  try {
+    const pollutedEntry = { ...packedEntry };
+    delete pollutedEntry.id;
+    delete pollutedEntry.fingerprint;
+    pollutedEntry.unrelatedId = "ignored";
+    pollutedEntry.unrelatedFingerprint = "ignored";
+    assert.throws(
+      () => learnings.validateLearningEntry(pollutedEntry),
+      /entry fields.*exactly/i
+    );
+    assert.throws(
+      () =>
+        learnings.validateLearningEntryStamps([
+          { harmless: true, stillHarmless: true },
+        ]),
+      /exactly.*id.*fingerprint/i
+    );
+  } finally {
+    delete Object.prototype.id;
+    delete Object.prototype.fingerprint;
+  }
+  assert.throws(
+    () =>
+      learnings.validateLearningEntryStamps([
+        { id: "A".repeat(129), fingerprint: packedVictim },
+      ]),
+    /id exceeds max stable token bytes 128/i
+  );
+  const inheritedProvenance = [];
+  inheritedProvenance.length = 1;
+  Object.defineProperty(Object.prototype, "0", {
+    configurable: true,
+    writable: true,
+    value: { value: "issue:#forged" },
+  });
+  try {
+    assert.throws(
+      () =>
+        learnings.validateLearningEntry({
+          ...packedEntry,
+          provenance: inheritedProvenance,
+        }),
+      /provenance array/i
+    );
+  } finally {
+    delete Object.prototype["0"];
+  }
+
+  const legacy = `# Project Learnings\n\n<!-- lisa-learnings-contract:v1 -->\n\n\`\`\`jsonl\n{"id":"learner-base","rule":"Keep packed compatibility executable.","why":"Consumers install the tarball, not the source tree.","provenance":["issue:#2015"],"first_learned":"2026-08-26","last_confirmed":"2026-08-26","confidence":"high"}\n\`\`\`\n`;
+  const normalized = learnings.parseLearningsDocument(legacy);
+  assert.equal(normalized.sourceVersion, 1);
+  assert.equal(normalized.entries[0].fingerprint, "learner-base");
+  const oversizedLegacyId = "a".repeat(7007);
+  const oversizedLegacy = `# Project Learnings\n\n<!-- lisa-learnings-contract:v1 -->\n\n\`\`\`jsonl\n${JSON.stringify({ ...JSON.parse(legacy.split("\n")[5]), id: oversizedLegacyId, rule: "r", why: "w".repeat(20), provenance: ["p"] })}\n\`\`\`\n`;
+  assert.equal(Buffer.byteLength(oversizedLegacy, "utf8"), 7226);
+  assert.throws(
+    () => learnings.parseLearningsDocument(oversizedLegacy),
+    /id exceeds max stable token bytes 128/i
+  );
+
+  const project = path.join(temporary, "consumer");
+  await mkdir(project);
+  const base = {
+    ...normalized.entries[0],
+    fingerprint: "learner-base-fingerprint",
+  };
+  await learnings.persistLearningEntry(project, base);
+  const observed = { id: base.id, fingerprint: base.fingerprint };
+  const stale = [];
+  const candidates = Array.from({ length: 9 }, (_unused, index) => ({
+    ...base,
+    id: `learner-candidate-${index}`,
+    fingerprint: `learner-candidate-fingerprint-${index}`,
+    rule: `Packed writer candidate ${index}.`,
+  }));
+  await Promise.all(
+    candidates.map(candidate =>
+      learnings.persistConsolidatedLearning(project, candidate, {
+        supersede: [observed],
+        onStaleSupersede: targets => stale.push(targets),
+      })
+    )
+  );
+
+  const ledgerPath = path.join(project, ".lisa", "PROJECT_LEARNINGS.md");
+  const afterRace = learnings.parseLearningsFile(
+    await readFile(ledgerPath, "utf8")
+  );
+  assert.equal(afterRace.length, 9);
+  assert.equal(stale.length, 8);
+  const current = afterRace.find(entry => entry.id === base.id);
+  assert.ok(current);
+
+  const chained = {
+    ...current,
+    id: "learner-chained-candidate",
+    fingerprint: "learner-chained-fingerprint",
+    rule: "A current stamp may rewrite the stable identity.",
+  };
+  await learnings.persistConsolidatedLearning(project, chained, {
+    supersede: [{ id: current.id, fingerprint: current.fingerprint }],
+  });
+  const beforeCollision = await readFile(ledgerPath, "utf8");
+  await assert.rejects(
+    learnings.persistLearningEntry(project, {
+      ...chained,
+      id: "learner-duplicate-content",
+    }),
+    /duplicate learning fingerprint/i
+  );
+  assert.equal(await readFile(ledgerPath, "utf8"), beforeCollision);
+
+  const ancestor = normalized.entries[0];
+  const ours = { ...ancestor, fingerprint: "learner-ours-fingerprint" };
+  const theirs = { ...ancestor, fingerprint: "learner-theirs-fingerprint" };
+  const merged = learnings.mergeLearningsDocuments(
+    legacy,
+    learnings.renderLearningsFile([ours]),
+    learnings.renderLearningsFile([theirs])
+  );
+  assert.equal(merged.kind, "merged");
+  const mergedEntries = learnings.parseLearningsFile(merged.content);
+  assert.equal(mergedEntries.length, 2);
+  assert.equal(new Set(mergedEntries.map(entry => entry.fingerprint)).size, 2);
+  assert.equal(learnings.projectLearnings(mergedEntries).omittedCount, 0);
+
+  const overflowProject = path.join(temporary, "overflow-consumer");
+  await mkdir(overflowProject);
+  const overflowEntry = index => ({
+    id: `packed-overflow-${index}`,
+    fingerprint: `packed-overflow-${index}`,
+    rule: `Packed overflow entry ${index}.`,
+    why: "Compact packed proof.",
+    provenance: [`issue:#${index}`],
+    first_learned: packedDate,
+    last_confirmed: packedDate,
+    confidence: "high",
+  });
+  for (
+    let index = 0;
+    index < learnings.LEARNINGS_CONTRACT.maxEntries;
+    index += 1
+  ) {
+    await learnings.persistLearningEntry(overflowProject, overflowEntry(index));
+  }
+  const firstDrop = overflowEntry(20);
+  await assert.rejects(
+    learnings.persistLearningEntry(overflowProject, firstDrop),
+    /was preserved/
+  );
+  const sameIdFork = { ...overflowEntry(21), id: firstDrop.id };
+  await assert.rejects(
+    learnings.persistLearningEntry(overflowProject, sameIdFork),
+    /was preserved/
+  );
+  const overflow = await learnings.readLearningsOverflow(overflowProject);
+  assert.deepEqual(
+    overflow.entries.map(entry => [entry.id, entry.fingerprint]),
+    [
+      [firstDrop.id, firstDrop.fingerprint],
+      [sameIdFork.fingerprint, sameIdFork.fingerprint],
+    ]
+  );
+  const beforeOverflowCollision = await readFile(overflow.file, "utf8");
+  await assert.rejects(
+    learnings.persistLearningEntry(overflowProject, {
+      ...overflowEntry(22),
+      fingerprint: firstDrop.fingerprint,
+    }),
+    /duplicate learning fingerprint/i
+  );
+  assert.equal(await readFile(overflow.file, "utf8"), beforeOverflowCollision);
+
+  console.log(
+    "[EVIDENCE: packed-learnings-contract] v1=migratable legacy=bounded entries=hardened stamps=hardened race=9 stale=8 chain=stable collision=atomic overflow=fork merge=2 projection=bounded"
+  );
+} finally {
+  await rm(temporary, { recursive: true, force: true });
+}