@yemi33/minions 0.1.2291 → 0.1.2293

package/docs/README.md

@@ -58,6 +58,7 @@ Architecture, design proposals, and lifecycle references for people working on t
 
 Operational runbooks for engine operators and fleet maintainers.
 
+- [notes.md](notes.md) — Consolidated team knowledge: patterns, conventions, bugs, and build findings accumulated from agent inbox notes. Read by the engine and injected into agent prompts as shared context.
 - [auto-discovery.md](auto-discovery.md) — Auto-discovery and execution pipeline: the per-tick orchestration loop and the four work-discovery sources.
 - [diagnostics-memory.md](diagnostics-memory.md) — Operator runbook for the in-process memory + perf observability surface: `/api/diagnostics/memory[/history]`, `/api/diagnostics/heap-snapshot` guard-token capture, `MEMORY_BASELINE` log emissions, `--cpu-prof`/`--heap-prof` capture, and the `test/perf/soak.test.js` heap-growth regression gate.
 - [engine-restart.md](engine-restart.md) — How agents survive an engine restart: state persistence, the 20-minute startup grace period, and orphan reattachment via PID files and `live-output.log`.

package/docs/command-center.md

@@ -63,7 +63,7 @@ Any violation rejects the **whole turn** with a typed error envelope (`code: 'in
 | Runtime | `imageInput` | Behavior |
 |---------|--------------|----------|
 | claude  | `true`  | Images delivered as an Anthropic Messages `content[]` envelope of base64 blocks over stream-json (`claude.buildPrompt` + `--input-format stream-json` in `claude.buildArgs`). |
-| copilot | `false` | Degraded — `engine/llm.js` `_resolveImageOpts` returns a typed `model-unavailable` envelope ("switch ccCli to a runtime with imageInput"). The Copilot worker-pool path also drops images. |
+| copilot | `true` | Images materialized to tmp files and passed as `--attachment <path>` (W-mqv7324u0021db5d). **Exception:** when the opt-in Copilot worker pool is active (`engine.ccUseWorkerPool: true`), images are silently dropped — the pool's ACP `stream()` does not forward images; a note in `_invokeCcStream` marks this intentional. |
 | codex   | `false` | Degraded — same typed `model-unavailable` envelope. |
 
 `_resolveImageOpts` (in `engine/llm.js`) is pure and unit-tested: it forwards the `images` list only when `runtime.capabilities.imageInput` is truthy, otherwise returns the typed error so CC/doc-chat surface the envelope instead of a silently-text-only reply. `_spawnProcess` re-applies the same gate (`if (!caps.imageInput) adapterOpts.images = undefined`) as defense in depth. Image **filenames** are run through the untrusted-input fence (`buildSource('cc-image-filename', …)`) before they reach prompt text, since they are user-supplied.

package/docs/plan-lifecycle.md

@@ -118,7 +118,7 @@ The verify agent does not call `archivePlan()`. After the testing guide and any
 
 Manual archive behavior:
 
-1. For PRD JSON files, the dashboard moves `prd/<file>.json` to `prd/archive/<file>.json`, marks the archived JSON with `status: "archived"` and `archivedAt`, and removes or neutralizes the active `.backup` sidecar so `safeJson()` cannot restore a stale completed PRD on restart.
+1. For PRD JSON files, the dashboard archives them **in-place** in `prd/` — the file is not moved. The JSON is stamped with `archived: true`, `status: "archived"`, and `archivedAt` (Phase 10 step 4.2b). The `.backup` sidecar is left intact since the file keeps its canonical path.
 2. If the PRD has `source_plan`, the matching `plans/<source>.md` is moved to `plans/archive/<source>.md`.
 3. Pending or queued work items linked to the archived PRD are cancelled with `_cancelledBy: "plan-archived"`; completed work items remain as history.
 4. Plan worktrees are cleaned up by the archive handler/lifecycle cleanup path.
@@ -148,7 +148,9 @@ After PRs are created, humans can leave comments containing `@minions` to trigge
 |------|---------|
 | `plans/*.md` | Source plans awaiting review or linked to active PRDs |
 | `prd/*.json` | Active PRDs with materializable items |
-| `plans/archive/`, `prd/archive/` | Manually archived plans and PRDs |
+| `plans/archive/` | Manually archived plan Markdown files |
+| `prd/*.json` (with `archived: true`) | PRDs archived in-place (Phase 10 step 4.2b — not moved; flag + `archivedAt` stamped on the file) |
+| `prd/archive/` | Legacy: PRDs archived before Phase 10 (still scanned for filename-collision detection) |
 | `playbooks/verify.md` | Verification task playbook |
 | `playbooks/implement.md` | Implementation playbook |
 | `playbooks/plan-to-prd.md` | Plan → PRD conversion playbook |

package/docs/runtime-adapters.md

@@ -81,6 +81,7 @@ real behavioral split appears between adapters.
 | `resumePromptCarryover` | ✗ | ✓ | ✓ | CC resume turns prepend recent visible Q&A in stdin when the runtime session store is opaque to Minions. |
 | `resumeBookkeepingTurn` | ✓ | — | — | Claude CLI injects a synthetic "Continue from where you left off." meta turn on `--resume`; CC prompts must tell the model not to treat it as user intent. |
 | `streamConsumer` | ✓ | ✓ | ✓ | Adapter implements `createStreamConsumer(ctx)` — required by `engine/llm.js` accumulator. |
+| `imageInput` | ✓ | ✓ | ✗ | Runtime accepts `images: [{mimeType, dataBase64}]` and delivers them to the CLI. Claude: Anthropic Messages base64 content-block envelope via `--input-format stream-json`. Copilot: base64 payloads materialized to tmp files and passed as `--attachment <path>` (W-mqv7324u0021db5d). When false, `_resolveImageOpts` returns a typed `model-unavailable` error. |
 
 When a behavior is genuinely uniform across all current adapters, it still gets
 a flag if a future runtime might disagree — flags are cheap.

package/engine/cli.js

@@ -624,6 +624,19 @@ const commands = {
       }
     } catch (err) { e.log('warn', `note-link backfill failed: ${err.message}`); }
 
+    // Backfill work_items.prd_item_id (the WI↔PRD-item FK) for rows that predate
+    // the dual-write stamp (#546) or that a JSON re-hydrate transiently zeroed.
+    // Set-based, only touches NULL FKs, no-op once everything is linked — safe on
+    // every boot. Lets the Phase-10 render join move off the feature-id string.
+    try {
+      const fk = require('./prd-store').backfillWorkItemPrdItemIds();
+      const filled = (fk.byWorkItemId || 0) + (fk.bySourcePlan || 0);
+      if (fk.ok && filled > 0) {
+        e.log('info', `Backfilled prd_item_id on ${filled} work item(s) (${fk.byWorkItemId} by workItemId, ${fk.bySourcePlan} by sourcePlan)`);
+        console.log(`  Linked ${filled} work item(s) to their PRD item.`);
+      }
+    } catch (err) { e.log('warn', `prd_item_id backfill failed: ${err.message}`); }
+
     // Auto-heal projects missing workSources (cloned-repo / hand-rolled-config
     // footgun): without this block, discoverFromWorkItems / discoverFromPrs
     // bail silently and the engine looks healthy but never dispatches. The

package/engine/features.js

@@ -92,6 +92,21 @@ const FEATURES = {
     addedIn: '0.1.2090',
     expires: '2026-12-01',
   },
+  // prdJoinFromFk — Phase 10 step 4.3 render-join flip. When ON, getPrdInfo
+  // resolves each PRD item to its work item via the stable SQL FK
+  // (work_items.prd_item_id → prd_items.id) instead of the fragile feature-id
+  // string match (work_item.id === feature.id), which bleeds when a live and an
+  // archived PRD share a feature id (footgun #7). PURELY ADDITIVE: the FK lookup
+  // is tried first and falls back to the exact legacy id match for any row not
+  // yet stamped, when the flag is OFF, or when SQL is unavailable — so flipping
+  // it off (or a NULL FK) reproduces current behavior byte-for-byte. Requires
+  // prdReadsFromSql (the FK only exists in SQL). Temporary migration gate.
+  'prdJoinFromFk': {
+    description: 'Resolve the dashboard WI↔PRD-item join via the SQL FK (work_items.prd_item_id) instead of the feature-id string match. Additive with a string-match fallback; reversible (Phase 10 step 4.3).',
+    default: true,
+    addedIn: '0.1.2232',
+    expires: '2026-12-01',
+  },
 };
 
 const ENV_TRUTHY = new Set(['1', 'true', 'on', 'yes']);

package/engine/lifecycle.js

@@ -2496,7 +2496,7 @@ function fixCompletionChangedBranch(structuredCompletion) {
 }
 
 function normalizePrFixBranchName(branch) {
-  return String(branch || '').trim().replace(/^refs\/heads\//, '');
+  return String(branch || '').trim().replace(/^refs\/heads\//i, '');
 }
 
 function getPrFixBaselineHead(pr) {
@@ -6300,6 +6300,8 @@ module.exports = {
   enrollPrFromCanonicalId,
   _setEnrollmentGhRunnerForTest,
   // W-mqtplpk6001oe6d5 — stamp PR ref + workItemId onto WI and PRD item at completion time.
+  // W-mqvwzgi500259361 — exported for unit testing case-insensitive refs/heads/ stripping.
+  normalizePrFixBranchName,
   stampWiPrRef,
   stampPrdItemWorkItemId,
   // M003 — auto-dispatch live-validation WI after coding WI completion.

package/engine/pipeline.js

@@ -57,7 +57,7 @@ function getPipeline(id) {
 
 function savePipeline(pipeline) {
   if (!fs.existsSync(PIPELINES_DIR)) fs.mkdirSync(PIPELINES_DIR, { recursive: true });
-  safeWrite(path.join(PIPELINES_DIR, pipeline.id + '.json'), pipeline);
+  mutateJsonFileLocked(path.join(PIPELINES_DIR, pipeline.id + '.json'), () => pipeline, { defaultValue: {} });
 }
 
 function deletePipeline(id) {

package/engine/prd-store.js

@@ -74,7 +74,10 @@ function _upsertPrd(db, filename, archived, prd, now) {
       sourcePlan, prd.sourcePlanModifiedAt || prd.source_plan_modified_at || null,
       JSON.stringify(prd), now, prdId,
     );
-    db.prepare('DELETE FROM prd_items WHERE prd_id=?').run(prdId);
+    // NOTE: prd_items are NOT bulk-deleted here — they are upserted by their
+    // stable (prd_id, feature_id) key below so prd_items.id survives a re-mirror
+    // (work_items.prd_item_id is an FK to it). prd_verify_prs carries no FK, so
+    // a cheap delete+reinsert is fine.
     db.prepare('DELETE FROM prd_verify_prs WHERE prd_id=?').run(prdId);
   } else {
     const info = db.prepare(`
@@ -91,18 +94,73 @@ function _upsertPrd(db, filename, archived, prd, now) {
     prdId = Number(info.lastInsertRowid);
   }
 
-  const insItem = db.prepare(`
+  // Upsert prd_items by their STABLE natural key (prd_id, feature_id) instead of
+  // delete-all-then-reinsert, so the surrogate prd_items.id is durable across
+  // re-mirrors. work_items.prd_item_id is an FK to this id; churning it (via
+  // AUTOINCREMENT on every mirror) dangled every stamped row — the Phase-10
+  // step-4 join-flip blocker. feature_id-less legacy items can't be keyed (NULL
+  // != NULL under UNIQUE), so they are swept + reinserted each time; they are
+  // never FK targets, so their churn is harmless.
+  const features = Array.isArray(prd.missing_features) ? prd.missing_features : [];
+  const keepIds = [];
+  for (const ft of features) { if (ft && typeof ft === 'object' && ft.id) keepIds.push(String(ft.id)); }
+  // Drop id-bearing items no longer on the PRD, plus all NULL-feature_id rows.
+  if (keepIds.length) {
+    db.prepare(
+      `DELETE FROM prd_items WHERE prd_id=? AND feature_id IS NOT NULL AND feature_id NOT IN (${keepIds.map(() => '?').join(',')})`
+    ).run(prdId, ...keepIds);
+  } else {
+    db.prepare('DELETE FROM prd_items WHERE prd_id=? AND feature_id IS NOT NULL').run(prdId);
+  }
+  db.prepare('DELETE FROM prd_items WHERE prd_id=? AND feature_id IS NULL').run(prdId);
+
+  const upsertItem = db.prepare(`
     INSERT INTO prd_items (prd_id, feature_id, name, project, status, type, work_item_id, data, updated_at)
     VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
+    ON CONFLICT(prd_id, feature_id) DO UPDATE SET
+      name=excluded.name, project=excluded.project, status=excluded.status,
+      type=excluded.type, work_item_id=excluded.work_item_id,
+      data=excluded.data, updated_at=excluded.updated_at
   `);
-  const features = Array.isArray(prd.missing_features) ? prd.missing_features : [];
+  const insItemNoId = db.prepare(`
+    INSERT INTO prd_items (prd_id, feature_id, name, project, status, type, work_item_id, data, updated_at)
+    VALUES (?, NULL, ?, ?, ?, ?, ?, ?, ?)
+  `);
+  const selItemId = db.prepare('SELECT id FROM prd_items WHERE prd_id=? AND feature_id=?');
+  // Stamp the materialized work item(s) → prd_items.id. The two backref paths
+  // mirror the migration-015 backfill: an explicit feature.workItemId (set by
+  // lifecycle.stampPrdItemWorkItemId), and the sourcePlan(=PRD filename) +
+  // feature-id equality the legacy JSON model joined on. Self-heals on every
+  // mirror, so a WI materialized after this PRD was first seen still gets its FK.
+  const stampWiById = db.prepare(
+    'UPDATE work_items SET prd_item_id=? WHERE id=? AND (prd_item_id IS NULL OR prd_item_id<>?)'
+  );
+  const stampWiBySourcePlan = db.prepare(
+    `UPDATE work_items SET prd_item_id=? WHERE json_extract(data,'$.sourcePlan')=? AND json_extract(data,'$.id')=? AND (prd_item_id IS NULL OR prd_item_id<>?)`
+  );
   for (const ft of features) {
     if (!ft || typeof ft !== 'object') continue;
+    const proj = ft.project || prd.project || null;
+    const explicitWi = ft.workItemId || ft.work_item_id || null;
+    if (!ft.id) {
+      try {
+        insItemNoId.run(prdId, ft.name || null, proj, ft.status || null, ft.type || null,
+          explicitWi, JSON.stringify(ft), now);
+      } catch { /* best-effort */ }
+      continue;
+    }
+    const featureId = String(ft.id);
     try {
-      insItem.run(prdId, ft.id || null, ft.name || null, ft.project || prd.project || null,
-        ft.status || null, ft.type || null, ft.workItemId || ft.work_item_id || null,
-        JSON.stringify(ft), now);
-    } catch { /* dup (prd_id, feature_id) → skip */ }
+      upsertItem.run(prdId, featureId, ft.name || null, proj, ft.status || null, ft.type || null,
+        explicitWi, JSON.stringify(ft), now);
+    } catch { continue; }
+    let prdItemId;
+    try { prdItemId = selItemId.get(prdId, featureId)?.id; } catch { /* ignore */ }
+    if (!prdItemId) continue;
+    if (explicitWi) {
+      try { stampWiById.run(prdItemId, String(explicitWi), prdItemId); } catch { /* best-effort */ }
+    }
+    try { stampWiBySourcePlan.run(prdItemId, filename, featureId, prdItemId); } catch { /* best-effort */ }
   }
 
   const insVp = db.prepare(`
@@ -235,6 +293,53 @@ function reconcilePrdsFromDisk() {
   return { ok: true, mirrored, removed };
 }
 
+// Idempotent boot sweep: populate work_items.prd_item_id for rows that predate
+// the dual-write stamp (everything materialized before #546), or that a JSON
+// re-hydrate transiently zeroed. The per-mirror stamp in _upsertPrd only fires
+// for PRDs that are re-mirrored, and reconcile is mtime-gated — so a PRD that
+// never changes again never re-stamps its historical WIs. This catch-up is
+// set-based (two SQL UPDATEs, no per-row loop), only touches NULL FKs, and
+// prefers the live (archived=0) prd_item when a basename exists in both buckets.
+// Safe to run on every boot — a no-op once every matchable WI is linked.
+// Returns { ok, byWorkItemId, bySourcePlan } counts.
+function backfillWorkItemPrdItemIds() {
+  let db;
+  try { db = require('./db').getDb(); } catch { return { ok: false }; }
+  try {
+    return require('./db').withTransaction(db, () => {
+      // Path 1 — explicit prd_items.work_item_id backref (most precise).
+      const r1 = db.prepare(`
+        UPDATE work_items SET prd_item_id = (
+          SELECT pi.id FROM prd_items pi JOIN prds p ON p.id = pi.prd_id
+          WHERE pi.work_item_id = work_items.id
+          ORDER BY p.archived ASC, pi.id ASC LIMIT 1
+        )
+        WHERE prd_item_id IS NULL
+          AND EXISTS (SELECT 1 FROM prd_items pi WHERE pi.work_item_id = work_items.id)
+      `).run();
+      // Path 2 — sourcePlan(=PRD filename) + feature-id equality on the WI blob,
+      // for WIs the PRD side never stamped with an explicit workItemId.
+      const r2 = db.prepare(`
+        UPDATE work_items SET prd_item_id = (
+          SELECT pi.id FROM prd_items pi JOIN prds p ON p.id = pi.prd_id
+          WHERE p.filename = json_extract(work_items.data, '$.sourcePlan')
+            AND pi.feature_id = json_extract(work_items.data, '$.id')
+          ORDER BY p.archived ASC, pi.id ASC LIMIT 1
+        )
+        WHERE prd_item_id IS NULL
+          AND EXISTS (
+            SELECT 1 FROM prd_items pi JOIN prds p ON p.id = pi.prd_id
+            WHERE p.filename = json_extract(work_items.data, '$.sourcePlan')
+              AND pi.feature_id = json_extract(work_items.data, '$.id')
+          )
+      `).run();
+      return { ok: true, byWorkItemId: Number(r1.changes) || 0, bySourcePlan: Number(r2.changes) || 0 };
+    });
+  } catch (e) {
+    return { ok: false, reason: e && e.message };
+  }
+}
+
 // List every PRD row as { filename, archived, plan } where plan is the parsed
 // data blob. Sorted by (archived, filename) for deterministic consumer order
 // (matches the disk-scan order once the caller also sorts). Read-flip source.
@@ -258,6 +363,7 @@ module.exports = {
   removePrdFromSql,
   readPrdFromSql,
   reconcilePrdsFromDisk,
+  backfillWorkItemPrdItemIds,
   listPrdRows,
   _reconcileMtimes, // exported for tests (reset between cases)
   _upsertPrd, // exported for tests

package/engine/queries.js

@@ -2151,6 +2151,16 @@ function getPrdInfo(config) {
     { dir: path.join(PRD_DIR, 'archive'), archived: true },
   ];
 
+  // Phase 10 step 4.3 — FK-based WI↔PRD-item join scaffolding. `prdItemIdByKey`
+  // maps `${physicalArchived} ${filename} ${feature_id}` → prd_items.id so each
+  // PRD item carries its stable surrogate id; built below once the SQL-read
+  // decision is known. Declared here so processPrd's closure can stamp
+  // `_prdItemId` at push time. Null when the FK join is off / SQL unavailable →
+  // the per-item join falls back to the legacy feature-id string match.
+  let useFkJoin = false;
+  try { useFkJoin = require('./features').isFeatureOn('prdJoinFromFk', config); } catch { /* fall back to string join */ }
+  let prdItemIdByKey = null;
+
   // Phase 10 step 3 (read-flip) — both the disk scan and the SQL-mirror source
   // funnel each PRD through this one processor, so the consumed shape
   // (existingPrds / verifyPrsByPlan / allPrdItems) is identical by construction.
@@ -2187,6 +2197,11 @@ function getPrdInfo(config) {
       }));
     }
     for (const f of plan.missing_features) {
+      // Stamp the stable surrogate prd_items.id, keyed on the PHYSICAL archived
+      // bit (the `archived` param == SQL location), not the logical `isArch`.
+      const prdItemId = (prdItemIdByKey && f && f.id != null)
+        ? (prdItemIdByKey.get(`${archived ? 1 : 0} ${pf} ${f.id}`) ?? null)
+        : null;
       allPrdItems.push({
         ...f, _source: pf, _planStatus: plan.status || 'active',
         _planSummary: plan.plan_summary || pf, _planProject: plan.project || '',
@@ -2195,6 +2210,7 @@ function getPrdInfo(config) {
         _planStale: planStale || plan.planStale || false, _lastSyncedFromPlan: plan.lastSyncedFromPlan || null,
         _prdUpdatedAt: new Date(mtimeMs).toISOString(),
         _prdCompletedAt: plan.completedAt || '',
+        _prdItemId: prdItemId,
       });
     }
   };
@@ -2202,6 +2218,21 @@ function getPrdInfo(config) {
   let useSqlReads = false;
   try { useSqlReads = require('./features').isFeatureOn('prdReadsFromSql', config); } catch { /* fall back to disk */ }
 
+  // Build the feature → prd_items.id map before the read loop so processPrd can
+  // stamp _prdItemId. Only when the FK join is on AND we're SQL-sourced (the FK
+  // lives only in SQL). Best-effort: any failure leaves prdItemIdByKey null and
+  // the join silently falls back to the legacy feature-id string match.
+  if (useFkJoin && useSqlReads) {
+    try {
+      const db = require('./db').getDb();
+      const map = new Map();
+      for (const r of db.prepare('SELECT p.filename AS filename, p.archived AS archived, pi.feature_id AS feature_id, pi.id AS id FROM prd_items pi JOIN prds p ON p.id = pi.prd_id WHERE pi.feature_id IS NOT NULL').all()) {
+        map.set(`${r.archived ? 1 : 0} ${r.filename} ${r.feature_id}`, r.id);
+      }
+      prdItemIdByKey = map;
+    } catch { prdItemIdByKey = null; }
+  }
+
   if (useSqlReads) {
     // SQL source: reconcile the mirror from disk (catches PRDs written outside
     // the dual-write chokepoint — plan-to-prd agent direct writes, archive
@@ -2293,6 +2324,29 @@ function getPrdInfo(config) {
     }
   } catch { /* optional */ }
 
+  // Phase 10 step 4.3 — index the (JSON-sourced) work items by their stable SQL
+  // FK so the per-item join can resolve PRD item → WI by prd_items.id rather
+  // than the feature-id string. The FK itself lives only in SQL (it's not in the
+  // JSON mirror), so read work_items.prd_item_id once and join it to the WI
+  // objects we already loaded, keyed by wi.id (first-wins, matching allWiById).
+  // Best-effort: null map → the join falls back to the legacy string match.
+  let wiByPrdItemId = null;
+  if (useFkJoin && useSqlReads && prdItemIdByKey) {
+    try {
+      const db = require('./db').getDb();
+      const fkByWiId = new Map();
+      for (const r of db.prepare('SELECT id, prd_item_id FROM work_items WHERE prd_item_id IS NOT NULL').all()) {
+        if (!fkByWiId.has(r.id)) fkByWiId.set(r.id, r.prd_item_id);
+      }
+      const map = {};
+      for (const wi of Object.values(allWiById)) {
+        const fk = fkByWiId.get(wi.id);
+        if (fk != null && map[fk] === undefined) map[fk] = wi;
+      }
+      wiByPrdItemId = map;
+    } catch { wiByPrdItemId = null; }
+  }
+
   // PR-to-PRD linking — derived from PR.prdItems (single source of truth).
   // getPullRequests includes records from unconfigured project subdirs so PRD
   // links can resolve to last-known status even after a project is removed.
@@ -2388,7 +2442,14 @@ function getPrdInfo(config) {
   // Augment each item with execution metadata from the work item.
   const statusDisplay = { pending: 'missing', dispatched: 'in-progress' };
   for (const item of items) {
-    const wi = wiById[item.id];
+    // Phase 10 step 4.3 — resolve the work item via the stable SQL FK
+    // (item._prdItemId → prd_items.id → work_items.prd_item_id) first; fall back
+    // to the legacy feature-id string match (wiById[item.id]) for unstamped rows,
+    // a NULL FK, the flag OFF, or SQL unavailable. The fallback reproduces the
+    // pre-flip behavior exactly, so a live+archived feature-id collision (footgun
+    // #7) is corrected when the FK is present and degrades to legacy otherwise.
+    const wi = (wiByPrdItemId && item._prdItemId != null && wiByPrdItemId[item._prdItemId])
+      || wiById[item.id];
     // PRD 'updated'/'missing' = intentional rework signal — takes priority over a done work item (#930).
     // Otherwise work item status is source of truth when available (PRD JSON may lag behind).
     // If PRD says dispatched/failed but no work item exists, treat as pending (orphaned — #779)

package/engine/shared.js

@@ -1005,14 +1005,18 @@ function _routeJsonReadToSql(p) {
  * stale `.backup` is actively harmful. See its JSDoc for selection guidance.
  */
 
-// PL-prd-no-backup (W-mqub65ez0004b3bd) — archived PRDs under prd/archive/ are
-// permanently removed terminal artifacts. They must NOT auto-restore from a
-// stale `.backup` sidecar (W-mouptdh1000h9f39: archived PRD came back and
-// re-dispatched work). Intentionally NARROW to prd/archive/ only:
-//   - prd/archive/*.json → no backup write + no safeJson restore
-//   - prd/*.json (root-level canonical)  → normal backup/restore lifecycle
-//     preserved so concurrent-sweep loss is recoverable (prd-rename-race.test.js).
-const _NO_BACKUP_JSON_RE = /(?:^|[\\/])prd[\\/]archive[\\/][^\\/]+\.json$/i;
+// PL-prd-no-backup (W-mqub65ez0004b3bd) — PRD `.json` files must NOT auto-restore
+// from a stale `.backup` sidecar: a removed/archived PRD coming back and
+// re-dispatching work is the original resurrection footgun (W-mouptdh1000h9f39).
+// Phase 10 step 4.3 — BROADENED from prd/archive/ to ALL prd/*.json (root +
+// archive). The root-level backup lifecycle previously survived ONLY to recover
+// the enforceDeclaredPlanProject rename-race (a concurrent sweep deleting a
+// just-renamed PRD); that rename is now retired and SQL is the canonical store,
+// so the sidecar is pure resurrection fuel with no remaining benefit:
+//   - prd/*.json AND prd/archive/*.json → no backup write + no safeJson restore.
+// Matches a single filename segment under prd/ (optionally one archive/ level),
+// so nested non-PRD paths and plans/ (which are .md) are unaffected.
+const _NO_BACKUP_JSON_RE = /(?:^|[\\/])prd[\\/](?:archive[\\/])?[^\\/]+\.json$/i;
 function _isNoBackupJsonPath(p) {
   return typeof p === 'string' && _NO_BACKUP_JSON_RE.test(p);
 }

package/engine/work-items-store.js

@@ -273,6 +273,18 @@ function _fileContentHash(filePath) {
 
 function _hydrateScopeFromJson(db, scope) {
   const jsonItems = _readJsonArrayFallback(scope);
+  // Preserve the SQL-only prd_item_id FK across the delete+reinsert. It is owned
+  // by the PRD dual-write (prd-store.js#_upsertPrd), NOT represented in the JSON
+  // mirror, so a naive rebuild-from-JSON silently zeroes it for the whole scope —
+  // regressing the WI↔PRD link every time the JSON diverges (the count would
+  // oscillate as the PRD mirror re-stamps and the next hydrate wipes it again).
+  // Snapshot before the DELETE, restore onto the rows that survive the rebuild.
+  const fkById = new Map();
+  try {
+    for (const r of db.prepare('SELECT id, prd_item_id FROM work_items WHERE scope = ? AND prd_item_id IS NOT NULL').all(scope)) {
+      fkById.set(r.id, r.prd_item_id);
+    }
+  } catch { /* column absent on a pre-v15 schema — nothing to preserve */ }
   // DELETE before re-insert: callers that wrote a smaller JSON file
   // (test cleanup() removing items) must end up with a smaller SQL state.
   db.prepare('DELETE FROM work_items WHERE scope = ?').run(scope);
@@ -298,6 +310,14 @@ function _hydrateScopeFromJson(db, scope) {
       now,
     );
   }
+  // Restore the FK for rows that still exist after the rebuild (a WI dropped
+  // from the JSON is gone, and its link goes with it — correct).
+  if (fkById.size) {
+    const restore = db.prepare('UPDATE work_items SET prd_item_id = ? WHERE scope = ? AND id = ?');
+    for (const [id, fk] of fkById) {
+      try { restore.run(fk, scope, id); } catch { /* best effort */ }
+    }
+  }
 }
 
 function applyWorkItemsMutation(scope, mutator) {
@@ -375,4 +395,5 @@ module.exports = {
   dropScope,
   _filePathForScope,
   _mirrorJsonFromSql,
+  _hydrateScopeFromJson, // exported for testing (prd_item_id-preservation)
 };

package/engine.js

@@ -196,9 +196,15 @@ function cleanupTempAgent(agentId) {
   try {
     const agentDir = path.join(AGENTS_DIR, agentId);
     // Keep output archive but remove temp agent directory (live-output.log etc.)
-    fs.rmSync(agentDir, { recursive: true, force: true });
+    // Use _retryFsOp so Windows EBUSY/EPERM (AV / file indexer lock) is retried.
+    shared._retryFsOp(
+      () => fs.rmSync(agentDir, { recursive: true, force: true }),
+      `cleanupTempAgent(${agentId})`
+    );
     log('info', `Temp agent ${agentId} cleaned up`);
-  } catch { /* cleanup */ }
+  } catch (err) {
+    log('warn', `cleanupTempAgent: fs.rmSync failed for ${agentId}: ${err && err.message}`);
+  }
 }
 
 // Per-tick cache of refs that failed to fetch — avoids repeating 30s ETIMEDOUT for same missing ref
@@ -974,7 +980,17 @@ async function runWorktreeAdd(rootDir, worktreePath, addArgs, gitOpts, worktreeC
           await shared.shellSafeGit(['worktree', 'remove', '--force', worktreePath], { ...gitOpts, cwd: rootDir, timeout: 30000 });
         } catch (rmErr) {
           log('warn', `runWorktreeAdd: worktree remove after partial checkout failed (${rmErr.message}) — falling back to fs-level cleanup`);
-          try { fs.rmSync(worktreePath, { recursive: true, force: true }); } catch {}
+          try {
+            // W-mqw3g5be000s934f: use _retryFsOp so transient Windows EBUSY/EPERM from
+            // antivirus or GVFS virtual-filesystem handles do not silently abandon cleanup.
+            shared._retryFsOp(
+              () => fs.rmSync(worktreePath, { recursive: true, force: true }),
+              `runWorktreeAdd: fs-level cleanup ${worktreePath}`,
+              { attempts: 3, baseMs: 100 },
+            );
+          } catch (fsErr) {
+            log('warn', `runWorktreeAdd: fs-level cleanup failed after retries (${fsErr.message}) — worktree directory may be leaked: ${worktreePath}`);
+          }
           try { await shared.shellSafeGit(['worktree', 'prune'], { ...gitOpts, cwd: rootDir, timeout: 15000 }); } catch {}
         }
         const incompleteErr = new Error(`GVFS incomplete checkout at ${worktreePath}: worktree add exited cleanly but no source files were checked out (only .git was written). Retriable.`);
@@ -2933,10 +2949,10 @@ async function spawnAgent(dispatchItem, config) {
                   try { await shared.shellSafeGit(['worktree', 'prune'], { ..._gitOpts, cwd: rootDir, timeout: 15000 }); } catch { /* optional */ }
                   removeStaleIndexLock(rootDir);
                   // Clean up partial worktree directory from the failed -b
-                  // attempt. Keep this a RAW fs.rmSync: this husk pre-dates
-                  // `git worktree add`, so git does not yet own it and
-                  // shared.removeWorktree's `git worktree remove --force` would
-                  // no-op on it (P-b3d9a162). Two guards before the delete:
+                  // attempt. This husk pre-dates `git worktree add`, so git
+                  // does not yet own it and shared.removeWorktree's
+                  // `git worktree remove --force` would no-op on it
+                  // (P-b3d9a162). Two guards before the delete:
                   //   1. isWorktreePathLive (fail-open): skip if another
                   //      dispatch raced onto this path — it returns true on a
                   //      SQL outage, so we leak the husk rather than nuke a
@@ -2945,6 +2961,7 @@ async function spawnAgent(dispatchItem, config) {
                   //      carries its own `.git` DIRECTORY (a mis-pointed
                   //      worktreePath onto a real repo) — a linked worktree's
                   //      `.git` is a FILE, so this never blocks a real husk.
+                  // W-mqw3i9vp000tcfaf: use shared._retryFsOp so Windows EBUSY/EPERM doesn't brick the branch.
                   try {
                     const _huskGit = path.join(worktreePath, '.git');
                     let _huskIsRealRepo = false;
@@ -2952,9 +2969,14 @@ async function spawnAgent(dispatchItem, config) {
                     if (fs.existsSync(worktreePath)
                         && !_huskIsRealRepo
                         && !shared.isWorktreePathLive(worktreePath, { excludeDispatchId: id })) {
-                      fs.rmSync(worktreePath, { recursive: true, force: true });
+                      shared._retryFsOp(
+                        () => fs.rmSync(worktreePath, { recursive: true, force: true }),
+                        `husk cleanup before -b retry ${branchName}`
+                      );
                     }
-                  } catch { /* optional */ }
+                  } catch (rmErr) {
+                    log('warn', `spawnAgent: husk cleanup failed before -b retry for ${branchName}: ${rmErr.message?.split('\n')[0]}`);
+                  }
                   try {
                     await runWorktreeAdd(rootDir, worktreePath, ['-b', branchName, _freshCreateBase], _worktreeGitOpts, 0);
                   } catch (e1b) {
@@ -5863,54 +5885,6 @@ function materializePlansAsWorkItems(config) {
       return mutator(current) || current;
     }, { defaultValue: fallback || {}, ...options });
   };
-  const declaredProjectPrdFilename = (fileName, projectName) => {
-    const match = String(fileName || '').match(/-(\d{4}-\d{2}-\d{2}(?:-\d+)?)\.json$/);
-    if (!match) return null;
-    return safePrdFilenameForProject(projectName, match[1]);
-  };
-  const migratePrdFilenameReferences = (oldFileName, newFileName) => {
-    if (!oldFileName || !newFileName || oldFileName === newFileName) return 0;
-    let migrated = 0;
-    const wiPaths = new Set([path.join(MINIONS_DIR, 'work-items.json')]);
-    for (const project of getProjects(config)) wiPaths.add(projectWorkItemsPath(project));
-    for (const wiPath of wiPaths) {
-      if (!fs.existsSync(wiPath)) continue;
-      mutateWorkItems(wiPath, (items) => {
-        for (const wi of items) {
-          if (!wi || typeof wi !== 'object') continue;
-          if (wi.sourcePlan === oldFileName) {
-            wi.sourcePlan = newFileName;
-            migrated++;
-          }
-          if (wi._artifacts?.sourcePlan === oldFileName) {
-            wi._artifacts.sourcePlan = newFileName;
-            migrated++;
-          }
-        }
-        return items;
-      });
-    }
-    if (fs.existsSync(DISPATCH_PATH)) {
-      mutateDispatch((dispatch) => {
-        for (const queue of ['pending', 'active', 'completed']) {
-          for (const entry of dispatch[queue] || []) {
-            const metaItem = entry?.meta?.item;
-            if (!metaItem || typeof metaItem !== 'object') continue;
-            if (metaItem.sourcePlan === oldFileName) {
-              metaItem.sourcePlan = newFileName;
-              migrated++;
-            }
-            if (metaItem._prdFilename === oldFileName) {
-              metaItem._prdFilename = newFileName;
-              migrated++;
-            }
-          }
-        }
-        return dispatch;
-      });
-    }
-    return migrated;
-  };
   const enforceDeclaredPlanProject = (fileName, currentPlan) => {
     if (!currentPlan?.source_plan) return { fileName, plan: currentPlan };
     let declaredProject = '';
@@ -5938,95 +5912,18 @@ function materializePlansAsWorkItems(config) {
       return planData;
     }, { skipWriteIfUnchanged: true });
 
-    let nextFileName = fileName;
-    const desiredFileName = declaredProjectPrdFilename(fileName, declaredProject);
-    if (desiredFileName && desiredFileName.toLowerCase() !== String(fileName).toLowerCase()) {
-      const fromPath = path.join(PRD_DIR, fileName);
-      const desiredPath = shared.sanitizePath(desiredFileName, PRD_DIR);
-      // W-mq8qdai6 — refuse to rename into a name owned by an archived PRD.
-      // `shared.uniquePath` only scans the live `prd/` directory; without this
-      // guard, an archived PRD with the canonical "<project>-<date>.json" name
-      // would silently collide with the rename target. The dashboard joins
-      // work items to PRDs by basename (sourcePlan === <prd filename>), so a
-      // live↔archive name collision bleeds the archived PRD's done items,
-      // verify task, and PRs into the live PRD's view (the bug-fix-plan vs
-      // minions-opg-2026-06-10 incident).
-      const archiveBasenames = new Set(
-        safeReadDir(path.join(PRD_DIR, 'archive'))
-          .filter(f => f.endsWith('.json'))
-          .map(f => f.toLowerCase())
-      );
-      if (archiveBasenames.has(desiredFileName.toLowerCase())) {
-        log('warn', `Plan project enforcement: skipping rename of ${fileName} to ${desiredFileName} — would collide with archived PRD`);
-      } else {
-        const toPath = shared.uniquePath(desiredPath);
-        const toBasename = path.basename(toPath);
-        if (archiveBasenames.has(toBasename.toLowerCase())) {
-          // uniquePath bumped past a live conflict (e.g. <name>-2.json) but
-          // the bumped name itself is owned by an archived PRD — same risk.
-          log('warn', `Plan project enforcement: skipping rename of ${fileName} to ${toBasename} — would collide with archived PRD`);
-        } else {
-          // W-mqfevwrm — Serialize the rename behind file locks on BOTH the
-          // source and destination paths so a concurrent `mutateJsonFileLocked`
-          // on either name cannot interleave (creating a divergent ghost PRD
-          // or losing data). Move the `.backup` sidecar along with the `.json`
-          // so the canonical name retains its restore-from-backup safety net
-          // — without this, deleting the renamed `.json` (concurrent sweep,
-          // plan-completion purge, etc.) leaves no `.backup` under the new
-          // name and the PRD is permanently lost (live incident 2026-06-15:
-          // `minions-opg-2026-06-10.json` gone, only the OLD-name `.backup`
-          // survived, all 9 WIs orphaned). The OLD-name `.backup` is removed
-          // so it cannot resurrect a ghost PRD on a future read of the OLD
-          // path (W-mouptdh1000h9f39-style landmine).
-          try {
-            withFileLock(`${fromPath}.lock`, () => {
-              withFileLock(`${toPath}.lock`, () => {
-                // Re-verify under both locks — another tick may have raced
-                // ahead and either deleted the source or claimed the dest.
-                if (!fs.existsSync(fromPath)) {
-                  log('warn', `Plan project enforcement: source ${fileName} disappeared before rename (raced)`);
-                  return;
-                }
-                if (fs.existsSync(toPath)) {
-                  log('warn', `Plan project enforcement: destination ${toBasename} appeared before rename (raced) — skipping`);
-                  return;
-                }
-                fs.renameSync(fromPath, toPath);
-                // Move the `.backup` sidecar along with the primary `.json`
-                // so safeJson's restore-from-backup path stays viable under
-                // the new canonical name. Both ops are best-effort: missing
-                // `.backup` is normal (nothing to move) and an unlink failure
-                // on the OLD-name `.backup` is logged but non-fatal.
-                const fromBackup = `${fromPath}.backup`;
-                const toBackup = `${toPath}.backup`;
-                if (fs.existsSync(fromBackup)) {
-                  try {
-                    fs.renameSync(fromBackup, toBackup);
-                  } catch (be) {
-                    // Fall back to copy + unlink if rename across the same dir
-                    // surfaces a Windows EPERM (AV/Search Indexer hold).
-                    try {
-                      fs.copyFileSync(fromBackup, toBackup);
-                      try { fs.unlinkSync(fromBackup); } catch { /* best-effort */ }
-                    } catch (ce) {
-                      log('warn', `Plan project enforcement: could not move .backup sidecar ${fileName}.backup → ${toBasename}.backup: ${be.message} / ${ce.message}`);
-                    }
-                  }
-                }
-                nextFileName = toBasename;
-                const migrated = migratePrdFilenameReferences(fileName, nextFileName);
-                if (migrated > 0) log('info', `Plan project enforcement: migrated ${migrated} PRD reference(s) from ${fileName} to ${nextFileName}`);
-                changed = true;
-              });
-            });
-          } catch (e) {
-            log('warn', `Plan project enforcement: could not rename ${fileName} to ${toBasename}: ${e.message}`);
-          }
-        }
-      }
-    }
-    if (changed) log('info', `Plan project enforcement: preserved declared project "${declaredProject}" for ${nextFileName}`);
-    return { fileName: nextFileName, plan: normalizedPlan };
+    // Phase 10 step 4.3 — the PRD filename is NO LONGER renamed to the canonical
+    // "<project>-<date>.json". That rename existed only to keep the basename
+    // identity (sourcePlan === <prd filename>) and the live↔archive collision
+    // guard (footgun #7) working. With the surrogate-key model — archive is an
+    // in-place flag (no move) and the WI↔PRD-item join is the stable SQL FK
+    // (work_items.prd_item_id), not the basename — the rename is dead weight and
+    // its own machinery (the `.backup` sidecar move, migratePrdFilenameReferences,
+    // the rename-race window that prd-rename-race.test.js guarded) is retired.
+    // The SEMANTIC enforcement above (declared project on the PRD + its features)
+    // still runs; only the file-move is gone.
+    if (changed) log('info', `Plan project enforcement: preserved declared project "${declaredProject}" for ${fileName}`);
+    return { fileName, plan: normalizedPlan };
   };
 
   // Enforce: PRDs must be .json — auto-rename .md files that contain valid PRD JSON

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@yemi33/minions",
-  "version": "0.1.2291",
+  "version": "0.1.2293",
   "description": "Multi-agent AI dev team that runs from ~/.minions/ — five autonomous agents share a single engine, dashboard, and knowledge base",
   "bin": {
     "minions": "bin/minions.js"