cleargate 0.10.0 → 0.11.1

package/dist/templates/cleargate-planning/.cleargate/knowledge/readiness-gates.md

@@ -6,7 +6,7 @@ This file is the single source of truth for ClearGate's machine-checkable readin
 
 ## Predicate Vocabulary
 
-There are exactly **6 predicate shapes**. No other shapes are recognized; a check string that does not match one of these forms throws a parse error at evaluation time.
+There are exactly **7 predicate shapes**. No other shapes are recognized; a check string that does not match one of these forms throws a parse error at evaluation time.
 
 **1. `frontmatter(<ref>).<field> <op> <value>`**
 Reads a frontmatter field from a document. `<ref>` is either `.` (the document being evaluated) or a frontmatter key whose value is a relative path to another document (e.g. `context_source`). `<op>` is one of `==`, `!=`, `>=`, `<=`. `<value>` is a literal string, number, or boolean. Example: `frontmatter(context_source).approved == true` reads the file named by the evaluated document's `context_source` key and asserts its `approved` field equals `true`.
@@ -15,7 +15,13 @@ Reads a frontmatter field from a document. `<ref>` is either `.` (the document b
 Performs a case-sensitive substring search on the document body (everything after the frontmatter block). The negated form `body does not contain` passes when the string is absent. Example: `body does not contain 'TBD'` fails if the literal string `TBD` appears anywhere in the body.
 
 **3. `section(<N>) has <count> <item-type>`**
-Splits the document body on `## ` heading boundaries (1-indexed) and counts items of a given type within section N. `<count>` is an expression like `≥1`, `≥3`, or `0` (exact zero). `<item-type>` is one of `checked-checkbox` (lines matching `- [x]`), `unchecked-checkbox` (lines matching `- [ ]`), or `listed-item` (lines matching `- ` regardless of checkbox state). Example: `section(2) has ≥1 checked-checkbox` asserts that the second `##` section contains at least one checked markdown checkbox.
+Splits the document body on `## ` heading boundaries (1-indexed) and counts items of a given type within section N. `<count>` is an expression like `≥1`, `≥3`, or `0` (exact zero). `<item-type>` is one of:
+- `checked-checkbox` — lines matching `- [x]`
+- `unchecked-checkbox` — lines matching `- [ ]`
+- `listed-item` — lines matching `- ` regardless of checkbox state (bullet-precise; use when checkbox/task-list semantics are required, e.g. DoD)
+- `declared-item` — any line that declares a structured item: bullet lines (`- ...`), table data rows (`| ... |` lines following a `|---|`-style separator within the section), or definition-list terms (lines matching `**Item:**`, `Item:`, `*Item*:` etc.). Use `declared-item` when the gate cares only that the author declared at least N entries in section N, regardless of presentation format (table vs bullet vs def-list).
+
+Example: `section(2) has ≥1 checked-checkbox` asserts that the second `##` section contains at least one checked markdown checkbox. Example: `section(3) has ≥1 declared-item` passes when §3 contains at least one bullet, table data row, or definition-list term.
 
 **4. `file-exists(<path>)`**
 Asserts that a file exists on disk at the given path, resolved relative to the project root. Example: `file-exists(.cleargate/knowledge/cleargate-protocol.md)` passes when that file is present in the working tree.
@@ -26,6 +32,9 @@ Reads `.cleargate/wiki/index.md` and asserts that the wiki index contains a refe
 **6. `status-of(<[[ID]]>) == <value>`**
 Resolves the given ID via the wiki index, reads that page's compiled frontmatter `status:` field, and compares it to `<value>`. Status values in the live corpus are textual strings (`Draft`, `Ready`, `Active`, `Done`) — not emoji. Example: `status-of([[EPIC-008]]) == Active` passes when EPIC-008's wiki page has `status: Active`. Note: this predicate returns `unknown` (evaluates to fail) when the wiki index is stale and the item is not yet compiled. Run `cleargate wiki build` before relying on `status-of` predicates.
 
+**7. `existing-surfaces-verified`**
+Closed-set predicate (no parameters). Locates the `## Existing Surfaces` section in the document body, extracts path-shaped substrings via regex, asserts each cited path exists on disk relative to the project root. Passes when section is absent (defers to `reuse-audit-recorded`) OR all cited paths exist OR section contains a "no overlap found" / "no existing surface" / "no prior implementation" / "audit returned empty" sentinel. Sandbox-rejected paths (escaping project root) are treated as missing. Example: `existing-surfaces-verified` against an Epic body whose `## Existing Surfaces` cites `cleargate-cli/src/lib/work-item-type.ts:detectWorkItemTypeFromFm` passes when that path exists.
+
 ---
 
 ## Severity Model
@@ -60,20 +69,26 @@ The asymmetry exists because Proposal documents are human-authored strategy arti
   transition: ready-for-decomposition
   severity: enforcing
   criteria:
-    - id: proposal-approved
+    - id: parent-approved-proposal
       check: "frontmatter(context_source).approved == true"
+      or_group: parent-approved
+    - id: parent-approved-initiative
+      check: "frontmatter(context_source).status == 'Triaged'"
+      or_group: parent-approved
     - id: no-tbds
       check: "body does not contain marker 'TBD'"
     - id: scope-in-populated
-      check: "section(2) has ≥1 listed-item"
+      check: "section(3) has ≥1 declared-item"
     - id: affected-files-declared
-      check: "section(4) has ≥1 listed-item"
+      check: "section(5) has ≥1 declared-item"
     - id: interrogation-resolved
       check: "body does not contain 'Unresolved'"
     - id: discovery-checked
       check: "frontmatter(.).context_source != null"
     - id: reuse-audit-recorded
       check: "body contains '## Existing Surfaces'"
+    - id: existing-surfaces-verified
+      check: "existing-surfaces-verified"
     - id: simplest-form-justified
       check: "body contains '## Why not simpler?'"
 ```
@@ -107,7 +122,7 @@ The asymmetry exists because Proposal documents are human-authored strategy arti
     - id: no-tbds
       check: "body does not contain marker 'TBD'"
     - id: implementation-files-declared
-      check: "section(3) has ≥1 listed-item"
+      check: "section(3) has ≥1 declared-item"
     - id: dod-declared
       check: "section(4) has ≥1 listed-item"
     - id: gherkin-present
@@ -116,6 +131,8 @@ The asymmetry exists because Proposal documents are human-authored strategy arti
       check: "frontmatter(.).context_source != null"
     - id: reuse-audit-recorded
       check: "body contains '## Existing Surfaces'"
+    - id: existing-surfaces-verified
+      check: "existing-surfaces-verified"
     - id: simplest-form-justified
       check: "body contains '## Why not simpler?'"
 ```
@@ -126,15 +143,17 @@ The asymmetry exists because Proposal documents are human-authored strategy arti
   severity: enforcing
   criteria:
     - id: blast-radius-populated
-      check: "section(2) has ≥1 listed-item"
+      check: "section(2) has ≥1 declared-item"
     - id: no-tbds
       check: "body does not contain marker 'TBD'"
     - id: sandbox-paths-declared
-      check: "section(2) has ≥1 listed-item"
+      check: "section(3) has ≥1 declared-item"
     - id: discovery-checked
       check: "frontmatter(.).context_source != null"
     - id: reuse-audit-recorded
       check: "body contains '## Existing Surfaces'"
+    - id: existing-surfaces-verified
+      check: "existing-surfaces-verified"
 ```
 
 ```yaml
@@ -143,7 +162,7 @@ The asymmetry exists because Proposal documents are human-authored strategy arti
   severity: enforcing
   criteria:
     - id: repro-steps-deterministic
-      check: "section(2) has ≥3 listed-item"
+      check: "section(2) has ≥3 declared-item"
     - id: severity-set
       check: "frontmatter(.).severity != null"
     - id: no-tbds
@@ -162,3 +181,16 @@ The asymmetry exists because Proposal documents are human-authored strategy arti
     - id: discovery-checked
       check: "frontmatter(.).context_source != null"
 ```
+
+```yaml
+- work_item_type: initiative
+  transition: ready-for-decomposition
+  severity: advisory
+  criteria:
+    - id: no-tbds
+      check: "body does not contain marker 'TBD'"
+    - id: user-flow-populated
+      check: "section(1) has ≥1 listed-item"
+    - id: success-criteria-populated
+      check: "section(5) has ≥1 listed-item"
+```

package/dist/templates/cleargate-planning/.cleargate/scripts/close_sprint.mjs

@@ -34,9 +34,10 @@
  *        atomicWrite pattern from update_state.mjs
  *
  * Test seams (CR-022 M1):
- *   CLEARGATE_SKIP_LIFECYCLE_CHECK=1  — skip Step 2.6 lifecycle reconciliation entirely
- *                                       (test environments where the CLI binary is present
- *                                       but real git history would produce drift false-positives).
+ *   CLEARGATE_SKIP_LIFECYCLE_CHECK=1  — skip Step 2.6 lifecycle reconciliation AND Step 2.6b
+ *                                       cross-sprint orphan drift check entirely (test
+ *                                       environments where the CLI binary is present but
+ *                                       real git history would produce drift false-positives).
  *   CLEARGATE_SKIP_WORKTREE_CHECK=1   — skip Step 2.7 entirely (test environments that cannot
  *                                       run git worktree list from a real git root).
  *   CLEARGATE_FORCE_WORKTREE_PATHS=p1,p2 — comma-separated fake worktree paths injected into
@@ -59,6 +60,9 @@
  *                                      --flashcard-cleanup scan in suggest_improvements.mjs.
  *   CLEARGATE_FLASHCARD_LOOKBACK=<N> — override 3-sprint default lookback for
  *                                      --flashcard-cleanup scan.
+ *   CLEARGATE_SKIP_BUNDLE_CHECK=1    — skip Step 3.5 bundle generation + size check entirely
+ *                                      (CR-036 test seam; analogous to CLEARGATE_SKIP_MERGE_CHECK).
+ *                                      Never use in production — Step 3.5 is v2-fatal in production.
  */
 
 import fs from 'node:fs';
@@ -153,7 +157,7 @@ function invokeScript(scriptName, scriptArgs, env) {
   });
 }
 
-function main() {
+async function main() {
   const args = process.argv.slice(2);
 
   if (args.length < 1) usage();
@@ -348,6 +352,61 @@ function main() {
     }
   }
 
+  // ── Step 2.6b: Cross-Sprint Orphan Drift Check (CR-048) ─────────────────────
+  // Detect items in pending-sync/ with non-terminal status whose state.json entry
+  // in any closed sprint shows Done — i.e., completed but never archived.
+  // v2: drift > 0 blocks close. v1: warn-only.
+  // Test seam: CLEARGATE_SKIP_LIFECYCLE_CHECK=1 also skips this step.
+  process.stdout.write('Step 2.6b: checking for cross-sprint orphan drift...\n');
+  if (process.env.CLEARGATE_SKIP_LIFECYCLE_CHECK !== '1') {
+    try {
+      const cliBin26b = path.join(REPO_ROOT, 'cleargate-cli', 'dist', 'cli.js');
+      if (fs.existsSync(cliBin26b)) {
+        // Dynamic import the compiled reconciler function
+        const reconcilerMod = await import(
+          path.join(REPO_ROOT, 'cleargate-cli', 'dist', 'lib', 'lifecycle-reconcile.js')
+        ).catch(() => null);
+
+        if (reconcilerMod && typeof reconcilerMod.reconcileCrossSprintOrphans === 'function') {
+          const deliveryRoot = path.join(REPO_ROOT, '.cleargate', 'delivery');
+          const sprintRunsRoot = path.join(REPO_ROOT, '.cleargate', 'sprint-runs');
+          const orphanResult = reconcilerMod.reconcileCrossSprintOrphans({ deliveryRoot, sprintRunsRoot });
+
+          if (orphanResult.drift.length > 0) {
+            process.stderr.write(
+              `Step 2.6b: ${orphanResult.drift.length} cross-sprint orphan(s) detected:\n`
+            );
+            for (const item of orphanResult.drift) {
+              process.stderr.write(
+                `  ${item.id} — status: ${item.pending_sync_status} in pending-sync, ` +
+                `state: ${item.state_json_state} in ${item.state_json_sprint}\n`
+              );
+            }
+            if (isV2) {
+              process.stderr.write(
+                'close_sprint: Step 2.6b FAILED — orphan drift blocks sprint close under v2.\n' +
+                '  Archive the listed items and re-run close_sprint.mjs.\n'
+              );
+              process.exit(1);
+            } else {
+              process.stdout.write('Step 2.6b warning (v1): orphan drift detected above — remediate before next sprint.\n');
+            }
+          } else {
+            process.stdout.write('Step 2.6b passed: no cross-sprint orphan drift.\n');
+          }
+        } else {
+          process.stdout.write('Step 2.6b skipped: reconcileCrossSprintOrphans not available in built CLI.\n');
+        }
+      } else {
+        process.stdout.write('Step 2.6b skipped: CLI binary not found (non-fatal).\n');
+      }
+    } catch (step26bErr) {
+      process.stderr.write(`Step 2.6b warning: orphan check unavailable: ${step26bErr.message}\n`);
+    }
+  } else {
+    process.stdout.write('Step 2.6b skipped: CLEARGATE_SKIP_LIFECYCLE_CHECK=1 set (test seam).\n');
+  }
+
   // ── Step 2.7: Worktree-Closed Check (CR-022 M1) ──────────────────────────
   // Block close if any .worktrees/STORY-* path is present.
   // v2 enforcing (exit 1); v1 advisory (warn + continue).
@@ -511,17 +570,40 @@ function main() {
   }
 
   // ── Step 3.5: Build curated Reporter context bundle ───────────────────────
-  process.stdout.write('Step 3.5: building Reporter context bundle...\n');
-  try {
-    invokeScript('prep_reporter_context.mjs', [sprintId], {
-      CLEARGATE_STATE_FILE: stateFile,
-      CLEARGATE_SPRINT_DIR: sprintDir,
-    });
-    process.stdout.write(`Step 3.5 passed: ${sprintDir}/.reporter-context.md ready.\n`);
-  } catch (err) {
-    // Non-fatal — Reporter falls back to source files
-    process.stderr.write(`Step 3.5 warning: prep_reporter_context.mjs failed: ${/** @type {Error} */ (err).message}\n`);
-    process.stderr.write('Reporter will fall back to broad-fetch context loading.\n');
+  const bundlePath = path.join(sprintDir, '.reporter-context.md');
+  const isEnforcingV2 = isV2 && state.execution_mode === 'v2';
+  const MIN_BUNDLE_BYTES = 2048;
+  if (process.env.CLEARGATE_SKIP_BUNDLE_CHECK === '1') {
+    process.stdout.write('Step 3.5 skipped: CLEARGATE_SKIP_BUNDLE_CHECK=1 set (test seam).\n');
+  } else {
+    process.stdout.write('Step 3.5: building Reporter context bundle...\n');
+    try {
+      invokeScript('prep_reporter_context.mjs', [sprintId], {
+        CLEARGATE_STATE_FILE: stateFile,
+        CLEARGATE_SPRINT_DIR: sprintDir,
+      });
+      if (!fs.existsSync(bundlePath)) {
+        throw new Error(`bundle not written at ${bundlePath}`);
+      }
+      const bundleSize = fs.statSync(bundlePath).size;
+      if (bundleSize < MIN_BUNDLE_BYTES) {
+        throw new Error(`bundle too small (${bundleSize}B < ${MIN_BUNDLE_BYTES}B): ${bundlePath}`);
+      }
+      process.stdout.write(`Step 3.5 passed: ${bundlePath} ready (${Math.round(bundleSize / 1024)}KB).\n`);
+    } catch (err) {
+      const msg = /** @type {Error} */ (err).message;
+      if (isEnforcingV2) {
+        process.stderr.write(
+          `close_sprint: Step 3.5 FAILED (v2 hard-block): ${msg}\n` +
+          `  Cannot dispatch Reporter without bundle. Fix prep_reporter_context.mjs or run with execution_mode: v1.\n` +
+          `  Diagnostic: node .cleargate/scripts/prep_reporter_context.mjs ${sprintId}\n`
+        );
+        process.exit(1);
+      } else {
+        process.stderr.write(`Step 3.5 warning (v1 advisory): ${msg}\n`);
+        process.stderr.write('Reporter will fall back to broad-fetch context loading.\n');
+      }
+    }
   }
 
   // ── Step 4: Orchestrator spawns Reporter separately ───────────────────────
@@ -700,4 +782,4 @@ function main() {
   }
 }
 
-main();
+await main();

package/dist/templates/cleargate-planning/.cleargate/scripts/gate-checks.json

@@ -1,13 +1,13 @@
 {
   "schema_version": 1,
   "qa": {
-    "typecheck": "npm run typecheck",
+    "typecheck": "cd cleargate-cli && npm run typecheck",
     "debug_patterns": ["console.log", "console.debug", "debugger"],
     "todo_patterns": ["TODO", "FIXME", "XXX"],
-    "test": "npm test"
+    "test": "cd cleargate-cli && npm test"
   },
   "arch": {
-    "typecheck": "npm run typecheck",
+    "typecheck": "cd cleargate-cli && npm run typecheck",
     "new_deps_check": true,
     "stray_env_files": [".env", ".env.local", ".env.production"],
     "file_count_report": true

package/dist/templates/cleargate-planning/.cleargate/scripts/init_sprint.mjs

@@ -2,7 +2,12 @@
 /**
  * init_sprint.mjs — Initialize a sprint state.json
  *
- * Usage: node init_sprint.mjs <sprint-id> --stories ID1,ID2,... [--force]
+ * Usage: node init_sprint.mjs <sprint-id> --stories ID1,ID2,... [--force] [--preserve-bounces]
+ *
+ * --preserve-bounces requires --force; reads existing state.json and carries
+ * forward qa_bounces / arch_bounces / state / lane / worktree per matching
+ * story-id. Items not in the new --stories list are dropped. Useful when
+ * mid-sprint dogfood / rerun must not lose kickback history.
  *
  * Creates .cleargate/sprint-runs/<sprint-id>/state.json with initial state
  * "Ready to Bounce" for each story. Refuses if state.json already exists
@@ -117,6 +122,7 @@ function main() {
   }
 
   const force = args.includes('--force');
+  const preserveBounces = args.includes('--preserve-bounces');
 
   const sprintDir = path.join(REPO_ROOT, '.cleargate', 'sprint-runs', sprintId);
   const stateFile = path.join(sprintDir, 'state.json');
@@ -128,6 +134,20 @@ function main() {
     process.exit(1);
   }
 
+  // --- CR-049-followup: Preserve bounce counters when --force re-inits an in-flight sprint ---
+  // Default --force overwrites everything (initial design). With --preserve-bounces,
+  // qa_bounces / arch_bounces / state are read from the existing state.json and merged
+  // back per-story. Only Ready-to-Bounce default fields get reset.
+  let preserved = {};
+  if (force && preserveBounces && fs.existsSync(stateFile)) {
+    try {
+      const existing = JSON.parse(fs.readFileSync(stateFile, 'utf8'));
+      preserved = existing.stories || {};
+    } catch (err) {
+      process.stderr.write(`WARN: --preserve-bounces could not read existing state.json: ${err.message}\n`);
+    }
+  }
+
   // --- Read execution_mode from sprint frontmatter ---
   const sprintFilePath = findSprintFile(REPO_ROOT, sprintId);
   const executionMode = sprintFilePath ? readExecutionMode(sprintFilePath) : 'v1';
@@ -159,17 +179,18 @@ function main() {
   const now = new Date().toISOString();
   const stories = {};
   for (const id of storyIds) {
+    const carry = preserved[id] || {};
     stories[id] = {
-      state: 'Ready to Bounce',
-      qa_bounces: 0,
-      arch_bounces: 0,
-      worktree: null,
+      state: carry.state ?? 'Ready to Bounce',
+      qa_bounces: carry.qa_bounces ?? 0,
+      arch_bounces: carry.arch_bounces ?? 0,
+      worktree: carry.worktree ?? null,
       updated_at: now,
-      notes: '',
-      lane: 'standard',
-      lane_assigned_by: 'migration-default',
-      lane_demoted_at: null,
-      lane_demotion_reason: null,
+      notes: carry.notes ?? '',
+      lane: carry.lane ?? 'standard',
+      lane_assigned_by: carry.lane_assigned_by ?? 'migration-default',
+      lane_demoted_at: carry.lane_demoted_at ?? null,
+      lane_demotion_reason: carry.lane_demotion_reason ?? null,
     };
   }
 
@@ -189,6 +210,61 @@ function main() {
   fs.writeFileSync(tmpFile, JSON.stringify(state, null, 2) + '\n', 'utf8');
   fs.renameSync(tmpFile, stateFile);
 
+  // --- CR-045: Write sprint-context.md from template ---
+  // Reads .cleargate/templates/sprint_context.md, substitutes frontmatter fields,
+  // optionally splices the sprint goal from the sprint plan §0, and writes atomically.
+  // Skip if file already exists and --force was not passed (idempotency-safe re-init).
+  const ctxTemplate = path.join(REPO_ROOT, '.cleargate', 'templates', 'sprint_context.md');
+  const ctxOut = path.join(sprintDir, 'sprint-context.md');
+
+  if (!fs.existsSync(ctxOut) || force) {
+    let ctxContent;
+    try {
+      ctxContent = fs.readFileSync(ctxTemplate, 'utf8');
+    } catch {
+      // Template absent — log warning and continue (non-fatal)
+      process.stderr.write(
+        `WARN: sprint-context.md template not found at ${ctxTemplate}; skipping sprint-context.md write.\n`
+      );
+      ctxContent = null;
+    }
+
+    if (ctxContent !== null) {
+      // Substitute frontmatter placeholders
+      ctxContent = ctxContent.replace(/sprint_id:\s*["']?S-NN["']?/, `sprint_id: "${sprintId}"`);
+      ctxContent = ctxContent.replace(/created_at:\s*["']?YYYY-MM-DDTHH:MM:SSZ["']?/, `created_at: "${now}"`);
+      ctxContent = ctxContent.replace(/last_updated:\s*["']?YYYY-MM-DDTHH:MM:SSZ["']?/, `last_updated: "${now}"`);
+
+      // Optionally extract sprint goal from sprint plan §0.
+      // Regex matches `- **Sprint Goal:** <text>` within first 200 lines (after H1).
+      // Falls back to placeholder if absent — non-fatal.
+      if (sprintFilePath) {
+        try {
+          const planContent = fs.readFileSync(sprintFilePath, 'utf8');
+          const planLines = planContent.split('\n').slice(0, 200);
+          const goalLine = planLines.find((l) => /^- \*\*Sprint Goal:\*\* (.+)$/.test(l.trim()));
+          if (goalLine) {
+            const goalMatch = goalLine.trim().match(/^- \*\*Sprint Goal:\*\* (.+)$/);
+            if (goalMatch) {
+              const goalText = goalMatch[1].trim();
+              ctxContent = ctxContent.replace(
+                '_(populated by orchestrator from sprint plan §0 at kickoff)_',
+                goalText
+              );
+            }
+          }
+        } catch {
+          // Goal extraction failed — leave placeholder; non-fatal
+        }
+      }
+
+      // Write atomically via tmpFile + renameSync (mirrors state.json pattern)
+      const ctxTmp = `${ctxOut}.tmp.${process.pid}`;
+      fs.writeFileSync(ctxTmp, ctxContent, 'utf8');
+      fs.renameSync(ctxTmp, ctxOut);
+    }
+  }
+
   process.stdout.write(`Initialized state.json for sprint ${sprintId} with ${storyIds.length} stories\n`);
 }