cleargate 0.8.2 → 0.11.0

package/templates/cleargate-planning/.cleargate/knowledge/mid-sprint-triage-rubric.md

@@ -0,0 +1,160 @@
+# Mid-Sprint Triage Rubric
+
+**CR-047 · SPRINT-23 · Authoritative Reference**
+
+This document is the authoritative rubric for classifying mid-sprint user input. It complements the operational routing table in SKILL.md §C.10 (new rubric section). Read this doc to understand *why* a class exists; read SKILL.md §C.11 (post-CR-047 renumber) to see *how* routing works in practice.
+
+---
+
+## Overview
+
+When a user injects input during an active sprint (between story kickoff and story merge), the orchestrator must classify it before routing. Unclassified input leads to either silent scope creep or unnecessary story restarts. The four classes below are mutually exclusive and exhaustive.
+
+**Classifier aid:** `cleargate-cli/src/lib/triage-classifier.ts` exports a `classify()` pure function that performs keyword-heuristic pre-classification. Output is advisory — the orchestrator confirms before acting. `confidence: 'low'` always requires human verification.
+
+---
+
+## Class 1: Bug
+
+**Definition:** The user reports that existing implemented behaviour is incorrect, broken, or regressed against the current story's acceptance spec. A bug does NOT introduce new requirements — it identifies a gap between the spec and the actual behaviour.
+
+**Keywords (heuristic):** broken, crashes, doesn't work, does not work, regression, nothing works, not working, failed, failure, error, exception, bug, defect, broke.
+
+**Boundary cases:**
+- "The button is broken" → Bug (existing behaviour broken vs spec).
+- "The button should also glow" → NOT Bug; this is Scope Change (new requirement).
+- "After the deploy nothing works" → Bug (regression language).
+
+**Worked examples:**
+
+1. "The login button is broken — it throws a 500 error instead of returning 401."
+   → Class: Bug · Route: re-open story, Dev fixes, QA re-verifies.
+
+2. "After the deploy the email sending stopped working."
+   → Class: Bug · Route: same as above.
+
+**Routing rules:**
+
+- Increment `qa_bounces` via `update_state.mjs <story-id> --qa-bounce`.
+- Re-open the story; Developer fixes; QA re-verifies (full loop).
+- Log in sprint §4 Execution Log: date + story ID + one-line description.
+- Human approval: NOT required (orchestrator routes autonomously).
+
+**Bounce-counter impact:** `qa_bounces++`. If `qa_bounces ≥ 3` → `Escalated`, halt.
+
+---
+
+## Class 2: Spec Clarification
+
+**Definition:** The user asks a question or requests clarification about an existing requirement without adding new scope. The story's acceptance Gherkin remains unchanged after the clarification — at most, ambiguous language in §1.2 gets updated in place.
+
+**Keywords (heuristic):** what does, what is, clarify, clarification, is the same as, the same as, same as, mean in, mean by, what do you mean, does it include, is this, should it.
+
+**Boundary cases:**
+- "What does 'eligible' mean in §3?" → Clarification (no new scope).
+- "Is 'merged' the same as 'closed'?" → Clarification (terminology disambiguation).
+- "What does 'eligible' mean — and should we also check for X?" → SPLIT: Clarification + Scope Change. Handle separately.
+
+**Worked examples:**
+
+1. "What does 'eligible' mean in the eligibility check requirement?"
+   → Class: Clarification · Update §1.2 with the answer; no story restart.
+
+2. "Is 'merged' the same as 'closed' for the purposes of the state machine?"
+   → Class: Clarification · Add a definition note; no counter impact.
+
+**Routing rules:**
+
+- No counter increment.
+- Update §1.2 (Acceptance Criteria) in place with the clarified definition.
+- Re-run only the impacted test(s) (not full loop).
+- Log in sprint §4 Execution Log: date + story ID + one-line clarification.
+- Human approval: NOT required for terminology clarifications. Required if the clarification reveals a spec gap (surface to human before updating §1.2).
+
+**Bounce-counter impact:** None.
+
+---
+
+## Class 3: Scope Change
+
+**Definition:** The user introduces a net-new requirement that was not in the original story spec. Even if "obvious" or "related", if the Gherkin would need a new scenario, it is Scope Change.
+
+**Keywords (heuristic):** also need, we also need, plus add, additionally, new requirement, add a, add an, plus, as well, in addition, new feature, extend with.
+
+**Boundary cases:**
+- "We also need a CSV export" → Scope Change (new feature).
+- "Plus add audit logging" → Scope Change (additional requirement).
+- "The export is broken" → NOT Scope Change; this is Bug.
+
+**Worked examples:**
+
+1. "We also need a CSV export alongside the existing JSON export."
+   → Class: Scope Change · Quarantine into new story in `pending-sync/`. Current story unchanged.
+
+2. "Plus add audit logging for all admin actions."
+   → Class: Scope Change · Same quarantine routing.
+
+**Routing rules:**
+
+- **Quarantine by default.** Create a new Story file in `.cleargate/delivery/pending-sync/` for the next sprint.
+- Current story proceeds UNCHANGED.
+- **Goal-critical override:** if the new requirement is critical to the active sprint goal, escalate to human: *"This scope-change is goal-critical: the sprint goal is `<verbatim>` and without this change, the goal will not be met. Add to current sprint? (Adding mid-sprint requires explicit ack.)"*
+- Log in sprint §4 Execution Log: date + new story ID + one-line description.
+- Human approval: REQUIRED for mid-sprint addition; NOT required for quarantine.
+
+**Bounce-counter impact:** None (quarantine path). If mid-sprint addition approved: treat as a new story dispatch (all counters reset for the new story).
+
+---
+
+## Class 4: Approach Change
+
+**Definition:** The user proposes a different implementation technique or technology for the same spec. The acceptance Gherkin remains identical — only the *how* changes, not the *what*.
+
+**Keywords (heuristic):** instead of, switch to, different way, different approach, replace with, rather than, alternative, migrate to.
+
+**Boundary cases:**
+- "Instead of polling, switch to websockets" → Approach Change (same behaviour spec, different mechanism).
+- "Switch to Postgres instead of Redis for invite storage" → Approach Change (storage backend, same API).
+- "Instead of storing invites, delete them" → NOT Approach Change; this is Scope Change (spec changes).
+
+**Worked examples:**
+
+1. "Instead of polling the API every 5 seconds, switch to websockets for real-time updates."
+   → Class: Approach Change · No counter; reset Developer context; re-spawn with updated approach note.
+
+2. "Use a different algorithm — BFS instead of DFS for the graph traversal."
+   → Class: Approach Change · Same routing.
+
+**Routing rules:**
+
+- No counter increment.
+- Reset Developer context (re-spawn Developer with the updated approach in the dispatch prompt).
+- Story Gherkin and acceptance criteria remain UNCHANGED.
+- Log in sprint §4 Execution Log: date + story ID + one-line approach delta.
+- Human approval: NOT required if the approach is technically equivalent. Required if the approach change affects cost, timeline, or cross-story surfaces.
+
+**Bounce-counter impact:** None.
+
+---
+
+## Routing Summary Table
+
+| Class | Trigger | Counter | Human Approval | Routing Action |
+|---|---|---|---|---|
+| Bug | Defect vs spec | `qa_bounces++` | No | Re-open story; Dev fix; QA re-verify |
+| Spec Clarification | Ambiguity question | None | No (yes if spec gap) | Update §1.2 in place; re-run impacted test |
+| Scope Change | Net-new requirement | None | YES for mid-sprint add | Quarantine to next sprint (default); escalate if goal-critical |
+| Approach Change | Different impl, same spec | None | No (yes if cross-story impact) | Reset Dev context; re-spawn with updated approach |
+
+---
+
+## Cross-References
+
+- **SKILL.md §C.10** — NEW Mid-Sprint Triage section (operational routing table, added by CR-047).
+- **SKILL.md §C.11** — Mid-cycle User Input table (pre-CR-047 §C.10; renumbered to §C.11 by this CR).
+- **`cleargate-cli/src/lib/triage-classifier.ts`** — keyword-heuristic classifier (advisory, not authoritative).
+- **SKILL.md §C.3.5** — TPV Gate (Architect-only wiring check between QA-Red and Developer).
+
+---
+
+_This document is append-only. Add new examples or boundary cases at the bottom of the relevant class block. Do not restructure class ordering — it matches the classifier priority (bug → approach → scope → clarification)._

package/templates/cleargate-planning/.cleargate/knowledge/readiness-gates.md

@@ -6,7 +6,7 @@ This file is the single source of truth for ClearGate's machine-checkable readin
 
 ## Predicate Vocabulary
 
-There are exactly **6 predicate shapes**. No other shapes are recognized; a check string that does not match one of these forms throws a parse error at evaluation time.
+There are exactly **7 predicate shapes**. No other shapes are recognized; a check string that does not match one of these forms throws a parse error at evaluation time.
 
 **1. `frontmatter(<ref>).<field> <op> <value>`**
 Reads a frontmatter field from a document. `<ref>` is either `.` (the document being evaluated) or a frontmatter key whose value is a relative path to another document (e.g. `context_source`). `<op>` is one of `==`, `!=`, `>=`, `<=`. `<value>` is a literal string, number, or boolean. Example: `frontmatter(context_source).approved == true` reads the file named by the evaluated document's `context_source` key and asserts its `approved` field equals `true`.
@@ -15,7 +15,13 @@ Reads a frontmatter field from a document. `<ref>` is either `.` (the document b
 Performs a case-sensitive substring search on the document body (everything after the frontmatter block). The negated form `body does not contain` passes when the string is absent. Example: `body does not contain 'TBD'` fails if the literal string `TBD` appears anywhere in the body.
 
 **3. `section(<N>) has <count> <item-type>`**
-Splits the document body on `## ` heading boundaries (1-indexed) and counts items of a given type within section N. `<count>` is an expression like `≥1`, `≥3`, or `0` (exact zero). `<item-type>` is one of `checked-checkbox` (lines matching `- [x]`), `unchecked-checkbox` (lines matching `- [ ]`), or `listed-item` (lines matching `- ` regardless of checkbox state). Example: `section(2) has ≥1 checked-checkbox` asserts that the second `##` section contains at least one checked markdown checkbox.
+Splits the document body on `## ` heading boundaries (1-indexed) and counts items of a given type within section N. `<count>` is an expression like `≥1`, `≥3`, or `0` (exact zero). `<item-type>` is one of:
+- `checked-checkbox` — lines matching `- [x]`
+- `unchecked-checkbox` — lines matching `- [ ]`
+- `listed-item` — lines matching `- ` regardless of checkbox state (bullet-precise; use when checkbox/task-list semantics are required, e.g. DoD)
+- `declared-item` — any line that declares a structured item: bullet lines (`- ...`), table data rows (`| ... |` lines following a `|---|`-style separator within the section), or definition-list terms (lines matching `**Item:**`, `Item:`, `*Item*:` etc.). Use `declared-item` when the gate cares only that the author declared at least N entries in section N, regardless of presentation format (table vs bullet vs def-list).
+
+Example: `section(2) has ≥1 checked-checkbox` asserts that the second `##` section contains at least one checked markdown checkbox. Example: `section(3) has ≥1 declared-item` passes when §3 contains at least one bullet, table data row, or definition-list term.
 
 **4. `file-exists(<path>)`**
 Asserts that a file exists on disk at the given path, resolved relative to the project root. Example: `file-exists(.cleargate/knowledge/cleargate-protocol.md)` passes when that file is present in the working tree.
@@ -26,6 +32,9 @@ Reads `.cleargate/wiki/index.md` and asserts that the wiki index contains a refe
 **6. `status-of(<[[ID]]>) == <value>`**
 Resolves the given ID via the wiki index, reads that page's compiled frontmatter `status:` field, and compares it to `<value>`. Status values in the live corpus are textual strings (`Draft`, `Ready`, `Active`, `Done`) — not emoji. Example: `status-of([[EPIC-008]]) == Active` passes when EPIC-008's wiki page has `status: Active`. Note: this predicate returns `unknown` (evaluates to fail) when the wiki index is stale and the item is not yet compiled. Run `cleargate wiki build` before relying on `status-of` predicates.
 
+**7. `existing-surfaces-verified`**
+Closed-set predicate (no parameters). Locates the `## Existing Surfaces` section in the document body, extracts path-shaped substrings via regex, asserts each cited path exists on disk relative to the project root. Passes when section is absent (defers to `reuse-audit-recorded`) OR all cited paths exist OR section contains a "no overlap found" / "no existing surface" / "no prior implementation" / "audit returned empty" sentinel. Sandbox-rejected paths (escaping project root) are treated as missing. Example: `existing-surfaces-verified` against an Epic body whose `## Existing Surfaces` cites `cleargate-cli/src/lib/work-item-type.ts:detectWorkItemTypeFromFm` passes when that path exists.
+
 ---
 
 ## Severity Model
@@ -60,16 +69,28 @@ The asymmetry exists because Proposal documents are human-authored strategy arti
   transition: ready-for-decomposition
   severity: enforcing
   criteria:
-    - id: proposal-approved
+    - id: parent-approved-proposal
       check: "frontmatter(context_source).approved == true"
+      or_group: parent-approved
+    - id: parent-approved-initiative
+      check: "frontmatter(context_source).status == 'Triaged'"
+      or_group: parent-approved
     - id: no-tbds
       check: "body does not contain marker 'TBD'"
     - id: scope-in-populated
-      check: "section(2) has ≥1 listed-item"
+      check: "section(3) has ≥1 declared-item"
     - id: affected-files-declared
-      check: "section(4) has ≥1 listed-item"
+      check: "section(5) has ≥1 declared-item"
     - id: interrogation-resolved
       check: "body does not contain 'Unresolved'"
+    - id: discovery-checked
+      check: "frontmatter(.).context_source != null"
+    - id: reuse-audit-recorded
+      check: "body contains '## Existing Surfaces'"
+    - id: existing-surfaces-verified
+      check: "existing-surfaces-verified"
+    - id: simplest-form-justified
+      check: "body contains '## Why not simpler?'"
 ```
 
 ```yaml
@@ -87,6 +108,8 @@ The asymmetry exists because Proposal documents are human-authored strategy arti
       check: "body does not contain marker 'TBD'"
     - id: interrogation-resolved
       check: "body does not contain 'Unresolved'"
+    - id: discovery-checked
+      check: "frontmatter(.).context_source != null"
 ```
 
 ```yaml
@@ -99,11 +122,19 @@ The asymmetry exists because Proposal documents are human-authored strategy arti
     - id: no-tbds
       check: "body does not contain marker 'TBD'"
     - id: implementation-files-declared
-      check: "section(3) has ≥1 listed-item"
+      check: "section(3) has ≥1 declared-item"
     - id: dod-declared
       check: "section(4) has ≥1 listed-item"
     - id: gherkin-present
       check: "body contains 'Scenario:'"
+    - id: discovery-checked
+      check: "frontmatter(.).context_source != null"
+    - id: reuse-audit-recorded
+      check: "body contains '## Existing Surfaces'"
+    - id: existing-surfaces-verified
+      check: "existing-surfaces-verified"
+    - id: simplest-form-justified
+      check: "body contains '## Why not simpler?'"
 ```
 
 ```yaml
@@ -112,11 +143,17 @@ The asymmetry exists because Proposal documents are human-authored strategy arti
   severity: enforcing
   criteria:
     - id: blast-radius-populated
-      check: "section(2) has ≥1 listed-item"
+      check: "section(2) has ≥1 declared-item"
     - id: no-tbds
       check: "body does not contain marker 'TBD'"
     - id: sandbox-paths-declared
-      check: "section(2) has ≥1 listed-item"
+      check: "section(3) has ≥1 declared-item"
+    - id: discovery-checked
+      check: "frontmatter(.).context_source != null"
+    - id: reuse-audit-recorded
+      check: "body contains '## Existing Surfaces'"
+    - id: existing-surfaces-verified
+      check: "existing-surfaces-verified"
 ```
 
 ```yaml
@@ -125,9 +162,35 @@ The asymmetry exists because Proposal documents are human-authored strategy arti
   severity: enforcing
   criteria:
     - id: repro-steps-deterministic
-      check: "section(2) has ≥3 listed-item"
+      check: "section(2) has ≥3 declared-item"
     - id: severity-set
       check: "frontmatter(.).severity != null"
     - id: no-tbds
       check: "body does not contain marker 'TBD'"
+    - id: discovery-checked
+      check: "frontmatter(.).context_source != null"
+```
+
+```yaml
+- work_item_type: sprint
+  transition: ready-for-execution
+  severity: enforcing
+  criteria:
+    - id: risk-table-populated
+      check: "body contains '| Mitigation'"
+    - id: discovery-checked
+      check: "frontmatter(.).context_source != null"
+```
+
+```yaml
+- work_item_type: initiative
+  transition: ready-for-decomposition
+  severity: advisory
+  criteria:
+    - id: no-tbds
+      check: "body does not contain marker 'TBD'"
+    - id: user-flow-populated
+      check: "section(1) has ≥1 listed-item"
+    - id: success-criteria-populated
+      check: "section(5) has ≥1 listed-item"
 ```

package/templates/cleargate-planning/.cleargate/knowledge/sprint-closeout-checklist.md

@@ -0,0 +1,71 @@
+# Sprint Closeout Doc & Metadata Refresh Checklist
+
+> Read at sprint close (Gate 4 ack). Each item names a surface that may need
+> updating based on what shipped this sprint. Trigger conditions tell you when
+> review is required vs when the surface can be skipped.
+>
+> Use `node .cleargate/scripts/prep_doc_refresh.mjs <sprint-id>` to generate
+> a per-sprint tailored checklist that pre-checks items based on actual
+> changed files in the sprint window.
+
+### 1. Project READMEs
+| Surface | Trigger condition |
+|---|---|
+| `README.md` | Any feature shipped that changes user-visible product behavior |
+| `cleargate-cli/README.md` | Any change to `cleargate-cli/src/commands/*.ts` |
+| `cleargate-planning/README.md` | Any change under `cleargate-planning/` |
+| `mcp/README.md` | Any change under `mcp/src/` (note: nested repo; check separately) |
+| `admin/README.md` | Any change under `admin/` (currently stub) |
+
+### 2. CHANGELOG files (Common-Changelog format per STORY-016-03)
+| Surface | Trigger condition |
+|---|---|
+| `cleargate-cli/CHANGELOG.md` | Any user-visible change in `cleargate-cli/` (CLI surface, error messages, package contents) |
+| `mcp/CHANGELOG.md` | Any user-visible change in `mcp/` (if file exists) |
+
+### 3. Manifest / package metadata
+| Surface | Trigger condition |
+|---|---|
+| `cleargate-planning/MANIFEST.json` | Any change to `.claude/agents/*.md`, `.cleargate/templates/*`, `.cleargate/knowledge/*`, or `.cleargate/scripts/*`. Run `cleargate doctor` to verify scaffold registry. |
+| `cleargate-cli/package.json` | Version bump only if releasing this sprint (release lane is separate from sprint close) |
+| `mcp/package.json` | Version bump only if releasing this sprint |
+
+### 4. CLAUDE.md "Active state" subsection
+| Surface | Trigger condition |
+|---|---|
+| `CLAUDE.md` lines containing "Active state (as of YYYY-MM-DD)" | Any EPIC / CR / Bug / Hotfix archived this sprint, OR any stack version bumped |
+| `cleargate-planning/CLAUDE.md` mirror | Same edit as live (CLEARGATE-tag-block region only — outside-block diverges intentionally) |
+
+### 5. Wiki surfaces (auto-rebuilt by PostToolUse hooks; verify after close)
+| Surface | Verify by |
+|---|---|
+| `.cleargate/wiki/active-sprint.md` | Read top of file; confirm sprint ID, status, and date are current |
+| `.cleargate/wiki/index.md` | Read; confirm new artifacts (epics, stories, CRs) appear in the relevant sections |
+| `.cleargate/wiki/product-state.md` | Read; confirm shipped capabilities are listed |
+| `.cleargate/wiki/roadmap.md` | Read; confirm closed sprint moved from Active to Completed section |
+
+### 6. INDEX surfaces (manual updates)
+| Surface | Trigger condition |
+|---|---|
+| `.cleargate/INDEX.md` | If maintained as a curated roadmap; update when sprint closes |
+| `.cleargate/delivery/INDEX.md` | Update epic/sprint map when new artifacts archived |
+
+### 7. Frontmatter version stamps
+| Surface | Action |
+|---|---|
+| Any `.cleargate/templates/*.md` modified this sprint | Run `cleargate stamp <path>` to bump `updated_at_version` |
+| `.cleargate/knowledge/cleargate-protocol.md` (post-EPIC-024 slim) | Same |
+| `.cleargate/knowledge/cleargate-enforcement.md` (post-EPIC-024 split) | Same |
+| Any other `.cleargate/knowledge/*.md` modified | Same |
+
+### 8. Knowledge doc cross-references
+| Surface | Action |
+|---|---|
+| Any knowledge doc that cites `§N` of protocol or enforcement.md | Verify post-rewrite resolution still works (covered for SPRINT-17 specifically by STORY-024-02; revisit if any future § reorganization happens) |
+
+### 9. Mirror parity audit
+| Surface | Action |
+|---|---|
+| `cleargate-planning/.claude/` | `diff -r .claude/agents/ cleargate-planning/.claude/agents/` empty (excluding skills/ flashcards/, hooks/, settings.json which differ intentionally) |
+| `cleargate-planning/.cleargate/templates/` | `diff -r .cleargate/templates/ cleargate-planning/.cleargate/templates/` empty |
+| `cleargate-planning/.cleargate/knowledge/` | `diff -r .cleargate/knowledge/ cleargate-planning/.cleargate/knowledge/` empty |

package/templates/cleargate-planning/.cleargate/scripts/assert_story_files.mjs

@@ -71,7 +71,7 @@ function extractDeliverablesSection(content) {
  *
  * STORY before CR/BUG/EPIC/HOTFIX, PROPOSAL before PROP — BUG-010 longest-first rule.
  */
-function extractWorkItemIds(text) {
+export function extractWorkItemIds(text) {
   const re = /(STORY-\d+-\d+|(CR|BUG|EPIC|HOTFIX)-\d+|(PROPOSAL|PROP)-\d+)/g;
   const raw = [];
   let m;
@@ -87,7 +87,7 @@ function extractWorkItemIds(text) {
  * Check whether pending-sync OR archive contains a file matching <ID>_*.md
  * Returns the matching absolute path or null.
  */
-function findWorkItemFile(repoRoot, workItemId) {
+export function findWorkItemFile(repoRoot, workItemId) {
   const searchDirs = [
     path.join(repoRoot, '.cleargate', 'delivery', 'pending-sync'),
     path.join(repoRoot, '.cleargate', 'delivery', 'archive'),
@@ -207,6 +207,28 @@ function main() {
 
   const sprintFilePath = path.resolve(args[0]);
 
+  // CR-027: --emit-json flag — extract work-item IDs and print JSON to stdout.
+  // Used by sprint.ts checkPerItemReadinessGates (path-a shell-out protocol).
+  // When this flag is present, skip the file-presence/approval triage and exit 0.
+  if (args.includes('--emit-json')) {
+    let content;
+    try {
+      content = fs.readFileSync(sprintFilePath, 'utf8');
+    } catch (err) {
+      process.stderr.write(`Error: cannot read sprint file: ${err.message}\n`);
+      process.exit(2);
+    }
+    const section = extractDeliverablesSection(content);
+    if (section === null) {
+      // No §1 section found — emit empty list (not an error for this flag)
+      process.stdout.write(JSON.stringify({ workItemIds: [] }) + '\n');
+      process.exit(0);
+    }
+    const workItemIds = extractWorkItemIds(section);
+    process.stdout.write(JSON.stringify({ workItemIds }) + '\n');
+    process.exit(0);
+  }
+
   // Allow test-isolation override of execution_mode
   const execMode = process.env.CLEARGATE_EXEC_MODE ?? 'v2';