forgedock 1.1.7 → 1.1.8

package/commands/work-on/investigate.md

@@ -90,6 +90,171 @@ Print these blocks to stdout before Phase 1A begins. During Phase 1B (step 3 —
 
 ---
 
+## Phase 0.6: Forge Ledger Pre-Recall <!-- Added: forge#1740 -->
+
+**Goal**: Query the Forge Ledger knowledge index for prior cards matching this issue's title terms, affected files, and symbols — **before reading any code**. Inject above-threshold results with explicit delta-verification framing so the investigator builds on prior knowledge rather than re-deriving it.
+
+**This phase is non-blocking** — if the index is absent, empty, or returns no match above threshold, log a single line and proceed to Phase 1A. Never stall the pipeline for recall.
+
+**Relationship to Phase 0.5**: Phase 0.5 retrieves structured priors from the Gist-based FORGE:MEMORY_INDEX (coarse keyword scoring over MEMORY_ENTRY lines). Phase 0.6 queries the Forge Ledger index (BM25 over structured knowledge cards, indexed by `build-knowledge-index.mjs`). Both are complementary and both run — neither replaces the other.
+
+### Step 1: Probe for index
+
+```bash
+RECALL_PATH="${REPO_PATH:-$(git rev-parse --show-toplevel 2>/dev/null)}/bin/recall.mjs"
+LEDGER_AVAILABLE=0
+
+if [ -f "$RECALL_PATH" ]; then
+  # Quick probe: does the index exist and have at least one card?
+  PROBE=$(node "$RECALL_PATH" --doctor 2>/dev/null | grep "^Total cards:" | grep -v "^Total cards:    0" || true)
+  [ -n "$PROBE" ] && LEDGER_AVAILABLE=1
+fi
+
+if [ "$LEDGER_AVAILABLE" -eq 0 ]; then
+  echo "[recall] Forge Ledger index absent or empty — skipping Phase 0.6, proceeding to Phase 1A"
+  # → Continue to Phase 1A
+fi
+```
+
+### Step 2: Build combined query (title terms + symbols + affected files)
+
+Extract query components from the issue body and title. The recall CLI accepts free text (BM25 ranked) plus repeated `--file` flags (exact-match boost):
+
+```bash
+# Extract title keywords: lowercase, remove stop words, keep noun/verb tokens (≥4 chars)
+ISSUE_TITLE=$(gh issue view {NUMBER} {GH_FLAG} --json title --jq '.title' 2>/dev/null || echo '')
+TITLE_TERMS=$(echo "$ISSUE_TITLE" | tr '[:upper:]' '[:lower:]' \
+  | sed 's/[^a-z0-9 ]/ /g' \
+  | tr ' ' '\n' \
+  | grep -E '^[a-z]{4,}$' \
+  | grep -vE '^(with|that|this|from|into|have|will|when|then|than|them|they|been|were|also|only|does|some|each|more|over|such|both|most|other|many|after|about|should|would|could|their|these|those|which|where|there|being|while|using|since|until|before|under|above)$' \
+  | sort -u | head -10 | tr '\n' ' ' | xargs)
+
+# Extract affected files from the issue body's ## Affected Files section
+ISSUE_BODY=$(gh issue view {NUMBER} {GH_FLAG} --json body --jq '.body' 2>/dev/null || echo '')
+AFFECTED_FILES_RAW=$(echo "$ISSUE_BODY" \
+  | sed -n '/^## Affected Files/,/^## /p' \
+  | grep -oE '`[^`]+\.(md|mjs|js|ts|py|sh|yaml|yml|json)`' \
+  | tr -d '`' | sort -u | head -5)
+
+# Extract symbols: backtick-quoted identifiers from the issue body (≥4 chars, camelCase or snake_case)
+SYMBOLS_RAW=$(echo "$ISSUE_BODY" \
+  | grep -oE '`[A-Za-z_][A-Za-z0-9_]{3,}`' \
+  | tr -d '`' | sort -u | head -5)
+
+# Build --file flags (one per affected file)
+FILE_FLAGS=""
+while IFS= read -r f; do
+  [ -n "$f" ] && FILE_FLAGS="$FILE_FLAGS --file $f"
+done <<< "$AFFECTED_FILES_RAW"
+
+# Build --symbol flag (first symbol only — recall supports one --symbol)
+SYMBOL_FLAG=""
+FIRST_SYMBOL=$(echo "$SYMBOLS_RAW" | head -1)
+[ -n "$FIRST_SYMBOL" ] && SYMBOL_FLAG="--symbol $FIRST_SYMBOL"
+```
+
+### Step 3: Run recall query and capture results
+
+```bash
+RECALL_RESULTS=""
+RECALL_ISSUE_CITATIONS=""
+
+if [ "$LEDGER_AVAILABLE" -eq 1 ] && [ -n "$TITLE_TERMS$FILE_FLAGS$SYMBOL_FLAG" ]; then
+  echo "[recall] Querying Forge Ledger: terms='${TITLE_TERMS}' files='${AFFECTED_FILES_RAW}' symbol='${FIRST_SYMBOL}'"
+
+  # Run combined query: title terms (free text) + file flags + symbol + min-score threshold
+  # --json for machine-readable output; --min-score 0.3 filters weak matches (noise gate)
+  RECALL_JSON=$(node "$RECALL_PATH" \
+    $TITLE_TERMS \
+    $FILE_FLAGS \
+    $SYMBOL_FLAG \
+    --k 5 \
+    --min-score 0.3 \
+    --json \
+    2>/dev/null || echo '[]')
+
+  # Count results
+  RECALL_COUNT=$(echo "$RECALL_JSON" | node -e "
+    let d = ''; process.stdin.on('data', c => d += c);
+    process.stdin.on('end', () => {
+      try { console.log(JSON.parse(d).length); } catch { console.log(0); }
+    });
+  " 2>/dev/null || echo "0")
+
+  if [ "${RECALL_COUNT:-0}" -gt 0 ]; then
+    echo "[recall] Found ${RECALL_COUNT} prior card(s) above threshold — injecting with delta-verification framing"
+
+    # Extract issue citations for the Prior Investigations field in the FORGE:INVESTIGATOR comment
+    RECALL_ISSUE_CITATIONS=$(echo "$RECALL_JSON" | node -e "
+      let d = ''; process.stdin.on('data', c => d += c);
+      process.stdin.on('end', () => {
+        try {
+          const cards = JSON.parse(d);
+          const seen = new Set();
+          const cites = cards
+            .filter(c => { if (seen.has(c.issue)) return false; seen.add(c.issue); return true; })
+            .map(c => '#' + c.issue)
+            .join(', ');
+          console.log(cites);
+        } catch { console.log(''); }
+      });
+    " 2>/dev/null || echo '')
+
+    # Format cards as human-readable text for injection (not raw JSON)
+    RECALL_FORMATTED=$(echo "$RECALL_JSON" | node -e "
+      let d = ''; process.stdin.on('data', c => d += c);
+      process.stdin.on('end', () => {
+        try {
+          const cards = JSON.parse(d);
+          const lines = [];
+          for (const c of cards) {
+            lines.push('── ' + c.kind.toUpperCase() + ' from #' + c.issue + ' (score: ' + c.score.toFixed(2) + ') ──');
+            if (c.rootCause)  lines.push('Root Cause: ' + c.rootCause);
+            if (c.prevention) lines.push('Fix/Prevention: ' + c.prevention);
+            if (c.pattern)    lines.push('Pattern: ' + c.pattern);
+            if (c.verdict)    lines.push('Verdict: ' + c.verdict + ' (' + (c.confidence || '?') + ')');
+            if (c.paths && c.paths.length) lines.push('Files: ' + c.paths.slice(0,3).join(', '));
+          }
+          console.log(lines.join('\n'));
+        } catch (e) { console.log(''); }
+      });
+    " 2>/dev/null || echo '')
+
+    # Cap at ≤ 2K chars (same budget as context C0 Gist summaries)
+    RECALL_RESULTS=$(echo "$RECALL_FORMATTED" | head -c 2048)
+  else
+    echo "[recall] No prior cards above threshold (min-score 0.3) — investigating from scratch"
+  fi
+fi
+```
+
+### Step 4: Inject recall results into investigation context
+
+If `RECALL_RESULTS` is non-empty, print the following block to stdout **before Phase 1A begins**. The investigation steps in Phase 1B MUST treat this as starting context — verify deltas against current code rather than re-deriving the prior model.
+
+**If `RECALL_RESULTS` is empty**: skip this step entirely and proceed to Phase 1A.
+
+```
+[FORGE:RECALL_PRIOR — Forge Ledger pre-recall results]
+Prior cards matched this issue above threshold (min-score 0.3).
+
+CRITICAL FRAMING: These cards describe prior findings on the same files/symbols.
+  → Verify the deltas against current code — do NOT re-derive the prior model.
+  → Stale cards (status: stale) are excluded by default — the query runs without
+     --include-stale. Do not treat any card as authoritative without confirming
+     the finding still exists in the current code.
+  → Cite confirmed priors in the FORGE:INVESTIGATOR "Prior Investigations" field.
+
+${RECALL_RESULTS}
+
+[END FORGE:RECALL_PRIOR]
+```
+
+Store `RECALL_ISSUE_CITATIONS` in a variable — it is used in Phase 1C to populate the `**Prior Investigations (via recall)**` field in the FORGE:INVESTIGATOR comment.
+
+---
+
 ## Phase 1A: Load Issue & Check Resume State
 
 ```bash
@@ -192,6 +357,20 @@ bash {REPO_PATH}/scripts/code-index.sh query --domain {DOMAIN_LABEL} --repo-path
 
 The comment MUST include `<!-- INVESTIGATION:COMPLETE -->` at the very end, AFTER all required sections are present. This marker signals the investigation finished successfully.
 
+**CODEC PATH (forge#1727)**: Construct the annotation body via the protocol codec — do NOT hand-roll the `<!-- FORGE:INVESTIGATOR -->` header. Use `forge-annotation.sh write INVESTIGATOR --field ...` or `node packages/protocol/src/cli.js emit INVESTIGATOR --field ...` to produce the opening tag and completion sentinel. Fill in the Markdown body sections below. The full pattern:
+
+```bash
+# Build the annotation body via codec (escaping and sentinel handled by codec)
+ANNOTATION_BODY=$(node packages/protocol/src/cli.js emit INVESTIGATOR \
+  --field "Verdict={VERDICT}" \
+  --field "Confidence={CONFIDENCE}" \
+  --field "Severity={SEVERITY}" \
+  --field "Task Type={TASK_TYPE}" \
+  --field "Decomposition Assessment={YES|NO} — {reason}")
+# ANNOTATION_BODY now has opening tag + required fields + INVESTIGATION:COMPLETE sentinel.
+# Append the Markdown body sections to it before posting.
+```
+
 Before posting, resolve the attribution annotation link from `forge.yaml`:
 
 ```bash
@@ -234,6 +413,7 @@ gh issue comment {NUMBER} {GH_FLAG} --body "<!-- FORGE:INVESTIGATOR -->
 **Last touched**: {hash — author — date — subject}
 **Pickaxe hits (prior fixes / regressions)**: {commit(s) found via \`git log -S\`/\`-G\`, or 'None found' — max 5}
 {This field is MANDATORY — populate from the git blame + pickaxe commands in step 4/5. If a file is newly created (no history), write 'New file — no history.'}
+**Prior Investigations (via recall)**: {Comma-separated issue citations from \`RECALL_ISSUE_CITATIONS\` (e.g. '#1172, #1243 — building on, not repeating'), or 'None — no Forge Ledger match above threshold' if \`RECALL_RESULTS\` was empty. Use the \`RECALL_ISSUE_CITATIONS\` variable populated in Phase 0.6.}
 
 ### Recommendation
 {what to build/fix, concrete and actionable}
@@ -541,6 +721,47 @@ fi
 
 ## Phase 1D: Update Labels & Return Verdict
 
+### 1D.0: Finding Lifecycle Label Transition (MANDATORY — run before workflow label update)
+
+**Purpose**: Wire the investigation verdict into the finding-validation lifecycle. If the issue under investigation is a review-finding (carries `needs-validation`), translate the verdict into `validated` or `false-positive` using `transition-label.sh --validate`. This is the primary mechanism that resolves the `needs-validation → validated/false-positive` lifecycle gap. <!-- Added: forge#1730 -->
+
+**Run this block regardless of verdict (CONFIRMED, PARTIAL, INVALID) — the verdict-to-label mapping handles all cases:**
+
+```bash
+# Check if this issue is a finding awaiting validation
+ISSUE_LABELS=$(gh issue view {NUMBER} {GH_FLAG} --json labels --jq '[.labels[].name] | join(",")' 2>/dev/null || echo "")
+
+if echo "$ISSUE_LABELS" | grep -q "needs-validation"; then
+  echo "Issue #{NUMBER} has needs-validation — applying verdict label transition..."
+  RESOLUTION=$(resolve_script 'transition-label')
+  TIER="${RESOLUTION%%:*}"
+  SCRIPT_PATH="${RESOLUTION#*:}"
+  case "$TIER" in
+    adaptive|universal)
+      bash "$SCRIPT_PATH" --validate {VERDICT} {NUMBER} {GH_FLAG} || true
+      ;;
+    prose)
+      # Prose fallback: apply label transition directly via gh
+      if [ "{VERDICT}" = "CONFIRMED" ]; then
+        gh issue edit {NUMBER} {GH_FLAG} --add-label "validated" --remove-label "needs-validation" 2>/dev/null || true
+        echo "Applied: needs-validation → validated (verdict: CONFIRMED)"
+      else
+        gh issue edit {NUMBER} {GH_FLAG} --add-label "false-positive" --remove-label "needs-validation" 2>/dev/null || true
+        echo "Applied: needs-validation → false-positive (verdict: {VERDICT})"
+      fi
+      ;;
+  esac
+else
+  echo "Issue #{NUMBER} does not have needs-validation — no finding lifecycle transition needed"
+fi
+```
+
+**Verdict → label mapping**: `CONFIRMED` → `validated`; `PARTIAL`, `NOT-CONFIRMED`, `INVALID` → `false-positive`.
+
+**Idempotency**: `transition-label.sh --validate` is a no-op if the issue already has `validated` or `false-positive`, or if it lacks `needs-validation`. Safe to call on any issue.
+
+---
+
 **CONFIRMED or PARTIAL with decompose: NO**:
 ```bash
 gh issue edit {NUMBER} {GH_FLAG} --add-label "workflow:ready-to-build" --remove-label "workflow:investigating"

package/commands/work-on.md

@@ -62,6 +62,7 @@ Orchestrator for the full issue lifecycle: investigate → decompose (if needed)
 - `workflow:merged` label is set
 - `workflow:invalid` label is set
 - `needs-human` label is set
+- `workflow:awaiting-merge` label is set (remediated + re-reviewed, awaiting a human merge decision — see #1810) <!-- Added: forge#1810 -->
 - `workflow:decomposed` label is set (sub-issues spawned)
 - Issue state is CLOSED with terminal label
 
@@ -216,7 +217,7 @@ fi
 Worktree/branch-already-exists and stale-label conditions are surfaced later (Phase 3E worktree creation and the `## Error Handling` section) with their own recovery guidance in `docs/site/troubleshooting.md`.
 
 ### 0A: Parse input
-Extract project prefix and issue number. If `next`/`pick`: list open issues sorted by priority, skip `needs-human` and `workflow:decomposed`, pick highest priority.
+Extract project prefix and issue number. If `next`/`pick`: list open issues sorted by priority, skip `needs-human`, `workflow:decomposed`, and `workflow:awaiting-merge`, pick highest priority.
 
 **Optional pre-flight**: Before committing to the full pipeline, run `/scope {NUMBER}` to get a complexity estimate (affected files, blast radius, risk flags, and decomposition recommendation). Especially useful for large or ambiguous issues.
 
@@ -234,7 +235,7 @@ gh issue comment {NUMBER} {GH_FLAG} --body "<!-- FORGE:HEARTBEAT -->
 
 **Also post at major phase entry points** (Phases 1, 3, and 5) — replace `Phase 0` with the correct phase name in each case. These mid-pipeline heartbeats ensure the stall detector sees recent activity during long phases (e.g., a build phase running for 20 minutes is not falsely classified as stalled). Inline snippets are embedded at Phase 1A, Phase 3A, and Phase 5A — agents resuming mid-pipeline encounter them without reading this section. <!-- Added: forge#740 -->
 
-**Skip if**: Issue already has a terminal label (`workflow:merged`, `workflow:invalid`, `needs-human`) — no heartbeat needed on a completed issue.
+**Skip if**: Issue already has a terminal label (`workflow:merged`, `workflow:invalid`, `needs-human`, `workflow:awaiting-merge`) — no heartbeat needed on a completed issue.
 
 ### 0B: Load issue + existing context
 ```bash
@@ -242,7 +243,7 @@ gh issue view {NUMBER} {GH_FLAG} --json number,title,body,labels,state,comments,
 gh api repos/{GH_REPO}/issues/{NUMBER}/comments --jq '.[] | {id: .id, author: .user.login, body: .body}'
 ```
 
-**Check**: state (closed → STOP), terminal labels (`workflow:merged`/`workflow:invalid` → STOP), existing agent comments (`FORGE:INVESTIGATOR`, `FORGE:DECOMPOSED`, `FORGE:CONTRACT`, `FORGE:BUILDER`, `FORGE:TRAJECTORY`, `FORGE:DECISION_RECORD`), parent tracker status, sub-issue status.
+**Check**: state (closed → STOP), terminal labels (`workflow:merged`/`workflow:invalid`/`workflow:awaiting-merge` → STOP), existing agent comments (`FORGE:INVESTIGATOR`, `FORGE:DECOMPOSED`, `FORGE:CONTRACT`, `FORGE:BUILDER`, `FORGE:TRAJECTORY`, `FORGE:DECISION_RECORD`), parent tracker status, sub-issue status.
 
 **Determine resume point**: No comments → Phase 1. Investigation exists + ready-to-build → Phase 3. Builder:COMPLETE + no PR → Phase 4. Builder without :COMPLETE (partial/interrupted build) + no PR → Phase 3 (partial-build cleanup). Builder + PR open → Phase 5. PR merged + issue open → Phase 6.
 
@@ -471,12 +472,12 @@ case "$TIER" in
   adaptive|universal) bash "$SCRIPT_PATH" {NUMBER} {GH_FLAG} investigating ;;
   prose)
     gh issue edit {NUMBER} {GH_FLAG} --add-label "workflow:investigating" \
-      --remove-label "workflow:ready-to-build,workflow:building,workflow:in-review,workflow:merged,workflow:invalid,workflow:decomposed" 2>/dev/null || true
+      --remove-label "workflow:ready-to-build,workflow:building,workflow:in-review,workflow:awaiting-merge,workflow:merged,workflow:invalid,workflow:decomposed" 2>/dev/null || true
     ;;
 esac
 ```
 
-**Post Phase 1 heartbeat** (skip if issue already has a terminal label — `workflow:merged`, `workflow:invalid`, `needs-human`):
+**Post Phase 1 heartbeat** (skip if issue already has a terminal label — `workflow:merged`, `workflow:invalid`, `needs-human`, `workflow:awaiting-merge`):
 ```bash
 PHASE_START_TIMESTAMP=$(date -u +%Y-%m-%dT%H:%M:%SZ)
 gh issue comment {NUMBER} {GH_FLAG} --body "<!-- FORGE:HEARTBEAT -->
@@ -748,7 +749,7 @@ case "$TIER" in
   adaptive|universal) bash "$SCRIPT_PATH" {NUMBER} {GH_FLAG} ready-to-build ;;
   prose)
     gh issue edit {NUMBER} {GH_FLAG} --add-label "workflow:ready-to-build" \
-      --remove-label "workflow:investigating,workflow:building,workflow:in-review,workflow:merged,workflow:invalid,workflow:decomposed" 2>/dev/null || true
+      --remove-label "workflow:investigating,workflow:building,workflow:in-review,workflow:awaiting-merge,workflow:merged,workflow:invalid,workflow:decomposed" 2>/dev/null || true
     ;;
 esac
 ```
@@ -797,7 +798,7 @@ case "$TIER" in
   adaptive|universal) bash "$SCRIPT_PATH" {NUMBER} {GH_FLAG} decomposed ;;
   prose)
     gh issue edit {NUMBER} {GH_FLAG} --add-label "workflow:decomposed" \
-      --remove-label "workflow:investigating,workflow:ready-to-build,workflow:building,workflow:in-review,workflow:merged,workflow:invalid" 2>/dev/null || true
+      --remove-label "workflow:investigating,workflow:ready-to-build,workflow:building,workflow:in-review,workflow:awaiting-merge,workflow:merged,workflow:invalid" 2>/dev/null || true
     ;;
 esac
 ```
@@ -821,7 +822,7 @@ case "$TIER" in
   adaptive|universal) bash "$SCRIPT_PATH" {NUMBER} {GH_FLAG} invalid ;;
   prose)
     gh issue edit {NUMBER} {GH_FLAG} --add-label "workflow:invalid" \
-      --remove-label "workflow:investigating,workflow:ready-to-build,workflow:building,workflow:in-review,workflow:merged,workflow:decomposed" 2>/dev/null || true
+      --remove-label "workflow:investigating,workflow:ready-to-build,workflow:building,workflow:in-review,workflow:awaiting-merge,workflow:merged,workflow:decomposed" 2>/dev/null || true
     ;;
 esac
 gh issue close {NUMBER} {GH_FLAG} --comment "Closing as invalid: {reason from investigation}"
@@ -927,7 +928,7 @@ case "$TIER" in
   adaptive|universal) bash "$SCRIPT_PATH" {NUMBER} {GH_FLAG} decomposed ;;
   prose)
     gh issue edit {NUMBER} {GH_FLAG} --add-label "workflow:decomposed" \
-      --remove-label "workflow:investigating,workflow:ready-to-build,workflow:building,workflow:in-review,workflow:merged,workflow:invalid" 2>/dev/null || true
+      --remove-label "workflow:investigating,workflow:ready-to-build,workflow:building,workflow:in-review,workflow:awaiting-merge,workflow:merged,workflow:invalid" 2>/dev/null || true
     ;;
 esac
 ```
@@ -958,7 +959,7 @@ fi
 
 ### 3A: Re-read state from GitHub (MANDATORY)
 
-**Post Phase 3 heartbeat** (skip if issue already has a terminal label — `workflow:merged`, `workflow:invalid`, `needs-human`):
+**Post Phase 3 heartbeat** (skip if issue already has a terminal label — `workflow:merged`, `workflow:invalid`, `needs-human`, `workflow:awaiting-merge`):
 ```bash
 PHASE_START_TIMESTAMP=$(date -u +%Y-%m-%dT%H:%M:%SZ)
 gh issue comment {NUMBER} {GH_FLAG} --body "<!-- FORGE:HEARTBEAT -->
@@ -1262,7 +1263,7 @@ case "$TIER" in
   adaptive|universal) bash "$SCRIPT_PATH" {NUMBER} {GH_FLAG} building ;;
   prose)
     gh issue edit {NUMBER} {GH_FLAG} --add-label "workflow:building" \
-      --remove-label "workflow:investigating,workflow:ready-to-build,workflow:in-review,workflow:merged,workflow:invalid,workflow:decomposed" 2>/dev/null || true
+      --remove-label "workflow:investigating,workflow:ready-to-build,workflow:in-review,workflow:awaiting-merge,workflow:merged,workflow:invalid,workflow:decomposed" 2>/dev/null || true
     ;;
 esac
 ```
@@ -1768,7 +1769,7 @@ case "$TIER" in
   adaptive|universal) bash "$SCRIPT_PATH" {NUMBER} {GH_FLAG} in-review ;;
   prose)
     gh issue edit {NUMBER} {GH_FLAG} --add-label "workflow:in-review" \
-      --remove-label "workflow:investigating,workflow:ready-to-build,workflow:building,workflow:merged,workflow:invalid,workflow:decomposed" 2>/dev/null || true
+      --remove-label "workflow:investigating,workflow:ready-to-build,workflow:building,workflow:awaiting-merge,workflow:merged,workflow:invalid,workflow:decomposed" 2>/dev/null || true
     ;;
 esac
 ```
@@ -1779,7 +1780,7 @@ esac
 
 ### 5A: Re-read state from GitHub (MANDATORY)
 
-**Post Phase 5 heartbeat** (skip if issue already has a terminal label — `workflow:merged`, `workflow:invalid`, `needs-human`):
+**Post Phase 5 heartbeat** (skip if issue already has a terminal label — `workflow:merged`, `workflow:invalid`, `needs-human`, `workflow:awaiting-merge`):
 ```bash
 PHASE_START_TIMESTAMP=$(date -u +%Y-%m-%dT%H:%M:%SZ)
 gh issue comment {NUMBER} {GH_FLAG} --body "<!-- FORGE:HEARTBEAT -->
@@ -1924,7 +1925,7 @@ case "$TIER" in
   adaptive|universal) bash "$SCRIPT_PATH" {NUMBER} {GH_FLAG} merged ;;
   prose)
     gh issue edit {NUMBER} {GH_FLAG} --add-label "workflow:merged" \
-      --remove-label "workflow:investigating,workflow:ready-to-build,workflow:building,workflow:in-review,workflow:invalid,workflow:decomposed" 2>/dev/null || true
+      --remove-label "workflow:investigating,workflow:ready-to-build,workflow:building,workflow:in-review,workflow:awaiting-merge,workflow:invalid,workflow:decomposed" 2>/dev/null || true
     ;;
 esac
 ```

package/docs/FORGE-PROTOCOL.md

@@ -13,3 +13,104 @@
 > and usable in any product with attribution. This older draft was mistakenly
 > marked AGPL-3.0; the protocol description itself has always been intended for
 > open, license-compatible adoption.
+
+---
+
+## Forge Ledger — Derived Artifact of the Protocol
+
+The **Forge Ledger** is the persistent, file/symbol-keyed knowledge index built from FORGE
+annotations. It is a _derived artifact_ of the protocol: the ledger is rebuilt by re-parsing
+the annotation corpus on GitHub (source of truth), not by maintaining separate authoritative
+state.
+
+### Architecture
+
+```
+GitHub Issues/PRs (annotation corpus, source of truth)
+          │
+          ▼
+scripts/build-knowledge-index.mjs   ← incremental indexer (watermarked by issue.updated_at)
+          │   uses packages/protocol/src/parse.js (FORGE annotation parser)
+          ▼
+~/.forge/index/
+  knowledge.jsonl    ← card records (one JSON per line, keyed by issue/kind/seq)
+  postings.json      ← BM25-lite inverted index (term → card IDs with per-field tf)
+  manifest.json      ← issue → content-hash map + watermark + card count
+  renames.jsonl      ← git rename log (old path → new path for query-time resolution)
+          │
+          ▼ (mirrored to orphan branch)
+forge-knowledge branch  ← versioned, diffable, hydratable by any clone or CI
+```
+
+### Card Schema
+
+Cards are keyed by provenance (`{issue}/{kind}/{seq}`), never by file path. Paths are
+attributes maintained by the indexer; renames are metadata updates, not re-keys.
+
+```json
+{
+  "schemaVersion": 1,
+  "id": "1370/pattern/0",
+  "kind": "pattern",
+  "issue": 1370,
+  "pattern": "...",
+  "rootCause": "...",
+  "prevention": "...",
+  "paths": ["bin/runner.mjs"],
+  "symbols": ["scrubEnv"],
+  "anchor": { "path": "bin/runner.mjs", "symbol": "scrubEnv" },
+  "status": "fresh",
+  "createdAt": "2026-07-04"
+}
+```
+
+### CLI
+
+```
+# Query by file path (exact lookup):
+forge recall --file bin/runner.mjs --json
+
+# Query by symbol:
+forge recall --symbol scrubEnv --json
+
+# Free-text BM25 search:
+forge recall "env scrub proc environ" --k 5
+
+# Index health:
+forge recall --doctor
+
+# Rebuild index from scratch:
+node scripts/build-knowledge-index.mjs --full-rebuild
+
+# Incremental update (used by close phase):
+node scripts/build-knowledge-index.mjs --issue {NUMBER}
+```
+
+### Invariants
+
+- **Full rebuild = incremental parity**: running a full rebuild produces byte-identical
+  card content to a sequence of incremental runs over the same issue range.
+- **Idempotency**: re-indexing the same issue twice produces the same card content (card IDs
+  are deterministic; content is a pure function of current annotation state).
+- **GitHub wins**: the index is always rebuildable from the annotation corpus alone. Deleting
+  `~/.forge/index/` loses nothing — a rebuild restores the same state.
+- **Non-blocking**: ledger indexing at close time never stalls the pipeline. Failure is logged
+  and the close phase continues.
+
+### Staleness Detection (tiered, cheapest first)
+
+1. Current blob sha of `anchor.path` matches stored blob sha → **FRESH**
+2. `anchor.symbol` is still defined in the current file → **FRESH**
+3. Rolling-hash scan finds `snippetHash` elsewhere in file → **MOVED** (re-anchor)
+4. `snippetHash` found in a file touched by rename log → **MOVED** (re-anchor path)
+5. Otherwise → **STALE** (excluded from binding injection; recallable with `--include-stale`)
+
+### Integration Points
+
+- **`commands/work-on/build/context.md` Phase C1**: uses `forge recall --file` for exact
+  file-path lookup; falls back to live `gh issue list --search` when index is absent.
+- **`commands/work-on/close.md` Phase C5.3**: calls `build-knowledge-index.mjs --issue`
+  after merge to index the just-closed issue; advances watermark.
+
+See issue [#1732](https://github.com/RapierCraftStudios/ForgeDock/issues/1732) for the
+implementation history and design decisions.

package/docs/spec/forge-protocol-v1.md

@@ -449,6 +449,43 @@ artifact so a sub-issue builder can fetch upstream context directly.
 <!-- FORGE:PRIOR_GIST: {https://example.com/artifact} -->
 ```
 
+#### Security: Gist Visibility — Normative Requirement <!-- Added: forge#1729 -->
+
+**All FORGE gist types (`FORGE:KNOWLEDGE_GIST`, `FORGE:MILESTONE_INDEX`, `FORGE:PRIOR_GIST`,
+and the memory-index Gist referenced by `FORGE:MEMORY_INDEX` in `close.md`) MUST be created
+secret. Publishing them as public Gists is a protocol violation.**
+
+Rationale: these artifacts embed investigation findings — root causes, file paths, affected
+service names, and security details. A public Gist exposes this content to a world-readable
+URL with no access control. `gh gist create` is secret by default; the requirement is that
+no pipeline spec or operator script must pass `--public` when creating any of these artifacts.
+
+**Enforcement**: The `pre-tool-use.mjs` hook (Rule 3) hard-blocks `gh gist create --public`
+in pipeline sessions. The `doctor-pipeline-state.sh` check I8 audits existing annotation-referenced
+gists and reports a CRITICAL finding if any are public.
+
+**Migration** (if a public gist is found): GitHub does not support changing a Gist's visibility
+in place. To replace a public pipeline gist with a secret one:
+
+```bash
+# 1. Save current content
+gh gist view {GIST_ID} > /tmp/gist_migrate.md
+[ -s /tmp/gist_migrate.md ] || { echo "ABORT: empty save"; exit 1; }
+
+# 2. Delete the public gist
+gh gist delete {GIST_ID} --yes
+
+# 3. Recreate secret (no --public) with same description
+gh gist create /tmp/gist_migrate.md --desc "{ORIGINAL_DESCRIPTION}"
+# gh gist create without --public creates a secret gist.
+
+# 4. Update the FORGE annotation on the issue to point to the new URL
+```
+
+**Override**: set `FORGE_ALLOW_PUBLIC_GIST=1` in the operator's shell environment before
+starting Claude Code only if a public gist is genuinely required (e.g., sharing a demo
+artifact). This must not be used for pipeline memory gists.
+
 ### 4.3 Control and error markers
 
 These markers carry no Markdown body; their presence is the signal.
@@ -1143,16 +1180,71 @@ separate, MIT-licensed npm package:
 **`@forgedock/protocol`** — available at `packages/protocol/` in the ForgeDock
 repository.
 
-It provides:
+### 13.1 Normative codec path
+
+**`packages/protocol/` is the normative write and read path for all FORGE
+annotations.** Hand-rolled annotation construction (embedding raw
+`<!-- FORGE:TYPE -->` strings in inline heredocs or `--body` blocks) is
+**non-conformant** because it bypasses the codec's escaping and round-trip
+guarantees. Every producer MUST route annotation construction through one of:
+
+1. **`forge-annotation.sh write <TYPE> --field KEY=VALUE ...`** (AGPL-licensed,
+   ships with the npm package at `scripts/forge-annotation.sh`), or
+2. **`node packages/protocol/src/cli.js emit <TYPE> --field KEY=VALUE ...`**
+   (MIT-licensed, part of the `@forgedock/protocol` package)
+
+Both paths delegate to `emit()` from `packages/protocol/src/emit.js`, which
+applies the complete escaping and sanitization policy (§3.5 injection hardening).
+
+### 13.2 JavaScript API
+
+```js
+import { parse, validate, emit, emitPartial } from '@forgedock/protocol';
+```
 
 - `parse(commentBody)` — extracts all annotations from a comment body string
 - `validate(annotation)` — checks conformance per §3–4; returns `{ valid, errors, warnings }`
-- `emit(type, fields)` — produces well-formed annotation strings
+- `emit(type, fields)` — produces a well-formed annotation string; applies all §3.5 escaping
 - `emitPartial(type)` — produces the partial sentinel for a type (§3.3)
 
-A conformance suite in `packages/protocol/fixtures/` covers all 13 reserved
-types with valid and invalid examples. The CLI runner can be invoked against any
-fixtures directory:
+### 13.3 CLI subcommands
+
+The CLI at `packages/protocol/src/cli.js` exposes three modes:
+
+```bash
+# Emit — print a well-formed annotation body to stdout
+node packages/protocol/src/cli.js emit INVESTIGATOR \
+  --field Verdict=CONFIRMED --field Confidence=HIGH \
+  --field Severity=MEDIUM --field "Task Type=Bug Fix" \
+  --field "Decomposition Assessment=NO — single fix."
+
+# Parse — read a comment body from stdin, extract a field value
+echo '<!-- FORGE:INVESTIGATOR -->...' | \
+  node packages/protocol/src/cli.js parse --type INVESTIGATOR --field Verdict
+# → CONFIRMED
+
+# Conformance runner (legacy — no subcommand keyword)
+node packages/protocol/src/cli.js fixtures/
+```
+
+The `emit` subcommand also supports the Base64url machine-surface form for
+`FORGE:CARD` annotations (design decision 2026-07-08: encoding beats escaping):
+
+```bash
+node packages/protocol/src/cli.js emit CARD \
+  --field issue=1370 --field status=merged --b64
+# → <!-- FORGE:CARD: v1 sha:7234c2d8 b64:eyJ... -->
+```
+
+The Base64url alphabet `[A-Za-z0-9_-]` cannot contain `>`, `<`, or `!`, making
+`-->`, `--!>`, and `<!--` structurally unrepresentable in the machine surface.
+
+### 13.4 Conformance suite
+
+A conformance suite in `packages/protocol/fixtures/` covers all reserved types
+with valid and invalid examples, including adversarial payloads that reproduce
+historical escaping vulnerabilities. The suite can be run against any fixtures
+directory:
 
 ```bash
 npx @forgedock/protocol fixtures/

package/docs/spec/label-state-machine.md

@@ -14,11 +14,14 @@ See also: [FORGE Annotation Protocol §6](forge-protocol-v1.md#6-label-state-mac
 | `workflow:ready-to-build` | Investigation complete, build not started | Investigator agent |
 | `workflow:building` | Build phase active | Builder agent |
 | `workflow:in-review` | PR created, review active | Orchestrator agent |
+| `workflow:awaiting-merge` | Remediated + re-reviewed, awaiting a human merge decision | Review-pr Phase 8 (auto-merge guard) |
 | `workflow:merged` | PR merged, issue closed | Close phase agent |
 | `workflow:invalid` | Issue closed as invalid | Investigator agent |
 | `workflow:decomposed` | Issue decomposed into sub-issues | Decomposer agent |
 | `needs-human` | Pipeline blocked, human intervention required | Any agent on error |
 
+`workflow:awaiting-merge` vs `needs-human`: both are terminal (the pipeline stops advancing the issue automatically), but they mean different things. `needs-human` means the pipeline hit a condition it cannot resolve on its own (genuinely blocked — conflicting PR, failed verdict, calibration/trust escalation, etc.) and a human must diagnose and act. `workflow:awaiting-merge` means the opposite: the PR was previously escalated to `needs-human`, has since been remediated and re-reviewed to a clean `APPROVED` verdict with no mergeability blockers, but does not yet meet the automated auto-land bar (see forge#1809 Q1) — a human only needs to click merge, not diagnose a problem. `scripts/transition-label.sh` clears `needs-human` (best-effort) on every transition, so moving an issue onto `workflow:awaiting-merge` also clears any stale `needs-human` label.
+
 ## Transitions
 
 ```
@@ -27,6 +30,7 @@ See also: [FORGE Annotation Protocol §6](forge-protocol-v1.md#6-label-state-mac
   → workflow:ready-to-build     [Phase 2: Investigation complete]
   → workflow:building           [Phase 3: Build started]
   → workflow:in-review          [Phase 4: PR created]
+  → workflow:awaiting-merge [TERMINAL] [Phase 5/review-pr: re-reviewed after needs-human, awaiting human merge]
   → workflow:merged  [TERMINAL] [Phase 5: PR merged]
   → workflow:invalid [TERMINAL] [Any phase: closed as invalid]
   → workflow:decomposed [TERMINAL] [Phase 2: split into sub-issues]
@@ -40,6 +44,7 @@ Processing stops when any of these labels is set:
 - `workflow:merged`
 - `workflow:invalid`
 - `workflow:decomposed`
+- `workflow:awaiting-merge`
 - `needs-human`
 
 ## Label Exclusivity
@@ -49,7 +54,7 @@ At most one `workflow:*` label should be active on an issue at any time. When tr
 ```bash
 gh issue edit {NUMBER} -R {REPO} \
   --add-label "workflow:building" \
-  --remove-label "workflow:investigating,workflow:ready-to-build,workflow:in-review,workflow:merged,workflow:invalid,workflow:decomposed"
+  --remove-label "workflow:investigating,workflow:ready-to-build,workflow:in-review,workflow:awaiting-merge,workflow:merged,workflow:invalid,workflow:decomposed"
 ```
 
 This is enforced by `scripts/transition-label.sh` — use `resolve_script 'transition-label'` rather than calling `gh issue edit` directly when possible.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "forgedock",
-  "version": "1.1.7",
+  "version": "1.1.8",
   "description": "Autonomous dev pipeline for Claude Code. GitHub as institutional memory — every issue, PR, and annotation survives context resets and makes the next agent smarter.",
   "bin": {
     "forgedock": "bin/forgedock.mjs"
@@ -45,7 +45,7 @@
     "README.md"
   ],
   "scripts": {
-    "test": "node --test \"bin/tests/**/*.test.mjs\"",
+    "test": "node --test \"bin/tests/**/*.test.mjs\" \"packages/protocol/test/**/*.test.mjs\"",
     "docs:dev": "vitepress dev docs/site",
     "docs:build": "vitepress build docs/site",
     "docs:preview": "vitepress preview docs/site"