forgedock 1.1.6 → 1.1.8

package/commands/orchestrate/phase-6-report.md

@@ -21,20 +21,122 @@ done
 
 Aggregate into the batch-level analytics for Step 6B.
 
-**Also collect the machine-readable summary cards** (`<!-- FORGE:CARD {json} -->`, embedded in
-each issue's `FORGE:TRAJECTORY` comment by `/work-on` close phase). These power the per-issue and
-batch cards in Step 6C:
+**Also collect the machine-readable summary cards** (embedded in each issue's `FORGE:TRAJECTORY`
+comment by `/work-on` close phase as a `<!-- FORGE:CARD: v1 sha:... b64:... -->` line). These
+power the per-issue and batch cards in Step 6C.
+
+**CODEC PATH (forge#1727)**: Cards are now Base64url-encoded. Use the protocol CLI's `parse`
+subcommand to decode them — do NOT use the old `sed -n 's/.*<!-- FORGE:CARD \(.*\) -->.*/\1/p'`
+regex extraction, which only worked for the deprecated inline-JSON form:
 
 ```bash
 CARDS=""
 for NUM in {all_completed_issue_numbers}; do
-  CARD=$(gh api repos/{GH_REPO}/issues/${NUM}/comments \
-    --jq '.[] | select(.body | contains("FORGE:CARD")) | .body' 2>/dev/null \
-    | sed -n 's/.*<!-- FORGE:CARD \(.*\) -->.*/\1/p' | head -1)
-  [ -n "$CARD" ] && CARDS="${CARDS}${CARD}"$'\n'
+  # Fetch the TRAJECTORY comment body for this issue
+  TRAJ_BODY=$(gh api repos/{GH_REPO}/issues/${NUM}/comments \
+    --jq '[.[] | select(.body | contains("FORGE:CARD:"))] | last | .body // ""' 2>/dev/null || true)
+  if [ -n "$TRAJ_BODY" ]; then
+    # Decode via protocol CLI — handles both Base64url form (forge#1727) and gracefully
+    # exits 1 (skips) for pre-migration inline-JSON entries.
+    CARD=$(echo "$TRAJ_BODY" | node packages/protocol/src/cli.js parse --type CARD 2>/dev/null || true)
+    [ -n "$CARD" ] && CARDS="${CARDS}${CARD}"$'\n'
+  fi
+done
+# CARDS is now a newline-delimited list of per-issue JSON objects decoded from FORGE:CARD payloads.
+# Skip any issue whose card is absent — pre-card runs or non-merged terminal states degrade gracefully.
+```
+
+### Step 6A.5: Collect merge-ready PRs (`workflow:awaiting-merge`) <!-- Added: forge#1811 -->
+
+**Why a separate collection pass**: `workflow:awaiting-merge` issues are open (their PR hasn't
+merged yet), so they are excluded from `{all_completed_issue_numbers}` above — that set only
+contains issues that reached a closed/merged terminal state. Iterate the full batch issue set
+(`{all_batch_issue_numbers}` — same prose-variable used by the batch-wide loops in
+`phase-4-execution.md`) instead, and filter to the awaiting-merge label.
+
+For each awaiting-merge issue, resolve its open PR using the `"Closes #N"`-anchored search
+established by forge#1634/#1646 — **do not** use a bare-number `search "${NUM} in:body"`, which
+can false-positive on changelogs or cross-references and resolve the wrong PR:
+
+```bash
+MERGE_READY_PRS=()   # each entry: "NUM|PR_NUMBER|TITLE"
+
+for NUM in {all_batch_issue_numbers}; do
+  IS_AWAITING=$(gh issue view "$NUM" -R {GH_REPO} --json labels \
+    --jq '[.labels[].name | select(. == "workflow:awaiting-merge")] | length' 2>/dev/null || echo "0")
+  [ "$IS_AWAITING" -gt 0 ] || continue
+
+  TITLE=$(gh issue view "$NUM" -R {GH_REPO} --json title --jq '.title' 2>/dev/null || echo "")
+
+  # Anchor on "Closes #N" first (forge#1634/#1646 precedent).
+  PR_NUM=$(gh pr list -R {GH_REPO} --state open --search "\"Closes #${NUM}\" in:body" \
+    --json number,updatedAt --jq 'sort_by(.updatedAt) | last | .number' 2>/dev/null || echo "")
+  if [ -z "$PR_NUM" ]; then
+    # Fallback: a PR may close the issue via "Fixes #N"/"Resolves #N" rather than
+    # "Closes #N". Use the bare number only to generate candidates, then RE-VALIDATE
+    # each candidate's body against an anchored (Closes|Fixes|Resolves) #N word-boundary
+    # regex before accepting it — never accept a bare mention (changelog / cross-reference),
+    # which could resolve the wrong PR into the one-shot batch-merge command. This mirrors
+    # the recover-orphans.md re-validation precedent. <!-- forge#1822 -->
+    PR_NUM=$(gh pr list -R {GH_REPO} --state open --search "${NUM} in:body" \
+      --json number,updatedAt,body \
+      --jq --arg n "${NUM}" '[ .[] | select(.body | test("(?i)(close[sd]?|fix(e[sd])?|resolve[sd]?)\\s+#" + $n + "\\b")) ] | sort_by(.updatedAt) | last | .number // empty' 2>/dev/null || echo "")
+  fi
+
+  [ -n "$PR_NUM" ] && MERGE_READY_PRS+=("${NUM}|${PR_NUM}|${TITLE}")
 done
-# CARDS is now a newline-delimited list of per-issue JSON objects (skip any issue whose
-# card is absent — pre-card runs or non-merged terminal states degrade gracefully).
+# MERGE_READY_PRS now holds one entry per merge-ready PR in the batch (empty array if none).
+# Feeds the "### Merge-Ready" report block and the Implementation Results table row (Step 6B).
+
+# Pre-build the batch-merge command (rendered verbatim as {MERGE_READY_CMD} in Step 6B —
+# computed here, not inside the report template, so Step 6B stays plain template text with no
+# nested code fences of its own). `gh pr merge` accepts only ONE PR selector per invocation,
+# so chain one `gh pr merge` per PR with && rather than passing a space-separated list (which
+# `gh` rejects). <!-- forge#1838 review: BUG-3 -->
+MERGE_READY_CMD=""
+if [ "${#MERGE_READY_PRS[@]}" -gt 0 ]; then
+  for _entry in "${MERGE_READY_PRS[@]}"; do
+    _pr=$(printf '%s' "$_entry" | cut -d'|' -f2)
+    [ -z "$_pr" ] && continue
+    [ -n "$MERGE_READY_CMD" ] && MERGE_READY_CMD="${MERGE_READY_CMD} && "
+    MERGE_READY_CMD="${MERGE_READY_CMD}gh pr merge ${_pr} --merge -R {GH_REPO}"
+  done
+fi
+```
+
+### Step 6A.6: Collect blocked-on-human-merge dependents (`blocked-on-human-merge`) <!-- Added: forge#1812 -->
+
+**Why a separate collection pass**: A `blocked-on-human-merge` issue is a *dependent* of a `GATED`
+predecessor (`needs-human` or `workflow:awaiting-merge` — see `phase-4-execution.md` Step 4B's
+Predecessor Classification), not itself blocked or failed. It has no PR of its own yet — it hasn't
+even been dispatched. It is excluded from both `{all_completed_issue_numbers}` (never ran) and
+`MERGE_READY_PRS` (Step 6A.5, which only covers issues that themselves reached `workflow:awaiting-merge`).
+Reporting it as `⏭ Skipped (dep)` would be wrong — that label means the predecessor `FAILED` and the
+work is abandoned; this state means the work is queued and will auto-dispatch on merge, no manual
+re-run required.
+
+```bash
+BLOCKED_ON_MERGE=()   # each entry: "NUM|GATING_PRED|GATING_PR|TITLE"
+
+for NUM in {all_batch_issue_numbers}; do
+  IS_BLOCKED=$(gh issue view "$NUM" -R {GH_REPO} --json labels \
+    --jq '[.labels[].name | select(. == "blocked-on-human-merge")] | length' 2>/dev/null || echo "0")
+  [ "$IS_BLOCKED" -gt 0 ] || continue
+
+  TITLE=$(gh issue view "$NUM" -R {GH_REPO} --json title --jq '.title' 2>/dev/null || echo "")
+  GATING_PRED=$(gh api repos/{GH_REPO}/issues/${NUM}/comments \
+    --jq '[.[] | select(.body | contains("FORGE:BLOCKED_ON_HUMAN_MERGE"))] | last | (.body | capture("Gating predecessor\\*\\*: #(?<p>[0-9]+)").p) // ""' 2>/dev/null || echo "")
+  GATING_PR=""
+  if [ -n "$GATING_PRED" ]; then
+    GATING_PR=$(gh pr list -R {GH_REPO} --state open --search "\"Closes #${GATING_PRED}\" in:body" \
+      --json number --jq '.[0].number // empty' 2>/dev/null || echo "")
+  fi
+
+  BLOCKED_ON_MERGE+=("${NUM}|${GATING_PRED}|${GATING_PR}|${TITLE}")
+done
+# BLOCKED_ON_MERGE now holds one entry per dependent queued behind a human-gated predecessor
+# (empty array if none). Feeds the "Blocked-on-Merge" report block and the Implementation
+# Results table row (Step 6B).
 ```
 
 ### Step 6B: Present consolidated report
@@ -61,7 +163,62 @@ done
 | #{N1} | {title} | from #{INV1} | ✓ Merged | #{PR} | staging |
 | #{C} | {title} | original | ✗ Invalid | — | — |
 | #{D} | {title} | original | ⚠ Blocked | — | — |
+| #{F} | {title} | original | ⏸ Awaiting Merge | #{PR} | staging |
 | #{E} | {title} | original | ⏭ Skipped (dep) | — | — |
+| #{G} | {title} | original | 🔗 Blocked-on-Merge (gated by #{PRED}) | — | — |
+
+`⏸ Awaiting Merge` (`workflow:awaiting-merge`) is structurally distinct from `⚠ Blocked`
+(`needs-human`): a Blocked row means the pipeline hit something it cannot resolve on its own
+and a human must diagnose it; an Awaiting Merge row means the PR was already remediated and
+re-reviewed to a clean APPROVED verdict — a human only needs to click merge. Never render an
+awaiting-merge PR as `⚠ Blocked`.
+
+`🔗 Blocked-on-Merge` (`blocked-on-human-merge`, forge#1812) is distinct from all three of the
+above: it is a *dependent* of a GATED predecessor (one currently `⚠ Blocked` or `⏸ Awaiting
+Merge`) — the dependent issue itself was never dispatched and has no problem of its own. It is
+also distinct from `⏭ Skipped (dep)`, which means the predecessor `FAILED` outright and the work
+was abandoned. A Blocked-on-Merge row means the work is queued and will auto-dispatch the instant
+the gating predecessor's PR merges — no manual `/orchestrate` re-run required. Never render a
+blocked-on-human-merge dependent as `⏭ Skipped (dep)`.
+
+{IF `MERGE_READY_PRS` (Step 6A.5) is non-empty:}
+
+### Merge-Ready — {N} PR(s) awaiting only a human merge
+
+These PRs were remediated and re-reviewed to a clean `APPROVED` verdict after an earlier
+`needs-human` escalation. They do not need further diagnosis — merge them to land the batch.
+
+| # | Issue | PR | Title |
+|---|-------|----|----|
+{one row per `MERGE_READY_PRS` entry: | {n} | #{NUM} | #{PR} | {title} |}
+
+**One action to land all {N}:** `{MERGE_READY_CMD}`
+
+(`{MERGE_READY_CMD}` is pre-built in Step 6A.5 by chaining one `gh pr merge <pr> --merge` per PR
+in `MERGE_READY_PRS` with `&&` — `gh pr merge` takes a single PR selector, so a space-separated
+list would be rejected. Render it verbatim here as plain text, no code fence.)
+
+{ELSE: omit this section entirely — do not print an empty "Merge-Ready" heading.}
+
+{IF `BLOCKED_ON_MERGE` (Step 6A.6) is non-empty:}
+
+### Blocked-on-Merge — {N} issue(s) queued behind a human-gated predecessor
+
+These issues are dependents of a predecessor that is currently `⚠ Blocked` or `⏸ Awaiting Merge`.
+They were never dispatched — no work has started on them — and they need no action right now.
+Each will auto-dispatch the instant its gating predecessor's PR merges (live-session wake via
+`phase-4-execution.md` Step 4B item 6.6, or next-`/orchestrate`-invocation wake via
+`phase-3-dependency.md`'s wake/compaction reconstruction). This batch is a **paused drain**, not
+a complete one — re-running `/orchestrate` after merging the gating PR(s) below will pick these up.
+
+| # | Issue | Gated By | Gating PR | Title |
+|---|-------|----------|-----------|-------|
+{one row per `BLOCKED_ON_MERGE` entry: | {n} | #{NUM} | #{GATING_PRED} | #{GATING_PR} | {title} |}
+
+**To unblock**: merge the gating PR(s) referenced above, then re-run `/orchestrate` (or let the
+current session's live wake pick them up automatically if it is still running).
+
+{ELSE: omit this section entirely — do not print an empty "Blocked-on-Merge" heading.}
 
 ### Review-Spawned Issues
 
@@ -91,6 +248,8 @@ done
 - **Review findings**: {N} spawned, {M} resolved in-batch, {K} swept, {J} deferred (cosmetic), {L} deferred (gen2)
 - **Succeeded**: {N} issues resolved (implementation + sweep)
 - **Failed**: {N} issues need attention
+- **Merge-ready**: {#MERGE_READY_PRS[@]:-0} PRs awaiting only a human merge (see "Merge-Ready" section above)
+- **Blocked-on-merge**: {#BLOCKED_ON_MERGE[@]:-0} issues queued behind a human-gated predecessor, will auto-dispatch on merge (see "Blocked-on-Merge" section above)
 - **Skipped**: {N} issues (dependency failures)
 
 ### Post-Batch Cleanup
@@ -124,6 +283,35 @@ done
 | Findings validated | {N} |
 | False positives | {N} ({%}) |
 | Anomalies flagged | {N} |
+| Budget limit | ${BUDGET_LIMIT:-uncapped} |
+| Projected spend (dispatched issues) | ${PROJECTED_SPEND:-—} |
+| Actual spend (best-effort telemetry) | ${ACTUAL_SPEND:-—} |
+| Issues deferred (budget ceiling) | {#DEFERRED_BUDGET_ISSUES[@]:-0} |
+| ε-reserve issued (no-prior dispatch) | {EPSILON_DISPATCHED:-no} |
+| **Value-weighted throughput** | **{VALUE_THROUGHPUT:-—} value-pts/USD** |
+
+`value-weighted throughput` = Σ(priority_weight × danger_zone_weight) for **merged** issues ÷ actual spend (USD). A higher value means more high-priority issues were resolved per dollar. Trend this across runs via `/pipeline-health` to measure the economic scheduling ROI. <!-- Added: forge#1743 -->
+
+**Compute value-weighted throughput** (populate before rendering the table):
+
+```bash
+# Collect value scores for merged issues (ISSUE_VALUE[] populated in Step 3E.5)
+MERGED_VALUE_SUM="0"
+for NUM in {all_completed_issue_numbers}; do
+  IS_MERGED=$(gh issue view "$NUM" -R {GH_REPO} --json labels \
+    --jq '[.labels[].name | select(. == "workflow:merged")] | length' 2>/dev/null || echo "0")
+  if [ "$IS_MERGED" -gt 0 ] && [ -n "${ISSUE_VALUE[$NUM]:-}" ]; then
+    MERGED_VALUE_SUM=$(echo "scale=4; $MERGED_VALUE_SUM + ${ISSUE_VALUE[$NUM]}" | bc 2>/dev/null \
+      || echo "$MERGED_VALUE_SUM")
+  fi
+done
+
+if [ "${ACTUAL_SPEND:-0}" != "0" ] && echo "${ACTUAL_SPEND:-0}" | grep -qP '^\d+(\.\d+)?$'; then
+  VALUE_THROUGHPUT=$(echo "scale=2; $MERGED_VALUE_SUM / $ACTUAL_SPEND" | bc 2>/dev/null || echo "—")
+else
+  VALUE_THROUGHPUT="—"  # no spend telemetry available
+fi
+```
 
 **Domain breakdown**: {N} scraping, {N} frontend, {N} billing, ...
 **Routing**: {N} fast-lane, {N} feature-lane
@@ -137,6 +325,7 @@ done
 - Contract divergences > 20% → investigation quality may need improvement
 - **Idle% > 50%** → agents stalling at phase boundaries; check work-on routing loop
 - **Avg resumes > 1** → orchestrator having to compensate for agent stops
+- **Value-weighted throughput trending down** → high-value issues deferred or blocked; check Step 3E.5 score distribution
 
 ### Next Steps
 {If milestone and all issues done: "Milestone ready to ship. Run `/milestone ship {slug}` when ready. The ship command includes a pre-merge hunk-loss audit (Step 2.5) that detects staging-only hunks in milestone-modified files and rebases the milestone branch to absorb them before creating the PR — protecting against squash-merge regressions."}

package/commands/pipeline-health/phase-2-metrics.md

@@ -45,7 +45,70 @@ gh issue list -R $REPO --state closed --label "review-finding" --json number,cre
   --jq "[.[] | select(.createdAt > \"$SINCE\")] | length"
 ```
 
-**False positive rate** = invalid / total closed. Target: < 10%.
+**False positive rate (proxy)** = invalid / total closed. Target: < 10%.
+
+**Note**: This proxy uses `workflow:invalid` as a stand-in for false-positive. Phase 2C.5 reports the actual false-positive rate from the `false-positive` label. Both metrics should be reported; they should converge as validation coverage increases.
+
+### 2C.5: Validation coverage (finding lifecycle health) <!-- Added: forge#1730 -->
+
+Measures whether the `needs-validation → validated/false-positive` lifecycle is operating. This is the primary health signal for the review system's calibration corpus.
+
+```bash
+# All finding-issue types: review-finding, staging-review, audit-finding
+FINDING_LABELS="review-finding staging-review audit-finding"
+
+TOTAL_FINDINGS=0
+TOTAL_VALIDATED=0
+TOTAL_FALSE_POSITIVE=0
+TOTAL_NEEDS_VALIDATION=0
+
+for LABEL in $FINDING_LABELS; do
+  COUNT=$(gh issue list -R $REPO --state all --label "$LABEL" --limit 1000 --json number \
+    --jq 'length' 2>/dev/null || echo 0)
+  TOTAL_FINDINGS=$((TOTAL_FINDINGS + COUNT))
+
+  VAL=$(gh issue list -R $REPO --state all --label "$LABEL,validated" --limit 1000 --json number \
+    --jq 'length' 2>/dev/null || echo 0)
+  TOTAL_VALIDATED=$((TOTAL_VALIDATED + VAL))
+
+  FP=$(gh issue list -R $REPO --state all --label "$LABEL,false-positive" --limit 1000 --json number \
+    --jq 'length' 2>/dev/null || echo 0)
+  TOTAL_FALSE_POSITIVE=$((TOTAL_FALSE_POSITIVE + FP))
+
+  NV=$(gh issue list -R $REPO --state all --label "$LABEL,needs-validation" --limit 1000 --json number \
+    --jq 'length' 2>/dev/null || echo 0)
+  TOTAL_NEEDS_VALIDATION=$((TOTAL_NEEDS_VALIDATION + NV))
+done
+
+TOTAL_RESOLVED=$((TOTAL_VALIDATED + TOTAL_FALSE_POSITIVE))
+if [ "$TOTAL_FINDINGS" -gt 0 ]; then
+  VALIDATION_COVERAGE_PCT=$(echo "scale=1; $TOTAL_RESOLVED * 100 / $TOTAL_FINDINGS" | bc 2>/dev/null || echo "N/A")
+  if [ "$TOTAL_RESOLVED" -gt 0 ]; then
+    FALSE_POSITIVE_RATE_PCT=$(echo "scale=1; $TOTAL_FALSE_POSITIVE * 100 / $TOTAL_RESOLVED" | bc 2>/dev/null || echo "N/A")
+  else
+    FALSE_POSITIVE_RATE_PCT="N/A (no resolved findings)"
+  fi
+else
+  VALIDATION_COVERAGE_PCT="N/A"
+  FALSE_POSITIVE_RATE_PCT="N/A"
+fi
+
+echo "=== Validation Coverage (2C.5) ==="
+echo "Total finding-issues:      $TOTAL_FINDINGS (review-finding + staging-review + audit-finding)"
+echo "  validated:               $TOTAL_VALIDATED"
+echo "  false-positive:          $TOTAL_FALSE_POSITIVE"
+echo "  resolved (total):        $TOTAL_RESOLVED"
+echo "  needs-validation (open): $TOTAL_NEEDS_VALIDATION"
+echo "Validation coverage:       ${VALIDATION_COVERAGE_PCT}% (target: ≥ 95% for newly closed findings)"
+echo "False-positive rate:       ${FALSE_POSITIVE_RATE_PCT}% of resolved findings (actual label-based rate)"
+```
+
+**Metrics to compute**:
+- **Validation coverage**: (validated + false-positive) / total finding-issues. Target: ≥ 95% for newly closed findings.
+- **False-positive rate (actual)**: false-positive / (validated + false-positive). Reports true negative rate, not the `workflow:invalid` proxy from 2C.
+- **Unresolved finding count**: Issues still holding `needs-validation` — actionable target for `/cleanup labels` backfill.
+
+**Why this matters**: The `needs-validation` label was previously a dead end — applied at creation, never resolved. With the lifecycle wired (#1730), these metrics become the primary signal for review system calibration quality. Validation coverage < 95% indicates pipeline runs are not completing the verdict transition; false-positive rate is now a reportable number instead of undefined.
 
 ### 2D: Build failure rate
 
@@ -494,6 +557,136 @@ gh issue list -R $FORGE_REPO --state all --label "audit-finding" \
 - **Failure category distribution**: breakdown by failure point (INVESTIGATION, IMPLEMENTATION, REVIEW, REVIEW_FALSE_NEG, DEPLOY_GATE, etc.). No target — use for diagnosis. High IMPLEMENTATION counts indicate builder gaps; high REVIEW_FALSE_NEG counts indicate review agent gaps.
 - **Mean time to detect (MTD)**: average days from PR merge to audit-finding creation. Target: < 7 days. Long MTD means defects linger undetected in production.
 
+### 2M.5: Review intensity distribution and escaped-defect rate by tier <!-- Added: forge#1745 -->
+
+This metric is only meaningful once `scripts/calibration.mjs --provenance` has been publishing to the `forge-knowledge` branch for at least one 30-day shadow-mode window. Before that point, all PRs have `INTENSITY_TIER=SHADOW` in their FORGE:REVIEW annotations and both tier buckets will be empty — this is expected and correct.
+
+**Why this matters**: Review spend should track risk. This metric verifies that the trust escalation system is working as intended: proven cells should have lower escaped-defect rates than novel cells, and overall review token cost per PR should trend down while escaped-defect rate stays flat or falls.
+
+**Success metric**: Review token cost per PR trending down (fewer agents on proven PRs) while escaped-defect rate (14-day post-merge review-findings) stays flat or falls; zero escapes attributable to a dropped agent that the full panel would have run.
+
+```bash
+# Collect FORGE:REVIEW annotation INTENSITY_TIER fields from merged PRs in the analysis window.
+# Source: the APPROVED:/CHANGES REQUESTED: PR comment from review-pr.md Phase 7B.
+# Field format (set since forge#1745): "**Intensity tier**: PROVEN | **Cell**: `...` | **Shadow mode**: false"
+#
+# This relies on MERGED_PRS (list of PR numbers) being available from Phase 2E.
+# If not available, fall back to gh pr list.
+if [ -z "${MERGED_PRS:-}" ]; then
+  MERGED_PRS=$(gh pr list -R "$FORGE_REPO" --state merged --limit 200 \
+    --json number,mergedAt \
+    --jq "[.[] | select(.mergedAt > \"$SINCE\")] | .[].number" 2>/dev/null || echo "")
+fi
+
+# Accumulators per intensity tier
+TIER_NOVEL_COUNT=0
+TIER_PROVEN_COUNT=0
+TIER_SHADOW_COUNT=0
+TIER_NOVEL_NEEDS_HUMAN_COUNT=0
+TIER_NOVEL_ESCAPES=0
+TIER_PROVEN_ESCAPES=0
+
+# Also track escaped defect counts per tier using the audit-finding ledger from 2L.
+# AUDIT_FINDINGS must be set from Phase 2L (contains audit-finding issues in window).
+AUDIT_FINDINGS_AVAILABLE="${AUDIT_FINDINGS:+true}"
+
+while IFS= read -r PR_NUM; do
+  [ -z "$PR_NUM" ] && continue
+
+  # Read the FORGE:REVIEW comment from this PR to extract INTENSITY_TIER
+  REVIEW_COMMENT=$(gh api "repos/${FORGE_REPO}/issues/${PR_NUM}/comments" \
+    --jq '[.[] | select(
+      (.body | test("APPROVED:|CHANGES REQUESTED:"; "i")) and
+      (.body | contains("Intensity tier"))
+    )] | last | .body // ""' 2>/dev/null || echo "")
+
+  if [ -z "$REVIEW_COMMENT" ]; then
+    TIER_SHADOW_COUNT=$((TIER_SHADOW_COUNT + 1))
+    continue
+  fi
+
+  PR_TIER=$(echo "$REVIEW_COMMENT" | grep -oP '(?<=\*\*Intensity tier\*\*: )[A-Z_]+' | head -1 || echo "SHADOW")
+  case "$PR_TIER" in
+    PROVEN)              TIER_PROVEN_COUNT=$((TIER_PROVEN_COUNT + 1)) ;;
+    NOVEL_NEEDS_HUMAN)   TIER_NOVEL_NEEDS_HUMAN_COUNT=$((TIER_NOVEL_NEEDS_HUMAN_COUNT + 1))
+                         TIER_NOVEL_COUNT=$((TIER_NOVEL_COUNT + 1)) ;;
+    NOVEL)               TIER_NOVEL_COUNT=$((TIER_NOVEL_COUNT + 1)) ;;
+    *)                   TIER_SHADOW_COUNT=$((TIER_SHADOW_COUNT + 1)) ;;
+  esac
+
+  # Cross-reference escaped defects for this PR's files (from 2L AUDIT_FINDINGS)
+  # A REVIEW_FALSE_NEG audit-finding referencing PR #{PR_NUM} counts as an escape for this tier.
+  if [ "$AUDIT_FINDINGS_AVAILABLE" = "true" ]; then
+    ESCAPE_COUNT=$(echo "$AUDIT_FINDINGS" | \
+      jq --argjson pr "$PR_NUM" '[.[] | select(
+        (.body | test("\\*\\*Failure point\\*\\*:.*REVIEW_FALSE_NEG")) and
+        (.body | test("#" + ($pr | tostring)))
+      )] | length' 2>/dev/null || echo "0")
+
+    if [ "${ESCAPE_COUNT:-0}" -gt 0 ]; then
+      case "$PR_TIER" in
+        PROVEN) TIER_PROVEN_ESCAPES=$((TIER_PROVEN_ESCAPES + ESCAPE_COUNT)) ;;
+        NOVEL|NOVEL_NEEDS_HUMAN) TIER_NOVEL_ESCAPES=$((TIER_NOVEL_ESCAPES + ESCAPE_COUNT)) ;;
+      esac
+    fi
+  fi
+done <<< "$MERGED_PRS"
+
+TIER_TOTAL=$((TIER_NOVEL_COUNT + TIER_PROVEN_COUNT + TIER_SHADOW_COUNT + TIER_NOVEL_NEEDS_HUMAN_COUNT))
+
+echo ""
+echo "=== Review Intensity Distribution (forge#1745 — provenance trust) ==="
+echo "Window: ${SINCE} → now | Total PRs: ${TIER_TOTAL}"
+echo ""
+echo "Tier distribution:"
+printf "  %-22s %4d  (%s%%)\n" "PROVEN (de-escalated):" "$TIER_PROVEN_COUNT" \
+  "$([ "$TIER_TOTAL" -gt 0 ] && echo "scale=0; $TIER_PROVEN_COUNT * 100 / $TIER_TOTAL" | bc || echo "0")"
+printf "  %-22s %4d  (%s%%)\n" "NOVEL (full panel):" "$TIER_NOVEL_COUNT" \
+  "$([ "$TIER_TOTAL" -gt 0 ] && echo "scale=0; $TIER_NOVEL_COUNT * 100 / $TIER_TOTAL" | bc || echo "0")"
+printf "  %-22s %4d  (%s%%)\n" "SHADOW (no action yet):" "$TIER_SHADOW_COUNT" \
+  "$([ "$TIER_TOTAL" -gt 0 ] && echo "scale=0; $TIER_SHADOW_COUNT * 100 / $TIER_TOTAL" | bc || echo "0")"
+
+echo ""
+echo "Escaped-defect rate by tier (REVIEW_FALSE_NEG audit-findings):"
+if [ "$TIER_PROVEN_COUNT" -gt 0 ]; then
+  PROVEN_ESCAPE_RATE=$(echo "scale=1; $TIER_PROVEN_ESCAPES * 100 / $TIER_PROVEN_COUNT" | bc 2>/dev/null || echo "N/A")
+  echo "  PROVEN: ${TIER_PROVEN_ESCAPES} escapes / ${TIER_PROVEN_COUNT} PRs = ${PROVEN_ESCAPE_RATE}% (target: ≤ review-wide escape rate)"
+else
+  echo "  PROVEN: no PRs in this tier yet"
+fi
+if [ "$TIER_NOVEL_COUNT" -gt 0 ]; then
+  NOVEL_ESCAPE_RATE=$(echo "scale=1; $TIER_NOVEL_ESCAPES * 100 / $TIER_NOVEL_COUNT" | bc 2>/dev/null || echo "N/A")
+  echo "  NOVEL:  ${TIER_NOVEL_ESCAPES} escapes / ${TIER_NOVEL_COUNT} PRs = ${NOVEL_ESCAPE_RATE}%"
+else
+  echo "  NOVEL: no PRs in this tier yet"
+fi
+
+# Health signal: if PROVEN escape rate > NOVEL escape rate, the de-escalation is leaking.
+# This should never happen in steady state — flag it if it does.
+if [ "$TIER_PROVEN_COUNT" -gt 0 ] && [ "$TIER_NOVEL_COUNT" -gt 0 ]; then
+  PROVEN_ESC_CMP=$(echo "scale=3; $TIER_PROVEN_ESCAPES / $TIER_PROVEN_COUNT" | bc 2>/dev/null || echo "0")
+  NOVEL_ESC_CMP=$(echo "scale=3; $TIER_NOVEL_ESCAPES / $TIER_NOVEL_COUNT" | bc 2>/dev/null || echo "0")
+  if awk "BEGIN{exit !($PROVEN_ESC_CMP > $NOVEL_ESC_CMP)}" 2>/dev/null; then
+    echo ""
+    echo "⚠ WARNING: PROVEN tier escape rate (${PROVEN_ESC_CMP}) exceeds NOVEL tier (${NOVEL_ESC_CMP})."
+    echo "  This indicates that de-escalation is suppressing agents that would have caught defects."
+    echo "  Investigate: run /pipeline-health --detail and check which cells are PROVEN with recent escapes."
+    echo "  Consider resetting affected provenance cells or raising the PROVEN threshold."
+  fi
+fi
+
+# Export for downstream phases (phase-3-analyze.md health score)
+export TIER_NOVEL_COUNT TIER_PROVEN_COUNT TIER_SHADOW_COUNT
+export TIER_NOVEL_ESCAPES TIER_PROVEN_ESCAPES
+```
+
+**Metrics to report in Phase 5 health summary**:
+- **Intensity distribution**: PROVEN / NOVEL / SHADOW counts and percentages over the analysis window
+- **Escaped-defect rate by tier**: REVIEW_FALSE_NEG audit-finding rate per tier (target: PROVEN ≤ overall rate, NOVEL ≥ overall rate before de-escalation takes effect, PROVEN never exceeds NOVEL)
+- **Shadow mode PRs**: count of PRs where intensity tier was computed but not acted on (should decrease after 30-day window)
+
+**Note**: If `TRUST_SHADOW_MODE=true` is still set in forge.yaml, all PRs will appear as SHADOW tier in this metric. This is correct — shadow mode has not yet been enforced. After setting `trust_shadow_mode: false`, tiers will populate.
+
 ### 2M: Session discovery and date-gated filtering (conversation transcript analytics)
 
 This phase locates conversation JSONL files for pipeline command sessions (work-on, orchestrate, review-pr, quality-gate, review-pr-staging) that ran within the analysis window. It produces `SESSION_JSONL_FILES` — consumed by downstream Conversation Transcript Analytics phases to extract flow metrics (phase durations, tool patterns, subagent efficiency, stalls, rate-limits).
@@ -1727,3 +1920,114 @@ fi
 
 ---
 
+### 2R: Confidence Calibration Table (from run outcomes)
+
+**Purpose**: Surfaces whether the pipeline's stated confidence levels predict actual outcomes. A HIGH-confidence issue that frequently triggers review-findings or reverts within 14 days indicates systematic overconfidence — the investigator's confidence is not tracking reality. Conversely, needs-human routing on issues that almost always survive indicates overcaution. This section reads the calibration table produced by `scripts/calibration.mjs` and published to the `forge-knowledge` branch. <!-- Added: forge#1741 -->
+
+**Skip if**: forge-knowledge branch is absent or `calibration/table.json` does not exist on it — emit the unavailability note below and continue.
+
+**Read calibration table**:
+```bash
+# Read calibration table from forge-knowledge branch (fail-safe: any error → skip)
+CALIBRATION_TABLE=""
+CALIBRATION_AVAILABLE=false
+
+if git show-ref --quiet "refs/remotes/origin/${FORGE_KNOWLEDGE_BRANCH:-forge-knowledge}" 2>/dev/null || \
+   git ls-remote --exit-code origin "${FORGE_KNOWLEDGE_BRANCH:-forge-knowledge}" >/dev/null 2>&1; then
+  RAW_TABLE=$(git show "origin/${FORGE_KNOWLEDGE_BRANCH:-forge-knowledge}:calibration/table.json" 2>/dev/null || echo '')
+  if [ -n "$RAW_TABLE" ]; then
+    CALIBRATION_TABLE="$RAW_TABLE"
+    CALIBRATION_AVAILABLE=true
+    CALIB_COMPUTED_AT=$(echo "$CALIBRATION_TABLE" | jq -r '.computedAt // "unknown"' 2>/dev/null || echo 'unknown')
+    CALIB_TOTAL_RUNS=$(echo "$CALIBRATION_TABLE" | jq -r '.totalRuns // 0' 2>/dev/null || echo '0')
+    CALIB_WINDOW_DAYS=$(echo "$CALIBRATION_TABLE" | jq -r '.windowDays // 14' 2>/dev/null || echo '14')
+    CALIB_MIN_SAMPLES=$(echo "$CALIBRATION_TABLE" | jq -r '.minSamples // 10' 2>/dev/null || echo '10')
+    echo "Calibration table loaded: ${CALIB_TOTAL_RUNS} runs, computed ${CALIB_COMPUTED_AT}"
+  fi
+fi
+
+if [ "$CALIBRATION_AVAILABLE" = false ]; then
+  echo "Phase 2R: Calibration table unavailable (forge-knowledge:calibration/table.json not found)"
+  echo "  To compute: node scripts/calibration.mjs --repo ${REPO} --publish"
+  # Store unavailability for Phase 5A report
+  CALIBRATION_AVAILABILITY_NOTE="⚠ Calibration table unavailable — run: node scripts/calibration.mjs --repo ${REPO} --publish"
+fi
+```
+
+**Render calibration table** (when available):
+
+```bash
+if [ "$CALIBRATION_AVAILABLE" = true ]; then
+  echo "## Confidence Calibration Table"
+  echo "**Source**: forge-knowledge:calibration/table.json"
+  echo "**Computed**: ${CALIB_COMPUTED_AT}"
+  echo "**Outcome window**: ${CALIB_WINDOW_DAYS} days | **Min samples for trust**: ${CALIB_MIN_SAMPLES}"
+  echo "**Total runs in corpus**: ${CALIB_TOTAL_RUNS}"
+  echo ""
+
+  # Extract and render per-cell rows; flag overconfidence and overcaution cells
+  OVERCONFIDENCE_CELLS=""
+  OVERCAUTION_CELLS=""
+
+  echo "$CALIBRATION_TABLE" | jq -r '
+    .rows[] |
+    [
+      .taskType,
+      .confidence,
+      (if .trusted then (.survivalRate * 100 | floor | tostring) + "%" else "—(N<" + (.sampleCount | tostring) + ")" end),
+      (.sampleCount | tostring),
+      (if .trusted then
+        (if .flag == "overconfidence" then "⚠ OVERCONFIDENCE" elif .flag == "overcaution-candidate" then "ℹ OVERCAUTION?" else "✅ OK" end)
+      else "ℹ untrusted" end)
+    ] | @tsv
+  ' 2>/dev/null | while IFS=$'\t' read -r taskType confidence survivalRate sampleCount status; do
+    printf "| %-20s | %-8s | %-18s | %-7s | %s\n" \
+      "$taskType" "$confidence" "$survivalRate" "$sampleCount" "$status"
+
+    # Collect flagged cells for summary
+    if [[ "$status" == *"OVERCONFIDENCE"* ]]; then
+      OVERCONFIDENCE_CELLS="${OVERCONFIDENCE_CELLS}\n  - ${taskType} × ${confidence}: ${survivalRate} survival (${sampleCount} runs)"
+    elif [[ "$status" == *"OVERCAUTION"* ]]; then
+      OVERCAUTION_CELLS="${OVERCAUTION_CELLS}\n  - ${taskType} × ${confidence}: ${survivalRate} survival (${sampleCount} runs)"
+    fi
+  done
+
+  # Emit flag summaries
+  if [ -n "$OVERCONFIDENCE_CELLS" ]; then
+    echo ""
+    echo "### ⚠ Overconfidence Cells (HIGH confidence, survival < 80%)"
+    echo "These task-type × confidence combinations claim HIGH confidence but have poor real-world outcomes."
+    echo "The review threshold for these cells should be tightened (consider needs-human routing)."
+    echo -e "$OVERCONFIDENCE_CELLS"
+  fi
+
+  if [ -n "$OVERCAUTION_CELLS" ]; then
+    echo ""
+    echo "### ℹ Overcaution Candidates (survival > 95% — needs-human may be unnecessary)"
+    echo "These cells have very high survival rates. Needs-human routing on them may be unnecessary,"
+    echo "but any loosening of behavior requires the eval-gate to pass on the affected task-type corpus."
+    echo "Do NOT loosen automatically — flag for human review."
+    echo -e "$OVERCAUTION_CELLS"
+  fi
+
+  # 30-day trend (compare against prior report's PRIOR_CALIBRATION_COMPUTED_AT if available)
+  if [ -n "${PRIOR_CALIBRATION_COMPUTED_AT:-}" ]; then
+    echo ""
+    echo "### Calibration Trend"
+    echo "Prior table computed: ${PRIOR_CALIBRATION_COMPUTED_AT}"
+    echo "Current table computed: ${CALIB_COMPUTED_AT}"
+    echo "(Trend comparison requires prior report data — compare manually if prior available)"
+  fi
+fi
+```
+
+**Metrics to surface in Phase 5A report**:
+- **Calibration data coverage**: `${CALIB_TOTAL_RUNS}` runs in corpus (target: > 20 for meaningful cells)
+- **Overconfidence cells**: count of cells where HIGH-confidence survival < 80%
+- **Overcaution candidates**: count of cells where survival > 95% (informational — loosening requires eval-gate)
+- **Table freshness**: `computedAt` timestamp — flag as stale if > 7 days old
+
+**Why this matters**: Static confidence thresholds in review-pr.md (Phase 7B blocking criteria) are prose rules with no feedback from outcomes. The calibration table is the mechanism by which the pipeline learns whether its stated confidence is tracking reality. A task-type with persistent overconfidence should have its review intensity increased regardless of the agent's stated confidence. <!-- Added: forge#1741 -->
+
+---
+