@mmerterden/multi-agent-pipeline 12.7.0 → 12.8.0

package/pipeline/multi-agent-refs/phases/phase-4-review.md

@@ -112,11 +112,16 @@ Cost ledger: `phase-4.repo_map_emitted bytes=N budget=B cache_hit=true|false`  -
 
 Before dispatching reviewers, run the deterministic diff risk scorer and inject the top-N files into each reviewer's prompt as a priority hint. **Advisory only  -  never gates the pipeline.** Heuristic, zero LLM, runs in well under a second.
 
+Score the diff **once, in full** (no `--top`): Steps 1.76/1.77 need every scored file (a shrinking test file ranked 20th; whether ANY file is high-stakes). The top-N hint is derived from that report, not a second git walk.
+
 ```bash
-RISK_JSON=$(node pipeline/scripts/diff-risk-score.mjs \
+RISK_FULL=$(node pipeline/scripts/diff-risk-score.mjs \
   --base "$BASE_BRANCH" --head HEAD \
-  --task-id "$TASK_ID" --top 5 2>/dev/null)
-echo "$RISK_JSON" | node pipeline/scripts/validate-diff-risk.mjs - >/dev/null 2>&1 || RISK_JSON=""
+  --task-id "$TASK_ID" 2>/dev/null)
+echo "$RISK_FULL" | node pipeline/scripts/validate-diff-risk.mjs - >/dev/null 2>&1 || RISK_FULL=""
+
+# Priority hint for the reviewer prompts: top 5 of the full report.
+RISK_JSON=$([ -n "$RISK_FULL" ] && jq -c '.files |= (sort_by(-.score) | .[:5])' <<< "$RISK_FULL" || echo "")
 ```
 
 **Signals & weights** (see `pipeline/schemas/diff-risk.schema.json`):
@@ -151,6 +156,35 @@ pipeline/scripts/log-metric.sh "$TASK_ID" 4 review.diff_risk \
 
 **Opt-out**: `prefs.global.diffRiskAdvisory = false` skips this step entirely (no script invocation, no priority block injection). Default `true` because the cost is bounded and the signal-to-noise has been measured against the golden-task fixture set.
 
+#### Step 1.76  -  Test-integrity gate (produces BLOCKING findings)
+
+Step 1.75 uses `test_lines_removed` as an advisory hint only  -  too weak for what it detects: a suite made green by deleting tests instead of fixing code. This turns the signal into blocking findings triage must adjudicate. Pure function of the full report, no git, no LLM.
+
+```bash
+TEST_INTEGRITY_JSON=$(printf '%s' "$RISK_FULL" | node pipeline/scripts/test-integrity-gate.mjs 2>/dev/null || echo "")
+TI_COUNT=$(jq -r '.count // 0' <<< "${TEST_INTEGRITY_JSON:-{\}}" 2>/dev/null || echo 0)
+[ "$TI_COUNT" -gt 0 ] && pipeline/scripts/log-metric.sh "$TASK_ID" 4 review.test_integrity findings="$TI_COUNT"
+```
+
+`findings[]` are reviewer-shaped (`test_integrity`, `blocking`), so they merge into the reviewer findings at Step 3.0 and need no triage-prompt or `validate-triage.mjs` change. Triage keeps each blocking unless the removal is justified per the immutable-test rule (spec changed AND commit body names the test) → `deferred[]`.
+
+The gate never blocks the phase; it *emits* blocking findings. Empty or unreadable input yields zero findings and the phase continues. Feed it the FULL report  -  `--top` hides a shrinking test file below the cut. **No opt-out**: a run that can switch off its own anti-reward-hacking control cannot be trusted to report a pass.
+
+#### Step 1.77  -  Reviewer scope (cost gate)
+
+On a trivial diff every reviewer agrees and the extra models plus triage are paid for nothing. Reviewer count comes from the same report, no LLM.
+
+```bash
+SCOPE_JSON=$(printf '%s' "$RISK_FULL" | node pipeline/scripts/review-scope.mjs 2>/dev/null \
+  || echo '{"scope":"full","reason":"no risk report - failing safe"}')
+REVIEW_SCOPE=$(jq -r '.scope // "full"' <<< "$SCOPE_JSON")
+pipeline/scripts/log-metric.sh "$TASK_ID" 4 review.scope scope="$REVIEW_SCOPE"
+```
+
+`single` (Reviewer 1 only) requires **all** of: churn <= 20 lines, `totals.max_score` < 3.0, and no `security_path` / `migration` / `public_api` / `no_test_change` / `test_lines_removed` on any file. Anything else → `full`.
+
+Fails safe in one direction only: empty report, parse failure or validator rejection all resolve to `full`, because skipping a reviewer trades coverage for cost. `consensus.reviewerCount` records what actually ran, so a single-reviewer run never reads as cross-model agreement. Opt-out: `prefs.global.reviewScopeGate = false` forces `full`.
+
 #### Step 1.8  -  Figma visual-fidelity context (when task carries a Figma reference)
 
 When `state.evidence.figma[]` is non-empty, the reviewer subagents MUST receive the captured screenshot URLs / paths and the canonical-component name (from `state.evidence.figma[i].screenshotUrl` and `state.evidence.figma[i].codeConnectSnippets[0].componentName` when present) so they can compare visual fidelity. Pass them inline in each reviewer prompt under a `## Figma evidence` block, one row per frame.
@@ -182,6 +216,8 @@ Phase 4 sends the same diff to every reviewer and then to triage, so the diff is
 
 Launch Agent instances **in parallel** using the shared `code-reviewer` subagent definition (`~/.claude/agents/code-reviewer.md`). The reviewer set is determined by the host CLI  -  GPT-5.4 is only available on Copilot CLI, so Claude Code skips that reviewer and runs a 2-model parallel review; Copilot CLI runs all three.
 
+**Scope from Step 1.77.** `$REVIEW_SCOPE == "single"` → dispatch **Reviewer 1 only**, and skip Step 2.5 + 3.6 (both no-ops with one reviewer). `"full"` (default + fail-safe) → the whole set below. Either way record the count in `consensus.reviewerCount`.
+
 | Reviewer   | subagent_type     | Model               | Focus                             | Skills Referenced                             | Where it runs        |
 | ---------- | ----------------- | ------------------- | --------------------------------- | --------------------------------------------- | -------------------- |
 | Reviewer 1 | `code-reviewer`   | `claude-fable-5` (Claude Code) / `claude-opus-4-8` (Copilot CLI) | Deep security + architecture      | `api-security-best-practices`, `architecture` | Both CLIs            |
@@ -259,9 +295,18 @@ Exit 0 = valid. Exit 2 = contradiction (approved=true with blocking findings)  -
 
 Opt-in empirical layer: when `prefs.global.verifyByTest.enabled` is `true`, accepted blocking findings additionally go through Step 3.7 (verify-by-test), which tries to reproduce each one with a minimal failing test before the Phase 3 rework loop fires. Full wiring: `$HOME/.claude/multi-agent-refs/features/verify-by-test.md`.
 
+##### 3.0 Merge the deterministic findings in
+
+Append the Step 1.76 test-integrity findings to the reviewer findings before counting, so they are adjudicated like any reviewer finding rather than resolved by triage never seeing them:
+
+```bash
+MERGED=$(jq -s '.[0] + (.[1].findings // [])' \
+  <(printf '%s' "$REVIEWER_FINDINGS_JSON") <(printf '%s' "${TEST_INTEGRITY_JSON:-{\}}"))
+```
+
 ##### 3.1 Short-circuit: no findings
 
-If merged findings `length === 0`, **skip triage**: write empty result `{"accepted": [], "deferred": [], "rejected": [], "approved": true}`, log, proceed to Phase 5.
+If **merged** findings `length === 0`, **skip triage**: write empty result `{"accepted": [], "deferred": [], "rejected": [], "approved": true}`, log, proceed to Phase 5. Note this is the merged count from 3.0: a run with zero reviewer findings but a non-empty test-integrity set must NOT short-circuit.
 
 ##### 3.2 Launch triage agent
 

package/pipeline/schemas/prefs.schema.json

@@ -960,6 +960,12 @@
           "default": true,
           "description": "v8.3+ - Phase 4 Step 1.75 advisory diff risk scoring. When enabled, `pipeline/scripts/diff-risk-score.mjs` runs before reviewer dispatch and the top-N risk-ranked files are injected into each reviewer's prompt as a priority hint (security paths, public API surfaces, untested source changes, schema migrations). Heuristic, deterministic, no LLM cost. Default ON - the run is sub-second and never gates the pipeline. Flip to false to skip the script and the prompt injection entirely."
         },
+        "reviewScopeGate": {
+          "type": "boolean",
+          "default": true,
+          "description": "v12.8+ - Phase 4 Step 1.77 reviewer-scope gate. Decides the reviewer count from the deterministic diff-risk report via `pipeline/scripts/review-scope.mjs`: a diff under 20 lines of churn with max_score < 3.0 and no security_path / migration / public_api / no_test_change / test_lines_removed signal runs ONE reviewer instead of the full CLI-aware set (2 on Claude Code, 3 on Copilot CLI). No LLM. Fails safe in one direction only - any error, empty report or validator rejection resolves to the full set, because skipping a reviewer trades coverage for cost. Set false to force the full set on every diff.",
+          "$comment": "Shipped inert for a release: the script existed, was unit- and smoke-tested, and no phase doc referenced it, so every diff paid for the full reviewer set. Wired in Step 1.77; smoke-gate-wiring.sh keeps it reachable."
+        },
         "priorArtEnrichment": {
           "type": "object",
           "additionalProperties": false,

package/pipeline/scripts/_smoke-root.sh

@@ -0,0 +1,61 @@
+#!/usr/bin/env bash
+#
+# _smoke-root.sh  -  resolve the pipeline tree for a smoke that must run in BOTH
+# the repo checkout and an end-user install.
+#
+# Why this exists: the repo and an install have different shapes.
+#
+#   repo checkout          install (~/.claude or ~/.copilot)
+#   ------------------     --------------------------------
+#   pipeline/scripts/      scripts/
+#   pipeline/schemas/      schemas/
+#   pipeline/multi-agent-refs/  multi-agent-refs/
+#   pipeline/commands/     commands/
+#   package.json           (absent)
+#
+# A smoke that hardcodes `REPO_ROOT="$(dirname "$0")/../.."` and then reads
+# `$REPO_ROOT/pipeline/...` resolves to `$HOME/pipeline/...` once installed, so
+# every path check fails. Whoever ran it sees a red report and concludes the
+# install is broken. Source this file instead and read the MA_* variables.
+#
+# Usage (from any script in pipeline/scripts/):
+#   . "$(dirname "${BASH_SOURCE[0]}")/_smoke-root.sh"
+#   grep -q needle "$MA_REFS/phases/phase-4-review.md"
+#
+# Exports:
+#   MA_LAYOUT     "repo" | "install"
+#   MA_ROOT       repo root, or the install root (~/.claude)
+#   MA_SCRIPTS    directory holding this script
+#   MA_SCHEMAS    JSON schemas
+#   MA_REFS       multi-agent-refs
+#   MA_COMMANDS   slash-command tree
+#   MA_SKILLS     skills tree
+#
+# Every exported directory is verified to exist; a missing one is left empty so
+# the caller can skip that check rather than report a false failure.
+
+MA_SCRIPTS="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+
+# The repo is the only layout with pipeline/ beside package.json. Checking both
+# rules out an install that happens to sit next to an unrelated pipeline/ dir.
+_ma_candidate="$(cd "$MA_SCRIPTS/../.." && pwd)"
+if [ -f "$_ma_candidate/package.json" ] && [ -d "$_ma_candidate/pipeline" ]; then
+  MA_LAYOUT="repo"
+  MA_ROOT="$_ma_candidate"
+  _ma_base="$_ma_candidate/pipeline"
+else
+  MA_LAYOUT="install"
+  MA_ROOT="$(cd "$MA_SCRIPTS/.." && pwd)"
+  _ma_base="$MA_ROOT"
+fi
+unset _ma_candidate
+
+_ma_dir() { [ -d "$1" ] && echo "$1" || echo ""; }
+
+MA_SCHEMAS="$(_ma_dir "$_ma_base/schemas")"
+MA_REFS="$(_ma_dir "$_ma_base/multi-agent-refs")"
+MA_COMMANDS="$(_ma_dir "$_ma_base/commands")"
+MA_SKILLS="$(_ma_dir "$_ma_base/skills")"
+unset _ma_base
+
+export MA_LAYOUT MA_ROOT MA_SCRIPTS MA_SCHEMAS MA_REFS MA_COMMANDS MA_SKILLS

package/pipeline/scripts/audit-log.sh

@@ -80,3 +80,28 @@ LINE="{\"ts\":\"${TS}\",\"event\":\"$(json_escape "$EVENT")\",\"service\":\"$(js
 LINE="${LINE}}"
 
 echo "$LINE" >> "$AUDIT_FILE" 2>/dev/null || true
+
+# Opportunistic rotation. audit-log-rotate.sh documents a launchd/cron install,
+# but nothing ever told a user to set that up, so an unrotated trail would grow
+# without bound now that credential-store.sh writes an event on every PAT lookup.
+# Self-triggering removes the operator step: check a cheap size threshold and hand
+# off to the real rotator, which is atomic against concurrent appenders (it moves
+# the live inode first). Silent and non-blocking - telemetry never fails a run.
+ROTATE_AT_BYTES="${AUDIT_ROTATE_AT_BYTES:-1048576}"
+_audit_size() {
+  case "$(uname -s)" in
+    Darwin|*BSD*) stat -f %z "$1" 2>/dev/null || echo 0 ;;
+    *)            stat -c %s "$1" 2>/dev/null || echo 0 ;;
+  esac
+}
+if [ "${AUDIT_SELF_ROTATE:-1}" = "1" ] && [ -f "$AUDIT_FILE" ]; then
+  SIZE="$(_audit_size "$AUDIT_FILE")"
+  if [ "${SIZE:-0}" -ge "$ROTATE_AT_BYTES" ]; then
+    ROTATOR="$(dirname "${BASH_SOURCE[0]:-$0}")/audit-log-rotate.sh"
+    # The rotator derives AUDIT_FILE from AUDIT_DIR, so pass the DIRECTORY.
+    # Handing it AUDIT_FILE would leave it rotating the default path while a
+    # caller-overridden trail kept growing.
+    [ -f "$ROTATOR" ] &&
+      AUDIT_DIR="$(dirname "$AUDIT_FILE")" bash "$ROTATOR" >/dev/null 2>&1 || true
+  fi
+fi

package/pipeline/scripts/smoke-cross-cli-behavior.sh

@@ -190,22 +190,36 @@ fi
 # (the orchestrator dispatches per the doc), so lock the CONTRACT here against
 # drift across the phase doc, the schema, and the consensus block.
 echo "→ reviewer-count contract (Claude=2, Copilot=3)"
-REPO_ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
-P4="$REPO_ROOT/pipeline/multi-agent-refs/phases/phase-4-review.md"
-REVSCHEMA="$REPO_ROOT/pipeline/schemas/reviewer-output.schema.json"
-TRSCHEMA="$REPO_ROOT/pipeline/schemas/triage-output.schema.json"
-if grep -qiE "Claude Code (dispatches|=|:) ?2|2-model" "$P4" && grep -qiE "Copilot CLI (dispatches|=|:) ?3|3-model" "$P4"; then
+# `/multi-agent:update` runs this smoke on the user's machine, where the tree is
+# ~/.claude/{schemas,multi-agent-refs} and NOT <root>/pipeline/*. Resolving the
+# root by hand here reported three phantom failures in an install, which reads as
+# a broken installation. _smoke-root.sh handles both layouts.
+# shellcheck source=pipeline/scripts/_smoke-root.sh
+. "$(dirname "${BASH_SOURCE[0]}")/_smoke-root.sh"
+P4="${MA_REFS:+$MA_REFS/phases/phase-4-review.md}"
+REVSCHEMA="${MA_SCHEMAS:+$MA_SCHEMAS/reviewer-output.schema.json}"
+TRSCHEMA="${MA_SCHEMAS:+$MA_SCHEMAS/triage-output.schema.json}"
+
+if [ -z "$P4" ] || [ ! -f "$P4" ]; then
+  echo "  ↷ SKIP: phase-4-review.md not present in this $MA_LAYOUT layout"
+elif grep -qiE "Claude Code (dispatches|=|:) ?2|2-model" "$P4" && grep -qiE "Copilot CLI (dispatches|=|:) ?3|3-model" "$P4"; then
   pass "phase-4-review declares Claude=2 / Copilot=3 reviewers"
 else
   fail "phase-4-review does not declare the CLI-aware reviewer count"
 fi
-if grep -qi "Copilot CLI" "$REVSCHEMA" && grep -qi "Claude" "$REVSCHEMA"; then
+
+if [ -z "$REVSCHEMA" ] || [ ! -f "$REVSCHEMA" ]; then
+  echo "  ↷ SKIP: reviewer-output.schema.json not present in this $MA_LAYOUT layout"
+elif grep -qi "Copilot CLI" "$REVSCHEMA" && grep -qi "Claude" "$REVSCHEMA"; then
   pass "reviewer-output schema notes the CLI-aware reviewer set"
 else
   fail "reviewer-output schema missing the CLI-aware note"
 fi
+
 # consensus.reviewerCount must accommodate both 2 and 3 (min 1, no max < 3)
-if grep -q '"reviewerCount"' "$TRSCHEMA" && grep -qi "Claude Code = 2, Copilot CLI = 3" "$TRSCHEMA"; then
+if [ -z "$TRSCHEMA" ] || [ ! -f "$TRSCHEMA" ]; then
+  echo "  ↷ SKIP: triage-output.schema.json not present in this $MA_LAYOUT layout"
+elif grep -q '"reviewerCount"' "$TRSCHEMA" && grep -qi "Claude Code = 2, Copilot CLI = 3" "$TRSCHEMA"; then
   pass "triage consensus.reviewerCount documents 2 (Claude) and 3 (Copilot)"
 else
   fail "triage consensus.reviewerCount does not document the 2/3 split"

package/pipeline/eval/golden-tasks/01-ios-bugfix-darkmode/expected/phase-1-analysis.json

@@ -1,25 +0,0 @@
-{
-  "stack": {
-    "primary": "ios",
-    "language": "Swift 6",
-    "framework": "SwiftUI"
-  },
-  "touchedAreas": [
-    {
-      "path": "Sources/Auth/LoginView.swift",
-      "why": "Hardcoded Color.white yerine DesignSystem.Color.Primary.onBackground kullanacak"
-    },
-    {
-      "path": "Sources/Auth/LoginViewTests.swift",
-      "why": "Dark mode snapshot + ViewInspector contrast assertion eklenecek"
-    }
-  ],
-  "risks": [
-    {
-      "risk": "LoginView başka ekran tarafından preview alanı olarak kullanılıyor olabilir  -  token değişimi preview snapshot'larını kırabilir",
-      "severity": "low",
-      "mitigation": "Preview varyantlarını snapshot'lar ile koruyarak incremental değiştir"
-    }
-  ],
-  "summary": "LoginView'de primary button rengi hardcoded Color.white. Token sistemi ile değiştirilip light+dark mode için ayrık snapshot testleri eklenecek. Değişim iki dosya ile sınırlı, regresyon riski düşük."
-}

package/pipeline/eval/golden-tasks/01-ios-bugfix-darkmode/expected/phase-2-plan.json

@@ -1,30 +0,0 @@
-{
-  "approach": "TDD-first. Önce dark-mode contrast için snapshot + ViewInspector testi yazılır (RED), sonra LoginView'de hardcoded renk token çağrısıyla değiştirilir (GREEN). Refactor: aynı dosyada ikinci hardcoded renk varsa tek geçişte temizlenir.",
-  "tasks": [
-    {
-      "id": "T1",
-      "title": "Add failing dark mode snapshot + contrast ViewInspector test",
-      "type": "test",
-      "files": ["Sources/Auth/LoginViewTests.swift"],
-      "dependsOn": [],
-      "acceptanceCriteria": [
-        "Snapshot test for dark mode renders primary button",
-        "ViewInspector asserts foreground color != Color.white when env.colorScheme == .dark"
-      ]
-    },
-    {
-      "id": "T2",
-      "title": "Replace hardcoded Color.white with DesignSystem token",
-      "type": "code",
-      "files": ["Sources/Auth/LoginView.swift"],
-      "dependsOn": ["T1"],
-      "acceptanceCriteria": [
-        "No Color(red:..., green:..., blue:...) or Color.white remains in LoginView",
-        "Primary button uses DesignSystem.Color.Primary.onBackground",
-        "All snapshot + inspector tests pass"
-      ]
-    }
-  ],
-  "approved": true,
-  "userFeedback": null
-}

package/pipeline/eval/golden-tasks/01-ios-bugfix-darkmode/expected/phase-4-review.json

@@ -1,20 +0,0 @@
-[
-  {
-    "reviewer": "opus",
-    "findings": [
-      {
-        "severity": "blocking",
-        "file": "Sources/Auth/LoginView.swift",
-        "line": 42,
-        "issue": "Hardcoded .font(.system(size: 14)) kaldı  -  aynı fix sırasında typography token'ına da bağlanmalı",
-        "fix": "Replace with .typographyStyle(.body1)  -  same pattern as the color token migration"
-      }
-    ],
-    "approved": false
-  },
-  {
-    "reviewer": "sonnet",
-    "findings": [],
-    "approved": true
-  }
-]

package/pipeline/eval/golden-tasks/01-ios-bugfix-darkmode/expected/phase-4-triage.json

@@ -1,15 +0,0 @@
-{
-  "accepted": [
-    {
-      "severity": "blocking",
-      "file": "Sources/Auth/LoginView.swift",
-      "line": 42,
-      "issue": "Hardcoded .font(.system(size: 14)) kaldı  -  aynı fix sırasında typography token'ına da bağlanmalı",
-      "fix": "Replace with .typographyStyle(.body1)  -  same pattern as the color token migration",
-      "reviewer": "opus"
-    }
-  ],
-  "deferred": [],
-  "rejected": [],
-  "approved": false
-}

package/pipeline/eval/golden-tasks/01-ios-bugfix-darkmode/metadata.json

@@ -1,14 +0,0 @@
-{
-  "focus": "SwiftUI bugfix path  -  verifies that Phase 4 picks up an in-scope but adjacent blocker (token drift) even when the primary issue is resolved.",
-  "exercises": [
-    "Phase 1 stack detection (iOS / Swift 6 / SwiftUI)",
-    "Phase 2 TDD ordering (test before code)",
-    "Phase 4 Reviewer-1 catches adjacent token drift; Reviewer-2 approves base fix",
-    "Triage keeps the blocker accepted, no deferrals, not approved"
-  ],
-  "regressionsCaught": [
-    "If stack detection drops SwiftUI from the framework field, this fixture fails the analysis schema check",
-    "If planning schema drops the 'dependsOn' field, T2 loses its order guarantee",
-    "If Phase 4 no longer tolerates Reviewer-1 blocker + Reviewer-2 approval (approved divergence), triage fixture fails"
-  ]
-}

package/pipeline/eval/golden-tasks/01-ios-bugfix-darkmode/task.json

@@ -1,12 +0,0 @@
-{
-  "title": "LoginView dark mode contrast fix",
-  "description": "Dark mode'da primary button text'i zemin rengiyle düşük kontrast veriyor. Tokens switch'i çalışmıyor çünkü LoginView hardcoded Color.white kullanıyor. Design tokens'a geri bağla.",
-  "inputType": "free-text",
-  "expectedStack": "ios",
-  "expectedLanguage": "Swift",
-  "expectedFramework": "SwiftUI",
-  "expectedBlockers": 1,
-  "expectedDeferrals": 0,
-  "baseBranch": "develop",
-  "taskBranch": "bugfix/loginview-dark-mode-contrast"
-}

package/pipeline/eval/golden-tasks/02-android-feature-compose/expected/phase-1-analysis.json

@@ -1,29 +0,0 @@
-{
-  "stack": {
-    "primary": "android",
-    "language": "Kotlin 2.1",
-    "framework": "Jetpack Compose"
-  },
-  "touchedAreas": [
-    {
-      "path": "feature/orders/src/main/java/com/example/orders/OrdersScreen.kt",
-      "why": "PullToRefreshBox UI eklenecek, ViewModel.refresh() tetiklenecek"
-    },
-    {
-      "path": "feature/orders/src/main/java/com/example/orders/OrdersViewModel.kt",
-      "why": "isRefreshing state flow eklenecek  -  PullToRefreshBox'ın state binding'i için"
-    },
-    {
-      "path": "feature/orders/src/test/java/com/example/orders/OrdersScreenTest.kt",
-      "why": "Pull-to-refresh compose testi eklenecek"
-    }
-  ],
-  "risks": [
-    {
-      "risk": "Material3 PullToRefreshBox henüz beta API  -  1.3.x'de signature değişti",
-      "severity": "medium",
-      "mitigation": "Existing libs.versions.toml'de material3 sürümünü teyit et, gerekirse bump et"
-    }
-  ],
-  "summary": "OrdersScreen Compose ekranına Material3 pull-to-refresh eklenir. ViewModel'de zaten refresh() var; UI binding + isRefreshing state + Compose testi yeterli. Material3 API surface'i beta olduğu için version check gerekir, risk orta."
-}

package/pipeline/eval/golden-tasks/02-android-feature-compose/expected/phase-2-plan.json

@@ -1,43 +0,0 @@
-{
-  "approach": "ViewModel state genişletilir (isRefreshing StateFlow), UI PullToRefreshBox'a bağlanır, Compose test yazılır. TDD: önce Compose testi refresh davranışını doğrulamalı (RED), sonra ViewModel + UI bağlanır.",
-  "tasks": [
-    {
-      "id": "T1",
-      "title": "Add failing Compose test for pull-to-refresh trigger",
-      "type": "test",
-      "files": ["feature/orders/src/test/java/com/example/orders/OrdersScreenTest.kt"],
-      "dependsOn": [],
-      "acceptanceCriteria": [
-        "Test performTouchInput { swipeDown() } on OrdersScreen",
-        "Asserts ViewModel.refresh() is called",
-        "Test initially fails because PullToRefreshBox is not wired"
-      ]
-    },
-    {
-      "id": "T2",
-      "title": "Add isRefreshing StateFlow to OrdersViewModel",
-      "type": "code",
-      "files": ["feature/orders/src/main/java/com/example/orders/OrdersViewModel.kt"],
-      "dependsOn": ["T1"],
-      "acceptanceCriteria": [
-        "isRefreshing: StateFlow<Boolean> exposed",
-        "refresh() sets isRefreshing = true, then false on completion"
-      ]
-    },
-    {
-      "id": "T3",
-      "title": "Wire PullToRefreshBox in OrdersScreen",
-      "type": "code",
-      "files": ["feature/orders/src/main/java/com/example/orders/OrdersScreen.kt"],
-      "dependsOn": ["T2"],
-      "acceptanceCriteria": [
-        "PullToRefreshBox wraps the list",
-        "isRefreshing bound from ViewModel state",
-        "onRefresh calls viewModel::refresh",
-        "Compose test from T1 passes"
-      ]
-    }
-  ],
-  "approved": true,
-  "userFeedback": null
-}

package/pipeline/eval/golden-tasks/02-android-feature-compose/expected/phase-4-review.json

@@ -1,35 +0,0 @@
-[
-  {
-    "reviewer": "opus",
-    "findings": [
-      {
-        "severity": "blocking",
-        "file": "feature/orders/src/main/java/com/example/orders/OrdersViewModel.kt",
-        "line": 28,
-        "issue": "refresh() içinde isRefreshing = false finally bloğunda değil  -  exception fırlatıldığında spinner sonsuza dek takılır",
-        "fix": "Wrap the body in try/finally and set _isRefreshing.value = false in finally"
-      },
-      {
-        "severity": "important",
-        "file": "feature/orders/src/main/java/com/example/orders/OrdersScreen.kt",
-        "line": 0,
-        "issue": "Yeni modifier chain lazılarla LazyColumn semantics test matcher'larını kırabilir  -  Compose stability endişesi var",
-        "fix": "Measure recomposition count with Compose Compiler Reports before merging"
-      }
-    ],
-    "approved": false
-  },
-  {
-    "reviewer": "sonnet",
-    "findings": [
-      {
-        "severity": "important",
-        "file": "feature/orders/src/main/java/com/example/orders/OrdersScreen.kt",
-        "line": 54,
-        "issue": "PullToRefreshState hoisted yerine composable içinde tutuluyor  -  configuration change'de state kaybı",
-        "fix": "Use rememberSaveable or hoist the state up to ViewModel"
-      }
-    ],
-    "approved": false
-  }
-]

package/pipeline/eval/golden-tasks/02-android-feature-compose/expected/phase-4-triage.json

@@ -1,35 +0,0 @@
-{
-  "accepted": [
-    {
-      "severity": "blocking",
-      "file": "feature/orders/src/main/java/com/example/orders/OrdersViewModel.kt",
-      "line": 28,
-      "issue": "refresh() içinde isRefreshing = false finally bloğunda değil  -  exception fırlatıldığında spinner sonsuza dek takılır",
-      "fix": "Wrap the body in try/finally and set _isRefreshing.value = false in finally",
-      "reviewer": "opus"
-    },
-    {
-      "severity": "important",
-      "file": "feature/orders/src/main/java/com/example/orders/OrdersScreen.kt",
-      "line": 54,
-      "issue": "PullToRefreshState hoisted yerine composable içinde tutuluyor  -  configuration change'de state kaybı",
-      "fix": "Use rememberSaveable or hoist the state up to ViewModel",
-      "reviewer": "sonnet"
-    }
-  ],
-  "deferred": [
-    {
-      "finding": {
-        "severity": "important",
-        "file": "feature/orders/src/main/java/com/example/orders/OrdersScreen.kt",
-        "line": 0,
-        "issue": "Yeni modifier chain lazılarla LazyColumn semantics test matcher'larını kırabilir  -  Compose stability endişesi var",
-        "fix": "Measure recomposition count with Compose Compiler Reports before merging",
-        "reviewer": "opus"
-      },
-      "reason": "Out of scope for this task  -  recomposition audit is a performance sweep, not a correctness gate for adding pull-to-refresh"
-    }
-  ],
-  "rejected": [],
-  "approved": false
-}

package/pipeline/eval/golden-tasks/02-android-feature-compose/metadata.json

@@ -1,14 +0,0 @@
-{
-  "focus": "Android/Compose feature path with mixed-verdict triage  -  one finding deferred out-of-scope, two accepted (one blocker + one important cross-reviewer match).",
-  "exercises": [
-    "Phase 1 stack detection (android / Kotlin 2.1 / Jetpack Compose)",
-    "Phase 2 3-task decomposition with clear dependency chain (T1 test → T2 VM → T3 UI)",
-    "Phase 4 Reviewer-1 + Reviewer-2 surface overlapping concerns at different lines",
-    "Triage deferred bucket works (important but out-of-scope finding parked, not actioned)"
-  ],
-  "regressionsCaught": [
-    "If triage schema removes the 'deferred' bucket, this fixture fails",
-    "If reviewer enum drops a required reviewer, accepted[] loses its reviewer attribution and schema fails",
-    "If planning schema drops 'dependsOn', T1→T2→T3 ordering becomes unverifiable"
-  ]
-}

package/pipeline/eval/golden-tasks/02-android-feature-compose/task.json

@@ -1,12 +0,0 @@
-{
-  "title": "Add pull-to-refresh on OrdersScreen",
-  "description": "Compose OrdersScreen'de swipe-to-refresh yok. Material3 PullToRefreshBox eklenmeli, ViewModel refresh() zaten var  -  UI tarafı bağlanmadı.",
-  "inputType": "free-text",
-  "expectedStack": "android",
-  "expectedLanguage": "Kotlin",
-  "expectedFramework": "Jetpack Compose",
-  "expectedBlockers": 1,
-  "expectedDeferrals": 1,
-  "baseBranch": "develop",
-  "taskBranch": "feature/orders-pull-to-refresh"
-}

package/pipeline/eval/golden-tasks/03-backend-python-ratelimit/expected/phase-1-analysis.json

@@ -1,29 +0,0 @@
-{
-  "stack": {
-    "primary": "backend",
-    "language": "Python 3.13",
-    "framework": "FastAPI"
-  },
-  "touchedAreas": [
-    {
-      "path": "app/middleware/rate_limit.py",
-      "why": "New sliding-window per-IP limiter implemented as ASGI middleware"
-    },
-    {
-      "path": "app/api/auth.py",
-      "why": "Login router registers the limiter and returns 429 + Retry-After on breach"
-    },
-    {
-      "path": "tests/test_rate_limit.py",
-      "why": "Covers allowed-under-limit, blocked-over-limit, and window reset"
-    }
-  ],
-  "risks": [
-    {
-      "risk": "An in-memory counter does not share state across Gunicorn/Uvicorn workers, so the limit is per-process not per-cluster",
-      "severity": "medium",
-      "mitigation": "Document the single-process assumption now; back it with Redis when the service scales horizontally"
-    }
-  ],
-  "summary": "Brute-force on POST /auth/login is unthrottled. Add a sliding-window per-IP limiter as middleware, wire it into the auth router, and return 429 with Retry-After. Scope is three files; the main caveat is multi-worker state sharing, deferred to a Redis-backed follow-up."
-}

package/pipeline/eval/golden-tasks/03-backend-python-ratelimit/expected/phase-2-plan.json

@@ -1,42 +0,0 @@
-{
-  "approach": "TDD-first. Write a failing test that fires N+1 requests from one IP and asserts the last one returns 429 with Retry-After (RED). Implement the sliding-window limiter middleware (GREEN). Wire it into the auth router and assert allowed traffic still passes (refactor + integration).",
-  "tasks": [
-    {
-      "id": "T1",
-      "title": "Add failing rate-limit tests (under limit, over limit, window reset)",
-      "type": "test",
-      "files": ["tests/test_rate_limit.py"],
-      "dependsOn": [],
-      "acceptanceCriteria": [
-        "N requests under the limit all return 200",
-        "The N+1th request within the window returns 429 with a Retry-After header",
-        "After the window elapses, requests are allowed again"
-      ]
-    },
-    {
-      "id": "T2",
-      "title": "Implement sliding-window per-IP limiter middleware",
-      "type": "code",
-      "files": ["app/middleware/rate_limit.py"],
-      "dependsOn": ["T1"],
-      "acceptanceCriteria": [
-        "Tracks request timestamps per client IP in a sliding window",
-        "Emits 429 with Retry-After once the threshold is exceeded",
-        "Window bookkeeping is safe under concurrent requests"
-      ]
-    },
-    {
-      "id": "T3",
-      "title": "Register the limiter on the auth login route",
-      "type": "code",
-      "files": ["app/api/auth.py"],
-      "dependsOn": ["T2"],
-      "acceptanceCriteria": [
-        "POST /auth/login passes through the limiter",
-        "Normal login traffic under the limit is unaffected"
-      ]
-    }
-  ],
-  "approved": true,
-  "userFeedback": null
-}

package/pipeline/eval/golden-tasks/03-backend-python-ratelimit/expected/phase-4-review.json

@@ -1,20 +0,0 @@
-[
-  {
-    "reviewer": "opus",
-    "findings": [
-      {
-        "severity": "blocking",
-        "file": "app/middleware/rate_limit.py",
-        "line": 38,
-        "issue": "The per-IP window dict is read-modified-written without a lock; under concurrent requests two coroutines can both pass the check and under-count, letting bursts through",
-        "fix": "Guard the read-modify-write with an asyncio.Lock keyed per IP, or use an atomic counter structure"
-      }
-    ],
-    "approved": false
-  },
-  {
-    "reviewer": "sonnet",
-    "findings": [],
-    "approved": true
-  }
-]