@lvlup-sw/exarchos 2.4.3 → 2.6.0

package/skills/generic/shepherd/references/assess-checklist.md

@@ -0,0 +1,170 @@
+# Assessment Checklist
+
+Detailed steps for gathering PR status during each shepherd iteration.
+
+## 1. Identify PRs
+
+Read PR URLs from workflow state:
+```
+mcp__plugin_exarchos_exarchos__exarchos_workflow({ action: "get", featureId: "<id>", fields: ["synthesis", "artifacts"] })
+```
+
+Extract PR numbers from URLs (e.g., `https://github.com/owner/repo/pull/123` → `123`).
+
+If no PRs in state, check GitHub:
+```bash
+gh pr list --json number,baseRefName,headRefName,url
+```
+
+## 2. CI Check Status
+
+For each PR, use GitHub MCP:
+```
+mcp__plugin_github_github__pull_request_read({
+  method: "get_status",
+  owner: "<owner>",
+  repo: "<repo>",
+  pullNumber: <number>
+})
+```
+
+Classification:
+| state | Status |
+|-------|--------|
+| `SUCCESS` | pass |
+| `NEUTRAL`, `SKIPPED` | pass (ignorable) |
+| `FAILURE`, `ERROR` | fail |
+| `EXPECTED` | pass (status check) |
+| `PENDING` | pending |
+
+**Aggregate rule:** ALL checks must pass. Any `FAILURE` or `ERROR` → CI fails.
+
+**Wait for pending:** If checks are still running, inform the user and suggest waiting. Do NOT treat pending as failure unless it has been pending for an unreasonable time (>30 minutes).
+
+## 3. Formal Review Status
+
+Check for formal reviews (APPROVED, CHANGES_REQUESTED, etc.) via GitHub MCP:
+```
+mcp__plugin_github_github__pull_request_read({
+  method: "get_reviews",
+  owner: "<owner>",
+  repo: "<repo>",
+  pullNumber: <number>
+})
+```
+
+Review classification:
+| state | Meaning |
+|-------|---------|
+| `APPROVED` | Reviewer approved |
+| `CHANGES_REQUESTED` | Reviewer wants changes |
+| `COMMENTED` | Non-blocking comment |
+| `PENDING` | Review started but not submitted |
+| `DISMISSED` | Review was dismissed |
+
+**Aggregate rule:** No `CHANGES_REQUESTED` reviews from any reviewer. `COMMENTED` and `PENDING` are non-blocking.
+
+**NOTE:** Formal review status alone is INSUFFICIENT. Many automated reviewers (Sentry, CodeRabbit) leave inline comments without submitting a formal review. You MUST also check inline review comments (step 4).
+
+## 4. Inline Review Comments (CRITICAL)
+
+**This is the most commonly missed dimension.** Sentry, CodeRabbit, and other bots leave inline review comments that are independent of formal review status. A PR can show "no reviews" while having 10 unaddressed inline comments.
+
+**Read ALL inline review comments for each PR via GitHub MCP:**
+```
+mcp__plugin_github_github__pull_request_read({
+  method: "get_review_comments",
+  owner: "<owner>",
+  repo: "<repo>",
+  pullNumber: <number>
+})
+```
+
+**Identify comment sources:**
+
+| Bot login | Reviewer | What they flag |
+|-----------|----------|----------------|
+| `sentry[bot]` | Sentry | Bug predictions, security vulnerabilities, runtime errors |
+| `github-actions[bot]` | GitHub Actions | CI/gate checks, usually informational |
+| `coderabbitai[bot]` | CodeRabbit | Code review suggestions, refactoring, best practices |
+| Any other login | Human reviewer | Direct feedback requiring response |
+
+**Determine which comments are addressed:**
+
+A comment thread is "addressed" if it has at least one reply (another comment with `in_reply_to_id` matching the original comment's `id`).
+
+Build a per-source summary:
+```
+sentry: 2 total, 2 replied
+human: 3 total, 1 replied  ← 2 UNADDRESSED
+coderabbit: 5 total, 5 replied
+```
+
+**Any unaddressed comment = assessment fails.** Every thread needs a reply — either confirming a fix, explaining a design decision, or acknowledging for a future phase.
+
+## 5. Stack Health
+
+Check the branch stack state:
+```bash
+gh pr list --json number,baseRefName,headRefName,state
+```
+
+Verify:
+- All expected branches are present and have PRs
+- Base branch targeting is correct (bottom of stack targets `main`)
+- Each PR's base matches its parent in the stack
+- No outdated branches needing rebase
+
+If base branch has advanced:
+```bash
+git fetch origin
+git rebase origin/<base>
+git push --force-with-lease
+```
+
+## 6. Merge Readiness
+
+Check if `--merge-when-ready` is active via GitHub MCP:
+```
+mcp__plugin_github_github__pull_request_read({
+  method: "get",
+  owner: "<owner>",
+  repo: "<repo>",
+  pullNumber: <number>
+})
+```
+The response includes `autoMergeRequest` — if null, merge-when-ready is not set.
+
+If `autoMergeRequest` is null, merge-when-ready is not set. Re-enable:
+```bash
+gh pr merge <number> --auto --squash
+```
+
+## 7. Aggregate and Report
+
+Build the status table covering ALL dimensions:
+
+```markdown
+## PR Status — Iteration <N>
+
+| PR | CI | Formal Reviews | Inline Comments | Stack | Merge Queue |
+|----|-----|---------------|-----------------|-------|-------------|
+| #621 | pass | none | 1 Sentry (replied) | healthy | enqueued |
+| #622 | pass | none | — | healthy | enqueued |
+| #623 | pass | CR: commented | 1 CodeRabbit (replied) | healthy | enqueued |
+| #624 | fail (lint) | CR: commented | 3 human (2 unaddressed), 4 CodeRabbit (replied) | healthy | blocked |
+| #625 | pass | none | 2 Sentry (unaddressed) | healthy | enqueued |
+
+### Unaddressed Comments
+1. **PR #624 — Reviewer:** `resolveEvalsDir` should use injected config (eval-run.ts:14)
+2. **PR #624 — Reviewer:** unused `dataset` parameter (eval-run.ts:48)
+3. **PR #625 — Sentry:** TracePatternGrader reads wrong field (trace-pattern.ts:26)
+4. **PR #625 — Sentry:** exact-match structural mismatch (suite.json:22)
+
+### Recommended Actions
+1. Fix Sentry bugs in #625 (trace field name, exact-match config)
+2. Reply to DI concern on #624 with Phase 2 rationale
+3. Reply to dataset param concern as intentional forward-compat
+4. Fix lint error in #624
+5. Resubmit stack after fixes
+```

package/skills/generic/shepherd/references/escalation-criteria.md

@@ -0,0 +1,53 @@
+# Escalation Criteria
+
+When to stop the shepherd loop and escalate to the user.
+
+## Automatic Escalation Triggers
+
+| Trigger | Condition | Action |
+|---------|-----------|--------|
+| Iteration limit | `currentIteration >= maxIterations` (default 5) | Pause and report summary |
+| Persistent CI failure | Same check fails across 3+ consecutive iterations | Report as likely flaky or systemic |
+| Review loop | Same reviewer requests changes 2+ times after fixes | Escalate — may need design discussion |
+| Conflicting feedback | Two reviewers give contradictory guidance | Escalate — human decision needed |
+| Access failure | GitHub MCP or `gh` CLI returns auth errors | Report — credentials may need refresh |
+| `assess_stack` returns `escalate` | Composite action determined escalation needed | Report action items to user |
+
+## Escalation Report Format
+
+When escalating, provide:
+
+```markdown
+## Shepherd Escalation — Iteration <N>/<max>
+
+**Reason:** <trigger from table above>
+
+### Persistent Issues
+- <Issue 1>: failed in iterations <list>, last fix attempted: <description>
+- <Issue 2>: ...
+
+### Actions Taken
+- Iteration 1: <summary>
+- Iteration 2: <summary>
+- ...
+
+### Recommendation
+<Suggest next steps — e.g., "re-run with higher limit", "needs human review of X", "flaky test should be skipped">
+```
+
+## User Override
+
+The user can override iteration limits: `/exarchos:shepherd --max-iterations 10`
+
+After escalation, the user may:
+1. Ask to continue with a higher limit
+2. Manually resolve the blocking issue, then re-run `/exarchos:shepherd`
+3. Accept the current state and proceed to `/exarchos:cleanup`
+
+## Non-Escalation Cases
+
+These situations should NOT trigger escalation:
+- CI checks still pending (wait and re-assess)
+- Minor CodeRabbit suggestions (acknowledge and move on)
+- Informational `github-actions[bot]` comments (safe to skip)
+- Stack needs routine restack (fix automatically)

package/skills/generic/shepherd/references/fix-strategies.md

@@ -0,0 +1,289 @@
+# Fix Strategies
+
+How to address common issues found during shepherd assessment.
+
+## Decision: Fix Directly vs. Delegate
+
+| Condition | Approach |
+|-----------|----------|
+| Single file, < 20 lines changed | Fix directly in the stack branch |
+| Multiple files, contained concern | Fix directly if < 5 files |
+| Cross-cutting or architectural | Delegate via `/exarchos:delegate --pr-fixes [PR_URL]` |
+| Test changes needed | Fix directly (keep TDD cycle tight) |
+
+**Default to fixing directly** — delegation adds overhead. Only delegate when the fix scope warrants it.
+
+## Remediation Event Emission
+
+When fixing CI failures or addressing review comments that require code changes, emit remediation events to track self-correction metrics in CodeQualityView.
+
+**When a fix attempt is made** (after applying a code change for a CI failure or review finding):
+```
+mcp__plugin_exarchos_exarchos__exarchos_event({
+  action: "append",
+  stream: "<featureId>",
+  event: {
+    type: "remediation.attempted",
+    data: {
+      taskId: "<taskId>",
+      skill: "shepherd",
+      gateName: "<failing-check-name-or-review-source>",
+      attemptNumber: <N>,
+      strategy: "direct-fix"
+    }
+  }
+})
+```
+
+**When the next iteration confirms the fix resolved the issue:**
+```
+mcp__plugin_exarchos_exarchos__exarchos_event({
+  action: "append",
+  stream: "<featureId>",
+  event: {
+    type: "remediation.succeeded",
+    data: {
+      taskId: "<taskId>",
+      skill: "shepherd",
+      gateName: "<check-name-or-review-source>",
+      totalAttempts: <N>,
+      finalStrategy: "direct-fix"
+    }
+  }
+})
+```
+
+These events feed `selfCorrectionRate` and `avgRemediationAttempts` metrics in CodeQualityView. Emit `remediation.attempted` each time you push a fix, and `remediation.succeeded` when the subsequent assess cycle confirms the issue is resolved.
+
+## CI Failures
+
+### Lint / Format
+
+1. Read the failure details via GitHub MCP:
+   ```
+   mcp__plugin_github_github__pull_request_read({
+     method: "get_status", owner: "<owner>", repo: "<repo>", pullNumber: <number>
+   })
+   ```
+2. Checkout the failing branch:
+   ```bash
+   git checkout <branch-name>
+   ```
+3. Run the linter locally to reproduce:
+   ```bash
+   npm run lint    # or project-specific command
+   ```
+4. Fix the issues
+5. Commit and push:
+   ```bash
+   git add <fixed-files>
+   git commit --amend -m "fix: lint errors"
+   git push --force-with-lease
+   ```
+
+### Test Failures
+
+1. Identify which tests failed from CI output
+2. Checkout the branch and reproduce locally:
+   ```bash
+   npm run test:run
+   ```
+3. Fix the failing tests (maintain TDD — don't delete tests, fix the code or update test expectations if the behavior changed intentionally)
+4. Verify all tests pass locally before pushing
+5. Commit and resubmit
+
+### Build / TypeCheck Failures
+
+1. Reproduce locally:
+   ```bash
+   npm run build && npm run typecheck
+   ```
+2. Fix type errors or build issues
+3. Commit and resubmit
+
+### Flaky Tests
+
+If a test passes locally but fails in CI:
+1. Check if it's a known flaky test
+2. Re-run CI: `gh pr checks <number> --watch` (or push an empty commit to retrigger)
+3. If consistently flaky, fix the test or mark it with a skip annotation and create a follow-up issue
+
+## Addressing Inline Review Comments
+
+**Every inline review comment on every PR must be addressed with a reply.** This applies to ALL sources — Sentry, CodeRabbit, humans, and any other bot that leaves comments.
+
+### Reading Comments
+
+Read all inline review comments for a PR via GitHub MCP:
+```
+mcp__plugin_github_github__pull_request_read({
+  method: "get_review_comments",
+  owner: "<owner>",
+  repo: "<repo>",
+  pullNumber: <number>
+})
+```
+
+Filter to find unaddressed root comments (threads with no replies):
+- Root comments have `in_reply_to_id: null`
+- A thread is addressed if any comment has `in_reply_to_id` equal to the root comment's `id`
+
+### Replying to Comments
+
+Use GitHub MCP tools to reply:
+```
+mcp__plugin_github_github__add_reply_to_pull_request_comment({
+  owner: "<owner>",
+  repo: "<repo>",
+  pullNumber: <number>,
+  commentId: <numeric_comment_id>,
+  body: "<response>"
+})
+```
+
+### Response Categories
+
+For each comment, determine the appropriate response:
+
+| Category | Action | Reply template |
+|----------|--------|---------------|
+| Real bug | Fix the code, then reply | "Fixed — [description of fix]. Test added: `TestName`." |
+| Valid suggestion (implement) | Apply the change, then reply | "Fixed — [description of change]." |
+| Valid suggestion (defer) | Reply with rationale | "Acknowledged — [rationale]. Tracked for Phase N / follow-up." |
+| Intentional design choice | Reply explaining | "Intentional — [explanation of why the current approach is correct]." |
+| Already fixed (outdated) | Reply confirming | "Fixed in [commit/PR description] — [brief explanation]." |
+| False positive | Reply explaining | "[Explanation of why this doesn't apply in this context]." |
+
+### Sentry Comments
+
+Sentry's `[bot]` leaves **bug predictions** — AI-generated analysis of potential runtime issues. These appear as inline review comments with severity tags (CRITICAL, MEDIUM, etc.).
+
+**Sentry comments deserve careful attention because they often identify real bugs** (field name mismatches, type coercion issues, null reference risks).
+
+How to handle:
+1. Read the full comment body — Sentry includes a "Suggested Fix" section
+2. Evaluate whether the bug is real:
+   - Check if the code path is actually reachable
+   - Check if the field names/types match what the data actually provides
+   - Check existing tests — does any test exercise this path?
+3. If real: fix the bug, add a test, reply confirming
+4. If false positive: reply explaining why (e.g., "This path is guarded by X" or "The field is validated at Y before reaching this code")
+
+**Common Sentry findings:**
+- Field name mismatches between producers and consumers
+- Missing null checks on optional fields
+- Type mismatches (string vs. enum, array vs. object)
+- Unreachable error paths due to upstream validation
+
+### CodeRabbit Comments
+
+CodeRabbit leaves detailed code review suggestions with severity indicators. It re-reviews automatically on push, so code fixes may auto-resolve threads.
+
+How to handle:
+1. Read all CodeRabbit comments, noting severity (Critical, Major, Minor)
+2. Critical/Major: Must address — fix or provide strong rationale for not fixing
+3. Minor: Fix if low-effort, otherwise acknowledge
+4. CodeRabbit marks threads as "Addressed in commits" when it detects the code changed — but always verify with a reply
+
+**Common CodeRabbit findings:**
+- Error handling gaps (missing try/catch, bare catches)
+- Code duplication (DRY violations)
+- Style/naming suggestions
+- Performance optimizations
+- Security concerns
+
+### Human Reviewer Comments
+
+Human comments require the most careful handling:
+1. Read comments carefully — understand the full context
+2. For required changes: fix the code, reply confirming
+3. For questions: answer directly on the PR
+4. For suggestions: discuss or implement, reply with decision
+5. For approval with minor nits: fix nits, note the approval
+
+### GitHub Actions Bot Comments
+
+`github-actions[bot]` typically posts automated gate results (review-gate, CI summaries). These are usually **informational** and don't require replies. However, if the gate check shows a failure, investigate the cause.
+
+## Stack Issues
+
+### Needs Rebase
+
+When the base branch (usually `main`) has advanced:
+```bash
+git rebase origin/<base>
+git push --force-with-lease
+```
+
+If rebase has conflicts:
+1. Resolve conflicts in each affected file
+2. `git add <resolved-files>` then continue:
+   ```bash
+   git rebase --continue
+   ```
+3. After resolution, push: `git push --force-with-lease`
+
+### Wrong Base Branch
+
+If a PR targets the wrong base:
+```bash
+gh pr edit <number> --base <correct-base>
+git rebase origin/<correct-base>
+git push --force-with-lease
+```
+
+### Stack Reconstruction
+
+If the stack is in a broken state:
+```typescript
+exarchos_orchestrate({
+  action: "reconstruct_stack"
+})
+```
+
+Then resubmit.
+
+## Commit Strategy for Fixes
+
+When making fixes to stack branches:
+
+1. **Checkout the target branch:**
+   ```bash
+   git checkout <branch-name>
+   ```
+
+2. **Apply fixes and amend:**
+   ```bash
+   git add <fixed-files>
+   git commit --amend -m "fix: <description>"
+   ```
+
+3. **Rebase dependent branches (bottom-up, onto updated parent):**
+   ```bash
+   git rebase <updated-parent-branch>
+   ```
+
+4. **Push the fixes:**
+   ```bash
+   git push --force-with-lease
+   ```
+
+**IMPORTANT:** After pushing, verify auto-merge is still enabled: `gh pr view <number> --json autoMergeRequest`.
+
+## Responding on PRs
+
+When addressing feedback, reply to each comment thread individually using the GitHub MCP `add_reply_to_pull_request_comment` tool. This ensures:
+- Each reviewer sees their specific feedback was acknowledged
+- GitHub marks threads as having replies
+- The PR audit trail shows every concern was addressed
+
+For bulk summaries after a round of fixes, post a general PR comment via GitHub MCP:
+```
+mcp__plugin_github_github__add_issue_comment({
+  owner: "<owner>",
+  repo: "<repo>",
+  issue_number: <number>,
+  body: "Addressed review feedback:\n- Fixed Sentry bug: ...\n- Replied to DI concern...\n\nAll inline review threads have replies."
+})
+```
+Fallback (if MCP token lacks write scope): `gh pr comment <number> --body "..."`

package/skills/generic/shepherd/references/gate-event-emission.md

@@ -0,0 +1,47 @@
+# Gate Event Emission
+
+`gate.executed` events track CI check results for quality analysis in CodeQualityView.
+
+## Event Format
+
+```javascript
+{
+  type: "gate.executed",
+  data: {
+    gateName: "<check-name>",
+    layer: "CI",
+    passed: <true|false>,
+    duration: <duration-ms-if-available>,
+    details: {
+      skill: "shepherd",
+      commit: "<head-sha>"
+    }
+  }
+}
+```
+
+## Emission Source
+
+The `assess_stack` composite action (`exarchos_orchestrate`) automatically emits `gate.executed` events for each CI check it observes. The shepherd skill does **not** need to emit these manually — they are handled internally by `assess_stack`.
+
+If `assess_stack` is unavailable (fallback mode), emit manually via:
+```javascript
+mcp__plugin_exarchos_exarchos__exarchos_event({
+  action: "append",
+  stream: "<featureId>",
+  event: {
+    type: "gate.executed",
+    data: {
+      gateName: "<check-name>",
+      layer: "CI",
+      passed: <true|false>,
+      duration: <duration-ms-if-available>,
+      details: { skill: "shepherd", commit: "<head-sha>" }
+    }
+  }
+})
+```
+
+## Downstream Consumer
+
+CodeQualityView tracks gate pass rates and detects quality regressions. The `gatePassRate` metric per skill drives the quality signal surfaced in Step 0 of the shepherd loop.

package/skills/generic/shepherd/references/shepherd-event-schemas.md

@@ -0,0 +1,56 @@
+# Shepherd Event Schemas
+
+Event data schemas for the four shepherd lifecycle events. All events are appended via `exarchos_event append` to the workflow's stream.
+
+## shepherd.started
+
+Emitted automatically by `assess_stack` on first invocation.
+
+| Field | Type | Description |
+|-------|------|-------------|
+| featureId | string | Workflow feature ID |
+
+## shepherd.iteration
+
+Emitted after each assess → fix → resubmit cycle.
+
+| Field | Type | Description |
+|-------|------|-------------|
+| iteration | number (int, ≥0) | Zero-based iteration counter |
+| prsAssessed | number (int, ≥0) | Count of PRs assessed this iteration |
+| fixesApplied | number (int, ≥0) | Count of fixes applied this iteration |
+| status | string | Iteration outcome (e.g., "resubmitted", "waiting", "escalated") |
+
+```javascript
+mcp__plugin_exarchos_exarchos__exarchos_event({
+  action: "append",
+  stream: "<featureId>",
+  event: {
+    type: "shepherd.iteration",
+    data: {
+      iteration: 1,
+      prsAssessed: 1,
+      fixesApplied: 2,
+      status: "resubmitted"
+    }
+  },
+  idempotencyKey: "<featureId>:shepherd.iteration:<n>"
+})
+```
+
+## shepherd.approval_requested
+
+Emitted when all checks pass and approval is requested.
+
+| Field | Type | Description |
+|-------|------|-------------|
+| prUrl | string | URL of the PR ready for approval |
+
+## shepherd.completed
+
+Emitted automatically by `assess_stack` when a PR is detected as merged.
+
+| Field | Type | Description |
+|-------|------|-------------|
+| prUrl | string | URL of the merged PR |
+| outcome | string | Completion outcome (e.g., "merged") |