devflow-kit 3.3.0 → 3.4.0

package/dist/learning-off/commands/bug-analysis.md

@@ -0,0 +1,420 @@
+---
+description: Proactive bug finding with static and semantic analysis — hunts real bugs in changed code before merge
+---
+# Bug Analysis Command
+
+Run a proactive bug analysis on the current branch by combining static analysis tools with parallel semantic analyzers, then synthesizing results into an actionable bug report. Supports incremental analysis, timestamped report directories, and `/resolve` compatibility.
+
+## Usage
+
+```
+/bug-analysis              (analyze current branch — incremental if prior analysis exists)
+/bug-analysis --full       (force full analysis, ignore incremental state)
+/bug-analysis --no-static  (skip static analysis track, semantic-only)
+```
+
+## Phases
+
+### Phase 1: Pre-flight
+
+**Produces:** BRANCH_INFO, PR_DESCRIPTION, EVIDENCE_POLICY, ISSUE_REQUIRED, APPLY_CONVENTIONS, REQUIRE_NON_AUTHOR_APPROVAL
+
+**Resolve the evidence policy once per run**, from the repository root, before any step reads the values:
+
+```bash
+node "$HOME/.devflow/scripts/resolve-evidence-policy.cjs" 2>/dev/null; echo "exit=$?"
+```
+
+Accept the output only when it is exactly two lines: `exit=0` last and, before it, one line of the form `EVIDENCE_POLICY=<required|standard> SOURCE=<file|worktree|default|invalid|error> REF=<branch|none>[ WARN=<remote-unavailable|invalid-file|raised-by-compliance|pr-changes-policy>[,…]] ISSUE_REQUIRED=<true|false> APPLY_CONVENTIONS=<true|false> REQUIRE_NON_AUTHOR_APPROVAL=<true|false>` — these fields, in this order, nothing else, where `<branch>` is a branch name such as `main`. **Anything else** (a non-zero exit, no line, extra text, or a missing, reordered or unlisted field or value) ⇒ use `EVIDENCE_POLICY=required SOURCE=error REF=none ISSUE_REQUIRED=true APPLY_CONVENTIONS=true REQUIRE_NON_AUTHOR_APPROVAL=true` instead.
+
+Set `EVIDENCE_POLICY`, `ISSUE_REQUIRED`, `APPLY_CONVENTIONS` and `REQUIRE_NON_AUTHOR_APPROVAL` from the accepted line. Pass agents only the three mechanism inputs, never `EVIDENCE_POLICY`. Report `Evidence policy: {EVIDENCE_POLICY} (source: {SOURCE})`, plus any `WARN` tokens as advisory, once in the final report.
+
+**Docs root (D-DOCS-ROOT).** Every `.devflow/docs/` path this command reads or writes lives at the checkout's toplevel, never under the directory the session started in. Resolve `{worktree}` from the start directory — `WORKTREE_PATH` if provided, otherwise cwd (`devflow:worktree-support`) — by running
+
+```bash
+git -C "{start}" rev-parse --show-toplevel
+```
+
+and using its one-line output. If the command fails (outside a git repository), `{worktree}` is the start directory itself. Every docs path below is written `{worktree}/.devflow/docs/…`; a repo-relative docs path handed to an agent always travels with a `WORKTREE_PATH` naming the checkout it is relative to.
+
+Render the test-plan block from `/implement`'s evidence file, and from nothing else:
+1. `branch_slug` is `git branch --show-current` with every `/` replaced by `-`.
+2. Only when `branch_slug` matches `^[A-Za-z0-9._-]{1,200}$` and the file exists, run (the path double-quoted):
+
+```bash
+node "$HOME/.devflow/scripts/verify-evidence.cjs" render --plan "{worktree}/.devflow/docs/evidence-{branch_slug}.md"; echo "exit=$?"
+```
+
+3. On `exit=0`, `PR_TEST_PLAN_BLOCK` is its stdout byte for byte without that `exit=` line; in every other case it is `(none)`. Nothing here waits on it: the Git agent pastes it only behind its own check, and only into a PR it creates.
+
+Spawn Git agent:
+
+```
+Agent(subagent_type="Git", run_in_background=false):
+"OPERATION: ensure-pr-ready
+PR_TEST_PLAN_BLOCK: {PR_TEST_PLAN_BLOCK verbatim, or (none)}
+APPLY_CONVENTIONS: {APPLY_CONVENTIONS}
+Validate branch, commit if needed, push, create PR if needed.
+Return: branch, base_branch, branch-slug, PR#"
+```
+
+**Extract from response:** `branch`, `base_branch`, `branch_slug`, `pr_number`.
+
+**Fetch PR body** (after extracting `pr_number`):
+```bash
+PR_DESCRIPTION=$(gh pr view {pr_number} --json body --jq '.body' 2>/dev/null || echo "(none)")
+```
+If `pr_number` is absent or the command fails, set `PR_DESCRIPTION` to `(none)`.
+
+### Phase 2: Static Analysis
+
+**Requires:** BRANCH_INFO
+
+#### Step 2a: Incremental Detection & Timestamp Setup
+
+**Produces:** DIFF_RANGE, ANALYSIS_DIR
+**Requires:** BRANCH_INFO
+
+1. Check `{worktree}/.devflow/docs/bug-analysis/{branch-slug}/.last-analysis-head`:
+   - **If exists AND `--full` NOT set:**
+     - Read the SHA from the file
+     - Verify reachable: `git cat-file -t {sha}` — if exit code non-zero (rebase invalidated SHA), fall through to full
+     - If SHA == current HEAD → "No new commits since last analysis. Use --full for a full re-analysis." Stop.
+     - Set `DIFF_RANGE` to `{sha}...HEAD`
+   - **If not exists, unreachable SHA, or `--full`:**
+     - Set `DIFF_RANGE` to `{base_branch}...HEAD`
+2. Generate timestamp: `YYYY-MM-DD_HHMM`. If directory already exists (same-minute collision), append seconds (`YYYY-MM-DD_HHMMSS`).
+3. Create timestamped analysis directory: `mkdir -p "{worktree}/.devflow/docs/bug-analysis/{branch-slug}/{timestamp}/"`
+4. Set `ANALYSIS_DIR` to that path.
+
+#### Step 2b: Check Changed Files
+
+**Requires:** DIFF_RANGE
+
+```bash
+CHANGED_FILES=$(git diff --name-only {DIFF_RANGE})
+```
+
+Store result as `CHANGED_FILES` — used throughout Steps 2d and Phase 4 to avoid repeated git invocations and ensure consistency.
+
+If output is empty → "No changes to analyze." Stop.
+
+#### Step 2c: Tool Availability Check
+
+**Produces:** STATIC_TOOL_STATUS
+
+Skip if `--no-static` flag provided.
+
+```bash
+SEMGREP_AVAILABLE=$(which semgrep 2>/dev/null && echo "yes" || echo "no")
+SNYK_AVAILABLE=$(which snyk 2>/dev/null && echo "yes" || echo "no")
+CODEQL_AVAILABLE=$(which codeql 2>/dev/null && echo "yes" || echo "no")
+```
+
+If all are `no`: warn user "No static analysis tools found. Proceeding with semantic analysis only. To enable static analysis, install semgrep (`pip install semgrep`) or snyk (`npm install -g snyk`)." Set `STATIC_FINDINGS` to `(none)`.
+
+If some are available: note which tools will run.
+
+#### Step 2d: Tiered Static Analysis
+
+**Produces:** STATIC_FINDINGS
+**Requires:** STATIC_TOOL_STATUS, DIFF_RANGE, ANALYSIS_DIR
+
+Skip if `--no-static` flag provided or all tools unavailable.
+
+Run available tools on the changed files (from `CHANGED_FILES` computed in Step 2b).
+
+**Semgrep and Snyk run in parallel** — launch both in the background, then wait for both before proceeding to CodeQL. CodeQL is conditional and sequential (it needs Semgrep/Snyk results to decide whether to run).
+
+**Semgrep** (if available):
+```bash
+# tr '\n' '\0' + xargs -0 is portable across GNU and BSD xargs (macOS ships BSD xargs, which lacks -d)
+echo "$CHANGED_FILES" | tr '\n' '\0' | xargs -0 timeout 300 semgrep scan --config auto --sarif --quiet 2>/dev/null
+```
+Parse SARIF output → extract findings.
+
+**Snyk Code** (if available):
+```bash
+# Run a single project-level scan; filter SARIF results programmatically to CHANGED_FILES before LLM processing.
+# Per-file invocation via xargs would invoke snyk O(n) times and --file is for dependency scanning, not source code.
+SNYK_SARIF=$(timeout 300 snyk code test --sarif 2>/dev/null)
+# Programmatic filter: keep only results whose file path appears in CHANGED_FILES.
+# This is defense-in-depth — filtering at the data layer before the LLM agent sees any output.
+SNYK_FILTERED=$(echo "$SNYK_SARIF" | jq --argjson files "$(echo "$CHANGED_FILES" | jq -R . | jq -s .)" \
+  '.runs[].results |= map(select(.locations[].physicalLocation.artifactLocation.uri as $uri | $files | index($uri) != null))' \
+  2>/dev/null || echo "$SNYK_SARIF")
+```
+Parse `SNYK_FILTERED` SARIF → extract findings. If `jq` is unavailable, fall back to the raw `SNYK_SARIF` and note the limitation.
+
+**CodeQL** (if available AND (`--full` OR Semgrep/Snyk found HIGH/CRITICAL findings)):
+```bash
+# Use a unique temp directory per run to prevent symlink attacks and concurrent-process clobbering
+CODEQL_TMP=$(mktemp -d)
+# Register cleanup trap immediately after mktemp — ensures rm -rf runs on EXIT, SIGTERM, and SIGINT
+# even if the session is interrupted before reaching the explicit rm -rf below
+trap 'rm -rf "${CODEQL_TMP}"' EXIT INT TERM
+timeout 600 codeql database create "${CODEQL_TMP}/db" --language={detected-language} --source-root=. 2>/dev/null && \
+timeout 600 codeql database analyze "${CODEQL_TMP}/db" --format=sarif-latest --output="${CODEQL_TMP}/results.sarif" 2>/dev/null
+# Capture exit status before cleanup so cleanup doesn't mask failures
+CODEQL_EXIT=$?
+# Parse SARIF output BEFORE cleanup — rm -rf destroys results.sarif
+CODEQL_SARIF=$(cat "${CODEQL_TMP}/results.sarif" 2>/dev/null || echo "")
+# Explicit cleanup (trap also covers this path; explicit rm is belt-and-suspenders)
+rm -rf "${CODEQL_TMP}"
+trap - EXIT INT TERM
+```
+Parse `CODEQL_SARIF` → extract findings. If database creation fails, `CODEQL_EXIT` is non-zero — skip CodeQL findings and note it. The `trap` guarantees cleanup on SIGTERM/SIGINT/EXIT so orphaned temp directories do not accumulate across interrupted sessions.
+
+**Normalize** all findings to unified table, cap at top 50 by severity. Truncate each Description entry to 200 characters maximum to bound the serialized size of `STATIC_FINDINGS`:
+
+| Tool | File:Line | CWE | Severity | Title | Description |
+|------|-----------|-----|----------|-------|-------------|
+| {tool} | {file}:{line} | {CWE or —} | {CRITICAL/HIGH/MEDIUM/LOW} | {title} | {description truncated to 200 chars} |
+
+Write ALL raw findings to `{ANALYSIS_DIR}/static-findings.md`. Set `STATIC_FINDINGS` to the top-50 table.
+
+If no tool produced findings: set `STATIC_FINDINGS` to `(none)`.
+
+### Phase 3: Context Loading
+
+**Produces:** FEATURE_KNOWLEDGE, FEATURE_KNOWLEDGE_RULES, PLAN_CONTEXT, ACCEPTANCE_RULES
+
+#### Feature Knowledge
+
+### Load Feature Knowledge
+
+Resolve `{worktree}` as the checkout's toplevel, because feature knowledge bases are committed with the branch (D-PROMPT-ROOT): from the start directory — `WORKTREE_PATH` if provided, otherwise cwd (`devflow:worktree-support`) — run
+
+```bash
+git -C "{start}" rev-parse --show-toplevel
+```
+
+and use its one-line output. If the command fails (outside a git repository), `{worktree}` is the start directory itself. All paths below are relative to `{worktree}`.
+
+**Step 1 — Read the index cache:**
+
+Attempt to read `{worktree}/.devflow/features/index.md`. Each line follows the format:
+
+```
+- **{slug}** — {areas} — {Use-when description}
+```
+
+If `index.md` exists and contains at least one entry line, use it for relevance matching.
+
+**Step 2 — Fallback: glob frontmatter (if `index.md` is absent or empty):**
+
+Glob `{worktree}/.devflow/features/*/KNOWLEDGE.md`. For each file found, read only its YAML frontmatter block (between the opening and closing `---` delimiters). The frontmatter fields `name`, `description`, and `directories` are the authoritative relevance surface — `index.md` is only a cache.
+
+**Step 3 — Pick relevant KBs:**
+
+Match the current task area and description against each index line (or frontmatter `description` + `directories` on fallback). Select entries whose documented area overlaps the current task. This is a relevance judgment — prefer specificity over breadth.
+
+**Step 4 — Read each selected KB's Rules:**
+
+For each selected entry, `{kb}` is `{worktree}/.devflow/features/{slug}/KNOWLEDGE.md`:
+
+1. List its `##` headings with line numbers through Bash: `command grep -n '^## ' "{kb}"`. This only locates sections; the text of a KB comes from the Read view alone.
+2. Read the `## Rules` range (its line to the next heading) with the Read tool, using `offset` and `limit`, and choose the one to three bullets most relevant to the current task. The choice is yours, made per KB.
+3. If the KB has no `## Rules` section, choose one to three entries from its `## Anti-Patterns` or `## Gotchas` range the same way, and label them by that section's name instead of an ID.
+
+When a KB contradicts the code you observe, **trust the code** — the code is the freshness mechanism; the KB may lag behind. A missing Rules section, a missing heading list and `(none)` are legitimate states, not errors.
+
+**Step 5 — Set FEATURE_KNOWLEDGE and FEATURE_KNOWLEDGE_RULES:**
+
+Write one block per selected KB. Paste each bullet verbatim from the Read view, never from a shell view. The path is relative to the checkout root; an agent resolves it under `WORKTREE_PATH` when one is provided.
+
+```
+--- Feature knowledge: {slug} ---
+KB: .devflow/features/{slug}/KNOWLEDGE.md
+Rules:
+- **KB-AP-2** {bullet text, verbatim}
+- **KB-INV-1** {bullet text, verbatim}
+Headings: L5 Rules · L40 Overview · L62 Anti-Patterns · L118 Key Files
+```
+
+A KB with no Rules section labels its entries `Rules ({section name}):` and gives them no ID. `FEATURE_KNOWLEDGE` is these blocks; `FEATURE_KNOWLEDGE_RULES` is the same blocks without the `Headings:` line. Both come from this one selection, and each spawn names the variable its recipient takes. If no KBs exist, no KBs are relevant, or `.devflow/features/` is absent, set both to `(none)`.
+
+**One git call, then direct reads — no `.cjs` script.** After resolving `{worktree}`, this step is 1 index read (or N frontmatter reads on fallback), plus one heading listing and one Rules read per selected KB, bounded by KB count.
+
+#### Plan Artifact
+
+1. List `{worktree}/.devflow/docs/design/*.md` — sort descending by filename (timestamps are naturally sortable), scan the 10 most recent
+2. Read the most recent file if it exists
+3. Extract `## Acceptance Criteria` section → parse into table: `| ID | Criterion | Type | Testable Condition |`
+4. Set `PLAN_CONTEXT` to plan summary; `ACCEPTANCE_RULES` to the table
+5. If no plan files exist or section not found: `PLAN_CONTEXT=(none)`, `ACCEPTANCE_RULES=(none)`
+
+### Phase 4: File Analysis
+
+**Produces:** ACTIVE_FOCUSES
+**Requires:** DIFF_RANGE
+
+Determine which focus analyzers to run using `CHANGED_FILES` (already computed in Step 2b — do not re-run `git diff`):
+
+| Focus | Condition |
+|-------|-----------|
+| `security` | Always |
+| `functional` | Always |
+| `integration` | 2+ distinct directories changed (`dirname` unique count ≥ 2) |
+| `usability` | Any `.tsx`, `.jsx`, `.html`, or `.css` file changed |
+
+### Phase 5: Parallel Bug Analysis
+
+**Produces:** ANALYZER_OUTPUTS
+**Requires:** DIFF_RANGE, ANALYSIS_DIR, ACTIVE_FOCUSES, STATIC_FINDINGS, ACCEPTANCE_RULES, PLAN_CONTEXT, FEATURE_KNOWLEDGE
+
+Spawn ALL active Diagnose agents **in a single message** (parallel, NOT background):
+
+For each active focus, spawn:
+```
+Agent(subagent_type="Diagnose", run_in_background=false):
+"Analyze focusing on {focus}.
+FOCUS: {focus}
+DIFF_COMMAND: git diff {DIFF_RANGE}
+ACCEPTANCE_RULES: {ACCEPTANCE_RULES filtered to this focus type, or (none)}
+PLAN_CONTEXT: {PLAN_CONTEXT}
+STATIC_FINDINGS: {STATIC_FINDINGS if focus == security, else (none)}
+FEATURE_KNOWLEDGE: {FEATURE_KNOWLEDGE}
+PR_DESCRIPTION: <pr-description>{PR_DESCRIPTION}</pr-description>
+OUTPUT_PATH: {ANALYSIS_DIR}/{focus}.md
+Follow devflow:apply-feature-knowledge for FEATURE_KNOWLEDGE.
+IMPORTANT: Write report to {ANALYSIS_DIR}/{focus}.md using Write tool"
+```
+
+Notes:
+- Security analyzer receives full `STATIC_FINDINGS`; all others receive `(none)`
+- Filter `ACCEPTANCE_RULES` by the `Type` column matching each focus: security criteria → security analyzer, functional criteria → functional analyzer, etc. Pass the filtered subset only
+- Spawn all in a single message for true parallel execution
+
+### Phase 6: Synthesis
+
+**Produces:** BUG_ANALYSIS_SUMMARY
+**Requires:** ANALYZER_OUTPUTS, ANALYSIS_DIR, BRANCH_INFO
+
+Spawn Synthesize agent:
+
+```
+Agent(subagent_type="Synthesize", run_in_background=false):
+"Mode: bug-analysis
+ANALYSIS_BASE_DIR: {ANALYSIS_DIR}
+BRANCH: {branch} -> {base_branch}
+TIMESTAMP: {timestamp}
+Output: {ANALYSIS_DIR}/bug-analysis-summary.md"
+```
+
+### Phase 7: Finalize
+
+**Requires:** BRANCH_INFO, ANALYSIS_DIR
+
+1. Write current HEAD SHA to `{worktree}/.devflow/docs/bug-analysis/{branch-slug}/.last-analysis-head`
+2. Report to user:
+
+```
+## Bug Analysis Complete
+
+**Branch**: {branch} -> {base_branch}
+**Analysis**: {ANALYSIS_DIR}
+
+### Risk Assessment: {risk_level}
+
+{brief_reasoning}
+
+### Bug Counts
+| Category | CRITICAL | HIGH | MEDIUM | LOW | Total |
+|----------|----------|------|--------|-----|-------|
+| Security | {n} | {n} | {n} | {n} | {n} |
+| Functional | {n} | {n} | {n} | {n} | {n} |
+| Integration | {n} | {n} | {n} | {n} | {n} |
+| Usability | {n} | {n} | {n} | {n} | {n} |
+
+### Top Findings
+{List top 3-5 bugs by severity and confidence}
+
+### Artifacts
+- Bug report: {ANALYSIS_DIR}/bug-analysis-summary.md
+- Per-focus reports: {ANALYSIS_DIR}/{security|functional|integration|usability}.md
+- Static findings: {ANALYSIS_DIR}/static-findings.md (if static analysis ran)
+
+{if any CRITICAL or HIGH bugs found:}
+Run `/resolve` to process and fix these findings.
+```
+
+## Architecture
+
+```
+/bug-analysis (orchestrator — spawns agents only)
+│
+├─ Phase 1: Pre-flight
+│  └─ Git agent (ensure-pr-ready)
+│
+├─ Phase 2: Static Analysis
+│  ├─ Step 2a: Incremental detection + timestamp setup
+│  ├─ Step 2b: Check changed files (stop if none)
+│  ├─ Step 2c: Tool availability check
+│  └─ Step 2d: Tiered static analysis (semgrep → snyk → codeql)
+│              Write static-findings.md
+│
+├─ Phase 3: Context Loading
+│  ├─ Feature knowledge load → FEATURE_KNOWLEDGE
+│  └─ {worktree}/.devflow/docs/design/*.md → PLAN_CONTEXT + ACCEPTANCE_RULES
+│
+├─ Phase 4: File Analysis
+│  └─ Detect active focuses (security + functional always; integration + usability conditional)
+│
+├─ Phase 5: Bug Analysis (PARALLEL)
+│  ├─ Diagnose agent: security (+ STATIC_FINDINGS)
+│  ├─ Diagnose agent: functional
+│  ├─ Diagnose agent: integration (conditional)
+│  └─ Diagnose agent: usability (conditional)
+│
+├─ Phase 6: Synthesis
+│  └─ Synthesize agent (mode: bug-analysis)
+│
+└─ Phase 7: Finalize
+   ├─ Write .last-analysis-head
+   └─ Display results + suggest /resolve if blocking bugs found
+```
+
+## Edge Cases
+
+| Case | Handling |
+|------|----------|
+| No new commits since last analysis | Stop: "No new commits since last analysis. Use --full for a full re-analysis." |
+| Rebase invalidates `.last-analysis-head` SHA | `git cat-file -t` check fails → fallback to full diff |
+| Zero changed files in DIFF_RANGE | Stop: "No changes to analyze." |
+| Same-minute analysis collision | `mkdir` with seconds suffix (`YYYY-MM-DD_HHMMSS`) |
+| All static tools unavailable | Warn, proceed with semantic-only analysis |
+| `--no-static` flag | Skip Phase 2c and 2d entirely; `STATIC_FINDINGS=(none)` |
+| `--full` flag | Bypass incremental detection (Step 2a), run full diff from base |
+| CodeQL database creation fails | Skip CodeQL, note in output, continue with other tools |
+| Static tool produces no findings | `STATIC_FINDINGS=(none)` — normal, proceed with semantic analysis |
+| No plan artifact found | `PLAN_CONTEXT=(none)`, `ACCEPTANCE_RULES=(none)` — proceed without acceptance criteria |
+| `integration` focus skipped | Not spawned when only 1 directory changed |
+| `usability` focus skipped | Not spawned when no UI files changed |
+
+## Resolve Compatibility
+
+Run `/resolve` after `/bug-analysis` to fix identified bugs. `/resolve` automatically detects and uses bug analysis reports when no code review report exists.
+
+## Principles
+
+1. **Orchestration only** — Command spawns agents, doesn't do analysis work itself
+2. **Parallel, not background** — Analyzers spawn in one message with `run_in_background=false`
+3. **Static + semantic** — Two complementary tracks, each validates the other
+4. **Incremental by default** — Only analyze new changes unless `--full` specified
+5. **Verify before reporting** — Diagnose agents self-verify every finding
+6. **Honest reporting** — Display risk level and counts directly from synthesis
+
+## Phase Completion Checklist
+
+Before reporting results, verify every phase was executed:
+
+- [ ] Phase 1: Pre-flight → BRANCH_INFO captured, PR_DESCRIPTION fetched (or `(none)`)
+- [ ] Phase 2: Static Analysis → STATIC_FINDINGS captured (or `(none)` if skipped/no tools/no findings); ANALYSIS_DIR created; CHANGED_FILES populated
+- [ ] Phase 3: Context Loading → FEATURE_KNOWLEDGE loaded (or `(none)`), PLAN_CONTEXT and ACCEPTANCE_RULES captured (or `(none)`)
+- [ ] Phase 4: File Analysis → ACTIVE_FOCUSES determined (security + functional always present)
+- [ ] Phase 5: Parallel Bug Analysis → ANALYZER_OUTPUTS captured per active focus; all reports written to ANALYSIS_DIR
+- [ ] Phase 6: Synthesis → BUG_ANALYSIS_SUMMARY written to `{ANALYSIS_DIR}/bug-analysis-summary.md`
+- [ ] Phase 7: Finalize → `.last-analysis-head` updated; results displayed to user
+
+If any phase is unchecked, execute it before proceeding.