lastlight 0.1.8 → 0.2.0

package/skills/security-review/SKILL.md

@@ -0,0 +1,213 @@
+---
+name: security-review
+description: Diff-scoped security review of SDLC concerns GitHub's scanners miss — workflow/CI hardening, auth changes, secret handling, supply-chain churn. Files one dated summary issue with a task-list of findings. Use on a security cron or when asked to scan a repo.
+version: 2.0.0
+tags: [github, security, review]
+---
+
+# Security Review
+
+Review **what changed in the repo since the last scan** with a security lens,
+focused on SDLC concerns that GitHub's built-in scanners (Dependabot, Code
+Scanning, Secret Scanning) and Renovate don't cover. File **one summary issue
+per run**, dated, containing a GitHub task list of any findings. Honour
+`SECURITY.md` to suppress accepted risks and false positives.
+
+This is **not** a general vulnerability scanner. It does not run `npm audit`
+(Dependabot does). It does not run `semgrep --config auto` over the whole tree
+(Code Scanning does). It looks at the diff since the prior scan and surfaces what
+humans introduced — CI hardening, auth changes, secret handling in new code,
+supply-chain churn.
+
+The summary issue's structure is the machine-parsed contract a maintainer's
+later comment is processed against (the `security-feedback` skill). It is defined
+in **[references/issue-format.md](references/issue-format.md)** — follow it
+exactly when composing the issue (§9).
+
+## Context
+
+- `context.repo` — `owner/name` to scan
+- `context.deliverSlackSummary` — if true, output a one-line Slack summary as the final response
+- `context.issueDir` — directory for the run summary file (e.g. `.lastlight/security-<date>`)
+
+## Procedure
+
+### 1. Clone, find the prior-scan anchor, read SECURITY.md
+
+1. Clone the target repo via `github_clone_repo`.
+2. **Prior scan anchor.** Query issues labelled `security-scan` (open OR closed),
+   newest first. Read the latest one's body for the
+   `<!-- lastlight-security-scan-ts: ... -->` comment and use that ISO-8601
+   timestamp as `priorScanTs`. No prior scan issue → `priorScanTs = now - 30 days`
+   (bootstrap floor — keeps the first run finite).
+3. Read `SECURITY.md` at the repo root (if present): per-tool severity floors
+   (default `medium`; skip `low`/`info`), the accepted-risks table, and the
+   false-positives table (both keyed by finding fingerprint).
+
+### 1.5. Compute the changeset
+
+Most weeks the diff is dominated by Renovate/Dependabot churn — strip it so the
+scope is what humans wrote.
+
+1. `git log --since="${priorScanTs}" --pretty=format:'%H|%an|%ae|%s'`.
+2. Drop a commit when **either**: author email matches
+   `*[bot]@users.noreply.github.com` and name is `dependabot[bot]` /
+   `renovate[bot]` / `github-actions[bot]`; OR the subject starts with
+   `chore(deps)`, `chore(deps-dev)`, `build(deps)`, `build(deps-dev)`, `fix(deps)`.
+3. Accumulate changed files from the surviving commits
+   (`git diff-tree --no-commit-id --name-only -r ${sha}`) into a deduped
+   `changedFiles` set.
+4. Drop entries that are **only** lockfiles (`package-lock.json`, `pnpm-lock.yaml`,
+   `yarn.lock`, `bun.lockb`, `Gemfile.lock`, `poetry.lock`, `uv.lock`,
+   `Cargo.lock`, `composer.lock`) — catches lockfile changes mixed into human commits.
+5. Build `commitsReviewed` = surviving commits as `{ shortSha, subject }` (used in the scope note).
+
+**Early exit.** If `changedFiles` is empty after filtering, **stop**: run no
+scanner, create no issue, emit no Slack message. Write the run summary file (§10)
+recording "no relevant changes since prior scan" and return.
+
+### 2. Ensure labels exist
+
+`github_create_label` for each (idempotent — ignore 422):
+
+| Label | Color | Purpose |
+|-------|-------|---------|
+| `security` | `ee0701` | Any security-related issue |
+| `security-scan` | `fbca04` | The per-run summary issue |
+| `p0-critical` | `b60205` | Severity |
+| `p1-high` | `d93f0b` | Severity |
+| `p2-medium` | `fbca04` | Severity |
+| `p3-low` | `0e8a16` | Severity |
+
+### 3. Run change-scoped scanners
+
+Three sources, all narrowed to the changeset. Do **not** run `npm audit` or
+`semgrep --config auto .` over the whole tree (covered elsewhere; the noise
+drowns the signal).
+
+- **Gitleaks (commit range):**
+  `gitleaks detect --source . --log-opts="--since=${priorScanTs}" --report-format json --report-path /tmp/gitleaks.json`
+  — secrets introduced in the new history.
+- **Semgrep (changed files only):**
+  `semgrep --config auto --json $(printf -- '--include=%s ' "${changedFiles[@]}")`
+  — only files that changed; far less noise.
+- **Claude SDLC review** — the unique value-add. Read `git diff ${priorScanTs}..HEAD`
+  plus the current contents of changed files against the checklist below. Each
+  match becomes a finding with `tool: "claude"` and the §4 severity.
+
+  - **GitHub Actions / CI** (`.github/workflows/*.yml`): actions pinned by
+    floating ref (`@main`/`@master`/`@v1`) not a commit SHA; `pull_request_target`
+    that checks out the PR head; missing top-level/job `permissions:` block;
+    `${{ secrets.* }}` interpolated into shell/`run:`/echo where it can land in
+    logs; untrusted PR body/title/branch interpolated into a `run:` block.
+  - **Dockerfile / compose**: base images on floating tags (no digest) introduced
+    here; new `RUN curl … | sh` / `wget … | bash`; new `--privileged`/`--cap-add`,
+    removed `security_opt`/`read_only` hardening; new host-exposed ports without need.
+  - **Auth / authorization**: modified middleware, route guards, role checks, CORS,
+    JWT verification, OAuth handlers, webhook signature verification (HMAC compare,
+    `crypto.timingSafeEqual` replaced with `===`).
+  - **Secret handling in new code**: new `process.env.*` reads whose value flows
+    into a log or HTTP response; new code logging Authorization headers/cookies/tokens;
+    hardcoded key/URL-shaped literals gitleaks missed.
+  - **Shell exec on attacker-influenced args**: new `execSync`/`exec`/`spawn` where
+    any argument is non-static (concatenation, interpolation, request-derived).
+  - **Supply-chain churn** (`package.json` diff): **new** top-level
+    `dependencies`/`devDependencies` entries (not version bumps — filtered at §1.5);
+    flag package name + publisher, higher severity for typosquat-shaped names;
+    removed integrity controls (`npm ci` → `npm install`, dropped `--ignore-scripts`
+    or provenance flags).
+  - **Release / publish flows**: changes to publish scripts, `npm publish`, release
+    CI steps, signing keys — anything touching what users download.
+
+  If the only changes are docs/tests/unrelated config and none of the above
+  applied, the run legitimately has **no findings** — proceed to §8.
+
+### 4. Normalize findings
+
+Convert each to: `{ fingerprint, severity, tool, rule, file, line, title,
+language, snippet, explanation, suggestedFix }`.
+
+- `fingerprint` = `sha1(tool + ":" + rule + ":" + file + ":" + 3-line-context)`, lowercase hex.
+- `severity` ∈ `p0-critical | p1-high | p2-medium | p3-low`.
+- `tool` ∈ `npm-audit | semgrep | gitleaks | claude` (lowercase, hyphenated).
+- `rule` = tool-native id, as-is. `file` = repo-relative, forward slashes.
+- `line` = 1-based; `0` when not line-scoped. `title` = one line, no backticks/asterisks.
+
+Severity mapping: npm-audit critical/high/moderate/low → p0/p1/p2/p3;
+semgrep ERROR/WARNING/INFO → p1/p2/p3; gitleaks (all) → p1-high;
+claude critical/high/medium/low → p0/p1/p2/p3.
+
+### 5. Apply severity floor
+
+Drop findings below the SECURITY.md floor (default `medium` → drops `p3-low`).
+
+### 6. Filter accepted risks / false positives
+
+Drop findings whose fingerprint prefix (first 16 hex chars) appears in the
+SECURITY.md accepted-risks or false-positives tables.
+
+### 7. Sort and cap
+
+Sort by `(severity rank p0<p1<p2<p3, then file asc, then line asc)`.
+
+The issue body has a hard 65,536-char GitHub limit. Cap: keep **ALL** `p0-critical`
+and **ALL** `p1-high`; keep at most **10** of `p2-medium`+`p3-low` combined (the
+first 10 after the sort). Assign 1-based `item` numbers top-to-bottom across the
+**kept** findings. `overflow` = survived-filtering minus kept; surface it in the
+overflow note.
+
+The cap intentionally puts no ceiling on critical/high — if a misconfigured
+scanner emits hundreds of highs and the body still overflows, raise the
+`SECURITY.md` floor or tune the scanner's rule severities (fix at the source, not
+by hiding real findings).
+
+### 8. Early exit: no findings
+
+If the filtered-and-capped list is empty, **do not** create the summary issue.
+Write the run summary file (§10) recording why and return **silently** — no Slack
+message. The cron is intentionally low-noise: only actual findings surface.
+
+### 9. Compose and create the summary issue
+
+Render the body per [references/issue-format.md](references/issue-format.md) and
+`github_create_issue` with `title: Security scan — {YYYY-MM-DD}` (UTC),
+`labels: ["security", "security-scan"]`. Record `summaryIssueNumber`. Do **not**
+touch prior `security-scan` issues — each scan is a point-in-time snapshot.
+
+### 10. Write the run summary file
+
+Write `{issueDir}/security-summary.md`:
+
+```markdown
+# Security Scan Summary — {repo}
+
+**Date**: {YYYY-MM-DD}
+**Prior scan anchor**: {priorScanTs} (issue #{priorScanIssueNumber} or "bootstrap floor")
+**Commits reviewed**: {N} (after filtering Renovate/Dependabot/lockfile-only)
+**Changed files**: {nFiles}
+**Summary issue**: #{summaryIssueNumber} (or "none — no findings")
+
+**Scanner raw counts**: gitleaks: {n}, semgrep: {n}, claude: {n}
+**After severity floor**: {n}
+**After SECURITY.md filtering**: {n} (filed)
+**Suppressed**: {n} (accepted: {nA}, false-positive: {nFP})
+{if overflow > 0}: **Overflow**: {overflow} lower-severity findings omitted (cap: ALL critical/high + first 10 medium/low)
+```
+
+On a §1.5 early exit, still write the file and state `**Early exit**: no human
+commits since prior scan`.
+
+### 11. Slack summary (optional)
+
+If `context.deliverSlackSummary` is true, output as the final response:
+
+- **With findings:**
+  ```
+  *Security scan: {repo}* — {n} findings filed in #{summaryIssueNumber} ({N} commits since {priorScanDate})
+  Critical: {nC} · High: {nH} · Medium: {nM} · Low: {nL}
+  ```
+- **No findings** (scanners ran clean) or **§1.5 early exit** → emit nothing;
+  silence matches the cron's low-noise design. The run summary file still records
+  what was reviewed.
+
+Otherwise output the run summary file contents as the final response.

package/skills/security-review/references/issue-format.md

@@ -0,0 +1,302 @@
+# § Issue format
+
+This is the **contract** between `security-review` (producer) and
+`security-feedback` (consumer). Every rule here is machine-parsed; do not
+deviate. `security-feedback` is staged into a separate workspace and cannot read
+this file, so it keeps its own copy of the row/severity grammar — **if you change
+the grammar here, update `skills/security-feedback/SKILL.md` in lockstep.**
+
+## Title
+
+```
+Security scan — YYYY-MM-DD
+```
+
+- Exactly one em-dash (` — `, U+2014), surrounded by single spaces.
+- Date is the scan's UTC date in ISO form.
+- Same-day re-scans produce a second issue with the same title. GitHub disambiguates by issue number; the scanner never edits a prior-run issue.
+
+## Body
+
+The body is assembled from eight blocks, in this exact order, separated by blank lines:
+
+```
+{header comments}
+
+{intro paragraph}
+
+{how-to-respond section}
+
+{summary table}
+
+{suppression note}
+
+{scope note}
+
+{overflow note — omitted when overflow == 0}
+
+{findings sections}
+```
+
+### Block 1 — header comments
+
+Three HTML comments, each on its own line, in this exact order:
+
+```
+<!-- lastlight-security-scan-version: 1 -->
+<!-- lastlight-security-scan-date: YYYY-MM-DD -->
+<!-- lastlight-security-scan-ts: YYYY-MM-DDTHH:MM:SSZ -->
+```
+
+- `version` is a format version. Bump if the structure changes incompatibly — `security-feedback` will check this and refuse to parse unknown versions.
+- `date` matches the title.
+- `ts` is an ISO-8601 UTC timestamp with second precision (no milliseconds).
+
+### Block 2 — intro paragraph
+
+Exactly one paragraph, with the commit count and short-SHA range substituted:
+
+```
+Reviewing {N} commits since {priorScanDate} ({firstShortSha}..{lastShortSha}). Findings here focus on SDLC and workflow changes — Dependabot, GitHub Code Scanning, and Renovate handle the rest. Tick the box once the underlying issue is resolved or recorded in `SECURITY.md`.
+```
+
+`{N}` is the size of `commitsReviewed`. `{priorScanDate}` is the YYYY-MM-DD of the prior scan (or the bootstrap floor). When `N == 1`, both short SHAs are the same — render as `({onlyShortSha})` rather than `({sha}..{sha})`.
+
+### Block 3 — how-to-respond section
+
+Verbatim, including the heading:
+
+```
+## How to respond
+
+**Preferred flow** — tick the boxes on the findings you want broken out, then comment:
+
+- `@last-light create issues` — files one issue per **ticked** finding (default)
+
+**Other shortcuts:**
+
+- `@last-light create issues for the criticals` — every Critical finding (ticked or not)
+- `@last-light create issues for the highs` — same, for High
+- `@last-light create issues for items 1, 3, 5` — specific items by number (1-based, top to bottom)
+- `@last-light create issues for all` — every finding in this scan
+- `@last-light accept-risk for item N: <reason>` — suppress this finding in future scans
+- `@last-light false-positive for item N: <reason>` — suppress this finding in future scans
+- Comment freely to ask questions or discuss
+```
+
+(Item positions in commands map to the `item:N` HTML-comment markers defined below. Ticking a box in GitHub's UI rewrites the row from `[ ]` to `[x]` — the feedback skill treats that as your selection.)
+
+### Block 4 — summary table
+
+Verbatim header, with numbers substituted. Always include all four severity rows, even when the count is 0.
+
+`{nC}`, `{nH}`, `{nM}`, `{nL}` and `{nTotal}` are **TRUE counts** (post-filtering, pre-cap) — i.e. how many findings of each severity actually survived the SECURITY.md filtering, regardless of whether each individual row is listed below the cap. The same numbers appear in the `### 🔴 Critical ({nC})` etc. section headers. The overflow note (Block 6) communicates how many of those counts were truncated from the listed rows.
+
+```
+## Summary
+
+| Severity | Count |
+|----------|------:|
+| Critical | {nC} |
+| High     | {nH} |
+| Medium   | {nM} |
+| Low      | {nL} |
+| **Total**| **{nTotal}** |
+```
+
+### Block 5 — suppression note
+
+A single line:
+
+```
+Suppressed by `SECURITY.md`: {nSuppressed} (accepted: {nA}, false-positives: {nFP}). Below severity floor: {nFloor}.
+```
+
+Set each count to 0 when N/A. Emit the line unconditionally so the structure is stable.
+
+### Block 6 — scope note
+
+Always emit, immediately after the suppression note. Lists the human (non-bot) commits actually reviewed, so a maintainer can see what diff produced these findings:
+
+```
+> Commits reviewed: {short-sha-1} {subject-1}, {short-sha-2} {subject-2}, …
+```
+
+Cap at 10 entries; if there are more, append ` +{N} more` after the last item. Subjects are truncated to 60 chars with `…` if longer. Renovate/Dependabot commits are filtered out at § 1.5 and never appear here.
+
+### Block 7 — overflow note
+
+Emit **only** when `overflow > 0`:
+
+```
+> **Note** — {overflow} lower-severity findings are not listed here. The cap is: ALL critical and high, plus the first 10 medium/low (after sort). Tighten `SECURITY.md` severity floors or break out items from this scan, then re-run to surface the rest.
+```
+
+### Block 8 — findings sections
+
+Four sections, in this **exact order** (Critical → High → Medium → Low). Always emit all four headers, even when a section has zero findings — the feedback skill relies on stable anchors.
+
+The header counts (`{nC}` etc.) are the **true** post-filter counts, identical to those in Block 4's summary table. The rows listed under each header are subject to the § 7 cap: critical and high are always complete; medium + low are truncated to the first 10 combined. When a section is partially listed, append `(showing first N of {nM})` after the marker — see the per-section header rule below.
+
+```
+## Findings
+
+### 🔴 Critical ({nC})
+
+{rows or "_No findings._"}
+
+### 🟠 High ({nH})
+
+{rows or "_No findings._"}
+
+### 🟡 Medium ({nM}){if truncated: " (showing first {kM} of {nM})"}
+
+{rows or "_No findings._"}
+
+### 🟢 Low ({nL}){if truncated: " (showing first {kL} of {nL})"}
+
+{rows or "_No findings._"}
+```
+
+Where `kM` and `kL` are the actual rows listed in this issue (sum of the two ≤ 10). When `kM == nM` or `kL == nL` (no truncation in that section), omit the parenthetical.
+
+### Finding-row grammar
+
+Every finding is exactly two lines: a task-list row, then a `<details>` block (one blank line between rows within a section).
+
+The task-list row is **one physical line** with this exact shape:
+
+```
+- [ ] <!-- item:N fp:FINGERPRINT --> **TITLE** — `FILE:LINE` (TOOL · `RULE`)
+```
+
+Matched by this canonical regex (multiline, case-sensitive) — covers all three row states:
+
+```
+/^- \[([ x])\] <!-- item:(\d+) fp:([0-9a-f]{8,}) --> (?:~~)?\*\*(.+?)\*\* — `([^`]+):(\d+)` \(([a-z][a-z0-9-]*) · `([^`]+)`\)(?:~~ → #(\d+))?$/m
+```
+
+Capture groups, in order:
+
+1. `checkbox` — `" "` (unticked) or `"x"` (ticked or broken-out)
+2. `itemNumber` (1-based across all severities)
+3. `fingerprint` (lowercase hex, ≥ 8 chars)
+4. `title` (plain text; no backticks, no asterisks)
+5. `file` (no backticks, forward-slash path)
+6. `line` (integer; use `0` when not line-scoped)
+7. `tool` (lowercase, hyphenated — e.g. `npm-audit`, `semgrep`, `gitleaks`, `claude`)
+8. `rule` (the tool's native rule id; may contain dots, hyphens)
+9. `subIssueNumber` — present **only** when the row has been broken out to a sub-issue; `undefined` otherwise
+
+Derived state (the feedback skill computes these from the captures):
+
+| State | Written as | `checkbox` | `subIssueNumber` |
+|-------|------------|------------|------------------|
+| **pending** | `- [ ] <!-- item:N fp:FP --> **TITLE** — …` | `" "` | `undefined` |
+| **user-ticked** (maintainer clicked the box in GitHub's UI) | `- [x] <!-- item:N fp:FP --> **TITLE** — …` | `"x"` | `undefined` |
+| **broken-out** (feedback skill created a sub-issue) | `- [x] <!-- item:N fp:FP --> ~~**TITLE** — …~~ → #SUBISSUE` | `"x"` | the sub-issue number |
+
+Rules:
+
+- `alreadyBrokenOut` ≡ `subIssueNumber != null`. Broken-out rows are immutable — the feedback skill never re-opens them, never touches their checkbox, never re-creates sub-issues from them.
+- `userTicked` ≡ `checkbox === "x" && subIssueNumber == null`. These are the candidates the default `@last-light create issues` command selects.
+- When creating sub-issues from ticked rows, the feedback skill transitions each row from **user-ticked** → **broken-out** by wrapping the visible text in `~~…~~` and appending ` → #{subIssueNumber}`. The checkbox stays `[x]`; the strikethrough + link is the canonical broken-out marker.
+- Un-ticking (moving a user-ticked row back to `[ ]`) is fine — the row just becomes pending again. The scanner doesn't police this.
+
+The per-finding detail block follows immediately on the next line:
+
+````
+<details><summary>Details</summary>
+
+```{LANGUAGE}
+{SNIPPET}
+```
+
+{EXPLANATION}
+
+**Suggested fix:** {SUGGESTED_FIX}
+
+</details>
+````
+
+Rules:
+- `LANGUAGE` is the fenced-code language tag; empty string when unknown.
+- `SNIPPET` is the code excerpt; no surrounding fences, no trailing blank line inside the fence.
+- `EXPLANATION` and `SUGGESTED_FIX` are markdown strings; they may contain their own fenced code blocks and line breaks.
+- The `<details>` block ends with `</details>` on its own line.
+
+## Worked example
+
+A scan with 1 critical + 1 high finding renders like:
+
+````markdown
+<!-- lastlight-security-scan-version: 1 -->
+<!-- lastlight-security-scan-date: 2026-04-21 -->
+<!-- lastlight-security-scan-ts: 2026-04-21T10:00:00Z -->
+
+Reviewing 3 commits since 2026-04-14 (a1b2c3d..f9e8d7c). Findings here focus on SDLC and workflow changes — Dependabot, GitHub Code Scanning, and Renovate handle the rest. Tick the box once the underlying issue is resolved or recorded in `SECURITY.md`.
+
+## How to respond
+
+- `@last-light create issues for the criticals` — file individual issues for every Critical finding
+- `@last-light create issues for the highs` — same, for High
+- `@last-light create issues for items 1, 3, 5` — file issues for specific items by number (1-based, top to bottom)
+- `@last-light create issues for all` — every finding in this scan
+- `@last-light accept-risk for item N: <reason>` — suppress this finding in future scans
+- `@last-light false-positive for item N: <reason>` — suppress this finding in future scans
+- Comment freely to ask questions or discuss
+
+## Summary
+
+| Severity | Count |
+|----------|------:|
+| Critical | 1 |
+| High     | 1 |
+| Medium   | 0 |
+| Low      | 0 |
+| **Total**| **2** |
+
+Suppressed by `SECURITY.md`: 0 (accepted: 0, false-positives: 0). Below severity floor: 0.
+
+> Commits reviewed: a1b2c3d wire user-supplied repo into git clone, e5f6a7b add admin shell endpoint, f9e8d7c rotate webhook secret
+
+## Findings
+
+### 🔴 Critical (1)
+
+- [ ] <!-- item:1 fp:abc123def4567890 --> **Command injection in git clone** — `mcp-github-app/src/index.js:42` (semgrep · `javascript.lang.security.exec-shell-command`)
+<details><summary>Details</summary>
+
+```javascript
+execSync(`git clone ${userInput}`)
+```
+
+`userInput` originates from an HTTP request and is concatenated directly into a shell command, allowing arbitrary command execution.
+
+**Suggested fix:** use `execFileSync('git', ['clone', userInput])` so arguments aren't re-parsed by a shell.
+
+</details>
+
+### 🟠 High (1)
+
+- [ ] <!-- item:2 fp:def456abc7890123 --> **Hardcoded API key in config** — `src/config.ts:18` (gitleaks · `generic-api-key`)
+<details><summary>Details</summary>
+
+```typescript
+const API_KEY = "sk_live_abc123..."
+```
+
+A live API key is committed to the repo. Anyone with read access to the repo (or the git history of a former branch) can use it.
+
+**Suggested fix:** move the key to an environment variable (`process.env.API_KEY`) and rotate the exposed one immediately.
+
+</details>
+
+### 🟡 Medium (0)
+
+_No findings._
+
+### 🟢 Low (0)
+
+_No findings._
+````

package/skills/verify/SKILL.md

@@ -0,0 +1,118 @@
+---
+name: verify
+description: Test a behaviour claim as an investigator and report whether the evidence confirms or refutes it — CONFIRMED / REFUTED / INCONCLUSIVE with bash-captured evidence. Use when asked to verify a claim, prove a fix works, or check that a PR does what it says.
+version: 1.0.0
+tags: [github, verify, testing, evidence]
+---
+
+# Verify
+
+Test a specific claim about behaviour and report whether the evidence supports
+or refutes it. The deliverable is a **verdict with evidence**, not an opinion.
+
+This skill uses the **building** skill for installing dependencies and running
+the repo's build/test commands in the sandbox.
+
+> **Scope (text-evidence path).** In the default text phase the sandbox agent
+> has bash, file read, and the github tools — no browser, no screenshots, no
+> video. Prove claims with what bash can capture: test output, command
+> stdout/stderr, `curl` against a dev-server you start, log excerpts, file
+> contents, exit codes. Driving a real browser and attaching screenshots is the
+> separate `browser-qa` skill, which is staged only in the gated browser phase
+> running on the docker QA image (Playwright + Chromium). If that skill isn't in
+> your catalogue and a claim genuinely can only be shown in a rendered UI, say
+> so under **INCONCLUSIVE** rather than guessing — the browser phase will pick
+> it up when available.
+
+## Ground rule — investigator, not advocate
+
+Your job is to find out whether the claim is **true**, not to make it look true.
+A conclusive "this is broken" with clear evidence is exactly as valuable as a
+"this works". Specifically:
+
+- **Never fabricate, hardcode, mock, or stage evidence** to match an expected
+  outcome. Do not edit the code under test to make a result appear.
+- If the behaviour you observe **contradicts** the claim, that is the result —
+  report it as REFUTED with the evidence inline. Don't bury it.
+- **Don't retry a failing test hoping for a different answer.** Retry only after
+  *changing* the environment or procedure (wrong branch, missing build step),
+  and only when you have reason to believe the test setup — not the code — was
+  wrong. One unexpected result that reproduces is a finding.
+
+## Parse the target
+
+The Context block gives you the claim and (usually) a PR or issue. The claim may
+arrive as:
+
+- A **direct claim** — "ESC cancels streaming in bash mode", "the `--fork` flag
+  creates a new session".
+- A **PR reference + claim** — verify the claim against that PR's head.
+- A **PR reference only** — read the PR description + diff and identify the most
+  important, most testable claim it makes; state which claim you chose.
+
+When a PR/issue is given, read its description and diff for context (from the
+local checkout — see the **building** skill's workspace notes; don't pull the
+diff through the API).
+
+## Decide what would convince a skeptic
+
+Name the single behaviour to observe and the evidence that settles it:
+
+- **Functional claim** (a feature works, a flow completes) → run it and capture
+  the output / exit code / resulting state.
+- **Regression / fix claim** ("no longer clears the screen", "stops erroring") →
+  show the **before and after**: reproduce on the base ref, then show it gone on
+  the head ref. Both states must appear in the evidence — an off-camera "it's
+  fixed" doesn't count.
+- **Output/encoding claim** → capture the exact bytes/text (`xxd`, `od -c`, raw
+  stdout) rather than describing them.
+
+## Run the test
+
+Follow the **building** skill to install and build. Then exercise the claim with
+the minimal sequence that demonstrates it, capturing evidence at the decisive
+step. Start a dev-server in the background and `curl` it when the claim is about
+a running service; run the CLI directly when it's a CLI claim; run the targeted
+test when the repo already encodes the behaviour as a test.
+
+If the environment blocks a clean test (missing dependency the sandbox can't
+provide, build failure you can't resolve, infra that won't run), capture what
+blocked it — that drives an **INCONCLUSIVE** result, not a guess.
+
+## Report
+
+Produce your report as your **final message** in this shape — the workflow posts
+it for you (don't `github_add_issue_comment` yourself; that would double-post).
+**Keep it concise**: a one-line environment summary, the decisive evidence
+(commands + key output, not every line), and a one-paragraph conclusion — not an
+exhaustive walkthrough.
+
+```
+## Verify: <claim>
+
+**Environment:** <branch / commit, package manager, how the app was run>
+
+### Evidence
+<commands run + their captured output — fenced. Include both states for a
+before/after claim.>
+
+### Conclusion
+**CONFIRMED** | **REFUTED** | **INCONCLUSIVE**
+
+<one paragraph: what the evidence shows and why it settles the claim.>
+```
+
+- **REFUTED** — state the expected behaviour (from the claim), the observed
+  behaviour (from the evidence), include the evidence inline, and note any
+  environmental factors (branch, commit, OS). This is a valuable finding, not a
+  failed run.
+- **INCONCLUSIVE** — report exactly what blocked the test and what would be
+  needed to resolve it. Do not guess at the outcome.
+
+## Do NOT
+
+- Retry a failing test more than once without changing the environment/procedure.
+- Hardcode expected output, mock responses, or edit the code under test to force
+  a result.
+- Assume unexpected behaviour means *you* made a mistake — it may be a real bug.
+- Omit evidence that contradicts the claim.

package/workflows/answer.yaml

@@ -0,0 +1,37 @@
+kind: answer
+name: answer
+description: |
+  Answer a question issue directly. Used when a newly-opened issue asks for
+  information, an explanation, or a comparison rather than a code change — the
+  bot researches (repo docs + web search) and posts a single sourced answer
+  comment, labels the issue `question`, and leaves it open. No agent brief, no
+  ready-for-agent, no PR.
+
+  Delivery is uniform: the agent's final message (the answer) is captured as
+  `answerResult` and posted by the harness via `postComment` — as a comment on
+  the originating issue for GitHub-initiated runs, or into the Slack thread for
+  Slack-initiated runs. The skill therefore does NOT post the answer itself; it
+  only applies the `question` label when answering a GitHub issue.
+
+  Triggered by:
+    - issue.opened webhook when the router classifies the issue as a question
+    - a Slack message classified as a question against a managed repo
+    - manual override via routes.github.issue_answer / routes.slack.answer
+
+phases:
+  - name: phase_0
+    label: Context
+    type: context
+
+  - name: answer
+    label: Answer
+    prompt: prompts/answer.md
+    skill: issue-answer
+    model: "{{models.answer}}"
+    web_search: true
+    unrestricted_egress: true
+    output_var: answerResult
+    messages:
+      on_start: "Researching and drafting an answer..."
+      on_success: "{{answerResult}}"
+      on_failure: "I couldn't put together an answer — see the logs for details."