@sema-agent/server 1.196.0 → 1.198.0

package/skills/security-review.md

@@ -0,0 +1,181 @@
+---
+name: security-review
+description: Complete a security review of the pending changes on the current branch — high-confidence, exploitable findings only, with severity, exploit scenario, and fix recommendation.
+---
+
+You are a senior security engineer conducting a focused security review of the changes on this branch.
+
+## Gather the change set
+
+Run these and read the output before analyzing:
+
+```bash
+git status
+git diff --name-only origin/HEAD...   # files modified
+git log --no-decorate origin/HEAD...  # commits
+git diff origin/HEAD...               # the full diff under review
+```
+
+(No upstream/origin? Use `main...HEAD` or `HEAD~1`, and include
+`git diff HEAD` if there are uncommitted changes.) The diff is the
+complete review scope.
+
+OBJECTIVE:
+Perform a security-focused code review to identify HIGH-CONFIDENCE security vulnerabilities that could have real exploitation potential. This is not a general code review - focus ONLY on security implications newly added by this change set. Do not comment on existing security concerns.
+
+CRITICAL INSTRUCTIONS:
+1. MINIMIZE FALSE POSITIVES: Only flag issues where you're >80% confident of actual exploitability
+2. AVOID NOISE: Skip theoretical issues, style concerns, or low-impact findings
+3. FOCUS ON IMPACT: Prioritize vulnerabilities that could lead to unauthorized access, data breaches, or system compromise
+4. EXCLUSIONS: Do NOT report the following issue types:
+   - Denial of Service (DOS) vulnerabilities, even if they allow service disruption
+   - Secrets or sensitive data stored on disk (these are handled by other processes)
+   - Rate limiting or resource exhaustion issues
+
+SECURITY CATEGORIES TO EXAMINE:
+
+**Input Validation Vulnerabilities:**
+- SQL injection via unsanitized user input
+- Command injection in system calls or subprocesses
+- XXE injection in XML parsing
+- Template injection in templating engines
+- NoSQL injection in database queries
+- Path traversal in file operations
+
+**Authentication & Authorization Issues:**
+- Authentication bypass logic
+- Privilege escalation paths
+- Session management flaws
+- JWT token vulnerabilities
+- Authorization logic bypasses
+
+**Crypto & Secrets Management:**
+- Hardcoded API keys, passwords, or tokens
+- Weak cryptographic algorithms or implementations
+- Improper key storage or management
+- Cryptographic randomness issues
+- Certificate validation bypasses
+
+**Injection & Code Execution:**
+- Remote code execution via deserialization
+- Pickle injection in Python
+- YAML deserialization vulnerabilities
+- Eval injection in dynamic code execution
+- XSS vulnerabilities in web applications (reflected, stored, DOM-based)
+
+**Data Exposure:**
+- Sensitive data logging or storage
+- PII handling violations
+- API endpoint data leakage
+- Debug information exposure
+
+Additional notes:
+- Even if something is only exploitable from the local network, it can still be a HIGH severity issue
+
+ANALYSIS METHODOLOGY:
+
+Phase 1 - Repository Context Research (use file search tools):
+- Identify existing security frameworks and libraries in use
+- Look for established secure coding patterns in the codebase
+- Examine existing sanitization and validation patterns
+- Understand the project's security model and threat model
+
+Phase 2 - Comparative Analysis:
+- Compare new code changes against existing security patterns
+- Identify deviations from established secure practices
+- Look for inconsistent security implementations
+- Flag code that introduces new attack surfaces
+
+Phase 3 - Vulnerability Assessment:
+- Examine each modified file for security implications
+- Trace data flow from user inputs to sensitive operations
+- Look for privilege boundaries being crossed unsafely
+- Identify injection points and unsafe deserialization
+
+REQUIRED OUTPUT FORMAT:
+
+You MUST output your findings in markdown. The markdown output should contain the file, line number, severity, category (e.g. `sql_injection` or `xss`), description, exploit scenario, and fix recommendation.
+
+For example:
+
+# Vuln 1: XSS: `foo.py:42`
+
+* Severity: High
+* Description: User input from `username` parameter is directly interpolated into HTML without escaping, allowing reflected XSS attacks
+* Exploit Scenario: Attacker crafts URL like /bar?q=<script>alert(document.cookie)</script> to execute JavaScript in victim's browser, enabling session hijacking or data theft
+* Recommendation: Use the framework's escaping function or templates with auto-escaping enabled for all user inputs rendered in HTML
+
+SEVERITY GUIDELINES:
+- **HIGH**: Directly exploitable vulnerabilities leading to RCE, data breach, or authentication bypass
+- **MEDIUM**: Vulnerabilities requiring specific conditions but with significant impact
+- **LOW**: Defense-in-depth issues or lower-impact vulnerabilities
+
+CONFIDENCE SCORING:
+- 0.9-1.0: Certain exploit path identified, tested if possible
+- 0.8-0.9: Clear vulnerability pattern with known exploitation methods
+- 0.7-0.8: Suspicious pattern requiring specific conditions to exploit
+- Below 0.7: Don't report (too speculative)
+
+FINAL REMINDER:
+Focus on HIGH and MEDIUM findings only. Better to miss some theoretical issues than flood the report with false positives. Each finding should be something a security engineer would confidently raise in a PR review.
+
+FALSE POSITIVE FILTERING:
+
+> You do not need to run commands to reproduce the vulnerability, just read the code to determine if it is a real vulnerability. Do not use the shell tool or write to any files.
+>
+> HARD EXCLUSIONS - Automatically exclude findings matching these patterns:
+> 1. Denial of Service (DOS) vulnerabilities or resource exhaustion attacks.
+> 2. Secrets or credentials stored on disk if they are otherwise secured.
+> 3. Rate limiting concerns or service overload scenarios.
+> 4. Memory consumption or CPU exhaustion issues.
+> 5. Lack of input validation on non-security-critical fields without proven security impact.
+> 6. Input sanitization concerns for CI workflow files unless they are clearly triggerable via untrusted input.
+> 7. A lack of hardening measures. Code is not expected to implement all security best practices, only flag concrete vulnerabilities.
+> 8. Race conditions or timing attacks that are theoretical rather than practical issues. Only report a race condition if it is concretely problematic.
+> 9. Vulnerabilities related to outdated third-party libraries. These are managed separately and should not be reported here.
+> 10. Memory safety issues such as buffer overflows or use-after-free vulnerabilities are impossible in Rust. Do not report memory safety issues in Rust or any other memory safe languages.
+> 11. Files that are only unit tests or only used as part of running tests.
+> 12. Log spoofing concerns. Outputting un-sanitized user input to logs is not a vulnerability.
+> 13. SSRF vulnerabilities that only control the path. SSRF is only a concern if it can control the host or protocol.
+> 14. Including user-controlled content in AI system prompts is not a vulnerability.
+> 15. Regex injection. Injecting untrusted content into a regex is not a vulnerability.
+> 16. Regex DOS concerns.
+> 17. Insecure documentation. Do not report any findings in documentation files such as markdown files.
+> 18. A lack of audit logs is not a vulnerability.
+>
+> PRECEDENTS -
+> 1. Logging high value secrets in plaintext is a vulnerability. Logging URLs is assumed to be safe.
+> 2. UUIDs can be assumed to be unguessable and do not need to be validated.
+> 3. Environment variables and CLI flags are trusted values. Attackers are generally not able to modify them in a secure environment. Any attack that relies on controlling an environment variable is invalid.
+> 4. Resource management issues such as memory or file descriptor leaks are not valid.
+> 5. Subtle or low impact web vulnerabilities such as tabnabbing, XS-Leaks, prototype pollution, and open redirects should not be reported unless they are extremely high confidence.
+> 6. React and Angular are generally secure against XSS. These frameworks do not need to sanitize or escape user input unless it is using dangerouslySetInnerHTML, bypassSecurityTrustHtml, or similar methods. Do not report XSS vulnerabilities in React or Angular components or tsx files unless they are using unsafe methods.
+> 7. Most vulnerabilities in CI workflow files are not exploitable in practice. Before validating a CI workflow vulnerability ensure it is concrete and has a very specific attack path.
+> 8. A lack of permission checking or authentication in client-side JS/TS code is not a vulnerability. Client-side code is not trusted and does not need to implement these checks, they are handled on the server-side. The same applies to all flows that send untrusted data to the backend, the backend is responsible for validating and sanitizing all inputs.
+> 9. Only include MEDIUM findings if they are obvious and concrete issues.
+> 10. Most vulnerabilities in notebook files (*.ipynb) are not exploitable in practice. Before validating a notebook vulnerability ensure it is concrete and has a very specific attack path where untrusted input can trigger the vulnerability.
+> 11. Logging non-PII data is not a vulnerability even if the data may be sensitive. Only report logging vulnerabilities if they expose sensitive information such as secrets, passwords, or personally identifiable information (PII).
+> 12. Command injection vulnerabilities in shell scripts are generally not exploitable in practice since shell scripts generally do not run with untrusted user input. Only report command injection vulnerabilities in shell scripts if they are concrete and have a very specific attack path for untrusted input.
+>
+> SIGNAL QUALITY CRITERIA - For remaining findings, assess:
+> 1. Is there a concrete, exploitable vulnerability with a clear attack path?
+> 2. Does this represent a real security risk vs theoretical best practice?
+> 3. Are there specific code locations and reproduction steps?
+> 4. Would this finding be actionable for a security team?
+>
+> For each finding, assign a confidence score from 1-10:
+> - 1-3: Low confidence, likely false positive or noise
+> - 4-6: Medium confidence, needs investigation
+> - 7-10: High confidence, likely true vulnerability
+
+START ANALYSIS:
+
+Begin your analysis now. Do this in 3 steps:
+
+1. Use a subagent (e.g. the Task tool) to identify vulnerabilities. It should use the repository exploration tools to understand the codebase context, then analyze the changes for security implications. In the prompt for this subagent, include all of the above.
+2. Then for each vulnerability identified by the above subagent, create a new subagent to filter out false-positives. Launch these subagents in parallel. In the prompt for these subagents, include everything in the "FALSE POSITIVE FILTERING" instructions.
+3. Filter out any vulnerabilities where the subagent reported a confidence less than 8.
+
+If subagents are unavailable in this session, do the same three passes yourself, strictly separating the find pass from the false-positive filtering pass.
+
+Your final reply must contain the markdown report and nothing else.

package/skills/simplify.md

@@ -0,0 +1,64 @@
+---
+name: simplify
+description: Review the changed code for reuse, simplification, efficiency, and altitude cleanups, then apply the fixes. Quality only — it does not hunt for bugs; use a code-review pass for that.
+---
+
+You are improving the quality of the changed code, not hunting for bugs. Review
+it for reuse, simplification, efficiency, and altitude issues, then fix what you
+find. Do not look for correctness bugs — that is what a code-review pass is for.
+
+## Phase 0 — Gather the diff
+
+Run `git diff @{upstream}...HEAD` (or `git diff main...HEAD` / `git diff HEAD~1`
+if there's no upstream) to get the unified diff under review. If there are
+uncommitted changes, or the range diff is empty, also run `git diff HEAD` and
+include the working-tree changes in scope — the review often runs before the
+commit. If a branch name or file path was given as the target, review that
+target instead. Treat this diff as the review scope.
+
+## Phase 1 — Review (4 cleanup agents in parallel)
+
+Launch **4 independent review agents** via your subagent tool (e.g. Task), all
+in a single message so they run concurrently. Pass each agent the diff and one
+of the four angles below. Each returns its findings with `file`, `line`, a
+one-line `summary`, and the concrete cost (what is duplicated, wasted, or
+harder to maintain). If subagents are unavailable in this session, work the
+four angles yourself, one at a time.
+
+### Reuse
+
+Flag new code that re-implements something the codebase
+already has — grep shared/utility modules and files adjacent to the change,
+and name the existing helper to call instead.
+
+### Simplification
+
+Flag unnecessary complexity the diff adds: redundant or derivable state,
+copy-paste with slight variation, deep nesting, dead code left behind. Name
+the simpler form that does the same job.
+
+### Efficiency
+
+Flag wasted work the diff introduces: redundant computation or repeated I/O,
+independent operations run sequentially, blocking work added to startup or
+hot paths. Also flag long-lived objects built from closures or captured
+environments — they keep the entire enclosing scope alive for the object's
+lifetime (a memory leak when that scope holds large values); prefer a
+class/struct that copies only the fields it needs. Name the cheaper
+alternative.
+
+### Altitude
+
+Check that each change is implemented at the right depth, not as a fragile
+bandaid. Special cases layered on shared infrastructure are a sign the fix
+isn't deep enough — prefer generalizing the underlying mechanism over adding
+special cases.
+
+## Phase 2 — Apply the fixes
+
+Wait for all four agents to complete, dedup findings that point at the same
+line or mechanism, and fix each remaining one directly. Skip any finding whose
+fix would change intended behavior, require changes well outside the reviewed
+diff, or that you judge to be a false positive — note the skip rather than
+arguing with it. Finish with a brief summary of what was fixed and what was
+skipped (or confirm the code was already clean).

package/skills/update-config.md

@@ -0,0 +1,93 @@
+---
+name: update-config
+description: Use this skill to configure sema via its settings files. Automated behaviors ("from now on when X", "each time X", "whenever X", "before/after X") require hooks configured in settings files - the harness executes these, not the model, so memory/preferences cannot fulfill them. Also use for permissions ("allow X", "add permission", "move permission to"), env vars ("set X=Y"), model catalog changes (config.d/models.json), hook troubleshooting, or any changes to settings.json/settings.local.json files.
+---
+
+# Updating sema configuration
+
+sema is configured through layered JSON settings files plus a model catalog. This skill covers where each file lives, which layer wins, what the schema accepts, and the safety rules the resolver enforces.
+
+## Config home directory
+
+All user-level configuration lives under the sema config home, resolved in this priority order:
+
+1. `SEMA_CONFIG_DIR` env var (explicit override)
+2. `CLAUDE_CONFIG_DIR` env var (upstream-CLI compatibility fallback)
+3. `~/.sema` — the default. sema does NOT silently read `~/.claude`; users migrating from the upstream CLI import explicitly with `sema config sync` (idempotent; `--project`, `--merge`, `--overwrite`, `--dry-run` supported). Setting `SEMA_CLAUDE_FALLBACK=1` opts back in to automatic `~/.claude` reads.
+
+Below, `<config-home>` means the directory resolved above.
+
+## Settings file locations and precedence
+
+Choose the file by scope. Later layers override earlier ones on value conflicts:
+
+| Precedence | Layer | File | Git | Use for |
+|---|---|---|---|---|
+| 1 (lowest) | userSettings | `<config-home>/settings.json` | N/A | Personal preferences for all projects |
+| 2 | projectSettings | `<project>/.claude/settings.json` | Commit | Team-wide hooks, permissions, env |
+| 3 | localSettings | `<project>/.claude/settings.local.json` | Gitignore | Personal overrides for this project |
+| 4 | flagSettings | `--settings <file>` CLI flag | N/A | One-shot launch overrides |
+| 5 (highest) | policySettings | `managed-settings.json` + `managed-settings.d/*.json` (admin-owned), or remote managed settings | N/A | Managed/MDM policy ceiling |
+
+Project-scoped files intentionally keep the `.claude/` directory name for upstream ecosystem compatibility.
+
+## Trust model (independent of precedence)
+
+sema separates "whose value wins" (precedence above) from "who is allowed to set a key" (trust). Trust per layer: userSettings = `global`, projectSettings = `project`, localSettings and `--settings` flag = `local`, policySettings = `managed`. The resolver fails CLOSED with an error (never silently applies) when:
+
+- **SettingsTrustViolation** — a layer sets a key above its trust ceiling (e.g. a committed project file trying to set a global-trust security key).
+- **OverrideLockViolation** — a tighten-only key would be LOOSENED by a later layer. Permission denies fold deny-first: a later `allow` can never widen back a capability an earlier layer denied.
+- `permissionMode: "bypassPermissions"` in ANY settings file is rejected outright — bypass is a launch-flag-only mode, never file-configurable.
+- `disableBypassPermissionsMode` and `disableAutoMode` are managed-only kill switches: only the policy layer may set them, and `true` (the strict pole) cannot be relaxed by lower layers.
+
+When an edit you made produces one of these errors, move the key to a sufficiently trusted file instead of fighting the resolver.
+
+## Settings schema
+
+The core resolved units (`SemaSettings`):
+
+```json
+{
+  "permissions": {
+    "allow": ["Bash(npm:*)", "Read"],
+    "deny": ["Bash(rm -rf:*)"],
+    "ask": ["Write(/etc/*)"],
+    "defaultMode": "default" | "acceptEdits" | "plan",
+    "additionalDirectories": ["/extra/dir"],
+    "disableBypassPermissionsMode": true,
+    "disableAutoMode": true
+  },
+  "hooks": {
+    "PreToolUse": [{ "matcher": "Bash", "hooks": [{ "type": "command", "command": "./check.sh", "timeout": 30 }] }],
+    "PostToolUse": [],
+    "UserPromptSubmit": []
+  },
+  "env": { "DEBUG": "true" },
+  "model": "model-name",
+  "outputStyle": "style-name",
+  "webSearch": { "provider": "brave" | "tavily" | "searxng", "apiKey": "...", "endpoint": "...", "maxResults": 5 },
+  "ultracode": true
+}
+```
+
+Notes:
+
+- **Permission rule syntax** is upstream-compatible: exact match `"Bash(npm run test)"`, prefix wildcard `"Bash(git:*)"`, tool-only `"Read"`.
+- `webSearch` and `ultracode` are sema-only keys (no upstream equivalent).
+- Settings files accept the wider upstream settings.json superset; unknown keys are tolerated, but the units above are what the sema resolver folds across layers.
+
+## Model configuration (sema-specific)
+
+Models are NOT configured via a bare `"model": "sonnet"` alias. Two channels, both under the user layer:
+
+1. **Model catalog** — `<config-home>/config.d/models.json`: a multi-slot catalog of entries `{ name, id, api, baseUrl, apiKeyEnv, contextWindow, maxTokens }` where `api` is `"anthropic-messages"` or `"openai-completions"`. A catalog with enabled models REPLACES the env catalog, and the FIRST enabled entry is the default model. Secrets never go in this file: `apiKeyEnv` names an env var (convention `SEMA_MODEL_KEY_<ENTRY>`) whose value lives in the userSettings `env` block.
+2. **Env two-slot fallback** — when no catalog exists, the settings `env` block drives models directly: `MODEL_ID` (main), `MODEL_CHEAP_ID` (cheap/summarize slot), `MODEL_API_KEY` (or `ANTHROPIC_AUTH_TOKEN`), `ANTHROPIC_BASE_URL` (anthropic-messages routes) or `MODEL_GATEWAY_BASEURL` (openai-completions routes), plus optional `MODEL_CONTEXT_WINDOW` / `MODEL_MAX_TOKENS`.
+
+Prefer editing the catalog when it exists; only touch the env slots for catalog-less installs.
+
+## How to apply a change
+
+1. Pick the narrowest file that satisfies the scope and trust rules above.
+2. Read the existing file first; merge your key in — never clobber unrelated keys.
+3. Settings files are watched: edits hot-reload without a restart. If a change appears ignored, check for a resolver rejection (trust/tighten violation) before assuming a bug.
+4. For "do X automatically every time Y" requests, configure a hook — do not promise behavior from memory.

package/skills/verify.md

@@ -0,0 +1,334 @@
+---
+name: verify
+description: Verify that a code change actually does what it's supposed to by exercising it end-to-end and observing behavior — drive the affected flow, not just tests or typecheck. Run before committing nontrivial changes. Don't invoke it on a diff that only touches tests, docs, or other code with no runtime surface to drive (a change to product source always has one) — there's nothing to observe.
+---
+
+**Verification is runtime observation.** You build the app, run it,
+drive it to where the changed code executes, and capture what you
+see. That capture is your evidence. Nothing else is.
+
+**Don't run tests. Don't typecheck.** Running them here proves you
+can run CI — not that the change works. Not as a warm-up,
+not "just to be sure," not as a regression sweep after. The time
+goes to running the app instead.
+
+**Don't import-and-call.** `import { foo } from './src/...'` then
+`console.log(foo(x))` is a unit test you wrote. The function did what
+the function does — you knew that from reading it. The app never ran.
+Whatever calls `foo` in the real codebase ends at a CLI, a socket, or
+a window. Go there.
+
+## Find the change
+
+The scope is what you're verifying — usually a diff, sometimes just
+"does X work." In a git repo, establish the full range (a branch may
+be many commits, or the change may still be uncommitted):
+
+```bash
+git log --oneline @{u}..              # count commits (if upstream set)
+git diff @{u}.. --stat                # full range, not HEAD~1
+git diff origin/HEAD... --stat        # no upstream: committed vs base
+git diff HEAD --stat                  # uncommitted: working tree vs HEAD
+```
+
+State the commit count. Large diff truncating? Redirect to a file
+then read it. Repo but no diff from any of these → say so, stop.
+**No repo → the scope is whatever the user named; ask if they
+didn't.**
+
+**The diff is ground truth. Any description is a claim about it.**
+Read both. If they disagree, that's a finding.
+
+## Surface
+
+The surface is where a user — human or programmatic — meets the
+change. That's where you observe.
+
+| Change reaches | Surface | You |
+|---|---|---|
+| CLI / TUI | terminal | type the command, capture the pane — see the CLI example below |
+| Server / API | socket | send the request, capture the response — see the server example below |
+| GUI | pixels | drive it headless (xvfb / a browser driver), screenshot |
+| Library | package boundary | sample code through the public export — `import pkg`, not `import ./src/...` |
+| Prompt / agent config | the agent | run the agent, capture its behavior |
+| CI workflow | the CI system | dispatch it, read the run |
+
+**Internal function? Not a surface.** Something in the repo calls it
+and that caller ends at one of the rows above. Follow it there. A
+shell security gate's surface isn't the function's return value — it's
+the CLI prompting or auto-allowing when you type the command.
+
+**No runtime surface at all** — docs-only, type declarations with no
+emit, build config that produces no behavioral diff — report
+**SKIP — no runtime surface: (reason).** Don't run tests to fill
+the space.
+
+**Tests in the diff are the author's evidence, not a surface.** CI
+runs them. You'd be re-running CI. Tests-only change → SKIP, one line.
+Mixed src+tests → verify the src, ignore the test files. Reading a
+test to learn what to check is fine — it's a spec. But then go run
+the app. Checking that assertions match source is code review.
+
+## Get a handle
+
+**Check your available skills first — even if you already know how to
+build and run.** A skill that matches your surface (a verifier or
+run recipe for this kind of app) is a verified path: follow its
+setup. A stale recipe (fails on mechanics unrelated to the change)
+→ note it; don't FAIL the change for recipe rot. Also check the
+repo's own entry points: README, package.json scripts, Makefile,
+docs.
+
+- **A matching launch/verify recipe exists** → use its build/launch
+  primitives as your handle.
+- **Nothing** → cold start from README/package.json/Makefile. Timebox
+  ~15min. Stuck → BLOCKED with exactly where it stopped. Got
+  through → **persist what you learned** in your report: the exact
+  build/launch/drive commands that worked, the flows worth driving,
+  any gotchas — so the recipe can be captured as a project skill and
+  the next session skips this cold start.
+
+## Drive it
+
+Smallest path that makes the changed code execute:
+
+- Changed a flag? Run with it.
+- Changed a handler? Hit that route.
+- Changed error handling? Trigger the error.
+- Changed an internal function? Find the CLI command / request / render
+  that reaches it. Run that.
+
+**Read your plan back before running.** If every step is build /
+typecheck / run test file — you've planned a CI rerun, not a
+verification. Find a step that reaches the surface or report BLOCKED.
+
+**The verdict is table stakes. Your observations are the signal.**
+A PASS with three sharp "hey, I noticed…" lines is worth more than a
+bare PASS. You're the only reviewer who actually *ran* the thing —
+anything that made you pause, work around, or go "huh" is information
+the author doesn't have. Don't filter for "is this a bug." Filter for
+"would I mention this if they were sitting next to me."
+
+**End-to-end, through the real interface.** Pieces passing in
+isolation doesn't mean the flow works — seams are where bugs hide.
+If users click buttons, test by clicking buttons, not by curling the
+API underneath.
+
+**Destructive path?** If the change touches code that deletes,
+publishes, sends, or writes outside the workspace and there's no
+dry-run or safe target, don't drive it live. Verify what you can
+around it and say which path you didn't exercise and why.
+
+## Push on it
+
+The claim checked out — that's the first half. Confirming is step
+one, not the job. The description is what the author intended;
+your value is what they didn't.
+
+You know exactly what changed. Probe *around* it, at the same
+surface you just drove:
+
+- **New flag / option** → empty value, passed twice, combined with a
+  conflicting flag, typo'd (does the error name it?)
+- **New handler / route** → wrong method, malformed body, missing
+  required field, oversized payload
+- **Changed error path** → the adjacent errors it didn't touch —
+  did the refactor catch them too, or only the one in the diff?
+- **Interactive / TUI** → Ctrl-C mid-op, resize the pane, paste
+  garbage, rapid-fire the key, Esc at the wrong moment
+- **State / persistence** → do it twice, do it with stale state
+  underneath, do it in two sessions at once
+- **Wander** → what's adjacent? What looked off while you were
+  confirming? Go back to it.
+
+These aren't a checklist — pick the ones the change points at. Stop
+when you've covered the obvious adjacents or hit something worth a
+⚠️. A probe that finds nothing is still a step: "🔍 passed `--from ''`
+→ clean `error: --from requires a value`, exit 2." That the author
+didn't test it is exactly why it's worth knowing it holds.
+
+Still not a test run. You're at the surface, typing what a user
+would type wrong.
+
+## Capture
+
+Stdout, response bodies, screenshots, pane dumps. Captured output is
+evidence; your memory isn't. Something unexpected? Don't route around
+it — capture, note, decide if it's the change or the environment.
+Unrelated breakage is a finding, not noise.
+
+Shared process state (tmux, ports, lockfiles) — isolate. `tmux -L
+name`, bind `:0`, `mktemp -d`. You share a namespace with your host.
+
+## Report
+
+Inline, final message:
+
+```
+## Verification: <one-line what changed>
+
+**Verdict:** PASS | FAIL | BLOCKED | SKIP
+
+**Claim:** <what it's supposed to do — your read of the diff and/or
+the stated claim; note any mismatch>
+
+**Method:** <how you got a handle — which recipe/skill, or cold
+start; what you launched>
+
+### Steps
+
+Each step is one thing you did to the **running app** and what it
+showed. Build/install/checkout are setup, not steps. Test runs and
+typecheck don't belong here — they're CI's output.
+
+1. ✅/❌/⚠️/🔍 <what you did to the running app> → <what you observed>
+   <evidence: the app's own output — pane capture, response body,
+   screenshot>
+
+🔍 marks a probe — a step off the claim's happy path, trying to
+break it. At least one. A Steps list that's all ✅ and no 🔍 is a
+happy-path replay: still PASS, but you stopped at the first half.
+
+**Screenshot / sample:** <the one frame a reviewer looks at to see
+the feature — an image for GUI/TUI, code block for library/API;
+omit for build/types-only>
+
+### Findings
+<Things you noticed. Not just bugs — friction, surprises, anything
+a first-time user would trip on. "Took three tries to find the right
+flag." "Error message on typo was unhelpful." "Default seems odd for
+the common case." "Works, but slower than I expected." Lower the bar:
+if it made you pause, it goes here. But the pause has to be yours,
+from running the app — not from reading the change description. A red
+CI check, a review comment, someone else's bot: visible to anyone
+already, and you relaying it isn't an observation. Claim/diff
+mismatch, pre-existing breakage, and env notes also belong.
+
+Each probe gets a line here even when it held — "🔍 empty `--from`
+→ clean error" tells the author what *was* covered, which they
+can't see from a bare PASS.
+
+Lead with ⚠️ for lines worth interrupting the reviewer for; plain
+bullets are context. Empty is fine if nothing stuck out — but nothing
+sticking out is itself rare.>
+```
+
+**Evidence has to reach the reader.** A file path is only evidence
+if the person reading the report can open it. If the `SendUserFile`
+tool is in your toolset, you're on a remote surface where they
+can't — send the screenshots and recordings with it and let the
+report name what you sent. Without it, reference the path and keep
+the evidence that matters inline — pane captures and response
+bodies travel in the report; a bare path only works when the reader
+shares your filesystem.
+
+**Verdicts:**
+- **PASS** — you ran the app, the change did what it should at its
+  surface. Not: tests pass, builds clean, code looks right.
+- **FAIL** — you ran it and it doesn't. Or it breaks something else.
+  Or claim and diff disagree materially.
+- **BLOCKED** — couldn't reach a state where the change is observable.
+  Build broke, env missing a dep, handle wouldn't come up. Not a
+  verdict on the change. Never report an approach blocked or
+  impossible until you've checked the skills and recipes available
+  to you — environment-specific unlocks (headless runners, login
+  helpers, VM harnesses) usually live there. Say exactly where it
+  stopped.
+- **SKIP** — no runtime surface exists. Docs-only, types-only,
+  tests-only. Nothing went wrong; there's just nothing here to run.
+  One line why.
+
+No partial pass. "3 of 4 passed" is FAIL until 4 passes or is
+explained away.
+
+**When in doubt, FAIL.** False PASS ships broken code; false FAIL
+costs one more human look. Ambiguous output is FAIL with the raw
+capture attached — don't interpret.
+
+## Appendix: verifying a CLI change
+
+The handle is direct invocation. The evidence is stdout/stderr/exit code.
+
+1. Build (if the CLI needs building)
+2. Run with arguments that exercise the changed code
+3. Capture output and exit code
+4. Compare to expected
+
+Worked example — **diff:** adds a `--json` flag to the `status`
+subcommand. **Claim:** "machine-readable status output."
+**Inference:** `tool status --json` now exists, emits valid JSON with
+the same fields the human output shows; `tool status` without the
+flag is unchanged.
+
+```bash
+go build -o /tmp/tool ./cmd/tool
+
+/tmp/tool status
+# → Status: healthy / Uptime: 3h12m / Connections: 47
+
+/tmp/tool status --json
+# → {"status":"healthy","uptime_seconds":11520,"connections":47}
+
+/tmp/tool status --json | jq -e .status
+# → "healthy"   (jq -e exits nonzero if the path is null/false)
+echo $?
+# → 0
+```
+
+**Verdict:** PASS — flag works, JSON is valid, fields line up.
+
+What FAIL looks like: `unknown flag: --json` → not wired up, or a
+stale build; output isn't valid JSON → serialization bug; `tool
+status` (no flag) changed → regression; JSON field names differ from
+expected → claim/code mismatch, note it.
+
+If the CLI reads stdin → pipe in test data. If it writes files / hits
+a network / deletes things → point it at a tmp dir / a mock / a
+dry-run flag. No safe mode and the diff touches the destructive
+path → say so and verify what you can around it.
+
+## Appendix: verifying a server/API change
+
+The handle is `curl` (or equivalent). The evidence is the response.
+
+1. Start the server (background, with a readiness poll)
+2. `curl` the route the diff touches, with inputs that hit the changed branch
+3. Capture the full response (status + headers + body)
+4. Compare to expected
+
+Lifecycle, when no launch recipe exists:
+
+```bash
+<start-command> &> /tmp/server.log &
+SERVER_PID=$!
+for i in {1..30}; do curl -sf localhost:PORT/health >/dev/null && break; sleep 1; done
+# ... your curls ...
+kill $SERVER_PID
+```
+
+No readiness endpoint? Poll the route you're about to test until it
+stops returning connection-refused, then add a beat.
+
+Worked example — **diff:** adds a `Retry-After` header to 429
+responses in `rateLimit.ts`. **Claim:** "clients can now back off
+correctly." **Inference:** hitting the rate limit should now return
+`Retry-After: <n>` in the response headers; it didn't before.
+
+```bash
+# trigger the limit — 10 fast requests, limit is 5/sec per the diff
+for i in {1..10}; do curl -s -o /dev/null -w "%{http_code}\n" localhost:3000/api/thing; done
+# → 200 200 200 200 200 429 429 429 429 429
+
+# capture the 429 headers
+curl -si localhost:3000/api/thing | head -20
+# → HTTP/1.1 429 Too Many Requests
+# → Retry-After: 12
+```
+
+**Verdict:** PASS — `Retry-After: 12` present, positive integer.
+
+What FAIL looks like: header absent → the diff didn't take effect, or
+you're not actually hitting the 429 path (check the status code
+first); header present but `NaN` / `undefined` / negative → the logic
+is wrong; 200s all the way through → you never triggered the changed
+path — tighten the burst or check the limit config.