counselors 0.4.12 → 0.5.0

package/README.md

@@ -118,6 +118,137 @@ counselors run -t opus,opus,opus "Review this"  # Run the same tool multiple tim
 | `--json` | Output manifest as JSON |
 | `-o, --output-dir <dir>` | Base output directory |
 
+### `loop [prompt]`
+
+Multi-round dispatch — agents iterate, seeing prior outputs each round.
+
+Each round dispatches to all tools in parallel. Starting from round 2, each agent receives the outputs from all prior rounds, so it can build on previous analysis and avoid repeating findings.
+
+```text
+input: user prompt/focus (e.g.: "focus on the auth module", "look at the sidebar component")
+  |
+  +--> with --preset:
+  |      [repo discovery phase] --> [prompt-writing phase] --> execution prompt (includes boilerplate)
+  +--> without --preset:
+         inline arg prompt:
+           default: [repo discovery phase] --> [prompt-writing phase] --> enhanced execution prompt
+           opt-out: --no-inline-enhancement (skip discovery/prompt-writing)
+         file/stdin prompt: used as provided (discovery/prompt-writing skipped)
+
+all modes: execution boilerplate is always appended
+
+execution prompt
+      |
+      v
++------------------------------- loop rounds -------------------------------+
+| round 1: dispatch to all selected tools in parallel                       |
+|          write per-tool outputs + round notes                             |
+|                                                                           |
+| round N>1: execution prompt + references to prior round outputs           |
+|            (new findings, challenge/refine prior findings)                |
+|            dispatch in parallel, write outputs + notes                    |
+|                                                                           |
+| stop when:                                                                |
+| - max rounds reached, or                                                  |
+| - duration expires, or                                                    |
+| - convergence threshold reached, or                                       |
+| - user aborts (Ctrl+C after current round)                                |
++---------------------------------------------------------------------------+
+      |
+      v
+final notes + run manifest
+```
+
+```text
+Round behavior:
+
+round 1 prompt = base execution prompt
+
+
+round N prompt = base execution prompt
+               // Base execution prompt is amended with...
+               + "Prior Round Outputs" section
+               + @refs to recent prior tool outputs
+               + instruction to avoid duplicate findings, challenge/refine 
+                 prior claims, and expand from prior leads
+```
+
+```bash
+counselors loop "Find and fix test gaps in src/auth/" --rounds 5
+counselors loop --duration 30m "Hunt for edge cases"
+counselors loop --preset bughunt "src/api" --tools opus,codex
+counselors loop --preset hotspots "critical request path" --group smart
+counselors loop --list-presets
+```
+
+| Flag | Description |
+|------|-------------|
+| `--rounds <N>` | Number of dispatch rounds (default: 3) |
+| `--duration <time>` | Max total duration (e.g. `"30m"`, `"1h"`). If set without `--rounds`, runs unlimited rounds until time expires |
+| `--preset <name-or-path>` | Use a built-in preset (e.g. `"bughunt"`) or a custom `.yml/.yaml` preset file |
+| `--list-presets` | List built-in presets and exit |
+| `--no-inline-enhancement` | For non-preset inline prompts, skip discovery + prompt-writing enhancement |
+
+Plus all `run` flags: `-f`, `-t`, `-g`, `--context`, `--read-only`, `--dry-run`, `--json`, `-o`.
+
+**SIGINT handling:** First Ctrl+C finishes the current round gracefully. Second Ctrl+C force-exits immediately.
+
+**Presets** provide domain-specific multi-round workflows.
+
+Built-ins:
+- `bughunt` — bugs, edge cases, and missing test coverage
+- `security` — exploitable vulnerabilities and high-impact security flaws
+- `invariants` — impossible states and state synchronization problems
+- `regression` — behavior changes likely to break existing callers/users
+- `contracts` — mismatches between API producers and consumers
+- `hotspots` — high-impact bottlenecks, including O(n^2)+ patterns
+
+Custom presets (code-grounded):
+
+```yaml
+name: auth-audit
+description: |
+  Audit authentication and authorization code paths for real issues.
+  Ground every claim in repository evidence.
+  For each finding, include concrete file paths and explain the exact control/data flow.
+  Do not speculate about behavior that is not visible in code.
+defaultRounds: 3
+defaultReadOnly: bestEffort
+```
+
+```bash
+counselors loop --preset ./presets/auth-audit.yml "src/auth and middleware"
+counselors loop --preset ./presets/auth-audit.yml "session + token flows" --dry-run
+```
+
+Guidelines for "truth of the code" presets:
+- Write `description` so findings must cite concrete evidence (file paths, functions, branches, tests).
+- Require the agent to separate observed behavior from assumptions and call out unknowns explicitly.
+- Ask for reproducible checks (commands/tests) for each high-confidence claim.
+- Keep the focus target narrow in the prompt argument (specific dirs, modules, or request paths).
+
+### `mkdir [prompt]`
+
+Create a counselors output directory and optionally write `prompt.md` without dispatching.
+
+If you do not provide a prompt (arg, `-f`, or stdin), `mkdir` creates only the containing directory.
+
+Useful when an orchestrating agent wants counselors to own output-dir creation and just return paths.
+
+```bash
+counselors mkdir --json
+counselors mkdir "Review the auth flow for edge cases" --json
+echo "prompt" | counselors mkdir --json
+cat prompt.md | counselors mkdir --json
+counselors mkdir -f prompt.md --json
+```
+
+The JSON output includes:
+- `outputDir`
+- `promptFilePath` (`null` when no prompt was provided)
+- `slug`
+- `promptSource` (`none`, `inline`, `file`, or `stdin`)
+
 ### `init`
 
 Interactive setup wizard. Discovers installed AI CLIs, lets you pick tools and models, runs validation tests.
@@ -163,6 +294,14 @@ counselors cleanup --dry-run --older-than 7d
 counselors cleanup --older-than 36h --yes
 ```
 
+### `config`
+
+Print the config file path and the full resolved configuration as JSON.
+
+```bash
+counselors config
+```
+
 ### `tools`
 
 Manage configured tools.
@@ -274,6 +413,24 @@ Each run creates a directory under your configured output directory (`defaults.o
 
 If the `{slug}` directory already exists, counselors appends a timestamp suffix to avoid collisions.
 
+For multi-round runs (`loop`), each round gets its own subdirectory:
+
+```
+<outputDir>/{slug}/
+  round-1/
+    prompt.md
+    {tool-id}.md
+    {tool-id}.stderr
+    round-notes.md
+  round-2/
+    prompt.md              # augmented with prior round outputs
+    {tool-id}.md
+    round-notes.md
+  ...
+  final-notes.md           # combined notes across all rounds
+  run.json                 # manifest with rounds array
+```
+
 ## Skill / slash command
 
 Install `/counselors` as a skill in Claude Code or other agents:
@@ -286,7 +443,7 @@ counselors skill
 counselors agent
 ```
 
-The skill template provides a multi-phase workflow: gather context, select agents, assemble prompt, dispatch via `counselors run`, read results, and synthesize a combined answer.
+The skill template provides a multi-phase workflow: gather context, select agents, choose dispatch mode (`run` vs `loop`), assemble prompt/focus, create prompt files via `counselors mkdir` when needed, dispatch, read results, and synthesize a combined answer.
 
 ## How is this different from...?
 
@@ -374,6 +531,12 @@ Codex also found 2 bugs all agents acknowledged: dedup by name drops valid sugge
 | Phase ordering | All agree: keep phases 4 and 5 separate, add a Phase 0 for compat checker |
 | `renderer.metrics` hack | All agree: high to extremely high risk of breakage in 0.4.0 |
 
+**Topic: Multi-round test gap hunting** — _`loop --preset test`_
+
+> counselors loop --preset test --scope src/auth/ --rounds 3
+
+Round 1 discovers the test landscape and finds initial gaps. Round 2 reads the round-1 reports and hunts for edge cases the first round missed. Round 3 goes deeper on anything still uncovered. Each agent independently builds on prior findings without repeating them.
+
 ## Security
 
 - **Environment allowlisting**: Child processes only receive allowlisted environment variables (PATH, HOME, API keys, proxy settings, etc.) — no full `process.env` leak.

package/assets/presets/bughunt.yml

@@ -0,0 +1,33 @@
+name: bughunt
+description: |
+  You are hunting for real correctness bugs, edge-case failures, and missing tests that would allow regressions.
+
+  Prioritize:
+  - User-visible correctness failures over style issues
+  - High-blast-radius bugs over speculative nits
+  - Findings likely to produce meaningful failing tests
+
+  Look for:
+  - Logic errors: wrong conditionals, off-by-one, incorrect defaults, null/undefined handling gaps
+  - Boundary and error-path failures: empty inputs, max/min values, partial failures, cleanup/rollback gaps
+  - Validation and contract gaps: unchecked inputs, missing type guards, mismatched return assumptions
+  - Concurrency/order bugs: TOCTOU, race conditions, shared mutable state hazards, invalid async state transitions
+  - Resource-lifecycle bugs: unclosed handles, unreleased locks, dangling listeners, swallowed exceptions in finally blocks
+  - Missing test coverage on risky branches: error handlers, retry logic, fallback paths, migration paths
+
+  Multi-round rule:
+  - Prioritize novel findings not already reported in prior rounds.
+  - If you repeat a prior finding, add new evidence, sharper impact analysis, or a better test strategy.
+
+  For each finding, include:
+  - severity: critical | high | medium | low
+  - confidence: high | medium | low
+  - location: file path + function/method name
+  - evidence: concrete code pattern and why it can fail at runtime
+  - impact: user/system consequence if triggered
+  - minimal fix: smallest safe change
+  - test idea: a concrete failing test scenario (inputs + expected behavior)
+
+  Skip trivial style comments unless they hide a correctness bug.
+defaultRounds: 3
+defaultReadOnly: enforced

package/assets/presets/contracts.yml

@@ -0,0 +1,16 @@
+name: contracts
+description: |
+  You are auditing for API contract drift across server handlers, shared types, clients, validators, and tests.
+
+  Look for:
+  - Request/response field mismatches between API handlers and consumers (missing, renamed, retyped, or re-nested fields)
+  - Optional vs required drift across schemas, runtime validators, and TypeScript/PHP/Python types
+  - Enum and status value drift across backend models, API serializers, and frontend/client assumptions
+  - Inconsistent error contracts: different status codes or error payload shapes for similar failure classes
+  - Versioning and backward-compatibility breaks (silent behavior changes, removed fields, stricter parsing)
+  - Serialization mismatches (date/time formats, number/string coercion, nullability handling)
+  - Documentation/examples/spec files that no longer match actual implementation behavior
+
+  For each issue found, include the producer and consumer locations, describe the concrete contract mismatch, and explain the runtime impact. Suggest a contract test or integration test that would fail before the fix and pass after it.
+defaultRounds: 3
+defaultReadOnly: enforced

package/assets/presets/hotspots.yml

@@ -0,0 +1,35 @@
+name: hotspots
+description: |
+  You are auditing for high-impact performance bottlenecks, with emphasis on asymptotic complexity and scaling behavior.
+
+  Prioritize:
+  - Hot-path issues over cold-path micro-optimizations
+  - Large wins with low implementation risk
+  - Evidence-backed findings over speculative tuning
+
+  Look for:
+  - Accidental O(n^2)+ patterns: nested scans, repeated sort/filter passes, per-item linear lookups
+  - N+1 access patterns across database, API, filesystem, queue, or cache boundaries
+  - Unbounded traversal/fan-out work that grows poorly with input size
+  - Repeated expensive work that should be cached, memoized, batched, or precomputed
+  - Serialization/parsing churn on hot paths (JSON encode/decode loops, repeated cloning/transforms)
+  - Large allocations/copies in tight loops instead of incremental reuse
+  - Missing pagination, streaming, chunking, or backpressure that causes latency/memory spikes
+
+  Multi-round rule:
+  - Prioritize novel hotspots not already reported in prior rounds.
+  - If you repeat a hotspot, add stronger evidence, better complexity analysis, or a lower-risk fix.
+
+  For each finding, include:
+  - severity: critical | high | medium | low
+  - confidence: high | medium | low
+  - location: file path + function/method name
+  - evidence: exact code path and operation causing the cost
+  - complexity: define input variables (for example n, m) and estimate before/after Big-O
+  - impact: expected latency/throughput/memory effect and where it appears
+  - minimal fix: smallest safe change (index/map usage, batching, caching, pagination, etc.)
+  - validation idea: benchmark/profiling or test strategy to confirm the gain
+
+  Skip tiny micro-optimizations unless they are clearly on a critical hot path.
+defaultRounds: 4
+defaultReadOnly: enforced

package/assets/presets/invariants.yml

@@ -0,0 +1,17 @@
+name: invariants
+description: |
+  You are auditing a codebase for state synchronization issues, impossible states, and state management anti-patterns.
+
+  Look for:
+  - Boolean explosion: multiple booleans creating 2^n states where many combinations are impossible (e.g. isLoading && isError both true)
+  - Impossible states: bags of optionals instead of discriminated unions — types that allow combinations that should never exist
+  - Magic strings: string literals used for status/state comparisons instead of enums or constants
+  - Status mismatches: database enums not matching code enums (different spelling, different count, different casing)
+  - Duplicated state: the same data stored in multiple locations that can get out of sync
+  - Derived state stored: computed values persisted when they could be calculated on the fly (e.g. totalCount stored instead of items.length)
+  - Missing state machines: complex multi-step flows or status fields with 4+ values managed with ad-hoc conditionals instead of explicit state machines
+  - Single source of truth violations: the same authoritative data defined in multiple places (validation rules duplicated client/server, type definitions copied across files, permissions checked in both frontend and backend with different logic)
+
+  For each issue found, include the file path, the specific code pattern, and what can go wrong. Suggest the minimal fix — discriminated union, enum extraction, computed getter, or state machine. Focus on drift that can cause real bugs at runtime, not theoretical concerns.
+defaultRounds: 3
+defaultReadOnly: enforced

package/assets/presets/regression.yml

@@ -0,0 +1,16 @@
+name: regression
+description: |
+  You are auditing a codebase for regression risk with emphasis on behavior changes that can break existing users or dependent systems.
+
+  Look for:
+  - Contract drift in function signatures, return shapes, event payloads, or CLI flags that callers may rely on
+  - Removed or weakened guards (validation, authorization, null checks) that previously prevented invalid states
+  - Refactors that changed control flow or ordering semantics in subtle ways (initialization order, retry order, cleanup timing)
+  - Partial migrations where old and new code paths can diverge in behavior
+  - Error handling regressions: swallowed exceptions, changed status codes, missing rollback/cleanup in failure paths
+  - Feature flag or config default changes that alter runtime behavior without clear migration handling
+  - Tests that assert implementation details but miss observable behavior, leaving real regressions undetected
+
+  For each issue found, include file path and function/method name, explain the user-visible regression risk, and suggest a concrete failing test that would catch it. Prioritize high-blast-radius risks over low-impact code style concerns.
+defaultRounds: 3
+defaultReadOnly: enforced

package/assets/presets/security.yml

@@ -0,0 +1,17 @@
+name: security
+description: |
+  You are a security engineer reviewing a codebase with an attacker's mindset. Your goal is to find exploitable vulnerabilities, not theoretical concerns.
+
+  Look for:
+  - Injection flaws: SQL, NoSQL, OS command, LDAP, or template injection from unsanitized user input reaching queries, shells, or eval
+  - Broken authentication: weak password handling, missing rate limiting, session fixation, credential exposure in logs or error messages
+  - Broken access control: missing authorization checks, insecure direct object references (IDOR), privilege escalation, path traversal
+  - Sensitive data exposure: secrets in source code, unencrypted storage or transit, excessive data in API responses, PII in logs
+  - Cross-site scripting (XSS): reflected, stored, or DOM-based — user input reaching HTML, attributes, or JavaScript without escaping
+  - Insecure deserialization: untrusted data passed to deserializers (pickle, unserialize, JSON.parse of executable content, yaml.load)
+  - Security misconfiguration: verbose error messages leaking internals, debug mode in production, default credentials, overly permissive CORS
+  - Missing cryptographic controls: hardcoded keys, weak algorithms (MD5, SHA1 for passwords), predictable tokens, improper random number generation
+
+  For each vulnerability found, include the file path, the vulnerable code pattern, how an attacker would exploit it, and the specific fix. Prioritize findings by exploitability — a real injection flaw matters more than a missing security header.
+defaultRounds: 3
+defaultReadOnly: enforced