@nklisch/pi-agile-workflow 0.15.3

package/skills/perf-scout/SKILL.md

@@ -0,0 +1,404 @@
+---
+name: perf-scout
+description: >
+  Cross-domain performance *idea* generator — a speculative brainstorm engine, not a profiler. Use
+  when the user asks for performance ideas, optimization angles, "how could this be faster", "what
+  would a game engine / database / GPU do here", a perf brainstorm, "ways to speed this up", or a
+  broad sweep of possible speedups. Fans parallel highest-thinking-model scouts across strategy lenses
+  (algorithmic, memory/locality, parallelism/SIMD, GPU, caching, I/O, distributed, game-engine,
+  database internals, compiler/runtime, approximation) through scanner agents, each surfacing *candidate* optimizations
+  borrowed from demanding domains as unvalidated hypotheses — never asserting gains are real. Then a
+  different model class (via peeragent, or a fresh max-effort sub-agent) prunes weak ideas and
+  surfaces missed angles. Writes a ranked report and parks ideas to .work/. Distinct from perf-design
+  (which measures and asserts — hand validated ideas there), bug-scan (correctness), and
+  refactor-design (structure).
+---
+
+# Perf-Scout
+
+You run a **cross-pollination idea engine** for performance. You take the
+hardest-won strategies from the most demanding domains — AAA game engines,
+database and storage internals, HPC, high-frequency trading, distributed
+systems, GPU programming, compilers — and ask, for the code in front of you:
+*could any of these apply here?*
+
+You detect the stack and workload shape, choose relevant **strategy lenses**,
+dispatch **one deep scanner/scout agent per lens in parallel** (each at the highest
+thinking model available, using a generic sub-agent prompted with the scanner posture from `../principles/references/subagents.md`),
+aggregate and rank the candidate ideas, then run the
+whole deck past a **different model class** to prune the weak ideas and surface
+the angles your first pass missed. The output is a ranked deck of *candidate*
+optimizations and parked backlog ideas.
+
+## What this is — and what it is NOT
+
+This skill **generates ideas**. It does not measure, profile, or prove anything.
+
+- Every idea is a **hypothesis**: "borrowing X from how databases / game engines
+  / GPUs solve this, *this code might* go faster." You never assert a gain is
+  real — you can't, because you didn't measure it. Labelling ideas as speculative
+  is a feature, not a hedge: it's what makes a 40-idea brainstorm safe to read.
+- Each idea carries a **validation path** — usually "hand to
+  `/agile-workflow:perf-design <id>` to actually profile and confirm." That skill
+  *measures and asserts*; this one *ideates and hypothesizes*. They're a pair.
+- Distinct siblings: **perf-design** (profile, measure, assert, design the fix),
+  **bug-scan** (correctness bugs), **refactor-design** (structure/abstraction).
+  When the user wants *what to investigate*, that's you. When they want *what's
+  actually slow and proven*, that's perf-design.
+
+Why speculation is worth shipping: the most valuable optimization is usually the
+one you didn't think to consider. A different domain's playbook — a GPU's memory
+coalescing, a game engine's entity-component layout, an LSM tree's write
+batching — applied to an ordinary web service is exactly the angle a single mind
+on a deadline never reaches for. This skill's job is to reach for all of them.
+
+## Highest-thinking-model mandate
+
+Idea quality scales hard with model strength, and this skill deliberately trades
+tokens for insight. Run **every** scanner/scout agent at maximum: use the host's strongest
+read-only scout/reviewer setting, and use extra-high reasoning where the host
+exposes it. Do not downgrade "simple" lenses. This does not replace the
+peeragent cross-model pass: the peer reviewer still runs at the deepest effort
+its agent supports (see Phase 5). Large deck reviews can take 10 to 30 minutes;
+do not assume a quiet process has hung after only a few minutes.
+
+If you ever feel tempted to save tokens by downgrading a model here, don't — that
+trade is the opposite of what this skill is for.
+
+## Modes (standalone only)
+
+There is no release-gate mode. Speculative ideas are not ship blockers; they're
+investigation candidates. The path to action runs through perf-design or scope.
+
+- `/agile-workflow:perf-scout` — scout the whole repo
+- `/agile-workflow:perf-scout <path>` — scope to a directory or glob (`src/render/`, `**/*.rs`)
+- `/agile-workflow:perf-scout <NL scope>` (e.g. "the request hot path", "the data ingest pipeline") — interpret against the codebase
+- `/agile-workflow:perf-scout --lenses a,b,c` — restrict to named lenses (skip selection)
+- `/agile-workflow:perf-scout --no-peer` — skip the cross-model pass (warn: it's the headline value)
+- `/agile-workflow:perf-scout --park-all` — park every tier including long-shots (default parks only Investigate-first + Worth-a-look)
+- `/agile-workflow:perf-scout --no-park` — write the report only, skip backlog parking
+
+## The strategy lenses (and their references)
+
+Each scout loads exactly one of these — that's the progressive-disclosure move
+that keeps each scout focused and your context lean. Each reference catalogs
+~10-15 strategies with the domain they're borrowed from, the code signals that
+*suggest* (never prove) the strategy might apply, the speculative win, the
+cost/risk, and how to validate.
+
+| Lens | Reference | When to load |
+|---|---|---|
+| Algorithmic & data structures | [references/algorithmic-and-data-structures.md](references/algorithmic-and-data-structures.md) | Almost always. Loops, repeated work, nested iteration, sorting/searching, recomputation, growing collections. |
+| Memory & data locality | [references/memory-and-data-locality.md](references/memory-and-data-locality.md) | CPU-bound hot paths, large object graphs, per-element processing, structs/arrays of records, pointer-heavy traversal. |
+| Parallelism & vectorization | [references/parallelism-and-vectorization.md](references/parallelism-and-vectorization.md) | Independent work done sequentially, map/reduce shapes, numeric kernels, multi-core idle, large batch processing. |
+| GPU & accelerators | [references/gpu-and-accelerators.md](references/gpu-and-accelerators.md) | Massively-parallel regular numeric work, large array/matrix/tensor math, image/signal processing, existing GPU/CUDA/Metal code, or compute-bound kernels that could move off the CPU. |
+| Caching & memoization | [references/caching-and-memoization.md](references/caching-and-memoization.md) | Repeated identical computation/requests, expensive pure functions, hot read paths, derived values recomputed per call. |
+| I/O & batching | [references/io-and-batching.md](references/io-and-batching.md) | DB/network/disk calls, per-item I/O in loops, serialization, syscalls, N+1 patterns, chatty service boundaries. |
+| Distributed systems | [references/distributed-systems.md](references/distributed-systems.md) | Multi-node/service code, queues, RPC, replication, cross-region calls, coordination, fan-out/fan-in. |
+| Game-engine & realtime | [references/game-engine-and-realtime.md](references/game-engine-and-realtime.md) | Per-frame/per-tick loops, simulations, spatial queries, entity collections, rendering, soft real-time budgets, allocation churn in a loop. |
+| Database & storage internals | [references/database-and-storage-internals.md](references/database-and-storage-internals.md) | Custom storage/query/index code, in-memory stores, scans/filters/joins, write-heavy paths, on-disk formats. |
+| Compiler, runtime & language | [references/compiler-runtime-and-language.md](references/compiler-runtime-and-language.md) | Hot inner loops, allocation/GC pressure, dynamic dispatch, boxing, interpreters, string building, reflection. |
+| Approximation & precomputation | [references/approximation-and-precomputation.md](references/approximation-and-precomputation.md) | Exact answers where approximate would do, membership/cardinality/quantile queries, similarity search, expensive lookups computable ahead of time. |
+
+## Phase 1: Stack & shape discovery
+
+Detect what you're scouting — structurally, with no profiling (you never measure
+in this skill):
+
+- **Languages & runtimes** — extensions, manifests (`package.json`, `Cargo.toml`,
+  `go.mod`, `pyproject.toml`, `pom.xml`, `*.csproj`, `CMakeLists.txt`).
+- **Workload shape** — pick the dominant one(s); it steers lens relevance:
+  - *Request/response service* (HTTP/RPC handlers) → I/O, caching, distributed.
+  - *Batch / data pipeline* (ETL, jobs, stream processors) → algorithmic, parallelism, I/O, approximation.
+  - *Real-time / simulation / rendering / game loop* → game-engine, memory-locality, parallelism.
+  - *Library / CPU kernel / engine internals* (parsers, codecs, DB, compiler) → algorithmic, memory-locality, compiler/runtime, database-internals.
+- **Hot-ish regions** — entry points and inner loops *likely* to dominate: API
+  routes, the main loop, per-request/per-event/per-frame code, nested loops,
+  serialization, query builders. You're guessing where the heat is, not
+  measuring it — say so.
+- **Existing perf posture** — note any benchmarks, caches, pools, or SIMD already
+  present, so scouts don't re-propose what's already there.
+
+Summarize the stack and shape in 5-7 lines. This summary goes into every scout
+brief.
+
+## Phase 2: Lens selection
+
+Map the detected stack and workload shape to the 11 lenses. Mark each **most
+relevant**, **relevant**, or **skip**.
+
+**structured question tool checkpoint** (multi-select): show the lenses with relevance
+annotations. Recommend the most-relevant set for a focused scout, or all 11 for a
+full cross-domain sweep. Default to all "most relevant" + "relevant" if the user
+doesn't pick. If invoked with `--lenses a,b,c`, skip the prompt and use exactly
+those.
+
+Run at least 4 lenses — fewer than that defeats the cross-pollination purpose.
+Always include `algorithmic-and-data-structures` unless explicitly excluded; it's
+relevant to nearly all code.
+
+## Phase 3: Fan-out idea generation
+
+For each selected lens, spawn **one parallel scanner/scout agent in a single message**
+so they run concurrently. Use a generic sub-agent prompted with the scanner posture from `../principles/references/subagents.md`,
+the host's generic/general-purpose subagent prompted with the scanner/scout
+posture from `../principles/references/subagents.md`, and extra-high reasoning
+when available; otherwise use the same-host read-only ideation fallback.
+
+### Scope (passed into every scout)
+
+Resolve the scope to a concrete file list with `git ls-files` (plus the path
+filter or NL interpretation, if any). Cap absurd scopes: if the repo is huge,
+focus scouts on the hot-ish regions from Phase 1 and say so in the report.
+
+### Scout brief template
+
+Each scout gets this brief. Read it carefully — the framing (hypotheses, not
+findings; borrowed-from attribution; validation path) is what makes the output
+honest:
+
+> You are a **perf-scout scanner agent** for the **<lens>** lens. Follow the
+> agile-workflow scanner contract: source-read-only, no fixes, no recursive
+> sub-agents. Your job is to
+> generate *candidate* performance ideas — speculative hypotheses, not proven
+> wins. You do not measure or profile. You surface angles worth investigating.
+>
+> **Reference (load FIRST)**: `<absolute path to references/<lens>.md>`
+> Read the whole file. It is your catalog of strategies, where each is borrowed
+> from, and the code signals that *suggest* it might apply.
+>
+> **Scope** — consider ONLY these files:
+> ```
+> <file list>
+> ```
+>
+> **Stack & workload shape** (from orchestrator):
+> ```
+> <stack summary from Phase 1>
+> ```
+>
+> **Already present** (don't re-propose what exists): `<caches/pools/SIMD/etc., or "none noted">`
+>
+> **Method**:
+> 1. Load the lens reference. Internalize its strategies and detection signals.
+> 2. Web-search 1-3 times for current, domain-specific techniques for THIS stack
+>    and workload (e.g. "data-oriented design ECS cache locality 2025", "io_uring
+>    batching Rust", "DuckDB vectorized execution technique"). Bring in strong
+>    ideas from the literature even if not in the reference — attribute them.
+> 3. Read the in-scope code through this lens. For each place the lens *could*
+>    apply, form a candidate idea. Prefer high-leverage, non-obvious angles
+>    borrowed from a demanding domain over generic "add a cache" advice.
+> 4. For each candidate idea, record the fields below. Be concrete about WHERE
+>    (cite `file:line` or `file:function`) and WHAT (the specific transform).
+> 5. Be honest about applicability. An idea you're only 30% sure fits is still
+>    worth surfacing — just mark it **Speculative**. Do NOT fabricate file
+>    locations; if an idea is general ("the whole ingest path could be batched"),
+>    cite the representative site.
+>
+> **Per-idea fields**:
+> - **Title** (one line — the idea, not the problem)
+> - **Strategy** (named strategy from the reference, or "new" with a one-line source)
+> - **Borrowed from** (the domain this technique comes from — game engines, LSM trees, GPU, HFT, etc.)
+> - **Location** (`file:line` or `file:function` — the representative site)
+> - **The idea** (2-4 sentences: the specific transform you'd try here)
+> - **Why it might help** (the speculative mechanism — *might*, never *will*)
+> - **Leverage / Applicability / Cost** (see the ranking rubric — High/Med/Low each)
+> - **Validate by** (how to confirm it's real: the metric to measure, the benchmark to write, "profile via perf-design")
+> - **Risk** (correctness, complexity, memory, or behavior-change concerns — one line)
+>
+> **Rules**:
+> - These are hypotheses. Never claim a speedup is real or guaranteed. Use
+>   "might", "could", "is a candidate for".
+> - Cite a representative `file:line`/`function` for every idea.
+> - Don't propose what already exists (see "Already present").
+> - Quality over quantity: 5 sharp, non-obvious, well-located ideas beat 25
+>   generic ones. Aim for the ideas a specialist from that domain would spot.
+> - Empty is valid: if a lens genuinely doesn't apply to this code, say so.
+> - Don't implement anything. Ideas only.
+>
+> **Output** — a single markdown doc:
+> ```
+> ## Lens: <name>
+> ## Applies here?: <one-line — strongly / partially / barely, and why>
+> ## Candidate ideas
+>
+> ### Idea 1
+> - **Title**: ...
+> - **Strategy**: ...
+> - **Borrowed from**: ...
+> - **Location**: `file:line`
+> - **The idea**: ...
+> - **Why it might help**: ...
+> - **Leverage**: High|Med|Low · **Applicability**: Likely|Plausible|Speculative · **Cost**: Low|Med|High
+> - **Validate by**: ...
+> - **Risk**: ...
+> ```
+> Followed by:
+> ```
+> ## Summary
+> - Files considered: <n>
+> - Strategies applied: <list>
+> - Ideas: High-leverage=<n>, Medium=<n>, Low=<n>
+> ```
+
+### Wave coordination
+
+- Dispatch all selected scouts in a **single message** (parallel).
+- If a scout errors, record it as a gap in the report — don't blindly re-run.
+- If a scout returns >20 ideas, ask it (via follow-up message) to keep the top 15 by
+  Leverage × Applicability. A flood usually means generic pattern-matching rather
+  than the sharp domain-borrowed ideas you want.
+
+## Phase 4: Aggregate, dedupe, cluster, rank
+
+Collect every scout's ideas, then:
+
+1. **Dedupe** — same `file:line` + same essential transform across two lenses →
+   keep one, note "also surfaced by: <lens>". Two *different* transforms at the
+   same site are two ideas; keep both.
+2. **Cluster** — group by file/subsystem. A subsystem with many ideas may deserve
+   a single "rethink this layer's data model" idea rather than ten micro-ideas.
+3. **Rank** — score each idea with the rubric in
+   [references/idea-ranking.md](references/idea-ranking.md): a **Priority tier**
+   (Investigate-first / Worth-a-look / Long-shot) from Leverage × Applicability
+   vs. Cost × Risk. Read that reference; don't invent your own scale.
+4. **Score each lens 0-10** for *opportunity density* — how much plausible,
+   high-leverage headroom this lens found (see the ranking reference). This is an
+   opportunity signal, NOT a quality grade of the code.
+
+## Phase 5: Cross-model peer pass (the headline)
+
+A different model has different blind spots. The whole point of this pass is to
+catch the ideas your scouts collectively missed and prune the ones that don't
+hold up — exactly what a fresh, differently-trained mind is good at. Run it by
+default; only skip on `--no-peer` (and warn that you're skipping the best part).
+
+Read **[references/peer-review-pass.md](references/peer-review-pass.md)** for the
+full mechanics. In brief:
+
+1. **Pick the reviewer.** Prefer a **different model class** via peeragent
+   (resolve the wrapper from the peeragent plugin location — never assume it's on
+   PATH) to maximize blind-spot diversity — pick the concrete target from the
+   host→peer pairing in [../principles/references/models.md](../principles/references/models.md)
+   (Claude host → codex/gemini/zai; Codex host → claude opus/gemini/zai; etc.).
+   Same-class peers add little; in that case use the fallback. A top-tier reasoning
+   peer may take 10 to 30 minutes for a large deck review; no return after a few
+   minutes is not a hang. This pass is the **adversarial** phase; for a deep or
+   complex deck, precede it with a **completeness/advisory** pass from a *second*
+   different class if available (see the two-phase ordering in
+   [../principles/references/models.md](../principles/references/models.md) §6).
+2. **Fallback** when peeragent is unavailable, fails, or would be same-class:
+   spawn a **fresh max-effort reviewer sub-agent** when available that sees only
+   the codebase, the lens catalog, and the idea deck — NOT your scouts'
+   reasoning — so it reviews with independent context. This is the "max compute
+   and quality subagent" fallback.
+3. **Ask the reviewer for three things** (write the deck to a temp file, pass via
+   `--prompt-file`): (a) which ideas are weak, implausible, or wrong, and why;
+   (b) **gaps** — high-leverage angles, strategies, or whole lenses the deck
+   missed; (c) new candidate ideas in the same hypothesis framing. Include the
+   no-recursion instruction. Don't lead the reviewer toward your conclusions.
+4. **Integrate honestly.** Prune ideas you agree are weak (note them as "pruned by
+   peer: <reason>" — don't silently delete). Add gap ideas, attributed
+   `source: peer-<agent>`. If the peer names a missed *lens*, you may fan out one
+   more scout (Phase 3) for it, then re-rank.
+5. **One pass is usually enough** — idea generation isn't a converging artifact
+   the way a design doc is. Loop a second time only if the peer opened a whole new
+   high-value direction.
+
+Record what the peer pruned and added; it goes in the report so the user sees the
+cross-model value.
+
+## Phase 6: Output
+
+Two outputs: a ranked report AND parked backlog ideas (unless `--no-park`).
+
+### 6a. Park ideas to backlog (default)
+
+Skip entirely on `--no-park`. Otherwise:
+
+1. **Verify substrate.** If `.work/CONVENTIONS.md` is missing, the project isn't
+   on agile-workflow — skip parking, note it in the report, and tell the user to
+   run `/agile-workflow:convert` if they want substrate tracking. (The report is
+   still the primary deliverable; parking degrades gracefully.)
+2. **Choose which ideas to park.** By default, park only **Investigate-first** and
+   **Worth-a-look** ideas — these are the candidates someone might actually action.
+   Long-shots stay in the report only, so the backlog doesn't fill with speculative
+   noise. Pass `--park-all` to park every tier including long-shots. The report
+   always lists all tiers regardless.
+3. **Build the existing-ideas index** for idempotency. The canonical dedupe key is
+   (`idea_location`, `idea_lens`) — the same key used everywhere in this skill:
+   ```bash
+   mkdir -p /tmp; : > /tmp/perfscout-existing.txt
+   for f in .work/backlog/perf-scout-*.md; do
+     [ -f "$f" ] || continue
+     grep -q '^idea_origin: scout$' "$f" || continue
+     loc=$(grep -m1 '^idea_location:' "$f" | awk '{print $2}')
+     lens=$(grep -m1 '^idea_lens:' "$f" | awk '{print $2}')
+     [ -n "$loc" ] && echo "$loc $lens $f" >> /tmp/perfscout-existing.txt
+   done
+   ```
+4. **Park each selected idea** as `.work/backlog/perf-scout-<slug>.md` using
+   [references/parked-item-template.md](references/parked-item-template.md). Skip
+   any idea whose (`file:line`, `idea_lens`) pair already appears in the index;
+   tally "duplicates skipped". On slug collision, suffix `-2`, `-3`. **Record the
+   exact path of every file you create** — you'll stage those explicitly next.
+5. **Commit** the newly-created files in one commit. Stage only the paths you
+   created this run (never a `perf-scout-*.md` glob — that can sweep pre-existing
+   or unrelated edits; never `git add -A`):
+   ```bash
+   git add <each new file path created this run>
+   git commit -m "perf-scout: park <N> ideas (<scope>)"
+   ```
+
+### 6b. Write the report
+
+Write `perf-scout-report.md` at repo root using
+[references/report-template.md](references/report-template.md). Include: the
+stack & shape profile, the per-lens opportunity-density table, ideas grouped by
+**Priority tier** (each with borrowed-from, location, validation path, and parked
+id), a "Top 5 to investigate first" callout, the cross-model peer summary
+(pruned + added), lenses skipped + why, and the parking summary.
+
+### 6c. Summarize to the user
+
+- The stack & workload shape you scouted.
+- Idea counts by Priority tier (Investigate-first / Worth-a-look / Long-shot).
+- The **Top 5 to investigate first**, one line each with `file:line` and the
+  domain each is borrowed from.
+- What the cross-model peer pass pruned and added (the headline value).
+- Path to the report; parked count + duplicates skipped (or why parking was skipped).
+- A reminder that **every idea is an unvalidated hypothesis** — nothing here is a
+  measured win yet.
+- **structured question tool** (next steps):
+  - "Hand the top idea(s) to `/agile-workflow:perf-design` to profile and validate"
+  - "Promote a tier into tracked work via `/agile-workflow:scope`"
+  - "Dive deeper into one lens (re-scout with `--lenses <lens>`)"
+  - "Done for now — leave the parked ideas in backlog"
+
+## Guardrails
+
+- **Never assert a gain is real.** You didn't measure it. Every idea is a
+  hypothesis with a validation path. This is the load-bearing rule — the skill is
+  only safe to use *because* it's honest about being speculative.
+- **Always attribute borrowed-from.** The cross-domain provenance is the value;
+  "this is how a game engine / LSM tree / GPU does it" is what makes an idea worth
+  reading. An idea with no source domain is probably a generic one — sharpen it.
+- **Every idea has a validation path.** If you can't say how to confirm it, you
+  haven't thought it through. Usually: hand to perf-design.
+- **The ideation happens in the scanner/scout agents, not here.** Your job is discovery,
+  dispatch, aggregation, the peer pass, and the report. Don't re-derive a scout's
+  work in the orchestrator — that throws away the progressive-disclosure win.
+- **Always fan out, always at max model.** One scout per lens, parallel, Opus /
+  xhigh. Never serialize, never downgrade.
+- **Run the peer pass by default.** A different model class is the single best
+  source of the angles you missed. Skipping it (`--no-peer`) should feel like a
+  loss, and you should say so.
+- **Quality over quantity.** A deck of 12 sharp, well-located, domain-borrowed
+  hypotheses beats 60 generic "add a cache / parallelize it" lines. Prune
+  aggressively in Phase 4 and let the peer prune more.
+- **Don't measure, don't profile, don't fix.** Measurement is perf-design.
+  Fixing is implement. This skill stops at ranked, parked ideas.
+- **Respect scope.** If the user gave a path or NL scope, don't expand past it.
+- **Idempotent re-runs.** Index existing `perf-scout-*` backlog ideas before
+  parking; skip ideas already captured at the same (`file:line`, `idea_lens`) —
+  the one canonical dedupe key, used here and in the parked-item template.

package/skills/perf-scout/agents/openai.yaml

@@ -0,0 +1,6 @@
+interface:
+  display_name: "Perf Scout"
+  short_description: "Use Perf Scout for repo workflows"
+  default_prompt: "Use $perf-scout for this repository."
+policy:
+  allow_implicit_invocation: false

package/skills/perf-scout/references/algorithmic-and-data-structures.md

@@ -0,0 +1,126 @@
+# Algorithmic & Data Structures — Perf-Scout Strategy Lens
+
+> **When to load this lens**: almost always. Any code with loops, repeated work,
+> nested iteration, searching/sorting, recomputation, or growing collections.
+>
+> **Where these ideas come from**: classical algorithms, competitive programming,
+> database query planners, and the "do less work" school of optimization. The
+> highest-leverage wins almost always live here — a better complexity class beats
+> any micro-optimization. Every entry is a *candidate* hypothesis, not a proven
+> win. Cite a representative `file:line`, attribute the source, give a validation
+> path.
+
+## Detection signals (where this lens might apply — not proof)
+- Nested loops over the same or related collections (`for ... for ...`), especially O(n²) shapes.
+- Linear scans to find/membership-test (`.find`, `.filter(...)[0]`, `in list`, `indexOf`) inside a loop.
+- Repeated identical computation across iterations or calls with the same inputs.
+- Sorting inside a loop, or re-sorting data that's already mostly ordered.
+- Building strings/collections by repeated concatenation or append-with-grow.
+- Recomputing aggregates (sum, max, count) from scratch after small mutations.
+- Recursion with overlapping subproblems; full recomputation on each call.
+
+## Strategies
+
+### 1. Lower the complexity class
+- **Borrowed from**: algorithm design fundamentals.
+- **The idea**: replace an O(n²) (or worse) shape with O(n log n) or O(n) by choosing a structure that answers the inner question in O(1)/O(log n) — hash set/map for membership, sorted array + binary search, heap for top-k.
+- **Code signals**: a linear `find`/`includes`/`in` inside a loop; double loops comparing all pairs.
+- **Speculative win**: superlinear-to-linear collapse; the only kind of win that *grows* with input size.
+- **Cost / risk**: extra memory for the index; index build cost must amortize over enough queries.
+- **Validate by**: benchmark across input sizes to confirm the slope changed, not just the constant — hand to perf-design.
+
+### 2. Precompute / build an index once
+- **Borrowed from**: database indexing.
+- **The idea**: pay once to build a lookup structure (map, sorted index, prefix-sum array, inverted index) so each later query is cheap, instead of scanning every time.
+- **Code signals**: the same collection scanned repeatedly with different keys; range/aggregate queries recomputed.
+- **Speculative win**: turns N scans into 1 build + N cheap lookups.
+- **Cost / risk**: staleness if the source mutates; memory; only wins when queries ≫ rebuilds.
+- **Validate by**: count actual query-to-rebuild ratio; benchmark build + query vs. scan.
+
+### 3. Memoize / cache pure computation
+- **Borrowed from**: dynamic programming, functional programming.
+- **The idea**: cache results of expensive pure functions keyed by arguments; reuse on repeat calls.
+- **Code signals**: a deterministic function called repeatedly with a small set of distinct inputs.
+- **Speculative win**: collapses repeated work to one computation per distinct input.
+- **Cost / risk**: unbounded cache growth; correctness if inputs aren't truly the full key. (Caching lens covers eviction.)
+- **Validate by**: measure hit rate on real input distribution.
+
+### 4. Incrementalize instead of recompute
+- **Borrowed from**: incremental computation, spreadsheet engines, React reconciliation.
+- **The idea**: maintain a result as inputs change (delta update) rather than recomputing from scratch — running sums, dirty-flag recompute, diff-and-patch.
+- **Code signals**: full recompute of an aggregate/derived structure after a small mutation; "recalculate everything" on any change.
+- **Speculative win**: O(change) instead of O(total) per update.
+- **Cost / risk**: incremental logic is subtler and easier to get wrong than full recompute.
+- **Validate by**: benchmark with realistic mutation/read ratios; property-test incremental == full.
+
+### 5. Amortize with the right growable structure
+- **Borrowed from**: amortized analysis, dynamic arrays.
+- **The idea**: replace repeated O(n) reallocation/shift with an amortized-O(1) structure — pre-sized buffers, ring buffers, deques, gap buffers, string builders.
+- **Code signals**: `array.shift()`/insert-at-front in a loop, string `+=` in a loop, list grown one element at a time without reserve.
+- **Speculative win**: removes a hidden O(n²) from naive concatenation/insertion.
+- **Cost / risk**: minimal; mostly an API swap.
+- **Validate by**: benchmark on large inputs where the quadratic bites.
+
+### 6. Pick a structure that matches the access pattern
+- **Borrowed from**: data-structure theory.
+- **The idea**: match structure to dominant operation — heap for top-k/priority, balanced tree / skip list for ordered range queries, trie for prefix matching, union-find for connectivity, segment/Fenwick tree for range aggregates, bitset for dense membership.
+- **Code signals**: sorting just to take the top few; repeated range/prefix/aggregate queries done by scan.
+- **Speculative win**: each query drops from O(n) to O(log n) or O(1).
+- **Cost / risk**: more code; structure must fit the real query mix.
+- **Validate by**: enumerate the real operation mix; benchmark the candidate structure.
+
+### 7. SIMD-probed open-addressing hash table
+- **Borrowed from**: Google Abseil Swiss tables and Facebook folly F14 (`abseil.io/about/design/swisstables`, `facebook/folly` F14.md).
+- **The idea**: when a hot hash map is the bottleneck, a Swiss/F14-style table stores tiny per-slot control bytes contiguously and scans a 16- or 14-slot group with a single SIMD instruction, so most lookups skip key comparisons and tolerate ~87% (Swiss) / 12-of-14 (F14) load factors with short probe chains.
+- **Code signals**: a profile dominated by `std::unordered_map`/chaining-map lookups or rehash churn; pointer-chasing per probe; high-frequency point lookups on a large map.
+- **Speculative win**: lookups might drop several-fold and memory shrink versus a node-based chaining map, because the SIMD group scan replaces per-node comparisons and indirection.
+- **Cost / risk**: open addressing perturbs iteration order and reference stability; pathological hashes still degrade; only matters if map ops actually dominate the profile.
+- **Validate by**: swap in the candidate map (e.g. `absl::flat_hash_map`, `folly::F14`) behind the same interface and A/B the hot path; profile via perf-design.
+
+### 8. Cache-friendly ordered index (adaptive radix tree)
+- **Borrowed from**: the Adaptive Radix Tree (ART) in the HyPer main-memory DB (Leis et al., ICDE 2013, `db.in.tum.de/~leis/papers/ART.pdf`).
+- **The idea**: when you need *ordered* lookups plus range scans, an adaptive radix tree sizes each node (4/16/48/256 children) to its actual fanout and uses path compression, giving trie-style O(key-length) lookups with far less space and pointer-chasing than a balanced BST — and unlike a hash map it preserves order for range queries.
+- **Code signals**: a balanced BST / red-black map on the hot path for ordered or prefix/range access; a sorted array re-bisected on every query with frequent inserts; sort-then-range patterns.
+- **Speculative win**: lookups might approach hash-map speed while keeping order; the HyPer paper measured ART near 2x a tuned hash-table/red-black combo on its workload — treat that as a candidate, not a promise.
+- **Cost / risk**: implementation complexity; wins are key-distribution-dependent; a plain hash map still beats it when order is never needed.
+- **Validate by**: benchmark ART (e.g. an `art`/`ART`-style library) against the current ordered structure on the real key set and query mix; profile via perf-design.
+
+### 9. Compressed bitmap for set-heavy membership and intersection
+- **Borrowed from**: Roaring bitmaps, used in Apache Lucene, Druid, Spark, ClickHouse, Pinot (`roaringbitmap.org`, "Better bitmap performance with Roaring", arXiv:1402.6407).
+- **The idea**: represent large integer-ID sets as a Roaring bitmap (per-64K-chunk array/bitmap/run containers) so AND/OR/ANDNOT run as native word-parallel bitwise ops, replacing per-element hash-set loops or list intersections — fast on dense sets, compact on sparse ones, no manual tuning.
+- **Code signals**: filtering/joining by intersecting large ID sets via nested loops or `HashSet.retainAll`; postings-list / tag / permission / candidate-row set algebra; many `contains` checks over dense integer keys.
+- **Speculative win**: intersections might run multiple-fold faster and far smaller than hash-set or sorted-list approaches, since whole 64-bit words are combined at once instead of element-by-element.
+- **Cost / risk**: only fits integer-keyed (or dictionary-encoded) sets; serialization/format lock-in; no benefit on tiny sets.
+- **Validate by**: model real IDs as roaring bitmaps and benchmark the set-op hot path vs. the current approach at production cardinalities; profile via perf-design.
+
+### 10. Reformulate the problem
+- **Borrowed from**: math/algorithmic insight, query optimization.
+- **The idea**: change *what* you compute — process in a different order, transform to a cheaper equivalent (e.g. sort-then-scan instead of all-pairs, two-pointer instead of nested loop, prefix sums instead of repeated range sums, bucket/counting sort when keys are bounded).
+- **Code signals**: brute-force all-pairs; "for each X, search all Y"; repeated overlapping range work.
+- **Speculative win**: often the biggest single jump — a different formulation can erase a whole loop.
+- **Cost / risk**: requires understanding the problem, not just the code; highest insight cost.
+- **Validate by**: prove equivalence on the domain; benchmark.
+
+### 11. Short-circuit and prune
+- **Borrowed from**: search algorithms (alpha-beta, branch-and-bound).
+- **The idea**: exit early, prune branches that can't matter, order checks cheapest-first / most-discriminating-first, bound the search.
+- **Code signals**: full scans that could stop at first match; expensive predicate evaluated before a cheap one; computing all results when only existence/first/any is used.
+- **Speculative win**: average-case savings, sometimes large, with no structural change.
+- **Cost / risk**: low; watch correctness of the early-exit condition.
+- **Validate by**: benchmark on representative (not worst-case) inputs.
+
+### 12. Batch the work, not the items
+- **Borrowed from**: vectorized databases, BLAS.
+- **The idea**: process many items per operation instead of one-at-a-time — group, sort-and-merge, do set operations once over the whole collection rather than per element.
+- **Code signals**: per-element work that re-derives shared context each time; one-row-at-a-time processing of a bulk dataset.
+- **Speculative win**: amortizes fixed per-call overhead across the batch.
+- **Cost / risk**: larger working set; latency vs. throughput tradeoff.
+- **Validate by**: benchmark batch vs. per-item at realistic sizes.
+
+### 13. Use cheaper exactness (lazy / streaming)
+- **Borrowed from**: lazy evaluation, streaming/iterator pipelines.
+- **The idea**: avoid materializing intermediate collections; stream through transforms; compute only what's consumed (take-n, first, any).
+- **Code signals**: `map().filter().slice(0,k)` materializing the whole list before slicing; building a big intermediate to use a fraction.
+- **Speculative win**: avoids work for unconsumed elements; lower peak memory.
+- **Cost / risk**: laziness can obscure cost and complicate debugging.
+- **Validate by**: measure how much of the intermediate is actually consumed; benchmark.