@vextlabs/theron-cli 0.2.1 → 0.4.0

package/skills/ab-test.md

@@ -0,0 +1,89 @@
+---
+name: ab-test
+description: Design and analyze a controlled A/B test end-to-end — covering metric + MDE definition, power/sample-size/duration, randomization unit, pre-registration, SRM + novelty checks, effect estimation with confidence intervals, multiple-comparison correction, and a go/no-go decision; invoke when a user wants to run or evaluate any controlled experiment on product, model, UI, pricing, ranking, or policy changes.
+allowed-tools: Read, Bash, Write
+---
+
+═══ HARD RULES ═══
+
+1. NEVER peek and stop early. The moment you run the primary analysis is fixed at pre-registration; stopping because p < 0.05 appeared inflates false-positive rate to 26%+ at just two unplanned looks. If continuous monitoring is required, use a sequential/SPRT design with a pre-registered α-spending function.
+2. NEVER change the primary metric after data collection begins. Secondary metrics may be added to the exploratory tier but never promoted to primary post-hoc.
+3. NEVER pool variants that "look similar" post-hoc. Each planned pairwise comparison is pre-registered; unplanned merges are p-hacking.
+4. NEVER interpret "no statistical significance" as "no effect." Report the 95% CI width against the MDE to characterize precisely what was and was not ruled out.
+5. NEVER skip the Sample Ratio Mismatch (SRM) check. An SRM invalidates causal inference regardless of how compelling the metric lift looks.
+6. NEVER report a point estimate without its confidence interval and the achieved power at the observed effect size.
+7. NEVER run the experiment shorter than the pre-registered duration. Minimum is max(computed_duration, 14 days) to capture at least two full weekly seasonal cycles. The 14-day floor is firm; 7 days is not sufficient.
+
+═══ PHASE A — DEFINE THE QUESTION AND PRIMARY METRIC ═══
+
+A1. Write a single falsifiable hypothesis: "Changing [X] will increase/decrease [metric] by at least [MDE] for [population] within [time window]."
+A2. Select ONE primary metric that is (a) directly causally linked to the treatment, (b) measurable at the randomization unit without aggregation ambiguity, (c) has a known historical distribution (mean, variance, or proportion) from at least 4 weeks of pre-experiment data. Do not accept a composite "North Star" metric as primary unless it has a closed-form variance estimator — composites typically require the delta method or bootstrap.
+A3. Enumerate secondary metrics: supporting metrics expected to move in the same direction as the primary, and guardrail metrics that must not degrade beyond a pre-specified absolute threshold. Guardrail violations trigger a hard NO-GO regardless of the primary metric result.
+A4. Specify the Minimum Detectable Effect (MDE): the smallest effect that is business-meaningful, not the smallest the test can detect at current traffic. Express in absolute terms (e.g., +0.5 pp conversion) AND relative terms (e.g., +3.3% lift on a 15% base rate). If the team cannot articulate the MDE, halt here — no power calculation is meaningful without it.
+A5. Document the null and alternative hypotheses formally. Default to two-sided unless a directional prior is pre-registered before any data is seen and the one-sided framing is explicitly justified (e.g., treatment can only improve latency, never harm it).
+
+═══ PHASE B — POWER, SAMPLE SIZE, AND DURATION ═══
+
+B1. Choose significance level α (default 0.05, two-sided) and power 1−β (default 0.80; use 0.90 for high-stakes or irreversible rollout decisions where a missed effect is costly).
+B2. Compute required sample size N per variant using the appropriate formula for the metric type:
+    - Proportion (conversion, CTR): n = 2 · (z_{α/2} + z_β)² · p̄(1−p̄) / δ²  where p̄ = (p_control + p_treatment)/2, δ = MDE in absolute pp.
+    - Continuous (revenue, latency, session length): n = 2 · (z_{α/2} + z_β)² · σ² / δ²  where σ² is the per-unit variance from historical data.
+    - Ratio metrics (revenue-per-user, clicks-per-impression): apply the delta method to estimate variance from historical data; do not use the naive ratio of aggregate means as the estimator.
+B3. Compute duration: duration (days) = N_total / (daily_eligible_users × traffic_fraction). Apply the hard floor: required_duration = max(computed_duration, 14 days). Document the traffic ramp schedule and whether the 14-day floor or the computed duration is binding.
+B4. If computed duration exceeds 8 weeks, revisit the design rather than relaxing α. Options in priority order: (1) increase the MDE to a still-meaningful but larger threshold and document the business rationale, (2) apply CUPED variance reduction (see Phase G) which can reduce N by 30–70% when pre-experiment correlation ρ > 0.4, (3) switch to a pre-registered sequential/SPRT design with an O'Brien-Fleming or Pocock α-spending function. Do not simply raise α to 0.10 to make the test shorter — that inflates false positives without reducing required N meaningfully for proportions.
+B5. Document all power-analysis inputs: baseline rate or mean, variance source and historical window, MDE, α, β, traffic fraction, ramp schedule, and the N calculation tool or script used.
+
+═══ PHASE C — RANDOMIZATION DESIGN ═══
+
+C1. Choose the randomization unit: the entity that receives a consistent treatment for the experiment's full duration. For user-facing features use user_id; for infrastructure or request-level experiments use request_id only if there is strictly zero carry-over between requests (stateless path); for marketplace experiments use the side of the market that controls the outcome variable (supply or demand, not both simultaneously).
+C2. Validate unit independence: if treatment on unit A changes outcomes for unit B (network effects, shared recommendation pools, leaderboards, shared cache), switch to cluster randomization and recompute N with the design effect: DEFF = 1 + (m−1)ρ, where m = mean cluster size and ρ = intra-cluster correlation estimated from historical data.
+C3. Use a stable deterministic hashing function (e.g., SHA256 of experiment_id + unit_id modulo num_buckets) for assignment. Never use random(seed=timestamp) — it creates re-assignment on page reload or service restart. Isolate the experiment namespace: use a per-experiment salt (e.g., hash(experiment_id + unit_id)) so that bucket assignments across concurrent experiments are statistically independent.
+C4. Stratify or block on major confounders (platform OS, country, user cohort, account age decile) when they explain >20% of primary metric variance; stratification reduces required N or increases effective power at no traffic cost.
+C5. Holdout rule: units assigned at bucketing time remain in their assigned bucket for the full experiment window, even if they become ineligible mid-experiment (intent-to-treat analysis). Exclusions must be pre-registered, not reactive.
+
+═══ PHASE D — PRE-REGISTRATION ═══
+
+D1. Before any treatment is exposed to users, write and lock a pre-registration document containing: hypothesis (A1), primary metric + formula, secondary metrics and guardrail thresholds, randomization unit and assignment mechanism, N per variant, planned duration and end date, analysis method (estimator + CI method), multiple-comparison correction method, and explicit decision criteria (what result → SHIP / ITERATE / KILL / EXTEND).
+D2. Commit this document to version control with a timestamp before the experiment launches. The commit hash is the audit trail. Store in a shared experiment log directory indexed by experiment_id.
+D3. Specify the analysis date and trigger explicitly: "Primary analysis will be run on [ISO date] after [N] unique units have been exposed for at least 24 hours, whichever is later."
+D4. For sequential designs, pre-register the α-spending function (e.g., O'Brien-Fleming), all planned look times, and the maximum sample size at which the experiment is stopped regardless of result.
+
+═══ PHASE E — INSTRUMENTATION AND LAUNCH ═══
+
+E1. Verify logging before any traffic is exposed: confirm that assignment events (unit_id, variant, timestamp, experiment_id) and all outcome events are being emitted, persisted, and joinable on the same key. Run a 24-hour shadow log with zero traffic to validate the pipeline end-to-end.
+E2. Ramp gradually: launch at 1–5% of eligible traffic, hold for at least one hour, and check error rates and guardrail metrics against baseline before expanding. This ramp check is an operational health check — it is NOT an A/A significance test and must not be used to make an early ship/kill call.
+E3. Monitor guardrail metrics daily via simple threshold alerts, not p-value gates. Monitoring the primary metric during the run is peeking and inflates false-positive rate (Rule 1).
+E4. Log all external events that could confound results (marketing campaigns, infrastructure incidents, seasonality events, competitor product launches) in the shared experiment log with timestamps. These are required for the post-experiment validity review.
+
+═══ PHASE F — SRM AND VALIDITY CHECKS ═══
+
+F1. Sample Ratio Mismatch (SRM): at analysis time, run a chi-squared test on observed assignment counts vs. expected counts under the target split. If p < 0.01, the experiment is causally invalid — do not analyze the primary metric until the SRM is explained and resolved. Common causes: bot or crawler traffic included in assignment, logging loss asymmetric across variants, sticky-bucketing failures, exposure logging before assignment completes, or assignment-to-logging lag differences.
+F2. Pre-experiment A/A balance check: using only pre-experiment data, compare the primary metric baseline across the assigned variants (pseudo-randomize on historical units using the same hashing function). A significant imbalance (p < 0.05) on a pre-experiment window metric is a red flag for assignment mechanism failure, not a genuine treatment effect.
+F3. Novelty and primacy effect check: segment results by experiment day and by user tenure (new vs. established users). If the treatment lift decays monotonically over the experiment window or is concentrated entirely in the first 3 days, the effect is likely novelty-driven. Require the effect to be present and non-trivial in the final third of the experiment window's calendar days before classifying it as durable.
+F4. SUTVA (Stable Unit Treatment Value Assumption) check: compare outcomes of control units that interacted with treated units vs. control units that did not. A statistically significant difference signals interference violations; escalate to a cluster or switchback experiment design.
+
+═══ PHASE G — EFFECT ESTIMATION ═══
+
+G1. Use the pre-registered estimator exclusively. For proportions: difference in proportions with normal approximation CI (use exact Fisher or Wilson CI if n < 1000 per variant). For continuous metrics: two-sample Welch t-test or OLS with pre-registered covariates. Do not switch estimators post-hoc.
+G2. CUPED variance reduction (Controlled-experiment Using Pre-Experiment Data): θ̂_CUPED = (ȳ_T − ȳ_C) − θ·(x̄_T − x̄_C), where θ = Cov(Y, X) / Var(X) estimated from the CONTROL ARM ONLY (not pooled, to avoid contamination), and X is the pre-experiment value of the primary metric for the same unit. CUPED reduces variance; it does not change the point estimate in expectation, only narrows the CI. Apply only if pre-registered.
+G3. Report all of the following: point estimate, 95% CI (lower, upper), two-sided p-value, achieved power at the observed effect size (to quantify what was ruled out), and the absolute business impact at current volume (e.g., "+0.4 pp CTR; at 30M daily eligible users = +120K additional clicks/day").
+G4. For ratio metrics (revenue/user, clicks/impression): compute SE using the delta method — Var(R) ≈ (1/μ_D²)·[Var(N) + R²·Var(D) − 2R·Cov(N,D)] — or bootstrap with ≥1000 iterations. Never use the naive ratio of aggregate sums as the standard error input.
+G5. The CI is the primary statistical output. The p-value is secondary and must always be reported alongside the CI, never in isolation.
+
+═══ PHASE H — MULTIPLE COMPARISONS ═══
+
+H1. Apply a correction for all pre-registered simultaneous comparisons. Default: Benjamini-Hochberg (BH) FDR control at q = 0.05 when testing 3+ secondary metrics; BH assumes test statistics are independent or positively correlated (PRDS condition), which holds for most product metrics that move together. Use Bonferroni when the tests are planned AND there is a specific need to control the family-wise error rate (FWER) rather than FDR — Bonferroni is valid regardless of correlation structure but is conservative when tests are correlated.
+H2. The primary metric is always tested at the full pre-registered α. Secondary metrics use the adjusted α output from the correction procedure. Guardrail metrics use pre-specified absolute thresholds, not adjusted p-value gates.
+H3. Exploratory metrics (post-hoc segmentation, unexpected segments, drill-downs discovered after unblinding) are hypothesis-generating only. Label them explicitly as EXPLORATORY in all outputs. Never make a ship/kill decision on exploratory findings; register them as hypotheses for the next experiment.
+H4. For multi-variant tests (A/B/C/…): compare each treatment variant to control using Dunnett's test (or BH on the set of treatment-vs-control contrasts), not all pairwise. This preserves FWER against the control comparison without the unnecessary power penalty of testing all pairs.
+
+═══ PHASE I — DECISION ═══
+
+I1. Apply the pre-registered decision rule (D1). Do not deviate based on visual dashboard inspection, stakeholder pressure, or proximity to a release deadline. The rule was set when the stakes were clear and the data was unseen — that is its entire value.
+I2. SHIP if: primary metric effect size has CI fully above zero (or above the pre-specified positive threshold), no guardrail metric has violated its pre-specified threshold, SRM check passed, and novelty check passed.
+I3. ITERATE if: effect is directional (point estimate positive) but the CI crosses zero AND the lower bound of the CI is above the negative MDE threshold (i.e., the treatment is not meaningfully harmful) AND there is a concrete, mechanistically grounded hypothesis for improving the treatment variant.
+I4. KILL if: any guardrail metric violated its pre-registered threshold; SRM failed and cannot be explained by a known logging artifact; or the CI's upper bound is below the MDE (power analysis confirms a meaningful effect was ruled out, not merely not observed).
+I5. EXTEND (one time only, pre-registered before unblinding) if: the required N was not reached due to lower-than-forecast traffic AND the pre-registered end date has not yet passed. Lock the new end date in version control before looking at any unblinded data. Never extend because the result is inconclusive after the pre-registered end date.
+I6. Write a post-experiment record for every experiment: what was tested, what was found (including CIs), what was shipped, what was learned about the metric relationship and variance. Archive in the experiment log. Variance and baseline rate estimates from completed experiments are the most accurate inputs for future power calculations in the same product surface.
+
+KEY PRINCIPLE: The experiment design is a contract — sign it before seeing the data, honor it without revision after.

package/skills/api-design.md

@@ -0,0 +1,175 @@
+---
+name: api-design
+description: Design clean, evolvable APIs — start from consumer use cases, make illegal states unrepresentable, explicit errors, versioning and backward-compatibility.
+allowed-tools: Read, Write, Edit, Grep, Glob
+---
+
+## Phase 0 — Grep for existing conventions FIRST
+Before writing a single type, grep the repo for naming patterns, error shapes, pagination contracts, and auth patterns already in use. New APIs must match the grain of what exists.
+
+```
+Grep for: "export type.*Result" | "export interface.*Response" | "export type.*Error"
+Grep for: cursor | offset | page | limit  (pagination convention)
+Grep for: "401\|403\|404" in route handlers  (error-response shape)
+Grep for: "Bearer\|x-api-key\|Authorization"  (auth header convention)
+```
+
+Hard rule: if you invent a new naming or error convention, you own migrating everything to it. Prefer conforming.
+
+---
+
+## Phase 1 — Write the call site FIRST (design-by-example)
+Before any implementation, write 3–5 concrete example usages as if you are the consumer:
+
+```ts
+// Example 1: happy path
+const result = await receipts.create({ agentId: "theron-cyber", claim: "...", sig: key })
+if (!result.ok) return handleError(result.error)  // typed discriminated union
+console.log(result.value.receiptId)
+
+// Example 2: list with pagination
+const page = await receipts.list({ agentId, limit: 50, cursor: prev.nextCursor })
+
+// Example 3: illegal state that must be rejected at the type level
+receipts.create({ agentId: undefined })  // should not compile
+```
+
+Ask: does this call site feel obvious? Is the intent readable? Are required vs optional params clear from types alone? Iterate here before writing any implementation.
+
+---
+
+## Phase 2 — Model the domain
+1. Name the RESOURCES (nouns): what are the durable entities? (Receipt, Agent, Run, Finding, Artifact)
+2. Name the OPERATIONS (verbs): create / get / list / update / delete / publish / verify — choose from a fixed vocabulary.
+3. State the INVARIANTS: what must always be true? (A receipt must have a valid sig. An agent must belong to an org. A run cannot be deleted while active.)
+4. Draw the ownership graph: who owns what? (Org → Agent → Run → Finding). This drives URL nesting depth and authz checks.
+
+Hard rule: no more than two levels of nesting in a REST path (`/orgs/:orgId/agents/:agentId`). Deeper = flatten with a query param.
+
+---
+
+## Phase 3 — Choose the right shape
+- **REST** for CRUD over resources with standard HTTP semantics (receipts, agents, runs). Use nouns, HTTP verbs (GET/POST/PATCH/DELETE), 200/201/204/400/401/403/404/409/422/500.
+- **RPC / action endpoint** for operations that are not CRUD (POST `/runs/:id/cancel`, POST `/receipts/:id/verify`). Name them as verb-noun.
+- **GraphQL** only if the consumer needs flexible field selection across many related resources in one round-trip. Overhead rarely justified for internal/CLI APIs.
+- **Library API (TypeScript functions)** for SDK-internal logic, tool contracts, and anything called in-process. Prefer typed functions over string-keyed maps.
+
+---
+
+## Phase 4 — Make illegal states unrepresentable
+1. Use discriminated unions for result types — no mixed `{ data?, error? }` bags:
+   ```ts
+   type Result<T, E = ApiError> = { ok: true; value: T } | { ok: false; error: E }
+   ```
+2. Use branded types or `z.string().uuid()` to prevent mixing IDs:
+   ```ts
+   type ReceiptId = string & { readonly __brand: "ReceiptId" }
+   ```
+3. Encode state machines in the type (a `Run` in state `"completed"` has a `completedAt`; in `"pending"` it does not — use a union of state-shapes, not optional fields).
+4. Never use `any`. Never accept `unknown` without a Zod parse at the boundary.
+
+---
+
+## Phase 5 — Explicit, typed errors (never silent)
+Every error must carry: `code` (stable machine-readable string), `message` (human), optional `field` for validation errors, optional `retryAfter` for rate limits.
+
+```ts
+type ApiError =
+  | { code: "NOT_FOUND";       message: string }
+  | { code: "UNAUTHORIZED";    message: string }
+  | { code: "FORBIDDEN";       message: string }
+  | { code: "VALIDATION";      message: string; fields: Record<string, string> }
+  | { code: "CONFLICT";        message: string }
+  | { code: "RATE_LIMITED";    message: string; retryAfter: number }
+  | { code: "INTERNAL";        message: string }
+```
+
+Hard rule: HTTP 500 must never leak stack traces or internal state to the client. Log internally, return a `INTERNAL` code externally.
+
+---
+
+## Phase 6 — Collections: pagination, filtering, sorting
+Use CURSOR-based pagination (not offset) for any collection that can grow:
+```ts
+// Request
+{ limit?: number; cursor?: string; filter?: { agentId?: string; since?: string } }
+// Response
+{ items: T[]; nextCursor: string | null; total?: number }
+```
+
+- Default `limit` to something sane (20–50). Cap it (max 200).
+- Cursors are opaque base64-encoded server state. Never expose internal IDs as cursors.
+- Filtering params use the same field names as the resource fields — no surprise aliases.
+- Sorting: `sort=createdAt:desc` as a single string param. Default sort must be documented.
+
+---
+
+## Phase 7 — Mutating endpoints: idempotency + safety
+- POST endpoints that create resources must accept an optional `idempotencyKey` (client-generated UUID). Return the same response on replay within 24h.
+- PATCH must be partial (only send what changes). PUT replaces the whole resource — avoid PUT unless you mean it.
+- DELETE must be idempotent: deleting an already-deleted resource returns 204, not 404.
+- Destructive operations (delete, purge, revoke) must require explicit confirmation param OR a two-step (request → confirm token → execute).
+
+---
+
+## Phase 8 — Security: authz at the boundary
+1. Authenticate FIRST (parse + verify the bearer token/API key) before touching any DB or running any logic.
+2. Authorize at the resource level: after fetching the resource, assert the caller's org/role owns it. Never trust a URL param to scope access.
+3. Validate ALL inputs with Zod (or equivalent) at the route handler entry point. Reject before processing.
+4. Rate-limit at the gateway or middleware layer, not inside business logic.
+5. Never return a resource that belongs to a different org, even in error messages ("receipt not found" not "receipt belongs to another org").
+
+---
+
+## Phase 9 — Versioning + backward-compatibility
+- URL version prefix (`/v1/`) for REST. Bump only on BREAKING changes.
+- Additive changes (new optional field, new endpoint, new enum value) are NOT breaking — ship them freely.
+- Breaking changes (remove field, rename field, change type, change semantics) require a new version OR a deprecation window (min 90 days, deprecation header on every response).
+- Add `Deprecation: true` + `Sunset: <date>` response headers on deprecated endpoints.
+- Keep a `CHANGELOG.md` entry for every change: `[added]`, `[deprecated]`, `[removed]`, `[breaking]`.
+- Hard rule: never remove a field from a response without a deprecation period. Consumers break silently.
+
+---
+
+## Phase 10 — Minimal required params + least-surprise defaults
+- Every required param must be impossible to omit (type-enforced, not runtime-guarded).
+- Every optional param must have a documented default that is the overwhelmingly common case.
+- If a caller must pass the same param on every call, it belongs in the client constructor, not the method signature.
+- Boolean flags are a smell — prefer an enum that names the intent (`mode: "strict" | "lenient"` not `strict: boolean`).
+
+---
+
+## Phase 11 — Document the contract
+For each endpoint/function, write inline:
+1. One-line purpose.
+2. All params: name, type, required/optional, default, constraints.
+3. Response shape with field semantics.
+4. All error codes that can be returned and when.
+5. One working example (request + response).
+
+For REST APIs, emit an OpenAPI 3.1 spec (`openapi.yaml`) generated from the types — do not hand-write it separately from the code.
+
+---
+
+## Phase 12 — Validate against use cases before finalizing
+Return to the example call sites from Phase 1. For each:
+- Can it be expressed with the designed API without workarounds?
+- Are there any surprise required params the consumer wouldn't know at call time?
+- Does the error path give the consumer enough information to recover or display a useful message?
+- Would a new engineer reading only the call site understand what it does?
+
+If any answer is no, revise the design — not the examples.
+
+---
+
+## Hard rules summary
+1. Call site first. Implementation is a detail.
+2. Match existing conventions — grep before inventing.
+3. Discriminated Result unions. No mixed `data?/error?` bags.
+4. Illegal states must not compile.
+5. Errors are typed and machine-readable (`code` field).
+6. Cursor pagination for all collections.
+7. Authz at the boundary, on every request, on the actual resource.
+8. Additive change is free. Breaking change costs a version bump or 90-day deprecation.
+9. Never leak internals in error responses.
+10. Document the contract in the code, not in a separate doc that goes stale.

package/skills/architecture-design.md

@@ -0,0 +1,185 @@
+---
+name: architecture-design
+description: Design a system from the non-functional requirements + load numbers — components and boundaries, storage/consistency tradeoffs, failure modes, ADRs, and design-for-change.
+allowed-tools: Read, Write, Grep, Glob
+---
+
+## Phase 0 — Read the repo before designing anything
+
+1. Grep for existing conventions: naming patterns, error shapes, transport layers, auth, DB schema, env var conventions.
+2. Identify what already exists that the new system can use or must integrate with. Designing in a vacuum produces orphan systems.
+3. State the single sentence the design must satisfy: "Support X doing Y with Z constraint." If you cannot write this sentence, stop and clarify.
+
+---
+
+## Phase 1 — Nail the requirements (functional + non-functional)
+
+4. List functional requirements as user-facing behaviors: "User can create a receipt. Receipt is verifiable offline. Agent can query its own run history."
+5. List the non-functional requirements that actually drive architecture — these are the ones architects miss:
+   - **Scale**: peak QPS read / peak QPS write / concurrent users / data volume at launch / at 12 months / at 36 months
+   - **Latency**: p50 / p99 targets per critical path (interactive vs. background)
+   - **Availability**: acceptable downtime per month (99.9% = 43 min/mo; 99.99% = 4 min/mo — know the number before picking topology)
+   - **Consistency**: can users see stale data? For how long? Can you lose writes? (CAP forces a choice — name it)
+   - **Durability**: what data can never be lost? (receipts, payments, audit logs are non-negotiable; ephemeral cache is fine to lose)
+   - **Security**: who are the threat actors? What must be encrypted at rest / in transit? What must be auditable?
+   - **Cost envelope**: $/month ceiling, cost sensitivity (bandwidth-heavy? storage-heavy? compute-heavy?)
+   - **Evolvability**: what is the most likely next feature? Design for that specific change, not for infinite unknowns
+   - **Compliance / regulatory**: GDPR, SOC2, HIPAA, export control — flag if any apply before choosing storage region or vendor
+6. Separate hard constraints (must-haves, deal-breakers) from soft preferences. Hard constraints are immovable; preferences are trade-off inputs.
+
+---
+
+## Phase 2 — Estimate the load (architecture follows the numbers)
+
+7. Do back-of-envelope math. Write it down explicitly — guesses buried in prose are not reviewable:
+   ```
+   Users:          10K DAU at launch → 100K at 12mo
+   Write QPS:      10K DAU × 5 writes/day / 86400s ≈ 0.6 QPS → rounds to ~1 QPS (comfortable single Postgres)
+   Read QPS:       10× write ratio = ~6 QPS peak → trivial, no read replica needed yet
+   Storage:        5KB per receipt × 10K users × 365 days = ~18GB/year → single DB volume, no sharding
+   Growth trigger: at 100K users writes hit ~6 QPS sustained → revisit at that milestone
+   ```
+8. From the numbers, derive what the architecture DOES NOT need yet. Explicitly state what you are deferring and at what load threshold to revisit. This prevents premature optimization.
+9. Identify the ONE bottleneck most likely to be hit first. Design around it. Everything else is secondary.
+
+---
+
+## Phase 3 — Identify core components, responsibilities, and boundaries
+
+10. List every major component as a box with ONE clear responsibility statement. If you need "and" to describe it, split it.
+11. Draw the dependency arrows. Dependencies must be acyclic at the component level. Cycles = wrong decomposition — break them with an interface or an event.
+12. Apply the test: could you swap out component X without touching Y? If not, the boundary is wrong.
+13. Separate stateless from stateful components. Stateless can be scaled horizontally and replaced trivially. Stateful components need replication, failover, and migration strategies — treat them with extra care.
+14. Name the INTERFACES between components (not implementations). The interface is the contract; the component is a detail.
+
+Typical component taxonomy for reference (do not copy blindly — omit what does not apply):
+- **Gateway / edge**: auth enforcement, rate limiting, routing, TLS termination
+- **API layer**: request validation, orchestration, response shaping — no business logic
+- **Domain services**: business rules, aggregates, invariant enforcement
+- **Workers / queue consumers**: async background jobs, retries, idempotency
+- **Data stores**: primary DB, cache, blob store, search index, event log
+- **External integrations**: third-party APIs, webhooks — always behind an anti-corruption layer
+
+---
+
+## Phase 4 — Data model and storage choice
+
+15. Model the data around access patterns, not around entities. Ask "what queries will this system run?" before writing a schema.
+16. Identify the access patterns for each entity: point lookup by ID? range scan by time? full-text search? aggregation? Each pattern pulls toward a different storage type.
+17. Choose storage type by workload shape:
+    - **Relational (Postgres/Neon)**: transactions, foreign keys, flexible queries, moderate scale, strong consistency — default choice until proven wrong
+    - **Key-value (Redis/Upstash)**: ephemeral state, sessions, leases, rate-limit counters, sub-millisecond reads — never use as a primary store for durable data
+    - **Blob / object store (R2/S3)**: large binary objects, model weights, artifacts, media — never query inside blobs
+    - **Time-series / append-only log**: audit trails, event streams, telemetry — immutable by design
+    - **Search index (Postgres full-text / Elastic)**: keyword search, fuzzy match — derived, always reconstructible from the primary store
+18. State the CAP/consistency tradeoff explicitly:
+    - "We choose consistency over availability for receipts — a 503 is better than a corrupt receipt."
+    - "We choose availability over consistency for agent status — stale status for 30s is acceptable."
+19. Design the schema migrations strategy upfront: can you add columns without downtime? (Yes for Postgres add-column with default.) Can you rename columns? (No, requires a multi-step deploy.) Know the answer before you ship.
+20. Identify what data is owned by this system vs. what it reads from another system. Do not store data you do not own — subscribe to events or call the owning service.
+
+---
+
+## Phase 5 — Data flow and critical paths
+
+21. Trace the critical path for the top 3 user-facing operations end-to-end: every hop, every I/O, every transformation.
+22. Count the number of synchronous I/O calls on each critical path. Every added hop adds latency and a failure point. Minimize hops on the hot path.
+23. Identify which steps MUST be synchronous (user is waiting) vs. which can be async (fire-and-forget, queue, or eventual). Move as much as possible off the synchronous path.
+24. Draw the write path separately from the read path. They often have different consistency and latency requirements and can be optimized independently.
+25. Identify fan-out: does one write trigger writes to multiple downstream systems? Fan-out is a reliability multiplier — if any downstream fails, what happens?
+
+---
+
+## Phase 6 — Key tradeoffs — name them explicitly, pick with reasons
+
+Do not hide tradeoffs in vague language. State each one as "Option A vs Option B — we pick A because X, accepting the downside of Y."
+
+26. **Sync vs. async**: sync = simpler, easier to reason about, couples latency; async = decoupled, resilient, harder to debug and trace.
+27. **Strong vs. eventual consistency**: strong = correct but slower and harder to scale; eventual = faster and more available but requires idempotency and conflict resolution everywhere.
+28. **Monolith vs. services**: monolith = simpler ops, refactor freely, one deploy, no network hops; services = independent deploy, isolated failure, harder to trace and test.
+29. **Normalize vs. denormalize**: normalized = consistent, smaller writes, expensive reads requiring joins; denormalized = fast reads, write amplification, consistency burden moves to application code.
+30. **Push vs. pull**: push (webhooks, events) = low latency, complex fan-out, receiver must be available; pull (polling) = simple, receiver controls load, higher latency.
+31. **Build vs. buy**: build = control, cost at scale, maintenance burden; buy = speed, vendor lock-in, ongoing cost.
+
+---
+
+## Phase 7 — Failure modes and degradation strategy
+
+32. For each component, ask: "What happens when this is unavailable?" Design the degraded behavior explicitly — do not leave it to chance.
+33. Apply bulkhead pattern: isolate failure domains so that a failure in component X does not cascade to Y. Separate thread pools, connection pools, and circuit breakers per downstream dependency.
+34. Define timeouts everywhere: every outbound network call must have a timeout. "Infinite wait" is not a valid strategy. Set timeouts based on the SLA of the caller, not the SLA of the callee.
+35. Define retry policy per operation type:
+    - Idempotent reads: retry with exponential backoff + jitter, max 3 attempts
+    - Non-idempotent writes: do NOT retry automatically — require idempotency key + deduplication at the receiver
+    - Background jobs: retry with backoff, dead-letter queue after N failures, alert on DLQ depth
+36. Idempotency is not optional for any write that can be retried. Every mutation endpoint must accept an idempotency key or be idempotent by nature (PUT, DELETE).
+37. Backpressure: if a downstream consumer is slower than the producer, the queue grows unbounded. Name the backpressure mechanism: queue depth limit, rate limiting at ingress, or consumer auto-scaling trigger.
+38. Define the runbook for the top 3 failure scenarios now, not after the first outage:
+    - DB primary goes down → failover to replica, expected downtime, data loss window
+    - Queue backs up → consumer scale-out trigger, alert threshold
+    - External API is unavailable → circuit open, cached response or graceful error to user
+
+---
+
+## Phase 8 — Identify the riskiest assumption and de-risk it first
+
+39. List every assumption the design rests on. Rank by: (probability of being wrong) × (cost if wrong).
+40. The top-ranked assumption is the one to validate BEFORE writing a single line of production code. Build the smallest possible spike — a throwaway prototype, a load test, a proof-of-concept query — to confirm or refute it.
+41. Common high-risk assumptions: "this third-party API will have the latency we need," "Postgres can handle this query at 100× current load," "the queue consumer can keep up with peak write throughput," "users will do X not Y."
+42. Document the result of the de-risking experiment. If the assumption was wrong, revise the design before committing to an implementation.
+
+---
+
+## Phase 9 — Architecture Decision Records (ADRs)
+
+For every significant decision, write a micro-ADR inline or in `docs/adr/`:
+
+```
+## ADR-NNN: <title>
+**Date**: YYYY-MM-DD
+**Status**: Accepted | Superseded by ADR-NNN
+
+**Context**: <the forces at play, the problem being solved, constraints>
+**Decision**: <what we decided, stated precisely>
+**Consequences**: <what becomes easier, what becomes harder, what we accept as a known cost>
+```
+
+43. Decisions that always need an ADR: storage engine choice, consistency model, sync vs. async, service boundary, auth mechanism, API versioning strategy, caching strategy.
+44. ADRs are immutable — when a decision changes, write a new ADR marked "Supersedes ADR-NNN." Never edit an accepted ADR to change its decision.
+
+---
+
+## Phase 10 — Design for change
+
+45. Ask for each component: "What is the most likely change in the next 6 months?" Then verify the component boundary isolates that change — a change inside it must not ripple outward.
+46. Likely-to-change things: pricing model, auth provider, storage backend, third-party API contract, business rules. Wrap each behind an interface or adapter — the rest of the system depends on the abstraction, not the implementation.
+47. Stable abstractions principle: depend on stable things (interfaces, domain concepts) not volatile things (vendor SDKs, specific DB query syntax).
+48. Feature flags are a design choice, not an afterthought. Any capability that may need to be toggled, rolled out gradually, or killed quickly must be flag-gated from day one.
+49. Schema migrations must be backward-compatible for at least one full deploy cycle: add before remove, nullable before required, two-phase rename (add new column → backfill → switch reads → drop old).
+
+---
+
+## Phase 11 — Anti-over-engineering check
+
+50. Before finalizing, apply these tests:
+    - Do you actually have the load that justifies sharding, caching, or a queue? If not, remove it.
+    - Does the number of services match the number of independent deploy/scale requirements? If not, merge.
+    - Is there a simpler data model that handles 90% of the use cases? Take it.
+    - Can this be a single Postgres table with an index instead of a separate service? Check seriously.
+    - Could a junior engineer understand and operate this system without you? If not, simplify.
+51. Write down what you deliberately chose NOT to build and why. This is as important as what you built — it prevents well-intentioned additions from violating the design.
+
+---
+
+## Hard rules
+
+1. Architecture follows the numbers. Estimate load before choosing topology.
+2. Name every tradeoff and pick with reasons. Hidden tradeoffs become bugs.
+3. Failure modes are first-class. Every component needs a degradation answer.
+4. Idempotency is not optional for any write that can fail and be retried.
+5. De-risk the riskiest assumption before writing production code.
+6. One ADR per significant decision, immutable after acceptance.
+7. Design for the specific likely change, not infinite unknowns.
+8. Avoid the scale you do not have. State the threshold that triggers revisiting.
+9. Stateful components are expensive — minimize their number and isolate them.
+10. Match existing repo conventions. A design that ignores the grain of the codebase becomes a maintenance island.

package/skills/business-case.md

@@ -0,0 +1,77 @@
+---
+name: business-case
+description: Build a rigorous business case for a decision or investment: size the problem and opportunity, model costs/benefits/risks across options including do-nothing, surface assumptions explicitly, and deliver a single prioritized recommendation with leading indicators and the strongest counterargument pre-empted.
+allowed-tools: Read, WebSearch, Write
+---
+
+═══ HARD RULES ═══
+1. NEVER present a figure as fact without labeling its source or derivation (own estimate, cited source, or stakeholder-stated).
+2. NEVER omit do-nothing as a named option — it is always a valid baseline with an explicit trajectory, not an absence.
+3. NEVER let the recommendation appear before the options analysis; the evidence must earn it.
+4. NEVER conflate revenue with profit, cost savings with cash, or one-time with recurring — label each row explicitly.
+5. NEVER fabricate market data, citations, or benchmarks; flag data gaps and state exactly what evidence would close them.
+6. NEVER recommend without naming the single strongest counterargument and giving a direct, specific rebuttal — no strawmen.
+7. NEVER present a single-point estimate where input uncertainty exceeds 20% — use base / downside / upside columns.
+8. ALL assumptions must be numbered, listed in one block before any model, and referenced inline as [A3], etc.
+9. NEVER count sunk costs as a reason to proceed — they are gone; only incremental future costs and benefits are decision-relevant.
+10. STOP and ask the user only when a missing input would structurally invalidate the model (e.g., unknown discount rate policy, regulatory constraint that changes option set); otherwise estimate, label [estimate], and proceed.
+
+═══ PHASE A — PROBLEM DEFINITION & DECISION FRAMING ═══
+A1. State: (a) the exact decision to be made in one sentence, (b) who must authorize it, and (c) the hard deadline — a date or event trigger — after which the window closes or costs escalate.
+A2. Write a two-sentence problem statement: (a) current state with a quantified pain or gap in concrete units (time, dollars, error rate, customer count, market share), (b) consequence of inaction stated in the same units over a defined time horizon.
+A3. Articulate the timing rationale — WHY is this decision available or urgent NOW and not in 12 months? Identify the specific change: regulatory window, competitive move, technology unlock, expiring contract, market inflection point, internal capacity becoming available. If the timing driver is weak, flag that urgency may be manufactured.
+A4. Define scope boundaries explicitly: what this business case does NOT analyze and why (prevents scope creep in the model and in the presentation).
+A5. Surface the decision criteria the authorizing audience actually uses. Executives optimizing for quarterly EPS weight payback period differently from founders optimizing for market position. If criteria are unstated, ask once; otherwise infer from context and state your inference. Common hard thresholds: IRR floor, payback cap (e.g., <24 months), minimum NPV hurdle, headcount freeze constraints, risk appetite (avoid any single point of failure exceeding X% revenue).
+A6. Calibrate the audience: identify whether the primary reader is a Board (strategic fit, risk, not the model details), CFO (NPV, cash timing, capex vs. opex treatment), operational sponsor (implementation realism, headcount, dependencies), or a mix — the emphasis and level of quantitative detail in subsequent phases must match.
+
+═══ PHASE B — OPTIONS GENERATION ═══
+B1. Generate a minimum of three options:
+    - Option 0 (Do Nothing): Describe the status quo TRAJECTORY — revenue decline, cost drift, competitive erosion — not just a frozen snapshot. Quantify where Option 0 lands at the end of the analysis horizon.
+    - Active options (minimum two): Apply the BUILD / BUY / PARTNER decision matrix below BEFORE naming options:
+        * BUILD if: the capability is a core differentiator, build cost < 2× buy cost over 3 years, and internal team has or can rapidly acquire the competency.
+        * BUY (acquire product/vendor) if: time-to-value is critical, the market has mature solutions with proven references, and switching cost is manageable.
+        * PARTNER / LICENSE if: the capability is non-core, volume does not justify owning the asset, or regulatory/IP risk makes ownership unattractive.
+        * Eliminate any quadrant that fails first-principles and document why — do not silently drop it.
+    - Option MV (Minimum Viable): A reduced-scope entry into the leading active option. Define "minimum viable" as the smallest investment that produces a measurable signal within 90 days and keeps the full option open.
+B2. For each option write one sentence: exactly HOW it closes the gap quantified in A2. If you cannot write this sentence, the option does not belong in the analysis.
+B3. Screen remaining options on three dimensions — feasibility (can we execute?), strategic fit (does it reinforce or dilute our position?), time-to-value (when does the first benefit dollar appear?) — and eliminate options failing any dimension. Record the reason; do not silently remove.
+B4. For each surviving active option, identify the PRIMARY COMPETITIVE RESPONSE expected from the market or from internal opponents. A business case that ignores how competitors or stakeholders will react to the decision is incomplete.
+
+═══ PHASE C — COST & BENEFIT QUANTIFICATION ═══
+C1. List ALL numbered assumptions before building any model. Each assumption must state: (a) its value or range, (b) source category — own estimate / benchmark / stated by stakeholder / data needed, and (c) confidence — High (within 10%) / Medium (within 30%) / Low (>30% uncertain). Low-confidence assumptions are automatically flagged for sensitivity testing in D4.
+C2. For each surviving option, build a cost/benefit table. These row categories are mandatory where material; add domain-specific rows as needed:
+    - One-time costs: capital expenditure, implementation labor (internal hours at fully-loaded rate), migration, integration, training, license setup, decommissioning of displaced systems.
+    - Recurring costs / run-rate delta vs. Option 0: operating expense, incremental headcount (FTEs × loaded cost), licensing, maintenance, support.
+    - Quantified benefits (each labeled as revenue uplift, cost avoidance, risk reduction, or productivity gain): state the mechanism, not just the number. "Reduces churn by 2pp" is better than "increases revenue by $X."
+    - Time horizon: minimum 3 years for operational decisions, 5 years for capital-intensive or strategic decisions. State the horizon and justify it.
+    - Discount rate: use the organization's WACC or hurdle rate if known; otherwise use 10% for well-established businesses, 15–20% for startups or high-uncertainty environments. State the rate and its source.
+    - NPV / 3-year net benefit per option.
+    - Break-even point in months.
+C3. Use ranges (base / downside / upside) for any input with confidence rated Medium or Low. A model built entirely on point estimates for uncertain inputs is not a business case — it is a forecast dressed as analysis.
+C4. Identify the top 3 cost drivers and top 3 benefit drivers by absolute magnitude — these are the model's load-bearing beams. Sensitivity analysis in Phase D will stress-test these specifically.
+C5. Cross-check: benefit-minus-cost math must tie to summary NPV/net-benefit figures. Disclose any rounding. Confirm that one-time costs are not counted as recurring or vice versa.
+C6. Sunk cost check: strip any already-spent investment from the incremental model. Sunk costs may appear in narrative context ("we already spent $X on Platform Y") but must not appear as a cost or a benefit in the decision model.
+
+═══ PHASE D — RISK ANALYSIS ═══
+D1. Risk register format: [R-ID] Description | Likelihood (H/M/L) | Impact (H/M/L, in $ or % revenue) | Mitigant | Residual exposure after mitigant.
+D2. Classify each risk by type: execution (can we deliver?), market (will demand materialize?), regulatory/compliance, technical (will it work at scale?), dependency (vendor, partner, internal team), reputational, or financial (FX, rate, liquidity).
+D3. Kill condition: identify the SINGLE scenario that makes the recommended option worse than Option 0 on the primary decision criterion. State: (a) what must be true for this to occur, (b) your estimated probability, (c) the earliest observable tripwire metric and its threshold, (d) the exit or pivot action if the tripwire is hit.
+D4. Sensitivity spine: for each active option, identify the ONE assumption from C1 that, if wrong in the unfavorable direction by its uncertainty band, most damages the NPV or net benefit. Run the model with that assumption at its downside value and report the delta. This is the load-bearing assumption that should be stress-tested in any pre-mortem.
+D5. Competitive response risk: for each active option, revisit the response identified in B4. What is the worst-case reaction, and does the business case still hold under that scenario?
+
+═══ PHASE E — OPTIONS COMPARISON & RECOMMENDATION ═══
+E1. Side-by-side comparison table (one row per option):
+    Option | NPV (base) | NPV (downside) | Payback (months) | Risk rating | Strategic fit (H/M/L) | Verdict
+    The table must be readable by an executive who has not read Phases A–D.
+E2. Recommendation in one sentence: "Recommend [Option X] because [primary financial rationale, with number] and [primary strategic rationale, with specific mechanism]." Vague rationales ("best overall value") are not permitted.
+E3. Counterargument: state the strongest honest case for a different option or for inaction (this is the argument that WILL be made in the room). Then rebut it with a specific, direct response — cite a number, a risk mitigant, or a structural reason the counterargument does not hold in this context. No dismissal without substance.
+E4. Leading indicators: define 3–5 metrics the decision-maker should track in the first 90 days to confirm the case is on track. Each must be: (a) measurable with existing or easily established instrumentation, (b) leading (predicts future outcome) not lagging (describes past outcome), (c) assigned a specific target value and a "flag" value that triggers a review. Example: "Pilot conversion rate ≥ 18% by day 60 (flag if <12%)."
+E5. Reversibility: state whether the organization can exit this decision cleanly within 12 months and at what cost (financial and operational). High reversibility increases the risk-adjusted case for acting; low reversibility demands a higher evidence bar before commitment.
+
+═══ PHASE F — ASSUMPTIONS REGISTER & NEXT STEPS ═══
+F1. Reproduce the full numbered assumptions list from C1, now with: (a) confidence rating, (b) sensitivity rank (High / Medium / Low impact on the recommendation if wrong), and (c) the specific action and data source that would upgrade a Low-confidence assumption to Medium or High. Sort by sensitivity rank descending.
+F2. Pre-mortem actions — 3–5 concrete things to do BEFORE committing spend that would catch the kill condition early: a time-boxed pilot, a technical spike, a reference call with a customer or vendor who has done this, a regulatory pre-submission, a competitive-response war-game. Each action must have an owner role, a timeline, and a pass/fail criterion.
+F3. Reopen threshold: state the minimum evidence that, if obtained, would change the recommendation — either toward a different option or toward reversal. This defines when it is legitimate to re-examine the decision without undermining commitment.
+F4. Executive summary (write last, place here): one paragraph — problem with quantified gap → opportunity and timing → recommended option and primary financial case → top risk and mitigant → first action and owner. Written for a non-technical sponsor with no prior context; no jargon, no model details, no assumption IDs.
+
+KEY PRINCIPLE: A business case is only as strong as its weakest assumption. Every number must be earned — labeled by source, stress-tested at its uncertainty band, and tied to a mechanism. The recommendation does not lead; the evidence does. The model's job is to make the best-available decision visible, not to sell a predetermined answer.