swe-workflow-skills 0.1.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (74) hide show
  1. package/README.md +58 -17
  2. package/VERSION +1 -1
  3. package/catalog.json +113 -8
  4. package/commands/role.md +14 -2
  5. package/install.mjs +23 -3
  6. package/package.json +2 -2
  7. package/roles.json +72 -7
  8. package/scripts/resolve.mjs +8 -3
  9. package/skills/ai-evaluation/SKILL.md +112 -0
  10. package/skills/ai-evaluation/evals/evals.json +40 -0
  11. package/skills/ai-evaluation/references/eval-tooling.md +63 -0
  12. package/skills/architecture-design/SKILL.md +3 -1
  13. package/skills/brainstorming/SKILL.md +92 -0
  14. package/skills/brainstorming/evals/evals.json +40 -0
  15. package/skills/browser-verification/SKILL.md +95 -0
  16. package/skills/browser-verification/evals/evals.json +40 -0
  17. package/skills/build-vs-buy/SKILL.md +109 -0
  18. package/skills/build-vs-buy/evals/evals.json +40 -0
  19. package/skills/cicd-pipeline/SKILL.md +1 -0
  20. package/skills/code-archaeology/SKILL.md +103 -0
  21. package/skills/code-archaeology/evals/evals.json +41 -0
  22. package/skills/code-reviewing/SKILL.md +13 -3
  23. package/skills/compliance-privacy/SKILL.md +113 -0
  24. package/skills/compliance-privacy/evals/evals.json +41 -0
  25. package/skills/compliance-privacy/references/obligations-map.md +69 -0
  26. package/skills/data-pipeline-design/SKILL.md +116 -0
  27. package/skills/data-pipeline-design/evals/evals.json +41 -0
  28. package/skills/data-pipeline-design/references/orchestration.md +64 -0
  29. package/skills/data-quality/SKILL.md +107 -0
  30. package/skills/data-quality/evals/evals.json +39 -0
  31. package/skills/data-quality/references/tooling.md +45 -0
  32. package/skills/deployment-checklist/SKILL.md +2 -1
  33. package/skills/dx-audit/SKILL.md +97 -0
  34. package/skills/dx-audit/evals/evals.json +40 -0
  35. package/skills/effort-estimation/SKILL.md +24 -67
  36. package/skills/exploratory-data-analysis/SKILL.md +110 -0
  37. package/skills/exploratory-data-analysis/evals/evals.json +43 -0
  38. package/skills/feature-planning/SKILL.md +5 -1
  39. package/skills/finops-cost-optimization/SKILL.md +109 -0
  40. package/skills/finops-cost-optimization/evals/evals.json +40 -0
  41. package/skills/git-workflow/SKILL.md +31 -14
  42. package/skills/llm-app-engineering/SKILL.md +102 -0
  43. package/skills/llm-app-engineering/evals/evals.json +40 -0
  44. package/skills/llm-app-engineering/references/patterns.md +70 -0
  45. package/skills/ml-experiment-tracking/SKILL.md +1 -1
  46. package/skills/ml-pipeline-design/SKILL.md +8 -2
  47. package/skills/ml-pipeline-design/evals/evals.json +10 -0
  48. package/skills/mobile-architecture/SKILL.md +106 -0
  49. package/skills/mobile-architecture/evals/evals.json +40 -0
  50. package/skills/mobile-release/SKILL.md +109 -0
  51. package/skills/mobile-release/evals/evals.json +41 -0
  52. package/skills/notebook-to-production/SKILL.md +89 -0
  53. package/skills/notebook-to-production/evals/evals.json +42 -0
  54. package/skills/plan-execution/SKILL.md +115 -0
  55. package/skills/plan-execution/evals/evals.json +55 -0
  56. package/skills/project-documentation/SKILL.md +8 -1
  57. package/skills/project-review/SKILL.md +13 -3
  58. package/skills/release-management/SKILL.md +97 -0
  59. package/skills/release-management/evals/evals.json +56 -0
  60. package/skills/release-management/references/release-tooling.md +106 -0
  61. package/skills/resilience-engineering/SKILL.md +121 -0
  62. package/skills/resilience-engineering/evals/evals.json +40 -0
  63. package/skills/security-audit/SKILL.md +14 -3
  64. package/skills/skill-router/SKILL.md +57 -4
  65. package/skills/statistical-analysis/SKILL.md +93 -0
  66. package/skills/statistical-analysis/evals/evals.json +43 -0
  67. package/skills/strategic-review/SKILL.md +16 -2
  68. package/skills/subagent-orchestration/SKILL.md +100 -0
  69. package/skills/subagent-orchestration/evals/evals.json +40 -0
  70. package/skills/technical-debt-review/SKILL.md +12 -2
  71. package/skills/threat-modeling/SKILL.md +101 -0
  72. package/skills/threat-modeling/evals/evals.json +40 -0
  73. package/skills/threat-modeling/references/stride-mitigations.md +90 -0
  74. package/skills/writing-skills/SKILL.md +48 -16
@@ -0,0 +1,40 @@
1
+ {
2
+ "skill_name": "resilience-engineering",
3
+ "evals": [
4
+ {
5
+ "id": 1,
6
+ "prompt": "Our e-commerce platform runs on one Postgres instance, a Node API, and Redis, all in a single AWS availability zone. Leadership asked: 'what happens if AWS has a bad day?' Design our resilience strategy.",
7
+ "expected_output": "Runs failure-mode analysis per dependency (down/slow/partial/data-corruption, single points of failure), sets explicit targets first (availability tier, RTO/RPO per store) noting cost scales with the numbers, applies stability patterns at boundaries (timeouts, retries with backoff+jitter on idempotent ops, circuit breakers, graceful degradation deciding in advance what turns off), designs data recovery (automated backups, isolated copy, scheduled restore tests), and proposes verification via small chaos experiments and a game day rather than declaring the design done",
8
+ "assertions": [
9
+ "Performs failure-mode analysis across the dependencies, covering slow/partial failures and data corruption — not just total outage — and identifies single points of failure",
10
+ "Sets explicit RTO/RPO and availability targets before choosing mechanisms, noting that cost scales with the targets (backups vs standby vs multi-AZ/region)",
11
+ "Applies concrete stability patterns: timeouts, retries with backoff and jitter (idempotent only), circuit breakers, and pre-decided graceful degradation",
12
+ "Treats backups as unproven until restored — schedules restore testing and keeps a copy isolated from the blast radius",
13
+ "Proposes verifying with chaos experiments (hypothesis-driven, small blast radius) and/or a game day rehearsing the runbooks"
14
+ ]
15
+ },
16
+ {
17
+ "id": 2,
18
+ "prompt": "We already have retries everywhere — every service retries failed calls 5 times. Last month a 30-second blip in our auth service somehow became a 45-minute full-platform outage. Make us more resilient.",
19
+ "expected_output": "Diagnoses the retry storm / cascade: unbudgeted retries amplified load (5 retries per hop compound multiplicatively across the chain), likely no timeouts or breakers so threads piled onto a hung dependency; prescribes retry budgets, exponential backoff with jitter, idempotency checks, circuit breakers to fail fast, load shedding at the edge, and bulkheads to contain blast radius — showing that MORE retrying is the opposite of resilience here",
20
+ "assertions": [
21
+ "Identifies the naive retries as an amplifier (retry storm / cascading failure), including how retries compound across call-chain hops",
22
+ "Recommends retry budgets and exponential backoff with jitter, restricted to idempotent operations, instead of fixed retry counts",
23
+ "Adds circuit breakers and timeouts so a hung or flapping dependency fails fast instead of tying up resources",
24
+ "Recommends load shedding/backpressure and/or bulkheads to contain the blast radius of a degraded dependency",
25
+ "Frames the fix as controlling and bounding retries — explicitly countering the assumption that more retrying equals more resilience"
26
+ ]
27
+ },
28
+ {
29
+ "id": 3,
30
+ "prompt": "Production is down RIGHT NOW — the primary database failed over but the app servers are still erroring and customers can't check out. What do we do?",
31
+ "expected_output": "Recognizes an active incident under time pressure — hands off to incident-response for triage/mitigation/comms rather than starting a peacetime resilience design; may note that post-recovery, the failure feeds back into failure-mode analysis and a game-day-style review via retrospective",
32
+ "assertions": [
33
+ "Recognizes this as an active production incident, not a resilience design exercise",
34
+ "Refers to the incident-response skill/workflow for triage, mitigation, and communication",
35
+ "Does not respond with a peacetime workflow (failure-mode analysis, chaos experiments, DR design) as the immediate answer",
36
+ "States or implies the boundary: resilience-engineering is peacetime design and rehearsal; incident-response handles the live fire (with learnings looping back afterward)"
37
+ ]
38
+ }
39
+ ]
40
+ }
@@ -1,7 +1,10 @@
1
1
  ---
2
2
  name: security-audit
3
- description: "Comprehensive security analysis — OWASP Top 10, auth/authz flows, injection vulnerabilities, data exposure, secrets detection, dependency CVEs, hardening recommendations. Triggers: security audit, vulnerability, is this secure, security review, pentest prep, OWASP, harden this, check for vulnerabilities, injection, XSS, CSRF, auth security."
3
+ description: "Comprehensive security analysis — OWASP Top 10, auth/authz flows, injection vulnerabilities, data exposure, secrets detection, dependency CVEs, hardening recommendations. Reviews EXISTING code/config design-time analysis of a system not yet built threat-modeling."
4
+ when_to_use: "Triggers: security audit, vulnerability, is this secure, security review, pentest prep, OWASP, harden this, check for vulnerabilities, injection, XSS, CSRF, auth security."
4
5
  model: opus
6
+ context: fork
7
+ agent: general-purpose
5
8
  allowed-tools: Read, Grep, Glob, Write, Edit, WebFetch, WebSearch
6
9
  ---
7
10
 
@@ -11,7 +14,7 @@ Perform a structured security assessment of an application. This skill goes far
11
14
 
12
15
  ## Scope Boundary
13
16
 
14
- This skill analyzes code, configuration, and architecture for security vulnerabilities through static analysis and design review. It does not replace penetration testing, but it prepares code for one by catching the issues a pentester would find.
17
+ This skill analyzes code, configuration, and architecture for security vulnerabilities through static analysis and design review. It does not replace penetration testing, but it prepares code for one by catching the issues a pentester would find. It is the *reactive* half of the pair: for design-time analysis of a system that isn't built yet (trust boundaries, STRIDE, abuse cases), use `threat-modeling` — and when a threat model exists, use it as this audit's checklist.
15
18
 
16
19
  ## ⛔ The Iron Law
17
20
 
@@ -31,7 +34,7 @@ Before reviewing code line by line, map what's exposed:
31
34
  - **External integrations**: Third-party APIs, OAuth providers, payment processors
32
35
  - **Infrastructure**: Cloud services, databases, caches, queues
33
36
 
34
- Present the attack surface map to the user. Gaps in understanding here mean gaps in the audit.
37
+ Record the attack surface map as the first section of the report — gaps in understanding here mean gaps in the audit, so anything you could not map (unreachable config, undocumented integrations) goes under Open questions rather than being assumed safe.
35
38
 
36
39
  ### Step 2: Assess by OWASP Top 10
37
40
 
@@ -113,6 +116,14 @@ Suggest using the `dependency-management` skill for remediation planning.
113
116
 
114
117
  Output the audit report using the template at [templates/security-report.md](templates/security-report.md). Organize findings by severity, with Critical and High items first.
115
118
 
119
+ **Write the full report to a file** (default a gitignored location, e.g.
120
+ `.local/security-audit-<date>.md`) and state its path in your final summary — this
121
+ skill runs in a forked context: only the summary returns, everything unwritten is
122
+ lost. The summary must lead with the Critical/High count. Judgment calls that need
123
+ user input (unclear trust boundary, whether a data store is in scope, unverifiable
124
+ control) go in an **Open questions** section of the report — never silently assumed
125
+ safe.
126
+
116
127
  ## What This Skill Does NOT Cover
117
128
 
118
129
  - Runtime penetration testing (requires running the application)
@@ -1,6 +1,7 @@
1
1
  ---
2
2
  name: skill-router
3
- description: "Orchestrator and entry point for the swe-workflow skills library — consult FIRST when starting any non-trivial software task; most skills load name-only and only activate when invoked here. Triggers: starting a feature, planning, an architecture or design decision, implementing, debugging, reviewing, refactoring, testing, security, deployment, an incident, shipping, or unsure which skill fits. Routes intent to the right skill(s) and invokes them by name; shows the Golden Path chains."
3
+ description: "Orchestrator and entry point for the swe-workflow skills library — consult FIRST when starting any non-trivial software task; most skills load name-only and only activate when invoked here. Routes intent to the right skill(s) and invokes them by name; shows the Golden Path chains."
4
+ when_to_use: "Triggers: starting a feature, planning, an architecture or design decision, implementing, debugging, reviewing, refactoring, testing, security, deployment, an incident, shipping, or unsure which skill fits."
4
5
  model: haiku
5
6
  allowed-tools: Read, Grep, Glob, Skill
6
7
  ---
@@ -87,7 +88,9 @@ installed skill by name, regardless of role.
87
88
  ## Catalog by SDLC phase
88
89
 
89
90
  ### Plan & Define
91
+ - **brainstorming** — divergent Socratic ideation *before* a spec exists; opens options, then hands off
90
92
  - **feature-planning** — break a feature into scoped tasks, acceptance criteria, dependencies
93
+ - **build-vs-buy** — build in-house vs vendor/SaaS vs adopt OSS: TCO, lock-in, exit costs
91
94
  - **prd-writing** — write a PRD / RFC / tech spec to align on the WHAT and WHY
92
95
  - **project-proposal** — business case / budget / go-no-go before a project starts
93
96
  - **strategic-review** — vision, positioning, defensible wedge, live competitive/market analysis before going public
@@ -102,14 +105,19 @@ installed skill by name, regardless of role.
102
105
  - **ui-ux-design** — user flows, wireframes, loading/error/empty states, responsive
103
106
  - **frontend-architecture** — component hierarchy, state management, design tokens
104
107
  - **accessibility-design** — WCAG, ARIA, keyboard nav, focus, screen readers
108
+ - **threat-modeling** — design-time security: trust boundaries, STRIDE, abuse cases, mitigations
109
+ - **compliance-privacy** — GDPR/SOC 2 obligations: data mapping, retention/deletion, DSRs, controls
105
110
  - **configuration-strategy** — env config, secrets management, feature-flag hierarchy
106
111
  - **dependency-impact-analysis** — blast radius of a change *before* implementing
107
112
  - **dependency-management** — evaluate, audit (CVEs), and upgrade libraries
108
113
 
109
114
  ### Build & Test
115
+ - **plan-execution** — execute an APPROVED plan: checkpoints, verification evidence, drift → re-plan
110
116
  - **tdd-workflow** — NEW code, test-first, red-green-refactor
111
117
  - **test-suite-design** — add tests to EXISTING code, coverage strategy
112
118
  - **test-data-strategy** — factories, synthetic data, property-based, contract testing
119
+ - **browser-verification** — drive the real UI to prove a web change works: console/network/state evidence
120
+ - **subagent-orchestration** — fan work out across subagents/worktrees; verify and synthesize results
113
121
  - **git-workflow** — commit messages, PR descriptions, branching strategy
114
122
  - **project-documentation** — README, contributing guide, changelog, docstrings
115
123
 
@@ -120,6 +128,8 @@ installed skill by name, regardless of role.
120
128
  - **refactoring** — systematic, test-protected code improvement
121
129
  - **technical-debt-review** — codebase health, hotspots, remediation roadmap
122
130
  - **project-review** — whole-project execution health: scope alignment, roadmap adherence, implementation maturity, evidence-it-works
131
+ - **code-archaeology** — understand unfamiliar/legacy code first: history mining, tracing, characterization tests
132
+ - **dx-audit** — developer-experience audit: feedback loops, CI wait, flakes, onboarding, tooling friction
123
133
 
124
134
  ### Diagnose & Fix
125
135
  - **bug-investigating** — systematic debugging, reproduce → isolate → hypothesize → verify
@@ -129,27 +139,50 @@ installed skill by name, regardless of role.
129
139
  - **rollback-strategy** — safe rollback plans, irreversible-change detection
130
140
  - **containerization** — Dockerfiles, docker-compose, Kubernetes manifests
131
141
  - **cicd-pipeline** — CI/CD pipelines, quality gates (GitHub Actions, GitLab CI)
142
+ - **release-management** — semver, changelog, tagging, publish gates, registry publishing, release automation (right-sized to the project)
132
143
  - **infrastructure-as-code** — Terraform, CloudFormation, Pulumi, CDK
133
144
  - **deployment-repo** — GitOps polyrepo orchestration, version pinning, promotion
134
145
  - **gitops-delivery** — ArgoCD / Flux declarative delivery, drift detection
135
146
  - **observability-design** — SLI/SLO/SLA, OpenTelemetry, structured logging, alerting
147
+ - **resilience-engineering** — failure modes, stability patterns, chaos experiments, DR/RTO/RPO, game days
148
+ - **finops-cost-optimization** — cloud spend: allocation, unit economics, rightsizing, commitments, egress
136
149
  - **incident-response** — ACTIVE production incident: triage, mitigate, communicate
137
150
 
151
+ ### Mobile
152
+ - **mobile-architecture** — platform choice, navigation, offline-first/sync, state, push, fleet constraints
153
+ - **mobile-release** — signing, store review, staged rollouts, no-rollback playbook, beta channels
154
+
138
155
  ### Reflect
139
156
  - **retrospective** — sprint retros, project / incident post-mortems, action items
140
157
 
141
158
  ### MLOps
142
- - **ml-pipeline-design** — training & data pipelines, feature engineering, orchestration
159
+ - **ml-pipeline-design** — ML training & feature pipelines, point-in-time correctness, orchestration
143
160
  - **ml-experiment-tracking** — MLflow / W&B / DVC, run comparison, reproducibility
144
161
  - **ml-model-deployment** — serving, monitoring, drift detection, safe rollouts
145
162
 
163
+ ### AI / LLM Apps
164
+ - **llm-app-engineering** — prompt/context engineering, RAG architecture, agent design, memory
165
+ - **ai-evaluation** — golden datasets, RAG metrics, LLM-as-judge design, eval gates in CI, online A/B
166
+
167
+ ### Data Engineering
168
+ - **data-pipeline-design** — batch/streaming ELT, dbt layering, Airflow/Dagster, idempotency, backfill
169
+ - **data-quality** — dbt tests, expectations, data contracts, freshness, lineage / blast radius
170
+
171
+ ### Data Science
172
+ - **exploratory-data-analysis** — profile an unfamiliar dataset: missingness structure, distributions, leakage, hypotheses
173
+ - **statistical-analysis** — hypothesis tests, experiment design/power, confidence intervals, multiple-comparison discipline
174
+ - **notebook-to-production** — refactor analysis notebooks to tested, parameterized, scheduled production code
175
+
146
176
  ## Golden Path workflow chains
147
177
 
148
178
  When work spans phases, chain skills rather than improvising:
149
179
 
150
180
  **New feature**
151
- `feature-planning` → `architecture-design` (if structural) → `data-modeling`
152
- (if schema) → `tdd-workflow` `code-reviewing` `deployment-checklist`
181
+ `brainstorming` (if the idea is still fuzzy) → `feature-planning`
182
+ `architecture-design` (if structural) → `data-modeling` (if schema)
183
+ `threat-modeling` (if new trust boundaries) → `plan-execution` (work the
184
+ approved plan; wraps `tdd-workflow` per task) → `code-reviewing` →
185
+ `deployment-checklist`
153
186
 
154
187
  **Bug / incident**
155
188
  `incident-response` (if prod is down) → `bug-investigating` → `tdd-workflow`
@@ -159,6 +192,26 @@ When work spans phases, chain skills rather than improvising:
159
192
  `technical-debt-review` → `refactoring` → `dependency-impact-analysis`
160
193
  (blast radius) → `test-suite-design` (if coverage is thin)
161
194
 
195
+ **Ship a release**
196
+ `git-workflow` (commits carry the bump intent) → `release-management` (version,
197
+ changelog, tag, publish gate, registry) → `deployment-checklist` (if it deploys) →
198
+ `rollback-strategy`. `cicd-pipeline` automates the gate as a pipeline stage.
199
+ App-store releases (signing, review, staged rollout) → `mobile-release`.
200
+
201
+ **LLM feature**
202
+ `feature-planning` → `llm-app-engineering` (prompt/RAG/agent design) →
203
+ `ai-evaluation` (golden set, judge, CI gate) → `deployment-checklist`.
204
+ `ml-model-deployment` joins when serving your own model.
205
+
206
+ **Analytics / data pipeline**
207
+ `data-pipeline-design` (ELT, layering, orchestration) → `data-modeling`
208
+ (marts schema) → `data-quality` (tests, contracts, freshness) →
209
+ `observability-design` (pipeline SLOs, alerting).
210
+ Notebook to production: `exploratory-data-analysis` (understand the data) →
211
+ `statistical-analysis` (test the hypotheses) → `notebook-to-production`
212
+ (modules, tests, scheduling); if the notebook trains a model →
213
+ `ml-pipeline-design` instead.
214
+
162
215
  **Pre-public / pre-milestone review**
163
216
  `strategic-review` (vision, positioning, market) → `project-review` (scope,
164
217
  roadmap, implementation, evidence) → synthesis → `artifact-design` (interactive
@@ -0,0 +1,93 @@
1
+ ---
2
+ name: statistical-analysis
3
+ description: "Design and analyze experiments and statistical tests — test selection with stated assumptions, sample size and power, effect sizes and confidence intervals over bare p-values, and pitfall discipline (multiple comparisons, p-hacking, peeking/optional stopping). Owns experiment statistics generally; live A/B evaluation of AI/LLM apps (quality metrics, judges, feedback) → ai-evaluation."
4
+ when_to_use: "Triggers: hypothesis test, t-test, chi-square, p-value, statistical significance, confidence interval, sample size, power analysis, design an experiment, A/B test, multiple comparisons, is this difference real."
5
+ model: opus
6
+ allowed-tools: Read, Grep, Glob, Write, Edit, Bash
7
+ ---
8
+
9
+ # Statistical Analysis
10
+
11
+ Get the statistics of a question right — the test, its assumptions, the
12
+ uncertainty, and the traps — so a decision rests on evidence rather than on a
13
+ lone p-value. The deliverable is a decision-ready answer: effect size,
14
+ uncertainty, assumptions, caveats.
15
+
16
+ ## Workflow
17
+
18
+ ### 1. Frame the question as a hypothesis
19
+
20
+ State the null and alternative in plain words before touching data ("the new
21
+ flow's conversion equals the old flow's"). Identify the unit of analysis
22
+ (user? session? order?) — mismatching it with the randomization unit silently
23
+ breaks independence.
24
+
25
+ ### 2. Design before data (when the experiment hasn't run yet)
26
+
27
+ - Choose the primary metric and **pre-register it** — one primary, the rest
28
+ explicitly secondary/exploratory.
29
+ - Compute **sample size from power**: minimum detectable effect worth acting
30
+ on, α (typically 0.05), power (typically 0.8). Run until n is reached —
31
+ don't stop early on a good-looking result.
32
+ - Randomize at the unit you'll analyze; check the split is balanced on key
33
+ covariates.
34
+
35
+ ### 3. Choose the test and state its assumptions
36
+
37
+ Match the test to the data (two proportions → two-proportion z-test / χ²;
38
+ means → t-test or its nonparametric fallback; counts → Poisson/χ²; paired data
39
+ → paired tests). **Always state the assumptions and check the ones you can**:
40
+ independence of observations (the usual casualty — repeated measures per user,
41
+ network effects, shared fixtures), adequate counts for normal approximations
42
+ (rule of thumb: ≥10 successes and failures per arm), variance assumptions for
43
+ t-tests. If an assumption fails, switch tests rather than pretending.
44
+
45
+ ### 4. Report effect size and interval, not a verdict
46
+
47
+ Work the actual numbers. Report the effect (absolute and relative difference)
48
+ with a **confidence interval**, and the p-value as supporting detail — never a
49
+ bare "significant/not significant". The CI is what the decision needs: a CI of
50
+ [+0.1%, +1.7%] and a CI of [+0.8%, +1.0%] are both "significant" and mean very
51
+ different things.
52
+
53
+ ### 5. Run the pitfall gate
54
+
55
+ Before reporting anything, check:
56
+
57
+ - **Peeking / optional stopping**: was n fixed in advance? Repeatedly testing
58
+ as data arrives inflates false positives badly. If peeking happened, say so
59
+ and treat the result as suggestive (or use sequential methods designed for it).
60
+ - **Multiple comparisons**: k tests at α=0.05 ≈ k/20 false positives expected.
61
+ Correct (Holm/Bonferroni for strict control, Benjamini-Hochberg FDR when many
62
+ metrics), and label post-hoc findings as exploratory — they are hypotheses
63
+ for the *next* experiment, not wins from this one.
64
+ - **Subgroup fishing**: subgroup effects found after the fact need the same
65
+ exploratory label and a confirmatory follow-up.
66
+ - **Survivorship / selection**: does the analyzed sample still represent the
67
+ population the decision is about?
68
+
69
+ ### 6. Translate to the decision
70
+
71
+ Distinguish statistical from practical significance: a real +0.05% lift may
72
+ not pay for the complexity; a non-significant result with a wide CI is "we
73
+ don't know yet", not "no effect". Give the ship/don't-ship framing with the
74
+ caveats attached, and say what additional data would resolve remaining
75
+ uncertainty.
76
+
77
+ ## Principles
78
+
79
+ - **KISS**: a well-checked z-test beats a fancy model with unexamined
80
+ assumptions.
81
+ - **Pre-register or label**: every metric is either declared up front or
82
+ reported as exploratory — no third category.
83
+ - **Uncertainty is the product**: any answer without an interval is opinion
84
+ with extra steps.
85
+
86
+ ## Cross-skill boundaries
87
+
88
+ - Live A/B testing of an AI/LLM app — defining answer quality, LLM-as-judge,
89
+ human feedback loops, eval gates → **ai-evaluation** (this skill still owns
90
+ the underlying experiment statistics if asked).
91
+ - Generating the hypotheses to test → **exploratory-data-analysis**.
92
+ - Tracking and comparing model-training runs → **ml-experiment-tracking**.
93
+ - Defining what business metric to move → **metrics-and-okrs**.
@@ -0,0 +1,43 @@
1
+ {
2
+ "skill_name": "statistical-analysis",
3
+ "evals": [
4
+ {
5
+ "id": 1,
6
+ "prompt": "We ran an experiment: 2,400 users saw the new checkout flow and converted at 6.8%; 2,350 saw the old one and converted at 5.9%. My PM wants to ship it today — is this difference real? How confident can we be?",
7
+ "expected_output": "Should treat this as a two-proportion hypothesis test: state the null, run or outline the test with the actual numbers, report a confidence interval on the difference, ask whether the sample size was fixed in advance (peeking), and separate statistical from practical significance before giving a ship recommendation.",
8
+ "assertions": [
9
+ "Frames the question as a hypothesis test (two-proportion z-test or equivalent) and states the null hypothesis",
10
+ "Works the actual numbers given (or sets up the exact computation) rather than answering qualitatively",
11
+ "Reports uncertainty as a confidence interval on the difference, not only a p-value or a yes/no verdict",
12
+ "Asks or checks whether the sample size was fixed in advance and flags the peeking/optional-stopping problem",
13
+ "Distinguishes statistical significance from practical/business significance",
14
+ "States the assumptions behind the chosen test (independent samples, adequate counts)",
15
+ "Gives a clear ship/don't-ship framing with the statistical caveats attached"
16
+ ]
17
+ },
18
+ {
19
+ "id": 2,
20
+ "prompt": "We measured 20 different metrics in our experiment and 2 came out significant at p < 0.05. Which ones should we report as wins?",
21
+ "expected_output": "Should identify the multiple-comparisons problem (about one false positive is expected by chance at that rate), recommend a correction (Holm/Bonferroni or FDR), distinguish pre-registered primary metrics from exploratory ones, and treat the 2 hits as hypotheses for a confirmatory follow-up — not report them as wins.",
22
+ "assertions": [
23
+ "Identifies the multiple-comparisons problem explicitly — with 20 tests, roughly one false positive at p < 0.05 is expected by chance",
24
+ "Recommends a concrete correction (Bonferroni, Holm, or FDR / Benjamini-Hochberg) and notes the trade-off between them",
25
+ "Warns against reporting the 2 nominal hits as wins (p-hacking / cherry-picking)",
26
+ "Distinguishes pre-registered primary metrics from exploratory findings",
27
+ "Recommends treating the surviving hits as hypotheses for a follow-up confirmatory experiment",
28
+ "Does not simply declare the two significant metrics real effects"
29
+ ]
30
+ },
31
+ {
32
+ "id": 3,
33
+ "prompt": "We're about to launch a new system prompt for our support chatbot and want to A/B test it live against the current one to see which gives better answers. Can you set this up?",
34
+ "expected_output": "Should recognize this as online A/B evaluation of an LLM app and hand off to the ai-evaluation skill for the evaluation design (answer-quality metrics, LLM-as-judge or human feedback, eval gates); statistical-analysis contributes at most the underlying experiment statistics (sample size, significance), not the AI evaluation design.",
35
+ "assertions": [
36
+ "Recognizes this as AI-app evaluation territory and hands off to or invokes the ai-evaluation skill",
37
+ "Raises AI-specific measurement concerns (defining answer quality, LLM-as-judge or human feedback signals) rather than treating it as a pure two-sample statistics problem",
38
+ "Does not respond with only a generic A/B test design ignoring how answer quality gets measured",
39
+ "If it retains anything, retains only the experiment statistics (sample size, significance testing) as this skill's contribution"
40
+ ]
41
+ }
42
+ ]
43
+ }
@@ -1,7 +1,10 @@
1
1
  ---
2
2
  name: strategic-review
3
- description: "Review a project's strategic position before going public, launching, or raising — vision, mission, value proposition, scope positioning, the defensible wedge, and a live competitive / market comparative analysis. Triggers: strategic review, positioning, go public, go-to-market, market analysis, competitive landscape, value proposition, is there a moat, who are our competitors, platform absorption risk, market positioning, comparable products. Use project-review for execution/roadmap/implementation health; delegates deep market scans to deep-research."
3
+ description: "Review a project's strategic position before going public, launching, or raising — vision, mission, value proposition, scope positioning, the defensible wedge, and a live competitive / market comparative analysis. Use project-review for execution/roadmap/implementation health; delegates deep market scans to deep-research."
4
+ when_to_use: "Triggers: strategic review, positioning, go public, go-to-market, market analysis, competitive landscape, value proposition, is there a moat, who are our competitors, platform absorption risk, market positioning, comparable products."
4
5
  model: opus
6
+ context: fork
7
+ agent: general-purpose
5
8
  allowed-tools: Read, Grep, Glob, WebFetch, WebSearch, Write, Edit
6
9
  ---
7
10
 
@@ -60,6 +63,9 @@ Survey the field with current evidence, not memory:
60
63
  absorb the category.
61
64
  - For a deep, multi-source, fact-checked sweep, **delegate to `deep-research`** and
62
65
  fold its cited report in. For a lighter pass, use `WebSearch`/`WebFetch` directly.
66
+ (This skill runs in a forked context — if the `deep-research` skill is not
67
+ invocable from here, do the lighter pass yourself and note under Open questions
68
+ that a deep-research sweep is the recommended follow-up.)
63
69
  - **Tag each comparable** with its relationship: **competitor**, **complement**,
64
70
  or **integration target** — and what it means for this project.
65
71
  - Label every finding confirmed-vs-inferred and date it.
@@ -86,8 +92,16 @@ Integrate the above into a decision instrument:
86
92
  default is options + a recommendation.
87
93
  - **A readiness picture** mapped to the project's own gates, if it has them.
88
94
 
95
+ **Write the full review to a file** — default a gitignored path (e.g.
96
+ `.local/strategic-review-<date>.md`) — and state its path in your final summary.
97
+ This skill runs in a forked context: only the summary returns, everything unwritten
98
+ is lost. The summary leads with the thesis verdict and the top weak point.
99
+ Anything that needs the user's judgment (which strategic fork to take, an
100
+ unverifiable market assumption, missing vision docs) goes in an **Open questions**
101
+ section of the report — never silently decided.
102
+
89
103
  For the rendered deliverable (interactive HTML, scorecards, a forks comparison
90
- panel), hand off to `artifact-design`; default to a gitignored path (e.g. `.local/`).
104
+ panel), hand off to `artifact-design`.
91
105
 
92
106
  ## Principles Applied
93
107
 
@@ -0,0 +1,100 @@
1
+ ---
2
+ name: subagent-orchestration
3
+ description: "Decompose big or parallelizable work across subagents — when to fan out vs stay solo, scoping self-contained sub-tasks, prompting agents that don't share your context, isolating parallel edits (git worktrees), skeptically verifying and synthesizing results you didn't produce. Triggers: use subagents, parallelize this, fan out, orchestrate agents, multi-agent, spawn agents, worktree, delegate to agents, split this across agents, agent team. Sequencing checkpoints of an approved plan → plan-execution (orchestration parallelizes within its batches)."
4
+ model: sonnet
5
+ allowed-tools: Read, Grep, Glob, Write, Edit, Bash
6
+ ---
7
+
8
+ # Subagent Orchestration
9
+
10
+ Structure work across multiple agents so the result is *more* trustworthy than
11
+ one agent's pass — not just faster. The two failure modes this skill counters:
12
+ fanning out work that needed shared context (agents produce incompatible
13
+ pieces), and trusting subagent reports as if they were evidence (they're
14
+ claims). Exact tool names and mechanics vary by harness and move fast — apply
15
+ these principles through whatever agent-spawning surface is available.
16
+ Boundary: `plan-execution` sequences an approved plan's checkpoints;
17
+ orchestration is *how a batch* gets executed when it parallelizes.
18
+
19
+ ## Workflow
20
+
21
+ ### Step 1: Decide Fan-Out vs Solo — Honestly
22
+
23
+ Fan out when the work is **divisible into independent pieces** (per-file
24
+ migration, multi-angle research, review dimensions), when you need
25
+ **independent perspectives** (adversarial verification, judge panels), or when
26
+ the reading wouldn't fit one context (broad audits). Stay solo when the task
27
+ needs **one coherent design voice** (an API's shape, a refactor's structure),
28
+ when pieces are tightly coupled, or when the task is small — orchestration has
29
+ overhead, and a fleet of agents building a feature without shared design
30
+ produces expensive incoherence. Sub-agents also can't ask you clarifying
31
+ questions mid-flight: ambiguity you'd resolve interactively must be resolved
32
+ *before* the fan-out.
33
+
34
+ ### Step 2: Decompose into Self-Contained Tasks
35
+
36
+ Each sub-task needs: a **crisp goal** with a definition of done, **explicit
37
+ inputs** (files, constraints, conventions), **no shared mutable state** with
38
+ its siblings, and an **output contract** (exactly what to return, in what
39
+ form). If two tasks keep needing to talk to each other, they're one task —
40
+ re-cut the boundaries. Interfaces first: when pieces must fit together, fix
41
+ the contract between them *yourself, up front*, and hand each agent the
42
+ contract rather than hoping parallel guesses converge.
43
+
44
+ ### Step 3: Prompt for a Cold Start
45
+
46
+ The subagent knows nothing you don't say — none of your conversation, your
47
+ constraints, or your taste. State the context it needs (or say exactly which
48
+ files to read first), the conventions to follow, the boundaries ("do NOT touch
49
+ X"), and the output contract. Ask for **raw findings and artifacts** (data,
50
+ diffs, file paths, failure output) rather than summaries — you need material
51
+ you can verify, not reassurance. A vague prompt costs a full agent-run to
52
+ discover; read your prompt as a stranger before sending it.
53
+
54
+ ### Step 4: Isolate Parallel Writers
55
+
56
+ Agents editing the same working tree concurrently corrupt each other. Reads
57
+ parallelize freely; **writes need isolation** — one writer per area, or git
58
+ worktrees (each agent on its own branch/copy, merged deliberately afterward).
59
+ Budget for the merge: parallel edits that touch shared files turn saved
60
+ wall-clock into conflict resolution. If the merge looks expensive, re-cut
61
+ Step 2 along file boundaries instead.
62
+
63
+ ### Step 5: Verify Like You Didn't Write It — Because You Didn't
64
+
65
+ A subagent's "done, tests pass" is a report, not evidence
66
+ (`verification-before-completion` applies with extra force). Run the proving
67
+ command on their artifact yourself; spot-read the diff (`code-reviewing` for
68
+ substantial changes). For *findings* (bugs, research claims), use structure:
69
+ **adversarial verification** (a second agent prompted to *refute* each
70
+ finding), **diverse lenses** (verifiers checking different failure modes beat
71
+ N identical passes), and majority voting where individual judgments are noisy.
72
+ Expect and discard duplicates and false positives from parallel finders —
73
+ that's normal yield, not failure.
74
+
75
+ ### Step 6: Synthesize — the Step That Makes It One Answer
76
+
77
+ Merging results is real work you own: dedup overlapping findings, reconcile
78
+ contradictions (two agents asserting opposite things means *you* investigate,
79
+ not average), integrate parallel diffs and re-run the full suite on the
80
+ combined result, and write the coherent summary. The whole was your task; the
81
+ agents only did the parts. Close with a completeness check: did every
82
+ sub-task actually complete, and what fell between the cracks of the
83
+ decomposition?
84
+
85
+ ## Principles Applied
86
+
87
+ - **Decomposition quality bounds output quality**: bad task boundaries can't
88
+ be fixed by more agents.
89
+ - **Reports aren't evidence**: verify artifacts, adversarially where stakes
90
+ warrant — trust is for people, not processes.
91
+ - **Parallelize reads freely, writes deliberately**: isolation is cheap before
92
+ the fan-out and expensive after.
93
+
94
+ ## Cross-Skill References
95
+
96
+ - `plan-execution` — the checkpoint discipline; orchestration executes its batches
97
+ - `feature-planning` — producing the task breakdown worth parallelizing
98
+ - `verification-before-completion` — the evidence gate applied to every agent's claim
99
+ - `code-reviewing` — reviewing substantial subagent diffs before integration
100
+ - `git-workflow` — branch/merge hygiene for worktree-isolated work
@@ -0,0 +1,40 @@
1
+ {
2
+ "skill_name": "subagent-orchestration",
3
+ "evals": [
4
+ {
5
+ "id": 1,
6
+ "prompt": "I need to migrate 40 API endpoint files from our old error-handling pattern to the new Result-type pattern. The transformation is mechanical but each file has quirks. Can we parallelize this across subagents?",
7
+ "expected_output": "Confirms good fan-out shape (divisible, independent files, mechanical core), decomposes with an explicit output contract and conventions per agent, fixes the shared pattern/contract up front (an exemplar transformation), prompts for cold start (files to read, boundaries, exact return format), isolates parallel writers (per-file ownership or worktrees), then verifies the combined result itself — running the full suite on the merged output and spot-reviewing diffs — rather than trusting per-agent 'done' reports",
8
+ "assertions": [
9
+ "Evaluates the task's fan-out fitness (independent per-file work, mechanical transformation) rather than parallelizing by default",
10
+ "Fixes the shared contract up front — an exemplar of the new pattern/conventions each agent receives — so parallel results converge",
11
+ "Prompts agents for a cold start: explicit inputs, boundaries (files they own, what not to touch), and an exact output contract",
12
+ "Isolates parallel writes (one agent per file/area or git worktrees) to avoid corrupting the shared working tree",
13
+ "Verifies the merged result directly (full test suite on the combined diff, spot-reading changes) instead of trusting subagent completion reports"
14
+ ]
15
+ },
16
+ {
17
+ "id": 2,
18
+ "prompt": "Designing our new public API is taking long. Let's speed it up: spawn 6 subagents — one designs the auth endpoints, one the user endpoints, one the billing endpoints, one picks the pagination style, one the error format, one versioning. Then we'll glue their outputs together.",
19
+ "expected_output": "Pushes back on the decomposition: an API's shape needs one coherent design voice — pagination, error format, versioning, and resource conventions are cross-cutting decisions that must be fixed once, not designed in parallel and glued; recommends deciding the cross-cutting contract first (solo or via api-design), then fanning out only genuinely independent work (e.g. drafting per-resource specs against the fixed contract, or independent reviews of a draft), warning that parallel guesses produce incompatible pieces and expensive reconciliation",
20
+ "assertions": [
21
+ "Identifies that the proposed decomposition splits tightly-coupled, cross-cutting design decisions (pagination, errors, versioning, conventions) that need a single coherent voice",
22
+ "Recommends fixing the shared contract/conventions first, before any fan-out, rather than gluing independently-designed pieces afterward",
23
+ "Explains the failure mode: parallel agents without a shared contract produce incompatible outputs and costly reconciliation",
24
+ "Offers a legitimate parallel structure instead — e.g. per-resource spec drafting against the fixed contract, or a multi-perspective review/judge panel of a single draft",
25
+ "Does not simply execute the 6-way split as requested"
26
+ ]
27
+ },
28
+ {
29
+ "id": 3,
30
+ "prompt": "Here's our approved 5-phase implementation plan for the billing feature. Execute it phase by phase, verifying each phase before moving to the next.",
31
+ "expected_output": "Recognizes this as sequential checkpointed execution of an approved plan — refers to plan-execution for the checkpoint/verification/drift discipline, noting orchestration only enters if a batch inside the plan parallelizes; states the boundary between sequencing a plan and fanning out sub-tasks",
32
+ "assertions": [
33
+ "Recognizes the request as checkpointed sequential execution of an approved plan, not a parallelization problem",
34
+ "Refers to the plan-execution skill for the checkpoint, verification-evidence, and drift discipline",
35
+ "Does not spawn subagents or propose a fan-out as the primary answer",
36
+ "States or implies the boundary: plan-execution sequences the plan's checkpoints; subagent-orchestration is how an individual batch runs when it genuinely parallelizes"
37
+ ]
38
+ }
39
+ ]
40
+ }
@@ -1,7 +1,10 @@
1
1
  ---
2
2
  name: technical-debt-review
3
- description: "Strategic codebase health assessment — identify hotspots, categorize debt, produce remediation roadmap. Triggers: technical debt, tech debt, debt review, codebase health, hotspots, debt assessment, remediation plan, what should we fix first, debt roadmap, code rot, legacy code audit."
3
+ description: "Strategic codebase health assessment — identify hotspots, categorize debt, produce remediation roadmap."
4
+ when_to_use: "Triggers: technical debt, tech debt, debt review, codebase health, hotspots, debt assessment, remediation plan, what should we fix first, debt roadmap, code rot, legacy code audit."
4
5
  model: opus
6
+ context: fork
7
+ agent: general-purpose
5
8
  allowed-tools: Read, Grep, Glob, Write, Edit
6
9
  ---
7
10
 
@@ -34,7 +37,7 @@ Don't try to review everything. Focus on where pain is concentrated.
34
37
  - Tests that only test happy paths (no edge cases, no error cases)
35
38
  - Test files significantly longer than the code they test (over-specified tests that break on refactoring)
36
39
 
37
- Ask the user: Where do engineers slow down? What parts of the codebase do people avoid touching? Where do bugs keep appearing?
40
+ If the request came with team pain points (where engineers slow down, code people avoid touching, where bugs keep appearing), weight those hotspots first. This skill runs in a forked context and cannot ask mid-run — when that input is missing, proceed on the code signals above and list "where does the team actually hurt?" under Open questions in the report.
38
41
 
39
42
  ## Step 2: Categorize by Debt Type and Severity
40
43
 
@@ -99,6 +102,13 @@ Output a prioritized action plan:
99
102
 
100
103
  Use [templates/debt-audit.md](templates/debt-audit.md) for the full audit format.
101
104
 
105
+ **Write the full audit to a file** (default a gitignored location, e.g.
106
+ `.local/debt-review-<date>.md`) and state its path in your final summary — this
107
+ skill runs in a forked context: only the summary returns, everything unwritten is
108
+ lost. Judgment calls that need user input (e.g. which subsystem matters most,
109
+ unknown team pain points) go in an **Open questions** section of the report, never
110
+ guessed silently.
111
+
102
112
  ## Principles Applied
103
113
 
104
114
  - **KISS**: Simplify complexity first — complex code is the root cause of most other debt