swe-workflow-skills 0.2.0 → 0.3.0

package/skills/git-workflow/SKILL.md

@@ -1,6 +1,7 @@
 ---
 name: git-workflow
-description: "Write commit messages, PR descriptions, and manage branching strategy following conventional commits. Triggers: write a commit message, commit this, PR description, pull request, branching strategy, git workflow, squash commits, rebase, conventional commits, how should I commit this, review staged changes."
+description: "Write commit messages, PR descriptions, and manage branching strategy following conventional commits."
+when_to_use: "Triggers: write a commit message, commit this, PR description, pull request, branching strategy, git workflow, squash commits, rebase, conventional commits, how should I commit this, review staged changes."
 model: haiku
 allowed-tools: Read, Grep, Glob, Write, Edit
 ---
@@ -13,11 +14,13 @@ Help write clear commit messages, structured PR descriptions, and manage branchi
 
 ### Step 1: Analyze the Changes
 
-Read the staged changes to understand what was done:
+Currently staged (live at skill load; empty when nothing is staged, this isn't a
+git repo, or injection is disabled):
 
-```bash
-git diff --staged
-```
+!`git diff --staged --stat 2>/dev/null || true`
+
+If the summary above is empty or not enough to understand the change, run
+`git diff --staged` yourself for the full diff.
 
 Identify:
 - **What changed?** (files modified, functions added/removed, logic altered)
@@ -68,14 +71,17 @@ Before committing, check:
 
 ### Step 1: Analyze the Branch
 
-Read the changes in the branch:
+Branch state vs the default branch (live at skill load; tries `main` then
+`master`):
 
-```bash
-git log main..HEAD --oneline
-git diff main...HEAD --stat
-```
+!`git log main..HEAD --oneline 2>/dev/null || git log master..HEAD --oneline 2>/dev/null || true`
+
+!`git diff main...HEAD --stat 2>/dev/null || git diff master...HEAD --stat 2>/dev/null || true`
 
-Understand the full scope of changes across all commits.
+If the output above is empty or the repo uses a different default branch, run the
+equivalents against the actual base branch. Understand the full scope of changes
+across all commits — run the full `git diff <base>...HEAD` when the stat summary
+isn't enough.
 
 ### Step 2: Write the PR Description
 
@@ -106,9 +112,10 @@ Help set up or improve branching conventions. Ask about team size and release ca
 
 **For small teams (1-5 devs) or continuous deployment:**
 - `main` — always deployable
-- `feat/description` — feature branches, short-lived (1-3 days)
-- `fix/description` — bug fix branches
-- Merge to main via PR, deploy from main
+- Branch naming: `feat/<description>`, `fix/<description>` — short-lived (1-3 days)
+- Merge to main via PR — **squash-merge by default** (one clean commit per logical
+  change on main); one approval is the right review bar at this size
+- Deploy from main
 
 **For medium teams (5-15 devs) or scheduled releases:**
 - `main` — production

package/skills/ml-pipeline-design/SKILL.md

@@ -1,13 +1,19 @@
 ---
 name: ml-pipeline-design
-description: "Design reproducible ML training and data pipelines — ingestion, validation, feature engineering, training, evaluation, continuous training orchestration. Triggers: training pipeline, ML data pipeline, feature engineering, ETL for ML, continuous training, data validation, feature store, preprocessing, notebook to pipeline, orchestrate training, Kubeflow, pipeline DAG, point-in-time features. Analytics/BI ELT and dbt warehouse pipelines → data-pipeline-design."
+description: "Design reproducible ML training and data pipelines — ingestion, validation, feature engineering, training, evaluation, continuous training orchestration. Analytics/BI ELT and dbt warehouse pipelines → data-pipeline-design; refactoring general analysis/reporting notebooks (no model training) to production code → notebook-to-production."
+when_to_use: "Triggers: training pipeline, ML data pipeline, feature engineering, ETL for ML, continuous training, data validation, feature store, preprocessing, notebook to training pipeline, orchestrate training, Kubeflow, pipeline DAG, point-in-time features."
 model: opus
 allowed-tools: Read, Grep, Glob, Write, Edit, Bash
 ---
 
 # ML Pipeline Design
 
-Design reproducible, automated ML pipelines that transform notebooks into production-grade workflows. A pipeline is the difference between "I ran this notebook and got good results" and "this system produces, validates, and deploys models automatically."
+Design reproducible, automated ML pipelines that transform training notebooks into production-grade workflows. A pipeline is the difference between "I ran this notebook and got good results" and "this system produces, validates, and deploys models automatically."
+
+Scope check first: if the notebook does NOT train a model — it's analysis,
+KPIs, or reporting — this is not a training pipeline; use the
+`notebook-to-production` skill (modules, tests, parameterization, scheduling)
+instead.
 
 ## Why Pipelines Matter
 

package/skills/ml-pipeline-design/evals/evals.json

@@ -29,6 +29,16 @@
         "Recommends the pipeline should FAIL on bad data, not train on it",
         "Provides concrete implementation (Great Expectations or Pandera)"
       ]
+    },
+    {
+      "id": 3,
+      "prompt": "I have a Jupyter notebook that pulls last month's sales from the warehouse, computes exec KPIs, and emails a report. I want it to run on a schedule instead of me running it by hand — how do I turn this notebook into production code?",
+      "expected_output": "Should recognize there is no model training involved — this is a general analysis/reporting notebook — and hand off to the notebook-to-production skill (modules, tests, parameterization, scheduling) rather than designing an ML training pipeline.",
+      "assertions": [
+        "Recognizes no model training is involved, so this is not an ML training pipeline",
+        "Hands off to or invokes the notebook-to-production skill for the refactor",
+        "Does not propose ML-specific machinery (feature stores, training DAGs, model evaluation gates) for a reporting notebook"
+      ]
     }
   ]
 }

package/skills/notebook-to-production/SKILL.md

@@ -0,0 +1,89 @@
+---
+name: notebook-to-production
+description: "Refactor analysis notebooks into production-grade code — triage what deserves productionizing, extract modules with tests, parameterize hardcoded values, pin environments for reproducibility, and schedule unattended runs with alerting. Owns general analysis/reporting notebooks; notebooks that feed model TRAINING (feature engineering + training DAGs, scheduled retraining) → ml-pipeline-design."
+when_to_use: "Triggers: productionize this notebook, notebook to script, notebook to module, refactor my notebook, parameterize notebook, papermill, schedule this analysis, reproducible analysis, notebook is a mess."
+model: opus
+allowed-tools: Read, Grep, Glob, Write, Edit, Bash
+---
+
+# Notebook to Production
+
+Turn exploratory notebooks into code that runs unattended, produces the same
+result twice, and fails loudly. The refactor is incremental — the notebook
+keeps working at every step — never a from-scratch rewrite.
+
+## Workflow
+
+### 1. Triage — not every notebook earns this
+
+Productionize by **re-run frequency × business impact**. A notebook someone
+re-runs by hand every quarter is the first candidate; a one-off exploration is
+not — **archive it reproducibly instead**: the query or a data snapshot, a
+pinned environment (`pip freeze` at minimum), and a one-line header (question,
+finding, date, author). "Archived" without the data recipe and environment is
+deleted-with-extra-steps.
+
+For a backlog of many notebooks, also: extract logic duplicated across them
+into a shared utility package (one fix, not N), and set a lightweight
+convention for future notebooks likely to graduate — imports at top,
+parameters in the first cell, logic in functions — so the next migration is
+cheap.
+
+### 2. Extract logic into modules
+
+Move transformations out of cells into importable, typed functions in a real
+package (`src/`), leaving the notebook as a thin driver that imports and
+calls. Separate I/O (load/save) from transformation logic — pure
+transformation functions are what you can test. Restartability check: does it
+run top-to-bottom in a fresh kernel? Hidden execution-order state is the first
+bug this flushes out.
+
+### 3. Parameterize everything hardcoded
+
+Dates, paths, connection strings, thresholds → function arguments driven by a
+config file or CLI args (or papermill parameters if the notebook itself stays
+the execution vehicle). The run for "last month" and the backfill for "March
+2024" must be the same code with different parameters.
+
+### 4. Test the transformations
+
+The extracted pure functions get unit tests: known-input → known-output cases,
+plus edge cases the data will eventually throw (empty frame, nulls, duplicate
+keys, timezone boundaries). A small golden sample of real data as a fixture
+catches most regressions. No tests, no schedule — an untested unattended job
+is a silent-corruption machine.
+
+### 5. Pin the environment
+
+Reproducibility means the same code AND the same dependencies: a lock file
+(`uv lock`, `poetry.lock`, `pip-compile`) or a container image. Pin the data
+too where possible — snapshot inputs or record query + as-of timestamp, so a
+rerun of last month's report produces last month's numbers.
+
+### 6. Schedule and operate
+
+Size the scheduler to the team: cron or a CI schedule (GitHub Actions) first;
+an orchestrator (Airflow/Dagster) only when there are real DAG dependencies
+across jobs. Unattended means operated: fail loudly (alert on failure AND on
+suspicious output — empty result, row count off), log enough to debug a 6am
+failure, and make the job idempotent so a retry can't double-append or
+double-send.
+
+## Principles
+
+- **Incremental (KISS)**: extract one section at a time; the notebook keeps
+  running throughout.
+- **YAGNI**: no orchestrator, feature store, or microservice for a weekly
+  report — cron + a tested module is production.
+- **SRP**: load, transform, and render are separate functions with separate
+  tests.
+
+## Cross-skill boundaries
+
+- The notebook trains a model (feature engineering, training, evaluation,
+  retraining schedule) → **ml-pipeline-design** — training pipelines have
+  their own stage/validation/gating design.
+- Recurring ELT into a warehouse feeding many consumers → **data-pipeline-design**.
+- Still figuring out what the analysis should say → **exploratory-data-analysis**.
+- CI mechanics for the scheduled run → **cicd-pipeline**; test design depth →
+  **test-suite-design**.

package/skills/notebook-to-production/evals/evals.json

@@ -0,0 +1,42 @@
+{
+  "skill_name": "notebook-to-production",
+  "evals": [
+    {
+      "id": 1,
+      "prompt": "My analysis notebook pulls monthly sales data, cleans it, computes KPI aggregates, and generates charts for a report. Management wants this delivered automatically every Monday morning. It's 800 lines of cells with hardcoded dates and file paths. How do I productionize it?",
+      "expected_output": "Should refactor incrementally: extract the logic into importable modules, parameterize the hardcoded dates/paths, add tests for the transformations, pin the environment for reproducibility, schedule the Monday run with failure alerting — not rewrite from scratch.",
+      "assertions": [
+        "Extracts notebook logic into importable modules/functions rather than leaving everything in cells",
+        "Parameterizes the hardcoded dates and paths (config file, CLI args, or papermill-style notebook parameters)",
+        "Adds tests for the extracted transformation logic",
+        "Pins the environment for reproducibility (lock file, pinned requirements, or a container)",
+        "Sets up scheduling for the Monday run sized to the team (cron or CI schedule before reaching for a heavy orchestrator)",
+        "Addresses failure handling and alerting for the unattended run",
+        "Proposes an incremental refactor path rather than a from-scratch rewrite"
+      ]
+    },
+    {
+      "id": 2,
+      "prompt": "Our team has 30+ research notebooks accumulated over two years. Some are one-off explorations, some get re-run every quarter by hand. We can't productionize everything — what should we do?",
+      "expected_output": "Should triage rather than migrate everything: productionize the recurring high-impact notebooks first, archive one-offs with enough context to be reproducible, extract shared utilities for duplicated logic, and set a lightweight standard for future notebooks.",
+      "assertions": [
+        "Triages the notebooks instead of proposing to productionize all 30+ — one-off explorations stay notebooks",
+        "Prioritizes by re-run frequency and business impact (the quarterly hand-run ones first)",
+        "Recommends archiving one-offs with enough context to reproduce them later (data snapshot or query, pinned environment)",
+        "Proposes extracting shared utilities for logic duplicated across notebooks",
+        "Suggests a lightweight convention for future notebooks that are likely to need productionizing",
+        "Does not propose a big-bang migration of everything"
+      ]
+    },
+    {
+      "id": 3,
+      "prompt": "I have a notebook that does feature engineering on our user events and trains a ranking model. I want to turn it into a pipeline that retrains automatically every week.",
+      "expected_output": "Should recognize this notebook feeds model TRAINING — feature engineering, training, evaluation, scheduled retraining — and hand off to the ml-pipeline-design skill for the training-pipeline DAG, rather than doing a general notebook-to-module refactor.",
+      "assertions": [
+        "Recognizes the notebook feeds model training and hands off to or invokes the ml-pipeline-design skill",
+        "Names training-pipeline concerns (pipeline DAG stages, data validation, evaluation gate, retraining schedule) as ml-pipeline-design territory",
+        "Does not respond with only a generic notebook-to-module refactor as the primary answer"
+      ]
+    }
+  ]
+}

package/skills/project-documentation/SKILL.md

@@ -1,6 +1,7 @@
 ---
 name: project-documentation
-description: "Write and maintain project docs — README, contributing guides, API docs, changelogs, inline docs. Triggers: write a README, document this project, create documentation, contributing guide, changelog, API docs, how do I document, this needs docs, onboarding docs, JSDoc, docstrings."
+description: "Write and maintain project docs — README, contributing guides, API docs, changelogs, inline docs."
+when_to_use: "Triggers: write a README, document this project, create documentation, contributing guide, changelog, API docs, how do I document, this needs docs, onboarding docs, JSDoc, docstrings."
 model: haiku
 allowed-tools: Read, Grep, Glob, Write, Edit
 ---
@@ -87,9 +88,11 @@ Follow Keep a Changelog conventions:
 
 See [templates/changelog.md](templates/changelog.md) for the format.
 
-Accumulate entries under `[Unreleased]` as changes land; promoting them to a version
-heading happens at release time — version choice, tagging, and automated changelog
-generation are `release-management`'s domain.
+Accumulate entries under `[Unreleased]` as changes land. Writing the entries is this
+skill's work even when they must be reconstructed from git history — read the log
+since the last tag, drop trivial commits (merges, typo fixes), and translate the rest
+into user-perspective entries under the sections above. Hand off only the release
+mechanics (version choice, tagging, publish automation) to `release-management`.
 
 ## Workflow: Inline Documentation
 

package/skills/project-review/SKILL.md

@@ -1,7 +1,10 @@
 ---
 name: project-review
-description: "Review a built project's execution health before a milestone — scope alignment, roadmap / execution-plan adherence, implementation maturity (what's production-ready vs stub/deferred), and the evidence it actually works (tests, coverage, validation results, changelog). Triggers: project review, execution review, are we on track, implementation vs roadmap, scope drift, readiness review, what's actually built, validation results review, pre-launch review. Use strategic-review for vision/positioning/market; technical-debt-review for a pure code-health audit."
+description: "Review a built project's execution health before a milestone — scope alignment, roadmap / execution-plan adherence, implementation maturity (what's production-ready vs stub/deferred), and the evidence it actually works (tests, coverage, validation results, changelog). Use strategic-review for vision/positioning/market; technical-debt-review for a pure code-health audit."
+when_to_use: "Triggers: project review, execution review, are we on track, implementation vs roadmap, scope drift, readiness review, what's actually built, validation results review, pre-launch review."
 model: opus
+context: fork
+agent: general-purpose
 allowed-tools: Read, Grep, Glob, WebFetch, Write, Edit
 ---
 
@@ -90,9 +93,16 @@ Produce ranked, severity-tagged findings, each tied to its evidence:
 - **An execution scorecard** — maturity, roadmap-adherence, evidence-strength,
   each rated with a one-line justification, so the reader gets the picture at a glance.
 
+**Write the full report to a file** — default a gitignored location (e.g.
+`.local/project-review-<date>.md`) unless told otherwise — and state its path in
+your final summary. This skill runs in a forked context: only the summary returns
+to the main conversation, so anything not written to disk is lost. If a judgment
+call needs user input (ambiguous scope boundary, missing baseline docs, which
+milestone to measure against), do **not** guess silently — record it in an
+**Open questions** section of the report and mention it in the summary.
+
 For the rendered deliverable (interactive HTML report, dashboards, sortable
-findings table), hand off to `artifact-design`. Default to writing it to a
-gitignored location (e.g. `.local/`) unless told otherwise.
+findings table), hand off to `artifact-design`.
 
 ## Principles Applied
 

package/skills/security-audit/SKILL.md

@@ -1,7 +1,10 @@
 ---
 name: security-audit
-description: "Comprehensive security analysis — OWASP Top 10, auth/authz flows, injection vulnerabilities, data exposure, secrets detection, dependency CVEs, hardening recommendations. Triggers: security audit, vulnerability, is this secure, security review, pentest prep, OWASP, harden this, check for vulnerabilities, injection, XSS, CSRF, auth security. Reviews EXISTING code/config — design-time analysis of a system not yet built → threat-modeling."
+description: "Comprehensive security analysis — OWASP Top 10, auth/authz flows, injection vulnerabilities, data exposure, secrets detection, dependency CVEs, hardening recommendations. Reviews EXISTING code/config — design-time analysis of a system not yet built → threat-modeling."
+when_to_use: "Triggers: security audit, vulnerability, is this secure, security review, pentest prep, OWASP, harden this, check for vulnerabilities, injection, XSS, CSRF, auth security."
 model: opus
+context: fork
+agent: general-purpose
 allowed-tools: Read, Grep, Glob, Write, Edit, WebFetch, WebSearch
 ---
 
@@ -31,7 +34,7 @@ Before reviewing code line by line, map what's exposed:
 - **External integrations**: Third-party APIs, OAuth providers, payment processors
 - **Infrastructure**: Cloud services, databases, caches, queues
 
-Present the attack surface map to the user. Gaps in understanding here mean gaps in the audit.
+Record the attack surface map as the first section of the report — gaps in understanding here mean gaps in the audit, so anything you could not map (unreachable config, undocumented integrations) goes under Open questions rather than being assumed safe.
 
 ### Step 2: Assess by OWASP Top 10
 
@@ -113,6 +116,14 @@ Suggest using the `dependency-management` skill for remediation planning.
 
 Output the audit report using the template at [templates/security-report.md](templates/security-report.md). Organize findings by severity, with Critical and High items first.
 
+**Write the full report to a file** (default a gitignored location, e.g.
+`.local/security-audit-<date>.md`) and state its path in your final summary — this
+skill runs in a forked context: only the summary returns, everything unwritten is
+lost. The summary must lead with the Critical/High count. Judgment calls that need
+user input (unclear trust boundary, whether a data store is in scope, unverifiable
+control) go in an **Open questions** section of the report — never silently assumed
+safe.
+
 ## What This Skill Does NOT Cover
 
 - Runtime penetration testing (requires running the application)

package/skills/skill-router/SKILL.md

@@ -1,6 +1,7 @@
 ---
 name: skill-router
-description: "Orchestrator and entry point for the swe-workflow skills library — consult FIRST when starting any non-trivial software task; most skills load name-only and only activate when invoked here. Triggers: starting a feature, planning, an architecture or design decision, implementing, debugging, reviewing, refactoring, testing, security, deployment, an incident, shipping, or unsure which skill fits. Routes intent to the right skill(s) and invokes them by name; shows the Golden Path chains."
+description: "Orchestrator and entry point for the swe-workflow skills library — consult FIRST when starting any non-trivial software task; most skills load name-only and only activate when invoked here. Routes intent to the right skill(s) and invokes them by name; shows the Golden Path chains."
+when_to_use: "Triggers: starting a feature, planning, an architecture or design decision, implementing, debugging, reviewing, refactoring, testing, security, deployment, an incident, shipping, or unsure which skill fits."
 model: haiku
 allowed-tools: Read, Grep, Glob, Skill
 ---
@@ -167,6 +168,11 @@ installed skill by name, regardless of role.
 - **data-pipeline-design** — batch/streaming ELT, dbt layering, Airflow/Dagster, idempotency, backfill
 - **data-quality** — dbt tests, expectations, data contracts, freshness, lineage / blast radius
 
+### Data Science
+- **exploratory-data-analysis** — profile an unfamiliar dataset: missingness structure, distributions, leakage, hypotheses
+- **statistical-analysis** — hypothesis tests, experiment design/power, confidence intervals, multiple-comparison discipline
+- **notebook-to-production** — refactor analysis notebooks to tested, parameterized, scheduled production code
+
 ## Golden Path workflow chains
 
 When work spans phases, chain skills rather than improvising:
@@ -201,6 +207,10 @@ App-store releases (signing, review, staged rollout) → `mobile-release`.
 `data-pipeline-design` (ELT, layering, orchestration) → `data-modeling`
 (marts schema) → `data-quality` (tests, contracts, freshness) →
 `observability-design` (pipeline SLOs, alerting).
+Notebook to production: `exploratory-data-analysis` (understand the data) →
+`statistical-analysis` (test the hypotheses) → `notebook-to-production`
+(modules, tests, scheduling); if the notebook trains a model →
+`ml-pipeline-design` instead.
 
 **Pre-public / pre-milestone review**
 `strategic-review` (vision, positioning, market) → `project-review` (scope,

package/skills/statistical-analysis/SKILL.md

@@ -0,0 +1,93 @@
+---
+name: statistical-analysis
+description: "Design and analyze experiments and statistical tests — test selection with stated assumptions, sample size and power, effect sizes and confidence intervals over bare p-values, and pitfall discipline (multiple comparisons, p-hacking, peeking/optional stopping). Owns experiment statistics generally; live A/B evaluation of AI/LLM apps (quality metrics, judges, feedback) → ai-evaluation."
+when_to_use: "Triggers: hypothesis test, t-test, chi-square, p-value, statistical significance, confidence interval, sample size, power analysis, design an experiment, A/B test, multiple comparisons, is this difference real."
+model: opus
+allowed-tools: Read, Grep, Glob, Write, Edit, Bash
+---
+
+# Statistical Analysis
+
+Get the statistics of a question right — the test, its assumptions, the
+uncertainty, and the traps — so a decision rests on evidence rather than on a
+lone p-value. The deliverable is a decision-ready answer: effect size,
+uncertainty, assumptions, caveats.
+
+## Workflow
+
+### 1. Frame the question as a hypothesis
+
+State the null and alternative in plain words before touching data ("the new
+flow's conversion equals the old flow's"). Identify the unit of analysis
+(user? session? order?) — mismatching it with the randomization unit silently
+breaks independence.
+
+### 2. Design before data (when the experiment hasn't run yet)
+
+- Choose the primary metric and **pre-register it** — one primary, the rest
+  explicitly secondary/exploratory.
+- Compute **sample size from power**: minimum detectable effect worth acting
+  on, α (typically 0.05), power (typically 0.8). Run until n is reached —
+  don't stop early on a good-looking result.
+- Randomize at the unit you'll analyze; check the split is balanced on key
+  covariates.
+
+### 3. Choose the test and state its assumptions
+
+Match the test to the data (two proportions → two-proportion z-test / χ²;
+means → t-test or its nonparametric fallback; counts → Poisson/χ²; paired data
+→ paired tests). **Always state the assumptions and check the ones you can**:
+independence of observations (the usual casualty — repeated measures per user,
+network effects, shared fixtures), adequate counts for normal approximations
+(rule of thumb: ≥10 successes and failures per arm), variance assumptions for
+t-tests. If an assumption fails, switch tests rather than pretending.
+
+### 4. Report effect size and interval, not a verdict
+
+Work the actual numbers. Report the effect (absolute and relative difference)
+with a **confidence interval**, and the p-value as supporting detail — never a
+bare "significant/not significant". The CI is what the decision needs: a CI of
+[+0.1%, +1.7%] and a CI of [+0.8%, +1.0%] are both "significant" and mean very
+different things.
+
+### 5. Run the pitfall gate
+
+Before reporting anything, check:
+
+- **Peeking / optional stopping**: was n fixed in advance? Repeatedly testing
+  as data arrives inflates false positives badly. If peeking happened, say so
+  and treat the result as suggestive (or use sequential methods designed for it).
+- **Multiple comparisons**: k tests at α=0.05 ≈ k/20 false positives expected.
+  Correct (Holm/Bonferroni for strict control, Benjamini-Hochberg FDR when many
+  metrics), and label post-hoc findings as exploratory — they are hypotheses
+  for the *next* experiment, not wins from this one.
+- **Subgroup fishing**: subgroup effects found after the fact need the same
+  exploratory label and a confirmatory follow-up.
+- **Survivorship / selection**: does the analyzed sample still represent the
+  population the decision is about?
+
+### 6. Translate to the decision
+
+Distinguish statistical from practical significance: a real +0.05% lift may
+not pay for the complexity; a non-significant result with a wide CI is "we
+don't know yet", not "no effect". Give the ship/don't-ship framing with the
+caveats attached, and say what additional data would resolve remaining
+uncertainty.
+
+## Principles
+
+- **KISS**: a well-checked z-test beats a fancy model with unexamined
+  assumptions.
+- **Pre-register or label**: every metric is either declared up front or
+  reported as exploratory — no third category.
+- **Uncertainty is the product**: any answer without an interval is opinion
+  with extra steps.
+
+## Cross-skill boundaries
+
+- Live A/B testing of an AI/LLM app — defining answer quality, LLM-as-judge,
+  human feedback loops, eval gates → **ai-evaluation** (this skill still owns
+  the underlying experiment statistics if asked).
+- Generating the hypotheses to test → **exploratory-data-analysis**.
+- Tracking and comparing model-training runs → **ml-experiment-tracking**.
+- Defining what business metric to move → **metrics-and-okrs**.

package/skills/statistical-analysis/evals/evals.json

@@ -0,0 +1,43 @@
+{
+  "skill_name": "statistical-analysis",
+  "evals": [
+    {
+      "id": 1,
+      "prompt": "We ran an experiment: 2,400 users saw the new checkout flow and converted at 6.8%; 2,350 saw the old one and converted at 5.9%. My PM wants to ship it today — is this difference real? How confident can we be?",
+      "expected_output": "Should treat this as a two-proportion hypothesis test: state the null, run or outline the test with the actual numbers, report a confidence interval on the difference, ask whether the sample size was fixed in advance (peeking), and separate statistical from practical significance before giving a ship recommendation.",
+      "assertions": [
+        "Frames the question as a hypothesis test (two-proportion z-test or equivalent) and states the null hypothesis",
+        "Works the actual numbers given (or sets up the exact computation) rather than answering qualitatively",
+        "Reports uncertainty as a confidence interval on the difference, not only a p-value or a yes/no verdict",
+        "Asks or checks whether the sample size was fixed in advance and flags the peeking/optional-stopping problem",
+        "Distinguishes statistical significance from practical/business significance",
+        "States the assumptions behind the chosen test (independent samples, adequate counts)",
+        "Gives a clear ship/don't-ship framing with the statistical caveats attached"
+      ]
+    },
+    {
+      "id": 2,
+      "prompt": "We measured 20 different metrics in our experiment and 2 came out significant at p < 0.05. Which ones should we report as wins?",
+      "expected_output": "Should identify the multiple-comparisons problem (about one false positive is expected by chance at that rate), recommend a correction (Holm/Bonferroni or FDR), distinguish pre-registered primary metrics from exploratory ones, and treat the 2 hits as hypotheses for a confirmatory follow-up — not report them as wins.",
+      "assertions": [
+        "Identifies the multiple-comparisons problem explicitly — with 20 tests, roughly one false positive at p < 0.05 is expected by chance",
+        "Recommends a concrete correction (Bonferroni, Holm, or FDR / Benjamini-Hochberg) and notes the trade-off between them",
+        "Warns against reporting the 2 nominal hits as wins (p-hacking / cherry-picking)",
+        "Distinguishes pre-registered primary metrics from exploratory findings",
+        "Recommends treating the surviving hits as hypotheses for a follow-up confirmatory experiment",
+        "Does not simply declare the two significant metrics real effects"
+      ]
+    },
+    {
+      "id": 3,
+      "prompt": "We're about to launch a new system prompt for our support chatbot and want to A/B test it live against the current one to see which gives better answers. Can you set this up?",
+      "expected_output": "Should recognize this as online A/B evaluation of an LLM app and hand off to the ai-evaluation skill for the evaluation design (answer-quality metrics, LLM-as-judge or human feedback, eval gates); statistical-analysis contributes at most the underlying experiment statistics (sample size, significance), not the AI evaluation design.",
+      "assertions": [
+        "Recognizes this as AI-app evaluation territory and hands off to or invokes the ai-evaluation skill",
+        "Raises AI-specific measurement concerns (defining answer quality, LLM-as-judge or human feedback signals) rather than treating it as a pure two-sample statistics problem",
+        "Does not respond with only a generic A/B test design ignoring how answer quality gets measured",
+        "If it retains anything, retains only the experiment statistics (sample size, significance testing) as this skill's contribution"
+      ]
+    }
+  ]
+}

package/skills/strategic-review/SKILL.md

@@ -1,7 +1,10 @@
 ---
 name: strategic-review
-description: "Review a project's strategic position before going public, launching, or raising — vision, mission, value proposition, scope positioning, the defensible wedge, and a live competitive / market comparative analysis. Triggers: strategic review, positioning, go public, go-to-market, market analysis, competitive landscape, value proposition, is there a moat, who are our competitors, platform absorption risk, market positioning, comparable products. Use project-review for execution/roadmap/implementation health; delegates deep market scans to deep-research."
+description: "Review a project's strategic position before going public, launching, or raising — vision, mission, value proposition, scope positioning, the defensible wedge, and a live competitive / market comparative analysis. Use project-review for execution/roadmap/implementation health; delegates deep market scans to deep-research."
+when_to_use: "Triggers: strategic review, positioning, go public, go-to-market, market analysis, competitive landscape, value proposition, is there a moat, who are our competitors, platform absorption risk, market positioning, comparable products."
 model: opus
+context: fork
+agent: general-purpose
 allowed-tools: Read, Grep, Glob, WebFetch, WebSearch, Write, Edit
 ---
 
@@ -60,6 +63,9 @@ Survey the field with current evidence, not memory:
   absorb the category.
 - For a deep, multi-source, fact-checked sweep, **delegate to `deep-research`** and
   fold its cited report in. For a lighter pass, use `WebSearch`/`WebFetch` directly.
+  (This skill runs in a forked context — if the `deep-research` skill is not
+  invocable from here, do the lighter pass yourself and note under Open questions
+  that a deep-research sweep is the recommended follow-up.)
 - **Tag each comparable** with its relationship: **competitor**, **complement**,
   or **integration target** — and what it means for this project.
 - Label every finding confirmed-vs-inferred and date it.
@@ -86,8 +92,16 @@ Integrate the above into a decision instrument:
   default is options + a recommendation.
 - **A readiness picture** mapped to the project's own gates, if it has them.
 
+**Write the full review to a file** — default a gitignored path (e.g.
+`.local/strategic-review-<date>.md`) — and state its path in your final summary.
+This skill runs in a forked context: only the summary returns, everything unwritten
+is lost. The summary leads with the thesis verdict and the top weak point.
+Anything that needs the user's judgment (which strategic fork to take, an
+unverifiable market assumption, missing vision docs) goes in an **Open questions**
+section of the report — never silently decided.
+
 For the rendered deliverable (interactive HTML, scorecards, a forks comparison
-panel), hand off to `artifact-design`; default to a gitignored path (e.g. `.local/`).
+panel), hand off to `artifact-design`.
 
 ## Principles Applied
 

package/skills/technical-debt-review/SKILL.md

@@ -1,7 +1,10 @@
 ---
 name: technical-debt-review
-description: "Strategic codebase health assessment — identify hotspots, categorize debt, produce remediation roadmap. Triggers: technical debt, tech debt, debt review, codebase health, hotspots, debt assessment, remediation plan, what should we fix first, debt roadmap, code rot, legacy code audit."
+description: "Strategic codebase health assessment — identify hotspots, categorize debt, produce remediation roadmap."
+when_to_use: "Triggers: technical debt, tech debt, debt review, codebase health, hotspots, debt assessment, remediation plan, what should we fix first, debt roadmap, code rot, legacy code audit."
 model: opus
+context: fork
+agent: general-purpose
 allowed-tools: Read, Grep, Glob, Write, Edit
 ---
 
@@ -34,7 +37,7 @@ Don't try to review everything. Focus on where pain is concentrated.
 - Tests that only test happy paths (no edge cases, no error cases)
 - Test files significantly longer than the code they test (over-specified tests that break on refactoring)
 
-Ask the user: Where do engineers slow down? What parts of the codebase do people avoid touching? Where do bugs keep appearing?
+If the request came with team pain points (where engineers slow down, code people avoid touching, where bugs keep appearing), weight those hotspots first. This skill runs in a forked context and cannot ask mid-run — when that input is missing, proceed on the code signals above and list "where does the team actually hurt?" under Open questions in the report.
 
 ## Step 2: Categorize by Debt Type and Severity
 
@@ -99,6 +102,13 @@ Output a prioritized action plan:
 
 Use [templates/debt-audit.md](templates/debt-audit.md) for the full audit format.
 
+**Write the full audit to a file** (default a gitignored location, e.g.
+`.local/debt-review-<date>.md`) and state its path in your final summary — this
+skill runs in a forked context: only the summary returns, everything unwritten is
+lost. Judgment calls that need user input (e.g. which subsystem matters most,
+unknown team pain points) go in an **Open questions** section of the report, never
+guessed silently.
+
 ## Principles Applied
 
 - **KISS**: Simplify complexity first — complex code is the root cause of most other debt