swe-workflow-skills 0.1.0 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (59) hide show
  1. package/README.md +57 -17
  2. package/VERSION +1 -1
  3. package/catalog.json +95 -5
  4. package/package.json +1 -1
  5. package/roles.json +58 -7
  6. package/skills/ai-evaluation/SKILL.md +111 -0
  7. package/skills/ai-evaluation/evals/evals.json +40 -0
  8. package/skills/ai-evaluation/references/eval-tooling.md +63 -0
  9. package/skills/architecture-design/SKILL.md +3 -1
  10. package/skills/brainstorming/SKILL.md +92 -0
  11. package/skills/brainstorming/evals/evals.json +40 -0
  12. package/skills/browser-verification/SKILL.md +95 -0
  13. package/skills/browser-verification/evals/evals.json +40 -0
  14. package/skills/build-vs-buy/SKILL.md +109 -0
  15. package/skills/build-vs-buy/evals/evals.json +40 -0
  16. package/skills/cicd-pipeline/SKILL.md +1 -0
  17. package/skills/code-archaeology/SKILL.md +103 -0
  18. package/skills/code-archaeology/evals/evals.json +41 -0
  19. package/skills/compliance-privacy/SKILL.md +113 -0
  20. package/skills/compliance-privacy/evals/evals.json +41 -0
  21. package/skills/compliance-privacy/references/obligations-map.md +69 -0
  22. package/skills/data-pipeline-design/SKILL.md +116 -0
  23. package/skills/data-pipeline-design/evals/evals.json +41 -0
  24. package/skills/data-pipeline-design/references/orchestration.md +64 -0
  25. package/skills/data-quality/SKILL.md +107 -0
  26. package/skills/data-quality/evals/evals.json +39 -0
  27. package/skills/data-quality/references/tooling.md +45 -0
  28. package/skills/deployment-checklist/SKILL.md +2 -1
  29. package/skills/dx-audit/SKILL.md +97 -0
  30. package/skills/dx-audit/evals/evals.json +40 -0
  31. package/skills/feature-planning/SKILL.md +5 -1
  32. package/skills/finops-cost-optimization/SKILL.md +109 -0
  33. package/skills/finops-cost-optimization/evals/evals.json +40 -0
  34. package/skills/git-workflow/SKILL.md +10 -0
  35. package/skills/llm-app-engineering/SKILL.md +102 -0
  36. package/skills/llm-app-engineering/evals/evals.json +40 -0
  37. package/skills/llm-app-engineering/references/patterns.md +70 -0
  38. package/skills/ml-experiment-tracking/SKILL.md +1 -1
  39. package/skills/ml-pipeline-design/SKILL.md +1 -1
  40. package/skills/mobile-architecture/SKILL.md +106 -0
  41. package/skills/mobile-architecture/evals/evals.json +40 -0
  42. package/skills/mobile-release/SKILL.md +109 -0
  43. package/skills/mobile-release/evals/evals.json +41 -0
  44. package/skills/plan-execution/SKILL.md +115 -0
  45. package/skills/plan-execution/evals/evals.json +55 -0
  46. package/skills/project-documentation/SKILL.md +4 -0
  47. package/skills/release-management/SKILL.md +97 -0
  48. package/skills/release-management/evals/evals.json +56 -0
  49. package/skills/release-management/references/release-tooling.md +106 -0
  50. package/skills/resilience-engineering/SKILL.md +121 -0
  51. package/skills/resilience-engineering/evals/evals.json +40 -0
  52. package/skills/security-audit/SKILL.md +2 -2
  53. package/skills/skill-router/SKILL.md +46 -3
  54. package/skills/subagent-orchestration/SKILL.md +100 -0
  55. package/skills/subagent-orchestration/evals/evals.json +40 -0
  56. package/skills/threat-modeling/SKILL.md +101 -0
  57. package/skills/threat-modeling/evals/evals.json +40 -0
  58. package/skills/threat-modeling/references/stride-mitigations.md +90 -0
  59. package/skills/writing-skills/SKILL.md +27 -11
@@ -0,0 +1,107 @@
1
+ ---
2
+ name: data-quality
3
+ description: "Keep warehouse and pipeline data trustworthy — dbt tests, expectations (Great Expectations/Soda), data contracts with upstream producers, source freshness, schema-drift and volume anomaly detection, lineage and blast radius, severity and alerting. Triggers: data quality, bad data in dashboards, data validation, dbt tests, data contract, schema drift, stale data, freshness check, duplicate rows, nulls in the warehouse, data lineage, data SLA, upstream schema change broke us. Generating test/synthetic data → test-data-strategy; schema design → data-modeling."
4
+ model: sonnet
5
+ allowed-tools: Read, Grep, Glob, Write, Edit, Bash
6
+ ---
7
+
8
+ # Data Quality
9
+
10
+ Make bad data **loud**: caught at the boundary where it enters, blocking what
11
+ must not ship, alerting someone who owns the fix — instead of being discovered
12
+ by a stakeholder in a dashboard. Boundaries: *generating* fake/test data is
13
+ `test-data-strategy`; *designing* the schema is `data-modeling`; this skill
14
+ keeps the real data flowing through pipelines trustworthy.
15
+
16
+ ## Workflow
17
+
18
+ ### Step 1: Start From Incidents, Not Inventory
19
+
20
+ Don't blanket every column with tests. List the recent data incidents (wrong
21
+ dashboard numbers, silent breakage) and the tables where an error costs real
22
+ money or trust — revenue marts, ML features, anything an exec reads. Each
23
+ known symptom maps to a check: nulls → `not_null`, duplicates → `unique`,
24
+ staleness → freshness, broken joins → `relationships`. Wins from day one, and
25
+ the suite grows the same way tests do after bugs — every new incident adds a
26
+ check that would have caught it.
27
+
28
+ ### Step 2: Check at the Right Layer — Boundary First
29
+
30
+ Where a check runs matters as much as what it checks:
31
+
32
+ - **Ingestion boundary (sources)**: schema, freshness, volume, primary-key
33
+ integrity — catch bad inputs *before* transforms consume them. One failed
34
+ source check beats fifty downstream test failures at 6 a.m.
35
+ - **Transform layers (staging → marts)**: uniqueness/not-null on the grain of
36
+ each model, relationship tests on joins, accepted values on enums.
37
+ - **Consumer edge (marts/metrics)**: business-rule assertions — "revenue is
38
+ never negative", "row count within N% of the 7-day average".
39
+
40
+ In dbt this is: source `freshness` + source tests, generic tests on models
41
+ (`unique`, `not_null`, `relationships`, `accepted_values`), singular SQL tests
42
+ for business rules. Dedicated tools (Great Expectations, Soda) earn their keep
43
+ outside dbt-land or when non-engineers must own the rules:
44
+ [references/tooling.md](references/tooling.md).
45
+
46
+ ### Step 3: Assign Severity — What Blocks vs What Warns
47
+
48
+ A suite where everything fails the build gets muted within a month; a suite
49
+ where nothing blocks is decoration. Decide per check:
50
+
51
+ - **error** — data is unusable or contaminating (duplicate keys in a revenue
52
+ mart, failed contract at ingestion): the pipeline stops, downstream models
53
+ don't build on poisoned input.
54
+ - **warn** — degraded but usable (slightly stale, volume dip within reason):
55
+ alert, keep flowing.
56
+
57
+ dbt: `severity:` config with `error_if`/`warn_if` thresholds. Revisit
58
+ severities after each incident and each false alarm.
59
+
60
+ ### Step 4: Route Failures to an Owner
61
+
62
+ A failing check nobody sees is identical to no check. Every table (or domain)
63
+ has a named owner; alerts go where that owner works (Slack channel per domain,
64
+ pager only for `error`-severity on critical marts), and each alert says what
65
+ failed, on which table, and the blast radius (Step 5). Track
66
+ time-to-detection and time-to-fix for data incidents — that's the metric this
67
+ whole skill improves (`observability-design` for the SLO framing).
68
+
69
+ ### Step 5: Use Lineage to Scope Blast Radius
70
+
71
+ When a check fails, the first question is "what downstream is now wrong?" —
72
+ lineage (dbt's DAG, `dbt ls --select my_model+`, or a catalog tool) answers
73
+ which models, dashboards, and consumers sit downstream of the failure. Use it
74
+ to: prioritize fixes by consumer impact, notify affected dashboard owners
75
+ proactively, and decide what to rebuild after the fix. If you can't answer
76
+ "what breaks if this table is wrong?", invest in lineage before more checks.
77
+
78
+ ### Step 6: Contract the Upstream Boundary
79
+
80
+ Downstream tests detect what upstream producers break; **contracts prevent
81
+ it.** When an external team owns the source: agree schema + semantics + SLA
82
+ explicitly (a versioned artifact — dbt model contracts, JSON Schema/protobuf in
83
+ a registry), enforce at ingestion so violations fail loudly at entry, and shift
84
+ left — contract validation in the *producer's* CI, so a breaking rename fails
85
+ their build, not your 6 a.m. run. Breaking changes get a version bump and a
86
+ deprecation window, coordinated like an API change
87
+ (`dependency-impact-analysis` thinking, applied to data). Back it with
88
+ schema-drift detection and volume/distribution anomaly checks for what
89
+ enforcement can't see.
90
+
91
+ ## Principles Applied
92
+
93
+ - **YAGNI**: checks earn their place by a failure mode you've seen or can't
94
+ afford — blanket-testing every column produces alert fatigue, the failure
95
+ mode that kills quality programs.
96
+ - **Fail loudly at the boundary**: the cost of bad data grows with every layer
97
+ it flows through undetected.
98
+ - **DRY**: one contract at the source replaces N teams' defensive checks
99
+ against the same breakage.
100
+
101
+ ## Cross-Skill References
102
+
103
+ - `data-pipeline-design` — the pipeline these checks are embedded in
104
+ - `data-modeling` — constraints and grain the checks assert (unique keys exist by design)
105
+ - `test-data-strategy` — generating synthetic/fixture data for testing pipelines
106
+ - `observability-design` — alerting design, SLOs, avoiding alert fatigue
107
+ - `incident-response` — when bad data has already reached production consumers
@@ -0,0 +1,39 @@
1
+ {
2
+ "skill_name": "data-quality",
3
+ "evals": [
4
+ {
5
+ "id": 1,
6
+ "prompt": "Stakeholders keep finding bad numbers in our dashboards — null customer IDs, duplicated orders, and data that's a day stale. We're on dbt + Snowflake. Set up data quality checks.",
7
+ "expected_output": "Maps each symptom to a concrete check (not_null/unique dbt tests, source freshness), places checks at the ingestion boundary first and critical marts second, assigns severity (error blocks, warn alerts), routes failures to an owner, and uses lineage to scope which downstream models and dashboards a failure affects",
8
+ "assertions": [
9
+ "Maps each reported symptom to a concrete check: not_null/unique dbt tests for the IDs and duplicates, and source freshness checks for the staleness",
10
+ "Places checks at the right layers: the source/ingestion boundary first, then critical marts — not only on the final dashboards",
11
+ "Assigns severity levels (error vs warn) and defines what blocks the pipeline versus what only alerts",
12
+ "Sets up alerting and ownership so failures reach someone who acts, instead of failing silently",
13
+ "Uses lineage to scope blast radius — which downstream models or dashboards a failing check affects"
14
+ ]
15
+ },
16
+ {
17
+ "id": 2,
18
+ "prompt": "An upstream team owns the events we ingest and keeps renaming columns and changing types without telling us. Our pipeline breaks silently every few weeks. What should we put in place?",
19
+ "expected_output": "Proposes a producer/consumer data contract (schema, types, semantics, SLA) enforced at the ingestion boundary so violations fail loudly at entry; shifts validation left into the producer's CI or publish path; adds versioning/deprecation windows for breaking changes; and backstops enforcement with schema-drift and volume/distribution anomaly detection",
20
+ "assertions": [
21
+ "Proposes an explicit data contract between producer and consumer (schema, types, semantics, SLAs), not just more downstream tests",
22
+ "Enforces the schema at the ingestion boundary so violations fail loudly at entry instead of propagating silently",
23
+ "Shifts checks left: contract validation in the producer's CI or at publish time",
24
+ "Covers change management for breaking changes: contract versioning or a deprecation window",
25
+ "Adds detection for what enforcement misses: schema-drift alerts or anomaly checks on volume/distributions"
26
+ ]
27
+ },
28
+ {
29
+ "id": 3,
30
+ "prompt": "We need realistic but fake customer data to seed staging so QA can test the checkout flow — GDPR means we can't copy production.",
31
+ "expected_output": "Recognizes this as generating synthetic/test data, not validating production data — hands off to test-data-strategy (factories, synthetic generation, anonymization) rather than proposing expectations, contracts, or freshness checks",
32
+ "assertions": [
33
+ "Recognizes this is about generating test/synthetic data, not validating production data quality",
34
+ "Refers to the test-data-strategy skill for factories, synthetic data generation, and GDPR-safe anonymization",
35
+ "Does not answer with data-quality machinery (expectations, contracts, freshness checks) as the solution"
36
+ ]
37
+ }
38
+ ]
39
+ }
@@ -0,0 +1,45 @@
1
+ # Data Quality Tooling — Selection Notes
2
+
3
+ Tool capabilities shift quickly; verify against current docs. The placement
4
+ logic is stable.
5
+
6
+ ## Picking the enforcement layer
7
+
8
+ | Tool | Shape | Choose when |
9
+ |---|---|---|
10
+ | **dbt tests** | YAML generic tests + singular SQL tests, run with the transforms | You're already on dbt. Zero new infrastructure; tests version with the models; severity/thresholds built in. The default. |
11
+ | **dbt model contracts** | Enforced column names/types on a model; breaks the *producer's* build | A dbt model is consumed across team boundaries — the contract fails the producing project's CI, which is the shift-left goal. |
12
+ | **Great Expectations** | Python expectation suites + data docs, runs anywhere | Checks must run outside the warehouse/dbt (files, in-flight DataFrames, non-dbt pipelines), or you need its rich expectation vocabulary and generated documentation. Heavier to operate. |
13
+ | **Soda (Core/Cloud)** | Declarative checks YAML (SodaCL), CLI + SaaS | Non-engineers own quality rules, or checks span multiple stores with one syntax; Cloud adds anomaly detection and incident routing. |
14
+ | **Elementary / observability platforms (Monte Carlo, Metaplane…)** | Monitor the warehouse from metadata: freshness, volume, schema drift, distribution anomalies | You want detection you didn't hand-write — anomaly monitoring across many tables — and (platforms) column-level lineage + incident management. Elementary is the dbt-native OSS entry point. |
15
+
16
+ Layering that works in practice: **dbt tests + source freshness** as the
17
+ foundation; **contracts** at team boundaries; **one anomaly/observability
18
+ layer** for the unknown-unknowns. Adding a second tool in the same layer buys
19
+ maintenance, not coverage.
20
+
21
+ ## Contract mechanics by boundary type
22
+
23
+ - **Event streams**: schema registry (Avro/Protobuf/JSON Schema) with
24
+ compatibility mode set (backward at minimum); producers can't publish a
25
+ breaking schema. The strongest enforcement available — use it if events are
26
+ the interface.
27
+ - **Warehouse-to-warehouse (dbt ↔ dbt)**: dbt model contracts + versioned
28
+ models (`v2` with a deprecation window) on the shared models.
29
+ - **API/file drops from another org**: JSON Schema validated at ingestion;
30
+ reject-and-alert (dead-letter the bad batch) rather than best-effort
31
+ parsing.
32
+ - The social half matters as much: a named producer contact, a change-notice
33
+ channel, and the contract in a repo both teams can PR.
34
+
35
+ ## Anomaly checks worth having before "ML-powered" anything
36
+
37
+ - Row count vs trailing 7/28-day window (volume drops are the most common
38
+ silent failure).
39
+ - Freshness vs declared SLA per source.
40
+ - Schema diff vs last run (added/removed/retyped columns) with an explicit
41
+ allowlist flow.
42
+ - Null-rate and distinct-count drift on business-critical columns.
43
+
44
+ These four catch the large majority of real incidents; statistical/ML anomaly
45
+ detection is a refinement, not a starting point.
@@ -69,7 +69,7 @@ Walk through each section. Check items by reading code, running commands, and ve
69
69
  #### Documentation
70
70
  - [ ] README updated if setup steps changed
71
71
  - [ ] API documentation updated for new/changed endpoints
72
- - [ ] CHANGELOG updated (if project uses one)
72
+ - [ ] Release cut correctly, if this deploy ships one — version, changelog, tag consistent (see `release-management`)
73
73
  - [ ] Breaking changes are clearly communicated
74
74
 
75
75
  ### Step 3: Report
@@ -126,6 +126,7 @@ See [references/pre-deploy-gates.md](references/pre-deploy-gates.md) for environ
126
126
 
127
127
  ## Cross-Skill References
128
128
 
129
+ - `release-management` — cutting the versioned release (semver, changelog, tag, publish) that this checklist deploys
129
130
  - `rollback-strategy` — design the rollback plan before deploying (required for High-risk changes)
130
131
  - `configuration-strategy` — verify all environment variables and feature flags are configured correctly
131
132
  - `incident-response` — use if the deployment causes a production incident
@@ -0,0 +1,97 @@
1
+ ---
2
+ name: dx-audit
3
+ description: "Audit and improve developer experience — inner-loop feedback (build/test/reload times), CI wait and queue times, flaky tests, local environment setup, onboarding time-to-first-PR, tooling friction, docs discoverability; produce a prioritized remediation plan. Triggers: developer experience, DX audit, devex, builds are slow, CI takes forever, flaky tests, onboarding takes weeks, tooling friction, developer productivity, engineers are frustrated. Code health/debt hotspots → technical-debt-review; designing one pipeline → cicd-pipeline."
4
+ model: sonnet
5
+ allowed-tools: Read, Grep, Glob, Write, Edit, Bash
6
+ ---
7
+
8
+ # DX Audit
9
+
10
+ Find and fix the friction that taxes every engineer, every day. Developer
11
+ experience compounds: a 10-minute CI queue on a 20-person team burns weeks of
12
+ engineering time per month, and worse, it changes *behavior* — engineers batch
13
+ changes, skip test runs, and avoid touching areas with slow feedback.
14
+ Boundary: `technical-debt-review` audits the *code's* health; this skill audits
15
+ the *workflow around* the code — loops, tooling, environments, and onboarding.
16
+
17
+ ## Workflow
18
+
19
+ ### Step 1: Measure the Loops — Numbers Before Anecdotes
20
+
21
+ Quantify the feedback loops before collecting opinions, so the audit ranks by
22
+ data and the improvement is provable later:
23
+
24
+ - **Inner loop**: cold build, incremental build, single-test run, hot-reload —
25
+ time them yourself in the actual repo.
26
+ - **Outer loop**: CI wall-clock (queue + run, from real recent pipelines), PR
27
+ time-to-first-review and time-to-merge, deploy lead time (DORA overlap —
28
+ `metrics-and-okrs`).
29
+ - **Flake rate**: retries and "re-run job" clicks per week; a 5%-flaky suite
30
+ means engineers already ignore red.
31
+ - **Onboarding**: time from laptop to first merged PR for recent joiners.
32
+
33
+ ### Step 2: Collect the Friction Log
34
+
35
+ Then ask the engineers — a lightweight survey or five 15-minute conversations:
36
+ "what wastes your time every day?", "what do you avoid doing because it's
37
+ painful?", "what would you fix first?". The second question matters most:
38
+ avoidance signals (never running the full suite locally, dreading a service's
39
+ local setup, batching deploys) reveal costs the timers can't see. Recurring
40
+ themes + Step-1 numbers = the finding list.
41
+
42
+ ### Step 3: Audit the Inner Loop
43
+
44
+ The highest-frequency loop gets the deepest look: Is the dev environment
45
+ reproducible in one command (or is setup a wiki page of drift-prone steps)?
46
+ Can you run one test in seconds without the world? Does hot-reload actually
47
+ work? Is the IDE experience functional (types, go-to-def, lint-on-save)? Are
48
+ common tasks (reset DB, seed data, run service X) scripted or tribal
49
+ knowledge? Every manual step here multiplies by invocations-per-day.
50
+
51
+ ### Step 4: Audit the Outer Loop
52
+
53
+ - **CI**: queue time vs run time (capacity vs speed problems differ), cache
54
+ hit rates, test parallelization/sharding, whether PRs run only affected
55
+ targets — delegate pipeline redesign to `cicd-pipeline`.
56
+ - **Flakes**: quarantine policy (a flaky test in the merge gate is a tax on
57
+ everyone), ownership for fixing, `test-suite-design` for structural causes.
58
+ - **Review latency**: are PRs waiting on people or on process (required
59
+ approvals, codeowner bottlenecks)?
60
+ - **Docs**: can an engineer answer "how do I X here?" without interrupting a
61
+ senior — README freshness, runbooks, `project-documentation`.
62
+
63
+ ### Step 5: Prioritize by Tax, Not by Annoyance
64
+
65
+ Score each finding as **time-cost × frequency × people affected** (daily
66
+ 10-second cuts beat monthly 10-minute ones), weighted by behavior distortion
67
+ (anything that makes engineers *skip verification* is top-tier regardless of
68
+ seconds). Produce the remediation list: quick wins first (cache config, test
69
+ sharding, a `make dev` script) to build credibility, structural items
70
+ (environment overhaul, CI capacity) as scheduled work with owners.
71
+
72
+ ### Step 6: Prove It and Keep It
73
+
74
+ Re-measure the Step-1 numbers after the fixes — DX work is uniquely easy to
75
+ verify, so verify it. Then keep the loop honest: track 2–3 DX metrics
76
+ continuously (CI p50/p95, flake rate, onboarding time), set a budget
77
+ ("CI > 10 min is a defect"), and re-run the friction survey a couple of times
78
+ a year. DX decays by default — every new service adds setup steps and CI
79
+ minutes unless something pushes back.
80
+
81
+ ## Principles Applied
82
+
83
+ - **Frequency beats magnitude**: optimize the loop that runs 50×/day before
84
+ the one that runs weekly.
85
+ - **Avoidance is the loudest signal**: what engineers won't do reveals more
86
+ than what they complain about.
87
+ - **Measured before and after**: a DX improvement that can't show the numbers
88
+ moved is an opinion.
89
+
90
+ ## Cross-Skill References
91
+
92
+ - `technical-debt-review` — the code-health complement to this workflow audit
93
+ - `cicd-pipeline` — redesigning the pipeline the audit flagged
94
+ - `test-suite-design` — structural fixes for slow or flaky suites
95
+ - `project-documentation` — the docs/onboarding gaps surfaced in Step 4
96
+ - `metrics-and-okrs` — DORA metrics and making DX goals measurable
97
+ - `retrospective` — mining recurring friction from team retros
@@ -0,0 +1,40 @@
1
+ {
2
+ "skill_name": "dx-audit",
3
+ "evals": [
4
+ {
5
+ "id": 1,
6
+ "prompt": "Our 25-person engineering team feels slow. CI takes ~35 minutes, people complain about local setup, and new hires seem to take forever to become productive. Run a DX audit and tell us what to fix.",
7
+ "expected_output": "Measures the loops first (inner-loop timings, CI queue vs run time, flake rate, PR cycle time, onboarding time-to-first-merge), collects a friction log from engineers with attention to avoidance behavior, audits inner loop (one-command env, single-test speed) and outer loop (CI caching/sharding, flake quarantine, review latency, docs), prioritizes by time-cost × frequency × people with quick wins first, and commits to re-measuring the same numbers after fixes",
8
+ "assertions": [
9
+ "Quantifies the feedback loops with real measurements (build/test times, CI queue vs run, flake rate, PR cycle, onboarding time) before prescribing fixes",
10
+ "Collects friction from the engineers themselves (survey/interviews) and pays attention to avoidance behavior, not just complaints",
11
+ "Audits both the inner loop (env setup, single-test runs, hot reload, scripted common tasks) and the outer loop (CI caching/parallelization, flaky-test policy, review latency, docs)",
12
+ "Prioritizes findings by frequency × time-cost × people affected, sequencing quick wins before structural work, with owners",
13
+ "Plans to re-measure the baseline numbers after remediation and keep 2–3 DX metrics tracked continuously"
14
+ ]
15
+ },
16
+ {
17
+ "id": 2,
18
+ "prompt": "Management says our DX is fine because nobody's complaining, but I notice engineers never run the full test suite locally, deploys pile up on Fridays, and everyone SSHes into a shared staging box to test things. Is there a DX problem here?",
19
+ "expected_output": "Reads the avoidance signals as the loudest DX evidence: skipping the local suite implies it's too slow/flaky (verification is being skipped — top-tier severity), deploy batching implies the deploy loop is painful or risky, shared-staging testing implies local environments can't reproduce reality; quantifies each to confirm, and explains why absence of complaints doesn't mean absence of tax (normalized friction becomes invisible)",
20
+ "assertions": [
21
+ "Identifies the described behaviors as avoidance signals that indicate DX problems despite the absence of complaints",
22
+ "Infers the likely underlying causes: local suite too slow or flaky, painful/risky deploy loop, local environments unable to reproduce reality",
23
+ "Flags that engineers skipping verification (not running tests locally) is especially severe because it distorts quality behavior",
24
+ "Proposes measuring to confirm (suite runtime, flake rate, deploy lead time, env setup) rather than acting on inference alone",
25
+ "Explains why normalized friction stops generating complaints (habituation), so surveys must ask about avoidance, not just satisfaction"
26
+ ]
27
+ },
28
+ {
29
+ "id": 3,
30
+ "prompt": "Our checkout module is a nightmare of spaghetti code — cyclomatic complexity through the roof, no tests, God classes everywhere. Audit it and give us a prioritized cleanup roadmap.",
31
+ "expected_output": "Recognizes this as a code-health/debt assessment, not a workflow-friction audit — refers to technical-debt-review for the hotspot analysis and remediation roadmap, stating the boundary: dx-audit covers the workflow around the code (loops, tooling, environments); technical-debt-review covers the code itself",
32
+ "assertions": [
33
+ "Recognizes the request as a code-health/technical-debt assessment, not a developer-workflow audit",
34
+ "Refers to the technical-debt-review skill for the hotspot analysis and cleanup roadmap",
35
+ "Does not produce a DX workflow audit (CI timings, onboarding, tooling friction) as the answer",
36
+ "States or implies the boundary: dx-audit audits the workflow around the code; technical-debt-review audits the code's health"
37
+ ]
38
+ }
39
+ ]
40
+ }
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: feature-planning
3
- description: "Break features into well-scoped tasks with acceptance criteria, risk assessment, and dependency mapping. Triggers: plan this, break this down, scope this feature, create tasks for, sprint planning, how should I implement this feature, user stories, acceptance criteria, dependency mapping, feature breakdown."
3
+ description: "Break features into well-scoped tasks with acceptance criteria, risk assessment, and dependency mapping. Triggers: plan this, break this down, scope this feature, create tasks for, sprint planning, how should I implement this feature, user stories, acceptance criteria, dependency mapping, feature breakdown. Creates the plan — executing an already-approved plan checkpoint by checkpoint → plan-execution."
4
4
  model: sonnet
5
5
  allowed-tools: Read, Grep, Glob, Write, Edit
6
6
  ---
@@ -82,3 +82,7 @@ Output the complete plan using the template at [templates/plan.md](templates/pla
82
82
  If the feature involves architectural decisions (new services, database changes, API design), suggest using the `architecture-design` skill before finalizing the plan.
83
83
 
84
84
  If the feature requires data model changes, suggest using the `data-modeling` skill for that portion.
85
+
86
+ If the idea itself is still fuzzy — the user is exploring what to build rather than how — run `brainstorming` first; this skill converges, that one diverges.
87
+
88
+ Once the plan is approved and execution begins, the `plan-execution` skill takes over: checkpointed batches, verification evidence per checkpoint, and drift-triggered re-planning (which loops back here).
@@ -0,0 +1,109 @@
1
+ ---
2
+ name: finops-cost-optimization
3
+ description: "Understand and reduce cloud spend — cost visibility and allocation (tagging, ownership), unit economics, rightsizing, autoscaling, commitment discounts (reserved instances, savings plans), storage/egress traps, architecture cost review, budgets and anomaly alerts. Triggers: cloud costs, AWS bill, cut costs, cost optimization, FinOps, why is our bill so high, rightsizing, reserved instances, savings plan, egress fees, cost per customer. Whether to build or buy a capability → build-vs-buy; making code faster → performance-optimization."
4
+ model: sonnet
5
+ allowed-tools: Read, Grep, Glob, Write, Edit, Bash, WebFetch, WebSearch
6
+ ---
7
+
8
+ # FinOps & Cost Optimization
9
+
10
+ Treat cloud cost as an engineering metric — observable, attributed, and
11
+ optimized with the same rigor as latency. The failure mode this skill counters:
12
+ nobody owns the bill, so it only gets attention as a quarterly surprise, and
13
+ the response is a panicked freeze instead of systematic reduction. Boundaries:
14
+ `performance-optimization` makes code faster (sometimes cheaper as a side
15
+ effect); `build-vs-buy` decides whether to run the capability at all; this
16
+ skill optimizes what you *do* run.
17
+
18
+ ## Workflow
19
+
20
+ ### Step 1: Visibility Before Optimization
21
+
22
+ You can't cut what you can't attribute. Establish first:
23
+
24
+ - **Allocation**: tag/label resources by service, team, and environment;
25
+ measure the untagged remainder and drive it down — an 40%-unattributed bill
26
+ makes every later step guesswork.
27
+ - **The big rocks**: rank spend by service and by resource type. Cloud bills
28
+ follow a power law; the top 5 line items usually carry most of the total.
29
+ Optimizing anything outside them is procrastination with charts.
30
+ - **Trend vs events**: is spend growing with usage (fine, if unit costs hold),
31
+ or did it step-change with a deploy (an incident with a dollar signature)?
32
+
33
+ Pull real billing data (cost explorer exports, `infracost` on IaC diffs, the
34
+ provider CLIs) — never optimize from memory of last quarter's bill.
35
+
36
+ ### Step 2: Define Unit Economics
37
+
38
+ Total spend is noise; **cost per unit of value** — per customer, per request,
39
+ per GB processed, per model inference — is signal. A bill that doubles while
40
+ customers triple is a win. Pick 1–3 unit metrics with the team, compute them
41
+ from the Step-1 allocation, and make them the number optimization is judged by.
42
+ This is also what connects engineering to the business conversation
43
+ (`metrics-and-okrs`).
44
+
45
+ ### Step 3: Harvest the Waste (no architecture changes)
46
+
47
+ The reliable first wins, in typical order of return:
48
+
49
+ - **Orphans**: unattached volumes/IPs, forgotten environments, snapshots-forever,
50
+ idle dev clusters running nights and weekends (schedule them off).
51
+ - **Rightsizing**: instances/databases provisioned for imagined peak — check
52
+ actual utilization (single-digit CPU on a large instance is a purchase order
53
+ for the cloud provider, not capacity planning).
54
+ - **Storage classes & lifecycle**: hot-tier data nobody has read in a year;
55
+ logs and backups with no expiry — lifecycle policies are one-line fixes.
56
+ - **Non-prod parity**: staging doesn't need production's instance sizes,
57
+ replica counts, or retention.
58
+
59
+ ### Step 4: Buy Commitment for the Stable Baseline
60
+
61
+ After rightsizing (never before — you'd lock in the waste): cover the stable,
62
+ predictable baseline with reserved instances / savings plans / committed-use
63
+ discounts, and keep the spiky remainder on-demand or spot (spot for
64
+ interruption-tolerant work: batch, CI, stateless workers). Start conservative
65
+ (cover well under 100% of the baseline) and ratchet up as forecasts prove out —
66
+ over-commitment converts a variable cost into a fixed regret.
67
+
68
+ ### Step 5: Review the Architecture's Cost Shape
69
+
70
+ Some spend is structural, and only architecture changes move it:
71
+
72
+ - **Egress and cross-zone/region chatter** — the classic invisible line item;
73
+ co-locate chatty services, cache at the edge, compress.
74
+ - **Per-invocation pricing at scale** — serverless that grew past the
75
+ crossover where instances are cheaper (and vice versa: idle instances that
76
+ should be serverless).
77
+ - **Managed-premium vs ops-burden** — the managed service's markup is often
78
+ worth it (that's a `build-vs-buy` call when it gets big).
79
+ - **Data pipelines** — scan-based pricing (warehouse queries over unpartitioned
80
+ tables) and always-on streaming for hourly-batch needs (`data-pipeline-design`).
81
+
82
+ Cost these with current provider pricing (verify live — pricing changes), and
83
+ record significant trade-offs as ADRs (`architecture-design`).
84
+
85
+ ### Step 6: Keep It Continuous
86
+
87
+ - **Budgets + anomaly alerts** per team/service, routed to the owning team —
88
+ a cost regression is a defect, detected in days not quarters.
89
+ - **Cost in the deploy loop**: IaC cost diffs (e.g. infracost) in PRs makes
90
+ cost visible at decision time.
91
+ - **A monthly 30-minute review** of the unit metrics and top movers beats an
92
+ annual cost-cutting project.
93
+
94
+ ## Principles Applied
95
+
96
+ - **Attribution before optimization** — unowned costs never stay cut.
97
+ - **Unit economics over totals** — growth should raise the bill; only rising
98
+ *unit* cost is a problem.
99
+ - **Biggest line item first** — a 10% cut of the top item beats a 90% cut of
100
+ the tenth.
101
+
102
+ ## Cross-Skill References
103
+
104
+ - `build-vs-buy` — whether to run the capability at all; managed-vs-self-hosted TCO
105
+ - `performance-optimization` — efficiency work that reduces compute per request
106
+ - `architecture-design` — recording cost-driven structural decisions as ADRs
107
+ - `observability-design` — the metrics/alerting machinery cost signals ride on
108
+ - `data-pipeline-design` — cost-shaping batch/streaming and warehouse workloads
109
+ - `metrics-and-okrs` — tying unit economics to business metrics
@@ -0,0 +1,40 @@
1
+ {
2
+ "skill_name": "finops-cost-optimization",
3
+ "evals": [
4
+ {
5
+ "id": 1,
6
+ "prompt": "Our AWS bill jumped from $18k to $31k/month over the last two quarters and leadership wants it 'fixed'. Where do we start?",
7
+ "expected_output": "Starts with visibility, not cuts: attribute the spend (tagging/allocation), rank the top line items, and separate usage-driven growth from step-change regressions; defines unit economics (cost per customer/request) as the real judge; then harvests waste in order of return (orphans, rightsizing from actual utilization, storage lifecycle, non-prod schedules), only then commitment discounts on the post-rightsizing baseline, plus budgets/anomaly alerts so the next jump is caught in days",
8
+ "assertions": [
9
+ "Establishes visibility and attribution first (tagging/allocation, ranking top line items from real billing data) before proposing any cuts",
10
+ "Distinguishes usage-driven growth from step-change regressions and/or introduces unit economics (cost per customer/request) rather than treating the total as the metric",
11
+ "Proposes concrete waste harvesting: orphaned resources, rightsizing based on actual utilization, storage lifecycle policies, or scheduling non-prod environments off",
12
+ "Recommends commitment discounts (RIs/savings plans) only after rightsizing, covering a conservative stable baseline",
13
+ "Sets up continuous guardrails: budgets and anomaly alerts routed to owning teams, and/or cost checks in the deploy loop"
14
+ ]
15
+ },
16
+ {
17
+ "id": 2,
18
+ "prompt": "Our bill is dominated by data transfer: about 40% is inter-AZ traffic and NAT gateway charges between our microservices, and the rest of the top items are a Kafka cluster and a data warehouse where queries scan whole tables. Rightsizing won't move these. What are our options?",
19
+ "expected_output": "Recognizes structural/architectural cost shape: co-locating chatty services or reworking cross-AZ topology (weighing the availability trade-off), VPC endpoints / NAT alternatives, compression; warehouse scan costs via partitioning/clustering and incremental models; evaluates whether always-on streaming matches the actual latency need; recommends costing options with live pricing and recording the trade-offs as ADRs since these decisions trade resilience/latency for money",
20
+ "assertions": [
21
+ "Recognizes these as architecture-shaped costs that rightsizing cannot fix and addresses the egress/inter-AZ problem specifically (co-location, topology, VPC endpoints, compression)",
22
+ "Notes the availability/resilience trade-off in reducing cross-AZ traffic rather than recommending single-AZ blindly",
23
+ "Addresses the warehouse scan costs structurally (partitioning, clustering, incremental models) rather than only suggesting fewer queries",
24
+ "Questions whether the always-on streaming infrastructure matches the actual latency requirement (batch vs streaming)",
25
+ "Recommends verifying costs against current provider pricing and/or recording significant cost-driven decisions as ADRs"
26
+ ]
27
+ },
28
+ {
29
+ "id": 3,
30
+ "prompt": "Our checkout API's p99 latency is 4 seconds and profiling shows an N+1 query pattern plus a missing index. Can you help optimize it?",
31
+ "expected_output": "Recognizes this as code/query performance work, not cloud-spend optimization — refers to performance-optimization for the N+1 and indexing work, noting the boundary: performance-optimization makes code faster (sometimes cheaper as a side effect); finops-cost-optimization attributes and reduces cloud spend",
32
+ "assertions": [
33
+ "Recognizes the request as application performance optimization, not cloud cost management",
34
+ "Refers to the performance-optimization skill for the N+1/indexing work",
35
+ "Does not produce a FinOps workflow (tagging, rightsizing, commitments) as the answer",
36
+ "States or implies the boundary: performance-optimization owns making code fast; finops-cost-optimization owns understanding and reducing spend"
37
+ ]
38
+ }
39
+ ]
40
+ }
@@ -52,6 +52,10 @@ Use the Conventional Commits format — see [references/conventions.md](referenc
52
52
  - Breaking changes: `BREAKING CHANGE: description`
53
53
  - Co-authors: `Co-authored-by: Name <email>`
54
54
 
55
+ Commit types are not just style — in automated release flows they drive the version:
56
+ `fix` → PATCH, `feat` → MINOR, `feat!`/`BREAKING CHANGE:` → MAJOR. A mislabeled type
57
+ mis-versions the next release (see `release-management`).
58
+
55
59
  ### Step 3: Validate
56
60
 
57
61
  Before committing, check:
@@ -125,3 +129,9 @@ See [references/conventions.md](references/conventions.md) for branch naming con
125
129
  - **KISS**: One commit = one logical change. Don't bundle unrelated changes.
126
130
  - **DRY**: If you're writing the same commit message pattern repeatedly, you might be making commits too granular or too broad.
127
131
  - **Functional Independence**: Each PR should be independently deployable when possible. Avoid PRs that depend on other unmerged PRs.
132
+
133
+ ## Cross-Skill References
134
+
135
+ - `release-management` — cutting the release these commits accumulate into: semver, changelog, tagging, publishing
136
+ - `code-reviewing` — reviewing the PR the description introduces
137
+ - `verification-before-completion` — run the proving commands before pushing or opening the PR
@@ -0,0 +1,102 @@
1
+ ---
2
+ name: llm-app-engineering
3
+ description: "Design and build LLM-powered applications — prompt and context engineering, RAG architecture (chunking, hybrid retrieval, reranking, grounded prompts), agent design (tool surfaces, planning, sub-agents), and memory. Triggers: build a chatbot, LLM app, AI assistant, prompt engineering, system prompt, RAG design, chunking, embeddings, vector database, semantic search, AI agent, tool calling, agent memory, context window, hallucinations. Measuring quality → ai-evaluation; model serving/inference infra → ml-model-deployment."
4
+ model: opus
5
+ allowed-tools: Read, Grep, Glob, Write, Edit, Bash
6
+ ---
7
+
8
+ # LLM App Engineering
9
+
10
+ Design LLM applications as **systems, not prompts**: what goes into the context
11
+ window, what the model is allowed to do, and what happens when it's wrong. The
12
+ model is the one component you don't control — engineer everything around it.
13
+ This skill designs and builds; proving a change helped is `ai-evaluation`'s job,
14
+ and no design below is "done" until it has an eval harness.
15
+
16
+ ## Workflow
17
+
18
+ ### Step 1: Choose the Simplest Sufficient Architecture
19
+
20
+ Escalate only when the previous rung measurably fails:
21
+
22
+ 1. **Single prompt** — the task fits in one call with instructions + input.
23
+ 2. **+ RAG** — the answer needs knowledge that doesn't fit in the prompt or
24
+ changes independently of deploys.
25
+ 3. **+ Tools** — the model must act (query, search, write) — not just answer.
26
+ 4. **Agent loop** — the path to the goal is genuinely unpredictable in advance.
27
+ Most "agents" that fail are workflows that should have been a fixed pipeline
28
+ with one or two LLM steps.
29
+
30
+ ### Step 2: Engineer the Prompt as an Interface
31
+
32
+ - Structure: role and constraints → domain context → task → output format.
33
+ Put stable content first (prompt caching) and the volatile input last.
34
+ - State what to do on failure: "if the context doesn't contain the answer, say
35
+ so" — the unhappy path is where LLM apps lose user trust.
36
+ - Version prompts in git like code; a prompt change is a deploy and goes
37
+ through the same eval gate (`ai-evaluation`).
38
+
39
+ ### Step 3: RAG — Design Retrieval Before Generation
40
+
41
+ Most bad RAG answers are retrieval failures wearing a generation costume.
42
+
43
+ - **Chunking**: split on document structure (headings, sections, one
44
+ ticket+resolution per chunk), not fixed character counts; attach metadata
45
+ (source, date, product area) for filtering and citations.
46
+ - **Retrieval**: hybrid (BM25 + vector) as the default — pure vector search
47
+ misses exact identifiers, SKUs, and error codes; filter by metadata, then
48
+ **rerank** the candidate set (retrieve ~20–50, keep the top 5–10).
49
+ - **Context budget** (decide, don't default): how many chunks fit, in what
50
+ order (most relevant near the question), the relevance floor below which a
51
+ chunk is dropped, and the behavior when nothing clears it — say "I don't
52
+ know" rather than pad the context with noise.
53
+ - **Grounding prompt**: answer only from the provided context, cite which
54
+ source supports each claim, refuse when unsupported.
55
+
56
+ ### Step 4: Agents — Design the Tool Surface and the Loop
57
+
58
+ - **Tools**: few and sharply distinct beat many and overlapping — every tool
59
+ description competes for the model's attention on every step. Name by
60
+ intent (`search_orders`, not `api_call`); make each tool's failure return a
61
+ message the model can act on, not a stack trace.
62
+ - **Decomposition**: long tasks fail as one monolithic loop with 25 tools;
63
+ split into phases or sub-agents, each with a scoped toolset, and pass
64
+ structured summaries between them — not raw transcripts.
65
+ - **Guardrails**: cap iterations, budget tokens per task, and validate
66
+ side-effecting tool calls (or require confirmation) — an agent retrying a
67
+ failed write in a loop is the classic production incident.
68
+
69
+ ### Step 5: Memory — Externalize State
70
+
71
+ The context window is a scarce cache, not a database. For long-horizon work:
72
+ keep a plan/scratchpad file the agent re-reads and updates; summarize or
73
+ compact stale history instead of letting it scroll off; store durable facts
74
+ (user preferences, decisions) outside the window and retrieve them like RAG.
75
+
76
+ ### Step 6: Wire In Evaluation Before Shipping
77
+
78
+ Hand the design to `ai-evaluation`: a golden dataset (including unanswerable
79
+ and adversarial cases), RAG-stage metrics (retrieval vs generation scored
80
+ separately), agent task-success rate, and a CI regression gate. "The answers
81
+ feel better" is not evidence; a design without an eval harness is a prototype.
82
+
83
+ Deeper pattern detail (chunking table, reranking, memory patterns, failure
84
+ modes): [references/patterns.md](references/patterns.md).
85
+
86
+ ## Principles Applied
87
+
88
+ - **KISS**: every architecture rung you skip is a class of failure you never
89
+ have to debug. Workflows beat agents when the path is predictable.
90
+ - **YAGNI**: no vector DB before checking whether the corpus fits in the
91
+ prompt; no agent framework before a plain loop with 3 tools fails.
92
+ - **SRP for tools**: one tool, one capability — overlapping tools force the
93
+ model to guess, and it will guess differently each run.
94
+
95
+ ## Cross-Skill References
96
+
97
+ - `ai-evaluation` — golden datasets, RAG metrics, judges, eval gates (the
98
+ measurement half of every step above)
99
+ - `ml-model-deployment` — serving, monitoring, and drift once the app ships
100
+ - `api-design` — the contract around the LLM feature (streaming, errors, timeouts)
101
+ - `security-audit` — prompt injection, data exfiltration via tools, output handling
102
+ - `architecture-design` — ADRs for costly-to-reverse choices (vector store, framework)