swe-workflow-skills 0.1.0 → 0.2.0

package/skills/ai-evaluation/evals/evals.json

@@ -0,0 +1,40 @@
+{
+  "skill_name": "ai-evaluation",
+  "evals": [
+    {
+      "id": 1,
+      "prompt": "We've built a RAG chatbot over our internal docs and we're planning to launch in two weeks. How should we evaluate it before launch? Someone on the team mentioned ragas.",
+      "expected_output": "Designs an evaluation plan: a golden dataset grounded in real user questions, the RAG triad (faithfulness, answer relevance, context precision/recall) separating retrieval from generation metrics, a concrete tool choice (ragas/deepeval/promptfoo) with rationale, LLM-as-judge reliability measures, and the eval suite wired in as a regression gate rather than a one-off check",
+      "assertions": [
+        "Proposes building a golden/evaluation dataset grounded in real user questions or the docs, rather than evaluating ad hoc",
+        "Covers the RAG triad or equivalent — faithfulness/groundedness, answer relevance, and retrieval/context quality — and distinguishes retrieval metrics from generation metrics",
+        "Recommends a concrete evaluation tool (ragas, deepeval, or promptfoo) and explains its role",
+        "Addresses LLM-as-judge reliability with at least one concrete measure: rubric design, position/self-preference bias, majority voting, or calibrating the judge against human spot-checks",
+        "Proposes running the evals as a repeatable regression gate (in CI or before each prompt/pipeline change), not only as a one-off pre-launch check"
+      ]
+    },
+    {
+      "id": 2,
+      "prompt": "Our LLM feature summarizes long sales calls. We have no ground-truth summaries and no users yet. How do we evaluate quality with zero labeled data?",
+      "expected_output": "Does not treat missing labels as a blocker: proposes reference-free evaluation (rubric-based LLM-as-judge, groundedness against the source call, pairwise comparison between variants), bootstraps a small human-labeled golden set, calibrates the judge against human spot-checks, and plans online signals (feedback, A/B) once users exist",
+      "assertions": [
+        "Does not claim evaluation is impossible without labels — proposes reference-free approaches such as rubric-based LLM-as-judge, groundedness against the source transcript, or pairwise comparison between variants",
+        "Recommends bootstrapping a small human-labeled golden set (even tens of examples) and explains what it buys",
+        "Designs the judge with an explicit rubric or criteria decomposition rather than a bare overall score",
+        "Mentions checking the judge against human spot-checks (calibration/agreement) or using voting to reduce judge noise",
+        "Plans for online signals after launch (user feedback, A/B testing) as a complement to offline evals"
+      ]
+    },
+    {
+      "id": 3,
+      "prompt": "Set up MLflow for our fine-tuning experiments so we can log runs and compare hyperparameters across the team.",
+      "expected_output": "Recognizes this as experiment-tracking infrastructure, not evaluation design — hands off to ml-experiment-tracking for the MLflow setup and run comparison, noting the boundary: tracking records and compares runs, while ai-evaluation defines what and how to measure",
+      "assertions": [
+        "Recognizes the request is about experiment-tracking/run-comparison infrastructure, not about defining evaluation metrics or datasets",
+        "Refers to the ml-experiment-tracking skill for the MLflow setup and run comparison",
+        "Does not produce an evaluation-design workflow (golden datasets, judges, RAG metrics) as the answer to a tracking-infrastructure request",
+        "States the boundary: tracking records and compares runs, while evaluation design defines what and how to measure"
+      ]
+    }
+  ]
+}

package/skills/ai-evaluation/references/eval-tooling.md

@@ -0,0 +1,63 @@
+# Evaluation Tooling — Selection and Judge-Design Detail
+
+Tool landscapes move fast; verify against each project's current docs before
+committing. What follows is the decision logic, which is stable.
+
+## Choosing the eval tool
+
+Pick **one** primary tool. They overlap heavily; running two means maintaining
+two golden-set formats and two CI integrations for no added signal.
+
+| Tool | Shape | Choose when |
+|---|---|---|
+| **ragas** | Python library of RAG-specific metrics (faithfulness, answer relevancy, context precision/recall, noise sensitivity) | The system is RAG and you want the standard names for the RAG triad with minimal ceremony. Integrates with LangChain/LlamaIndex datasets. |
+| **deepeval** | pytest-style framework — metrics as assertions, `assert_test(test_case, [metric])` | The team already lives in pytest and wants evals in the same runner and CI report as unit tests. Broad metric set (RAG triad, hallucination, bias, custom G-Eval rubrics). |
+| **promptfoo** | Config-driven (YAML) prompt/model comparison matrix + web viewer, CLI-first | The main activity is comparing prompt variants or models side-by-side; also does red-teaming. Good for non-Python stacks (Node CLI). |
+| **Roll your own** | A loop + a judge prompt + a JSON of cases | The metrics are product-specific rubrics anyway and a framework would only wrap your judge calls. Fine — the golden set and calibration discipline matter more than the harness. |
+
+Platform suites (LangSmith, Langfuse, Braintrust, W&B Weave) bundle eval +
+tracing + datasets; prefer them when the team already uses that platform for
+observability — the dataset/annotation UI is the real value.
+
+## RAG metric cheat sheet
+
+| Failure you saw | Stage | Metric to add |
+|---|---|---|
+| Answer cites facts not in the docs | Generation | Faithfulness / groundedness |
+| Answer is on-topic-ish but doesn't answer | Generation | Answer relevance |
+| Right doc exists but wasn't retrieved | Retrieval | Context recall |
+| Retrieved chunks are mostly padding | Retrieval | Context precision |
+| Correct refusal expected, model answered anyway | Both | Unanswerable-set accuracy |
+
+Debug order: retrieval metrics first. Generation metrics computed over bad
+context are noise — fix recall before tuning prompts.
+
+## LLM-as-judge bias catalog
+
+| Bias | Symptom | Counter |
+|---|---|---|
+| Position | Prefers whichever answer is shown first | Randomize/swap order, judge twice, keep only consistent verdicts |
+| Self-preference | Rates its own model family's style higher | Judge with a different model family than the generator |
+| Verbosity | Longer answer wins regardless of substance | Instruct explicitly; normalize or cap lengths; per-criterion rubric |
+| Sycophancy toward the prompt | Scores what the question implies is right | Hide metadata (which variant is "new"); blind the judge |
+| Scale drift | 7/10 today ≠ 7/10 next month | Binary/3-point criteria; compare deltas on the same rubric version |
+
+## Judge calibration procedure
+
+1. Sample 20–30 golden cases; have a human score them with the same rubric.
+2. Run the judge on the same cases (k=3 voting).
+3. Compute simple agreement (percent match per criterion). Below ~80%: the
+   rubric is ambiguous — tighten criteria wording, add anchored examples
+   (a passing and a failing answer per criterion), re-run.
+4. Re-calibrate whenever the rubric, judge model, or task distribution changes.
+
+## Online evaluation notes
+
+- **A/B tests**: randomize per user (not per request) for conversational
+  products; pick the primary metric before launch; expect judge-metric ↔
+  user-metric divergence and investigate it — it usually means the rubric is
+  measuring the wrong thing.
+- **Human feedback**: thumbs up/down rates are biased (angry users click more);
+  use as a trend signal and a failure-mining source, not an absolute score.
+- **Failure mining**: a weekly pass over flagged conversations feeding the
+  golden set is the highest-ROI eval activity that exists.

package/skills/architecture-design/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: architecture-design
-description: "Guide architectural and structural decisions using Architecture Decision Records (ADRs) — new services, pattern choices, database selection, component boundaries, state management, costly-to-reverse decisions. Triggers: how should I architect, which pattern, design decision, should I use X or Y, system design, ADR, component design, architectural trade-off."
+description: "Guide architectural and structural decisions using Architecture Decision Records (ADRs) — new services, pattern choices, database selection, component boundaries, state management, costly-to-reverse decisions. Triggers: how should I architect, which pattern, design decision, should I use X or Y, system design, ADR, component design, architectural trade-off. Owns internal-structure decisions — build vs buy/adopt a vendor or OSS capability → build-vs-buy."
 model: opus
 allowed-tools: Read, Grep, Glob, Write, Edit
 ---
@@ -78,6 +78,8 @@ ADRs are immutable records. If a decision is superseded, create a new ADR that r
 
 For visual architecture documentation (system diagrams, runtime flows, infrastructure topology), see the `architecture-documentation` skill.
 
+When the decision is whether a capability should be built at all versus bought or adopted (vendor/SaaS/OSS, TCO, lock-in, exit costs), use `build-vs-buy` — it produces the same ADR-style record, but for sourcing rather than structure.
+
 ## When to Split Decisions
 
 If the analysis reveals multiple independent decisions bundled together, split them into separate ADRs. For example, "How should we handle caching?" might split into:

package/skills/brainstorming/SKILL.md

@@ -0,0 +1,92 @@
+---
+name: brainstorming
+description: "Facilitate divergent ideation before committing to a spec — Socratic questioning, widening the option space, challenging assumptions and inherited constraints, then converging on candidates. Triggers: brainstorm, let's explore ideas, what are our options, help me think through, generate ideas, what could we build, not sure what I want, explore approaches, ideate. Once ideas converge, hand off to prd-writing (WHAT/WHY) or feature-planning (breakdown) — this skill opens options; those close them."
+model: sonnet
+allowed-tools: Read, Grep, Glob, Write, Edit
+---
+
+# Brainstorming
+
+Divergent exploration *before* convergent specification. The most expensive
+failure in product work is efficiently building the wrong thing — this skill
+exists to widen the option space and test the framing before anything gets
+specced. It sits upstream of `prd-writing` and `feature-planning`: they converge
+on one answer; this skill makes sure the right answers were on the table first.
+
+The mode is **Socratic**: the user's tacit knowledge of their domain beats any
+generated list. Ask questions that surface it; generate to provoke, not to
+replace their thinking.
+
+## Workflow
+
+### Step 1: Frame the Real Problem
+
+Before generating anything, ask: what problem, for whom, why now, and what would
+"solved" look like? Then challenge the framing — the stated problem is often a
+solution in disguise ("we need a dashboard" usually means "someone can't answer
+a question"). Ask "what does that get you?" until you hit the underlying need.
+If the user arrives with a solution already chosen, brainstorm the *problem
+behind it* before brainstorming within it.
+
+### Step 2: Separate Real Constraints from Inherited Ones
+
+List the constraints, then interrogate each: is it physics/budget/law (real), or
+habit/assumption/"we've always done it this way" (inherited)? Explicitly mark
+which are negotiable. Most breakthrough options live behind a constraint that
+turned out to be assumed.
+
+### Step 3: Diverge — Quantity First, Judgment Deferred
+
+Generate broadly, and do not evaluate mid-generation — criticism during
+divergence kills the unusual options that make the exercise worthwhile. Mix
+techniques rather than exhausting one:
+
+- **Vary the axes**: who else could this serve, what adjacent problem does it
+  solve, how else could it be delivered, when/where else does it apply?
+- **Invert**: how would we guarantee this fails? Each failure mode implies an option.
+- **Analogize**: how do other domains solve the structural equivalent?
+- **Recombine**: force-pair two existing ideas or systems.
+
+Alternate generating with asking — offer a batch, then ask a question that
+mines the user's context for the next batch.
+
+### Step 4: Widen Once More (the Wildcards)
+
+Before converging, deliberately add the options nobody proposes on their own:
+the **do-nothing / process-change** option (no build at all), the **1/10-scope**
+version (what's the smallest thing that dents the problem?), the **10x**
+version (what if this were the company's main bet?), and the
+**buy-instead-of-build** option (delegate that comparison to `build-vs-buy` if
+it becomes a contender).
+
+### Step 5: Converge with Explicit Criteria
+
+Cluster related ideas, then name the selection criteria *before* scoring
+(typical set: impact, effort, reversibility, confidence) — criteria chosen after
+seeing favorites just rationalize them. Score roughly, pick 1–3 candidates, and
+keep the rest in a **parking lot** with a one-line reason each — parked, not
+deleted; today's reject is next quarter's answer.
+
+### Step 6: Hand Off
+
+Write a one-paragraph brief per surviving candidate (problem, idea, why it won,
+open questions) and route it onward: `prd-writing` for requirements,
+`feature-planning` for task breakdown, `project-proposal` for a business case,
+`architecture-design` if the open questions are structural.
+
+## Principles Applied
+
+- **Defer judgment during divergence; be ruthless during convergence** — mixing
+  the modes does both badly.
+- **Ask > tell**: a question that unlocks the user's knowledge outperforms ten
+  generated ideas.
+- **YAGNI applies at convergence, not divergence** — generating a wild option
+  costs a line; building it is what needs justification.
+
+## Cross-Skill References
+
+- `prd-writing` — turn the winning idea into WHAT/WHY requirements
+- `feature-planning` — break the chosen idea into scoped tasks
+- `project-proposal` — wrap a candidate in a business case for a go/no-go
+- `build-vs-buy` — when a candidate could be bought or adopted instead of built
+- `architecture-design` — when the surviving questions are structural

package/skills/brainstorming/evals/evals.json

@@ -0,0 +1,40 @@
+{
+  "skill_name": "brainstorming",
+  "evals": [
+    {
+      "id": 1,
+      "prompt": "We want to improve developer onboarding at our company — new hires take about two months to ship their first meaningful change. I'm not sure where to start. Can we brainstorm?",
+      "expected_output": "Runs a Socratic divergent session: probes the problem first (where the two months actually go, for whom, what 'onboarded' means), separates real from inherited constraints, generates a broad and varied option set without evaluating mid-stream, deliberately includes wildcards (do-nothing/process-change, tiny-scope, buy-instead-of-build), then converges with named criteria and hands the 1–3 survivors to prd-writing or feature-planning with a parking lot for the rest",
+      "assertions": [
+        "Asks clarifying/Socratic questions about the underlying problem (where the time goes, for whom, what success looks like) before or while generating ideas, rather than immediately listing solutions",
+        "Generates a genuinely diverse option set spanning different kinds of interventions (e.g. docs, tooling, process, people/mentoring), not variations of a single idea",
+        "Does not evaluate, score, or eliminate ideas during the divergent generation phase",
+        "Includes at least one wildcard option such as a no-build/process change, a radically smaller-scope version, or buying/adopting instead of building",
+        "Converges using explicitly named selection criteria (e.g. impact, effort, reversibility) and identifies 1–3 candidates plus a parking lot, with a handoff toward prd-writing, feature-planning, or project-proposal"
+      ]
+    },
+    {
+      "id": 2,
+      "prompt": "I've decided we need to build a custom internal chat app for our 40-person company. Let's brainstorm features for it.",
+      "expected_output": "Widens before narrowing: probes the problem behind the already-chosen solution (what's failing with current communication, why existing tools don't fit), surfaces that 'build a custom chat app' is a solution not a problem, raises alternatives including buying/adopting existing tools (routing a serious buy-vs-build comparison to build-vs-buy), distinguishes real from assumed constraints — and still delivers the requested feature ideation rather than refusing or lecturing",
+      "assertions": [
+        "Probes the problem behind the chosen solution (what communication need is unmet, why existing tools fall short) instead of only listing features",
+        "Surfaces at least one alternative to building a custom chat app, such as adopting or configuring an existing tool, and mentions build-vs-buy for that comparison",
+        "Questions or tests the constraints/assumptions behind the decision (e.g. why custom, why now) and distinguishes real constraints from inherited ones",
+        "Still provides the requested divergent feature ideation for the chat app rather than refusing to engage with the user's framing",
+        "Does not converge on a final feature list without naming selection criteria first"
+      ]
+    },
+    {
+      "id": 3,
+      "prompt": "We've settled on building a CLI tool that syncs our design tokens to code. Break it into tasks with acceptance criteria and dependencies so the team can start next sprint.",
+      "expected_output": "Recognizes that ideation is complete and the user is asking for convergent task breakdown — refers to feature-planning for the scoped tasks, acceptance criteria, and dependency mapping rather than reopening divergent exploration",
+      "assertions": [
+        "Recognizes the request as convergent planning of an already-chosen idea, not ideation",
+        "Refers to the feature-planning skill for the task breakdown, acceptance criteria, and dependencies",
+        "Does not restart divergent idea generation or question the settled decision as the main response",
+        "States or implies the boundary: brainstorming opens the option space before a decision; feature-planning structures the work after it"
+      ]
+    }
+  ]
+}

package/skills/browser-verification/SKILL.md

@@ -0,0 +1,95 @@
+---
+name: browser-verification
+description: "Verify a web app change by driving the real UI in a browser — reproduce the user flow end to end, watch console errors and network failures, exercise loading/error/empty states, capture screenshots as evidence. Use before claiming a UI change works. Triggers: check it in the browser, verify the UI, click through the flow, console errors, test it like a user, smoke test the frontend. Designing persistent test suites → test-suite-design; the generic done-gate → verification-before-completion (this is its browser-specific practice)."
+model: sonnet
+allowed-tools: Read, Grep, Glob, Write, Edit, Bash
+---
+
+# Browser Verification
+
+Prove a web change works by *being the user*, not by reading the diff. Passing
+unit tests and a clean build say the code is consistent with itself — they say
+nothing about the page actually rendering, the handler actually firing, or the
+API call actually succeeding with real wiring. This skill is the
+browser-specific practice of `verification-before-completion`: the claim "the
+UI works" requires having driven the UI, in this session, with the evidence
+captured. Boundary: `test-suite-design` builds *persistent* automated suites;
+this skill is the verification pass you run on the change in front of you
+(what it finds often becomes a suite case afterward).
+
+## Workflow
+
+### Step 1: Define the Flow That Proves the Change
+
+Name the user-visible behavior the change promises ("submitting the form shows
+the success toast and the new record appears in the list") and the shortest
+real path to it. That flow — not "the page loads" — is what verification means.
+Include what should *stop* happening for bug fixes (the old broken behavior,
+reproduced first if possible).
+
+### Step 2: Run the Real App in a Realistic State
+
+Launch the actual stack the change ships in (dev server + backend + seeded
+data), not a mental model of it. State matters: an empty database hides list
+rendering bugs; a logged-out session hides authz redirects. Seed what the flow
+needs — including a user in the right role.
+
+### Step 3: Drive the Flow Like a User
+
+Automate the drive with a browser tool (Playwright or the environment's
+browser automation) or click it manually — but *interact*: navigate, type into
+the fields, submit, wait for the response, observe the outcome. Prefer
+selectors a user would recognize (roles, labels, text) over brittle internals.
+Loading a URL and seeing HTTP 200 is not driving the flow — most UI bugs live
+after first paint.
+
+### Step 4: Watch All Three Evidence Channels
+
+The page can *look* right while failing. During the drive, check:
+
+- **Console** — errors and warnings that fired during the flow (a caught-and-
+  swallowed error today is the un-reproducible bug report next week).
+- **Network** — the requests the flow triggered: right endpoints, right
+  payloads, right status codes; no 4xx/5xx, no CORS failures, no retry loops.
+- **UI state** — the visible outcome: content correct, no layout collapse, no
+  infinite spinner masquerading as "loading".
+
+### Step 5: Exercise Beyond the Happy Path
+
+The states users hit that developers don't: **error** (kill the backend or
+force a 500 — does the UI degrade or white-screen?), **empty** (zero items),
+**loading** (throttle and look — flicker? layout shift?), **repeat actions**
+(double-submit, back button, refresh mid-flow), and at least one non-desktop
+viewport if the change touches layout. Scope to what the change plausibly
+affects — a copy tweak doesn't need the full matrix; a checkout-flow change
+does. (`ui-ux-design` owns designing these states; this skill verifies them.)
+
+### Step 6: Capture Evidence and Report Precisely
+
+Screenshot the key outcomes (success state, plus anything broken), keep the
+console/network findings, and report in evidence terms: "drove signup →
+dashboard with a fresh user; form validates, POST /users returns 201, toast
+renders, zero console errors — screenshot attached" beats "the UI works".
+Anything found but not fixed is reported, not silently absorbed. If the flow
+proved valuable to verify, feed it to `test-suite-design` as a candidate e2e
+case so next time it verifies itself.
+
+## Principles Applied
+
+- **The rendered page is the truth**: code review and unit tests approximate
+  it; only the browser confirms it.
+- **Interaction over inspection**: most UI bugs are reachable only by doing
+  what a user does, in order.
+- **Evidence, not impressions**: a screenshot with zero console errors is a
+  verification; "looked fine" is a mood.
+
+## Cross-Skill References
+
+- `verification-before-completion` — the generic evidence gate this skill
+  instantiates for web UIs
+- `test-suite-design` — turning one-off verification flows into persistent e2e coverage
+- `ui-ux-design` — designing the loading/error/empty states verified here
+- `accessibility-design` — the deeper a11y pass when the change touches
+  interaction patterns
+- `bug-investigating` — when verification finds something broken and the cause
+  isn't obvious

package/skills/browser-verification/evals/evals.json

@@ -0,0 +1,40 @@
+{
+  "skill_name": "browser-verification",
+  "evals": [
+    {
+      "id": 1,
+      "prompt": "I just finished implementing the new signup form (client validation + POST to /api/users + success toast). Unit tests pass. Can you verify it actually works in the browser before I open the PR?",
+      "expected_output": "Defines the proving flow (fill form → submit → 201 → toast → user exists), runs the real stack with seeded/realistic state, drives the flow interactively (typing, submitting — not just loading the page), watches console errors, network requests (endpoint, payload, status), and UI state, exercises validation errors and double-submit, and reports with evidence (screenshots, zero-console-errors statement, observed network results) rather than 'looks good'",
+      "assertions": [
+        "Defines the user-visible flow that proves the change (form fill → submit → success), not just that the page loads",
+        "Runs the actual app stack with realistic state and drives the flow interactively — typing into fields and submitting, via automation or manual clicking",
+        "Checks all three evidence channels: console errors, network requests (endpoint/payload/status), and the rendered UI outcome",
+        "Exercises non-happy paths relevant to the change, such as validation errors and double-submit or backend failure",
+        "Reports in evidence terms — screenshots and observed results (e.g. 'POST /api/users returned 201, toast rendered, zero console errors') — before declaring it works"
+      ]
+    },
+    {
+      "id": 2,
+      "prompt": "QA says the orders dashboard 'sometimes shows nothing' but every time I open it, it looks fine and there are no errors in our logs. Verify whether the page is actually broken.",
+      "expected_output": "Recognizes 'works on my visit' isn't verification of the failure condition: reproduces the states where lists show nothing — empty data, slow API (throttled loading state), failed request (5xx/CORS with the error swallowed client-side), auth/role differences, race on refresh/back — while watching the browser console and network tab (server logs won't show client-side failures), and either reproduces it with evidence or reports precisely which states were exercised and ruled out",
+      "assertions": [
+        "Recognizes that the page rendering fine on one visit does not rule out state-dependent failure, and enumerates candidate states: empty data, slow/failed API responses, auth/role differences, or refresh/back races",
+        "Uses browser-side evidence (console, network tab) explicitly noting that server logs miss client-side failures like swallowed errors or CORS",
+        "Manipulates state to reproduce — seeding empty data, throttling, or forcing API failure — rather than only reloading the page",
+        "Checks the loading and empty states specifically (infinite spinner or blank-on-empty are classic 'shows nothing' causes)",
+        "Concludes with evidence: either a reproduced failure with its trigger, or a precise list of exercised-and-passing states — not 'seems fine'"
+      ]
+    },
+    {
+      "id": 3,
+      "prompt": "We need proper automated e2e coverage for our whole checkout funnel — browse, cart, payment, confirmation — running in CI on every PR. Set up the test strategy.",
+      "expected_output": "Recognizes this as designing a persistent automated test suite — refers to test-suite-design for the e2e strategy/infrastructure (and cicd-pipeline for the CI wiring), stating the boundary: browser-verification is the one-off evidence pass on the change in front of you; persistent suites are test-suite-design's domain",
+      "assertions": [
+        "Recognizes the request as persistent automated e2e suite design, not a one-off verification pass",
+        "Refers to the test-suite-design skill for the suite strategy (optionally cicd-pipeline for CI integration)",
+        "Does not respond with a single manual/ad-hoc browser verification session as the answer",
+        "States or implies the boundary: browser-verification verifies the change at hand; test-suite-design builds the lasting coverage"
+      ]
+    }
+  ]
+}

package/skills/build-vs-buy/SKILL.md

@@ -0,0 +1,109 @@
+---
+name: build-vs-buy
+description: "Decide whether to build in-house, buy a vendor/SaaS product, or adopt OSS — core-vs-commodity framing, multi-year total cost of ownership, integration and maintenance burden, vendor viability, lock-in and exit costs. Triggers: build vs buy, build or buy, vendor evaluation, SaaS vs in-house, should we adopt this open source, TCO, total cost of ownership, lock-in, exit cost, make or buy, off the shelf. Internal-structure decisions (patterns, service boundaries, ADRs for your own code) → architecture-design."
+model: opus
+allowed-tools: Read, Grep, Glob, Write, Edit, WebFetch, WebSearch
+---
+
+# Build vs Buy
+
+Make the build/buy/adopt decision on total cost and strategic fit, not on
+whichever option the room already favors. The two systematic errors this skill
+exists to counter: engineers underestimate the *ongoing* cost of building
+(maintenance routinely exceeds the initial build within a few years), and
+everyone underestimates the *exit* cost of buying (discovered only when
+leaving). Boundary: this skill decides whether the capability comes from
+outside; how you structure your own code is `architecture-design`'s ADR
+territory, and vetting an individual library is `dependency-management`.
+
+## Workflow
+
+### Step 1: Frame the Capability — Core or Commodity?
+
+Name the capability precisely, then ask the question that dominates the whole
+decision: **does doing this better than anyone else win you customers?** If yes
+(core differentiator), building earns strategic control; if it's commodity
+plumbing (auth, feature flags, payments, analytics infra, CRUD admin), every
+engineer-month spent building it is a month not spent on what differentiates
+you. Then split requirements into *must-have* and *nice-to-have* — a vendor
+covering 90% of must-haves usually beats a build covering 100% of a wishlist.
+
+### Step 2: Inventory the Real Options
+
+There are usually more than two:
+
+- **Build** in-house
+- **Buy** a vendor/SaaS product
+- **Adopt** OSS self-hosted (you're not buying — you're taking an operations job)
+- **Hybrid** — buy/adopt the commodity core, build the thin differentiating layer
+- **Defer** — do nothing yet; a legitimate option when the need is unproven
+
+Verify current facts (pricing, licensing, maintenance status) with live sources
+— vendor pricing pages and OSS repos, not memory.
+
+### Step 3: TCO Over 3 Years — Not the Sticker Price
+
+Cost each finalist over a 3-year horizon; year-1 comparisons systematically
+favor building (the build cost is visible, the ownership cost isn't):
+
+- **Build**: initial engineering + **ongoing ownership** (maintenance, on-call,
+  security patching, feature parity as needs grow — commonly comparable to the
+  initial build per year for actively-used systems) + **opportunity cost** (what
+  those engineers would otherwise ship — usually the largest and least-counted line).
+- **Buy**: subscription *at projected usage*, not today's (model the pricing
+  curve at 3× growth — usage-based pricing is where vendors recover their
+  discounts) + integration + admin.
+- **Adopt (OSS)**: hosting + operations + upgrades + security response + the
+  expertise to run it; "free" describes the license, not the ownership.
+- **Everyone forgets**: migration of existing data/flows, training, compliance
+  and security review, vendor management overhead.
+
+### Step 4: Risk — Lock-in, Exit, and Viability
+
+Write the **exit plan now**, while you have leverage — the exit cost you can't
+estimate is the one that traps you:
+
+- What does leaving take? Data export (format? completeness? egress fees?),
+  proprietary APIs/features you'd re-implement, contract terms (lock-in period,
+  auto-renewal, price-increase caps).
+- **Vendor viability**: funding/profitability, roadmap alignment, acquisition
+  risk, what happens to your data if they fold.
+- **OSS health**: maintainer bus factor, release cadence, CVE responsiveness,
+  license (copyleft implications, and watch relicensing-to-source-available
+  history in the space) — delegate a deep package-level check to `dependency-management`.
+- **Build risk** is real too: key-person dependency, and the "internal vendor"
+  failure mode — a team obligated to maintain a product nobody funds.
+
+### Step 5: Decide and Record
+
+Score the finalists in a weighted matrix — criteria and weights fixed *before*
+scoring (after, the matrix just rationalizes the favorite). State a clear
+recommendation with the dominant reasons, name the runner-up and what would
+have flipped the decision, and set a **revisit trigger** ("revisit if usage
+exceeds X / pricing changes >Y% / the maintainer goes dark"). Record it like an
+ADR — decision, context, alternatives, consequences — so the next team
+understands *why*, not just *what*.
+
+### Step 6: De-risk Before Committing
+
+For close calls or large commitments, run a time-boxed pilot against your top
+must-haves and your real data volumes before signing multi-year terms or
+staffing a build. A two-week PoC is the cheapest insurance in this workflow —
+and pilot the *exit* too (run the data export once and look at what comes out).
+
+## Principles Applied
+
+- **Commodity → buy bias; differentiator → build bias** — most wrong decisions
+  invert this because building is more fun than integrating.
+- **Compare TCO, not sticker price** — the build's cost is mostly after launch;
+  the vendor's is mostly at renewal and exit.
+- **A decision without a revisit trigger is dogma** — pricing, vendors, and
+  your scale all change.
+
+## Cross-Skill References
+
+- `architecture-design` — internal-structure decisions and the ADR format
+- `dependency-management` — deep evaluation of a specific library/package
+- `project-proposal` — wrap the decision in a business case for approval
+- `effort-estimation` — sizing the build option honestly
+- `strategic-review` — when the decision touches positioning or the moat

package/skills/build-vs-buy/evals/evals.json

@@ -0,0 +1,40 @@
+{
+  "skill_name": "build-vs-buy",
+  "evals": [
+    {
+      "id": 1,
+      "prompt": "We need feature flags for our product (SaaS, ~15 engineers, growing fast). Should we build our own flag service, buy LaunchDarkly, or self-host an open-source option like Unleash? Help me decide.",
+      "expected_output": "Frames feature flags as commodity infrastructure (core-vs-differentiator test), compares all three options on a multi-year TCO including ongoing ownership for build, projected-usage pricing for buy, and operations burden for self-hosted OSS, examines lock-in and exit costs plus vendor/OSS health, and lands on a clear recommendation recorded like an ADR with a revisit trigger",
+      "assertions": [
+        "Applies the core-vs-commodity framing — asks or determines whether feature flags differentiate this product (they don't) and lets that bias the decision",
+        "Compares build, buy, and OSS-adopt on total cost of ownership over a multi-year horizon, including ongoing maintenance/ownership for build and operations burden for self-hosted OSS — not just sticker price",
+        "Models the buy option at projected usage/growth, not only today's seat or flag count",
+        "Addresses lock-in and exit costs (data/config export, proprietary features, contract terms) and vendor or OSS project health",
+        "Concludes with a clear recommendation, the runner-up and what would flip it, and a revisit trigger — recorded in an ADR-like decision format"
+      ]
+    },
+    {
+      "id": 2,
+      "prompt": "Our data team already spent two quarters building about 60% of an in-house analytics pipeline (ingestion works, transformation layer is half done). Meanwhile Fivetran + Snowflake + dbt Cloud would cover everything we planned for roughly $8k/month. Do we finish the build or switch? Leadership hates the idea of throwing away six months of work.",
+      "expected_output": "Treats the six months as sunk cost and evaluates only forward-looking costs: remaining build effort plus perpetual ownership (maintenance, on-call, feature parity) and the team's opportunity cost, versus migration effort plus the subscription at projected data volumes — including the vendor stack's lock-in/exit profile — and gives a clear recommendation with reasoning rather than an 'it depends' hedge",
+      "assertions": [
+        "Explicitly identifies the completed 60% as sunk cost and excludes it from the forward-looking comparison",
+        "Compares remaining-build-plus-ongoing-ownership cost (maintenance, on-call, opportunity cost of the data team) against migration-plus-subscription cost",
+        "Models the managed stack's subscription at projected data growth and examines its lock-in and exit costs, not just the $8k/month figure",
+        "Considers what the team would work on instead (opportunity cost) as a first-class factor in the decision",
+        "Delivers a clear, justified recommendation with a revisit trigger instead of an evasive 'it depends'"
+      ]
+    },
+    {
+      "id": 3,
+      "prompt": "Should our order service use event sourcing or a plain CRUD model, and is it time to split it out of the monolith into its own microservice?",
+      "expected_output": "Recognizes these as internal-structure architectural decisions about the team's own code — pattern choice and service boundaries — and refers to architecture-design for an ADR-driven trade-off analysis, without running a vendor/TCO evaluation",
+      "assertions": [
+        "Recognizes the request as internal architecture (pattern choice, service boundaries), not a build/buy/adopt sourcing decision",
+        "Refers to the architecture-design skill and its ADR workflow for the decision",
+        "Does not produce a vendor evaluation, TCO analysis, or lock-in assessment for this request",
+        "States or implies the boundary: build-vs-buy decides where a capability comes from; architecture-design decides how your own system is structured"
+      ]
+    }
+  ]
+}

package/skills/cicd-pipeline/SKILL.md

@@ -83,5 +83,6 @@ Quality gates prevent bad code from advancing:
 - `deployment-repo` — for multi-service systems, the deployment repo handles system-level CI (contract tests, E2E, promotion)
 - `gitops-delivery` — pull-based delivery as an alternative to CI-driven `kubectl apply` / `helm upgrade`
 - `deployment-checklist` — pre-deployment verification gates to incorporate into the pipeline
+- `release-management` — the release stage itself: version bumps, changelog, tagging, publish gates, registry publishing (the pipeline automates what that skill decides)
 - `security-audit` — security scanning stages and SAST/DAST integration
 - `verification-before-completion` — run the proving commands locally before relying on the pipeline to catch failures