vinta-ai-workflows 0.3.0 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +84 -0
- package/README.md +23 -0
- package/package.json +1 -1
- package/schemas/vinta-ai-workflows-config.v1.schema.json +43 -0
- package/skills/vinta-analyze-codebase/SKILL.md +15 -1
- package/skills/vinta-bootstrap-ai-tools/SKILL.md +37 -8
- package/skills/vinta-derive-skills/SKILL.md +44 -3
- package/skills/vinta-derive-skills/resources/write-unit-test-packs/runners/jest.md +49 -0
- package/skills/vinta-derive-skills/resources/write-unit-test-packs/runners/pytest.md +36 -0
- package/skills/vinta-derive-skills/resources/write-unit-test-packs/runners/vitest.md +43 -0
- package/skills/vinta-derive-skills/resources/write-unit-test-packs/stacks/django.md +36 -0
- package/skills/vinta-derive-skills/resources/write-unit-test-packs/stacks/fastapi.md +34 -0
- package/skills/vinta-derive-skills/resources/write-unit-test-packs/stacks/flask.md +31 -0
- package/skills/vinta-derive-skills/resources/write-unit-test-packs/stacks/medplum-test.globalSetup.ts +54 -0
- package/skills/vinta-derive-skills/resources/write-unit-test-packs/stacks/medplum-test.setup.ts +39 -0
- package/skills/vinta-derive-skills/resources/write-unit-test-packs/stacks/medplum.md +47 -0
- package/skills/vinta-derive-skills/resources/write-unit-test-packs/stacks/nextjs.md +27 -0
- package/skills/vinta-derive-skills/resources/write-unit-test-packs/stacks/prisma.md +31 -0
- package/skills/vinta-derive-skills/resources/write-unit-test-packs/stacks/python-package.md +28 -0
- package/skills/vinta-derive-skills/resources/write-unit-test-packs/stacks/react-router.md +25 -0
- package/skills/vinta-derive-skills/resources/write-unit-test-packs/stacks/react.md +29 -0
- package/skills/vinta-derive-skills/resources/write-unit-test-packs/stacks/tanstack-start.md +32 -0
- package/skills/vinta-derive-skills/resources/write-unit-test-packs/stacks/typescript-package.md +32 -0
- package/skills/vinta-derive-skills/resources/write-unit-test-template.md +44 -0
package/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,90 @@ All notable changes to `vinta-ai-workflows` are documented in this file.
|
|
|
5
5
|
The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/);
|
|
6
6
|
the project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
7
|
|
|
8
|
+
## [0.4.0] — 2026-07-31
|
|
9
|
+
|
|
10
|
+
### Added
|
|
11
|
+
|
|
12
|
+
- **`write-unit-test` foundation skill (template-rendered, default-on when a
|
|
13
|
+
unit-test framework is detected).** A new foundation skill that ships into
|
|
14
|
+
target projects' `ai-tools/skills/write-unit-test/` to write durable unit
|
|
15
|
+
tests. The body enforces six universal rules regardless of framework — mock
|
|
16
|
+
only genuine externals (never the unit under test / DB / local collaborators),
|
|
17
|
+
assert the **full** expected output (not counts or truthiness), clean up
|
|
18
|
+
created data or run inside a rolled-back transaction, never hit an external
|
|
19
|
+
service that can't run in dev/test, keep test logic literal (no loops /
|
|
20
|
+
conditionals recomputing the expectation), and decouple from implementation
|
|
21
|
+
internals — plus a self-review checklist and a "run it green, then break it to
|
|
22
|
+
see it go red" step.
|
|
23
|
+
- Rendered from
|
|
24
|
+
[`skills/vinta-derive-skills/resources/write-unit-test-template.md`](skills/vinta-derive-skills/resources/write-unit-test-template.md)
|
|
25
|
+
(bucket B). New placeholders: `{{FRAMEWORK_PACKS_LIST}}`,
|
|
26
|
+
`{{FRAMEWORK_PACK_LOADER_BLOCK}}`, `{{PROJECT_TEST_PREFERENCES_BLOCK}}`,
|
|
27
|
+
`{{ADDITIONAL_CONVENTIONS_BLOCK}}` (reuses the existing `{{TEST_CMD}}` /
|
|
28
|
+
`{{NEW_TEST_CMD_PATTERN}}` / `{{SCOPED_TEST_NOTE}}` / `{{PROJECT_NAME}}` /
|
|
29
|
+
`{{STACK_SUMMARY}}` set). All four new placeholders are documented in the
|
|
30
|
+
`vinta-derive-skills` placeholder table.
|
|
31
|
+
- **Best-practice packs on two independent axes** (the framework-specific
|
|
32
|
+
"sub-skills"), verbatim references under
|
|
33
|
+
[`skills/vinta-derive-skills/resources/write-unit-test-packs/`](skills/vinta-derive-skills/resources/write-unit-test-packs/).
|
|
34
|
+
`derive-skills` copies only what the project matches into
|
|
35
|
+
`ai-tools/skills/write-unit-test/resources/packs/` so framework-specific
|
|
36
|
+
advice never ships to a project that doesn't use that framework (a FastAPI
|
|
37
|
+
repo never receives Django guidance; a pure library gets no web-framework
|
|
38
|
+
pack):
|
|
39
|
+
- **Runner packs** (`packs/runners/`, stack-agnostic — structure /
|
|
40
|
+
assertions / mocking): `pytest.md`, `vitest.md`, `jest.md`. One selected
|
|
41
|
+
by `inventory.tests.unit_framework`.
|
|
42
|
+
- **Stack packs** (`packs/stacks/`, DB isolation / framework client / domain
|
|
43
|
+
mocks): `django.md`, `fastapi.md`, `flask.md`, `medplum.md`, `react.md`,
|
|
44
|
+
`nextjs.md`, `tanstack-start.md`, `react-router.md`, `prisma.md`,
|
|
45
|
+
`python-package.md`, `typescript-package.md`. Every pack whose stack the inventory matched
|
|
46
|
+
ships (a project can match several — e.g. Next.js + React, or TanStack
|
|
47
|
+
Start + React); a repo with no web framework (a standalone TS/Python package like
|
|
48
|
+
an open-source lib) gets its `*-package.md` and no more; when nothing
|
|
49
|
+
matches, the runner pack ships alone. Missing packs for a present
|
|
50
|
+
framework are recorded as a gap (candidate for a new pack via
|
|
51
|
+
`add-foundation-skill`).
|
|
52
|
+
- **Stack packs may carry a one-time project-setup step**, applied at derive
|
|
53
|
+
time (not per test). Medplum ships the two Vitest setup files
|
|
54
|
+
`medplum-test.globalSetup.ts` + `medplum-test.setup.ts` (FHIR search-parameter
|
|
55
|
+
indexing shared across workers via `provide`/`inject`, so `MockClient`
|
|
56
|
+
search filtering works) — the derive flow installs them and registers both
|
|
57
|
+
`test.globalSetup` + `test.setupFiles` in the Vitest config; the `medplum.md`
|
|
58
|
+
pack only references it. Bundled verbatim (dated headers) from the team's
|
|
59
|
+
`medplum-snippet-catalog` (`TestSetup/`) so bootstrap stays offline-safe.
|
|
60
|
+
- **Schema:** new `foundation_skills.write-unit-test` enum
|
|
61
|
+
(`enabled` / `disabled`) and a new `skills.write-unit-test` config block —
|
|
62
|
+
`test_style` (functions / classes / framework-default), `data_setup`
|
|
63
|
+
(factories / fixtures / inline / framework-default), `assertion_style`
|
|
64
|
+
(plain-assert / framework-methods / framework-default), `db_isolation`
|
|
65
|
+
(transaction-rollback / truncate / recreate / framework-default), and a
|
|
66
|
+
free-form `additional_conventions[]`. All optional; every field defaults to
|
|
67
|
+
`framework-default`.
|
|
68
|
+
- **Bootstrap:** `write-unit-test` is **default-on, not an opt-in** — the
|
|
69
|
+
interview sets `foundation_skills.write-unit-test: enabled` whenever a
|
|
70
|
+
unit-test framework was detected, `disabled` otherwise. New Project-conventions
|
|
71
|
+
interview item **C.9 (Test conventions for `write-unit-test`)** captures the
|
|
72
|
+
four preferences + extras, pre-filled from inference on existing projects.
|
|
73
|
+
(The former C.9 "Agent model tiers" is now **C.10**; its Step 0.5 references
|
|
74
|
+
were updated.) Step 0.5 emits the `skills.write-unit-test` block; the outputs
|
|
75
|
+
tree, the three-buckets paragraph, and the `Replace with Vinta foundation
|
|
76
|
+
version` foundation-shape list all now list `write-unit-test`.
|
|
77
|
+
- **Inference:** `vinta-analyze-codebase` §6 (Tests) now reads a sample of the
|
|
78
|
+
existing unit tests and reports a `tests.conventions` block
|
|
79
|
+
(`test_style` / `data_setup` / `assertion_style` / `db_isolation` / `extras`,
|
|
80
|
+
each `framework-default` when mixed / greenfield) that seeds the C.9 defaults.
|
|
81
|
+
- **Review-driven refinements** (PR #21): two universal rules added — **kill
|
|
82
|
+
nondeterminism** (clock / randomness / timezone / ordering / concurrency → no
|
|
83
|
+
flaky tests) and **cover the branches** (aim 95%+ branch coverage); the
|
|
84
|
+
full-value assertion rule now carves out **brittle human-facing text** (assert
|
|
85
|
+
a stable anchor — error code, role, i18n key — for error/UI strings, not exact
|
|
86
|
+
prose). Runner packs gained a **record/replay cassette** note for complex APIs
|
|
87
|
+
like LLM endpoints (PollyJS / `nock.back` for JS, `vcrpy` / `betamax` for
|
|
88
|
+
Python). The Medplum pack gained a **`MockClient` limitations** section
|
|
89
|
+
(pre-seeded example resources skew unfiltered searches, partial search-param
|
|
90
|
+
support, server-only behavior not enforced — defer those to integration/e2e).
|
|
91
|
+
|
|
8
92
|
## [0.3.0] — 2026-07-17
|
|
9
93
|
|
|
10
94
|
### Fixed
|
package/README.md
CHANGED
|
@@ -241,6 +241,7 @@ Land at `ai-tools/skills/<name>/SKILL.md`. Always-on unless flagged optional.
|
|
|
241
241
|
| `amend-plan` | always (generated) | History-rewriting companion to `implement-plan` — revises in-flight plans, amends prior-phase commits, force-pushes, rebases stacked downstream branches. Reuses `review-phase` + `implement-phase`'s shared prompt loop. |
|
|
242
242
|
| [`handoff`](skills/vinta-derive-skills/resources/foundation-skills/handoff/SKILL.md) | always | Session-continuation handoff doc → `.vinta-ai-workflows/handoffs/`. **Write** captures goal, verified-vs-unverified state, decisions + rejected alternatives, landmines, and the single next step — gathered from the repo, never memory. **Resume** verifies a handoff's claims against the repo before continuing from its next step. |
|
|
243
243
|
| [`deslop-comments`](skills/vinta-derive-skills/resources/foundation-skills/deslop-comments/SKILL.md) | always | Rewrite comments + doc blocks touched during a task into Simple English (strips AI-slop vocabulary + negative framing; comment-only — no renames, no behavior change). `review-phase` Layer 2 depends on it; also invokable standalone. |
|
|
244
|
+
| `write-unit-test` | default-on (generated) | Write durable unit tests for a unit of behavior, enforcing six framework-agnostic rules + this project's captured test conventions, with runner- and stack-specific best-practice packs. Ships automatically whenever a unit-test framework is detected (set `disabled` to opt out). Generated from a template; config under `skills.write-unit-test.*`. See below. |
|
|
244
245
|
| `systematic-debugging` | **opt-in** | Root-cause-first debugging with project-specific repro commands + MCP evidence-gathering (error tracking, traces, logs, metrics, alerts). Renders from a catalogue of observability MCP servers the user declares. |
|
|
245
246
|
| `add-e2e-test` | **opt-in** | Add an e2e test. Body covers e2e framework, page-object pattern, auth/storage-state, seed helpers, tenant scoping, screenshot conventions. |
|
|
246
247
|
| `add-env-var` | **opt-in** | Propagate a new env var through every layer (`.env.example`, build tool envPrefix, build cache hash, app config, AGENTS.md, CI, deploy injection). |
|
|
@@ -249,6 +250,28 @@ Land at `ai-tools/skills/<name>/SKILL.md`. Always-on unless flagged optional.
|
|
|
249
250
|
| [`thermo-nuclear-code-quality-review`](skills/vinta-derive-skills/resources/foundation-skills/thermo-nuclear-code-quality-review/SKILL.md) | **opt-in** | Deliberately harsh structural-maintainability audit of a diff — abstraction quality, giant files, spaghetti-condition growth — hunting "code-judo" reframes that collapse whole branches / helpers / modes / layers. Read-only; hands each fix to `fixer`. `review-phase` Layer 3 runs a condensed lens every phase and escalates to this full audit on core-architecture / ~1000-line / structurally-smelly phases. |
|
|
250
251
|
| `handoff-to-client` | **opt-in** | (API-only repos) Generate a self-contained API-change handoff doc for the client teams consuming the repo's API — every endpoint / operation added / changed / deprecated / removed vs the default branch, with request/response shapes, auth + error changes, breaking-change flags, one example per operation, and per-platform migration notes. Template-rendered from `skills.handoff-to-client.*` config (client platforms, API style, spec path). |
|
|
251
252
|
|
|
253
|
+
#### `write-unit-test` in depth
|
|
254
|
+
|
|
255
|
+
Agents write tests that pass but assert too little, mock everything, leave data behind, or couple to internals — this skill exists so you don't have to babysit that. It bakes in **six framework-agnostic rules** that hold for every test in every stack:
|
|
256
|
+
|
|
257
|
+
1. **Mock only genuine externals** — never the unit under test, its local collaborators, or the DB.
|
|
258
|
+
2. **Assert the full expected value**, not counts or truthiness (`assert result`, `len(x) == 3` are too weak).
|
|
259
|
+
3. **Leave no trace** — transaction-rollback where available, else clean up in teardown (runs on failure too).
|
|
260
|
+
4. **No external service** that can't run in dev/test — stub at the boundary.
|
|
261
|
+
5. **Test logic stays literal** — no loops/conditionals recomputing the expected value.
|
|
262
|
+
6. **Decouple from internals** — assert observable state/output; survive a behavior-preserving rewrite.
|
|
263
|
+
7. **Kill nondeterminism** — control every entropy source (clock, randomness, timezone/locale, iteration/DB order, concurrency); a flaky test is worse than none. Don't over-fit brittle text either — assert a stable anchor (error code, role, i18n key) for error messages and UI copy, not the exact churning prose.
|
|
264
|
+
8. **Cover the branches** — aim high (95%+ branch coverage): every error path, guard, and edge gets a case. Coverage is the floor; correct assertions are the point.
|
|
265
|
+
|
|
266
|
+
Plus a "run it green, then break the code to watch it go red" step and a name-for-the-behavior / one-reason-to-fail convention.
|
|
267
|
+
|
|
268
|
+
**Framework guidance ships as best-practice packs on two independent axes**, so a project only ever receives advice for frameworks it actually uses (a FastAPI repo never gets Django guidance; a pure library gets no web-framework pack):
|
|
269
|
+
|
|
270
|
+
- **Runner pack** (`resources/packs/runners/`) — how to structure, assert, and mock. One, picked by the detected test runner: `pytest`, `vitest`, or `jest`.
|
|
271
|
+
- **Stack pack(s)** (`resources/packs/stacks/`) — DB isolation, framework client, domain mocks. Every matched stack's pack ships (a project can match several — e.g. Next.js + React, or TanStack Start + React): `django`, `fastapi`, `flask`, `medplum`, `react`, `nextjs`, `tanstack-start`, `react-router`, `prisma`, plus pure-library packs `python-package` / `typescript-package` for framework-less packages.
|
|
272
|
+
|
|
273
|
+
**Project preferences are captured at bootstrap** (and inferred from an existing suite, then confirmed) under `skills.write-unit-test.*`: `test_style` (classes vs functions), `data_setup` (factories / fixtures / inline), `assertion_style`, `db_isolation`, plus free-form `additional_conventions`. Each defaults to `framework-default`, deferring to the pack. New frameworks slot in by adding a pack under [`write-unit-test-packs/`](skills/vinta-derive-skills/resources/write-unit-test-packs/) and a detection signal — no schema change.
|
|
274
|
+
|
|
252
275
|
### Foundation sub-agents (project-agnostic)
|
|
253
276
|
|
|
254
277
|
Land at `ai-tools/agents/<name>.yaml` (canonical YAML; `setup-ai-tools.mjs` emits per-vendor copies into `.claude/agents/`, `.cursor/`, `.codex/`, `.github/`). Always-on trio.
|
package/package.json
CHANGED
|
@@ -232,6 +232,11 @@
|
|
|
232
232
|
"description": "Generate a markdown API-change handoff document for client teams consuming an API-only repo — endpoints added/changed/deprecated/removed on the current branch vs the default branch, with request/response shapes, breaking-change flags, and per-platform migration notes. Template-rendered (bucket B) from `vinta-derive-skills/resources/handoff-to-client-template.md` and opt-in (bucket C) — only ships when the bootstrap interview confirms an API-only repo with separate client codebases. When `enabled`, project-level configuration lives under `skills.handoff-to-client` (client platforms, API style, spec path, output dir).",
|
|
233
233
|
"type": "string",
|
|
234
234
|
"enum": ["enabled", "disabled"]
|
|
235
|
+
},
|
|
236
|
+
"write-unit-test": {
|
|
237
|
+
"description": "Write durable unit tests for a unit of behavior — mock only genuine externals, assert the full expected output (not counts/truthiness), clean up created data or run inside a rolled-back transaction, never hit an external service that can't run in dev/test, keep test logic literal, and decouple from implementation internals. Template-rendered (bucket B) from `vinta-derive-skills/resources/write-unit-test-template.md`, with best-practice packs copied verbatim into the skill's `resources/packs/` on two independent axes — one runner pack (`runners/`: pytest / vitest / jest) selected by the detected test framework, plus a stack pack per matched stack (`stacks/`: django / fastapi / flask / medplum / react / nextjs / tanstack-start / react-router / prisma / python-package / typescript-package) so framework-specific advice ships only when that framework is present. Default-on: bootstrap sets `enabled` whenever `vinta-analyze-codebase` detects a unit-test framework, `disabled` when none is found (there's nothing to render against). When `enabled`, project-level test conventions live under `skills.write-unit-test` (test style, data setup, assertion style, DB isolation, inferred extra conventions).",
|
|
238
|
+
"type": "string",
|
|
239
|
+
"enum": ["enabled", "disabled"]
|
|
235
240
|
}
|
|
236
241
|
}
|
|
237
242
|
},
|
|
@@ -479,6 +484,44 @@
|
|
|
479
484
|
"default": "one-off-runs/"
|
|
480
485
|
}
|
|
481
486
|
}
|
|
487
|
+
},
|
|
488
|
+
"write-unit-test": {
|
|
489
|
+
"description": "Configuration for the `write-unit-test` foundation skill. Optional when `foundation_skills.write-unit-test` is `enabled` — every field defaults to `framework-default`, deferring to the shipped framework pack. At bootstrap, values are inferred from the existing test suite (see `vinta-analyze-codebase` inventory `tests.conventions`) and confirmed by the user; on a greenfield project they're asked directly. Rendered into the skill body at derive time.",
|
|
490
|
+
"type": "object",
|
|
491
|
+
"additionalProperties": false,
|
|
492
|
+
"properties": {
|
|
493
|
+
"test_style": {
|
|
494
|
+
"description": "Whether new tests are written as standalone functions or grouped in test classes. `functions` (pytest-style `def test_x()`), `classes` (TestCase / describe-block grouping), or `framework-default` (defer to the framework pack — functions for pytest/vitest/jest, classes for Django TestCase). Inferred from the dominant shape in the existing suite.",
|
|
495
|
+
"type": "string",
|
|
496
|
+
"enum": ["functions", "classes", "framework-default"],
|
|
497
|
+
"default": "framework-default"
|
|
498
|
+
},
|
|
499
|
+
"data_setup": {
|
|
500
|
+
"description": "How test data is constructed. `factories` (factory classes like factory_boy / FactoryBot / builder helpers — preferred when the project has them), `fixtures` (framework fixtures / fixture files), `inline` (built in the test body), or `framework-default`. Inferred from `tests.test_data_helpers` and the dominant pattern in the suite.",
|
|
501
|
+
"type": "string",
|
|
502
|
+
"enum": ["factories", "fixtures", "inline", "framework-default"],
|
|
503
|
+
"default": "framework-default"
|
|
504
|
+
},
|
|
505
|
+
"assertion_style": {
|
|
506
|
+
"description": "Which assertion form new tests use. `plain-assert` (Python plain `assert` / pytest introspection, or JS `expect`), `framework-methods` (`unittest` `self.assertEqual` family), or `framework-default`. Inferred from the existing suite — a pytest suite that already uses plain `assert` sets `plain-assert`; a `TestCase`-heavy suite sets `framework-methods`.",
|
|
507
|
+
"type": "string",
|
|
508
|
+
"enum": ["plain-assert", "framework-methods", "framework-default"],
|
|
509
|
+
"default": "framework-default"
|
|
510
|
+
},
|
|
511
|
+
"db_isolation": {
|
|
512
|
+
"description": "How data created by a test is undone between tests. `transaction-rollback` (each test runs in a transaction rolled back at teardown — preferred where supported), `truncate` (tables truncated between tests, e.g. Django TransactionTestCase), `recreate` (schema/DB rebuilt), or `framework-default`. Inferred from the test base classes / fixtures in use.",
|
|
513
|
+
"type": "string",
|
|
514
|
+
"enum": ["transaction-rollback", "truncate", "recreate", "framework-default"],
|
|
515
|
+
"default": "framework-default"
|
|
516
|
+
},
|
|
517
|
+
"additional_conventions": {
|
|
518
|
+
"description": "Free-form project-specific test conventions the four structured fields can't capture — inferred from the existing suite at bootstrap (e.g. `co-locate tests next to source as *.test.ts`, `one assertion per test`, `name tests should_<behavior>_when_<condition>`, `mock the network with our recorded-cassette helper`) or supplied by the user. Rendered verbatim into the skill body's project-conventions section. Empty array is fine — the universal rules + framework pack already cover the essentials.",
|
|
519
|
+
"type": "array",
|
|
520
|
+
"items": { "type": "string", "minLength": 1 },
|
|
521
|
+
"uniqueItems": true,
|
|
522
|
+
"default": []
|
|
523
|
+
}
|
|
524
|
+
}
|
|
482
525
|
}
|
|
483
526
|
}
|
|
484
527
|
}
|
|
@@ -62,6 +62,8 @@ Match top-level deps against this table:
|
|
|
62
62
|
| `flask` | Flask |
|
|
63
63
|
| `pytest` | pytest |
|
|
64
64
|
| `mypy`, `ruff`, `black` | Python lint/type tools |
|
|
65
|
+
| `@tanstack/react-start`, `@tanstack/start` | TanStack Start (check `@tanstack/react-router`) |
|
|
66
|
+
| `react-router`, `react-router-dom`, `@react-router/*` | React Router |
|
|
65
67
|
| `tRPC`, `@trpc/*` | tRPC |
|
|
66
68
|
| `prisma`, `@prisma/client` | Prisma |
|
|
67
69
|
| `drizzle-orm` | Drizzle |
|
|
@@ -94,6 +96,12 @@ For each app/package: name + path + framework (cross-ref the [Frameworks (from d
|
|
|
94
96
|
- Unit / integration: where are they? `apps/*/src/**/*.test.ts`, `tests/`, `__tests__/`, `<pkg>/tests/unit`, `<pkg>/tests/integration`. Glob.
|
|
95
97
|
- E2E: `e2e/`, `playwright/`, `cypress/`. Read fixtures + setup files for: auth strategy, seed helpers, tenant scoping in seeds.
|
|
96
98
|
- Test data: factories, fixtures, mocks. Note conventions (`@medplum/mock`, `factory_boy`, MSW, etc).
|
|
99
|
+
- **Unit-test conventions (feeds `write-unit-test`).** Read a representative sample of existing unit tests (not e2e) and infer the **dominant** pattern for each axis below. These pre-fill `skills.write-unit-test.*` in the bootstrap interview (group C.9), so the shipped skill matches the suite instead of imposing a house style. Report `framework-default` for any axis where the suite is mixed or there's no signal (greenfield):
|
|
100
|
+
- **test_style** — `functions` (standalone `def test_x` / bare `it(...)`) vs `classes` (`TestCase` subclasses, `describe` grouping). Which dominates?
|
|
101
|
+
- **data_setup** — `factories` (factory_boy / FactoryBot / builder helpers — cross-ref `test_data_helpers`), `fixtures` (pytest fixtures / fixture files / `setUpTestData`), or `inline` (objects built in the test body).
|
|
102
|
+
- **assertion_style** — `plain-assert` (Python `assert` / JS `expect`) vs `framework-methods` (`self.assertEqual`-family).
|
|
103
|
+
- **db_isolation** — `transaction-rollback` (Django `TestCase`, pytest-django `db`, SQLAlchemy savepoint fixtures), `truncate` (`TransactionTestCase`), or `recreate`. Read the test base classes / DB fixtures to tell.
|
|
104
|
+
- **extras** — any other recurring convention worth carrying (test file co-location + naming pattern, one-assertion-per-test, a house HTTP-mock helper / recorded-cassette pattern, custom assertion helpers). List them; they seed `additional_conventions`.
|
|
97
105
|
|
|
98
106
|
### 7. Deploy / release model
|
|
99
107
|
|
|
@@ -155,7 +163,7 @@ This is a load-bearing input for the rest of the bootstrap flow — surface ever
|
|
|
155
163
|
For each skill, also classify:
|
|
156
164
|
|
|
157
165
|
- `vinta-managed` — directory name starts with `vinta-` (installed by `vinta-ai-workflows`; can be left alone or refreshed via that CLI).
|
|
158
|
-
- `foundation-shape` — name matches the Vinta foundation set (`plan-feature`, `create-spec`, `create-qa-use-cases`, `implement-plan`, `implement-phase`, `review-phase`, `integrate-phase`, `amend-plan`, `add-e2e-test`, `add-env-var`). `implement-phase` / `review-phase` / `integrate-phase` are the plan-execution sub-skills co-shipped with `implement-plan`.
|
|
166
|
+
- `foundation-shape` — name matches the Vinta foundation set (`plan-feature`, `create-spec`, `create-qa-use-cases`, `implement-plan`, `implement-phase`, `review-phase`, `integrate-phase`, `amend-plan`, `add-e2e-test`, `add-env-var`, `write-unit-test`). `implement-phase` / `review-phase` / `integrate-phase` are the plan-execution sub-skills co-shipped with `implement-plan`.
|
|
159
167
|
- `project-custom` — anything else; written by the team.
|
|
160
168
|
|
|
161
169
|
**Sub-agents** — list every agent file with its name + description:
|
|
@@ -273,6 +281,12 @@ tests:
|
|
|
273
281
|
e2e_framework: playwright | cypress | none
|
|
274
282
|
e2e_paths: [<path>]
|
|
275
283
|
test_data_helpers: [<file>] # factories, mock clients, seed scripts
|
|
284
|
+
conventions: # inferred dominant patterns → skills.write-unit-test.* (framework-default when mixed / no signal)
|
|
285
|
+
test_style: functions | classes | framework-default
|
|
286
|
+
data_setup: factories | fixtures | inline | framework-default
|
|
287
|
+
assertion_style: plain-assert | framework-methods | framework-default
|
|
288
|
+
db_isolation: transaction-rollback | truncate | recreate | framework-default
|
|
289
|
+
extras: [<free-form recurring convention>] # seeds additional_conventions; [] when none
|
|
276
290
|
|
|
277
291
|
deploy:
|
|
278
292
|
targets: [vercel, aws-ecs, heroku, k8s, medplum-bots, npm-publish, ...]
|
|
@@ -81,7 +81,19 @@ These bleed across sub-skills, so capture once now:
|
|
|
81
81
|
|
|
82
82
|
Lands in `.vinta-ai-workflows.yaml` `policies.commit_strategy`. Drives the `implement-plan` template's branch / commit / PR-opening flow + the `amend-plan` skill's supported-strategies check.
|
|
83
83
|
|
|
84
|
-
9. **
|
|
84
|
+
9. **Test conventions for `write-unit-test`.** The `write-unit-test` foundation skill ships **by default whenever a unit-test framework is detected** (item 2 above) — it's not an opt-in in group D. Capture the project's test conventions so the rendered skill matches the suite instead of imposing a house style. Land them under `skills.write-unit-test.*` in `.vinta-ai-workflows.yaml`.
|
|
85
|
+
|
|
86
|
+
**On an existing project, infer first, then confirm.** [vinta-analyze-codebase](../vinta-analyze-codebase/SKILL.md#6-tests)'s `tests.conventions` block reports the dominant patterns it read from the suite (test style, data setup, assertion style, DB isolation, plus any recurring extras). Present those inferred values as the defaults and ask only "these match? anything to change?". On a greenfield project with no tests yet, ask directly (recommend `framework-default` for all four — the shipped framework pack already encodes the idiomatic choice). Skip this item entirely when no test framework was detected (`foundation_skills.write-unit-test: disabled`).
|
|
87
|
+
|
|
88
|
+
Four structured `AskUserQuestion` fields (each offers a `Framework default` option — pick it to defer to the pack):
|
|
89
|
+
- **Test style** — `Standalone functions`, `Test classes`, `Framework default`. → `test_style`.
|
|
90
|
+
- **Test-data setup** — `Factory classes / builders`, `Fixtures`, `Inline in the test`, `Framework default`. → `data_setup`.
|
|
91
|
+
- **Assertion style** — `Plain assert / expect`, `Framework assertion methods (self.assertEqual …)`, `Framework default`. → `assertion_style`.
|
|
92
|
+
- **DB isolation** — `Transaction rollback`, `Truncate between tests`, `Recreate schema/DB`, `Framework default`. → `db_isolation`.
|
|
93
|
+
|
|
94
|
+
**Additional conventions (optional).** Open prose: *"Any recurring test conventions the four fields above don't capture? (e.g. 'co-locate tests as `*.test.ts` next to source', 'name tests `should_<behavior>_when_<condition>`', 'mock HTTP with our recorded-cassette helper'.)"* On an existing project, seed this from the extras `vinta-analyze-codebase` surfaced and let the user trim. Each entry lands in `skills.write-unit-test.additional_conventions[]`. Leave empty when there's nothing project-specific — the universal rules + framework pack already cover the essentials.
|
|
95
|
+
|
|
96
|
+
10. **Agent model tiers (optional).** The plan-execution family picks the **implementer** model per phase from the plan itself, but the **reviewer**, **fixer**, and the mechanical steps (**worktree prep**, **opening the PR / integrate**) have no model input unless the project sets one. Offer to pin each to a **tier** (1–4) into the same table `plan-feature` uses — `ai-tools/skills/plan-feature/resources/ai-models.yaml` — so review runs on a capable model while cheap mechanical work runs on a cheap one. `AskUserQuestion`: `Use recommended defaults (reviewer 3, fixer 2, worktree prep 1, integrate 1)`, `Customize each`, `Leave unset — every spawn uses the runtime default`. On `Customize each`, ask a per-role tier (1–4) for `reviewer` / `fixer` / `worktree_prep` / `integrate`. Land the chosen tiers in `.vinta-ai-workflows.yaml` `agent_models`; omit any role the user leaves unset (unset → runtime default). Recommend the defaults for most teams; recommend `Leave unset` only when the runtime exposes a single model anyway. Note `worktree_prep` is a no-op unless `prepare-worktree` is enabled, and setting `worktree_prep` / `integrate` makes `implement-plan` delegate those steps to a cheap subagent instead of running them inline.
|
|
85
97
|
|
|
86
98
|
### D. Optional foundation skills
|
|
87
99
|
|
|
@@ -144,7 +156,7 @@ For each artifact, read it (frontmatter + body), then ask the user via `AskUserQ
|
|
|
144
156
|
- `Keep in current vendor path, don't touch` — leaves it where it is. AGENTS.md may reference it; downstream skill setup won't manage it.
|
|
145
157
|
- `Drop` — delete (rare; usually the user wants to migrate).
|
|
146
158
|
|
|
147
|
-
Foundation-shape skills (name matches `plan-feature`, `create-spec`, `create-qa-use-cases`, `implement-plan`, `implement-phase`, `review-phase`, `integrate-phase`, `amend-plan`, `add-e2e-test`, `add-env-var`, `add-one-off-script`, `prepare-worktree`, `thermo-nuclear-code-quality-review`, `deslop-comments`, `handoff`, `handoff-to-client`) get an extra option: `Replace with Vinta foundation version` — overwrites with the canonical foundation content, preserving the user's name. Useful when the existing version is stale. (`implement-phase` / `review-phase` / `integrate-phase` are the plan-execution sub-skills co-shipped with `implement-plan`; replace them as a unit.)
|
|
159
|
+
Foundation-shape skills (name matches `plan-feature`, `create-spec`, `create-qa-use-cases`, `implement-plan`, `implement-phase`, `review-phase`, `integrate-phase`, `amend-plan`, `add-e2e-test`, `add-env-var`, `add-one-off-script`, `prepare-worktree`, `thermo-nuclear-code-quality-review`, `deslop-comments`, `handoff`, `handoff-to-client`, `write-unit-test`) get an extra option: `Replace with Vinta foundation version` — overwrites with the canonical foundation content, preserving the user's name. Useful when the existing version is stale. (`implement-phase` / `review-phase` / `integrate-phase` are the plan-execution sub-skills co-shipped with `implement-plan`; replace them as a unit.)
|
|
148
160
|
|
|
149
161
|
- **Sub-agents** (each under any vendor `agents/` dir):
|
|
150
162
|
- `Migrate to ai-tools/agents/<name>.yaml` — converts vendor-specific format → canonical YAML; `setup-ai-tools.mjs` re-emits per-vendor copies.
|
|
@@ -266,20 +278,21 @@ foundation_skills:
|
|
|
266
278
|
deslop-comments: enabled # always ships — review-phase Layer 2 comment-hygiene + fix loop depend on it
|
|
267
279
|
handoff: enabled # always ships — project-agnostic session-continuation handoff docs
|
|
268
280
|
handoff-to-client: <Optional foundation skills → handoff-to-client answer → enabled | disabled; only asked for API-only repos>
|
|
281
|
+
write-unit-test: <enabled when a unit-test framework was detected (Project conventions → Test framework(s)); disabled when none found — default-on, not an opt-in>
|
|
269
282
|
|
|
270
283
|
foundation_agents:
|
|
271
284
|
implementer: enabled
|
|
272
285
|
reviewer: enabled
|
|
273
286
|
fixer: enabled
|
|
274
287
|
|
|
275
|
-
# Only emit the roles the user chose in the "Agent model tiers" question (C.
|
|
288
|
+
# Only emit the roles the user chose in the "Agent model tiers" question (C.10).
|
|
276
289
|
# Omit the whole block on "Leave unset"; omit any individual role left unset.
|
|
277
290
|
# Each value is a tier (1–4) into ai-tools/skills/plan-feature/resources/ai-models.yaml.
|
|
278
291
|
agent_models:
|
|
279
|
-
reviewer: <C.
|
|
280
|
-
fixer: <C.
|
|
281
|
-
worktree_prep: <C.
|
|
282
|
-
integrate: <C.
|
|
292
|
+
reviewer: <C.10 → reviewer tier, default 3>
|
|
293
|
+
fixer: <C.10 → fixer tier, default 2>
|
|
294
|
+
worktree_prep: <C.10 → worktree_prep tier, default 1; omit when prepare-worktree disabled>
|
|
295
|
+
integrate: <C.10 → integrate tier, default 1>
|
|
283
296
|
|
|
284
297
|
stacks: <Stack detection → matched stacks>
|
|
285
298
|
stack_specialist_agents: <empty unless user supplied templates>
|
|
@@ -305,6 +318,17 @@ skills:
|
|
|
305
318
|
api_spec_path: <handoff-to-client follow-up — repo-relative spec path + regen command; omit when none>
|
|
306
319
|
output_dir: .vinta-ai-workflows/client-handoffs
|
|
307
320
|
|
|
321
|
+
# Only emit this block when foundation_skills.write-unit-test = enabled.
|
|
322
|
+
# Values come from Project conventions → C.9 (Test conventions), inferred from the
|
|
323
|
+
# existing suite (analyze-codebase tests.conventions) and confirmed. Omit any field
|
|
324
|
+
# left at framework-default and the whole block when there's nothing but defaults.
|
|
325
|
+
write-unit-test:
|
|
326
|
+
test_style: <C.9 → functions | classes | framework-default>
|
|
327
|
+
data_setup: <C.9 → factories | fixtures | inline | framework-default>
|
|
328
|
+
assertion_style: <C.9 → plain-assert | framework-methods | framework-default>
|
|
329
|
+
db_isolation: <C.9 → transaction-rollback | truncate | recreate | framework-default>
|
|
330
|
+
additional_conventions: <C.9 → free-form array of extra conventions inferred/supplied; [] when none>
|
|
331
|
+
|
|
308
332
|
# Only emit this block when foundation_skills.prepare-worktree = enabled.
|
|
309
333
|
prepare-worktree:
|
|
310
334
|
worktree_root: <prepare-worktree follow-up — `.claude/worktrees` | `../<repo>-wt-`>
|
|
@@ -388,6 +412,11 @@ ai-tools/
|
|
|
388
412
|
│ ├── add-env-var/SKILL.md ← optional — only if user opts in
|
|
389
413
|
│ ├── systematic-debugging/SKILL.md ← optional — only if user opts in (template-rendered)
|
|
390
414
|
│ ├── handoff-to-client/SKILL.md ← optional — only for API-only repos, if user opts in (template-rendered; config under skills.handoff-to-client.*)
|
|
415
|
+
│ ├── write-unit-test/ ← default-on when a unit-test framework was detected (template-rendered; config under skills.write-unit-test.*)
|
|
416
|
+
│ │ ├── SKILL.md
|
|
417
|
+
│ │ └── resources/packs/ ← verbatim packs for the detected runner + matched stacks only
|
|
418
|
+
│ │ ├── runners/<runner>.md ← one: pytest.md / vitest.md / jest.md
|
|
419
|
+
│ │ └── stacks/<stack>.md ← only matched: django / fastapi / flask / medplum / react / nextjs / tanstack-start / react-router / prisma / python-package / typescript-package
|
|
391
420
|
│ ├── add-one-off-script/ ← optional — only if user opts in (verbatim copy + bundled BaseOneOffScript + LocalRuntime templates)
|
|
392
421
|
│ │ ├── SKILL.md
|
|
393
422
|
│ │ └── resources/
|
|
@@ -424,7 +453,7 @@ Plus the symlinks + per-vendor generated files, set up by the install step.
|
|
|
424
453
|
Foundation skills break into three buckets — see [vinta-derive-skills](../vinta-derive-skills/SKILL.md) for the full mechanics:
|
|
425
454
|
|
|
426
455
|
- **Always copy verbatim**: `plan-feature`, `create-spec`, `create-qa-use-cases`, `deslop-comments`, `handoff`. Bundled with the bootstrap skill set; project-agnostic enough to ship as-is (with light path scrubs). `deslop-comments` always ships because `review-phase`'s Layer 2 comment-hygiene check + fix loop dispatch it. `handoff` always ships because its session-continuation body is fully project-agnostic.
|
|
427
|
-
- **Always generate**: the plan-execution unit — `implement-plan` (conductor) + its co-shipped sub-skills `implement-phase` / `review-phase` / `integrate-phase`, plus `amend-plan` (conductor). Bodies have too much project-specific content (test commands, branch convention, PR + co-author policy, agent dispatch) — generated from parameterized templates + shared partials using interview answers + inventory. The sub-skills are not independently opt-in; they always ship with the conductors.
|
|
456
|
+
- **Always generate**: the plan-execution unit — `implement-plan` (conductor) + its co-shipped sub-skills `implement-phase` / `review-phase` / `integrate-phase`, plus `amend-plan` (conductor). Bodies have too much project-specific content (test commands, branch convention, PR + co-author policy, agent dispatch) — generated from parameterized templates + shared partials using interview answers + inventory. The sub-skills are not independently opt-in; they always ship with the conductors. `write-unit-test` is also template-rendered but **conditionally generated** — it ships whenever a unit-test framework was detected (not an opt-in question, not ask-first): the rendered body bakes in the project's test command + captured `skills.write-unit-test.*` conventions, and the verbatim packs land in its `resources/packs/` — one runner pack for the detected test framework plus a stack pack per matched stack (Django / FastAPI / Flask / Medplum / React / Next.js / TanStack Start / React Router / Prisma / pure-package), so framework-specific advice only ships when that framework is present. A few stack packs also apply a one-time project-setup step at derive time (e.g. Medplum wires the two FHIR-indexing Vitest setup files — `test.globalSetup.ts` + `test.setup.ts` — into the Vitest config so `MockClient` search filtering works) — a project mutation done once, not per test.
|
|
428
457
|
- **Optional, ask first**: `add-e2e-test`, `add-env-var`, `systematic-debugging`, `add-one-off-script`, `prepare-worktree`, `thermo-nuclear-code-quality-review`, `handoff-to-client`. Skipped by default; orchestrator asks via `AskUserQuestion` whether the project has the relevant flow at all. `add-e2e-test` / `add-env-var`: if yes + user has a template → copy + adapt; if yes + no template → draft from scratch via interview; if no → don't ship. `systematic-debugging`: if yes → render the bundled template plus the per-tool MCP catalogue blocks for the observability tools selected in its follow-up; if no → don't ship. `add-one-off-script`: if yes → copy the bundled SKILL.md verbatim plus the language-specific `BaseOneOffScript` template (`one_off_script_base.py` / `one_off_script_base.ts`) chosen via its follow-up; if no → don't ship. `prepare-worktree`: if yes → copy the bundled SKILL.md verbatim, populate `skills.prepare-worktree.*` defaults from its follow-ups, and (when the user opted in via the worktree-default follow-up) flip `run_options.implement-plan.use_worktree` to `true` so `implement-plan`'s Step 0 question (c) defaults to yes; if no → don't ship. `thermo-nuclear-code-quality-review`: if yes → copy the bundled SKILL.md verbatim (no follow-up config); if no → don't ship. `handoff-to-client`: asked only for API-only repos; if yes → render the bundled template using `skills.handoff-to-client.*` (client platforms, API style, spec path, output dir) from its follow-ups; if no → don't ship.
|
|
429
458
|
|
|
430
459
|
Stack-specific skills + agents land in the target only when the user provides templates for them. If they don't have templates yet, the orchestrator records the detected stacks + skill categories as a TODO list the user can address later via [vinta-derive-skills](../vinta-derive-skills/SKILL.md) / [vinta-derive-subagents](../vinta-derive-subagents/SKILL.md) standalone runs.
|
|
@@ -26,7 +26,7 @@ Before drafting any new SKILL.md, walk through every entry in `existing_ai_artif
|
|
|
26
26
|
- **Migrate to `ai-tools/skills/<name>/`** — `git mv` the existing skill folder (with all its resources) into the canonical layout. After move, scan the body for hard-coded vendor paths that no longer apply (`.cursor/skills/...` self-references, etc.) and rewrite to the new path. `setup-ai-tools.mjs` will re-link to the chosen vendors.
|
|
27
27
|
- **Keep in current vendor path, don't touch** — leave it where it is; AGENTS.md may reference it; downstream skill setup won't manage it. **Don't ship a foundation duplicate that would shadow it.**
|
|
28
28
|
- **Drop** — log removal; don't emit anything.
|
|
29
|
-
- **Replace with Vinta foundation version** (foundation-shape only — `plan-feature`, `create-spec`, `create-qa-use-cases`, `implement-plan`, `add-e2e-test`, `add-env-var`, `add-one-off-script`, `prepare-worktree`, `thermo-nuclear-code-quality-review`, `deslop-comments`, `handoff`, `handoff-to-client`) — proceed with the bucket A / B / C flow below for that name. The user has explicitly opted into overwriting their version.
|
|
29
|
+
- **Replace with Vinta foundation version** (foundation-shape only — `plan-feature`, `create-spec`, `create-qa-use-cases`, `implement-plan`, `add-e2e-test`, `add-env-var`, `add-one-off-script`, `prepare-worktree`, `thermo-nuclear-code-quality-review`, `deslop-comments`, `handoff`, `handoff-to-client`, `write-unit-test`) — proceed with the bucket A / B / C flow below for that name. The user has explicitly opted into overwriting their version.
|
|
30
30
|
- **`vinta-managed`** (any skill whose dir starts with `vinta-`) — leave alone. These come from the `vinta-ai-workflows` CLI; `vinta-derive-skills` doesn't manage them.
|
|
31
31
|
|
|
32
32
|
For each name in the foundation set: only emit it (verbatim copy / generate from template / interview-draft) when no existing skill of that name is being migrated or kept. If the user said `Migrate` or `Keep` for an existing `plan-feature/SKILL.md`, **do not overwrite**.
|
|
@@ -67,7 +67,7 @@ The **always-on unit** is `plan-feature` + `create-spec` + `open-pr-from-context
|
|
|
67
67
|
|
|
68
68
|
After copying, scan each for project-specific path references that no longer match the target (the bundled copies may still carry source-repo paths such as `<source-repo>/ai-plans/` or `apps/<service>/`). Replace with the target's paths from the inventory.
|
|
69
69
|
|
|
70
|
-
### B. Generate — the **plan-execution unit** + `systematic-debugging` from templates
|
|
70
|
+
### B. Generate — the **plan-execution unit** + `systematic-debugging` + `write-unit-test` from templates
|
|
71
71
|
|
|
72
72
|
Project-specific skills, generated from templates because their bodies cite real test commands, branch conventions, agent dispatch table, PR / co-author policy, or selected MCP tools.
|
|
73
73
|
|
|
@@ -85,6 +85,41 @@ The five are a **co-shipped unit**: `implement-plan` is always generated, so `im
|
|
|
85
85
|
|
|
86
86
|
- **`handoff-to-client`** — generate a markdown API-change handoff document for the client teams consuming an API-only repo: endpoints/operations added / changed / deprecated / removed on the current branch vs the default branch, with request/response shapes, breaking-change flags, and per-platform migration notes. From [resources/handoff-to-client-template.md](resources/handoff-to-client-template.md). **Opt-in** — only generated when `foundation_skills.handoff-to-client` is `enabled` in `.vinta-ai-workflows.yaml` (the bootstrap interview asks only when the repo looks API-only with separate client codebases). The rendered body bakes in `skills.handoff-to-client.*` config: client platforms, API style, optional API spec path, output dir. Standalone — no cross-links into the plan-execution unit.
|
|
87
87
|
|
|
88
|
+
- **`write-unit-test`** — write durable unit tests for a unit of behavior, enforcing the universal rules (mock only genuine externals, assert the full expected output not counts/truthiness, clean up created data or roll back the transaction, never hit an external service unavailable in dev/test, keep test logic literal, decouple from internals) plus this project's captured test conventions. From [resources/write-unit-test-template.md](resources/write-unit-test-template.md). **Default-on, not ask-first** — generated whenever `foundation_skills.write-unit-test` is `enabled`, which bootstrap sets automatically when `vinta-analyze-codebase` detected a unit-test framework (`disabled` → skip; nothing to render against). The rendered body bakes in `skills.write-unit-test.*` config: test style, data setup, assertion style, DB isolation, plus any inferred `additional_conventions`. Standalone — no cross-links into the plan-execution unit.
|
|
89
|
+
|
|
90
|
+
**Ship the packs too — two axes, matched independently.** Alongside the rendered `SKILL.md`, copy the **verbatim** packs the project needs from [resources/write-unit-test-packs/](resources/write-unit-test-packs/) into `ai-tools/skills/write-unit-test/resources/packs/`. There are two kinds, selected on **different** signals so framework-specific advice only ships when its framework is actually present:
|
|
91
|
+
|
|
92
|
+
- **Runner pack** (`resources/packs/runners/<runner>.md` → shipped to `.../packs/runners/`) — the test-runner idioms (structure, assertions, mocking). Select **one (or, in a polyglot repo, one per language)** by `inventory.tests.unit_framework`: `pytest` → `pytest.md`; `vitest` → `vitest.md`; `jest` → `jest.md`. Runner packs are **stack-agnostic** — they contain no Django/FastAPI/Medplum specifics.
|
|
93
|
+
- **Stack pack(s)** (`resources/packs/stacks/<stack>.md` → shipped to `.../packs/stacks/`) — DB isolation, framework client/test-client, domain mocks. Ship **every** pack whose stack the inventory matched (a project can match several — e.g. `nextjs` + `react`, or `tanstack-start` + `react`). Match on `inventory.frameworks` / `inventory.stacks`:
|
|
94
|
+
|
|
95
|
+
| Detected signal | Stack pack |
|
|
96
|
+
|---|---|
|
|
97
|
+
| Django | `django.md` |
|
|
98
|
+
| FastAPI | `fastapi.md` |
|
|
99
|
+
| Flask | `flask.md` |
|
|
100
|
+
| Medplum (`@medplum/*`) | `medplum.md` |
|
|
101
|
+
| React (renders UI) | `react.md` |
|
|
102
|
+
| Next.js | `nextjs.md` |
|
|
103
|
+
| TanStack Start (`@tanstack/react-start`) | `tanstack-start.md` |
|
|
104
|
+
| React Router (`react-router` / `@react-router/*`) | `react-router.md` |
|
|
105
|
+
| Prisma (`@prisma/client`) | `prisma.md` |
|
|
106
|
+
| Standalone Python package (no web framework) | `python-package.md` |
|
|
107
|
+
| Standalone TS package / lib (no framework) | `typescript-package.md` |
|
|
108
|
+
|
|
109
|
+
**Only ship what matched — never dump the whole directory.** A FastAPI project must not receive `django.md`; a pure-library project gets its `*-package.md` and **no** web-framework pack. When no stack pack matches (e.g. a plain script repo), ship the runner pack alone — the rendered body already falls back to the universal rules + the nearest existing tests. When the runner or a clearly-present stack has no pack here, ship nothing for it and record the gap in the orchestrator's final summary (a candidate for a new pack via the source repo's `add-foundation-skill` flow). The `{{FRAMEWORK_PACK_LOADER_BLOCK}}` + `{{FRAMEWORK_PACKS_LIST}}` placeholders are rendered from exactly the set of packs you shipped.
|
|
110
|
+
|
|
111
|
+
**Some stack packs need a one-time project-setup step, done here — not per test.** A pack may require a project-level test-setup fixture that must exist before any test runs; wire it once at derive time so no individual test has to. Currently:
|
|
112
|
+
|
|
113
|
+
- **Medplum** (Vitest) — `MockClient` search filtering (`searchResources` / `searchOne` with filters) silently returns nothing unless the FHIR structure definitions + search parameters are indexed in the test process. When you ship `medplum.md`, also install the **two** bundled Vitest setup files and wire both hooks:
|
|
114
|
+
- Copy [resources/write-unit-test-packs/stacks/medplum-test.globalSetup.ts](resources/write-unit-test-packs/stacks/medplum-test.globalSetup.ts) → the project's setup dir (commonly `src/test.globalSetup.ts`) and register it in `vitest.config.*` under `test.globalSetup`. It indexes the bundles once in the main process and shares `globalSchema.types` to every worker via `project.provide`.
|
|
115
|
+
- Copy [resources/write-unit-test-packs/stacks/medplum-test.setup.ts](resources/write-unit-test-packs/stacks/medplum-test.setup.ts) → `src/test.setup.ts` and register it under `test.setupFiles`. It `inject`s the shared index and assigns it at **module scope** (not `beforeAll` — timing matters).
|
|
116
|
+
- **Deps:** `@medplum/definitions` as a devDependency; Vitest **3+** (the `provide`/`inject` API); Node **≥ 22.4** or add `ws` for the WebSocket polyfill (@medplum/core v5). Flag any of these that are missing.
|
|
117
|
+
- **Version alignment:** `@medplum/core`, `@medplum/definitions`, `@medplum/mock`, `@medplum/fhirtypes` must be on matching versions.
|
|
118
|
+
- **Idempotent + merge, don't clobber:** if the project already has `test.setup.ts` / `test.globalSetup.ts`, merge the indexing + inject/provide logic in rather than overwriting; if it already does this, do nothing. Create the files + point the runner config at them only when absent.
|
|
119
|
+
- **Jest-based Medplum project** (rare — the catalog is Vitest-first): there's no bundled Jest variant; index the FHIR bundles in a single `setupFilesAfterEnv` file instead and record it as a follow-up to add a Jest snippet to the catalogue.
|
|
120
|
+
|
|
121
|
+
This is a project mutation — do it once at bootstrap. Canonical source is the team's snippet catalogue (`vintasoftware/medplum-snippet-catalog`, `TestSetup/`); the bundled copies (dated in their headers) keep bootstrap offline-safe — prefer the live catalogue if it has advanced. The rendered `medplum.md` only *mentions* the override so tests don't re-index.
|
|
122
|
+
|
|
88
123
|
Every plan-execution shell + `systematic-debugging` consumes the placeholder set below — render each by substituting from the inventory + Step 0 interview answers. `systematic-debugging` reuses the same set plus two extra placeholders (`{{OBSERVABILITY_MCP_BLOCK}}`, `{{OBSERVABILITY_MCP_LIST}}`) rendered from the catalogue.
|
|
89
124
|
|
|
90
125
|
#### Rendering the plan-execution unit
|
|
@@ -150,6 +185,10 @@ The remaining per-shell placeholders:
|
|
|
150
185
|
| `{{CLIENT_PLATFORMS_LIST}}` | `skills.handoff-to-client.client_platforms` (handoff-to-client only) | comma-separated, e.g. `iOS (Swift), Android (Kotlin), web SPA (React)` |
|
|
151
186
|
| `{{CLIENT_HANDOFF_DIR}}` | `skills.handoff-to-client.output_dir` (handoff-to-client only) | `.vinta-ai-workflows/client-handoffs` (default) |
|
|
152
187
|
| `{{API_SPEC_BLOCK}}` | derived from `skills.handoff-to-client.api_spec_path` (handoff-to-client only) | when set: a paragraph instructing the agent to regenerate/read the spec at that path on both base and branch and diff the two as the primary change-enumeration source (cross-checking against the code diff). When absent: a one-line note that no machine-readable spec exists, so shapes come from route/serializer code alone. |
|
|
188
|
+
| `{{FRAMEWORK_PACKS_LIST}}` | derived from the packs shipped (write-unit-test only) | comma-separated names of the runner + stack packs that landed under `resources/packs/`, e.g. `pytest + Django` or `vitest, React, TanStack Start`. Renders `the project's test framework` when only the runner shipped with no stack pack. |
|
|
189
|
+
| `{{FRAMEWORK_PACK_LOADER_BLOCK}}` | derived from the packs shipped (write-unit-test only) | bullets pointing at each shipped pack, runner first then stacks: `- Runner: **pytest** → read [resources/packs/runners/pytest.md](resources/packs/runners/pytest.md).` then `- Stack: **Django** → read [resources/packs/stacks/django.md](resources/packs/stacks/django.md).` (one bullet per matched stack). When only the runner shipped, emit just its bullet plus `- No stack pack matched — apply the universal rules and mirror the nearest existing tests.` When no pack shipped at all, render that single fallback line alone. |
|
|
190
|
+
| `{{PROJECT_TEST_PREFERENCES_BLOCK}}` | derived from `skills.write-unit-test.{test_style,data_setup,assertion_style,db_isolation}` (write-unit-test only) | one bullet per field whose value is **not** `framework-default`, phrased as an instruction: `test_style: classes` → `- Group tests in test classes (not standalone functions).`; `data_setup: factories` → `- Build test data with factory classes / builders, not inline literals or fixture files.`; `assertion_style: plain-assert` → `- Use plain \`assert\` (not \`self.assertEqual\`-family methods).`; `db_isolation: transaction-rollback` → `- Isolate DB state with transaction rollback (each test in a rolled-back transaction).`. Emit `- No project-specific overrides — follow the framework pack's defaults.` when all four are `framework-default`. |
|
|
191
|
+
| `{{ADDITIONAL_CONVENTIONS_BLOCK}}` | derived from `skills.write-unit-test.additional_conventions[]` (write-unit-test only) | when non-empty: a `### Additional project conventions` sub-block listing each entry as a bullet, verbatim. Empty string (whole block omitted) when the array is empty. Starts with a leading blank line so it slots cleanly under the preferences block. |
|
|
153
192
|
| `{{DEPENDENCY_LICENSE_BLOCK}}` | derived from `policies.dependency_licenses` | Top-level `## Adding new third-party dependencies` section. Empty string when `enforcement: off` or the block is absent. When `block` / `warn`: short paragraph naming the enforcement mode + the forbidden SPDX list + the pre-install check (`npm view <pkg> license`, PyPI metadata, etc.) + a one-line pointer to the **Dependency licenses** section in [AGENTS.md](AGENTS.md) for the full table including `allowed_overrides` and `notes`. See [resources/dependency-license-block.md](resources/dependency-license-block.md) for the canonical body. |
|
|
154
193
|
| `{{DEPENDENCY_LICENSE_LAYER1_CHECK}}` | derived from `policies.dependency_licenses` | Layer 1 review checklist item. Empty string when `enforcement: off`. When `block` / `warn`: `6. **Dependency license scan**: \`git diff package.json pyproject.toml ...\` (project-relevant manifests) — for every added dep look up its SPDX license (\`npm view <pkg> license\`, PyPI metadata, repo \`LICENSE\`). A license in \`policies.dependency_licenses.forbidden_spdx\` and not in \`allowed_overrides\` is a BLOCKER (when \`block\`) or a SHOULD-FIX (when \`warn\`). A missing / \`UNKNOWN\` / undeclared license is **always a BLOCKER** regardless of enforcement mode — there is no override to silently bless undisclosed terms.` |
|
|
155
194
|
| `{{DEPENDENCY_LICENSE_RULE_LINE}}` | derived from `policies.dependency_licenses` | Important-rules bullet. Empty string when `enforcement: off`. When `block`: `- **License check before any new dep.** Refuse \`npm add\` / \`pnpm add\` / \`pip install\` / \`poetry add\` / \`uv add\` / \`cargo add\` / \`go get\` when the package's SPDX license is in the forbidden list — see AGENTS.md **Dependency licenses**. User can grant a one-off override after acknowledging the violation; record the override in \`policies.dependency_licenses.allowed_overrides\` before re-running.` When `warn`: same line but `Proceed but record the violation in the phase report` instead of `Refuse`. |
|
|
@@ -221,6 +260,7 @@ Render each template (expand includes → substitute placeholders → write), pr
|
|
|
221
260
|
- `ai-tools/skills/amend-plan/SKILL.md` (from `plan-execution/shell/amend-plan-template.md`)
|
|
222
261
|
- `ai-tools/skills/systematic-debugging/SKILL.md` (from `systematic-debugging-template.md`) — only when `foundation_skills.systematic-debugging: enabled`
|
|
223
262
|
- `ai-tools/skills/handoff-to-client/SKILL.md` (from `handoff-to-client-template.md`) — only when `foundation_skills.handoff-to-client: enabled`
|
|
263
|
+
- `ai-tools/skills/write-unit-test/SKILL.md` (from `write-unit-test-template.md`) **plus** `ai-tools/skills/write-unit-test/resources/packs/runners/<runner>.md` + `.../packs/stacks/<stack>.md` (verbatim from `write-unit-test-packs/`, only the detected runner + matched stacks) — only when `foundation_skills.write-unit-test: enabled` (default-on when a test framework was detected)
|
|
224
264
|
|
|
225
265
|
The first five are the **plan-execution unit** — render them together (they share partials + placeholders + cross-links); a change to one partial ripples to every shell that includes it.
|
|
226
266
|
|
|
@@ -321,4 +361,5 @@ After all skills written:
|
|
|
321
361
|
4. Foundation skills (plan-feature, create-spec, open-pr-from-context — plus create-qa-use-cases when e2e is enabled) reference each other correctly via `[name](../<name>/SKILL.md)` links. When e2e is disabled, confirm `plan-feature` has **no** surviving `<!-- e2e:start/end -->` markers and no dangling link to `create-qa-use-cases` / `add-e2e-test`.
|
|
322
362
|
5. **Plan-execution unit intact.** All five files exist; no surviving `<!-- include -->` markers; the conductor's `[implement-phase]` / `[review-phase]` / integrate-phase links resolve, and `amend-plan` + `systematic-debugging` (when shipped) → `[review-phase]` resolve. Under `ask`: both `integrate-phase-stacked` + `integrate-phase-modular` exist and the conductor's dispatch line names both.
|
|
323
363
|
6. `implement-plan` (+ the plan-execution sub-skills) cite real commands + the project's actual code host + branch / PR / co-author policy.
|
|
324
|
-
7.
|
|
364
|
+
7. **`write-unit-test` (when shipped)** renders with no surviving `{{PLACEHOLDER}}`, cites the project's real test command, and every pack path in its `{{FRAMEWORK_PACK_LOADER_BLOCK}}` resolves to a file that actually landed under `ai-tools/skills/write-unit-test/resources/packs/`. One runner pack; stack packs only for frameworks the project actually uses (no `django.md` in a FastAPI repo, no web-framework pack in a pure-library repo). **Medplum projects**: confirm both FHIR-indexing setup files exist and are registered in `vitest.config.*` (`test.globalSetup` + `test.setupFiles`), `@medplum/definitions` is a devDependency, and no test re-indexes.
|
|
365
|
+
8. Setup script ([vinta-install-ai-tools-setup](../vinta-install-ai-tools-setup/SKILL.md)) runs cleanly with the new skills in place.
|
|
@@ -0,0 +1,49 @@
|
|
|
1
|
+
# Jest runner pack
|
|
2
|
+
|
|
3
|
+
Test-runner idioms for Jest. Stack-agnostic — framework-client and domain-mock specifics live in the **stack pack**, loaded alongside this one. The Testing Library / DOM notes apply only when the project renders UI. Read with the skill's universal rules. (Jest and Vitest share most idioms; the differences are noted.)
|
|
4
|
+
|
|
5
|
+
## Structure & style
|
|
6
|
+
|
|
7
|
+
- **`describe` / `it` (or `test`) with the behavior in the name.** `it("returns zero for an empty cart")`. The string is the spec.
|
|
8
|
+
- **Assert full values with the right matcher.** `expect(result).toEqual(expected)` for deep value equality, `toBe` for primitives/identity, `toStrictEqual` when `undefined` fields / class identity matter. Avoid `toBeTruthy()` / `toBeDefined()` as the *only* assertion. For arrays assert contents (and order when it matters), not just `.toHaveLength(n)`.
|
|
9
|
+
- Error paths: `expect(fn).toThrow(Message)` — assert the error, not just that it threw.
|
|
10
|
+
|
|
11
|
+
## Table-drive instead of looping
|
|
12
|
+
|
|
13
|
+
- Use `it.each([...])` / `describe.each` — each row is its own test with a literal expected value. Never loop inside one test recomputing the expectation.
|
|
14
|
+
|
|
15
|
+
## Test data
|
|
16
|
+
|
|
17
|
+
- **Prefer small factory/builder helpers over inline object literals** when the project has them — `makeUser({ email })` returns a valid object with only the relevant field overridden. Keep overrides minimal.
|
|
18
|
+
- No factory convention → a local `base` object with per-test spread beats copy-pasted full literals.
|
|
19
|
+
|
|
20
|
+
## Mocking & externals
|
|
21
|
+
|
|
22
|
+
- **Mock only genuine externals, at the boundary.** `jest.mock("./module")`, `jest.fn()` for callbacks, `jest.spyOn(obj, "method")` for one method. Don't mock the unit under test or pure helpers.
|
|
23
|
+
- **Reset between tests.** Set `clearMocks: true` (and consider `restoreMocks: true`) in the Jest config, or `afterEach(() => jest.restoreAllMocks())`. Stale mock state between tests is a top flake source.
|
|
24
|
+
- **HTTP:** use **MSW** (`setupServer`) to intercept at the network layer and return canned responses — the real client path runs. Never let a real request out. Prefer MSW over hand-mocking `fetch`/`axios`.
|
|
25
|
+
- **Complex APIs (e.g. LLM endpoints):** when hand-writing responses is impractical, record real traffic once and replay it from a committed cassette — [PollyJS](https://github.com/Netflix/pollyjs) or `nock`'s `nock.back`. Scrub secrets/PII from the recording, commit it, and never re-hit the live API in CI. Re-record deliberately when the contract changes.
|
|
26
|
+
- **Time:** `jest.useFakeTimers()` + `jest.setSystemTime(...)`, `jest.useRealTimers()` in teardown. Don't assert on `Date.now()` live.
|
|
27
|
+
- Prefer dependency injection over `jest.mock` when practical — fewer module-graph surprises.
|
|
28
|
+
|
|
29
|
+
## UI / component tests
|
|
30
|
+
|
|
31
|
+
- When the project renders UI, the component-testing idioms (Testing Library queries, `user-event`, asserting output not internals) live in the **React stack pack** — follow it there. Pure-logic units don't need it.
|
|
32
|
+
|
|
33
|
+
## Isolation
|
|
34
|
+
|
|
35
|
+
- Keep test files independent; don't share mutable module-level state. Clean up any DOM/global you touched in `afterEach`.
|
|
36
|
+
- DB-backed tests: wrap each in a transaction rolled back in `afterEach`, or reset to a known seed. Never leave rows behind.
|
|
37
|
+
|
|
38
|
+
## Jest-specific notes
|
|
39
|
+
|
|
40
|
+
- `jest.mock(...)` is **hoisted** above imports; a mock factory can't reference outer variables unless they're prefixed `mock`. Return values from inside the factory.
|
|
41
|
+
- Automock is off by default — keep it off; explicit mocks are clearer.
|
|
42
|
+
- `jest.mock("module")` replaces the whole module for the file; use `jest.requireActual` to keep the real parts you don't mean to stub.
|
|
43
|
+
|
|
44
|
+
## Pitfalls
|
|
45
|
+
|
|
46
|
+
- Asserting on a mock's return you just configured — tests the mock, not the code.
|
|
47
|
+
- Forgetting `await` on async expectations (`await expect(p).resolves...`) — the assertion never runs; the test passes hollow.
|
|
48
|
+
- Snapshot sprawl: giant `toMatchSnapshot()` blobs get blindly re-recorded. Assert the fields that matter by value.
|
|
49
|
+
- `toEqual` ignores `undefined` properties — use `toStrictEqual` when their presence/absence is part of the contract.
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# pytest runner pack
|
|
2
|
+
|
|
3
|
+
Test-runner idioms for pytest. Stack-agnostic — DB isolation and framework-client/mock specifics live in the **stack pack** (Django / FastAPI / Flask / …), loaded alongside this one. Read with the skill's universal rules.
|
|
4
|
+
|
|
5
|
+
## Structure & style
|
|
6
|
+
|
|
7
|
+
- **Plain functions + fixtures, not `unittest.TestCase`.** `def test_x():` with fixtures is the idiomatic form. Use a plain `class TestThing:` only when it genuinely groups (shared parametrization, a natural noun) — never subclass `TestCase` in new pytest code.
|
|
8
|
+
- **Fixtures over `setUp`/`tearDown`.** Fixtures compose, scope (`function`/`module`/`session`), and `yield` for teardown. Put shared ones in `conftest.py`.
|
|
9
|
+
- **Plain `assert`, not `self.assertEqual`.** pytest rewrites `assert` to show both sides. `assert result == expected`. Unordered collections: compare `sorted(...)` or sets. Approximate floats: `pytest.approx`.
|
|
10
|
+
|
|
11
|
+
## Parametrize instead of looping
|
|
12
|
+
|
|
13
|
+
- `@pytest.mark.parametrize("input,expected", [...])` — each row is its own test with a **literal** expected value. Add `ids=` when the repr is opaque. Never loop inside one test recomputing the expectation.
|
|
14
|
+
|
|
15
|
+
## Test data
|
|
16
|
+
|
|
17
|
+
- **Prefer `factory_boy` factories over inline dicts** when the project has them — `UserFactory(email="x@y.z")` spells out only the relevant field and stays correct as the model grows. `Factory.build()` (no persistence) vs `Factory.create()` (persisted). Override only the fields the assertion depends on.
|
|
18
|
+
- No factories → a small local constructor/helper beats hand-assembled deep dicts repeated across tests.
|
|
19
|
+
|
|
20
|
+
## Mocking & externals
|
|
21
|
+
|
|
22
|
+
- **`mocker` (`pytest-mock`)** — `mocker.patch("module.where_its_used.thing")`, patching where the name is *looked up*, not where it's defined. Auto-undone at teardown.
|
|
23
|
+
- **HTTP:** `responses` (requests) / `respx` (httpx) — assert the outbound request, return a canned response. `pytest-socket` (`--disable-socket`) is a good CI guard against stray real calls.
|
|
24
|
+
- **Complex APIs (e.g. LLM endpoints):** when hand-writing responses is impractical, record real traffic once and replay it from a committed cassette — `vcrpy` (`@pytest.mark.vcr`) or `betamax`. Scrub secrets/PII from the cassette, commit it, and never re-hit the live API in CI. Re-record deliberately when the contract changes.
|
|
25
|
+
- **Time/randomness:** `freezegun` (`@freeze_time`) or inject a clock; seed or patch `random`.
|
|
26
|
+
- Don't patch your own pure functions or the unit under test — that tests the mock.
|
|
27
|
+
|
|
28
|
+
## DB isolation
|
|
29
|
+
|
|
30
|
+
- Wrap each DB-touching test in a transaction rolled back at teardown, via a fixture — the exact mechanism is stack-specific (see the stack pack). Never `DELETE FROM` in setup as a cleanup crutch, and never depend on test ordering.
|
|
31
|
+
|
|
32
|
+
## Pitfalls
|
|
33
|
+
|
|
34
|
+
- Patching the definition site instead of the SUT's namespace (`from lib import thing` → patch `sut.thing`) — the patch silently does nothing.
|
|
35
|
+
- `assert mock.called` as the only assertion — asserts an interaction, not a result.
|
|
36
|
+
- Comparing objects without value equality — give them a dataclass `__eq__` or assert the fields you mean.
|
|
@@ -0,0 +1,43 @@
|
|
|
1
|
+
# Vitest runner pack
|
|
2
|
+
|
|
3
|
+
Test-runner idioms for Vitest. Stack-agnostic — framework-client and domain-mock specifics (e.g. Medplum's `MockClient`) live in the **stack pack**, loaded alongside this one. The Testing Library / DOM notes apply only when the project renders UI. Read with the skill's universal rules.
|
|
4
|
+
|
|
5
|
+
## Structure & style
|
|
6
|
+
|
|
7
|
+
- **`describe` / `it` (or `test`) with the behavior in the name.** `it("returns zero for an empty cart")`. The string is the spec; the failure line should read as a sentence.
|
|
8
|
+
- **Assert full values with the right matcher.** `expect(result).toEqual(expected)` for deep value equality (objects/arrays), `toBe` for primitives/identity, `toStrictEqual` when `undefined` fields and class identity matter. Avoid `toBeTruthy()` / `toBeDefined()` as the *only* assertion — a wrong-but-truthy value passes. For arrays assert contents (and order when it matters), not just `.toHaveLength(n)`.
|
|
9
|
+
- Use `expect(fn).toThrow(Message)` for error paths; assert the error, not just that it threw.
|
|
10
|
+
|
|
11
|
+
## Table-drive instead of looping
|
|
12
|
+
|
|
13
|
+
- Use `it.each([...])` / `describe.each` to cover many cases — each row is its own test with a literal expected value. Never `for`-loop inside one test recomputing the expectation.
|
|
14
|
+
|
|
15
|
+
## Test data
|
|
16
|
+
|
|
17
|
+
- **Prefer small factory/builder helpers over inline object literals** when the project has them — `makeUser({ email })` returns a valid object with only the relevant field overridden, and survives new required fields on the type. Keep overrides minimal: set only what the assertion depends on.
|
|
18
|
+
- When there's no factory convention, a local `const base = {...}` with per-test spread (`{ ...base, qty: 2 }`) beats copy-pasting full literals across tests.
|
|
19
|
+
|
|
20
|
+
## Mocking & externals
|
|
21
|
+
|
|
22
|
+
- **Mock only genuine externals, at the boundary.** `vi.mock("./module")` for a whole module, `vi.fn()` for a passed-in callback, `vi.spyOn(obj, "method")` for one method. Don't mock the unit under test or pure helpers that run fine.
|
|
23
|
+
- **Reset between tests.** Set `test: { clearMocks: true }` (or `restoreMocks: true`) in the Vitest config, or `afterEach(() => vi.restoreAllMocks())` — stale mock state leaking between tests is a top flake source.
|
|
24
|
+
- **HTTP:** use **MSW** (`setupServer`) to intercept at the network layer and return canned responses — the code under test runs its real fetch/axios path. Never let a real request out. Prefer MSW over mocking `fetch` directly (which couples to the client).
|
|
25
|
+
- **Complex APIs (e.g. LLM endpoints):** when hand-writing responses is impractical, record real traffic once and replay it from a committed cassette — [PollyJS](https://github.com/Netflix/pollyjs) or `nock`'s `nock.back`. Scrub secrets/PII from the recording, commit it, and never re-hit the live API in CI. Re-record deliberately when the contract changes.
|
|
26
|
+
- **Time:** `vi.useFakeTimers()` + `vi.setSystemTime(...)`, and `vi.useRealTimers()` in teardown. Don't assert on `Date.now()` live.
|
|
27
|
+
- **Modules with side effects on import:** prefer dependency injection over `vi.mock` when you can — fewer module-graph surprises.
|
|
28
|
+
|
|
29
|
+
## UI / component tests
|
|
30
|
+
|
|
31
|
+
- When the project renders UI, the component-testing idioms (Testing Library queries, `user-event`, asserting output not internals) live in the **React stack pack** — follow it there. Pure-logic units don't need it.
|
|
32
|
+
|
|
33
|
+
## Isolation
|
|
34
|
+
|
|
35
|
+
- Vitest isolates test files by default; keep it. Don't share mutable module-level state between tests. Clean up any DOM/global you touched in `afterEach`.
|
|
36
|
+
- For DB-backed integration tests, wrap each test in a transaction rolled back in `afterEach`, or reset to a known seed — never leave rows behind.
|
|
37
|
+
|
|
38
|
+
## Pitfalls
|
|
39
|
+
|
|
40
|
+
- Asserting on a mock's return you just configured — tests the mock, not the code.
|
|
41
|
+
- `vi.mock` factory hoisting surprises: `vi.mock` is hoisted above imports; reference values via the factory, not outer closures, or use `vi.hoisted`.
|
|
42
|
+
- Forgetting `await` on async expectations (`expect(promise).resolves...`) — the assertion never runs and the test passes hollow.
|
|
43
|
+
- Snapshot sprawl: giant `toMatchSnapshot()` blobs get blindly re-recorded. Assert the fields that matter by value.
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# Django stack pack
|
|
2
|
+
|
|
3
|
+
Django-specific testing — DB isolation, ORM data, settings, the test client, external boundaries. Loaded **only when the project is Django**, alongside the runner pack (pytest or unittest). Read with the skill's universal rules.
|
|
4
|
+
|
|
5
|
+
## DB isolation & base class
|
|
6
|
+
|
|
7
|
+
- **`TestCase` (or pytest-django's `db` fixture) wraps each test in a transaction rolled back at teardown** — the default, fast path, zero manual cleanup.
|
|
8
|
+
- **`TransactionTestCase` / `@pytest.mark.django_db(transaction=True)`** truncates instead of rolling back — use **only** when the code commits its own transactions, tests `on_commit` hooks, or uses `select_for_update`. Markedly slower; not the default.
|
|
9
|
+
- **`SimpleTestCase`** forbids DB access — use it to prove a unit is DB-free.
|
|
10
|
+
- Never `Model.objects.all().delete()` in setup as a cleanup crutch; rely on rollback.
|
|
11
|
+
|
|
12
|
+
## Test data
|
|
13
|
+
|
|
14
|
+
- **Prefer `factory_boy` or `model_bakery` recipes over fixture files or inline `Model.objects.create(...)` walls** — `UserFactory(is_staff=True)` builds a valid instance spelling out only the relevant field. Fixture files drift from the schema silently.
|
|
15
|
+
- With `TestCase`, put shared read-only data in **`setUpTestData(cls)`** (runs once per class inside the outer transaction), not `setUp` (per test). Only the fields the assertion needs.
|
|
16
|
+
|
|
17
|
+
## Assertions
|
|
18
|
+
|
|
19
|
+
- `self.assertEqual(response.json(), {...})` / `assertEqual(obj.status, "paid")` — full value, not `assertTrue(obj)` or a lone `Model.objects.count()`.
|
|
20
|
+
- Use the specific ones for signal: `assertQuerySetEqual`, `assertContains`/`assertNotContains` (body + status), `assertRaisesMessage`, `assertNumQueries` (guard a hot path).
|
|
21
|
+
- **`obj.refresh_from_db()` before asserting persisted state** — the in-memory instance hides DB defaults, signals, `auto_now`.
|
|
22
|
+
|
|
23
|
+
## Externals
|
|
24
|
+
|
|
25
|
+
- **`unittest.mock.patch`** where the name is *used* (`patch("myapp.services.stripe_client")`), not where defined.
|
|
26
|
+
- **Email:** assert on `django.core.mail.outbox` (the test runner's `locmem` backend), don't mock `send`.
|
|
27
|
+
- **HTTP:** `responses` / `respx` — never a real outbound call.
|
|
28
|
+
- **Settings/time:** `@override_settings(...)` / `self.settings(...)`; `freezegun` or an injected clock.
|
|
29
|
+
- **Test client:** `self.client` exercises the real URL/middleware/view stack — good. Mock only the external service the view calls, not the view.
|
|
30
|
+
|
|
31
|
+
## Pitfalls
|
|
32
|
+
|
|
33
|
+
- `TransactionTestCase` "to be safe" — slower for no reason. Default to `TestCase`.
|
|
34
|
+
- `on_commit` hooks never fire under `TestCase` (the transaction rolls back) — use `TestCase.captureOnCommitCallbacks(execute=True)` or `TransactionTestCase`.
|
|
35
|
+
- A lone `assertEqual(Model.objects.count(), 1)` — asserts a count, not that the right object with the right fields exists.
|
|
36
|
+
- Asserting on `obj` after `save()` without `refresh_from_db()`.
|
|
@@ -0,0 +1,34 @@
|
|
|
1
|
+
# FastAPI stack pack
|
|
2
|
+
|
|
3
|
+
FastAPI-specific testing — the test client, dependency overrides, async, DB isolation. Loaded **only when the project is FastAPI**, alongside the runner pack (usually pytest). Read with the skill's universal rules.
|
|
4
|
+
|
|
5
|
+
## Test client & dependencies
|
|
6
|
+
|
|
7
|
+
- **`TestClient(app)`** (Starlette, sync) exercises the real routing/validation/response stack. For async-native tests use **`httpx.AsyncClient(transport=ASGITransport(app=app))`**.
|
|
8
|
+
- **Override dependencies with `app.dependency_overrides[dep] = fake`**, not by monkeypatching internals — inject a test DB session, a stubbed auth principal, a fake external client. Clear overrides in teardown (a fixture that yields then pops).
|
|
9
|
+
- Don't mock the endpoint function; drive it through the client and override only what crosses the boundary.
|
|
10
|
+
|
|
11
|
+
## Async
|
|
12
|
+
|
|
13
|
+
- Use `pytest-asyncio` (`@pytest.mark.asyncio`) or `anyio`. Await the real coroutine; don't wrap sync mocks around async code. Mock async collaborators with `AsyncMock`.
|
|
14
|
+
|
|
15
|
+
## DB isolation
|
|
16
|
+
|
|
17
|
+
- Inject the DB session via a dependency and override it with a **session bound to a transaction rolled back in teardown** (SQLAlchemy: connection → `begin()` → session → rollback). Each test starts clean; no manual delete.
|
|
18
|
+
- Prefer a real (SQLite/Postgres) test DB over mocking the ORM — mocking the session tests the mock.
|
|
19
|
+
|
|
20
|
+
## Externals
|
|
21
|
+
|
|
22
|
+
- **HTTP:** `respx` (httpx) / `responses` (requests) — never a real outbound call.
|
|
23
|
+
- **Settings:** override via a `Settings` dependency or env fixture, not by mutating a global.
|
|
24
|
+
- **Time/randomness:** `freezegun` or an injected clock; seed `random`.
|
|
25
|
+
|
|
26
|
+
## Assertions
|
|
27
|
+
|
|
28
|
+
- Assert `response.status_code` **and** `response.json()` by full value — not just the status. Validate against the response model's shape, not a lone key.
|
|
29
|
+
|
|
30
|
+
## Pitfalls
|
|
31
|
+
|
|
32
|
+
- Forgetting to clear `app.dependency_overrides` — leaks a fake into the next test.
|
|
33
|
+
- Mixing sync `TestClient` with async DB drivers — use `AsyncClient` + async session instead.
|
|
34
|
+
- Asserting only the status code while the body is wrong.
|
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
# Flask stack pack
|
|
2
|
+
|
|
3
|
+
Flask-specific testing — the test client, app/request context, DB isolation. Loaded **only when the project is Flask**, alongside the runner pack (usually pytest). Read with the skill's universal rules.
|
|
4
|
+
|
|
5
|
+
## App, client & context
|
|
6
|
+
|
|
7
|
+
- Build the app from an **app factory** with a test config (`create_app({"TESTING": True, ...})`) so each test gets a clean instance; expose it as a fixture.
|
|
8
|
+
- **`app.test_client()`** exercises the real routing/blueprint/error-handler stack — drive endpoints through it, don't call view functions directly.
|
|
9
|
+
- Code that touches `current_app` / `g` / `session` needs an active context — use `with app.app_context():` / `with app.test_request_context():` in the fixture. Don't fake the context.
|
|
10
|
+
|
|
11
|
+
## DB isolation
|
|
12
|
+
|
|
13
|
+
- **SQLAlchemy:** bind the session to a connection-level transaction rolled back in teardown (connection → `begin()` → scoped session → `rollback` + `close`). Each test starts clean; no manual delete.
|
|
14
|
+
- Prefer a real test DB over mocking the ORM.
|
|
15
|
+
|
|
16
|
+
## Externals
|
|
17
|
+
|
|
18
|
+
- **Auth:** log in through the client (post to the login route or set the session via `client.session_transaction()`), not by patching the auth check.
|
|
19
|
+
- **HTTP:** `responses` / `respx` — never a real outbound call.
|
|
20
|
+
- **Extensions (mail, tasks):** use the extension's test/suppress mode (`mail.record_messages()`, eager Celery) and assert on what it captured, rather than mocking send.
|
|
21
|
+
- **Time/randomness:** `freezegun` or an injected clock; seed `random`.
|
|
22
|
+
|
|
23
|
+
## Assertions
|
|
24
|
+
|
|
25
|
+
- Assert `response.status_code` **and** the body (`response.get_json()` / rendered content) by full value. For JSON APIs assert the whole payload, not one key.
|
|
26
|
+
|
|
27
|
+
## Pitfalls
|
|
28
|
+
|
|
29
|
+
- Working outside an app/request context and getting `RuntimeError: working outside of ...` — push the right context in the fixture.
|
|
30
|
+
- Reusing one app instance with mutable global state across tests — the factory per test avoids it.
|
|
31
|
+
- Asserting only the status code while the body is wrong.
|
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
// @ts-nocheck
|
|
2
|
+
// Bundled verbatim from vintasoftware/medplum-snippet-catalog → TestSetup/test.globalSetup.ts
|
|
3
|
+
// Source: https://github.com/vintasoftware/medplum-snippet-catalog/tree/main/TestSetup
|
|
4
|
+
// Retrieved: 2026-07-31. The catalog is canonical — if it has advanced past this copy, prefer it.
|
|
5
|
+
// Install as (e.g.) `src/test.globalSetup.ts` and register in Vitest config `test.globalSetup`.
|
|
6
|
+
|
|
7
|
+
// Assert that the versions of @medplum/core, @medplum/definitions, @medplum/mock and
|
|
8
|
+
// @medplum/fhirtypes match.
|
|
9
|
+
import type { IndexedStructureDefinition } from '@medplum/core';
|
|
10
|
+
import type { Bundle, SearchParameter } from '@medplum/fhirtypes';
|
|
11
|
+
import type { TestProject } from 'vitest/node';
|
|
12
|
+
|
|
13
|
+
export default async function setup(project: TestProject): Promise<void> {
|
|
14
|
+
// @medplum/core v5 references WebSocket at module-load time. This file runs in the Vitest
|
|
15
|
+
// main process, where WebSocket is only guaranteed as a global on Node >= 22.4. It has to be
|
|
16
|
+
// polyfilled *before* @medplum/core is loaded, which is why the imports below are dynamic and
|
|
17
|
+
// only the (erased) type imports sit at the top of the file.
|
|
18
|
+
if (typeof globalThis.WebSocket === 'undefined') {
|
|
19
|
+
try {
|
|
20
|
+
const ws = await import('ws');
|
|
21
|
+
(globalThis as Record<string, unknown>).WebSocket = ws.default ?? ws.WebSocket;
|
|
22
|
+
} catch {
|
|
23
|
+
throw new Error(
|
|
24
|
+
'@medplum/core needs a global WebSocket. Upgrade to Node >= 22.4, or add "ws" as a dev dependency.'
|
|
25
|
+
);
|
|
26
|
+
}
|
|
27
|
+
}
|
|
28
|
+
|
|
29
|
+
const { globalSchema, indexSearchParameterBundle, indexStructureDefinitionBundle } = await import('@medplum/core');
|
|
30
|
+
const { readJson, SEARCH_PARAMETER_BUNDLE_FILES } = await import('@medplum/definitions');
|
|
31
|
+
|
|
32
|
+
// Initialize FHIR
|
|
33
|
+
indexStructureDefinitionBundle(readJson('fhir/r4/profiles-types.json') as Bundle);
|
|
34
|
+
indexStructureDefinitionBundle(readJson('fhir/r4/profiles-resources.json') as Bundle);
|
|
35
|
+
indexStructureDefinitionBundle(readJson('fhir/r4/profiles-medplum.json') as Bundle);
|
|
36
|
+
for (const filename of SEARCH_PARAMETER_BUNDLE_FILES) {
|
|
37
|
+
indexSearchParameterBundle(readJson(filename) as Bundle<SearchParameter>);
|
|
38
|
+
}
|
|
39
|
+
|
|
40
|
+
// `globalSchema.types` is plain JSON-serializable data, so it can be handed to every test
|
|
41
|
+
// worker instead of each one re-reading and re-indexing the search parameter bundles.
|
|
42
|
+
// See test.setup.ts for the consuming side.
|
|
43
|
+
project.provide('globalSchemaTypes', globalSchema.types);
|
|
44
|
+
}
|
|
45
|
+
|
|
46
|
+
export function teardown(): void {
|
|
47
|
+
// Nothing to do.
|
|
48
|
+
}
|
|
49
|
+
|
|
50
|
+
declare module 'vitest' {
|
|
51
|
+
export interface ProvidedContext {
|
|
52
|
+
globalSchemaTypes: IndexedStructureDefinition['types'];
|
|
53
|
+
}
|
|
54
|
+
}
|
package/skills/vinta-derive-skills/resources/write-unit-test-packs/stacks/medplum-test.setup.ts
ADDED
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
// @ts-nocheck
|
|
2
|
+
// Bundled verbatim from vintasoftware/medplum-snippet-catalog → TestSetup/test.setup.ts
|
|
3
|
+
// Source: https://github.com/vintasoftware/medplum-snippet-catalog/tree/main/TestSetup
|
|
4
|
+
// Retrieved: 2026-07-31. The catalog is canonical — if it has advanced past this copy, prefer it.
|
|
5
|
+
// Install as (e.g.) `src/test.setup.ts` and register in Vitest config `test.setupFiles`.
|
|
6
|
+
// Requires its companion `test.globalSetup.ts` in `test.globalSetup` (it provides globalSchemaTypes).
|
|
7
|
+
|
|
8
|
+
// Assert that the versions of @medplum/core, @medplum/definitions, @medplum/mock and
|
|
9
|
+
// @medplum/fhirtypes match.
|
|
10
|
+
import { globalSchema } from '@medplum/core';
|
|
11
|
+
import { inject } from 'vitest';
|
|
12
|
+
|
|
13
|
+
// This runs at module scope on purpose, not inside beforeAll. Vitest runs setup files once per
|
|
14
|
+
// test file, but a test file's own module body is evaluated *before* any beforeAll hook
|
|
15
|
+
// registered here. Fixtures built at module scope would otherwise search against an empty
|
|
16
|
+
// schema and silently match nothing:
|
|
17
|
+
//
|
|
18
|
+
// const medplum = new MockClient();
|
|
19
|
+
// await medplum.createResource({ resourceType: 'Patient', ... });
|
|
20
|
+
// const preloaded = await medplum.searchResources('Patient', 'family=Xu'); // 0 results
|
|
21
|
+
const globalSchemaTypes = inject('globalSchemaTypes');
|
|
22
|
+
|
|
23
|
+
if (!globalSchemaTypes) {
|
|
24
|
+
throw new Error(
|
|
25
|
+
'globalSchemaTypes was not provided. Add test.globalSetup.ts to `globalSetup` in your Vitest config.'
|
|
26
|
+
);
|
|
27
|
+
}
|
|
28
|
+
|
|
29
|
+
// Search parameters were indexed once by test.globalSetup.ts. `globalSchema.types` holds
|
|
30
|
+
// nothing but plain data, so it survives the trip from the main process to this worker.
|
|
31
|
+
// This is what MockClient's search matching reads, and it is the whole reason this file
|
|
32
|
+
// exists: without it, filters like `family=` or `birthdate=` silently match nothing.
|
|
33
|
+
globalSchema.types = globalSchemaTypes;
|
|
34
|
+
|
|
35
|
+
// Structure definitions are deliberately *not* indexed here. MockClient indexes them itself
|
|
36
|
+
// on its first write, so doing it up front only moves the same ~35 MB parse earlier.
|
|
37
|
+
// The one exception: calling validateResource (or anything else that reads @medplum/core's
|
|
38
|
+
// internal type store) before any resource has been created throws "Unknown data type".
|
|
39
|
+
// Index them in that individual test if you need it.
|
|
@@ -0,0 +1,47 @@
|
|
|
1
|
+
# Medplum stack pack
|
|
2
|
+
|
|
3
|
+
Medplum-specific testing — the mock client, FHIR resources, bots, React providers. Loaded **only when the project uses Medplum** (`@medplum/core` / `@medplum/react`), alongside the runner pack (Vitest / Jest). Read with the skill's universal rules.
|
|
4
|
+
|
|
5
|
+
## Mock client, never a real server
|
|
6
|
+
|
|
7
|
+
- **`MockClient` from `@medplum/mock`** stands in for `MedplumClient` — it holds an in-memory FHIR store, so `createResource` / `readResource` / `searchResources` run for real against fake data. Never point tests at a real Medplum server; there is no exception.
|
|
8
|
+
- Give each test its own `MockClient` (or reset between tests) so resources one test creates don't leak into the next.
|
|
9
|
+
- Seed only the resources the assertion needs; let the mock default the rest.
|
|
10
|
+
|
|
11
|
+
## MockClient limitations — it is not a full FHIR server
|
|
12
|
+
|
|
13
|
+
`MockClient` is an in-memory approximation. A green MockClient test does not prove the real server behaves the same — know the gaps:
|
|
14
|
+
|
|
15
|
+
- **It comes pre-seeded.** MockClient ships with example resources already created (Patients, Practitioners, and others). An **unfiltered** `searchResources('Patient')` returns those *plus* what your test created — so **never assert on unfiltered search counts**. Filter to what your test made (a unique `identifier` / tag), or assert the specific resource by id/fields.
|
|
16
|
+
- **Search support is partial.** Even with the FHIR index wired up (below), not every search parameter, modifier, chained search, `_include` / `_revinclude`, or `_filter` matches server semantics. Prefer filtering on simple, well-supported params in tests; if a filter behaves oddly, check server semantics before assuming your code is wrong.
|
|
17
|
+
- **Server-only behavior isn't enforced.** Access policies / auth scoping, subscriptions, and some operations (`$everything`, GraphQL, batch/transaction edge cases) plus bot-execution nuances are approximated or absent. Anything whose contract depends on the server enforcing it belongs in an integration/e2e test, not a MockClient unit test.
|
|
18
|
+
- **Consult the team's MockClient extensions + internal Medplum testing docs** for the current maintained limitation list and any local wrappers that patch gaps — they track the Medplum version. Canonical setup + helpers live in the `medplum-snippet-catalog`.
|
|
19
|
+
|
|
20
|
+
## Search filtering — indexed once at setup, never per test
|
|
21
|
+
|
|
22
|
+
- **`MockClient` search filters only work if the FHIR search parameters are indexed in the test process.** Symptom: a resource comes back from an unfiltered `searchResources`, but `searchResources` / `searchOne` **with** a filter (`family=`, `birthdate=`, …) silently returns nothing. That means the index is missing, not that your data is wrong.
|
|
23
|
+
- This is wired **once** at bootstrap via two Vitest setup files: `test.globalSetup.ts` (indexes the bundles once in the main process, shares `globalSchema.types` to workers) registered under `test.globalSetup`, and `test.setup.ts` (assigns the shared index at module scope) registered under `test.setupFiles`. **Do not add `indexStructureDefinitionBundle` / `indexSearchParameterBundle` inside a test or a per-test `beforeAll`** — rely on the project-level setup. If filters mysteriously fail, check both files are registered in `vitest.config.*`, not that each test indexes.
|
|
24
|
+
- One deliberate exception: `validateResource` (or anything reading @medplum/core's type store) **before any resource is created** throws `Unknown data type` — index structure definitions in that one test if you hit it.
|
|
25
|
+
- Version alignment matters: `@medplum/core`, `@medplum/definitions`, `@medplum/mock`, and `@medplum/fhirtypes` must be on **matching versions** — a skew causes confusing index/search failures.
|
|
26
|
+
|
|
27
|
+
## FHIR resource assertions
|
|
28
|
+
|
|
29
|
+
- Assert on the **resource fields you care about by value** — `patient.name[0].family`, `observation.valueQuantity.value` — not just `resource.id` truthiness or a search count. A wrong resource with an id still passes a truthiness check.
|
|
30
|
+
- When asserting a search result, assert the matched resources' contents, not only `bundle.entry.length`.
|
|
31
|
+
- Build resources as typed `@medplum/fhirtypes` objects (or a small factory helper), not loosely-typed literals copied across tests.
|
|
32
|
+
|
|
33
|
+
## Bots
|
|
34
|
+
|
|
35
|
+
- A bot's `handler(medplum, event)` is a plain function — unit-test it by calling it with a `MockClient` and a constructed `BotEvent` input, then assert the resources it wrote (read them back from the mock) and the value it returned.
|
|
36
|
+
- Mock outbound integrations the bot calls (external HTTP, `sendEmail`) at the boundary; don't mock the `medplum` client — use `MockClient`.
|
|
37
|
+
|
|
38
|
+
## React components (with `@medplum/react`)
|
|
39
|
+
|
|
40
|
+
- Wrap rendered components in **`MedplumProvider` with a `MockClient`** (Testing Library). Query by role/text; assert rendered output. See the runner pack's Testing Library notes.
|
|
41
|
+
- Let components fetch through the `MockClient`; use `await screen.findBy...` for the async load rather than mocking the fetch.
|
|
42
|
+
|
|
43
|
+
## Pitfalls
|
|
44
|
+
|
|
45
|
+
- Sharing one `MockClient` across tests without reset — resources bleed between tests.
|
|
46
|
+
- Asserting `resource.id` is defined instead of the fields the code set.
|
|
47
|
+
- Mocking `MedplumClient` methods by hand instead of using `MockClient` — you re-implement the store and test the mock.
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
# Next.js stack pack
|
|
2
|
+
|
|
3
|
+
Next.js (App Router) testing — what to unit-test vs leave to e2e, route handlers, server code. Loaded **only when the project uses Next.js**, alongside the runner pack (Vitest / Jest) and, for UI, the React stack pack. Read with the skill's universal rules.
|
|
4
|
+
|
|
5
|
+
## Test the logic, not the framework
|
|
6
|
+
|
|
7
|
+
- **Unit-test the units the framework calls**, not the framework's rendering: route-handler functions, server actions, data-loading helpers, and pure client components. Full-page rendering, navigation, streaming, and middleware are e2e territory (Playwright) — don't fake the Next runtime to force them into a unit test.
|
|
8
|
+
|
|
9
|
+
## Route handlers & server actions
|
|
10
|
+
|
|
11
|
+
- A **Route Handler** (`GET`/`POST` in `route.ts`) is a plain async function taking a `Request` — call it directly with a constructed `new Request(url, { method, body })` and assert the returned `Response` (`await res.json()` + `res.status`) by full value. No server boot needed.
|
|
12
|
+
- A **Server Action** is a plain async function — call it with its args and assert what it returned / wrote (mock the DB/service boundary, not the action). Don't assert on `revalidatePath`/`redirect` internals; assert the effect.
|
|
13
|
+
- Inject or mock external services (DB, upstream APIs) at the boundary with MSW / a fake client, not by faking `next/*` internals.
|
|
14
|
+
|
|
15
|
+
## Client components & hooks
|
|
16
|
+
|
|
17
|
+
- Test client components with the React stack pack (Testing Library). Mock `next/navigation` (`useRouter`, `useSearchParams`, `usePathname`) with light fakes when a component reads them — assert the visible result, not that `router.push` was called, unless navigation *is* the contract.
|
|
18
|
+
|
|
19
|
+
## Server components
|
|
20
|
+
|
|
21
|
+
- Async Server Components that just fetch + compose are thin — extract the data logic into a plain function and unit-test that. Rendering the RSC tree itself is not well-served by unit tests today; cover it with e2e.
|
|
22
|
+
|
|
23
|
+
## Pitfalls
|
|
24
|
+
|
|
25
|
+
- Trying to render a whole App-Router page in a unit test and mocking half of Next to do it — brittle; use e2e.
|
|
26
|
+
- Asserting `redirect()` / `revalidatePath()` were called instead of the observable outcome.
|
|
27
|
+
- Leaving real `fetch` calls in a Server Component test — stub at the network layer (MSW) or test the extracted function.
|
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
# Prisma stack pack
|
|
2
|
+
|
|
3
|
+
Prisma (data layer) testing — DB isolation, real-DB vs mock, seeding. Loaded **only when the project uses Prisma** (`@prisma/client`), alongside the runner pack (Vitest / Jest) and any web-framework stack pack. Read with the skill's universal rules.
|
|
4
|
+
|
|
5
|
+
## Real test DB over a mocked client
|
|
6
|
+
|
|
7
|
+
- **Test queries against a real (SQLite/Postgres) test database**, not a mocked `PrismaClient`. Mocking the client (`jest-mock-extended`, `prisma-mock`) means your `where`/relations/constraints/`select` never run — the test passes on a query that would throw in production. Point `DATABASE_URL` at a disposable test DB and apply migrations in global setup (`prisma migrate deploy` / `db push`).
|
|
8
|
+
- **Mock only at the port, not the client.** If the unit under test is pure business logic that takes a repository/port, inject a fake repo — don't mock `PrismaClient`. If you're testing the query/persistence itself, use the real DB.
|
|
9
|
+
|
|
10
|
+
## DB isolation
|
|
11
|
+
|
|
12
|
+
- **Wrap each test in a transaction rolled back at teardown** — an interactive `$transaction` handle passed to the code, rolled back after (helpers like `@chax-at/transactional-prisma-testing` automate it), or a per-test truncation of the tables touched. Each test starts clean; no manual delete scattered around.
|
|
13
|
+
- **`$disconnect()` in teardown** (global) — leaked connections exhaust the pool and hang the suite.
|
|
14
|
+
- Never depend on test ordering or on rows another test created.
|
|
15
|
+
|
|
16
|
+
## Test data
|
|
17
|
+
|
|
18
|
+
- Seed with small factory helpers that spell out only the fields the assertion needs and let schema defaults fill the rest — not deep nested `create` literals copy-pasted across tests.
|
|
19
|
+
- Reset to a known baseline between tests via the rollback/truncate above, not a shared mutable seed.
|
|
20
|
+
|
|
21
|
+
## Assertions
|
|
22
|
+
|
|
23
|
+
- **Read the row back and assert its fields by value** (`await prisma.user.findUnique(...)` → assert the object), not `count` alone — a count passes for the wrong row. For relations, `include` them and assert their contents.
|
|
24
|
+
- After a write, assert the persisted state from a fresh read, not the in-memory input you passed in.
|
|
25
|
+
|
|
26
|
+
## Pitfalls
|
|
27
|
+
|
|
28
|
+
- Mocking `PrismaClient` then asserting the mock was called — tests the mock, not the query.
|
|
29
|
+
- `expect(await prisma.x.count()).toBe(1)` as the whole assertion — asserts a count, not the right record.
|
|
30
|
+
- No `$disconnect()` / shared client across tests — connection leaks and cross-test bleed.
|
|
31
|
+
- Truncating in `beforeEach` as a cleanup crutch instead of transaction rollback — slower and hides ordering bugs.
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
# Python package (library) stack pack
|
|
2
|
+
|
|
3
|
+
Testing a pure Python library/package — no web framework, no app DB, shipped to consumers (e.g. an open-source package). Loaded **only when the project is a standalone package**, alongside the pytest runner pack. Read with the skill's universal rules.
|
|
4
|
+
|
|
5
|
+
## Test the public API
|
|
6
|
+
|
|
7
|
+
- Test the package's **public surface** — the names in `__init__.py` / documented exports — the way a consumer imports and calls them. Don't reach into private modules (`_internal`) to test them directly; if a private helper needs its own test, that's a hint it should be public or covered through the public path.
|
|
8
|
+
- Backward compatibility is the contract: a test pinning documented behavior is what stops a refactor from breaking downstream users.
|
|
9
|
+
|
|
10
|
+
## Keep it dependency-light & deterministic
|
|
11
|
+
|
|
12
|
+
- A library test should need only the package + `pytest` (+ its declared test extras). Don't pull a DB, a broker, or network into a unit test — if a feature integrates with those, test the seam with a fake/in-memory double the consumer could also use.
|
|
13
|
+
- No hidden global state between tests: reset module-level singletons/caches your code exposes; don't rely on import order.
|
|
14
|
+
|
|
15
|
+
## Data & purity
|
|
16
|
+
|
|
17
|
+
- Most of a library is pure functions — arrange literal inputs, assert literal outputs by full value. Parametrize to cover edge cases (empty, boundary, unicode, `None`).
|
|
18
|
+
- For anything time/random/env-dependent, inject the dependency (clock, rng, `os.environ` reader) so tests pass a fake — libraries especially should not read ambient globals.
|
|
19
|
+
|
|
20
|
+
## Version / matrix awareness
|
|
21
|
+
|
|
22
|
+
- If the package supports multiple Python versions or an optional-dependency matrix (`tox` / `nox`), write tests that don't assume the newest syntax/behavior, and guard optional-dependency paths with `pytest.importorskip` so the suite runs in every configured environment.
|
|
23
|
+
|
|
24
|
+
## Pitfalls
|
|
25
|
+
|
|
26
|
+
- Testing `_private` internals — couples the test to today's structure and breaks on refactor.
|
|
27
|
+
- A unit test that imports a heavy optional dependency unconditionally — fails in the minimal matrix env.
|
|
28
|
+
- Asserting truthiness of a returned object instead of its value/shape.
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
# React Router stack pack
|
|
2
|
+
|
|
3
|
+
React Router testing — data-router loaders/actions and routing-aware components. Loaded **only when the project uses React Router** (`react-router` / `react-router-dom` / `@react-router/*`), alongside the runner pack and, for UI, the React stack pack. Read with the skill's universal rules.
|
|
4
|
+
|
|
5
|
+
## Loaders & actions are plain functions
|
|
6
|
+
|
|
7
|
+
- In data-router / framework mode, a route's **`loader` and `action` are plain async functions** receiving `{ request, params, context }`. Export and call them directly; assert what they return / throw. A thrown `redirect(...)` is part of the contract — assert its `Location`, don't assert on router internals.
|
|
8
|
+
- Mock the boundary the loader/action hits (DB, upstream API) with a fake client / MSW. Build the `request` with `new Request(url, { method, body })` for actions.
|
|
9
|
+
|
|
10
|
+
## Routing-aware components
|
|
11
|
+
|
|
12
|
+
- Components that call routing hooks (`useNavigate`, `useParams`, `useLoaderData`, `<Link>`) need a router in the test. Use **`createMemoryRouter([...routes], { initialEntries: [...] })` + `<RouterProvider>`** (data router), seeding the route with `loader: () => data` so `useLoaderData` returns what the component reads. For simple hook-only components, wrapping in **`<MemoryRouter initialEntries={[...]}>`** is enough.
|
|
13
|
+
- Assert the **visible result** of navigation (the destination content rendered, the URL via the test router), not that `navigate` was called — unless firing navigation *is* the unit's whole job.
|
|
14
|
+
|
|
15
|
+
## Externals & isolation
|
|
16
|
+
|
|
17
|
+
- Stub HTTP at the network layer with MSW; never a real outbound call.
|
|
18
|
+
- Build a **fresh memory router/history per test** — a shared router leaks navigation/loader state and causes order-dependence.
|
|
19
|
+
|
|
20
|
+
## Pitfalls
|
|
21
|
+
|
|
22
|
+
- Rendering a component that uses `useNavigate` / `<Link>` without a router wrapper → it throws; wrap it in a memory router.
|
|
23
|
+
- Asserting `navigate("/x")` was called instead of asserting the `/x` content rendered.
|
|
24
|
+
- Testing loaders by rendering the whole route tree instead of calling the loader function directly.
|
|
25
|
+
- Reusing one router across tests — stale location/loader data flips results by order.
|
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
# React stack pack
|
|
2
|
+
|
|
3
|
+
Component-testing idioms with React Testing Library. Loaded **only when the project renders React UI**, alongside the runner pack (Vitest / Jest). Read with the skill's universal rules.
|
|
4
|
+
|
|
5
|
+
## Query & interact like a user
|
|
6
|
+
|
|
7
|
+
- **Query by role/label/text**, not test-ids or class names: `getByRole("button", { name: /save/i })`. Test-ids couple the test to markup. Reserve `data-testid` for the rare element with no accessible handle.
|
|
8
|
+
- **`getBy`** (must exist) vs **`queryBy`** (assert absence) vs **`findBy`** (async, awaited). Use the right one — `getBy` throwing is a clearer failure than a null deref.
|
|
9
|
+
- Drive interactions with **`user-event`** (`await userEvent.click(...)`), not raw `fireEvent` — it models real focus/typing.
|
|
10
|
+
|
|
11
|
+
## Assert output, not internals
|
|
12
|
+
|
|
13
|
+
- Assert what the user sees (rendered text, presence/absence, disabled state), not component state, not that a hook or handler ran, not render counts. The test should survive a refactor that preserves behavior.
|
|
14
|
+
- Don't reach into instance internals or snapshot the whole tree as the only assertion — assert the elements that matter.
|
|
15
|
+
|
|
16
|
+
## Data & async
|
|
17
|
+
|
|
18
|
+
- Mock HTTP at the network layer with **MSW** (see the runner pack), so components run their real fetch path. For a component that loads data, `await screen.findBy...` the settled UI rather than mocking the fetching hook.
|
|
19
|
+
- Pure logic (reducers, formatters, custom hooks without UI) — test directly, no render. Hooks: `renderHook` + `act`.
|
|
20
|
+
|
|
21
|
+
## Isolation
|
|
22
|
+
|
|
23
|
+
- Clean up is automatic per test with the Testing Library auto-cleanup; don't share rendered state between tests. Reset mocks in `afterEach` (see runner pack).
|
|
24
|
+
|
|
25
|
+
## Pitfalls
|
|
26
|
+
|
|
27
|
+
- `getByTestId` everywhere — brittle, and hides accessibility gaps. Prefer role/label.
|
|
28
|
+
- Asserting a handler prop was called instead of the visible result of calling it.
|
|
29
|
+
- Missing `await` on `findBy` / `userEvent` — the assertion runs before the UI settles and passes hollow.
|
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
# TanStack Start stack pack
|
|
2
|
+
|
|
3
|
+
TanStack Start testing — server functions, route loaders, and isomorphic logic. Loaded **only when the project uses TanStack Start** (`@tanstack/react-start`), alongside the runner pack (Vitest) and, for UI, the React stack pack. Read with the skill's universal rules.
|
|
4
|
+
|
|
5
|
+
## Test the logic, not the framework
|
|
6
|
+
|
|
7
|
+
- **Unit-test the units the framework calls**, not the full route/SSR render: server functions, route `loader` / `beforeLoad` functions, and pure components. Full-page rendering, streaming, hydration, and navigation are e2e territory (Playwright) — don't fake the Start runtime to force them into a unit test.
|
|
8
|
+
|
|
9
|
+
## Server functions (`createServerFn`)
|
|
10
|
+
|
|
11
|
+
- A server function's **handler is a plain async function** — export it (or the plain function it wraps) and call it directly with constructed input; assert the returned value by full shape. Don't spin up the server to test it.
|
|
12
|
+
- Mock what the handler calls at the boundary (DB, upstream APIs) with a fake client / MSW, not the `createServerFn` machinery.
|
|
13
|
+
- Validate the input-validator (`.validator(...)`) separately with valid + invalid inputs — it's a pure function.
|
|
14
|
+
|
|
15
|
+
## Route loaders & context
|
|
16
|
+
|
|
17
|
+
- A route's `loader` / `beforeLoad` is a plain async function receiving `{ context, params, ... }` — call it with a constructed argument and assert what it returns / throws (e.g. a `redirect()`), passing a fake `context` (auth principal, injected client). Don't assert on router internals; assert the returned data or the thrown redirect's target.
|
|
18
|
+
|
|
19
|
+
## Client components & router hooks
|
|
20
|
+
|
|
21
|
+
- Test client components with the React stack pack (Testing Library). For components using router hooks (`useLoaderData`, `useNavigate`, `useParams`), render them under a **test router** built with `createRouter` + a `createMemoryHistory({ initialEntries: [...] })` and `RouterProvider`, seeding the route with the data the component reads. Assert the visible result, not that `navigate` was called (unless navigation *is* the contract).
|
|
22
|
+
|
|
23
|
+
## Externals & isolation
|
|
24
|
+
|
|
25
|
+
- Stub HTTP at the network layer with MSW; never a real outbound call in a unit test.
|
|
26
|
+
- Fresh router/history per test — don't share navigation state between tests.
|
|
27
|
+
|
|
28
|
+
## Pitfalls
|
|
29
|
+
|
|
30
|
+
- Booting the Start server or rendering a whole route in a unit test and mocking half the framework — brittle; use e2e.
|
|
31
|
+
- Asserting a `redirect()` / `navigate()` happened instead of the observable outcome (the data, the rendered target).
|
|
32
|
+
- Leaving a real `fetch` in a server-function or loader test — stub at the boundary or test the extracted function.
|
package/skills/vinta-derive-skills/resources/write-unit-test-packs/stacks/typescript-package.md
ADDED
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
# TypeScript package (library) stack pack
|
|
2
|
+
|
|
3
|
+
Testing a pure TypeScript library/package — no framework, no app DB, shipped to consumers (npm package, workspace lib). Loaded **only when the project is a standalone package**, alongside the runner pack (Vitest / Jest). Read with the skill's universal rules.
|
|
4
|
+
|
|
5
|
+
## Test the public API
|
|
6
|
+
|
|
7
|
+
- Import from the package **entry point** (`exports` / `main` / the package name in a monorepo), the way a consumer does — not deep relative paths into `src/internal`. That's what pins the published contract; a green suite that imports internals can still ship a broken public API.
|
|
8
|
+
- If a private helper needs its own test, consider whether it should be exported, or cover it through the public path.
|
|
9
|
+
|
|
10
|
+
## Types are part of the contract
|
|
11
|
+
|
|
12
|
+
- Type-level behavior (generics, inference, conditional types) won't be caught by value assertions. When it matters, add compile-time type tests with **`tsd`** / `expectTypeOf` (Vitest) / `vitest`'s type-testing — assert that a call *doesn't* compile when it shouldn't and infers the right type when it should.
|
|
13
|
+
|
|
14
|
+
## Keep it dependency-light & deterministic
|
|
15
|
+
|
|
16
|
+
- A library test needs only the package + the runner. Don't pull a DB/network in; test integration seams with an in-memory fake the consumer could also use.
|
|
17
|
+
- Reset any module-level singleton/cache the package exposes between tests; don't rely on import order.
|
|
18
|
+
|
|
19
|
+
## Build outputs (when relevant)
|
|
20
|
+
|
|
21
|
+
- If the package ships both ESM and CJS, a small smoke test that imports the built entry in each format catches packaging/`exports`-map breakage that source-only tests miss.
|
|
22
|
+
|
|
23
|
+
## Data & purity
|
|
24
|
+
|
|
25
|
+
- Most of a library is pure — literal inputs, assert literal outputs with `toEqual` / `toStrictEqual` (use `toStrictEqual` when `undefined` fields / class identity are part of the contract). Table-drive edge cases (empty, boundary, unicode).
|
|
26
|
+
- Inject clock/rng/env for anything ambient so tests pass a fake.
|
|
27
|
+
|
|
28
|
+
## Pitfalls
|
|
29
|
+
|
|
30
|
+
- Testing via deep `src/...` imports — green tests, broken published `exports`.
|
|
31
|
+
- Only value tests on a type-heavy API — the type regressions ship silently. Add type tests.
|
|
32
|
+
- `toEqual` when `undefined`-property presence matters — use `toStrictEqual`.
|
|
@@ -0,0 +1,44 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: write-unit-test
|
|
3
|
+
description: Write a unit test for a function, method, class, or module in {{PROJECT_NAME}} ({{STACK_SUMMARY}}) — mock only genuine externals, assert the full output (not counts/truthiness), clean up created data or roll back the transaction, never hit a service unavailable in dev/test, keep test logic literal, stay decoupled from internals. Applies this project's test preferences and the runner + stack packs for {{FRAMEWORK_PACKS_LIST}}. Use on "add a test", "write a unit test", "cover this with tests", or when new behavior lands uncovered.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Write a unit test
|
|
7
|
+
|
|
8
|
+
A test should fail when the behavior breaks and pass when it's correct — nothing more.
|
|
9
|
+
|
|
10
|
+
## Steps
|
|
11
|
+
|
|
12
|
+
1. **Read the unit's contract** — inputs, outputs, side effects, error paths. Test the contract, not the implementation.
|
|
13
|
+
2. **Match the nearest existing tests** — location, naming, helpers — unless they break a rule below.
|
|
14
|
+
3. **Load the packs** that shipped for this project — the test-**runner** pack (how to structure/assert/mock) plus the **stack** pack(s) matching what you're testing (DB isolation, framework client, domain mocks). Follow them alongside the rules:
|
|
15
|
+
|
|
16
|
+
{{FRAMEWORK_PACK_LOADER_BLOCK}}
|
|
17
|
+
|
|
18
|
+
4. **Write it, then prove it.** Run in isolation (`{{NEW_TEST_CMD_PATTERN}}`) — green. Break the code (or invert an assertion), re-run — red for the right reason. Revert. Run the suite (`{{TEST_CMD}}`){{SCOPED_TEST_NOTE}} — still green.
|
|
19
|
+
|
|
20
|
+
## Rules
|
|
21
|
+
|
|
22
|
+
1. **Mock only genuine externals** (third-party HTTP, payment/email/SMS, clock, randomness). Never mock the unit under test, its local collaborators, the DB, or your own pure functions.
|
|
23
|
+
2. **Assert the full value**, not counts or truthiness. `assert result`, `toBeTruthy()`, `len(x) == 3` are too weak. Assert contents and order for collections. **But don't over-fit brittle text** — for human-facing strings (error messages, UI copy, i18n) assert a stable anchor (an error code/type, a role, an i18n key, a substring), not the exact prose that churns.
|
|
24
|
+
3. **Leave no trace** — transaction-rollback harness where available; else clean up in teardown, and make teardown run on failure. Never depend on another test's data.
|
|
25
|
+
4. **No external service** that can't run in dev/test. Stub at the boundary.
|
|
26
|
+
5. **Test logic stays literal** — no loops/conditionals computing the expected value. Parametrize to cover cases; keep each expectation a literal.
|
|
27
|
+
6. **Decouple from internals** — assert observable state/output, not private methods or call counts. The test survives a behavior-preserving rewrite.
|
|
28
|
+
7. **One behavior per test**, named for what it pins (`returns_zero_for_empty_cart`).
|
|
29
|
+
8. **Kill nondeterminism** — control every entropy source (clock, randomness, timezone/locale, iteration/DB order, concurrency). Freeze/inject them; a test that passes only sometimes is worse than none.
|
|
30
|
+
9. **Cover the branches, not just the happy path** — aim high (95%+ branch coverage): each error path, guard, and edge (empty, boundary, null) gets a case. Coverage is the floor, correct assertions are the point.
|
|
31
|
+
|
|
32
|
+
## Project preferences
|
|
33
|
+
|
|
34
|
+
Apply these; `framework-default` defers to the pack. If the existing suite contradicts a preference, follow the preference for new tests and flag it — don't rewrite old tests.
|
|
35
|
+
|
|
36
|
+
{{PROJECT_TEST_PREFERENCES_BLOCK}}
|
|
37
|
+
{{ADDITIONAL_CONVENTIONS_BLOCK}}
|
|
38
|
+
|
|
39
|
+
## Pitfalls
|
|
40
|
+
|
|
41
|
+
- Asserting on a mock you set up — tests the mock, not the code.
|
|
42
|
+
- Giant snapshots as a substitute for asserting the fields that matter.
|
|
43
|
+
- `delete all rows` in setup instead of rollback — masks leaked state, races other tests.
|
|
44
|
+
- Testing the framework/ORM/language instead of your behavior on top of it.
|