@boopkit/cap-testing 0.5.2 → 0.5.4

package/README.md

@@ -12,7 +12,7 @@ Agent skills plus a minimal Jest + ts-jest scaffold. The scaffold is intentional
 
 Copied into your project on install:
 
-- `jest.config.js` — ts-jest preset, Node test environment, test matcher at `tests/**/*.test.ts`
+- `jest.config.js` — ts-jest preset, Node test environment, test matcher at `tests/**/*.test.ts`, and the worker settings the suite runs under (`maxWorkers: 4`, `workerIdleMemoryLimit: '500MB'`). Those two spread test files across a pool of worker processes Jest recycles as they grow, so a long suite's memory stays bounded instead of climbing until it dies. They live in the config rather than in the `test` script so that focused runs — `npx jest tests/foo.test.ts` — inherit them too. Because files now run concurrently, tests must namespace whatever they create; drop `maxWorkers` to 1 if a suite is not isolation-safe yet.
 - `tsconfig.test.json` — extends your project tsconfig and includes `tests/`
 - `tests/app.test.ts` — a starter test asserting the Express app boots and has body parsing wired
 
@@ -20,7 +20,7 @@ Copied into your project on install:
 
 | Script | Command |
 |--------|---------|
-| `test` | `jest --runInBand` |
+| `test` | `jest` |
 | `test:watch` | `jest --watch` |
 
 ### Skills installed
@@ -48,7 +48,7 @@ Copied into `skills/` on install:
 Run the suite:
 
 ```bash
-npm test           # full suite, serialized
+npm test           # full suite
 npm run test:watch # watch mode
 ```
 

package/capability.json

@@ -20,7 +20,7 @@
   ],
   "scripts": {
     "test": {
-      "cmd": "jest --runInBand",
+      "cmd": "jest",
       "description": "Run all tests"
     },
     "test:watch": {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@boopkit/cap-testing",
-  "version": "0.5.2",
+  "version": "0.5.4",
   "description": "Jest testing setup and testing skills for AI agents",
   "files": [
     "scaffold/",

package/scaffold/jest.config.js

@@ -6,4 +6,28 @@ module.exports = {
   transform: {
     '^.+\\.ts$': ['ts-jest', { tsconfig: 'tsconfig.test.json' }],
   },
+
+  // Run test files across a small pool of worker processes that Jest recycles.
+  //
+  // `workerIdleMemoryLimit` is the load-bearing setting and it is easy to drop
+  // by accident. Without it Jest runs every file in its own main process — never
+  // recycling it — whenever `maxWorkers <= 1`, and that process's heap grows for
+  // the whole run until a long suite dies of it. Setting a memory limit is what
+  // overrides that decision: it forces real workers and gives Jest a threshold at
+  // which to replace one. `maxWorkers` on its own does not do this, and neither
+  // did the `--runInBand` this replaced — that flag asks for exactly the
+  // un-recycled main process described above.
+  //
+  // The limit is compared against V8 `heapUsed`, not RSS, and is checked between
+  // test files. Budget for real memory use somewhat above `maxWorkers` times this
+  // number, and note that a single file exceeding it is not caught mid-flight.
+  //
+  // Four workers is a deliberate middle setting; six is comfortable on a developer
+  // machine with memory to spare. The constraint on raising it is not the machine,
+  // it is your tests: files now run CONCURRENTLY, so anything sharing a database,
+  // a fixture directory, or a well-known key with another test file will collide.
+  // Namespace what each test creates. Drop to `maxWorkers: 1` if a suite is not
+  // isolation-safe yet — that still gets you the recycling, just without overlap.
+  maxWorkers: 4,
+  workerIdleMemoryLimit: '500MB',
 };

package/skills/boop-testing-tdd-light/SKILL.md

@@ -12,7 +12,7 @@ Red → Green → Refactor. Document "before" and "after" states.
 2. **Green**: Implement the minimum change. Add any additional tests.
 3. **Verify**: Re-run all tests from steps 1-2. Type-checking alone is **not sufficient** — functional tests must pass. Check for regressions in related functionality. Verify from the user's perspective — don't just test internal state. For example, if the feature writes files the user interacts with, verify those files exist and are correct.
 
-Test tools: `npm run test` (Jest), `npm run tester` (CLI — see `npm run tester -- help`). For UI changes, coordinate manual browser testing.
+Test tools: `npm test` for the whole suite, `npx jest <path>` for a focused run. Go through `npm test` rather than the runner binary when you want the suite — the script and `jest.config.js` together carry the settings the suite is tuned for, including how many files run at once and how much memory a worker may hold before Jest replaces it, and a bare runner invocation drops whatever lives only in the script. The general testing skill explains why those settings matter and which of them is load-bearing. If the project defines its own test commands beyond these, its `package.json` is the authority. For UI changes, coordinate manual browser testing.
 
 ## Picking the right test shape
 

package/skills/boop-testing-testing/SKILL.md

@@ -58,6 +58,59 @@ A skip carrying neither a ticket reference nor a `SKIP-OK:` reason is a conventi
 
 ---
 
+## Which tests to run, and when
+
+**The rule.** A change runs the focused tests for what it touched, plus the project's whole-program static check. The full suite runs once per branch, on the work that completes it. The `boop-project-mgmt-beads-planning` skill carries the canonical wording to put on issues; this section is the reasoning behind it, and the two have to keep agreeing.
+
+**Why the whole suite is the wrong tool mid-change.** The problem is localization, first and foremost. Break a shared dependency and the suite reports it as a cascade of failures in tests that never mention the thing you changed, and you are then reading a wall of red to reconstruct a fact the type-checker would have told you outright, naming a file and a line. A static check plus a search for callers answers "does everything that uses this still hold?" both faster and more precisely than the suite does.
+
+Second, it is slower than the focused run, though see the caution below about how much weight that carries.
+
+Third, a suite already red for an unrelated reason gives you nothing to compare against, so mid-change runs on a red tree get retried and re-read rather than acted on.
+
+**Do not rest the rule on the suite being slow.** A suite's wall-clock time is a property of how the runner is invoked as much as of the tests themselves, and it can fall by an order of magnitude on a single configuration change — running the files in a recyclable worker pool rather than in one long-lived process is the common case. A rule argued from "the suite takes tens of minutes" quietly expires the day somebody fixes the runner, and then it reads as superstition. Argue it from instrument fit, which does not expire.
+
+**What we have not measured.** Nobody has measured how often a focused-tests-only run misses a break that the full suite would have caught, weighed against the time the narrow runs save. Until somebody does, this rule rests on the shippable-unit argument and on instrument fit, and on nothing else. That is worth stating plainly rather than dressing up.
+
+### Go through the project's test script, not a bare runner invocation
+
+Run the suite the way the project defines it — `npm test` — rather than reaching past it to the runner binary. The script and the runner's config file together carry the settings the suite is tuned for: how many files run at once, and how much memory a worker may hold before the runner replaces it. A bare `npx jest` picks up the config file but drops anything that lives only in the script.
+
+Those settings are not decoration. With Jest specifically, `--maxWorkers=1` **does nothing on its own** — Jest runs every file in its own main process, and never recycles it, whenever `maxWorkers <= 1`. Setting `workerIdleMemoryLimit` is what forces a real worker process and gives Jest a threshold at which to replace it; it is the load-bearing setting, and a long suite that grows its heap until it dies is what its absence looks like. Anyone "simplifying" the two settings down to the worker cap has restored the bug.
+
+Keep those settings in the runner's config file rather than in the `test` script, so that the focused runs this section asks for inherit them too. `boop add testing` scaffolds them that way, at four workers and a 500MB per-worker ceiling.
+
+**A worker pool makes test isolation a requirement, not a nicety.** More than one worker means test *files* run at the same time, in separate processes, against the same backing services. Anything two files share and do not namespace — a table one of them truncates in setup, a fixture directory, a well-known key or account id — is now a race, and it surfaces as a suite that passes alone and fails in a full run. That is the same symptom the shared-dev-database section below describes, reached by a different route, and it invites the same wrong fix of weakening the assertion that caught it. Namespace what each test creates. Where a suite is not isolation-safe yet, set `maxWorkers: 1`: that keeps the recycling, which is the part that fixes the memory, and gives up only the overlap.
+
+## Integration tests never touch the shared dev database
+
+A test or migration that needs a real database must not run against the checkout's inherited connection string.
+In a worktree that is the operator's live dev database, and **file isolation is not data isolation**.
+Measured cost: a loop's cutover migration ran against the shared dev database and destroyed the operator's live data.
+An agent that applies migrations by hand, outside the test harness, must still create and point at a throwaway database first, and say so in its run record.
+
+Make the harness handle this so nobody has to remember it.
+Ask git whether the run is in a linked worktree, and in one, derive a per-worktree test database name from the worktree's absolute path — the same name on every run in that worktree, so the one-time migration cost is paid once.
+The main checkout keeps the ordinary shared test database, and an explicit test-database override still wins over both, so it remains the escape hatch when you want a particular database.
+
+Without that derivation, two concurrent runs both resolve to one test database and each suite's setup truncates the other's rows.
+What that produces is not an obvious collision: a suite passes alone, fails in a full run, and fails on unmodified `HEAD` too, which reads as pre-existing flakiness in your own code and invites an agent to weaken real assertions until the phantom failures stop.
+If you see that pattern, check which database the run resolved to before you touch an assertion.
+
+## A suite that passes alone and fails in a full run is a suspect, not a verdict
+
+Inter-suite interference — leaked state, a shared fixture, a revision conflict under concurrency — produces exactly the shape of a real regression.
+Rerun before you touch an assertion, and never make a red test green by weakening what it asserts.
+The failure that invites that is exactly the failure that is lying to you.
+
+If a suite is a known intermittent offender, say so in writing where the next reader will look, for the same reason the shared-database hazard is written down: an undocumented intermittent failure gets "fixed" by deleting the assertion that caught it.
+
+## A guard is only a guard if it can tell it is talking to your application
+
+A test that gates its live cases on a generic liveness probe — a plain health endpoint returning ok — is trusting that nothing else could answer.
+Measured: an unrelated process on the operator's machine answered that probe while returning 404 for every real route, so cases that should have skipped ran and failed.
+Make the guard assert something only this application would say.
+
 ## Generic Testing Approach
 
 These principles apply across all subsystems.
@@ -168,4 +221,4 @@ When code produces a schema or structure that is validated by an external system
 ## References
 
 - [development.skill.md](development.skill.md) — Light TDD Flow, completion criteria
-- [jest.config.js](../jest.config.js) — Jest configuration
+- [jest.config.js](../../jest.config.js) — Jest configuration, at the project root this capability installs it to

package/skills/boop-testing-testing-express-web/SKILL.md

@@ -20,6 +20,8 @@ npx jest tests/services/foo --no-coverage                   # Pattern match
 npx jest tests/services/foo.test.ts -t "handles replay"     # By test name
 ```
 
+The focused forms are safe to reach for: the concurrency and per-worker memory settings live in `jest.config.js`, not in the `test` script, so `npx jest <path>` runs under the same settings the full suite does. Which of these you should be running on a given piece of work is a separate question — see "Which tests to run, and when" in the parent skill.
+
 **Prerequisites**: integration tests run against the project's real backing services rather than mocks, so whatever the app talks to at runtime — the database first of all — has to be reachable from the machine running the tests. Which services those are depends on the capabilities the project has installed; its README and environment template are the authority.
 
 ## Test File Layout