@warlock.js/core 4.13.0 → 4.15.0

package/skills/test-service/SKILL.md

@@ -1,13 +1,21 @@
 ---
 name: test-service
-description: 'Pure unit tests against services, repositories, models, and use-cases — `setupTest({ connectors })` bootstraps each Vitest worker with its own DB/cache connections so you can call your code directly. Triggers: `setupTest`, `src/test-setup.ts`, `tests.connectors`, `Application.setEnvironment`; "unit-test a service", "test a repository query", "vitest setupFiles", "skip connectors for pure-logic tests"; typical import `import { setupTest } from "@warlock.js/core/tests"`. Skip: HTTP integration — `@warlock.js/core/test-http/SKILL.md`; warlock add test scaffold — `@warlock.js/core/write-cli-command/SKILL.md`; competing tooling: jest direct, `supertest`, `nock`.'
+description: 'Pure unit tests against services, repositories, models, and use-cases — `setupTest({ connectors })` bootstraps the framework with its own DB/cache connections so you can call your code directly, and `teardownTest()` closes it. Triggers: `setupTest`, `teardownTest`, `src/test-setup.ts`, `tests.connectors`, `tests.setupTimeout`, `Application.setEnvironment`; "unit-test a service", "test a repository query", "vitest setupFiles", "skip connectors for pure-logic tests"; typical import `import { setupTest, teardownTest } from "@warlock.js/core/tests"`. Skip: HTTP integration — `@warlock.js/core/test-http/SKILL.md`; warlock add test scaffold — `@warlock.js/core/write-cli-command/SKILL.md`; competing tooling: jest direct, `supertest`, `nock`.'
 ---
 
 # Warlock — test a service
 
-For unit tests, you import the thing under test and call it directly. No HTTP, no fetch, no controllers. Framework testing in Warlock is about getting your **service layer** under test efficiently — and that means each Vitest worker needs its own bootstrapped framework with a DB connection.
+For unit tests, you import the thing under test and call it directly. No HTTP, no fetch, no controllers. Framework testing in Warlock is about getting your **service layer** under test efficiently — and that means your tests need a bootstrapped framework with a DB connection.
 
-`setupTest()` is the one-call bootstrap that gives each worker that environment.
+`setupTest()` is the one-call bootstrap that provides that environment; `teardownTest()` closes it.
+
+⚠ **Corrected in 4.14.0 — `setupTest` is CALLED once per TEST FILE, not once per worker.** Every version of this skill through 4.13.0 said "per worker", the generated `src/test-setup.ts` carries a `Per-Worker Test Setup` comment saying the same thing, and **both were wrong.** Vitest runs `setupFiles` before **each test file**, and the setup module's registry is rebuilt every time — measured across all four `pool` × `isolate` combinations.
+
+⛔ **If your project was generated before 4.14.0, fix BOTH the comment and the call.** The comment is false, **and** the generated `setupTest({ connectors: true })` is now an *explicit* value that overrides your `src/config/tests.ts`. **Bare `setupTest()` is the correct call.**
+
+**The lifetime is FILE-SCOPED, on purpose.** Your setup file bootstraps the framework and its `afterAll(teardownTest)` closes it, once per test file. **One owner, one pairing — correct under every pool, every isolation setting, and watch mode.**
+
+⚠ **A worker-scoped lifetime is possible and is deliberately not shipped yet.** Lifecycle state now lives in the worker runtime, so leaving the framework running would let every file in a worker share one bootstrap. Two things block claiming it: under `pool: "threads"` we cannot observe whether Node reclaims a torn-down thread's sockets and pools, and **in watch mode Vitest reuses workers between reruns, so there is no recycle and no cleanup owner.** It gets taken when the real cost is measured and the integration is chosen, not inherited. See *Lifecycle and repeated calls* below.
 
 ⚠ **Changed in 4.13.0 — the import is a subpath now.** `setupTest` used to be re-exported from the package root; it is not any more, because that put the test helpers into every application's production module graph. `import { setupTest } from "@warlock.js/core"` now fails with *"has no exported member"* — **add `/tests` to the specifier and nothing else changes.**
 
@@ -34,9 +42,9 @@ describe("registerUserService", () => {
 });
 ```
 
-No `beforeAll(setupTest)` in this file — the project's `src/test-setup.ts` (registered as `setupFiles` in `vite.config.ts`) already ran it once per worker before any test executed.
+No `beforeAll(setupTest)` in this file — the project's `src/test-setup.ts` (registered as `setupFiles` in `vite.config.ts`) already ran it **before this file's tests executed**, as it does before every test file.
 
-## `setupTest({ connectors })` — the worker bootstrap
+## `setupTest({ connectors })` — the bootstrap
 
 ```ts
 import { setupTest } from "@warlock.js/core/tests";
@@ -51,10 +59,10 @@ What it does (in order):
 3. Runs `bootstrap()` — env, app, prestart hooks.
 4. Initializes the `filesOrchestrator` (module/route/config discovery, no file watching).
 5. Loads all `src/config/*.ts` files.
-6. Reads `tests.connectors` from config (overrides the parameter if set).
+6. Resolves the connector selection — **an explicit parameter wins, then `tests.connectors` from config, then the `true` default.** See *Selecting connectors* below.
 7. Starts the chosen connectors — but **never `http`** when you pass a boolean. HTTP is the global-setup's job.
 
-The result: each worker has its own DB/cache/logger/storage connections. Models save, repositories query, services run. Same code as production, just isolated to the test process.
+The result: DB/cache/logger/storage connections your code can use. Models save, repositories query, services run. Same code as production, just isolated to the test process.
 
 ### The `connectors` parameter
 
@@ -64,9 +72,9 @@ The result: each worker has its own DB/cache/logger/storage connections. Models
 | `false`                | None                                                             | Pure logic tests with no DB / cache touches (parsers, validators, util functions). |
 | `["database", "cache"]` | Just those, in that order                                        | A test that only needs DB but not, say, the storage driver. |
 
-The default `true` is the sane choice. Reach for `false` when the unit you're testing genuinely doesn't talk to any framework subsystem — pulling up a DB connection per worker just to test a string parser is wasted setup time.
+The default `true` is the sane choice. Reach for `false` when the unit you're testing genuinely doesn't talk to any framework subsystem — pulling up a DB connection **for every file** just to test a string parser is wasted setup time.
 
-### Override via config — `src/config/tests.ts`
+### Selecting connectors — `src/config/tests.ts`
 
 ```ts title="src/config/tests.ts"
 const testsConfigurations = {
@@ -76,7 +84,33 @@ const testsConfigurations = {
 export default testsConfigurations;
 ```
 
-If `tests.connectors` is set, **it wins over the `setupTest({ connectors })` parameter**. Use this when every test file in the project agrees on the same minimal connector list — saves repeating the explicit array in `test-setup.ts`.
+⚠ **BREAKING in 4.14.0 — the precedence flipped.**
+
+| | Order |
+|---|---|
+| **4.13.0 and earlier** | `tests.connectors` config **>** `setupTest({ connectors })` parameter **>** `true` |
+| **4.14.0 onward** | **explicit `setupTest({ connectors })` parameter** **>** `tests.connectors` config **>** `true` |
+
+**An explicit call-site value now beats project config.** If your project sets `tests.connectors` *and* some test file passes `connectors` explicitly, **that file will start a different connector set after upgrading.** Search for `setupTest({` across your tests before you upgrade — a call passing `connectors` was previously ignored and is now honoured.
+
+**"Explicit" means you supplied a non-`undefined` value.** Both of these fall through to config:
+
+```ts
+await setupTest();                       // → tests.connectors, else true
+await setupTest({});                     // → tests.connectors, else true
+await setupTest({ connectors: undefined }); // → tests.connectors, else true — NOT "start none"
+```
+
+The `undefined` rule is deliberate: an optional variable that happens to be `undefined` must not silently erase your project config.
+
+```ts
+await setupTest({ connectors: false });        // → none, even if config says otherwise
+await setupTest({ connectors: ["database"] }); // → exactly that, even if config differs
+```
+
+⚠ **The generated `src/test-setup.ts` calls `setupTest()` with no argument, on purpose.** If you "helpfully" change it to `setupTest({ connectors: true })`, you have made it explicit and **erased the `tests.connectors` layer for the whole project.**
+
+Use `tests.connectors` when every test file agrees on the same minimal list — it saves repeating the array, and individual files can still override it.
 
 ## Project wiring — `src/test-setup.ts` + `vite.config.ts`
 
@@ -84,14 +118,22 @@ The `warlock add test` feature creates both files. The standard wiring:
 
 ```ts title="src/test-setup.ts"
 /**
- * Per-Worker Test Setup
- * Runs in EACH Vitest worker thread before tests execute.
+ * Test Setup
+ * Runs before EACH test file — not once per worker.
  */
-import { setupTest } from "@warlock.js/core/tests";
+import { afterAll } from "vitest";
+import { setupTest, teardownTest } from "@warlock.js/core/tests";
 
-await setupTest({ connectors: true });
+await setupTest();
+afterAll(teardownTest);
 ```
 
+⛔ **Three things changed here in 4.14.0. If you generated this file earlier, replace all three — it is not a comment fix.**
+
+1. **`afterAll(teardownTest)` is new and mandatory.** Nothing else closes the framework your tests started. This is what makes the lifetime file-scoped and owned rather than left running.
+2. **The call is now bare `setupTest()`, not `setupTest({ connectors: true })`.** Under the flipped precedence, passing `true` is an *explicit* value and would override `tests.connectors` for **every file in the project.** Bare means "whatever this project configured, else the default".
+3. **The comment used to say `Per-Worker Test Setup` / "Runs in EACH Vitest worker thread".** False — see the top of this skill.
+
 ```ts title="vite.config.ts"
 import { lowerStage3Decorators } from "@warlock.js/core/vite";
 import mongezVite from "@mongez/vite";
@@ -101,7 +143,7 @@ export default defineConfig({
   plugins: [lowerStage3Decorators(), mongezVite()],
   test: {
     globalSetup: "./src/test-global-setup.ts",  // ← HTTP server (see test-http skill)
-    setupFiles: ["./src/test-setup.ts"],         // ← runs setupTest per worker
+    setupFiles: ["./src/test-setup.ts"],         // ← runs setupTest before EACH test file
     environment: "node",
     globals: false,
     include: ["src/app/**/*.test.ts"],
@@ -206,7 +248,9 @@ afterEach(async () => {
 });
 ```
 
-Vitest runs tests in a single worker file sequentially, so an `afterEach` truncate gives each test a clean slate. For cross-file isolation, run the suite with `vitest --pool=forks --maxWorkers=N` and rely on the per-worker connection — each file's data stays within its worker until the run ends.
+Vitest runs the tests within one file sequentially, so an `afterEach` truncate gives each test a clean slate.
+
+⚠ **Cross-*file* isolation is not solved by this.** Separate workers get separate **connections**, not separate **rows** — two files pointed at the same database see each other's committed data regardless of pool or worker count. **Truncate what your file wrote; don't assume the worker boundary did it for you.** Real data isolation (DB-per-worker, transaction-per-test) is a separate piece of work and is not in this release.
 
 ### Skipping connectors for pure logic tests
 
@@ -226,14 +270,109 @@ describe("slugify", () => {
 });
 ```
 
-`setupTest` is idempotent per worker (`isSetupComplete` flag) — calling it again with different options after `src/test-setup.ts` already ran is a no-op. **That includes a `connectors: false` call: if `src/test-setup.ts` already ran `setupTest()` in this worker, the example above changes nothing.** To genuinely skip connectors, either set `tests.connectors: false` in config (project-wide) or rely on the default in `src/test-setup.ts` being what you want most of the time.
+⛔ **BREAKING in 4.14.0 — the example above now REJECTS if your project has a `src/test-setup.ts`.**
+
+Through 4.13.0 a second `setupTest` call with different options was a **silent no-op** — you asked for no connectors, got all of them, and nothing told you. In 4.14.0 a conflicting call **rejects with an error naming both the active and the requested selection**, because silently ignoring what you asked for is worse than failing.
+
+**If `src/test-setup.ts` already ran `setupTest()` in this file, use one of these instead:**
+
+```ts
+// 1. Tear down first, then set up differently — and PUT IT BACK when the file ends.
+import { afterAll, beforeAll } from "vitest";
+import { setupTest, teardownTest } from "@warlock.js/core/tests";
+
+beforeAll(async () => {
+  await teardownTest();
+  await setupTest({ connectors: false });
+});
+
+afterAll(async () => {
+  await teardownTest();      // ← REQUIRED, see below
+});
+```
+
+⛔ **The `afterAll` is not optional** — without it you leave a `connectors: false` runtime ready when the file ends.
+
+⚠ **This pattern interacts with the `afterAll(teardownTest)` in your setup file, and the relative ordering of the two has not been verified.** `teardownTest` is idempotent, so whichever runs second finds an idle lifecycle and no-ops — but **do not build anything on a particular order until someone has measured it.** This is the strongest argument for option 2 below.
+
+```ts
+// 2. Or don't call setupTest at all — a pure-logic test needs nothing from it.
+//    The connectors your setup file started are already running; you simply don't use them.
+```
+
+**3. Or set `tests.connectors: false` in `src/config/tests.ts`** if no test file in the project needs connectors.
+
+**Option 2 is usually right, and option 1 is easy to get wrong.** Tearing down and re-bootstrapping costs a full framework startup — twice, once for your file and once for the next — to avoid a DB connection you were never going to use. **Reach for option 1 only when a connector's mere presence breaks the thing you're testing**, not to save setup time.
+
+## Lifecycle and repeated calls
+
+`setupTest` / `teardownTest` are a pair. **The harness that calls one owns calling the other in the same context.**
+
+| Call | Behaviour |
+|---|---|
+| `setupTest(x)` while idle | bootstraps |
+| `setupTest(x)` while already ready with the **same** effective options | no-op |
+| `setupTest(y)` while ready or starting with **different** effective options | ⛔ **rejects**, naming active vs requested |
+| two concurrent `setupTest(x)` calls | share one startup |
+| `setupTest` after a failed setup | allowed — a failed setup unwinds and returns to idle |
+| `teardownTest()` while idle | no-op |
+| two concurrent `teardownTest()` calls | share one shutdown |
+| `setupTest(y)` after a **successful** teardown | allowed, different options fine |
+
+**"Same options" is compared by meaning, not by literal value** — connector arrays are deduplicated and compared as sets, so `["cache", "database"]` and `["database", "cache", "cache"]` are the same selection.
+
+⚠ **A failed shutdown poisons the lifecycle.** If `teardownTest()` rejects because the shutdown layer reported a failure, later `setupTest` calls **refuse until the Vitest worker is recycled** or a retried teardown fully succeeds. This is deliberate: a cleared flag does not prove that ports, sockets, pools or timers actually closed, and pretending otherwise hands you a "clean" run built on a leaked runtime.
+
+⚠ **What it cannot detect:** `connectorsManager.shutdown()` catches and logs individual connector failures internally. Those never reach this lifecycle, so they never poison it. It surfaces what that layer reports — no more.
+
+### `tests.setupTimeout` — the setup attempt is bounded
+
+**New in 4.14.0.** A setup attempt that never settles used to leave the lifecycle stuck in `starting` and take the worker down with an out-of-memory crash. It is now bounded.
+
+```ts title="src/config/tests.ts"
+const testsConfigurations = {
+  connectors: ["database", "logger"],
+  setupTimeout: 120000,   // milliseconds — this is the default
+};
+
+export default testsConfigurations;
+```
+
+**Default: `120000` (two minutes)** — far above a healthy cold start, below the point where you'd stop watching the terminal. When it expires:
+
+```
+setupTest() did not finish within 120000ms and is stuck in the "starting" state. The
+lifecycle is now poisoned: whatever that attempt had already started is not known to be
+closed, so later setupTest() calls refuse until the Vitest worker is recycled. If your
+cold start is legitimately slower than this, raise the bound with `tests.setupTimeout`
+in `src/config/tests.ts` — milliseconds, default 120000.
+```
+
+1. **It bounds the setup ATTEMPT, not teardown separately.** `teardownTest()` awaits the same attempt, so it inherits the bound — **one timer, not two.** A second teardown-side deadline was tried and rejected: it re-introduced the unbounded re-entry this whole guard exists to remove.
+2. **Expiry poisons the lifecycle**, it does not return to `idle`. The attempt may have started connectors nobody can now account for, so pretending the runtime is clean would be worse than refusing.
+3. **⛔ An invalid `setupTimeout` throws, naming the bad value.** Zero, negative and non-numeric all fail loudly rather than falling back to the default — a silent fallback would hide a typo behind a working suite.
+4. **The bound is measured from when the attempt started**, not from when config was read. `tests.setupTimeout` is only readable after `loadConfigFiles()`, which happens *inside* the window being bounded; re-arming naively would give you the default plus your configured value.
+
+⚠ **What is proven and what is not.** The nine guards above were each seen to fail under their own mutation. **But every spec injects its scheduler**, so the default *value* is tested while the production timer — and whether its `unref` actually releases the worker — is not. **And no spec observes a real hang**: the stuck attempt is a mock gate, not a socket that never returns. These prove what the lifecycle *decides*, not what a genuinely wedged connector does.
+
+### State is per worker runtime, not per module
+
+The lifecycle state lives in the worker runtime, not in a module variable. That matters because **Vitest rebuilds the module registry between test files while the worker itself keeps running** — so a module-level flag resets exactly where live DB connections and pools survive. Scope:
+
+- **`pool: "forks"`** — state is per worker **process**.
+- **`pool: "threads"`** — state is per worker **thread**. It does **not** cross threads; `globalThis` is per realm, not per process.
+
+In both cases the guard's scope matches the resource's scope, which is the point.
 
-⚠ **Config beats the parameter.** If `tests.connectors` is set at all, `setupTest({ connectors })` cannot override it — the config value wins. That is the current contract, not an accident; a per-call override is under discussion for a later release.
+⚠ **Not guaranteed:** under `threads` with `isolate: true`, Vitest tears the thread down while the process lives on. **Whether Node reclaims that thread's sockets and pools is unmeasured**, and nothing in this lifecycle can observe it.
 
 ## Gotchas
 
-- **`setupTest` is idempotent per worker.** Second + later calls early-return. You can't "swap" the connector set mid-run — the first call wins, including the one in `src/test-setup.ts`. Choose your worker default carefully.
-- **Per-worker connections are separate from the HTTP server's connections.** A row inserted by a service-level test is on the worker's connection; the HTTP test server has its own. They don't see each other unless they're both pointing at the same physical DB and the inserting test has already committed.
+- **`setupTest` runs per test file, not per worker** — every version of this skill through 4.13.0 said otherwise. **You pay one framework bootstrap per test file**, which is what 4.13.0 already cost; the difference is that it is now a chosen lifetime rather than a side effect of a module flag resetting.
+- ⛔ **Never delete the `afterAll(teardownTest)` from your setup file.** Without it nothing closes what `setupTest` opened, and the connectors outlive the file that started them.
+- ⛔ **You can't swap the connector set by calling `setupTest` again — it rejects now.** Through 4.13.0 the second call was silently ignored. Tear down first, or don't call it.
+- **These connections are separate from the HTTP test server's.** A row inserted by a service-level test is on this connection; the HTTP test server has its own. They only see each other if both point at the same physical DB **and** the inserting test has committed.
+- **`setupTest` takes over the process's `connectorsManager` for its lifetime.** Teardown is manager-wide, so **mixing `setupTest` with manually started connectors is unsupported** — teardown may close yours too.
 - **`NODE_ENV` is set to `"test"`** by `setupTest`. Code that branches on `Application.isProduction` / `Application.isDevelopment` sees `false` for both. If your tests need production-like config (cookies, CORS), set those values in `src/config/*.ts` explicitly under the test branch — don't rely on the env flag.
 - **No HTTP from this layer.** `setupTest({ connectors: true })` never starts the HTTP connector by design. Don't try to `request.app.http` your way to a fetch test — use the `test-http` skill instead.
 - **Don't import `vitest-setup` from `@warlock.js/core/src/...`.** The public surface is `import { setupTest } from "@warlock.js/core/tests"`. Reaching into source paths breaks when the package layout shifts.