@gfdelarue/testkit 0.3.0

package/README.md

@@ -0,0 +1,1125 @@
+<p align="center">
+  <img src="assets/logo.png" alt="Testkit logo" width="240">
+</p>
+
+# @gfdelarue/testkit
+
+`@gfdelarue/testkit` is the orchestrator and managed distribution for product test
+suites. It discovers `*.testkit.ts` files, infers suite ownership from the
+filesystem, starts local services, provisions Docker-managed local Postgres
+databases, schedules suites, and collects results and artifacts.
+
+Testkit exists to replace false confidence with inspectable execution evidence.
+It cannot prove that an entire application works. It can make narrower claims
+that deserve trust: these tests genuinely executed; these were the runtimes,
+fixtures, and inputs supplied to the run; this is what passed, failed, was
+skipped, or did not run; and this is the evidence behind the result. Testkit
+retains provenance, logs, and verifiable artifacts so those claims can be
+audited rather than accepted on the strength of a green check mark.
+
+Configure Testkit with an optional repo-root `testkit.config.ts`.
+
+## CI gate
+
+Give the installed CLI one stable package script and use that script locally and
+in CI:
+
+```json
+{
+  "scripts": {
+    "testkit": "testkit run"
+  }
+}
+```
+
+```bash
+npm ci
+npm run testkit
+```
+
+The command exits zero only when at least one test file completed and every
+required file passed. Test failures, infrastructure failures, interrupted or
+not-run files, an all-skipped selection, and an empty selection all fail the
+gate. `npm run testkit -- --allow-empty` is an explicit escape hatch for a job
+whose empty selection is intentional; it should not be the default CI command.
+
+Testkit writes the immutable, verifiable run bundle under
+`.testkit/results/runs/<run-id>/`. Upload `.testkit/results` with `if: always()`
+when CI evidence should survive a failed job.
+
+Publishing that evidence to Testkit Cloud is an additional explicit step. Set
+`TESTKIT_CLOUD_PROJECT_ID` and the secret `TESTKIT_CLOUD_TOKEN`, then run:
+
+```bash
+npx testkit cloud publish
+```
+
+The command verifies the immutable bundle before upload. Ordinary runs never
+contact Cloud, and the presence of `CI` does not authorize publication.
+
+## Connect to Cloud
+
+For interactive use, approve your CLI through the browser once:
+
+```sh
+npx testkit cloud login
+npx testkit cloud project create
+npx testkit cloud publish --project PROJECT_ID --run RUN_ID
+```
+
+Login works outside Git. Project creation infers the repository from Git origin
+or accepts `--repository owner/repo`. It resumes after any required provider
+consent. `cloud list`, `cloud show`, and `cloud apply --file` provide discovery,
+settings and product control. See the [Cloud control boundary](docs/testkit-cloud-boundary.md#complete-cloud-control)
+and [complete operation reference](https://github.com/gfdelarue/testkit-cloud/blob/master/docs/cloud-control.md).
+Explicit `TESTKIT_CLOUD_TOKEN` still takes precedence over the saved account.
+
+## Execution engines
+
+Testkit has three explicit built-in engines rather than one implicit runtime:
+
+- `k6` runs `int`, `e2e`, `load`, and `dal` files with native k6
+  and xk6-sql semantics.
+- `node-vitest` runs purpose-built `data` and `contract` files with Node and
+  Vitest.
+- `playwright` runs `ui` files with Playwright and a managed Chromium.
+
+Every test file must export `defineFile({ engine: "..." })`. The filename
+suffix supplies the public test type; Testkit validates that explicit pair and
+derives a runner profile. It never infers a default engine or falls back to a
+different engine.
+
+Consumers declare only `@gfdelarue/testkit`. The root package owns the three
+adapters and exact JavaScript tools it executes. The k6 engine ships as
+separately versioned npm packages built from
+[gfdelarue/testkit-k6](https://github.com/gfdelarue/testkit-k6); npm installs
+the compatible k6 target package and its shared source package through the
+root's optional dependency graph, and Testkit resolves that direct executable
+without consulting `PATH`. Node/Vitest and Playwright resolve their managed public packages from the
+installed Testkit root, so product-local tool copies do not select Testkit's
+runtime.
+
+Playwright browser installation is explicit:
+
+```bash
+npx @gfdelarue/testkit setup playwright
+npx @gfdelarue/testkit setup playwright --with-deps # CI and fresh hosts
+```
+
+Setup and execution share one canonical Playwright registry. Normal runs never
+install or download a browser. See [Engine Architecture](docs/engine-architecture.md)
+for the authoring, installation, trust, and failure contracts.
+
+Managed commands use the same Execa-backed adapter on hosted Linux and Windows
+runners. Testkit passes exact argv, cwd, and environment values, bounds captured
+output, applies cancellation and timeouts, and asks the platform adapter to
+terminate descendants. Persisted recovery identities remain birth-fenced so a
+later cleanup does not signal a reused process ID.
+
+The five k6 target packages share one exact-version source package that carries
+the AGPL corresponding source archive and licenses. Runtime evidence records
+the package name and version and the k6 version observed by executing the
+installed binary. The
+[testkit-k6 README](https://github.com/gfdelarue/testkit-k6#readme) documents
+the reproducible build, the expected executable digests, and the license
+notes.
+
+Managed task environment is engine input rather than ambient host state. k6
+suites read it through native `__ENV`; Node/Vitest data and contract callbacks
+read immutable, enumerable `context.env`; and Playwright suites can read the
+same contract from `readPlaywrightContext()?.env`. Managed Node processes also
+expose each value as an immutable, non-enumerable `process.env.KEY` property so
+libraries that read `process.env` directly see it, but non-Node child processes
+do not receive consumer values unless the suite explicitly passes them.
+
+Consumer environment names are checked case-insensitively. Every engine
+reserves `APPDATA`, `ComSpec`, `HOME`, `LANG`, `LC_ALL`, `LOCALAPPDATA`, `PATH`,
+`PATHEXT`, `SystemRoot`, `TEMP`, `TMP`, `TMPDIR`, `USERPROFILE`,
+`XDG_CACHE_HOME`, `XDG_CONFIG_HOME`, and `XDG_DATA_HOME`. In addition:
+
+- k6 reserves the `K6_` and `TESTKIT_` prefixes plus `BASE_URL` and `GODEBUG`;
+- Node/Vitest reserves the `TESTKIT_` and `VITEST_` prefixes plus `BASE_URL`,
+  `FORCE_COLOR`, `NODE_OPTIONS`, `NO_COLOR`, and `UPDATE_SNAPSHOT`;
+- Playwright reserves the `TESTKIT_`, `PLAYWRIGHT_`, `PWTEST_`, and `PW_TEST_`
+  prefixes plus `BASE_URL`, `FORCE_COLOR`, `NODE_OPTIONS`, and `NO_COLOR`.
+
+Case-variant duplicates and invalid names or NUL-containing values fail instead
+of being normalized. k6 accepts at most 128 consumer entries and 16,384
+environment-block character units. Node/Vitest accepts at most 512 entries and
+4 MiB of UTF-8 key/value data. Playwright accepts at most 512 entries and 2 MiB
+of UTF-8 key/value data. The smaller k6 limit is required because native k6
+receives consumer values in its physical process environment; Node/Vitest and
+Playwright transport them through owned runtime-input files. See
+[Engine Architecture](docs/engine-architecture.md#managed-environment) for the
+complete transport and aggregate-input limits.
+
+## Usage
+
+```bash
+cd my-product
+
+# Run every testkit-managed suite in batch mode
+npx @gfdelarue/testkit run
+
+# Inspect discovered tests without running them
+npx @gfdelarue/testkit discover
+npx @gfdelarue/testkit discover --output-mode verbose
+npx @gfdelarue/testkit discover --json > .testkit/discovery.json
+npx @gfdelarue/testkit discover --output .testkit/discovery.json
+
+# Filter by type
+npx @gfdelarue/testkit --type int
+npx @gfdelarue/testkit --type dal
+npx @gfdelarue/testkit --type e2e
+npx @gfdelarue/testkit --type int,e2e,dal
+npx @gfdelarue/testkit --type ui
+
+# Parallel file execution
+npx @gfdelarue/testkit --task-concurrency 8
+
+# One file-level wall clock budget for every suite file
+npx @gfdelarue/testkit --file-timeout-seconds 60
+
+# Specific service / suite
+npx @gfdelarue/testkit --service frontend --type ui -s navigation
+npx @gfdelarue/testkit --service api --type int -s health
+npx @gfdelarue/testkit --type int,e2e,dal -s dal:queries
+
+# Exact file
+npx @gfdelarue/testkit --type int --file __testkit__/health.int.testkit.ts
+
+# Temporarily ignore repo-declared skip rules
+npx @gfdelarue/testkit --ignore-skip-rules --file __testkit__/billing.int.testkit.ts
+
+# Lifecycle
+npx @gfdelarue/testkit status
+npx @gfdelarue/testkit lint
+npx @gfdelarue/testkit destroy
+npx @gfdelarue/testkit cleanup
+npx @gfdelarue/testkit cleanup --resources --dry-run
+
+
+# Diagnostics: refresh or verify the source schema cache
+npx @gfdelarue/testkit db schema refresh --service api
+npx @gfdelarue/testkit db schema verify --service api
+```
+
+Database cleanup is scoped to the selected product. `cleanup --resources --global`
+is retired; use `--dir <product> cleanup --resources` for each product. After
+deleting a product directory, remove its orphaned containers and volumes
+explicitly through Docker.
+
+Use `execution.taskConcurrency` in config and `--task-concurrency` on the CLI.
+`execution.workers`, `presets.nodeNext({ workers })` and `--workers` remain
+deprecated aliases through 0.2.x and will be removed in 0.3.0. When both names are
+provided in config or on the CLI, their values must match. The CLI value overrides
+config after those checks. See the [execution settings](testkit-config-schema.md#execution).
+
+Batch `run` output stays intentionally short: one line per completed file,
+concise failure blocks, and a final summary. Service logs, captured runtime
+output, and emitted artifacts are persisted under `.testkit/results/`.
+
+Testkit produces an immutable local run bundle and, when requested, a Cloud
+publication receipt. `testkit run` prints `Run ID: <id>` after sealing in text
+output and returns `runId` in `--json` output, for passed and failed runs.
+Publish that exact evidence with `testkit cloud publish --run <id>`. Omitting
+`--run` selects the newest bundle for interactive use. Historical summaries
+and trends belong to Testkit Cloud.
+
+For CI, preserve the run exit status and publish the ID from that invocation:
+
+```sh
+run_exit=0
+testkit run --json > testkit-run.json || run_exit=$?
+run_id=$(node -e 'const r = JSON.parse(require("node:fs").readFileSync("testkit-run.json", "utf8")); if (typeof r.runId !== "string" || !r.runId) process.exit(1); process.stdout.write(r.runId)') || {
+  if [ "$run_exit" -ne 0 ]; then exit "$run_exit"; fi
+  exit 1
+}
+publish_exit=0
+testkit cloud publish --run "$run_id" || publish_exit=$?
+if [ "$publish_exit" -ne 0 ]; then
+  echo "Cloud publication failed with exit $publish_exit" >&2
+fi
+exit "$run_exit"
+```
+
+Set `TESTKIT_CLOUD_PROJECT_ID` and `TESTKIT_CLOUD_TOKEN` in the CI environment.
+`TESTKIT_CLOUD_URL`, or the `--url` flag on `cloud publish`, `cloud apply`, and
+`cloud snapshot`, changes the Testkit Cloud origin. The bearer token and the
+run bundle are sent only to that origin, so protect the variable like the
+token itself. Leave it unset to use `https://testkit.cloud`.
+A failed run can still publish a valid sealed bundle. Publication reports its
+own exit status and does not change the test outcome. This recipe reports a
+publication failure and retains the run status as the job status. See the
+[Cloud boundary](docs/testkit-cloud-boundary.md) and
+[breaking-change migration](https://github.com/gfdelarue/testkit/blob/master/docs/release-notes.md).
+
+Test execution also maintains a scheduler cache at `.testkit/timings.json`.
+Completed file-level task durations are used to rank future runs with a
+longest-estimated-duration-first policy, so slow files start earlier when
+workers are available. Run artifacts include compact scheduler metadata under
+`planning` so ordering decisions are inspectable.
+
+## Tooling Adapters
+
+`testkit` also ships tool-specific config helpers so consumer repos do not need
+repo-local runtime policy files just to cooperate with managed runs.
+
+```ts
+// drizzle.config.ts
+import { defineConfig } from "@gfdelarue/testkit/drizzle";
+
+export default defineConfig({
+  schema: "./src/db/schema/index.ts",
+  out: "./src/db/migrations",
+  dialect: "postgresql",
+  dbCredentials: {
+    url: process.env.DATABASE_URL!,
+  },
+});
+```
+
+```ts
+// vitest.config.ts
+import { defineConfig } from "@gfdelarue/testkit/tooling/vitest";
+
+export default defineConfig({
+  test: {
+    include: ["src/**/*.test.ts"],
+  },
+});
+```
+
+```ts
+// playwright.config.ts
+import { defineTestkitConfig, devices } from "@gfdelarue/testkit/playwright";
+
+export default defineTestkitConfig(
+  {
+    testDir: "./__testkit__",
+    projects: [{ name: "chromium", use: { ...devices["Desktop Chrome"] } }],
+  },
+  {
+    dotenvFiles: [".env.local"],
+  },
+);
+```
+
+Managed UI suites use `@gfdelarue/testkit/playwright` for Testkit's Playwright
+helpers and re-exported authoring API. They may also import exact public exports
+from the managed `@playwright/test`, `playwright`, and `playwright-core`
+packages. Testkit pins both runtime resolution and type resolution, so
+product-local Playwright installs cannot split the test registry or supply a
+different compile-time API.
+
+For scripts and app-runtime code, `@gfdelarue/testkit/env` provides shared helpers
+for managed runtime detection, dotenv loading, and local-database safety:
+
+```ts
+import { assertLocalDatabaseUrl, loadDotenvFiles, shouldLoadDotenv } from "@gfdelarue/testkit/env";
+
+if (shouldLoadDotenv()) {
+  loadDotenvFiles({ files: [".env", ".env.local"] });
+}
+
+assertLocalDatabaseUrl(process.env, "seed.ts");
+```
+
+## Setup
+
+See the [configuration schema](testkit-config-schema.md) for the complete,
+strict `testkit.config.ts` contract.
+
+Create `testkit.config.ts` at repo root:
+
+```ts
+import { app, database, defineConfig, toolchain } from "@gfdelarue/testkit/config";
+
+export default defineConfig({
+  execution: {
+    taskConcurrency: 8,
+    fileTimeoutSeconds: 60,
+  },
+  lint: {
+    rules: {
+      unitTestBoundaries: true,
+    },
+  },
+  fingerprints: {
+    exclude: ["next-env.d.ts"],
+  },
+  toolchains: {
+    frontendNode: toolchain.node({
+      cwd: "frontend",
+      detect: "auto",
+      install: "download",
+    }),
+  },
+  services: {
+    api: app.node({
+      cwd: ".",
+      entry: "src/index.ts",
+      port: 3004,
+      envFiles: [".env.testkit"],
+      database: database.postgres({
+        runtimeConnections: 20,
+        sourceSchema: database.schema.fromEnv("PRODUCTION_DATABASE_URL"),
+        template: {
+          inputs: ["db/schema.sql", "scripts/seed.ts"],
+          migrate: [{ kind: "sql-file", path: "db/schema.sql" }],
+          seed: [{ kind: "command", run: "npm run db:seed" }],
+          verify: [{ kind: "module", target: "src/testkit/verify-seed.ts#verifySeed" }],
+        },
+      }),
+      runtime: {
+        instances: 1,
+        maxConcurrentTasks: 4,
+      },
+    }),
+    frontend: app.next({
+      cwd: "frontend",
+      mode: "start",
+      port: 3000,
+      dependsOn: ["api"],
+      envFiles: ["frontend/.env.testkit"],
+      env: {
+        values: {
+          NEXT_PUBLIC_API_URL: "{baseUrl:api}",
+        },
+      },
+      runtime: {
+        instances: 1,
+        maxConcurrentTasks: 2,
+        toolchain: "frontendNode",
+      },
+    }),
+  },
+});
+```
+
+Every test file declares its execution engine and any file-local scheduling
+metadata next to the test:
+
+```ts
+import { defineFile } from "@gfdelarue/testkit";
+
+export const testkit = defineFile({
+  engine: "k6",
+  skip: "Billing is currently unavailable locally",
+  locks: ["global-worker-loop"],
+});
+```
+
+The engine value is a required literal. The supported public authoring paths
+are `@gfdelarue/testkit/k6`, `@gfdelarue/testkit/node-vitest/data`,
+`@gfdelarue/testkit/node-vitest/contract`, and `@gfdelarue/testkit/playwright`.
+Node/Vitest and Playwright suites may also import exact public APIs directly
+from Testkit's managed `vitest`, `@playwright/test`, `playwright`, and
+`playwright-core` packages.
+There are no legacy aliases or implicit-runner compatibility paths.
+
+`testkit.config.ts` is optional for simple repos, but it is the primary escape hatch
+for:
+
+- worker count and per-file runtime budget
+- per-file wall clock timeout budget
+- multi-service graphs
+- local runtime instance counts
+- per-runtime concurrent task caps
+- repo-managed Node toolchains for prepare/start commands
+- one-time runtime preparation steps for stable shared servers
+- local DB binding configuration
+- source-backed schema verification
+- template database migrate / seed / verify stages
+- explicit per-file or per-suite locks
+- named HTTP suite profiles
+- repo-level fingerprint include/exclude policy
+- repo-declared suite/file skip policies with explicit reasons
+
+`runtime.prepare` is the generic build-once hook for shared runtimes. It runs
+once per runtime generation before local services start, fingerprints declared
+inputs, and writes cache state under the service runtime directory. This is the
+right way to move expensive browser targets from `next dev` / watch mode to
+stable build-and-start flows.
+
+Setup steps (`runtime.prepare`, database `template`, and other Testkit-run
+Node commands) inherit `NODE_OPTIONS`. `TESTKIT_SETUP_NODE_OPTIONS` appends
+extra flags to `NODE_OPTIONS` for those steps only, for example a larger heap
+for a heavy migration. Anything preloaded through it runs inside your product's
+setup processes, so set it deliberately.
+
+Testkit-owned Docker resources are labelled at creation time so they can be
+found again even if product-local `.testkit` state is deleted. `testkit destroy`
+removes labelled resources for the current product after service state is
+destroyed. `testkit cleanup --resources` cleans stale resources for the current
+product. Both commands filter Docker inventory by the product ownership label
+and preserve other products, including idle caches. Host-local Postgres data
+lives in a Testkit-labelled named Docker
+volume instead of an image-declared anonymous volume, so cleanup can account for
+and remove it safely. Each created volume receives a never-reused random
+generation in both its name and labels; cleanup re-inspects the complete
+generation immediately before Docker's name-based removal. Docker has no
+immutable volume ID or conditional-delete API, so direct Docker-daemon
+principals are part of the host trust boundary and must not replace that
+unguessable name during the final removal call. Testkit serializes database
+mutations for the same product; independent products can proceed concurrently.
+Use `cleanup --resources --dry-run` to preview the selected product's cleanup.
+
+`database.template` is the database-side equivalent for reusable template DB
+state. When `database.sourceSchema` is configured, Testkit treats the configured
+source database as the schema source of truth. A normal `testkit run` resolves a
+commit-aware source schema cache under
+`.testkit/db/<service>/source-schemas/`, applies that cached schema to the local
+template DB, runs local template setup, and verifies that the replayed local
+schema still matches the source dump. If local replay differs, Testkit refreshes
+from the source once for the current cache key and retries. If it still differs,
+the run fails with schema diagnostics under `.testkit/results/schema`.
+
+Source schema cache keys are derived automatically from repo state:
+
+- clean git worktrees use `commits/<sha>`
+- dirty git worktrees use `dirty/<sha>-<fingerprint>`
+- non-git directories use `nogit/<fingerprint>`
+
+Dirty worktree fingerprints compare the complete index to `HEAD`, then compare
+the physical worktree bytes, node types, and executable modes to the index.
+This intentionally detects changes hidden by Git stat flags, sparse checkout,
+or clean/smudge and EOL normalization. For untracked files, repository
+`.gitignore` files are respected; host-global excludes, `core.excludesFile`, and
+`.git/info/exclude` do not alter Testkit's cache identity.
+
+Fingerprint Git commands disable replacement objects and optional locks and do
+not read system or user Git configuration. Git path records are decoded from
+their byte-preserving quoted form and must be valid UTF-8. Uninitialized
+gitlinks and special filesystem nodes fail explicitly instead of being assigned
+an incomplete or ambiguous identity. Testkit reads the complete HEAD, index,
+untracked, submodule, and physical snapshot twice and fails if the observations
+differ.
+
+Physical reads open regular files by descriptor and verify every pathname
+component before and after use; on POSIX, ancestor directory descriptors remain
+open during the read. Node does not expose a portable `openat`/Windows reparse-
+handle traversal API, so this detects ordinary replacement races but is not a
+security boundary against a same-user adversary able to perform a perfectly
+timed replace-and-restore ABA attack. Run Testkit against source trees that are
+not concurrently controlled by an untrusted local process.
+
+Testkit-owned outputs are always excluded from fingerprints: `.testkit/`,
+and `.next-testkit/`. `testkit.config.ts` is normal repo
+configuration and is not excluded automatically. For user-managed generated
+files, add repo-level `fingerprints.exclude` patterns;
+`fingerprints.include` can opt specific paths back in below an excluded
+directory, including through wildcard directory patterns. Fingerprint patterns
+use `/` as the only separator; a single backslash is rejected when the config
+loads. A doubled backslash addresses a literal POSIX backslash in a filename
+(for example, `"fixtures\\\\literal.txt"` in `testkit.config.ts` source).
+
+Branch names and worktree paths are recorded as metadata but do not affect clean
+commit cache keys, so branch renames and clean worktrees at the same commit
+reuse the same source schema. Dirty worktrees are isolated by content
+fingerprint so local experiments cannot overwrite a clean commit baseline.
+
+Template setup executes in three explicit phases:
+
+- `migrate`
+- `seed`
+- `verify`
+
+Schema drift is checked after each successful `migrate` and `seed` step, and
+`verify` only runs once local replay matches the source schema. Source refreshes
+use Dockerized `pg_dump --schema-only --no-owner --no-privileges`, so
+seed/reference data is never written into the baseline and host `pg_dump`
+installations are not used. Keep schema-changing setup in its own step where
+possible; a single command that changes schema and then fails before exiting
+cannot be refreshed at the midpoint.
+
+Source schema refreshes are intentionally single-connection and pooler-safe.
+If a Neon pooled source URL is configured, Testkit rewrites it to the matching
+direct Neon endpoint before running Dockerized `pg_dump` and records the
+original/resolved host classifications beside the resolved cache entry. Unknown
+PgBouncer/pooler URLs fail closed; configure a direct source URL for those
+providers. Concurrent refreshes for the same service and cache key are
+serialized with a cache-local lock so multiple Testkit processes do not
+stampede the source database. Testkit also maintains
+`.testkit/db/<service>/source-schemas/index.json` and prunes old inactive cache
+entries automatically.
+
+For most repos, prefer declarative step objects directly inside
+`database.postgres({ template: ... })` and `runtime.prepare.steps`.
+The supported shapes are:
+
+- `{ kind: "command", run: "..." }`
+- `{ kind: "sql-file", path: "..." }`
+- `{ kind: "module", target: "file.ts#exportName" }`
+
+Local files and ordinary external packages remain valid module targets.
+Testkit-owned `@gfdelarue/testkit` targets are validated against Testkit's exact
+configured-step module registry while config is loaded, so a misspelled or
+private Testkit subpath fails before preparation or database setup starts.
+
+`runtime.toolchain` is the first-class way to make those prepare/start commands
+run under the correct Node toolchain instead of whatever `node`/`npm` happened
+to launch `testkit`. Node toolchains support:
+
+- host verification mode: `install: "require-host"`
+- cached repo-local provisioning mode: `install: "download"`
+- auto-detection from:
+  - `package.json#volta.node`
+  - `.nvmrc`
+  - `.node-version`
+  - `.tool-versions` (`nodejs`)
+  - `package.json#engines.node`
+  - `package.json#volta.npm`
+  - `package.json#packageManager`
+  - `package.json#engines.npm`
+
+Downloaded toolchains resolve only through Testkit's target-specific trusted
+Node release manifest. The archive URL, compressed size, SHA-256, archive
+shape, unpacked size, and complete installed-tree digest are fixed before any
+download. An unlisted version or platform fails closed. The verified cache is
+kept under `.testkit/_toolchains/node`; external cache redirects, linked cache
+ancestors, modified installed content, and escaping executable links are
+rejected. `runtime.toolchain.cwd` is always a normalized product-relative path
+and cannot contain a parent-directory segment. The trusted official Linux Node
+archives are glibc builds; `install: "download"` fails before fetch or cache
+mutation on musl instead of pretending those binaries are compatible.
+
+Example:
+
+```ts
+toolchains: {
+  frontendNode: toolchain.node({
+    cwd: "frontend",
+    detect: "auto",
+    install: "download",
+  }),
+},
+services: {
+  frontend: app.next({
+    cwd: "frontend",
+    port: 3000,
+    runtime: {
+      toolchain: "frontendNode",
+    },
+  }),
+}
+```
+
+Every terminal run is sealed under `.testkit/results/runs/<run-id>/` with a
+canonical `run.json`, `provenance.json`, `manifest.json`, and the run's exact
+artifact/log/setup files. Payloads are SHA-256 addressed under
+`.testkit/results/objects/sha256`, so repeated bytes are deduplicated without
+removing their logical per-run paths. `.testkit/results/latest.json` remains a
+convenience projection; it is not the historical authority.
+
+Use the artifact archive config to opt into pruning or atomic publication to
+an external directory. Local retention is indefinite by default:
+
+```ts
+export default defineConfig({
+  artifacts: {
+    retention: {
+      maxRuns: 50,
+      maxAgeDays: 30,
+      preserveSuites: ["ui:performance"],
+    },
+    publication: {
+      directory: "/var/lib/testkit-run-archive",
+      required: true,
+    },
+  },
+  services: {
+    // ...
+  },
+});
+```
+
+Retention decisions and publication receipts are recorded under
+`.testkit/results/retention` and `.testkit/results/publications`. A required
+publication failure fails the command but leaves the verified local bundle in
+place. Discover and verify retained evidence with:
+
+```sh
+testkit artifacts --list --json
+testkit artifacts --run <run-id> --verify --json
+testkit artifacts --run <run-id> path/to/test.testkit.ts --json
+```
+
+Programmatic consumers can import `listRunBundles`, `loadRunBundleArtifact`,
+`loadRunBundleManifest`, and `verifyRunBundle` from
+`@gfdelarue/testkit/workspace`.
+
+The run bundle is also the boundary for Testkit Cloud. The local runner never
+reads regression state or mutates GitHub. See
+[`docs/testkit-cloud-boundary.md`](docs/testkit-cloud-boundary.md) for ownership,
+ingestion, and migration rules.
+
+Use `testkit cloud snapshot` to read versioned Cloud regression state. Apply
+reviewed mutations from a file with `testkit cloud apply --file commands.json`;
+the file must contain a schema version, idempotency key, expected snapshot
+version, and commands.
+
+## Authoring
+
+### Node setup beside a test
+
+A test such as `tests/webhook.int.testkit.ts` can have a sibling
+`tests/webhook.int.testkit.setup.mjs`. Testkit runs its named `setup` export,
+or its default export, before starting the test engine:
+
+```js
+// tests/webhook.int.testkit.setup.mjs
+export async function setup({ sidecars }) {
+  const capture = await sidecars.httpCapture({ path: "/webhook" });
+  return {
+    data: { webhookUrl: capture.url, requestsUrl: capture.requestsUrl },
+  };
+}
+```
+
+Setup runs in a Node worker thread with its own JavaScript state. It can use
+Node APIs, prepare external resources, and start HTTP capture listeners.
+The context contains `deadlineMs`, `signal`, `env`, `leaseDir`, `productDir`,
+`serviceName`, `testFile`, and `sidecars`. The worker shares the host process's
+permissions. It is not a sandbox for untrusted code.
+
+Return plain JSON, or `{ data, cleanup }` with an optional cleanup function.
+Testkit passes the data to the engine as `setupData` and records it with the
+setup module path and sidecar handles. Bare `null` or `undefined`, and a
+cleanup-only result, give empty data. `{ data: null }` preserves `null`.
+Data must encode to at most 1 MiB of JSON. Invalid data reports its JSON path,
+such as `$.items[1]`, without invoking property getters.
+
+Setup shares the task's absolute deadline. On failure or cancellation, Testkit
+runs registered cleanup and closes its sidecars. Once setup succeeds, those
+resources remain available until task cleanup. Cleanup callbacks run in reverse
+registration order. The runner uses the task timeout for cleanup, capped at
+60 seconds. A hung worker is terminated so it cannot keep its listeners open.
+
+Thrown setup errors, deadline expiry, and invalid data fail the task with the
+setup module path. Consumer cleanup errors fail the task; Testkit cleanup or
+worker failures count as infrastructure failures. Cleanup remains idempotent.
+HTTP capture retains at most 100 requests by default, configurable up to 1,000.
+Each body defaults to 1 MiB, configurable up to 8 MiB, and retained bodies total
+at most 8 MiB per listener. A setup can create up to 16 listeners, each with at
+most 256 connections.
+
+Discovery tries sibling `.setup.mjs`, sibling `.setup.js`, then the matching
+`.mjs` and `.js` files in `__setup__`, in that order. For the example above,
+the fallback is `tests/__setup__/webhook.int.testkit.mjs`.
+
+### HTTP suites
+
+```ts
+import { defineFile } from "@gfdelarue/testkit";
+import { defineHttpSuite, expect } from "@gfdelarue/testkit/k6";
+
+export const testkit = defineFile({ engine: "k6" });
+
+const suite = defineHttpSuite(({ rawReq }) => {
+  const response = rawReq.get("/health");
+  expect.status(response, 200, "health returns 200");
+});
+
+export const options = suite.options;
+export const setup = suite.setup;
+export default suite.exec;
+```
+
+Testkit helper suites are authored as native-shaped k6 modules: re-export the
+helper's `options` and `setup`, and default-export its `exec` function. Testkit
+does not inspect source text or invent k6 lifecycle exports.
+
+Suite-owned authentication lives in the k6 module. `auth.setup` receives the
+resolved env and a raw HTTP client, returns a session bundle of named actors,
+and `auth.headers` turns the bundle plus the requested actor into headers:
+
+```ts
+import { defineFile } from "@gfdelarue/testkit";
+import { defineHttpSuite } from "@gfdelarue/testkit/k6";
+
+import { organizationHeaders, sessionAuth } from "../helpers/session.js";
+
+export const testkit = defineFile({ engine: "k6" });
+
+const suite = defineHttpSuite(
+  {
+    auth: sessionAuth(["reviewer", "outsider"]),
+    rawHeaders: organizationHeaders,
+  },
+  ({ actor, actors, req }) => {
+    req.get("/api/auth/session");
+    req("GET", "/api/auth/session");
+    actor?.req.get("/api/auth/session");
+    req.as("outsider").get("/api/auth/session");
+    actors.get("reviewer").rawReq.get("/api/auth/session");
+  },
+);
+
+export const options = suite.options;
+export const setup = suite.setup;
+export default suite.exec;
+```
+
+```ts
+// __testkit__/helpers/session.ts
+import type { AuthAdapter, AuthSessionBundle } from "@gfdelarue/testkit/k6";
+
+export function sessionAuth(actorNames: string[]): AuthAdapter<AuthSessionBundle> {
+  return {
+    setup({ env, http }) {
+      const actors = Object.fromEntries(
+        actorNames.map((actorName, actorIndex) => [
+          actorName,
+          { actorIndex, actorName, session: login(http, env.BASE, actorName) },
+        ]),
+      );
+      return { actors, primaryActor: actorNames[0] };
+    },
+    headers(bundle, { actor }) {
+      const session = bundle?.actors[actor ?? bundle.primaryActor ?? ""]?.session;
+      return session ? { Authorization: `Bearer ${session.jwt}` } : {};
+    },
+  };
+}
+```
+
+Named HTTP profiles in `testkit.config.ts` stay JSON-only for k6 suites, because
+the orchestrator serializes them across the Node/k6 boundary. Node/Vitest data
+profiles are loaded in-process and may define `auth` and header functions
+directly. See the [config reference](testkit-config-schema.md#profileshttp).
+
+DAL suites:
+
+```ts
+import { defineFile } from "@gfdelarue/testkit";
+import { defineDalFixtures, defineDalSuite } from "@gfdelarue/testkit/k6";
+
+export const testkit = defineFile({ engine: "k6" });
+
+const fixtures = defineDalFixtures(({ db, fixtureScope }) => ({
+  widget() {
+    return fixtureScope.seed("widget", "primary", { name: "Primary Widget" }, () => {
+      const widgetId = fixtureScope.id("widget");
+      db.exec(`
+        INSERT INTO widgets (id, name)
+        VALUES ('${widgetId}', 'Primary Widget')
+      `);
+      return { widgetId };
+    });
+  },
+}));
+
+const suite = defineDalSuite({ fixtures }, ({ db, fixtureScope, fixtures }) => {
+  const widget = fixtures.widget();
+  db.query(`select id from widgets where id = '${widget.widgetId}'`);
+  fixtureScope.records();
+});
+
+export const options = suite.options;
+export const setup = suite.setup;
+export default suite.exec;
+```
+
+DAL files for the same service database are serialized by default to avoid
+exhausting Postgres connection limits during highly parallel runs.
+Runtime services also reserve a configurable Postgres connection budget for each
+active runtime slot. Use `database.postgres({ runtimeConnections })` for the
+service pool size and `database.postgres({ maxConnections,
+reservedConnections })` for the backing Postgres capacity.
+
+`defineDalFixtures(...)` is the package-owned DAL seeding model. It gives every
+suite a deterministic `fixtureScope` with:
+
+- `fixtureScope.id(label)` / `uuid(label)`
+- `fixtureScope.slug(label)`
+- `fixtureScope.email(label)`
+- `fixtureScope.string(label, options)`
+- `fixtureScope.token(label)`
+- `fixtureScope.seed(kind, key, signature, create)`
+- `fixtureScope.records()`
+
+`testkit` enforces strict fixture behavior:
+
+- one logical fixture per `kind + key`
+- identical reseeds reuse the same seeded value
+- conflicting reseeds fail immediately
+- dependency cycles fail immediately
+- seeded fixture records are persisted as a `testkit.dal-fixtures` artifact
+
+The first-class k6 authoring namespace:
+
+```ts
+import { checks, expect, parse, waitFor } from "@gfdelarue/testkit/k6";
+
+const suite = defineHttpSuite(({ actor, rawReq, req }) => {
+  const me = req.get("/api/v1/me");
+  expect.status(me, 200, "authenticated session loads");
+  expect.json(me, (body) => typeof body.data?.id === "string", "response includes user id");
+
+  checks.authGate(rawReq, "sessions", {
+    get: ["/api/v1/sessions"],
+  });
+
+  actor?.rawHeaders();
+
+  const upload = req.multipart.post("/api/v1/uploads", {
+    fields: { name: "fixture" },
+    files: [
+      {
+        field: "file",
+        data: "hello",
+        filename: "fixture.txt",
+        contentType: "text/plain",
+      },
+    ],
+  });
+
+  expect.status(upload, 201, "upload is accepted");
+  parse.safeJson(upload);
+});
+```
+
+Use native k6 imports for checks, groups, metrics, and low-level HTTP:
+
+```ts
+import { check, group } from "k6";
+import http from "k6/http";
+import { Rate, Trend } from "k6/metrics";
+```
+
+The [k6 authoring guide](https://github.com/gfdelarue/testkit/blob/master/docs/k6-authoring-layer.md)
+describes the optional evidence recorder and migration from retired Testkit
+primitive, metric, option, and assertion aliases.
+
+`waitFor()` consumes the file budget configured by `execution.fileTimeoutSeconds`.
+Consumers should not set local timeout values in test files.
+
+```ts
+import { parse, waitFor } from "@gfdelarue/testkit/k6";
+
+const response = waitFor(
+  () => req.get("/api/v1/jobs/123"),
+  (res) => parse.json(res).data?.status === "completed",
+  { description: "job 123 to complete" },
+);
+```
+
+## Discovery
+
+`testkit` discovers suites from `__testkit__/` directories.
+
+Place each `__testkit__/` directory as a peer of the thing being tested. The
+target is the smallest owned directory or package surface that honestly contains
+the behavior under test. This is a single target-relative rule, not a list of
+allowed or forbidden locations.
+
+Examples:
+
+- tests for `src/api/routes/*` live in `src/api/routes/__testkit__/*`
+- tests for `src/db/*` live in `src/db/__testkit__/*`
+- tests for `frontend/src/components/layout/*` live in
+  `frontend/src/components/layout/__testkit__/*`
+- tests for package-root behavior such as `package.json`, root scripts, release
+  shape, or the whole package contract live in `__testkit__/*`
+
+Do not put tests in a higher-level `__testkit__/` merely because it is
+convenient. For example, a repository-root `__testkit__/` is appropriate for
+repository/package-level behavior, but it is a dumping ground if it contains
+tests for `src/api/routes/*`, `src/db/*`, or a specific frontend component tree.
+
+Do not create child directories inside a `__testkit__/` directory. The
+`__testkit__/` directory already names the owner boundary; additional grouping
+folders create a second, non-production hierarchy that weakens ownership. Use
+clear flat filenames instead, and colocate another `__testkit__/` at a lower
+production owner when a narrower owner exists.
+
+Concrete layouts:
+
+- `src/api/routes/__testkit__/auth-me.int.testkit.ts`
+- `src/db/__testkit__/sessions-count-type.dal.testkit.ts`
+- `frontend/src/components/layout/__testkit__/navigation.ui.testkit.ts`
+- `src/internal/handler/repos/__testkit__/crud.int.testkit.ts`
+
+`testkit` uses these suffixes automatically:
+
+- `*.int.testkit.ts`
+- `*.e2e.testkit.ts`
+- `*.contract.testkit.ts`
+- `*.data.testkit.ts`
+- `*.dal.testkit.ts`
+- `*.load.testkit.ts`
+- `*.ui.testkit.ts`
+
+The suffix never chooses an engine. Each discovered file must also contain one
+root-package `defineFile(...)` declaration with a supported literal engine.
+Unsupported pairs fail discovery; for example, a `*.ui.testkit.ts` file cannot
+declare `engine: "k6"`.
+
+See [docs/test-types.md](docs/test-types.md) for the canonical type model.
+`*.contract.testkit.ts` files are Node-backed service/runtime contract suites.
+They use `contractScenario(...)` from
+`@gfdelarue/testkit/node-vitest/contract`, declare `engine: "node-vitest"`, and run
+through the `node-vitest-contract` profile for checks that need Node APIs but
+are not data-flow evidence suites. They are native Vitest suites:
+`contractScenario(...)` is the managed-context convenience, while direct public
+Vitest runner APIs remain valid for behavior such as `test.skip(...)` and
+`test.todo(...)`.
+
+`*.data.testkit.ts` files are Testkit data-flow/data-contract suites, not raw
+Vitest files with a Testkit suffix. They must use `dataScenario(...)` from
+`@gfdelarue/testkit/node-vitest/data`, declare `engine: "node-vitest"`, and record
+structured data-flow evidence through the data context. Supporting
+`count(...)`/`event(...)` calls are not enough by themselves; model actors,
+systems, records, transitions, external calls, payloads, budgets, cache/LLM/RPC
+usage, or reconciliation evidence. Data suites run through the
+`node-vitest-data` profile. Use `contract` for plain Node assertions and
+Node-side service or protocol contracts, `int` or `e2e` for k6-backed HTTP
+integration, and `ui` when a browser is part of the assertion.
+Data suites use Vitest directly for assertions, hooks, and mocking.
+`dataScenario(...)` owns case registration: `testkit lint` reports native
+registration imports in data suites, and the data runner rejects any case that
+was not registered through the DSL.
+
+Ownership is inferred from:
+
+- the deepest matching service root from `services.<name>.local.cwd`
+- optional `services.<name>.discovery.roots` overrides for shared-root edge cases
+
+Suite names are inferred from the colocated path:
+
+- `auth/__testkit__/*.int.testkit.ts` => `auth`
+- `routes/__testkit__/auth-me.int.testkit.ts` => `routes`
+
+Discovery is also a first-class CLI/API surface:
+
+- `testkit discover`
+  - human-first compact output with service -> type -> suite -> file hierarchy
+- `testkit discover --output-mode verbose`
+  - explicit paths, IDs, locks, dependencies, and skip reasons
+- `testkit discover --json`
+  - machine-readable output with stable enums, canonical paths, and summary data
+- `testkit discover --output .testkit/discovery.json`
+  - writes the same machine-readable JSON document to a file artifact
+
+Compact mode prefers derived human labels such as `Agent Configs Auth Gate`
+instead of printing long file paths as the primary row label. Exact paths remain
+available in verbose and JSON output.
+
+The public API is exported from `@gfdelarue/testkit/workspace`:
+
+```ts
+import { discoverTests } from "@gfdelarue/testkit/workspace";
+
+const result = await discoverTests({
+  dir: process.cwd(),
+  runnableOnly: true,
+  diagnostics: "report",
+});
+```
+
+JSON and file output are machine-first. Each discovered file carries a stable
+identifier plus canonical metadata such as:
+
+- `id`
+- `path`
+- `service`
+- `suiteName`
+- `testType`
+- `engineId`
+- `profileId`
+- `contract`
+- `skipped`
+- `skipReason`
+- `locks`
+- `dependsOn`
+- `displayName`
+
+For cross-run counts, last outcomes, durations, or trends, use Testkit Cloud.
+See the [status and history migration](https://github.com/gfdelarue/testkit/blob/master/docs/release-notes.md#local-status-and-history-removal)
+for the removed discovery fields and consumer migration.
+
+## Local Databases
+
+`@gfdelarue/testkit` provisions Docker-managed local Postgres automatically for
+services that define `database: database.postgres(...)`.
+
+Docker is also the required Postgres client runtime for Testkit-owned database
+operations. Testkit runs `psql` and `pg_dump` inside the configured Postgres
+client image instead of using host-installed binaries, which keeps local runs
+consistent across Linux, macOS, and native Windows. Override the client image
+with `TESTKIT_POSTGRES_CLIENT_IMAGE` when needed.
+
+Client containers for a Testkit-owned database join that database container's
+network namespace (`--network container:<id>`) and connect over loopback. The
+route does not depend on the Docker daemon sharing the Testkit process's host
+network, so template steps, schema snapshots, and `database.steps` helpers work
+the same under Docker Desktop, rootless Docker, and Docker-in-Docker. Clients
+for external databases, such as `sourceSchema` URLs, still connect from the
+platform's host address.
+
+- normal test runs always use transient host-managed Postgres
+- template databases are cached
+- runtime databases are cloned from templates when binding is `per-runtime`
+- shared databases are reused when binding is `shared`
+- source schema caches are refreshed only from the configured source database
+- clean commits, dirty worktrees, and non-git directories get separate source
+  schema cache entries automatically
+- template fingerprints are derived automatically from env files, source schema
+  cache, migrate/seed config, and repo contents
+
+`db schema refresh` forces a source database dump into the `.testkit` source
+schema cache. `db schema verify` prepares local templates and verifies local
+replay against the cached/refreshed source schema. `--skip-schema-source-verify`
+is available as a narrow escape hatch when users need to run tests while schema
+verification is temporarily blocked.
+
+## Development Tests
+
+```bash
+npm test
+npm run test:unit
+npm run test:integration
+npm run test:system
+npm run test:live
+npm run test:integration -- fixture-package
+```
+
+The four lanes use Vitest directly. Unit tests (source and pure tooling) run in
+parallel; integration and system files run serially; live tests are explicitly
+opt-in and serial. Pass ordinary Vitest filters after npm's `--` separator.
+`npm test` runs the first three lanes and never collects live cases.
+
+Live tests cover credentials, external services, network-acquired runtimes,
+alternate toolchains, and Docker/platform matrices. An explicitly selected live
+test fails clearly when its prerequisites are unavailable; it does not silently
+skip.
+
+Repository commands use npm's native timing support, enabled project-wide by
+the tracked `.npmrc`. Build, lint, and release modes support `help`; release
+provides `bump` and `check` modes.
+Every merge to `master` publishes the next patch version. The existing
+pre-merge CI checks provide verification. The
+[release workflow](https://github.com/gfdelarue/testkit/blob/master/docs/release-gates.md) tags the merge and publishes through npm
+trusted publishing; merge a `package.json` bump to start a new minor or major.
+
+The static audit retains package/lock/runtime policy, engine dependency/import
+intent, retired engine/product references, minimal repository traversal, and
+the dependency direction between top-level `src/` modules. The edge table in
+`scripts/lint/audits/import-direction.ts` mirrors the import graph exactly: an
+import outside the table fails lint, and so does an entry that no import
+exercises. Module direction is the one structural rule lint enforces; other
+implementation-shape audits stay out of lint.
+
+JavaScript modules under `test/fixtures/**` describe fixture behavior and keep
+their fixture-owned paths. Outside fixtures, only `bin/testkit.mjs` and the
+documented raw Node process probe remain `.mjs`; the probe is a program, not a
+Vitest suite.
+
+GitHub Actions run a narrow installed Node/Vitest consumer smoke on dedicated
+Orthrus Ubuntu x64 and Windows x64 self-hosted runners for pull requests from
+this repository and manual dispatches. Fork pull requests skip this workflow's
+jobs. Local `npm test`,
+`npm run test:unit`, `npm run test:integration`, and `npm run test:system` remain
+the full developer entrypoints. PostgreSQL and Docker compatibility are explicit
+opt-in dispatcher cases. The smoke workflow is not five-platform release proof.