@gfdelarue/testkit 0.4.5 → 0.5.0

package/docs/bug-records.md

@@ -45,10 +45,10 @@ Two overlapping refund requests for one order both pay out.
 ## Reproduce
 
 ```sh
-npx testkit run --file src/checkout/__testkit__/refund.int.testkit.ts
+npx testkit run --file src/checkout/__testkit__/refund.contract.testkit.ts
 ```
 
-Test: `src/checkout/__testkit__/refund.int.testkit.ts`, "a refund pays out once".
+Test: `src/checkout/__testkit__/refund.contract.testkit.ts`, "a refund pays out once".
 Found at `4f0c2d1e9b7a4c3d8e6f5a4b3c2d1e0f9a8b7c6d`.
 Fault: `src/checkout/refunds.ts`, lines 41 to 58.
 
@@ -66,9 +66,9 @@ Fault: `src/checkout/refunds.ts`, lines 41 to 58.
   "escape": "no-test",
   "caughtBy": "campaign",
   "campaign": "refunds-2026-10",
-  "testType": "int",
+  "testType": "contract",
   "test": {
-    "file": "src/checkout/__testkit__/refund.int.testkit.ts",
+    "file": "src/checkout/__testkit__/refund.contract.testkit.ts",
     "title": "a refund pays out once"
   },
   "foundAt": "4f0c2d1e9b7a4c3d8e6f5a4b3c2d1e0f9a8b7c6d",
@@ -86,29 +86,40 @@ Fault: `src/checkout/refunds.ts`, lines 41 to 58.
 
 The reproduce command runs from the repository root. When `testkit.config.ts` is in a subdirectory, the command carries `--dir <directory>` and the `--file` path is relative to that directory. The `Test:` and `Fault:` lines always give paths from the repository root.
 
-A fault on one line reads `line 41`.
+When `testkit bug file` ran the test against [supplied values](../README.md#run-against-a-supplied-environment), the command ends with the `--base-url`, `--endpoint`, `--env-from`, and `--environment-file` flags it was given, as given. They hold no raw secret: `--env-from` names host variables, the environment file stays outside the issue, and `testkit bug file` accepts a URL only without credentials, a query, or a fragment, and an endpoint name only of letters, digits, dot, underscore, and hyphen. It refuses `--database` and `--env`, whose values the command would otherwise carry, and a relative `--environment-file` unless it runs from the repository root. It checks these values, and that every `--env-from` host variable is set, before it contacts the tracker. The command is written for a POSIX shell: flags and simple mappings appear unquoted, and any argument that needs quoting, such as a URL with brackets or a path with a space, is quoted for that shell. The facts block does not change.
 
-An issue for a k6 script that fails as a whole covers the whole file. Its facts block has no `test.title`, and its `Test:` line quotes the `test.title` of the bug record as the finder's description of the script.
+A fault on one line reads `line 41`.
 
 ```markdown
-Test: `src/health/__testkit__/health.int.testkit.ts`, the whole script, recorded as "the health endpoint answers".
+Test: `src/health/__testkit__/health.contract.testkit.ts`, the whole script, recorded as "the health endpoint answers".
 ```
 
-Testkit redacts the secrets it recognises in the title, in every section, and in `test.title`. It does so when it records the bug and again when it files the issue. It recognises these forms.
+Testkit redacts the secrets it recognises in the title, in every section, and in `test.title`. It does so when it records the bug and again when it files the issue. `testkit run` applies the same rules when it stores a run result. Testkit recognises these forms.
 
 - The value of a `KEY=value` assignment whose key names a secret.
 - The password in a URL.
 - The value of an `Authorization`, `Proxy-Authorization`, `Cookie`, or `Set-Cookie` header.
-- A JSON string value whose key names a secret.
+- A JSON string value whose key names a secret, including JSON that is itself inside a JSON string, as in `{"message":"{\"token\":\"…\"}"}`. Testkit reads the key's escapes, so `"\u0074oken"` names a secret too.
+- The value of a single-quoted `'password': '…'` member, in any letter case. It ends at the next apostrophe that no backslash precedes, or with the line.
 
-A key names a secret when it contains `PASSWORD`, `TOKEN`, `SECRET`, `PRIVATE_KEY`, `AUTH`, `COOKIE`, `SESSION`, or `API_KEY` in any letter case. Testkit cannot recognise every secret, so the finder must still keep credentials, tokens, and passwords out of the record.
+A key names a secret when it contains `PASSWORD`, `TOKEN`, `SECRET`, `PRIVATE_KEY`, `AUTH`, `COOKIE`, `SESSION`, or `API_KEY` in any letter case. Still keep credentials, tokens, and passwords out of the record.
 
-An assigned value ends at the first space, with three exceptions.
+An assigned value ends at the first space, with four exceptions.
 
 - A value that starts with `"` or `'` ends at the next such quote on its line. When the line has no such quote, the value ends with the line.
+- A quoted value inside a JSON string (`\"`), or inside JSON that is itself inside a JSON string (`\\\"`), ends at the next quote escaped to the same depth. A quote escaped more deeply belongs to the value. The value ends sooner at a quote escaped less deeply, such as the quote that closes the JSON string, at an escaped line break at its own depth or less, or with the line. A value quoted with an escaped apostrophe, `\'`, works the same way. JSON doesn't escape apostrophes, so each JSON layer only doubles the backslashes (`\\'`). A plain apostrophe inside the value belongs to it.
 - A value that starts with `Basic`, `Bearer`, `Digest`, `Negotiate`, or `Token` and then a space or a tab ends with the line.
 - Under a key that ends in `_URL`, Testkit keeps a value that starts with a scheme, such as `https://`, and redacts the URL's password only.
 
+When `testkit run` stores a result, it redacts JSON one string value at a time. That covers text that is a JSON object, array, or string, and a line whose text from its first `{` or `[` is JSON. Each string value gets the rules above, and so does JSON inside a string. A value that runs to a space or to the end of its line therefore ends with its string, and the record's other values stay as they are. Other text gets the rules as written, and so does JSON in two cases:
+
+- a rule reading the whole line would redact across the line's first `{` or `[`;
+- a secret Testkit resolved from the run's environment, whole or cut short, crosses the JSON's edge or spans more than one string.
+
+A string that holds such a secret in escaped form is redacted as decoded text when the escaped text decodes to the secret, or, for a secret cut short, to the start of one, and redacted whole otherwise. Where these rules and a resolved secret overlap, Testkit redacts every character that either would redact.
+
+Testkit writes `<redacted>` in place of the value. A quoted value keeps its quotes, as in `PASSWORD="<redacted>"`, so redacting the text again changes nothing. A `<redacted>` that the text already holds counts as part of the value, so `TOKEN=<redacted>example-secret` becomes `TOKEN=<redacted>`. A secret resolved from the run's environment can itself contain part of `<redacted>`, or a character Testkit reserves. Such a secret can match text that redaction creates, so where it takes part, `testkit run` redacts the whole line of a log, or the whole JSON string, when it stores the result. When such a secret spans lines, or takes part only across lines, `testkit run` redacts every line between the nearest JSON lines instead, or the whole text when it holds none.
+
 A URL password is the text between `://user:` and the last `@` before the next `/` or space. The user can be empty, as in `redis://:password@cache`. Testkit does not recognise a password that holds a `/`, or a token that stands alone as the user, as in `https://token@host`.
 
 Testkit does not rewrite a path or an area's name. `testkit bug record` rejects a record whose `test.file` or `fault.path` holds text that Testkit redacts. `testkit bug file` stops when the area's name or the reproduce command holds such text.
@@ -214,29 +225,25 @@ An area id matches `^[a-z0-9]+(?:-[a-z0-9]+)*$` and is at most 40 characters lon
 
 ## Known failures
 
-A failed file reports its failures in one of three forms.
+A failed file reports its failures in one of two forms.
 
-| Failure         | When                                                                                                                                                                                              |
-| --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| Named failure   | The engine names the test that failed. One failure for each title.                                                                                                                                |
-| Script failure  | A k6 script fails as a whole: it calls `fail()`, throws while it runs, fails a k6 `check()`, or fails a threshold. One per file.                                                                  |
-| Unnamed failure | The file fails any other way, such as a crash, a timeout, a file that does not load, a Vitest error outside a test, a Playwright error outside a spec, or a k6 script that throws during `setup`. |
+| Failure         | When                                                                                                                                                   |
+| --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
+| Named failure   | The engine names the test that failed. One failure for each title.                                                                                     |
+| Unnamed failure | The file fails any other way, such as a crash, a timeout, a file that does not load, a Vitest error outside a test, a Playwright error outside a spec. |
 
 A named failure's title comes from the engine.
 
-| Engine     | Title                                                                                                                            |
-| ---------- | -------------------------------------------------------------------------------------------------------------------------------- |
-| Vitest     | The names of the `describe` blocks around the test and the test's own name, joined by single spaces.                             |
-| Playwright | The names of the `describe` blocks around the spec and the spec's own title, joined by single spaces.                            |
-| k6         | The label of a failed assertion from `@gfdelarue/testkit/k6`, or the assertion's request and default title when it has no label. |
+| Engine     | Title                                                                                                 |
+| ---------- | ----------------------------------------------------------------------------------------------------- |
+| Vitest     | The names of the `describe` blocks around the test and the test's own name, joined by single spaces.  |
+| Playwright | The names of the `describe` blocks around the spec and the spec's own title, joined by single spaces. |
 
 A Playwright title holds neither the file path nor the project name. A spec that fails in two projects is one named failure.
 
-A k6 assertion on a response has a default title when the script gives it no label. Those assertions are `expect.status`, `expect.statusOneOf`, `expect.notStatus`, `expect.json`, `expect.jsonPath`, `expect.error.shape`, and `expect.error.message`. Testkit names such a failure by its request and the default title, as in `GET /health > status is 201`. Inside a k6 `group`, the group's name comes first, as in `orders > GET /health > status is 201`. Any other assertion with no label keeps its default title, such as `row count is 2`. A label names its failure as written, even when it reads like a default title. Two failures in one file whose titles are equal are one named failure.
-
 When a file reports named failures, `testkit bug file` rejects a bug record whose `test.title` is not one of their titles. Its message lists the titles a bug record can use.
 
-Testkit files no bug under a title that spans lines or holds text it redacts as a secret, so no issue with a `test.title` covers such a failure. `testkit run` redacts some secrets when it stores a run result and writes `<redacted>` in their place. A title that the run result stores with `<redacted>` is such a title. Rename the test, or give the assertion a label, so that its title is one line and holds no such text. [Issue body](#issue-body) lists what Testkit redacts.
+Testkit files no bug under a title that spans lines or holds text it redacts as a secret, so no issue with a `test.title` covers such a failure. `testkit run` redacts those secrets when it stores a run result and writes `<redacted>` in their place. A title that the run result stores with `<redacted>` is such a title. Rename the test, or give the assertion a label, so that its title is one line and holds no such text. [Issue body](#issue-body) lists what Testkit redacts.
 
 Vitest reports an error outside a test when a hook such as `beforeAll` throws, or when code throws after its test has returned. The file then reports one unnamed failure beside its named failures.
 
@@ -250,22 +257,15 @@ A file also reports one unnamed failure beside its named failures when the engin
 - A Playwright config sets `failOnFlakyTests` and a spec passes only on a retry. A spec that passes on a retry is no named failure, so no issue can cover it.
 - A Playwright spec fails in a file other than the selected one, such as a spec in a project that the selected file's project depends on or a spec in a teardown project. That spec is no named failure of the selected file, so no issue can cover it. A spec's file is the file that Playwright loaded. A compiled file whose source map names the selected file is still another file.
 - Testkit cannot keep a file that a Playwright spec attached, or cannot store the evidence of the Playwright run.
-- A k6 process whose thresholds failed ends on a signal or with an exit code other than 99.
-- A k6 process whose script threw an error ends on a signal or with an exit code other than 0 or 99, or leaves no summary that Testkit can read.
-
-A k6 script reports a script failure and no named failure when it fails only through `fail()`, a thrown error, a k6 `check()`, or a threshold. `testkit bug file` then files one issue for the whole file, and the issue's facts carry no `test.title`.
-
-A k6 script reports its named failures and one script failure when it also fails that way beside the assertions the run names. Testkit tells a k6 `check()` from its own assertions by count. The k6 summary counts every failed check, and each failed assertion from `@gfdelarue/testkit/k6` is one failed check. When the summary's count differs from the number of failed assertions the run names, or the summary carries no count, the file has a script failure.
 
 A failure is known when an open issue with the `testkit` label covers it. The issue's `test.file` must equal the failing file's path from the repository root.
 
 | Failure         | Covered by                                                                                            |
 | --------------- | ----------------------------------------------------------------------------------------------------- |
 | Named failure   | An issue whose `test.title` equals the failure's title, or an issue whose facts have no `test.title`. |
-| Script failure  | An issue whose facts have no `test.title`.                                                            |
 | Unnamed failure | No issue.                                                                                             |
 
-An issue with a `test.title` covers that one title and nothing else. An issue with no `test.title` covers the whole file. That is every named failure in the file and its script failure, and never an unnamed failure. An unnamed failure is always new. Closing an issue makes the next failure it covered new.
+An issue with a `test.title` covers that one title and nothing else. An issue with no `test.title` covers the whole file. That is every named failure in the file, and never an unnamed failure. An unnamed failure is always new. Closing an issue makes the next failure it covered new.
 
 When Testkit cannot read the issues, every failure is new.
 
@@ -282,7 +282,7 @@ When Testkit cannot read the issues, every failure is new.
 | Target URL  | The CI run's URL, when the CI provider gives one.                                                                         |
 | Description | See below. At most 140 characters.                                                                                        |
 
-The description has one of three forms.
+The description has one of two forms.
 
 | Run                              | Description                |
 | -------------------------------- | -------------------------- |
@@ -294,12 +294,12 @@ The counts are failures. A script failure counts as one failure. An unnamed fail
 
 Testkit redacts the secrets it recognises in the reason. [Issue body](#issue-body) lists the forms.
 
-`testkit run` redacts some secrets when it stores the run result, and that redaction can stop before the secret's end. The description therefore ends at the first `<redacted>` in the stored reason. A redacted URL password is the exception. `testkit run` redacts a URL password whole, so the description continues past it.
+A run result stored before Testkit 0.5.0 can keep the rest of a secret after a `<redacted>`. The description therefore ends at the first `<redacted>` in the stored reason, keeping a quote that directly follows it. A redacted URL password is the exception. `testkit run` always redacted a URL password whole, so the description continues past it.
 
 ## Pull request comment
 
-`testkit ci report` keeps one comment on the run's pull request and updates it in place. The comment's first line is `<!-- testkit-ci-report -->`, or `<!-- testkit-ci-report:<context> -->` when the context is not `testkit`. Testkit finds its comment by that line.
+`testkit ci report` keeps one comment on the run's pull request and updates it in place. The comment's first line is `<!-- testkit-ci-report -->`, or `<!-- testkit-ci-report:<context> -->` when the context is not `testkit`. Testkit updates the comment that starts with that line, and adds one when no comment does, so each context keeps its own comment.
 
 The comment names each failure by its file and its title. Testkit redacts the secrets it recognises in both, in the reason for a run that failed outside its tests, and in the reason it could not read the open bugs. [Issue body](#issue-body) lists the forms.
 
-A file, a title, or a run's reason that the run result stores with `<redacted>` ends there in the comment and in the step summary, as it does in the [commit status](#commit-status). A title such as `rejects PASSWORD=<redacted> with 401` shows as `rejects PASSWORD=<redacted>`.
+A file, a title, or a run's reason that the run result stores with `<redacted>` ends there in the comment and in the step summary, as it does in the [commit status](#commit-status). A title such as `rejects PASSWORD=<redacted> with 401` shows as `rejects PASSWORD=<redacted>`, and `rejects PASSWORD="<redacted>" with 401` shows as `rejects PASSWORD="<redacted>"`.

package/docs/engines.md

@@ -0,0 +1,213 @@
+# Engines
+
+Testkit runs tests with two engines: `node-vitest` and `playwright`.
+[Execution engines](../README.md#execution-engines) says which files each one
+runs. This page covers what a test author or a CI setup needs from each engine:
+native authoring, typechecking, the environment a test receives, Playwright
+attachments, and installation.
+
+## Engine-native authoring
+
+Each engine keeps its own test language. The root package declares a typed
+file's engine, and each engine's subpath exposes that engine's API.
+
+Node/Vitest runs plain Vitest test files, and data and contract suites:
+
+```ts
+import { defineFile } from "@gfdelarue/testkit";
+import { dataScenario } from "@gfdelarue/testkit/node-vitest/data";
+import { expect } from "vitest";
+
+export const testkit = defineFile({ engine: "node-vitest" });
+
+dataScenario("records checkout traffic", ({ count, record }) => {
+  record("checkout", { count: 1 });
+  count("orders", 1);
+  expect(1).toBe(1);
+});
+```
+
+Playwright runs UI suites and keeps ordinary Playwright configuration:
+
+```ts
+import { defineFile } from "@gfdelarue/testkit";
+import { expect, test } from "@playwright/test";
+
+export const testkit = defineFile({ engine: "playwright" });
+
+test("shows the account", async ({ page }) => {
+  await page.goto("/account");
+  await expect(page.getByRole("heading", { name: "Account" })).toBeVisible();
+});
+```
+
+## Typechecking
+
+`testkit typecheck` groups the discovered test files by engine and by their
+nearest tsconfig, and checks each group with native `tsc --noEmit`. Plain Vitest
+test files written in TypeScript join the Node/Vitest program, and plain
+JavaScript files are skipped. Config and Node/Vitest use Node types and ES2022;
+Playwright adds DOM and DOM.Iterable. Each check has a deadline and can be
+cancelled. When a program fails, the report includes the programs that passed
+before it.
+
+Testkit supports consumer TypeScript 5.9.3 and 6.0.2. Each program uses the
+compiler installed beside its nearest tsconfig or in an ancestor directory, and
+otherwise Testkit's bundled compiler. Config checking starts at the product
+root. Runtime toolchain settings don't select the typecheck compiler.
+
+TypeScript follows transitive imports and workspace links and reports type
+errors. Programs inherit the consumer's compiler options and aliases, but a
+tsconfig's `files`, `include`, and project references don't add test roots.
+Testkit's authoring declarations and its pinned engine packages take precedence
+over conflicting consumer packages. A helper shared by several engines must
+typecheck in each engine's program. Testkit's generated configs don't set
+`baseUrl`, so they need no deprecation suppression and you never edit them.
+Aliases inherited from a config with `baseUrl` keep their target directory and
+fallback order.
+
+### Playwright support files
+
+A Playwright program always includes the discovered UI suites and the nearest
+`playwright.config.*` above the owning group's roots. Typechecking never runs
+that config. For support files that the config names as strings or chooses at
+run time, add a `tsconfig.playwright.json` in the test file's directory or an
+ancestor inside the product root:
+
+```json
+{
+  "extends": "./tsconfig.json",
+  "files": ["global/setup.ts", "global/teardown.ts"],
+  "include": ["fixtures/**/*.ts", "reporters/**/*.ts", "projects/**/*.ts"],
+  "exclude": ["projects/generated/**"]
+}
+```
+
+The nearest companion takes precedence over the ordinary tsconfig for that
+Playwright scope. Its `extends`, `files`, `include`, and `exclude` select the
+support files that join the Playwright program. Use it for global setup and
+teardown, reporters, fixtures, and dependency and teardown projects, including
+every variant of a multi-project config. An empty companion can supply options
+alone. Without a companion, Testkit checks the implicit roots, what they import,
+and the existing Next ambient and fixture defaults. Project references don't
+run.
+
+TypeScript follows imported helpers, but it can't find a file that only a
+runtime string or function names. Include those files in the companion.
+
+## Environment
+
+The environment declared for a test is part of the test's input. Each engine
+exposes it through its own authoring surface:
+
+- Node/Vitest data and contract callbacks read it from immutable `context.env`.
+- Playwright files read immutable `context.env` from `readPlaywrightContext()`
+  when Testkit runs them.
+
+Outside a Testkit run, a Node/Vitest authoring context has an empty `env`, and
+`readPlaywrightContext()` returns `null`.
+
+Typed Node/Vitest files, and plain files under `--vitest-runner managed`, also
+see each value as an immutable, non-enumerable `process.env.KEY` property, so
+libraries that read `process.env` see it. `process.env` isn't the enumerable
+list of the test's values, though. A Node child that inherits Testkit's
+bootstrap sees the values too. A non-Node child process receives only the
+values the test passes it. Plain files under the repository's Vitest get the
+values as ordinary environment variables, which tests and child processes can
+read, enumerate, change, and delete like any other.
+
+Test files of every engine may import `@gfdelarue/testkit/environment` and read
+named endpoints, databases, and handles with `endpoint(name)`, `database(name)`,
+and `handle(name)`. See
+[Named endpoints and databases](../README.md#named-endpoints-and-databases) and
+[Handles](../README.md#handles). `testkit.config.ts` and lifecycle step modules
+may not import it. In Node, a handle's operations use `node:http`, so a test
+that replaces `fetch` doesn't reroute them.
+
+Testkit checks the consumer environment's names case-insensitively. Empty
+names, names containing `=` or NUL, values containing NUL, and names that
+differ only in case fail instead of being normalized. Every engine reserves
+the keys `APPDATA`, `ComSpec`, `HOME`, `LANG`, `LC_ALL`, `LOCALAPPDATA`, `PATH`,
+`PATHEXT`, `SystemRoot`, `TEMP`, `TMP`, `TMPDIR`, `USERPROFILE`,
+`XDG_CACHE_HOME`, `XDG_CONFIG_HOME`, and `XDG_DATA_HOME`.
+
+Each engine also reserves these prefixes and keys, and limits the consumer
+environment:
+
+| Engine        | Reserved prefixes                                | Other reserved keys                                                                                                                                                                                                                                                                                                                                                                 | Consumer environment limit                                                                 |
+| ------------- | ------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ |
+| `node-vitest` | none                                             | `BASE_URL`, `FORCE_COLOR`, `NODE_OPTIONS`, `NO_COLOR`, `UPDATE_SNAPSHOT`, `VITEST_POOL_ID`, `VITEST_WORKER_ID`, `TESTKIT_ACTIVE`, `TESTKIT_DATA_ARTIFACT_DIR`, `TESTKIT_INVOCATION_ID`, `TESTKIT_LEASE_DIR`, `TESTKIT_LEASE_ID`, `TESTKIT_MODE`, `TESTKIT_NODE_VITEST_INPUT_FILE`, `TESTKIT_NODE_VITEST_INPUT_SHA256`, `TESTKIT_RUNTIME_ID`, `TESTKIT_TEST_FILE`, `TESTKIT_TEST_ID` | 512 entries and 4 MiB of UTF-8 key/value data; 8 MiB for all of the test's inputs together |
+| `playwright`  | `TESTKIT_`, `PLAYWRIGHT_`, `PWTEST_`, `PW_TEST_` | `BASE_URL`, `FORCE_COLOR`, `NODE_OPTIONS`, `NO_COLOR`                                                                                                                                                                                                                                                                                                                               | 512 entries and 2 MiB of UTF-8 key/value data; 4 MiB for all of the test's inputs together |
+
+Prefixes and keys match in any letter case. The processes Testkit starts have
+separate environment limits, which consumer values don't count toward: 24,576
+character units for Testkit's own Vitest, and 64 entries and 24 KiB for
+Playwright. The repository's Vitest starts with the shell's environment and the
+group's values, so only the operating system's limit applies to it. The
+reservations keep consumer
+configuration from overriding the endpoint, cancellation, deadline, reporter,
+module-resolution, and native-runner controls Testkit sets.
+
+## Compilation cache
+
+Node/Vitest and Playwright reuse Node's compilation cache across a run's files.
+Only compiled code is shared: evaluated modules, globals, and runner state stay
+in each test process. The cache lasts for one run. Setting Node's own cache controls or enabling V8 coverage
+turns it off, and a cache Testkit can't use falls back to running without one.
+
+Testkit rejects a file it reads for a test, such as a fixture, when a symlink
+leads it outside the product directory. A file inside the product whose name
+starts with `..`, such as `..fixtures/input.json`, is fine.
+
+## Playwright reports and attachments
+
+Testkit adds its reporter after the product's reporters, including JSON
+reporters, so the product's reporters keep running.
+
+A Playwright attachment may hold at most 64 MiB, and one file's attachments at
+most 256 MiB in total, whether the test attached a body or a path. An
+attachment over a limit fails the task with `PLAYWRIGHT_ATTACHMENT_LIMIT`,
+which names the attachment, the limit, and the bytes it held, and takes
+precedence over the spec's own result. An attachment that can't be read fails
+the task as an infrastructure failure when no spec failed. Either way, the
+results, assertion details, and accepted attachments of the rest of the file
+are kept, and every rejected attachment stays in the evidence. Cancellation,
+timeouts, and process failures keep their precedence over both.
+
+## Installation
+
+Consumers declare one dependency, `@gfdelarue/testkit`. It installs Vitest
+4.1.11 and Playwright 1.59.1. Plain Vitest test files run with the product's own
+Vitest 4.x and its config; see
+[Plain Vitest test files](../README.md#plain-vitest-test-files). Otherwise a
+product's copies of Vitest, Vite, esbuild, TypeScript, or Playwright don't
+replace the versions Testkit runs.
+
+Keep optional dependencies enabled because JavaScript tools such as esbuild
+use platform-specific optional packages. The product's lockfile makes the
+install reproducible. Testkit never downloads a missing package during a run.
+
+Browser installation is explicit:
+
+```text
+testkit setup playwright
+testkit setup playwright --with-deps
+testkit setup playwright --force
+```
+
+The command runs Playwright's own installer for Chromium. A run never installs
+a browser. When Chromium fails to launch, the task fails as an infrastructure
+failure with Playwright's own cause. The product's Playwright global setup and
+teardown still run.
+
+To recover a broken install, reinstall from the lockfile with optional
+dependencies included. To recover a broken browser, run
+`testkit setup playwright --force`, adding `--with-deps` on a host that lacks
+the browser's system libraries.
+
+### Supported platforms
+
+- Linux: Node/Vitest supports glibc and musl. Playwright doesn't support
+  musl and fails before setup or a run starts.
+- macOS: arm64 and x64.
+- Windows: x64 on Windows 10 or Windows Server 2016 and newer.

package/docs/test-types.md

@@ -1,14 +1,22 @@
 # Test Types
 
-Testkit exposes seven product-level test types:
+A test file is either a plain Vitest test file or a typed file.
+
+Plain Vitest test files have no test type. A file such as `src/sum.test.ts`
+imports `test` and `expect` from `vitest` and needs no Testkit suffix,
+`__testkit__` directory, `defineFile`, or wrapper. Unit tests belong here, and
+so does any Node-side test that reads its environment from `process.env` or
+`@gfdelarue/testkit/environment`. These files run under the general
+`node-vitest` profile, and a bug filed from one records no `testType`. See
+[Plain Vitest test files](../README.md#plain-vitest-test-files) for which files
+Testkit runs.
+
+Typed files carry one of three product-level test types in their filename
+suffix:
 
 - `ui`: browser-level UI checks for user-visible pages and interactions.
-- `e2e`: service-spanning flows that exercise real runtime behavior without driving a browser.
-- `int`: integration checks around routes, services, queues, jobs, or other runtime boundaries.
 - `contract`: Node-backed service/runtime contract checks that need Node APIs but are not data-flow evidence suites.
 - `data`: data-flow and data-contract checks for throughput, payload shape, conversation/request traces, and cost/latency budgets.
-- `dal`: data access and persistence checks.
-- `load`: load, soak, or pressure checks.
 
 Consumers choose the type that describes the product behavior and declare the
 execution engine in each file's `defineFile(...)` metadata. Testkit validates
@@ -16,21 +24,20 @@ that pair against its engine catalog and resolves the exact runner profile.
 
 The current catalog is deliberately explicit:
 
-| Test type            | Engine        | Runner profile         |
-| -------------------- | ------------- | ---------------------- |
-| `int`, `e2e`, `load` | `k6`          | `k6-http`              |
-| `dal`                | `k6`          | `k6-dal`               |
-| `data`               | `node-vitest` | `node-vitest-data`     |
-| `contract`           | `node-vitest` | `node-vitest-contract` |
-| `ui`                 | `playwright`  | `playwright-ui`        |
+| Test type          | Engine        | Runner profile         |
+| ------------------ | ------------- | ---------------------- |
+| none, a plain file | `node-vitest` | `node-vitest`          |
+| `data`             | `node-vitest` | `node-vitest-data`     |
+| `contract`         | `node-vitest` | `node-vitest-contract` |
+| `ui`               | `playwright`  | `playwright-ui`        |
 
-Every file must import `defineFile` from the root package and export one direct
-metadata declaration:
+Every typed file must import `defineFile` from the root package and export one
+direct metadata declaration:
 
 ```ts
 import { defineFile } from "@gfdelarue/testkit";
 
-export const testkit = defineFile({ engine: "k6" });
+export const testkit = defineFile({ engine: "node-vitest" });
 ```
 
 The engine is not inferred from the suffix, imports, or repo configuration. A
@@ -38,24 +45,22 @@ missing declaration, dynamic engine value, or unsupported pair is a discovery
 error. The profile is derived from the validated pair and is carried unchanged
 through scheduling, results, and immutable run evidence.
 
-Run selection by `--file`, `--service`, `--type`, or `--suite` narrows the files
-whose engine metadata is read and validated. Testkit uses discovered paths,
-canonical type suffixes, and suite names to select candidates; each selected
-file must still declare a valid engine. Invalid metadata in an unselected file
+Run selection by `--file`, `--group`, `--type`, or `--suite` narrows the files
+whose engine metadata is read and validated. A `--type` that names test types
+and typed suite selectors select typed files only; `--type all` includes plain
+files. Testkit uses discovered paths, canonical type
+suffixes, and suite names to select candidates; each selected typed file must
+still declare a valid engine. Invalid metadata in an unselected file
 does not block that focused run. Root configuration, service dependencies, and
-configured skip, requirement, and isolation selectors remain validated against
+group skip, requirement, and isolation selectors remain validated against
 the complete discovered project. `discover`, `doctor`, and `typecheck` validate
 metadata across the whole project.
 
 Canonical filename suffixes:
 
 - `*.ui.testkit.ts`
-- `*.e2e.testkit.ts`
 - `*.contract.testkit.ts`
 - `*.data.testkit.ts`
-- `*.int.testkit.ts`
-- `*.dal.testkit.ts`
-- `*.load.testkit.ts`
 
 UI tests may import Playwright's public browser primitives directly from the
 managed native package:
@@ -112,15 +117,12 @@ data-flow suite.
 Data suites run as Node processes managed by Testkit's `node-vitest` engine and
 should use configured service URLs for data-flow evidence. If a check requires
 service-local loopback, `serviceContextUrl`, DOM rendering, browser automation,
-or raw app-server process access, use `int`, `e2e`, or `ui` instead.
-
-Use `int` for ordinary service boundary checks that fit the k6 HTTP profile,
-`contract` for Node-backed service/runtime contracts that would otherwise look
-like Vitest, `e2e` for product journeys, `dal` for persistence
-contracts, and `ui` when the browser is part of the assertion. k6 suites retain
-native k6 options, scenarios, executors, VUs, thresholds, and lifecycle hooks;
-Testkit orchestrates them without translating those concepts into the other
-engines.
+or raw app-server process access, use a plain Vitest test, `contract`, or `ui` instead.
+
+Use a plain Vitest test file for unit tests and other Node-side checks that need
+no Testkit context, `contract` for typed service or protocol contracts, and `ui`
+when the browser is part of the assertion. Plain Vitest files can exercise HTTP,
+persistence, and service-spanning flows without a type label.
 
 The retired `@gfdelarue/testkit/runtime`, `@gfdelarue/testkit/data`,
 `@gfdelarue/testkit/contract`, `@gfdelarue/testkit/ui`, and

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@gfdelarue/testkit",
-  "version": "0.4.5",
+  "version": "0.5.0",
   "description": "Test orchestration platform for product acceptance and QA workflows",
   "license": "SEE LICENSE IN LICENSE",
   "type": "module",
@@ -12,59 +12,59 @@
     "access": "public",
     "registry": "https://registry.npmjs.org/"
   },
-  "types": "./dist/index.d.ts",
+  "types": "./dist/types/index.d.ts",
   "exports": {
     ".": {
-      "types": "./dist/index.d.ts",
+      "types": "./dist/types/index.d.ts",
       "default": "./dist/index.js"
     },
-    "./k6": {
-      "types": "./dist/engines/k6/runtime/index.d.ts",
-      "default": "./dist/engines/k6/runtime/index.js"
-    },
     "./config": {
-      "types": "./dist/config/authoring/index.d.ts",
+      "types": "./dist/types/config.d.ts",
       "default": "./dist/config/authoring/index.js"
     },
     "./config/database-steps": {
-      "types": "./dist/database/steps.d.ts",
+      "types": "./dist/types/config-database-steps.d.ts",
       "default": "./dist/database/steps.js"
     },
     "./drizzle": {
-      "types": "./dist/database/drizzle/index.d.ts",
+      "types": "./dist/types/drizzle.d.ts",
       "default": "./dist/database/drizzle/index.js"
     },
     "./env": {
-      "types": "./dist/config/runtime/env/index.d.ts",
+      "types": "./dist/types/env.d.ts",
       "default": "./dist/config/runtime/env/index.js"
     },
+    "./environment": {
+      "types": "./dist/types/environment.d.ts",
+      "default": "./dist/engines/environment/index.js"
+    },
     "./playwright": {
-      "types": "./dist/engines/playwright/runtime/index.d.ts",
+      "types": "./dist/types/playwright.d.ts",
       "default": "./dist/engines/playwright/runtime/index.js"
     },
     "./tooling/vitest": {
-      "types": "./dist/toolchains/vitest/index.d.ts",
+      "types": "./dist/types/tooling-vitest.d.ts",
       "default": "./dist/toolchains/vitest/index.js"
     },
     "./node-vitest/data": {
-      "types": "./dist/engines/node-vitest/runtime/data/index.d.ts",
+      "types": "./dist/types/node-vitest-data.d.ts",
       "default": "./dist/engines/node-vitest/runtime/data/index.js"
     },
     "./node-vitest/contract": {
-      "types": "./dist/engines/node-vitest/runtime/contract/index.d.ts",
+      "types": "./dist/types/node-vitest-contract.d.ts",
       "default": "./dist/engines/node-vitest/runtime/contract/index.js"
     },
     "./workspace": {
-      "types": "./dist/app/inspect/index.d.ts",
+      "types": "./dist/types/workspace.d.ts",
       "default": "./dist/app/inspect/index.js"
     },
     "./package.json": "./package.json",
     "./config/build-assets": {
-      "types": "./dist/config/runtime/build-assets.d.ts",
+      "types": "./dist/types/config-build-assets.d.ts",
       "default": "./dist/config/runtime/build-assets.js"
     },
     "./config/next-runtime-tsconfig": {
-      "types": "./dist/config/runtime/next-runtime-tsconfig.d.ts",
+      "types": "./dist/types/config-next-runtime-tsconfig.d.ts",
       "default": "./dist/config/runtime/next-runtime-tsconfig.js"
     }
   },
@@ -87,7 +87,7 @@
     "bin/",
     "dist/",
     "docs/bug-records.md",
-    "docs/engine-architecture.md",
+    "docs/engines.md",
     "docs/test-types.md",
     "testkit-config-schema.md"
   ],
@@ -102,18 +102,18 @@
     "@gfdelarue/code-budget": "0.1.2",
     "@homebridge/node-pty-prebuilt-multiarch": "^0.13.1",
     "@kitschpatrol/tokei": "2.0.2",
-    "@types/k6": "1.6.0",
     "@types/picomatch": "4.0.3",
     "@types/yauzl": "3.4.0",
+    "dts-bundle-generator": "9.5.1",
     "globals": "^17.7.0",
+    "pg": "8.23.1",
     "prettier": "3.9.6",
     "strip-ansi": "^7.1.2",
     "tsx": "4.23.12",
     "typescript-eslint": "^8.65.0",
     "typescript6": "npm:typescript@6.0.2",
     "typescript7": "npm:typescript@7.0.2",
-    "verdaccio": "6.10.3",
-    "xk6-sql": "https://github.com/grafana/xk6-sql/archive/refs/tags/v1.0.6.tar.gz"
+    "verdaccio": "6.10.3"
   },
   "dependencies": {
     "@playwright/test": "1.59.1",
@@ -138,13 +138,6 @@
     "wrap-ansi": "^10.0.0",
     "yauzl": "3.4.0"
   },
-  "optionalDependencies": {
-    "@gfdelarue/testkit-engine-k6-darwin-arm64": "1.0.0",
-    "@gfdelarue/testkit-engine-k6-darwin-x64": "1.0.0",
-    "@gfdelarue/testkit-engine-k6-linux-arm64": "1.0.0",
-    "@gfdelarue/testkit-engine-k6-linux-x64": "1.0.0",
-    "@gfdelarue/testkit-engine-k6-win32-x64": "1.0.0"
-  },
   "overrides": {
     "postcss": "8.5.24"
   },