@gfdelarue/testkit 0.4.5 → 0.6.0

package/docs/test-types.md

@@ -1,14 +1,25 @@
 # Test Types
 
-Testkit exposes seven product-level test types:
+A test file is either a plain Vitest test file or a typed file.
+
+Plain Vitest test files have no test type. A file such as `src/sum.test.ts`
+imports `test` and `expect` from `vitest` and needs no Testkit suffix,
+`__testkit__` directory, `defineFile`, or wrapper. Unit tests belong here, and
+so does any Node-side test that reads its environment from `process.env` or
+`@gfdelarue/testkit/environment`. These files run under the general
+`node-vitest` profile, and a bug filed from one records no `testType`. See
+[Plain Vitest test files](../README.md#plain-vitest-test-files) for which files
+Testkit runs.
+
+A plain Vitest test records data-flow evidence, such as metrics, sequences,
+records, and budgets, with `dataFlow(context)` from
+`@gfdelarue/testkit/node-vitest/data`. Recording evidence gives the test no
+type and no setup. See
+[Record data-flow evidence](../README.md#record-data-flow-evidence).
+
+Typed files carry one product-level test type in their filename suffix:
 
 - `ui`: browser-level UI checks for user-visible pages and interactions.
-- `e2e`: service-spanning flows that exercise real runtime behavior without driving a browser.
-- `int`: integration checks around routes, services, queues, jobs, or other runtime boundaries.
-- `contract`: Node-backed service/runtime contract checks that need Node APIs but are not data-flow evidence suites.
-- `data`: data-flow and data-contract checks for throughput, payload shape, conversation/request traces, and cost/latency budgets.
-- `dal`: data access and persistence checks.
-- `load`: load, soak, or pressure checks.
 
 Consumers choose the type that describes the product behavior and declare the
 execution engine in each file's `defineFile(...)` metadata. Testkit validates
@@ -16,21 +27,18 @@ that pair against its engine catalog and resolves the exact runner profile.
 
 The current catalog is deliberately explicit:
 
-| Test type            | Engine        | Runner profile         |
-| -------------------- | ------------- | ---------------------- |
-| `int`, `e2e`, `load` | `k6`          | `k6-http`              |
-| `dal`                | `k6`          | `k6-dal`               |
-| `data`               | `node-vitest` | `node-vitest-data`     |
-| `contract`           | `node-vitest` | `node-vitest-contract` |
-| `ui`                 | `playwright`  | `playwright-ui`        |
+| Test type          | Engine        | Runner profile  |
+| ------------------ | ------------- | --------------- |
+| none, a plain file | `node-vitest` | `node-vitest`   |
+| `ui`               | `playwright`  | `playwright-ui` |
 
-Every file must import `defineFile` from the root package and export one direct
-metadata declaration:
+Every typed file must import `defineFile` from the root package and export one
+direct metadata declaration:
 
 ```ts
 import { defineFile } from "@gfdelarue/testkit";
 
-export const testkit = defineFile({ engine: "k6" });
+export const testkit = defineFile({ engine: "playwright" });
 ```
 
 The engine is not inferred from the suffix, imports, or repo configuration. A
@@ -38,24 +46,18 @@ missing declaration, dynamic engine value, or unsupported pair is a discovery
 error. The profile is derived from the validated pair and is carried unchanged
 through scheduling, results, and immutable run evidence.
 
-Run selection by `--file`, `--service`, `--type`, or `--suite` narrows the files
-whose engine metadata is read and validated. Testkit uses discovered paths,
-canonical type suffixes, and suite names to select candidates; each selected
-file must still declare a valid engine. Invalid metadata in an unselected file
+Run selection by `--file`, `--group`, `--type`, or `--suite` narrows the files
+whose engine metadata is read and validated. `--type ui` and typed suite
+selectors select typed files only, `--type plain` selects plain files only, and
+`--type all` selects both. Testkit uses discovered paths, canonical type
+suffixes, and suite names to select candidates; each selected typed file must
+still declare a valid engine. Invalid metadata in an unselected file
 does not block that focused run. Root configuration, service dependencies, and
-configured skip, requirement, and isolation selectors remain validated against
+group skip, requirement, and isolation selectors remain validated against
 the complete discovered project. `discover`, `doctor`, and `typecheck` validate
 metadata across the whole project.
 
-Canonical filename suffixes:
-
-- `*.ui.testkit.ts`
-- `*.e2e.testkit.ts`
-- `*.contract.testkit.ts`
-- `*.data.testkit.ts`
-- `*.int.testkit.ts`
-- `*.dal.testkit.ts`
-- `*.load.testkit.ts`
+The canonical filename suffix is `*.ui.testkit.ts`.
 
 UI tests may import Playwright's public browser primitives directly from the
 managed native package:
@@ -67,60 +69,10 @@ import { expect, test } from "@playwright/test";
 export const testkit = defineFile({ engine: "playwright" });
 ```
 
-Data suites import data-flow primitives from Testkit:
-
-```ts
-import { defineFile } from "@gfdelarue/testkit";
-import { dataScenario } from "@gfdelarue/testkit/node-vitest/data";
-import { expect } from "vitest";
-
-export const testkit = defineFile({ engine: "node-vitest" });
-```
-
-Contract suites import Node-backed contract primitives from Testkit:
-
-```ts
-import { defineFile } from "@gfdelarue/testkit";
-import { contractScenario } from "@gfdelarue/testkit/node-vitest/contract";
-import { expect } from "vitest";
-
-export const testkit = defineFile({ engine: "node-vitest" });
-```
-
-Use `contract` when the behavior under test is a service/runtime boundary that
-needs Node APIs, fixtures, file IO, or richer protocol checks, but does not need
-Testkit data-flow evidence. Contract files are native Vitest suites:
-`contractScenario(...)` provides the managed Testkit contract context, while
-direct public runner APIs remain available for native behavior such as
-`test.skip(...)` and `test.todo(...)`. Contract suites run as Node processes
-managed by Testkit and should talk to configured service URLs.
-
-Use `data` when the behavior under test is the shape, cost, volume, latency, or
-movement of data through a real product flow. A data suite is not a generic
-Vitest migration target: write `dataScenario(...)` cases that record Testkit
-structured data-flow evidence with the provided data context, such as metrics,
-budgets, traces, fixtures, HTTP payload measurements, cache behavior, LLM usage,
-actors, systems, records, transitions, external calls, and reconciliation
-evidence.
-Data scenarios use Vitest directly for assertions, hooks, and mocking.
-`dataScenario(...)` owns case registration and evidence: lint reports native
-registration imports, and the runtime rejects cases registered any other way.
-Weak bookkeeping such as a single `count(...)`, `event(...)`, or `finding(...)`
-is useful as supporting detail, but it is not enough by itself to make a suite a
-data-flow suite.
-
-Data suites run as Node processes managed by Testkit's `node-vitest` engine and
-should use configured service URLs for data-flow evidence. If a check requires
-service-local loopback, `serviceContextUrl`, DOM rendering, browser automation,
-or raw app-server process access, use `int`, `e2e`, or `ui` instead.
-
-Use `int` for ordinary service boundary checks that fit the k6 HTTP profile,
-`contract` for Node-backed service/runtime contracts that would otherwise look
-like Vitest, `e2e` for product journeys, `dal` for persistence
-contracts, and `ui` when the browser is part of the assertion. k6 suites retain
-native k6 options, scenarios, executors, VUs, thresholds, and lifecycle hooks;
-Testkit orchestrates them without translating those concepts into the other
-engines.
+Use a plain Vitest test file for unit tests, other Node-side checks, and
+service or protocol contracts, and `ui` when the browser is part of the
+assertion. Plain Vitest files can exercise HTTP,
+persistence, and service-spanning flows without a type label.
 
 The retired `@gfdelarue/testkit/runtime`, `@gfdelarue/testkit/data`,
 `@gfdelarue/testkit/contract`, `@gfdelarue/testkit/ui`, and

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@gfdelarue/testkit",
-  "version": "0.4.5",
+  "version": "0.6.0",
   "description": "Test orchestration platform for product acceptance and QA workflows",
   "license": "SEE LICENSE IN LICENSE",
   "type": "module",
@@ -12,59 +12,55 @@
     "access": "public",
     "registry": "https://registry.npmjs.org/"
   },
-  "types": "./dist/index.d.ts",
+  "types": "./dist/types/index.d.ts",
   "exports": {
     ".": {
-      "types": "./dist/index.d.ts",
+      "types": "./dist/types/index.d.ts",
       "default": "./dist/index.js"
     },
-    "./k6": {
-      "types": "./dist/engines/k6/runtime/index.d.ts",
-      "default": "./dist/engines/k6/runtime/index.js"
-    },
     "./config": {
-      "types": "./dist/config/authoring/index.d.ts",
+      "types": "./dist/types/config.d.ts",
       "default": "./dist/config/authoring/index.js"
     },
     "./config/database-steps": {
-      "types": "./dist/database/steps.d.ts",
+      "types": "./dist/types/config-database-steps.d.ts",
       "default": "./dist/database/steps.js"
     },
     "./drizzle": {
-      "types": "./dist/database/drizzle/index.d.ts",
+      "types": "./dist/types/drizzle.d.ts",
       "default": "./dist/database/drizzle/index.js"
     },
     "./env": {
-      "types": "./dist/config/runtime/env/index.d.ts",
+      "types": "./dist/types/env.d.ts",
       "default": "./dist/config/runtime/env/index.js"
     },
+    "./environment": {
+      "types": "./dist/types/environment.d.ts",
+      "default": "./dist/engines/environment/index.js"
+    },
     "./playwright": {
-      "types": "./dist/engines/playwright/runtime/index.d.ts",
+      "types": "./dist/types/playwright.d.ts",
       "default": "./dist/engines/playwright/runtime/index.js"
     },
     "./tooling/vitest": {
-      "types": "./dist/toolchains/vitest/index.d.ts",
+      "types": "./dist/types/tooling-vitest.d.ts",
       "default": "./dist/toolchains/vitest/index.js"
     },
     "./node-vitest/data": {
-      "types": "./dist/engines/node-vitest/runtime/data/index.d.ts",
+      "types": "./dist/types/node-vitest-data.d.ts",
       "default": "./dist/engines/node-vitest/runtime/data/index.js"
     },
-    "./node-vitest/contract": {
-      "types": "./dist/engines/node-vitest/runtime/contract/index.d.ts",
-      "default": "./dist/engines/node-vitest/runtime/contract/index.js"
-    },
     "./workspace": {
-      "types": "./dist/app/inspect/index.d.ts",
+      "types": "./dist/types/workspace.d.ts",
       "default": "./dist/app/inspect/index.js"
     },
     "./package.json": "./package.json",
     "./config/build-assets": {
-      "types": "./dist/config/runtime/build-assets.d.ts",
+      "types": "./dist/types/config-build-assets.d.ts",
       "default": "./dist/config/runtime/build-assets.js"
     },
     "./config/next-runtime-tsconfig": {
-      "types": "./dist/config/runtime/next-runtime-tsconfig.d.ts",
+      "types": "./dist/types/config-next-runtime-tsconfig.d.ts",
       "default": "./dist/config/runtime/next-runtime-tsconfig.js"
     }
   },
@@ -87,7 +83,8 @@
     "bin/",
     "dist/",
     "docs/bug-records.md",
-    "docs/engine-architecture.md",
+    "docs/engines.md",
+    "docs/release-notes.md",
     "docs/test-types.md",
     "testkit-config-schema.md"
   ],
@@ -102,18 +99,18 @@
     "@gfdelarue/code-budget": "0.1.2",
     "@homebridge/node-pty-prebuilt-multiarch": "^0.13.1",
     "@kitschpatrol/tokei": "2.0.2",
-    "@types/k6": "1.6.0",
     "@types/picomatch": "4.0.3",
     "@types/yauzl": "3.4.0",
+    "dts-bundle-generator": "9.5.1",
     "globals": "^17.7.0",
+    "pg": "8.23.1",
     "prettier": "3.9.6",
     "strip-ansi": "^7.1.2",
     "tsx": "4.23.12",
     "typescript-eslint": "^8.65.0",
     "typescript6": "npm:typescript@6.0.2",
     "typescript7": "npm:typescript@7.0.2",
-    "verdaccio": "6.10.3",
-    "xk6-sql": "https://github.com/grafana/xk6-sql/archive/refs/tags/v1.0.6.tar.gz"
+    "verdaccio": "6.10.3"
   },
   "dependencies": {
     "@playwright/test": "1.59.1",
@@ -138,13 +135,6 @@
     "wrap-ansi": "^10.0.0",
     "yauzl": "3.4.0"
   },
-  "optionalDependencies": {
-    "@gfdelarue/testkit-engine-k6-darwin-arm64": "1.0.0",
-    "@gfdelarue/testkit-engine-k6-darwin-x64": "1.0.0",
-    "@gfdelarue/testkit-engine-k6-linux-arm64": "1.0.0",
-    "@gfdelarue/testkit-engine-k6-linux-x64": "1.0.0",
-    "@gfdelarue/testkit-engine-k6-win32-x64": "1.0.0"
-  },
   "overrides": {
     "postcss": "8.5.24"
   },