@gfdelarue/testkit 0.4.4 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (817) hide show
  1. package/README.md +752 -462
  2. package/bin/testkit-windows-owner-arm64.exe +0 -0
  3. package/bin/testkit-windows-owner-x64.exe +0 -0
  4. package/dist/app/bugs/categories.js +1 -1
  5. package/dist/app/bugs/ci-report.js +2 -2
  6. package/dist/app/bugs/ci-summary.js +3 -3
  7. package/dist/app/bugs/draft.js +4 -4
  8. package/dist/app/bugs/facts.js +2 -2
  9. package/dist/app/bugs/file.js +1 -1
  10. package/dist/app/bugs/gate.js +1 -1
  11. package/dist/app/bugs/guide.js +3 -3
  12. package/dist/app/bugs/issue.js +4 -4
  13. package/dist/app/bugs/known-failures.js +1 -1
  14. package/dist/app/bugs/record.js +2 -2
  15. package/dist/app/bugs/run-reading.js +1 -1
  16. package/dist/app/bugs/supplied-reproduce.js +1 -0
  17. package/dist/app/campaigns/campaign.js +1 -1
  18. package/dist/app/cleanup/index.js +1 -1
  19. package/dist/app/db-schema/prepare.js +1 -1
  20. package/dist/app/destroy/index.js +1 -1
  21. package/dist/app/discover/index.js +2 -2
  22. package/dist/app/doctor/index.js +1 -1
  23. package/dist/app/environments/config-driven.js +1 -0
  24. package/dist/app/environments/environment-file.js +1 -0
  25. package/dist/app/environments/file-setup.js +1 -0
  26. package/dist/app/environments/routing.js +1 -0
  27. package/dist/app/environments/static-values.js +1 -0
  28. package/dist/app/environments/supplied-input.js +1 -0
  29. package/dist/app/environments/supplied.js +1 -0
  30. package/dist/app/environments/task-databases.js +2 -0
  31. package/dist/app/file-selection.js +1 -1
  32. package/dist/app/inspect/artifacts.js +1 -1
  33. package/dist/app/inspect/index.js +1 -1
  34. package/dist/app/lint/index.js +1 -1
  35. package/dist/app/lint/native.js +1 -1
  36. package/dist/app/lint/options.js +1 -1
  37. package/dist/app/lint/rules/data.js +1 -1
  38. package/dist/app/lint/rules/engine-imports.js +1 -1
  39. package/dist/app/lint/rules/syntax.js +1 -1
  40. package/dist/app/lint/rules/unit.js +1 -1
  41. package/dist/app/lint/shared.js +1 -1
  42. package/dist/app/package-inspection.js +1 -1
  43. package/dist/app/reports/files.js +1 -0
  44. package/dist/app/reports/prepare.js +2 -0
  45. package/dist/app/reports/publish.js +1 -0
  46. package/dist/app/run/caller-redaction.js +1 -0
  47. package/dist/app/run/engine-evidence-persistence.js +1 -1
  48. package/dist/app/run/engine-execution-failures.js +1 -1
  49. package/dist/app/run/engine-execution-reporting.js +1 -1
  50. package/dist/app/run/engine-execution.js +1 -1
  51. package/dist/app/run/engine-orchestration.js +1 -1
  52. package/dist/app/run/environment.js +1 -0
  53. package/dist/app/run/failure-details.js +1 -1
  54. package/dist/app/run/live-run.js +1 -1
  55. package/dist/app/run/orchestrator.js +1 -1
  56. package/dist/app/run/planning.js +1 -1
  57. package/dist/app/run/request.js +1 -1
  58. package/dist/app/run/results.js +1 -1
  59. package/dist/app/run/run-events.js +1 -1
  60. package/dist/app/run/run-finalization.js +1 -1
  61. package/dist/app/run/runtime-fixtures.js +1 -1
  62. package/dist/app/run/scheduler/estimates.js +1 -1
  63. package/dist/app/run/scheduler/identity.js +1 -1
  64. package/dist/app/run/scheduler/index.js +1 -1
  65. package/dist/app/run/scheduler/locks.js +1 -0
  66. package/dist/app/run/selection.js +1 -1
  67. package/dist/app/run/setup/runtime-node-setup-protocol.js +1 -1
  68. package/dist/app/run/setup/runtime-node-setup-worker.js +1 -1
  69. package/dist/app/run/setup/runtime-node-setup.js +2 -2
  70. package/dist/app/run/task-batches.js +1 -1
  71. package/dist/app/run/task-leases.js +1 -0
  72. package/dist/app/run/worker-loop.js +1 -1
  73. package/dist/app/status/runs.js +1 -1
  74. package/dist/app/tracker/index.js +1 -1
  75. package/dist/app/typecheck/index.js +4 -4
  76. package/dist/cli/command-catalog.js +1 -1
  77. package/dist/cli/command-flags.js +1 -1
  78. package/dist/cli/command-registry.js +1 -1
  79. package/dist/cli/commands/artifacts.js +2 -2
  80. package/dist/cli/commands/bug/file.js +2 -2
  81. package/dist/cli/commands/bug/record.js +2 -2
  82. package/dist/cli/commands/discover.js +1 -1
  83. package/dist/cli/commands/report/prepare.js +5 -0
  84. package/dist/cli/commands/report/publish.js +2 -0
  85. package/dist/cli/commands/run.js +2 -2
  86. package/dist/cli/commands/typecheck.js +1 -1
  87. package/dist/cli/commands/workspace.js +1 -1
  88. package/dist/cli/framework/errors.js +1 -1
  89. package/dist/cli/framework/execute.js +7 -6
  90. package/dist/cli/framework/parser.js +1 -1
  91. package/dist/cli/json-errors.js +1 -1
  92. package/dist/cli/operations/run/operation.js +2 -2
  93. package/dist/cli/renderers/bug/text.js +1 -1
  94. package/dist/cli/renderers/discover/report.js +1 -1
  95. package/dist/cli/renderers/doctor/text.js +1 -1
  96. package/dist/cli/renderers/run/events.js +2 -2
  97. package/dist/cli/renderers/run/formatting.js +1 -1
  98. package/dist/cli/renderers/run/status.js +1 -1
  99. package/dist/cli/renderers/run/text-reporter.js +24 -22
  100. package/dist/cli/terminal/colors.js +1 -1
  101. package/dist/cli/terminal/layout.js +2 -2
  102. package/dist/config/authoring/http-methods.js +1 -1
  103. package/dist/config/authoring/index.js +1 -1
  104. package/dist/config/authoring/preset-env.js +1 -1
  105. package/dist/config/loader/config-loader.js +1 -1
  106. package/dist/config/loader/database.js +1 -1
  107. package/dist/config/loader/decode-execution.js +1 -1
  108. package/dist/config/loader/dependencies.js +1 -1
  109. package/dist/config/loader/discover-tests.js +1 -1
  110. package/dist/config/loader/discovery-summary.js +1 -1
  111. package/dist/config/loader/discovery.js +2 -2
  112. package/dist/config/loader/env.js +1 -1
  113. package/dist/config/loader/execution-model.js +1 -1
  114. package/dist/config/loader/file-metadata.js +1 -1
  115. package/dist/config/loader/groups.js +1 -0
  116. package/dist/config/loader/index.js +1 -1
  117. package/dist/config/loader/load.js +1 -1
  118. package/dist/config/loader/managed-configs.js +1 -1
  119. package/dist/config/loader/owned-discovery.js +1 -0
  120. package/dist/config/loader/path-policy.js +1 -1
  121. package/dist/config/loader/runtime-isolation.js +1 -1
  122. package/dist/config/loader/runtime.js +1 -1
  123. package/dist/config/loader/skip-config.js +1 -1
  124. package/dist/config/loader/task-policy.js +1 -0
  125. package/dist/config/loader/test-selection.js +1 -1
  126. package/dist/config/loader/validation.js +1 -1
  127. package/dist/database/activity-gate.js +1 -1
  128. package/dist/database/admin.js +2 -2
  129. package/dist/database/bindings.js +1 -0
  130. package/dist/database/cleanup.js +1 -1
  131. package/dist/database/docker-metadata.js +3 -3
  132. package/dist/database/docker.js +3 -3
  133. package/dist/database/index.js +1 -1
  134. package/dist/database/lifecycle-cleanup.js +1 -1
  135. package/dist/database/local-postgres.js +1 -1
  136. package/dist/database/naming.js +1 -1
  137. package/dist/database/postgres/connection-check.js +1 -0
  138. package/dist/database/postgres/protocol.js +2 -0
  139. package/dist/database/schema-operations.js +1 -1
  140. package/dist/database/schema-source.js +3 -3
  141. package/dist/database/source-schema-env.js +1 -0
  142. package/dist/database/source-url.js +2 -2
  143. package/dist/database/state-files.js +1 -1
  144. package/dist/database/state.js +1 -1
  145. package/dist/database/steps.js +3 -3
  146. package/dist/domain/file-timeout.js +1 -1
  147. package/dist/domain/identifiers.js +1 -1
  148. package/dist/domain/operation-display.js +1 -1
  149. package/dist/domain/retired-test-types.js +1 -0
  150. package/dist/domain/runtime-isolation-contract.js +1 -1
  151. package/dist/domain/suite-selection.js +1 -1
  152. package/dist/domain/test-outcomes.js +1 -1
  153. package/dist/domain/test-types.js +1 -1
  154. package/dist/engines/adapters/evidence-files.js +1 -1
  155. package/dist/engines/adapters/evidence-identity.js +1 -1
  156. package/dist/engines/adapters/execution.js +1 -1
  157. package/dist/engines/adapters/json-bounds.js +2 -0
  158. package/dist/engines/adapters/native-report.js +1 -1
  159. package/dist/engines/api/execution-scope.js +1 -1
  160. package/dist/engines/api/execution.js +1 -1
  161. package/dist/engines/api/index.js +1 -1
  162. package/dist/engines/api/runtime-artifact.js +1 -1
  163. package/dist/engines/bundle-cache.js +1 -1
  164. package/dist/engines/catalog.js +1 -1
  165. package/dist/engines/contracts/environment-values.js +1 -0
  166. package/dist/engines/contracts/fixture-manifest.js +1 -1
  167. package/dist/engines/definitions.js +1 -1
  168. package/dist/engines/environment/handles.js +1 -0
  169. package/dist/engines/environment/index.js +1 -0
  170. package/dist/engines/factory.js +1 -1
  171. package/dist/engines/installations/node-vitest.js +1 -1
  172. package/dist/engines/installations/package-resolution.js +1 -1
  173. package/dist/engines/installations/playwright-resolution-child.js +1 -1
  174. package/dist/engines/installations/playwright.js +1 -1
  175. package/dist/engines/managed-module-resolution-policy-input.js +1 -1
  176. package/dist/engines/managed-module-resolution-policy.js +1 -1
  177. package/dist/engines/managed-module-resolution.js +2 -2
  178. package/dist/engines/manager.js +1 -1
  179. package/dist/engines/node-vitest/adapter/adapter.js +1 -1
  180. package/dist/engines/node-vitest/adapter/driver.js +2 -0
  181. package/dist/engines/node-vitest/adapter/evidence.js +2 -2
  182. package/dist/engines/node-vitest/adapter/failure-details.js +4 -2
  183. package/dist/engines/node-vitest/adapter/reporter.js +1 -1
  184. package/dist/engines/node-vitest/adapter/workspace.js +31 -192
  185. package/dist/engines/node-vitest/inventory/index.js +2 -0
  186. package/dist/engines/node-vitest/inventory/list-files.js +3 -0
  187. package/dist/engines/node-vitest/runtime/contract/index.js +1 -1
  188. package/dist/engines/node-vitest/runtime/data/http.js +1 -1
  189. package/dist/engines/node-vitest/runtime/data/index.js +1 -1
  190. package/dist/engines/node-vitest/runtime/data/report.js +5 -6
  191. package/dist/engines/node-vitest/runtime/data/scenario.js +2 -2
  192. package/dist/engines/node-vitest/runtime/runtime-input.js +1 -1
  193. package/dist/engines/node-vitest/selection.js +1 -0
  194. package/dist/engines/playwright/adapter/adapter.js +1 -1
  195. package/dist/engines/playwright/adapter/config.js +4 -4
  196. package/dist/engines/playwright/adapter/evidence-validation.js +1 -1
  197. package/dist/engines/playwright/adapter/evidence.js +1 -1
  198. package/dist/engines/playwright/adapter/reporter.js +1 -1
  199. package/dist/engines/playwright/runtime/config-file.js +1 -1
  200. package/dist/engines/playwright/runtime/config.js +1 -1
  201. package/dist/engines/playwright/runtime/runtime-input.js +43 -20
  202. package/dist/engines/runtime-input-file.js +1 -1
  203. package/dist/environment/lifecycle-artifacts.js +1 -1
  204. package/dist/environment/lifecycle-controller.js +1 -1
  205. package/dist/environment/lifecycle.js +1 -1
  206. package/dist/environment/ports.js +1 -0
  207. package/dist/environment/ready-environment.js +1 -1
  208. package/dist/environment/run-environment.js +1 -1
  209. package/dist/environment/run-stopping.js +1 -1
  210. package/dist/environment/runtime-contexts.js +1 -1
  211. package/dist/environment/runtime-manager.js +1 -1
  212. package/dist/environment/runtime-planning.js +1 -1
  213. package/dist/environment/runtime-preparation.js +1 -1
  214. package/dist/environment/services.js +3 -1
  215. package/dist/environment/state.js +1 -1
  216. package/dist/environment/template.js +1 -1
  217. package/dist/evidence/artifact-config.js +1 -1
  218. package/dist/evidence/artifact-validation.js +1 -1
  219. package/dist/evidence/artifacts.js +2 -2
  220. package/dist/evidence/bundles.js +1 -1
  221. package/dist/evidence/environment-record.js +1 -0
  222. package/dist/evidence/index.js +1 -1
  223. package/dist/evidence/logs.js +2 -2
  224. package/dist/evidence/operation-timings.js +1 -1
  225. package/dist/evidence/read-fields.js +1 -1
  226. package/dist/evidence/read-run.js +1 -1
  227. package/dist/evidence/reading.js +1 -1
  228. package/dist/evidence/redaction.js +4 -0
  229. package/dist/evidence/reporting.js +1 -1
  230. package/dist/evidence/run-model.js +1 -0
  231. package/dist/evidence/storage.js +2 -2
  232. package/dist/evidence/timing.js +1 -1
  233. package/dist/kernel/error-serialization.js +1 -1
  234. package/dist/kernel/redaction-targets.js +1 -0
  235. package/dist/kernel/redaction.js +4 -1
  236. package/dist/kernel/unknown-values.js +1 -1
  237. package/dist/process/environment.js +1 -1
  238. package/dist/process/execa-bundle.json +3 -3
  239. package/dist/process/files/json-file.js +1 -1
  240. package/dist/process/files/paths.js +1 -1
  241. package/dist/process/host/metadata.js +1 -1
  242. package/dist/process/host/platform.js +1 -1
  243. package/dist/process/host-process.js +1 -1
  244. package/dist/process/index.js +1 -1
  245. package/dist/process/lifecycle-fence.js +1 -1
  246. package/dist/process/locks.js +4 -4
  247. package/dist/process/port-listener.js +2 -0
  248. package/dist/process/process-group.js +1 -1
  249. package/dist/process/process-identity.js +1 -1
  250. package/dist/process/process.js +33 -33
  251. package/dist/process/readiness.js +1 -1
  252. package/dist/process/ready-deadline.js +1 -0
  253. package/dist/process/recovery.js +1 -1
  254. package/dist/process/runs/manifest.js +1 -1
  255. package/dist/process/runs/ownership.js +1 -1
  256. package/dist/process/runs/product-state.js +1 -1
  257. package/dist/process/runs/stop.js +1 -1
  258. package/dist/process/service-processes.js +1 -1
  259. package/dist/process/stoppable-settlement.js +1 -0
  260. package/dist/process/url-redaction.js +1 -0
  261. package/dist/process/windows-cli-owner.js +3 -3
  262. package/dist/process/windows-native.js +1 -1
  263. package/dist/repo/ci-environment.js +1 -1
  264. package/dist/repo/fingerprint-policy.js +1 -1
  265. package/dist/repo/managed-git.js +1 -1
  266. package/dist/repo/physical-tree.js +1 -1
  267. package/dist/repo/publish-directory.js +3 -0
  268. package/dist/repo/state.js +4 -4
  269. package/dist/steps/build-config.js +1 -1
  270. package/dist/steps/configured-step-testkit-modules.js +1 -1
  271. package/dist/steps/configured-steps.js +1 -1
  272. package/dist/steps/environment.js +1 -1
  273. package/dist/steps/index.js +1 -1
  274. package/dist/steps/run.js +2 -2
  275. package/dist/toolchains/archive.js +1 -1
  276. package/dist/toolchains/index.js +3 -3
  277. package/dist/toolchains/node-provisioning.js +2 -2
  278. package/dist/toolchains/semver.js +1 -1
  279. package/dist/toolchains/vitest/index.js +1 -1
  280. package/dist/tracker/redaction.js +1 -2
  281. package/dist/{config/runtime/build-assets.d.ts → types/config-build-assets.d.ts} +5 -3
  282. package/dist/{database/steps.d.ts → types/config-database-steps.d.ts} +7 -7
  283. package/dist/{config/runtime/next-runtime-tsconfig.d.ts → types/config-next-runtime-tsconfig.d.ts} +5 -3
  284. package/dist/types/config.d.ts +501 -0
  285. package/dist/types/drizzle.d.ts +13 -0
  286. package/dist/{config/runtime/env/index.d.ts → types/env.d.ts} +22 -20
  287. package/dist/types/environment.d.ts +40 -0
  288. package/dist/types/index.d.ts +21 -0
  289. package/dist/types/node-vitest-contract.d.ts +75 -0
  290. package/dist/types/node-vitest-data.d.ts +407 -0
  291. package/dist/types/playwright.d.ts +130 -0
  292. package/dist/types/tooling-vitest.d.ts +13 -0
  293. package/dist/types/workspace.d.ts +617 -0
  294. package/docs/bug-records.md +83 -81
  295. package/docs/engines.md +213 -0
  296. package/docs/test-types.md +35 -33
  297. package/package.json +22 -29
  298. package/testkit-config-schema.md +403 -171
  299. package/dist/app/bugs/areas.d.ts +0 -38
  300. package/dist/app/bugs/assessment.d.ts +0 -9
  301. package/dist/app/bugs/categories.d.ts +0 -41
  302. package/dist/app/bugs/ci-report.d.ts +0 -21
  303. package/dist/app/bugs/ci-summary.d.ts +0 -13
  304. package/dist/app/bugs/draft.d.ts +0 -70
  305. package/dist/app/bugs/facts.d.ts +0 -48
  306. package/dist/app/bugs/file.d.ts +0 -45
  307. package/dist/app/bugs/gate.d.ts +0 -22
  308. package/dist/app/bugs/guide.d.ts +0 -20
  309. package/dist/app/bugs/issue.d.ts +0 -18
  310. package/dist/app/bugs/known-failures.d.ts +0 -49
  311. package/dist/app/bugs/record.d.ts +0 -14
  312. package/dist/app/bugs/run-reading.d.ts +0 -51
  313. package/dist/app/bugs/store.d.ts +0 -34
  314. package/dist/app/campaigns/brief.d.ts +0 -7
  315. package/dist/app/campaigns/campaign.d.ts +0 -14
  316. package/dist/app/campaigns/index.d.ts +0 -38
  317. package/dist/app/campaigns/store.d.ts +0 -5
  318. package/dist/app/campaigns/summary.d.ts +0 -13
  319. package/dist/app/cleanup/index.d.ts +0 -52
  320. package/dist/app/db-schema/prepare.d.ts +0 -11
  321. package/dist/app/db-schema/refresh.d.ts +0 -17
  322. package/dist/app/db-schema/verify.d.ts +0 -11
  323. package/dist/app/destroy/index.d.ts +0 -16
  324. package/dist/app/discover/index.d.ts +0 -8
  325. package/dist/app/doctor/index.d.ts +0 -17
  326. package/dist/app/file-selection.d.ts +0 -1
  327. package/dist/app/inspect/artifacts.d.ts +0 -30
  328. package/dist/app/inspect/index.d.ts +0 -112
  329. package/dist/app/lint/index.d.ts +0 -28
  330. package/dist/app/lint/native.d.ts +0 -10
  331. package/dist/app/lint/options.d.ts +0 -67
  332. package/dist/app/lint/policy.d.ts +0 -21
  333. package/dist/app/lint/rules/data.d.ts +0 -3
  334. package/dist/app/lint/rules/engine-imports.d.ts +0 -3
  335. package/dist/app/lint/rules/syntax.d.ts +0 -4
  336. package/dist/app/lint/rules/unit.d.ts +0 -3
  337. package/dist/app/lint/shared.d.ts +0 -36
  338. package/dist/app/package-inspection.d.ts +0 -20
  339. package/dist/app/run/endpoint-readiness.d.ts +0 -43
  340. package/dist/app/run/endpoint-readiness.js +0 -1
  341. package/dist/app/run/engine-evidence-persistence.d.ts +0 -39
  342. package/dist/app/run/engine-execution-cleanup.d.ts +0 -8
  343. package/dist/app/run/engine-execution-cleanup.js +0 -1
  344. package/dist/app/run/engine-execution-failures.d.ts +0 -41
  345. package/dist/app/run/engine-execution-observer.d.ts +0 -21
  346. package/dist/app/run/engine-execution-reporting.d.ts +0 -17
  347. package/dist/app/run/engine-execution.d.ts +0 -63
  348. package/dist/app/run/engine-failure-stack.d.ts +0 -1
  349. package/dist/app/run/engine-orchestration.d.ts +0 -49
  350. package/dist/app/run/failure-details.d.ts +0 -37
  351. package/dist/app/run/live-run.d.ts +0 -27
  352. package/dist/app/run/operation-progress.d.ts +0 -13
  353. package/dist/app/run/orchestrator.d.ts +0 -68
  354. package/dist/app/run/planning-types.d.ts +0 -68
  355. package/dist/app/run/planning.d.ts +0 -15
  356. package/dist/app/run/playwright-configuration.d.ts +0 -2
  357. package/dist/app/run/playwright-configuration.js +0 -1
  358. package/dist/app/run/request.d.ts +0 -30
  359. package/dist/app/run/results.d.ts +0 -161
  360. package/dist/app/run/run-events.d.ts +0 -144
  361. package/dist/app/run/run-finalization.d.ts +0 -52
  362. package/dist/app/run/run-guards.d.ts +0 -3
  363. package/dist/app/run/runtime-failures.d.ts +0 -18
  364. package/dist/app/run/runtime-fixtures.d.ts +0 -2
  365. package/dist/app/run/runtime-task-types.d.ts +0 -20
  366. package/dist/app/run/scheduler/estimates.d.ts +0 -32
  367. package/dist/app/run/scheduler/identity.d.ts +0 -9
  368. package/dist/app/run/scheduler/index.d.ts +0 -124
  369. package/dist/app/run/scheduler/observations.d.ts +0 -20
  370. package/dist/app/run/selection.d.ts +0 -10
  371. package/dist/app/run/setup/runtime-node-setup-errors.d.ts +0 -13
  372. package/dist/app/run/setup/runtime-node-setup-protocol.d.ts +0 -85
  373. package/dist/app/run/setup/runtime-node-setup-worker.d.ts +0 -1
  374. package/dist/app/run/setup/runtime-node-setup.d.ts +0 -48
  375. package/dist/app/run/task-batches.d.ts +0 -23
  376. package/dist/app/run/worker-loop.d.ts +0 -57
  377. package/dist/app/setup/playwright.d.ts +0 -22
  378. package/dist/app/status/index.d.ts +0 -9
  379. package/dist/app/status/model.d.ts +0 -29
  380. package/dist/app/status/runs.d.ts +0 -99
  381. package/dist/app/stop/index.d.ts +0 -14
  382. package/dist/app/tracker/index.d.ts +0 -8
  383. package/dist/app/tracker/remote.d.ts +0 -13
  384. package/dist/app/tracker/token.d.ts +0 -7
  385. package/dist/app/typecheck/index.d.ts +0 -63
  386. package/dist/cli/command-catalog.d.ts +0 -137
  387. package/dist/cli/command-flags.d.ts +0 -19
  388. package/dist/cli/command-registry.d.ts +0 -3
  389. package/dist/cli/commands/artifacts.d.ts +0 -48
  390. package/dist/cli/commands/bug/file.d.ts +0 -16
  391. package/dist/cli/commands/bug/guide.d.ts +0 -5
  392. package/dist/cli/commands/bug/record.d.ts +0 -9
  393. package/dist/cli/commands/campaign/finish.d.ts +0 -10
  394. package/dist/cli/commands/campaign/start.d.ts +0 -7
  395. package/dist/cli/commands/campaign/status.d.ts +0 -13
  396. package/dist/cli/commands/ci/report.d.ts +0 -15
  397. package/dist/cli/commands/cleanup.d.ts +0 -13
  398. package/dist/cli/commands/db/schema/refresh.d.ts +0 -2
  399. package/dist/cli/commands/db/schema/verify.d.ts +0 -2
  400. package/dist/cli/commands/destroy.d.ts +0 -9
  401. package/dist/cli/commands/discover.d.ts +0 -2
  402. package/dist/cli/commands/doctor.d.ts +0 -2
  403. package/dist/cli/commands/guard/unsafe-entrypoint.d.ts +0 -2
  404. package/dist/cli/commands/lint.d.ts +0 -2
  405. package/dist/cli/commands/run/status.d.ts +0 -5
  406. package/dist/cli/commands/run/stop.d.ts +0 -7
  407. package/dist/cli/commands/run.d.ts +0 -9
  408. package/dist/cli/commands/setup/playwright.d.ts +0 -2
  409. package/dist/cli/commands/status.d.ts +0 -5
  410. package/dist/cli/commands/stop.d.ts +0 -12
  411. package/dist/cli/commands/typecheck.d.ts +0 -2
  412. package/dist/cli/commands/workspace.d.ts +0 -2
  413. package/dist/cli/entrypoint.d.ts +0 -1
  414. package/dist/cli/framework/command.d.ts +0 -21
  415. package/dist/cli/framework/definitions.d.ts +0 -100
  416. package/dist/cli/framework/errors.d.ts +0 -6
  417. package/dist/cli/framework/execute.d.ts +0 -14
  418. package/dist/cli/framework/help.d.ts +0 -11
  419. package/dist/cli/framework/index.d.ts +0 -3
  420. package/dist/cli/framework/parser.d.ts +0 -6
  421. package/dist/cli/index.d.ts +0 -4
  422. package/dist/cli/json-errors.d.ts +0 -3
  423. package/dist/cli/operations/db/schema/refresh/operation.d.ts +0 -7
  424. package/dist/cli/operations/db/schema/verify/operation.d.ts +0 -7
  425. package/dist/cli/operations/run/operation.d.ts +0 -24
  426. package/dist/cli/process-signal.d.ts +0 -1
  427. package/dist/cli/renderers/bug/text.d.ts +0 -8
  428. package/dist/cli/renderers/campaign/text.d.ts +0 -2
  429. package/dist/cli/renderers/ci/text.d.ts +0 -2
  430. package/dist/cli/renderers/cleanup/format.d.ts +0 -14
  431. package/dist/cli/renderers/db-schema/text.d.ts +0 -4
  432. package/dist/cli/renderers/destroy/text.d.ts +0 -4
  433. package/dist/cli/renderers/discover/report.d.ts +0 -7
  434. package/dist/cli/renderers/discover/text.d.ts +0 -3
  435. package/dist/cli/renderers/doctor/text.d.ts +0 -2
  436. package/dist/cli/renderers/lint/text.d.ts +0 -2
  437. package/dist/cli/renderers/run/events.d.ts +0 -8
  438. package/dist/cli/renderers/run/failure.d.ts +0 -8
  439. package/dist/cli/renderers/run/formatting.d.ts +0 -79
  440. package/dist/cli/renderers/run/status.d.ts +0 -6
  441. package/dist/cli/renderers/run/text-reporter.d.ts +0 -19
  442. package/dist/cli/renderers/status/text.d.ts +0 -2
  443. package/dist/cli/renderers/stop/text.d.ts +0 -7
  444. package/dist/cli/renderers/typecheck/text.d.ts +0 -2
  445. package/dist/cli/terminal/colors.d.ts +0 -10
  446. package/dist/cli/terminal/layout.d.ts +0 -11
  447. package/dist/cli/version-drift.d.ts +0 -25
  448. package/dist/config/authoring/build-config-contract.d.ts +0 -27
  449. package/dist/config/authoring/configured-step-contract.d.ts +0 -18
  450. package/dist/config/authoring/http-execution-contract.d.ts +0 -70
  451. package/dist/config/authoring/http-methods.d.ts +0 -18
  452. package/dist/config/authoring/index.d.ts +0 -381
  453. package/dist/config/authoring/preset-env.d.ts +0 -19
  454. package/dist/config/authoring/runtime.d.ts +0 -64
  455. package/dist/config/loader/bugs.d.ts +0 -15
  456. package/dist/config/loader/config-loader.d.ts +0 -9
  457. package/dist/config/loader/database-materialization.d.ts +0 -19
  458. package/dist/config/loader/database-roles.d.ts +0 -2
  459. package/dist/config/loader/database.d.ts +0 -55
  460. package/dist/config/loader/decode-discovery.d.ts +0 -5
  461. package/dist/config/loader/decode-execution.d.ts +0 -9
  462. package/dist/config/loader/dependencies.d.ts +0 -7
  463. package/dist/config/loader/discover-tests.d.ts +0 -111
  464. package/dist/config/loader/discovery-summary.d.ts +0 -29
  465. package/dist/config/loader/discovery.d.ts +0 -54
  466. package/dist/config/loader/env.d.ts +0 -4
  467. package/dist/config/loader/execution-model.d.ts +0 -16
  468. package/dist/config/loader/file-metadata.d.ts +0 -19
  469. package/dist/config/loader/index.d.ts +0 -10
  470. package/dist/config/loader/load.d.ts +0 -91
  471. package/dist/config/loader/managed-configs.d.ts +0 -23
  472. package/dist/config/loader/path-policy.d.ts +0 -9
  473. package/dist/config/loader/paths.d.ts +0 -5
  474. package/dist/config/loader/runtime-isolation.d.ts +0 -44
  475. package/dist/config/loader/runtime.d.ts +0 -25
  476. package/dist/config/loader/skip-config.d.ts +0 -44
  477. package/dist/config/loader/test-selection.d.ts +0 -15
  478. package/dist/config/loader/validation.d.ts +0 -16
  479. package/dist/config/loader/values.d.ts +0 -4
  480. package/dist/config/loader/values.js +0 -1
  481. package/dist/database/activity-gate.d.ts +0 -22
  482. package/dist/database/admin.d.ts +0 -31
  483. package/dist/database/cleanup.d.ts +0 -36
  484. package/dist/database/configured-steps.d.ts +0 -27
  485. package/dist/database/constants.d.ts +0 -11
  486. package/dist/database/docker-metadata.d.ts +0 -3
  487. package/dist/database/docker.d.ts +0 -65
  488. package/dist/database/drizzle/index.d.ts +0 -13
  489. package/dist/database/fingerprint.d.ts +0 -12
  490. package/dist/database/index.d.ts +0 -39
  491. package/dist/database/lifecycle-cleanup.d.ts +0 -7
  492. package/dist/database/local-postgres.d.ts +0 -71
  493. package/dist/database/naming.d.ts +0 -17
  494. package/dist/database/postgres/index.d.ts +0 -71
  495. package/dist/database/progress.d.ts +0 -6
  496. package/dist/database/roles.d.ts +0 -19
  497. package/dist/database/runtime-config.d.ts +0 -16
  498. package/dist/database/runtime-reservations.d.ts +0 -19
  499. package/dist/database/schema-operations.d.ts +0 -18
  500. package/dist/database/schema-source.d.ts +0 -96
  501. package/dist/database/source-refresh-lock.d.ts +0 -8
  502. package/dist/database/source-url.d.ts +0 -22
  503. package/dist/database/state-files.d.ts +0 -13
  504. package/dist/database/state.d.ts +0 -26
  505. package/dist/database/template-steps.d.ts +0 -39
  506. package/dist/domain/file-timeout.d.ts +0 -28
  507. package/dist/domain/identifiers.d.ts +0 -9
  508. package/dist/domain/operation-display.d.ts +0 -10
  509. package/dist/domain/operation-reason.d.ts +0 -5
  510. package/dist/domain/postgres-route.d.ts +0 -4
  511. package/dist/domain/runtime-identity.d.ts +0 -1
  512. package/dist/domain/runtime-isolation-contract.d.ts +0 -5
  513. package/dist/domain/service-name.d.ts +0 -2
  514. package/dist/domain/suite-selection.d.ts +0 -20
  515. package/dist/domain/test-outcomes.d.ts +0 -3
  516. package/dist/domain/test-types.d.ts +0 -16
  517. package/dist/engines/adapters/evidence-files.d.ts +0 -32
  518. package/dist/engines/adapters/evidence-identity.d.ts +0 -12
  519. package/dist/engines/adapters/execution.d.ts +0 -42
  520. package/dist/engines/adapters/native-report.d.ts +0 -25
  521. package/dist/engines/adapters/node-startup-cache.d.ts +0 -8
  522. package/dist/engines/adapters/workspace.d.ts +0 -4
  523. package/dist/engines/api/capabilities.d.ts +0 -31
  524. package/dist/engines/api/capabilities.js +0 -1
  525. package/dist/engines/api/evidence.d.ts +0 -38
  526. package/dist/engines/api/execution-observer.d.ts +0 -8
  527. package/dist/engines/api/execution-scope.d.ts +0 -17
  528. package/dist/engines/api/execution.d.ts +0 -101
  529. package/dist/engines/api/index.d.ts +0 -5
  530. package/dist/engines/api/k6-runtime-input.d.ts +0 -26
  531. package/dist/engines/api/k6-runtime-input.js +0 -1
  532. package/dist/engines/api/runtime-artifact.d.ts +0 -19
  533. package/dist/engines/bundle-cache.d.ts +0 -32
  534. package/dist/engines/catalog.d.ts +0 -27
  535. package/dist/engines/contracts/fixture-manifest.d.ts +0 -34
  536. package/dist/engines/contracts/json-bounds.d.ts +0 -36
  537. package/dist/engines/contracts/json-bounds.js +0 -2
  538. package/dist/engines/contracts/k6-repository-configuration.d.ts +0 -12
  539. package/dist/engines/contracts/k6-repository-configuration.js +0 -1
  540. package/dist/engines/contracts/test-context.d.ts +0 -15
  541. package/dist/engines/contracts/test-context.js +0 -1
  542. package/dist/engines/definitions.d.ts +0 -22
  543. package/dist/engines/factory.d.ts +0 -6
  544. package/dist/engines/generated-performance.d.ts +0 -1
  545. package/dist/engines/installation-catalog.d.ts +0 -27
  546. package/dist/engines/installation-catalog.js +0 -1
  547. package/dist/engines/installations/k6.d.ts +0 -98
  548. package/dist/engines/installations/k6.js +0 -1
  549. package/dist/engines/installations/node-vitest.d.ts +0 -33
  550. package/dist/engines/installations/package-requirements.d.ts +0 -28
  551. package/dist/engines/installations/package-resolution.d.ts +0 -51
  552. package/dist/engines/installations/playwright-resolution-child.d.ts +0 -1
  553. package/dist/engines/installations/playwright.d.ts +0 -75
  554. package/dist/engines/installations/types.d.ts +0 -10
  555. package/dist/engines/k6/adapter/bundle.d.ts +0 -25
  556. package/dist/engines/k6/adapter/bundle.js +0 -112
  557. package/dist/engines/k6/adapter/evidence.d.ts +0 -15
  558. package/dist/engines/k6/adapter/evidence.js +0 -4
  559. package/dist/engines/k6/adapter/executor.d.ts +0 -13
  560. package/dist/engines/k6/adapter/executor.js +0 -1
  561. package/dist/engines/k6/adapter/index.d.ts +0 -1
  562. package/dist/engines/k6/adapter/index.js +0 -1
  563. package/dist/engines/k6/adapter/input.d.ts +0 -21
  564. package/dist/engines/k6/adapter/input.js +0 -1
  565. package/dist/engines/k6/adapter/native-summary.d.ts +0 -11
  566. package/dist/engines/k6/adapter/native-summary.js +0 -1
  567. package/dist/engines/k6/adapter/repository-configuration.d.ts +0 -2
  568. package/dist/engines/k6/adapter/repository-configuration.js +0 -1
  569. package/dist/engines/k6/authoring-type-paths.d.ts +0 -3
  570. package/dist/engines/k6/authoring-type-paths.js +0 -1
  571. package/dist/engines/k6/runtime/ambient-types/bundle-manifest.json +0 -146
  572. package/dist/engines/k6/runtime/ambient-types/k6/LICENSE +0 -21
  573. package/dist/engines/k6/runtime/ambient-types/k6/browser/index.d.ts +0 -6332
  574. package/dist/engines/k6/runtime/ambient-types/k6/crypto/index.d.ts +0 -218
  575. package/dist/engines/k6/runtime/ambient-types/k6/data/index.d.ts +0 -16
  576. package/dist/engines/k6/runtime/ambient-types/k6/encoding/index.d.ts +0 -46
  577. package/dist/engines/k6/runtime/ambient-types/k6/execution/index.d.ts +0 -127
  578. package/dist/engines/k6/runtime/ambient-types/k6/experimental/csv/index.d.ts +0 -96
  579. package/dist/engines/k6/runtime/ambient-types/k6/experimental/fs/index.d.ts +0 -104
  580. package/dist/engines/k6/runtime/ambient-types/k6/experimental/redis/index.d.ts +0 -821
  581. package/dist/engines/k6/runtime/ambient-types/k6/experimental/streams/index.d.ts +0 -198
  582. package/dist/engines/k6/runtime/ambient-types/k6/global.d.ts +0 -83
  583. package/dist/engines/k6/runtime/ambient-types/k6/html/index.d.ts +0 -1325
  584. package/dist/engines/k6/runtime/ambient-types/k6/http/index.d.ts +0 -806
  585. package/dist/engines/k6/runtime/ambient-types/k6/index.d.ts +0 -126
  586. package/dist/engines/k6/runtime/ambient-types/k6/metrics/index.d.ts +0 -66
  587. package/dist/engines/k6/runtime/ambient-types/k6/net/grpc/index.d.ts +0 -247
  588. package/dist/engines/k6/runtime/ambient-types/k6/options/index.d.ts +0 -459
  589. package/dist/engines/k6/runtime/ambient-types/k6/package.json +0 -79
  590. package/dist/engines/k6/runtime/ambient-types/k6/secrets/index.d.ts +0 -36
  591. package/dist/engines/k6/runtime/ambient-types/k6/timers/index.d.ts +0 -47
  592. package/dist/engines/k6/runtime/ambient-types/k6/websockets/index.d.ts +0 -324
  593. package/dist/engines/k6/runtime/ambient-types/k6/ws/index.d.ts +0 -283
  594. package/dist/engines/k6/runtime/ambient-types/xk6-sql/LICENSE +0 -201
  595. package/dist/engines/k6/runtime/ambient-types/xk6-sql/index.d.ts +0 -397
  596. package/dist/engines/k6/runtime/ambient-types/xk6-sql/package.json +0 -15
  597. package/dist/engines/k6/runtime/artifacts.d.ts +0 -2
  598. package/dist/engines/k6/runtime/artifacts.js +0 -1
  599. package/dist/engines/k6/runtime/checks.d.ts +0 -17
  600. package/dist/engines/k6/runtime/checks.js +0 -1
  601. package/dist/engines/k6/runtime/dal-fixtures.d.ts +0 -24
  602. package/dist/engines/k6/runtime/dal-fixtures.js +0 -1
  603. package/dist/engines/k6/runtime/dal-suite.d.ts +0 -24
  604. package/dist/engines/k6/runtime/dal-suite.js +0 -1
  605. package/dist/engines/k6/runtime/dal.d.ts +0 -10
  606. package/dist/engines/k6/runtime/dal.js +0 -1
  607. package/dist/engines/k6/runtime/fixtures.d.ts +0 -3
  608. package/dist/engines/k6/runtime/fixtures.js +0 -1
  609. package/dist/engines/k6/runtime/http-assertions.d.ts +0 -19
  610. package/dist/engines/k6/runtime/http-assertions.js +0 -1
  611. package/dist/engines/k6/runtime/http-checks.d.ts +0 -19
  612. package/dist/engines/k6/runtime/http-checks.js +0 -1
  613. package/dist/engines/k6/runtime/http-suite-runtime.d.ts +0 -103
  614. package/dist/engines/k6/runtime/http-suite-runtime.js +0 -1
  615. package/dist/engines/k6/runtime/http.d.ts +0 -15
  616. package/dist/engines/k6/runtime/http.js +0 -1
  617. package/dist/engines/k6/runtime/index.d.ts +0 -11
  618. package/dist/engines/k6/runtime/index.js +0 -1
  619. package/dist/engines/k6/runtime/input.d.ts +0 -3
  620. package/dist/engines/k6/runtime/input.js +0 -1
  621. package/dist/engines/k6/runtime/k6-repository-configuration.d.ts +0 -9
  622. package/dist/engines/k6/runtime/k6-repository-configuration.js +0 -1
  623. package/dist/engines/k6/runtime/node-setup.d.ts +0 -13
  624. package/dist/engines/k6/runtime/node-setup.js +0 -1
  625. package/dist/engines/k6/runtime/runtime-globals.d.ts +0 -10
  626. package/dist/engines/k6/runtime/runtime.d.ts +0 -54
  627. package/dist/engines/k6/runtime/runtime.js +0 -1
  628. package/dist/engines/k6/runtime/shared/assertion-details.d.ts +0 -19
  629. package/dist/engines/k6/runtime/shared/assertion-details.js +0 -2
  630. package/dist/engines/k6/runtime/shared/db-normalization.d.ts +0 -19
  631. package/dist/engines/k6/runtime/shared/db-normalization.js +0 -1
  632. package/dist/engines/k6/runtime/shared/error-body.d.ts +0 -3
  633. package/dist/engines/k6/runtime/shared/error-body.js +0 -1
  634. package/dist/engines/k6/runtime/shared/file-data.d.ts +0 -1
  635. package/dist/engines/k6/runtime/shared/file-data.js +0 -1
  636. package/dist/engines/k6/runtime/shared/fixture-engine.d.ts +0 -8
  637. package/dist/engines/k6/runtime/shared/fixture-engine.js +0 -2
  638. package/dist/engines/k6/runtime/shared/http-check-plan.d.ts +0 -29
  639. package/dist/engines/k6/runtime/shared/http-check-plan.js +0 -1
  640. package/dist/engines/k6/runtime/shared/http-parsing.d.ts +0 -10
  641. package/dist/engines/k6/runtime/shared/http-parsing.js +0 -3
  642. package/dist/engines/k6/runtime/shared/ordering.d.ts +0 -2
  643. package/dist/engines/k6/runtime/shared/ordering.js +0 -1
  644. package/dist/engines/k6/runtime/shared/properties.d.ts +0 -1
  645. package/dist/engines/k6/runtime/shared/properties.js +0 -1
  646. package/dist/engines/k6/runtime/suite.d.ts +0 -4
  647. package/dist/engines/k6/runtime/suite.js +0 -1
  648. package/dist/engines/k6/runtime/types.d.ts +0 -244
  649. package/dist/engines/k6/runtime/types.js +0 -1
  650. package/dist/engines/k6/runtime/xk6-sql-driver-postgres.d.ts +0 -7
  651. package/dist/engines/k6/runtime/xk6-sql-driver-postgres.js +0 -1
  652. package/dist/engines/managed-module-resolution-policy-input.d.ts +0 -9
  653. package/dist/engines/managed-module-resolution-policy.d.ts +0 -25
  654. package/dist/engines/managed-module-resolution.d.ts +0 -13
  655. package/dist/engines/manager.d.ts +0 -25
  656. package/dist/engines/node-vitest/adapter/adapter.d.ts +0 -16
  657. package/dist/engines/node-vitest/adapter/consumer-typescript-paths.d.ts +0 -17
  658. package/dist/engines/node-vitest/adapter/consumer-typescript-paths.js +0 -3
  659. package/dist/engines/node-vitest/adapter/evidence.d.ts +0 -33
  660. package/dist/engines/node-vitest/adapter/failure-details.d.ts +0 -3
  661. package/dist/engines/node-vitest/adapter/index.d.ts +0 -1
  662. package/dist/engines/node-vitest/adapter/reporter.d.ts +0 -7
  663. package/dist/engines/node-vitest/adapter/resolution.d.ts +0 -12
  664. package/dist/engines/node-vitest/adapter/resolution.js +0 -1
  665. package/dist/engines/node-vitest/adapter/workspace.d.ts +0 -19
  666. package/dist/engines/node-vitest/runtime/contract/index.d.ts +0 -64
  667. package/dist/engines/node-vitest/runtime/data/http.d.ts +0 -4
  668. package/dist/engines/node-vitest/runtime/data/index.d.ts +0 -8
  669. package/dist/engines/node-vitest/runtime/data/primitives.d.ts +0 -1
  670. package/dist/engines/node-vitest/runtime/data/report.d.ts +0 -3
  671. package/dist/engines/node-vitest/runtime/data/runner.d.ts +0 -4
  672. package/dist/engines/node-vitest/runtime/data/scenario.d.ts +0 -12
  673. package/dist/engines/node-vitest/runtime/data/types.d.ts +0 -412
  674. package/dist/engines/node-vitest/runtime/runtime-input.d.ts +0 -64
  675. package/dist/engines/playwright/adapter/adapter.d.ts +0 -18
  676. package/dist/engines/playwright/adapter/attachments.d.ts +0 -26
  677. package/dist/engines/playwright/adapter/config.d.ts +0 -45
  678. package/dist/engines/playwright/adapter/evidence-validation.d.ts +0 -12
  679. package/dist/engines/playwright/adapter/evidence.d.ts +0 -32
  680. package/dist/engines/playwright/adapter/failure-details.d.ts +0 -2
  681. package/dist/engines/playwright/adapter/index.d.ts +0 -3
  682. package/dist/engines/playwright/adapter/reporter.d.ts +0 -12
  683. package/dist/engines/playwright/runtime/config-file.d.ts +0 -2
  684. package/dist/engines/playwright/runtime/config.d.ts +0 -24
  685. package/dist/engines/playwright/runtime/index.d.ts +0 -4
  686. package/dist/engines/playwright/runtime/runtime-input.d.ts +0 -36
  687. package/dist/engines/playwright/runtime/sandbox.d.ts +0 -84
  688. package/dist/engines/runtime-input-file.d.ts +0 -27
  689. package/dist/environment/lifecycle-artifacts.d.ts +0 -7
  690. package/dist/environment/lifecycle-controller.d.ts +0 -70
  691. package/dist/environment/lifecycle.d.ts +0 -29
  692. package/dist/environment/maintenance.d.ts +0 -68
  693. package/dist/environment/planning-types.d.ts +0 -35
  694. package/dist/environment/readiness.d.ts +0 -12
  695. package/dist/environment/readiness.js +0 -1
  696. package/dist/environment/ready-environment.d.ts +0 -19
  697. package/dist/environment/run-environment.d.ts +0 -38
  698. package/dist/environment/run-stopping.d.ts +0 -13
  699. package/dist/environment/runtime-contexts.d.ts +0 -33
  700. package/dist/environment/runtime-manager.d.ts +0 -95
  701. package/dist/environment/runtime-planning.d.ts +0 -16
  702. package/dist/environment/runtime-preparation.d.ts +0 -30
  703. package/dist/environment/runtime-service.d.ts +0 -18
  704. package/dist/environment/services.d.ts +0 -48
  705. package/dist/environment/state.d.ts +0 -18
  706. package/dist/environment/status-runtime.d.ts +0 -53
  707. package/dist/environment/template.d.ts +0 -180
  708. package/dist/evidence/artifact-config.d.ts +0 -14
  709. package/dist/evidence/artifact-validation.d.ts +0 -5
  710. package/dist/evidence/artifacts.d.ts +0 -75
  711. package/dist/evidence/bundles.d.ts +0 -27
  712. package/dist/evidence/ci-provenance.d.ts +0 -9
  713. package/dist/evidence/contracts.d.ts +0 -154
  714. package/dist/evidence/file-evidence.d.ts +0 -23
  715. package/dist/evidence/finalization.d.ts +0 -9
  716. package/dist/evidence/index.d.ts +0 -8
  717. package/dist/evidence/logs.d.ts +0 -31
  718. package/dist/evidence/operation-timings.d.ts +0 -60
  719. package/dist/evidence/read-bundle.d.ts +0 -4
  720. package/dist/evidence/read-fields.d.ts +0 -22
  721. package/dist/evidence/read-provenance.d.ts +0 -2
  722. package/dist/evidence/read-redaction.d.ts +0 -2
  723. package/dist/evidence/read-retention.d.ts +0 -2
  724. package/dist/evidence/read-run.d.ts +0 -2
  725. package/dist/evidence/reading.d.ts +0 -28
  726. package/dist/evidence/reporting-types.d.ts +0 -330
  727. package/dist/evidence/reporting.d.ts +0 -3
  728. package/dist/evidence/secure-file-copy.d.ts +0 -24
  729. package/dist/evidence/setup-progress.d.ts +0 -63
  730. package/dist/evidence/step-output.d.ts +0 -16
  731. package/dist/evidence/storage-inventory.d.ts +0 -27
  732. package/dist/evidence/storage.d.ts +0 -27
  733. package/dist/evidence/timing.d.ts +0 -23
  734. package/dist/file.d.ts +0 -11
  735. package/dist/index.d.ts +0 -4
  736. package/dist/kernel/error-serialization.d.ts +0 -38
  737. package/dist/kernel/redaction.d.ts +0 -2
  738. package/dist/kernel/unknown-values.d.ts +0 -6
  739. package/dist/kernel/work-metrics.d.ts +0 -6
  740. package/dist/process/capture-output.d.ts +0 -7
  741. package/dist/process/command-shell.d.ts +0 -1
  742. package/dist/process/environment-overlay.d.ts +0 -17
  743. package/dist/process/environment.d.ts +0 -33
  744. package/dist/process/files/contained-path.d.ts +0 -8
  745. package/dist/process/files/directory-status.d.ts +0 -14
  746. package/dist/process/files/dotenv.d.ts +0 -2
  747. package/dist/process/files/file-snapshot.d.ts +0 -34
  748. package/dist/process/files/json-file.d.ts +0 -12
  749. package/dist/process/files/paths.d.ts +0 -1
  750. package/dist/process/files/product-dir.d.ts +0 -1
  751. package/dist/process/files/temp-dir.d.ts +0 -11
  752. package/dist/process/host/metadata.d.ts +0 -6
  753. package/dist/process/host/platform.d.ts +0 -57
  754. package/dist/process/host-command.d.ts +0 -4
  755. package/dist/process/host-inspection-command.d.ts +0 -13
  756. package/dist/process/host-process.d.ts +0 -19
  757. package/dist/process/index.d.ts +0 -9
  758. package/dist/process/lifecycle-fence.d.ts +0 -11
  759. package/dist/process/locks.d.ts +0 -28
  760. package/dist/process/process-contracts.d.ts +0 -78
  761. package/dist/process/process-group.d.ts +0 -12
  762. package/dist/process/process-identity.d.ts +0 -27
  763. package/dist/process/process-limits.d.ts +0 -6
  764. package/dist/process/process.d.ts +0 -12
  765. package/dist/process/readiness.d.ts +0 -37
  766. package/dist/process/recovery.d.ts +0 -3
  767. package/dist/process/runs/inspection.d.ts +0 -5
  768. package/dist/process/runs/manifest.d.ts +0 -86
  769. package/dist/process/runs/ownership.d.ts +0 -11
  770. package/dist/process/runs/product-state.d.ts +0 -2
  771. package/dist/process/runs/schema.d.ts +0 -3
  772. package/dist/process/runs/stop.d.ts +0 -19
  773. package/dist/process/runtime-socket.d.ts +0 -4
  774. package/dist/process/service-processes.d.ts +0 -13
  775. package/dist/process/shutdown.d.ts +0 -10
  776. package/dist/process/windows-cli-owner.d.ts +0 -2
  777. package/dist/process/windows-native.d.ts +0 -6
  778. package/dist/process/windows-process-identity.d.ts +0 -4
  779. package/dist/repo/ci-environment.d.ts +0 -7
  780. package/dist/repo/fingerprint-policy.d.ts +0 -15
  781. package/dist/repo/git-path.d.ts +0 -2
  782. package/dist/repo/history.d.ts +0 -17
  783. package/dist/repo/managed-git.d.ts +0 -10
  784. package/dist/repo/metadata.d.ts +0 -3
  785. package/dist/repo/physical-tree.d.ts +0 -37
  786. package/dist/repo/remote.d.ts +0 -6
  787. package/dist/repo/state.d.ts +0 -26
  788. package/dist/repo/tree-listing.d.ts +0 -10
  789. package/dist/steps/build-config.d.ts +0 -35
  790. package/dist/steps/configured-step-testkit-modules.d.ts +0 -19
  791. package/dist/steps/configured-steps.d.ts +0 -51
  792. package/dist/steps/environment.d.ts +0 -84
  793. package/dist/steps/index.d.ts +0 -2
  794. package/dist/steps/module-runner.d.ts +0 -1
  795. package/dist/steps/run.d.ts +0 -36
  796. package/dist/toolchains/archive.d.ts +0 -53
  797. package/dist/toolchains/esbuild.d.ts +0 -9
  798. package/dist/toolchains/host-node-support.d.ts +0 -3
  799. package/dist/toolchains/index.d.ts +0 -114
  800. package/dist/toolchains/managed-node-releases.d.ts +0 -28
  801. package/dist/toolchains/node-cache.d.ts +0 -20
  802. package/dist/toolchains/node-integrity.d.ts +0 -18
  803. package/dist/toolchains/node-provisioning.d.ts +0 -29
  804. package/dist/toolchains/semver.d.ts +0 -13
  805. package/dist/toolchains/shared-operation.d.ts +0 -11
  806. package/dist/toolchains/timeouts.d.ts +0 -1
  807. package/dist/toolchains/vitest/index.d.ts +0 -13
  808. package/dist/tracker/gitea.d.ts +0 -14
  809. package/dist/tracker/github.d.ts +0 -14
  810. package/dist/tracker/http.d.ts +0 -33
  811. package/dist/tracker/index.d.ts +0 -4
  812. package/dist/tracker/redaction.d.ts +0 -4
  813. package/dist/tracker/requests.d.ts +0 -17
  814. package/dist/tracker/responses.d.ts +0 -24
  815. package/dist/tracker/types.d.ts +0 -51
  816. package/docs/engine-architecture.md +0 -654
  817. /package/dist/engines/{k6/runtime/runtime-globals.js → api/profiles.js} +0 -0
package/README.md CHANGED
@@ -4,17 +4,19 @@
4
4
 
5
5
  # @gfdelarue/testkit
6
6
 
7
- `@gfdelarue/testkit` is the orchestrator and managed distribution for product test
8
- suites. It discovers `*.testkit.ts` files, infers suite ownership from the
9
- filesystem, starts local services, provisions Docker-managed local Postgres
10
- databases, schedules suites, and collects results and artifacts. Coding agents
11
- use it to write tests and hunt bugs, and CI uses it to run those tests. It needs
12
- no account or server.
7
+ `@gfdelarue/testkit` is a test runner and CLI for coding agents and CI. It runs
8
+ a repository's ordinary Vitest test files, such as `src/sum.test.ts`, as they
9
+ are, and typed `*.testkit.ts` suites for Playwright and Node. It assigns
10
+ tests to groups by directory, starts only the local services a group uses,
11
+ provisions Docker-managed local Postgres databases when a group needs one,
12
+ schedules the files, and collects results and artifacts. It starts no service
13
+ for a test whose group uses none. Coding agents use Testkit to write tests
14
+ and hunt bugs, and CI uses it to run those tests. It needs no account or
15
+ server.
13
16
 
14
17
  Testkit exists to replace false confidence with inspectable execution evidence.
15
- It cannot prove that an entire application works. It can make narrower claims
16
- that deserve trust: these tests genuinely executed; these were the runtimes,
17
- fixtures, and inputs supplied to the run; this is what passed, failed, was
18
+ It makes claims that deserve trust: these tests genuinely executed; these were
19
+ the runtimes, fixtures, and inputs supplied to the run; this is what passed, failed, was
18
20
  skipped, or did not run; and this is the evidence behind the result. Testkit
19
21
  retains provenance, logs, and verifiable artifacts so those claims can be
20
22
  audited rather than accepted on the strength of a green check mark.
@@ -111,6 +113,23 @@ only when the test fails. It stops when tracked files have uncommitted changes,
111
113
  when no remote branch contains the commit, or when an open issue already covers
112
114
  the test. The token needs permission to create issues and labels.
113
115
 
116
+ `testkit bug record` saves the `--vitest-config`, `--vitest-project`, and
117
+ `--vitest-runner` it was given in the draft, and asks for `--vitest-project`
118
+ when two Vitest projects run the test's file. `testkit bug file` reruns with
119
+ them, and the issue's reproduce command carries them. With
120
+ `TESTKIT_REQUIRE_COMPLETE_SUITE=1`, it reruns every file the selection lists,
121
+ then reads the recorded failure.
122
+
123
+ `testkit bug file` takes the same [supplied values](#run-against-a-supplied-environment)
124
+ as `testkit run` and reruns the test on them. The issue's reproduce command
125
+ carries those flags, so `testkit bug file` refuses inputs that would put a raw
126
+ secret in it. It takes `--env-from` and `--environment-file`, and takes
127
+ `--base-url` and `--endpoint` only for URLs without credentials, a query, or a
128
+ fragment. It refuses `--database` and `--env`; pass those values through
129
+ `--env-from` or the environment file. The command runs from the repository
130
+ root, so a relative `--environment-file` needs `testkit bug file` run from
131
+ there too.
132
+
114
133
  ### Name the product's areas
115
134
 
116
135
  Add `areas` to `testkit.config.ts` so that each issue names the part of the
@@ -174,30 +193,97 @@ claims, how to work, and the bug record format. Give that output to the agent.
174
193
  summary for the pull request that adds the tests to
175
194
  `.testkit/campaigns/<id>/summary.md` and prints it.
176
195
 
196
+ ## Prepare and publish a report bundle
197
+
198
+ Prepare a new directory from your own template, recorded bug IDs, and selected
199
+ run artifacts. All inputs are optional. Repeat `--bug` for related findings and
200
+ `--artifact` for supporting files.
201
+
202
+ ```bash
203
+ npx testkit artifacts --run <run-id> --json
204
+ npx testkit report prepare ./draft-report \
205
+ --template ./report-template.md \
206
+ --instructions ./reporting-instructions.md \
207
+ --bug <bug-id> --run <run-id> --artifact <ref.path>
208
+ ```
209
+
210
+ Preparation copies the template unchanged to `report.md`, or creates an empty
211
+ file. It puts recorded facts in `context.json` and selected, verified run files
212
+ under `evidence/`. Drafting instructions are printed for the author and are not
213
+ copied into the bundle. Write and review the narrative yourself. Templates have
214
+ no special syntax, and preparation never overwrites an existing directory.
215
+
216
+ Publish that directory, or any directory you prepared yourself, to an explicit
217
+ repository, branch, and bundle path:
218
+
219
+ ```bash
220
+ npx testkit report publish ./draft-report \
221
+ --repository https://github.com/your-team/private-reports \
222
+ --branch main --path reports/refund-race --json
223
+ ```
224
+
225
+ Use a full Gitea repository URL to publish there. `owner/name` uses the host of
226
+ the checkout's origin. Authentication uses the same environment variables and
227
+ Git credential helper as `testkit bug file`; the token needs repository write
228
+ access. Both commands support `--dir` to select the Testkit checkout and
229
+ `--json` for structured output. Local input paths are relative to your current
230
+ directory. Publication needs Git installed and works without Testkit config.
231
+
232
+ Publication replaces the chosen bundle directory, including removing files
233
+ that are no longer in your local bundle. It preserves other repository paths
234
+ and leaves your checkout, index, and branch untouched. The destination branch
235
+ must exist unless the repository is empty. A concurrent branch update rejects
236
+ the push; review the new branch state before trying again. Fetching requests
237
+ a blob-free snapshot and monitors temporary Git storage against a 256 MiB
238
+ budget, with possible brief overshoot between checks. Use a smaller reports
239
+ repository if it exceeds that budget. An unchanged bundle
240
+ returns its existing revision without creating a commit.
241
+
242
+ The result includes the exact `revision` and a provider `url` pinned to it.
243
+ `--message` sets the commit message. Commits use the standard `GIT_AUTHOR_*` and
244
+ `GIT_COMMITTER_*` identity variables when set, otherwise `Testkit
245
+ <testkit@localhost>`. Keep the revision used for a submission. A failed push can
246
+ have an uncertain result after a lost connection, so check the reported commit
247
+ at the destination before retrying.
248
+
249
+ Nested text and binary files keep their bytes and names, without line-ending
250
+ conversion or Git LFS filters. Publication accepts ordinary files up to 16 MiB
251
+ each, 64 MiB total, and 1000 files. Directory depth is limited to 32, with at
252
+ most 128 directories including the repository root and destination parents.
253
+ Symlinks, Git metadata, traversal paths, and
254
+ filenames containing backslashes or line breaks are rejected. Filenames must
255
+ be UTF-8. Empty directories are not stored by Git. Review everything in the
256
+ selected directory before publishing it, including hidden files.
257
+
258
+ Neither command needs Turnstone, a paid feature, or a fixed report format.
259
+ To view the result in Turnstone, connect the destination repository and select
260
+ the same branch and the parent folder, such as `reports`. The author submits
261
+ the report to the chosen platform.
262
+
177
263
  ## Execution engines
178
264
 
179
- Testkit has three explicit built-in engines rather than one implicit runtime:
180
-
181
- - `k6` runs `int`, `e2e`, `load`, and `dal` files with native k6
182
- and xk6-sql semantics.
183
- - `node-vitest` runs purpose-built `data` and `contract` files with Node and
184
- Vitest.
185
- - `playwright` runs `ui` files with Playwright and a managed Chromium.
186
-
187
- Every test file must export `defineFile({ engine: "..." })`. The filename
188
- suffix supplies the public test type; Testkit validates that explicit pair and
189
- derives a runner profile. It never infers a default engine or falls back to a
190
- different engine.
191
-
192
- Consumers declare only `@gfdelarue/testkit`. The root package owns the three
193
- adapters and exact JavaScript tools it executes. The k6 engine ships as
194
- separately versioned npm packages built from
195
- [gfdelarue/testkit-k6](https://github.com/gfdelarue/testkit-k6); npm installs
196
- the compatible k6 target package and its shared source package through the
197
- root's optional dependency graph, and Testkit resolves that direct executable
198
- without consulting `PATH`. Node/Vitest and Playwright resolve their managed public packages from the
199
- installed Testkit root, so product-local tool copies do not select Testkit's
200
- runtime.
265
+ Testkit has two built-in engines:
266
+
267
+ - `node-vitest` runs plain Vitest test files and purpose-built `data` and
268
+ `contract` files with Node and Vitest.
269
+ - `playwright` runs `ui` files with Playwright and managed Chromium.
270
+
271
+ A plain Vitest test file needs no Testkit suffix, directory, metadata, or
272
+ wrapper. It keeps its native `vitest` imports and its repository paths, has no
273
+ test type, and runs under the engine's general `node-vitest` profile. Testkit
274
+ does not evaluate `testkit.config.ts` in the test's process, and it installs the
275
+ values of the services the test's group uses before the test's imports
276
+ evaluate. See [Plain Vitest test files](#plain-vitest-test-files) for which
277
+ files Testkit runs.
278
+
279
+ Every typed `*.testkit.ts` file must export `defineFile({ engine: "..." })`. The
280
+ filename suffix supplies the public test type; Testkit validates that explicit
281
+ pair and derives a runner profile. It never infers a default engine for a typed
282
+ file or falls back to a different engine.
283
+
284
+ Consumers declare only `@gfdelarue/testkit`, which installs the exact versions
285
+ of the tools its engines run. A product's own copies of Vitest, Playwright, or
286
+ their tools don't change what Testkit runs.
201
287
 
202
288
  Playwright browser installation is explicit:
203
289
 
@@ -206,26 +292,16 @@ npx @gfdelarue/testkit setup playwright
206
292
  npx @gfdelarue/testkit setup playwright --with-deps # CI and fresh hosts
207
293
  ```
208
294
 
209
- Setup and execution share one canonical Playwright registry. Normal runs never
210
- install or download a browser. See [Engine Architecture](docs/engine-architecture.md)
211
- for the authoring, installation, trust, and failure contracts.
212
-
213
- Managed commands use the same Execa-backed adapter on hosted Linux and Windows
214
- runners. Testkit passes exact argv, cwd, and environment values, bounds captured
215
- output, applies cancellation and timeouts, and asks the platform adapter to
216
- terminate descendants. Persisted recovery identities remain birth-fenced so a
217
- later cleanup does not signal a reused process ID.
218
-
219
- The five k6 target packages share one exact-version source package that carries
220
- the AGPL corresponding source archive and licenses. Runtime evidence records
221
- the package name and version and the k6 version observed by executing the
222
- installed binary. The
223
- [testkit-k6 README](https://github.com/gfdelarue/testkit-k6#readme) documents
224
- the reproducible build, the expected executable digests, and the license
225
- notes.
226
-
227
- Managed task environment is engine input rather than ambient host state. k6
228
- suites read it through native `__ENV`; Node/Vitest data and contract callbacks
295
+ Normal runs never install or download a browser.
296
+ [Installation](docs/engines.md#installation) covers what an install contains,
297
+ the supported platforms, and how to recover a broken one.
298
+ [Typechecking](docs/engines.md#typechecking) covers the supported TypeScript
299
+ versions and `tsconfig.playwright.json`.
300
+
301
+ Testkit bounds the output it captures from each command it runs, and asks the
302
+ command and its descendants to stop on cancellation or timeout.
303
+
304
+ Managed task environment is engine input. Node/Vitest data and contract callbacks
229
305
  read immutable, enumerable `context.env`; and Playwright suites can read the
230
306
  same contract from `readPlaywrightContext()?.env`. Managed Node processes also
231
307
  expose each value as an immutable, non-enumerable `process.env.KEY` property so
@@ -237,21 +313,21 @@ reserves `APPDATA`, `ComSpec`, `HOME`, `LANG`, `LC_ALL`, `LOCALAPPDATA`, `PATH`,
237
313
  `PATHEXT`, `SystemRoot`, `TEMP`, `TMP`, `TMPDIR`, `USERPROFILE`,
238
314
  `XDG_CACHE_HOME`, `XDG_CONFIG_HOME`, and `XDG_DATA_HOME`. In addition:
239
315
 
240
- - k6 reserves the `K6_` and `TESTKIT_` prefixes plus `BASE_URL` and `GODEBUG`;
241
- - Node/Vitest reserves the `TESTKIT_` and `VITEST_` prefixes plus `BASE_URL`,
242
- `FORCE_COLOR`, `NODE_OPTIONS`, `NO_COLOR`, and `UPDATE_SNAPSHOT`;
316
+ - Node/Vitest reserves `BASE_URL`, `FORCE_COLOR`, `NODE_OPTIONS`, `NO_COLOR`,
317
+ `UPDATE_SNAPSHOT`, `VITEST_POOL_ID`, `VITEST_WORKER_ID`, and the `TESTKIT_`
318
+ keys it sets itself, listed in
319
+ [Environment](docs/engines.md#environment);
320
+ other `TESTKIT_` and `VITEST_` keys, such as
321
+ `TESTKIT_REQUIRE_COMPLETE_SUITE`, pass through;
243
322
  - Playwright reserves the `TESTKIT_`, `PLAYWRIGHT_`, `PWTEST_`, and `PW_TEST_`
244
323
  prefixes plus `BASE_URL`, `FORCE_COLOR`, `NODE_OPTIONS`, and `NO_COLOR`.
245
324
 
246
325
  Case-variant duplicates and invalid names or NUL-containing values fail instead
247
- of being normalized. k6 accepts at most 128 consumer entries and 16,384
248
- environment-block character units. Node/Vitest accepts at most 512 entries and
326
+ of being normalized. Node/Vitest accepts at most 512 entries and
249
327
  4 MiB of UTF-8 key/value data. Playwright accepts at most 512 entries and 2 MiB
250
- of UTF-8 key/value data. The smaller k6 limit is required because native k6
251
- receives consumer values in its physical process environment; Node/Vitest and
252
- Playwright transport them through owned runtime-input files. See
253
- [Engine Architecture](docs/engine-architecture.md#managed-environment) for the
254
- complete transport and aggregate-input limits.
328
+ of UTF-8 key/value data. [Environment](docs/engines.md#environment) lists the
329
+ complete limits, and [Named endpoints and databases](#named-endpoints-and-databases)
330
+ covers the environment API.
255
331
 
256
332
  ## Usage
257
333
 
@@ -268,10 +344,9 @@ npx @gfdelarue/testkit discover --json > .testkit/discovery.json
268
344
  npx @gfdelarue/testkit discover --output .testkit/discovery.json
269
345
 
270
346
  # Filter by type
271
- npx @gfdelarue/testkit --type int
272
- npx @gfdelarue/testkit --type dal
273
- npx @gfdelarue/testkit --type e2e
274
- npx @gfdelarue/testkit --type int,e2e,dal
347
+ npx @gfdelarue/testkit --type contract
348
+ npx @gfdelarue/testkit --type data
349
+ npx @gfdelarue/testkit --type contract,data
275
350
  npx @gfdelarue/testkit --type ui
276
351
 
277
352
  # Parallel file execution
@@ -280,16 +355,17 @@ npx @gfdelarue/testkit --task-concurrency 8
280
355
  # One file-level wall clock budget for every suite file
281
356
  npx @gfdelarue/testkit --file-timeout-seconds 60
282
357
 
283
- # Specific service / suite
284
- npx @gfdelarue/testkit --service frontend --type ui -s navigation
285
- npx @gfdelarue/testkit --service api --type int -s health
286
- npx @gfdelarue/testkit --type int,e2e,dal -s dal:queries
358
+ # Specific group / suite
359
+ npx @gfdelarue/testkit --group frontend --type ui -s navigation
360
+ npx @gfdelarue/testkit --group api --type contract -s health
361
+ npx @gfdelarue/testkit --type contract,data -s contract:queries
287
362
 
288
363
  # Exact file
289
- npx @gfdelarue/testkit --type int --file __testkit__/health.int.testkit.ts
364
+ npx @gfdelarue/testkit --type contract --file __testkit__/health.contract.testkit.ts
365
+ npx @gfdelarue/testkit run --file src/sum.test.ts
290
366
 
291
367
  # Temporarily ignore repo-declared skip rules
292
- npx @gfdelarue/testkit --ignore-skip-rules --file __testkit__/billing.int.testkit.ts
368
+ npx @gfdelarue/testkit --ignore-skip-rules --file __testkit__/billing.contract.testkit.ts
293
369
 
294
370
  # Lifecycle
295
371
  npx @gfdelarue/testkit status
@@ -304,6 +380,16 @@ npx @gfdelarue/testkit db schema refresh --service api
304
380
  npx @gfdelarue/testkit db schema verify --service api
305
381
  ```
306
382
 
383
+ A `--type` that names test types, such as `--type contract,ui`, and typed suite
384
+ selectors such as `contract:health` select typed files only. A run without them, or
385
+ with `--type all`, also runs plain Vitest test files. To run every typed file
386
+ and no plain one, name all three types: `--type ui,contract,data`.
387
+
388
+ `run`, `discover`, `artifacts`, and `workspace` select test groups with
389
+ `--group`. On those commands `--service` is a deprecated alias for `--group`
390
+ and prints a notice on stderr. `status`, `destroy`, `cleanup`, and `db schema`
391
+ take `--service` to name a service.
392
+
307
393
  Database cleanup is scoped to the selected product. `cleanup --resources --global`
308
394
  is retired; use `--dir <product> cleanup --resources` for each product. After
309
395
  deleting a product directory, remove its orphaned containers and volumes
@@ -319,15 +405,125 @@ Batch `run` output stays intentionally short: one line per completed file,
319
405
  concise failure blocks, and a final summary. Service logs, captured runtime
320
406
  output, and emitted artifacts are persisted under `.testkit/results/`.
321
407
 
408
+ Text output is coloured in a terminal and plain otherwise. As in Node,
409
+ `NO_COLOR` turns colour off and `FORCE_COLOR` overrides it: `0` or `false`
410
+ turns colour off and any other value turns it on.
411
+
322
412
  Testkit seals an immutable local run bundle. `testkit run` prints
323
413
  `Run ID: <id>` after sealing in text output and returns `runId` in `--json`
324
414
  output, for passed and failed runs.
325
415
 
326
- Test execution also maintains a scheduler cache at `.testkit/timings.json`.
327
- Completed file-level task durations are used to rank future runs with a
328
- longest-estimated-duration-first policy, so slow files start earlier when
329
- workers are available. Run artifacts include compact scheduler metadata under
330
- `planning` so ordering decisions are inspectable.
416
+ Testkit starts the files that took longest in earlier runs first, so slow
417
+ files start while workers are free. The run document records the order under
418
+ `planning`.
419
+
420
+ ### Run against a supplied environment
421
+
422
+ `testkit run` can take the values its tests need instead of provisioning them.
423
+ Every selected group then runs on those values alone. Testkit starts none of
424
+ the group's services, creates no database, and runs no `.setup.*` module beside
425
+ a test. It also ignores the env values and env files those services declare, so
426
+ supply every value the tests read. Groups, selection, skip rules, and other
427
+ policy work as before.
428
+
429
+ ```bash
430
+ npx @gfdelarue/testkit run --group checks \
431
+ --base-url http://127.0.0.1:3000 \
432
+ --endpoint api=http://127.0.0.1:3001 \
433
+ --database store=postgres://app:password@127.0.0.1:5432/store \
434
+ --env FEATURE_FLAG=on \
435
+ --env-from API_TOKEN=CI_API_TOKEN
436
+
437
+ npx @gfdelarue/testkit run --environment-file supplied.json
438
+ ```
439
+
440
+ - `--base-url URL` supplies the default endpoint, which tests receive as
441
+ `BASE_URL`.
442
+ - `--endpoint NAME=URL` supplies a named endpoint, usually named after the
443
+ service it stands in for.
444
+ - `--database NAME=URL` supplies a named PostgreSQL database.
445
+ - `--env NAME=VALUE` supplies an environment value.
446
+ - `--env-from NAME=HOST_KEY` supplies the value of host variable `HOST_KEY`
447
+ under `NAME`.
448
+ - `--environment-file PATH` reads values, prepared setup data, and handle
449
+ descriptors from a file.
450
+
451
+ Every flag except `--base-url` and `--environment-file` can be repeated. Tests
452
+ read named values with `endpoint(name)` and `database(name)` from
453
+ `@gfdelarue/testkit/environment`. An `--env`, `--env-from`, or file `env` entry
454
+ named `BASE_URL` or `DATABASE_URL` sets the default endpoint or database, and an
455
+ empty database URL means no database. Without an explicit default, a group's
456
+ `BASE_URL` and `DATABASE_URL` are those of the first service in its `uses` that
457
+ has a supplied endpoint or database. Either way the default is also available
458
+ by the group's name, as in a configured run, so it may not differ from a named
459
+ endpoint or database with that name.
460
+
461
+ Testkit reads host variables only through `--env-from`. A `BASE_URL` or
462
+ `DATABASE_URL` already set in the shell does not switch a run to supplied
463
+ values. Two inputs may define the same value only if they agree. Any other
464
+ repeat, such as `--base-url` and `--env BASE_URL` with different URLs, or a
465
+ missing `--env-from` host variable, fails the run before it starts. Before a
466
+ test starts, each supplied endpoint must give a valid HTTP response. A 4xx or
467
+ 5xx response counts as valid.
468
+
469
+ Each supplied database must also accept a connection before a test starts.
470
+ Testkit connects, signs in with the URL's credentials, runs `SELECT 1`, and
471
+ disconnects. It handles trust, password, md5, and SCRAM-SHA-256 authentication.
472
+ TLS follows the URL's `sslmode` as in libpq. The default, `prefer`, tries TLS
473
+ first and then a plain connection, and `allow` tries them the other way round.
474
+ `verify-ca` and `verify-full` need `sslrootcert`, set to a CA file or to
475
+ `system`, and `sslrootcert=system` alone means `verify-full`. `host=/path`
476
+ connects through the Unix socket in that directory and ignores `sslmode`. The
477
+ check reads only the URL. It ignores `PG*` variables, `~/.pgpass`, and
478
+ `~/.postgresql/root.crt`, and it presents no client certificate. A database
479
+ that is unreachable, fails TLS, rejects the credentials, does not exist, or is
480
+ not PostgreSQL fails the run before any test starts. The message names the
481
+ database, never its URL.
482
+
483
+ The environment file is JSON:
484
+
485
+ ```json
486
+ {
487
+ "version": 1,
488
+ "baseUrl": "http://127.0.0.1:3000",
489
+ "databaseUrl": "postgres://app:password@127.0.0.1:5432/app",
490
+ "endpoints": { "api": "http://127.0.0.1:3001" },
491
+ "databases": { "store": "postgres://app:password@127.0.0.1:5432/store" },
492
+ "env": { "FEATURE_FLAG": "on" },
493
+ "ui": {},
494
+ "setupData": { "actor": "Grace" },
495
+ "handles": {
496
+ "webhook": {
497
+ "kind": "http-capture",
498
+ "url": "http://127.0.0.1:41000/webhook",
499
+ "requestsUrl": "http://127.0.0.1:41000/__testkit__/<capability>/requests",
500
+ "resetUrl": "http://127.0.0.1:41000/__testkit__/<capability>/requests"
501
+ }
502
+ }
503
+ }
504
+ ```
505
+
506
+ Only `version` is required, and this Testkit reads version 1. `baseUrl`,
507
+ `databaseUrl`, and `ui` may be `null`, as an environment rendered by Testkit
508
+ writes them, and `null` there counts as absent. Tests receive
509
+ `setupData` as the data a setup module would have returned: `{}` when the file
510
+ leaves it out, and `null` when the file sets it to `null`. `handles` entries must each name a `kind`, and tests reach them with
511
+ [`handle(name)`](#handles). An `http-capture` entry holds the `url`,
512
+ `requestsUrl`, and `resetUrl` that a capture sidecar reports, and gives the
513
+ test the same handle a setup module's capture would. Other kinds are handed on,
514
+ and `handle()` throws `EnvironmentHandleError` with `"unsupported"` for them. Any other field, such as
515
+ service or database configuration, is rejected. Errors name the file and the
516
+ field and do not repeat the file's contents.
517
+
518
+ A run records the values its tests received in this shape. Each acquisition in
519
+ `run.json`'s `environment` section has them under `values`, with secrets and
520
+ capture capabilities redacted. Its `setupData` is the supplied setup data or
521
+ what the file's setup module returned, and `{}` without either. Sealing redacts
522
+ setup data under a key that names a secret or that `redact` lists, and records
523
+ the rest as returned, so list any other sensitive key in `redact`. With
524
+ `"version": 1` added, an
525
+ acquisition's `values` are an environment file that gives a second run the
526
+ same values, unless redaction changed one of them.
331
527
 
332
528
  ## Tooling Adapters
333
529
 
@@ -363,23 +559,17 @@ export default defineConfig({
363
559
  // playwright.config.ts
364
560
  import { defineTestkitConfig, devices } from "@gfdelarue/testkit/playwright";
365
561
 
366
- export default defineTestkitConfig(
367
- {
368
- testDir: "./__testkit__",
369
- projects: [{ name: "chromium", use: { ...devices["Desktop Chrome"] } }],
370
- },
371
- {
372
- dotenvFiles: [".env.local"],
373
- },
374
- );
562
+ export default defineTestkitConfig({
563
+ testDir: "./__testkit__",
564
+ projects: [{ name: "chromium", use: { ...devices["Desktop Chrome"] } }],
565
+ });
375
566
  ```
376
567
 
377
568
  Managed UI suites use `@gfdelarue/testkit/playwright` for Testkit's Playwright
378
569
  helpers and re-exported authoring API. They may also import exact public exports
379
570
  from the managed `@playwright/test`, `playwright`, and `playwright-core`
380
- packages. Testkit pins both runtime resolution and type resolution, so
381
- product-local Playwright installs cannot split the test registry or supply a
382
- different compile-time API.
571
+ packages. Those imports resolve to Testkit's pinned versions at run time and
572
+ when typechecking, whatever Playwright version the product installs.
383
573
 
384
574
  For scripts and app-runtime code, `@gfdelarue/testkit/env` provides shared helpers
385
575
  for managed runtime detection, dotenv loading, and local-database safety:
@@ -428,7 +618,6 @@ export default defineConfig({
428
618
  api: app.node({
429
619
  cwd: ".",
430
620
  entry: "src/index.ts",
431
- port: 3004,
432
621
  envFiles: [".env.testkit"],
433
622
  database: database.postgres({
434
623
  runtimeConnections: 20,
@@ -448,7 +637,6 @@ export default defineConfig({
448
637
  frontend: app.next({
449
638
  cwd: "frontend",
450
639
  mode: "start",
451
- port: 3000,
452
640
  dependsOn: ["api"],
453
641
  envFiles: ["frontend/.env.testkit"],
454
642
  env: {
@@ -466,28 +654,48 @@ export default defineConfig({
466
654
  });
467
655
  ```
468
656
 
469
- Every test file declares its execution engine and any file-local scheduling
470
- metadata next to the test:
657
+ Every service Testkit starts listens on a port the OS assigns for that run.
658
+ Testkit passes it to the service as `PORT` and as `{port}` in `start`, `env`,
659
+ `baseUrl`, and `readyUrl`, so two runs of the same repository on one host, such
660
+ as two CI jobs on a runner or two agent worktrees, never compete for a port. A
661
+ service that exits before it becomes ready gets a fresh port and up to three
662
+ attempts in total. Each run's `run.json` lists every service's actual URL, port,
663
+ and attempts under `environment.services`. A config that sets `local.port`, the
664
+ `port` option of `app.node` or `app.next`, or a literal port in `local.baseUrl`
665
+ or `local.readyUrl` fails to load; delete it and use `{port}`.
666
+
667
+ Every typed test file declares its execution engine and any file-local
668
+ scheduling metadata next to the test. A plain Vitest test file declares
669
+ neither:
471
670
 
472
671
  ```ts
473
672
  import { defineFile } from "@gfdelarue/testkit";
474
673
 
475
674
  export const testkit = defineFile({
476
- engine: "k6",
675
+ engine: "node-vitest",
477
676
  skip: "Billing is currently unavailable locally",
478
- locks: ["global-worker-loop"],
479
677
  });
480
678
  ```
481
679
 
482
- The engine value is a required literal. The supported public authoring paths
483
- are `@gfdelarue/testkit/k6`, `@gfdelarue/testkit/node-vitest/data`,
680
+ A file can also name `locks`.
681
+ [Give each test its own data](#give-each-test-its-own-data) says when to use
682
+ one.
683
+
684
+ In a typed file, the engine value is a required literal. The supported public
685
+ authoring paths
686
+ are `@gfdelarue/testkit/node-vitest/data`,
484
687
  `@gfdelarue/testkit/node-vitest/contract`, and `@gfdelarue/testkit/playwright`.
485
688
  Node/Vitest and Playwright suites may also import exact public APIs directly
486
689
  from Testkit's managed `vitest`, `@playwright/test`, `playwright`, and
487
690
  `playwright-core` packages.
488
- There are no legacy aliases or implicit-runner compatibility paths.
489
-
490
- `testkit.config.ts` is optional for simple repos, but it is the primary escape hatch
691
+ There are no legacy aliases or implicit-runner compatibility paths. A typed
692
+ Node/Vitest file resolves `paths` from its nearest `tsconfig.json`, through
693
+ Vite's `resolve.tsconfigPaths`.
694
+
695
+ `testkit.config.ts` is optional. Without it, Testkit runs every test file with
696
+ zero setup: it starts no server, provisions no database, and loads no `.env`
697
+ file. A test that needs a server or a database gets one from a service that
698
+ `testkit.config.ts` declares. The config is also the primary escape hatch
491
699
  for:
492
700
 
493
701
  - worker count and per-file runtime budget
@@ -506,10 +714,9 @@ for:
506
714
  - repo-declared suite/file skip policies with explicit reasons
507
715
 
508
716
  `runtime.prepare` is the generic build-once hook for shared runtimes. It runs
509
- once per runtime generation before local services start, fingerprints declared
510
- inputs, and writes cache state under the service runtime directory. This is the
511
- right way to move expensive browser targets from `next dev` / watch mode to
512
- stable build-and-start flows.
717
+ once per runtime generation before local services start, and runs again when
718
+ its declared inputs change. This is the right way to move expensive browser
719
+ targets from `next dev` / watch mode to stable build-and-start flows.
513
720
 
514
721
  Setup steps (`runtime.prepare`, database `template`, and other Testkit-run
515
722
  Node commands) inherit `NODE_OPTIONS`. `TESTKIT_SETUP_NODE_OPTIONS` appends
@@ -521,18 +728,12 @@ Testkit-owned Docker resources are labelled at creation time so they can be
521
728
  found again even if product-local `.testkit` state is deleted. `testkit destroy`
522
729
  removes labelled resources for the current product after service state is
523
730
  destroyed. `testkit cleanup --resources` cleans stale resources for the current
524
- product. Both commands filter Docker inventory by the product ownership label
525
- and preserve other products, including idle caches. Host-local Postgres data
526
- lives in a Testkit-labelled named Docker
527
- volume instead of an image-declared anonymous volume, so cleanup can account for
528
- and remove it safely. Each created volume receives a never-reused random
529
- generation in both its name and labels; cleanup re-inspects the complete
530
- generation immediately before Docker's name-based removal. Docker has no
531
- immutable volume ID or conditional-delete API, so direct Docker-daemon
532
- principals are part of the host trust boundary and must not replace that
533
- unguessable name during the final removal call. Testkit serializes database
534
- mutations for the same product; independent products can proceed concurrently.
535
- Use `cleanup --resources --dry-run` to preview the selected product's cleanup.
731
+ product. Both commands remove only the current product's resources and leave
732
+ other products' alone, including their idle caches. Host-local Postgres data
733
+ lives in a Testkit-labelled named Docker volume, so cleanup can find and remove
734
+ it. Database changes for one product run one at a time; other products don't
735
+ wait for them. Use `cleanup --resources --dry-run` to preview the selected
736
+ product's cleanup.
536
737
 
537
738
  `database.template` is the database-side equivalent for reusable template DB
538
739
  state. When `database.sourceSchema` is configured, Testkit treats the configured
@@ -544,34 +745,16 @@ schema still matches the source dump. If local replay differs, Testkit refreshes
544
745
  from the source once for the current cache key and retries. If it still differs,
545
746
  the run fails with schema diagnostics under `.testkit/results/schema`.
546
747
 
547
- Source schema cache keys are derived automatically from repo state:
548
-
549
- - clean git worktrees use `commits/<sha>`
550
- - dirty git worktrees use `dirty/<sha>-<fingerprint>`
551
- - non-git directories use `nogit/<fingerprint>`
552
-
553
- Dirty worktree fingerprints compare the complete index to `HEAD`, then compare
554
- the physical worktree bytes, node types, and executable modes to the index.
555
- This intentionally detects changes hidden by Git stat flags, sparse checkout,
556
- or clean/smudge and EOL normalization. For untracked files, repository
557
- `.gitignore` files are respected; host-global excludes, `core.excludesFile`, and
558
- `.git/info/exclude` do not alter Testkit's cache identity.
559
-
560
- Fingerprint Git commands disable replacement objects and optional locks and do
561
- not read system or user Git configuration. Git path records are decoded from
562
- their byte-preserving quoted form and must be valid UTF-8. Uninitialized
563
- gitlinks and special filesystem nodes fail explicitly instead of being assigned
564
- an incomplete or ambiguous identity. Testkit reads the complete HEAD, index,
565
- untracked, submodule, and physical snapshot twice and fails if the observations
566
- differ.
567
-
568
- Physical reads open regular files by descriptor and verify every pathname
569
- component before and after use; on POSIX, ancestor directory descriptors remain
570
- open during the read. Node does not expose a portable `openat`/Windows reparse-
571
- handle traversal API, so this detects ordinary replacement races but is not a
572
- security boundary against a same-user adversary able to perform a perfectly
573
- timed replace-and-restore ABA attack. Run Testkit against source trees that are
574
- not concurrently controlled by an untrusted local process.
748
+ Testkit keys the source schema cache on the repository's state. A clean
749
+ worktree shares the entry for its commit. A dirty worktree or a directory
750
+ outside Git gets its own entry, keyed on the content of its files.
751
+
752
+ The key changes when a file's bytes, type, or executable mode changes, including
753
+ changes Git's own status can hide, such as sparse checkout or line-ending
754
+ normalization, and when a submodule moves. Untracked files are included unless
755
+ the repository's `.gitignore` files exclude them; global or per-clone Git
756
+ excludes don't count. A path Testkit can't read, or a tree that changes while
757
+ Testkit reads it, fails the run instead of producing a key.
575
758
 
576
759
  Testkit-owned outputs are always excluded from fingerprints: `.testkit/`,
577
760
  and `.next-testkit/`. `testkit.config.ts` is normal repo
@@ -596,22 +779,16 @@ Template setup executes in three explicit phases:
596
779
 
597
780
  Schema drift is checked after each successful `migrate` and `seed` step, and
598
781
  `verify` only runs once local replay matches the source schema. Source refreshes
599
- use Dockerized `pg_dump --schema-only --no-owner --no-privileges`, so
600
- seed/reference data is never written into the baseline and host `pg_dump`
601
- installations are not used. Keep schema-changing setup in its own step where
602
- possible; a single command that changes schema and then fails before exiting
603
- cannot be refreshed at the midpoint.
604
-
605
- Source schema refreshes are intentionally single-connection and pooler-safe.
606
- If a Neon pooled source URL is configured, Testkit rewrites it to the matching
607
- direct Neon endpoint before running Dockerized `pg_dump` and records the
608
- original/resolved host classifications beside the resolved cache entry. Unknown
609
- PgBouncer/pooler URLs fail closed; configure a direct source URL for those
610
- providers. Concurrent refreshes for the same service and cache key are
611
- serialized with a cache-local lock so multiple Testkit processes do not
612
- stampede the source database. Testkit also maintains
613
- `.testkit/db/<service>/source-schemas/index.json` and prunes old inactive cache
614
- entries automatically.
782
+ dump the schema only, in Docker, so seed and reference data never enter the
783
+ baseline and no host `pg_dump` is needed. Keep schema-changing setup in its own
784
+ step where possible; a single command that changes schema and then fails before
785
+ exiting cannot be refreshed at the midpoint.
786
+
787
+ A source schema refresh uses one connection. A Neon pooled source URL is
788
+ rewritten to the matching direct Neon endpoint. Other PgBouncer or pooler URLs
789
+ fail; configure a direct source URL for those providers. Refreshes of the same
790
+ cache entry run one at a time, even from separate Testkit processes, and
791
+ Testkit prunes old inactive cache entries automatically.
615
792
 
616
793
  For most repos, prefer declarative step objects directly inside
617
794
  `database.postgres({ template: ... })` and `runtime.prepare.steps`.
@@ -621,10 +798,9 @@ The supported shapes are:
621
798
  - `{ kind: "sql-file", path: "..." }`
622
799
  - `{ kind: "module", target: "file.ts#exportName" }`
623
800
 
624
- Local files and ordinary external packages remain valid module targets.
625
- Testkit-owned `@gfdelarue/testkit` targets are validated against Testkit's exact
626
- configured-step module registry while config is loaded, so a misspelled or
627
- private Testkit subpath fails before preparation or database setup starts.
801
+ Local files and ordinary external packages remain valid module targets. An
802
+ unsupported or private `@gfdelarue/testkit` target fails when the config loads,
803
+ before preparation or database setup starts.
628
804
 
629
805
  `runtime.toolchain` is the first-class way to make those prepare/start commands
630
806
  run under the correct Node toolchain instead of whatever `node`/`npm` happened
@@ -642,16 +818,12 @@ to launch `testkit`. Node toolchains support:
642
818
  - `package.json#packageManager`
643
819
  - `package.json#engines.npm`
644
820
 
645
- Downloaded toolchains resolve only through Testkit's target-specific trusted
646
- Node release manifest. The archive URL, compressed size, SHA-256, archive
647
- shape, unpacked size, and complete installed-tree digest are fixed before any
648
- download. An unlisted version or platform fails closed. The verified cache is
649
- kept under `.testkit/_toolchains/node`; external cache redirects, linked cache
650
- ancestors, modified installed content, and escaping executable links are
651
- rejected. `runtime.toolchain.cwd` is always a normalized product-relative path
652
- and cannot contain a parent-directory segment. The trusted official Linux Node
653
- archives are glibc builds; `install: "download"` fails before fetch or cache
654
- mutation on musl instead of pretending those binaries are compatible.
821
+ `install: "download"` supports the Node versions and platforms Testkit lists,
822
+ and fails before downloading for any other. Downloads are cached under
823
+ `.testkit/_toolchains/node`, and a modified cache entry fails instead of
824
+ running. `runtime.toolchain.cwd` is a product-relative path and cannot contain
825
+ a parent-directory segment. The official Linux Node archives are glibc builds,
826
+ so `install: "download"` fails on musl before it downloads anything.
655
827
 
656
828
  Example:
657
829
 
@@ -666,7 +838,6 @@ toolchains: {
666
838
  services: {
667
839
  frontend: app.next({
668
840
  cwd: "frontend",
669
- port: 3000,
670
841
  runtime: {
671
842
  toolchain: "frontendNode",
672
843
  },
@@ -681,6 +852,77 @@ artifact/log/setup files. Payloads are SHA-256 addressed under
681
852
  removing their logical per-run paths. `.testkit/results/latest.json` remains a
682
853
  convenience projection; it is not the historical authority.
683
854
 
855
+ `run.json` is schema 20. `groups` holds each group's suites and files.
856
+ `environment` is the run's environment record. For each acquisition, it holds:
857
+
858
+ - the source that prepared it: `configured`, `static`, `supplied`, or `none`;
859
+ - the tasks it served and the services it started;
860
+ - each file's setup;
861
+ - how it was released;
862
+ - the values its tests received.
863
+
864
+ It also holds each service's start attempts, the service and setup logs, the
865
+ run's capacity and database backend, and the values tests asked for that the
866
+ environment lacked. Runs that Testkit 0.4.x stored as schema 19 stay readable
867
+ through `testkit artifacts`, `ci report`, known-failure checks, and the
868
+ workspace API.
869
+
870
+ Sealing replaces secrets with `<redacted>` in the run record and in every text
871
+ file of the bundle. A key names a secret when it holds `PASSWORD`, `TOKEN`,
872
+ `SECRET`, `PRIVATE_KEY`, `AUTH`, `COOKIE`, `SESSION`, or `API_KEY` in any letter
873
+ case. Sealing redacts:
874
+
875
+ - A `KEY=value` assignment under such a key, an `Authorization`,
876
+ `Proxy-Authorization`, `Cookie`, or `Set-Cookie` header value, a JSON string
877
+ value under such a key, even when the key is spelled with escapes, a
878
+ single-quoted `'password': '…'` value, and a URL's password, wherever they
879
+ appear. A quoted value ends at its closing quote and keeps its quotes, and a
880
+ value that starts with an auth scheme such as `Bearer` ends with its line.
881
+ A secret in one JSON value leaves the record's other values as they are.
882
+ - The value of each such key, and of each key that `redact` names, in the
883
+ environment Testkit runs in. When the value is a URL, its password counts
884
+ too.
885
+ - The values the run's environment resolved: each value under such a key or a
886
+ key that `redact` names, and the password of each URL. A PostgreSQL URL's
887
+ password may also be a `password` query parameter, as libpq reads it. This
888
+ covers config `env`, env files, `local.env`, dependency services, base,
889
+ readiness, UI, and database URLs, `--env`, `--env-from`, and an
890
+ `--environment-file`. A value under such a key that is a URL counts only for
891
+ its password.
892
+ - The unguessable segment in each [HTTP capture's](#handles) operation URLs,
893
+ from a setup module's capture or an `--environment-file` handle.
894
+
895
+ Name a secret's key in `redact` when it holds none of the words above:
896
+
897
+ ```ts
898
+ export default defineConfig({
899
+ redact: ["STRIPE_SK", "PGPASS"],
900
+ });
901
+ ```
902
+
903
+ Testkit matches the names in any letter case, in a configured run and a
904
+ supplied run alike. `redact` adds keys whose values sealing redacts; the forms
905
+ in the first item still use the words above.
906
+
907
+ A value shorter than eight characters is not redacted on its own. Sealing also
908
+ redacts a value's JSON-escaped forms, each line of a value that spans lines,
909
+ and keys in the free-form data and metadata that tests attach. When output cuts a secret short and marks
910
+ the cut with `…` or `...`, sealing redacts the part that shows. `testkit run
911
+ --json` and the run's progress output apply the same redaction.
912
+
913
+ Some secrets stay visible:
914
+
915
+ - What a setup module prints goes to the terminal as written.
916
+ - Sealing does not look inside compressed or binary files, such as a Playwright
917
+ `trace.zip`. It refuses to seal a binary file that holds one of the values
918
+ above in plain bytes.
919
+ - When a run's process is killed before it seals the run, `testkit stop`,
920
+ `testkit cleanup`, or the next run seals it from another process. That
921
+ process does not have the values the run's environment resolved or the
922
+ `redact` list, so it redacts only the first item and its own environment's
923
+ values under the words above. The bundle's `archive.redaction.status` is then
924
+ `source-targets-unavailable` instead of `applied`.
925
+
684
926
  Use the artifact archive config to opt into pruning or atomic publication to
685
927
  an external directory. Local retention is indefinite by default:
686
928
 
@@ -724,253 +966,320 @@ Programmatic consumers can import `listRunBundles`, `loadRunBundleArtifact`,
724
966
 
725
967
  ## Authoring
726
968
 
727
- ### Node setup beside a test
969
+ ### Named endpoints and databases
728
970
 
729
- A test such as `tests/webhook.int.testkit.ts` can have a sibling
730
- `tests/webhook.int.testkit.setup.mjs`. Testkit runs its named `setup` export,
731
- or its default export, before starting the test engine:
971
+ `@gfdelarue/testkit/environment` gives a test the URL and database of each
972
+ service its group uses, by the service's name:
973
+
974
+ ```ts
975
+ import { database, endpoint } from "@gfdelarue/testkit/environment";
976
+
977
+ const response = await fetch(new URL("/health", endpoint("api")));
978
+ const reportsDatabase = database("reports");
979
+ ```
980
+
981
+ Both functions return strings. A service's database is the one Testkit
982
+ provisions for it, or the `DATABASE_URL` its `env` declares. The module works
983
+ in Vitest and Playwright files and in the modules they import.
984
+ Testkit installs the values before any of that code loads, so module-level code
985
+ and native hooks can read them, such as a Playwright config and its
986
+ `globalSetup` or a Vitest `beforeAll`.
987
+
988
+ A name with no value throws `EnvironmentValueMissingError`, with `kind`,
989
+ `valueName`, and the `available` names. `valueName` is `null` when the missing
990
+ value is the run's default URL. Outside a Testkit run every call throws it.
991
+
992
+ A test that fails on this error, or a file whose imports throw it, ends as an
993
+ infrastructure failure in Vitest and Playwright. The run fails, and
994
+ `testkit bug file` refuses to file it. A test that catches the error and
995
+ carries on is unaffected. When the run gives a test no base URL, a Playwright
996
+ `page.goto()` to a relative URL ends the same way.
997
+
998
+ `BASE_URL` and `DATABASE_URL` hold the values of the first service
999
+ in `uses` that has each one. Playwright gives `BASE_URL` and `baseURL` in
1000
+ canonical form, so an origin ends with `/`. `endpoint()` returns the URL
1001
+ unchanged in every engine.
1002
+
1003
+ The group's default endpoint and database are also available by the group's
1004
+ name, whether the run starts its services or takes supplied values. For a group
1005
+ `checks` that uses `api`, `endpoint("checks")` and `endpoint("api")` return the
1006
+ same URL. A group whose default differs from a used service's value with the
1007
+ group's name, such as a group `api` with `uses: ["web", "api"]`, gets no
1008
+ values: its files end as an infrastructure failure that names the group, and
1009
+ other groups still run.
1010
+
1011
+ ### Handles
1012
+
1013
+ `handle(name)` from `@gfdelarue/testkit/environment` gives a test an object for
1014
+ something its environment prepared. Today that is an HTTP capture, which a
1015
+ [setup module](#node-setup-beside-a-test) starts with `sidecars.httpCapture()`.
1016
+ The handle is named after the capture's `id`:
732
1017
 
733
1018
  ```js
734
- // tests/webhook.int.testkit.setup.mjs
1019
+ // tests/webhook.contract.testkit.setup.mjs
735
1020
  export async function setup({ sidecars }) {
736
- const capture = await sidecars.httpCapture({ path: "/webhook" });
737
- return {
738
- data: { webhookUrl: capture.url, requestsUrl: capture.requestsUrl },
739
- };
1021
+ await sidecars.httpCapture({ id: "webhook", path: "/webhook" });
740
1022
  }
741
1023
  ```
742
1024
 
743
- Setup runs in a Node worker thread with its own JavaScript state. It can use
744
- Node APIs, prepare external resources, and start HTTP capture listeners.
745
- The context contains `deadlineMs`, `signal`, `env`, `leaseDir`, `productDir`,
746
- `serviceName`, `testFile`, and `sidecars`. The worker shares the host process's
747
- permissions. It is not a sandbox for untrusted code.
748
-
749
- Return plain JSON, or `{ data, cleanup }` with an optional cleanup function.
750
- Testkit passes the data to the engine as `setupData` and records it with the
751
- setup module path and sidecar handles. Bare `null` or `undefined`, and a
752
- cleanup-only result, give empty data. `{ data: null }` preserves `null`.
753
- Data must encode to at most 1 MiB of JSON. Invalid data reports its JSON path,
754
- such as `$.items[1]`, without invoking property getters.
755
-
756
- Setup shares the task's absolute deadline. On failure or cancellation, Testkit
757
- runs registered cleanup and closes its sidecars. Once setup succeeds, those
758
- resources remain available until task cleanup. Cleanup callbacks run in reverse
759
- registration order. The runner uses the task timeout for cleanup, capped at
760
- 60 seconds. A hung worker is terminated so it cannot keep its listeners open.
761
-
762
- Thrown setup errors, deadline expiry, and invalid data fail the task with the
763
- setup module path. Consumer cleanup errors fail the task; Testkit cleanup or
764
- worker failures count as infrastructure failures. Cleanup remains idempotent.
765
- HTTP capture retains at most 100 requests by default, configurable up to 1,000.
766
- Each body defaults to 1 MiB, configurable up to 8 MiB, and retained bodies total
767
- at most 8 MiB per listener. A setup can create up to 16 listeners, each with at
768
- most 256 connections.
769
-
770
- Discovery tries sibling `.setup.mjs`, sibling `.setup.js`, then the matching
771
- `.mjs` and `.js` files in `__setup__`, in that order. For the example above,
772
- the fallback is `tests/__setup__/webhook.int.testkit.mjs`.
773
-
774
- ### HTTP suites
775
-
776
1025
  ```ts
777
- import { defineFile } from "@gfdelarue/testkit";
778
- import { defineHttpSuite, expect } from "@gfdelarue/testkit/k6";
779
-
780
- export const testkit = defineFile({ engine: "k6" });
781
-
782
- const suite = defineHttpSuite(({ rawReq }) => {
783
- const response = rawReq.get("/health");
784
- expect.status(response, 200, "health returns 200");
785
- });
1026
+ // tests/webhook.contract.testkit.ts
1027
+ import { handle } from "@gfdelarue/testkit/environment";
786
1028
 
787
- export const options = suite.options;
788
- export const setup = suite.setup;
789
- export default suite.exec;
1029
+ const webhook = handle("webhook");
1030
+ await placeOrder({ callbackUrl: webhook.url });
1031
+ const [delivery] = await webhook.requests();
1032
+ await webhook.reset();
790
1033
  ```
791
1034
 
792
- Testkit helper suites are authored as native-shaped k6 modules: re-export the
793
- helper's `options` and `setup`, and default-export its `exec` function. Testkit
794
- does not inspect source text or invent k6 lifecycle exports.
795
-
796
- Suite-owned authentication lives in the k6 module. `auth.setup` receives the
797
- resolved env and a raw HTTP client, returns a session bundle of named actors,
798
- and `auth.headers` turns the bundle plus the requested actor into headers:
1035
+ A capture handle has `kind` (`"http-capture"`), `url` (where the product sends
1036
+ its requests), `requests()`, which returns the captured requests oldest first,
1037
+ and `reset()`, which forgets them. Both methods return a Promise in
1038
+ Vitest and Playwright. Each call fails after 10 seconds if the capture hasn't
1039
+ answered in full.
1040
+
1041
+ A handle is a reference, not a connection. It can list or reset its own capture
1042
+ and nothing else. The capture's operation URLs hold an unguessable segment, so
1043
+ holding the handle is the authority. Testkit keeps that segment out of the
1044
+ run's results, artifacts, and events, and the capture refuses any other
1045
+ operation. When Testkit releases the file's environment, the capture closes and
1046
+ the handle expires. Calling a handle never starts a sidecar or a service.
1047
+
1048
+ - A name with no handle throws `EnvironmentValueMissingError` with `kind`
1049
+ `"handle"`, and the test ends as an infrastructure failure, as for a missing
1050
+ endpoint.
1051
+ - Every capture created without an `id` gets the id `http-capture`. If two do,
1052
+ `handle("http-capture")` throws `EnvironmentHandleAmbiguousError` with the
1053
+ `count`. Give each capture a distinct `id`.
1054
+ - A call that the capture refuses, such as one from a stale or forged
1055
+ reference, throws `EnvironmentHandleError` with `reason` `"refused"`. A call
1056
+ whose capture no longer answers throws it with `"unavailable"`, and a
1057
+ descriptor this Testkit cannot use throws it with `"invalid"` or
1058
+ `"unsupported"`. These end the test as ordinary failures.
1059
+
1060
+ ### Give each test its own data
1061
+
1062
+ Testkit runs test files at the same time. Files on one runtime share its
1063
+ services and its database. Write each test so that it passes beside any other
1064
+ test, in any order.
1065
+
1066
+ - Create the rows the test needs, and filter every query and update to them.
1067
+ Don't depend on rows that another test creates, and don't count, update, or
1068
+ delete across a whole table. Every test sees the rows from `template.seed`,
1069
+ so read them and leave them unchanged.
1070
+ - When the test ends, leave its rows in place. Testkit resets the databases it
1071
+ provisions. By default, every run starts from a fresh clone of the template
1072
+ database. See [`database.reset`](testkit-config-schema.md#database).
1073
+ - Clean up only what Testkit doesn't own, such as an account in a third-party
1074
+ sandbox. Register the cleanup as soon as the test creates the account. Give
1075
+ the account a unique name, so that a leftover from a cleanup that never ran
1076
+ can't collide with the next run.
799
1077
 
800
1078
  ```ts
1079
+ import { randomUUID } from "node:crypto";
801
1080
  import { defineFile } from "@gfdelarue/testkit";
802
- import { defineHttpSuite } from "@gfdelarue/testkit/k6";
803
-
804
- import { organizationHeaders, sessionAuth } from "../helpers/session.js";
1081
+ import { contractScenario } from "@gfdelarue/testkit/node-vitest/contract";
1082
+ import { expect, onTestFinished } from "vitest";
805
1083
 
806
- export const testkit = defineFile({ engine: "k6" });
807
-
808
- const suite = defineHttpSuite(
809
- {
810
- auth: sessionAuth(["reviewer", "outsider"]),
811
- rawHeaders: organizationHeaders,
812
- },
813
- ({ actor, actors, req }) => {
814
- req.get("/api/auth/session");
815
- req("GET", "/api/auth/session");
816
- actor?.req.get("/api/auth/session");
817
- req.as("outsider").get("/api/auth/session");
818
- actors.get("reviewer").rawReq.get("/api/auth/session");
819
- },
820
- );
1084
+ import { sandbox } from "../payments-sandbox.js";
821
1085
 
822
- export const options = suite.options;
823
- export const setup = suite.setup;
824
- export default suite.exec;
825
- ```
1086
+ export const testkit = defineFile({ engine: "node-vitest" });
826
1087
 
827
- ```ts
828
- // __testkit__/helpers/session.ts
829
- import type { AuthAdapter, AuthSessionBundle } from "@gfdelarue/testkit/k6";
1088
+ contractScenario("a refund reaches the sandbox customer", async ({ http }) => {
1089
+ const customer = await sandbox.createCustomer({ email: `refund-${randomUUID()}@example.test` });
1090
+ onTestFinished(() => sandbox.deleteCustomer(customer.id));
830
1091
 
831
- export function sessionAuth(actorNames: string[]): AuthAdapter<AuthSessionBundle> {
832
- return {
833
- setup({ env, http }) {
834
- const actors = Object.fromEntries(
835
- actorNames.map((actorName, actorIndex) => [
836
- actorName,
837
- { actorIndex, actorName, session: login(http, env.BASE, actorName) },
838
- ]),
839
- );
840
- return { actors, primaryActor: actorNames[0] };
841
- },
842
- headers(bundle, { actor }) {
843
- const session = bundle?.actors[actor ?? bundle.primaryActor ?? ""]?.session;
844
- return session ? { Authorization: `Bearer ${session.jwt}` } : {};
845
- },
846
- };
847
- }
1092
+ const response = await http().post("/api/refunds", { customerId: customer.id, amountCents: 500 });
1093
+ expect(response.status).toBe(201);
1094
+ });
848
1095
  ```
849
1096
 
850
- Named HTTP profiles in `testkit.config.ts` stay JSON-only for k6 suites, because
851
- the orchestrator serializes them across the Node/k6 boundary. Node/Vitest data
852
- profiles are loaded in-process and may define `auth` and header functions
853
- directly. See the [config reference](testkit-config-schema.md#profileshttp).
1097
+ A [Node setup module](#node-setup-beside-a-test) returns its cleanup with its
1098
+ data. Testkit runs only the cleanup that `setup` returns. If a step in `setup`
1099
+ can throw after `setup` creates something, undo the creation before rethrowing.
854
1100
 
855
- DAL suites:
1101
+ A test that has to count or change a whole table needs a database that other
1102
+ tests don't write to. Give the test a group of its own, and have the group use
1103
+ a service with its own database. Below, the `ledger` group owns the test files
1104
+ under `src/ledger`, and its tests get the database of `ledger-db`, built from
1105
+ the same schema:
856
1106
 
857
1107
  ```ts
858
- import { defineFile } from "@gfdelarue/testkit";
859
- import { defineDalFixtures, defineDalSuite } from "@gfdelarue/testkit/k6";
860
-
861
- export const testkit = defineFile({ engine: "k6" });
862
-
863
- const fixtures = defineDalFixtures(({ db, fixtureScope }) => ({
864
- widget() {
865
- return fixtureScope.seed("widget", "primary", { name: "Primary Widget" }, () => {
866
- const widgetId = fixtureScope.id("widget");
867
- db.exec(`
868
- INSERT INTO widgets (id, name)
869
- VALUES ('${widgetId}', 'Primary Widget')
870
- `);
871
- return { widgetId };
872
- });
873
- },
874
- }));
875
-
876
- const suite = defineDalSuite({ fixtures }, ({ db, fixtureScope, fixtures }) => {
877
- const widget = fixtures.widget();
878
- db.query(`select id from widgets where id = '${widget.widgetId}'`);
879
- fixtureScope.records();
880
- });
881
-
882
- export const options = suite.options;
883
- export const setup = suite.setup;
884
- export default suite.exec;
1108
+ services: {
1109
+ api: app.node({
1110
+ // ...
1111
+ }),
1112
+ "ledger-db": database.fixture({
1113
+ template: { migrate: [{ kind: "sql-file", path: "db/schema.sql" }] },
1114
+ }),
1115
+ },
1116
+ groups: {
1117
+ api: { roots: ["src"], uses: ["api"] },
1118
+ ledger: { roots: ["src/ledger"], uses: ["ledger-db"] },
1119
+ },
885
1120
  ```
886
1121
 
887
- DAL files for the same service database are serialized by default to avoid
888
- exhausting Postgres connection limits during highly parallel runs.
889
- Runtime services also reserve a configurable Postgres connection budget for each
890
- active runtime slot. Use `database.postgres({ runtimeConnections })` for the
891
- service pool size and `database.postgres({ maxConnections,
892
- reservedConnections })` for the backing Postgres capacity.
1122
+ `locks` is an escape hatch for a resource that can't be duplicated, such as a
1123
+ payment provider's sandbox account with one webhook URL that some tests change.
1124
+ Files that name the same lock never run at the same time within one run.
1125
+ Testkit doesn't share locks between runs, such as two CI jobs. Two tests that
1126
+ need a lock because they share rows need their own rows instead.
893
1127
 
894
- `defineDalFixtures(...)` is the package-owned DAL seeding model. It gives every
895
- suite a deterministic `fixtureScope` with:
1128
+ ```ts
1129
+ export const testkit = defineFile({ engine: "node-vitest", locks: ["payments-sandbox-webhook"] });
1130
+ ```
896
1131
 
897
- - `fixtureScope.id(label)` / `uuid(label)`
898
- - `fixtureScope.slug(label)`
899
- - `fixtureScope.email(label)`
900
- - `fixtureScope.string(label, options)`
901
- - `fixtureScope.token(label)`
902
- - `fixtureScope.seed(kind, key, signature, create)`
903
- - `fixtureScope.records()`
1132
+ ### Node setup beside a test
904
1133
 
905
- `testkit` enforces strict fixture behavior:
1134
+ A test such as `tests/webhook.contract.testkit.ts` can have a sibling
1135
+ `tests/webhook.contract.testkit.setup.mjs`. Testkit runs its named `setup` export,
1136
+ or its default export, before starting the test engine:
906
1137
 
907
- - one logical fixture per `kind + key`
908
- - identical reseeds reuse the same seeded value
909
- - conflicting reseeds fail immediately
910
- - dependency cycles fail immediately
911
- - seeded fixture records are persisted as a `testkit.dal-fixtures` artifact
1138
+ ```js
1139
+ // tests/webhook.contract.testkit.setup.mjs
1140
+ export async function setup({ sidecars }) {
1141
+ const capture = await sidecars.httpCapture({ path: "/webhook" });
1142
+ return {
1143
+ data: { webhookUrl: capture.url, requestsUrl: capture.requestsUrl },
1144
+ };
1145
+ }
1146
+ ```
912
1147
 
913
- The first-class k6 authoring namespace:
1148
+ Setup is part of preparing the file's environment. Testkit runs it once, after
1149
+ it acquires the services the file's group uses and before the test engine
1150
+ starts. Testkit never batches a Playwright file that has setup with other
1151
+ Playwright files. Files without setup still batch.
914
1152
 
915
- ```ts
916
- import { checks, expect, parse, waitFor } from "@gfdelarue/testkit/k6";
917
-
918
- const suite = defineHttpSuite(({ actor, rawReq, req }) => {
919
- const me = req.get("/api/v1/me");
920
- expect.status(me, 200, "authenticated session loads");
921
- expect.json(me, (body) => typeof body.data?.id === "string", "response includes user id");
922
-
923
- checks.authGate(rawReq, "sessions", {
924
- get: ["/api/v1/sessions"],
925
- });
926
-
927
- actor?.rawHeaders();
928
-
929
- const upload = req.multipart.post("/api/v1/uploads", {
930
- fields: { name: "fixture" },
931
- files: [
932
- {
933
- field: "file",
934
- data: "hello",
935
- filename: "fixture.txt",
936
- contentType: "text/plain",
937
- },
938
- ],
939
- });
1153
+ Setup runs with its own JavaScript state. It can use Node APIs, prepare
1154
+ external resources, and start HTTP capture listeners. Tests
1155
+ reach a capture by its `id` through [`handle()`](#handles). A capture's
1156
+ `requestsUrl` and `resetUrl` hold an unguessable segment that Testkit keeps out
1157
+ of the run's records, and its capture `path` may not start with `/__testkit__/`.
1158
+ The context contains `deadlineMs`, `signal`, `env`, `leaseDir`, `productDir`,
1159
+ `serviceName`, `testFile`, and `sidecars`. `serviceName` holds the name of the
1160
+ test's group. Setup has the same permissions as the Testkit process. It is not
1161
+ a sandbox for untrusted code.
940
1162
 
941
- expect.status(upload, 201, "upload is accepted");
942
- parse.safeJson(upload);
943
- });
944
- ```
1163
+ A run on [supplied values](#run-against-a-supplied-environment) does not run
1164
+ setup modules. Its tests receive the environment file's `setupData` instead.
945
1165
 
946
- Use native k6 imports for checks, groups, metrics, and low-level HTTP:
1166
+ Return plain JSON, or `{ data, cleanup }` with an optional cleanup function.
1167
+ Testkit passes the data to the engine as `setupData` and records it with the
1168
+ setup module path and sidecar handles. Bare `null` or `undefined`, and a
1169
+ cleanup-only result, give empty data. `{ data: null }` preserves `null`.
1170
+ Data must encode to at most 1 MiB of JSON. Invalid data reports its JSON path,
1171
+ such as `$.items[1]`.
947
1172
 
948
- ```ts
949
- import { check, group } from "k6";
950
- import http from "k6/http";
951
- import { Rate, Trend } from "k6/metrics";
952
- ```
1173
+ Setup shares the task's absolute deadline. On failure or cancellation, Testkit
1174
+ runs registered cleanup and closes its sidecars. Once setup succeeds, those
1175
+ resources remain available until the test and its native hooks finish and
1176
+ Testkit releases the file's environment. Cleanup callbacks run in reverse
1177
+ registration order. The runner uses the task timeout for cleanup, capped at
1178
+ 60 seconds. Testkit stops a setup that hangs past that limit, which closes its
1179
+ listeners.
953
1180
 
954
- The [k6 authoring guide](https://github.com/gfdelarue/testkit/blob/master/docs/k6-authoring-layer.md)
955
- describes the optional evidence recorder and migration from retired Testkit
956
- primitive, metric, option, and assertion aliases.
1181
+ Thrown setup errors, deadline expiry, and invalid data fail the task with the
1182
+ setup module path. Consumer cleanup errors fail the task; a failure in
1183
+ Testkit's own cleanup, or a setup that stops unexpectedly, counts as an
1184
+ infrastructure failure. Cleanup remains idempotent.
1185
+ HTTP capture retains at most 100 requests by default, configurable up to 1,000.
1186
+ Each body defaults to 1 MiB, configurable up to 8 MiB, and retained bodies total
1187
+ at most 8 MiB per listener. A setup can create up to 16 listeners, each with at
1188
+ most 256 connections.
957
1189
 
958
- `waitFor()` consumes the file budget configured by `execution.fileTimeoutSeconds`.
959
- Consumers should not set local timeout values in test files.
1190
+ Discovery tries sibling `.setup.mjs`, sibling `.setup.js`, then the matching
1191
+ `.mjs` and `.js` files in `__setup__`, in that order. For the example above,
1192
+ the fallback is `tests/__setup__/webhook.contract.testkit.mjs`.
960
1193
 
961
- ```ts
962
- import { parse, waitFor } from "@gfdelarue/testkit/k6";
1194
+ ### HTTP and persistence checks
963
1195
 
964
- const response = waitFor(
965
- () => req.get("/api/v1/jobs/123"),
966
- (res) => parse.json(res).data?.status === "completed",
967
- { description: "job 123 to complete" },
968
- );
969
- ```
1196
+ Use plain Vitest files for ordinary HTTP, service, and database checks. Read
1197
+ configured URLs with `endpoint(name)` and `database(name)` from
1198
+ `@gfdelarue/testkit/environment`, then use `fetch` or your application's
1199
+ database client. Use `contractScenario` when a typed contract file needs the
1200
+ managed contract context. Use `dataScenario` for structured data-flow evidence.
970
1201
 
971
1202
  ## Discovery
972
1203
 
973
- `testkit` discovers suites from `__testkit__/` directories.
1204
+ Testkit discovers plain Vitest test files and typed suites.
1205
+
1206
+ ### Plain Vitest test files
1207
+
1208
+ Testkit runs plain Vitest test files with the repository's own Vitest and its
1209
+ config, as `vitest run` does from the product directory. It loads the root
1210
+ `vitest.config.*` or `vite.config.*` the way Vitest does and lists the files
1211
+ that config's projects select, without running global setup. Nested configs and
1212
+ package scripts have no effect. Without a config, Vitest's defaults select files
1213
+ that match `**/*.{test,spec}.?(c|m)[jt]s?(x)`, outside `node_modules` and
1214
+ `.git`.
1215
+
1216
+ `discover`, `run`, `typecheck`, `bug record`, and `bug file` take three flags
1217
+ that change the selection:
1218
+
1219
+ - `--vitest-config <path>` uses that config, relative to the product
1220
+ directory. A missing file, or one outside the product directory, fails before
1221
+ any test runs.
1222
+ - `--vitest-project <name>` runs only that project. An unknown name fails and
1223
+ lists the projects.
1224
+ - `--vitest-runner managed` runs plain files with Testkit's own Vitest and its
1225
+ default rules, without the repository's config.
1226
+
1227
+ Testkit supports Vitest 4.x. Another major, a broken installation, a root
1228
+ Vitest config that fails to load, or a Vitest that `package.json` declares but
1229
+ that is not installed fails before any test runs, naming the version and path.
1230
+ `--vitest-runner managed` avoids the repository's Vitest. A run that selects
1231
+ only typed files, with `--type` or with typed `--file` paths, `bug record` or
1232
+ `bug file` for a typed test, and the commands that select no tests (`status`,
1233
+ `run status`, `doctor`, `destroy`, `cleanup`, and `db schema`) don't load the
1234
+ repository's Vitest, so they still work. A product that neither installs nor
1235
+ declares Vitest has no plain test files.
1236
+
1237
+ All plain files in a run go to one Vitest invocation, so root global setup,
1238
+ teardown, and the repository's reporters run as often as they do under
1239
+ `vitest run`, and Testkit's reporter runs beside them. The invocation's deadline
1240
+ is the sum of its files' timeouts; Vitest's own test and hook timeouts apply
1241
+ inside it. A file that two projects select runs in both, its result combines
1242
+ both runs, and its evidence names each run's project. Files from different
1243
+ groups run together when their groups give them the same services, values, and
1244
+ locks; none may have a setup module or isolation other than `shared`.
1245
+ Otherwise the run stops before any file runs and names the groups and what
1246
+ differs. Services, locks, isolation, or a setup module stop it before Testkit
1247
+ provisions anything; values stop it before Vitest starts. Select one group with
1248
+ `--group`.
1249
+
1250
+ Plain files see the host environment Testkit started with: `HOME`, `PATH`,
1251
+ temporary and XDG directories, locale, `CI`, `FORCE_COLOR`, `NO_COLOR`, and any
1252
+ other variable reach config loading, setup files, and the tests as they do
1253
+ under `vitest run`. Unlike `vitest run`, ambient `BASE_URL` and `DATABASE_URL`
1254
+ never reach them; those come only from the group's environment, from
1255
+ provisioned services or supplied values. Testkit loads no env files for plain
1256
+ files, and Vitest's own env handling and `test.env` work as usual. Testkit puts
1257
+ its own `--import` in front of the shell's `NODE_OPTIONS`. Preloads in it see the
1258
+ shell's environment and the group's values; as under `vitest run`, Vitest sets
1259
+ `TEST`, `VITEST`, and `NODE_ENV` after them. `--vitest-runner managed` leaves the
1260
+ shell's `NODE_OPTIONS` out.
1261
+
1262
+ Two settings narrow the files Vitest selects: `discovery.exclude` in `testkit.config.ts`, and
1263
+ a declared group's `exclude`. Testkit's built-in exclusions for typed files,
1264
+ such as `dist`, `build`, and `coverage`, `discovery.roots`, and a legacy
1265
+ service's exclusions do not apply to plain files.
1266
+
1267
+ A plain file belongs to the group whose roots contain it, or to the implicit
1268
+ `root` group, which uses no service. Legacy service ownership never claims a
1269
+ plain file. Testkit starts no service for a plain file in `root`.
1270
+
1271
+ The plain files in one directory form a suite named after that directory,
1272
+ relative to the group's root. In `root`, `--suite src` selects
1273
+ `src/sum.test.ts` and `src/sum.spec.ts`. A file directly in a group's root names
1274
+ its own suite, so `sum.test.ts` in a group rooted at `src` is in suite `sum`.
1275
+ When two suites in a group would share a name, Testkit lengthens the later
1276
+ ones with parent directories, such as `api-src`, and numbers them if that is
1277
+ not enough. `testkit typecheck` checks plain TypeScript test files and skips
1278
+ plain JavaScript ones, as `tsc` does without `allowJs`.
1279
+
1280
+ ### Typed suites
1281
+
1282
+ `testkit` discovers typed suites from `__testkit__/` directories.
974
1283
 
975
1284
  Place each `__testkit__/` directory as a peer of the thing being tested. The
976
1285
  target is the smallest owned directory or package surface that honestly contains
@@ -999,25 +1308,22 @@ production owner when a narrower owner exists.
999
1308
 
1000
1309
  Concrete layouts:
1001
1310
 
1002
- - `src/api/routes/__testkit__/auth-me.int.testkit.ts`
1003
- - `src/db/__testkit__/sessions-count-type.dal.testkit.ts`
1311
+ - `src/api/routes/__testkit__/auth-me.contract.testkit.ts`
1312
+ - `src/db/__testkit__/sessions-count-type.contract.testkit.ts`
1004
1313
  - `frontend/src/components/layout/__testkit__/navigation.ui.testkit.ts`
1005
- - `src/internal/handler/repos/__testkit__/crud.int.testkit.ts`
1314
+ - `src/internal/handler/repos/__testkit__/crud.contract.testkit.ts`
1006
1315
 
1007
1316
  `testkit` uses these suffixes automatically:
1008
1317
 
1009
- - `*.int.testkit.ts`
1010
- - `*.e2e.testkit.ts`
1011
1318
  - `*.contract.testkit.ts`
1012
1319
  - `*.data.testkit.ts`
1013
- - `*.dal.testkit.ts`
1014
- - `*.load.testkit.ts`
1015
1320
  - `*.ui.testkit.ts`
1016
1321
 
1017
- The suffix never chooses an engine. Each discovered file must also contain one
1018
- root-package `defineFile(...)` declaration with a supported literal engine.
1322
+ The suffix never chooses an engine. Each discovered typed file must also
1323
+ contain one root-package `defineFile(...)` declaration with a supported literal
1324
+ engine.
1019
1325
  Unsupported pairs fail discovery; for example, a `*.ui.testkit.ts` file cannot
1020
- declare `engine: "k6"`.
1326
+ declare `engine: "node-vitest"`.
1021
1327
 
1022
1328
  See [docs/test-types.md](docs/test-types.md) for the canonical type model.
1023
1329
  `*.contract.testkit.ts` files are Node-backed service/runtime contract suites.
@@ -1036,30 +1342,61 @@ structured data-flow evidence through the data context. Supporting
1036
1342
  `count(...)`/`event(...)` calls are not enough by themselves; model actors,
1037
1343
  systems, records, transitions, external calls, payloads, budgets, cache/LLM/RPC
1038
1344
  usage, or reconciliation evidence. Data suites run through the
1039
- `node-vitest-data` profile. Use `contract` for plain Node assertions and
1040
- Node-side service or protocol contracts, `int` or `e2e` for k6-backed HTTP
1041
- integration, and `ui` when a browser is part of the assertion.
1345
+ `node-vitest-data` profile. Use a plain Vitest test file for plain Node
1346
+ assertions and HTTP or database integration, `contract` for typed service or
1347
+ protocol contracts, and `ui` when a browser is part of the assertion.
1042
1348
  Data suites use Vitest directly for assertions, hooks, and mocking.
1043
1349
  `dataScenario(...)` owns case registration: `testkit lint` reports native
1044
1350
  registration imports in data suites, and the data runner rejects any case that
1045
1351
  was not registered through the DSL.
1046
1352
 
1047
- Ownership is inferred from:
1353
+ Test groups own test files by directory. Declare them under `groups` in
1354
+ `testkit.config.ts`. Each group lists its repo-relative `roots` and the services
1355
+ its tests `use`:
1048
1356
 
1049
- - the deepest matching service root from `services.<name>.local.cwd`
1050
- - optional `services.<name>.discovery.roots` overrides for shared-root edge cases
1357
+ ```ts
1358
+ export default defineConfig({
1359
+ services: {
1360
+ api: app.node({ cwd: "services/api", start: "node dist/server.js" }),
1361
+ web: app.next({ cwd: "apps/web", dependsOn: ["api"] }),
1362
+ },
1363
+ groups: {
1364
+ api: { roots: ["services/api"], uses: ["api"] },
1365
+ web: { roots: ["apps/web"], uses: ["web", "api"] },
1366
+ },
1367
+ });
1368
+ ```
1369
+
1370
+ - A file belongs to the group with the deepest root that contains it. When two
1371
+ groups have roots at the same depth, discovery reports `ambiguous_test`.
1372
+ - Files that no group's roots claim belong to the implicit group `root`, which
1373
+ uses no service. With no `testkit.config.ts`, every file belongs to `root`.
1374
+ - A run starts the services that the selected files' groups use, with their
1375
+ `dependsOn`, and nothing else. A test gets `BASE_URL`, `DATABASE_URL`, and
1376
+ `env` values only from the services its group uses. When two of them set the
1377
+ same name, the first in `uses` wins. Each used service's URL and database
1378
+ are also available by name; see
1379
+ [Named endpoints and databases](#named-endpoints-and-databases).
1380
+ - `skip`, `requirements`, and `runtimeIsolation` are group fields. See the
1381
+ [config reference](testkit-config-schema.md#groups).
1382
+
1383
+ A config with `services` and no `groups` key uses legacy ownership. Each
1384
+ service owns the typed test files under its `local.cwd` and
1385
+ `discovery.roots`, as a group of the same name that uses that service. Testkit
1386
+ prints a notice on stderr when legacy ownership gives a file to a service.
1387
+ Legacy ownership will be removed; declare `groups` instead.
1051
1388
 
1052
1389
  Suite names are inferred from the colocated path:
1053
1390
 
1054
- - `auth/__testkit__/*.int.testkit.ts` => `auth`
1055
- - `routes/__testkit__/auth-me.int.testkit.ts` => `routes`
1391
+ - `auth/__testkit__/*.contract.testkit.ts` => `auth`
1392
+ - `routes/__testkit__/auth-me.contract.testkit.ts` => `routes`
1056
1393
 
1057
1394
  Discovery is also a first-class CLI/API surface:
1058
1395
 
1059
1396
  - `testkit discover`
1060
- - human-first compact output with service -> type -> suite -> file hierarchy
1397
+ - human-first compact output with group -> type -> suite -> file hierarchy
1061
1398
  - `testkit discover --output-mode verbose`
1062
- - explicit paths, IDs, locks, dependencies, and skip reasons
1399
+ - group roots and uses, explicit paths, IDs, locks, and skip reasons
1063
1400
  - `testkit discover --json`
1064
1401
  - machine-readable output with stable enums, canonical paths, and summary data
1065
1402
  - `testkit discover --output .testkit/discovery.json`
@@ -1088,7 +1425,7 @@ identifier plus canonical metadata such as:
1088
1425
  - `path`
1089
1426
  - `service`
1090
1427
  - `suiteName`
1091
- - `testType`
1428
+ - `testType`, absent for a plain Vitest test file
1092
1429
  - `engineId`
1093
1430
  - `profileId`
1094
1431
  - `contract`
@@ -1099,8 +1436,10 @@ identifier plus canonical metadata such as:
1099
1436
  - `displayName`
1100
1437
 
1101
1438
  Discovery output has no cross-run counts, last outcomes, durations, or trends.
1102
- See the [status and history migration](https://github.com/gfdelarue/testkit/blob/master/docs/release-notes.md#local-status-and-history-removal)
1103
- for the removed discovery fields and consumer migration.
1439
+ Testkit 0.2.1 removed `testkit.status.json`, `.testkit/history/`, and
1440
+ discovery's `history`, `files[].history`, and `summary.latestResults` fields.
1441
+ To check one run, read its bundle under `.testkit/results/runs/<run-id>/`.
1442
+ Testkit ignores an old `.testkit/history/` directory and leaves it in place.
1104
1443
 
1105
1444
  ## Local Databases
1106
1445
 
@@ -1113,13 +1452,10 @@ client image instead of using host-installed binaries, which keeps local runs
1113
1452
  consistent across Linux, macOS, and native Windows. Override the client image
1114
1453
  with `TESTKIT_POSTGRES_CLIENT_IMAGE` when needed.
1115
1454
 
1116
- Client containers for a Testkit-owned database join that database container's
1117
- network namespace (`--network container:<id>`) and connect over loopback. The
1118
- route does not depend on the Docker daemon sharing the Testkit process's host
1119
- network, so template steps, schema snapshots, and `database.steps` helpers work
1120
- the same under Docker Desktop, rootless Docker, and Docker-in-Docker. Clients
1121
- for external databases, such as `sourceSchema` URLs, still connect from the
1122
- platform's host address.
1455
+ Template steps, schema snapshots, and `database.steps` helpers work the same
1456
+ under Docker Desktop, rootless Docker, and Docker-in-Docker. Clients for
1457
+ external databases, such as `sourceSchema` URLs, connect from the platform's
1458
+ host address, so those databases must be reachable from there.
1123
1459
 
1124
1460
  - normal test runs always use transient host-managed Postgres
1125
1461
  - template databases are cached
@@ -1131,8 +1467,12 @@ platform's host address.
1131
1467
  - template fingerprints are derived automatically from env files, source schema
1132
1468
  cache, migrate/seed config, and repo contents
1133
1469
 
1134
- The service process receives its provisioned database as `DATABASE_URL`. Tasks
1135
- get it only through declared `env` values: use the `{dbUrl}`, `{dbHost}`,
1470
+ The service process receives its provisioned database as `DATABASE_URL`. Every
1471
+ task gets the same `DATABASE_URL` when its service declares a database, and its
1472
+ service's URL as `BASE_URL` when the service declares a server, whatever its
1473
+ engine. The provisioned `DATABASE_URL` replaces one that the service's `env`
1474
+ declares. A test whose group uses several services with databases reads each
1475
+ one with `database(name)` from `@gfdelarue/testkit/environment`. Declared `env` values can also use the `{dbUrl}`, `{dbHost}`,
1136
1476
  `{dbPort}`, `{dbName}`, `{dbUser}`, `{dbPassword}`, and `{dbSsl}` tokens, with
1137
1477
  `:serviceName` to read the database of a service listed in `dependsOn`. Testkit
1138
1478
  resolves them from the database provisioned for the current run. They are
@@ -1143,53 +1483,3 @@ schema cache. `db schema verify` prepares local templates and verifies local
1143
1483
  replay against the cached/refreshed source schema. `--skip-schema-source-verify`
1144
1484
  is available as a narrow escape hatch when users need to run tests while schema
1145
1485
  verification is temporarily blocked.
1146
-
1147
- ## Development Tests
1148
-
1149
- ```bash
1150
- npm test
1151
- npm run test:unit
1152
- npm run test:integration
1153
- npm run test:system
1154
- npm run test:live
1155
- npm run test:integration -- fixture-package
1156
- ```
1157
-
1158
- The four lanes use Vitest directly. Unit tests (source and pure tooling) run in
1159
- parallel; integration and system files run serially; live tests are explicitly
1160
- opt-in and serial. Pass ordinary Vitest filters after npm's `--` separator.
1161
- `npm test` runs the first three lanes and never collects live cases.
1162
-
1163
- Live tests cover credentials, external services, network-acquired runtimes,
1164
- alternate toolchains, and Docker/platform matrices. An explicitly selected live
1165
- test fails clearly when its prerequisites are unavailable; it does not silently
1166
- skip.
1167
-
1168
- Repository commands use npm's native timing support, enabled project-wide by
1169
- the tracked `.npmrc`. Build, lint, and release modes support `help`; release
1170
- provides `bump` and `check` modes.
1171
- Every merge to `master` publishes the next patch version. The existing
1172
- pre-merge CI checks provide verification. The
1173
- [release workflow](https://github.com/gfdelarue/testkit/blob/master/docs/release-gates.md) tags the merge and publishes through npm
1174
- trusted publishing; merge a `package.json` bump to start a new minor or major.
1175
-
1176
- The static audit retains package/lock/runtime policy, engine dependency/import
1177
- intent, retired engine/product references, minimal repository traversal, and
1178
- the dependency direction between top-level `src/` modules. The edge table in
1179
- `scripts/lint/audits/import-direction.ts` mirrors the import graph exactly: an
1180
- import outside the table fails lint, and so does an entry that no import
1181
- exercises. Module direction is the one structural rule lint enforces; other
1182
- implementation-shape audits stay out of lint.
1183
-
1184
- JavaScript modules under `test/fixtures/**` describe fixture behavior and keep
1185
- their fixture-owned paths. Outside fixtures, only `bin/testkit.mjs` and the
1186
- documented raw Node process probe remain `.mjs`; the probe is a program, not a
1187
- Vitest suite.
1188
-
1189
- GitHub Actions run a narrow installed Node/Vitest consumer smoke on dedicated
1190
- Orthrus Ubuntu x64 and Windows x64 self-hosted runners for pull requests from
1191
- this repository and manual dispatches. Fork pull requests skip this workflow's
1192
- jobs. Local `npm test`,
1193
- `npm run test:unit`, `npm run test:integration`, and `npm run test:system` remain
1194
- the full developer entrypoints. PostgreSQL and Docker compatibility are explicit
1195
- opt-in dispatcher cases. The smoke workflow is not five-platform release proof.