@gfdelarue/testkit 0.4.5 → 0.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (787) hide show
  1. package/README.md +724 -495
  2. package/bin/testkit-windows-owner-arm64.exe +0 -0
  3. package/bin/testkit-windows-owner-x64.exe +0 -0
  4. package/dist/app/bugs/ci-report.js +2 -2
  5. package/dist/app/bugs/ci-summary.js +3 -3
  6. package/dist/app/bugs/draft.js +3 -3
  7. package/dist/app/bugs/file.js +1 -1
  8. package/dist/app/bugs/gate.js +1 -1
  9. package/dist/app/bugs/guide.js +3 -3
  10. package/dist/app/bugs/issue.js +4 -4
  11. package/dist/app/bugs/known-failures.js +1 -1
  12. package/dist/app/bugs/record.js +1 -1
  13. package/dist/app/bugs/run-reading.js +1 -1
  14. package/dist/app/bugs/supplied-reproduce.js +1 -0
  15. package/dist/app/campaigns/campaign.js +1 -1
  16. package/dist/app/campaigns/index.js +2 -2
  17. package/dist/app/cleanup/index.js +1 -1
  18. package/dist/app/db-schema/prepare.js +1 -1
  19. package/dist/app/destroy/index.js +1 -1
  20. package/dist/app/discover/index.js +2 -2
  21. package/dist/app/doctor/index.js +1 -1
  22. package/dist/app/environments/config-driven.js +1 -1
  23. package/dist/app/environments/environment-file.js +1 -0
  24. package/dist/app/environments/file-setup.js +1 -0
  25. package/dist/app/environments/routing.js +1 -1
  26. package/dist/app/environments/static-values.js +1 -0
  27. package/dist/app/environments/supplied-input.js +1 -0
  28. package/dist/app/environments/supplied.js +1 -1
  29. package/dist/app/environments/task-databases.js +2 -0
  30. package/dist/app/inspect/artifacts.js +1 -1
  31. package/dist/app/inspect/index.js +1 -1
  32. package/dist/app/lint/index.js +1 -1
  33. package/dist/app/lint/native.js +1 -1
  34. package/dist/app/lint/options.js +1 -1
  35. package/dist/app/lint/rules/engine-imports.js +1 -1
  36. package/dist/app/lint/rules/syntax.js +1 -1
  37. package/dist/app/lint/rules/unit.js +1 -1
  38. package/dist/app/reports/prepare.js +1 -1
  39. package/dist/app/run/caller-redaction.js +1 -0
  40. package/dist/app/run/engine-execution-failures.js +1 -1
  41. package/dist/app/run/engine-execution-reporting.js +1 -1
  42. package/dist/app/run/engine-execution.js +1 -1
  43. package/dist/app/run/engine-orchestration.js +1 -1
  44. package/dist/app/run/environment.js +1 -1
  45. package/dist/app/run/failure-details.js +1 -1
  46. package/dist/app/run/live-run.js +1 -1
  47. package/dist/app/run/orchestrator.js +1 -1
  48. package/dist/app/run/planning.js +1 -1
  49. package/dist/app/run/request.js +1 -1
  50. package/dist/app/run/results.js +1 -1
  51. package/dist/app/run/run-events.js +1 -1
  52. package/dist/app/run/run-finalization.js +1 -1
  53. package/dist/app/run/scheduler/estimates.js +1 -1
  54. package/dist/app/run/scheduler/identity.js +1 -1
  55. package/dist/app/run/scheduler/index.js +1 -1
  56. package/dist/app/run/setup/runtime-node-setup-protocol.js +1 -1
  57. package/dist/app/run/setup/runtime-node-setup-worker.js +1 -1
  58. package/dist/app/run/setup/runtime-node-setup.js +2 -2
  59. package/dist/app/run/task-batches.js +1 -1
  60. package/dist/app/run/worker-loop.js +1 -1
  61. package/dist/app/status/model.js +1 -1
  62. package/dist/app/status/runs.js +1 -1
  63. package/dist/app/typecheck/index.js +4 -4
  64. package/dist/cli/command-catalog.js +1 -1
  65. package/dist/cli/command-flags.js +1 -1
  66. package/dist/cli/commands/artifacts.js +2 -2
  67. package/dist/cli/commands/bug/file.js +2 -2
  68. package/dist/cli/commands/bug/record.js +2 -2
  69. package/dist/cli/commands/discover.js +1 -1
  70. package/dist/cli/commands/guard/unsafe-entrypoint.js +1 -1
  71. package/dist/cli/commands/run.js +2 -2
  72. package/dist/cli/commands/typecheck.js +1 -1
  73. package/dist/cli/commands/workspace.js +1 -1
  74. package/dist/cli/framework/errors.js +1 -1
  75. package/dist/cli/framework/execute.js +7 -6
  76. package/dist/cli/framework/parser.js +1 -1
  77. package/dist/cli/operations/run/operation.js +2 -2
  78. package/dist/cli/renderers/bug/text.js +1 -1
  79. package/dist/cli/renderers/discover/report.js +1 -1
  80. package/dist/cli/renderers/run/events.js +2 -2
  81. package/dist/cli/renderers/run/formatting.js +1 -1
  82. package/dist/cli/renderers/run/status.js +1 -1
  83. package/dist/cli/renderers/run/text-reporter.js +24 -22
  84. package/dist/cli/terminal/colors.js +1 -1
  85. package/dist/cli/terminal/layout.js +2 -2
  86. package/dist/config/authoring/database-fixture-discovery.js +1 -0
  87. package/dist/config/authoring/http-methods.js +1 -1
  88. package/dist/config/authoring/index.js +1 -1
  89. package/dist/config/loader/config-loader.js +1 -1
  90. package/dist/config/loader/decode-discovery.js +1 -1
  91. package/dist/config/loader/decode-execution.js +1 -1
  92. package/dist/config/loader/dependencies.js +1 -1
  93. package/dist/config/loader/discover-tests.js +1 -1
  94. package/dist/config/loader/discovery-summary.js +1 -1
  95. package/dist/config/loader/discovery.js +2 -2
  96. package/dist/config/loader/env.js +1 -1
  97. package/dist/config/loader/file-metadata.js +1 -1
  98. package/dist/config/loader/groups.js +2 -1
  99. package/dist/config/loader/index.js +1 -1
  100. package/dist/config/loader/load.js +1 -1
  101. package/dist/config/loader/managed-configs.js +1 -1
  102. package/dist/config/loader/owned-discovery.js +1 -0
  103. package/dist/config/loader/path-policy.js +1 -1
  104. package/dist/config/loader/runtime-isolation.js +1 -1
  105. package/dist/config/loader/runtime.js +1 -1
  106. package/dist/config/loader/skip-config.js +1 -1
  107. package/dist/config/loader/task-policy.js +1 -0
  108. package/dist/config/loader/test-selection.js +1 -1
  109. package/dist/config/loader/validation.js +1 -1
  110. package/dist/database/admin.js +2 -2
  111. package/dist/database/bindings.js +1 -0
  112. package/dist/database/cleanup.js +1 -1
  113. package/dist/database/configured-steps.js +1 -1
  114. package/dist/database/docker-metadata.js +3 -3
  115. package/dist/database/docker.js +3 -3
  116. package/dist/database/index.js +1 -1
  117. package/dist/database/local-postgres.js +1 -1
  118. package/dist/database/naming.js +1 -1
  119. package/dist/database/postgres/connection-check.js +1 -0
  120. package/dist/database/postgres/protocol.js +2 -0
  121. package/dist/database/schema-operations.js +1 -1
  122. package/dist/database/schema-source.js +3 -3
  123. package/dist/database/source-schema-env.js +1 -0
  124. package/dist/database/state-files.js +1 -1
  125. package/dist/database/state.js +1 -1
  126. package/dist/database/steps.js +3 -3
  127. package/dist/domain/file-timeout.js +1 -1
  128. package/dist/domain/identifiers.js +1 -1
  129. package/dist/domain/retired-test-types.js +1 -0
  130. package/dist/domain/runtime-isolation-contract.js +1 -1
  131. package/dist/domain/suite-selection.js +1 -1
  132. package/dist/domain/test-types.js +1 -1
  133. package/dist/engines/adapters/evidence-files.js +1 -1
  134. package/dist/engines/adapters/execution.js +1 -1
  135. package/dist/engines/adapters/json-bounds.js +2 -0
  136. package/dist/engines/adapters/native-report.js +1 -1
  137. package/dist/engines/api/execution.js +1 -1
  138. package/dist/engines/api/index.js +1 -1
  139. package/dist/engines/api/runtime-artifact.js +1 -1
  140. package/dist/engines/catalog.js +1 -1
  141. package/dist/engines/contracts/environment-values.js +1 -0
  142. package/dist/engines/definitions.js +1 -1
  143. package/dist/engines/environment/handles.js +1 -0
  144. package/dist/engines/environment/index.js +1 -0
  145. package/dist/engines/factory.js +1 -1
  146. package/dist/engines/installations/node-vitest.js +1 -1
  147. package/dist/engines/installations/package-resolution.js +1 -1
  148. package/dist/engines/installations/playwright.js +1 -1
  149. package/dist/engines/managed-module-resolution.js +8 -4
  150. package/dist/engines/manager.js +1 -1
  151. package/dist/engines/node-vitest/adapter/adapter.js +1 -1
  152. package/dist/engines/node-vitest/adapter/driver.js +2 -0
  153. package/dist/engines/node-vitest/adapter/evidence.js +2 -2
  154. package/dist/engines/node-vitest/adapter/failure-details.js +4 -2
  155. package/dist/engines/node-vitest/adapter/reporter.js +1 -1
  156. package/dist/engines/node-vitest/adapter/workspace.js +17 -195
  157. package/dist/engines/node-vitest/inventory/index.js +2 -0
  158. package/dist/engines/node-vitest/inventory/list-files.js +3 -0
  159. package/dist/engines/node-vitest/runtime/data/index.js +1 -1
  160. package/dist/engines/node-vitest/runtime/data/report.js +5 -6
  161. package/dist/engines/node-vitest/runtime/data/scenario.js +2 -2
  162. package/dist/engines/node-vitest/runtime/runtime-input.js +1 -1
  163. package/dist/engines/node-vitest/selection.js +1 -0
  164. package/dist/engines/playwright/adapter/adapter.js +1 -1
  165. package/dist/engines/playwright/adapter/config.js +3 -3
  166. package/dist/engines/playwright/adapter/evidence-validation.js +1 -1
  167. package/dist/engines/playwright/adapter/evidence.js +1 -1
  168. package/dist/engines/playwright/adapter/reporter.js +1 -1
  169. package/dist/engines/playwright/runtime/config-file.js +1 -1
  170. package/dist/engines/playwright/runtime/config.js +1 -1
  171. package/dist/engines/playwright/runtime/managed-test.js +1 -0
  172. package/dist/engines/playwright/runtime/runtime-input.js +29 -10
  173. package/dist/engines/runtime-input-file.js +1 -1
  174. package/dist/environment/lifecycle-controller.js +1 -1
  175. package/dist/environment/lifecycle.js +1 -1
  176. package/dist/environment/ready-environment.js +1 -1
  177. package/dist/environment/run-environment.js +1 -1
  178. package/dist/environment/run-stopping.js +1 -1
  179. package/dist/environment/runtime-contexts.js +1 -1
  180. package/dist/environment/runtime-manager.js +1 -1
  181. package/dist/environment/runtime-planning.js +1 -1
  182. package/dist/environment/runtime-preparation.js +1 -1
  183. package/dist/environment/services.js +3 -3
  184. package/dist/environment/template.js +1 -1
  185. package/dist/evidence/artifact-config.js +1 -1
  186. package/dist/evidence/artifacts.js +2 -2
  187. package/dist/evidence/bundles.js +1 -1
  188. package/dist/evidence/environment-record.js +1 -1
  189. package/dist/evidence/index.js +1 -1
  190. package/dist/evidence/maintenance.js +1 -0
  191. package/dist/evidence/read-run.js +1 -1
  192. package/dist/evidence/reading.js +1 -1
  193. package/dist/evidence/redaction.js +4 -0
  194. package/dist/evidence/reporting.js +1 -1
  195. package/dist/evidence/run-model.js +1 -0
  196. package/dist/evidence/storage.js +2 -2
  197. package/dist/evidence/timing.js +1 -1
  198. package/dist/kernel/error-serialization.js +1 -1
  199. package/dist/kernel/known-keys.js +1 -0
  200. package/dist/kernel/redaction-targets.js +1 -0
  201. package/dist/kernel/redaction.js +4 -1
  202. package/dist/kernel/unknown-values.js +1 -1
  203. package/dist/process/environment.js +1 -1
  204. package/dist/process/execa-bundle.json +3 -3
  205. package/dist/process/files/json-file.js +1 -1
  206. package/dist/process/host/platform.js +1 -1
  207. package/dist/process/host-process.js +1 -1
  208. package/dist/process/index.js +1 -1
  209. package/dist/process/locks.js +4 -4
  210. package/dist/process/port-listener.js +2 -0
  211. package/dist/process/process-group.js +1 -1
  212. package/dist/process/process.js +32 -32
  213. package/dist/process/readiness.js +1 -1
  214. package/dist/process/ready-deadline.js +1 -0
  215. package/dist/process/runs/ownership.js +1 -1
  216. package/dist/process/runs/stop.js +1 -1
  217. package/dist/process/service-processes.js +1 -1
  218. package/dist/process/windows-cli-owner.js +3 -3
  219. package/dist/process/windows-native.js +1 -1
  220. package/dist/repo/fingerprint-policy.js +1 -1
  221. package/dist/repo/managed-git.js +1 -1
  222. package/dist/repo/physical-tree.js +1 -1
  223. package/dist/repo/state.js +4 -4
  224. package/dist/steps/build-config.js +1 -1
  225. package/dist/steps/configured-step-testkit-modules.js +1 -1
  226. package/dist/steps/configured-steps.js +1 -1
  227. package/dist/steps/environment.js +1 -1
  228. package/dist/steps/index.js +1 -1
  229. package/dist/toolchains/index.js +3 -3
  230. package/dist/toolchains/semver.js +1 -1
  231. package/dist/toolchains/vitest/index.js +1 -1
  232. package/dist/tracker/redaction.js +1 -2
  233. package/dist/{config/runtime/build-assets.d.ts → types/config-build-assets.d.ts} +5 -3
  234. package/dist/{database/steps.d.ts → types/config-database-steps.d.ts} +7 -7
  235. package/dist/{config/runtime/next-runtime-tsconfig.d.ts → types/config-next-runtime-tsconfig.d.ts} +5 -3
  236. package/dist/types/config.d.ts +422 -0
  237. package/dist/types/drizzle.d.ts +13 -0
  238. package/dist/{config/runtime/env/index.d.ts → types/env.d.ts} +22 -20
  239. package/dist/types/environment.d.ts +46 -0
  240. package/dist/types/index.d.ts +21 -0
  241. package/dist/types/node-vitest-data.d.ts +191 -0
  242. package/dist/types/playwright.d.ts +130 -0
  243. package/dist/types/tooling-vitest.d.ts +13 -0
  244. package/dist/types/workspace.d.ts +633 -0
  245. package/docs/bug-records.md +35 -40
  246. package/docs/engines.md +259 -0
  247. package/docs/release-notes.md +2116 -0
  248. package/docs/test-types.md +37 -85
  249. package/package.json +22 -32
  250. package/testkit-config-schema.md +391 -260
  251. package/dist/app/bugs/areas.d.ts +0 -38
  252. package/dist/app/bugs/assessment.d.ts +0 -9
  253. package/dist/app/bugs/categories.d.ts +0 -41
  254. package/dist/app/bugs/ci-report.d.ts +0 -21
  255. package/dist/app/bugs/ci-summary.d.ts +0 -13
  256. package/dist/app/bugs/draft.d.ts +0 -70
  257. package/dist/app/bugs/facts.d.ts +0 -49
  258. package/dist/app/bugs/file.d.ts +0 -45
  259. package/dist/app/bugs/gate.d.ts +0 -22
  260. package/dist/app/bugs/guide.d.ts +0 -20
  261. package/dist/app/bugs/issue.d.ts +0 -18
  262. package/dist/app/bugs/known-failures.d.ts +0 -49
  263. package/dist/app/bugs/record.d.ts +0 -14
  264. package/dist/app/bugs/run-reading.d.ts +0 -51
  265. package/dist/app/bugs/store.d.ts +0 -34
  266. package/dist/app/campaigns/brief.d.ts +0 -7
  267. package/dist/app/campaigns/campaign.d.ts +0 -14
  268. package/dist/app/campaigns/index.d.ts +0 -38
  269. package/dist/app/campaigns/store.d.ts +0 -5
  270. package/dist/app/campaigns/summary.d.ts +0 -13
  271. package/dist/app/cleanup/index.d.ts +0 -52
  272. package/dist/app/db-schema/prepare.d.ts +0 -11
  273. package/dist/app/db-schema/refresh.d.ts +0 -17
  274. package/dist/app/db-schema/verify.d.ts +0 -11
  275. package/dist/app/destroy/index.d.ts +0 -16
  276. package/dist/app/discover/index.d.ts +0 -8
  277. package/dist/app/doctor/index.d.ts +0 -17
  278. package/dist/app/environments/config-driven.d.ts +0 -5
  279. package/dist/app/environments/routing.d.ts +0 -6
  280. package/dist/app/environments/supplied.d.ts +0 -3
  281. package/dist/app/file-selection.d.ts +0 -1
  282. package/dist/app/inspect/artifacts.d.ts +0 -30
  283. package/dist/app/inspect/index.d.ts +0 -112
  284. package/dist/app/lint/index.d.ts +0 -28
  285. package/dist/app/lint/native.d.ts +0 -10
  286. package/dist/app/lint/options.d.ts +0 -66
  287. package/dist/app/lint/policy.d.ts +0 -21
  288. package/dist/app/lint/rules/data.d.ts +0 -3
  289. package/dist/app/lint/rules/data.js +0 -1
  290. package/dist/app/lint/rules/engine-imports.d.ts +0 -3
  291. package/dist/app/lint/rules/syntax.d.ts +0 -4
  292. package/dist/app/lint/rules/unit.d.ts +0 -3
  293. package/dist/app/lint/shared.d.ts +0 -34
  294. package/dist/app/package-inspection.d.ts +0 -20
  295. package/dist/app/reports/files.d.ts +0 -13
  296. package/dist/app/reports/prepare.d.ts +0 -20
  297. package/dist/app/reports/publish.d.ts +0 -21
  298. package/dist/app/run/endpoint-readiness.d.ts +0 -43
  299. package/dist/app/run/endpoint-readiness.js +0 -1
  300. package/dist/app/run/engine-evidence-persistence.d.ts +0 -39
  301. package/dist/app/run/engine-execution-cleanup.d.ts +0 -8
  302. package/dist/app/run/engine-execution-cleanup.js +0 -1
  303. package/dist/app/run/engine-execution-failures.d.ts +0 -41
  304. package/dist/app/run/engine-execution-observer.d.ts +0 -21
  305. package/dist/app/run/engine-execution-reporting.d.ts +0 -17
  306. package/dist/app/run/engine-execution.d.ts +0 -65
  307. package/dist/app/run/engine-failure-stack.d.ts +0 -1
  308. package/dist/app/run/engine-orchestration.d.ts +0 -30
  309. package/dist/app/run/environment.d.ts +0 -33
  310. package/dist/app/run/failure-details.d.ts +0 -37
  311. package/dist/app/run/live-run.d.ts +0 -29
  312. package/dist/app/run/operation-progress.d.ts +0 -13
  313. package/dist/app/run/orchestrator.d.ts +0 -75
  314. package/dist/app/run/planning-types.d.ts +0 -68
  315. package/dist/app/run/planning.d.ts +0 -14
  316. package/dist/app/run/request.d.ts +0 -30
  317. package/dist/app/run/results.d.ts +0 -161
  318. package/dist/app/run/run-events.d.ts +0 -144
  319. package/dist/app/run/run-finalization.d.ts +0 -56
  320. package/dist/app/run/run-guards.d.ts +0 -3
  321. package/dist/app/run/runtime-failures.d.ts +0 -18
  322. package/dist/app/run/runtime-fixtures.d.ts +0 -2
  323. package/dist/app/run/runtime-task-types.d.ts +0 -20
  324. package/dist/app/run/scheduler/estimates.d.ts +0 -32
  325. package/dist/app/run/scheduler/identity.d.ts +0 -9
  326. package/dist/app/run/scheduler/index.d.ts +0 -124
  327. package/dist/app/run/scheduler/locks.d.ts +0 -6
  328. package/dist/app/run/scheduler/observations.d.ts +0 -20
  329. package/dist/app/run/selection.d.ts +0 -9
  330. package/dist/app/run/setup/runtime-node-setup-errors.d.ts +0 -13
  331. package/dist/app/run/setup/runtime-node-setup-protocol.d.ts +0 -85
  332. package/dist/app/run/setup/runtime-node-setup-worker.d.ts +0 -1
  333. package/dist/app/run/setup/runtime-node-setup.d.ts +0 -48
  334. package/dist/app/run/task-batches.d.ts +0 -23
  335. package/dist/app/run/task-leases.d.ts +0 -15
  336. package/dist/app/run/worker-loop.d.ts +0 -66
  337. package/dist/app/setup/playwright.d.ts +0 -22
  338. package/dist/app/status/index.d.ts +0 -9
  339. package/dist/app/status/model.d.ts +0 -29
  340. package/dist/app/status/runs.d.ts +0 -99
  341. package/dist/app/stop/index.d.ts +0 -14
  342. package/dist/app/tracker/index.d.ts +0 -13
  343. package/dist/app/tracker/remote.d.ts +0 -13
  344. package/dist/app/tracker/token.d.ts +0 -7
  345. package/dist/app/typecheck/index.d.ts +0 -63
  346. package/dist/cli/command-catalog.d.ts +0 -156
  347. package/dist/cli/command-flags.d.ts +0 -19
  348. package/dist/cli/command-registry.d.ts +0 -3
  349. package/dist/cli/commands/artifacts.d.ts +0 -48
  350. package/dist/cli/commands/bug/file.d.ts +0 -16
  351. package/dist/cli/commands/bug/guide.d.ts +0 -5
  352. package/dist/cli/commands/bug/record.d.ts +0 -9
  353. package/dist/cli/commands/campaign/finish.d.ts +0 -10
  354. package/dist/cli/commands/campaign/start.d.ts +0 -7
  355. package/dist/cli/commands/campaign/status.d.ts +0 -13
  356. package/dist/cli/commands/ci/report.d.ts +0 -15
  357. package/dist/cli/commands/cleanup.d.ts +0 -13
  358. package/dist/cli/commands/db/schema/refresh.d.ts +0 -2
  359. package/dist/cli/commands/db/schema/verify.d.ts +0 -2
  360. package/dist/cli/commands/destroy.d.ts +0 -9
  361. package/dist/cli/commands/discover.d.ts +0 -2
  362. package/dist/cli/commands/doctor.d.ts +0 -2
  363. package/dist/cli/commands/guard/unsafe-entrypoint.d.ts +0 -2
  364. package/dist/cli/commands/lint.d.ts +0 -2
  365. package/dist/cli/commands/report/prepare.d.ts +0 -12
  366. package/dist/cli/commands/report/publish.d.ts +0 -13
  367. package/dist/cli/commands/run/status.d.ts +0 -5
  368. package/dist/cli/commands/run/stop.d.ts +0 -7
  369. package/dist/cli/commands/run.d.ts +0 -9
  370. package/dist/cli/commands/setup/playwright.d.ts +0 -2
  371. package/dist/cli/commands/status.d.ts +0 -5
  372. package/dist/cli/commands/stop.d.ts +0 -12
  373. package/dist/cli/commands/typecheck.d.ts +0 -2
  374. package/dist/cli/commands/workspace.d.ts +0 -2
  375. package/dist/cli/entrypoint.d.ts +0 -1
  376. package/dist/cli/framework/command.d.ts +0 -21
  377. package/dist/cli/framework/definitions.d.ts +0 -100
  378. package/dist/cli/framework/errors.d.ts +0 -6
  379. package/dist/cli/framework/execute.d.ts +0 -14
  380. package/dist/cli/framework/help.d.ts +0 -11
  381. package/dist/cli/framework/index.d.ts +0 -3
  382. package/dist/cli/framework/parser.d.ts +0 -6
  383. package/dist/cli/index.d.ts +0 -4
  384. package/dist/cli/json-errors.d.ts +0 -3
  385. package/dist/cli/operations/db/schema/refresh/operation.d.ts +0 -7
  386. package/dist/cli/operations/db/schema/verify/operation.d.ts +0 -7
  387. package/dist/cli/operations/run/operation.d.ts +0 -26
  388. package/dist/cli/process-signal.d.ts +0 -1
  389. package/dist/cli/renderers/bug/text.d.ts +0 -8
  390. package/dist/cli/renderers/campaign/text.d.ts +0 -2
  391. package/dist/cli/renderers/ci/text.d.ts +0 -2
  392. package/dist/cli/renderers/cleanup/format.d.ts +0 -14
  393. package/dist/cli/renderers/db-schema/text.d.ts +0 -4
  394. package/dist/cli/renderers/destroy/text.d.ts +0 -4
  395. package/dist/cli/renderers/discover/report.d.ts +0 -7
  396. package/dist/cli/renderers/discover/text.d.ts +0 -3
  397. package/dist/cli/renderers/doctor/text.d.ts +0 -2
  398. package/dist/cli/renderers/lint/text.d.ts +0 -2
  399. package/dist/cli/renderers/run/events.d.ts +0 -8
  400. package/dist/cli/renderers/run/failure.d.ts +0 -8
  401. package/dist/cli/renderers/run/formatting.d.ts +0 -79
  402. package/dist/cli/renderers/run/status.d.ts +0 -6
  403. package/dist/cli/renderers/run/text-reporter.d.ts +0 -19
  404. package/dist/cli/renderers/status/text.d.ts +0 -2
  405. package/dist/cli/renderers/stop/text.d.ts +0 -7
  406. package/dist/cli/renderers/typecheck/text.d.ts +0 -2
  407. package/dist/cli/terminal/colors.d.ts +0 -10
  408. package/dist/cli/terminal/layout.d.ts +0 -11
  409. package/dist/cli/version-drift.d.ts +0 -25
  410. package/dist/config/authoring/build-config-contract.d.ts +0 -27
  411. package/dist/config/authoring/configured-step-contract.d.ts +0 -18
  412. package/dist/config/authoring/http-execution-contract.d.ts +0 -70
  413. package/dist/config/authoring/http-methods.d.ts +0 -18
  414. package/dist/config/authoring/index.d.ts +0 -372
  415. package/dist/config/authoring/preset-env.d.ts +0 -19
  416. package/dist/config/authoring/runtime.d.ts +0 -64
  417. package/dist/config/authoring/runtime.js +0 -1
  418. package/dist/config/loader/bugs.d.ts +0 -15
  419. package/dist/config/loader/config-loader.d.ts +0 -9
  420. package/dist/config/loader/database-materialization.d.ts +0 -19
  421. package/dist/config/loader/database-roles.d.ts +0 -2
  422. package/dist/config/loader/database.d.ts +0 -55
  423. package/dist/config/loader/decode-discovery.d.ts +0 -5
  424. package/dist/config/loader/decode-execution.d.ts +0 -7
  425. package/dist/config/loader/dependencies.d.ts +0 -7
  426. package/dist/config/loader/discover-tests.d.ts +0 -111
  427. package/dist/config/loader/discovery-summary.d.ts +0 -29
  428. package/dist/config/loader/discovery.d.ts +0 -53
  429. package/dist/config/loader/env.d.ts +0 -4
  430. package/dist/config/loader/execution-model.d.ts +0 -15
  431. package/dist/config/loader/file-metadata.d.ts +0 -19
  432. package/dist/config/loader/groups.d.ts +0 -33
  433. package/dist/config/loader/index.d.ts +0 -10
  434. package/dist/config/loader/load.d.ts +0 -89
  435. package/dist/config/loader/managed-configs.d.ts +0 -23
  436. package/dist/config/loader/path-policy.d.ts +0 -9
  437. package/dist/config/loader/paths.d.ts +0 -5
  438. package/dist/config/loader/runtime-isolation.d.ts +0 -44
  439. package/dist/config/loader/runtime.d.ts +0 -16
  440. package/dist/config/loader/skip-config.d.ts +0 -44
  441. package/dist/config/loader/test-selection.d.ts +0 -15
  442. package/dist/config/loader/validation.d.ts +0 -15
  443. package/dist/config/loader/values.d.ts +0 -3
  444. package/dist/config/loader/values.js +0 -1
  445. package/dist/database/activity-gate.d.ts +0 -22
  446. package/dist/database/admin.d.ts +0 -31
  447. package/dist/database/cleanup.d.ts +0 -36
  448. package/dist/database/configured-steps.d.ts +0 -27
  449. package/dist/database/constants.d.ts +0 -11
  450. package/dist/database/docker-metadata.d.ts +0 -3
  451. package/dist/database/docker.d.ts +0 -65
  452. package/dist/database/drizzle/index.d.ts +0 -13
  453. package/dist/database/fingerprint.d.ts +0 -12
  454. package/dist/database/index.d.ts +0 -39
  455. package/dist/database/lifecycle-cleanup.d.ts +0 -8
  456. package/dist/database/local-postgres.d.ts +0 -71
  457. package/dist/database/naming.d.ts +0 -17
  458. package/dist/database/postgres/index.d.ts +0 -71
  459. package/dist/database/progress.d.ts +0 -6
  460. package/dist/database/roles.d.ts +0 -19
  461. package/dist/database/runtime-config.d.ts +0 -16
  462. package/dist/database/runtime-reservations.d.ts +0 -19
  463. package/dist/database/schema-operations.d.ts +0 -18
  464. package/dist/database/schema-source.d.ts +0 -96
  465. package/dist/database/source-refresh-lock.d.ts +0 -8
  466. package/dist/database/source-url.d.ts +0 -21
  467. package/dist/database/state-files.d.ts +0 -13
  468. package/dist/database/state.d.ts +0 -26
  469. package/dist/database/template-steps.d.ts +0 -39
  470. package/dist/domain/file-timeout.d.ts +0 -25
  471. package/dist/domain/identifiers.d.ts +0 -9
  472. package/dist/domain/operation-display.d.ts +0 -10
  473. package/dist/domain/operation-reason.d.ts +0 -5
  474. package/dist/domain/postgres-route.d.ts +0 -4
  475. package/dist/domain/runtime-identity.d.ts +0 -1
  476. package/dist/domain/runtime-isolation-contract.d.ts +0 -5
  477. package/dist/domain/service-name.d.ts +0 -2
  478. package/dist/domain/suite-selection.d.ts +0 -20
  479. package/dist/domain/test-outcomes.d.ts +0 -2
  480. package/dist/domain/test-types.d.ts +0 -16
  481. package/dist/engines/adapters/evidence-files.d.ts +0 -32
  482. package/dist/engines/adapters/evidence-identity.d.ts +0 -12
  483. package/dist/engines/adapters/execution.d.ts +0 -41
  484. package/dist/engines/adapters/native-report.d.ts +0 -25
  485. package/dist/engines/adapters/node-startup-cache.d.ts +0 -8
  486. package/dist/engines/adapters/workspace.d.ts +0 -4
  487. package/dist/engines/api/capabilities.d.ts +0 -21
  488. package/dist/engines/api/capabilities.js +0 -1
  489. package/dist/engines/api/evidence.d.ts +0 -38
  490. package/dist/engines/api/execution-observer.d.ts +0 -8
  491. package/dist/engines/api/execution-scope.d.ts +0 -17
  492. package/dist/engines/api/execution.d.ts +0 -101
  493. package/dist/engines/api/index.d.ts +0 -5
  494. package/dist/engines/api/k6-runtime-input.d.ts +0 -26
  495. package/dist/engines/api/k6-runtime-input.js +0 -1
  496. package/dist/engines/api/runtime-artifact.d.ts +0 -19
  497. package/dist/engines/bundle-cache.d.ts +0 -32
  498. package/dist/engines/catalog.d.ts +0 -27
  499. package/dist/engines/contracts/fixture-manifest.d.ts +0 -34
  500. package/dist/engines/contracts/json-bounds.d.ts +0 -36
  501. package/dist/engines/contracts/json-bounds.js +0 -2
  502. package/dist/engines/contracts/k6-repository-configuration.d.ts +0 -12
  503. package/dist/engines/contracts/k6-repository-configuration.js +0 -1
  504. package/dist/engines/contracts/test-context.d.ts +0 -15
  505. package/dist/engines/contracts/test-context.js +0 -1
  506. package/dist/engines/definitions.d.ts +0 -23
  507. package/dist/engines/factory.d.ts +0 -6
  508. package/dist/engines/generated-performance.d.ts +0 -1
  509. package/dist/engines/installation-catalog.d.ts +0 -27
  510. package/dist/engines/installation-catalog.js +0 -1
  511. package/dist/engines/installations/k6.d.ts +0 -98
  512. package/dist/engines/installations/k6.js +0 -1
  513. package/dist/engines/installations/node-vitest.d.ts +0 -33
  514. package/dist/engines/installations/package-requirements.d.ts +0 -28
  515. package/dist/engines/installations/package-resolution.d.ts +0 -51
  516. package/dist/engines/installations/playwright-resolution-child.d.ts +0 -1
  517. package/dist/engines/installations/playwright.d.ts +0 -75
  518. package/dist/engines/installations/types.d.ts +0 -10
  519. package/dist/engines/k6/adapter/bundle.d.ts +0 -25
  520. package/dist/engines/k6/adapter/bundle.js +0 -112
  521. package/dist/engines/k6/adapter/evidence.d.ts +0 -15
  522. package/dist/engines/k6/adapter/evidence.js +0 -4
  523. package/dist/engines/k6/adapter/executor.d.ts +0 -13
  524. package/dist/engines/k6/adapter/executor.js +0 -1
  525. package/dist/engines/k6/adapter/index.d.ts +0 -1
  526. package/dist/engines/k6/adapter/index.js +0 -1
  527. package/dist/engines/k6/adapter/input.d.ts +0 -21
  528. package/dist/engines/k6/adapter/input.js +0 -1
  529. package/dist/engines/k6/adapter/native-summary.d.ts +0 -11
  530. package/dist/engines/k6/adapter/native-summary.js +0 -1
  531. package/dist/engines/k6/adapter/repository-configuration.d.ts +0 -2
  532. package/dist/engines/k6/adapter/repository-configuration.js +0 -1
  533. package/dist/engines/k6/authoring-type-paths.d.ts +0 -3
  534. package/dist/engines/k6/authoring-type-paths.js +0 -1
  535. package/dist/engines/k6/runtime/ambient-types/bundle-manifest.json +0 -146
  536. package/dist/engines/k6/runtime/ambient-types/k6/LICENSE +0 -21
  537. package/dist/engines/k6/runtime/ambient-types/k6/browser/index.d.ts +0 -6332
  538. package/dist/engines/k6/runtime/ambient-types/k6/crypto/index.d.ts +0 -218
  539. package/dist/engines/k6/runtime/ambient-types/k6/data/index.d.ts +0 -16
  540. package/dist/engines/k6/runtime/ambient-types/k6/encoding/index.d.ts +0 -46
  541. package/dist/engines/k6/runtime/ambient-types/k6/execution/index.d.ts +0 -127
  542. package/dist/engines/k6/runtime/ambient-types/k6/experimental/csv/index.d.ts +0 -96
  543. package/dist/engines/k6/runtime/ambient-types/k6/experimental/fs/index.d.ts +0 -104
  544. package/dist/engines/k6/runtime/ambient-types/k6/experimental/redis/index.d.ts +0 -821
  545. package/dist/engines/k6/runtime/ambient-types/k6/experimental/streams/index.d.ts +0 -198
  546. package/dist/engines/k6/runtime/ambient-types/k6/global.d.ts +0 -83
  547. package/dist/engines/k6/runtime/ambient-types/k6/html/index.d.ts +0 -1325
  548. package/dist/engines/k6/runtime/ambient-types/k6/http/index.d.ts +0 -806
  549. package/dist/engines/k6/runtime/ambient-types/k6/index.d.ts +0 -126
  550. package/dist/engines/k6/runtime/ambient-types/k6/metrics/index.d.ts +0 -66
  551. package/dist/engines/k6/runtime/ambient-types/k6/net/grpc/index.d.ts +0 -247
  552. package/dist/engines/k6/runtime/ambient-types/k6/options/index.d.ts +0 -459
  553. package/dist/engines/k6/runtime/ambient-types/k6/package.json +0 -79
  554. package/dist/engines/k6/runtime/ambient-types/k6/secrets/index.d.ts +0 -36
  555. package/dist/engines/k6/runtime/ambient-types/k6/timers/index.d.ts +0 -47
  556. package/dist/engines/k6/runtime/ambient-types/k6/websockets/index.d.ts +0 -324
  557. package/dist/engines/k6/runtime/ambient-types/k6/ws/index.d.ts +0 -283
  558. package/dist/engines/k6/runtime/ambient-types/xk6-sql/LICENSE +0 -201
  559. package/dist/engines/k6/runtime/ambient-types/xk6-sql/index.d.ts +0 -397
  560. package/dist/engines/k6/runtime/ambient-types/xk6-sql/package.json +0 -15
  561. package/dist/engines/k6/runtime/artifacts.d.ts +0 -2
  562. package/dist/engines/k6/runtime/artifacts.js +0 -1
  563. package/dist/engines/k6/runtime/checks.d.ts +0 -17
  564. package/dist/engines/k6/runtime/checks.js +0 -1
  565. package/dist/engines/k6/runtime/dal-fixtures.d.ts +0 -24
  566. package/dist/engines/k6/runtime/dal-fixtures.js +0 -1
  567. package/dist/engines/k6/runtime/dal-suite.d.ts +0 -24
  568. package/dist/engines/k6/runtime/dal-suite.js +0 -1
  569. package/dist/engines/k6/runtime/dal.d.ts +0 -10
  570. package/dist/engines/k6/runtime/dal.js +0 -1
  571. package/dist/engines/k6/runtime/fixtures.d.ts +0 -3
  572. package/dist/engines/k6/runtime/fixtures.js +0 -1
  573. package/dist/engines/k6/runtime/http-assertions.d.ts +0 -19
  574. package/dist/engines/k6/runtime/http-assertions.js +0 -1
  575. package/dist/engines/k6/runtime/http-checks.d.ts +0 -19
  576. package/dist/engines/k6/runtime/http-checks.js +0 -1
  577. package/dist/engines/k6/runtime/http-suite-runtime.d.ts +0 -103
  578. package/dist/engines/k6/runtime/http-suite-runtime.js +0 -1
  579. package/dist/engines/k6/runtime/http.d.ts +0 -15
  580. package/dist/engines/k6/runtime/http.js +0 -1
  581. package/dist/engines/k6/runtime/index.d.ts +0 -11
  582. package/dist/engines/k6/runtime/index.js +0 -1
  583. package/dist/engines/k6/runtime/input.d.ts +0 -3
  584. package/dist/engines/k6/runtime/input.js +0 -1
  585. package/dist/engines/k6/runtime/k6-repository-configuration.d.ts +0 -9
  586. package/dist/engines/k6/runtime/k6-repository-configuration.js +0 -1
  587. package/dist/engines/k6/runtime/node-setup.d.ts +0 -13
  588. package/dist/engines/k6/runtime/node-setup.js +0 -1
  589. package/dist/engines/k6/runtime/runtime-globals.d.ts +0 -10
  590. package/dist/engines/k6/runtime/runtime.d.ts +0 -54
  591. package/dist/engines/k6/runtime/runtime.js +0 -1
  592. package/dist/engines/k6/runtime/shared/assertion-details.d.ts +0 -19
  593. package/dist/engines/k6/runtime/shared/assertion-details.js +0 -2
  594. package/dist/engines/k6/runtime/shared/db-normalization.d.ts +0 -19
  595. package/dist/engines/k6/runtime/shared/db-normalization.js +0 -1
  596. package/dist/engines/k6/runtime/shared/error-body.d.ts +0 -3
  597. package/dist/engines/k6/runtime/shared/error-body.js +0 -1
  598. package/dist/engines/k6/runtime/shared/file-data.d.ts +0 -1
  599. package/dist/engines/k6/runtime/shared/file-data.js +0 -1
  600. package/dist/engines/k6/runtime/shared/fixture-engine.d.ts +0 -8
  601. package/dist/engines/k6/runtime/shared/fixture-engine.js +0 -2
  602. package/dist/engines/k6/runtime/shared/http-check-plan.d.ts +0 -29
  603. package/dist/engines/k6/runtime/shared/http-check-plan.js +0 -1
  604. package/dist/engines/k6/runtime/shared/http-parsing.d.ts +0 -10
  605. package/dist/engines/k6/runtime/shared/http-parsing.js +0 -3
  606. package/dist/engines/k6/runtime/shared/ordering.d.ts +0 -2
  607. package/dist/engines/k6/runtime/shared/ordering.js +0 -1
  608. package/dist/engines/k6/runtime/shared/properties.d.ts +0 -1
  609. package/dist/engines/k6/runtime/shared/properties.js +0 -1
  610. package/dist/engines/k6/runtime/suite.d.ts +0 -4
  611. package/dist/engines/k6/runtime/suite.js +0 -1
  612. package/dist/engines/k6/runtime/types.d.ts +0 -244
  613. package/dist/engines/k6/runtime/types.js +0 -1
  614. package/dist/engines/k6/runtime/xk6-sql-driver-postgres.d.ts +0 -7
  615. package/dist/engines/k6/runtime/xk6-sql-driver-postgres.js +0 -1
  616. package/dist/engines/managed-module-resolution-policy-input.d.ts +0 -8
  617. package/dist/engines/managed-module-resolution-policy.d.ts +0 -25
  618. package/dist/engines/managed-module-resolution.d.ts +0 -13
  619. package/dist/engines/manager.d.ts +0 -25
  620. package/dist/engines/node-vitest/adapter/adapter.d.ts +0 -16
  621. package/dist/engines/node-vitest/adapter/consumer-typescript-paths.d.ts +0 -17
  622. package/dist/engines/node-vitest/adapter/consumer-typescript-paths.js +0 -3
  623. package/dist/engines/node-vitest/adapter/evidence.d.ts +0 -33
  624. package/dist/engines/node-vitest/adapter/failure-details.d.ts +0 -3
  625. package/dist/engines/node-vitest/adapter/index.d.ts +0 -1
  626. package/dist/engines/node-vitest/adapter/reporter.d.ts +0 -7
  627. package/dist/engines/node-vitest/adapter/resolution.d.ts +0 -12
  628. package/dist/engines/node-vitest/adapter/resolution.js +0 -1
  629. package/dist/engines/node-vitest/adapter/workspace.d.ts +0 -19
  630. package/dist/engines/node-vitest/runtime/contract/index.d.ts +0 -64
  631. package/dist/engines/node-vitest/runtime/contract/index.js +0 -1
  632. package/dist/engines/node-vitest/runtime/data/http.d.ts +0 -4
  633. package/dist/engines/node-vitest/runtime/data/http.js +0 -1
  634. package/dist/engines/node-vitest/runtime/data/index.d.ts +0 -8
  635. package/dist/engines/node-vitest/runtime/data/primitives.d.ts +0 -1
  636. package/dist/engines/node-vitest/runtime/data/report.d.ts +0 -3
  637. package/dist/engines/node-vitest/runtime/data/runner.d.ts +0 -4
  638. package/dist/engines/node-vitest/runtime/data/runner.js +0 -1
  639. package/dist/engines/node-vitest/runtime/data/scenario.d.ts +0 -11
  640. package/dist/engines/node-vitest/runtime/data/types.d.ts +0 -384
  641. package/dist/engines/node-vitest/runtime/runtime-input.d.ts +0 -64
  642. package/dist/engines/playwright/adapter/adapter.d.ts +0 -18
  643. package/dist/engines/playwright/adapter/attachments.d.ts +0 -26
  644. package/dist/engines/playwright/adapter/config.d.ts +0 -45
  645. package/dist/engines/playwright/adapter/evidence-validation.d.ts +0 -9
  646. package/dist/engines/playwright/adapter/evidence.d.ts +0 -32
  647. package/dist/engines/playwright/adapter/failure-details.d.ts +0 -2
  648. package/dist/engines/playwright/adapter/index.d.ts +0 -3
  649. package/dist/engines/playwright/adapter/reporter.d.ts +0 -12
  650. package/dist/engines/playwright/runtime/config-file.d.ts +0 -2
  651. package/dist/engines/playwright/runtime/config.d.ts +0 -24
  652. package/dist/engines/playwright/runtime/index.d.ts +0 -4
  653. package/dist/engines/playwright/runtime/runtime-input.d.ts +0 -36
  654. package/dist/engines/playwright/runtime/sandbox.d.ts +0 -84
  655. package/dist/engines/runtime-input-file.d.ts +0 -27
  656. package/dist/environment/lifecycle-artifacts.d.ts +0 -7
  657. package/dist/environment/lifecycle-controller.d.ts +0 -60
  658. package/dist/environment/lifecycle.d.ts +0 -32
  659. package/dist/environment/maintenance.d.ts +0 -68
  660. package/dist/environment/planning-types.d.ts +0 -33
  661. package/dist/environment/ports.d.ts +0 -10
  662. package/dist/environment/ready-environment.d.ts +0 -16
  663. package/dist/environment/run-environment.d.ts +0 -39
  664. package/dist/environment/run-stopping.d.ts +0 -13
  665. package/dist/environment/runtime-contexts.d.ts +0 -37
  666. package/dist/environment/runtime-manager.d.ts +0 -87
  667. package/dist/environment/runtime-planning.d.ts +0 -16
  668. package/dist/environment/runtime-preparation.d.ts +0 -30
  669. package/dist/environment/runtime-service.d.ts +0 -12
  670. package/dist/environment/services.d.ts +0 -56
  671. package/dist/environment/state.d.ts +0 -15
  672. package/dist/environment/status-runtime.d.ts +0 -53
  673. package/dist/environment/template.d.ts +0 -164
  674. package/dist/evidence/artifact-config.d.ts +0 -14
  675. package/dist/evidence/artifact-validation.d.ts +0 -4
  676. package/dist/evidence/artifacts.d.ts +0 -75
  677. package/dist/evidence/bundles.d.ts +0 -27
  678. package/dist/evidence/ci-provenance.d.ts +0 -9
  679. package/dist/evidence/contracts.d.ts +0 -154
  680. package/dist/evidence/environment-record.d.ts +0 -22
  681. package/dist/evidence/file-evidence.d.ts +0 -23
  682. package/dist/evidence/finalization.d.ts +0 -9
  683. package/dist/evidence/index.d.ts +0 -8
  684. package/dist/evidence/logs.d.ts +0 -31
  685. package/dist/evidence/operation-timings.d.ts +0 -60
  686. package/dist/evidence/read-bundle.d.ts +0 -4
  687. package/dist/evidence/read-fields.d.ts +0 -21
  688. package/dist/evidence/read-provenance.d.ts +0 -2
  689. package/dist/evidence/read-redaction.d.ts +0 -2
  690. package/dist/evidence/read-retention.d.ts +0 -2
  691. package/dist/evidence/read-run.d.ts +0 -2
  692. package/dist/evidence/reading.d.ts +0 -28
  693. package/dist/evidence/reporting-types.d.ts +0 -349
  694. package/dist/evidence/reporting.d.ts +0 -3
  695. package/dist/evidence/secure-file-copy.d.ts +0 -24
  696. package/dist/evidence/setup-progress.d.ts +0 -63
  697. package/dist/evidence/step-output.d.ts +0 -16
  698. package/dist/evidence/storage-inventory.d.ts +0 -27
  699. package/dist/evidence/storage.d.ts +0 -25
  700. package/dist/evidence/timing.d.ts +0 -23
  701. package/dist/file.d.ts +0 -11
  702. package/dist/index.d.ts +0 -4
  703. package/dist/kernel/error-serialization.d.ts +0 -40
  704. package/dist/kernel/redaction.d.ts +0 -2
  705. package/dist/kernel/unknown-values.d.ts +0 -11
  706. package/dist/kernel/work-metrics.d.ts +0 -6
  707. package/dist/process/capture-output.d.ts +0 -7
  708. package/dist/process/command-shell.d.ts +0 -1
  709. package/dist/process/environment-overlay.d.ts +0 -17
  710. package/dist/process/environment.d.ts +0 -33
  711. package/dist/process/files/contained-path.d.ts +0 -8
  712. package/dist/process/files/directory-status.d.ts +0 -14
  713. package/dist/process/files/dotenv.d.ts +0 -2
  714. package/dist/process/files/file-snapshot.d.ts +0 -34
  715. package/dist/process/files/json-file.d.ts +0 -11
  716. package/dist/process/files/paths.d.ts +0 -3
  717. package/dist/process/files/product-dir.d.ts +0 -1
  718. package/dist/process/files/temp-dir.d.ts +0 -11
  719. package/dist/process/host/metadata.d.ts +0 -6
  720. package/dist/process/host/platform.d.ts +0 -57
  721. package/dist/process/host-command.d.ts +0 -4
  722. package/dist/process/host-inspection-command.d.ts +0 -13
  723. package/dist/process/host-process.d.ts +0 -19
  724. package/dist/process/index.d.ts +0 -9
  725. package/dist/process/lifecycle-fence.d.ts +0 -10
  726. package/dist/process/locks.d.ts +0 -28
  727. package/dist/process/process-contracts.d.ts +0 -78
  728. package/dist/process/process-group.d.ts +0 -12
  729. package/dist/process/process-identity.d.ts +0 -27
  730. package/dist/process/process-limits.d.ts +0 -6
  731. package/dist/process/process.d.ts +0 -12
  732. package/dist/process/readiness.d.ts +0 -43
  733. package/dist/process/recovery.d.ts +0 -3
  734. package/dist/process/runs/inspection.d.ts +0 -5
  735. package/dist/process/runs/manifest.d.ts +0 -86
  736. package/dist/process/runs/ownership.d.ts +0 -11
  737. package/dist/process/runs/product-state.d.ts +0 -2
  738. package/dist/process/runs/schema.d.ts +0 -3
  739. package/dist/process/runs/stop.d.ts +0 -19
  740. package/dist/process/runtime-socket.d.ts +0 -4
  741. package/dist/process/service-processes.d.ts +0 -13
  742. package/dist/process/shutdown.d.ts +0 -10
  743. package/dist/process/stoppable-settlement.d.ts +0 -26
  744. package/dist/process/url-redaction.d.ts +0 -1
  745. package/dist/process/windows-cli-owner.d.ts +0 -2
  746. package/dist/process/windows-native.d.ts +0 -12
  747. package/dist/process/windows-process-identity.d.ts +0 -4
  748. package/dist/repo/ci-environment.d.ts +0 -7
  749. package/dist/repo/fingerprint-policy.d.ts +0 -13
  750. package/dist/repo/git-path.d.ts +0 -2
  751. package/dist/repo/history.d.ts +0 -17
  752. package/dist/repo/managed-git.d.ts +0 -10
  753. package/dist/repo/metadata.d.ts +0 -3
  754. package/dist/repo/physical-tree.d.ts +0 -37
  755. package/dist/repo/publish-directory.d.ts +0 -20
  756. package/dist/repo/remote.d.ts +0 -6
  757. package/dist/repo/state.d.ts +0 -26
  758. package/dist/repo/tree-listing.d.ts +0 -10
  759. package/dist/steps/build-config.d.ts +0 -34
  760. package/dist/steps/configured-step-testkit-modules.d.ts +0 -19
  761. package/dist/steps/configured-steps.d.ts +0 -51
  762. package/dist/steps/environment.d.ts +0 -82
  763. package/dist/steps/index.d.ts +0 -2
  764. package/dist/steps/module-runner.d.ts +0 -1
  765. package/dist/steps/run.d.ts +0 -36
  766. package/dist/toolchains/archive.d.ts +0 -53
  767. package/dist/toolchains/esbuild.d.ts +0 -9
  768. package/dist/toolchains/host-node-support.d.ts +0 -3
  769. package/dist/toolchains/index.d.ts +0 -114
  770. package/dist/toolchains/managed-node-releases.d.ts +0 -28
  771. package/dist/toolchains/node-cache.d.ts +0 -20
  772. package/dist/toolchains/node-integrity.d.ts +0 -18
  773. package/dist/toolchains/node-provisioning.d.ts +0 -29
  774. package/dist/toolchains/semver.d.ts +0 -13
  775. package/dist/toolchains/shared-operation.d.ts +0 -11
  776. package/dist/toolchains/timeouts.d.ts +0 -1
  777. package/dist/toolchains/vitest/index.d.ts +0 -13
  778. package/dist/tracker/gitea.d.ts +0 -14
  779. package/dist/tracker/github.d.ts +0 -14
  780. package/dist/tracker/http.d.ts +0 -33
  781. package/dist/tracker/index.d.ts +0 -4
  782. package/dist/tracker/redaction.d.ts +0 -4
  783. package/dist/tracker/requests.d.ts +0 -17
  784. package/dist/tracker/responses.d.ts +0 -24
  785. package/dist/tracker/types.d.ts +0 -51
  786. package/docs/engine-architecture.md +0 -682
  787. /package/dist/engines/{k6/runtime/runtime-globals.js → api/profiles.js} +0 -0
package/README.md CHANGED
@@ -4,17 +4,19 @@
4
4
 
5
5
  # @gfdelarue/testkit
6
6
 
7
- `@gfdelarue/testkit` is the orchestrator and managed distribution for product test
8
- suites. It discovers `*.testkit.ts` files, infers suite ownership from the
9
- filesystem, starts local services, provisions Docker-managed local Postgres
10
- databases, schedules suites, and collects results and artifacts. Coding agents
11
- use it to write tests and hunt bugs, and CI uses it to run those tests. It needs
12
- no account or server.
7
+ `@gfdelarue/testkit` is a test runner and CLI for coding agents and CI. It runs
8
+ a repository's ordinary Vitest test files, such as `src/sum.test.ts`, as they
9
+ are, and typed `*.testkit.ts` suites for Playwright and Node. It assigns
10
+ tests to groups by directory, starts only the local services a group uses,
11
+ provisions Docker-managed local Postgres databases when a group needs one,
12
+ schedules the files, and collects results and artifacts. It starts no service
13
+ for a test whose group uses none. Coding agents use Testkit to write tests
14
+ and hunt bugs, and CI uses it to run those tests. It needs no account or
15
+ server.
13
16
 
14
17
  Testkit exists to replace false confidence with inspectable execution evidence.
15
- It cannot prove that an entire application works. It can make narrower claims
16
- that deserve trust: these tests genuinely executed; these were the runtimes,
17
- fixtures, and inputs supplied to the run; this is what passed, failed, was
18
+ It makes claims that deserve trust: these tests genuinely executed; these were
19
+ the runtimes, fixtures, and inputs supplied to the run; this is what passed, failed, was
18
20
  skipped, or did not run; and this is the evidence behind the result. Testkit
19
21
  retains provenance, logs, and verifiable artifacts so those claims can be
20
22
  audited rather than accepted on the strength of a green check mark.
@@ -41,8 +43,9 @@ npm run testkit
41
43
 
42
44
  The command exits zero only when at least one test file completed and every
43
45
  required file passed. Test failures, infrastructure failures, interrupted or
44
- not-run files, an all-skipped selection, and an empty selection all fail the
45
- gate. `npm run testkit -- --allow-empty` is an explicit escape hatch for a job
46
+ not-run files, a skipped file, and an empty selection all fail the gate. A file
47
+ that a `skip` rule or its own `skip` metadata skips fails the gate even when
48
+ every other file passes, and `--allow-empty` doesn't change that. `npm run testkit -- --allow-empty` is an explicit escape hatch for a job
46
49
  whose empty selection is intentional; it should not be the default CI command.
47
50
 
48
51
  Testkit writes the immutable, verifiable run bundle under
@@ -111,6 +114,23 @@ only when the test fails. It stops when tracked files have uncommitted changes,
111
114
  when no remote branch contains the commit, or when an open issue already covers
112
115
  the test. The token needs permission to create issues and labels.
113
116
 
117
+ `testkit bug record` saves the `--vitest-config`, `--vitest-project`, and
118
+ `--vitest-runner` it was given in the draft, and asks for `--vitest-project`
119
+ when two Vitest projects run the test's file. `testkit bug file` reruns with
120
+ them, and the issue's reproduce command carries them. With
121
+ `TESTKIT_REQUIRE_COMPLETE_SUITE=1`, it reruns every file the selection lists,
122
+ then reads the recorded failure.
123
+
124
+ `testkit bug file` takes the same [supplied values](#run-against-a-supplied-environment)
125
+ as `testkit run` and reruns the test on them. The issue's reproduce command
126
+ carries those flags, so `testkit bug file` refuses inputs that would put a raw
127
+ secret in it. It takes `--env-from` and `--environment-file`, and takes
128
+ `--base-url` and `--endpoint` only for URLs without credentials, a query, or a
129
+ fragment. It refuses `--database` and `--env`; pass those values through
130
+ `--env-from` or the environment file. The command runs from the repository
131
+ root, so a relative `--environment-file` needs `testkit bug file` run from
132
+ there too.
133
+
114
134
  ### Name the product's areas
115
135
 
116
136
  Add `areas` to `testkit.config.ts` so that each issue names the part of the
@@ -243,28 +263,27 @@ the report to the chosen platform.
243
263
 
244
264
  ## Execution engines
245
265
 
246
- Testkit has three explicit built-in engines rather than one implicit runtime:
247
-
248
- - `k6` runs `int`, `e2e`, `load`, and `dal` files with native k6
249
- and xk6-sql semantics.
250
- - `node-vitest` runs purpose-built `data` and `contract` files with Node and
251
- Vitest.
252
- - `playwright` runs `ui` files with Playwright and a managed Chromium.
253
-
254
- Every test file must export `defineFile({ engine: "..." })`. The filename
255
- suffix supplies the public test type; Testkit validates that explicit pair and
256
- derives a runner profile. It never infers a default engine or falls back to a
257
- different engine.
258
-
259
- Consumers declare only `@gfdelarue/testkit`. The root package owns the three
260
- adapters and exact JavaScript tools it executes. The k6 engine ships as
261
- separately versioned npm packages built from
262
- [gfdelarue/testkit-k6](https://github.com/gfdelarue/testkit-k6); npm installs
263
- the compatible k6 target package and its shared source package through the
264
- root's optional dependency graph, and Testkit resolves that direct executable
265
- without consulting `PATH`. Node/Vitest and Playwright resolve their managed public packages from the
266
- installed Testkit root, so product-local tool copies do not select Testkit's
267
- runtime.
266
+ Testkit has two built-in engines:
267
+
268
+ - `node-vitest` runs plain Vitest test files with Node and Vitest.
269
+ - `playwright` runs `ui` files with Playwright and managed Chromium.
270
+
271
+ A plain Vitest test file needs no Testkit suffix, directory, metadata, or
272
+ wrapper. It keeps its native `vitest` imports and its repository paths, has no
273
+ test type, and runs under the engine's general `node-vitest` profile. Testkit
274
+ does not evaluate `testkit.config.ts` in the test's process, and it installs the
275
+ values of the services the test's group uses before the test's imports
276
+ evaluate. See [Plain Vitest test files](#plain-vitest-test-files) for which
277
+ files Testkit runs.
278
+
279
+ Every typed `*.testkit.ts` file must export `defineFile({ engine: "..." })`. The
280
+ filename suffix supplies the public test type; Testkit validates that explicit
281
+ pair and derives a runner profile. It never infers a default engine for a typed
282
+ file or falls back to a different engine.
283
+
284
+ Consumers declare only `@gfdelarue/testkit`, which installs the exact versions
285
+ of the tools its engines run. A product's own copies of Vitest, Playwright, or
286
+ their tools don't change what Testkit runs.
268
287
 
269
288
  Playwright browser installation is explicit:
270
289
 
@@ -273,28 +292,17 @@ npx @gfdelarue/testkit setup playwright
273
292
  npx @gfdelarue/testkit setup playwright --with-deps # CI and fresh hosts
274
293
  ```
275
294
 
276
- Setup and execution share one canonical Playwright registry. Normal runs never
277
- install or download a browser. See [Engine Architecture](docs/engine-architecture.md)
278
- for the authoring, installation, trust, and failure contracts.
279
-
280
- Managed commands use the same Execa-backed adapter on hosted Linux and Windows
281
- runners. Testkit passes exact argv, cwd, and environment values, bounds captured
282
- output, applies cancellation and timeouts, and asks the platform adapter to
283
- terminate descendants. Persisted recovery identities remain birth-fenced so a
284
- later cleanup does not signal a reused process ID.
285
-
286
- The five k6 target packages share one exact-version source package that carries
287
- the AGPL corresponding source archive and licenses. Runtime evidence records
288
- the package name and version and the k6 version observed by executing the
289
- installed binary. The
290
- [testkit-k6 README](https://github.com/gfdelarue/testkit-k6#readme) documents
291
- the reproducible build, the expected executable digests, and the license
292
- notes.
293
-
294
- Managed task environment is engine input rather than ambient host state. k6
295
- suites read it through native `__ENV`; Node/Vitest data and contract callbacks
296
- read immutable, enumerable `context.env`; and Playwright suites can read the
297
- same contract from `readPlaywrightContext()?.env`. Managed Node processes also
295
+ Normal runs never install or download a browser.
296
+ [Installation](docs/engines.md#installation) covers what an install contains,
297
+ the supported platforms, and how to recover a broken one.
298
+ [Typechecking](docs/engines.md#typechecking) covers the supported TypeScript
299
+ versions and `tsconfig.playwright.json`.
300
+
301
+ Testkit bounds the output it captures from each command it runs, and asks the
302
+ command and its descendants to stop on cancellation or timeout.
303
+
304
+ Managed task environment is engine input. Playwright suites read it from
305
+ `readPlaywrightContext()?.env`. Managed Node processes also
298
306
  expose each value as an immutable, non-enumerable `process.env.KEY` property so
299
307
  libraries that read `process.env` directly see it, but non-Node child processes
300
308
  do not receive consumer values unless the suite explicitly passes them.
@@ -304,21 +312,23 @@ reserves `APPDATA`, `ComSpec`, `HOME`, `LANG`, `LC_ALL`, `LOCALAPPDATA`, `PATH`,
304
312
  `PATHEXT`, `SystemRoot`, `TEMP`, `TMP`, `TMPDIR`, `USERPROFILE`,
305
313
  `XDG_CACHE_HOME`, `XDG_CONFIG_HOME`, and `XDG_DATA_HOME`. In addition:
306
314
 
307
- - k6 reserves the `K6_` and `TESTKIT_` prefixes plus `BASE_URL` and `GODEBUG`;
308
- - Node/Vitest reserves the `TESTKIT_` and `VITEST_` prefixes plus `BASE_URL`,
309
- `FORCE_COLOR`, `NODE_OPTIONS`, `NO_COLOR`, and `UPDATE_SNAPSHOT`;
315
+ - Node/Vitest reserves `BASE_URL`, `FORCE_COLOR`, `NODE_OPTIONS`, `NO_COLOR`,
316
+ `UPDATE_SNAPSHOT`, `VITEST_POOL_ID`, `VITEST_WORKER_ID`, and the `TESTKIT_`
317
+ keys it sets itself, listed in
318
+ [Environment](docs/engines.md#environment);
319
+ other `TESTKIT_` and `VITEST_` keys, such as
320
+ `TESTKIT_REQUIRE_COMPLETE_SUITE`, pass through, though Vitest's own `TEST`,
321
+ `VITEST`, `VITEST_MODE`, `FORCE_TTY`, `MODE`, `DEV`, `PROD`, and `SSR`
322
+ replace a group's values;
310
323
  - Playwright reserves the `TESTKIT_`, `PLAYWRIGHT_`, `PWTEST_`, and `PW_TEST_`
311
324
  prefixes plus `BASE_URL`, `FORCE_COLOR`, `NODE_OPTIONS`, and `NO_COLOR`.
312
325
 
313
326
  Case-variant duplicates and invalid names or NUL-containing values fail instead
314
- of being normalized. k6 accepts at most 128 consumer entries and 16,384
315
- environment-block character units. Node/Vitest accepts at most 512 entries and
327
+ of being normalized. Node/Vitest accepts at most 512 entries and
316
328
  4 MiB of UTF-8 key/value data. Playwright accepts at most 512 entries and 2 MiB
317
- of UTF-8 key/value data. The smaller k6 limit is required because native k6
318
- receives consumer values in its physical process environment; Node/Vitest and
319
- Playwright transport them through owned runtime-input files. See
320
- [Engine Architecture](docs/engine-architecture.md#managed-environment) for the
321
- complete transport and aggregate-input limits.
329
+ of UTF-8 key/value data. [Environment](docs/engines.md#environment) lists the
330
+ complete limits, and [Named endpoints and databases](#named-endpoints-and-databases)
331
+ covers the environment API.
322
332
 
323
333
  ## Usage
324
334
 
@@ -335,28 +345,27 @@ npx @gfdelarue/testkit discover --json > .testkit/discovery.json
335
345
  npx @gfdelarue/testkit discover --output .testkit/discovery.json
336
346
 
337
347
  # Filter by type
338
- npx @gfdelarue/testkit --type int
339
- npx @gfdelarue/testkit --type dal
340
- npx @gfdelarue/testkit --type e2e
341
- npx @gfdelarue/testkit --type int,e2e,dal
342
348
  npx @gfdelarue/testkit --type ui
349
+ npx @gfdelarue/testkit --type plain
343
350
 
344
- # Parallel file execution
351
+ # Run up to 8 test tasks at once
345
352
  npx @gfdelarue/testkit --task-concurrency 8
346
353
 
347
354
  # One file-level wall clock budget for every suite file
348
355
  npx @gfdelarue/testkit --file-timeout-seconds 60
349
356
 
350
- # Specific service / suite
351
- npx @gfdelarue/testkit --service frontend --type ui -s navigation
352
- npx @gfdelarue/testkit --service api --type int -s health
353
- npx @gfdelarue/testkit --type int,e2e,dal -s dal:queries
357
+ # Specific group / suite
358
+ npx @gfdelarue/testkit --group frontend --type ui -s navigation
359
+ npx @gfdelarue/testkit --group api -s health
360
+ npx @gfdelarue/testkit --group api --type plain
361
+ npx @gfdelarue/testkit --type ui -s ui:checkout
354
362
 
355
363
  # Exact file
356
- npx @gfdelarue/testkit --type int --file __testkit__/health.int.testkit.ts
364
+ npx @gfdelarue/testkit --type ui --file __testkit__/checkout.ui.testkit.ts
365
+ npx @gfdelarue/testkit run --file src/sum.test.ts
357
366
 
358
367
  # Temporarily ignore repo-declared skip rules
359
- npx @gfdelarue/testkit --ignore-skip-rules --file __testkit__/billing.int.testkit.ts
368
+ npx @gfdelarue/testkit --ignore-skip-rules --file src/billing.test.ts
360
369
 
361
370
  # Lifecycle
362
371
  npx @gfdelarue/testkit status
@@ -371,14 +380,27 @@ npx @gfdelarue/testkit db schema refresh --service api
371
380
  npx @gfdelarue/testkit db schema verify --service api
372
381
  ```
373
382
 
383
+ `--type ui` and typed suite selectors such as `ui:checkout` select typed files
384
+ only. `--type plain` selects plain Vitest test files only. A run without
385
+ `--type`, or with `--type all`, runs both. `--type` narrows the other filters,
386
+ so `--group api --type plain` runs the `api` group's plain files and none of its
387
+ UI files.
388
+
389
+ `run`, `discover`, `artifacts`, `workspace`, and `typecheck` select test groups
390
+ with `--group`. `status`, `destroy`, `cleanup`, and `db schema` take `--service`
391
+ to name a service.
392
+
374
393
  Database cleanup is scoped to the selected product. `cleanup --resources --global`
375
394
  is retired; use `--dir <product> cleanup --resources` for each product. After
376
395
  deleting a product directory, remove its orphaned containers and volumes
377
396
  explicitly through Docker.
378
397
 
379
398
  Use `execution.taskConcurrency` in config and `--task-concurrency` on the CLI.
380
- `execution.workers`, `presets.nodeNext({ workers })` and `--workers` remain
381
- deprecated aliases through 0.2.x and will be removed in 0.3.0. When both names are
399
+ A task is a typed file, a batch of typed files, or one Vitest invocation of
400
+ plain files. Plain files in one invocation run with Vitest's file parallelism,
401
+ which `taskConcurrency` doesn't limit.
402
+ `execution.workers`, `presets.nodeNext({ workers })` and `--workers` are
403
+ deprecated aliases. When both names are
382
404
  provided in config or on the CLI, their values must match. The CLI value overrides
383
405
  config after those checks. See the [execution settings](testkit-config-schema.md#execution).
384
406
 
@@ -386,15 +408,125 @@ Batch `run` output stays intentionally short: one line per completed file,
386
408
  concise failure blocks, and a final summary. Service logs, captured runtime
387
409
  output, and emitted artifacts are persisted under `.testkit/results/`.
388
410
 
411
+ Text output is coloured in a terminal and plain otherwise. As in Node,
412
+ `NO_COLOR` turns colour off and `FORCE_COLOR` overrides it: `0` or `false`
413
+ turns colour off and any other value turns it on.
414
+
389
415
  Testkit seals an immutable local run bundle. `testkit run` prints
390
416
  `Run ID: <id>` after sealing in text output and returns `runId` in `--json`
391
417
  output, for passed and failed runs.
392
418
 
393
- Test execution also maintains a scheduler cache at `.testkit/timings.json`.
394
- Completed file-level task durations are used to rank future runs with a
395
- longest-estimated-duration-first policy, so slow files start earlier when
396
- workers are available. Run artifacts include compact scheduler metadata under
397
- `planning` so ordering decisions are inspectable.
419
+ Testkit starts the files that took longest in earlier runs first, so slow
420
+ files start while workers are free. The run document records the order under
421
+ `planning`.
422
+
423
+ ### Run against a supplied environment
424
+
425
+ `testkit run` can take the values its tests need instead of provisioning them.
426
+ Every selected group then runs on those values alone. Testkit starts none of
427
+ the group's services, creates no database, and runs no `.setup.*` module beside
428
+ a test. It also ignores the env values and env files those services declare, so
429
+ supply every value the tests read. Groups, selection, skip rules, and other
430
+ policy work as before.
431
+
432
+ ```bash
433
+ npx @gfdelarue/testkit run --group checks \
434
+ --base-url http://127.0.0.1:3000 \
435
+ --endpoint api=http://127.0.0.1:3001 \
436
+ --database store=postgres://app:password@127.0.0.1:5432/store \
437
+ --env FEATURE_FLAG=on \
438
+ --env-from API_TOKEN=CI_API_TOKEN
439
+
440
+ npx @gfdelarue/testkit run --environment-file supplied.json
441
+ ```
442
+
443
+ - `--base-url URL` supplies the default endpoint, which tests receive as
444
+ `BASE_URL`.
445
+ - `--endpoint NAME=URL` supplies a named endpoint, usually named after the
446
+ service it stands in for.
447
+ - `--database NAME=URL` supplies a named PostgreSQL database.
448
+ - `--env NAME=VALUE` supplies an environment value.
449
+ - `--env-from NAME=HOST_KEY` supplies the value of host variable `HOST_KEY`
450
+ under `NAME`.
451
+ - `--environment-file PATH` reads values, prepared setup data, and handle
452
+ descriptors from a file.
453
+
454
+ Every flag except `--base-url` and `--environment-file` can be repeated. Tests
455
+ read named values with `endpoint(name)` and `database(name)` from
456
+ `@gfdelarue/testkit/environment`. An `--env`, `--env-from`, or file `env` entry
457
+ named `BASE_URL` or `DATABASE_URL` sets the default endpoint or database, and an
458
+ empty database URL means no database. Without an explicit default, a group's
459
+ `BASE_URL` and `DATABASE_URL` are those of the first service in its `uses` that
460
+ has a supplied endpoint or database. Either way the default is also available
461
+ by the group's name, as in a configured run, so it may not differ from a named
462
+ endpoint or database with that name.
463
+
464
+ Testkit reads host variables only through `--env-from`. A `BASE_URL` or
465
+ `DATABASE_URL` already set in the shell does not switch a run to supplied
466
+ values. Two inputs may define the same value only if they agree. Any other
467
+ repeat, such as `--base-url` and `--env BASE_URL` with different URLs, or a
468
+ missing `--env-from` host variable, fails the run before it starts. Before a
469
+ test starts, each supplied endpoint must give a valid HTTP response. A 4xx or
470
+ 5xx response counts as valid.
471
+
472
+ Each supplied database must also accept a connection before a test starts.
473
+ Testkit connects, signs in with the URL's credentials, runs `SELECT 1`, and
474
+ disconnects. It handles trust, password, md5, and SCRAM-SHA-256 authentication.
475
+ TLS follows the URL's `sslmode` as in libpq. The default, `prefer`, tries TLS
476
+ first and then a plain connection, and `allow` tries them the other way round.
477
+ `verify-ca` and `verify-full` need `sslrootcert`, set to a CA file or to
478
+ `system`, and `sslrootcert=system` alone means `verify-full`. `host=/path`
479
+ connects through the Unix socket in that directory and ignores `sslmode`. The
480
+ check reads only the URL. It ignores `PG*` variables, `~/.pgpass`, and
481
+ `~/.postgresql/root.crt`, and it presents no client certificate. A database
482
+ that is unreachable, fails TLS, rejects the credentials, does not exist, or is
483
+ not PostgreSQL fails the run before any test starts. The message names the
484
+ database, never its URL.
485
+
486
+ The environment file is JSON:
487
+
488
+ ```json
489
+ {
490
+ "version": 1,
491
+ "baseUrl": "http://127.0.0.1:3000",
492
+ "databaseUrl": "postgres://app:password@127.0.0.1:5432/app",
493
+ "endpoints": { "api": "http://127.0.0.1:3001" },
494
+ "databases": { "store": "postgres://app:password@127.0.0.1:5432/store" },
495
+ "env": { "FEATURE_FLAG": "on" },
496
+ "ui": {},
497
+ "setupData": { "actor": "Grace" },
498
+ "handles": {
499
+ "webhook": {
500
+ "kind": "http-capture",
501
+ "url": "http://127.0.0.1:41000/webhook",
502
+ "requestsUrl": "http://127.0.0.1:41000/__testkit__/<capability>/requests",
503
+ "resetUrl": "http://127.0.0.1:41000/__testkit__/<capability>/requests"
504
+ }
505
+ }
506
+ }
507
+ ```
508
+
509
+ Only `version` is required, and this Testkit reads version 1. `baseUrl`,
510
+ `databaseUrl`, and `ui` may be `null`, as an environment rendered by Testkit
511
+ writes them, and `null` there counts as absent. Tests receive
512
+ `setupData` as the data a setup module would have returned: `{}` when the file
513
+ leaves it out, and `null` when the file sets it to `null`. `handles` entries must each name a `kind`, and tests reach them with
514
+ [`handle(name)`](#handles). An `http-capture` entry holds the `url`,
515
+ `requestsUrl`, and `resetUrl` that a capture sidecar reports, and gives the
516
+ test the same handle a setup module's capture would. Other kinds are handed on,
517
+ and `handle()` throws `EnvironmentHandleError` with `"unsupported"` for them. Any other field, such as
518
+ service or database configuration, is rejected. Errors name the file and the
519
+ field and do not repeat the file's contents.
520
+
521
+ A run records the values its tests received in this shape. Each acquisition in
522
+ `run.json`'s `environment` section has them under `values`, with secrets and
523
+ capture capabilities redacted. Its `setupData` is the supplied setup data or
524
+ what the file's setup module returned, and `{}` without either. Sealing redacts
525
+ setup data under a key that names a secret or that `redact` lists, and records
526
+ the rest as returned, so list any other sensitive key in `redact`. With
527
+ `"version": 1` added, an
528
+ acquisition's `values` are an environment file that gives a second run the
529
+ same values, unless redaction changed one of them.
398
530
 
399
531
  ## Tooling Adapters
400
532
 
@@ -430,23 +562,17 @@ export default defineConfig({
430
562
  // playwright.config.ts
431
563
  import { defineTestkitConfig, devices } from "@gfdelarue/testkit/playwright";
432
564
 
433
- export default defineTestkitConfig(
434
- {
435
- testDir: "./__testkit__",
436
- projects: [{ name: "chromium", use: { ...devices["Desktop Chrome"] } }],
437
- },
438
- {
439
- dotenvFiles: [".env.local"],
440
- },
441
- );
565
+ export default defineTestkitConfig({
566
+ testDir: "./__testkit__",
567
+ projects: [{ name: "chromium", use: { ...devices["Desktop Chrome"] } }],
568
+ });
442
569
  ```
443
570
 
444
571
  Managed UI suites use `@gfdelarue/testkit/playwright` for Testkit's Playwright
445
572
  helpers and re-exported authoring API. They may also import exact public exports
446
573
  from the managed `@playwright/test`, `playwright`, and `playwright-core`
447
- packages. Testkit pins both runtime resolution and type resolution, so
448
- product-local Playwright installs cannot split the test registry or supply a
449
- different compile-time API.
574
+ packages. Those imports resolve to Testkit's pinned versions at run time and
575
+ when typechecking, whatever Playwright version the product installs.
450
576
 
451
577
  For scripts and app-runtime code, `@gfdelarue/testkit/env` provides shared helpers
452
578
  for managed runtime detection, dotenv loading, and local-database safety:
@@ -541,25 +667,27 @@ and attempts under `environment.services`. A config that sets `local.port`, the
541
667
  `port` option of `app.node` or `app.next`, or a literal port in `local.baseUrl`
542
668
  or `local.readyUrl` fails to load; delete it and use `{port}`.
543
669
 
544
- Every test file declares its execution engine and any file-local scheduling
545
- metadata next to the test:
670
+ Every typed test file declares its execution engine and any file-local
671
+ scheduling metadata next to the test. A plain Vitest test file declares
672
+ neither:
546
673
 
547
674
  ```ts
548
675
  import { defineFile } from "@gfdelarue/testkit";
549
676
 
550
677
  export const testkit = defineFile({
551
- engine: "k6",
678
+ engine: "node-vitest",
552
679
  skip: "Billing is currently unavailable locally",
553
- locks: ["global-worker-loop"],
554
680
  });
555
681
  ```
556
682
 
557
- The engine value is a required literal. The supported public authoring paths
558
- are `@gfdelarue/testkit/k6`, `@gfdelarue/testkit/node-vitest/data`,
559
- `@gfdelarue/testkit/node-vitest/contract`, and `@gfdelarue/testkit/playwright`.
560
- Node/Vitest and Playwright suites may also import exact public APIs directly
561
- from Testkit's managed `vitest`, `@playwright/test`, `playwright`, and
562
- `playwright-core` packages.
683
+ A file can also name `locks`.
684
+ [Give each test its own data](#give-each-test-its-own-data) says when to use
685
+ one.
686
+
687
+ In a typed file, the engine value is a required literal. The supported public
688
+ authoring path is `@gfdelarue/testkit/playwright`. Playwright suites may also
689
+ import exact public APIs directly from Testkit's managed `@playwright/test`,
690
+ `playwright`, and `playwright-core` packages.
563
691
  There are no legacy aliases or implicit-runner compatibility paths.
564
692
 
565
693
  `testkit.config.ts` is optional. Without it, Testkit runs every test file with
@@ -584,10 +712,9 @@ for:
584
712
  - repo-declared suite/file skip policies with explicit reasons
585
713
 
586
714
  `runtime.prepare` is the generic build-once hook for shared runtimes. It runs
587
- once per runtime generation before local services start, fingerprints declared
588
- inputs, and writes cache state under the service runtime directory. This is the
589
- right way to move expensive browser targets from `next dev` / watch mode to
590
- stable build-and-start flows.
715
+ once per runtime generation before local services start, and runs again when
716
+ its declared inputs change. This is the right way to move expensive browser
717
+ targets from `next dev` / watch mode to stable build-and-start flows.
591
718
 
592
719
  Setup steps (`runtime.prepare`, database `template`, and other Testkit-run
593
720
  Node commands) inherit `NODE_OPTIONS`. `TESTKIT_SETUP_NODE_OPTIONS` appends
@@ -598,19 +725,14 @@ setup processes, so set it deliberately.
598
725
  Testkit-owned Docker resources are labelled at creation time so they can be
599
726
  found again even if product-local `.testkit` state is deleted. `testkit destroy`
600
727
  removes labelled resources for the current product after service state is
601
- destroyed. `testkit cleanup --resources` cleans stale resources for the current
602
- product. Both commands filter Docker inventory by the product ownership label
603
- and preserve other products, including idle caches. Host-local Postgres data
604
- lives in a Testkit-labelled named Docker
605
- volume instead of an image-declared anonymous volume, so cleanup can account for
606
- and remove it safely. Each created volume receives a never-reused random
607
- generation in both its name and labels; cleanup re-inspects the complete
608
- generation immediately before Docker's name-based removal. Docker has no
609
- immutable volume ID or conditional-delete API, so direct Docker-daemon
610
- principals are part of the host trust boundary and must not replace that
611
- unguessable name during the final removal call. Testkit serializes database
612
- mutations for the same product; independent products can proceed concurrently.
613
- Use `cleanup --resources --dry-run` to preview the selected product's cleanup.
728
+ destroyed. It leaves run results under `.testkit/results` and other files
729
+ under `.testkit`, such as the compiled config and timings, in place. `testkit cleanup --resources` cleans stale resources for the current
730
+ product. Both commands remove only the current product's resources and leave
731
+ other products' alone, including their idle caches. Host-local Postgres data
732
+ lives in a Testkit-labelled named Docker volume, so cleanup can find and remove
733
+ it. Database changes for one product run one at a time; other products don't
734
+ wait for them. Use `cleanup --resources --dry-run` to preview the selected
735
+ product's cleanup.
614
736
 
615
737
  `database.template` is the database-side equivalent for reusable template DB
616
738
  state. When `database.sourceSchema` is configured, Testkit treats the configured
@@ -622,34 +744,16 @@ schema still matches the source dump. If local replay differs, Testkit refreshes
622
744
  from the source once for the current cache key and retries. If it still differs,
623
745
  the run fails with schema diagnostics under `.testkit/results/schema`.
624
746
 
625
- Source schema cache keys are derived automatically from repo state:
626
-
627
- - clean git worktrees use `commits/<sha>`
628
- - dirty git worktrees use `dirty/<sha>-<fingerprint>`
629
- - non-git directories use `nogit/<fingerprint>`
630
-
631
- Dirty worktree fingerprints compare the complete index to `HEAD`, then compare
632
- the physical worktree bytes, node types, and executable modes to the index.
633
- This intentionally detects changes hidden by Git stat flags, sparse checkout,
634
- or clean/smudge and EOL normalization. For untracked files, repository
635
- `.gitignore` files are respected; host-global excludes, `core.excludesFile`, and
636
- `.git/info/exclude` do not alter Testkit's cache identity.
637
-
638
- Fingerprint Git commands disable replacement objects and optional locks and do
639
- not read system or user Git configuration. Git path records are decoded from
640
- their byte-preserving quoted form and must be valid UTF-8. Uninitialized
641
- gitlinks and special filesystem nodes fail explicitly instead of being assigned
642
- an incomplete or ambiguous identity. Testkit reads the complete HEAD, index,
643
- untracked, submodule, and physical snapshot twice and fails if the observations
644
- differ.
645
-
646
- Physical reads open regular files by descriptor and verify every pathname
647
- component before and after use; on POSIX, ancestor directory descriptors remain
648
- open during the read. Node does not expose a portable `openat`/Windows reparse-
649
- handle traversal API, so this detects ordinary replacement races but is not a
650
- security boundary against a same-user adversary able to perform a perfectly
651
- timed replace-and-restore ABA attack. Run Testkit against source trees that are
652
- not concurrently controlled by an untrusted local process.
747
+ Testkit keys the source schema cache on the repository's state. A clean
748
+ worktree shares the entry for its commit. A dirty worktree or a directory
749
+ outside Git gets its own entry, keyed on the content of its files.
750
+
751
+ The key changes when a file's bytes, type, or executable mode changes, including
752
+ changes Git's own status can hide, such as sparse checkout or line-ending
753
+ normalization, and when a submodule moves. Untracked files are included unless
754
+ the repository's `.gitignore` files exclude them; global or per-clone Git
755
+ excludes don't count. A path Testkit can't read, or a tree that changes while
756
+ Testkit reads it, fails the run instead of producing a key.
653
757
 
654
758
  Testkit-owned outputs are always excluded from fingerprints: `.testkit/`,
655
759
  and `.next-testkit/`. `testkit.config.ts` is normal repo
@@ -674,22 +778,16 @@ Template setup executes in three explicit phases:
674
778
 
675
779
  Schema drift is checked after each successful `migrate` and `seed` step, and
676
780
  `verify` only runs once local replay matches the source schema. Source refreshes
677
- use Dockerized `pg_dump --schema-only --no-owner --no-privileges`, so
678
- seed/reference data is never written into the baseline and host `pg_dump`
679
- installations are not used. Keep schema-changing setup in its own step where
680
- possible; a single command that changes schema and then fails before exiting
681
- cannot be refreshed at the midpoint.
682
-
683
- Source schema refreshes are intentionally single-connection and pooler-safe.
684
- If a Neon pooled source URL is configured, Testkit rewrites it to the matching
685
- direct Neon endpoint before running Dockerized `pg_dump` and records the
686
- original/resolved host classifications beside the resolved cache entry. Unknown
687
- PgBouncer/pooler URLs fail closed; configure a direct source URL for those
688
- providers. Concurrent refreshes for the same service and cache key are
689
- serialized with a cache-local lock so multiple Testkit processes do not
690
- stampede the source database. Testkit also maintains
691
- `.testkit/db/<service>/source-schemas/index.json` and prunes old inactive cache
692
- entries automatically.
781
+ dump the schema only, in Docker, so seed and reference data never enter the
782
+ baseline and no host `pg_dump` is needed. Keep schema-changing setup in its own
783
+ step where possible; a single command that changes schema and then fails before
784
+ exiting cannot be refreshed at the midpoint.
785
+
786
+ A source schema refresh uses one connection. A Neon pooled source URL is
787
+ rewritten to the matching direct Neon endpoint. Other PgBouncer or pooler URLs
788
+ fail; configure a direct source URL for those providers. Refreshes of the same
789
+ cache entry run one at a time, even from separate Testkit processes, and
790
+ Testkit prunes old inactive cache entries automatically.
693
791
 
694
792
  For most repos, prefer declarative step objects directly inside
695
793
  `database.postgres({ template: ... })` and `runtime.prepare.steps`.
@@ -699,10 +797,9 @@ The supported shapes are:
699
797
  - `{ kind: "sql-file", path: "..." }`
700
798
  - `{ kind: "module", target: "file.ts#exportName" }`
701
799
 
702
- Local files and ordinary external packages remain valid module targets.
703
- Testkit-owned `@gfdelarue/testkit` targets are validated against Testkit's exact
704
- configured-step module registry while config is loaded, so a misspelled or
705
- private Testkit subpath fails before preparation or database setup starts.
800
+ Local files and ordinary external packages remain valid module targets. An
801
+ unsupported or private `@gfdelarue/testkit` target fails when the config loads,
802
+ before preparation or database setup starts.
706
803
 
707
804
  `runtime.toolchain` is the first-class way to make those prepare/start commands
708
805
  run under the correct Node toolchain instead of whatever `node`/`npm` happened
@@ -720,16 +817,12 @@ to launch `testkit`. Node toolchains support:
720
817
  - `package.json#packageManager`
721
818
  - `package.json#engines.npm`
722
819
 
723
- Downloaded toolchains resolve only through Testkit's target-specific trusted
724
- Node release manifest. The archive URL, compressed size, SHA-256, archive
725
- shape, unpacked size, and complete installed-tree digest are fixed before any
726
- download. An unlisted version or platform fails closed. The verified cache is
727
- kept under `.testkit/_toolchains/node`; external cache redirects, linked cache
728
- ancestors, modified installed content, and escaping executable links are
729
- rejected. `runtime.toolchain.cwd` is always a normalized product-relative path
730
- and cannot contain a parent-directory segment. The trusted official Linux Node
731
- archives are glibc builds; `install: "download"` fails before fetch or cache
732
- mutation on musl instead of pretending those binaries are compatible.
820
+ `install: "download"` supports the Node versions and platforms Testkit lists,
821
+ and fails before downloading for any other. Downloads are cached under
822
+ `.testkit/_toolchains/node`, and a modified cache entry fails instead of
823
+ running. `runtime.toolchain.cwd` is a product-relative path and cannot contain
824
+ a parent-directory segment. The official Linux Node archives are glibc builds,
825
+ so `install: "download"` fails on musl before it downloads anything.
733
826
 
734
827
  Example:
735
828
 
@@ -758,6 +851,77 @@ artifact/log/setup files. Payloads are SHA-256 addressed under
758
851
  removing their logical per-run paths. `.testkit/results/latest.json` remains a
759
852
  convenience projection; it is not the historical authority.
760
853
 
854
+ `run.json` is schema 20. `groups` holds each group's suites and files.
855
+ `environment` is the run's environment record. For each acquisition, it holds:
856
+
857
+ - the source that prepared it: `configured`, `static`, `supplied`, or `none`;
858
+ - the tasks it served and the services it started;
859
+ - each file's setup;
860
+ - how it was released;
861
+ - the values its tests received.
862
+
863
+ It also holds each service's start attempts, the service and setup logs, the
864
+ run's capacity and database backend, and the values tests asked for that the
865
+ environment lacked. Runs that Testkit 0.4.x stored as schema 19 stay readable
866
+ through `testkit artifacts`, `ci report`, known-failure checks, and the
867
+ workspace API.
868
+
869
+ Sealing replaces secrets with `<redacted>` in the run record and in every text
870
+ file of the bundle. A key names a secret when it holds `PASSWORD`, `TOKEN`,
871
+ `SECRET`, `PRIVATE_KEY`, `AUTH`, `COOKIE`, `SESSION`, or `API_KEY` in any letter
872
+ case. Sealing redacts:
873
+
874
+ - A `KEY=value` assignment under such a key, an `Authorization`,
875
+ `Proxy-Authorization`, `Cookie`, or `Set-Cookie` header value, a JSON string
876
+ value under such a key, even when the key is spelled with escapes, a
877
+ single-quoted `'password': '…'` value, and a URL's password, wherever they
878
+ appear. A quoted value ends at its closing quote and keeps its quotes, and a
879
+ value that starts with an auth scheme such as `Bearer` ends with its line.
880
+ A secret in one JSON value leaves the record's other values as they are.
881
+ - The value of each such key, and of each key that `redact` names, in the
882
+ environment Testkit runs in. When the value is a URL, its password counts
883
+ too.
884
+ - The values the run's environment resolved: each value under such a key or a
885
+ key that `redact` names, and the password of each URL. A PostgreSQL URL's
886
+ password may also be a `password` query parameter, as libpq reads it. This
887
+ covers config `env`, env files, `local.env`, dependency services, base,
888
+ readiness, UI, and database URLs, `--env`, `--env-from`, and an
889
+ `--environment-file`. A value under such a key that is a URL counts only for
890
+ its password.
891
+ - The unguessable segment in each [HTTP capture's](#handles) operation URLs,
892
+ from a setup module's capture or an `--environment-file` handle.
893
+
894
+ Name a secret's key in `redact` when it holds none of the words above:
895
+
896
+ ```ts
897
+ export default defineConfig({
898
+ redact: ["STRIPE_SK", "PGPASS"],
899
+ });
900
+ ```
901
+
902
+ Testkit matches the names in any letter case, in a configured run and a
903
+ supplied run alike. `redact` adds keys whose values sealing redacts; the forms
904
+ in the first item still use the words above.
905
+
906
+ A value shorter than eight characters is not redacted on its own. Sealing also
907
+ redacts a value's JSON-escaped forms, each line of a value that spans lines,
908
+ and keys in the free-form data and metadata that tests attach. When output cuts a secret short and marks
909
+ the cut with `…` or `...`, sealing redacts the part that shows. `testkit run
910
+ --json` and the run's progress output apply the same redaction.
911
+
912
+ Some secrets stay visible:
913
+
914
+ - What a setup module prints goes to the terminal as written.
915
+ - Sealing does not look inside compressed or binary files, such as a Playwright
916
+ `trace.zip`. It refuses to seal a binary file that holds one of the values
917
+ above in plain bytes.
918
+ - When a run's process is killed before it seals the run, `testkit stop`,
919
+ `testkit cleanup`, or the next run seals it from another process. That
920
+ process does not have the values the run's environment resolved or the
921
+ `redact` list, so it redacts only the first item and its own environment's
922
+ values under the words above. The bundle's `archive.redaction.status` is then
923
+ `source-targets-unavailable` instead of `applied`.
924
+
761
925
  Use the artifact archive config to opt into pruning or atomic publication to
762
926
  an external directory. Local retention is indefinite by default:
763
927
 
@@ -801,254 +965,376 @@ Programmatic consumers can import `listRunBundles`, `loadRunBundleArtifact`,
801
965
 
802
966
  ## Authoring
803
967
 
804
- ### Node setup beside a test
968
+ ### Named endpoints and databases
805
969
 
806
- A test such as `tests/webhook.int.testkit.ts` can have a sibling
807
- `tests/webhook.int.testkit.setup.mjs`. Testkit runs its named `setup` export,
808
- or its default export, before starting the test engine:
970
+ `@gfdelarue/testkit/environment` gives a test the URL and database of each
971
+ service its group uses, by the service's name:
972
+
973
+ ```ts
974
+ import { database, endpoint } from "@gfdelarue/testkit/environment";
975
+
976
+ const response = await fetch(new URL("/health", endpoint("api")));
977
+ const reportsDatabase = database("reports");
978
+ ```
979
+
980
+ Both functions return strings. A service's database is the one Testkit
981
+ provisions for it, or the `DATABASE_URL` its `env` declares. The module works
982
+ in Vitest and Playwright files and in the modules they import.
983
+ Testkit installs the values before any of that code loads, so module-level code
984
+ and native hooks can read them, such as a Playwright config and its
985
+ `globalSetup` or a Vitest `beforeAll`.
986
+
987
+ A name with no value throws `EnvironmentValueMissingError`, with `kind`,
988
+ `valueName`, and the `available` names. `valueName` is `null` when the missing
989
+ value is the run's default URL. Outside a Testkit run every call throws it.
990
+
991
+ A test that fails on this error, or a file whose imports throw it, ends as an
992
+ infrastructure failure in Vitest and Playwright. The run fails, and
993
+ `testkit bug file` refuses to file it. A test that catches the error and
994
+ carries on is unaffected. When the run gives a test no base URL, a Playwright
995
+ `page.goto()` to a relative URL ends the same way.
996
+
997
+ `BASE_URL` and `DATABASE_URL` hold the values of the first service
998
+ in `uses` that has each one. Playwright gives `BASE_URL` and `baseURL` in
999
+ canonical form, so an origin ends with `/`. `endpoint()` returns the URL
1000
+ unchanged in every engine.
1001
+
1002
+ The group's default endpoint and database are also available by the group's
1003
+ name, whether the run starts its services or takes supplied values. For a group
1004
+ `checks` that uses `api`, `endpoint("checks")` and `endpoint("api")` return the
1005
+ same URL. A group whose default differs from a used service's value with the
1006
+ group's name, such as a group `api` with `uses: ["web", "api"]`, gets no
1007
+ values: its files end as an infrastructure failure that names the group, and
1008
+ other groups still run.
1009
+
1010
+ ### Handles
1011
+
1012
+ `handle(name)` from `@gfdelarue/testkit/environment` gives a test an object for
1013
+ something its environment prepared. Today that is an HTTP capture, which a
1014
+ [setup module](#node-setup-beside-a-test) starts with `sidecars.httpCapture()`.
1015
+ The handle is named after the capture's `id`:
809
1016
 
810
1017
  ```js
811
- // tests/webhook.int.testkit.setup.mjs
1018
+ // tests/webhook.test.setup.mjs
812
1019
  export async function setup({ sidecars }) {
813
- const capture = await sidecars.httpCapture({ path: "/webhook" });
814
- return {
815
- data: { webhookUrl: capture.url, requestsUrl: capture.requestsUrl },
816
- };
1020
+ await sidecars.httpCapture({ id: "webhook", path: "/webhook" });
817
1021
  }
818
1022
  ```
819
1023
 
820
- Setup runs in a Node worker thread with its own JavaScript state. It can use
821
- Node APIs, prepare external resources, and start HTTP capture listeners.
822
- The context contains `deadlineMs`, `signal`, `env`, `leaseDir`, `productDir`,
823
- `serviceName`, `testFile`, and `sidecars`. The worker shares the host process's
824
- permissions. It is not a sandbox for untrusted code.
825
-
826
- Return plain JSON, or `{ data, cleanup }` with an optional cleanup function.
827
- Testkit passes the data to the engine as `setupData` and records it with the
828
- setup module path and sidecar handles. Bare `null` or `undefined`, and a
829
- cleanup-only result, give empty data. `{ data: null }` preserves `null`.
830
- Data must encode to at most 1 MiB of JSON. Invalid data reports its JSON path,
831
- such as `$.items[1]`, without invoking property getters.
832
-
833
- Setup shares the task's absolute deadline. On failure or cancellation, Testkit
834
- runs registered cleanup and closes its sidecars. Once setup succeeds, those
835
- resources remain available until task cleanup. Cleanup callbacks run in reverse
836
- registration order. The runner uses the task timeout for cleanup, capped at
837
- 60 seconds. A hung worker is terminated so it cannot keep its listeners open.
838
-
839
- Thrown setup errors, deadline expiry, and invalid data fail the task with the
840
- setup module path. Consumer cleanup errors fail the task; Testkit cleanup or
841
- worker failures count as infrastructure failures. Cleanup remains idempotent.
842
- HTTP capture retains at most 100 requests by default, configurable up to 1,000.
843
- Each body defaults to 1 MiB, configurable up to 8 MiB, and retained bodies total
844
- at most 8 MiB per listener. A setup can create up to 16 listeners, each with at
845
- most 256 connections.
1024
+ ```ts
1025
+ // tests/webhook.test.ts
1026
+ import { handle } from "@gfdelarue/testkit/environment";
846
1027
 
847
- Discovery tries sibling `.setup.mjs`, sibling `.setup.js`, then the matching
848
- `.mjs` and `.js` files in `__setup__`, in that order. For the example above,
849
- the fallback is `tests/__setup__/webhook.int.testkit.mjs`.
1028
+ const webhook = handle("webhook");
1029
+ await placeOrder({ callbackUrl: webhook.url });
1030
+ const [delivery] = await webhook.requests();
1031
+ await webhook.reset();
1032
+ ```
850
1033
 
851
- ### HTTP suites
1034
+ A capture handle has `kind` (`"http-capture"`), `url` (where the product sends
1035
+ its requests), `requests()`, which returns the captured requests oldest first,
1036
+ and `reset()`, which forgets them. Both methods return a Promise in
1037
+ Vitest and Playwright. Each call fails after 10 seconds if the capture hasn't
1038
+ answered in full.
1039
+
1040
+ A handle is a reference, not a connection. It can list or reset its own capture
1041
+ and nothing else. The capture's operation URLs hold an unguessable segment, so
1042
+ holding the handle is the authority. Testkit keeps that segment out of the
1043
+ run's results, artifacts, and events, and the capture refuses any other
1044
+ operation. When Testkit releases the file's environment, the capture closes and
1045
+ the handle expires. Calling a handle never starts a sidecar or a service.
1046
+
1047
+ - A name with no handle throws `EnvironmentValueMissingError` with `kind`
1048
+ `"handle"`, and the test ends as an infrastructure failure, as for a missing
1049
+ endpoint.
1050
+ - Every capture created without an `id` gets the id `http-capture`. If two do,
1051
+ `handle("http-capture")` throws `EnvironmentHandleAmbiguousError` with the
1052
+ `count`. Give each capture a distinct `id`.
1053
+ - A call that the capture refuses, such as one from a stale or forged
1054
+ reference, throws `EnvironmentHandleError` with `reason` `"refused"`. A call
1055
+ whose capture no longer answers throws it with `"unavailable"`, and a
1056
+ descriptor this Testkit cannot use throws it with `"invalid"` or
1057
+ `"unsupported"`. These end the test as ordinary failures.
1058
+
1059
+ ### Give each test its own data
1060
+
1061
+ Testkit runs test files at the same time. Files on one runtime share its
1062
+ services and its database. Write each test so that it passes beside any other
1063
+ test, in any order.
1064
+
1065
+ - Create the rows the test needs, and filter every query and update to them.
1066
+ Don't depend on rows that another test creates, and don't count, update, or
1067
+ delete across a whole table. Every test sees the rows from `template.seed`,
1068
+ so read them and leave them unchanged.
1069
+ - When the test ends, leave its rows in place. Testkit resets the databases it
1070
+ provisions. By default, every run starts from a fresh clone of the template
1071
+ database. See [`database.reset`](testkit-config-schema.md#database).
1072
+ - Clean up only what Testkit doesn't own, such as an account in a third-party
1073
+ sandbox. Register the cleanup as soon as the test creates the account. Give
1074
+ the account a unique name, so that a leftover from a cleanup that never ran
1075
+ can't collide with the next run.
852
1076
 
853
1077
  ```ts
854
- import { defineFile } from "@gfdelarue/testkit";
855
- import { defineHttpSuite, expect } from "@gfdelarue/testkit/k6";
1078
+ import { randomUUID } from "node:crypto";
1079
+ import { endpoint } from "@gfdelarue/testkit/environment";
1080
+ import { expect, onTestFinished, test } from "vitest";
856
1081
 
857
- export const testkit = defineFile({ engine: "k6" });
1082
+ import { sandbox } from "../payments-sandbox.js";
858
1083
 
859
- const suite = defineHttpSuite(({ rawReq }) => {
860
- const response = rawReq.get("/health");
861
- expect.status(response, 200, "health returns 200");
862
- });
1084
+ test("a refund reaches the sandbox customer", async () => {
1085
+ const customer = await sandbox.createCustomer({ email: `refund-${randomUUID()}@example.test` });
1086
+ onTestFinished(() => sandbox.deleteCustomer(customer.id));
863
1087
 
864
- export const options = suite.options;
865
- export const setup = suite.setup;
866
- export default suite.exec;
1088
+ const response = await fetch(new URL("/api/refunds", endpoint("api")), {
1089
+ method: "POST",
1090
+ headers: { "content-type": "application/json" },
1091
+ body: JSON.stringify({ customerId: customer.id, amountCents: 500 }),
1092
+ });
1093
+ expect(response.status).toBe(201);
1094
+ });
867
1095
  ```
868
1096
 
869
- Testkit helper suites are authored as native-shaped k6 modules: re-export the
870
- helper's `options` and `setup`, and default-export its `exec` function. Testkit
871
- does not inspect source text or invent k6 lifecycle exports.
1097
+ A [Node setup module](#node-setup-beside-a-test) returns its cleanup with its
1098
+ data. Testkit runs only the cleanup that `setup` returns. If a step in `setup`
1099
+ can throw after `setup` creates something, undo the creation before rethrowing.
872
1100
 
873
- Suite-owned authentication lives in the k6 module. `auth.setup` receives the
874
- resolved env and a raw HTTP client, returns a session bundle of named actors,
875
- and `auth.headers` turns the bundle plus the requested actor into headers:
1101
+ A test that has to count or change a whole table needs a database that other
1102
+ tests don't write to. Give the test a group of its own, and have the group use
1103
+ a service with its own database. Below, the `ledger` group owns the test files
1104
+ under `src/ledger`, and its tests get the database of `ledger-db`, built from
1105
+ the same schema:
876
1106
 
877
1107
  ```ts
878
- import { defineFile } from "@gfdelarue/testkit";
879
- import { defineHttpSuite } from "@gfdelarue/testkit/k6";
880
-
881
- import { organizationHeaders, sessionAuth } from "../helpers/session.js";
882
-
883
- export const testkit = defineFile({ engine: "k6" });
1108
+ services: {
1109
+ api: app.node({
1110
+ // ...
1111
+ }),
1112
+ "ledger-db": database.fixture({
1113
+ template: { migrate: [{ kind: "sql-file", path: "db/schema.sql" }] },
1114
+ }),
1115
+ },
1116
+ groups: {
1117
+ api: { roots: ["src"], uses: ["api"] },
1118
+ ledger: { roots: ["src/ledger"], uses: ["ledger-db"] },
1119
+ },
1120
+ ```
884
1121
 
885
- const suite = defineHttpSuite(
886
- {
887
- auth: sessionAuth(["reviewer", "outsider"]),
888
- rawHeaders: organizationHeaders,
889
- },
890
- ({ actor, actors, req }) => {
891
- req.get("/api/auth/session");
892
- req("GET", "/api/auth/session");
893
- actor?.req.get("/api/auth/session");
894
- req.as("outsider").get("/api/auth/session");
895
- actors.get("reviewer").rawReq.get("/api/auth/session");
896
- },
897
- );
1122
+ `locks` is an escape hatch for a resource that can't be duplicated, such as a
1123
+ payment provider's sandbox account with one webhook URL that some tests change.
1124
+ Files that name the same lock never run at the same time within one run.
1125
+ Testkit doesn't share locks between runs, such as two CI jobs. Two tests that
1126
+ need a lock because they share rows need their own rows instead.
898
1127
 
899
- export const options = suite.options;
900
- export const setup = suite.setup;
901
- export default suite.exec;
1128
+ ```ts
1129
+ export const testkit = defineFile({ engine: "node-vitest", locks: ["payments-sandbox-webhook"] });
902
1130
  ```
903
1131
 
904
- ```ts
905
- // __testkit__/helpers/session.ts
906
- import type { AuthAdapter, AuthSessionBundle } from "@gfdelarue/testkit/k6";
1132
+ ### Node setup beside a test
907
1133
 
908
- export function sessionAuth(actorNames: string[]): AuthAdapter<AuthSessionBundle> {
1134
+ A test such as `tests/webhook.test.ts` can have a sibling
1135
+ `tests/webhook.test.setup.mjs`. Testkit runs its named `setup` export, or its
1136
+ default export, before starting the test engine:
1137
+
1138
+ ```js
1139
+ // tests/webhook.test.setup.mjs
1140
+ export async function setup({ sidecars }) {
1141
+ const capture = await sidecars.httpCapture({ path: "/webhook" });
909
1142
  return {
910
- setup({ env, http }) {
911
- const actors = Object.fromEntries(
912
- actorNames.map((actorName, actorIndex) => [
913
- actorName,
914
- { actorIndex, actorName, session: login(http, env.BASE, actorName) },
915
- ]),
916
- );
917
- return { actors, primaryActor: actorNames[0] };
918
- },
919
- headers(bundle, { actor }) {
920
- const session = bundle?.actors[actor ?? bundle.primaryActor ?? ""]?.session;
921
- return session ? { Authorization: `Bearer ${session.jwt}` } : {};
922
- },
1143
+ data: { webhookUrl: capture.url, requestsUrl: capture.requestsUrl },
923
1144
  };
924
1145
  }
925
1146
  ```
926
1147
 
927
- Named HTTP profiles in `testkit.config.ts` stay JSON-only for k6 suites, because
928
- the orchestrator serializes them across the Node/k6 boundary. Node/Vitest data
929
- profiles are loaded in-process and may define `auth` and header functions
930
- directly. See the [config reference](testkit-config-schema.md#profileshttp).
1148
+ Setup is part of preparing the file's environment. Testkit runs it once, after
1149
+ it acquires the services the file's group uses and before the test engine
1150
+ starts. Testkit never batches a Playwright file that has setup with other
1151
+ Playwright files, and a plain Vitest file that has setup runs in its own Vitest
1152
+ invocation. Files without setup still batch.
1153
+
1154
+ Setup runs with its own JavaScript state. It can use Node APIs, prepare
1155
+ external resources, and start HTTP capture listeners. Tests
1156
+ reach a capture by its `id` through [`handle()`](#handles). A capture's
1157
+ `requestsUrl` and `resetUrl` hold an unguessable segment that Testkit keeps out
1158
+ of the run's records, and its capture `path` may not start with `/__testkit__/`.
1159
+ The context contains `deadlineMs`, `signal`, `env`, `leaseDir`, `productDir`,
1160
+ `serviceName`, `testFile`, and `sidecars`. `serviceName` holds the name of the
1161
+ test's group. Setup has the same permissions as the Testkit process. It is not
1162
+ a sandbox for untrusted code.
931
1163
 
932
- DAL suites:
1164
+ A run on [supplied values](#run-against-a-supplied-environment) does not run
1165
+ setup modules. Its tests receive the environment file's `setupData` instead.
933
1166
 
934
- ```ts
935
- import { defineFile } from "@gfdelarue/testkit";
936
- import { defineDalFixtures, defineDalSuite } from "@gfdelarue/testkit/k6";
937
-
938
- export const testkit = defineFile({ engine: "k6" });
939
-
940
- const fixtures = defineDalFixtures(({ db, fixtureScope }) => ({
941
- widget() {
942
- return fixtureScope.seed("widget", "primary", { name: "Primary Widget" }, () => {
943
- const widgetId = fixtureScope.id("widget");
944
- db.exec(`
945
- INSERT INTO widgets (id, name)
946
- VALUES ('${widgetId}', 'Primary Widget')
947
- `);
948
- return { widgetId };
949
- });
950
- },
951
- }));
952
-
953
- const suite = defineDalSuite({ fixtures }, ({ db, fixtureScope, fixtures }) => {
954
- const widget = fixtures.widget();
955
- db.query(`select id from widgets where id = '${widget.widgetId}'`);
956
- fixtureScope.records();
957
- });
1167
+ Return plain JSON, or `{ data, cleanup }` with an optional cleanup function.
1168
+ Testkit passes the data to the engine as `setupData` and records it with the
1169
+ setup module path and sidecar handles. Tests read it with `setupData()` from
1170
+ `@gfdelarue/testkit/environment`, which returns `{}` when the file has no setup
1171
+ module and throws outside a Testkit run. Bare `null` or `undefined`, and a
1172
+ cleanup-only result, give empty data. `{ data: null }` preserves `null`.
1173
+ Data must encode to at most 1 MiB of JSON. Invalid data reports its JSON path,
1174
+ such as `$.items[1]`.
958
1175
 
959
- export const options = suite.options;
960
- export const setup = suite.setup;
961
- export default suite.exec;
962
- ```
1176
+ Setup shares the task's absolute deadline. On failure or cancellation, Testkit
1177
+ runs registered cleanup and closes its sidecars. Once setup succeeds, those
1178
+ resources remain available until the test and its native hooks finish and
1179
+ Testkit releases the file's environment. Cleanup callbacks run in reverse
1180
+ registration order. The runner uses the task timeout for cleanup, capped at
1181
+ 60 seconds. Testkit stops a setup that hangs past that limit, which closes its
1182
+ listeners.
963
1183
 
964
- DAL files run at the same time as other tests, including other DAL files of the
965
- same service. If two files share rows that one of them changes, give both the
966
- same lock, for example `defineFile({ engine: "k6", locks: ["widgets"] })`, and
967
- Testkit runs them one at a time. Runtime services reserve a configurable
968
- Postgres connection budget for each active runtime slot. Use `database.postgres({ runtimeConnections })` for the
969
- service pool size and `database.postgres({ maxConnections,
970
- reservedConnections })` for the backing Postgres capacity.
1184
+ Thrown setup errors, deadline expiry, and invalid data fail the task with the
1185
+ setup module path. Consumer cleanup errors fail the task; a failure in
1186
+ Testkit's own cleanup, or a setup that stops unexpectedly, counts as an
1187
+ infrastructure failure. Cleanup remains idempotent.
1188
+ HTTP capture retains at most 100 requests by default, configurable up to 1,000.
1189
+ Each body defaults to 1 MiB, configurable up to 8 MiB, and retained bodies total
1190
+ at most 8 MiB per listener. A setup can create up to 16 listeners, each with at
1191
+ most 256 connections.
971
1192
 
972
- `defineDalFixtures(...)` is the package-owned DAL seeding model. It gives every
973
- suite a deterministic `fixtureScope` with:
1193
+ Discovery tries sibling `.setup.mjs`, sibling `.setup.js`, then the matching
1194
+ `.mjs` and `.js` files in `__setup__`, in that order. For the example above,
1195
+ the fallback is `tests/__setup__/webhook.test.mjs`.
974
1196
 
975
- - `fixtureScope.id(label)` / `uuid(label)`
976
- - `fixtureScope.slug(label)`
977
- - `fixtureScope.email(label)`
978
- - `fixtureScope.string(label, options)`
979
- - `fixtureScope.token(label)`
980
- - `fixtureScope.seed(kind, key, signature, create)`
981
- - `fixtureScope.records()`
1197
+ ### HTTP and persistence checks
982
1198
 
983
- `testkit` enforces strict fixture behavior:
1199
+ Use plain Vitest files for ordinary HTTP, service, and database checks. Read
1200
+ configured URLs with `endpoint(name)` and `database(name)` from
1201
+ `@gfdelarue/testkit/environment`, then use `fetch` or your application's
1202
+ database client.
984
1203
 
985
- - one logical fixture per `kind + key`
986
- - identical reseeds reuse the same seeded value
987
- - conflicting reseeds fail immediately
988
- - dependency cycles fail immediately
989
- - seeded fixture records are persisted as a `testkit.dal-fixtures` artifact
1204
+ ### Record data-flow evidence
990
1205
 
991
- The first-class k6 authoring namespace:
1206
+ `dataFlow(context)` from `@gfdelarue/testkit/node-vitest/data` records what a
1207
+ plain Vitest test sees data do: metrics, counts, bytes, durations, money,
1208
+ ratios, sequences, records, actors, systems, transitions, events, findings,
1209
+ external calls, and cache, LLM, and RPC usage. Budgets check the recorded
1210
+ values, for example that a metric stays under a limit or that a sequence was
1211
+ recorded.
992
1212
 
993
1213
  ```ts
994
- import { checks, expect, parse, waitFor } from "@gfdelarue/testkit/k6";
995
-
996
- const suite = defineHttpSuite(({ actor, rawReq, req }) => {
997
- const me = req.get("/api/v1/me");
998
- expect.status(me, 200, "authenticated session loads");
999
- expect.json(me, (body) => typeof body.data?.id === "string", "response includes user id");
1000
-
1001
- checks.authGate(rawReq, "sessions", {
1002
- get: ["/api/v1/sessions"],
1003
- });
1004
-
1005
- actor?.rawHeaders();
1006
-
1007
- const upload = req.multipart.post("/api/v1/uploads", {
1008
- fields: { name: "fixture" },
1009
- files: [
1010
- {
1011
- field: "file",
1012
- data: "hello",
1013
- filename: "fixture.txt",
1014
- contentType: "text/plain",
1015
- },
1016
- ],
1214
+ import { endpoint } from "@gfdelarue/testkit/environment";
1215
+ import { dataFlow } from "@gfdelarue/testkit/node-vitest/data";
1216
+ import { test } from "vitest";
1217
+
1218
+ test("lists one page of orders", async (context) => {
1219
+ const flow = dataFlow(context);
1220
+ const response = await fetch(`${endpoint("api")}/orders?page=1`);
1221
+ const body = await response.text();
1222
+ flow.recordHttpRequest({
1223
+ method: "GET",
1224
+ path: "/orders",
1225
+ status: response.status,
1226
+ responseBytes: Buffer.byteLength(body),
1017
1227
  });
1018
-
1019
- expect.status(upload, 201, "upload is accepted");
1020
- parse.safeJson(upload);
1228
+ flow.expectMetric("http.response.bytes").atMost(64_000);
1021
1229
  });
1022
1230
  ```
1023
1231
 
1024
- Use native k6 imports for checks, groups, metrics, and low-level HTTP:
1025
-
1026
- ```ts
1027
- import { check, group } from "k6";
1028
- import http from "k6/http";
1029
- import { Rate, Trend } from "k6/metrics";
1030
- ```
1031
-
1032
- The [k6 authoring guide](https://github.com/gfdelarue/testkit/blob/master/docs/k6-authoring-layer.md)
1033
- describes the optional evidence recorder and migration from retired Testkit
1034
- primitive, metric, option, and assertion aliases.
1035
-
1036
- `waitFor()` consumes the file budget configured by `execution.fileTimeoutSeconds`.
1037
- Consumers should not set local timeout values in test files.
1038
-
1039
- ```ts
1040
- import { parse, waitFor } from "@gfdelarue/testkit/k6";
1041
-
1042
- const response = waitFor(
1043
- () => req.get("/api/v1/jobs/123"),
1044
- (res) => parse.json(res).data?.status === "completed",
1045
- { description: "job 123 to complete" },
1046
- );
1047
- ```
1232
+ - `dataFlow` needs the context of a running test. Every call in one attempt
1233
+ returns the same recorder, and a call without that context throws.
1234
+ - The library reads no URL, environment value, fixture file, or Testkit config.
1235
+ Read named URLs from `@gfdelarue/testkit/environment`, log in from test code,
1236
+ and read files with `fs`.
1237
+ - A failed budget fails its test when the test finishes. The failure detail has
1238
+ the kind `data-flow-budget` and a key made of the test's full title and the
1239
+ budget.
1240
+ - Testkit writes `data-flow.json` and `data-flow.md` for a file whose tests
1241
+ recorded something. Each such test is one scenario, from its final attempt. A
1242
+ test that records nothing adds nothing, and a file whose tests record
1243
+ nothing gets no data-flow artifacts.
1048
1244
 
1049
1245
  ## Discovery
1050
1246
 
1051
- `testkit` discovers suites from `__testkit__/` directories.
1247
+ Testkit discovers plain Vitest test files and typed suites.
1248
+
1249
+ ### Plain Vitest test files
1250
+
1251
+ Testkit runs plain Vitest test files with the repository's own Vitest and its
1252
+ config, as `vitest run` does from the product directory. It loads the root
1253
+ `vitest.config.*` or `vite.config.*` the way Vitest does and lists the files
1254
+ that config's projects select, without running global setup. Nested configs and
1255
+ package scripts have no effect. Without a config, Vitest's defaults select files
1256
+ that match `**/*.{test,spec}.?(c|m)[jt]s?(x)`, outside `node_modules` and
1257
+ `.git`.
1258
+
1259
+ `discover`, `run`, `typecheck`, `bug record`, and `bug file` take three flags
1260
+ that change the selection:
1261
+
1262
+ - `--vitest-config <path>` uses that config, relative to the product
1263
+ directory. A missing file, or one outside the product directory, fails before
1264
+ any test runs.
1265
+ - `--vitest-project <name>` runs only that project. An unknown name fails and
1266
+ lists the projects.
1267
+ - `--vitest-runner managed` runs plain files with Testkit's own Vitest and its
1268
+ default rules, without the repository's config.
1269
+
1270
+ Testkit supports Vitest 4.x. Another major, a broken installation, a root
1271
+ Vitest config that fails to load, or a Vitest that `package.json` declares but
1272
+ that is not installed fails before any test runs, naming the version and path.
1273
+ `--vitest-runner managed` avoids the repository's Vitest. A run that selects
1274
+ only typed files, with `--type ui` or with typed `--file` paths, `bug record` or
1275
+ `bug file` for a typed test, and the commands that select no tests (`status`,
1276
+ `run status`, `doctor`, `destroy`, `cleanup`, and `db schema`) don't load the
1277
+ repository's Vitest, so they still work. A product that neither installs nor
1278
+ declares Vitest has no plain test files.
1279
+
1280
+ Plain files run in one Vitest invocation per environment their groups give them.
1281
+ Groups that give their files the same values share one, such as `root` and the
1282
+ other groups that use no services. A group that uses a service Testkit starts
1283
+ gets an invocation of its own.
1284
+ Inside each invocation, root global setup, teardown, and the repository's
1285
+ reporters run as they do under `vitest run`, and Testkit's reporter runs beside
1286
+ them, so a run calls them once for each invocation. An invocation's deadline is
1287
+ the sum of its files' timeouts; Vitest's own test and hook timeouts apply inside
1288
+ it. A file that two projects select runs in both, its result combines both
1289
+ runs, and its evidence names each run's project. A file with a setup module,
1290
+ locks, or isolation other than `shared` runs in its own Vitest invocation with
1291
+ its own environment. Invocations run one after another unless
1292
+ `--task-concurrency` or `execution.taskConcurrency` is above 1. To run one group's
1293
+ files alone, select it with `--group`.
1294
+
1295
+ Plain files see the host environment Testkit started with: `HOME`, `PATH`,
1296
+ temporary and XDG directories, locale, `CI`, `FORCE_COLOR`, `NO_COLOR`, and any
1297
+ other variable reach config loading, setup files, and the tests as they do
1298
+ under `vitest run`. Unlike `vitest run`, ambient `BASE_URL` and `DATABASE_URL`
1299
+ never reach them; those come only from the group's environment, from
1300
+ provisioned services or supplied values. Testkit loads no env files for plain
1301
+ files, and Vitest's own env handling and `test.env` work as usual. Testkit puts
1302
+ its own `--import` in front of the shell's `NODE_OPTIONS`. Preloads in it see the
1303
+ shell's environment and the group's values; as under `vitest run`, Vitest sets
1304
+ `TEST`, `VITEST`, and `NODE_ENV` after them. `--vitest-runner managed` leaves the
1305
+ shell's `NODE_OPTIONS` out.
1306
+
1307
+ When Vitest detects a coding agent from the environment, such as `CLAUDECODE`,
1308
+ `AI_AGENT`, or `CURSOR_AGENT`, and the repository's Vitest config names no
1309
+ reporter, Vitest uses its agent reporter. That reporter prints no console output
1310
+ from a passing test, so the output is missing from the file's
1311
+ `node-vitest-stdout` artifact. A failing test's output still appears. Name a
1312
+ reporter in the Vitest config, such as `reporters: ["default"]`, to keep it.
1313
+
1314
+ Two settings narrow the files Vitest selects: `discovery.exclude` in `testkit.config.ts`, and
1315
+ a declared group's `exclude`. Testkit's built-in exclusions for typed files,
1316
+ such as `dist`, `build`, and `coverage`, and `discovery.roots` do not apply to
1317
+ plain files.
1318
+
1319
+ A plain file belongs to the group whose roots contain it, or to the implicit
1320
+ `root` group, which uses no service. Testkit starts no service for a plain
1321
+ file in `root`.
1322
+
1323
+ The plain files in one directory form a suite named after that directory,
1324
+ relative to the group's root. In `root`, `--suite src` selects
1325
+ `src/sum.test.ts` and `src/sum.spec.ts`. A file directly in a group's root names
1326
+ its own suite, so `sum.test.ts` in a group rooted at `src` is in suite `sum`.
1327
+ When two suites in a group would share a name, Testkit lengthens the later
1328
+ ones with parent directories, such as `api-src`, and numbers them if that is
1329
+ not enough. `testkit typecheck` checks plain TypeScript test files and skips
1330
+ plain JavaScript ones, as `tsc` does without `allowJs`. It adds the TypeScript
1331
+ files in each Vitest project's `setupFiles` to the program, so a test can use
1332
+ the matchers a setup file declares, such as those from
1333
+ `@testing-library/jest-dom/vitest`.
1334
+
1335
+ ### Typed suites
1336
+
1337
+ `testkit` discovers typed suites from `__testkit__/` directories.
1052
1338
 
1053
1339
  Place each `__testkit__/` directory as a peer of the thing being tested. The
1054
1340
  target is the smallest owned directory or package surface that honestly contains
@@ -1077,67 +1363,66 @@ production owner when a narrower owner exists.
1077
1363
 
1078
1364
  Concrete layouts:
1079
1365
 
1080
- - `src/api/routes/__testkit__/auth-me.int.testkit.ts`
1081
- - `src/db/__testkit__/sessions-count-type.dal.testkit.ts`
1082
1366
  - `frontend/src/components/layout/__testkit__/navigation.ui.testkit.ts`
1083
- - `src/internal/handler/repos/__testkit__/crud.int.testkit.ts`
1084
-
1085
- `testkit` uses these suffixes automatically:
1086
1367
 
1087
- - `*.int.testkit.ts`
1088
- - `*.e2e.testkit.ts`
1089
- - `*.contract.testkit.ts`
1090
- - `*.data.testkit.ts`
1091
- - `*.dal.testkit.ts`
1092
- - `*.load.testkit.ts`
1093
- - `*.ui.testkit.ts`
1368
+ `testkit` uses the `*.ui.testkit.ts` suffix automatically.
1094
1369
 
1095
- The suffix never chooses an engine. Each discovered file must also contain one
1096
- root-package `defineFile(...)` declaration with a supported literal engine.
1370
+ The suffix never chooses an engine. Each discovered typed file must also
1371
+ contain one root-package `defineFile(...)` declaration with a supported literal
1372
+ engine.
1097
1373
  Unsupported pairs fail discovery; for example, a `*.ui.testkit.ts` file cannot
1098
- declare `engine: "k6"`.
1374
+ declare `engine: "node-vitest"`.
1099
1375
 
1100
1376
  See [docs/test-types.md](docs/test-types.md) for the canonical type model.
1101
- `*.contract.testkit.ts` files are Node-backed service/runtime contract suites.
1102
- They use `contractScenario(...)` from
1103
- `@gfdelarue/testkit/node-vitest/contract`, declare `engine: "node-vitest"`, and run
1104
- through the `node-vitest-contract` profile for checks that need Node APIs but
1105
- are not data-flow evidence suites. They are native Vitest suites:
1106
- `contractScenario(...)` is the managed-context convenience, while direct public
1107
- Vitest runner APIs remain valid for behavior such as `test.skip(...)` and
1108
- `test.todo(...)`.
1109
-
1110
- `*.data.testkit.ts` files are Testkit data-flow/data-contract suites, not raw
1111
- Vitest files with a Testkit suffix. They must use `dataScenario(...)` from
1112
- `@gfdelarue/testkit/node-vitest/data`, declare `engine: "node-vitest"`, and record
1113
- structured data-flow evidence through the data context. Supporting
1114
- `count(...)`/`event(...)` calls are not enough by themselves; model actors,
1115
- systems, records, transitions, external calls, payloads, budgets, cache/LLM/RPC
1116
- usage, or reconciliation evidence. Data suites run through the
1117
- `node-vitest-data` profile. Use `contract` for plain Node assertions and
1118
- Node-side service or protocol contracts, `int` or `e2e` for k6-backed HTTP
1119
- integration, and `ui` when a browser is part of the assertion.
1120
- Data suites use Vitest directly for assertions, hooks, and mocking.
1121
- `dataScenario(...)` owns case registration: `testkit lint` reports native
1122
- registration imports in data suites, and the data runner rejects any case that
1123
- was not registered through the DSL.
1124
-
1125
- Ownership is inferred from:
1126
-
1127
- - the deepest matching service root from `services.<name>.local.cwd`
1128
- - optional `services.<name>.discovery.roots` overrides for shared-root edge cases
1377
+
1378
+ Test groups own test files by directory. Declare them under `groups` in
1379
+ `testkit.config.ts`. Each group lists its repo-relative `roots` and the services
1380
+ its tests `use`:
1381
+
1382
+ ```ts
1383
+ export default defineConfig({
1384
+ services: {
1385
+ api: app.node({ cwd: "services/api", start: "node dist/server.js" }),
1386
+ web: app.next({ cwd: "apps/web", dependsOn: ["api"] }),
1387
+ },
1388
+ groups: {
1389
+ api: { roots: ["services/api"], uses: ["api"] },
1390
+ web: { roots: ["apps/web"], uses: ["web", "api"] },
1391
+ },
1392
+ });
1393
+ ```
1394
+
1395
+ - A file belongs to the group with the deepest root that contains it. When two
1396
+ groups have roots at the same depth, discovery reports `ambiguous_test`.
1397
+ - Files that no group's roots claim belong to the implicit group `root`, which
1398
+ uses no service. With no `testkit.config.ts`, every file belongs to `root`.
1399
+ - A run starts the services that the selected files' groups use, with their
1400
+ `dependsOn`, and nothing else. A test gets `BASE_URL`, `DATABASE_URL`, and
1401
+ `env` values only from the services its group uses. When two of them set the
1402
+ same name, the first in `uses` wins. Each used service's URL and database
1403
+ are also available by name; see
1404
+ [Named endpoints and databases](#named-endpoints-and-databases).
1405
+ - `skip`, `requirements`, and `runtimeIsolation` are group fields. See the
1406
+ [config reference](testkit-config-schema.md#groups).
1407
+
1408
+ Services own no test files. A config with `services` and no `groups` key
1409
+ fails to load when a service's `local.cwd` or `discovery.roots` holds a typed
1410
+ test file, which that service owned before 0.6.0. The error lists the `groups`
1411
+ entries that keep those files with their services. `status`, `destroy`,
1412
+ `cleanup`, and `db schema` still run on it. Otherwise every file belongs to
1413
+ `root`.
1129
1414
 
1130
1415
  Suite names are inferred from the colocated path:
1131
1416
 
1132
- - `auth/__testkit__/*.int.testkit.ts` => `auth`
1133
- - `routes/__testkit__/auth-me.int.testkit.ts` => `routes`
1417
+ - `auth/__testkit__/*.ui.testkit.ts` => `auth`
1418
+ - `routes/__testkit__/checkout.ui.testkit.ts` => `routes`
1134
1419
 
1135
1420
  Discovery is also a first-class CLI/API surface:
1136
1421
 
1137
1422
  - `testkit discover`
1138
- - human-first compact output with service -> type -> suite -> file hierarchy
1423
+ - human-first compact output with group -> type -> suite -> file hierarchy
1139
1424
  - `testkit discover --output-mode verbose`
1140
- - explicit paths, IDs, locks, dependencies, and skip reasons
1425
+ - group roots and uses, explicit paths, IDs, locks, and skip reasons
1141
1426
  - `testkit discover --json`
1142
1427
  - machine-readable output with stable enums, canonical paths, and summary data
1143
1428
  - `testkit discover --output .testkit/discovery.json`
@@ -1166,7 +1451,7 @@ identifier plus canonical metadata such as:
1166
1451
  - `path`
1167
1452
  - `service`
1168
1453
  - `suiteName`
1169
- - `testType`
1454
+ - `testType`, absent for a plain Vitest test file
1170
1455
  - `engineId`
1171
1456
  - `profileId`
1172
1457
  - `contract`
@@ -1177,8 +1462,10 @@ identifier plus canonical metadata such as:
1177
1462
  - `displayName`
1178
1463
 
1179
1464
  Discovery output has no cross-run counts, last outcomes, durations, or trends.
1180
- See the [status and history migration](https://github.com/gfdelarue/testkit/blob/master/docs/release-notes.md#local-status-and-history-removal)
1181
- for the removed discovery fields and consumer migration.
1465
+ Testkit 0.2.1 removed `testkit.status.json`, `.testkit/history/`, and
1466
+ discovery's `history`, `files[].history`, and `summary.latestResults` fields.
1467
+ To check one run, read its bundle under `.testkit/results/runs/<run-id>/`.
1468
+ Testkit ignores an old `.testkit/history/` directory and leaves it in place.
1182
1469
 
1183
1470
  ## Local Databases
1184
1471
 
@@ -1191,13 +1478,10 @@ client image instead of using host-installed binaries, which keeps local runs
1191
1478
  consistent across Linux, macOS, and native Windows. Override the client image
1192
1479
  with `TESTKIT_POSTGRES_CLIENT_IMAGE` when needed.
1193
1480
 
1194
- Client containers for a Testkit-owned database join that database container's
1195
- network namespace (`--network container:<id>`) and connect over loopback. The
1196
- route does not depend on the Docker daemon sharing the Testkit process's host
1197
- network, so template steps, schema snapshots, and `database.steps` helpers work
1198
- the same under Docker Desktop, rootless Docker, and Docker-in-Docker. Clients
1199
- for external databases, such as `sourceSchema` URLs, still connect from the
1200
- platform's host address.
1481
+ Template steps, schema snapshots, and `database.steps` helpers work the same
1482
+ under Docker Desktop, rootless Docker, and Docker-in-Docker. Clients for
1483
+ external databases, such as `sourceSchema` URLs, connect from the platform's
1484
+ host address, so those databases must be reachable from there.
1201
1485
 
1202
1486
  - normal test runs always use transient host-managed Postgres
1203
1487
  - template databases are cached
@@ -1213,7 +1497,8 @@ The service process receives its provisioned database as `DATABASE_URL`. Every
1213
1497
  task gets the same `DATABASE_URL` when its service declares a database, and its
1214
1498
  service's URL as `BASE_URL` when the service declares a server, whatever its
1215
1499
  engine. The provisioned `DATABASE_URL` replaces one that the service's `env`
1216
- declares. Declared `env` values can also use the `{dbUrl}`, `{dbHost}`,
1500
+ declares. A test whose group uses several services with databases reads each
1501
+ one with `database(name)` from `@gfdelarue/testkit/environment`. Declared `env` values can also use the `{dbUrl}`, `{dbHost}`,
1217
1502
  `{dbPort}`, `{dbName}`, `{dbUser}`, `{dbPassword}`, and `{dbSsl}` tokens, with
1218
1503
  `:serviceName` to read the database of a service listed in `dependsOn`. Testkit
1219
1504
  resolves them from the database provisioned for the current run. They are
@@ -1224,59 +1509,3 @@ schema cache. `db schema verify` prepares local templates and verifies local
1224
1509
  replay against the cached/refreshed source schema. `--skip-schema-source-verify`
1225
1510
  is available as a narrow escape hatch when users need to run tests while schema
1226
1511
  verification is temporarily blocked.
1227
-
1228
- ## Development Tests
1229
-
1230
- ```bash
1231
- npm test
1232
- npm run test:unit
1233
- npm run test:integration
1234
- npm run test:system
1235
- npm run test:live
1236
- npm run test:integration -- fixture-package
1237
- ```
1238
-
1239
- The four lanes use Vitest directly. Unit tests (source and pure tooling) run in
1240
- parallel; integration and system files run serially; live tests are explicitly
1241
- opt-in and serial. Pass ordinary Vitest filters after npm's `--` separator.
1242
- `npm test` runs the first three lanes and never collects live cases.
1243
-
1244
- Live tests cover credentials, external services, network-acquired runtimes,
1245
- alternate toolchains, and Docker/platform matrices. An explicitly selected live
1246
- test fails clearly when its prerequisites are unavailable; it does not silently
1247
- skip.
1248
-
1249
- Repository commands use npm's native timing support, enabled project-wide by
1250
- the tracked `.npmrc`. Build, lint, and release modes support `help`; release
1251
- provides `bump`, `check`, `consumer-lock`, and `plan` modes.
1252
- A push to `master` publishes only when it raises `version` in `package.json`
1253
- and the new version is above npm's latest. To release, run
1254
- `npm run release -- bump <version>` and merge the pull request once its checks
1255
- pass. The
1256
- [publish workflow](https://github.com/gfdelarue/testkit/blob/master/docs/release-gates.md)
1257
- publishes that exact version through npm trusted publishing and tags the
1258
- merged commit. Any other merge publishes nothing.
1259
-
1260
- The static audit retains package/lock/runtime policy, engine dependency/import
1261
- intent, retired engine/product references, minimal repository traversal, the
1262
- dependency direction between top-level `src/` modules, and the test execution
1263
- boundary. An import outside the direction table in
1264
- `scripts/lint/audits/import-direction.ts` fails lint. The table may allow
1265
- directions that no import uses. Test execution code may import only test
1266
- execution and shared kernel code, apart from a committed list of existing
1267
- imports that only shrinks. The
1268
- [dependency direction notes](https://github.com/gfdelarue/testkit/blob/master/docs/dependency-direction.md)
1269
- describe both rules. Other implementation-shape audits stay out of lint.
1270
-
1271
- JavaScript modules under `test/fixtures/**` describe fixture behavior and keep
1272
- their fixture-owned paths. Outside fixtures, only `bin/testkit.mjs` and the
1273
- documented raw Node process probe remain `.mjs`; the probe is a program, not a
1274
- Vitest suite.
1275
-
1276
- GitHub Actions run a narrow installed Node/Vitest consumer smoke on dedicated
1277
- Orthrus Ubuntu x64 and Windows x64 self-hosted runners for pull requests from
1278
- this repository and manual dispatches. Fork pull requests skip this workflow's
1279
- jobs. Local `npm test`,
1280
- `npm run test:unit`, `npm run test:integration`, and `npm run test:system` remain
1281
- the full developer entrypoints. PostgreSQL and Docker compatibility are explicit
1282
- opt-in dispatcher cases. The smoke workflow is not five-platform release proof.