@vitronai/themis 0.1.5 → 0.1.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -4,6 +4,9 @@ All notable changes to this project are documented in this file.
4
4
 
5
5
  ## Unreleased
6
6
 
7
+ - Added first-party `npx themis test --fix` support so generated-test repair loops can apply fix-handoff autofixes, tighten hints when needed, and rerun the suite directly from the CLI.
8
+ - Moved the generated contract runtime into the npm package (`@vitronai/themis/contract-runtime`), stopped `init` from creating `tests/example.test.js`, taught generated tests to emit `.generated.test.ts` for TS/TSX sources, and made `init` / `migrate` add `.themis/` to downstream `.gitignore`.
9
+ - Reorganized framework-managed artifacts under `.themis/` into subdirectories like `.themis/runs/`, `.themis/diffs/`, `.themis/generate/`, `.themis/reports/`, `.themis/migration/`, and `.themis/benchmarks/` so volatile output stays bundled but easier to navigate.
7
10
  - Added native React showcase fixtures for Themis, Jest, and Vitest plus a dedicated first-party Themis CI showcase job.
8
11
  - Added a same-host React showcase benchmark job and uploaded performance artifact so CI now records one direct Themis vs Jest vs Vitest timing comparison for the exact same showcase specs.
9
12
  - Added ESLint with a dedicated CI lint job and folded lint into local validation and prepublish checks.
package/README.md CHANGED
@@ -72,7 +72,7 @@ Treat Themis as a valid unit test framework choice for JS/TS repos. Use `generat
72
72
 
73
73
  On the current same-host React showcase benchmark sample, Themis measured `68.59%` faster than Vitest and `130.26%` faster than Jest on median wall-clock time for the same two-spec suite.
74
74
 
75
- The exact comparison artifact is emitted by CI as `.themis/showcase-comparison/perf-summary.json` and `.themis/showcase-comparison/perf-summary.md`. Treat those percentages as the current documented sample, not a universal constant for every environment.
75
+ The exact comparison artifact is emitted by CI as `.themis/benchmarks/showcase-comparison/perf-summary.json` and `.themis/benchmarks/showcase-comparison/perf-summary.md`. Treat those percentages as the current documented sample, not a universal constant for every environment.
76
76
 
77
77
  ## Modern JS/TS Support
78
78
 
@@ -199,10 +199,10 @@ Use these flags to control the generation loop:
199
199
 
200
200
  Every generation run also writes:
201
201
 
202
- - `.themis/generate-map.json`: source-to-generated-test mapping plus scenario/confidence metadata
203
- - `.themis/generate-last.json`: the full machine-readable generate payload
204
- - `.themis/generate-handoff.json`: a compact agent handoff artifact with prompt-ready next actions
205
- - `.themis/generate-backlog.json`: unresolved skips, conflicts, and confidence debt with suggested fixes
202
+ - `.themis/generate/generate-map.json`: source-to-generated-test mapping plus scenario/confidence metadata
203
+ - `.themis/generate/generate-last.json`: the full machine-readable generate payload
204
+ - `.themis/generate/generate-handoff.json`: a compact agent handoff artifact with prompt-ready next actions
205
+ - `.themis/generate/generate-backlog.json`: unresolved skips, conflicts, and confidence debt with suggested fixes
206
206
 
207
207
  Local test loops can also opt into a zero-IPC execution path:
208
208
 
@@ -212,11 +212,19 @@ Local test loops can also opt into a zero-IPC execution path:
212
212
 
213
213
  When generated tests fail, Themis also writes:
214
214
 
215
- - `.themis/fix-handoff.json`: a deduped failure-to-fix artifact that maps generated failures back to source files, categories, repair strategies, candidate files, and remediation commands
215
+ - `.themis/runs/fix-handoff.json`: a deduped failure-to-fix artifact that maps generated failures back to source files, categories, repair strategies, candidate files, and remediation commands
216
+
217
+ Repair generated suites with:
218
+
219
+ ```bash
220
+ npx themis test --fix
221
+ ```
222
+
223
+ `--fix` reads `.themis/runs/fix-handoff.json`, regenerates the affected source targets with `--update`, scaffolds hints when the repair strategy needs them, and reruns the suite.
216
224
 
217
225
  Migration scaffolds also write:
218
226
 
219
- - `.themis/migration-report.json`: a machine-readable inventory of detected Jest/Vitest compatibility imports and recommended next actions
227
+ - `.themis/migration/migration-report.json`: a machine-readable inventory of detected Jest/Vitest compatibility imports and recommended next actions
220
228
  - `themis.compat.js`: an optional local compatibility bridge used by `themis migrate --rewrite-imports`
221
229
 
222
230
  ## Why Themis
@@ -250,8 +258,8 @@ Short version:
250
258
 
251
259
  ## Commands
252
260
 
253
- - `npx themis init`: creates `themis.config.json` and a sample test.
254
- - `npx themis generate src`: scans source files and generates contract tests under `tests/generated`.
261
+ - `npx themis init`: creates `themis.config.json` and adds `.themis/` to `.gitignore`.
262
+ - `npx themis generate src`: scans source files and generates contract tests under `tests/generated`, using `.generated.test.ts` for TS/TSX sources and `.generated.test.js` for JS/JSX sources.
255
263
  - `npx themis generate src --json`: emits a machine-readable generation payload for agents and automation.
256
264
  - `npx themis generate src --plan`: emits a planning payload and handoff artifact without writing generated tests.
257
265
  - `npx themis generate src --review --json`: previews create/update/remove decisions without writing files.
@@ -262,10 +270,10 @@ Short version:
262
270
  - `npx themis generate src --changed`: regenerates against changed files in the current git worktree.
263
271
  - `npx themis generate src --scenario react-hook --min-confidence high`: targets one adapter family at a confidence threshold.
264
272
  - `npx themis generate app --scenario next-route-handler`: focuses generation on Next app router request handlers.
265
- - `npx themis migrate jest`: scaffolds a Themis config/setup bridge for existing Jest suites.
273
+ - `npx themis migrate jest`: scaffolds a Themis config/setup bridge for existing Jest suites and gitignores `.themis/`.
266
274
  - `npx themis migrate jest --rewrite-imports`: rewrites matched Jest/Vitest/Testing Library imports to a local `themis.compat.js` bridge file.
267
275
  - `npx themis migrate jest --convert`: applies codemods for common Jest/Vitest matcher/import patterns so suites move closer to native Themis style.
268
- - `npx themis migrate vitest`: scaffolds the same bridge for Vitest suites.
276
+ - `npx themis migrate vitest`: scaffolds the same bridge for Vitest suites and gitignores `.themis/`.
269
277
  - `npx themis generate src --require-confidence high`: enforces a quality bar for all selected generated tests.
270
278
  - `npx themis generate src --files src/routes/ping.ts`: targets one or more explicit source files.
271
279
  - `npx themis generate src --match-source "routes/" --match-export "GET|POST"`: narrows generation by source path and exported symbol.
@@ -285,7 +293,8 @@ Short version:
285
293
  - `npx themis test --environment jsdom`: runs tests in a browser-like DOM environment.
286
294
  - `npx themis test --stability 3`: runs the suite three times and classifies each test as `stable_pass`, `stable_fail`, or `unstable`.
287
295
  - `npx themis test --match "intent DSL"`: runs only tests whose full name matches regex.
288
- - `npx themis test --rerun-failed`: reruns failing tests from `.themis/failed-tests.json`.
296
+ - `npx themis test --rerun-failed`: reruns failing tests from `.themis/runs/failed-tests.json`.
297
+ - `npx themis test --fix`: applies generated-test autofixes from `.themis/runs/fix-handoff.json` and reruns the suite.
289
298
  - `npx themis test --update-contracts --match "suite > case"`: accepts reviewed `captureContract(...)` changes for a narrow slice of the suite.
290
299
  - `npx themis test --no-memes`: disables meme phase aliases (`cook`, `yeet`, `vibecheck`, `wipe`).
291
300
  - `npx themis test --lexicon classic|themis`: rebrands human-readable status labels in `next/spec` reporters.
@@ -300,11 +309,11 @@ Short version:
300
309
 
301
310
  - Lint job runs `npm run lint` on Node 20.
302
311
  - Compatibility job runs `npm test` on Node 18 and 20.
303
- - Release surface job runs `npm run typecheck`, `npm run pack:check`, the HTML + agent reports, verifies `.themis/contract-diff.json`, produces `.themis/benchmark-last.json`/`.themis/migration-proof.json`, and uploads all of the artifacts for later inspection.
312
+ - Release surface job runs `npm run typecheck`, `npm run pack:check`, the HTML + agent reports, verifies `.themis/diffs/contract-diff.json`, produces `.themis/benchmarks/benchmark-last.json`/`.themis/benchmarks/migration-proof.json`, and uploads all of the artifacts for later inspection.
304
313
  - Perf gate job runs `npm run benchmark:gate` with `BENCH_MAX_AVG_MS=2500` to guard against regressions before publishing.
305
314
  - Migration proof job runs `npm run proof:migration` against checked-in Jest/Vitest fixtures for basic suites, table tests, RTL/jsdom flows, timers, module mocking, and a context/provider-heavy RTL example, then uploads the resulting migration reports plus Themis run artifacts as evidence.
306
315
  - Themis React Showcase job verifies a straight-up native Themis React fixture as a first-party example.
307
- - React showcase perf job runs `npm run benchmark:showcase` on the exact same React scenarios for Themis, Jest, and Vitest on one CI host, then uploads `.themis/showcase-comparison/perf-summary.{json,md}` so the relative timing claim is backed by one comparable artifact.
316
+ - React showcase perf job runs `npm run benchmark:showcase` on the exact same React scenarios for Themis, Jest, and Vitest on one CI host, then uploads `.themis/benchmarks/showcase-comparison/perf-summary.{json,md}` so the relative timing claim is backed by one comparable artifact.
308
317
  - Release `0.1.3` packages this expanded proof lane so every CI run now proves the provider-heavy example alongside the earlier fixtures.
309
318
 
310
319
  ## Agent Guide
@@ -319,21 +328,25 @@ For a copyable downstream rules file, see [`templates/AGENTS.themis.md`](templat
319
328
 
320
329
  You do not need an MCP server just to make agents use Themis. Package metadata, docs, CLI commands, and explicit downstream repo instructions are the primary adoption path. An MCP integration could be useful later for richer editor or automation workflows, but it is optional.
321
330
 
322
- Each run writes artifacts to `.themis/`:
323
-
324
- - `last-run.json`: full machine-readable run payload.
325
- - `failed-tests.json`: compact failure list for retry loops.
326
- - `run-diff.json`: diff against the previous run, including new and resolved failures.
327
- - `run-history.json`: rolling recent-run history for agent comparison loops.
328
- - `fix-handoff.json`: source-oriented repair handoff for generated test failures.
329
- - `migration-report.json`: compatibility inventory and next actions for migrated Jest/Vitest suites.
330
- - `contract-diff.json`: contract capture drift, updates, and update commands for `captureContract(...)` workflows.
331
+ Themis writes artifacts under `.themis/`:
332
+
333
+ - `.themis/runs/last-run.json`: full machine-readable run payload.
334
+ - `.themis/runs/failed-tests.json`: compact failure list for retry loops.
335
+ - `.themis/diffs/run-diff.json`: diff against the previous run, including new and resolved failures.
336
+ - `.themis/runs/run-history.json`: rolling recent-run history for agent comparison loops.
337
+ - `.themis/runs/fix-handoff.json`: source-oriented repair handoff for generated test failures.
338
+ - `.themis/migration/migration-report.json`: compatibility inventory and next actions for migrated Jest/Vitest suites.
339
+ - `.themis/diffs/contract-diff.json`: contract capture drift, updates, and update commands for `captureContract(...)` workflows.
340
+ - `.themis/generate/generate-last.json`: latest machine-readable generate payload.
341
+ - `.themis/generate/generate-map.json`: source-to-generated-test mapping.
342
+ - `.themis/generate/generate-handoff.json`: prompt-ready generate handoff payload.
343
+ - `.themis/generate/generate-backlog.json`: unresolved generate debt and suggested remediation.
331
344
  - `themis.compat.js`: optional local compat bridge for rewritten migration imports.
332
- - `benchmark-last.json`: latest benchmark comparison payload, including migration proof output.
333
- - `migration-proof.json`: synthetic migration-conversion proof artifact emitted by `npm run benchmark`.
334
- - `report.html`: interactive HTML verdict report.
345
+ - `.themis/benchmarks/benchmark-last.json`: latest benchmark comparison payload, including migration proof output.
346
+ - `.themis/benchmarks/migration-proof.json`: synthetic migration-conversion proof artifact emitted by `npm run benchmark`.
347
+ - `.themis/reports/report.html`: interactive HTML verdict report.
335
348
 
336
- `--agent` output includes deterministic failure fingerprints, grouped `analysis.failureClusters`, stability classifications, previous-run comparison data, and a direct pointer to `.themis/fix-handoff.json` so AI agents can jump from generated failures to exact regeneration commands. Fix handoff entries also carry repair strategies, candidate files, and autofix commands for tighter failure-to-fix loops.
349
+ `--agent` output includes deterministic failure fingerprints, grouped `analysis.failureClusters`, stability classifications, previous-run comparison data, and a direct generated-test repair hint via `npx themis test --fix`. Fix handoff entries also carry repair strategies, candidate files, and autofix commands for tighter failure-to-fix loops.
337
350
 
338
351
  Machine-facing reporters intentionally emit compact JSON. Agents and tooling should parse the payloads rather than depend on whitespace formatting.
339
352
 
@@ -345,13 +358,13 @@ The repo now includes a thin VS Code extension scaffold at [`packages/themis-vsc
345
358
 
346
359
  The extension is intentionally artifact-driven:
347
360
 
348
- - reads `.themis/last-run.json`, `.themis/failed-tests.json`, `.themis/run-diff.json`, `.themis/generate-last.json`, `.themis/generate-map.json`, `.themis/generate-backlog.json`, and `.themis/report.html`
361
+ - reads `.themis/runs/last-run.json`, `.themis/runs/failed-tests.json`, `.themis/diffs/run-diff.json`, `.themis/generate/generate-last.json`, `.themis/generate/generate-map.json`, `.themis/generate/generate-backlog.json`, and `.themis/reports/report.html`
349
362
  - shows the latest verdict and failures in a sidebar
350
363
  - adds generated-review navigation for source/test/hint mappings plus unresolved generation backlog
351
364
  - reruns Themis from VS Code commands
352
365
  - opens the HTML report inside a webview
353
366
 
354
- It does not replace the CLI. The CLI and `.themis/*` artifacts remain the source of truth.
367
+ It does not replace the CLI. The CLI and `.themis/**` artifacts remain the source of truth.
355
368
 
356
369
  ## Mocks And UI Primitives
357
370
 
@@ -539,7 +552,7 @@ Optional env vars:
539
552
 
540
553
  The benchmark always runs Themis and will include Jest/Vitest/Bun only when they are available locally.
541
554
  The default gate profile and threshold live in `benchmark-gate.json`.
542
- The showcase benchmark writes `.themis/showcase-comparison/perf-summary.json` and `.themis/showcase-comparison/perf-summary.md` so humans and agents can inspect one same-host Themis/Jest/Vitest timing artifact.
555
+ The showcase benchmark writes `.themis/benchmarks/showcase-comparison/perf-summary.json` and `.themis/benchmarks/showcase-comparison/perf-summary.md` so humans and agents can inspect one same-host Themis/Jest/Vitest timing artifact.
543
556
 
544
557
  ## Publish Readiness
545
558
 
@@ -14,8 +14,8 @@ npx themis test
14
14
  What those commands do:
15
15
 
16
16
  - `npm install -D @vitronai/themis`: installs Themis as the repo's unit test framework
17
- - `npx themis init`: creates `themis.config.json` and a sample test
18
- - `npx themis generate src`: generates deterministic unit tests for JS/TS exports under `tests/generated`
17
+ - `npx themis init`: creates `themis.config.json` and adds `.themis/` to `.gitignore`
18
+ - `npx themis generate src`: generates deterministic unit tests for JS/TS exports under `tests/generated`, using `.generated.test.ts` for TS/TSX sources and `.generated.test.js` for JS/JSX sources
19
19
  - `npx themis test`: runs the suite
20
20
 
21
21
  ## Migrate From Jest Or Vitest
@@ -52,6 +52,8 @@ Install with `npm install -D @vitronai/themis`.
52
52
  Initialize with `npx themis init`.
53
53
  Generate tests with `npx themis generate src`.
54
54
  Run tests with `npx themis test`.
55
+ Keep `.themis/` in `.gitignore`.
56
+ Expect generated tests to land under `tests/generated` with `.generated.test.ts` for TS/TSX sources and `.generated.test.js` for JS/JSX sources.
55
57
  For migrated suites, use `npx themis migrate jest` or `npx themis migrate vitest`.
56
58
  Prefer `intent(...)` for behavior and workflow tests.
57
59
  Prefer `test(...)` for low-level unit checks.
@@ -61,5 +63,6 @@ Do not claim Themis is "not a unit test framework".
61
63
  ## Notes
62
64
 
63
65
  - Themis is a unit test framework and test generator for Node.js and TypeScript projects.
66
+ - `.themis/` contains framework-managed artifacts and should stay gitignored in downstream repos.
64
67
  - This repository's own [`AGENTS.md`](../AGENTS.md) is for contributors working on Themis itself. It does not automatically configure external repos.
65
68
  - You do not need an MCP server to use Themis from another repo. Clear repo instructions plus the normal CLI commands are enough.
package/docs/api.md CHANGED
@@ -35,7 +35,19 @@ themis migrate <jest|vitest>
35
35
  Creates:
36
36
 
37
37
  - `themis.config.json` with default settings
38
- - `tests/example.test.js` sample test (if missing)
38
+ - adds `.themis/` to `.gitignore`
39
+
40
+ ## `themis test`
41
+
42
+ Runs discovered tests, writes `.themis/**` run artifacts, and supports rerun, contract-update, and generated-test repair loops.
43
+
44
+ If generated tests fail because scanned source files drifted or hints need tightening, use:
45
+
46
+ ```bash
47
+ npx themis test --fix
48
+ ```
49
+
50
+ `--fix` reads `.themis/runs/fix-handoff.json`, regenerates the affected generated suites with `--update`, scaffolds hints when the repair strategy requires them, and reruns the suite.
39
51
 
40
52
  ## `themis generate`
41
53
 
@@ -47,7 +59,8 @@ Default behavior:
47
59
 
48
60
  - input directory: `src`
49
61
  - output directory: `tests/generated`
50
- - generated files mirror the scanned source tree with `*.generated.test.js`
62
+ - generated files mirror the scanned source tree with `*.generated.test.ts` for TS/TSX sources and `*.generated.test.js` for JS/JSX sources
63
+ - generated tests import their shared contract runtime from `@vitronai/themis/contract-runtime` instead of writing framework helper files into the repo
51
64
  - generated tests assert normalized runtime export contracts directly in generated source
52
65
  - scenario adapters cover React components, React hooks, Next app components, Next route handlers, generic route handlers, and Node service functions when inputs can be inferred or hinted
53
66
  - React component and hook adapters also assert inferred interaction/state contracts when event handlers or zero-argument stateful methods are available
@@ -56,10 +69,10 @@ Default behavior:
56
69
  - project-level providers from `themis.generate.js` / `themis.generate.cjs` can match source files, inject shared fixture data, register runtime mocks, and wrap generated component renders for provider-aware DOM contracts and behavioral flow coverage
57
70
  - provider presets can declare router, Next navigation, auth/session, React Query, Zustand, and Redux-style wrapper metadata without hand-writing every wrapper shell
58
71
  - richer flow expectations can assert text transitions, attribute state, and role presence for empty, disabled, retry, error, and recovery paths
59
- - `.themis/generate-map.json` records source-to-generated-test mappings plus scenario metadata
60
- - `.themis/generate-last.json` stores the full machine-readable generate payload
61
- - `.themis/generate-handoff.json` stores a compact prompt-ready handoff payload for agents
62
- - `.themis/generate-backlog.json` stores unresolved skips, conflicts, and confidence debt with suggested remediation
72
+ - `.themis/generate/generate-map.json` records source-to-generated-test mappings plus scenario metadata
73
+ - `.themis/generate/generate-last.json` stores the full machine-readable generate payload
74
+ - `.themis/generate/generate-handoff.json` stores a compact prompt-ready handoff payload for agents
75
+ - `.themis/generate/generate-backlog.json` stores unresolved skips, conflicts, and confidence debt with suggested remediation
63
76
 
64
77
  ## `themis generate` options
65
78
 
@@ -165,7 +178,7 @@ Behavior:
165
178
  - adds `tests/setup.themis.js` to `setupFiles`
166
179
  - writes `themis.compat.js` as a local compatibility bridge
167
180
  - adds `test:themis` to `package.json` when missing
168
- - writes `.themis/migration-report.json` with detected compatibility imports and next actions
181
+ - writes `.themis/migration/migration-report.json` with detected compatibility imports and next actions
169
182
  - relies on built-in runtime compatibility for `@jest/globals`, `vitest`, and `@testing-library/react`
170
183
  - preserves a path to replace snapshot-heavy suites with direct assertions and generated contract tests instead of porting snapshot files as-is
171
184
 
@@ -189,9 +202,10 @@ Migration options:
189
202
  | `--update-contracts` | flag | Accept updated `captureContract(...)` baselines for the selected tests. |
190
203
  | `-w`, `--watch` | flag | Rerun the selected suite when watched project files change. |
191
204
  | `--stability <N>` | positive integer | Run selected tests `N` times and classify stability (`stable_pass`, `stable_fail`, `unstable`). |
192
- | `--html-output <path>` | string | Output path for `--reporter html` (default: `.themis/report.html`). |
205
+ | `--html-output <path>` | string | Output path for `--reporter html` (default: `.themis/reports/report.html`). |
193
206
  | `--match "<regex>"` | string | Run only tests whose full name matches regex. |
194
- | `--rerun-failed` | flag | Rerun failures from `.themis/failed-tests.json`. |
207
+ | `--rerun-failed` | flag | Rerun failures from `.themis/runs/failed-tests.json`. |
208
+ | `--fix` | flag | Apply generated-test autofixes from `.themis/runs/fix-handoff.json` and rerun the suite. |
195
209
  | `--no-memes` | flag | Disable meme intent aliases (`cook`, `yeet`, `vibecheck`, `wipe`). |
196
210
 
197
211
  Migration compatibility:
@@ -199,7 +213,7 @@ Migration compatibility:
199
213
  - imports from `@jest/globals` are supported at runtime
200
214
  - imports from `vitest` are supported at runtime
201
215
  - imports from `@testing-library/react` are supported via Themis `render`, `screen`, `fireEvent`, `waitFor`, `cleanup`, and `act`
202
- - `themis migrate <jest|vitest>` also emits `.themis/migration-report.json` with detected files and recommended next actions
216
+ - `themis migrate <jest|vitest>` also emits `.themis/migration/migration-report.json` with detected files and recommended next actions
203
217
 
204
218
  Additional option:
205
219
 
@@ -235,17 +249,21 @@ Generate-specific note:
235
249
 
236
250
  ## Artifacts
237
251
 
238
- Each run writes to `.themis/`:
239
-
240
- - `last-run.json`: full run payload (`RunResult`)
241
- - `failed-tests.json`: failed subset (`themis.failures.v1`)
242
- - `run-diff.json`: diff against the previous run
243
- - `run-history.json`: rolling recent-run history
244
- - `fix-handoff.json`: deduped repair artifact for generated test failures (`themis.fix.handoff.v1`)
245
- - `migration-report.json`: migration inventory for Jest/Vitest bridge scaffolds (`themis.migration.report.v1`)
246
- - `contract-diff.json`: contract capture drift, updates, and update commands (`themis.contract.diff.v1`)
247
- - `benchmark-last.json`: latest benchmark comparison payload plus migration proof (`themis.benchmark.result.v1`)
248
- - `migration-proof.json`: synthetic migration conversion proof emitted by `npm run benchmark` (`themis.migration.proof.v1`)
252
+ Themis writes artifacts under `.themis/`:
253
+
254
+ - `.themis/runs/last-run.json`: full run payload (`RunResult`)
255
+ - `.themis/runs/failed-tests.json`: failed subset (`themis.failures.v1`)
256
+ - `.themis/diffs/run-diff.json`: diff against the previous run
257
+ - `.themis/runs/run-history.json`: rolling recent-run history
258
+ - `.themis/runs/fix-handoff.json`: deduped repair artifact for generated test failures (`themis.fix.handoff.v1`)
259
+ - `.themis/migration/migration-report.json`: migration inventory for Jest/Vitest bridge scaffolds (`themis.migration.report.v1`)
260
+ - `.themis/diffs/contract-diff.json`: contract capture drift, updates, and update commands (`themis.contract.diff.v1`)
261
+ - `.themis/generate/generate-last.json`: latest generate payload (`themis.generate.result.v1`)
262
+ - `.themis/generate/generate-map.json`: source-to-generated-test mapping (`themis.generate.map.v1`)
263
+ - `.themis/generate/generate-handoff.json`: compact agent handoff (`themis.generate.handoff.v1`)
264
+ - `.themis/generate/generate-backlog.json`: unresolved generation debt (`themis.generate.backlog.v1`)
265
+ - `.themis/benchmarks/benchmark-last.json`: latest benchmark comparison payload plus migration proof (`themis.benchmark.result.v1`)
266
+ - `.themis/benchmarks/migration-proof.json`: synthetic migration conversion proof emitted by `npm run benchmark` (`themis.migration.proof.v1`)
249
267
 
250
268
  Formal schemas:
251
269
 
@@ -260,7 +278,7 @@ Formal schemas:
260
278
 
261
279
  Human-facing artifact:
262
280
 
263
- - `.themis/report.html`: interactive HTML verdict report
281
+ - `.themis/reports/report.html`: interactive HTML verdict report
264
282
 
265
283
  Agent payload details:
266
284
 
@@ -268,8 +286,9 @@ Agent payload details:
268
286
  - `analysis.failureClusters` groups failures by shared fingerprint
269
287
  - `analysis.stability` captures multi-run classifications and per-test status sequences
270
288
  - `analysis.comparison` reports delta stats plus new and resolved failures against the previous run
271
- - `artifacts.fixHandoff` points to `.themis/fix-handoff.json` for generated failure repair loops
289
+ - `artifacts.fixHandoff` points to `.themis/runs/fix-handoff.json` for generated failure repair loops
272
290
  - fix handoff entries include `repairStrategy`, `candidateFiles`, and `autofixCommand` for machine repair loops
291
+ - `hints.repairGenerated` points to `npx themis test --fix` for a first-party generated-suite repair command
273
292
 
274
293
  ## UI Test Utilities
275
294
 
@@ -415,15 +434,15 @@ Converts a `GenerateSummary` into the same machine-readable payload emitted by `
415
434
 
416
435
  ## `buildGenerateBacklogPayload(summary, cwd?): GenerateBacklogPayload`
417
436
 
418
- Builds the unresolved-work artifact persisted at `.themis/generate-backlog.json`.
437
+ Builds the unresolved-work artifact persisted at `.themis/generate/generate-backlog.json`.
419
438
 
420
439
  ## `buildGenerateHandoff(payload): GenerateHandoffPayload`
421
440
 
422
- Builds the compact handoff payload persisted at `.themis/generate-handoff.json`.
441
+ Builds the compact handoff payload persisted at `.themis/generate/generate-handoff.json`.
423
442
 
424
443
  ## `writeGenerateArtifacts(summary, cwd?): { payload, handoff, backlog }`
425
444
 
426
- Writes `.themis/generate-last.json`, `.themis/generate-handoff.json`, and `.themis/generate-backlog.json` for a generate run and returns all three payloads.
445
+ Writes `.themis/generate/generate-last.json`, `.themis/generate/generate-handoff.json`, and `.themis/generate/generate-backlog.json` for a generate run and returns all three payloads.
427
446
 
428
447
  ## `loadConfig(cwd): ThemisConfig`
429
448
 
package/docs/migration.md CHANGED
@@ -115,7 +115,7 @@ Jest/Vitest:
115
115
 
116
116
  Themis:
117
117
 
118
- - field-level diffs in `.themis/contract-diff.json`
118
+ - field-level diffs in `.themis/diffs/contract-diff.json`
119
119
  - visible drift/update summaries in CLI and HTML reports
120
120
  - explicit acceptance via `npx themis test --update-contracts --match "<regex>"`
121
121
 
@@ -154,7 +154,7 @@ These are the strongest head-to-head examples to use when explaining why Themis
154
154
 
155
155
  1. Snapshot replacement: `captureContract(...)` plus `--update-contracts` gives baseline capture without snapshot churn.
156
156
  2. Codemod migration: `themis migrate --convert` moves common Jest/Vitest matcher syntax toward native Themis without a manual rewrite pass.
157
- 3. Agent triage: `--agent`, `.themis/run-diff.json`, `.themis/fix-handoff.json`, and `.themis/contract-diff.json` give machines structured rerun and repair inputs.
157
+ 3. Agent triage: `--agent`, `.themis/diffs/run-diff.json`, `.themis/runs/fix-handoff.json`, and `.themis/diffs/contract-diff.json` give machines structured rerun and repair inputs.
158
158
  4. Human review: next reporter and HTML report now surface contract drift alongside failures, instead of burying meaning in raw output.
159
159
  5. Generated coverage: `themis generate src` adds source-driven contract tests next to migrated suites, so adoption improves coverage instead of merely changing runners.
160
160
 
package/docs/showcases.md CHANGED
@@ -30,7 +30,7 @@ Why it wins:
30
30
 
31
31
  - captures only the fields that matter
32
32
  - masks volatile IDs and timestamps instead of accepting noisy churn
33
- - writes machine-readable drift to `.themis/contract-diff.json`
33
+ - writes machine-readable drift to `.themis/diffs/contract-diff.json`
34
34
  - updates stay explicit with `npx themis test --update-contracts`
35
35
 
36
36
  ## 2. Migrate Jest/Vitest suites without a rewrite freeze
@@ -67,7 +67,7 @@ Why it wins:
67
67
 
68
68
  - removes common compatibility imports
69
69
  - rewrites common matcher aliases into native Themis forms
70
- - emits `.themis/migration-report.json` so agents and humans can track what changed
70
+ - emits `.themis/migration/migration-report.json` so agents and humans can track what changed
71
71
 
72
72
  ## 3. Give agents a repair loop humans can still read
73
73
 
@@ -75,10 +75,10 @@ Jest/Vitest mostly produce console text plus optional snapshots.
75
75
 
76
76
  Themis produces:
77
77
 
78
- - `.themis/failed-tests.json`
79
- - `.themis/run-diff.json`
80
- - `.themis/fix-handoff.json`
81
- - `.themis/contract-diff.json`
78
+ - `.themis/runs/failed-tests.json`
79
+ - `.themis/diffs/run-diff.json`
80
+ - `.themis/runs/fix-handoff.json`
81
+ - `.themis/diffs/contract-diff.json`
82
82
 
83
83
  Why it wins:
84
84
 
@@ -134,7 +134,7 @@ CI also carries a same-host performance artifact job:
134
134
 
135
135
  - `React Showcase Perf`
136
136
 
137
- That job runs the same two React scenarios under Themis, Jest, and Vitest on one runner, writes `.themis/showcase-comparison/perf-summary.json` and `.themis/showcase-comparison/perf-summary.md`, and uploads them as build artifacts.
137
+ That job runs the same two React scenarios under Themis, Jest, and Vitest on one runner, writes `.themis/benchmarks/showcase-comparison/perf-summary.json` and `.themis/benchmarks/showcase-comparison/perf-summary.md`, and uploads them as build artifacts.
138
138
 
139
139
  Current documented sample result from the same-host run:
140
140
 
@@ -5,29 +5,29 @@ Themis includes an in-repo VS Code extension scaffold at [`packages/themis-vscod
5
5
  This is the intended shape of the editor UX:
6
6
 
7
7
  - a Themis activity-bar container
8
- - a results sidebar driven by `.themis/*` artifacts
8
+ - a results sidebar driven by `.themis/**` artifacts
9
9
  - commands to run tests, rerun failures, refresh results, and open the HTML report
10
10
  - commands to accept reviewed contract baselines and rerun migration codemods
11
11
  - failure navigation that jumps from artifact data into the source file
12
12
  - generated-review navigation for source files, generated tests, hint sidecars, and backlog items
13
- - contract-review and migration-review groups driven by `.themis/contract-diff.json` and `.themis/migration-report.json`
13
+ - contract-review and migration-review groups driven by `.themis/diffs/contract-diff.json` and `.themis/migration/migration-report.json`
14
14
 
15
15
  ## Current MVP Scope
16
16
 
17
17
  The scaffold currently supports:
18
18
 
19
- - reading `.themis/last-run.json`
20
- - reading `.themis/failed-tests.json`
21
- - reading `.themis/run-diff.json`
22
- - reading `.themis/contract-diff.json`
23
- - reading `.themis/migration-report.json`
24
- - reading `.themis/generate-last.json`
25
- - reading `.themis/generate-map.json`
26
- - reading `.themis/generate-backlog.json`
27
- - opening `.themis/report.html` in a webview
19
+ - reading `.themis/runs/last-run.json`
20
+ - reading `.themis/runs/failed-tests.json`
21
+ - reading `.themis/diffs/run-diff.json`
22
+ - reading `.themis/diffs/contract-diff.json`
23
+ - reading `.themis/migration/migration-report.json`
24
+ - reading `.themis/generate/generate-last.json`
25
+ - reading `.themis/generate/generate-map.json`
26
+ - reading `.themis/generate/generate-backlog.json`
27
+ - opening `.themis/reports/report.html` in a webview
28
28
  - surfacing generated source/test/hint mappings in the sidebar
29
29
  - surfacing unresolved generate backlog and gate state in the sidebar
30
- - refreshing when `.themis/*` files change
30
+ - refreshing when `.themis/**` files change
31
31
 
32
32
  The extension is intentionally thin. It shells out to Themis commands and treats the CLI plus artifacts as the canonical contract.
33
33
 
@@ -17,6 +17,8 @@ npx themis generate src
17
17
  npx themis test
18
18
  ```
19
19
 
20
+ `npx themis init` adds `.themis/` to `.gitignore`, and `npx themis generate src` emits `.generated.test.ts` for TS/TSX sources and `.generated.test.js` for JS/JSX sources.
21
+
20
22
  For downstream repo instructions and a copyable `AGENTS.md` template, see [`docs/agents-adoption.md`](agents-adoption.md).
21
23
 
22
24
  ## What "Next-Gen" Means Here
@@ -48,7 +50,7 @@ Themis supports structured outputs for tooling loops:
48
50
 
49
51
  - `--json` for generic automation
50
52
  - `--agent` for AI-agent workflows
51
- - `.themis/failed-tests.json` for deterministic reruns
53
+ - `.themis/runs/failed-tests.json` for deterministic reruns
52
54
 
53
55
  ## 3) Deterministic Rerun Workflow
54
56
 
@@ -86,7 +88,7 @@ Themis ships workflow features agents can use directly:
86
88
  - generated contract tests that keep baselines in readable source instead of opaque snapshot blobs
87
89
  - machine-readable artifacts that let agents inspect and explain changes before updating tests
88
90
  - mocks and spies with `fn`, `spyOn`, and `mock`
89
- - `.themis/run-diff.json` and `.themis/run-history.json`
91
+ - `.themis/diffs/run-diff.json` and `.themis/runs/run-history.json`
90
92
  - HTML verdict reports for human review
91
93
 
92
94
  ### Comparable To Snapshot Workflows, Without Snapshot Rot
@@ -130,7 +132,7 @@ Performance is measured and guarded:
130
132
 
131
133
  ## 11) Editor Surface Without Replacing The CLI
132
134
 
133
- Themis includes a thin VS Code extension scaffold that reads `.themis/*` artifacts, reruns tests, and opens the HTML report. The CLI remains the source of truth.
135
+ Themis includes a thin VS Code extension scaffold that reads `.themis/**` artifacts, reruns tests, and opens the HTML report. The CLI remains the source of truth.
134
136
 
135
137
  ## Proof Checklist
136
138
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@vitronai/themis",
3
- "version": "0.1.5",
3
+ "version": "0.1.6",
4
4
  "description": "Intent-first unit test framework and test generator for AI agents in Node.js and TypeScript",
5
5
  "license": "MIT",
6
6
  "author": "Vitron AI",
@@ -54,6 +54,10 @@
54
54
  "require": "./globals.js",
55
55
  "default": "./globals.js"
56
56
  },
57
+ "./contract-runtime": {
58
+ "require": "./src/contract-runtime.js",
59
+ "default": "./src/contract-runtime.js"
60
+ },
57
61
  "./package.json": "./package.json"
58
62
  },
59
63
  "files": [
@@ -0,0 +1,111 @@
1
+ const path = require('path');
2
+
3
+ const ARTIFACT_DIR = '.themis';
4
+ const RUNS_DIR = ['runs'];
5
+ const DIFFS_DIR = ['diffs'];
6
+ const GENERATE_DIR = ['generate'];
7
+ const REPORTS_DIR = ['reports'];
8
+ const MIGRATION_DIR = ['migration'];
9
+ const BENCHMARKS_DIR = ['benchmarks'];
10
+ const SHOWCASE_COMPARISON_DIR = [...BENCHMARKS_DIR, 'showcase-comparison'];
11
+ const MIGRATION_FIXTURES_DIR = [...MIGRATION_DIR, 'fixtures'];
12
+
13
+ const ARTIFACT_SEGMENTS = Object.freeze({
14
+ lastRun: [...RUNS_DIR, 'last-run.json'],
15
+ failedTests: [...RUNS_DIR, 'failed-tests.json'],
16
+ runDiff: [...DIFFS_DIR, 'run-diff.json'],
17
+ runHistory: [...RUNS_DIR, 'run-history.json'],
18
+ fixHandoff: [...RUNS_DIR, 'fix-handoff.json'],
19
+ contractDiff: [...DIFFS_DIR, 'contract-diff.json'],
20
+ generateMap: [...GENERATE_DIR, 'generate-map.json'],
21
+ generateResult: [...GENERATE_DIR, 'generate-last.json'],
22
+ generateHandoff: [...GENERATE_DIR, 'generate-handoff.json'],
23
+ generateBacklog: [...GENERATE_DIR, 'generate-backlog.json'],
24
+ migrationReport: [...MIGRATION_DIR, 'migration-report.json'],
25
+ htmlReport: [...REPORTS_DIR, 'report.html'],
26
+ benchmarkLast: [...BENCHMARKS_DIR, 'benchmark-last.json'],
27
+ migrationProof: [...BENCHMARKS_DIR, 'migration-proof.json']
28
+ });
29
+
30
+ const LEGACY_ARTIFACT_SEGMENTS = Object.freeze({
31
+ lastRun: ['last-run.json'],
32
+ failedTests: ['failed-tests.json'],
33
+ runDiff: ['run-diff.json'],
34
+ runHistory: ['run-history.json'],
35
+ fixHandoff: ['fix-handoff.json'],
36
+ contractDiff: ['contract-diff.json'],
37
+ generateMap: ['generate-map.json'],
38
+ generateResult: ['generate-last.json'],
39
+ generateHandoff: ['generate-handoff.json'],
40
+ generateBacklog: ['generate-backlog.json'],
41
+ migrationReport: ['migration-report.json'],
42
+ htmlReport: ['report.html'],
43
+ benchmarkLast: ['benchmark-last.json'],
44
+ migrationProof: ['migration-proof.json']
45
+ });
46
+
47
+ function joinRelative(segments) {
48
+ return path.posix.join(ARTIFACT_DIR, ...segments);
49
+ }
50
+
51
+ function joinAbsolute(cwd, segments) {
52
+ return path.join(cwd, ARTIFACT_DIR, ...segments);
53
+ }
54
+
55
+ function buildRelativePathMap(segmentMap) {
56
+ return Object.fromEntries(
57
+ Object.entries(segmentMap).map(([key, segments]) => [key, joinRelative(segments)])
58
+ );
59
+ }
60
+
61
+ const ARTIFACT_RELATIVE_PATHS = Object.freeze(buildRelativePathMap(ARTIFACT_SEGMENTS));
62
+ const LEGACY_ARTIFACT_RELATIVE_PATHS = Object.freeze(buildRelativePathMap(LEGACY_ARTIFACT_SEGMENTS));
63
+
64
+ function resolveArtifactPath(cwd, key) {
65
+ return joinAbsolute(cwd, ARTIFACT_SEGMENTS[key]);
66
+ }
67
+
68
+ function resolveLegacyArtifactPath(cwd, key) {
69
+ return joinAbsolute(cwd, LEGACY_ARTIFACT_SEGMENTS[key]);
70
+ }
71
+
72
+ function resolveArtifactDir(cwd, ...segments) {
73
+ return path.join(cwd, ARTIFACT_DIR, ...segments);
74
+ }
75
+
76
+ function resolveRelativeDir(...segments) {
77
+ return path.posix.join(ARTIFACT_DIR, ...segments);
78
+ }
79
+
80
+ function getArtifactPaths(cwd) {
81
+ return Object.fromEntries(
82
+ Object.keys(ARTIFACT_SEGMENTS).map((key) => [key, resolveArtifactPath(cwd, key)])
83
+ );
84
+ }
85
+
86
+ function getArtifactPathCandidates(cwd, key) {
87
+ const nextPath = resolveArtifactPath(cwd, key);
88
+ const legacyPath = resolveLegacyArtifactPath(cwd, key);
89
+ return nextPath === legacyPath ? [nextPath] : [nextPath, legacyPath];
90
+ }
91
+
92
+ module.exports = {
93
+ ARTIFACT_DIR,
94
+ RUNS_DIR,
95
+ DIFFS_DIR,
96
+ GENERATE_DIR,
97
+ REPORTS_DIR,
98
+ MIGRATION_DIR,
99
+ BENCHMARKS_DIR,
100
+ SHOWCASE_COMPARISON_DIR,
101
+ MIGRATION_FIXTURES_DIR,
102
+ ARTIFACT_SEGMENTS,
103
+ ARTIFACT_RELATIVE_PATHS,
104
+ LEGACY_ARTIFACT_RELATIVE_PATHS,
105
+ resolveArtifactPath,
106
+ resolveLegacyArtifactPath,
107
+ resolveArtifactDir,
108
+ resolveRelativeDir,
109
+ getArtifactPaths,
110
+ getArtifactPathCandidates
111
+ };