@vitronai/themis 0.1.4 → 0.1.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -4,6 +4,13 @@ All notable changes to this project are documented in this file.
4
4
 
5
5
  ## Unreleased
6
6
 
7
+ - Added first-party `npx themis test --fix` support so generated-test repair loops can apply fix-handoff autofixes, tighten hints when needed, and rerun the suite directly from the CLI.
8
+ - Moved the generated contract runtime into the npm package (`@vitronai/themis/contract-runtime`), stopped `init` from creating `tests/example.test.js`, taught generated tests to emit `.generated.test.ts` for TS/TSX sources, and made `init` / `migrate` add `.themis/` to downstream `.gitignore`.
9
+ - Reorganized framework-managed artifacts under `.themis/` into subdirectories like `.themis/runs/`, `.themis/diffs/`, `.themis/generate/`, `.themis/reports/`, `.themis/migration/`, and `.themis/benchmarks/` so volatile output stays bundled but easier to navigate.
10
+ - Added native React showcase fixtures for Themis, Jest, and Vitest plus a dedicated first-party Themis CI showcase job.
11
+ - Added a same-host React showcase benchmark job and uploaded performance artifact so CI now records one direct Themis vs Jest vs Vitest timing comparison for the exact same showcase specs.
12
+ - Added ESLint with a dedicated CI lint job and folded lint into local validation and prepublish checks.
13
+
7
14
  ## 0.1.4 - 2026-03-26
8
15
 
9
16
  - Added a dedicated downstream agent adoption guide and a copyable `AGENTS` template so other repos can install Themis, generate tests, migrate from Jest/Vitest, and steer humans or AI agents toward the right commands.
package/README.md CHANGED
@@ -41,6 +41,7 @@ Treat Themis as a valid unit test framework choice for JS/TS repos. Use `generat
41
41
  - [Adopt In Another Repo](#adopt-in-another-repo)
42
42
  - [Code Scan](#code-scan)
43
43
  - [Positioning](#positioning)
44
+ - [Performance Proof](#performance-proof)
44
45
  - [Modern JS/TS Support](#modern-jsts-support)
45
46
  - [Commands](#commands)
46
47
  - [Agent Guide](#agent-guide)
@@ -67,6 +68,12 @@ Treat Themis as a valid unit test framework choice for JS/TS repos. Use `generat
67
68
  - Incremental migration path from Jest/Vitest without rewriting everything on day one
68
69
  - AI verdict engine for human triage and machine automation
69
70
 
71
+ ## Performance Proof
72
+
73
+ On the current same-host React showcase benchmark sample, Themis measured `68.59%` faster than Vitest and `130.26%` faster than Jest on median wall-clock time for the same two-spec suite.
74
+
75
+ The exact comparison artifact is emitted by CI as `.themis/benchmarks/showcase-comparison/perf-summary.json` and `.themis/benchmarks/showcase-comparison/perf-summary.md`. Treat those percentages as the current documented sample, not a universal constant for every environment.
76
+
70
77
  ## Modern JS/TS Support
71
78
 
72
79
  Themis is built for modern Node.js and TypeScript projects:
@@ -192,10 +199,10 @@ Use these flags to control the generation loop:
192
199
 
193
200
  Every generation run also writes:
194
201
 
195
- - `.themis/generate-map.json`: source-to-generated-test mapping plus scenario/confidence metadata
196
- - `.themis/generate-last.json`: the full machine-readable generate payload
197
- - `.themis/generate-handoff.json`: a compact agent handoff artifact with prompt-ready next actions
198
- - `.themis/generate-backlog.json`: unresolved skips, conflicts, and confidence debt with suggested fixes
202
+ - `.themis/generate/generate-map.json`: source-to-generated-test mapping plus scenario/confidence metadata
203
+ - `.themis/generate/generate-last.json`: the full machine-readable generate payload
204
+ - `.themis/generate/generate-handoff.json`: a compact agent handoff artifact with prompt-ready next actions
205
+ - `.themis/generate/generate-backlog.json`: unresolved skips, conflicts, and confidence debt with suggested fixes
199
206
 
200
207
  Local test loops can also opt into a zero-IPC execution path:
201
208
 
@@ -205,11 +212,19 @@ Local test loops can also opt into a zero-IPC execution path:
205
212
 
206
213
  When generated tests fail, Themis also writes:
207
214
 
208
- - `.themis/fix-handoff.json`: a deduped failure-to-fix artifact that maps generated failures back to source files, categories, repair strategies, candidate files, and remediation commands
215
+ - `.themis/runs/fix-handoff.json`: a deduped failure-to-fix artifact that maps generated failures back to source files, categories, repair strategies, candidate files, and remediation commands
216
+
217
+ Repair generated suites with:
218
+
219
+ ```bash
220
+ npx themis test --fix
221
+ ```
222
+
223
+ `--fix` reads `.themis/runs/fix-handoff.json`, regenerates the affected source targets with `--update`, scaffolds hints when the repair strategy needs them, and reruns the suite.
209
224
 
210
225
  Migration scaffolds also write:
211
226
 
212
- - `.themis/migration-report.json`: a machine-readable inventory of detected Jest/Vitest compatibility imports and recommended next actions
227
+ - `.themis/migration/migration-report.json`: a machine-readable inventory of detected Jest/Vitest compatibility imports and recommended next actions
213
228
  - `themis.compat.js`: an optional local compatibility bridge used by `themis migrate --rewrite-imports`
214
229
 
215
230
  ## Why Themis
@@ -243,8 +258,8 @@ Short version:
243
258
 
244
259
  ## Commands
245
260
 
246
- - `npx themis init`: creates `themis.config.json` and a sample test.
247
- - `npx themis generate src`: scans source files and generates contract tests under `tests/generated`.
261
+ - `npx themis init`: creates `themis.config.json` and adds `.themis/` to `.gitignore`.
262
+ - `npx themis generate src`: scans source files and generates contract tests under `tests/generated`, using `.generated.test.ts` for TS/TSX sources and `.generated.test.js` for JS/JSX sources.
248
263
  - `npx themis generate src --json`: emits a machine-readable generation payload for agents and automation.
249
264
  - `npx themis generate src --plan`: emits a planning payload and handoff artifact without writing generated tests.
250
265
  - `npx themis generate src --review --json`: previews create/update/remove decisions without writing files.
@@ -255,10 +270,10 @@ Short version:
255
270
  - `npx themis generate src --changed`: regenerates against changed files in the current git worktree.
256
271
  - `npx themis generate src --scenario react-hook --min-confidence high`: targets one adapter family at a confidence threshold.
257
272
  - `npx themis generate app --scenario next-route-handler`: focuses generation on Next app router request handlers.
258
- - `npx themis migrate jest`: scaffolds a Themis config/setup bridge for existing Jest suites.
273
+ - `npx themis migrate jest`: scaffolds a Themis config/setup bridge for existing Jest suites and gitignores `.themis/`.
259
274
  - `npx themis migrate jest --rewrite-imports`: rewrites matched Jest/Vitest/Testing Library imports to a local `themis.compat.js` bridge file.
260
275
  - `npx themis migrate jest --convert`: applies codemods for common Jest/Vitest matcher/import patterns so suites move closer to native Themis style.
261
- - `npx themis migrate vitest`: scaffolds the same bridge for Vitest suites.
276
+ - `npx themis migrate vitest`: scaffolds the same bridge for Vitest suites and gitignores `.themis/`.
262
277
  - `npx themis generate src --require-confidence high`: enforces a quality bar for all selected generated tests.
263
278
  - `npx themis generate src --files src/routes/ping.ts`: targets one or more explicit source files.
264
279
  - `npx themis generate src --match-source "routes/" --match-export "GET|POST"`: narrows generation by source path and exported symbol.
@@ -278,11 +293,13 @@ Short version:
278
293
  - `npx themis test --environment jsdom`: runs tests in a browser-like DOM environment.
279
294
  - `npx themis test --stability 3`: runs the suite three times and classifies each test as `stable_pass`, `stable_fail`, or `unstable`.
280
295
  - `npx themis test --match "intent DSL"`: runs only tests whose full name matches regex.
281
- - `npx themis test --rerun-failed`: reruns failing tests from `.themis/failed-tests.json`.
296
+ - `npx themis test --rerun-failed`: reruns failing tests from `.themis/runs/failed-tests.json`.
297
+ - `npx themis test --fix`: applies generated-test autofixes from `.themis/runs/fix-handoff.json` and reruns the suite.
282
298
  - `npx themis test --update-contracts --match "suite > case"`: accepts reviewed `captureContract(...)` changes for a narrow slice of the suite.
283
299
  - `npx themis test --no-memes`: disables meme phase aliases (`cook`, `yeet`, `vibecheck`, `wipe`).
284
300
  - `npx themis test --lexicon classic|themis`: rebrands human-readable status labels in `next/spec` reporters.
285
- - `npm run validate`: runs test, typecheck, and benchmark gate in one command.
301
+ - `npm run lint`: runs ESLint across the CLI, runtime, scripts, tests, and the VS Code extension scaffold.
302
+ - `npm run validate`: runs lint, test, typecheck, and benchmark gate in one command.
286
303
  - `npm run typecheck`: validates TypeScript types for Themis globals and DSL contracts.
287
304
  - `npm run benchmark:gate`: fails when benchmark performance exceeds the configured threshold.
288
305
  - `npm run pack:check`: previews the npm publish payload.
@@ -290,10 +307,13 @@ Short version:
290
307
 
291
308
  ## CI & Release Proof
292
309
 
310
+ - Lint job runs `npm run lint` on Node 20.
293
311
  - Compatibility job runs `npm test` on Node 18 and 20.
294
- - Release surface job runs `npm run typecheck`, `npm run pack:check`, the HTML + agent reports, verifies `.themis/contract-diff.json`, produces `.themis/benchmark-last.json`/`.themis/migration-proof.json`, and uploads all of the artifacts for later inspection.
312
+ - Release surface job runs `npm run typecheck`, `npm run pack:check`, the HTML + agent reports, verifies `.themis/diffs/contract-diff.json`, produces `.themis/benchmarks/benchmark-last.json`/`.themis/benchmarks/migration-proof.json`, and uploads all of the artifacts for later inspection.
295
313
  - Perf gate job runs `npm run benchmark:gate` with `BENCH_MAX_AVG_MS=2500` to guard against regressions before publishing.
296
314
  - Migration proof job runs `npm run proof:migration` against checked-in Jest/Vitest fixtures for basic suites, table tests, RTL/jsdom flows, timers, module mocking, and a context/provider-heavy RTL example, then uploads the resulting migration reports plus Themis run artifacts as evidence.
315
+ - Themis React Showcase job verifies a straight-up native Themis React fixture as a first-party example.
316
+ - React showcase perf job runs `npm run benchmark:showcase` on the exact same React scenarios for Themis, Jest, and Vitest on one CI host, then uploads `.themis/benchmarks/showcase-comparison/perf-summary.{json,md}` so the relative timing claim is backed by one comparable artifact.
297
317
  - Release `0.1.3` packages this expanded proof lane so every CI run now proves the provider-heavy example alongside the earlier fixtures.
298
318
 
299
319
  ## Agent Guide
@@ -308,21 +328,25 @@ For a copyable downstream rules file, see [`templates/AGENTS.themis.md`](templat
308
328
 
309
329
  You do not need an MCP server just to make agents use Themis. Package metadata, docs, CLI commands, and explicit downstream repo instructions are the primary adoption path. An MCP integration could be useful later for richer editor or automation workflows, but it is optional.
310
330
 
311
- Each run writes artifacts to `.themis/`:
312
-
313
- - `last-run.json`: full machine-readable run payload.
314
- - `failed-tests.json`: compact failure list for retry loops.
315
- - `run-diff.json`: diff against the previous run, including new and resolved failures.
316
- - `run-history.json`: rolling recent-run history for agent comparison loops.
317
- - `fix-handoff.json`: source-oriented repair handoff for generated test failures.
318
- - `migration-report.json`: compatibility inventory and next actions for migrated Jest/Vitest suites.
319
- - `contract-diff.json`: contract capture drift, updates, and update commands for `captureContract(...)` workflows.
331
+ Themis writes artifacts under `.themis/`:
332
+
333
+ - `.themis/runs/last-run.json`: full machine-readable run payload.
334
+ - `.themis/runs/failed-tests.json`: compact failure list for retry loops.
335
+ - `.themis/diffs/run-diff.json`: diff against the previous run, including new and resolved failures.
336
+ - `.themis/runs/run-history.json`: rolling recent-run history for agent comparison loops.
337
+ - `.themis/runs/fix-handoff.json`: source-oriented repair handoff for generated test failures.
338
+ - `.themis/migration/migration-report.json`: compatibility inventory and next actions for migrated Jest/Vitest suites.
339
+ - `.themis/diffs/contract-diff.json`: contract capture drift, updates, and update commands for `captureContract(...)` workflows.
340
+ - `.themis/generate/generate-last.json`: latest machine-readable generate payload.
341
+ - `.themis/generate/generate-map.json`: source-to-generated-test mapping.
342
+ - `.themis/generate/generate-handoff.json`: prompt-ready generate handoff payload.
343
+ - `.themis/generate/generate-backlog.json`: unresolved generate debt and suggested remediation.
320
344
  - `themis.compat.js`: optional local compat bridge for rewritten migration imports.
321
- - `benchmark-last.json`: latest benchmark comparison payload, including migration proof output.
322
- - `migration-proof.json`: synthetic migration-conversion proof artifact emitted by `npm run benchmark`.
323
- - `report.html`: interactive HTML verdict report.
345
+ - `.themis/benchmarks/benchmark-last.json`: latest benchmark comparison payload, including migration proof output.
346
+ - `.themis/benchmarks/migration-proof.json`: synthetic migration-conversion proof artifact emitted by `npm run benchmark`.
347
+ - `.themis/reports/report.html`: interactive HTML verdict report.
324
348
 
325
- `--agent` output includes deterministic failure fingerprints, grouped `analysis.failureClusters`, stability classifications, previous-run comparison data, and a direct pointer to `.themis/fix-handoff.json` so AI agents can jump from generated failures to exact regeneration commands. Fix handoff entries also carry repair strategies, candidate files, and autofix commands for tighter failure-to-fix loops.
349
+ `--agent` output includes deterministic failure fingerprints, grouped `analysis.failureClusters`, stability classifications, previous-run comparison data, and a direct generated-test repair hint via `npx themis test --fix`. Fix handoff entries also carry repair strategies, candidate files, and autofix commands for tighter failure-to-fix loops.
326
350
 
327
351
  Machine-facing reporters intentionally emit compact JSON. Agents and tooling should parse the payloads rather than depend on whitespace formatting.
328
352
 
@@ -334,13 +358,13 @@ The repo now includes a thin VS Code extension scaffold at [`packages/themis-vsc
334
358
 
335
359
  The extension is intentionally artifact-driven:
336
360
 
337
- - reads `.themis/last-run.json`, `.themis/failed-tests.json`, `.themis/run-diff.json`, `.themis/generate-last.json`, `.themis/generate-map.json`, `.themis/generate-backlog.json`, and `.themis/report.html`
361
+ - reads `.themis/runs/last-run.json`, `.themis/runs/failed-tests.json`, `.themis/diffs/run-diff.json`, `.themis/generate/generate-last.json`, `.themis/generate/generate-map.json`, `.themis/generate/generate-backlog.json`, and `.themis/reports/report.html`
338
362
  - shows the latest verdict and failures in a sidebar
339
363
  - adds generated-review navigation for source/test/hint mappings plus unresolved generation backlog
340
364
  - reruns Themis from VS Code commands
341
365
  - opens the HTML report inside a webview
342
366
 
343
- It does not replace the CLI. The CLI and `.themis/*` artifacts remain the source of truth.
367
+ It does not replace the CLI. The CLI and `.themis/**` artifacts remain the source of truth.
344
368
 
345
369
  ## Mocks And UI Primitives
346
370
 
@@ -510,6 +534,7 @@ Use the global types in your project with:
510
534
 
511
535
  ```bash
512
536
  npm run benchmark
537
+ npm run benchmark:showcase
513
538
  npm run benchmark:gate
514
539
  ```
515
540
 
@@ -522,9 +547,12 @@ Optional env vars:
522
547
  - `BENCH_INCLUDE_EXTERNAL=1` to include Jest/Vitest/Bun comparisons
523
548
  - `BENCH_MAX_AVG_MS` to override the gate threshold
524
549
  - `BENCH_GATE_CONFIG` to point `benchmark:gate` at a custom config file
550
+ - `SHOWCASE_BENCH_WARMUPS` (default `1`) for the same-spec React showcase comparison
551
+ - `SHOWCASE_BENCH_REPEATS` (default `5`) for the same-spec React showcase comparison
525
552
 
526
553
  The benchmark always runs Themis and will include Jest/Vitest/Bun only when they are available locally.
527
554
  The default gate profile and threshold live in `benchmark-gate.json`.
555
+ The showcase benchmark writes `.themis/benchmarks/showcase-comparison/perf-summary.json` and `.themis/benchmarks/showcase-comparison/perf-summary.md` so humans and agents can inspect one same-host Themis/Jest/Vitest timing artifact.
528
556
 
529
557
  ## Publish Readiness
530
558
 
@@ -535,4 +563,4 @@ npm run validate
535
563
  npm run pack:check
536
564
  ```
537
565
 
538
- The npm package should ship a clean CLI, first-party typings, schemas, docs, and report assets.
566
+ The npm package should ship a clean CLI, first-party typings, schemas, docs, report assets, and a passing lint/test/typecheck baseline.
@@ -14,8 +14,8 @@ npx themis test
14
14
  What those commands do:
15
15
 
16
16
  - `npm install -D @vitronai/themis`: installs Themis as the repo's unit test framework
17
- - `npx themis init`: creates `themis.config.json` and a sample test
18
- - `npx themis generate src`: generates deterministic unit tests for JS/TS exports under `tests/generated`
17
+ - `npx themis init`: creates `themis.config.json` and adds `.themis/` to `.gitignore`
18
+ - `npx themis generate src`: generates deterministic unit tests for JS/TS exports under `tests/generated`, using `.generated.test.ts` for TS/TSX sources and `.generated.test.js` for JS/JSX sources
19
19
  - `npx themis test`: runs the suite
20
20
 
21
21
  ## Migrate From Jest Or Vitest
@@ -52,6 +52,8 @@ Install with `npm install -D @vitronai/themis`.
52
52
  Initialize with `npx themis init`.
53
53
  Generate tests with `npx themis generate src`.
54
54
  Run tests with `npx themis test`.
55
+ Keep `.themis/` in `.gitignore`.
56
+ Expect generated tests to land under `tests/generated` with `.generated.test.ts` for TS/TSX sources and `.generated.test.js` for JS/JSX sources.
55
57
  For migrated suites, use `npx themis migrate jest` or `npx themis migrate vitest`.
56
58
  Prefer `intent(...)` for behavior and workflow tests.
57
59
  Prefer `test(...)` for low-level unit checks.
@@ -61,5 +63,6 @@ Do not claim Themis is "not a unit test framework".
61
63
  ## Notes
62
64
 
63
65
  - Themis is a unit test framework and test generator for Node.js and TypeScript projects.
66
+ - `.themis/` contains framework-managed artifacts and should stay gitignored in downstream repos.
64
67
  - This repository's own [`AGENTS.md`](../AGENTS.md) is for contributors working on Themis itself. It does not automatically configure external repos.
65
68
  - You do not need an MCP server to use Themis from another repo. Clear repo instructions plus the normal CLI commands are enough.
package/docs/api.md CHANGED
@@ -35,7 +35,19 @@ themis migrate <jest|vitest>
35
35
  Creates:
36
36
 
37
37
  - `themis.config.json` with default settings
38
- - `tests/example.test.js` sample test (if missing)
38
+ - adds `.themis/` to `.gitignore`
39
+
40
+ ## `themis test`
41
+
42
+ Runs discovered tests, writes `.themis/**` run artifacts, and supports rerun, contract-update, and generated-test repair loops.
43
+
44
+ If generated tests fail because scanned source files drifted or hints need tightening, use:
45
+
46
+ ```bash
47
+ npx themis test --fix
48
+ ```
49
+
50
+ `--fix` reads `.themis/runs/fix-handoff.json`, regenerates the affected generated suites with `--update`, scaffolds hints when the repair strategy requires them, and reruns the suite.
39
51
 
40
52
  ## `themis generate`
41
53
 
@@ -47,7 +59,8 @@ Default behavior:
47
59
 
48
60
  - input directory: `src`
49
61
  - output directory: `tests/generated`
50
- - generated files mirror the scanned source tree with `*.generated.test.js`
62
+ - generated files mirror the scanned source tree with `*.generated.test.ts` for TS/TSX sources and `*.generated.test.js` for JS/JSX sources
63
+ - generated tests import their shared contract runtime from `@vitronai/themis/contract-runtime` instead of writing framework helper files into the repo
51
64
  - generated tests assert normalized runtime export contracts directly in generated source
52
65
  - scenario adapters cover React components, React hooks, Next app components, Next route handlers, generic route handlers, and Node service functions when inputs can be inferred or hinted
53
66
  - React component and hook adapters also assert inferred interaction/state contracts when event handlers or zero-argument stateful methods are available
@@ -56,10 +69,10 @@ Default behavior:
56
69
  - project-level providers from `themis.generate.js` / `themis.generate.cjs` can match source files, inject shared fixture data, register runtime mocks, and wrap generated component renders for provider-aware DOM contracts and behavioral flow coverage
57
70
  - provider presets can declare router, Next navigation, auth/session, React Query, Zustand, and Redux-style wrapper metadata without hand-writing every wrapper shell
58
71
  - richer flow expectations can assert text transitions, attribute state, and role presence for empty, disabled, retry, error, and recovery paths
59
- - `.themis/generate-map.json` records source-to-generated-test mappings plus scenario metadata
60
- - `.themis/generate-last.json` stores the full machine-readable generate payload
61
- - `.themis/generate-handoff.json` stores a compact prompt-ready handoff payload for agents
62
- - `.themis/generate-backlog.json` stores unresolved skips, conflicts, and confidence debt with suggested remediation
72
+ - `.themis/generate/generate-map.json` records source-to-generated-test mappings plus scenario metadata
73
+ - `.themis/generate/generate-last.json` stores the full machine-readable generate payload
74
+ - `.themis/generate/generate-handoff.json` stores a compact prompt-ready handoff payload for agents
75
+ - `.themis/generate/generate-backlog.json` stores unresolved skips, conflicts, and confidence debt with suggested remediation
63
76
 
64
77
  ## `themis generate` options
65
78
 
@@ -165,7 +178,7 @@ Behavior:
165
178
  - adds `tests/setup.themis.js` to `setupFiles`
166
179
  - writes `themis.compat.js` as a local compatibility bridge
167
180
  - adds `test:themis` to `package.json` when missing
168
- - writes `.themis/migration-report.json` with detected compatibility imports and next actions
181
+ - writes `.themis/migration/migration-report.json` with detected compatibility imports and next actions
169
182
  - relies on built-in runtime compatibility for `@jest/globals`, `vitest`, and `@testing-library/react`
170
183
  - preserves a path to replace snapshot-heavy suites with direct assertions and generated contract tests instead of porting snapshot files as-is
171
184
 
@@ -189,9 +202,10 @@ Migration options:
189
202
  | `--update-contracts` | flag | Accept updated `captureContract(...)` baselines for the selected tests. |
190
203
  | `-w`, `--watch` | flag | Rerun the selected suite when watched project files change. |
191
204
  | `--stability <N>` | positive integer | Run selected tests `N` times and classify stability (`stable_pass`, `stable_fail`, `unstable`). |
192
- | `--html-output <path>` | string | Output path for `--reporter html` (default: `.themis/report.html`). |
205
+ | `--html-output <path>` | string | Output path for `--reporter html` (default: `.themis/reports/report.html`). |
193
206
  | `--match "<regex>"` | string | Run only tests whose full name matches regex. |
194
- | `--rerun-failed` | flag | Rerun failures from `.themis/failed-tests.json`. |
207
+ | `--rerun-failed` | flag | Rerun failures from `.themis/runs/failed-tests.json`. |
208
+ | `--fix` | flag | Apply generated-test autofixes from `.themis/runs/fix-handoff.json` and rerun the suite. |
195
209
  | `--no-memes` | flag | Disable meme intent aliases (`cook`, `yeet`, `vibecheck`, `wipe`). |
196
210
 
197
211
  Migration compatibility:
@@ -199,7 +213,7 @@ Migration compatibility:
199
213
  - imports from `@jest/globals` are supported at runtime
200
214
  - imports from `vitest` are supported at runtime
201
215
  - imports from `@testing-library/react` are supported via Themis `render`, `screen`, `fireEvent`, `waitFor`, `cleanup`, and `act`
202
- - `themis migrate <jest|vitest>` also emits `.themis/migration-report.json` with detected files and recommended next actions
216
+ - `themis migrate <jest|vitest>` also emits `.themis/migration/migration-report.json` with detected files and recommended next actions
203
217
 
204
218
  Additional option:
205
219
 
@@ -235,17 +249,21 @@ Generate-specific note:
235
249
 
236
250
  ## Artifacts
237
251
 
238
- Each run writes to `.themis/`:
239
-
240
- - `last-run.json`: full run payload (`RunResult`)
241
- - `failed-tests.json`: failed subset (`themis.failures.v1`)
242
- - `run-diff.json`: diff against the previous run
243
- - `run-history.json`: rolling recent-run history
244
- - `fix-handoff.json`: deduped repair artifact for generated test failures (`themis.fix.handoff.v1`)
245
- - `migration-report.json`: migration inventory for Jest/Vitest bridge scaffolds (`themis.migration.report.v1`)
246
- - `contract-diff.json`: contract capture drift, updates, and update commands (`themis.contract.diff.v1`)
247
- - `benchmark-last.json`: latest benchmark comparison payload plus migration proof (`themis.benchmark.result.v1`)
248
- - `migration-proof.json`: synthetic migration conversion proof emitted by `npm run benchmark` (`themis.migration.proof.v1`)
252
+ Themis writes artifacts under `.themis/`:
253
+
254
+ - `.themis/runs/last-run.json`: full run payload (`RunResult`)
255
+ - `.themis/runs/failed-tests.json`: failed subset (`themis.failures.v1`)
256
+ - `.themis/diffs/run-diff.json`: diff against the previous run
257
+ - `.themis/runs/run-history.json`: rolling recent-run history
258
+ - `.themis/runs/fix-handoff.json`: deduped repair artifact for generated test failures (`themis.fix.handoff.v1`)
259
+ - `.themis/migration/migration-report.json`: migration inventory for Jest/Vitest bridge scaffolds (`themis.migration.report.v1`)
260
+ - `.themis/diffs/contract-diff.json`: contract capture drift, updates, and update commands (`themis.contract.diff.v1`)
261
+ - `.themis/generate/generate-last.json`: latest generate payload (`themis.generate.result.v1`)
262
+ - `.themis/generate/generate-map.json`: source-to-generated-test mapping (`themis.generate.map.v1`)
263
+ - `.themis/generate/generate-handoff.json`: compact agent handoff (`themis.generate.handoff.v1`)
264
+ - `.themis/generate/generate-backlog.json`: unresolved generation debt (`themis.generate.backlog.v1`)
265
+ - `.themis/benchmarks/benchmark-last.json`: latest benchmark comparison payload plus migration proof (`themis.benchmark.result.v1`)
266
+ - `.themis/benchmarks/migration-proof.json`: synthetic migration conversion proof emitted by `npm run benchmark` (`themis.migration.proof.v1`)
249
267
 
250
268
  Formal schemas:
251
269
 
@@ -260,7 +278,7 @@ Formal schemas:
260
278
 
261
279
  Human-facing artifact:
262
280
 
263
- - `.themis/report.html`: interactive HTML verdict report
281
+ - `.themis/reports/report.html`: interactive HTML verdict report
264
282
 
265
283
  Agent payload details:
266
284
 
@@ -268,8 +286,9 @@ Agent payload details:
268
286
  - `analysis.failureClusters` groups failures by shared fingerprint
269
287
  - `analysis.stability` captures multi-run classifications and per-test status sequences
270
288
  - `analysis.comparison` reports delta stats plus new and resolved failures against the previous run
271
- - `artifacts.fixHandoff` points to `.themis/fix-handoff.json` for generated failure repair loops
289
+ - `artifacts.fixHandoff` points to `.themis/runs/fix-handoff.json` for generated failure repair loops
272
290
  - fix handoff entries include `repairStrategy`, `candidateFiles`, and `autofixCommand` for machine repair loops
291
+ - `hints.repairGenerated` points to `npx themis test --fix` for a first-party generated-suite repair command
273
292
 
274
293
  ## UI Test Utilities
275
294
 
@@ -415,15 +434,15 @@ Converts a `GenerateSummary` into the same machine-readable payload emitted by `
415
434
 
416
435
  ## `buildGenerateBacklogPayload(summary, cwd?): GenerateBacklogPayload`
417
436
 
418
- Builds the unresolved-work artifact persisted at `.themis/generate-backlog.json`.
437
+ Builds the unresolved-work artifact persisted at `.themis/generate/generate-backlog.json`.
419
438
 
420
439
  ## `buildGenerateHandoff(payload): GenerateHandoffPayload`
421
440
 
422
- Builds the compact handoff payload persisted at `.themis/generate-handoff.json`.
441
+ Builds the compact handoff payload persisted at `.themis/generate/generate-handoff.json`.
423
442
 
424
443
  ## `writeGenerateArtifacts(summary, cwd?): { payload, handoff, backlog }`
425
444
 
426
- Writes `.themis/generate-last.json`, `.themis/generate-handoff.json`, and `.themis/generate-backlog.json` for a generate run and returns all three payloads.
445
+ Writes `.themis/generate/generate-last.json`, `.themis/generate/generate-handoff.json`, and `.themis/generate/generate-backlog.json` for a generate run and returns all three payloads.
427
446
 
428
447
  ## `loadConfig(cwd): ThemisConfig`
429
448
 
package/docs/migration.md CHANGED
@@ -115,7 +115,7 @@ Jest/Vitest:
115
115
 
116
116
  Themis:
117
117
 
118
- - field-level diffs in `.themis/contract-diff.json`
118
+ - field-level diffs in `.themis/diffs/contract-diff.json`
119
119
  - visible drift/update summaries in CLI and HTML reports
120
120
  - explicit acceptance via `npx themis test --update-contracts --match "<regex>"`
121
121
 
@@ -154,7 +154,7 @@ These are the strongest head-to-head examples to use when explaining why Themis
154
154
 
155
155
  1. Snapshot replacement: `captureContract(...)` plus `--update-contracts` gives baseline capture without snapshot churn.
156
156
  2. Codemod migration: `themis migrate --convert` moves common Jest/Vitest matcher syntax toward native Themis without a manual rewrite pass.
157
- 3. Agent triage: `--agent`, `.themis/run-diff.json`, `.themis/fix-handoff.json`, and `.themis/contract-diff.json` give machines structured rerun and repair inputs.
157
+ 3. Agent triage: `--agent`, `.themis/diffs/run-diff.json`, `.themis/runs/fix-handoff.json`, and `.themis/diffs/contract-diff.json` give machines structured rerun and repair inputs.
158
158
  4. Human review: next reporter and HTML report now surface contract drift alongside failures, instead of burying meaning in raw output.
159
159
  5. Generated coverage: `themis generate src` adds source-driven contract tests next to migrated suites, so adoption improves coverage instead of merely changing runners.
160
160
 
package/docs/showcases.md CHANGED
@@ -30,7 +30,7 @@ Why it wins:
30
30
 
31
31
  - captures only the fields that matter
32
32
  - masks volatile IDs and timestamps instead of accepting noisy churn
33
- - writes machine-readable drift to `.themis/contract-diff.json`
33
+ - writes machine-readable drift to `.themis/diffs/contract-diff.json`
34
34
  - updates stay explicit with `npx themis test --update-contracts`
35
35
 
36
36
  ## 2. Migrate Jest/Vitest suites without a rewrite freeze
@@ -67,7 +67,7 @@ Why it wins:
67
67
 
68
68
  - removes common compatibility imports
69
69
  - rewrites common matcher aliases into native Themis forms
70
- - emits `.themis/migration-report.json` so agents and humans can track what changed
70
+ - emits `.themis/migration/migration-report.json` so agents and humans can track what changed
71
71
 
72
72
  ## 3. Give agents a repair loop humans can still read
73
73
 
@@ -75,10 +75,10 @@ Jest/Vitest mostly produce console text plus optional snapshots.
75
75
 
76
76
  Themis produces:
77
77
 
78
- - `.themis/failed-tests.json`
79
- - `.themis/run-diff.json`
80
- - `.themis/fix-handoff.json`
81
- - `.themis/contract-diff.json`
78
+ - `.themis/runs/failed-tests.json`
79
+ - `.themis/diffs/run-diff.json`
80
+ - `.themis/runs/fix-handoff.json`
81
+ - `.themis/diffs/contract-diff.json`
82
82
 
83
83
  Why it wins:
84
84
 
@@ -115,3 +115,38 @@ Why it wins:
115
115
  - one action accepts reviewed contracts
116
116
  - one action reruns migration codemods for the detected framework
117
117
  - humans and agents operate from the same source of truth
118
+
119
+ ## 6. Show native Themis React tests and a same-host runner comparison in CI
120
+
121
+ CI now carries a dedicated first-party Themis React showcase job:
122
+
123
+ - `Themis React Showcase`
124
+
125
+ The Themis fixture is a straight-up native Themis jsdom suite. The checked-in Jest and Vitest fixtures are still used by the comparison benchmark job, but they no longer need their own standalone workflow jobs.
126
+
127
+ Why it wins:
128
+
129
+ - the CI page still shows a first-party native Themis React example, not only migration proof
130
+ - the runner-to-runner timing comparison moves into one fair same-host artifact job instead of spreading timing across separate hosts
131
+ - humans and agents can inspect checked-in fixture sources under `tests/fixtures/showcase/`
132
+
133
+ CI also carries a same-host performance artifact job:
134
+
135
+ - `React Showcase Perf`
136
+
137
+ That job runs the same two React scenarios under Themis, Jest, and Vitest on one runner, writes `.themis/benchmarks/showcase-comparison/perf-summary.json` and `.themis/benchmarks/showcase-comparison/perf-summary.md`, and uploads them as build artifacts.
138
+
139
+ Current documented sample result from the same-host run:
140
+
141
+ - Themis median: `1110.14ms`
142
+ - Vitest median: `1871.59ms`
143
+ - Jest median: `2556.22ms`
144
+ - Themis was `68.59%` faster than Vitest
145
+ - Themis was `130.26%` faster than Jest
146
+
147
+ Why it wins:
148
+
149
+ - the timing claim comes from one comparable host instead of three separate jobs
150
+ - the uploaded artifact shows median, average, min, and max wall-clock samples for the exact same showcase specs
151
+ - the markdown summary makes it obvious whether Themis was faster or slower in that specific CI run
152
+ - the docs now state the current measured gap directly instead of making readers hunt through the artifact first
@@ -5,29 +5,29 @@ Themis includes an in-repo VS Code extension scaffold at [`packages/themis-vscod
5
5
  This is the intended shape of the editor UX:
6
6
 
7
7
  - a Themis activity-bar container
8
- - a results sidebar driven by `.themis/*` artifacts
8
+ - a results sidebar driven by `.themis/**` artifacts
9
9
  - commands to run tests, rerun failures, refresh results, and open the HTML report
10
10
  - commands to accept reviewed contract baselines and rerun migration codemods
11
11
  - failure navigation that jumps from artifact data into the source file
12
12
  - generated-review navigation for source files, generated tests, hint sidecars, and backlog items
13
- - contract-review and migration-review groups driven by `.themis/contract-diff.json` and `.themis/migration-report.json`
13
+ - contract-review and migration-review groups driven by `.themis/diffs/contract-diff.json` and `.themis/migration/migration-report.json`
14
14
 
15
15
  ## Current MVP Scope
16
16
 
17
17
  The scaffold currently supports:
18
18
 
19
- - reading `.themis/last-run.json`
20
- - reading `.themis/failed-tests.json`
21
- - reading `.themis/run-diff.json`
22
- - reading `.themis/contract-diff.json`
23
- - reading `.themis/migration-report.json`
24
- - reading `.themis/generate-last.json`
25
- - reading `.themis/generate-map.json`
26
- - reading `.themis/generate-backlog.json`
27
- - opening `.themis/report.html` in a webview
19
+ - reading `.themis/runs/last-run.json`
20
+ - reading `.themis/runs/failed-tests.json`
21
+ - reading `.themis/diffs/run-diff.json`
22
+ - reading `.themis/diffs/contract-diff.json`
23
+ - reading `.themis/migration/migration-report.json`
24
+ - reading `.themis/generate/generate-last.json`
25
+ - reading `.themis/generate/generate-map.json`
26
+ - reading `.themis/generate/generate-backlog.json`
27
+ - opening `.themis/reports/report.html` in a webview
28
28
  - surfacing generated source/test/hint mappings in the sidebar
29
29
  - surfacing unresolved generate backlog and gate state in the sidebar
30
- - refreshing when `.themis/*` files change
30
+ - refreshing when `.themis/**` files change
31
31
 
32
32
  The extension is intentionally thin. It shells out to Themis commands and treats the CLI plus artifacts as the canonical contract.
33
33
 
@@ -17,6 +17,8 @@ npx themis generate src
17
17
  npx themis test
18
18
  ```
19
19
 
20
+ `npx themis init` adds `.themis/` to `.gitignore`, and `npx themis generate src` emits `.generated.test.ts` for TS/TSX sources and `.generated.test.js` for JS/JSX sources.
21
+
20
22
  For downstream repo instructions and a copyable `AGENTS.md` template, see [`docs/agents-adoption.md`](agents-adoption.md).
21
23
 
22
24
  ## What "Next-Gen" Means Here
@@ -48,7 +50,7 @@ Themis supports structured outputs for tooling loops:
48
50
 
49
51
  - `--json` for generic automation
50
52
  - `--agent` for AI-agent workflows
51
- - `.themis/failed-tests.json` for deterministic reruns
53
+ - `.themis/runs/failed-tests.json` for deterministic reruns
52
54
 
53
55
  ## 3) Deterministic Rerun Workflow
54
56
 
@@ -86,7 +88,7 @@ Themis ships workflow features agents can use directly:
86
88
  - generated contract tests that keep baselines in readable source instead of opaque snapshot blobs
87
89
  - machine-readable artifacts that let agents inspect and explain changes before updating tests
88
90
  - mocks and spies with `fn`, `spyOn`, and `mock`
89
- - `.themis/run-diff.json` and `.themis/run-history.json`
91
+ - `.themis/diffs/run-diff.json` and `.themis/runs/run-history.json`
90
92
  - HTML verdict reports for human review
91
93
 
92
94
  ### Comparable To Snapshot Workflows, Without Snapshot Rot
@@ -130,7 +132,7 @@ Performance is measured and guarded:
130
132
 
131
133
  ## 11) Editor Surface Without Replacing The CLI
132
134
 
133
- Themis includes a thin VS Code extension scaffold that reads `.themis/*` artifacts, reruns tests, and opens the HTML report. The CLI remains the source of truth.
135
+ Themis includes a thin VS Code extension scaffold that reads `.themis/**` artifacts, reruns tests, and opens the HTML report. The CLI remains the source of truth.
134
136
 
135
137
  ## Proof Checklist
136
138
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@vitronai/themis",
3
- "version": "0.1.4",
3
+ "version": "0.1.6",
4
4
  "description": "Intent-first unit test framework and test generator for AI agents in Node.js and TypeScript",
5
5
  "license": "MIT",
6
6
  "author": "Vitron AI",
@@ -54,6 +54,10 @@
54
54
  "require": "./globals.js",
55
55
  "default": "./globals.js"
56
56
  },
57
+ "./contract-runtime": {
58
+ "require": "./src/contract-runtime.js",
59
+ "default": "./src/contract-runtime.js"
60
+ },
57
61
  "./package.json": "./package.json"
58
62
  },
59
63
  "files": [
@@ -75,14 +79,16 @@
75
79
  "themis": "bin/themis.js"
76
80
  },
77
81
  "scripts": {
82
+ "lint": "eslint .",
78
83
  "test": "node bin/themis.js test",
79
- "validate": "npm test && npm run typecheck && npm run benchmark:gate",
84
+ "validate": "npm run lint && npm test && npm run typecheck && npm run benchmark:gate",
80
85
  "typecheck": "tsc -p tsconfig.json --pretty false",
81
86
  "benchmark": "node scripts/benchmark.js",
87
+ "benchmark:showcase": "node scripts/benchmark-showcase-runners.js",
82
88
  "benchmark:gate": "node scripts/benchmark-gate.js",
83
89
  "proof:migration": "node scripts/verify-migration-fixtures.js",
84
90
  "pack:check": "npm pack --dry-run",
85
- "prepublishOnly": "npm test && npm run typecheck"
91
+ "prepublishOnly": "npm run lint && npm test && npm run typecheck"
86
92
  },
87
93
  "publishConfig": {
88
94
  "access": "public"
@@ -95,6 +101,13 @@
95
101
  "typescript": "^5.9.3"
96
102
  },
97
103
  "devDependencies": {
98
- "react": "^19.0.0"
104
+ "@testing-library/react": "^16.3.2",
105
+ "eslint": "^9.22.0",
106
+ "globals": "^17.4.0",
107
+ "jest": "^29.7.0",
108
+ "jest-environment-jsdom": "^29.7.0",
109
+ "react": "^19.0.0",
110
+ "react-dom": "^19.2.4",
111
+ "vitest": "^1.6.1"
99
112
  }
100
113
  }