@vitronai/themis 0.1.0-beta.4 → 0.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +11 -1
- package/README.md +42 -3
- package/docs/api.md +21 -2
- package/docs/migration.md +167 -0
- package/docs/publish.md +2 -0
- package/docs/release-checklist.md +28 -0
- package/docs/release-policy.md +11 -8
- package/docs/roadmap.md +18 -0
- package/docs/schemas/agent-result.v1.json +8 -2
- package/docs/schemas/contract-diff.v1.json +120 -0
- package/docs/showcases.md +117 -0
- package/docs/vscode-extension.md +13 -0
- package/docs/why-themis.md +34 -3
- package/globals.d.ts +1 -0
- package/index.d.ts +57 -0
- package/package.json +1 -1
- package/src/artifacts.js +63 -2
- package/src/assets/themisVerdictEngine.png +0 -0
- package/src/cli.js +16 -3
- package/src/contracts.js +330 -0
- package/src/migrate.js +104 -4
- package/src/reporter.js +79 -3
- package/src/runner.js +2 -0
- package/src/runtime.js +32 -3
- package/src/test-utils.js +13 -0
- package/src/worker.js +1 -0
package/CHANGELOG.md
CHANGED
|
@@ -4,10 +4,18 @@ All notable changes to this project are documented in this file.
|
|
|
4
4
|
|
|
5
5
|
## Unreleased
|
|
6
6
|
|
|
7
|
+
## 0.1.1 - 2026-03-23
|
|
8
|
+
|
|
9
|
+
### Changed
|
|
10
|
+
|
|
11
|
+
- Updated the primary Themis brand mark in the README/package surface to use `src/assets/themisVerdictEngine.png` while keeping the existing report imagery for HTML-report references and report-specific documentation.
|
|
12
|
+
|
|
13
|
+
## 0.1.0 - 2026-03-23
|
|
14
|
+
|
|
7
15
|
### Changed
|
|
8
16
|
|
|
9
17
|
- Added a first-party `themis generate` / `themis scan` code-scan flow that generates deterministic contract tests for exported modules, React components, hooks, Next app/router files, route handlers, and Node services; emits richer machine-readable `--json` results for agents; writes `.themis/generate-map.json`, `.themis/generate-last.json`, `.themis/generate-handoff.json`, `.themis/generate-backlog.json`, and `.themis/fix-handoff.json`; supports project-level `themis.generate.js` providers, interaction/state adapters for React components and hooks, automatic `--write-hints` scaffolding, targeted regeneration, planning mode, scenario/confidence steering, review/update/clean workflows, strict quality gates, and backlog remediation commands; and fails generated tests with direct regeneration guidance when scanned sources drift.
|
|
10
|
-
-
|
|
18
|
+
- Promoted Themis to the first stable `0.1.0` release and defined the stable CLI, artifact, and JS/TS package surface as the public contract baseline.
|
|
11
19
|
- Tightened npm/package positioning around Themis as an intent-first unit test framework for AI agents in Node.js and TypeScript.
|
|
12
20
|
- Updated publish-facing docs to consistently frame Themis as an AI verdict engine for human and agent review loops.
|
|
13
21
|
- Added `npm run pack:check` for npm publish payload validation.
|
|
@@ -30,6 +38,8 @@ All notable changes to this project are documented in this file.
|
|
|
30
38
|
- Added config-level `testIgnore` discovery patterns so repos can keep generated output, fixture sandboxes, and other local test noise out of default suite runs deterministically.
|
|
31
39
|
- Added an in-repo VS Code extension scaffold for artifact-driven result viewing, reruns, and HTML report opening.
|
|
32
40
|
- Expanded the VS Code extension scaffold with generated-review navigation for source/test/hint mappings and unresolved generation backlog.
|
|
41
|
+
- Expanded the VS Code extension scaffold with contract review and migration review actions backed by `.themis/contract-diff.json` and `.themis/migration-report.json`.
|
|
42
|
+
- Added contract-capture options (`normalize`, `maskPaths`, `sortArrays`), richer contract diff summaries, migration codemods, showcase docs, release checklists, and CI proof artifacts for benchmark and migration comparisons.
|
|
33
43
|
- Refreshed README, AGENTS, and supporting docs to match the current package scope, JS/TS feature set, artifact contracts, and extension surface.
|
|
34
44
|
|
|
35
45
|
## 0.1.0-alpha.1 - 2026-02-13
|
package/README.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# Themis
|
|
2
2
|
|
|
3
3
|
<p align="center">
|
|
4
|
-
<img src="src/assets/
|
|
4
|
+
<img src="src/assets/themisVerdictEngine.png" alt="Themis verdict engine mark" width="42" valign="middle">
|
|
5
5
|
<a href="https://github.com/vitron-ai/themis/actions/workflows/ci.yml">
|
|
6
6
|
<img src="https://img.shields.io/github/actions/workflow/status/vitron-ai/themis/ci.yml?branch=main&style=for-the-badge&label=THEMIS%20VERDICT%20PIPELINE&labelColor=111827&color=16a34a" alt="Themis verdict pipeline status" valign="middle">
|
|
7
7
|
</a>
|
|
@@ -32,6 +32,7 @@ It is built to be the best test loop for agent workflows: deterministic reruns,
|
|
|
32
32
|
- [Publish Readiness](#publish-readiness)
|
|
33
33
|
- [Why Themis](docs/why-themis.md)
|
|
34
34
|
- [API Reference](docs/api.md)
|
|
35
|
+
- [Showcase Comparisons](docs/showcases.md)
|
|
35
36
|
- [Release Policy](docs/release-policy.md)
|
|
36
37
|
- [Publish Guide](docs/publish.md)
|
|
37
38
|
- [Contributing](CONTRIBUTING.md)
|
|
@@ -42,6 +43,8 @@ It is built to be the best test loop for agent workflows: deterministic reruns,
|
|
|
42
43
|
- Best-in-class unit testing for AI agents in Node.js and TypeScript
|
|
43
44
|
- Deterministic execution with fast rerun loops
|
|
44
45
|
- Agent-native JSON and HTML reporting
|
|
46
|
+
- Structured contract workflows instead of opaque snapshot files
|
|
47
|
+
- Incremental migration path from Jest/Vitest without rewriting everything on day one
|
|
45
48
|
- AI verdict engine for human triage and machine automation
|
|
46
49
|
|
|
47
50
|
## Modern JS/TS Support
|
|
@@ -60,7 +63,7 @@ Themis is built for modern Node.js and TypeScript projects:
|
|
|
60
63
|
|
|
61
64
|
## Visuals
|
|
62
65
|
|
|
63
|
-
- Real Themis brand mark: [`src/assets/
|
|
66
|
+
- Real Themis brand mark: [`src/assets/themisVerdictEngine.png`](src/assets/themisVerdictEngine.png)
|
|
64
67
|
- HTML verdict report art: [`src/assets/themisReport.png`](src/assets/themisReport.png)
|
|
65
68
|
- Background art used by the report: [`src/assets/themisBg.png`](src/assets/themisBg.png)
|
|
66
69
|
|
|
@@ -121,6 +124,8 @@ Generated files land under `tests/generated` by default. Each generated test:
|
|
|
121
124
|
|
|
122
125
|
Themis also supports per-file generation hints with sidecars like `src/components/Button.themis.json` so humans and agents can provide props, component flows, args, route requests, and route context. When those sidecars do not exist yet, `--write-hints` can scaffold them automatically from the current source analysis.
|
|
123
126
|
|
|
127
|
+
This is the core alternative to snapshot-driven testing: generated and hand-written tests assert normalized contracts in readable source, so diffs stay reviewable and updates stay intentional.
|
|
128
|
+
|
|
124
129
|
For repo-wide generation defaults, add `themis.generate.js` or `themis.generate.cjs` at the project root. Providers in that file can match source paths, supply shared props/args/flow plans, register runtime mocks for generated UI scenarios, and wrap generated component renders so generated DOM contracts run inside the same provider shells humans use in app tests. Providers can also declare preset wrapper metadata for router, Next navigation, auth/session shells, React Query, Zustand, and Redux-style app state patterns, including route history/state, query status, auth permissions, and store selector/action metadata.
|
|
125
130
|
|
|
126
131
|
For CI and agent loops, Themis can also enforce generation quality instead of only writing files. Strict runs emit a structured backlog, fail on unresolved scan debt, and hand back exact remediation commands.
|
|
@@ -169,9 +174,17 @@ Migration scaffolds also write:
|
|
|
169
174
|
|
|
170
175
|
See [`docs/why-themis.md`](docs/why-themis.md) for positioning, differentiators, and community messaging.
|
|
171
176
|
|
|
177
|
+
Short version:
|
|
178
|
+
|
|
179
|
+
- Themis aims to deliver the benefits people reach for in snapshots, without snapshot rot.
|
|
180
|
+
- Prefer explicit, normalized contracts over broad output dumps.
|
|
181
|
+
- Keep changes reviewable through source assertions, machine-readable artifacts, and diff-oriented rerun workflows.
|
|
182
|
+
- See [`docs/showcases.md`](docs/showcases.md) for direct Jest/Vitest comparison examples.
|
|
183
|
+
|
|
172
184
|
## Reference Docs
|
|
173
185
|
|
|
174
186
|
- API reference: [`docs/api.md`](docs/api.md)
|
|
187
|
+
- Migration guide: [`docs/migration.md`](docs/migration.md)
|
|
175
188
|
- Release policy: [`docs/release-policy.md`](docs/release-policy.md)
|
|
176
189
|
- Publish guide: [`docs/publish.md`](docs/publish.md)
|
|
177
190
|
- VS Code extension notes: [`docs/vscode-extension.md`](docs/vscode-extension.md)
|
|
@@ -182,6 +195,7 @@ See [`docs/why-themis.md`](docs/why-themis.md) for positioning, differentiators,
|
|
|
182
195
|
- Generate backlog schema: [`docs/schemas/generate-backlog.v1.json`](docs/schemas/generate-backlog.v1.json)
|
|
183
196
|
- Fix handoff schema: [`docs/schemas/fix-handoff.v1.json`](docs/schemas/fix-handoff.v1.json)
|
|
184
197
|
- Failures artifact schema: [`docs/schemas/failures.v1.json`](docs/schemas/failures.v1.json)
|
|
198
|
+
- Contract diff schema: [`docs/schemas/contract-diff.v1.json`](docs/schemas/contract-diff.v1.json)
|
|
185
199
|
- Changelog: [`CHANGELOG.md`](CHANGELOG.md)
|
|
186
200
|
- Contributing: [`CONTRIBUTING.md`](CONTRIBUTING.md)
|
|
187
201
|
- Security: [`SECURITY.md`](SECURITY.md)
|
|
@@ -202,6 +216,7 @@ See [`docs/why-themis.md`](docs/why-themis.md) for positioning, differentiators,
|
|
|
202
216
|
- `npx themis generate app --scenario next-route-handler`: focuses generation on Next app router request handlers.
|
|
203
217
|
- `npx themis migrate jest`: scaffolds a Themis config/setup bridge for existing Jest suites.
|
|
204
218
|
- `npx themis migrate jest --rewrite-imports`: rewrites matched Jest/Vitest/Testing Library imports to a local `themis.compat.js` bridge file.
|
|
219
|
+
- `npx themis migrate jest --convert`: applies codemods for common Jest/Vitest matcher/import patterns so suites move closer to native Themis style.
|
|
205
220
|
- `npx themis migrate vitest`: scaffolds the same bridge for Vitest suites.
|
|
206
221
|
- `npx themis generate src --require-confidence high`: enforces a quality bar for all selected generated tests.
|
|
207
222
|
- `npx themis generate src --files src/routes/ping.ts`: targets one or more explicit source files.
|
|
@@ -223,6 +238,7 @@ See [`docs/why-themis.md`](docs/why-themis.md) for positioning, differentiators,
|
|
|
223
238
|
- `npx themis test --stability 3`: runs the suite three times and classifies each test as `stable_pass`, `stable_fail`, or `unstable`.
|
|
224
239
|
- `npx themis test --match "intent DSL"`: runs only tests whose full name matches regex.
|
|
225
240
|
- `npx themis test --rerun-failed`: reruns failing tests from `.themis/failed-tests.json`.
|
|
241
|
+
- `npx themis test --update-contracts --match "suite > case"`: accepts reviewed `captureContract(...)` changes for a narrow slice of the suite.
|
|
226
242
|
- `npx themis test --no-memes`: disables meme phase aliases (`cook`, `yeet`, `vibecheck`, `wipe`).
|
|
227
243
|
- `npx themis test --lexicon classic|themis`: rebrands human-readable status labels in `next/spec` reporters.
|
|
228
244
|
- `npm run validate`: runs test, typecheck, and benchmark gate in one command.
|
|
@@ -230,6 +246,12 @@ See [`docs/why-themis.md`](docs/why-themis.md) for positioning, differentiators,
|
|
|
230
246
|
- `npm run benchmark:gate`: fails when benchmark performance exceeds the configured threshold.
|
|
231
247
|
- `npm run pack:check`: previews the npm publish payload.
|
|
232
248
|
|
|
249
|
+
## CI & Release Proof
|
|
250
|
+
|
|
251
|
+
- Compatibility job runs `npm test` on Node 18 and 20.
|
|
252
|
+
- Release surface job runs `npm run typecheck`, `npm run pack:check`, the HTML + agent reports, verifies `.themis/contract-diff.json`, produces `.themis/benchmark-last.json`/`.themis/migration-proof.json`, and uploads all of the artifacts for later inspection.
|
|
253
|
+
- Perf gate job runs `npm run benchmark:gate` with `BENCH_MAX_AVG_MS=2500` to guard against regressions before publishing.
|
|
254
|
+
|
|
233
255
|
## Agent Guide
|
|
234
256
|
|
|
235
257
|
See [`AGENTS.md`](AGENTS.md) for the AI-agent test authoring contract used in this repository.
|
|
@@ -242,7 +264,10 @@ Each run writes artifacts to `.themis/`:
|
|
|
242
264
|
- `run-history.json`: rolling recent-run history for agent comparison loops.
|
|
243
265
|
- `fix-handoff.json`: source-oriented repair handoff for generated test failures.
|
|
244
266
|
- `migration-report.json`: compatibility inventory and next actions for migrated Jest/Vitest suites.
|
|
267
|
+
- `contract-diff.json`: contract capture drift, updates, and update commands for `captureContract(...)` workflows.
|
|
245
268
|
- `themis.compat.js`: optional local compat bridge for rewritten migration imports.
|
|
269
|
+
- `benchmark-last.json`: latest benchmark comparison payload, including migration proof output.
|
|
270
|
+
- `migration-proof.json`: synthetic migration-conversion proof artifact emitted by `npm run benchmark`.
|
|
246
271
|
- `report.html`: interactive HTML verdict report.
|
|
247
272
|
|
|
248
273
|
`--agent` output includes deterministic failure fingerprints, grouped `analysis.failureClusters`, stability classifications, previous-run comparison data, and a direct pointer to `.themis/fix-handoff.json` so AI agents can jump from generated failures to exact regeneration commands. Fix handoff entries also carry repair strategies, candidate files, and autofix commands for tighter failure-to-fix loops.
|
|
@@ -283,7 +308,21 @@ test('captures a stable UI contract', () => {
|
|
|
283
308
|
});
|
|
284
309
|
```
|
|
285
310
|
|
|
286
|
-
|
|
311
|
+
Contract capture for reviewable baselines:
|
|
312
|
+
|
|
313
|
+
```js
|
|
314
|
+
test('captures a stable response contract', () => {
|
|
315
|
+
const payload = {
|
|
316
|
+
status: 'ok',
|
|
317
|
+
flags: ['fast', 'deterministic']
|
|
318
|
+
};
|
|
319
|
+
|
|
320
|
+
captureContract('status payload', payload);
|
|
321
|
+
expect(payload.status).toBe('ok');
|
|
322
|
+
});
|
|
323
|
+
```
|
|
324
|
+
|
|
325
|
+
Themis intentionally avoids first-party snapshot-file workflows. Prefer direct assertions, generated contract tests, and explicit flow expectations over large opaque snapshots. The goal is comparable baseline coverage with better reviewability: normalized contracts, focused assertions, machine-readable artifacts, and intentional updates instead of broad snapshot re-acceptance.
|
|
287
326
|
|
|
288
327
|
Available globals:
|
|
289
328
|
|
package/docs/api.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Themis API Reference
|
|
2
2
|
|
|
3
|
-
This document defines the public API surface for Themis `0.1.0
|
|
3
|
+
This document defines the public API surface for Themis `0.1.0`.
|
|
4
4
|
|
|
5
5
|
## CLI
|
|
6
6
|
|
|
@@ -24,6 +24,8 @@ Creates:
|
|
|
24
24
|
|
|
25
25
|
Scans a source directory and writes generated Themis unit-layer tests.
|
|
26
26
|
|
|
27
|
+
Themis uses generation and explicit assertions as a contract-first alternative to snapshot-heavy workflows. The goal is comparable baseline coverage with more reviewable diffs, more intentional updates, and stronger machine-readable artifacts for agents.
|
|
28
|
+
|
|
27
29
|
Default behavior:
|
|
28
30
|
|
|
29
31
|
- input directory: `src`
|
|
@@ -138,6 +140,8 @@ Project-level provider modules are supported via `themis.generate.js` or `themis
|
|
|
138
140
|
|
|
139
141
|
Scaffolds an incremental migration bridge for existing Jest or Vitest suites.
|
|
140
142
|
|
|
143
|
+
This command is designed for incremental adoption, not forced rewrites. Teams can keep existing suites running under Themis first, then move touched tests toward native Themis contracts and `intent(...)` flows over time.
|
|
144
|
+
|
|
141
145
|
Behavior:
|
|
142
146
|
|
|
143
147
|
- writes or updates `themis.config.json`
|
|
@@ -146,10 +150,12 @@ Behavior:
|
|
|
146
150
|
- adds `test:themis` to `package.json` when missing
|
|
147
151
|
- writes `.themis/migration-report.json` with detected compatibility imports and next actions
|
|
148
152
|
- relies on built-in runtime compatibility for `@jest/globals`, `vitest`, and `@testing-library/react`
|
|
153
|
+
- preserves a path to replace snapshot-heavy suites with direct assertions and generated contract tests instead of porting snapshot files as-is
|
|
149
154
|
|
|
150
155
|
Migration options:
|
|
151
156
|
|
|
152
157
|
- `--rewrite-imports`: rewrites matched imports from `@jest/globals`, `vitest`, and `@testing-library/react` to the local `themis.compat.js` bridge
|
|
158
|
+
- `--convert`: removes common framework imports and rewrites common Jest/Vitest matcher patterns (`it`, `toStrictEqual`, `toContainEqual`, `toBeCalled*`) into Themis-native forms
|
|
153
159
|
|
|
154
160
|
## `themis test` options
|
|
155
161
|
|
|
@@ -163,6 +169,7 @@ Migration options:
|
|
|
163
169
|
| `--environment node\|jsdom` | string | Override the configured test environment. |
|
|
164
170
|
| `--isolation worker\|in-process` | string | Select worker isolation or a zero-IPC in-process execution mode. |
|
|
165
171
|
| `--cache` | flag | Enable file-level result caching for in-process local loops. |
|
|
172
|
+
| `--update-contracts` | flag | Accept updated `captureContract(...)` baselines for the selected tests. |
|
|
166
173
|
| `-w`, `--watch` | flag | Rerun the selected suite when watched project files change. |
|
|
167
174
|
| `--stability <N>` | positive integer | Run selected tests `N` times and classify stability (`stable_pass`, `stable_fail`, `unstable`). |
|
|
168
175
|
| `--html-output <path>` | string | Output path for `--reporter html` (default: `.themis/report.html`). |
|
|
@@ -185,10 +192,18 @@ Execution note:
|
|
|
185
192
|
|
|
186
193
|
- `--watch --isolation in-process --cache` is the fastest local rerun mode
|
|
187
194
|
- `--isolation worker` remains the safer mode for CI and global-heavy suites
|
|
195
|
+
- `--watch` is intended for short edit-run-review loops for both humans and AI agents
|
|
188
196
|
|
|
189
197
|
Snapshot note:
|
|
190
198
|
|
|
191
|
-
- Themis no longer supports first-party snapshot files or `-u` update flows.
|
|
199
|
+
- Themis no longer supports first-party snapshot files or `-u` update flows.
|
|
200
|
+
- Prefer direct assertions, generated contract tests, and explicit flow expectations.
|
|
201
|
+
- The intended replacement is comparable outcome with better reviewability: normalized contracts, readable source assertions, diff-oriented artifacts, and intentional regeneration.
|
|
202
|
+
- `captureContract(name, value)` writes a normalized baseline under `.themis/contracts/`, fails on drift by default, and pairs with `--update-contracts` for explicit acceptance.
|
|
203
|
+
- `captureContract(name, value, options)` also supports:
|
|
204
|
+
- `normalize(value)`: rewrite volatile payloads before persistence
|
|
205
|
+
- `maskPaths: string[]`: replace selected JSON-style paths such as `$.requestId`
|
|
206
|
+
- `sortArrays: true`: sort normalized array values for order-insensitive contracts
|
|
192
207
|
|
|
193
208
|
## Exit behavior
|
|
194
209
|
|
|
@@ -211,6 +226,9 @@ Each run writes to `.themis/`:
|
|
|
211
226
|
- `run-history.json`: rolling recent-run history
|
|
212
227
|
- `fix-handoff.json`: deduped repair artifact for generated test failures (`themis.fix.handoff.v1`)
|
|
213
228
|
- `migration-report.json`: migration inventory for Jest/Vitest bridge scaffolds (`themis.migration.report.v1`)
|
|
229
|
+
- `contract-diff.json`: contract capture drift, updates, and update commands (`themis.contract.diff.v1`)
|
|
230
|
+
- `benchmark-last.json`: latest benchmark comparison payload plus migration proof (`themis.benchmark.result.v1`)
|
|
231
|
+
- `migration-proof.json`: synthetic migration conversion proof emitted by `npm run benchmark` (`themis.migration.proof.v1`)
|
|
214
232
|
|
|
215
233
|
Formal schemas:
|
|
216
234
|
|
|
@@ -221,6 +239,7 @@ Formal schemas:
|
|
|
221
239
|
- `docs/schemas/generate-backlog.v1.json`
|
|
222
240
|
- `docs/schemas/fix-handoff.v1.json`
|
|
223
241
|
- `docs/schemas/failures.v1.json`
|
|
242
|
+
- `docs/schemas/contract-diff.v1.json`
|
|
224
243
|
|
|
225
244
|
Human-facing artifact:
|
|
226
245
|
|
|
@@ -0,0 +1,167 @@
|
|
|
1
|
+
# Migrating From Jest And Vitest
|
|
2
|
+
|
|
3
|
+
Themis is designed for incremental migration. Start by running existing suites under the Themis runtime, then convert touched tests toward native contracts and `intent(...)` flows as you work.
|
|
4
|
+
|
|
5
|
+
## Fast path
|
|
6
|
+
|
|
7
|
+
```bash
|
|
8
|
+
npx themis migrate jest
|
|
9
|
+
npx themis migrate jest --rewrite-imports
|
|
10
|
+
npx themis migrate jest --convert
|
|
11
|
+
npx themis test
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
Use `vitest` instead of `jest` for Vitest suites.
|
|
15
|
+
|
|
16
|
+
## Migration modes
|
|
17
|
+
|
|
18
|
+
- `themis migrate <jest|vitest>`: scaffold config, setup, compat bridge, and migration report.
|
|
19
|
+
- `--rewrite-imports`: point framework imports at `themis.compat.js`.
|
|
20
|
+
- `--convert`: remove common Jest/Vitest imports and rewrite common matcher/test patterns into Themis-native forms.
|
|
21
|
+
|
|
22
|
+
## Before And After
|
|
23
|
+
|
|
24
|
+
### 1. Runtime bridge first
|
|
25
|
+
|
|
26
|
+
Jest/Vitest:
|
|
27
|
+
|
|
28
|
+
```js
|
|
29
|
+
import { describe, test, expect } from '@jest/globals';
|
|
30
|
+
|
|
31
|
+
describe('math', () => {
|
|
32
|
+
test('adds', () => {
|
|
33
|
+
expect(1 + 1).toBe(2);
|
|
34
|
+
});
|
|
35
|
+
});
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
Themis after `migrate`:
|
|
39
|
+
|
|
40
|
+
```js
|
|
41
|
+
describe('math', () => {
|
|
42
|
+
test('adds', () => {
|
|
43
|
+
expect(1 + 1).toBe(2);
|
|
44
|
+
});
|
|
45
|
+
});
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
The suite keeps running, but Themis owns the runner, artifacts, rerun loop, and agent payloads.
|
|
49
|
+
|
|
50
|
+
### 2. Matcher codemods
|
|
51
|
+
|
|
52
|
+
Jest/Vitest:
|
|
53
|
+
|
|
54
|
+
```js
|
|
55
|
+
import { it, expect } from '@jest/globals';
|
|
56
|
+
|
|
57
|
+
it('tracks calls', () => {
|
|
58
|
+
const worker = fn();
|
|
59
|
+
worker('ok');
|
|
60
|
+
|
|
61
|
+
expect({ status: 'ok' }).toStrictEqual({ status: 'ok' });
|
|
62
|
+
expect([1, 2]).toContainEqual(2);
|
|
63
|
+
expect(worker).toBeCalledTimes(1);
|
|
64
|
+
});
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
Themis after `migrate --convert`:
|
|
68
|
+
|
|
69
|
+
```js
|
|
70
|
+
test('tracks calls', () => {
|
|
71
|
+
const worker = fn();
|
|
72
|
+
worker('ok');
|
|
73
|
+
|
|
74
|
+
expect({ status: 'ok' }).toEqual({ status: 'ok' });
|
|
75
|
+
expect([1, 2]).toContain(2);
|
|
76
|
+
expect(worker).toHaveBeenCalledTimes(1);
|
|
77
|
+
});
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
### 3. Snapshot replacement
|
|
81
|
+
|
|
82
|
+
Jest/Vitest snapshot flow:
|
|
83
|
+
|
|
84
|
+
```js
|
|
85
|
+
test('renders summary', () => {
|
|
86
|
+
const view = renderSummary();
|
|
87
|
+
expect(view).toMatchSnapshot();
|
|
88
|
+
});
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
Themis contract flow:
|
|
92
|
+
|
|
93
|
+
```js
|
|
94
|
+
test('renders summary', () => {
|
|
95
|
+
const view = renderSummary();
|
|
96
|
+
|
|
97
|
+
captureContract('summary view', {
|
|
98
|
+
heading: view.heading,
|
|
99
|
+
counts: view.counts,
|
|
100
|
+
flags: view.flags
|
|
101
|
+
});
|
|
102
|
+
|
|
103
|
+
expect(view.heading).toBe('Summary');
|
|
104
|
+
});
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
This keeps the baseline machine-readable and reviewable instead of hiding it in an opaque blob.
|
|
108
|
+
|
|
109
|
+
### 4. Better drift review
|
|
110
|
+
|
|
111
|
+
Jest/Vitest:
|
|
112
|
+
|
|
113
|
+
- large snapshot diffs
|
|
114
|
+
- broad “accept all changes” update flow
|
|
115
|
+
|
|
116
|
+
Themis:
|
|
117
|
+
|
|
118
|
+
- field-level diffs in `.themis/contract-diff.json`
|
|
119
|
+
- visible drift/update summaries in CLI and HTML reports
|
|
120
|
+
- explicit acceptance via `npx themis test --update-contracts --match "<regex>"`
|
|
121
|
+
|
|
122
|
+
### 5. Behavior-first UI coverage
|
|
123
|
+
|
|
124
|
+
Jest/Vitest often converges on DOM dumps or shallow expectations:
|
|
125
|
+
|
|
126
|
+
```js
|
|
127
|
+
test('button snapshot', () => {
|
|
128
|
+
const { container } = render(<Button />);
|
|
129
|
+
expect(container).toMatchSnapshot();
|
|
130
|
+
});
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
Themis pushes behavior and contract intent:
|
|
134
|
+
|
|
135
|
+
```js
|
|
136
|
+
test('button interaction contract', async () => {
|
|
137
|
+
render(<Button />);
|
|
138
|
+
|
|
139
|
+
fireEvent.click(screen.getByRole('button', { name: 'Save' }));
|
|
140
|
+
|
|
141
|
+
captureContract('button state', {
|
|
142
|
+
text: screen.getByRole('button', { name: 'Saved' }).textContent
|
|
143
|
+
});
|
|
144
|
+
|
|
145
|
+
await waitFor(() => {
|
|
146
|
+
expect(screen.getByText('Saved')).toBeInTheDocument();
|
|
147
|
+
});
|
|
148
|
+
});
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
## Showcase Examples
|
|
152
|
+
|
|
153
|
+
These are the strongest head-to-head examples to use when explaining why Themis is better for AI-agent loops:
|
|
154
|
+
|
|
155
|
+
1. Snapshot replacement: `captureContract(...)` plus `--update-contracts` gives baseline capture without snapshot churn.
|
|
156
|
+
2. Codemod migration: `themis migrate --convert` moves common Jest/Vitest matcher syntax toward native Themis without a manual rewrite pass.
|
|
157
|
+
3. Agent triage: `--agent`, `.themis/run-diff.json`, `.themis/fix-handoff.json`, and `.themis/contract-diff.json` give machines structured rerun and repair inputs.
|
|
158
|
+
4. Human review: next reporter and HTML report now surface contract drift alongside failures, instead of burying meaning in raw output.
|
|
159
|
+
5. Generated coverage: `themis generate src` adds source-driven contract tests next to migrated suites, so adoption improves coverage instead of merely changing runners.
|
|
160
|
+
|
|
161
|
+
## Recommended rollout
|
|
162
|
+
|
|
163
|
+
1. Run `themis migrate <jest|vitest>`.
|
|
164
|
+
2. Add `--rewrite-imports` if you want local explicit compat imports.
|
|
165
|
+
3. Add `--convert` to normalize the easy matcher/import cases immediately.
|
|
166
|
+
4. Replace snapshots with `captureContract(...)` or explicit assertions in files you touch.
|
|
167
|
+
5. Use `themis generate src` to add source-driven coverage in parallel with migrated suites.
|
package/docs/publish.md
CHANGED
|
@@ -16,6 +16,8 @@ This guide covers publishing Themis to npm.
|
|
|
16
16
|
3. Confirm the public messaging still matches the project:
|
|
17
17
|
- best-in-class unit test framework for AI agents in Node.js and TypeScript
|
|
18
18
|
- AI verdict engine for human and agent review loops
|
|
19
|
+
- contract-first alternative to snapshot-heavy test maintenance
|
|
20
|
+
- incremental migration path from Jest/Vitest with fast watch-mode rerun loops
|
|
19
21
|
4. Run:
|
|
20
22
|
|
|
21
23
|
```bash
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
# Release Checklist
|
|
2
|
+
|
|
3
|
+
This checklist tracks the release work for Themis `0.1.0` and future stable milestone cuts.
|
|
4
|
+
|
|
5
|
+
## Documentation polish
|
|
6
|
+
|
|
7
|
+
- confirm README, API reference, and VS Code docs clearly describe the stable story
|
|
8
|
+
- publish [`docs/showcases.md`](docs/showcases.md) with migration/comparison examples
|
|
9
|
+
- surface release expectations in [`docs/release-policy.md`](docs/release-policy.md)
|
|
10
|
+
- link to this checklist from the release policy and README so the team knows the remaining steps
|
|
11
|
+
|
|
12
|
+
## Gates & proof
|
|
13
|
+
|
|
14
|
+
- `npm test`
|
|
15
|
+
- `npm run typecheck`
|
|
16
|
+
- `npm run benchmark`
|
|
17
|
+
- `npm run benchmark:gate`
|
|
18
|
+
- verify `.themis/benchmark-last.json` and `.themis/migration-proof.json` are up to date
|
|
19
|
+
|
|
20
|
+
## Release actions
|
|
21
|
+
|
|
22
|
+
- confirm the release version (current target `0.1.0`)
|
|
23
|
+
- bump `package.json` and `packages/themis-vscode/package.json`
|
|
24
|
+
- update `docs/api.md` version header
|
|
25
|
+
- run `npm pack` / `npm publish --dry-run` if publishing
|
|
26
|
+
- tag `v0.1.0` and push the tag
|
|
27
|
+
- update the changelog/release notes for the stable release
|
|
28
|
+
- announce `0.1.0` with the improved story and benchmarks
|
package/docs/release-policy.md
CHANGED
|
@@ -4,17 +4,17 @@ This document defines release expectations for Themis.
|
|
|
4
4
|
|
|
5
5
|
## Current Stage
|
|
6
6
|
|
|
7
|
-
Current line: `0.1.
|
|
7
|
+
Current line: `0.1.x`
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
Stable means:
|
|
10
10
|
|
|
11
|
-
- the core CLI, artifact, and JS/TS package shape
|
|
12
|
-
- compatibility remains a deliberate goal across
|
|
13
|
-
- breaking changes
|
|
11
|
+
- the core CLI, artifact, and JS/TS package shape are the public contract
|
|
12
|
+
- compatibility remains a deliberate goal across stable patch releases
|
|
13
|
+
- breaking changes require intentional version bumps and migration notes
|
|
14
14
|
|
|
15
15
|
## Versioning Model
|
|
16
16
|
|
|
17
|
-
Themis follows semantic versioning
|
|
17
|
+
Themis follows semantic versioning:
|
|
18
18
|
|
|
19
19
|
- `0.1.0-alpha.N`: unstable early pre-release
|
|
20
20
|
- `0.1.0-beta.N`: pre-release with mostly stable product shape
|
|
@@ -22,7 +22,7 @@ Themis follows semantic versioning semantics with pre-release tags:
|
|
|
22
22
|
- `0.1.x`: backward-compatible fixes and minor enhancements
|
|
23
23
|
- `0.2.0+`: intentional breaking changes with migration notes
|
|
24
24
|
|
|
25
|
-
## Compatibility Rules During
|
|
25
|
+
## Compatibility Rules During Stable
|
|
26
26
|
|
|
27
27
|
- CLI flags should remain stable unless a clear product issue requires change.
|
|
28
28
|
- Programmatic API should prefer additive changes over renames or removals.
|
|
@@ -32,7 +32,7 @@ Themis follows semantic versioning semantics with pre-release tags:
|
|
|
32
32
|
## Deprecation Policy
|
|
33
33
|
|
|
34
34
|
- New deprecations are documented in `CHANGELOG.md`.
|
|
35
|
-
- Deprecated APIs should keep at least one
|
|
35
|
+
- Deprecated APIs should keep at least one stable release cycle before removal when practical.
|
|
36
36
|
- Meme aliases can be disabled any time with `--no-memes`; strict environments should use this flag.
|
|
37
37
|
|
|
38
38
|
## Quality Gates
|
|
@@ -41,8 +41,11 @@ Before publishing:
|
|
|
41
41
|
|
|
42
42
|
- `npm test`
|
|
43
43
|
- `npm run typecheck`
|
|
44
|
+
- `npm run benchmark`
|
|
44
45
|
- `npm run benchmark:gate`
|
|
45
46
|
|
|
47
|
+
Release prep steps for major milestones are tracked in [`docs/release-checklist.md`](docs/release-checklist.md).
|
|
48
|
+
|
|
46
49
|
## Publishing Policy
|
|
47
50
|
|
|
48
51
|
- Publish target: npm public package `@vitronai/themis`.
|
package/docs/roadmap.md
ADDED
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
## Next-phase focus
|
|
2
|
+
|
|
3
|
+
1. **Provider/flow depth**
|
|
4
|
+
- Expand `generate` detection to infer React Query usage, router contexts, and persisted store slices so generated tests wrap components/hooks with accurate provider shells.
|
|
5
|
+
- Surface async DOM flows in `tests/generated/*` with multi-stage user journeys, empty/loading/error states, and configurable timing fixtures so the runner can validate realistic UI transitions instead of static renders.
|
|
6
|
+
- Add documentation (and optionally VS Code actions) that show how to hook a project-level `themis.generate.js` or `.themis.json` provider configuration for shared auth/session/React Query clients.
|
|
7
|
+
|
|
8
|
+
2. **Migration helpers**
|
|
9
|
+
- Improve `themis migrate` to rewrite Jest/Vitest imports to the generated compatibility module, create prompt-ready diff artifacts, and log a migration report.
|
|
10
|
+
- Build a VS Code pane or CLI summary showing both the original Jest test and the new generated Themis contract, highlighting the migration delta in code and behavior.
|
|
11
|
+
- Provide a recipe in `docs/migration.md` for teams to adopt Themis incrementally, including a helper to wrap Jest tests inside Themis-generated asserts.
|
|
12
|
+
- Add a native contract-capture workflow that gives teams snapshot-comparable baseline coverage without reviving snapshot-file maintenance.
|
|
13
|
+
|
|
14
|
+
3. **Proof/polish**
|
|
15
|
+
- Document how to run the generator in "plan" mode for React/Next flows, share the artifact URLs, and point to the new `tests/fixtures/react-app` coverage as proof that CI-enforced flows now run cleanly under the real dependency graph.
|
|
16
|
+
- Consider adding benchmark entries showing the generator/migration loop time compared to Jest baseline.
|
|
17
|
+
|
|
18
|
+
These steps keep widening the gap versus Jest/Vitest and ensure our agent-and-human workflow stays irresistible.
|
|
@@ -25,7 +25,7 @@
|
|
|
25
25
|
"artifacts": {
|
|
26
26
|
"type": "object",
|
|
27
27
|
"additionalProperties": false,
|
|
28
|
-
"required": ["lastRun", "failedTests", "runDiff", "runHistory", "fixHandoff"],
|
|
28
|
+
"required": ["lastRun", "failedTests", "runDiff", "runHistory", "fixHandoff", "contractDiff"],
|
|
29
29
|
"properties": {
|
|
30
30
|
"lastRun": {
|
|
31
31
|
"type": "string"
|
|
@@ -41,6 +41,9 @@
|
|
|
41
41
|
},
|
|
42
42
|
"fixHandoff": {
|
|
43
43
|
"type": "string"
|
|
44
|
+
},
|
|
45
|
+
"contractDiff": {
|
|
46
|
+
"type": "string"
|
|
44
47
|
}
|
|
45
48
|
}
|
|
46
49
|
},
|
|
@@ -50,7 +53,7 @@
|
|
|
50
53
|
"hints": {
|
|
51
54
|
"type": "object",
|
|
52
55
|
"additionalProperties": false,
|
|
53
|
-
"required": ["rerunFailed", "targetedRerun", "diffLastRun", "repairGenerated"],
|
|
56
|
+
"required": ["rerunFailed", "targetedRerun", "diffLastRun", "repairGenerated", "reviewContracts"],
|
|
54
57
|
"properties": {
|
|
55
58
|
"rerunFailed": {
|
|
56
59
|
"type": "string"
|
|
@@ -63,6 +66,9 @@
|
|
|
63
66
|
},
|
|
64
67
|
"repairGenerated": {
|
|
65
68
|
"type": "string"
|
|
69
|
+
},
|
|
70
|
+
"reviewContracts": {
|
|
71
|
+
"type": "string"
|
|
66
72
|
}
|
|
67
73
|
}
|
|
68
74
|
}
|