statelyai 0.14.3 → 0.15.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +76 -0
- package/dist/bin.mjs +1 -1
- package/dist/{cli-D8zy3z4n.mjs → cli-CE7KhcOJ.mjs} +1527 -15
- package/dist/index.d.mts +172 -23
- package/dist/index.mjs +2 -2
- package/package.json +6 -5
package/README.md
CHANGED
|
@@ -26,6 +26,10 @@ npm install --global statelyai
|
|
|
26
26
|
|
|
27
27
|
| Goal | Command | Writes |
|
|
28
28
|
| ----------------------------------- | ---------------------------------------------------- | ------------------------------------------------------- |
|
|
29
|
+
| Calibrate reviewed behavior | `statelyai behavior calibrate --config config.json --output .scratch/calibration` | Native setup receipt awaiting inspected adoption |
|
|
30
|
+
| Run deterministic behavior evidence | `statelyai behavior check --config config.json --output .scratch/run` | Fresh local run artifact; checked baseline unchanged |
|
|
31
|
+
| Adopt inspected red or green evidence | `statelyai behavior adopt .scratch/run --config config.json` | Immutable checked state and exact config baseline |
|
|
32
|
+
| Test an application against a model | `statelyai test ./feature.campaign.mjs` | Local code execution, report and optional replay fixture |
|
|
29
33
|
| Process a saved verification report | `statelyai verify --from-report report.json` | Local report file |
|
|
30
34
|
| List local machines | `statelyai scan` | Nothing |
|
|
31
35
|
| Edit local machines visually | `statelyai open [file]` | Local source, when you save in the editor |
|
|
@@ -40,6 +44,14 @@ npm install --global statelyai
|
|
|
40
44
|
| Compare two machines | `statelyai diff <source> <target>` | Nothing |
|
|
41
45
|
| Fail on differences in CI | `statelyai diff <source> <target> --fail-on-changes` | Nothing |
|
|
42
46
|
|
|
47
|
+
## Test application behavior
|
|
48
|
+
|
|
49
|
+
<!-- trusted local campaign command from src/testCommand.ts -->
|
|
50
|
+
|
|
51
|
+
`statelyai test ./feature.campaign.mjs` runs graph paths and property testing through `@statelyai/model-testing`. Install that package in the campaign project. The campaign is trusted JavaScript with ordinary local permissions. The child process is not a sandbox.
|
|
52
|
+
|
|
53
|
+
Use `--report report.json --fixture regression.json` to save results and a concrete counterexample, then `--replay regression.json` after fixing the app. Exit codes are 0 for passed, 1 for a model/app discrepancy, and 2 for inconclusive execution. See [Test application behavior](../../docs/cli/test.md) for budgets and replay, or the [runnable browser example](../model-testing/README.md#verify-the-browser-example).
|
|
54
|
+
|
|
43
55
|
## Quick start
|
|
44
56
|
|
|
45
57
|
<!-- project initialization and sync flow from InitCommand, LinkCommand, PushCommand, PullCommand, and projectConfig.ts -->
|
|
@@ -282,6 +294,13 @@ safely and require manual migration.
|
|
|
282
294
|
|
|
283
295
|
<!-- command names, arguments, flags, defaults, and behavior from src/cli.ts#COMMANDS -->
|
|
284
296
|
|
|
297
|
+
### `behavior calibrate` / `behavior check` / `behavior adopt`
|
|
298
|
+
|
|
299
|
+
`check --config <path> --output <fresh-directory>` runs the selected tests and
|
|
300
|
+
writes an evidence artifact. `adopt <artifact-directory> --config <path>`
|
|
301
|
+
validates inspected red/green evidence and advances the exact checked baseline.
|
|
302
|
+
See [Behavior evidence in CI](#behavior-evidence-in-ci) for setup and exit codes.
|
|
303
|
+
|
|
285
304
|
### `login`
|
|
286
305
|
|
|
287
306
|
Store an OAuth credential or API key.
|
|
@@ -573,6 +592,63 @@ See [Verification reports](../../docs/cli/verify.md) for offline JSON/JUnit
|
|
|
573
592
|
processing and the programmatic executor hook. Live verification requires a host
|
|
574
593
|
adapter; no verification engine is bundled.
|
|
575
594
|
|
|
595
|
+
## Behavior evidence in CI
|
|
596
|
+
|
|
597
|
+
<!-- behavior check/adopt commands, config, statuses and artifacts from src/behaviorCi.ts -->
|
|
598
|
+
|
|
599
|
+
Connect existing Vitest tests with the repository's
|
|
600
|
+
[setup-behavior-ci skill](../../.agents/skills/setup-behavior-ci/SKILL.md).
|
|
601
|
+
The skill discovers real test names, writes a reviewed specification and model
|
|
602
|
+
estimate, selects source/test inputs, and generates a CI workflow. Users do not
|
|
603
|
+
author XState or maintain hashes. CI runs ordinary checked commands without an
|
|
604
|
+
LLM, authentication, or automatic Git writes.
|
|
605
|
+
|
|
606
|
+
```sh
|
|
607
|
+
statelyai behavior calibrate --config .behavior-ci/feature/config.json --output .scratch/calibration
|
|
608
|
+
statelyai behavior adopt .scratch/calibration --config .behavior-ci/feature/config.json
|
|
609
|
+
statelyai behavior check --config .behavior-ci/feature/config.json --output .scratch/behavior-run
|
|
610
|
+
statelyai behavior adopt .scratch/behavior-run --config .behavior-ci/feature/config.json
|
|
611
|
+
```
|
|
612
|
+
|
|
613
|
+
`check` captures Git revision/dirty bytes, selected input hashes, runner identity,
|
|
614
|
+
command argv, logs, raw native report and exact model/record references. It copies
|
|
615
|
+
the checked public FileStore into the fresh output directory and evolves only
|
|
616
|
+
that copy. First setup uses `baseline: null`; stage generated inputs before
|
|
617
|
+
checking, inspect output, then explicitly adopt and commit the checked state.
|
|
618
|
+
|
|
619
|
+
Exit codes describe **this run**. `0` means all retained active checks passed;
|
|
620
|
+
`1` means a mapped test failure, skipped or missing evidence; `2` means invalid
|
|
621
|
+
configuration, process, provenance, reporter or artifact evidence. Missing or
|
|
622
|
+
duplicate mappings, unknown statuses, unmapped selected cases in schema 2, changed source,
|
|
623
|
+
and exit/report disagreement cannot pass. Setup maps every selected native
|
|
624
|
+
case, including existing tests. A native test failure can be an assertion, runtime exception or timeout; inspect its detail
|
|
625
|
+
before diagnosing the cause. Historical failures remain readable
|
|
626
|
+
and do not make every future run fail. Omitting a requirement/check retains it;
|
|
627
|
+
removal requires an explicit retirement reason.
|
|
628
|
+
|
|
629
|
+
`summary.md` shows current verdict, Git/model/spec/implementation bindings,
|
|
630
|
+
changed and retained obligations, failures and historical core gaps. `run.json`,
|
|
631
|
+
`manifest.json`, `process.json`, raw report/logs and `store/.statelyai` preserve
|
|
632
|
+
the evidence. Producers may write optional diagnostics under `$STATELY_BEHAVIOR_OUTPUT/traces`; the child receives a unique absolute output path. Trace files remain in red/green artifacts and are checksum-validated on adoption. Invalid setup still writes a minimal error artifact when the
|
|
633
|
+
output directory can be safely allocated. Existing/populated output is refused.
|
|
634
|
+
|
|
635
|
+
`adopt` accepts inspected red and green artifacts against their exact checked
|
|
636
|
+
parent. It validates checksums, record/model digests, selected Git input bytes
|
|
637
|
+
(or unchanged local dirty bytes), path safety and immutable version conflicts,
|
|
638
|
+
then updates the config baseline after copying files. Repeating the same
|
|
639
|
+
adoption is idempotent. Divergent parents and error artifacts refuse adoption.
|
|
640
|
+
For schema 2 it verifies native normalization with the current trusted repository adapter. It never executes artifact code. Download and authenticate remote run artifacts
|
|
641
|
+
before adoption; checksums are corruption detection, not runner authentication.
|
|
642
|
+
|
|
643
|
+
Schema 2 adds a cited discovery inventory for seven input categories and a checked native calibration receipt. Checked authoritative intent files are contract-bound, so requirement-document edits invalidate calibration even before the structured spec changes. Initial calibration executes against the unchanged app; unresolved mismatches block adoption. A cited implementation-defect disposition can retain a known red baseline without changing its verdict. Changed model/spec/plan/mappings, adapter, test producers or runner identity require inspected recalibration. Ordinary app implementation changes retain the calibrated contract. Receipt snapshots preserve original revision evidence. Selected-check gaps remain visible; generated paths and executed cases are separate evidence. Schema 1 remains compatible and reports setup assurance unavailable.
|
|
644
|
+
|
|
645
|
+
Config and the single module-adapter protocol are documented in the
|
|
646
|
+
[setup contract](../../.agents/skills/setup-behavior-ci/references/contract.md).
|
|
647
|
+
Source scope is bounded to 200 files including checked state. It is a selected
|
|
648
|
+
dependency scope, not complete transitive coverage. Core execution remains
|
|
649
|
+
reported; review identity is unavailable. CI requires clean bound inputs and
|
|
650
|
+
actual HEAD equal to `GITHUB_SHA`; environment metadata alone is not remote proof.
|
|
651
|
+
|
|
576
652
|
## Self-hosting
|
|
577
653
|
|
|
578
654
|
<!-- self-host auth and URL behavior from LoginCommand, baseUrlFlags, OpenCommand, credential resolution, and projectConfig.ts -->
|
package/dist/bin.mjs
CHANGED