lawspec 0.22.0 → 0.23.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -6,6 +6,261 @@ to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
6
6
 
7
7
  ## [Unreleased]
8
8
 
9
+ ## [0.23.0] - 2026-10-12
10
+
11
+ ### Added
12
+
13
+ - F# and C# targets (`--target fsharp`, `--target csharp`) on .NET 10, with
14
+ xunit.v3 on Microsoft.Testing.Platform and `dotnet test`. F# gets a runtime
15
+ written for it: records and unions in LawSpec's order, exact integers,
16
+ persistent collections, `Result` for typed failures, abilities as
17
+ interfaces, `Task` for async, typed actors and supervisors, single-use
18
+ session ends, mailboxes, a virtual clock, remote evaluation and the secure
19
+ network on the shared wire format, and post-quantum crypto from .NET with
20
+ BouncyCastle where .NET lacks it. Properties run on FsCheck 3.4.0, driven
21
+ from the xUnit facts: LawSpec's refinement-directed generators are FsCheck
22
+ generators of shrink trees, so shrinks stay inside refinements, and FsCheck
23
+ runs, sizes, shrinks and replays (`LAWSPEC_SEED`) the cases; generator
24
+ bindings return FsCheck Arbitraries. Models, linearizability, scenarios and
25
+ the whole harness run around it. C# runs over it: the program's model is a generated F#
26
+ project, and adapters, entries, the embedding API and tests are C#, with
27
+ `LawSpec.CSharp` as the compatibility layer. Definitions whose values all
28
+ have native types run typed, over the program's own F# types with checked
29
+ .NET integers, and each spec handler is a native class with `Result`
30
+ try-forms and its state to read or restore (typed handlers keep it native).
31
+ Native bindings, `lawspec test`, `lawspec doctor`, the native payments
32
+ example and `lawspec-dev bench kafka` cover both targets. Every acceptance
33
+ suite passes on both, with its mutants rejected, and the tutorial lessons
34
+ have F# and C# implementations. Text in a Set or a KeyVal orders by code
35
+ point on .NET as everywhere: F#'s own string order (by UTF-16 unit) is
36
+ never observed, as items, keys, entries or a comparison. See the F# and C#
37
+ setup guides.
38
+ - The collections suite checks that text keys and items order by code point,
39
+ with a character outside the Basic Multilingual Plane beside a fullwidth
40
+ letter, on every target.
41
+ - `lawspec evidence` says which tested laws ran against stand-ins. It reads
42
+ an environment attestation, a versioned JSON document any tool may write
43
+ for one run: the environment, the run's id and times, and for each
44
+ dependency its endpoint, whether it was reached, and whether it was the
45
+ real dependency or a stand-in (with the stand-in's identity, version,
46
+ declared conformance level and optional history digest). A dependency is
47
+ matched by name to an ability whose production handler a law runs under
48
+ (`unit::Name` or `unit::ability::Name`) or to an adapter or bound native
49
+ function the law calls. Each such law whose tests ran gets a line, and
50
+ `--json` the fields `environment`, `environmentRun` and `against`; an
51
+ `ENVIRONMENT` section lists the dependencies. It is reporting only: no
52
+ status changes and no law means anything different.
53
+ - `lawspec evidence --attestation <file>` and `lawspec test --attestation
54
+ <file>`, or `attestation` in `lawspec.json`, name the attestation. `lawspec
55
+ test` keeps it with the run's results in `.lawspec/results`, so a later
56
+ `lawspec evidence` shows the attested run each law last passed in; a law it
57
+ touches runs again when its dependencies are wired differently.
58
+ - The compiler API takes an `attestation` in any request: the result's
59
+ `evidence` carries the fields above and `attestation` lists each dependency
60
+ with the obligations it touched. A malformed attestation, or an unknown
61
+ format or version, is a diagnostic with code `attestation`.
62
+ - Checked definitions and stateful spec handlers run as typed native code on
63
+ all eight targets wherever every value they touch has a typed form
64
+ (Bool, Unit, the bounded integers, Integer, Text, Bytes, List, Maybe,
65
+ Either, KeyVal, Set and data built from them); anything else runs on the
66
+ dynamic values, as before. A definition that performs an operation of
67
+ another ability, or calls one that does, runs typed too: it takes the
68
+ handler context first and performs through the handler in scope, so
69
+ LawTwin's Kafka client handlers (`kafkaProducer`, `kafkaConsumer`,
70
+ `kafkaTransactions`) run typed, and 829 of the contract's 865 definitions
71
+ do (only the IntSize and UIntPtr helpers stay dynamic). Python represents
72
+ data as tuples, lists as shared cons cells that know their length and
73
+ KeyVal and Set as persistent AVL trees, and turns self tail calls into
74
+ loops; JavaScript and TypeScript use classes with generated TypeScript
75
+ types, and a number for every integer that fits a safe integer, a BigInt
76
+ only beyond; Haskell uses strict data types, `Data.Map.Strict` and
77
+ `Data.Set`, and threads a handler's state through an `MVar`. Definitions that `fail with E`, `raise` and `prelude.attempt`, field
78
+ access and record update, bounded arithmetic, recursion on integers and
79
+ the sequence helpers run typed, and so does a spec handler of several
80
+ abilities with a computed start; a typed failure is the same panic or
81
+ exception the dynamic code raises, so each catches the other's.
82
+ `lawspec-dev bench <name> --typed` lists what stays dynamic, and why. Lists are
83
+ persistent cons lists that share their tails, KeyVal and Set persistent AVL
84
+ trees in the portable order, Integer stays in a machine word until it
85
+ outgrows it, data with fields is shared by reference, and a definition's
86
+ self tail calls become a loop. A handler's state is never copied: an
87
+ operation costs what it changes. Values convert directly between the typed
88
+ forms and each target's native types at a handler's interface, and through
89
+ the dynamic value where typed and dynamic code meet.
90
+ - Embedding entry points: a stateful spec handler can be made outside any law
91
+ and called from native code, `Handler::start()` in Rust,
92
+ `Start<Handler>()` in Go, the no-argument constructor in Java and Kotlin,
93
+ `Handler()` in Python, `new Handler()` in JavaScript and TypeScript and
94
+ the handler's function over a symbol context in Haskell. Its operations apply one at a time, so it can
95
+ be shared between threads. Each operation that fails with `E` also has a
96
+ form that returns the failure as a value: `try_op` gives `Result<T, E>` in
97
+ Rust, `TryOp` gives `(T, error)` with a `*LawSpecRaised[E]` in Go, `tryOp`
98
+ an `Outcome<T, E>` in Java and Kotlin, `Right` or `Left` in Python,
99
+ JavaScript and TypeScript, and `Either E T` in Haskell. A handler's state
100
+ can be read and replaced (`state`, `set_state` and their typed forms;
101
+ `<h>Snapshot` and `<h>Restore` in Haskell), and the state's type is
102
+ public. The new guide "Embed a generated model" shows how.
103
+ - `lawspec-dev bench <name>` runs a benchmark of generated code on all eight
104
+ targets. `bench/kafka` drives LawTwin's Kafka reference model (a
105
+ copy under its MIT licence, re-expressed with failing operations, one
106
+ handler for three abilities, fields and bounded arithmetic) through its
107
+ `cluster` handler: a million produces over 100 partitions, a consumer
108
+ group of ten fetching and committing with a rebalance, and ten thousand
109
+ transactions. Every definition of the model and the handler run typed.
110
+ It runs the whole workload at 0.53 (Rust), 1.03 (Go), 0.78 (Java) and 0.93
111
+ (Kotlin) million operations a second, holding about 190 to 270 MiB for the
112
+ million records, where 0.21.0's generated Rust managed 9 operations a
113
+ second, and 1.5 GiB, at two thousand. Python runs it at 118 thousand
114
+ operations a second (it ran 1,100 on the dynamic values, at twenty
115
+ thousand), JavaScript at 942 thousand and TypeScript at 964 thousand
116
+ (7,300 before) and Haskell at 1.83 million (57 before, at two thousand).
117
+ `lawspec-dev bench kafka --deterministic` generates the contract on each
118
+ target with and without a cache, cold and warm, and by `lawspec-core`, and
119
+ fails unless every run gives the same bytes.
120
+ - Fields: `b.balance` reads a field of a single-constructor value, and
121
+ `b with balance = v, owner = o` copies it with fields replaced.
122
+ - Handlers of several abilities: `handler reference for Broker, Admin with
123
+ state s :: S start e is ... end` answers every operation of each from one
124
+ state. A law that runs it for one of its abilities runs it for the others
125
+ it uses, made once per case.
126
+ - Sets of handlers: `handlers reference is cluster, client end`. Every law of
127
+ an ability a set covers runs under all of its handlers at once,
128
+ `` Broker: law [reference] ``, and so does every law that uses one of its
129
+ abilities.
130
+ - `ability Broker with Cluster is ... end`: native handlers made with a
131
+ resource. Each case acquires one `Cluster` and makes every such native
132
+ handler with it, so native handlers of one case share it.
133
+ - Ability operations may fail: `produce :: Batch -> Int64 fails with
134
+ KafkaError`. Spec handler clauses `raise`; native handlers raise the
135
+ runtime's `Fail`. `prelude.attempt` works in definitions.
136
+ - Handler clauses: a start value may call definitions and the prelude
137
+ (`start prelude.keyValOf []`); a tail `match` or `if` may update the state
138
+ in its branches; `op by f` makes a clause of a definition that gives
139
+ `Pair result next`.
140
+ - Sequences: `++`, `prelude.slice`, `take`, `drop`, `at`, `range`, `octets`,
141
+ `bytesOf`, `utf8`, `textOf` and `join` over `Text`, `Bytes` and `List`.
142
+ - Bounded arithmetic: `prelude.wrapping`, `prelude.saturating` and
143
+ `prelude.bounded` put an integer result into the bounded type the context
144
+ wants.
145
+ - Recursion on integers: a recursive call may move an integer argument
146
+ strictly toward a bound.
147
+ - The totality checker knows what guards say about calls, fields and lengths,
148
+ that a call lies in its result type's range, that lengths are not negative,
149
+ and the bounds of `prelude.rem` and `prelude.quot`.
150
+ - A text refined by a regex, directly or through a checked definition, is
151
+ generated from texts the regex matches instead of filtered from any text,
152
+ also in laws that mix it with record or list inputs.
153
+ - On Python, a unit named like another unit's package (`shop.tax` beside
154
+ `shop.tax.rates`) is the package's `__init__.py`.
155
+ - New acceptance suites `records`, `sequences` and `composition`.
156
+ - Erlang, Elixir and Gleam run the language gaps on their dynamic path:
157
+ handlers of several abilities and sets of handlers, operations that `fail
158
+ with`, natives made with a resource, computed starts and state updated in
159
+ branches, field access and record update, the sequence, `Bytes` and `Text`
160
+ operations, bounded arithmetic, and regex-shaped generation of refined text
161
+ fields. A handler of several abilities gives one native interface per
162
+ ability over its one state; each operation that fails has a try form
163
+ (`{ok, V}` or `{error, E}`, a `Result` in Gleam), and a spec handler with
164
+ state has `<handler>_state` and `<handler>_set_state`. The `records`,
165
+ `sequences`, `composition` and `contracts` suites run on all eleven
166
+ targets, and a `beam-embedding` suite checks the embedding forms.
167
+ - Vetting on Rice's Tax's review page: `make assess` records every pending
168
+ row, `make tax` serves the page on which each comment, page, decision and
169
+ reference is judged beside the code or page it explains, and `make audit`
170
+ says what is left. `lawspec-dev tax` runs the Rice's Tax checkout named by
171
+ `RICESTAX_HOME`; see the new how-to guide, Vet the canonical material.
172
+
173
+ ### Changed
174
+
175
+ - `canon.yaml` names the canon version its vetting files were written with,
176
+ 0.2.1, which both `make canon` and Rice's Tax require. Code reshaped in
177
+ 0.20.1 for canon's Haskell grammar has its original forms back, since canon
178
+ 0.2.1 reads them.
179
+ - Definitions that are the same up to their names, such as an import's
180
+ copies, share one generated body: Rust aliases the copies' functions, and
181
+ Java and Go delegate to it. Go packages hold only the bodies they reach,
182
+ and the native entry points of typed definitions convert their values
183
+ directly. LawTwin's Kafka contract generates 23,606 lines of
184
+ `lawspec_definitions.rs` instead of 83,157, 27,439 lines of Java
185
+ definitions instead of 80,444, 27,309 of Kotlin and Java instead of 87,537,
186
+ and 37,900 lines of Go definitions over its four packages instead of
187
+ 113,462.
188
+ - Generated Rust builds its data schema once: `lawspec_schema::schema()`
189
+ returns `&'static ls::Schema`.
190
+ - `lawspec-dev ci` runs its steps in parallel: `--jobs N` at a time
191
+ (`LAWSPEC_CI_JOBS`, half the processors by default), and `--jobs 1` runs
192
+ them one at a time as before. Each step declares the steps it follows and
193
+ what it holds (its target's `.integration` environment, its acceptance
194
+ project, the repository's `.stack-work`, Stack's snapshot database, the
195
+ runtime crate's Cargo target, a timing class for real-time suites), so
196
+ parallel steps never share what one of them changes, and every step still
197
+ runs after a failure. A line is printed as each step finishes, a terminal
198
+ shows the running steps, the summary names failures in step order, and
199
+ Ctrl-C stops every running step's process group. Cached steps are skipped
200
+ exactly as before. See Contribute to LawSpec.
201
+ - The npm package's `prepack` replaces `README.md` and `CHANGELOG.md` in
202
+ `npm/` by rename, and leaves identical copies alone, so concurrent packs
203
+ never read a half-written copy.
204
+ - `package-smoke` allows half an hour for exporting the examples, and
205
+ `lawspec-dev ci` runs it alone.
206
+
207
+ ### Fixed
208
+
209
+ - The predicates npm test makes its scratch project under `.artifacts`, so a
210
+ `generate --check` running beside it in parallel CI no longer sees it.
211
+ - `tools/bootstrap-integration.mjs` installs the crypto packages the crypto
212
+ and distribution suites need (`@noble/post-quantum` from the JavaScript and
213
+ TypeScript locks, `cryptography` for Python), so a new `.integration`
214
+ passes them.
215
+
216
+ - The acceptance harness strips ANSI escape sequences before it decides
217
+ whether a mutant failed to compile, so a coloured toolchain (Gleam under
218
+ `FORCE_COLOR`) no longer hides a compile failure behind its colour codes.
219
+ - Generation is deterministic: the on-disk cache is keyed by the build of
220
+ the compiler that wrote it (its executable's path, size and time) as well
221
+ as its version, so a rebuilt compiler no longer reads another build's
222
+ entries.
223
+ - Generated names are stable under unrelated edits: the typed functions a
224
+ handler's clauses call, and its start, are named from the definitions and
225
+ the handler (with a suffix from the digest of what they name where one is
226
+ needed), not from their position among the others.
227
+ - Generated code builds without warnings, or with targeted allow attributes,
228
+ on every target, also as a dependency of another project.
229
+ - A TypeScript type with several type arguments no longer has a trailing
230
+ comma.
231
+ - A spec handler clause that fails no longer poisons the handler on Rust:
232
+ later operations see the state as the failure left it.
233
+ - A field's refinement may call a checked definition of its unit
234
+ (`label :: (v :: Text where validName v)`).
235
+ - A record update or a rebuild proves a field's regex refinement, which the
236
+ copied field already met; text tests and the sequence helpers are facts
237
+ the checker knows.
238
+ - A definition that says it `fails with E` but never raises may be a
239
+ handler clause (`op by f`).
240
+ - A spec handler named like its ability (`counter` for `Counter`) is
241
+ `CounterSpec` where the two would share a name.
242
+ - A record whose field a regex constrains has boundary witnesses that match
243
+ the regex.
244
+ - Bounded arithmetic used by an import's definitions resolves in the
245
+ importing unit, packages included.
246
+ - Checking scales with a program's size: LawTwin's import of GitHub's whole
247
+ REST description (2,108 types, 1,232 operations, 87 laws) checks in
248
+ minutes, where it did not finish in 30. Field access is typed without
249
+ building its match, registries, indexes, audits and unit elaboration are
250
+ made once per program, and sets and maps replace quadratic list scans.
251
+ `lawspec-dev bench compile` times a scaled model. Field access and record
252
+ update are Core's own projection and update, read directly by validation,
253
+ evaluation and the totality audits, instead of a match binding every field
254
+ of the record: the scaled model of 100, 200 and 400 types checks in 22.5,
255
+ 46.9 and 129.9 seconds instead of 33.8, 95.5 and 340.2 (unoptimized
256
+ builds).
257
+ - Java and Kotlin generation accepts a field named `record`, `sealed`,
258
+ `permits` or `var`, which Java restricts only as type names.
259
+ - Prelude helpers such as `prelude.isRight` work in an ability's laws and in
260
+ handler clauses.
261
+ - A numeric literal in a `match` or `if` branch, or a list, takes its type
262
+ from the other branches or items.
263
+
9
264
  ## [0.22.0] - 2026-10-10
10
265
 
11
266
  ### Added
@@ -372,7 +627,7 @@ to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
372
627
  requirements the tests verify), `canonical_decisions.yaml` (the design
373
628
  decisions, each cited as `ref:KEY`) and `canonical_exemptions.yaml` sit at
374
629
  the root, and `canonical_vetting/` holds every comment, page, decision and
375
- reference as pending with owner review deferred until after the 0.21 refactor.
630
+ reference as pending with owner review deferred until 1.0.
376
631
  - `make canon` and `lawspec-dev canon` run canon over the repository, and the
377
632
  complete check runs it as its `canon` step. Set `CANON_HOME` to a canon
378
633
  checkout built with `stack build`; without one the step is skipped with a
@@ -1280,7 +1535,8 @@ A maintenance release. The language, generated code and API are unchanged from
1280
1535
  machine-sized bindings reject an architecture mismatch. Rust adapters take
1281
1536
  owned values.
1282
1537
 
1283
- [Unreleased]: https://github.com/brain-fuel/lawspec/compare/v0.22.0...HEAD
1538
+ [Unreleased]: https://github.com/brain-fuel/lawspec/compare/v0.23.0...HEAD
1539
+ [0.23.0]: https://github.com/brain-fuel/lawspec/compare/v0.22.0...v0.23.0
1284
1540
  [0.22.0]: https://github.com/brain-fuel/lawspec/compare/v0.21.0...v0.22.0
1285
1541
  [0.21.0]: https://github.com/brain-fuel/lawspec/compare/v0.20.1...v0.21.0
1286
1542
  [0.20.1]: https://github.com/brain-fuel/lawspec/compare/v0.20.0...v0.20.1
package/README.md CHANGED
@@ -30,7 +30,7 @@ Node 22+ and the selected target's build tools are required. No Haskell
30
30
  toolchain is needed. The reference platforms are macOS and Linux.
31
31
 
32
32
  ```sh
33
- npm install --save-dev lawspec@0.22.0
33
+ npm install --save-dev lawspec@0.23.0
34
34
  npx lawspec --version
35
35
  ```
36
36
 
@@ -40,8 +40,8 @@ LawSpec can create `package.json` and its test script:
40
40
  ```sh
41
41
  mkdir lawspec-example
42
42
  cd lawspec-example
43
- npm exec --package=lawspec@0.22.0 -- lawspec init --target javascript
44
- npm install --save-dev lawspec@0.22.0
43
+ npm exec --package=lawspec@0.23.0 -- lawspec init --target javascript
44
+ npm install --save-dev lawspec@0.23.0
45
45
  npx lawspec check
46
46
  npx lawspec explain 'example.atoi_codec::itoa and then atoi yields a'
47
47
  npx lawspec doctor
package/bin/lawspec.mjs CHANGED
@@ -20,14 +20,14 @@ import {
20
20
  atomicWrite,
21
21
  safePath,
22
22
  } from "../files.mjs";
23
- const VERSION = "0.22.0";
23
+ const VERSION = "0.23.0";
24
24
  const args = process.argv.slice(2);
25
25
  const verb = args.shift();
26
26
  const options = {};
27
27
  const positional = [];
28
28
  for (let i = 0; i < args.length; i++) {
29
29
  const arg = args[i];
30
- if (["--target", "--project", "--config", "--output", "--machine-bits", "--example", "--seed", "--report"].includes(arg)) {
30
+ if (["--target", "--project", "--config", "--output", "--machine-bits", "--example", "--seed", "--report", "--attestation"].includes(arg)) {
31
31
  if (!args[i + 1] || args[i + 1].startsWith("--"))
32
32
  throw new Error(`Missing value for ${arg}`);
33
33
  options[arg.slice(2)] = args[++i];
@@ -272,19 +272,49 @@ function explainExamples(law) {
272
272
  ex.expectations.map(e => ` expect ${showAssertion(e)}`).join("\n")
273
273
  ).join("\n");
274
274
  }
275
- // What lawspec test last recorded for each law, across targets.
275
+ // What lawspec test last recorded for each law, across targets, and the
276
+ // environment attestations its runs were given, by run id. A law passed in
277
+ // an attested run names that run; across targets, its latest passing run.
276
278
  async function lastRuns() {
277
279
  const folder = path.join(configRoot, ".lawspec", "results");
278
280
  const runs = new Map();
281
+ const attestations = new Map();
279
282
  for (const name of (await readOptionalDirectory(folder)) ?? []) {
280
283
  if (!name.endsWith(".json")) continue;
281
284
  const recorded = JSON.parse((await readOptional(path.join(folder, name))) ?? "{}");
285
+ for (const [id, attestation] of Object.entries(recorded.attestations ?? {})) attestations.set(id, attestation);
282
286
  for (const [law, run] of Object.entries(recorded.laws ?? {})) {
283
287
  const known = runs.get(law) ?? {};
284
- runs.set(law, { flaky: known.flaky || run.flaky, adequacy: run.adequacy ?? known.adequacy });
288
+ const latest = (known.passed ?? "") > (run.passed ?? "") ? known : run;
289
+ runs.set(law, { flaky: known.flaky || run.flaky, adequacy: run.adequacy ?? known.adequacy,
290
+ passed: latest.passed, environment: latest.environment });
285
291
  }
286
292
  }
287
- return runs;
293
+ return { runs, attestations };
294
+ }
295
+ // An environment attestation: a versioned JSON document any tool may write,
296
+ // recording for one run where each dependency was wired and whether it was
297
+ // the real dependency or a stand-in. --attestation names one, or lawspec.json
298
+ // does; the compiler checks it. It is reporting only and changes no evidence.
299
+ async function readAttestation(config) {
300
+ const file = options.attestation !== undefined ? path.resolve(options.attestation)
301
+ : config.attestation !== undefined ? path.resolve(configRoot, config.attestation) : null;
302
+ if (file === null) return null;
303
+ const text = await readOptional(file);
304
+ if (text === null) throw new Error(`No environment attestation at ${file}`);
305
+ try {
306
+ return JSON.parse(text);
307
+ } catch (error) {
308
+ throw new Error(`${file}: the environment attestation is not JSON: ${error.message}`);
309
+ }
310
+ }
311
+ // One attested dependency, as lawspec evidence shows it.
312
+ function dependencyText(d) {
313
+ const level = (c) => /^[0-9]+$/.test(c) ? `conformance level ${c}` : `conformance ${c}`;
314
+ return `${d.dependency}: ` + (d.standIn
315
+ ? `stand-in ${d.standIn.identity} ${d.standIn.version} (${level(d.standIn.conformance)}` +
316
+ (d.standIn.historyDigest ? `, history ${d.standIn.historyDigest}` : "") + ")"
317
+ : "real") + ` at ${d.endpoint}` + (d.reached ? "" : " (not reached when recorded)");
288
318
  }
289
319
  // One law's harness, as lawspec evidence shows it.
290
320
  function harnessText(item) {
@@ -358,12 +388,18 @@ async function runTests(compiler, input, selected, roots, config) {
358
388
  const summaries = [];
359
389
  const junit = [];
360
390
  const report = options.report === undefined ? null : parseReport(options.report);
391
+ const attestation = await readAttestation(config);
361
392
  for (const [i, target] of selected.entries()) {
362
393
  const root = roots[i];
363
394
  const planned = diagnostics(await compiler.planGeneration({
364
395
  ...input, target: target.language, sourceDir: target.sourceDir, testDir: target.testDir,
365
396
  nativeBindings: target.nativeBindings, minify: options.minify === true,
397
+ ...(attestation ? { attestation } : {}),
366
398
  }));
399
+ // The laws the attestation touches, each with how its dependencies
400
+ // were wired (the run's id and times aside).
401
+ const attested = new Map((planned.evidence ?? []).filter((item) => item.against)
402
+ .map((item) => [item.declaration, JSON.stringify([item.environment, item.against])]));
367
403
  const pending = await planWrites(root, planned.files);
368
404
  if (pending.changes.length)
369
405
  throw new Error(`${target.language}: generated files are out of date; run lawspec generate${options.minify ? " --minify" : ""}`);
@@ -376,6 +412,7 @@ async function runTests(compiler, input, selected, roots, config) {
376
412
  // Recorded values are spec data: a changed recording runs its laws again.
377
413
  environment: await environmentDigest(root, report_) + (await recordedDigest(recordedFolder)),
378
414
  project: await projectDigest(root, generated),
415
+ attested,
379
416
  });
380
417
  const identity = projectKey(root);
381
418
  const resultsFile = path.join(configRoot, ".lawspec", "results", `${target.language}-${identity}.json`);
@@ -476,6 +513,8 @@ async function runTests(compiler, input, selected, roots, config) {
476
513
  const label = entry.label;
477
514
  const observed = statistics.filter((s) => s.law === label);
478
515
  laws[entry.law] = { key: keys.get(entry.law), seed: Number(seedOf.get(entry.law) ?? seed), passed: new Date().toISOString(),
516
+ // A law the attestation touches names the run it passed in.
517
+ ...(attestation && attested.has(entry.law) ? { environment: attestation.run.id } : {}),
479
518
  ...(observed.some((s) => s.outcome === "flaky") ? { flaky: true } : {}),
480
519
  ...(observed.some((s) => s.cover || s.labels || s.classes) ? { adequacy: observed.filter((s) => s.cover || s.labels || s.classes)
481
520
  .map(({ test, cases, cover, classes, labels }) => ({ test, cases, cover, classes, labels })) } : {}) };
@@ -492,13 +531,19 @@ async function runTests(compiler, input, selected, roots, config) {
492
531
  await writeFile(databaseFile, JSON.stringify(database, null, 2) + "\n");
493
532
  await mkdir(path.dirname(resultsFile), { recursive: true });
494
533
  await writeFile(path.join(path.dirname(resultsFile), ".gitignore"), "*\n");
495
- await writeFile(resultsFile, JSON.stringify({ version: 1, laws }, null, 2) + "\n");
534
+ // Each attestation a recorded law ran under is kept with the results.
535
+ const attestations = Object.fromEntries(Object.entries({ ...(previous.attestations ?? {}),
536
+ ...(attestation ? { [attestation.run.id]: attestation } : {}) })
537
+ .filter(([id]) => Object.values(laws).some((run) => run.environment === id)));
538
+ await writeFile(resultsFile, JSON.stringify({ version: 1, laws,
539
+ ...(Object.keys(attestations).length ? { attestations } : {}) }, null, 2) + "\n");
496
540
  const flaky = statistics.filter((s) => s.outcome === "flaky").map((s) => s.law);
497
541
  const unmet = statistics.flatMap((s) => (s.cover ?? []).filter((c) => !c.met).map((c) => `${s.law}: cover ${c.required}% "${c.label}" (${c.observed}%)`));
498
542
  const benchmarks = statistics.filter((s) => s.benchmark);
499
543
  // How each parallel unit's tests actually ran at the same time.
500
544
  const parallelism = statistics.filter((s) => s.parallel).map(({ parallel, mode, workers }) => ({ unit: parallel, mode, workers }));
501
545
  summaries.push({ target: target.language, root: target.root, seed: Number(seed), ran: stale.map((e) => e.law),
546
+ ...(attestation ? { environment: attestation.environment } : {}),
502
547
  unchanged: chosen.length - stale.length, ok: !failed,
503
548
  ...(chosen.length !== planned.tests.length ? { deselected: planned.tests.length - chosen.length } : {}),
504
549
  ...(unrun.length ? { unrun: unrun.map((e) => e.law) } : {}),
@@ -514,7 +559,7 @@ async function runTests(compiler, input, selected, roots, config) {
514
559
  await writeFile(path.resolve(report.path), mergeJunit(junit));
515
560
  }
516
561
  output(options.json ? summaries : summaries.map((s) =>
517
- `${s.target}${summaries.filter(other => other.target === s.target).length > 1 ? ` (${s.root})` : ""}: ${s.ran.length ? `ran ${s.ran.length} law(s) with seed ${s.seed}` : "nothing to run"}` +
562
+ `${s.target}${summaries.filter(other => other.target === s.target).length > 1 ? ` (${s.root})` : ""}: ${s.ran.length ? `ran ${s.ran.length} law(s) with seed ${s.seed}${s.environment ? ` in environment ${s.environment}` : ""}` : "nothing to run"}` +
518
563
  `, ${s.unchanged} unchanged since their last passing run.${s.ok ? "" : " FAILED"}` +
519
564
  (s.deselected ? `\n${s.deselected} law(s) not selected by --tag or --exclude-tag.` : "") +
520
565
  (s.unrun ? `\nNo tests ran for ${s.unrun.join(", ")}; the runner matched none of their tests.` : "") +
@@ -615,7 +660,7 @@ async function main() {
615
660
  }
616
661
  if (!verb || ["help", "--help", "-h"].includes(verb)) {
617
662
  output(
618
- "LawSpec " + VERSION + "\nUsage: lawspec init --target <language> [--project <directory>] [--minify]\n lawspec check | doctor | explain <unit>::<law> | generate\n lawspec evidence [<unit> | <unit>::<declaration>]\n lawspec test [--target <language>] [--fresh] [--seed <n>] [--update-recorded] [--tag <t>] [--exclude-tag <t>] [--report junit=<path>] [--coverage] [--benchmarks]\n lawspec package [--project <package directory>]\n lawspec examples [--example payments] [--target <language>] [--output <directory>]\nOptions: --config <path>, --target <language>, --machine-bits <32|64>, --json, --no-cache\nGeneration: --dry-run, --check, --minify\nTargets: " +
663
+ "LawSpec " + VERSION + "\nUsage: lawspec init --target <language> [--project <directory>] [--minify]\n lawspec check | doctor | explain <unit>::<law> | generate\n lawspec evidence [<unit> | <unit>::<declaration>] [--attestation <file>]\n lawspec test [--target <language>] [--fresh] [--seed <n>] [--attestation <file>] [--update-recorded] [--tag <t>] [--exclude-tag <t>] [--report junit=<path>] [--coverage] [--benchmarks]\n lawspec package [--project <package directory>]\n lawspec examples [--example payments] [--target <language>] [--output <directory>]\nOptions: --config <path>, --target <language>, --machine-bits <32|64>, --json, --no-cache\nGeneration: --dry-run, --check, --minify\nTargets: " +
619
664
  targets.join(", "),
620
665
  );
621
666
  return;
@@ -662,6 +707,8 @@ async function main() {
662
707
  if ((options.fresh || options.seed !== undefined || options.tag || options["exclude-tag"] ||
663
708
  options.report !== undefined || options.coverage || options["update-recorded"] || options.benchmarks) && verb !== "test")
664
709
  throw new Error("--fresh, --seed, --update-recorded, --tag, --exclude-tag, --report, --coverage and --benchmarks apply to test");
710
+ if (options.attestation !== undefined && !["evidence", "test"].includes(verb))
711
+ throw new Error("--attestation applies to evidence and test");
665
712
  if (options.seed !== undefined && !/^[0-9]+$/.test(options.seed))
666
713
  throw new Error("--seed must be a whole number");
667
714
  if (verb === "init") return init();
@@ -704,6 +751,8 @@ async function main() {
704
751
  }
705
752
  if (config.machineBits !== undefined && ![32, 64].includes(config.machineBits)) throw new Error("machineBits must be 32 or 64");
706
753
  if (config.cache !== undefined && typeof config.cache !== "boolean") throw new Error("cache must be true or false");
754
+ if (config.attestation !== undefined && typeof config.attestation !== "string")
755
+ throw new Error("attestation must be the path of an environment attestation");
707
756
  const cache = await cacheDirectory(config);
708
757
  const input = {
709
758
  ...(cache === undefined ? {} : { cacheDirectory: cache }),
@@ -752,8 +801,23 @@ async function main() {
752
801
  return;
753
802
  }
754
803
  if (verb === "evidence") {
755
- const result = diagnostics(await compiler.check(input));
804
+ // An attestation named here, or in lawspec.json, says where the tests
805
+ // ran; otherwise each law shows the attested run it last passed in.
806
+ const named = await readAttestation(config);
807
+ const result = diagnostics(await compiler.check({ ...input, ...(named ? { attestation: named } : {}) }));
756
808
  let evidence = result.evidence;
809
+ const { runs, attestations } = await lastRuns();
810
+ const environments = named ? [result.attestation] : [];
811
+ if (!named) for (const [id, attestation] of attestations) {
812
+ const laws = new Set([...runs].filter(([, run]) => run.environment === id).map(([law]) => law));
813
+ if (!laws.size) continue;
814
+ const attested = diagnostics(await compiler.check({ ...input, attestation }));
815
+ const fields = new Map(attested.evidence.filter((item) => item.against && laws.has(item.declaration))
816
+ .map(({ declaration, environment, environmentRun, against }) => [declaration, { environment, environmentRun, against }]));
817
+ evidence = evidence.map((item) => item.stage === "law" && fields.has(item.declaration) ? { ...item, ...fields.get(item.declaration) } : item);
818
+ environments.push({ ...attested.attestation, dependencies: attested.attestation.dependencies
819
+ .map((d) => ({ ...d, obligations: d.obligations.filter((law) => laws.has(law)) })) });
820
+ }
757
821
  // Native bindings are configured per target; their obligations are added.
758
822
  for (const target of selected) {
759
823
  if (target.nativeBindings === undefined) continue;
@@ -767,7 +831,6 @@ async function main() {
767
831
  if (positional[0] && !evidence.length) throw new Error("No matching obligation");
768
832
  // The last run's outcome (lawspec test): a law that failed, then passed
769
833
  // on a retry, is flaky; what its generated cases covered is its adequacy.
770
- const runs = await lastRuns();
771
834
  evidence = evidence.map((item) => {
772
835
  const run = item.stage === "law" ? runs.get(item.declaration) : undefined;
773
836
  if (!run) return item;
@@ -785,12 +848,18 @@ async function main() {
785
848
  if (!items.length) return null;
786
849
  return `${label} (${items.length})\n` + items.map((item) =>
787
850
  ` ${item.stage} ${obligationName(item)}${item.target ? ` [${item.target}]` : ""}` +
788
- (item.claim ? `: ${item.claim.text}` : "") + `\n ${item.reason}`).join("\n");
851
+ (item.claim ? `: ${item.claim.text}` : "") + `\n ${item.reason}` +
852
+ (item.against ? `\n ran in environment ${item.environment} (run ${item.environmentRun.id}) against ` +
853
+ item.against.map(dependencyText).join("; ") : "")).join("\n");
789
854
  })
790
855
  .filter((section) => section !== null),
791
856
  // How each law is discharged: the harness plane, apart from what
792
857
  // the laws say.
793
- ...(harnessed.length ? [`HARNESS (${harnessed.length})\n` + harnessed.map(harnessText).join("\n")] : [])]
858
+ ...(harnessed.length ? [`HARNESS (${harnessed.length})\n` + harnessed.map(harnessText).join("\n")] : []),
859
+ // Where the tests ran: reporting only, so no status above changes.
860
+ ...environments.map((e) => `ENVIRONMENT ${e.environment} (run ${e.run.id}, recorded ${e.run.recordedAt})\n` +
861
+ e.dependencies.map((d) => ` ${dependencyText(d)}; ` + (d.obligations.length
862
+ ? `${d.obligations.length} tested obligation(s)` : "no tested obligation relies on it")).join("\n"))]
794
863
  .join("\n\n"),
795
864
  );
796
865
  return;