gdp-wired 0.2.0 → 0.3.0

package/README.md

@@ -2,12 +2,13 @@
 
 # gdp-wired
 
-Laws about stateful rules (budgets, caps, rate limits, access policies), proven in TypeScript by a small trusted
-kernel, with an untrusted prover that writes the proofs, an optional Lean twin that checks the kernel itself, and GPU
-searches that find counterexamples or the invariants a proof needs.
+An automatic prover for stateful rules: budgets, caps, rate limits, workflows, access policies. Write the rule and the
+laws it must keep; gdp-wired proves them with no proof to write, shows a real run that breaks them when they're false
+(searched on the GPU), checks every proof again in Lean, and ships the rule as JavaScript.
 
-You write the rule once with typed builders and state what must always hold. The library proves it by induction (or
-shows a real run that breaks it), and the same rule runs as an interpreter or ships as generated JavaScript.
+Proofs are checked by a small trusted kernel; the prover that writes them is untrusted, and the kernel's Lean twin is
+proven sound. On the three rules in [`compare/`](compare/README.md), the proof is a one- or two-line call and checks in
+25–44 ms, against 74–117 lines of hand proof and 137–161 ms in Bend, and 8–30 lines and 488–571 ms in Lean.
 
 ```sh
 npm install gdp-wired   # or bun add gdp-wired; plain JavaScript with types, Node 20+, Bun or a browser
@@ -33,34 +34,54 @@ Forget the cap (`s.plus(c)`) and the kernel refuses the proof; a search from the
 and `examples/domains/` holds 92 rules from nine domains (finance, access control, public services, operations,
 protocols, commerce, supply chains, games, education), each checked three ways.
 
-## Scope: where it replaces Bend, and where it doesn't
+## Scope: what it proves by itself, and where the wall is
 
-gdp-wired is meant to replace Bend where it matters in practice, not to be a language: laws about **stateful rules whose
-state is counters, totals, maps of counters and logs summed over time**, proven automatically, with the rule shipped as
-code. If the checker side finds new frontiers the scope may grow; until then:
+gdp-wired is an automatic prover, not a language: laws about **stateful rules** (counters, totals, maps, sets and lists
+of names, logs summed over time, steps in a row, runs that go on forever), proven without a proof to write, with the
+rule shipped as code. With Lean beside it, it says what Bend says; where a proof needs more than its kernel decides, the
+proof is Lean's, as a fact about a function the rule reads:
 
 | It expresses and proves | It does not (today) |
 | --- | --- |
-| Step machines over naturals: scalars, maps from naturals to naturals (one write per map per step), named step conditions, preconditions on params, assumptions per request | Lists, sets or strings as values: membership, subsets, equality of lists of names |
-| Invariants of the state and params, laws about every key of a map; step laws relating one step's before and after (`s'`, `M'[k]`: "only increases", "stays retired"), and k-step laws about k steps in a row ("two grants on one day add two", up to 16 steps) | Orderings over any number of steps ("a refund never follows a release", at any distance, needs a state flag), beyond progress |
-| Composition: rules side by side (`product`), taking turns (`interleave`), and one rule refining another (`refinement`: a forward simulation, the spec allowing every request it is sent), each built as a plain machine so every law above applies | Refinement where the spec needs more than one step per step of the implementation |
-| Laws about one key of a map at a time, or about scalars | Totals across a map's keys ("the sum of every client's holdings equals in-use") |
-| Histories: sums and queries over a log of requests, relations between them proven one record at a time, pruning | Unbounded recursion over data, quantifiers over sequences other than a machine's own history |
-| Decisions over facts: flags, one-of choices, optional facts, bounded naturals (`match` covers every case) | Datatypes beyond those encodings; division, modulo, reals |
-| Loops that halt by a rank or a tuple of ranks; folds over binary trees; progress laws ("some request makes this happen") | Arbitrary algorithms: there is no general recursion to prove things about |
-| Arithmetic: `+`, truncated `−`, `×` by a constant, `min`, `max`, `if`, products of two terms (proven by the kernel, less well by the prover) | Generated code for decisions alone (`compileJs` emits machines; a predicate emitter is planned) |
-
-Keep Bend where the specification is about **lists or sets of names** (resource scopes, capability subsets, normalization
-that must refine its input), where a proof needs **induction over data structures**, or where the shipped code must be
-the compiled proved source of an arbitrary algorithm. Use gdp-wired where the rule is a budget, a cap, a window, a
-workflow or a policy decision: it proves those without proof files, finds counterexample runs when they're false, and
-checks its own kernel in Lean.
+| Step machines over naturals: scalars, maps from naturals to naturals (a write at one key, or at every key, per map per step), named step conditions, preconditions on params, assumptions per request | Strings, and lists as values (a list in a map's cell, lists of lists) |
+| Lists of names (`.lists(...)`): positions and a length, `push`, `pop`, `set`, and `insertAt`/`removeAt` that keep the order (positions shift: a write at every key reading its key at k ∸ 1 or k + 1, which the runtimes decide on breakpoints); laws about every two positions or more (`forall((i, j) => ...)`: "strictly increasing", so no name twice; `forall((i, j, k) => ...)`: "any three cells repeat a value"), and step laws that an insert puts its name in, keeps every name, and a removal takes it out (`examples/lists.ts`, a scope kept in normal form) | |
+| Sets of names, as maps of 0/1 cells over names interned to numbers: a request carrying whole sets (`.carries`), conditions about every key (`everyKey`, `has`, `subsetOf`, `disjointFrom`, `sameAs`) anywhere in a step (each named as a flag of its own) and inside laws (pulled out as the law's key), the key read alone or plus or less a term and compared (`k < n`, `M[k + 1]`), writes at every key in one step (`each`); laws, step laws and k-step laws over them, each also a Lean theorem; a set's size over a universe of names (`within(n)`, `sizeBelow(n)`: the sum of its cells, exact for names interned below n; `examples/sizes.ts`) | A set's size over unbounded names (Lean's maps have no finite support, so no sum over every key) |
+| Some key (`someKey`): "something was added, so some name is in", proven through a witness, a term naming the key, that the prover finds among the state, the params, the law's key and the request (or the one the law names) | A witness no term of the state names (which a ghost can keep: the last key added, say) |
+| Functions from outside the rule (`fn`, and `divBy`, `modBy` for division and remainder by a constant): read like maps, known to the kernel only by their facts, which every goal may assume; the Lean export states the facts as the hypothesis they are, and proves them in Lean where the function has a Lean definition (`divBy` and `modBy` by `omega`); the GPU search and the GPU invariant synthesis tabulate each function over the keys they can read it at | Facts proven by gdp-wired (they are Lean's to prove) |
+| Invariants of the state and params, laws about every key of a map; step laws relating one step's before and after (`s'`, `M'[k]`: "only increases", "stays retired"), and k-step laws about k steps in a row ("two grants on one day add two", up to 16 steps); orderings at any distance through ghost state ("a refund never follows a release": `ghost` with `ever`, `count`); liveness under fairness (`leadsTo`: "every ticket issued is served", in every run that sends each helpful request infinitely often, by a rank that drops: one natural or several compared in order; several helpful requests, each in the region where it helps; a request naming some inputs and leaving the rest, and the sets it carries, free), for machines with maps, with sets, and with a history (its queries free, within its relations); a Lean theorem over runs, and under weak fairness too (each helpful request taken whenever it stays allowed, proven allowed in its region: its free inputs at any value, or at a witness the exporter finds among 0, the state and the params, a history's query among them; the sets it carries empty) | |
+| Composition: rules side by side (`product`), taking turns (`interleave`), and one rule refining another (`refinement`: a forward simulation, the spec allowing every request it is sent, steps the spec doesn't see as `stutter`, and `steps(spec, k)` where the spec takes several steps for one of the implementation's: a transfer at once refines a withdrawal then a deposit), each built as a plain machine so every law above applies; what a refinement means is a Lean theorem (`refines`: every reachable state of the implementation beside a reachable state of the spec, related by every law), for machines with maps and flags and for machines with sets (`srefines`: the sets a request carries shared by name, writes at every key, flags at every key) | Refinement where the number of spec steps depends on the state (beyond a bound k with skips) |
+| Laws about one key of a map at a time, or about scalars; totals across a map's keys through a ghost total ("in use equals what the clients hold") | Sums across a map's keys other than through a ghost total |
+| Histories: sums and queries over a log of requests, relations between them proven one record at a time, pruning | Quantifiers over sequences other than a machine's own history |
+| Decisions over facts: flags, one-of choices, optional facts, bounded naturals (`match` covers every case) | Datatypes beyond those encodings (except as a function's arguments: a list as a number a Lean function decodes); reals |
+| Loops that halt by a rank or a tuple of ranks; folds over binary trees; progress laws ("some request makes this happen"); recursion defined in Lean, entered as a function with its facts | Induction the prover would have to invent (the wall below) |
+| Arithmetic: `+`, truncated `−`, `×` by a constant, `min`, `max`, `if`, products of two terms (proven by the kernel, less well by the prover), division and remainder by a constant (`divBy`, `modBy`) | Generated code for decisions alone (`compileJs` emits machines; a predicate emitter is planned) |
+
+**The wall.** What stays outside is one of two things, and neither is a gap the next construct closes:
+
+- **Induction the prover would have to invent.** A property of a recursive structure you define (a tree, every prefix
+  of a list of lists) whose proof needs a lemma nobody stated. No automatic prover proves all of these: arithmetic with
+  induction is undecidable. Here the proof is written in Lean, and gdp-wired uses it: define the function in Lean,
+  prove its facts there, and give the rule `fn(arity, js, {facts, lean})`. The steps, the state, the maps and the laws
+  around it stay automatic, and the Lean export takes the facts as the hypothesis they are.
+- **An arbitrary algorithm shipped as its proved source.** gdp-wired ships the rule as JavaScript and runs a function's
+  own JavaScript beside its Lean definition (held to it by tests, not by a compiler). Bend compiles the proved source
+  itself.
+
+The engineering list is closed. Each other "does not" above has a route already: a ghost (a witness no term names, a
+sum across a map's keys), a bound with skips (spec steps that vary), an encoding (strings, datatypes as numbers a Lean
+function decodes), or Lean itself (a function you define and prove there). Use gdp-wired where the rule is a budget, a cap, a
+window, a workflow, a policy decision, an authority over sets of names or a list kept in normal form
+(`examples/sets.ts`, `examples/lists.ts`): it proves those without proof files, finds counterexample runs when they're
+false, and checks its own kernel in Lean. Keep Bend (or Lean) where the proof is the induction above.
 
 For a sense of the split: in one real codebase with 98 Bend laws and about 1,600 lines of hand proofs, the policy
 decisions, the compare-and-swap storage plans and the state/provider plans fit gdp-wired as it is (with names interned
 to numbers), and gdp-wired can prove more about storage than its Bend proofs state (no revision reuse across keys); the
-lifecycle machine's before/after laws are step laws (as in `release lifecycle` in `examples/`); the resource and authority checks, built on lists of names as
-sets, stay in Bend. `compare/` puts three rules side by side in gdp-wired, Bend and plain Lean, with line counts and
+lifecycle machine's before/after laws are step laws (as in `release lifecycle` in `examples/`); the authority checks,
+built on lists of names used as sets, fit as sets of names (all ten of its authority laws proven automatically, and the
+rule agrees with the compiled Bend on all 7,654 outcomes over four names); the resource checks, which normalize scopes
+that are lists of paths (lists of lists of names) and compare their prefixes, are induction over nested lists: the wall,
+so they stay in Bend, or go to Lean as functions with proven facts. `compare/` puts three rules side by side in gdp-wired, Bend and plain Lean, with line counts and
 check times.
 
 ## What's trusted
@@ -69,8 +90,8 @@ check times.
 | --- | --- | --- |
 | `kernel/kernel.ts`, `core/check.ts` | **Yes** | Turns a machine and its laws into goals and checks a proof tree for each: splits on linear atoms, a Fourier–Motzkin derivation at each leaf. Every rule (step machines with maps and flags, step laws and k-step laws, loops with ranks, progress, folds over trees) is a few moves of one small algebra of runs: conditions holding where a run is, a step taken from there, a conclusion read at the end or across the run. Histories become plain machines first. Refuses names a machine doesn't declare |
 | `kernel/prove.ts` | No | Grows the trees. If it's wrong, the kernel rejects the tree |
-| `lean-kernel/` | Optional | The kernel's twin in Lean 4.34.0, each rule proven sound (`Kern.sound`, `machine_sound`, `mmachine_sound`, `fmmachine_sound`, `loop_sound`, `fold_sound`, `progress_sound`, `steplaw_sound`, `kstep_sound`, `hmachine_sound`), axioms `propext`, `Classical.choice`, `Quot.sound` only. The exporter sends the kernel's real trees; tampered ones compute to false |
-| `search/`, `synth/`, `hw/` | No | GPU work: counterexample search over every short history, and invariant/rank synthesis. Every hit is replayed by the interpreter or proven by the kernel before it counts |
+| `lean-kernel/` | Optional | The kernel's twin in Lean 4.34.0, each rule proven sound (`Kern.sound`, `machine_sound`, `mmachine_sound`, `fmmachine_sound`, `loop_sound`, `fold_sound`, `progress_sound`, `steplaw_sound`, `kstep_sound`, `hmachine_sound`, `smachine_sound` with functions' facts and laws about several keys, `skstep_sound`, `leads_sound`, `leads_wf_sound` and `sleads_sound` for liveness over runs), what a ghost means (`ghost_laws`, `ghost_run`, `ever_meaning`) and what a refinement means (`refines`, `srefines`), axioms `propext`, `Classical.choice`, `Quot.sound` only. The exporter sends the kernel's real trees; tampered ones compute to false |
+| `search/`, `synth/`, `hw/` | No | GPU work: counterexample search over every short history (for laws, step laws and k-step laws, machines with a log of requests too), and invariant/rank synthesis. Every hit is replayed by the interpreter or proven by the kernel before it counts |
 | `core/dsl.ts`, `core/compose.ts`, the runtimes | No | The builders and combinators compile to the kernel's plain values; the interpreter and generated JavaScript are held to the reference evaluator by the conformance test, and the combinators to runs of their parts |
 
 How the untrusted parts are held to the trusted ones, automatically:
@@ -83,6 +104,9 @@ How the untrusted parts are held to the trusted ones, automatically:
   evaluator, and the same cases through Lean's own `evalE` (`lean-kernel/conformance.ts`).
 - **Examples** (`examples/`): every law proven by the kernel and re-checked by Lean, every rule run at random through
   both runtimes, and every broken variant broken by a real run (a refused proof alone could be the prover's weakness).
+- **One description, three readings**: a law or step law is proven by the kernel, read on one run by `core/watch.ts` (the
+  examples' runs, every replay), and searched for a breaking run by the Handwired search; `hw/check-steps.ts` holds the
+  search to the watcher on whole spaces, and all three to each other on random machines and laws.
 - **Tampering and mutation** (`kernel/tamper.test.ts`, `kernel/refusals.test.ts`, `kernel/mutants.ts`): real proofs
   broken structurally or offered for weakened goals that have a model must be rejected, every refusal the kernel makes is
   exercised, and every one-operator mutation of `kernel.ts` and `check.ts` is run against the tests to find what they
@@ -138,11 +162,30 @@ const failsClosed = trial.decide('a missing fact means no trial', ({spent}) => s
   machine reads.
 - **Arithmetic** over the naturals: `plus`, `minus` (truncated), `times(k)`, `mul` (a product of two terms), `min`,
   `max`, `when(cond, a, b)`, `count(cond)`; conditions `lt`, `atMost`, `eq`, `and`, `or`, `not`, `implies`.
+- **Sets and quantifiers**: `.carries(...)` for sets a request brings, `everyKey(k => ...)` (and `has`, `subsetOf`,
+  `disjointFrom`, `sameAs`) anywhere in a step or inside a law, `M.each(k => ...)` to write every key, and
+  `someKey(k => ..., witness?)` for "some key where", proven through a witness the prover finds or the one given.
+- **Functions** (`.fns({perMinute: divBy(60)})`, or `fn(arity, js, {facts, lean})` for your own): read like params that
+  take arguments, `perMinute(t)`; the kernel knows each only by its facts (`.facts(...)`, and a library function's
+  own), and the Lean export takes them as the hypothesis they are, proving them where the function has a Lean
+  definition. A function defined and proven in Lean is how a rule reaches past what the kernel decides: `examples/functions.ts`.
+- **Liveness** (`rule.leadsTo(name, {from, to, rank, helpful})`, `proveLeadsTo`): whenever `from` holds, `to` holds later in
+  every run that takes each helpful request infinitely often; proven by a rank no step raises while waiting and each
+  helpful request lowers in its region. `rank` may return several terms (compared in order) and `helpful` may be a list
+  of `{when, request}`; a request names only the inputs it fixes (`examples/liveness.ts`: batches, a history with a
+  query, a machine with sets).
 - **Loops** (`loop(...)`): state, `while`, `body`, a `rank` (or a tuple of ranks) that must drop each turn.
-- **Composition** (`gdp-wired/compose`): `product(a, b)`, `interleave(a, b)` and `refinement(impl, spec, {relation, witness})`
+- **Lean** (`gdp-wired/lean`): `toLean([{id, machine, laws, stepLaws}, ...])` writes the proofs as a Lean file, and
+  `leanProject(file)` a Lake project with the twin, so `lake build` checks them. Each law comes as the twin's theorem and
+  as a plain statement over named variables, e.g. `theorem budget.law0 (h : FMReach budget_machine ρ μ) : ρ «s» ≤
+  (ρ «b» + ρ «m»)`, for a Lean development to cite.
+- **Composition** (`gdp-wired/compose`): `product(a, b)`, `interleave(a, b)`, `refinement(impl, spec, {relation, witness, stutter})`,
+  `steps(spec, k)` (k spec steps as one, each skippable) and `ghost(m, {name: ever(...) | count(...) | total(m, map, name)})`
   build plain machines from machines (each side's state named `a_x`, `b_x`). A refinement's laws are the relation, proven
   by induction, and its step law says the spec allows the request `witness` sends it at each of the implementation's:
-  `examples/composition.ts` proves a meter that counts down refines one that counts up, and refuses two broken ones.
+  `examples/composition.ts` proves a meter that counts down refines one that counts up, and refuses two broken ones;
+  a two-phase meter refines it with stuttering; an escrow never refunds after a release (at any distance); and the
+  connection pool's in-use count is the total of every client's holdings.
 - **Running it**: `rule.interpreter()` is persistent (searches branch from any state); `rule.compiled()` is what an app
   ships: maps and the log updated in place, so a step costs the same however long the history. Both refuse params and
   inputs that aren't naturals below 2^53 and any value past 2^53 − 1, where the proofs over the naturals stop describing
@@ -194,7 +237,7 @@ its Lean twin in 1.35 s. What each proves, what each ships at runtime, and where
 | `next/` | Guards synthesized from laws, relational laws (migrations, tenant isolation, through `core/compose.ts`), a ReBAC audit |
 | `sat/` | Bit-vector laws: bit-blasted, solved, LRAT-checked in TypeScript, optionally by Lean |
 | `enum/`, `perf/` | The enumeration, and the benchmarks |
-| `examples/` | The builders at work: `domains/` (92 rules across nine domains, each with laws and broken variants), `harness.ts` (the triple check), `readme.ts`, `budget.ts`, `composition.ts` |
+| `examples/` | The builders at work: `domains/` (92 rules across nine domains, each with laws and broken variants), `harness.ts` (the triple check), `readme.ts`, `budget.ts`, `composition.ts`, `sets.ts`, `quantifiers.ts`, `functions.ts`, `lists.ts`, `sizes.ts`, `liveness.ts` |
 | `compare/` | The same three rules in gdp-wired, Bend and plain Lean, checked and measured |
 | `spec.ts`, `gdp.ts`, `circuit.ts`, `laws.ts` | The audit path: gdp-ts predicates written once, as trusted provers per request and as a Handwired circuit that decides every (user, project) pair at once |
 | `modal/` | Everything above as Modal jobs: `suite.json` is the regression suite |
@@ -204,7 +247,7 @@ its Lean twin in 1.35 s. What each proves, what each ships at runtime, and where
 ## Running
 
 The npm package is the library: the builders (`gdp-wired`), the runtimes (`gdp-wired/machine`), composition
-(`gdp-wired/compose`), the kernel
+(`gdp-wired/compose`), the Lean export (`gdp-wired/lean`), the kernel
 (`gdp-wired/kernel`), the prover (`gdp-wired/prove`), the derivation checker and solver (`gdp-wired/check`,
 `gdp-wired/solver`). Everything else here (examples, comparisons, the Lean twin, the GPU searches) runs from a clone and
 needs [Bun](https://bun.sh); Lean 4.34.0 for `lean/` and `lean-kernel/`; a WebGPU runtime (Chrome, or Dawn's node
@@ -218,7 +261,7 @@ bun test core/compile.test.ts core/runtime.test.ts kernel/history.test.ts
 cd kernel && bun run-kernel.ts && bun test kernel.test.ts
 ```
 
-The whole suite (110 jobs, about 15 minutes of wall time) runs on [Modal](https://modal.com):
+The whole suite (122 jobs, about 20 minutes of wall time) runs on [Modal](https://modal.com):
 
 ```sh
 modal run modal/run.py --jobs modal/suite.json --out runs/modal/suite && sh modal/reap.sh

package/dist/core/compose.d.ts

@@ -1,5 +1,5 @@
 import { type C, type E, type Law, type Machine } from './machine.js';
-import type { StepLaw } from '../kernel/kernel.js';
+import { type StepLaw } from '../kernel/kernel.js';
 export declare const side: (m: Machine, prefix: "a_" | "b_", c: C) => C;
 export declare function product(a: Machine, b: Machine, opts?: {
     name?: string;
@@ -15,10 +15,24 @@ export declare function interleave(a: Machine, b: Machine, opts?: {
 export declare function refinement(impl: Machine, spec: Machine, opts: {
     relation: Law[];
     witness: Record<string, E>;
+    stutter?: C;
     name?: string;
 }): {
     machine: Machine;
     laws: Law[];
     stepLaws: StepLaw[];
 };
+export declare function refinementSides(impl: Machine, spec: Machine): {
+    impl: Machine;
+    spec: Machine;
+};
 export declare function agree(a: Machine): Law[];
+export declare function steps(m: Machine, k: number): Machine;
+export type Ghost = {
+    start: E;
+    step: E;
+};
+export declare function ghost(m: Machine, vars: Record<string, Ghost>): Machine;
+export declare const ever: (name: string, cond: C) => Ghost;
+export declare const count: (name: string, cond: C) => Ghost;
+export declare function total(m: Machine, map: string, name: string): Ghost;

package/dist/core/compose.js

@@ -3,18 +3,24 @@
 // is trusted: a proof is about the machine built, and the combinators are tested against runs of their parts.
 //   product(a, b)      both step on every request (their params and inputs shared by name)
 //   interleave(a, b)   one of them steps on each request, `turn` picking which
-//   refinement(impl, spec, {relation, witness})
+//   refinement(impl, spec, {relation, witness, stutter})
 //                      impl refines spec: the spec, sent the request `witness` names for each of impl's, always allows it,
-//                      and `relation` holds of the two states at every step (a forward simulation)
+//                      and `relation` holds of the two states at every step (a forward simulation); where `stutter`
+//                      holds, impl's step is one the spec doesn't see, and the spec stays
+//   ghost(m, vars)     state that watches the machine without changing it: what happened before ("ever", "count") and
+//                      totals across a map's keys ("total"), for laws about whole histories
+//   steps(m, k)        k steps of m as one: the spec of a refinement that takes several steps for one of impl's
 // Each side's state, maps and flags are renamed a_x and b_x; params and inputs keep their names.
 import { all, any, bit, not, same, unreachable, x } from "./machine.js";
+import { substC, substE } from "../kernel/kernel.js";
+import { nameQuantifiers } from "./dsl.js";
 function rename(e, prefix, m, inputs) {
     switch (e.op) {
         case 'var': return m.params.includes(e.name) ? e : m.inputs.includes(e.name) ? (inputs[e.name] ?? e) : x(`${prefix}${e.name}`);
         case 'const': return e;
         case 'mul': return { ...e, a: rename(e.a, prefix, m, inputs) };
         case 'if': return { op: 'if', c: renameC(e.c, prefix, m, inputs), then: rename(e.then, prefix, m, inputs), else: rename(e.else, prefix, m, inputs) };
-        case 'get': return { op: 'get', map: `${prefix}${e.map}`, key: rename(e.key, prefix, m, inputs) };
+        case 'get': return { op: 'get', map: m.inputMaps?.includes(e.map) ? e.map : `${prefix}${e.map}`, key: rename(e.key, prefix, m, inputs) };
         case 'add':
         case 'sub':
         case 'min':
@@ -30,6 +36,8 @@ function renameC(c, prefix, m, inputs) {
         return { op: 'not', x: renameC(c.x, prefix, m, inputs) };
     if (c.op === 'and' || c.op === 'or')
         return { op: c.op, xs: c.xs.map(y => renameC(y, prefix, m, inputs)) };
+    if (c.op === 'all')
+        return { op: 'all', key: `${prefix}${c.key}`, c: renameC(c.c, prefix, m, inputs) }; // the key renamed with its side
     return { ...c, a: rename(c.a, prefix, m, inputs), b: rename(c.b, prefix, m, inputs) };
 }
 // A condition (or law) of one side, read in the composed machine.
@@ -46,6 +54,8 @@ function renamed(m, p, inputs, stays) {
             flags: Object.fromEntries(Object.entries(m.step.flags ?? {}).map(([n, c]) => [`${p}${n}`, renameC(c, p, m, inputs)])),
             state: Object.fromEntries(Object.entries(m.step.state ?? {}).map(([n, e]) => [`${p}${n}`, keep(rename(e, p, m, inputs), x(`${p}${n}`))])),
             maps: Object.fromEntries(Object.entries(m.step.maps ?? {}).map(([n, w]) => {
+                if ('each' in w)
+                    return [`${p}${n}`, { each: `${p}${w.each}`, value: keep(rename(w.value, p, m, inputs), { op: 'get', map: `${p}${n}`, key: x(`${p}${w.each}`) }) }];
                 const key = rename(w.key, p, m, inputs);
                 return [`${p}${n}`, { key, value: keep(rename(w.value, p, m, inputs), { op: 'get', map: `${p}${n}`, key }) }];
             })),
@@ -57,6 +67,8 @@ const join = (a, b) => ({
     state: { ...a.state, ...b.state }, maps: { ...a.maps, ...b.maps },
     step: { flags: { ...a.step.flags, ...b.step.flags }, state: { ...a.step.state, ...b.step.state }, maps: { ...a.step.maps, ...b.step.maps } },
 });
+// The maps requests carry: shared by name, as inputs are.
+const carriedBy = (...ms) => { const names = [...new Set(ms.flatMap(m => m.inputMaps ?? []))]; return names.length ? { inputMaps: names } : {}; };
 const conj = (cs) => { const xs = cs.filter((c) => c !== undefined); return xs.length ? all(...xs) : undefined; };
 // a and b stepped by the same requests. `bInputs` rewrites what b sees (other tenants' requests as no-ops, say).
 export function product(a, b, opts = {}) {
@@ -66,7 +78,7 @@ export function product(a, b, opts = {}) {
         name: opts.name ?? `${a.name} | ${b.name}`,
         params: [...new Set([...a.params, ...b.params, ...(opts.params ?? [])])],
         inputs: [...new Set([...a.inputs, ...b.inputs, ...(opts.inputs ?? [])])],
-        ...join(A, B),
+        ...join(A, B), ...carriedBy(a, b),
         ...(assume ? { assume } : {}),
         ...(pre ? { pre } : {}),
     };
@@ -84,7 +96,7 @@ export function interleave(a, b, opts = {}) {
         name: opts.name ?? `${a.name} || ${b.name}`,
         params: [...new Set([...a.params, ...b.params])],
         inputs: [...new Set([...a.inputs, ...b.inputs, turn])],
-        ...join(A, B),
+        ...join(A, B), ...carriedBy(a, b),
         assume: all(bit(turn), ...(A.assume ? [any(not(isA), A.assume)] : []), ...(B.assume ? [any(not(isB), B.assume)] : [])),
         ...(pre ? { pre } : {}),
     };
@@ -98,13 +110,16 @@ export function interleave(a, b, opts = {}) {
 export function refinement(impl, spec, opts) {
     if (!spec.inputs.every(i => i in opts.witness))
         throw new Error(`refinement: name every input of ${spec.name} (${spec.inputs.join(', ')})`);
-    const A = renamed(impl, 'a_', {}), B = renamed(spec, 'b_', opts.witness);
-    const allowedBy = B.assume;
+    if ((spec.inputMaps ?? []).some(mp => !(impl.inputMaps ?? []).includes(mp)))
+        throw new Error(`refinement: a map ${spec.name}'s requests carry is one ${impl.name}'s carry too (shared by name)`);
+    const A = renamed(impl, 'a_', {}), B = renamed(spec, 'b_', opts.witness, opts.stutter);
+    // A step the spec doesn't see needs no request it allows.
+    const allowedBy = B.assume && opts.stutter ? any(opts.stutter, B.assume) : B.assume;
     const machine = {
         name: opts.name ?? `${impl.name} refines ${spec.name}`,
         params: [...new Set([...impl.params, ...spec.params])],
         inputs: impl.inputs,
-        ...join(A, B),
+        ...join(A, B), ...carriedBy(impl, spec),
         ...(A.assume ? { assume: A.assume } : {}),
         ...(impl.pre ? { pre: impl.pre } : {}),
     };
@@ -112,6 +127,16 @@ export function refinement(impl, spec, opts) {
     const stepLaws = allowedBy ? [{ name: `${spec.name} allows the request it is sent`, then: allowedBy }] : [];
     return { machine, laws, stepLaws };
 }
+// A refinement's two sides as machines in its composite's names (for lean-kernel/Refine.lean, which builds the composite
+// from them): the implementation's parts named a_x, the spec's b_x, the spec's inputs b_<input> (which the composite
+// never reads: the witness stands for them there).
+export function refinementSides(impl, spec) {
+    const named = (m, p, inputs, inputNames) => {
+        const R = renamed(m, p, inputs);
+        return { name: m.name, params: m.params, inputs: inputNames, state: R.state, maps: R.maps, step: R.step, ...(m.inputMaps?.length ? { inputMaps: m.inputMaps } : {}), ...(R.assume ? { assume: R.assume } : {}), ...(m.pre ? { pre: m.pre } : {}) };
+    };
+    return { impl: named(impl, 'a_', {}, impl.inputs), spec: named(spec, 'b_', Object.fromEntries(spec.inputs.map(i => [i, x(`b_${i}`)])), spec.inputs.map(i => `b_${i}`)) };
+}
 // "a and b agree" in a product of a machine with another of its shape: every scalar equal, every map equal at every key.
 export function agree(a) {
     return [
@@ -119,3 +144,102 @@ export function agree(a) {
         ...Object.keys(a.maps ?? {}).map(n => ({ name: `a.${n}[k] = b.${n}[k]`, holds: same({ op: 'get', map: `a_${n}`, key: x('k') }, { op: 'get', map: `b_${n}`, key: x('k') }), forall: 'k' })),
     ];
 }
+// --- several steps as one --------------------------------------------------------------------------------------------
+// k steps of m in a row as one step: the j-th request read as <input>@s<j> (j from 1), each step skipped where its input
+// skip@s<j> is 1. Each step reads the state the one before it left (its flags decided there, every map read over the
+// writes before), so several writes to one map become one write at every key, its key compared with theirs (which the
+// runtimes decide on breakpoints). Its assumption: each step not skipped is one m allows. As the spec of a refinement,
+// the witness names each step's request and when it is skipped.
+export function steps(m, k) {
+    if (!Number.isInteger(k) || k < 1)
+        throw new Error('steps: k is a whole number at least 1');
+    if (m.history || m.inputMaps?.length)
+        throw new Error(`steps: ${m.name} has a history or maps a request carries (not taken here)`);
+    const at = (name, j) => `${name}@s${j}`;
+    let vars = Object.fromEntries(Object.keys(m.state).map(n => [n, x(n)]));
+    let maps = Object.fromEntries(Object.keys(m.maps ?? {}).map(map => [map, (key) => ({ op: 'get', map, key })]));
+    const allowed = [], written = new Set();
+    for (let j = 1; j <= k; j++) {
+        const skip = same(x(at('skip', j)), { op: 'const', value: 1 });
+        const sub = { vars: { ...vars, ...Object.fromEntries(m.inputs.map(i => [i, x(at(i, j))])) }, maps, flags: {} };
+        for (const [f, c] of Object.entries(m.step.flags ?? {}))
+            sub.flags[f] = substC(c, sub);
+        if (m.assume)
+            allowed.push(any(skip, substC(m.assume, sub)));
+        const kept = (now, old) => ({ op: 'if', c: skip, then: old, else: now });
+        const nextVars = { ...vars };
+        for (const [n, e] of Object.entries(m.step.state ?? {}))
+            nextVars[n] = kept(substE(e, sub), vars[n]);
+        const nextMaps = { ...maps };
+        for (const [map, w] of Object.entries(m.step.maps ?? {})) {
+            written.add(map);
+            const old = maps[map];
+            if ('each' in w)
+                nextMaps[map] = (key) => kept(substE(w.value, { ...sub, vars: { ...sub.vars, [w.each]: key } }), old(key));
+            else {
+                const wk = substE(w.key, sub), wv = substE(w.value, sub);
+                nextMaps[map] = (key) => kept({ op: 'if', c: same(key, wk), then: wv, else: old(key) }, old(key));
+            }
+        }
+        vars = nextVars;
+        maps = nextMaps;
+    }
+    const each = 'k$steps';
+    return nameQuantifiers({
+        name: `${m.name}, ${k} steps at once`, params: m.params,
+        inputs: Array.from({ length: k }, (_, i) => [...m.inputs.map(n => at(n, i + 1)), at('skip', i + 1)]).flat(),
+        state: m.state, ...(m.maps ? { maps: m.maps } : {}),
+        step: {
+            state: Object.fromEntries(Object.keys(m.step.state ?? {}).map(n => [n, vars[n]])),
+            ...(written.size ? { maps: Object.fromEntries([...written].map(map => [map, { each, value: maps[map](x(each)) }])) } : {}),
+        },
+        assume: all(...Array.from({ length: k }, (_, i) => bit(at('skip', i + 1))), ...allowed),
+        ...(m.pre ? { pre: m.pre } : {}),
+    });
+}
+export function ghost(m, vars) {
+    const taken = [...m.params, ...m.inputs, ...Object.keys(m.state), ...Object.keys(m.maps ?? {}), ...Object.keys(m.step.flags ?? {})];
+    for (const name of Object.keys(vars))
+        if (taken.includes(name))
+            throw new Error(`ghost: ${name} is already a name of ${m.name}`);
+    if (m.history)
+        throw new Error(`${m.name}: ghosts go on a machine without a history (its sums are ghosts of that kind already)`);
+    return {
+        ...m,
+        state: { ...m.state, ...Object.fromEntries(Object.entries(vars).map(([n, g]) => [n, g.start])) },
+        step: { ...m.step, state: { ...m.step.state, ...Object.fromEntries(Object.entries(vars).map(([n, g]) => [n, g.step])) } },
+    };
+}
+// Whether `cond` has held at some step so far (read where the step reads: the state before, the request, the flags).
+export const ever = (name, cond) => ({ start: { op: 'const', value: 0 }, step: { op: 'if', c: cond, then: { op: 'const', value: 1 }, else: x(name) } });
+// How many steps so far `cond` held at.
+export const count = (name, cond) => ({ start: { op: 'const', value: 0 }, step: { op: 'add', a: x(name), b: { op: 'if', c: cond, then: { op: 'const', value: 1 }, else: { op: 'const', value: 0 } } } });
+// The total of a map over every key: the map must start at 0 everywhere (so finitely many keys are nonzero), and each
+// step writes one key, so the total moves by the value written less the value it replaces. That difference is pushed
+// through the written term where its shape gives it (old + d adds d; old ∸ d takes min(d, old); an if moves each way
+// by its branch's), so a proof needs no fact about the total beyond the law it is about; any other term falls back to
+// total ∸ old + new (exact, since the total is at least any one cell).
+export function total(m, map, name) {
+    if ((m.maps ?? {})[map] !== 0)
+        throw new Error(`total: ${map} must start at 0 at every key`);
+    const w = m.step.maps?.[map];
+    if (!w)
+        return { start: { op: 'const', value: 0 }, step: x(name) };
+    if ('each' in w)
+        throw new Error(`total: ${map} is written at every key at once, which no one step's difference describes`);
+    const old = { op: 'get', map, key: w.key }, t = x(name), same = (a) => JSON.stringify(a) === JSON.stringify(old);
+    const moved = (e) => {
+        if (same(e))
+            return t;
+        if (e.op === 'add' && same(e.a))
+            return { op: 'add', a: t, b: e.b };
+        if (e.op === 'add' && same(e.b))
+            return { op: 'add', a: t, b: e.a };
+        if (e.op === 'sub' && same(e.a))
+            return { op: 'sub', a: t, b: { op: 'min', a: e.b, b: old } };
+        if (e.op === 'if')
+            return { op: 'if', c: e.c, then: moved(e.then), else: moved(e.else) };
+        return { op: 'add', a: { op: 'sub', a: t, b: old }, b: e };
+    };
+    return { start: { op: 'const', value: 0 }, step: moved(w.value) };
+}

package/dist/core/dsl.d.ts

@@ -1,5 +1,5 @@
-import { type C, type E, type Law, type Machine, type State } from './machine.js';
-import { type Goal, type Loop, type StepLaw, type Tree } from '../kernel/kernel.js';
+import { type C, type E, type Law, type Machine, type Request, type State } from './machine.js';
+import { type Goal, type Liveness, type Loop, type StepLaw, type Tree } from '../kernel/kernel.js';
 export type TermLike = Term | number;
 export declare const termOf: (t: TermLike) => E;
 export declare class Term {
@@ -87,7 +87,13 @@ export type MapWrite = {
     readonly map: string;
     readonly key: E;
     readonly value: E;
+} | {
+    readonly map: string;
+    readonly each: string;
+    readonly value: E;
 };
+export declare const everyKey: (f: (key: Term) => Cond) => Cond;
+export declare const someKey: (f: (key: Term) => Cond, witness?: TermLike) => Cond;
 export declare class MapRef {
     readonly name: string;
     readonly init: number;
@@ -95,16 +101,57 @@ export declare class MapRef {
     at(key: TermLike): Term;
     set(key: TermLike, value: TermLike): MapWrite;
     bump(key: TermLike, c?: Cond, by?: TermLike): MapWrite;
+    each(f: (key: Term) => TermLike): MapWrite;
+    has(key: TermLike): Cond;
+    subsetOf(other: MapRef): Cond;
+    disjointFrom(other: MapRef): Cond;
+    sameAs(other: MapRef): Cond;
+    within(n: number): Cond;
+    sizeBelow(n: number): Term;
+}
+export type ListOp = {
+    readonly when: Cond;
+    readonly at: (k: Term) => TermLike;
+    readonly len: TermLike;
+};
+export declare class ListRef {
+    readonly name: string;
+    readonly len: Term;
+    constructor(name: string, len: Term);
+    at(i: TermLike): Term;
+    has(x: TermLike): Cond;
+    sorted(): Cond;
+    push(x: TermLike, when?: Cond): ListOp;
+    pop(when?: Cond): ListOp;
+    set(i: TermLike, x: TermLike, when?: Cond): ListOp;
+    insertAt(i: TermLike, x: TermLike, when?: Cond): ListOp;
+    removeAt(i: TermLike, when?: Cond): ListOp;
+    apply(...ops: ListOp[]): Record<string, TermLike | MapWrite>;
+    within(): Forall;
+    increasing(): Forall;
 }
 export type Forall = {
-    readonly forall: (key: Term) => Cond;
+    readonly forall: (key: Term, ...more: Term[]) => Cond;
 };
-export declare const forall: (f: (key: Term) => Cond) => Forall;
+export declare function forall(f: (key: Term, key2: Term, key3: Term, key4: Term) => Cond): Forall;
 export type Relation = {
     readonly when: Cond;
     readonly holds: Cond;
 };
 export declare const relation: (holds: Cond, whenever?: Cond) => Relation;
+export declare function nameQuantifiers(m: Machine): Machine;
+export type FnDef = {
+    arity: number;
+    js: string | ((...args: number[]) => number);
+    lean?: string;
+    facts?: (f: (...args: TermLike[]) => Term) => Record<string, Cond | Forall>;
+};
+export declare const fn: (arity: number, js: FnDef["js"], opts?: {
+    lean?: string;
+    facts?: FnDef["facts"];
+}) => FnDef;
+export declare const divBy: (c: number) => FnDef;
+export declare const modBy: (c: number) => FnDef;
 type Kinds = Record<string, Kind<unknown>>;
 type ValuesOf<K extends Kinds> = {
     [n in keyof K]: K[n] extends Kind<infer V> ? V : never;
@@ -121,6 +168,10 @@ type Spec = {
     inputs: [string, Kind<unknown>][];
     state: [string, number | ((p: never) => TermLike)][];
     maps: [string, number][];
+    lists?: string[];
+    carried?: string[];
+    fns?: [string, FnDef][];
+    facts?: ((env: never) => Record<string, Cond | Forall>)[];
     flags: ((env: never) => Record<string, Cond>)[];
     step?: (env: never) => Record<string, TermLike | MapWrite | undefined>;
     pre: ((p: never) => Cond)[];
@@ -148,6 +199,18 @@ export declare class Rule<P extends Obj = {}, I extends Obj = {}, S extends Obj
     maps<T extends Record<string, number>>(init: T): Rule<P, I, S, M & {
         [k in keyof T]: MapRef;
     }, F, R, Su, Q>;
+    lists<const N extends string>(...names: N[]): Rule<P, I, S & {
+        [k in N as `${k}_len`]: Term;
+    }, M & {
+        [k in N]: ListRef;
+    }, F, R, Su, Q>;
+    fns<T extends Record<string, FnDef>>(defs: T): Rule<P & {
+        [k in keyof T]: (...args: TermLike[]) => Term;
+    }, I, S, M, F, R, Su, Q>;
+    facts(f: (p: P) => Record<string, Cond | Forall>): Rule<P, I, S, M, F, R, Su, Q>;
+    carries<const N extends string>(...names: N[]): Rule<P, I & {
+        [k in N]: MapRef;
+    }, S, M, F, R, Su, Q>;
     flags<T extends Record<string, Cond>>(f: (env: P & I & S & M & F & Su & Q) => T): Rule<P, I, S, M, F & {
         [k in keyof T]: Cond;
     }, R, Su, Q>;
@@ -165,8 +228,18 @@ export declare class Rule<P extends Obj = {}, I extends Obj = {}, S extends Obj
     relations(f: (env: P & I & Su & Q) => Relation[]): Rule<P, I, S, M, F, R, Su, Q>;
     prune(order: keyof R & string, after: (p: P) => TermLike): Rule<P, I, S, M, F, R, Su, Q>;
     law(name: string, f: (env: P & S & M & Su) => Cond | Forall): Law;
+    private lawOf;
     stepLaw(name: string, then: (before: P & I & S & M & F & Su & Q, after: S & M & Su) => Cond | Forall, when?: (before: P & I & S & M & F & Su & Q) => Cond): StepLaw;
     stepsLaw(name: string, k: number, then: (first: P & I & S & M & F & Su & Q, last: S & M & Su, requests: (I & Q)[]) => Cond | Forall, when?: (before: P & I & S & M & F & Su & Q, j: number) => Cond | undefined): StepLaw;
+    leadsTo(name: string, spec: {
+        from: (env: P & S & M & Su) => Cond;
+        to: (env: P & S & M & Su) => Cond;
+        rank: (env: P & S & M & Su) => TermLike | TermLike[];
+        helpful: ((env: P & S & M & Su) => Record<string, TermLike>) | {
+            when?: (env: P & S & M & Su) => Cond;
+            request: (env: P & S & M & Su) => Record<string, TermLike>;
+        }[];
+    }): Liveness;
     decide(name: string, given: (env: P & I) => Cond | Cond[], then: (env: P & I) => Cond): Decision;
     private kindAssumptions;
     private env;
@@ -175,10 +248,10 @@ export declare class Rule<P extends Obj = {}, I extends Obj = {}, S extends Obj
         [k in keyof P]: Raw;
     }): Record<string, number>;
     encodeInput(values: {
-        [k in keyof I]: Raw;
-    }): Record<string, number>;
+        [k in keyof I]: I[k] extends MapRef ? ReadonlySet<number> | readonly number[] | ReadonlyMap<number, number> : Raw;
+    }): Request;
     randomParams(rand: Rand, bound: number): Record<string, number>;
-    randomInput(rand: Rand, bound: number): Record<string, number>;
+    randomInput(rand: Rand, bound: number): Request;
     get orderedBy(): string | undefined;
     interpreter(): {
         init: (params: {
@@ -186,16 +259,12 @@ export declare class Rule<P extends Obj = {}, I extends Obj = {}, S extends Obj
         }) => State;
         queries: (params: {
             [x: string]: number;
-        }, st: State, input: {
-            [x: string]: number;
-        }) => {
+        }, st: State, input: Request) => {
             [x: string]: number;
         };
         step: (params: {
             [x: string]: number;
-        }, st: State, input: {
-            [x: string]: number;
-        }) => State;
+        }, given: State, input: Request) => State;
     };
     compiled(): {
         init: (params: object) => State & {
@@ -264,5 +333,7 @@ export type Verdict = {
 export declare function proveRule(r: Rule<Obj, Obj, Obj, Obj, Obj, Obj, Obj, Obj>, laws: Law[]): Verdict;
 export declare function proveSteps(r: Rule<Obj, Obj, Obj, Obj, Obj, Obj, Obj, Obj>, laws: Law[], stepLaws: StepLaw[]): Verdict;
 export type { StepLaw };
+export type { Liveness };
+export declare function proveLeadsTo(r: Rule<Obj, Obj, Obj, Obj, Obj, Obj, Obj, Obj>, laws: Law[], l: Liveness): Verdict;
 export declare const proveDecision: (d: Decision) => Verdict;
 export declare const proveLoop: (l: LoopRule<Obj, Obj, Obj>, laws: Law[]) => Verdict;