@panagos/jojoscript 0.9.3 → 1.0.0

package/README.md

@@ -1,13 +1,12 @@
-# JojoScript 0.9.3
+# JojoScript 1.0.0
 
-JojoScript is a small JavaScript-compatible language and runtime for
-expressive, composable pipelines. It adds lightweight syntax for declarations,
-functions, pattern matching, type annotations, and pipelines while preserving
-ordinary JavaScript. Its compiler uses a lightweight lexer and source
-transformations rather than a full AST, keeping the toolchain compact. The
-project also provides lazy and async collection operators, checkpointing,
-retries, metrics, pipeline graphing, CLI workflows, effect handlers, MCP
-support, and VS Code editor support.
+JojoScript is a lightweight language and runtime for data pipelines and agent
+workflows, built on JavaScript.
+
+It combines expressive pipeline syntax, lazy data processing, resumable
+execution and effect handling with MCP tool serving and A2A agent
+communication. It compiles to ordinary JavaScript and keeps the familiar
+JavaScript ecosystem accessible, without requiring a heavyweight framework.
 
 See [CHANGELOG.md](CHANGELOG.md) for release history and
 [example/](example/) for a complete runnable application that exercises
@@ -23,8 +22,17 @@ npm install @panagos/jojoscript
 
 ```bash
 jojo build <file.jojo|directory> [--out <directory>] [--watch]
-jojo run <file.jojo> [--out <directory>]
-jojo graph <file.jojo> [--format text|dot]
+jojo run <file.jojo> [--out <directory>] [--deterministic [--seed <n>]]
+jojo graph <file.jojo> [--format text|dot] [--pipeline <name>]
+jojo explain <file.jojo> [--format text|json] [--pipeline <name>]
+jojo plan <file.jojo> [--format json] [--pipeline <name>]
+jojo generate <plan.json>
+jojo check <file.jojo|plan.json> [--format text|json] [--machine]
+jojo describe <file.jojo> [--pipeline <name>]
+jojo simulate <file.jojo> [--input <records>] [--format text|json] [--pipeline <name>]
+jojo why <file.jojo> [--format text|json] [--pipeline <name>]
+jojo format <file.jojo|directory> [--check]
+jojo parse <file.jojo> [--json]
 jojo profile <file.jojo>
 jojo resume <file.jojo> [--out <directory>]
 jojo replay <file.jojo> --from <checkpoint> [--out <directory>]
@@ -36,7 +44,17 @@ jojo --help
 - `jojo build src` compiles every `.jojo` file under `src` to `src/dist/*.js` (or `--out <dir>`).
 - `jojo build src --watch` rebuilds whenever a `.jojo` file under `src` changes.
 - `jojo run app.jojo` compiles a single file and immediately runs it with Node — useful for a fast dev loop.
+- `jojo run app.jojo --deterministic --seed 42` runs the program in [deterministic execution mode](#deterministic-execution-mode): seeded `random()`, a fixed clock, sequential `parallel()`, and deterministic effect handlers only.
 - `jojo graph app.jojo` prints a static pipeline graph; use `--format dot` for Graphviz output.
+- `jojo explain app.jojo` describes what each pipeline means (stages, effects, properties) without running it; use `--format json` for tools and LLMs. See [Explaining pipelines](#explaining-pipelines).
+- `jojo plan app.jojo` prints a canonical JSON execution plan, and `jojo generate plan.json` turns a plan back into validated JojoScript. See [Machine-readable execution plans](#machine-readable-execution-plans).
+- `graph`, `explain`, and `plan` accept `--pipeline <name>` to select one pipeline by its (inferred or [declared](#named-pipelines)) name.
+- `jojo check app.jojo` (or `plan.json`) reports structured diagnostics with codes, lines, and suggestions, and exits 1 on errors; `--machine` prints JSON. See [Checking programs](#checking-programs).
+- `jojo describe app.jojo` prints one JSON document describing the pipelines, effects, functions, and types in a file. See [Describing programs](#describing-programs).
+- `jojo simulate app.jojo --input 1000000` estimates item counts, effect calls, and risks for a given input size without running anything. See [Simulating pipelines](#simulating-pipelines).
+- `jojo why app.jojo` answers questions such as "why is this pipeline not deterministic?" in plain language. See [Asking why](#asking-why).
+- `jojo format src` reformats `.jojo` files in place; `--check` lists unformatted files and exits 1. See [Formatting](#formatting).
+- `jojo parse app.jojo --json` lists every syntax construct (imports, effects, types, functions, pipelines, handle blocks, matches) with its line. See [Parsing programs](#parsing-programs).
 - `jojo profile app.jojo` executes a pipeline and prints per-node item counts, duration, and throughput.
 - `jojo resume app.jojo` reuses completed file-backed checkpoints; `jojo replay app.jojo --from name` starts from a selected checkpoint.
 - `jojo test app.jojo` compiles the file and runs its co-located `<name>.test.js` test when present.
@@ -137,6 +155,69 @@ async fn loadUsers() {
 }
 ```
 
+### Pure and effectful functions
+
+`pure` and `effectful` declare a function's role. They compile away to an
+ordinary function, and may follow `export` or surround `async`:
+
+```jojo
+effect storeUser(user)
+
+export pure fn normalizeUser(user) {
+  return { ...user, email: user.email.trim().toLowerCase() }
+}
+
+effectful fn saveUser(user) {
+  return storeUser(user)
+}
+```
+
+The compiler rejects a `pure fn` that uses a declared `effect` or an
+`effectful fn`, whether it calls it or passes it along (`map(saveUser)`):
+
+```
+pure fn "normalizeUser" uses effectful function "saveUser" (line 9)
+```
+
+The check is static and name-based. Strings, comments, property access
+(`user.saveUser`), and recursion are ignored, and calls to unannotated
+functions are allowed. `jojo explain` reports a stage that uses an
+`effectful fn` as `effectful` and the pipeline as non-deterministic, exactly as
+it does for declared effects. See
+[example/src/application/purity-example.jojo](example/src/application/purity-example.jojo).
+
+### Function contracts
+
+A block function can state what it expects and what it guarantees. Each
+clause goes on its own line between the parameter list and the opening `{`:
+
+```jojo
+fn applyDiscount(price, percent)
+  requires price >= 0
+  requires percent >= 0 && percent <= 100
+  ensures result >= 0 && result <= price
+{
+  return price - price * percent / 100
+}
+```
+
+`requires` clauses are checked on entry, in order, before the body runs.
+`ensures` clauses are checked after it, with the return value available as
+`result` (async functions are awaited first). The body runs inside an arrow
+function, so `this` and `arguments` keep referring to the declared function.
+A broken clause throws `ContractError`:
+
+```
+ContractError: Contract violated: applyDiscount requires percent >= 0 && percent <= 100
+```
+
+The error carries `functionName`, `kind` (`"requires"` or `"ensures"`), and
+`condition`, and is non-retryable, so `retry()` never repeats a call that
+broke its contract. Contract clauses keep the source line count. Contracts
+apply to block functions only, and a function using `ensures` should not
+have a parameter named `result`. See
+[example/src/application/contracts-example.jojo](example/src/application/contracts-example.jojo).
+
 ### Optional type annotations
 
 Parameters and return values can be annotated. Annotations are **never
@@ -198,8 +279,77 @@ category := match true {
 }
 ```
 
-`match` is a value-equality construct, not structural pattern matching:
-patterns are plain expressions compared with `===`.
+#### Structured patterns
+
+Object patterns match on the shape of a value. Arms can be separated by
+newlines instead of commas:
+
+```jojo
+message := match result {
+  { status: "ok", value } => value
+  { status: "error", error: { code: 404 } } => "not found"
+  { status: "error", error } => handle(error)
+}
+```
+
+Inside an object pattern:
+
+- a literal or expression (`status: "ok"`, `code: Status.OK`) is compared with `===`;
+- a bare name (`value`) binds that field, and the field must be present;
+- `key: name` binds the field under another name, and `key: { ... }` matches a nested object;
+- quoted keys (`"content-type": "json"`) are supported.
+
+A match with at least one object pattern may omit the `_` arm. If no arm
+matches, it throws `TypeError: No match arm matched the value`, so an
+unexpected shape fails loudly instead of returning `undefined`. A match whose
+patterns are all plain expressions still requires a final `_` arm and
+compiles exactly as before. See
+[example/src/application/structured-match-example.jojo](example/src/application/structured-match-example.jojo).
+
+### Tagged unions
+
+`type` declares a tagged union: a value that is exactly one of a fixed set of
+variants. Each variant has a constructor:
+
+```jojo
+type PaymentResult =
+  | Paid(amount)
+  | Declined(reason)
+  | Pending
+```
+
+becomes:
+
+```js
+const PaymentResult = defineType("PaymentResult", { Paid: ["amount"], Declined: ["reason"], Pending: [] }); const { Paid, Declined, Pending } = PaymentResult
+```
+
+`Paid(40)` returns the frozen plain object `{ tag: "Paid", amount: 40 }`, so
+values serialize to JSON unchanged. Constructors reject the wrong number of
+arguments (`Paid expects 1 argument (amount), got 0`), and fieldless variants
+are called with no arguments (`Pending()`). `export type` exports the type and
+all of its constructors. Variant names must start with an uppercase letter,
+variants and fields must be unique, and no field may be named `tag`.
+
+Constructor patterns match a variant by its tag and bind its fields by
+position, so they also match values decoded from JSON:
+
+```jojo
+message := match result {
+  Paid(amount) => `paid ${amount}`
+  Declined("insufficient funds") => "ask for another card"
+  Declined(reason) => `declined: ${reason}`
+  Pending() => "waiting for the bank"
+}
+```
+
+Pattern arguments can be literals, `_`, bindings, or nested object and
+constructor patterns. Like object patterns, a match over constructor patterns
+may omit `_` and throws `TypeError` when nothing matches. A capitalized call
+such as `Ok(v)` inside a `match` is always a constructor pattern. Avoid naming
+a variant `Error`, since its constructor would shadow JavaScript's `Error` in
+that module. See
+[example/src/application/tagged-unions-example.jojo](example/src/application/tagged-unions-example.jojo).
 
 ### Pipelines
 
@@ -222,6 +372,52 @@ A stage can have no arguments (`users |> unique`) or arguments
 inputs, including `await` expressions and pipelines inside `return`
 statements, function bodies, and object literals.
 
+#### Named pipelines
+
+A `pipeline` block declares a named, reusable pipeline. Its first line names
+the source, and the remaining lines are stages:
+
+```jojo
+export pipeline enrichOrders {
+  source orders
+
+  |> filter(order => order.active)
+  |> map(addVat)
+  |> checkpoint("enriched")
+  |> toArrayAsync
+  |> await
+}
+```
+
+It compiles to a function. An identifier source becomes its single parameter
+(`enrichOrders(orders)`); any other source expression gives a function with
+no parameters. A pipeline with a `|> await` stage (or an awaited source) becomes an
+`async` function; an `await` inside a stage callback does not. Also,
+blank lines inside the block are allowed. Tools see the pipeline under its
+declared name, so `jojo explain app.jojo --pipeline enrichOrders` (and the same
+option on `graph` and `plan`) selects it directly. See
+[example/src/application/named-pipeline-example.jojo](example/src/application/named-pipeline-example.jojo).
+
+#### Stage IDs
+
+A stage can be labelled with a trailing `@id` (letters, digits, `_`, and `-`,
+starting with a letter or `_`):
+
+```jojo
+saved := orders
+  |> filter(order => order.paid)   @paid-only
+  |> map(assignCarrier)             @pick-carrier
+  |> checkpoint("prepared")
+```
+
+The label is removed from the compiled code. It becomes the stage's node id in
+runtime metadata (checkpoint, inspect, parallel, batch, and `jojo profile`),
+in `jojo graph`, in `jojo explain` (`id` and `@pick-carrier` in the text), and
+in `jojo plan` (an `id` field that `jojo generate` writes back). Tools and
+LLMs can therefore refer to "the `@pick-carrier` stage" instead of a position
+in the source. IDs must be unique within a pipeline. They are not part of the
+pipeline's hash, so renaming a label never invalidates saved checkpoints.
+
 #### Placeholder argument
 
 By default the piped value is inserted as the stage's first argument. The
@@ -326,6 +522,381 @@ auto-imported):
 See [ITERATOR_PROPOSAL.md](ITERATOR_PROPOSAL.md) and [ITERATORS.md](ITERATORS.md)
 for the full laziness/memory model.
 
+## Explaining pipelines
+
+`jojo explain` lets a person, a tool, or an LLM ask JojoScript what a program
+means instead of inferring it from source. It is static: nothing is executed.
+
+```jojo
+effect enrichOrder(order)
+
+export async fn enrichPaidOrders(orders) {
+  return handle {
+    enrichOrder: order => ({ ...order, region: "EU" })
+  } {
+    return orders
+      |> filter(order => order.status == "paid")
+      |> parallel(4)
+      |> map(enrichOrder)
+      |> checkpoint("enriched-orders")
+      |> toArrayAsync
+      |> await
+  }
+}
+```
+
+```
+$ jojo explain orders.jojo
+PIPELINE enrichPaidOrders
+
+STAGES
+  1 source       orders
+  2 filter       pure
+  3 parallel     concurrency=4
+  4 map          effectful: enrichOrder
+  5 checkpoint   enriched-orders
+  6 toArrayAsync pure
+  7 await        control
+
+EFFECTS
+  enrichOrder (handler declared)
+
+PROPERTIES
+  lazy: true
+  parallel: true
+  resumable: true
+  deterministic: false
+```
+
+`jojo explain orders.jojo --format json` prints the same information as an
+array of `{ name, id, stages, effects, properties }` objects, and
+`explainPipelines(source)` / `renderExplanationText(explanation)` are exported
+from `@panagos/jojoscript` for programmatic use.
+
+- **name** is inferred: the variable or property a pipeline is assigned to
+  (`paid := orders |> ...`), the `fn` name of an expression function, or the
+  enclosing named function of a `return` pipeline; otherwise `pipeline-N`.
+- **stages** list each stage's type, raw `args`, referenced `effects`, and a
+  `detail`: the source expression, `concurrency=N` for `parallel`, `size=N` for
+  `batch`, the checkpoint name, `control` for `await`/`catch`/`orElse`/`pass`,
+  or `pure`/`effectful: <effects>`.
+- **effects** are the declared effects the pipeline references, each with the
+  same best-effort `staticallyHandled` signal as `jojo graph`.
+- **properties**: `lazy` (no `sort`/`sortBy`/`groupBy`/`partition`
+  materialization boundary), `parallel` (has a `parallel` stage), `resumable`
+  (has a `checkpoint` stage), `deterministic` (references no declared effect).
+
+Purity and determinism only reflect references to declared `effect`s and
+`effectful fn`s (see [Pure and effectful functions](#pure-and-effectful-functions)): a stage
+calling ordinary JavaScript such as `Math.random()` or `fetch()` directly is
+still reported as `pure`. Route external work through effects to make it
+visible. See
+[example/src/application/explain-example.jojo](example/src/application/explain-example.jojo).
+
+## Machine-readable execution plans
+
+`jojo plan` gives every pipeline a stable, simple JSON representation, and
+`jojo generate` turns such a plan back into JojoScript. A tool or an LLM can
+read a program as data, or write a plan and have JojoScript validate it:
+
+```
+Jojo source -> jojo plan -> JSON plan -> LLM
+LLM -> JSON plan -> jojo generate -> validated Jojo program
+```
+
+```jojo
+export fn topCustomers(customers) {
+  return customers
+    |> filter(customer => customer.active)
+    |> sortBy(customer => -customer.spend)
+    |> take(2)
+    |> map(customer => customer.name)
+    |> toArray
+}
+```
+
+```
+$ jojo plan customers.jojo --format=json
+{
+  "version": 1,
+  "pipelines": [
+    {
+      "pipeline": "topCustomers",
+      "id": "…",
+      "stages": [
+        { "type": "source", "expression": "customers" },
+        { "type": "filter", "expression": "customer => customer.active" },
+        { "type": "sortBy", "expression": "customer => -customer.spend" },
+        { "type": "take", "args": ["2"] },
+        { "type": "map", "expression": "customer => customer.name" },
+        { "type": "toArray" }
+      ]
+    }
+  ]
+}
+```
+
+(Printed with two-space indentation; condensed here.)
+
+The plan format, version 1:
+
+| Field | Meaning |
+|---|---|
+| `version` | Always `1` for this format |
+| `pipelines[].pipeline` | Inferred name (same rules as `jojo explain`), or `null` for an unnamed pipeline |
+| `pipelines[].id` | The pipeline's deterministic hash (informational, ignored by `generate`) |
+| `pipelines[].stages[0]` | `{ "type": "source", "expression": "..." }` |
+| `parallel` / `batch` | `concurrency` / `size` as a positive integer |
+| `checkpoint` | `name` as a string |
+| `map`, `filter`, `flatMap`, `tap`, `mapAsync`, `filterAsync`, `find`, `some`, `every`, `sortBy`, `distinctBy`, `groupBy`, `partition`, `catch` | `expression`: the single function argument as source text |
+| any other stage | `args`: argument source strings, omitted when there are none |
+| `.prop` / `.method()` | `{ "type": ".length" }` reads a property; `{ "type": ".trim", "args": [] }` calls it |
+| `id` (any stage after the source) | Optional [stage ID](#stage-ids) such as `"pick-carrier"`, unique within the pipeline |
+
+A stage whose setting is not a literal (`parallel(limit)`) keeps it in `args`
+instead, so a plan is always lossless. Arguments stay source text: the plan
+describes computation without evaluating it.
+
+`jojo generate plan.json` (or `planToJojo(plan)`) emits one
+`name := source |> ...` statement per pipeline. It rejects unknown fields,
+wrong types, misplaced `source` stages, and duplicate names with the exact path
+of the problem (for example
+`Invalid plan: pipelines[0].stages[2].concurrency must be a positive integer`),
+then compiles the program and checks that it parses as JavaScript, without
+running it. The generated program contains only the pipelines: their source
+variables, custom stage functions, and effects must be supplied by
+surrounding code. Validation is not a sandbox: expressions are code, so
+review a generated program as you would any code you did not write.
+
+`planPipelines(source)` and `planToJojo(plan)` are exported from
+`@panagos/jojoscript`. See
+[example/src/application/plan-example.jojo](example/src/application/plan-example.jojo).
+
+## Checking programs
+
+`jojo check` validates a program, or a JSON plan, without running it. It is
+designed for LLM-generated code: every problem is a structured diagnostic with
+a stable code, a line, and a concrete suggestion, instead of a raw stack trace.
+
+```
+$ jojo check generated.jojo
+ERROR E301
+line 8
+Call divide(1, 0) violates "requires b != 0"
+suggestion: Pass arguments to "divide" that satisfy b != 0
+
+1 error, 0 warnings
+```
+
+`--machine` (or `--format json`) prints the same result as JSON:
+
+```json
+{
+  "valid": false,
+  "errors": [
+    {
+      "code": "E301",
+      "severity": "error",
+      "line": 8,
+      "message": "Call divide(1, 0) violates \"requires b != 0\"",
+      "suggestion": "Pass arguments to \"divide\" that satisfy b != 0"
+    }
+  ],
+  "warnings": []
+}
+```
+
+The command exits with code 1 when there is at least one error, so a
+generate-check-fix loop can rely on the exit code.
+
+| Code | Severity | Meaning |
+|---|---|---|
+| `E100` | error | The compiler rejects the program (with its line when known) |
+| `E101` | error | The compiled JavaScript does not parse (checked by Node's own parser) |
+| `W102` | warning | A declared effect is used but no `handle { ... }` in the file handles it; a caller may still install one |
+| `E201` | error | A `pure fn` uses a declared effect or an `effectful fn` |
+| `E301` | error | A call with literal arguments breaks a `requires` clause |
+| `E401` | error | A `match` over a declared tagged union misses a variant and has no `_` arm |
+| `E402` | error | A constructor pattern has a different number of fields than its variant |
+| `E700` | error | `jojo check plan.json`: the plan is invalid (message names the exact field) |
+
+Contract clauses are only evaluated statically when they use nothing but the
+function's parameters, literals, and operators, so `jojo check` never runs
+program code. `checkSource(source)`, `checkPlan(plan)`, and
+`renderCheckText(result)` are exported from `@panagos/jojoscript`. See
+[example/src/application/check-example.jojo](example/src/application/check-example.jojo).
+
+## Describing programs
+
+`jojo describe` prints one self-describing JSON document for a file, so a tool
+or an LLM can learn what a program contains without reading its source:
+
+```json
+{
+  "language": "jojo",
+  "version": "1",
+  "pipelines": [
+    {
+      "name": "openInvoices",
+      "inputs": ["records"],
+      "effects": [],
+      "stages": [
+        { "id": "unpaid", "operation": "filter", "expression": "record => !record.paid" },
+        { "id": "classify", "operation": "map", "expression": "toInvoice" },
+        { "id": "node-3", "operation": "checkpoint", "name": "open-invoices" }
+      ],
+      "properties": { "lazy": true, "parallel": false, "resumable": true, "deterministic": true }
+    }
+  ],
+  "effects": [{ "name": "storeInvoice", "params": ["invoice"], "handled": true }],
+  "functions": [
+    { "name": "toInvoice", "line": 12, "purity": "pure", "requires": ["record.amount >= 0"], "ensures": [] }
+  ],
+  "types": [
+    { "name": "Invoice", "variants": [{ "name": "Open", "fields": ["id", "amount"] }, { "name": "Settled", "fields": ["id"] }] }
+  ]
+}
+```
+
+- `pipelines` uses the same names, effects, and properties as `jojo explain`,
+  and the same stage fields as `jojo plan`. A pipeline whose source is an
+  identifier lists it in `inputs`. A stage's `id` is its
+  [stage ID](#stage-ids), or the runtime node id (`node-3`) used by
+  `jojo graph` and `jojo profile`.
+- `effects` lists declared effects and whether a handler is statically
+  visible in the file.
+- `functions` lists functions that declare a purity or contracts.
+- `types` lists tagged unions with their variants and fields.
+
+`--pipeline <name>` limits `pipelines` to one pipeline. `describeSource(source)`
+is exported from `@panagos/jojoscript`. See
+[example/src/application/describe-example.jojo](example/src/application/describe-example.jojo).
+
+## Parsing programs
+
+`jojo parse <file.jojo> --json` prints the syntax-level view of a file: every
+JojoScript construct the compiler recognizes, with its line, ordered by line.
+Where `jojo describe` summarizes what a program means, `jojo parse` lists what
+it contains, so a tool or an LLM can locate and edit constructs precisely.
+
+```json
+{
+  "language": "jojo",
+  "version": "1",
+  "nodes": [
+    { "kind": "type", "line": 6, "name": "Decision", "exported": true,
+      "variants": [{ "name": "Approved", "fields": ["id"] }, { "name": "Escalated", "fields": ["id", "team"] }] },
+    { "kind": "function", "line": 10, "name": "decide", "params": ["refund"], "exported": false,
+      "async": false, "purity": null, "body": "expression", "requires": [], "ensures": [] },
+    { "kind": "match", "line": 13, "patterns": ["Approved(id)", "Escalated(id, team)"] },
+    { "kind": "pipeline", "line": 20, "name": "routeRefunds", "declared": false, "source": "refunds",
+      "stages": [{ "type": "map", "args": ["decide"], "id": null }] }
+  ]
+}
+```
+
+| Kind | Fields |
+|---|---|
+| `import` | `from`, `names` |
+| `effect` | `name`, `params` |
+| `type` | `name`, `exported`, `variants` (`name`, `fields`) |
+| `function` | `name`, `params` (as written, including defaults), `exported`, `async`, `purity`, `body` (`"block"` or `"expression"`), `requires`, `ensures` |
+| `pipeline` | `name`, `declared` (a `pipeline` block), `source`, `stages` (`type`, `args`, `id`) |
+| `handle` | `handlers` |
+| `match` | `patterns` |
+
+JojoScript has no AST by design: nodes come from the same finders the compiler
+uses, so `parse` never disagrees with what compiles. Invalid syntax is
+reported as a single `jojo:` error line. `parseSource(source)` is exported
+from `@panagos/jojoscript`. See
+[example/src/application/parse-example.jojo](example/src/application/parse-example.jojo).
+
+## Simulating pipelines
+
+`jojo simulate` answers "what would this pipeline do with N records?" before
+anything runs. No stage or effect is executed:
+
+```
+$ jojo simulate prices.jojo --input 250000 --pipeline pricedProducts
+SIMULATION pricedProducts
+
+INPUT
+  250,000 records
+
+OPERATIONS
+  2 filter     250,000 items in
+  3 parallel   ≤250,000 items in, 16 workers
+  4 map        ≤250,000 items in, ≤250,000 calls to fetchPrice
+  5 checkpoint ≤250,000 items in, saved as "priced"
+
+POTENTIAL PROBLEMS
+  ⚠ stage 4: up to 250,000 calls to effect "fetchPrice"
+  ⚠ stage 4: effect "fetchPrice" runs before checkpoint "priced": items processed before a failure are fetched again on retry (at-least-once)
+```
+
+Stages are numbered as in `jojo explain`, where the source is stage 1.
+Item counts flow through the stages as upper bounds (`≤` once a filtering
+stage such as `filter`, `unique`, or `distinctBy` has run). `take`, `skip`,
+`chunk`, `batch`, and `window` change them exactly; `flatMap` and `groupBy`
+make them unknown; `count`, `first`, `find`, `some`, `every`, and `reduce`
+produce one result. A stage that uses a declared effect or an `effectful fn`
+makes one call per item it receives.
+
+Potential problems reported:
+
+- 1,000 or more calls to one effect;
+- an effect that runs before a checkpoint, and so may run again for items
+  processed before a failure (see
+  [Effects and checkpoint/resume](#effects-and-checkpointresume));
+- a pipeline with effects but no checkpoint at all;
+- `sort`, `sortBy`, `groupBy`, or `partition` holding 10,000 or more items in
+  memory.
+
+The input defaults to 1,000 records. `--format json` prints the simulation as
+data, and `--pipeline <name>` selects one pipeline. `simulatePipelines(source,
+{ input })` and `renderSimulationText(simulation)` are exported from
+`@panagos/jojoscript`. See
+[example/src/application/simulate-example.jojo](example/src/application/simulate-example.jojo).
+
+## Asking why
+
+`jojo why` turns the static analysis into plain-language answers, numbered
+like `jojo explain` (the source is stage 1):
+
+```
+$ jojo why onboarding.jojo --pipeline verifiedCustomers
+WHY verifiedCustomers
+
+Why is this pipeline not deterministic?
+→ Stage 3 (mapAsync) uses effect verifyEmail; stage 5 (mapAsync) uses effectful function sendWelcome.
+
+Why is this pipeline lazy?
+→ Every stage passes items on one at a time, so no stage waits for the complete input.
+
+Why is this pipeline resumable?
+→ Checkpoint "verified" (stage 4) saves its items once they all complete; a rerun restores them instead of repeating stages 2-3.
+
+Why can't stage 3 be moved before stage 2?
+→ Stage 3 (mapAsync) uses effect verifyEmail and stage 2 (filter) decides which items reach it: moving it earlier would also run it for items that stage 2 drops.
+
+Why does checkpoint "verified" not guarantee replay safety?
+→ Effect verifyEmail runs before it (stage 3): if a run fails before "verified" completes, it runs again on retry (at-least-once, not exactly-once).
+
+Why can stage 5 repeat after a failure?
+→ Stage 5 (mapAsync) uses effectful function sendWelcome after the last checkpoint "verified": a rerun restores the checkpoint and runs it again for every restored item.
+```
+
+Every pipeline gets answers about determinism, laziness, and resumability.
+Additional questions appear when they apply: an effectful stage behind a
+filtering stage (`filter`, `filterAsync`, `unique`, `distinctBy`, `take`,
+`skip`) cannot be moved before it, a `parallel` stage explains what it
+parallelizes, and checkpoints explain which effects can run again on retry.
+Answers only reflect declared effects and `effectful fn`s. `--format json`
+prints `[{ name, answers: [{ question, answer }] }]`, and
+`whyPipelines(source)` / `renderWhyText(why)` are exported from
+`@panagos/jojoscript`. See
+[example/src/application/why-example.jojo](example/src/application/why-example.jojo).
+
 ## Effect Handlers
 
 > This is a deliberately simplified effect-handler model, not full
@@ -592,6 +1163,51 @@ declared itself `deterministic`, and the declaring effect's `sideEffecting`/
 `replayable` metadata (from `effect name(params)`'s options, or defaults —
 `sideEffecting: true`, `replayable: false`).
 
+### Effect authorization
+
+Applications can place a capability policy around a `handle` block with
+`withAuthorization(policy, fn)`. The policy receives the declared effect
+name, a shallow copy of its arguments, declared effect metadata, and the
+current execution context. It must explicitly return `true` or
+`{ allowed: true }` for the handler to run.
+
+```js
+import { withAuthorization } from "@panagos/jojoscript/runtime"
+
+const result = await withAuthorization(
+  invocation => invocation.name === "readTender"
+    ? true
+    : { allowed: false, reason: "Capability is not granted" },
+  () => withHandlers({
+    readTender: id => ({ id }),
+    approveOffer: offer => approve(offer)
+  }, () => program())
+)
+```
+
+Authorization is opt-in; programs that do not use `withAuthorization` retain
+their existing unrestricted effect behavior. When enabled, every active
+authorization scope must allow each resolved effect invocation before its
+handler starts. This means an inner `handle` block cannot bypass an outer
+policy. A missing, malformed, denied, or failed policy decision throws
+`EffectAuthorizationError`; denial is non-retryable, so `retry()` does not
+repeat it. An unhandled effect still throws `UnhandledEffectError` before any
+authorization policy runs.
+
+Authorization is evaluated before normal handlers, recording wrappers, and
+replay wrappers. Replay therefore does not bypass a policy, and a checkpoint
+resume only avoids authorization when it restores a completed checkpoint
+without invoking an upstream effect at all. Policies do not record arguments
+or decisions automatically, and this mechanism is not a sandbox: ordinary
+JavaScript and operations that do not pass through a declared effect are
+outside its enforcement boundary. Do not derive permission from model or
+remote-agent output; make policy decisions in trusted application code.
+
+See
+[example/src/effects/effect-authorization.jojo](example/src/effects/effect-authorization.jojo)
+for a procurement workflow that permits reads and supplier lookup while
+blocking invitations and offer approval before their handlers run.
+
 ### Effects and checkpoint/resume
 
 Consider:
@@ -629,6 +1245,13 @@ actually safe. See
 [example/src/effects/checkpoint-resume.jojo](example/src/effects/checkpoint-resume.jojo)
 and `test/effects-checkpoint.test.js` for both cases exercised end to end.
 
+File-backed checkpoint records are validated when they are loaded: their
+pipeline ID and checkpoint ID must match the requested checkpoint, `items`
+must be an array, and a present `itemCount` must match the saved item count.
+Malformed records fail clearly rather than resuming with untrusted data. See
+[example/src/effects/checkpoint-integrity.jojo](example/src/effects/checkpoint-integrity.jojo)
+for a compiled pipeline test of this behavior.
+
 ### Record/replay
 
 A handler map can be wrapped to record every call (arguments + result) to
@@ -653,6 +1276,44 @@ interface (`record`/`replayNext`), mirroring `CheckpointStore` — an
 extension point for a future backend, not a full production-grade
 persistence system.
 
+### Deterministic execution mode
+
+`now()` and `random()` are built-in capabilities, like `parallel()`. Outside
+any handler they behave like `Date.now()` and `Math.random()`, and a
+`handle { TIME: ..., RANDOM: ... }` block can replace them:
+
+```jojo
+import { now, random } from "@panagos/jojoscript/runtime"
+
+order := { id: Math.round(random() * 1000), createdAt: new Date(now()) }
+```
+
+`deterministic(fn, { seed, startTime })` runs `fn` so that the same inputs
+always produce the same outputs:
+
+- `random()` is a seeded generator (`seed` defaults to `1`);
+- `now()` starts at `startTime` (default `0`) and advances 1ms per read;
+- `parallel()` stages process one item at a time, in order;
+- every declared effect must resolve to a handler declared deterministic
+  (`{ handle, deterministic: true }`, or `replayHandlers(...)`); any other
+  handler throws a non-retryable `NondeterministicEffectError` before it runs;
+- checkpoints record `now()` as their `createdAt`, so reruns write identical
+  checkpoints.
+
+```js
+import { deterministic } from "@panagos/jojoscript/runtime"
+
+const first = await deterministic(() => generateOrders(3), { seed: 2026 })
+const second = await deterministic(() => generateOrders(3), { seed: 2026 })
+// first and second are deep-equal
+```
+
+`jojo run app.jojo --deterministic [--seed <n>]` runs a whole program, top-level
+code included, in this mode. Determinism covers JojoScript's capabilities:
+direct calls to `Date.now()`, `Math.random()`, or I/O outside declared effects
+are not intercepted. See
+[example/src/application/deterministic-example.jojo](example/src/application/deterministic-example.jojo).
+
 ### JavaScript interoperability
 
 A handler is just a normal JavaScript function — sync or async — and can
@@ -739,6 +1400,38 @@ never through `|>`) and also uses pipeline syntax with overlapping names,
 the compiler merges the auto-imported names into the existing import
 instead of emitting a second, colliding import declaration.
 
+### Deterministic serialization
+
+`toJson(value, { indent })` and `fromJson(text)` give JojoScript values one
+canonical JSON form:
+
+```jojo
+import { toJson, fromJson } from "@panagos/jojoscript/runtime"
+
+snapshot := toJson({ name: "John", id: 42, placedAt: new Date(0) })
+// {"id":42,"name":"John","placedAt":{"$type":"Date","value":"1970-01-01T00:00:00.000Z"}}
+
+order := fromJson(snapshot)   // order.placedAt is a Date again
+```
+
+- Object keys are sorted recursively, so equal values always serialize to
+  identical text, whatever order their properties were created in.
+- Values plain JSON would lose or change are written as `{ "$type": ... }`
+  objects: `Date`, `Map`, `Set`, `BigInt`, `undefined`, `NaN`, `Infinity`,
+  `-Infinity`, `-0`, and `Error` (name, message, own fields such as a
+  `ContractError`'s `kind`, and `cause`). `fromJson` restores each exactly.
+- An ordinary object that has its own `$type` key is wrapped, so tags are
+  never ambiguous. Tagged union values are plain objects and serialize as
+  they are. Objects with a `toJSON` method serialize its result.
+- Functions, symbols, and circular structures throw a `TypeError`.
+
+`FileCheckpointStore` writes and reads checkpoints in this format, so
+checkpointed dates, maps, and sets resume as themselves. Checkpoint files
+written as plain JSON before this change still load. Pipelines and execution
+plans have their own canonical form: the [`jojo plan`](#machine-readable-execution-plans)
+JSON. See
+[example/src/application/serialization-example.jojo](example/src/application/serialization-example.jojo).
+
 ### MCP support
 
 JojoScript includes a small MCP-style JSON-RPC runtime for Node.js. It does
@@ -749,18 +1442,121 @@ JavaScript functions.
 import { createMcpServer, runMcpStdioServer } from "jojoscript/runtime"
 
 const server = createMcpServer({
+  name: "example-server",
+  version: "1.0.0",
   tools: {
-    echo: args => ({ content: [{ type: "text", text: args.value }] })
+    echo: {
+      description: "Returns the supplied value.",
+      inputSchema: {
+        type: "object",
+        properties: {
+          value: { type: "string" }
+        },
+        required: ["value"],
+        additionalProperties: false
+      },
+      handler: args => ({
+        content: [{ type: "text", text: args.value }]
+      })
+    }
   }
 })
 
 await runMcpStdioServer(server)
 ```
 
-The server accepts `tools/call` requests and returns JSON-RPC responses on
-standard output. It is intentionally minimal: supporting tool registration,
-request dispatch, structured errors, and the stdin/stdout transport without any
-external NPM packages.
+The server supports the `initialize` handshake (protocol version
+`2025-06-18`), `notifications/initialized`, `ping`, `tools/list`, and
+`tools/call` over newline-delimited JSON-RPC on standard input/output. Clients
+must initialize before making requests. Tools declare an `inputSchema`; calls
+that do not conform are rejected before the handler runs. Handlers return a
+`CallToolResult` object with a `content` array and may return
+`structuredContent` when their tool declares an `outputSchema`. Thrown handler
+errors are returned as `CallToolResult` values with `isError: true`.
+
+It is intentionally minimal: supporting tool registration, schema validation,
+request dispatch, structured errors, and the stdin/stdout transport without
+external NPM packages. It remains an MCP-style runtime until its supported
+protocol surface has been verified against MCP conformance tests and real
+clients.
+
+### A2A client
+
+JojoScript provides a dependency-free A2A 1.0 JSON-RPC client. It discovers a
+remote agent through `/.well-known/agent-card.json`, selects its advertised
+JSON-RPC interface, sends messages, and retrieves/polls task results.
+
+```js
+import { createA2AClient } from "jojoscript/runtime"
+
+const agent = createA2AClient("https://agent.example")
+await agent.discover()
+
+const task = await agent.sendMessage({
+  role: "ROLE_USER",
+  parts: [{ kind: "data", data: { record: { id: "item-1" } } }],
+  metadata: { requestId: "request-1" }
+})
+const result = await agent.waitForTask(task, {
+  pollIntervalMs: 500,
+  maxPollIntervalMs: 5000,
+  timeoutMs: 30000
+})
+```
+
+`sendMessage`, `getTask`, and `cancelTask` map to the A2A `SendMessage`,
+`GetTask`, and `CancelTask` operations. Messages, task IDs, context IDs,
+metadata, typed parts, and artifacts are returned unchanged. `waitForTask`
+accepts either a task ID or an in-progress task, then returns the final task
+for every terminal state—including failed, rejected, canceled,
+input-required, and authentication-required tasks—so callers must inspect the
+state before treating it as successful.
+
+Waiting uses a five-minute default timeout, configurable polling and bounded
+backoff (`pollIntervalMs`, `maxPollIntervalMs`, and `backoffMultiplier`), and
+an optional `AbortSignal`. It rejects with an `A2AError` whose `code` clearly
+identifies timeouts, cancellation, network/HTTP failures, invalid discovery or
+JSON-RPC responses, and remote-agent errors. Pass `bearerToken`, custom
+headers, or an alternate `fetch` function in the client options. Discovery
+validates advertised JSON-RPC endpoints and only permits credential-free HTTP
+or HTTPS URLs.
+
+JojoScript intentionally provides a lightweight A2A client rather than an
+A2A SDK or agent server. Its protocol behavior is covered by local fixtures;
+real-agent interoperability remains a separate integration concern.
+
+### Resumable MCP-to-A2A enrichment
+
+[example/src/application/mcp-a2a-resumable-enrichment.jojo](example/src/application/mcp-a2a-resumable-enrichment.jojo)
+combines the MCP server, A2A client, asynchronous pipelines, and checkpoints
+in one tool. `enrich_records` accepts a batch of `{ id, title }` records,
+delegates enrichment to an A2A agent, checkpoints a completed batch, and
+returns both text content and structured records to the MCP client.
+
+```js
+import {
+  FileCheckpointStore,
+  runMcpStdioServer
+} from "jojoscript/runtime"
+import { createEnrichmentServer } from "./application/mcp-a2a-resumable-enrichment.js"
+
+const server = createEnrichmentServer({
+  agentUrl: process.env.A2A_AGENT_URL,
+  checkpointStore: new FileCheckpointStore(".jojo-checkpoints")
+})
+
+await runMcpStdioServer(server)
+```
+
+Use a stable `jobId` for retries of the same batch. The checkpoint is saved
+only after every record has been enriched, so retrying a completed batch uses
+the saved records without calling the A2A agent again. This is **at-least-once
+delegation**, not exactly-once: if an A2A agent completes a record and the
+process fails before the enclosing batch checkpoint commits, the retry invokes
+that agent again. Remote agents should therefore use idempotency keys derived
+from the record and job ID when duplicate work is unsafe. The matching
+integration test demonstrates both the duplicate-before-commit case and the
+no-repeat-after-commit case.
 
 ## Example
 
@@ -777,6 +1573,10 @@ running the same program with multiple handlers, deterministic testing,
 `parallel()` as an effect, effect visibility in the execution plan, and the
 checkpoint/resume interaction.
 
+[example/src/application/a2a-enrichment.jojo](example/src/application/a2a-enrichment.jojo)
+demonstrates enriching a record through A2A discovery, message dispatch, and
+task polling against a local test agent.
+
 ```bash
 cd example
 npm install
@@ -804,10 +1604,36 @@ const total = users
 boundaries because they need the complete input. See
 [ITERATORS.md](ITERATORS.md) for the complete proposal and memory model.
 
+## Formatting
+
+`jojo format <file.jojo|directory>` reformats files in place and prints each
+file it changed. `--check` changes nothing: it lists the files that need
+formatting and exits with code 1 if there are any, which suits CI and
+generate-format-check loops.
+
+The formatter is line-based, like the compiler: it normalizes indentation and
+whitespace but never rewrites expressions.
+
+- A line that opens brackets indents their contents one level, however many
+  it opens (`posts.reduce((groups, post) => {`).
+- `|>` stages, `requires` / `ensures` clauses, `| Variant` lines, ternary
+  `?` / `:` lines, leading `.method()` chains, and the line after a trailing
+  `:=` or `=>` are indented one extra level. Stages inside a
+  `pipeline name { ... }` block line up with its `source` line.
+- Trailing whitespace is removed, Windows line endings become `\n`, runs of
+  blank lines collapse to one, and the file ends with a single newline.
+- Block comments and template literal text are left untouched.
+
+`formatSource(source, { indent })` is exported from `@panagos/jojoscript`, and
+the VS Code extension's `Format Document` uses the same formatter. Every file
+under `example/src` is kept formatted by a test.
+
 ## Editor support
 
 See [editors/vscode](editors/vscode) for a VS Code extension providing syntax
-highlighting and a `Format Document` provider for `.jojo` files. It is not
+highlighting for all JojoScript syntax (pipelines, stage IDs, named
+pipelines, `match` patterns, tagged unions, effects, purity, and contracts)
+and a `Format Document` provider for `.jojo` files. It is not
 published to the marketplace; package it with `vsce package` and install the
 resulting `.vsix` locally.
 
@@ -815,8 +1641,6 @@ resulting `.vsix` locally.
 
 - **No source maps.** Stack traces from compiled `dist/*.js` point at the
   generated JavaScript, not the original `.jojo` source.
-- **No `jojo fmt` CLI command.** Formatting is only available through the
-  VS Code extension's `Format Document` command, not the `jojo` CLI.
 - **No regex-literal awareness in the lexer.** A regex containing `{`, `}`,
   `(`, or `)` can confuse brace/paren-depth-sensitive transforms (pipelines,
   `match`, destructuring, type annotations). Prefer `new RegExp(...)` in code
@@ -845,8 +1669,7 @@ resulting `.vsix` locally.
   actually consumed.** Constructing a lazy pipeline inside a `handle` block
   without consuming it there (no `toArray`/`toArrayAsync`/`reduce`/...) means
   effects pulled later, outside the block, won't see its handlers.
-- **Only `PARALLEL` is a built-in effect today.** The other capabilities
-  mentioned as a long-term direction (`IO`, `HTTP`, `FILE`, `DATABASE`,
-  `SLEEP`, `TIME`, `RANDOM`, `CHECKPOINT`) are not implemented — user-defined
+- **`PARALLEL`, `TIME`, and `RANDOM` are the only built-in effects.** The
+  other capabilities mentioned as a long-term direction (`IO`, `HTTP`, `FILE`,
+  `DATABASE`, `SLEEP`, `CHECKPOINT`) are not implemented; user-defined
   `effect` declarations cover the same need today.
-