@panagos/jojoscript 0.10.1 → 1.0.0

package/README.md

@@ -1,4 +1,4 @@
-# JojoScript 0.10.1
+# JojoScript 1.0.0
 
 JojoScript is a lightweight language and runtime for data pipelines and agent
 workflows, built on JavaScript.
@@ -22,8 +22,17 @@ npm install @panagos/jojoscript
 
 ```bash
 jojo build <file.jojo|directory> [--out <directory>] [--watch]
-jojo run <file.jojo> [--out <directory>]
-jojo graph <file.jojo> [--format text|dot]
+jojo run <file.jojo> [--out <directory>] [--deterministic [--seed <n>]]
+jojo graph <file.jojo> [--format text|dot] [--pipeline <name>]
+jojo explain <file.jojo> [--format text|json] [--pipeline <name>]
+jojo plan <file.jojo> [--format json] [--pipeline <name>]
+jojo generate <plan.json>
+jojo check <file.jojo|plan.json> [--format text|json] [--machine]
+jojo describe <file.jojo> [--pipeline <name>]
+jojo simulate <file.jojo> [--input <records>] [--format text|json] [--pipeline <name>]
+jojo why <file.jojo> [--format text|json] [--pipeline <name>]
+jojo format <file.jojo|directory> [--check]
+jojo parse <file.jojo> [--json]
 jojo profile <file.jojo>
 jojo resume <file.jojo> [--out <directory>]
 jojo replay <file.jojo> --from <checkpoint> [--out <directory>]
@@ -35,7 +44,17 @@ jojo --help
 - `jojo build src` compiles every `.jojo` file under `src` to `src/dist/*.js` (or `--out <dir>`).
 - `jojo build src --watch` rebuilds whenever a `.jojo` file under `src` changes.
 - `jojo run app.jojo` compiles a single file and immediately runs it with Node — useful for a fast dev loop.
+- `jojo run app.jojo --deterministic --seed 42` runs the program in [deterministic execution mode](#deterministic-execution-mode): seeded `random()`, a fixed clock, sequential `parallel()`, and deterministic effect handlers only.
 - `jojo graph app.jojo` prints a static pipeline graph; use `--format dot` for Graphviz output.
+- `jojo explain app.jojo` describes what each pipeline means (stages, effects, properties) without running it; use `--format json` for tools and LLMs. See [Explaining pipelines](#explaining-pipelines).
+- `jojo plan app.jojo` prints a canonical JSON execution plan, and `jojo generate plan.json` turns a plan back into validated JojoScript. See [Machine-readable execution plans](#machine-readable-execution-plans).
+- `graph`, `explain`, and `plan` accept `--pipeline <name>` to select one pipeline by its (inferred or [declared](#named-pipelines)) name.
+- `jojo check app.jojo` (or `plan.json`) reports structured diagnostics with codes, lines, and suggestions, and exits 1 on errors; `--machine` prints JSON. See [Checking programs](#checking-programs).
+- `jojo describe app.jojo` prints one JSON document describing the pipelines, effects, functions, and types in a file. See [Describing programs](#describing-programs).
+- `jojo simulate app.jojo --input 1000000` estimates item counts, effect calls, and risks for a given input size without running anything. See [Simulating pipelines](#simulating-pipelines).
+- `jojo why app.jojo` answers questions such as "why is this pipeline not deterministic?" in plain language. See [Asking why](#asking-why).
+- `jojo format src` reformats `.jojo` files in place; `--check` lists unformatted files and exits 1. See [Formatting](#formatting).
+- `jojo parse app.jojo --json` lists every syntax construct (imports, effects, types, functions, pipelines, handle blocks, matches) with its line. See [Parsing programs](#parsing-programs).
 - `jojo profile app.jojo` executes a pipeline and prints per-node item counts, duration, and throughput.
 - `jojo resume app.jojo` reuses completed file-backed checkpoints; `jojo replay app.jojo --from name` starts from a selected checkpoint.
 - `jojo test app.jojo` compiles the file and runs its co-located `<name>.test.js` test when present.
@@ -136,6 +155,69 @@ async fn loadUsers() {
 }
 ```
 
+### Pure and effectful functions
+
+`pure` and `effectful` declare a function's role. They compile away to an
+ordinary function, and may follow `export` or surround `async`:
+
+```jojo
+effect storeUser(user)
+
+export pure fn normalizeUser(user) {
+  return { ...user, email: user.email.trim().toLowerCase() }
+}
+
+effectful fn saveUser(user) {
+  return storeUser(user)
+}
+```
+
+The compiler rejects a `pure fn` that uses a declared `effect` or an
+`effectful fn`, whether it calls it or passes it along (`map(saveUser)`):
+
+```
+pure fn "normalizeUser" uses effectful function "saveUser" (line 9)
+```
+
+The check is static and name-based. Strings, comments, property access
+(`user.saveUser`), and recursion are ignored, and calls to unannotated
+functions are allowed. `jojo explain` reports a stage that uses an
+`effectful fn` as `effectful` and the pipeline as non-deterministic, exactly as
+it does for declared effects. See
+[example/src/application/purity-example.jojo](example/src/application/purity-example.jojo).
+
+### Function contracts
+
+A block function can state what it expects and what it guarantees. Each
+clause goes on its own line between the parameter list and the opening `{`:
+
+```jojo
+fn applyDiscount(price, percent)
+  requires price >= 0
+  requires percent >= 0 && percent <= 100
+  ensures result >= 0 && result <= price
+{
+  return price - price * percent / 100
+}
+```
+
+`requires` clauses are checked on entry, in order, before the body runs.
+`ensures` clauses are checked after it, with the return value available as
+`result` (async functions are awaited first). The body runs inside an arrow
+function, so `this` and `arguments` keep referring to the declared function.
+A broken clause throws `ContractError`:
+
+```
+ContractError: Contract violated: applyDiscount requires percent >= 0 && percent <= 100
+```
+
+The error carries `functionName`, `kind` (`"requires"` or `"ensures"`), and
+`condition`, and is non-retryable, so `retry()` never repeats a call that
+broke its contract. Contract clauses keep the source line count. Contracts
+apply to block functions only, and a function using `ensures` should not
+have a parameter named `result`. See
+[example/src/application/contracts-example.jojo](example/src/application/contracts-example.jojo).
+
 ### Optional type annotations
 
 Parameters and return values can be annotated. Annotations are **never
@@ -197,8 +279,77 @@ category := match true {
 }
 ```
 
-`match` is a value-equality construct, not structural pattern matching:
-patterns are plain expressions compared with `===`.
+#### Structured patterns
+
+Object patterns match on the shape of a value. Arms can be separated by
+newlines instead of commas:
+
+```jojo
+message := match result {
+  { status: "ok", value } => value
+  { status: "error", error: { code: 404 } } => "not found"
+  { status: "error", error } => handle(error)
+}
+```
+
+Inside an object pattern:
+
+- a literal or expression (`status: "ok"`, `code: Status.OK`) is compared with `===`;
+- a bare name (`value`) binds that field, and the field must be present;
+- `key: name` binds the field under another name, and `key: { ... }` matches a nested object;
+- quoted keys (`"content-type": "json"`) are supported.
+
+A match with at least one object pattern may omit the `_` arm. If no arm
+matches, it throws `TypeError: No match arm matched the value`, so an
+unexpected shape fails loudly instead of returning `undefined`. A match whose
+patterns are all plain expressions still requires a final `_` arm and
+compiles exactly as before. See
+[example/src/application/structured-match-example.jojo](example/src/application/structured-match-example.jojo).
+
+### Tagged unions
+
+`type` declares a tagged union: a value that is exactly one of a fixed set of
+variants. Each variant has a constructor:
+
+```jojo
+type PaymentResult =
+  | Paid(amount)
+  | Declined(reason)
+  | Pending
+```
+
+becomes:
+
+```js
+const PaymentResult = defineType("PaymentResult", { Paid: ["amount"], Declined: ["reason"], Pending: [] }); const { Paid, Declined, Pending } = PaymentResult
+```
+
+`Paid(40)` returns the frozen plain object `{ tag: "Paid", amount: 40 }`, so
+values serialize to JSON unchanged. Constructors reject the wrong number of
+arguments (`Paid expects 1 argument (amount), got 0`), and fieldless variants
+are called with no arguments (`Pending()`). `export type` exports the type and
+all of its constructors. Variant names must start with an uppercase letter,
+variants and fields must be unique, and no field may be named `tag`.
+
+Constructor patterns match a variant by its tag and bind its fields by
+position, so they also match values decoded from JSON:
+
+```jojo
+message := match result {
+  Paid(amount) => `paid ${amount}`
+  Declined("insufficient funds") => "ask for another card"
+  Declined(reason) => `declined: ${reason}`
+  Pending() => "waiting for the bank"
+}
+```
+
+Pattern arguments can be literals, `_`, bindings, or nested object and
+constructor patterns. Like object patterns, a match over constructor patterns
+may omit `_` and throws `TypeError` when nothing matches. A capitalized call
+such as `Ok(v)` inside a `match` is always a constructor pattern. Avoid naming
+a variant `Error`, since its constructor would shadow JavaScript's `Error` in
+that module. See
+[example/src/application/tagged-unions-example.jojo](example/src/application/tagged-unions-example.jojo).
 
 ### Pipelines
 
@@ -221,6 +372,52 @@ A stage can have no arguments (`users |> unique`) or arguments
 inputs, including `await` expressions and pipelines inside `return`
 statements, function bodies, and object literals.
 
+#### Named pipelines
+
+A `pipeline` block declares a named, reusable pipeline. Its first line names
+the source, and the remaining lines are stages:
+
+```jojo
+export pipeline enrichOrders {
+  source orders
+
+  |> filter(order => order.active)
+  |> map(addVat)
+  |> checkpoint("enriched")
+  |> toArrayAsync
+  |> await
+}
+```
+
+It compiles to a function. An identifier source becomes its single parameter
+(`enrichOrders(orders)`); any other source expression gives a function with
+no parameters. A pipeline with a `|> await` stage (or an awaited source) becomes an
+`async` function; an `await` inside a stage callback does not. Also,
+blank lines inside the block are allowed. Tools see the pipeline under its
+declared name, so `jojo explain app.jojo --pipeline enrichOrders` (and the same
+option on `graph` and `plan`) selects it directly. See
+[example/src/application/named-pipeline-example.jojo](example/src/application/named-pipeline-example.jojo).
+
+#### Stage IDs
+
+A stage can be labelled with a trailing `@id` (letters, digits, `_`, and `-`,
+starting with a letter or `_`):
+
+```jojo
+saved := orders
+  |> filter(order => order.paid)   @paid-only
+  |> map(assignCarrier)             @pick-carrier
+  |> checkpoint("prepared")
+```
+
+The label is removed from the compiled code. It becomes the stage's node id in
+runtime metadata (checkpoint, inspect, parallel, batch, and `jojo profile`),
+in `jojo graph`, in `jojo explain` (`id` and `@pick-carrier` in the text), and
+in `jojo plan` (an `id` field that `jojo generate` writes back). Tools and
+LLMs can therefore refer to "the `@pick-carrier` stage" instead of a position
+in the source. IDs must be unique within a pipeline. They are not part of the
+pipeline's hash, so renaming a label never invalidates saved checkpoints.
+
 #### Placeholder argument
 
 By default the piped value is inserted as the stage's first argument. The
@@ -325,6 +522,381 @@ auto-imported):
 See [ITERATOR_PROPOSAL.md](ITERATOR_PROPOSAL.md) and [ITERATORS.md](ITERATORS.md)
 for the full laziness/memory model.
 
+## Explaining pipelines
+
+`jojo explain` lets a person, a tool, or an LLM ask JojoScript what a program
+means instead of inferring it from source. It is static: nothing is executed.
+
+```jojo
+effect enrichOrder(order)
+
+export async fn enrichPaidOrders(orders) {
+  return handle {
+    enrichOrder: order => ({ ...order, region: "EU" })
+  } {
+    return orders
+      |> filter(order => order.status == "paid")
+      |> parallel(4)
+      |> map(enrichOrder)
+      |> checkpoint("enriched-orders")
+      |> toArrayAsync
+      |> await
+  }
+}
+```
+
+```
+$ jojo explain orders.jojo
+PIPELINE enrichPaidOrders
+
+STAGES
+  1 source       orders
+  2 filter       pure
+  3 parallel     concurrency=4
+  4 map          effectful: enrichOrder
+  5 checkpoint   enriched-orders
+  6 toArrayAsync pure
+  7 await        control
+
+EFFECTS
+  enrichOrder (handler declared)
+
+PROPERTIES
+  lazy: true
+  parallel: true
+  resumable: true
+  deterministic: false
+```
+
+`jojo explain orders.jojo --format json` prints the same information as an
+array of `{ name, id, stages, effects, properties }` objects, and
+`explainPipelines(source)` / `renderExplanationText(explanation)` are exported
+from `@panagos/jojoscript` for programmatic use.
+
+- **name** is inferred: the variable or property a pipeline is assigned to
+  (`paid := orders |> ...`), the `fn` name of an expression function, or the
+  enclosing named function of a `return` pipeline; otherwise `pipeline-N`.
+- **stages** list each stage's type, raw `args`, referenced `effects`, and a
+  `detail`: the source expression, `concurrency=N` for `parallel`, `size=N` for
+  `batch`, the checkpoint name, `control` for `await`/`catch`/`orElse`/`pass`,
+  or `pure`/`effectful: <effects>`.
+- **effects** are the declared effects the pipeline references, each with the
+  same best-effort `staticallyHandled` signal as `jojo graph`.
+- **properties**: `lazy` (no `sort`/`sortBy`/`groupBy`/`partition`
+  materialization boundary), `parallel` (has a `parallel` stage), `resumable`
+  (has a `checkpoint` stage), `deterministic` (references no declared effect).
+
+Purity and determinism only reflect references to declared `effect`s and
+`effectful fn`s (see [Pure and effectful functions](#pure-and-effectful-functions)): a stage
+calling ordinary JavaScript such as `Math.random()` or `fetch()` directly is
+still reported as `pure`. Route external work through effects to make it
+visible. See
+[example/src/application/explain-example.jojo](example/src/application/explain-example.jojo).
+
+## Machine-readable execution plans
+
+`jojo plan` gives every pipeline a stable, simple JSON representation, and
+`jojo generate` turns such a plan back into JojoScript. A tool or an LLM can
+read a program as data, or write a plan and have JojoScript validate it:
+
+```
+Jojo source -> jojo plan -> JSON plan -> LLM
+LLM -> JSON plan -> jojo generate -> validated Jojo program
+```
+
+```jojo
+export fn topCustomers(customers) {
+  return customers
+    |> filter(customer => customer.active)
+    |> sortBy(customer => -customer.spend)
+    |> take(2)
+    |> map(customer => customer.name)
+    |> toArray
+}
+```
+
+```
+$ jojo plan customers.jojo --format=json
+{
+  "version": 1,
+  "pipelines": [
+    {
+      "pipeline": "topCustomers",
+      "id": "…",
+      "stages": [
+        { "type": "source", "expression": "customers" },
+        { "type": "filter", "expression": "customer => customer.active" },
+        { "type": "sortBy", "expression": "customer => -customer.spend" },
+        { "type": "take", "args": ["2"] },
+        { "type": "map", "expression": "customer => customer.name" },
+        { "type": "toArray" }
+      ]
+    }
+  ]
+}
+```
+
+(Printed with two-space indentation; condensed here.)
+
+The plan format, version 1:
+
+| Field | Meaning |
+|---|---|
+| `version` | Always `1` for this format |
+| `pipelines[].pipeline` | Inferred name (same rules as `jojo explain`), or `null` for an unnamed pipeline |
+| `pipelines[].id` | The pipeline's deterministic hash (informational, ignored by `generate`) |
+| `pipelines[].stages[0]` | `{ "type": "source", "expression": "..." }` |
+| `parallel` / `batch` | `concurrency` / `size` as a positive integer |
+| `checkpoint` | `name` as a string |
+| `map`, `filter`, `flatMap`, `tap`, `mapAsync`, `filterAsync`, `find`, `some`, `every`, `sortBy`, `distinctBy`, `groupBy`, `partition`, `catch` | `expression`: the single function argument as source text |
+| any other stage | `args`: argument source strings, omitted when there are none |
+| `.prop` / `.method()` | `{ "type": ".length" }` reads a property; `{ "type": ".trim", "args": [] }` calls it |
+| `id` (any stage after the source) | Optional [stage ID](#stage-ids) such as `"pick-carrier"`, unique within the pipeline |
+
+A stage whose setting is not a literal (`parallel(limit)`) keeps it in `args`
+instead, so a plan is always lossless. Arguments stay source text: the plan
+describes computation without evaluating it.
+
+`jojo generate plan.json` (or `planToJojo(plan)`) emits one
+`name := source |> ...` statement per pipeline. It rejects unknown fields,
+wrong types, misplaced `source` stages, and duplicate names with the exact path
+of the problem (for example
+`Invalid plan: pipelines[0].stages[2].concurrency must be a positive integer`),
+then compiles the program and checks that it parses as JavaScript, without
+running it. The generated program contains only the pipelines: their source
+variables, custom stage functions, and effects must be supplied by
+surrounding code. Validation is not a sandbox: expressions are code, so
+review a generated program as you would any code you did not write.
+
+`planPipelines(source)` and `planToJojo(plan)` are exported from
+`@panagos/jojoscript`. See
+[example/src/application/plan-example.jojo](example/src/application/plan-example.jojo).
+
+## Checking programs
+
+`jojo check` validates a program, or a JSON plan, without running it. It is
+designed for LLM-generated code: every problem is a structured diagnostic with
+a stable code, a line, and a concrete suggestion, instead of a raw stack trace.
+
+```
+$ jojo check generated.jojo
+ERROR E301
+line 8
+Call divide(1, 0) violates "requires b != 0"
+suggestion: Pass arguments to "divide" that satisfy b != 0
+
+1 error, 0 warnings
+```
+
+`--machine` (or `--format json`) prints the same result as JSON:
+
+```json
+{
+  "valid": false,
+  "errors": [
+    {
+      "code": "E301",
+      "severity": "error",
+      "line": 8,
+      "message": "Call divide(1, 0) violates \"requires b != 0\"",
+      "suggestion": "Pass arguments to \"divide\" that satisfy b != 0"
+    }
+  ],
+  "warnings": []
+}
+```
+
+The command exits with code 1 when there is at least one error, so a
+generate-check-fix loop can rely on the exit code.
+
+| Code | Severity | Meaning |
+|---|---|---|
+| `E100` | error | The compiler rejects the program (with its line when known) |
+| `E101` | error | The compiled JavaScript does not parse (checked by Node's own parser) |
+| `W102` | warning | A declared effect is used but no `handle { ... }` in the file handles it; a caller may still install one |
+| `E201` | error | A `pure fn` uses a declared effect or an `effectful fn` |
+| `E301` | error | A call with literal arguments breaks a `requires` clause |
+| `E401` | error | A `match` over a declared tagged union misses a variant and has no `_` arm |
+| `E402` | error | A constructor pattern has a different number of fields than its variant |
+| `E700` | error | `jojo check plan.json`: the plan is invalid (message names the exact field) |
+
+Contract clauses are only evaluated statically when they use nothing but the
+function's parameters, literals, and operators, so `jojo check` never runs
+program code. `checkSource(source)`, `checkPlan(plan)`, and
+`renderCheckText(result)` are exported from `@panagos/jojoscript`. See
+[example/src/application/check-example.jojo](example/src/application/check-example.jojo).
+
+## Describing programs
+
+`jojo describe` prints one self-describing JSON document for a file, so a tool
+or an LLM can learn what a program contains without reading its source:
+
+```json
+{
+  "language": "jojo",
+  "version": "1",
+  "pipelines": [
+    {
+      "name": "openInvoices",
+      "inputs": ["records"],
+      "effects": [],
+      "stages": [
+        { "id": "unpaid", "operation": "filter", "expression": "record => !record.paid" },
+        { "id": "classify", "operation": "map", "expression": "toInvoice" },
+        { "id": "node-3", "operation": "checkpoint", "name": "open-invoices" }
+      ],
+      "properties": { "lazy": true, "parallel": false, "resumable": true, "deterministic": true }
+    }
+  ],
+  "effects": [{ "name": "storeInvoice", "params": ["invoice"], "handled": true }],
+  "functions": [
+    { "name": "toInvoice", "line": 12, "purity": "pure", "requires": ["record.amount >= 0"], "ensures": [] }
+  ],
+  "types": [
+    { "name": "Invoice", "variants": [{ "name": "Open", "fields": ["id", "amount"] }, { "name": "Settled", "fields": ["id"] }] }
+  ]
+}
+```
+
+- `pipelines` uses the same names, effects, and properties as `jojo explain`,
+  and the same stage fields as `jojo plan`. A pipeline whose source is an
+  identifier lists it in `inputs`. A stage's `id` is its
+  [stage ID](#stage-ids), or the runtime node id (`node-3`) used by
+  `jojo graph` and `jojo profile`.
+- `effects` lists declared effects and whether a handler is statically
+  visible in the file.
+- `functions` lists functions that declare a purity or contracts.
+- `types` lists tagged unions with their variants and fields.
+
+`--pipeline <name>` limits `pipelines` to one pipeline. `describeSource(source)`
+is exported from `@panagos/jojoscript`. See
+[example/src/application/describe-example.jojo](example/src/application/describe-example.jojo).
+
+## Parsing programs
+
+`jojo parse <file.jojo> --json` prints the syntax-level view of a file: every
+JojoScript construct the compiler recognizes, with its line, ordered by line.
+Where `jojo describe` summarizes what a program means, `jojo parse` lists what
+it contains, so a tool or an LLM can locate and edit constructs precisely.
+
+```json
+{
+  "language": "jojo",
+  "version": "1",
+  "nodes": [
+    { "kind": "type", "line": 6, "name": "Decision", "exported": true,
+      "variants": [{ "name": "Approved", "fields": ["id"] }, { "name": "Escalated", "fields": ["id", "team"] }] },
+    { "kind": "function", "line": 10, "name": "decide", "params": ["refund"], "exported": false,
+      "async": false, "purity": null, "body": "expression", "requires": [], "ensures": [] },
+    { "kind": "match", "line": 13, "patterns": ["Approved(id)", "Escalated(id, team)"] },
+    { "kind": "pipeline", "line": 20, "name": "routeRefunds", "declared": false, "source": "refunds",
+      "stages": [{ "type": "map", "args": ["decide"], "id": null }] }
+  ]
+}
+```
+
+| Kind | Fields |
+|---|---|
+| `import` | `from`, `names` |
+| `effect` | `name`, `params` |
+| `type` | `name`, `exported`, `variants` (`name`, `fields`) |
+| `function` | `name`, `params` (as written, including defaults), `exported`, `async`, `purity`, `body` (`"block"` or `"expression"`), `requires`, `ensures` |
+| `pipeline` | `name`, `declared` (a `pipeline` block), `source`, `stages` (`type`, `args`, `id`) |
+| `handle` | `handlers` |
+| `match` | `patterns` |
+
+JojoScript has no AST by design: nodes come from the same finders the compiler
+uses, so `parse` never disagrees with what compiles. Invalid syntax is
+reported as a single `jojo:` error line. `parseSource(source)` is exported
+from `@panagos/jojoscript`. See
+[example/src/application/parse-example.jojo](example/src/application/parse-example.jojo).
+
+## Simulating pipelines
+
+`jojo simulate` answers "what would this pipeline do with N records?" before
+anything runs. No stage or effect is executed:
+
+```
+$ jojo simulate prices.jojo --input 250000 --pipeline pricedProducts
+SIMULATION pricedProducts
+
+INPUT
+  250,000 records
+
+OPERATIONS
+  2 filter     250,000 items in
+  3 parallel   ≤250,000 items in, 16 workers
+  4 map        ≤250,000 items in, ≤250,000 calls to fetchPrice
+  5 checkpoint ≤250,000 items in, saved as "priced"
+
+POTENTIAL PROBLEMS
+  ⚠ stage 4: up to 250,000 calls to effect "fetchPrice"
+  ⚠ stage 4: effect "fetchPrice" runs before checkpoint "priced": items processed before a failure are fetched again on retry (at-least-once)
+```
+
+Stages are numbered as in `jojo explain`, where the source is stage 1.
+Item counts flow through the stages as upper bounds (`≤` once a filtering
+stage such as `filter`, `unique`, or `distinctBy` has run). `take`, `skip`,
+`chunk`, `batch`, and `window` change them exactly; `flatMap` and `groupBy`
+make them unknown; `count`, `first`, `find`, `some`, `every`, and `reduce`
+produce one result. A stage that uses a declared effect or an `effectful fn`
+makes one call per item it receives.
+
+Potential problems reported:
+
+- 1,000 or more calls to one effect;
+- an effect that runs before a checkpoint, and so may run again for items
+  processed before a failure (see
+  [Effects and checkpoint/resume](#effects-and-checkpointresume));
+- a pipeline with effects but no checkpoint at all;
+- `sort`, `sortBy`, `groupBy`, or `partition` holding 10,000 or more items in
+  memory.
+
+The input defaults to 1,000 records. `--format json` prints the simulation as
+data, and `--pipeline <name>` selects one pipeline. `simulatePipelines(source,
+{ input })` and `renderSimulationText(simulation)` are exported from
+`@panagos/jojoscript`. See
+[example/src/application/simulate-example.jojo](example/src/application/simulate-example.jojo).
+
+## Asking why
+
+`jojo why` turns the static analysis into plain-language answers, numbered
+like `jojo explain` (the source is stage 1):
+
+```
+$ jojo why onboarding.jojo --pipeline verifiedCustomers
+WHY verifiedCustomers
+
+Why is this pipeline not deterministic?
+→ Stage 3 (mapAsync) uses effect verifyEmail; stage 5 (mapAsync) uses effectful function sendWelcome.
+
+Why is this pipeline lazy?
+→ Every stage passes items on one at a time, so no stage waits for the complete input.
+
+Why is this pipeline resumable?
+→ Checkpoint "verified" (stage 4) saves its items once they all complete; a rerun restores them instead of repeating stages 2-3.
+
+Why can't stage 3 be moved before stage 2?
+→ Stage 3 (mapAsync) uses effect verifyEmail and stage 2 (filter) decides which items reach it: moving it earlier would also run it for items that stage 2 drops.
+
+Why does checkpoint "verified" not guarantee replay safety?
+→ Effect verifyEmail runs before it (stage 3): if a run fails before "verified" completes, it runs again on retry (at-least-once, not exactly-once).
+
+Why can stage 5 repeat after a failure?
+→ Stage 5 (mapAsync) uses effectful function sendWelcome after the last checkpoint "verified": a rerun restores the checkpoint and runs it again for every restored item.
+```
+
+Every pipeline gets answers about determinism, laziness, and resumability.
+Additional questions appear when they apply: an effectful stage behind a
+filtering stage (`filter`, `filterAsync`, `unique`, `distinctBy`, `take`,
+`skip`) cannot be moved before it, a `parallel` stage explains what it
+parallelizes, and checkpoints explain which effects can run again on retry.
+Answers only reflect declared effects and `effectful fn`s. `--format json`
+prints `[{ name, answers: [{ question, answer }] }]`, and
+`whyPipelines(source)` / `renderWhyText(why)` are exported from
+`@panagos/jojoscript`. See
+[example/src/application/why-example.jojo](example/src/application/why-example.jojo).
+
 ## Effect Handlers
 
 > This is a deliberately simplified effect-handler model, not full
@@ -704,6 +1276,44 @@ interface (`record`/`replayNext`), mirroring `CheckpointStore` — an
 extension point for a future backend, not a full production-grade
 persistence system.
 
+### Deterministic execution mode
+
+`now()` and `random()` are built-in capabilities, like `parallel()`. Outside
+any handler they behave like `Date.now()` and `Math.random()`, and a
+`handle { TIME: ..., RANDOM: ... }` block can replace them:
+
+```jojo
+import { now, random } from "@panagos/jojoscript/runtime"
+
+order := { id: Math.round(random() * 1000), createdAt: new Date(now()) }
+```
+
+`deterministic(fn, { seed, startTime })` runs `fn` so that the same inputs
+always produce the same outputs:
+
+- `random()` is a seeded generator (`seed` defaults to `1`);
+- `now()` starts at `startTime` (default `0`) and advances 1ms per read;
+- `parallel()` stages process one item at a time, in order;
+- every declared effect must resolve to a handler declared deterministic
+  (`{ handle, deterministic: true }`, or `replayHandlers(...)`); any other
+  handler throws a non-retryable `NondeterministicEffectError` before it runs;
+- checkpoints record `now()` as their `createdAt`, so reruns write identical
+  checkpoints.
+
+```js
+import { deterministic } from "@panagos/jojoscript/runtime"
+
+const first = await deterministic(() => generateOrders(3), { seed: 2026 })
+const second = await deterministic(() => generateOrders(3), { seed: 2026 })
+// first and second are deep-equal
+```
+
+`jojo run app.jojo --deterministic [--seed <n>]` runs a whole program, top-level
+code included, in this mode. Determinism covers JojoScript's capabilities:
+direct calls to `Date.now()`, `Math.random()`, or I/O outside declared effects
+are not intercepted. See
+[example/src/application/deterministic-example.jojo](example/src/application/deterministic-example.jojo).
+
 ### JavaScript interoperability
 
 A handler is just a normal JavaScript function — sync or async — and can
@@ -790,6 +1400,38 @@ never through `|>`) and also uses pipeline syntax with overlapping names,
 the compiler merges the auto-imported names into the existing import
 instead of emitting a second, colliding import declaration.
 
+### Deterministic serialization
+
+`toJson(value, { indent })` and `fromJson(text)` give JojoScript values one
+canonical JSON form:
+
+```jojo
+import { toJson, fromJson } from "@panagos/jojoscript/runtime"
+
+snapshot := toJson({ name: "John", id: 42, placedAt: new Date(0) })
+// {"id":42,"name":"John","placedAt":{"$type":"Date","value":"1970-01-01T00:00:00.000Z"}}
+
+order := fromJson(snapshot)   // order.placedAt is a Date again
+```
+
+- Object keys are sorted recursively, so equal values always serialize to
+  identical text, whatever order their properties were created in.
+- Values plain JSON would lose or change are written as `{ "$type": ... }`
+  objects: `Date`, `Map`, `Set`, `BigInt`, `undefined`, `NaN`, `Infinity`,
+  `-Infinity`, `-0`, and `Error` (name, message, own fields such as a
+  `ContractError`'s `kind`, and `cause`). `fromJson` restores each exactly.
+- An ordinary object that has its own `$type` key is wrapped, so tags are
+  never ambiguous. Tagged union values are plain objects and serialize as
+  they are. Objects with a `toJSON` method serialize its result.
+- Functions, symbols, and circular structures throw a `TypeError`.
+
+`FileCheckpointStore` writes and reads checkpoints in this format, so
+checkpointed dates, maps, and sets resume as themselves. Checkpoint files
+written as plain JSON before this change still load. Pipelines and execution
+plans have their own canonical form: the [`jojo plan`](#machine-readable-execution-plans)
+JSON. See
+[example/src/application/serialization-example.jojo](example/src/application/serialization-example.jojo).
+
 ### MCP support
 
 JojoScript includes a small MCP-style JSON-RPC runtime for Node.js. It does
@@ -962,10 +1604,36 @@ const total = users
 boundaries because they need the complete input. See
 [ITERATORS.md](ITERATORS.md) for the complete proposal and memory model.
 
+## Formatting
+
+`jojo format <file.jojo|directory>` reformats files in place and prints each
+file it changed. `--check` changes nothing: it lists the files that need
+formatting and exits with code 1 if there are any, which suits CI and
+generate-format-check loops.
+
+The formatter is line-based, like the compiler: it normalizes indentation and
+whitespace but never rewrites expressions.
+
+- A line that opens brackets indents their contents one level, however many
+  it opens (`posts.reduce((groups, post) => {`).
+- `|>` stages, `requires` / `ensures` clauses, `| Variant` lines, ternary
+  `?` / `:` lines, leading `.method()` chains, and the line after a trailing
+  `:=` or `=>` are indented one extra level. Stages inside a
+  `pipeline name { ... }` block line up with its `source` line.
+- Trailing whitespace is removed, Windows line endings become `\n`, runs of
+  blank lines collapse to one, and the file ends with a single newline.
+- Block comments and template literal text are left untouched.
+
+`formatSource(source, { indent })` is exported from `@panagos/jojoscript`, and
+the VS Code extension's `Format Document` uses the same formatter. Every file
+under `example/src` is kept formatted by a test.
+
 ## Editor support
 
 See [editors/vscode](editors/vscode) for a VS Code extension providing syntax
-highlighting and a `Format Document` provider for `.jojo` files. It is not
+highlighting for all JojoScript syntax (pipelines, stage IDs, named
+pipelines, `match` patterns, tagged unions, effects, purity, and contracts)
+and a `Format Document` provider for `.jojo` files. It is not
 published to the marketplace; package it with `vsce package` and install the
 resulting `.vsix` locally.
 
@@ -973,8 +1641,6 @@ resulting `.vsix` locally.
 
 - **No source maps.** Stack traces from compiled `dist/*.js` point at the
   generated JavaScript, not the original `.jojo` source.
-- **No `jojo fmt` CLI command.** Formatting is only available through the
-  VS Code extension's `Format Document` command, not the `jojo` CLI.
 - **No regex-literal awareness in the lexer.** A regex containing `{`, `}`,
   `(`, or `)` can confuse brace/paren-depth-sensitive transforms (pipelines,
   `match`, destructuring, type annotations). Prefer `new RegExp(...)` in code
@@ -1003,7 +1669,7 @@ resulting `.vsix` locally.
   actually consumed.** Constructing a lazy pipeline inside a `handle` block
   without consuming it there (no `toArray`/`toArrayAsync`/`reduce`/...) means
   effects pulled later, outside the block, won't see its handlers.
-- **Only `PARALLEL` is a built-in effect today.** The other capabilities
-  mentioned as a long-term direction (`IO`, `HTTP`, `FILE`, `DATABASE`,
-  `SLEEP`, `TIME`, `RANDOM`, `CHECKPOINT`) are not implemented — user-defined
+- **`PARALLEL`, `TIME`, and `RANDOM` are the only built-in effects.** The
+  other capabilities mentioned as a long-term direction (`IO`, `HTTP`, `FILE`,
+  `DATABASE`, `SLEEP`, `CHECKPOINT`) are not implemented; user-defined
   `effect` declarations cover the same need today.