@smthrs/plan 0.0.0-stage → 1.0.0-rc.13

package/docs/guides/diff-two-plans.md

@@ -0,0 +1,310 @@
+---
+title: "Diff two plans"
+description: "Compare a re-plan against the plan before it: what the verdict is, how a re-key is attributed to the field that caused it, how a step whose positional id moved is paired by its key, and why the report is in no digest."
+sidebar:
+  order: 5
+---
+
+"What is different about this run" is a set comparison, not archaeology.
+`PlanDiff.diff` is that comparison as a value.
+
+## Before you start
+
+The examples run with `node <file>.ts` on Node.js 26.4.0 or later and need no
+engine, database, or jj. Compiling a plan needs Effect's `Crypto` service, which
+`@effect/platform-node` supplies. In a project that installed `@smthrs/flows`,
+add this package and `@smthrs/flow` at the version of `@smthrs/flows` you have,
+plus the platform package:
+
+```bash
+npm install @smthrs/plan@1.0.0-rc.13 @smthrs/flow@1.0.0-rc.13 @effect/platform-node
+```
+
+The outputs below are real output from the release that install line names.
+
+## Compare a re-plan
+
+Create `diff.ts`. It compiles the two drafts from
+[Compile drafts into a plan](./compile-a-plan.md), edits one field of one
+draft's body, compiles again, and compares:
+
+```ts
+import * as NodeCrypto from "@effect/platform-node/NodeCrypto"
+import * as KeyMaterial from "@smthrs/plan/KeyMaterial"
+import * as Plan from "@smthrs/plan/Plan"
+import * as PlanDiff from "@smthrs/plan/PlanDiff"
+import * as Effect from "effect/Effect"
+
+const readPr: Plan.NodeDraft = {
+  id: "read-pr",
+  material: {
+    version: KeyMaterial.version,
+    kind: "sealed",
+    body: { action: "read-pr", pr: 4821 },
+    inputs: [],
+    layers: [],
+    capabilities: ["net:get"]
+  },
+  effects: { reads: [], writes: ["pr.json"], boundaryMode: "hard" }
+}
+
+const runTests: Plan.NodeDraft = {
+  id: "run-tests",
+  material: {
+    version: KeyMaterial.version,
+    kind: "sealed",
+    body: { action: "run-tests" },
+    inputs: [{ _tag: "Ref", from: "read-pr", path: [] }],
+    layers: [],
+    capabilities: []
+  },
+  effects: { reads: ["pr.json"], writes: ["report.json"], boundaryMode: "hard" }
+}
+
+const compile = (nodes: ReadonlyArray<Plan.NodeDraft>) =>
+  Plan.compile({ planId: "review-4821", flow: "example/Review", nodes })
+
+export const compare = Effect.gen(function*() {
+  const before = yield* compile([readPr, runTests])
+  const after = yield* compile([
+    readPr,
+    { ...runTests, material: { ...runTests.material, body: { action: "run-tests", jobs: 4 } } }
+  ])
+  const changes: PlanDiff.PlanDiff = PlanDiff.diff(before, after)
+  return changes.rekeyed.map((entry) => `${entry.id}: ${entry.changed.join(", ")}`)
+}).pipe(Effect.provide(NodeCrypto.layer))
+
+console.log(await Effect.runPromise(compare))
+```
+
+```bash
+node diff.ts
+```
+
+```text
+[ 'run-tests: body' ]
+```
+
+`compare` answers `["run-tests: body"]`. The edit moved one field of one node's
+body, so that node re-keyed and the report says which field did it.
+
+## The verdict is the key
+
+Two nodes with the same key are the same step, under the same id or a moved
+one. Nothing else needs saying, and `diff` says nothing else. Four lists come
+back:
+
+| Field       | Contents                                                              |
+| ----------- | --------------------------------------------------------------------- |
+| `added`     | Node ids in `next` that pair with no node in `previous`.              |
+| `removed`   | Node ids in `previous` that pair with no node in `next`.              |
+| `rekeyed`   | Nodes paired by id whose key moved, with `from`, `to`, and `changed`. |
+| `unchanged` | Node ids in `next` paired with a node of the same key.                |
+
+## The attribution is for a human
+
+`Rekeyed.changed` is a list of field labels: `["body", "input[1]"]`. It is
+derived by comparing declarations field by field, and it is deliberately part of
+no digest. Changing how the report reads can never change what a plan hashes to.
+
+The labels mirror the hashed declaration: the effect tier `kind`, the fields
+the hashed material body folds, and the input positions:
+
+```text
+kind  body  layers  capabilities  effects  version  nondeterministic  placement  input[n]
+```
+
+An `input[n]` entry appears when that input's declaration changed, and also when
+the declaration is unchanged but the node it references itself re-keyed. A node
+re-keyed purely by an upstream edit is therefore attributed to the input
+position that references it, even behind an unprojected `Pending`, rather than
+reported as nothing changed.
+
+`changed` is empty only when none of the compared fields moved, which means the
+key moved for a reason the comparison does not model. Treat an empty `changed`
+on a re-keyed node as a bug report, not as noise.
+
+## Node ids are positional
+
+A node id is the node's structural address in the body, such as
+`root.flow.andThen`, not the name of its step. Adding a step to a flow can move
+the steps around it to new addresses while their keys stay the same. `diff`
+therefore pairs nodes in three passes, each over the nodes no earlier pass
+paired:
+
+1. The same id with the same key: unchanged.
+2. The same key under a different id: the same step at a new address, reported
+   as unchanged under its id in `next`.
+3. The same id with a different key: re-keyed.
+
+Whatever is left in `next` is `added`, and whatever is left in `previous` is
+`removed`. `added`, `rekeyed`, and `unchanged` name nodes by their id in
+`next`; `removed` names nodes by their id in `previous`.
+
+Create `inserted.ts`. It compiles two steps, then the same two steps with a
+third inserted before them, so every id shifts by one:
+
+```ts
+import * as NodeCrypto from "@effect/platform-node/NodeCrypto"
+import * as KeyMaterial from "@smthrs/plan/KeyMaterial"
+import * as Plan from "@smthrs/plan/Plan"
+import * as PlanDiff from "@smthrs/plan/PlanDiff"
+import * as Effect from "effect/Effect"
+
+/** A step whose id is its position, as a flow's structural ids are. */
+const step = (position: number, action: string): Plan.NodeDraft => ({
+  id: `step-${position}`,
+  material: {
+    version: KeyMaterial.version,
+    kind: "sealed",
+    body: { action },
+    inputs: [],
+    layers: [],
+    capabilities: []
+  },
+  effects: { reads: [], writes: [], boundaryMode: "hard" }
+})
+
+const compile = (actions: ReadonlyArray<string>) =>
+  Plan.compile({
+    planId: "checks-1",
+    flow: "example/Checks",
+    nodes: actions.map((action, index) => step(index, action))
+  })
+
+const program = Effect.gen(function*() {
+  const previous = yield* compile(["lint", "test"])
+  const next = yield* compile(["format", "lint", "test"])
+  console.log(PlanDiff.diff(previous, next))
+}).pipe(Effect.provide(NodeCrypto.layer))
+
+await Effect.runPromise(program)
+```
+
+```bash
+node inserted.ts
+```
+
+```text
+{
+  added: [ 'step-0' ],
+  removed: [],
+  rekeyed: [],
+  unchanged: [ 'step-1', 'step-2' ]
+}
+```
+
+Only the inserted `format` step is new. `lint` and `test` moved from `step-0`
+and `step-1` to `step-1` and `step-2` with the keys they had, so they come back
+as `unchanged`.
+
+## Comparison uses the key's own JSON semantics
+
+Each compared field is projected through the same JSON mirror the keys are
+derived from, so two `Date` bodies a generation apart attribute to `body` rather
+than to nothing. The projection runs no accessor.
+
+A field the projection refuses compares by an identity token scoped to that node
+and field. The same value stays equal to itself, two distinct refused values
+never compare equal, and `diff` stays a total function even for a value
+canonical serialization would reject. `diff` never fails and never needs a
+service.
+
+## Reporting a re-plan
+
+The pairing this is built for is a re-plan of the same flow: compile the flow
+again against the current source and compare the result with the plan an
+operator last saw. `added` and `removed` are the shape change, `rekeyed` is the
+work that will not come from cache, and `unchanged` is everything a cache hit
+will skip. That reading holds when a step is added or removed and ids move,
+because `diff` pairs a moved step by its key, as
+[Node ids are positional](#node-ids-are-positional) shows.
+
+On a body built with `Flow.make`, `added` and `rekeyed` also list structural
+nodes: each `AndThen` that sequences steps, and the `FlowCall` at `root`. Their
+keys fold their children's keys, so they move whenever a child does, and they
+run no work. Count steps as the entries whose material body names an `action`.
+
+Create `replan.ts`. It appends a `Summarize` step to a two-step review and
+compares the two plans:
+
+```ts
+import * as NodeCrypto from "@effect/platform-node/NodeCrypto"
+import { Action, Flow, Graph } from "@smthrs/flow"
+import { Node } from "@smthrs/plan"
+import * as Plan from "@smthrs/plan/Plan"
+import * as PlanDiff from "@smthrs/plan/PlanDiff"
+import * as Effect from "effect/Effect"
+import * as Schema from "effect/Schema"
+
+const ReadPr = Action.make("review/ReadPr", { payload: { pr: Schema.Number }, success: Schema.String })
+const RunTests = Action.make("review/RunTests", { payload: { diff: Schema.String }, success: Schema.String })
+const Summarize = Action.make("review/Summarize", { payload: { report: Schema.String }, success: Schema.String })
+
+const review = (summarize: boolean) =>
+  Flow.make("review/Review", {
+    payload: { pr: Schema.Number },
+    success: Schema.String,
+    body: (payload) => {
+      const tested = ReadPr.call(payload).pipe(Node.bindPlanned((diff) => RunTests.call({ diff })))
+      return summarize ? tested.pipe(Node.bindPlanned((report) => Summarize.call({ report }))) : tested
+    }
+  })
+
+const compile = (summarize: boolean) =>
+  Plan.compile({
+    planId: "review-4821",
+    flow: "review/Review",
+    nodes: Graph.drafts(Graph.build(review(summarize), { pr: 4821 }))
+  })
+
+/** A step is a node whose material body names an action; the rest only compose steps. */
+const isStep = (node: Plan.PlanNode) => {
+  const body = node.material.body
+  return typeof body === "object" && body !== null && "action" in body
+}
+
+const program = Effect.gen(function*() {
+  const previous = yield* compile(false)
+  const next = yield* compile(true)
+  const changes = PlanDiff.diff(previous, next)
+  const steps = new Set(next.nodes.filter(isStep).map((node) => node.id))
+  console.log({
+    added: changes.added,
+    rekeyed: changes.rekeyed.map((entry) => entry.id),
+    addedSteps: changes.added.filter((id) => steps.has(id)),
+    rekeyedSteps: changes.rekeyed.filter((entry) => steps.has(entry.id)).map((entry) => entry.id)
+  })
+}).pipe(Effect.provide(NodeCrypto.layer))
+
+await Effect.runPromise(program)
+```
+
+```bash
+node replan.ts
+```
+
+```text
+{
+  added: [ 'root.flow.andThen', 'root.flow.then' ],
+  rekeyed: [ 'root.flow', 'root' ],
+  addedSteps: [ 'root.flow.then' ],
+  rekeyedSteps: []
+}
+```
+
+One step was added: `Summarize`, at `root.flow.then`. `ReadPr` and `RunTests`
+moved under the new `AndThen` with the keys they had, so they are `unchanged`.
+The other three entries are structure: two `AndThen` nodes and the `FlowCall`.
+
+The command line prints the same comparison through
+[`smthrs flow plan`](/cli/flow), and the plan card it renders is what
+[`smthrs approvals approve`](/cli/approvals) binds a decision to.
+
+## Next
+
+- [Inspect the plan a body builds](../../../flow/docs/guides/inspect-the-plan.md):
+  build a flow's graph and read its nodes before anything runs.
+- [Step keys](../concepts/step-keys.md): why an upstream edit re-keys a
+  downstream node whose own declaration did not change.
+- [The plan value](../concepts/plan-value.md): the digests a diff sits beside.

package/docs/installation.md

@@ -0,0 +1,75 @@
+---
+title: "Installation"
+description: "Install @smthrs/plan and satisfy its peer requirements."
+sidebar:
+  order: 1
+---
+
+## Requirements
+
+- Node.js 26.4.0 or later.
+- `effect` `4.0.5`. The package's schemas, layers, and error classes are
+  Effect values, so it uses the one `effect` your application installs.
+
+## Install
+
+```bash
+npm install @smthrs/plan@next effect
+```
+
+The `next` tag carries the 1.0 release candidates.
+
+That is enough to build node graphs, compile plans, and diff them. Compiling
+asks for Effect's `Crypto` service, which a platform package supplies:
+
+```bash
+pnpm add @effect/platform-node @effect/platform-node-shared
+```
+
+`@smthrs/crypto`, `@smthrs/canonical` and [`@smthrs/keys`](/api/keys) arrive as
+dependencies of this package; you do not install them yourself unless you use
+them directly.
+
+## What persistence adds
+
+This package performs no I/O. Recording a compiled plan is
+[`@smthrs/plan-store`](/api/plan-store), which owns the SQL and brings
+[`@smthrs/database`](/api/database) with it.
+
+## Import forms
+
+The root entry point re-exports every module as a namespace:
+
+```ts
+import { FileSet, Node, Plan, PlanDiff } from "@smthrs/plan"
+```
+
+Each module is also its own entry point. Prefer this form: it keeps a bundle to
+the modules you actually reach for.
+
+```ts
+import * as Node from "@smthrs/plan/Node"
+import * as Plan from "@smthrs/plan/Plan"
+```
+
+Both forms resolve to the same values.
+
+## What you cannot import
+
+`@smthrs/plan/internal/*` is blocked by the export map, and so is
+`@smthrs/plan/<Module>/index`. Node reports `ERR_PACKAGE_PATH_NOT_EXPORTED` for
+either.
+
+`@smthrs/plan/test/PlanFixtures` is the exception that is meant for test code:
+it builds drafts and compiles them, and
+[`@smthrs/plan-store`](/api/plan-store) drives the same fixtures against the
+persisted form.
+
+## Browser support
+
+The package is browser-safe: it resolves no `node:` built-in and binds no
+database. Compiling, diffing, and building node graphs all work in a browser.
+
+## Next
+
+[Quickstart](./quickstart.md) compiles a plan, appends to it, and diffs it.

package/docs/quickstart.md

@@ -0,0 +1,128 @@
+---
+title: "Quickstart"
+description: "Compile a two-node plan, read its step keys, append a third node at the next generation, and diff the two graphs."
+sidebar:
+  order: 2
+---
+
+This quickstart takes one plan from drafts to a keyed graph and grows it. By the
+end you will have a compiled graph with real step keys, a third node added at
+generation 1 with the first two untouched, and a diff that says exactly what
+changed.
+
+Nothing here executes a node or writes a row. A plan is inert: persisting one is
+[`@smthrs/plan-store`](/api/plan-store), and driving one is
+[`@smthrs/engine-store`](/api/engine-store)'s `PlanScheduler`.
+
+## Prerequisites
+
+- Node.js 26.4.0 or later.
+- A project with the packages installed: `npm install @smthrs/plan@next @effect/platform-node effect`.
+
+`@effect/platform-node` supplies Effect's `Crypto` service, which is the only
+thing compiling a plan asks for.
+
+## Declare two nodes
+
+Create `quickstart.ts`. A `Plan.NodeDraft` is a node without its key: an id, the
+key material that decides its identity, and the file effects it declares.
+
+```ts
+import * as KeyMaterial from "@smthrs/plan/KeyMaterial"
+import * as Plan from "@smthrs/plan/Plan"
+
+/** Everything hashed about one node. `sealed` is the tier that may be reused across runs. */
+const material = (body: unknown, inputs: ReadonlyArray<KeyMaterial.InputRef> = []): KeyMaterial.KeyMaterial => ({
+  version: KeyMaterial.version,
+  kind: "sealed",
+  body,
+  inputs,
+  layers: [],
+  capabilities: []
+})
+
+/** Two steps: the second consumes the first's result and reads the file it wrote. */
+const drafts: ReadonlyArray<Plan.NodeDraft> = [
+  {
+    id: "read-pr",
+    material: material({ action: "read-pr", pr: 4821 }),
+    effects: { reads: [], writes: ["pr.json"], boundaryMode: "hard" }
+  },
+  {
+    id: "run-tests",
+    material: material({ action: "run-tests" }, [{ _tag: "Ref", from: "read-pr", path: [] }]),
+    effects: { reads: ["pr.json"], writes: ["report.json"], boundaryMode: "hard" }
+  }
+]
+```
+
+The `Ref` input is the only place `run-tests` names `read-pr`. That one
+declaration becomes both the hashed dependency and the graph edge, so an edge
+and a key can never disagree about what a node consumes.
+
+## Compile, append, diff
+
+```ts
+import * as NodeCrypto from "@effect/platform-node/NodeCrypto"
+import * as PlanDiff from "@smthrs/plan/PlanDiff"
+import * as Effect from "effect/Effect"
+
+const program = Effect.gen(function*() {
+  const plan = yield* Plan.compile({ planId: "review-4821", flow: "example/Review", nodes: drafts })
+  console.log(plan.digest)
+
+  const grown = yield* Plan.append(plan, [{
+    id: "post-comment",
+    material: material({ action: "post-comment" }, [{ _tag: "Pending", from: "run-tests" }]),
+    effects: { reads: ["report.json"], writes: [], boundaryMode: "hard" }
+  }])
+
+  for (const node of grown.nodes) {
+    console.log(node.generation, node.id, node.key, node.dependsOn)
+  }
+  console.log(grown.generation, grown.baseDigest === plan.digest)
+  console.log(PlanDiff.diff(plan, grown).added)
+})
+
+await Effect.runPromise(program.pipe(Effect.provide(NodeCrypto.layer), Effect.orDie))
+```
+
+Run the file with your TypeScript runner. The digests on your machine match
+these, because a plan is a pure function of its declarations:
+
+```text
+key1_61edccc875ff69b798e0c3874d167f65ff0a250b9fe67d27558b854fa2832302
+0 read-pr key1_6353326ab2a03804d6acfe916debc4f04f09f9deb8f0e93c1ebea3cdd983db18 []
+0 run-tests key1_e58229e3be2f6abf6f818baf8577fe7045edba6f5e88097b633898478da93985 [ 'read-pr' ]
+1 post-comment key1_0ac2aef1d5d4226849c6009450ad987b47dc02cf39d39ab975db3c673698241d [ 'run-tests' ]
+1 true
+[ 'post-comment' ]
+```
+
+## What just happened
+
+`Plan.compile` put the drafts in topological order, substituted each dependency
+reference for the already computed key of the node it names, annotated the
+overlap between write sets, and derived the plan digest. It read no file, no
+clock, and no network: the only service it asked for was `Crypto`.
+
+`Plan.append` then produced generation 1. The two original nodes kept their id,
+key, edges, and generation byte for byte, and `post-comment` arrived pre-keyed
+against them. The `true` line is `baseDigest` still naming the digest a human
+would have approved, even though `digest` has moved on.
+
+Read `post-comment`'s edge: it depends on `run-tests`, which it never consumed a
+value from. Two separate declarations put it there. Its `Pending` input is an
+ordering reference, and it reads `report.json`, which `run-tests` writes, so the
+reader-after-writer pass would have ordered it behind the producer anyway.
+
+## Next steps
+
+- [The plan value](./concepts/plan-value.md): generations, digests, and why a
+  plan grows instead of being rewritten.
+- [Step keys](./concepts/step-keys.md): what goes into a key, what deliberately
+  does not, and why invalidation is re-keying.
+- [Diff two plans](./guides/diff-two-plans.md): report what a re-plan changed,
+  attributed to the field that changed it.
+- [Persist a plan](../../plan-store/docs/guides/persist-a-plan.md): the
+  append-only store, in production shape.

package/docs/testing.md

@@ -0,0 +1,73 @@
+---
+title: "Testing"
+description: "How to test code that builds plans: the one service to provide, a real in-memory store, and the assertions worth writing."
+---
+
+## Testing code that builds plans
+
+Compiling asks for Effect's `Crypto` service and nothing else, so a plan test
+needs one layer:
+
+```ts
+import * as NodeCrypto from "@effect/platform-node/NodeCrypto"
+import type * as Crypto from "effect/Crypto"
+import * as Effect from "effect/Effect"
+
+/** Provides concrete Node cryptography to a test Effect. */
+export const withCrypto = <A, E>(effect: Effect.Effect<A, E, Crypto.Crypto>): Effect.Effect<A, E> =>
+  Effect.provide(effect, NodeCrypto.layer)
+
+/** The same, for an Effect expected to fail: yields the typed error. */
+export const withCryptoFailure = <A, E>(effect: Effect.Effect<A, E, Crypto.Crypto>): Effect.Effect<E, A> =>
+  withCrypto(Effect.flip(effect))
+```
+
+Those two helpers are the whole harness. No clock, no filesystem, and no fake
+anything: a plan is a pure function of its declarations, so a plan test is an
+ordinary value test.
+
+Testing persistence is [`@smthrs/plan-store`](/api/plan-store)'s
+[testing page](../../plan-store/docs/testing.md). It drives the same
+fixtures this package ships at `@smthrs/plan/test/PlanFixtures`, so a draft
+builder is written once and used on both sides.
+
+### Assert on keys, not on shape
+
+A plan's value is that identity is computable. The assertions that earn their
+place say so:
+
+- An edit re-keys its node and its dependent cone, and nothing else.
+- A rename re-keys nothing, because ids are lookup addresses.
+- An ordering edge, a priority change, or a conflict annotation leaves every key
+  where it was.
+- Compiling the same drafts twice produces the same digest.
+
+### Mutate the caller's draft after compiling
+
+A compiled plan is a deep-frozen snapshot. The cheapest proof is to compile,
+mutate the object you passed in, and assert the stored material and the plan
+digest did not move. The values worth trying are a `Date`, a `URL`, and an
+object with a custom `toJSON`, because each one keys through a different path.
+
+### Keep fixtures out of test modules
+
+Importing a `*.test.ts` module registers its suites in the importer, so shared
+fixtures belong in an ordinary module. When one suite pulls a helper out of
+another suite's file, it re-runs every case in that file, and a fast suite
+inherits a slow one's runtime and its timeouts for work it never asked for.
+
+### Keep resource checks independent of wall time
+
+The package runs files serially, including the dense compiler cases with their
+original graph sizes, digest and edge-order assertions. FileSet's synchronous
+adversarial matcher tests charge thread CPU time against the existing 100 ms
+budget, excluding descheduling and work on other threads.
+
+The planned-value lint test builds a dedicated consumer fixture project against
+the real Node types. It asserts the exact conditional-truthiness diagnostics
+without asking the editor project service to discover other workspace projects.
+An isolated Node 24 run measured 6.96 seconds user plus 1.56 seconds system CPU
+with `/usr/bin/time -p`. This case alone allows 70 seconds: eight times that CPU
+cost, rounded up, for a heavily shared host. The package's 30-second default is
+unchanged. Coverage reports and V8 scratch files use a per-process directory so
+concurrent package invocations cannot clean each other's reports.