@pond-ts/process 0.69.0 → 0.70.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/API.md CHANGED
@@ -716,31 +716,31 @@ Typed dataflow graphs for pipelines whose **shape is data** (runtime-assembled,
716
716
  user-edited, one computation fanned out to several consumers). Chaining stays
717
717
  the default for pipelines known at authoring time — see the package README.
718
718
 
719
- | Group | Exports | Source |
720
- | ----------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------- |
721
- | Worker pool (Node) | `HostPool` (`start`, `run`, `close`, `size`, `inFlight`); types `HostPoolOptions`, `PoolSetup`, `PoolSetupConfig`; `toWire` / `fromWire`, types `WireResult`, `WireColumn` — subpath `@pond-ts/process/pool` | `packages/process/src/pool/index.ts` |
722
- | Ports | `Inlet`, `Outlet` (typed fields on `node.in` / `node.out`; `get()`, `peek()`, `version`, `connect`, `disconnect`) | `packages/process/src/port.ts` |
723
- | Nodes | `Node` (`in`, `out`, `dirty`, `error`, `invalidate()`), `defineNode` (reusable multi-output node type), `derive` (single-output, wired inline) | `packages/process/src/node.ts` |
724
- | Port declaration | `port<T>({ equals, defaultValue })`; types `PortSpec`, `PortSpecMap`, `PortValue`, `PortValues` | `packages/process/src/types.ts` |
725
- | Sources | `source<T>()` → `SourceNode` (`set()`), `fromLive(liveSource)` → `LiveSourceNode` (`dispose()`); `GraphSource` (bind contract — looser than core's `LiveSource`, accepts `LiveAggregation`), `SnapshotSource`, `NoInputs` | `packages/process/src/source.ts` |
726
- | Graph view | `Graph` (`Graph.from(...roots)`, `nodes`, `order()`, `edges()`, `toJSON()`); types `GraphEdge`, `GraphJson`, `GraphNodeJson`, `GraphEdgeJson` | `packages/process/src/graph.ts` |
727
- | Range output buffers | `prepareRange(length, keep, prior)` → `RangeOutput` (`values`, `bits`, `set`, `clear`) carrying `[0, keep)` forward as blocks — values **and** validity; `sealRange(out, length)` → `Float64Column`; `validityByteCount`. Reached from an op as `ctx.out[n]` | `packages/process/src/column.ts` |
728
- | Ranged recompute | `graph.setSourceFrom(series, changedFrom)` — declares which row first changed; `graph.recomputes` → `{ ranged, full }`. An op opts in with `OpDef.runRange(ctx)`, receiving `{ from, to, previous, previousView, out }` (type `RangeContext`) — write into `out` and return nothing for the block path alongside the usual context. Requires `lookback`. Falls back to a full `run` whenever anything is missing | `packages/process/src/plan/graph.ts` |
729
- | Node budget | `bind(series, { registry, budgetBytes })` — engine-wide cap on retained node values, LRU, enforced after each `run`; `graph.retainedBytes` / `graph.evictions` / `graph.enforceBudget()`. Unbounded when omitted. Skips a node whose consumer still holds its outlet | `packages/process/src/plan/graph.ts` |
730
- | Plan history | `requiredHistory(registry, plan)` → `{ known, rows?, undeclared, byOp }` — the minimum safe tail in rows, folded from per-op `OpDef.lookback`. Sums along nesting, maxes across siblings. `known: false` names ops with no declared lookback rather than defaulting to zero (type `HistoryResult`) | `packages/process/src/plan/history.ts` |
731
- | Column values | `packColumn` (values → packed `Float64Column`, NaN = missing), `columnBytes` (retained size, for a byte budget), `appendColumn` (column → series; boxing-free when gapless), `columnBuffers` / `columnFromBuffers` (the buffer pair a column is, for an isolate boundary; type `ColumnBuffers`), `columnView` (zero-copy borrowed read view for in-process folds; type `ColumnView`) | `packages/process/src/column.ts` |
732
- | Plan — registry | `createRegistry({ folds })` / `Registry` (`define`, `get`, `foldFor`, `outputsOf`, `resolveParams` (`{ validate: false }` applies defaults and skips every check), `byFamily`, `describe`, `toJsonSchema`), param builders `int` / `num` / `choice` / `flag`, `UnknownOpError`, `ParamError` | `packages/process/src/plan/registry.ts`, `params.ts` |
733
- | Plan — identity | `specId(registry, spec, { validate })` (content-addressed, param-order invariant, defaults materialized; `validate: false` is **total** — names a spec that would not compile in a separate `p1?:` namespace that cannot collide with a valid id; a valid spec's id is identical either way), `SpecIdOptions`, `refToId`, `explain`, `unitOf`, `columnsOf`, `dependsOn`, `outputKey` | `packages/process/src/plan/identity.ts` |
734
- | Plan — types | `Spec`, `Plan`, `Input` (column name \| `Spec` \| `PickedOutput`), `SpecRef`, `Def` (`OpDef` \| `FoldDef`), `OpContext`, `OpResult`, `FoldContext`, `FactBody`, `isFold`, `ParamDef`, `Params`, `Units`, `InputDef`, `OutputDef` | `packages/process/src/plan/types.ts` |
735
- | Plan — bind / run | `bind(series, { registry, units })` → `BoundGraph` (`compile`, `setSource`, `ids`, `series`, `columnOf`), `run(graph, { plan, select, onError })` → `RunResult`, `UnitError`, `UnknownColumnError` (a raw string input naming no column of the bound series, checked over the whole closure and re-checked on the warm path) | `packages/process/src/plan/graph.ts`, `run.ts` |
736
- | Plan — request/response | `RunRequest` (`PlanRequest` \| `SlotRequest`), `RunOptions`, `RunResult`, `Select` (`{ on, output?, name? }` — points at a node; what comes back is what that node produces), `ErrorPolicy`, `Fact` (carries `op`), `OutputInfo`, `Skipped` (`spec`, `select`, `reason`, `code` — the failure's kind, matching the error class a throw would have carried), `NodeTiming` (`slot`, `pulled`, `cached`, `ms`, `inputs`) | `packages/process/src/plan/run.ts` |
737
- | Plan — host | `createHost({ registry, units, sources })` → `Host` (`add`, `has`, `datasets`, `graphFor`, `run`, `runAsync`), `toWire`, `UnknownDatasetError`; local-string `Envelope` (`PlanEnvelope` \| `SlotEnvelope`), remote-capable `AsyncEnvelope` (`AsyncPlanEnvelope` \| `AsyncSlotEnvelope` \| `Envelope`), `DatasetInfo`, `WireResult` | `packages/process/src/plan/host.ts` |
738
- | Plan — slots | `expandSlots(slots, columns)` → `Map<slot, Spec>` (expands to the nested form, so ids match by construction; `slot#Output` picks one output), `SlotError`; types `SlotDef` (`{ op, params, in }`), `Slots` | `packages/process/src/plan/slots.ts` |
739
- | Plan — builder | `plan(from)` → low-level `PlanBuilder`; `process(registry, from)` → typed fluent `ProcessBuilder` (`column`, op methods, `outputs`), `BuilderError`; types `NodeHandle`, `OutputHandle`, `FluentColumnRef`, `SingleColumnNode`, `MultiColumnNode`, `ColumnSelection`, `FactRef`, `BuiltRequest` | `packages/process/src/plan/builder.ts`, `fluent.ts` |
740
- | Plan — async sources | `defineSource({ name, load })`, `createSourceRegistry()` / `SourceRegistry`, `sourceId`, `UnknownSourceError`; types `SourceRef`, `SourceParams`, `LoadedSource` (value + revision), `SourceLoadContext`, `SourceDef` | `packages/process/src/plan/source.ts` |
741
- | Plan — folds | `STANDARD_FOLDS` and the four it holds — `last`, `extremes`, `percentileRank`, `shape` — pre-registered by `createRegistry()`; each a plain `FoldDef`, so a consumer can `define` over one | `packages/process/src/plan/folds.ts` |
742
- | Errors | `ProcessError` (base; `code` — a stable per-class literal, minification-proof, also surfaced on `Skipped`), `CycleError`, `UnconnectedInputError`, `MissingOutputError`, `UnsetSourceError` | `packages/process/src/errors.ts` |
743
- | Node type helpers | `NodeSpec`, `NodeFactory`, `InletsFor`, `OutletsFor`, `OutletValue`, `SpecsForOutlets`, `DerivedOutput` | `packages/process/src/node.ts` |
719
+ | Group | Exports | Source |
720
+ | ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------- |
721
+ | Worker pool (Node) | `HostPool` (`start`, `run`, `close`, `size`, `inFlight`); types `HostPoolOptions`, `PoolSetup`, `PoolSetupConfig`; `toWire` / `fromWire`, types `WireResult`, `WireColumn` — subpath `@pond-ts/process/pool` | `packages/process/src/pool/index.ts` |
722
+ | Ports | `Inlet`, `Outlet` (typed fields on `node.in` / `node.out`; `get()`, `peek()`, `version`, `connect`, `disconnect`) | `packages/process/src/port.ts` |
723
+ | Nodes | `Node` (`in`, `out`, `dirty`, `error`, `invalidate()`), `defineNode` (reusable multi-output node type), `derive` (single-output, wired inline) | `packages/process/src/node.ts` |
724
+ | Port declaration | `port<T>({ equals, defaultValue })`; types `PortSpec`, `PortSpecMap`, `PortValue`, `PortValues` | `packages/process/src/types.ts` |
725
+ | Sources | `source<T>()` → `SourceNode` (`set()`), `fromLive(liveSource)` → `LiveSourceNode` (`dispose()`); `GraphSource` (bind contract — looser than core's `LiveSource`, accepts `LiveAggregation`), `SnapshotSource`, `NoInputs` | `packages/process/src/source.ts` |
726
+ | Graph view | `Graph` (`Graph.from(...roots)`, `nodes`, `order()`, `edges()`, `toJSON()`); types `GraphEdge`, `GraphJson`, `GraphNodeJson`, `GraphEdgeJson` | `packages/process/src/graph.ts` |
727
+ | Range output buffers | `prepareRange(length, keep, prior)` → `RangeOutput` (`values`, `bits`, `set`, `clear`) carrying `[0, keep)` forward as blocks — values **and** validity; `sealRange(out, length)` → `Float64Column`; `validityByteCount`. Reached from an op as `ctx.out[n]` | `packages/process/src/column.ts` |
728
+ | Ranged recompute | `graph.setSourceFrom(series, changedFrom)` — declares which row first changed; `graph.recomputes` → `{ ranged, full }`. An op opts in with `OpDef.runRange(ctx)`, receiving `{ from, to, previous, previousView, out }` (type `RangeContext`) — write into `out` and return nothing for the block path alongside the usual context. Requires `lookback`. Falls back to a full `run` whenever anything is missing | `packages/process/src/plan/graph.ts` |
729
+ | Node budget | `bind(series, { registry, budgetBytes })` — engine-wide cap on retained node values, LRU, enforced after each `run`; `graph.retainedBytes` / `graph.evictions` / `graph.enforceBudget()`. Unbounded when omitted. Skips a node whose consumer still holds its outlet | `packages/process/src/plan/graph.ts` |
730
+ | Plan history | `requiredHistory(registry, plan)` → `{ known, rows?, undeclared, byOp }` — the minimum safe tail in rows, folded from per-op `OpDef.lookback`. Sums along nesting, maxes across siblings. `known: false` names ops with no declared lookback rather than defaulting to zero (type `HistoryResult`) | `packages/process/src/plan/history.ts` |
731
+ | Column values | `packColumn` (values → packed `Float64Column`, NaN = missing), `columnBytes` (retained size, for a byte budget), `appendColumn` (column → series; boxing-free when gapless), `columnBuffers` / `columnFromBuffers` (the buffer pair a column is, for an isolate boundary; type `ColumnBuffers`), `columnView` (zero-copy borrowed read view for in-process folds; type `ColumnView`) | `packages/process/src/column.ts` |
732
+ | Plan — registry | `createRegistry({ folds })` / `Registry` (`define`, `get`, `foldFor`, `outputsOf`, `resolveParams` (`{ validate: false }` applies defaults and skips every check), `byFamily`, `describe`, `toJsonSchema`, `checkArity`), param builders `int` / `num` / `choice` / `flag`, `UnknownOpError`, `ParamError`, `ArityError` | `packages/process/src/plan/registry.ts`, `params.ts` |
733
+ | Plan — identity | `specId(registry, spec, { validate })` (content-addressed, param-order invariant, defaults materialized; `validate: false` is **total over arbitrary JSON** — names a spec that would not compile, including malformed shapes, in a separate `p1?:` namespace that cannot collide with a valid id; a valid spec's id is identical either way. Judges op existence, params **and arity**), `SpecIdOptions`, `refToId`, `explain`, `unitOf`, `columnsOf`, `dependsOn`, `outputKey` | `packages/process/src/plan/identity.ts` |
734
+ | Plan — types | `Spec`, `Plan`, `Input` (column name \| `Spec` \| `PickedOutput`), `SpecRef`, `Def` (`OpDef` \| `FoldDef`), `OpContext`, `OpResult`, `FoldContext`, `FactBody`, `isFold`, `ParamDef`, `Params`, `Units`, `InputDef`, `OutputDef` | `packages/process/src/plan/types.ts` |
735
+ | Plan — bind / run | `bind(series, { registry, units })` → `BoundGraph` (`compile`, `setSource`, `ids`, `series`, `columnOf`), `run(graph, { plan, select, onError })` → `RunResult`, `UnitError`, `UnknownColumnError` (a raw string input naming no column of the bound series, checked over the whole closure and re-checked on the warm path) | `packages/process/src/plan/graph.ts`, `run.ts` |
736
+ | Plan — request/response | `RunRequest` (`PlanRequest` \| `SlotRequest`), `RunOptions`, `RunResult`, `Select` (`{ on, output?, name? }` — points at a node; what comes back is what that node produces), `ErrorPolicy`, `Fact` (carries `op`), `OutputInfo`, `Skipped` (`spec` — echoed verbatim, `params`/`inputs` typed `unknown`; `select`, `reason`, `code` — the failure's kind, matching the error class a throw would have carried), `NodeTiming` (`slot`, `pulled`, `cached`, `ms`, `inputs`) | `packages/process/src/plan/run.ts` |
737
+ | Plan — host | `createHost({ registry, units, sources })` → `Host` (`add`, `has`, `datasets`, `graphFor`, `run`, `runAsync`), `toWire`, `UnknownDatasetError`; local-string `Envelope` (`PlanEnvelope` \| `SlotEnvelope`), remote-capable `AsyncEnvelope` (`AsyncPlanEnvelope` \| `AsyncSlotEnvelope` \| `Envelope`), `DatasetInfo`, `WireResult` | `packages/process/src/plan/host.ts` |
738
+ | Plan — slots | `expandSlots(slots, columns)` → `Map<slot, Spec>` (expands to the nested form, so ids match by construction; `slot#Output` picks one output), `SlotError`; types `SlotDef` (`{ op, params, in }`), `Slots` | `packages/process/src/plan/slots.ts` |
739
+ | Plan — builder | `plan(from)` → low-level `PlanBuilder`; `process(registry, from)` → typed fluent `ProcessBuilder` (`column`, op methods, `outputs`), `BuilderError`; types `NodeHandle`, `OutputHandle`, `FluentColumnRef`, `SingleColumnNode`, `MultiColumnNode`, `ColumnSelection`, `FactRef`, `BuiltRequest` | `packages/process/src/plan/builder.ts`, `fluent.ts` |
740
+ | Plan — async sources | `defineSource({ name, load })`, `createSourceRegistry()` / `SourceRegistry`, `sourceId`, `UnknownSourceError`; types `SourceRef`, `SourceParams`, `LoadedSource` (value + revision), `SourceLoadContext`, `SourceDef` | `packages/process/src/plan/source.ts` |
741
+ | Plan — folds | `STANDARD_FOLDS` and the four it holds — `last`, `extremes`, `percentileRank`, `shape` — pre-registered by `createRegistry()`; each a plain `FoldDef`, so a consumer can `define` over one | `packages/process/src/plan/folds.ts` |
742
+ | Errors | `ProcessError` (base; `code` — a stable per-class literal, minification-proof, also surfaced on `Skipped`), `CycleError`, `UnconnectedInputError`, `MissingOutputError`, `UnsetSourceError` | `packages/process/src/errors.ts` |
743
+ | Node type helpers | `NodeSpec`, `NodeFactory`, `InletsFor`, `OutletsFor`, `OutletValue`, `SpecsForOutlets`, `DerivedOutput` | `packages/process/src/node.ts` |
744
744
 
745
745
  Note: this package's `npm test` includes a `test:dts` step that typechecks the
746
746
  **emitted** `dist/*.d.ts` from a consumer's perspective (`test-dts/`,
package/CHANGELOG.md CHANGED
@@ -8,7 +8,8 @@ The `@pond-ts` packages — `pond-ts`, `@pond-ts/react`, `@pond-ts/charts`,
8
8
  under a single `v*` tag, so this file covers them all. Pre-1.0: minor bumps may
9
9
  include new features and type-level changes; patch bumps are strictly additive.
10
10
 
11
- [Unreleased]: https://github.com/pond-ts/pond/compare/v0.69.0...HEAD
11
+ [Unreleased]: https://github.com/pond-ts/pond/compare/v0.70.0...HEAD
12
+ [0.70.0]: https://github.com/pond-ts/pond/compare/v0.69.0...v0.70.0
12
13
  [0.69.0]: https://github.com/pond-ts/pond/compare/v0.68.0...v0.69.0
13
14
  [0.68.0]: https://github.com/pond-ts/pond/compare/v0.67.0...v0.68.0
14
15
  [0.67.0]: https://github.com/pond-ts/pond/compare/v0.66.0...v0.67.0
@@ -72,6 +73,71 @@ include new features and type-level changes; patch bumps are strictly additive.
72
73
 
73
74
  ## [Unreleased]
74
75
 
76
+ ## [0.70.0] — 2026-09-18
77
+
78
+ ### Added
79
+
80
+ - `@pond-ts/charts`: **`<BandChart sessionBreaks>`** — the same trading-axis
81
+ session break `<LineChart>` has had since v0.45.0. On a discontinuous
82
+ (`discontinuities` / `calendar`) axis the fill previously ran a near-vertical
83
+ sliver from one session's last sample to the next session's first, because
84
+ the collapsed overnight gap put them a pixel apart; `sessionBreaks` ends the
85
+ envelope at the close and re-starts it at the open, so a band and its centre
86
+ line break in step. A **scale** break, orthogonal to the NaN **data** gaps a
87
+ band always breaks at; default `false` (unchanged output). Decimation
88
+ composes: `decimateBand` folds each break instant into the pixel-column
89
+ edges and bakes a `NaN` sample at it, so no column merges two sessions'
90
+ envelopes. Stories `Axes/TradingTimeAxis / SessionBreaksBand` and
91
+ `Performance/Decimation / TradingSessionBreaksBand`.
92
+ - `@pond-ts/process`: **`ArityError`**, and **arity is now part of what `specId`
93
+ can judge**. A spec whose `inputs` count does not match the op's — including
94
+ one carrying no `inputs` at all — was named `p1:sma(;period=20)`, a _valid_
95
+ id for something that cannot compile, and then died at `compile` as a bare
96
+ `TypeError` reading `.length` of `undefined`, reaching the consumer as a
97
+ `Skipped` with **no `code`** (which under that contract means "op code
98
+ threw"). Arity is decidable from the registry alone, so strict `specId` now
99
+ raises `ArityError` and lenient mode marks the id `p1?:`.
100
+ `Registry.checkArity(op, inputs)` is the shared check `compile` uses too.
101
+
102
+ ### Changed
103
+
104
+ - `@pond-ts/financial`: **`StudyOutput.id` documents the `@pond-ts/process`
105
+ bridge**, and a new `test/catalog-process.test.ts` pins it. The field is a
106
+ *financial column suffix*; process's `OutputDef.id` is a *process outlet
107
+ id*; they share a name and are different namespaces. Process names its own
108
+ columns (`specId + OutputDef.id`) and matches an op's return to its outputs
109
+ positionally, so a study's own column names never reach it — which means a
110
+ registry bridging the two chooses its own suffixes, and for the **twelve**
111
+ multi-output studies that claim the bare prefix (`trix`, `superTrend`,
112
+ `klinger`, …) it must: process rejects `''` on a multi-output op, because
113
+ there the column would be named exactly the spec id, itself a legal column
114
+ reference. The map is `outputs.length > 1 && id === ''` → `'value'` — not
115
+ `id === ''`, which would rename all seventy-odd single-output columns for
116
+ nothing. Reported by a consumer that hit the throw at module load and
117
+ worked around it by hand. No API change: the descriptors, the studies and
118
+ the guard are all unchanged, and the round-trip test is what stops the two
119
+ packages drifting — `catalog.test.ts` validates a descriptor against its
120
+ *study*, so it is structurally blind to a cross-package disagreement.
121
+ - `@pond-ts/process`: **`Skipped.spec` echoes the request verbatim**, and its
122
+ `params` / `inputs` are typed `unknown` accordingly. The plan pass normalized
123
+ `params: null` to `{}`, so recomputing an id from the echo produced the
124
+ _defaulted spec's valid id_ — keying a broken persisted entry's report onto a
125
+ legitimate node. The selector pass echoed the original all along, so the two
126
+ passes disagreed. **Migration:** a consumer reading `entry.spec.params` now
127
+ narrows it (`entry.spec.params as Record<string, unknown>`, or a guard) —
128
+ which is the point, since the value may be exactly the malformed thing that
129
+ failed.
130
+ - `@pond-ts/process`: `specId(…, { validate: false })` is **total over
131
+ arbitrary JSON**, not just over well-typed specs. `params: null`, a
132
+ non-array `inputs`, and an input entry that is neither a column name nor a
133
+ spec each used to raise a `TypeError`; they are now named in the `p1?:`
134
+ namespace, distinctly enough that two differently-broken specs stay two ids.
135
+ Strict mode reports them as `ParamError` / `ArityError` / `ProcessError`
136
+ rather than crashing.
137
+
138
+ (All three reported by Tidal against 0.62.0, after adopting it.)
139
+
140
+
75
141
  ## [0.69.0] — 2026-09-13
76
142
 
77
143
  ### Added
package/dist/index.d.ts CHANGED
@@ -36,7 +36,7 @@ export { Graph } from './graph.js';
36
36
  export type { GraphEdge, GraphJson, GraphNodeJson, GraphEdgeJson, } from './graph.js';
37
37
  export { ProcessError, CycleError, UnconnectedInputError, MissingOutputError, } from './errors.js';
38
38
  export { createRegistry, Registry, int, num, choice, flag, } from './plan/registry.js';
39
- export { UnknownOpError, ParamError } from './plan/registry.js';
39
+ export { UnknownOpError, ParamError, ArityError } from './plan/registry.js';
40
40
  export type { DefMap } from './plan/registry.js';
41
41
  export { isFold } from './plan/types.js';
42
42
  export { STANDARD_FOLDS, last, extremes, percentileRank, shape, } from './plan/folds.js';
package/dist/index.js CHANGED
@@ -31,7 +31,7 @@ export { Graph } from './graph.js';
31
31
  export { ProcessError, CycleError, UnconnectedInputError, MissingOutputError, } from './errors.js';
32
32
  // ─── Plan layer ([PND-DEMOM0]) ──────────────────────────────────
33
33
  export { createRegistry, Registry, int, num, choice, flag, } from './plan/registry.js';
34
- export { UnknownOpError, ParamError } from './plan/registry.js';
34
+ export { UnknownOpError, ParamError, ArityError } from './plan/registry.js';
35
35
  export { isFold } from './plan/types.js';
36
36
  export { STANDARD_FOLDS, last, extremes, percentileRank, shape, } from './plan/folds.js';
37
37
  export { specId, refToId, explain, unitOf, columnsOf, dependsOn, outputKey, } from './plan/identity.js';
@@ -142,8 +142,8 @@ export declare class BoundGraph {
142
142
  * one, nothing is ever evicted.
143
143
  *
144
144
  * Validation happens here rather than at pull time so a bad plan is
145
- * rejected before any work: params first, then arity, then the typed
146
- * input check.
145
+ * rejected before any work: the op must exist, then arity, then params
146
+ * (all three via strict `specId`), then the typed input check.
147
147
  */
148
148
  compile(spec: Spec): Compiled;
149
149
  /** Reads one output column of a compiled spec, by output suffix. */
@@ -381,8 +381,8 @@ export class BoundGraph {
381
381
  * one, nothing is ever evicted.
382
382
  *
383
383
  * Validation happens here rather than at pull time so a bad plan is
384
- * rejected before any work: params first, then arity, then the typed
385
- * input check.
384
+ * rejected before any work: the op must exist, then arity, then params
385
+ * (all three via strict `specId`), then the typed input check.
386
386
  */
387
387
  compile(spec) {
388
388
  const id = specId(this.registry, spec);
@@ -413,9 +413,7 @@ export class BoundGraph {
413
413
  }
414
414
  const op = this.registry.get(spec.op);
415
415
  const params = this.registry.resolveParams(op, spec.params);
416
- if (spec.inputs.length !== op.inputs.length) {
417
- throw new ProcessError(`${spec.op} takes ${op.inputs.length} input(s), got ${spec.inputs.length}`);
418
- }
416
+ this.registry.checkArity(op, spec.inputs);
419
417
  // After arity — an input index past the declared list is an arity
420
418
  // problem, not a column one — and before the typed-unit pass, whose
421
419
  // answer for an absent column is a misleading 'unitless'.
@@ -34,13 +34,13 @@
34
34
  *
35
35
  * Both are pinned by tests.
36
36
  */
37
- import type { Registry } from './registry.js';
37
+ import { type Registry } from './registry.js';
38
38
  import type { Params, Spec, SpecRef, Units } from './types.js';
39
39
  /** Options for {@link specId}. */
40
40
  export interface SpecIdOptions {
41
41
  /**
42
- * Whether the op must exist and its params must be legal default
43
- * `true`.
42
+ * Whether the op must exist, its params must be legal, and its
43
+ * `inputs` count must match the op's arity — default `true`.
44
44
  *
45
45
  * Pass `false` to name a spec that would not compile. See
46
46
  * {@link specId} for why identity is separable from validity.
@@ -70,8 +70,10 @@ export interface SpecIdOptions {
70
70
  *
71
71
  * So `specId(registry, spec, { validate: false })` is **total**: an
72
72
  * unknown op keeps its given params verbatim, a known one still gets
73
- * its defaults applied and its keys sorted, and nothing throws.
74
- * Validity stays `compile`'s job.
73
+ * its defaults applied and its keys sorted, and nothing throws — a
74
+ * malformed shape is **named** (marked `p1?:`), not rejected. Validity
75
+ * stays `compile`'s job, with one exception decidable from the registry
76
+ * alone: arity, which strict mode judges here too (`ArityError`).
75
77
  *
76
78
  * **A valid spec has one id under either mode.** Canonicalization is
77
79
  * the same code path and `checkParam` never coerces, so the lenient id
@@ -34,6 +34,8 @@
34
34
  *
35
35
  * Both are pinned by tests.
36
36
  */
37
+ import { ProcessError } from '../errors.js';
38
+ import { ParamError } from './registry.js';
37
39
  import { isFold, isPicked, specOf } from './types.js';
38
40
  /** Id format version. Bumping it invalidates persisted ids deliberately. */
39
41
  const VERSION = 'p1';
@@ -51,6 +53,14 @@ function esc(v) {
51
53
  * after the version and can never equal one, whatever its params say.
52
54
  */
53
55
  const UNVALIDATED = '?';
56
+ /**
57
+ * Key under which a malformed `params` or `inputs` value is recorded in
58
+ * an unvalidated id, so the shape it actually had survives.
59
+ *
60
+ * Only ever emitted inside a `p1?:` id, so it cannot be confused with a
61
+ * declared param — and it is escaped like any other key there.
62
+ */
63
+ const MALFORMED = '!malformed';
54
64
  /**
55
65
  * Type-preserving encoding, used **only** inside an unvalidated id.
56
66
  *
@@ -93,8 +103,10 @@ function typedEsc(v) {
93
103
  *
94
104
  * So `specId(registry, spec, { validate: false })` is **total**: an
95
105
  * unknown op keeps its given params verbatim, a known one still gets
96
- * its defaults applied and its keys sorted, and nothing throws.
97
- * Validity stays `compile`'s job.
106
+ * its defaults applied and its keys sorted, and nothing throws — a
107
+ * malformed shape is **named** (marked `p1?:`), not rejected. Validity
108
+ * stays `compile`'s job, with one exception decidable from the registry
109
+ * alone: arity, which strict mode judges here too (`ArityError`).
98
110
  *
99
111
  * **A valid spec has one id under either mode.** Canonicalization is
100
112
  * the same code path and `checkParam` never coerces, so the lenient id
@@ -116,53 +128,129 @@ function typedEsc(v) {
116
128
  export function specId(registry, spec, options = {}) {
117
129
  return build(registry, spec, options.validate === false).id;
118
130
  }
119
- /** An id, and whether anything in its closure failed validation. */
131
+ /** True for an object literal a spec's `params` could legally be. */
132
+ function isParamBag(v) {
133
+ return typeof v === 'object' && v !== null && !Array.isArray(v);
134
+ }
135
+ /** True for an input that is a nested spec or a picked output. */
136
+ function isSpecLike(v) {
137
+ return (typeof v === 'object' &&
138
+ v !== null &&
139
+ ('op' in v || 'from' in v) &&
140
+ !Array.isArray(v));
141
+ }
142
+ /**
143
+ * An id, and whether anything in its closure failed validation.
144
+ *
145
+ * **Totality is over arbitrary JSON, not over well-typed `Spec`s.** The
146
+ * whole reason a consumer reaches for leniency is a persisted object
147
+ * that no longer fits — a dropped `inputs` key, a `null` where a param
148
+ * bag belongs, an input that came back as a bare `null`. Answering those
149
+ * with a `TypeError` is the same failure as throwing `ParamError`, one
150
+ * layer down, so each malformed shape is **named** here rather than
151
+ * crashed on: marked unvalidated, and encoded distinctly enough that two
152
+ * differently-broken specs stay two ids (Tidal, on 0.62.0).
153
+ */
120
154
  function build(registry, spec, lenient) {
121
155
  let unvalidated = false;
122
- // Inputs first: a nested spec that did not validate marks this one,
123
- // and the mark has to be known before the params are encoded.
124
- //
125
- // `?? []` because a spec arriving from persistence may be missing the
126
- // field entirely, and a mode whose promise is totality cannot answer a
127
- // dropped key with a TypeError. Strict mode reaches `compile`'s arity
128
- // check instead, which says what is actually wrong.
129
- const inputs = (spec.inputs ?? [])
156
+ const bad = () => {
157
+ unvalidated = true;
158
+ };
159
+ // The op first, because arity needs it. An unknown op under leniency
160
+ // leaves `op` undefined and nothing further is decidable about params
161
+ // or arity both are declared BY the definition.
162
+ const op = lenient && !registry.has(spec.op) ? undefined : registry.get(spec.op);
163
+ if (op === undefined)
164
+ bad();
165
+ // Arity is part of validity, and decidable from the registry alone —
166
+ // so a spec that fails it must not be named in the valid namespace.
167
+ // It used to be checked only at `compile`, which meant `p1:sma(;…)`
168
+ // named a spec that could not exist, and then `compile` read `.length`
169
+ // off `undefined` and raised a codeless `TypeError` (Tidal, 0.62.0).
170
+ if (op !== undefined) {
171
+ try {
172
+ registry.checkArity(op, spec.inputs);
173
+ }
174
+ catch (e) {
175
+ if (!lenient)
176
+ throw e;
177
+ bad();
178
+ }
179
+ }
180
+ // Inputs next: a nested spec that did not validate marks this one, and
181
+ // the mark has to be known before the params are encoded.
182
+ const rawInputs = Array.isArray(spec.inputs)
183
+ ? spec.inputs
184
+ : spec.inputs === undefined
185
+ ? // The key was dropped. Reads as an empty input list, which is
186
+ // what it is — the arity check above has already marked it.
187
+ (bad(), [])
188
+ : // Present but not a list. Encoded as one token so the shape it
189
+ // actually had survives rather than being flattened to "empty".
190
+ (bad(), [{ [MALFORMED]: spec.inputs }]);
191
+ const inputs = rawInputs
130
192
  .map((i) => {
131
193
  if (typeof i === 'string')
132
194
  return esc(i);
133
- // `#Lower` rather than a separate field: an input picking a
134
- // different output is a different computation, and the id is what
135
- // says so.
136
- const base = build(registry, specOf(i), lenient);
137
- if (base.unvalidated)
138
- unvalidated = true;
139
- return isPicked(i) ? `${base.id}#${esc(i.output)}` : base.id;
195
+ if (isSpecLike(i)) {
196
+ // `#Lower` rather than a separate field: an input picking a
197
+ // different output is a different computation, and the id is
198
+ // what says so.
199
+ const base = build(registry, specOf(i), lenient);
200
+ if (base.unvalidated)
201
+ unvalidated = true;
202
+ return isPicked(i)
203
+ ? `${base.id}#${esc(i.output)}`
204
+ : base.id;
205
+ }
206
+ // Neither a column name nor a spec — `null`, a number, an array.
207
+ if (!lenient) {
208
+ throw new ProcessError(`input of '${spec.op}' must be a column name or a spec, got ${JSON.stringify(i) ?? typeof i}`);
209
+ }
210
+ bad();
211
+ // A non-array `inputs` was wrapped above so its shape survives; keep
212
+ // the marker in the token, or `{ inputs: null }` and `{ inputs: [null] }`
213
+ // would mint the same id — two differently-broken specs, one chip.
214
+ return isParamBag(i) && MALFORMED in i
215
+ ? esc(`${MALFORMED}:${typeof i[MALFORMED]}:${String(i[MALFORMED])}`)
216
+ : typedEsc(i);
140
217
  })
141
218
  .join('+');
142
- let params;
143
- if (lenient && !registry.has(spec.op)) {
144
- unvalidated = true;
145
- params = spec.params ?? {};
219
+ // Params last, so the mark is settled before they are encoded.
220
+ let entries;
221
+ if (op === undefined) {
222
+ entries = Object.entries(isParamBag(spec.params) ? spec.params : {});
223
+ if (spec.params !== undefined && !isParamBag(spec.params)) {
224
+ bad();
225
+ entries = [[MALFORMED, spec.params]];
226
+ }
227
+ }
228
+ else if (spec.params !== undefined && !isParamBag(spec.params)) {
229
+ // `resolveParams` would read keys off it and throw a `TypeError`.
230
+ if (!lenient) {
231
+ throw new ParamError(`${spec.op} params must be an object, got ${JSON.stringify(spec.params) ?? typeof spec.params}`);
232
+ }
233
+ bad();
234
+ entries = [[MALFORMED, spec.params]];
146
235
  }
147
236
  else {
148
- const op = registry.get(spec.op);
149
237
  try {
150
238
  // The strict resolve first even under leniency: when it succeeds
151
239
  // the id is byte-identical to the validating one, which is the
152
240
  // whole contract. Only its failure moves this spec into the
153
241
  // unvalidated namespace.
154
- params = registry.resolveParams(op, spec.params);
242
+ entries = Object.entries(registry.resolveParams(op, spec.params));
155
243
  }
156
244
  catch (e) {
157
245
  if (!lenient)
158
246
  throw e;
159
- unvalidated = true;
160
- params = registry.resolveParams(op, spec.params, { validate: false });
247
+ bad();
248
+ entries = Object.entries(registry.resolveParams(op, spec.params, { validate: false }));
161
249
  }
162
250
  }
163
251
  const encodeKey = unvalidated ? esc : (k) => k;
164
252
  const encodeValue = unvalidated ? typedEsc : esc;
165
- const p = Object.entries(params)
253
+ const p = entries
166
254
  .sort(([a], [b]) => (a < b ? -1 : a > b ? 1 : 0))
167
255
  .map(([k, v]) => `${encodeKey(k)}=${encodeValue(v)}`)
168
256
  .join(',');
@@ -32,6 +32,22 @@ export declare class UnknownOpError extends ProcessError {
32
32
  export declare class ParamError extends ProcessError {
33
33
  static readonly code: string;
34
34
  }
35
+ /**
36
+ * Thrown when a spec's input list does not match the op's declared
37
+ * arity — including a spec carrying no `inputs` at all.
38
+ *
39
+ * Its own class because arity is decidable from the **registry alone**,
40
+ * with no data bound: it is part of what `specId` can judge, and a spec
41
+ * that fails it must not be named in the valid id namespace. Before
42
+ * this, a spec with no `inputs` was named `p1:sma(;period=20)` — a valid
43
+ * id for something that cannot compile — and then died at `compile` as a
44
+ * bare `TypeError` reading `.length` of undefined, reaching the consumer
45
+ * as a `Skipped` with no `code` at all, which under that contract means
46
+ * "op code threw" (Tidal, on 0.62.0).
47
+ */
48
+ export declare class ArityError extends ProcessError {
49
+ static readonly code: string;
50
+ }
35
51
  /** Op metadata as a picker or a tool catalog wants it. */
36
52
  export interface OpDescriptor {
37
53
  readonly name: string;
@@ -103,6 +119,14 @@ export declare class Registry<Defs extends DefMap = {}> {
103
119
  resolveParams(op: Def, given?: Readonly<Record<string, ParamValue>>, options?: {
104
120
  validate?: boolean;
105
121
  }): Params;
122
+ /**
123
+ * Checks a spec's input list against the op's declared arity.
124
+ *
125
+ * Lives here beside `get` and `resolveParams` — the three things
126
+ * decidable about a spec from the definition alone, before any data is
127
+ * bound — so `specId` and `compile` ask the same question once.
128
+ */
129
+ checkArity(op: Def, inputs: unknown): void;
106
130
  /** Grouped for a picker. */
107
131
  byFamily(): Map<string, OpDescriptor[]>;
108
132
  describe(): OpDescriptor[];
@@ -24,6 +24,22 @@ export class UnknownOpError extends ProcessError {
24
24
  export class ParamError extends ProcessError {
25
25
  static code = 'ParamError';
26
26
  }
27
+ /**
28
+ * Thrown when a spec's input list does not match the op's declared
29
+ * arity — including a spec carrying no `inputs` at all.
30
+ *
31
+ * Its own class because arity is decidable from the **registry alone**,
32
+ * with no data bound: it is part of what `specId` can judge, and a spec
33
+ * that fails it must not be named in the valid id namespace. Before
34
+ * this, a spec with no `inputs` was named `p1:sma(;period=20)` — a valid
35
+ * id for something that cannot compile — and then died at `compile` as a
36
+ * bare `TypeError` reading `.length` of undefined, reaching the consumer
37
+ * as a `Skipped` with no `code` at all, which under that contract means
38
+ * "op code threw" (Tidal, on 0.62.0).
39
+ */
40
+ export class ArityError extends ProcessError {
41
+ static code = 'ArityError';
42
+ }
27
43
  /**
28
44
  * Validates one param and returns it.
29
45
  *
@@ -206,6 +222,22 @@ export class Registry {
206
222
  }
207
223
  return out;
208
224
  }
225
+ /**
226
+ * Checks a spec's input list against the op's declared arity.
227
+ *
228
+ * Lives here beside `get` and `resolveParams` — the three things
229
+ * decidable about a spec from the definition alone, before any data is
230
+ * bound — so `specId` and `compile` ask the same question once.
231
+ */
232
+ checkArity(op, inputs) {
233
+ const want = op.inputs.length;
234
+ if (!Array.isArray(inputs)) {
235
+ throw new ArityError(`${op.name} takes ${want} input(s), got none — 'inputs' is ${inputs === undefined ? 'missing' : (JSON.stringify(inputs) ?? 'unset')}`);
236
+ }
237
+ if (inputs.length !== want) {
238
+ throw new ArityError(`${op.name} takes ${want} input(s), got ${inputs.length}`);
239
+ }
240
+ }
209
241
  /** Grouped for a picker. */
210
242
  byFamily() {
211
243
  const out = new Map();
@@ -20,7 +20,7 @@
20
20
  import type { Column, SeriesSchema, TimeSeries } from 'pond-ts';
21
21
  import type { BoundGraph } from './graph.js';
22
22
  import { type Slots } from './slots.js';
23
- import type { Input, Plan, SpecRef } from './types.js';
23
+ import type { Plan, SpecRef } from './types.js';
24
24
  /** What to do when a spec or a selector fails. Covers both, not just resolution. */
25
25
  export type ErrorPolicy = 'throw' | 'skip' | 'collect';
26
26
  /**
@@ -160,14 +160,22 @@ export interface NodeTiming {
160
160
  }
161
161
  export interface Skipped {
162
162
  /**
163
- * The spec that failed, echoed back — including `inputs`, because a
164
- * plan may hold two specs of the same op and a caller retrying needs
165
- * to know which one it was.
163
+ * The spec that failed, echoed back **verbatim** — including `inputs`,
164
+ * because a plan may hold two specs of the same op and a caller
165
+ * retrying needs to know which one it was.
166
+ *
167
+ * `params` and `inputs` are typed `unknown` because this is an echo of
168
+ * whatever arrived, and what arrives is exactly what may be malformed.
169
+ * The plan pass used to normalize — `params: null` came back as `{}` —
170
+ * which was not merely lossy: recomputing an id from the echo then
171
+ * produced the **defaulted spec's valid id**, so a broken persisted
172
+ * entry's report keyed onto a legitimate node (Tidal, on 0.62.0). The
173
+ * selector pass echoed the original all along, so the two disagreed.
166
174
  */
167
175
  readonly spec?: {
168
176
  op: string;
169
- params: Record<string, unknown>;
170
- inputs: readonly Input[];
177
+ params?: unknown;
178
+ inputs?: unknown;
171
179
  };
172
180
  readonly select?: Select;
173
181
  readonly reason: string;
package/dist/plan/run.js CHANGED
@@ -127,11 +127,7 @@ export function run(graph, request) {
127
127
  }
128
128
  catch (e) {
129
129
  fail(e, {
130
- spec: {
131
- op: spec.op,
132
- params: { ...(spec.params ?? {}) },
133
- inputs: spec.inputs,
134
- },
130
+ spec: { op: spec.op, params: spec.params, inputs: spec.inputs },
135
131
  });
136
132
  }
137
133
  }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@pond-ts/process",
3
- "version": "0.69.0",
3
+ "version": "0.70.0",
4
4
  "description": "Computations as data over pond-ts: processing graphs authored fluently or composed as JSON, resolved against a declared op vocabulary with content-addressed caching, provenance, and per-node timings. Experimental, pre-1.0.",
5
5
  "keywords": [
6
6
  "time-series",
@@ -62,7 +62,7 @@
62
62
  "verify": "npm run format:check && npm run build && npm test"
63
63
  },
64
64
  "peerDependencies": {
65
- "pond-ts": "^0.69.0"
65
+ "pond-ts": "^0.70.0"
66
66
  },
67
67
  "devDependencies": {
68
68
  "typescript": "^5.6.3",