pond-ts 0.33.0 → 0.34.1

package/CHANGELOG.md

@@ -8,7 +8,9 @@ The `@pond-ts` packages — `pond-ts`, `@pond-ts/react`, `@pond-ts/charts`, and
 them all. Pre-1.0: minor bumps may include new features and type-level changes;
 patch bumps are strictly additive.
 
-[Unreleased]: https://github.com/pjm17971/pond-ts/compare/v0.33.0...HEAD
+[Unreleased]: https://github.com/pjm17971/pond-ts/compare/v0.34.1...HEAD
+[0.34.1]: https://github.com/pjm17971/pond-ts/compare/v0.34.0...v0.34.1
+[0.34.0]: https://github.com/pjm17971/pond-ts/compare/v0.33.0...v0.34.0
 [0.33.0]: https://github.com/pjm17971/pond-ts/compare/v0.32.0...v0.33.0
 [0.32.0]: https://github.com/pjm17971/pond-ts/compare/v0.31.2...v0.32.0
 [0.31.2]: https://github.com/pjm17971/pond-ts/compare/v0.31.1...v0.31.2
@@ -28,6 +30,63 @@ patch bumps are strictly additive.
 [0.19.0]: https://github.com/pjm17971/pond-ts/compare/v0.18.0...v0.19.0
 [0.18.0]: https://github.com/pjm17971/pond-ts/compare/v0.17.1...v0.18.0
 
+## [0.34.1] — 2026-07-01
+
+A `pond-ts` core patch: fixes a performance regression introduced in 0.34.0.
+`@pond-ts/react`, `@pond-ts/charts`, and `@pond-ts/fit` carry no code
+changes — republished in lock-step; their `^0.34.0` peer ranges already
+admit this patch.
+
+### Fixed
+
+- **`TimeSeries.fromColumns` — `number[]` column conversion was ~7-18x
+  slower than intended.** The `null`/`NaN`-gap parity fix in 0.34.0
+  converted `number[]` columns via `Float64Array.from(raw, mapFn)`.
+  Supplying a map function forces V8's generic iterable-protocol path even
+  for a plain array, dramatically slower than a manual loop into a
+  preallocated buffer at 100k-element scale. Found while pointing the
+  `pond-columnar-ingest` wire-format spike at the real published package
+  instead of a local build. Fixed with a manual `for` loop in both the
+  key-column and value-column conversion paths — no behavior change, same
+  gap semantics, all 12 `fromColumns` tests unchanged and green. Added
+  `scripts/perf-from-columns.mjs` as the durable regression benchmark.
+  Measured: 100k × 7 cols, `number[]` columns, dense — 21.5ms → 2.9ms.
+  (#311)
+
+## [0.34.0] — 2026-07-01
+
+A `pond-ts` core release: the columnar/typed-array ingress driven by the
+Tidal wire-format spike. `@pond-ts/react`, `@pond-ts/charts`, and
+`@pond-ts/fit` carry no code changes — republished in lock-step (their
+`pond-ts` / `@pond-ts/react` peer ranges widen to `^0.34.0`).
+
+### Added
+
+- **`TimeSeries.fromColumns`** — the columnar (struct-of-arrays) ingress,
+  the counterpart to `fromJSON`'s row-tuple shape. Accepts either a plain
+  `number[]` or a `Float64Array` per column — one polymorphic door, so a
+  wire format only changes the *decoder*, not the ingest. `Float64Array`
+  columns are adopted directly (zero-copy); `number[]` columns are copied.
+  A `null`/`undefined` cell or a non-finite value (`NaN`/`Infinity`) is a
+  gap, identically across both input shapes. Enforces the same
+  non-decreasing key-order invariant as `fromJSON`. v1 scope: a `time`-kind
+  key and `number` value columns. (#310)
+
+  Measured against a wire-format spike (`tidal-app/pond-columnar-ingest`):
+  `fromColumns` collapses the ingest step that every non-rows path
+  previously paid (transpose → `fromJSON`) — 100k-point protobuf ingest
+  27.7ms → 2.8ms; JSON-columnar 27.0ms → 5.2ms. In a browser, decoding
+  off-main in a Web Worker and transferring the resulting `Float64Array`s
+  back keeps `fromColumns`'s adopt path on the main thread to ~9ms,
+  dropping a 500k-point ingest's worst animation-frame stall from ~50ms to
+  ~9ms.
+
+- **Docs** — a "Columnar ingest" section on the
+  [Creating series](https://pjm17971.github.io/pond-ts/docs/start-here/creating#columnar-ingest)
+  page covering both input shapes, the adopt-vs-copy/aliasing distinction,
+  missing-value and ordering semantics, and why this matters for
+  interactive charts; a pointer from Getting Started.
+
 ## [0.33.0] — 2026-06-30
 
 A `@pond-ts/charts` release: a label opt-out for annotations plus interaction

package/dist/batch/time-series.d.ts

@@ -87,6 +87,46 @@ export declare class TimeSeries<S extends SeriesSchema> {
          */
         sort?: boolean;
     }): TimeSeries<S>;
+    /**
+     * Example: `TimeSeries.fromColumns({ name, schema, columns })`.
+     *
+     * The **columnar** ingress — the struct-of-arrays counterpart to
+     * {@link TimeSeries.fromJSON} (which takes row tuples). Each entry in
+     * `columns` is one column's values, keyed by schema column name and aligned by
+     * index; every column must have the same length. Values may be a plain
+     * `number[]` (e.g. `JSON.parse` of a columnar wire payload) **or** a
+     * `Float64Array` (e.g. protobuf packed doubles / fixed-point after descale) —
+     * one polymorphic door, so the wire format only changes the *decoder*, not the
+     * ingest. A value cell is a gap (missing) iff it's `null`/`undefined` or
+     * non-finite (`NaN`/`Infinity`) — identical rule regardless of which array
+     * type supplied it. Note this is looser than `fromJSON`, which *rejects* a
+     * non-finite provided number outright rather than treating it as a gap.
+     *
+     * Skips the row-tuple materialization + per-row intake a columnar caller
+     * otherwise pays (transpose → `fromJSON`): each column packs directly, the key
+     * column is built from the first schema column, and the store is assembled via
+     * trusted construction (`ColumnarStore.fromTrustedStore` still validates kind +
+     * aligned length).
+     *
+     * **`Float64Array` inputs are adopted, not copied** (zero-copy — the point of
+     * the fast path): the resulting series' columns alias the caller's buffers.
+     * Mutating an adopted buffer after construction mutates the series. Pass a
+     * fresh buffer (e.g. straight off a decode) if this matters; `number[]`
+     * columns are always copied.
+     *
+     * **v1 scope:** a `time`-kind key + `number` value columns (the market-data /
+     * chart wire case). Other key kinds (`interval` / `timeRange`) and non-numeric
+     * value columns throw for now — extend as consumers need.
+     *
+     * @throws ValidationError on a missing column, a length mismatch, an
+     *   unsupported kind, a non-finite timestamp key, or an out-of-order
+     *   (decreasing) timestamp — keys must be non-decreasing, same as `fromJSON`.
+     */
+    static fromColumns<S extends SeriesSchema>(input: {
+        name: string;
+        schema: S;
+        columns: Record<string, ReadonlyArray<number | null | undefined> | Float64Array>;
+    }): TimeSeries<S>;
     /**
      * Example: `TimeSeries.fromEvents(events, { schema, name })`.
      * Builds a typed series from an array of `Event` instances. The events

package/dist/batch/time-series.js

@@ -18,7 +18,8 @@ import { TimeRange } from '../core/time-range.js';
 import { compareEventKeys } from '../core/temporal.js';
 import { PartitionedTimeSeries } from './partitioned-time-series.js';
 import { Sequence } from '../sequence/sequence.js';
-import { ColumnarStore, IntervalKeyColumn, TimeKeyColumn, TimeRangeKeyColumn, float64ColumnFromArray, stringColumnFromArray, withColumnAppended, withColumnsRenamed, withColumnsSelected, withKeyColumn, withRowRange, withRowSelection, } from '../columnar/index.js';
+import { ColumnarStore, Float64Column, IntervalKeyColumn, TimeKeyColumn, TimeRangeKeyColumn, float64ColumnFromArray, stringColumnFromArray, validityFromPredicate, withColumnAppended, withColumnsRenamed, withColumnsSelected, withKeyColumn, withRowRange, withRowSelection, } from '../columnar/index.js';
+import { ValidationError } from '../core/errors.js';
 import { SeriesStore } from '../live/series-store.js';
 import { parseDuration } from '../core/duration.js';
 import { resolveReducer, bucketStateFor, rollingStateFor, } from '../reducers/index.js';
@@ -485,6 +486,130 @@ export class TimeSeries {
             sort: input.sort ?? false,
         });
     }
+    /**
+     * Example: `TimeSeries.fromColumns({ name, schema, columns })`.
+     *
+     * The **columnar** ingress — the struct-of-arrays counterpart to
+     * {@link TimeSeries.fromJSON} (which takes row tuples). Each entry in
+     * `columns` is one column's values, keyed by schema column name and aligned by
+     * index; every column must have the same length. Values may be a plain
+     * `number[]` (e.g. `JSON.parse` of a columnar wire payload) **or** a
+     * `Float64Array` (e.g. protobuf packed doubles / fixed-point after descale) —
+     * one polymorphic door, so the wire format only changes the *decoder*, not the
+     * ingest. A value cell is a gap (missing) iff it's `null`/`undefined` or
+     * non-finite (`NaN`/`Infinity`) — identical rule regardless of which array
+     * type supplied it. Note this is looser than `fromJSON`, which *rejects* a
+     * non-finite provided number outright rather than treating it as a gap.
+     *
+     * Skips the row-tuple materialization + per-row intake a columnar caller
+     * otherwise pays (transpose → `fromJSON`): each column packs directly, the key
+     * column is built from the first schema column, and the store is assembled via
+     * trusted construction (`ColumnarStore.fromTrustedStore` still validates kind +
+     * aligned length).
+     *
+     * **`Float64Array` inputs are adopted, not copied** (zero-copy — the point of
+     * the fast path): the resulting series' columns alias the caller's buffers.
+     * Mutating an adopted buffer after construction mutates the series. Pass a
+     * fresh buffer (e.g. straight off a decode) if this matters; `number[]`
+     * columns are always copied.
+     *
+     * **v1 scope:** a `time`-kind key + `number` value columns (the market-data /
+     * chart wire case). Other key kinds (`interval` / `timeRange`) and non-numeric
+     * value columns throw for now — extend as consumers need.
+     *
+     * @throws ValidationError on a missing column, a length mismatch, an
+     *   unsupported kind, a non-finite timestamp key, or an out-of-order
+     *   (decreasing) timestamp — keys must be non-decreasing, same as `fromJSON`.
+     */
+    static fromColumns(input) {
+        const { name, schema, columns } = input;
+        // Key column (schema[0]). v1: point-in-time (`time`) keys only.
+        const keyDef = schema[0];
+        if (keyDef === undefined) {
+            throw new ValidationError('fromColumns: schema must have at least a key column');
+        }
+        if (keyDef.kind !== 'time') {
+            throw new ValidationError(`fromColumns: v1 supports a 'time' key; schema[0] '${keyDef.name}' is '${keyDef.kind}'`);
+        }
+        const keyRaw = columns[keyDef.name];
+        if (keyRaw === undefined) {
+            throw new ValidationError(`fromColumns: missing key column '${keyDef.name}'`);
+        }
+        // epoch-ms buffer; TimeKeyColumn asserts all finite. A manual loop, not
+        // `Float64Array.from(arr, mapFn)`: supplying a map function forces V8's
+        // generic iterable-protocol path even for a plain array, ~15-20x slower
+        // than a preallocated-buffer copy at 100k-element scale — measured, not
+        // theoretical (see the pond-columnar-ingest spike's ingest regression).
+        let begin;
+        if (keyRaw instanceof Float64Array) {
+            begin = keyRaw;
+        }
+        else {
+            begin = new Float64Array(keyRaw.length);
+            for (let j = 0; j < keyRaw.length; j += 1) {
+                const v = keyRaw[j];
+                begin[j] = v == null ? NaN : Number(v);
+            }
+        }
+        const count = begin.length;
+        // Throws on any non-finite timestamp.
+        const keys = new TimeKeyColumn(begin, count);
+        // Enforce the non-decreasing-key invariant that `fromJSON`'s
+        // `validateAndNormalize` guarantees. Trusted construction skips row
+        // materialization + kind re-validation, but NOT this correctness contract:
+        // bisect-based operators (crop, `atTime`, range queries) rely on it, so an
+        // unsorted columnar input must fail loudly here rather than build a silently
+        // broken series. One O(N) scan over already-finite values — negligible next
+        // to decode.
+        for (let j = 1; j < count; j += 1) {
+            if (begin[j] < begin[j - 1]) {
+                throw new ValidationError(`fromColumns: key column '${keyDef.name}' is out of order at index ${j} ` +
+                    `(${begin[j]} < ${begin[j - 1]}) — timestamps must be non-decreasing; pre-sort the columns`);
+            }
+        }
+        // Value columns — packed directly (missing-aware) from the arrays.
+        const columnMap = new Map();
+        for (let i = 1; i < schema.length; i += 1) {
+            const def = schema[i];
+            if (def.kind !== 'number') {
+                throw new ValidationError(`fromColumns: v1 supports 'number' value columns; column '${def.name}' is '${def.kind}'`);
+            }
+            const raw = columns[def.name];
+            if (raw === undefined) {
+                throw new ValidationError(`fromColumns: missing column '${def.name}'`);
+            }
+            if (raw.length !== count) {
+                throw new ValidationError(`fromColumns: column '${def.name}' length ${raw.length} does not match key length ${count}`);
+            }
+            // Normalize to a Float64Array either way — adopt if already typed (the
+            // fast path a protobuf / fixed-point decoder hits, zero-copy), else
+            // convert (`null`/`undefined` -> `NaN`) — then apply ONE validity rule
+            // to both: a cell is a gap iff it's non-finite. This must be identical
+            // regardless of input type: an earlier version used `float64ColumnFromArray`
+            // for the `number[]` branch, which treats a `NaN` *value* (as opposed to
+            // `null`) as defined-but-non-finite rather than missing, diverging from
+            // the `Float64Array` branch's `Number.isFinite` gap signal — the same
+            // wire value would silently mean different things depending on which
+            // array type decoded it.
+            // Manual loop, not `Float64Array.from(arr, mapFn)` — see the key-column
+            // comment above; the cost applies identically here.
+            let values;
+            if (raw instanceof Float64Array) {
+                values = raw;
+            }
+            else {
+                values = new Float64Array(count);
+                for (let j = 0; j < count; j += 1) {
+                    const v = raw[j];
+                    values[j] = v == null ? NaN : v;
+                }
+            }
+            const validity = validityFromPredicate(count, (j) => Number.isFinite(values[j]));
+            columnMap.set(def.name, new Float64Column(values, count, validity, validity === undefined));
+        }
+        const store = ColumnarStore.fromTrustedStore(schema, keys, columnMap);
+        return TimeSeries.#fromTrustedStore(name, schema, store);
+    }
     /**
      * Example: `TimeSeries.fromEvents(events, { schema, name })`.
      * Builds a typed series from an array of `Event` instances. The events

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pond-ts",
-  "version": "0.33.0",
+  "version": "0.34.1",
   "description": "TypeScript-first time series primitives",
   "license": "MIT",
   "repository": {