pond-ts 0.50.0 → 0.52.0

package/CHANGELOG.md

@@ -8,7 +8,9 @@ The `@pond-ts` packages — `pond-ts`, `@pond-ts/react`, `@pond-ts/charts`,
 tag, so this file covers them all. Pre-1.0: minor bumps may include new features
 and type-level changes; patch bumps are strictly additive.
 
-[Unreleased]: https://github.com/pond-ts/pond/compare/v0.50.0...HEAD
+[Unreleased]: https://github.com/pond-ts/pond/compare/v0.52.0...HEAD
+[0.52.0]: https://github.com/pond-ts/pond/compare/v0.51.0...v0.52.0
+[0.51.0]: https://github.com/pond-ts/pond/compare/v0.50.0...v0.51.0
 [0.50.0]: https://github.com/pond-ts/pond/compare/v0.49.0...v0.50.0
 [0.49.0]: https://github.com/pond-ts/pond/compare/v0.48.1...v0.49.0
 [0.48.1]: https://github.com/pond-ts/pond/compare/v0.48.0...v0.48.1
@@ -50,6 +52,190 @@ and type-level changes; patch bumps are strictly additive.
 
 ## [Unreleased]
 
+## [0.52.0] — 2026-07-23
+
+### Changed
+
+- **core / financial:** **Market-scale studies are now typed-array fast**
+  (the "SMA/EMA at 1M bars costs hundreds of ms" report). Three cuts along
+  the same path, all behaviour-preserving (identical values, warm-ups,
+  missing-cell semantics, and rejection errors; every fast path falls back
+  to the original sweep when it doesn't apply):
+  - **`smooth('ema')` columnar fast path** — on a packed numeric source
+    column the EMA recurrence runs straight off the typed buffer into a
+    typed result column via trusted construction (key + untouched columns
+    pass through zero-copy), replacing the per-row Event/tuple rebuild +
+    full-series intake re-pack. 1M rows: **530 ms → 4.4 ms (~120×)**.
+  - **`rolling({ count })` numeric fast path** — an all-built-in numeric
+    mapping over packed sources feeds the shared incremental reducer states
+    directly from the typed buffers and writes snapshots into typed columns
+    (no per-row snapshot arrays, no boxed accumulators, no post-pass
+    assert/re-pack). 1M rows, `avg`: **135 ms → 32 ms (~4×)**.
+  - **financial kernel reads columns, not events** — `rollingColumns` /
+    `columnValues` now read study inputs/outputs off the public column API
+    instead of materializing `series.events` (an Event + data object per
+    row, ~400 ms of pure overhead at 1M rows).
+  - End-to-end at 1M bars: `ema()` **603 ms → 2.5 ms (~240×)**, `sma()`
+    **569 ms → 56 ms (~10×)**, `bollinger()` **748 ms → 162 ms (~4.6×)**.
+    Durable benchmarks: `packages/core/scripts/perf-smooth-ema.mjs`,
+    `packages/financial/scripts/perf-studies.mjs`.
+
+### Added
+
+- **core:** **`TimeSeries.fromArrow(table, options?)` — ingest a decoded Apache
+  Arrow `Table`.** pond stays zero-dependency: bring your own Arrow
+  (`tableFromIPC(...)`) and hand the `Table` in; the input is duck-typed against
+  a small structural surface (`ArrowTableLike` / `ArrowVectorLike` / …, all
+  exported). Ingest is the zero-copy path — every `Float64` column's backing
+  `Float64Array` is adopted as-is (`Float32`/int columns convert; int64 value
+  columns recombine BigInt-free), and the schema is derived from the Arrow
+  fields. The time key is converted **BigInt-free**: Arrow's idiomatic int64
+  timestamps are recombined from their two int32 halves rather than
+  `Number(bigint)` per row — measured **~11× faster** on the time column (0.6ms
+  vs 6.8ms at 500k rows; `scripts/perf-from-arrow.mjs`). Options: `time` (key
+  column, default the `'time'` field), `timeUnit` (default read from the Arrow
+  Arrow type family — a `Timestamp`'s raw-unit int64 is scaled by its
+  `TimeUnit`; `Date32`/`Date64` arrive already normalized to epoch-ms and pass
+  through; overridable), `columns` (subset, in order), `name`, `sort`. Numeric
+  **and string** columns are supported — string columns (Arrow
+  `Utf8`) become dict-encoded `StringColumn`s; any other Arrow type
+  (list/struct) throws, naming it. A null time key throws; numeric nulls map to
+  `NaN` and string nulls to missing.
+- **core:** **`TimeSeries.fromColumns` / `ValueSeries.fromColumns` now accept
+  `string` value columns** (previously numeric-only), packed to dict-encoded
+  `StringColumn`s (`null`/`undefined` → missing) — the shared columnar-ingress
+  engine now dispatches on the schema kind. Other value kinds (`boolean`,
+  arrays) still throw.
+- **charts:** **`<ScatterChart decimate>` — dense scatter plots now decimate**
+  (PND-MARKDEC scatter half — the last un-decimated mark type). **Default
+  `true`.** When the marks are **uniform** (fixed size + colour, no data-driven
+  `radius`/`color`), **opaque**, and denser than the pixel grid, overlapping
+  marks collapse to one representative per **mark-radius cell** via a 2D
+  pixel-**occupancy** sweep. Scatter has no fill, so a line/bar's per-column
+  `[min, max]` envelope would erase interior points — the occupancy grid keeps
+  one mark per occupied cell instead, which is **visually lossless** for uniform
+  opaque marks at that density (same-cell marks overlap). Interaction (hover /
+  click / tracker) still reads **every source point**; the per-point selection
+  ring + labels are suppressed only on the decimated (dense) path. A
+  **translucent** fill (density-encoded — overlap _should_ build up) or a
+  data-driven size/colour keeps the full draw. `decimate={false}` draws every
+  mark; `{ threshold }` tunes the trigger. The occupancy sweep uses the affine
+  fast path for the per-point pixel mapping. Measured (SciChart-suite
+  point-update, real browser): **100k 18 → 73 fps (4×)**, and the ladder now
+  runs to **10M** points (previously dead by 1M). `drawScatter` now returns
+  `LayerDrawStats` (visible via `onDrawStats`).
+
+## [0.51.0] — 2026-07-22
+
+### Changed
+
+- **charts:** **`trackerPosition` is now a _followed_ position, not a hard pin —
+  enabling cross-chart cursor sync.** A live local hover wins over
+  `trackerPosition`, so the chart under the pointer shows its own cursor while
+  any chart without a local pointer follows the controlled time (mapped through
+  its own `xScale`, so it's correct across different zooms). This makes
+  **multi-chart dashboard cursor sync** fall out of the plain props: give every
+  `<ChartContainer>` the same `trackerPosition={sharedTime}` and set `sharedTime`
+  from each one's `onTrackerChanged` (clear it to `null` on the group's
+  `onPointerLeave`) — no "which chart is active" bookkeeping. **Behaviour
+  change:** previously a numeric `trackerPosition` overrode local hover, and
+  `trackerPosition={null}` force-hid the cursor; now `null` and `undefined` are
+  equivalent ("no controlled position") and a hovered chart always tracks its
+  pointer. To force a chart to never show a cursor, use `cursor="none"`. See the
+  "Synced cursors across charts" story. No type change (`number | null`).
+- **charts:** **Line / area draw is ~3× faster on stroke-bound frames**
+  (PND-AFFINE / PND-GRADX; 2026-07 external-bench profile). When the curve is
+  linear and both scales are affine (every y axis is `scaleLinear`; x is
+  `scaleLinear` / `scaleTime` / the gap-free default time axis), `drawLine` and
+  `drawArea` now map points with an inline `k·v + b` over the typed arrays
+  instead of a per-point d3-scale closure + d3-shape generator — a **visually
+  identical** draw (guarded by the decimation pixel-identity and per-layer
+  visual-regression e2e). A real-gap trading-time axis, or a non-linear curve,
+  transparently keeps the exact d3 path. Measured on a JS-only micro-bench
+  (`scripts/perf-affine.mjs`): line 1M 60.4 → 19.0 ms (3.2×), area 1M 132 →
+  38 ms (3.5×). Separately, the area fill gradient's full-series value extent is
+  now memoized per column buffer, so a y-zoom / pan repaint no longer re-walks
+  the whole series to find the gradient span. No API change.
+- **charts:** **A y-zoom / y-autorange repaint no longer re-decimates line /
+  area layers** (PND-DECKEY; same 2026-07 profile, finding 3). The M4
+  decimation output is a pure function of the source data, x-domain, device
+  width, threshold, and session breaks — it never reads the y-scale — so it is
+  identical across every y-only frame. `drawLine` / `drawArea` now memoize the
+  cull+decimate result per source series (one entry, keyed on the x-scale
+  object + width + threshold + breaks), so a y-zoom / live y-autorange frame
+  reuses the prior polyline instead of re-binning O(N) points; a pan / x-zoom
+  mints a fresh x-scale and correctly recomputes. Measured
+  (`scripts/perf-deckey.mjs`): the ~5 ms/frame decimation walk at 1M points is
+  eliminated on every y-only frame. No API change.
+
+### Added
+
+- **charts:** **`<BarChart decimate>` — dense column charts now decimate**
+  (PND-MARKDEC; 2026-07 profile, finding 4 — "column dead by 5M"). **Default
+  `true`**: once the visible **single-series** bars are denser than ~2 per device
+  pixel (each slot < ~1px), they're drawn as one per-column **envelope** rect —
+  the exact painted union `[min(value, baseline), max(value, baseline)]` of each
+  pixel column — instead of every bar, so a 100k–5M-bar column chart stays
+  interactive. **Visually lossless** at that density (a perf knob, not a style);
+  interaction still reads the source bars (`barAt`), and the per-bar
+  selection/hover highlight is suppressed only when decimated (a <1px bar's ring
+  isn't visible anyway). `decimate={false}` draws every bar; `{ threshold }`
+  tunes the samples-per-pixel factor. No-op for a stacked / multi-group
+  histogram (the low-count categorical path). `drawBars` now returns
+  `LayerDrawStats` (visible via `onDrawStats`). Measured
+  (`scripts/perf-markdec.mjs`, JS-only): the bar draw at 5M points drops
+  485 → 26 ms (18.9×), 100k drops 9.7 → 0.9 ms (10.7×) — with the larger
+  rasterization win on top, browser-side.
+- **charts:** **`panZoom` is now a three-way mode + a `bounds` extent.**
+  `<ChartContainer panZoom>` takes `'none'` / `'pan'` / `'panZoom'` (drag-only
+  vs. drag+wheel), with the old boolean kept as shorthand (`true` ⇒ `'panZoom'`,
+  `false` ⇒ `'none'`) — so existing charts are unchanged. A new `bounds`
+  (`[min, max]`) prop fences pan/zoom to an **outer** extent (panning into an
+  edge stops there keeping its span; zoom-out is capped at the whole span), the
+  companion
+  to the existing `minDuration` zoom-in floor — together they pin the reachable
+  window between an inner and outer bound. On a trading-time axis `bounds`
+  clamps in wall-clock ms. Purely additive; no type narrowing.
+- **charts:** **Draw-cost + decimation observability** — `<ChartContainer
+onDrawStats>` (PND-DECOBS; dashboard A/B friction, 2026-07-21). Fires a
+  `DrawStatsFrame` once per row-canvas repaint (keyed by an opaque `rowKey` for
+  multi-row attribution), one `LayerDrawInfo` per layer carrying its `as`,
+  measured `drawMs`, and — for a decimating layer (line / area / band / candle /
+  box) — `sourceCount` / `drawnCount` / `decimated`.
+  Compare `drawnCount` to `sourceCount` to see whether M4 engaged; read `drawMs`
+  for per-layer render cost. **Zero-overhead when unused** — the render loop
+  skips per-layer timing entirely unless a consumer subscribes. New exports:
+  `DrawStatsFrame`, `LayerDrawInfo`.
+
+### Fixed
+
+- **charts:** hovering no longer repaints the row data canvas on every cursor
+  mousemove. The container frame minted a fresh `timeRange` array identity per
+  rebuild (and the frame rebuilds per cursor move), which the Layers draw
+  callback — depending on `container.timeRange` — read as a domain change:
+  each hover frame re-fired the canvas draw effect, including per-layer M4
+  re-decimation (measured 105 repaints per 122 mousemove events; with
+  `decimate` off, hover fell to ~10 fps). The tuple is now identity-stable on
+  its endpoints, restoring the SVG-overlay cursor contract (0 repaints, full
+  frame rate while hovering). Found running uPlot's bench protocol against
+  pond-charts; guarded by a new hover-sweep perf invariant in
+  `e2e/perf-invariants.spec.ts`.
+- **charts:** hovering no longer re-renders cursor-independent components
+  (both `YAxis`, `Bar`/`Box`). The cursor position was a `ContainerFrame`
+  field, so every mousemove re-identified the whole (~50-field) frame and
+  re-rendered **all** its context consumers — even ones that never read the
+  cursor. The per-move cursor state (`cursorX`/`cursorY`/`cursorRowKey`) now
+  lives in a dedicated `CursorContext`; the frame stays identity-stable across
+  a hover, so only the genuine cursor consumers (the `Layers` overlay,
+  `XAxis` crosshair pill, `Legend` values) re-render. Measured: 4 → 2 React
+  commits per mousemove, ~25% less hover script time on the uPlot-bench
+  workload (the win scales with axis/row count). No API change — the split
+  types are internal (`PND-HOVCTX`, follow-up to the repaint fix above).
+- **charts:** corrected the `@pond-ts/charts` package-header doc comment, which
+  described a "chunked Path2D cache" render stage that was explored and
+  **deferred**, never built (it doesn't help the pan case, which re-decimates
+  every frame). The stale comment had misled a consumer's perf investigation.
+
 ## [0.50.0] — 2026-07-21
 
 ### Added

package/dist/batch/aggregate-columns.d.ts

@@ -1,5 +1,6 @@
 import type { Column } from '../columnar/index.js';
-import type { AggregateMap, AggregateOutputMap, AggregateReducer, ScalarKind, SeriesSchema } from '../schema/index.js';
+import { Float64Column } from '../columnar/index.js';
+import type { AggregateMap, AggregateOutputMap, AggregateReducer, RollingAlignment, ScalarKind, SeriesSchema } from '../schema/index.js';
 /**
  * Normalised column spec used by both batch and live aggregation paths.
  *
@@ -38,4 +39,32 @@ export declare function tryAggregateColumnarTimeKeyed<B extends {
     begin(): number;
     end(): number;
 }>(begins: Float64Array, getColumn: (name: string) => Column | undefined, buckets: ReadonlyArray<B>, columns: ReadonlyArray<AggregateColumnSpec>): Array<ReadonlyArray<unknown>> | null;
+/**
+ * **Columnar fast path for count-window `rolling()`** — the N-bar window
+ * financial studies compose on (SMA / Bollinger / rolling stats). When every
+ * mapped column is a built-in reducer producing a `'number'` output from a
+ * **packed `Float64Column`** source, the window sweep feeds the shared
+ * incremental rolling states (`rollingStateFor` — the same add / remove /
+ * snapshot arithmetic and non-finite skip policy as the generic path) values
+ * read straight off the typed buffers, and writes each snapshot into a typed
+ * result column. That removes the generic sweep's per-row costs: the
+ * `snapshotWindow` result-array allocation, the boxed accumulator rows, the
+ * polymorphic `col.read(i)` per add/remove, and the post-pass
+ * `assertColumnValuesMatchKind` + re-pack over boxed values (each written
+ * value is finite-asserted inline instead, same rejection class + message).
+ *
+ * Returns `null` — caller takes the generic sweep — when any column doesn't
+ * qualify: a custom-function reducer, a non-`'number'` output kind
+ * (`unique` / `samples` / `keep` over a non-number source / an explicit
+ * `kind` override), or a non-numeric / chunked / missing source column.
+ * All-or-nothing per call, matching {@link tryAggregateColumnarTimeKeyed}.
+ *
+ * Window shape replicates the generic count sweep exactly: rows are the unit
+ * (`count` is a bar count — no equal-key grouping), `lo` / `hi` are monotonic
+ * in the row index for every alignment (each row enters and leaves the window
+ * once, amortized O(1)), a centered even `count` biases one row toward the
+ * leading side, and a row whose window holds fewer than `minSamples` rows
+ * emits missing cells without consulting the reducer states.
+ */
+export declare function tryRollingCountColumnarNumeric(getColumn: (name: string) => Column | undefined, rowCount: number, columns: ReadonlyArray<AggregateColumnSpec>, count: number, alignment: RollingAlignment, minSamples: number): Float64Column[] | null;
 //# sourceMappingURL=aggregate-columns.d.ts.map

package/dist/batch/aggregate-columns.js

@@ -1,4 +1,6 @@
-import { resolveReducer } from '../reducers/index.js';
+import { resolveReducer, rollingStateFor } from '../reducers/index.js';
+import { Float64Column, bitmapByteCount, validityFromBits, } from '../columnar/index.js';
+import { ValidationError } from '../core/errors.js';
 /**
  * @internal — discriminator between an `AggregateOutputSpec` (`{ from,
  * using, kind? }`) and a bare reducer string/function passed in an
@@ -159,4 +161,137 @@ export function tryAggregateColumnarTimeKeyed(begins, getColumn, buckets, column
     }
     return rows;
 }
+/**
+ * **Columnar fast path for count-window `rolling()`** — the N-bar window
+ * financial studies compose on (SMA / Bollinger / rolling stats). When every
+ * mapped column is a built-in reducer producing a `'number'` output from a
+ * **packed `Float64Column`** source, the window sweep feeds the shared
+ * incremental rolling states (`rollingStateFor` — the same add / remove /
+ * snapshot arithmetic and non-finite skip policy as the generic path) values
+ * read straight off the typed buffers, and writes each snapshot into a typed
+ * result column. That removes the generic sweep's per-row costs: the
+ * `snapshotWindow` result-array allocation, the boxed accumulator rows, the
+ * polymorphic `col.read(i)` per add/remove, and the post-pass
+ * `assertColumnValuesMatchKind` + re-pack over boxed values (each written
+ * value is finite-asserted inline instead, same rejection class + message).
+ *
+ * Returns `null` — caller takes the generic sweep — when any column doesn't
+ * qualify: a custom-function reducer, a non-`'number'` output kind
+ * (`unique` / `samples` / `keep` over a non-number source / an explicit
+ * `kind` override), or a non-numeric / chunked / missing source column.
+ * All-or-nothing per call, matching {@link tryAggregateColumnarTimeKeyed}.
+ *
+ * Window shape replicates the generic count sweep exactly: rows are the unit
+ * (`count` is a bar count — no equal-key grouping), `lo` / `hi` are monotonic
+ * in the row index for every alignment (each row enters and leaves the window
+ * once, amortized O(1)), a centered even `count` biases one row toward the
+ * leading side, and a row whose window holds fewer than `minSamples` rows
+ * emits missing cells without consulting the reducer states.
+ */
+export function tryRollingCountColumnarNumeric(getColumn, rowCount, columns, count, alignment, minSamples) {
+    const specCount = columns.length;
+    const sources = [];
+    for (const spec of columns) {
+        if (typeof spec.reducer !== 'string')
+            return null; // custom function
+        if (spec.kind !== 'number')
+            return null; // non-numeric output kind
+        const source = getColumn(spec.source);
+        if (source === undefined)
+            return null; // missing source
+        if (source.kind !== 'number' || source.storage !== 'packed') {
+            return null; // non-numeric / chunked numeric source
+        }
+        sources.push(source);
+    }
+    // An all-finite fully-defined source feeds its reducer state the raw
+    // buffer values, so the `rollingStateFor` non-finite skip wrapper is
+    // provably an identity there — resolve the bare built-in state instead
+    // and drop one call layer from every add / remove / snapshot. Any other
+    // source keeps the wrapped state (same values, same skip policy).
+    const states = columns.map((spec, c) => {
+        const source = sources[c];
+        return source.allFinite && source.validity === undefined
+            ? resolveReducer(spec.reducer).rollingState()
+            : rollingStateFor(spec.reducer);
+    });
+    const srcValues = sources.map((col) => col._values);
+    const srcValidity = sources.map((col) => col.validity);
+    const outValues = columns.map(() => new Float64Array(rowCount));
+    const outBits = columns.map(() => new Uint8Array(bitmapByteCount(rowCount)));
+    const outDefined = new Array(specCount).fill(0);
+    // Feed the states exactly what the generic sweep's `col.read(index)`
+    // yields: `undefined` for a missing cell, the raw (possibly non-finite)
+    // number otherwise — the non-finite skip stays inside the shared state
+    // wrapper so the contributor set can't drift between the two paths.
+    const addAt = (index) => {
+        for (let c = 0; c < specCount; c += 1) {
+            const validity = srcValidity[c];
+            const value = validity === undefined || validity.isDefined(index)
+                ? srcValues[c][index]
+                : undefined;
+            states[c].add(index, value);
+        }
+    };
+    const removeAt = (index) => {
+        for (let c = 0; c < specCount; c += 1) {
+            const validity = srcValidity[c];
+            const value = validity === undefined || validity.isDefined(index)
+                ? srcValues[c][index]
+                : undefined;
+            states[c].remove(index, value);
+        }
+    };
+    const leftSpan = Math.floor((count - 1) / 2);
+    const rightSpan = count - 1 - leftSpan;
+    let windowStart = 0;
+    let windowEnd = 0;
+    for (let index = 0; index < rowCount; index += 1) {
+        let lo;
+        let hi;
+        if (alignment === 'trailing') {
+            lo = index - count + 1 < 0 ? 0 : index - count + 1;
+            hi = index;
+        }
+        else if (alignment === 'leading') {
+            lo = index;
+            hi = index + count - 1 < rowCount ? index + count - 1 : rowCount - 1;
+        }
+        else {
+            lo = index - leftSpan < 0 ? 0 : index - leftSpan;
+            hi = index + rightSpan < rowCount ? index + rightSpan : rowCount - 1;
+        }
+        while (windowEnd <= hi) {
+            addAt(windowEnd);
+            windowEnd += 1;
+        }
+        while (windowStart < lo) {
+            removeAt(windowStart);
+            windowStart += 1;
+        }
+        if (windowEnd - windowStart < minSamples)
+            continue; // missing row
+        for (let c = 0; c < specCount; c += 1) {
+            const v = states[c].snapshot();
+            if (v === undefined)
+                continue; // missing cell (e.g. all-missing window)
+            if (typeof v !== 'number' || !Number.isFinite(v)) {
+                // Same rejection class + message as the generic path's post-pass
+                // `assertColumnValuesMatchKind` (e.g. a `sum` overflow to Infinity)
+                // — checked at write time since there is no post-pass here.
+                throw new ValidationError(`rolling column '${columns[c].output}': result ${String(v)} is not a valid 'number' value`);
+            }
+            outValues[c][index] = v;
+            outBits[c][index >> 3] |= 1 << (index & 7);
+            outDefined[c] = outDefined[c] + 1;
+        }
+    }
+    return columns.map((_, c) => {
+        const validity = outDefined[c] === rowCount
+            ? undefined
+            : validityFromBits(outBits[c], rowCount);
+        // Every written cell was finite-asserted above → `allFinite`.
+        return new Float64Column(outValues[c], rowCount, validity, true);
+    });
+}
 //# sourceMappingURL=aggregate-columns.js.map

package/dist/batch/operators/from-arrow.d.ts

@@ -0,0 +1,101 @@
+import type { SeriesSchema } from '../../schema/index.js';
+import type { RawColumns } from './ingest-columns.js';
+/**
+ * Structural view of an Apache Arrow `Vector`. pond does **not** depend on
+ * `apache-arrow` — the caller brings their own Arrow (`tableFromIPC(...)`) and
+ * hands the decoded `Table` to {@link TimeSeries.fromArrow}. We duck-type the
+ * small slice of the vector surface we read: the length, the null count, and
+ * `toArray()` (which for a contiguous numeric column returns the backing typed
+ * array — the zero-copy handoff pond adopts).
+ */
+export interface ArrowVectorLike {
+    readonly length: number;
+    /** Number of null slots; `0` unlocks the fast (validity-free) path. */
+    readonly nullCount: number;
+    /**
+     * Backing values. For a **single-chunk** numeric column this is the typed
+     * array itself (`Float64Array`, `Int32Array`, `BigInt64Array`, …), adopted
+     * where possible; for a string column (Arrow `Utf8`) it is a plain `Array` of
+     * strings. Any other shape (list/struct) is rejected by
+     * {@link TimeSeries.fromArrow}. A **multi-chunk** vector (a table decoded from
+     * a multi-record-batch IPC stream) concatenates into a fresh array here — so
+     * such tables are correct but pay a copy, and the zero-copy/aliasing note
+     * does not apply to them.
+     */
+    toArray(): unknown;
+    /**
+     * Per-slot access, used only on the (rare) null-containing numeric slow path.
+     * Typed to match a real Arrow `Vector.get` across column kinds (a `Utf8`
+     * vector returns `string`), so a genuine `apache-arrow` `Table` satisfies
+     * this interface structurally.
+     */
+    get(index: number): number | bigint | string | null | undefined;
+}
+/** Structural view of an Arrow `Field` (`schema.fields[i]`). */
+export interface ArrowFieldLike {
+    readonly name: string;
+    /**
+     * The field's `DataType`. We read `typeId` (the Arrow `Type` enum) to detect
+     * a `Timestamp` (= 10), whose `toArray()` is raw-unit and so needs `unit`
+     * (a `TimeUnit`) to scale; every other kind is read straight off `toArray()`.
+     * Both are optional so a bare structural stand-in — and a real `DataType`,
+     * which carries far more — satisfy the shape.
+     */
+    readonly type: {
+        readonly typeId?: number;
+        readonly unit?: number;
+    };
+}
+/** Structural view of an Arrow `Schema`. */
+export interface ArrowSchemaLike {
+    readonly fields: ReadonlyArray<ArrowFieldLike>;
+}
+/** Structural view of an Arrow `Table` — the input to {@link TimeSeries.fromArrow}. */
+export interface ArrowTableLike {
+    readonly numRows: number;
+    readonly schema: ArrowSchemaLike;
+    getChild(name: string): ArrowVectorLike | null | undefined;
+}
+/** Options for {@link TimeSeries.fromArrow}. */
+export interface FromArrowOptions {
+    /** Series name. Default `'arrow'`. */
+    name?: string;
+    /**
+     * Which Arrow column is the time key. Default: the field named `'time'`.
+     * Throws if neither `time` is given nor a `'time'` field exists.
+     */
+    time?: string;
+    /**
+     * Unit of the time column's raw integer values. Default: read from the Arrow
+     * `Timestamp` field's `TimeUnit` when present, otherwise `'millisecond'`.
+     * An explicit value here always wins over the field's declared unit.
+     */
+    timeUnit?: ArrowTimeUnit;
+    /**
+     * Value columns to include, in order. Default: every non-time field. Each
+     * must be numeric (→ `Float64Column`) or a string column (→ `StringColumn`);
+     * any other Arrow type (list/struct/…) throws, naming it. Pass this to select
+     * a supported subset out of a mixed table.
+     */
+    columns?: readonly string[];
+    /**
+     * Sort rows by time key before construction (off by default — Arrow feeds
+     * are usually already time-ordered). Forwarded to `fromColumns`; disables
+     * zero-copy adoption when set (rows are permuted into fresh buffers).
+     */
+    sort?: boolean;
+}
+export type ArrowTimeUnit = 'second' | 'millisecond' | 'microsecond' | 'nanosecond';
+/**
+ * Translate an Arrow `Table` into the `{ name, schema, columns }` triple that
+ * `TimeSeries.fromColumns` ingests. Kept separate from the static so the
+ * conversion is unit-testable without the class and so the Arrow-shaped types
+ * live in one place. The returned columns are all `Float64Array`, so
+ * `fromColumns` takes its zero-copy adoption path throughout.
+ */
+export declare function arrowToColumns(table: ArrowTableLike, options?: FromArrowOptions): {
+    name: string;
+    schema: SeriesSchema;
+    columns: RawColumns;
+};
+//# sourceMappingURL=from-arrow.d.ts.map

package/dist/batch/operators/from-arrow.js

@@ -0,0 +1,297 @@
+import { ValidationError } from '../../core/errors.js';
+// Arrow's `TimeUnit` enum, keyed by its wire ordinal (unit read off a
+// `Timestamp` DataType). SECOND=0, MILLISECOND=1, MICROSECOND=2, NANOSECOND=3.
+const TIME_UNIT_BY_ORDINAL = {
+    0: 'second',
+    1: 'millisecond',
+    2: 'microsecond',
+    3: 'nanosecond',
+};
+// Multiplier taking one raw unit to milliseconds (pond's `Time` key is epoch
+// ms). Nanoseconds divide by 1e6, so sub-ms precision below the double's
+// integer range is discarded — expected at ms resolution.
+const MS_SCALE = {
+    second: 1000,
+    millisecond: 1,
+    microsecond: 1 / 1000,
+    nanosecond: 1 / 1_000_000,
+};
+// Arrow `Type` enum ordinal for `Timestamp` (a stable wire-level contract).
+// Only `Timestamp` needs unit scaling: its `toArray()` hands back the raw int64
+// values in the declared `TimeUnit`. Arrow's `Date32`/`Date64` `toArray()`, by
+// contrast, is already normalized to epoch-**ms** JS numbers by the logical
+// vector view — so a `Date` key is scale-1, NOT days/ms-from-a-DateUnit. (An
+// earlier version read `.unit` blind and scaled Date32 as seconds; verified
+// against real `apache-arrow` in `from-arrow.arrow.test.ts`, that was wrong.)
+const ARROW_TYPE_TIMESTAMP = 10;
+const TWO_POW_32 = 4294967296;
+/**
+ * Resolve the raw-unit → milliseconds multiplier for the time key. An explicit
+ * `timeUnit` always wins. Otherwise: a `Timestamp` (`typeId` 10, or a bare
+ * structural input carrying a `unit` but no `typeId` — overwhelmingly a
+ * timestamp) is scaled by its `TimeUnit`, since its `toArray()` is raw-unit
+ * int64. Everything else — `Date` (arrow already normalizes its `toArray()` to
+ * epoch-ms), plain int/float epoch-ms columns — is taken as milliseconds
+ * (scale 1).
+ */
+function resolveMsScale(field, override) {
+    if (override !== undefined)
+        return MS_SCALE[override];
+    const type = field.type ?? {};
+    const typeId = typeof type.typeId === 'number' ? type.typeId : undefined;
+    const unit = typeof type.unit === 'number' ? type.unit : undefined;
+    if (typeId === ARROW_TYPE_TIMESTAMP ||
+        (typeId === undefined && unit !== undefined)) {
+        return MS_SCALE[TIME_UNIT_BY_ORDINAL[unit ?? 1] ?? 'millisecond'];
+    }
+    return 1;
+}
+/**
+ * Numeric typed-array constructors Arrow's `toArray()` can hand back. A
+ * `BigInt64Array` / `BigUint64Array` is numeric but needs the BigInt-free
+ * recombination; the rest are plain-number typed arrays.
+ */
+function isBigIntArray(value) {
+    return value instanceof BigInt64Array || value instanceof BigUint64Array;
+}
+function isNumberTypedArray(value) {
+    return (value instanceof Float64Array ||
+        value instanceof Float32Array ||
+        value instanceof Int32Array ||
+        value instanceof Int16Array ||
+        value instanceof Int8Array ||
+        value instanceof Uint32Array ||
+        value instanceof Uint16Array ||
+        value instanceof Uint8Array ||
+        value instanceof Uint8ClampedArray);
+}
+// Detected once: is this host little-endian? Arrow IPC is always LE, but the
+// buffer a BigInt64Array exposes is in *host* byte order, so a big-endian host
+// (vanishingly rare for Node, but real) needs the two int32 halves swapped.
+const HOST_LITTLE_ENDIAN = new Uint8Array(new Uint32Array([1]).buffer)[0] === 1;
+/**
+ * Convert an int64 column to `Float64Array` **without BigInt**. We alias the
+ * buffer as `Int32Array` and recombine each pair `hi * 2^32 + (lo >>> 0)` —
+ * exact for the ±2^53 range that covers every realistic epoch timestamp
+ * (ms ≈ 1.7e12, µs ≈ 1.7e15). This is the ~30ms `Number(bigint)` ×N cost the
+ * fromArrow note called out, reclaimed: the per-element work is two array reads
+ * and a multiply-add, no BigInt boxing.
+ *
+ * Signedness follows the source: `BigUint64Array`'s high word is read unsigned
+ * (so a `Uint64` ≥ 2^63 stays positive rather than flipping negative);
+ * `BigInt64Array`'s stays signed (so pre-epoch / negative values recombine
+ * correctly). Half order follows {@link HOST_LITTLE_ENDIAN}. Values beyond
+ * ±2^53 (a huge id/count, or a nanosecond stamp) round to the nearest double —
+ * inherent to `Float64Column` storage, same as `Number(bigint)` would give.
+ */
+function int64ToFloat64(source, scale) {
+    const count = source.length;
+    const out = new Float64Array(count);
+    const halves = new Int32Array(source.buffer, source.byteOffset, count * 2);
+    const unsigned = source instanceof BigUint64Array;
+    const loOff = HOST_LITTLE_ENDIAN ? 0 : 1;
+    const hiOff = HOST_LITTLE_ENDIAN ? 1 : 0;
+    if (scale === 1) {
+        for (let j = 0; j < count; j += 1) {
+            const lo = halves[j * 2 + loOff] >>> 0;
+            const hiRaw = halves[j * 2 + hiOff];
+            const hi = unsigned ? hiRaw >>> 0 : hiRaw;
+            out[j] = hi * TWO_POW_32 + lo;
+        }
+    }
+    else {
+        for (let j = 0; j < count; j += 1) {
+            const lo = halves[j * 2 + loOff] >>> 0;
+            const hiRaw = halves[j * 2 + hiOff];
+            const hi = unsigned ? hiRaw >>> 0 : hiRaw;
+            out[j] = (hi * TWO_POW_32 + lo) * scale;
+        }
+    }
+    return out;
+}
+/**
+ * Read a numeric column carrying nulls, mapping null → `NaN` (missing). This is
+ * the slow path: per-element `get()`, and for an int64 column also the
+ * `Number(bigint)` boxing the two-int32 recombination avoids — the BigInt-free
+ * fast path is null-free-only. Acceptable given how rare a nulled int64 column
+ * is; a dense numeric column never lands here.
+ */
+function numericWithNulls(vector) {
+    const count = vector.length;
+    const out = new Float64Array(count);
+    for (let j = 0; j < count; j += 1) {
+        const v = vector.get(j);
+        out[j] = v == null ? NaN : Number(v);
+    }
+    return out;
+}
+/** Copy a string array's values, mapping null/undefined → missing. */
+function readStrings(raw) {
+    const out = new Array(raw.length);
+    for (let j = 0; j < raw.length; j += 1) {
+        const v = raw[j];
+        out[j] = v == null ? null : v;
+    }
+    return out;
+}
+/** Build a `Float64Array` from a plain number array, null/undefined → `NaN`. */
+function readNumbersFromArray(raw) {
+    const out = new Float64Array(raw.length);
+    for (let j = 0; j < raw.length; j += 1) {
+        const v = raw[j];
+        out[j] = v == null ? NaN : Number(v);
+    }
+    return out;
+}
+/**
+ * Classify a plain `Array` (what `toArray()` returns for non-typed-array
+ * columns) by its first defined element. Arrow hands back an `Array<string>`
+ * for `Utf8`/`LargeUtf8` and an `Array<number>` (epoch-ms) for `Date32`/`Date64`
+ * — the element type is the only reliable discriminator. All-null → `'number'`
+ * (an all-missing `Float64Column`); a non-string/number element → `'other'`
+ * (a list/struct vector), rejected by the caller.
+ */
+function classifyArray(raw) {
+    for (let j = 0; j < raw.length; j += 1) {
+        const v = raw[j];
+        if (v == null)
+            continue;
+        if (typeof v === 'number')
+            return 'number';
+        if (typeof v === 'string')
+            return 'string';
+        return 'other';
+    }
+    return 'number';
+}
+/**
+ * Read one value column, inferring its pond kind from what `toArray()` hands
+ * back. Numeric fast paths: `Float64Array` adopted as-is (zero copy), other
+ * number typed arrays copy through the `Float64Array` constructor (no map fn —
+ * stays off V8's slow iterable path), int64 recombines BigInt-free; a numeric
+ * column carrying nulls takes the per-element `get()` path (null → `NaN`). A
+ * plain `Array` is classified by content: `Array<string>` (Arrow `Utf8`) →
+ * `StringColumn` downstream (dict-encoded when it pays), `Array<number>` (a
+ * normalized `Date` column, epoch-ms) → `Float64Column`. Anything else (a
+ * list/struct vector) throws, naming the column.
+ */
+function readColumn(vector, name) {
+    const raw = vector.toArray();
+    if (isNumberTypedArray(raw)) {
+        if (vector.nullCount > 0)
+            return { kind: 'number', values: numericWithNulls(vector) };
+        return {
+            kind: 'number',
+            values: raw instanceof Float64Array ? raw : new Float64Array(raw),
+        };
+    }
+    if (isBigIntArray(raw)) {
+        if (vector.nullCount > 0)
+            return { kind: 'number', values: numericWithNulls(vector) };
+        return { kind: 'number', values: int64ToFloat64(raw, 1) };
+    }
+    if (Array.isArray(raw)) {
+        const kind = classifyArray(raw);
+        if (kind === 'string')
+            return { kind: 'string', values: readStrings(raw) };
+        if (kind === 'number')
+            return { kind: 'number', values: readNumbersFromArray(raw) };
+    }
+    throw new ValidationError(`fromArrow: value column '${name}' is neither numeric nor string — ` +
+        `fromArrow supports 'number' and 'string' columns; drop it or select a ` +
+        `supported subset via { columns: [...] }`);
+}
+/** Read the time column into an epoch-ms `Float64Array`, scaled by unit. */
+function readTimeColumn(vector, name, scale) {
+    if (vector.nullCount > 0) {
+        throw new ValidationError(`fromArrow: time column '${name}' has ${vector.nullCount} null value(s) ` +
+            `— time keys must be present`);
+    }
+    const raw = vector.toArray();
+    if (isBigIntArray(raw))
+        return int64ToFloat64(raw, scale);
+    if (raw instanceof Float64Array) {
+        if (scale === 1)
+            return raw;
+        const out = new Float64Array(raw.length);
+        for (let j = 0; j < raw.length; j += 1)
+            out[j] = raw[j] * scale;
+        return out;
+    }
+    if (isNumberTypedArray(raw)) {
+        const out = new Float64Array(raw);
+        if (scale !== 1)
+            for (let j = 0; j < out.length; j += 1)
+                out[j] *= scale;
+        return out;
+    }
+    // Arrow `Date32`/`Date64` `toArray()` yields a plain `Array<number>` already
+    // normalized to epoch-ms (scale is 1 for a Date key; see resolveMsScale).
+    if (Array.isArray(raw) && classifyArray(raw) === 'number') {
+        const out = readNumbersFromArray(raw);
+        if (scale !== 1)
+            for (let j = 0; j < out.length; j += 1)
+                out[j] *= scale;
+        return out;
+    }
+    throw new ValidationError(`fromArrow: time column '${name}' is not a numeric or temporal Arrow column`);
+}
+/**
+ * Translate an Arrow `Table` into the `{ name, schema, columns }` triple that
+ * `TimeSeries.fromColumns` ingests. Kept separate from the static so the
+ * conversion is unit-testable without the class and so the Arrow-shaped types
+ * live in one place. The returned columns are all `Float64Array`, so
+ * `fromColumns` takes its zero-copy adoption path throughout.
+ */
+export function arrowToColumns(table, options = {}) {
+    const fields = table.schema?.fields;
+    if (!fields || fields.length === 0) {
+        throw new ValidationError('fromArrow: table has no columns');
+    }
+    // Resolve the time column: explicit `time`, else a field named 'time'.
+    const timeName = options.time ??
+        (fields.some((f) => f.name === 'time') ? 'time' : undefined);
+    if (timeName === undefined) {
+        throw new ValidationError("fromArrow: no time column — pass { time: '<column>' } or include a " +
+            "field named 'time'");
+    }
+    const timeField = fields.find((f) => f.name === timeName);
+    if (timeField === undefined) {
+        throw new ValidationError(`fromArrow: time column '${timeName}' not found in the table schema`);
+    }
+    const timeVector = table.getChild(timeName);
+    if (timeVector == null) {
+        throw new ValidationError(`fromArrow: time column '${timeName}' has no vector`);
+    }
+    // Time unit → ms scale, gated on the Arrow type family (see resolveMsScale).
+    const scale = resolveMsScale(timeField, options.timeUnit);
+    // Value column selection: explicit `columns` (in order), else every non-time
+    // field. Every selected column must be numeric (checked in readValueColumn).
+    const valueNames = options.columns ??
+        fields.map((f) => f.name).filter((name) => name !== timeName);
+    const columns = {
+        [timeName]: readTimeColumn(timeVector, timeName, scale),
+    };
+    const schema = [
+        { name: timeName, kind: 'time' },
+    ];
+    for (const name of valueNames) {
+        if (name === timeName) {
+            throw new ValidationError(`fromArrow: column '${name}' is the time key and can't also be a ` +
+                `value column`);
+        }
+        const vector = table.getChild(name);
+        if (vector == null) {
+            throw new ValidationError(`fromArrow: column '${name}' not found in the table`);
+        }
+        const result = readColumn(vector, name);
+        columns[name] = result.values;
+        schema.push({ name, kind: result.kind });
+    }
+    return {
+        name: options.name ?? 'arrow',
+        schema: schema,
+        columns,
+    };
+}
+//# sourceMappingURL=from-arrow.js.map

package/dist/batch/operators/ingest-columns.d.ts

@@ -1,17 +1,24 @@
 import { ColumnarStore, type ColumnSchema, type KeyColumn } from '../../columnar/index.js';
-/** The per-column raw input the `fromColumns` doors accept. */
-export type RawColumns = Record<string, ReadonlyArray<number | null | undefined> | Float64Array>;
+/**
+ * The per-column raw input the `fromColumns` doors accept. A `'number'` column
+ * is a `number[]` (adopted-if-`Float64Array`); a `'string'` column is a
+ * `string[]` (`null`/`undefined` → missing). The kind is taken from the
+ * matching schema entry — the engine dispatches on it.
+ */
+export type RawColumns = Record<string, ReadonlyArray<number | null | undefined> | Float64Array | ReadonlyArray<string | null | undefined>>;
 /**
  * The shared columnar-ingress engine behind `TimeSeries.fromColumns` and
  * `ValueSeries.fromColumns`. Both doors are the same machine — normalize the
  * key column (adopt a `Float64Array` zero-copy, convert a `number[]`),
  * optionally sort by key (stable permutation, disables adoption), enforce the
- * non-decreasing-key invariant, pack each `'number'` value column
- * (`null`/`undefined` → `NaN`, one gap rule for both input types) — and differ
- * only in the key column they mint (`TimeKeyColumn` vs `ValueKeyColumn`) and
- * the words their errors use. `op` prefixes every message (so a throw names
- * the door the caller went through) and `keyNoun` names the key in the
- * out-of-order error (`timestamps` / `axis values`).
+ * non-decreasing-key invariant, pack each value column by its schema kind
+ * (`'number'` → `Float64Column`, `null`/`undefined`/non-finite → `NaN` gap;
+ * `'string'` → `StringColumn` via the shared dict-encode heuristic,
+ * `null`/`undefined` → missing) — and differ only in the key column they mint
+ * (`TimeKeyColumn` vs `ValueKeyColumn`) and the words their errors use. `op`
+ * prefixes every message (so a throw names the door the caller went through)
+ * and `keyNoun` names the key in the out-of-order error
+ * (`timestamps` / `axis values`).
  *
  * `makeKey` runs **before** the ordering scan, matching the original inline
  * order of checks: a non-finite key fails in the key-column constructor first,

package/dist/batch/operators/ingest-columns.js

@@ -1,16 +1,18 @@
-import { ColumnarStore, Float64Column, validityFromPredicate, } from '../../columnar/index.js';
+import { ColumnarStore, Float64Column, stringColumnFromArray, validityFromPredicate, } from '../../columnar/index.js';
 import { ValidationError } from '../../core/errors.js';
 /**
  * The shared columnar-ingress engine behind `TimeSeries.fromColumns` and
  * `ValueSeries.fromColumns`. Both doors are the same machine — normalize the
  * key column (adopt a `Float64Array` zero-copy, convert a `number[]`),
  * optionally sort by key (stable permutation, disables adoption), enforce the
- * non-decreasing-key invariant, pack each `'number'` value column
- * (`null`/`undefined` → `NaN`, one gap rule for both input types) — and differ
- * only in the key column they mint (`TimeKeyColumn` vs `ValueKeyColumn`) and
- * the words their errors use. `op` prefixes every message (so a throw names
- * the door the caller went through) and `keyNoun` names the key in the
- * out-of-order error (`timestamps` / `axis values`).
+ * non-decreasing-key invariant, pack each value column by its schema kind
+ * (`'number'` → `Float64Column`, `null`/`undefined`/non-finite → `NaN` gap;
+ * `'string'` → `StringColumn` via the shared dict-encode heuristic,
+ * `null`/`undefined` → missing) — and differ only in the key column they mint
+ * (`TimeKeyColumn` vs `ValueKeyColumn`) and the words their errors use. `op`
+ * prefixes every message (so a throw names the door the caller went through)
+ * and `keyNoun` names the key in the out-of-order error
+ * (`timestamps` / `axis values`).
  *
  * `makeKey` runs **before** the ordering scan, matching the original inline
  * order of checks: a non-finite key fails in the key-column constructor first,
@@ -85,12 +87,14 @@ export function ingestColumnsToStore(input) {
                 `pass { sort: true } or pre-sort the columns`);
         }
     }
-    // Value columns — packed directly (missing-aware) from the arrays.
+    // Value columns — packed directly (missing-aware) from the arrays,
+    // dispatched by the schema kind.
     const columnMap = new Map();
     for (let i = 1; i < schema.length; i += 1) {
         const def = schema[i];
-        if (def.kind !== 'number') {
-            throw new ValidationError(`${op}: v1 supports 'number' value columns; column '${def.name}' is '${def.kind}'`);
+        if (def.kind !== 'number' && def.kind !== 'string') {
+            throw new ValidationError(`${op}: supports 'number' and 'string' value columns; column ` +
+                `'${def.name}' is '${def.kind}'`);
         }
         const raw = columns[def.name];
         if (raw === undefined) {
@@ -99,6 +103,36 @@ export function ingestColumnsToStore(input) {
         if (raw.length !== count) {
             throw new ValidationError(`${op}: column '${def.name}' length ${raw.length} does not match key length ${count}`);
         }
+        if (def.kind === 'string') {
+            // String column → StringColumn (dict-encoded when it pays; see
+            // `stringColumnFromArray`). `null`/`undefined` are missing. When sorting,
+            // reorder into a fresh array through the key permutation first — strings
+            // are heap objects, so there's no zero-copy story to preserve anyway.
+            //
+            // The `columns` input type isn't correlated per-column with the schema
+            // kind (one `RawColumns` union covers both), so a numeric typed array can
+            // reach a `'string'` column at the type level. Reject that clear mismatch
+            // loudly rather than stringifying numbers — the caller crossed the schema
+            // wires. (A plain `number[]` handed to a string column is still trusted;
+            // per-cell kind-checking isn't worth the hot-path cost.)
+            if (ArrayBuffer.isView(raw)) {
+                throw new ValidationError(`${op}: string column '${def.name}' must be a string[] — got a ` +
+                    `typed array (a numeric buffer can't back a string column)`);
+            }
+            const rawStrings = raw;
+            let source;
+            if (order !== null) {
+                const reordered = new Array(count);
+                for (let j = 0; j < count; j += 1)
+                    reordered[j] = rawStrings[order[j]];
+                source = reordered;
+            }
+            else {
+                source = rawStrings;
+            }
+            columnMap.set(def.name, stringColumnFromArray(source));
+            continue;
+        }
         // Normalize to a Float64Array either way — adopt if already typed (the
         // fast path a protobuf / fixed-point decoder hits, zero-copy), else
         // convert (`null`/`undefined` -> `NaN`) — then apply ONE validity rule
@@ -111,30 +145,31 @@ export function ingestColumnsToStore(input) {
         // array type decoded it.
         // Manual loop, not `Float64Array.from(arr, mapFn)` — see the key-column
         // comment above; the cost applies identically here.
+        const numeric = raw;
         let values;
         if (order !== null) {
             // Sorting: reorder into a fresh buffer through the key permutation (no
             // zero-copy adoption — the rows are being moved). Same missing rule
             // (`null`/`undefined` → NaN) applied while remapping.
             values = new Float64Array(count);
-            if (raw instanceof Float64Array) {
+            if (numeric instanceof Float64Array) {
                 for (let j = 0; j < count; j += 1)
-                    values[j] = raw[order[j]];
+                    values[j] = numeric[order[j]];
             }
             else {
                 for (let j = 0; j < count; j += 1) {
-                    const v = raw[order[j]];
+                    const v = numeric[order[j]];
                     values[j] = v == null ? NaN : v;
                 }
             }
         }
-        else if (raw instanceof Float64Array) {
-            values = raw;
+        else if (numeric instanceof Float64Array) {
+            values = numeric;
         }
         else {
             values = new Float64Array(count);
             for (let j = 0; j < count; j += 1) {
-                const v = raw[j];
+                const v = numeric[j];
                 values[j] = v == null ? NaN : v;
             }
         }

package/dist/batch/time-series.d.ts

@@ -1,6 +1,7 @@
 import type { AlignSchema, MaterializeSchema, ArrayAggregateAppendSchema, ArrayAggregateReplaceSchema, ArrayColumnNameForSchema, ArrayExplodeAppendSchema, ArrayExplodeReplaceSchema, BaselineSchema, AggregateReducer, AggregateSchema, AppendColumn, OptionalNumberColumn, CollapseSchema, EventDataForSchema, EventForSchema, FirstColKind, IntervalKeyedSchema, JsonRowFormat, JoinManySchema, JoinSchema, JoinType, NumericColumnNameForSchema, NormalizedObjectRow, NormalizedRowForSchema, PivotByGroupSchema, PointRowForSchema, PrefixedJoinManySchema, PrefixedJoinSchema, ReduceResult, RenameMap, ValidatedAggregateMap } from '../schema/index.js';
 import type { RenameSchema, RollingAlignment, RollingSchema, ColumnValue, DedupeKeep, DiffSchema, FillMapping, FillStrategy, ScalarKind, ScalarValue, SmoothMethod, SmoothAppendSchema, SmoothSchema, SelectSchema, SeriesSchema, TimeKeyedSchema, TimeSeriesJsonInput, TimeSeriesInput, TimeRangeKeyedSchema, ValueColumnKindForName, ValueColumnNameForSchema, ValueColumnsForSchema, ValueKeyedSchema } from '../schema/index.js';
 import { type ScanStep } from './operators/scan.js';
+import { type ArrowTableLike, type FromArrowOptions } from './operators/from-arrow.js';
 import { ValueSeries } from './value-series.js';
 import { type BinSpec } from './by-column.js';
 import { type WindowSpec } from './rolling-by-column.js';
@@ -134,9 +135,10 @@ export declare class TimeSeries<S extends SeriesSchema> {
      * — clamp the key column yourself before ingest; the library won't mutate data
      * for you.)
      *
-     * **v1 scope:** a `time`-kind key + `number` value columns (the market-data /
-     * chart wire case). Other key kinds (`interval` / `timeRange`) and non-numeric
-     * value columns throw for now — extend as consumers need.
+     * **Value columns:** `number` (→ `Float64Column`; `Float64Array` adopted) and
+     * `string` (→ `StringColumn`, dict-encoded when it pays; `null`/`undefined`
+     * missing). Other key kinds (`interval` / `timeRange`) and other value kinds
+     * (`boolean` / array) throw for now — extend as consumers need.
      *
      * @throws ValidationError on a missing column, a length mismatch, an
      *   unsupported kind, or an out-of-order (decreasing) timestamp when `sort`
@@ -147,7 +149,7 @@ export declare class TimeSeries<S extends SeriesSchema> {
     static fromColumns<S extends SeriesSchema>(input: {
         name: string;
         schema: S;
-        columns: Record<string, ReadonlyArray<number | null | undefined> | Float64Array>;
+        columns: Record<string, ReadonlyArray<number | null | undefined> | Float64Array | ReadonlyArray<string | null | undefined>>;
         /**
          * Sort the rows by key before construction (off by default), for a columnar
          * payload whose rows aren't guaranteed ordered — the counterpart of
@@ -156,6 +158,43 @@ export declare class TimeSeries<S extends SeriesSchema> {
          */
         sort?: boolean;
     }): TimeSeries<S>;
+    /**
+     * Example: `TimeSeries.fromArrow(tableFromIPC(bytes), { time: 'ts' })`.
+     *
+     * Build a time series from a decoded Apache Arrow `Table`. pond does **not**
+     * depend on `apache-arrow` — bring your own (`tableFromIPC(...)` /
+     * `tableFromArrays(...)`) and hand the `Table` here; the input is duck-typed
+     * against the small {@link ArrowTableLike} slice we read. Ingest is the
+     * zero-copy path: every **single-chunk** `Float64` column's backing
+     * `Float64Array` is adopted as-is, and the time key is converted
+     * **BigInt-free** — Arrow's idiomatic int64 timestamps are recombined from
+     * their two int32 halves rather than `Number(bigint)` per row (which costs
+     * ~30ms at 500k rows). A table decoded from a multi-record-batch IPC stream
+     * has multi-chunk columns, whose `toArray()` concatenates into a fresh buffer
+     * — still correct, but a copy rather than an adopt.
+     *
+     * Column handling:
+     * - **Time key** — named by `time` (default: a field named `'time'`).
+     *   Timestamp/int/float/Date sources accepted and normalized to epoch ms: a
+     *   `Timestamp`'s raw-unit int64 is scaled by its `TimeUnit`; a `Date32`/
+     *   `Date64` already arrives as epoch-ms via Arrow's `toArray()` and passes
+     *   through; a plain int/float is taken as ms. `timeUnit` overrides.
+     * - **Value columns** — every non-time field by default, or the subset named
+     *   by `columns` (in order). Numeric columns (`Float32`/int convert to
+     *   `Float64Array`; int64 recombines BigInt-free; nulls → `NaN`) and string
+     *   columns (Arrow `Utf8` → `StringColumn`, dict-encoded when it pays; nulls →
+     *   missing) are supported. Any other Arrow type (list/struct/…) throws,
+     *   naming it. A null in the time key throws.
+     *
+     * The rows must be time-ordered (as `fromColumns` requires); pass
+     * `{ sort: true }` for an unordered table (disables zero-copy adoption).
+     *
+     * **Trust contract on the type parameter:** the runtime schema is derived
+     * from the Arrow fields. Supplying `S` (`fromArrow<MySchema>(...)`) is a
+     * downstream-typing convenience taken on trust — pond does not verify the
+     * Arrow fields match `S`, exactly as `fromEvents` trusts its schema.
+     */
+    static fromArrow<S extends SeriesSchema = SeriesSchema>(table: ArrowTableLike, options?: FromArrowOptions): TimeSeries<S>;
     /**
      * Example: `TimeSeries.fromEvents(events, { schema, name })`.
      * Builds a typed series from an array of `Event` instances. The events

package/dist/batch/time-series.js

@@ -1,8 +1,9 @@
-import { isAggregateOutputSpec, normalizeAggregateColumns, tryAggregateColumnarTimeKeyed, } from './aggregate-columns.js';
+import { isAggregateOutputSpec, normalizeAggregateColumns, tryAggregateColumnarTimeKeyed, tryRollingCountColumnarNumeric, } from './aggregate-columns.js';
 import { cumulativeOp, } from './operators/cumulative.js';
 import { scanOp } from './operators/scan.js';
 import { byValueOp } from './operators/by-value.js';
 import { ingestColumnsToStore } from './operators/ingest-columns.js';
+import { arrowToColumns, } from './operators/from-arrow.js';
 import { ValueSeries } from './value-series.js';
 import { diffRateOp } from './operators/diff-rate.js';
 import { fillOp } from './operators/fill.js';
@@ -19,7 +20,7 @@ import { TimeRange } from '../core/time-range.js';
 import { compareEventKeys } from '../core/temporal.js';
 import { PartitionedTimeSeries } from './partitioned-time-series.js';
 import { Sequence } from '../sequence/sequence.js';
-import { ColumnarStore, IntervalKeyColumn, TimeKeyColumn, TimeRangeKeyColumn, float64ColumnFromArray, stringColumnFromArray, withColumnAppended, withColumnsRenamed, withColumnsSelected, withKeyColumn, withRowRange, withRowSelection, } from '../columnar/index.js';
+import { ColumnarStore, Float64Column, IntervalKeyColumn, TimeKeyColumn, TimeRangeKeyColumn, bitmapByteCount, float64ColumnFromArray, stringColumnFromArray, validityFromBits, withColumnAppended, withColumnReplaced, withColumnsRenamed, withColumnsSelected, withKeyColumn, withRowRange, withRowSelection, } from '../columnar/index.js';
 import { ValidationError } from '../core/errors.js';
 import { SeriesStore } from '../live/series-store.js';
 import { parseDuration } from '../core/duration.js';
@@ -539,9 +540,10 @@ export class TimeSeries {
      * — clamp the key column yourself before ingest; the library won't mutate data
      * for you.)
      *
-     * **v1 scope:** a `time`-kind key + `number` value columns (the market-data /
-     * chart wire case). Other key kinds (`interval` / `timeRange`) and non-numeric
-     * value columns throw for now — extend as consumers need.
+     * **Value columns:** `number` (→ `Float64Column`; `Float64Array` adopted) and
+     * `string` (→ `StringColumn`, dict-encoded when it pays; `null`/`undefined`
+     * missing). Other key kinds (`interval` / `timeRange`) and other value kinds
+     * (`boolean` / array) throw for now — extend as consumers need.
      *
      * @throws ValidationError on a missing column, a length mismatch, an
      *   unsupported kind, or an out-of-order (decreasing) timestamp when `sort`
@@ -572,6 +574,51 @@ export class TimeSeries {
         });
         return TimeSeries.#fromTrustedStore(name, schema, store);
     }
+    /**
+     * Example: `TimeSeries.fromArrow(tableFromIPC(bytes), { time: 'ts' })`.
+     *
+     * Build a time series from a decoded Apache Arrow `Table`. pond does **not**
+     * depend on `apache-arrow` — bring your own (`tableFromIPC(...)` /
+     * `tableFromArrays(...)`) and hand the `Table` here; the input is duck-typed
+     * against the small {@link ArrowTableLike} slice we read. Ingest is the
+     * zero-copy path: every **single-chunk** `Float64` column's backing
+     * `Float64Array` is adopted as-is, and the time key is converted
+     * **BigInt-free** — Arrow's idiomatic int64 timestamps are recombined from
+     * their two int32 halves rather than `Number(bigint)` per row (which costs
+     * ~30ms at 500k rows). A table decoded from a multi-record-batch IPC stream
+     * has multi-chunk columns, whose `toArray()` concatenates into a fresh buffer
+     * — still correct, but a copy rather than an adopt.
+     *
+     * Column handling:
+     * - **Time key** — named by `time` (default: a field named `'time'`).
+     *   Timestamp/int/float/Date sources accepted and normalized to epoch ms: a
+     *   `Timestamp`'s raw-unit int64 is scaled by its `TimeUnit`; a `Date32`/
+     *   `Date64` already arrives as epoch-ms via Arrow's `toArray()` and passes
+     *   through; a plain int/float is taken as ms. `timeUnit` overrides.
+     * - **Value columns** — every non-time field by default, or the subset named
+     *   by `columns` (in order). Numeric columns (`Float32`/int convert to
+     *   `Float64Array`; int64 recombines BigInt-free; nulls → `NaN`) and string
+     *   columns (Arrow `Utf8` → `StringColumn`, dict-encoded when it pays; nulls →
+     *   missing) are supported. Any other Arrow type (list/struct/…) throws,
+     *   naming it. A null in the time key throws.
+     *
+     * The rows must be time-ordered (as `fromColumns` requires); pass
+     * `{ sort: true }` for an unordered table (disables zero-copy adoption).
+     *
+     * **Trust contract on the type parameter:** the runtime schema is derived
+     * from the Arrow fields. Supplying `S` (`fromArrow<MySchema>(...)`) is a
+     * downstream-typing convenience taken on trust — pond does not verify the
+     * Arrow fields match `S`, exactly as `fromEvents` trusts its schema.
+     */
+    static fromArrow(table, options = {}) {
+        const { name, schema, columns } = arrowToColumns(table, options);
+        return TimeSeries.fromColumns({
+            name,
+            schema: schema,
+            columns,
+            sort: options.sort ?? false,
+        });
+    }
     /**
      * Example: `TimeSeries.fromEvents(events, { schema, name })`.
      * Builds a typed series from an array of `Event` instances. The events
@@ -2205,6 +2252,23 @@ export class TimeSeries {
         // materialization, no per-row `Event`, no row re-validation/re-pack.
         const store = this.#store.store;
         const rowCount = store.length;
+        // Count-window numeric fast path: an all-built-in numeric mapping over
+        // packed sources builds its typed result columns in one sweep (see
+        // `tryRollingCountColumnarNumeric`) — none of the generic sweep's
+        // per-row snapshot arrays, boxed accumulators, or post-pass re-pack.
+        // `null` (a custom reducer, non-number output, chunked source) falls
+        // through to the generic sweep below.
+        if (countWindow !== null) {
+            const fastColumns = tryRollingCountColumnarNumeric((name) => store.columns.get(name), rowCount, columnSpecs, countWindow, alignment, minSamples);
+            if (fastColumns !== null) {
+                const fastOutColumns = new Map();
+                for (let c = 0; c < columnSpecs.length; c += 1) {
+                    fastOutColumns.set(columnSpecs[c].output, fastColumns[c]);
+                }
+                const fastStore = ColumnarStore.fromTrustedStore(resultSchema, store.keys, fastOutColumns);
+                return TimeSeries.#fromTrustedStore(this.name, resultSchema, fastStore);
+            }
+        }
         const sourceCols = columnSpecs.map((spec) => store.columns.get(spec.source));
         const reducerStates = columnSpecs.map((spec) => isBuiltInAggregateReducer(spec.reducer)
             ? createRollingReducerState(spec.reducer)
@@ -2434,11 +2498,6 @@ export class TimeSeries {
         const resultSchema = output === undefined
             ? makeSmoothSchema(this.schema, column)
             : makeSmoothSchema(this.schema, column, output);
-        const anchors = this.events.map((event) => eventAnchorTime(event.key()));
-        const sourceValues = this.events.map((event) => {
-            const raw = event.get(column);
-            return typeof raw === 'number' ? raw : undefined;
-        });
         if (method === 'ema') {
             // Rate parameter: `alpha` directly, or the financial `span` convention
             // (α = 2/(span+1) — a `span`-period EMA), exactly one of the two.
@@ -2488,6 +2547,63 @@ export class TimeSeries {
                 !Number.isFinite(minSamples)) {
                 throw new TypeError('ema smoothing requires minSamples to be a non-negative integer');
             }
+            // Columnar fast path: on a packed numeric source column the EMA
+            // recurrence runs straight off the typed buffer into a typed result
+            // column — no event materialization, no per-row Event / frozen-tuple
+            // allocation, and no full-series intake re-pack (the key and every
+            // untouched column pass through by reference). Emitted values are the
+            // exact sequence the event path emits (same reads, same arithmetic),
+            // and each one is asserted finite so the intake rejection the event
+            // path gets from its row re-pack is preserved. Non-packed sources
+            // (chunked storage) fall back to the event path below.
+            const emaSourceCol = this.#store.store.columns.get(column);
+            if (emaSourceCol !== undefined &&
+                emaSourceCol.kind === 'number' &&
+                emaSourceCol.storage === 'packed') {
+                const store = this.#store.store;
+                const packed = emaSourceCol;
+                const rowCount = store.length;
+                const src = packed._values;
+                const srcValidity = packed.validity;
+                const out = new Float64Array(rowCount);
+                const outBits = new Uint8Array(bitmapByteCount(rowCount));
+                let outDefined = 0;
+                let hasPrevious = false;
+                let previous = 0;
+                let seen = 0;
+                for (let i = 0; i < rowCount; i += 1) {
+                    if (srcValidity !== undefined && !srcValidity.isDefined(i)) {
+                        continue; // missing source cell — emitted value is missing too
+                    }
+                    const raw = src[i];
+                    const smoothed = hasPrevious
+                        ? alpha * raw + (1 - alpha) * previous
+                        : raw;
+                    previous = smoothed;
+                    hasPrevious = true;
+                    seen += 1;
+                    // Mask the emitted value during the length-preserving warm-up;
+                    // `previous` / `seen` advanced on the real value above.
+                    if (seen < minSamples)
+                        continue;
+                    if (!Number.isFinite(smoothed)) {
+                        throw new ValidationError(`smooth column '${output ?? column}': result ${smoothed} is not a valid 'number' value`);
+                    }
+                    out[i] = smoothed;
+                    outBits[i >> 3] |= 1 << (i & 7);
+                    outDefined += 1;
+                }
+                const outValidity = outDefined === rowCount
+                    ? undefined
+                    : validityFromBits(outBits, rowCount);
+                // Every defined cell was asserted finite above → `allFinite`.
+                const outColumn = new Float64Column(out, rowCount, outValidity, true);
+                const reshaped = output === undefined || output === column
+                    ? withColumnReplaced(store, column, outColumn)
+                    : withColumnAppended(store, output, outColumn);
+                const kept = warmup > 0 ? withRowRange(reshaped, warmup, rowCount) : reshaped;
+                return TimeSeries.#fromTrustedStore(this.name, resultSchema, kept);
+            }
             let previous;
             let seen = 0;
             const resultRows = this.events.map((event) => {
@@ -2521,6 +2637,14 @@ export class TimeSeries {
                 rows: keptRows,
             });
         }
+        // Anchor times + source values for the window-based methods
+        // (movingAverage / loess). Materialized here — below the ema branch —
+        // because ema never reads either (its fast path stays event-free).
+        const anchors = this.events.map((event) => eventAnchorTime(event.key()));
+        const sourceValues = this.events.map((event) => {
+            const raw = event.get(column);
+            return typeof raw === 'number' ? raw : undefined;
+        });
         if (method === 'loess') {
             if (!('span' in options)) {
                 throw new TypeError('loess smoothing requires a span option');

package/dist/batch/value-series.d.ts

@@ -57,10 +57,10 @@ export declare class ValueSeries<VS extends ValueSeriesSchema> {
      * stable sort every unordered snapshot wants (e.g. a keyed live feed that
      * delivers rows in update order, not axis order).
      *
-     * **v1 scope:** `number` value columns, matching `TimeSeries.fromColumns`.
+     * **Value columns:** `number` and `string`, matching `TimeSeries.fromColumns`.
      *
      * @throws ValidationError on a non-`'value'` axis kind, a missing column, a
-     *   length mismatch, a non-`'number'` value column, or an out-of-order axis
+     *   length mismatch, an unsupported value-column kind, or an out-of-order axis
      *   when `sort` is not set. Throws RangeError on a non-finite
      *   (`null`/`NaN`/`±Infinity`) axis cell — sorting can't make it valid — or
      *   a duplicate column name (the axis name repeated among the value columns).
@@ -68,7 +68,7 @@ export declare class ValueSeries<VS extends ValueSeriesSchema> {
     static fromColumns<VS extends ValueSeriesSchema>(input: {
         name: string;
         schema: VS;
-        columns: Record<string, ReadonlyArray<number | null | undefined> | Float64Array>;
+        columns: Record<string, ReadonlyArray<number | null | undefined> | Float64Array | ReadonlyArray<string | null | undefined>>;
         /**
          * Sort the rows by axis value before construction (off by default), for a
          * payload whose rows aren't guaranteed ordered. Stable; disables the

package/dist/batch/value-series.js

@@ -67,10 +67,10 @@ export class ValueSeries {
      * stable sort every unordered snapshot wants (e.g. a keyed live feed that
      * delivers rows in update order, not axis order).
      *
-     * **v1 scope:** `number` value columns, matching `TimeSeries.fromColumns`.
+     * **Value columns:** `number` and `string`, matching `TimeSeries.fromColumns`.
      *
      * @throws ValidationError on a non-`'value'` axis kind, a missing column, a
-     *   length mismatch, a non-`'number'` value column, or an out-of-order axis
+     *   length mismatch, an unsupported value-column kind, or an out-of-order axis
      *   when `sort` is not set. Throws RangeError on a non-finite
      *   (`null`/`NaN`/`±Infinity`) axis cell — sorting can't make it valid — or
      *   a duplicate column name (the axis name repeated among the value columns).

package/dist/index.d.ts

@@ -19,6 +19,7 @@ export { Time } from './core/time.js';
 export { TimeRange, toTimeRange } from './core/time-range.js';
 export { Sequence } from './sequence/sequence.js';
 export { TimeSeries, type KeyLike } from './batch/time-series.js';
+export type { ArrowTableLike, ArrowVectorLike, ArrowFieldLike, ArrowSchemaLike, ArrowTimeUnit, FromArrowOptions, } from './batch/operators/from-arrow.js';
 export { ValueSeries } from './batch/value-series.js';
 export { top } from './reducers/index.js';
 export { ValidationError } from './core/errors.js';

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pond-ts",
-  "version": "0.50.0",
+  "version": "0.52.0",
   "description": "TypeScript-first time series primitives",
   "license": "MIT",
   "repository": {
@@ -43,6 +43,7 @@
   "devDependencies": {
     "@babel/runtime": "^7.29.2",
     "@types/node": "^20.17.0",
+    "apache-arrow": "^21.2.0",
     "pondjs": "^0.9.0",
     "prettier": "^3.8.3",
     "typescript": "^5.6.3",