pond-ts 0.17.0 → 0.18.0

package/CHANGELOG.md

@@ -7,10 +7,187 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 file covers both packages. Pre-1.0: minor bumps may include new features and
 type-level changes; patch bumps are strictly additive.
 
-[Unreleased]: https://github.com/pjm17971/pond-ts/compare/v0.17.0...HEAD
+[Unreleased]: https://github.com/pjm17971/pond-ts/compare/v0.18.0...HEAD
+[0.18.0]: https://github.com/pjm17971/pond-ts/compare/v0.17.1...v0.18.0
 
 ## [Unreleased]
 
+## [0.18.0] — 2026-05-30
+
+This release graduates the **Phase 4.7 columnar substrate** from
+framework-internal (shipped piecemeal to `main` since v0.17.1) to a
+user-visible **public column API**, plus a column-native live buffer that
+fixes a high-partition-count OOM. Everything is additive except one
+documented breaking change (interval label kinds) and one documented
+behavior change (chunked-backed `pushMany` commit semantics). Pre-1.0: the
+column API is expected to keep moving toward its eventual shape.
+
+### Added
+
+- **Public column API (Phase 4.7 step 8).** A column-centric extraction
+  surface on `TimeSeries`, for high-throughput and charting consumers that
+  want typed-array access instead of per-`Event` iteration. Additive — every
+  existing row / `Event` API is unchanged.
+  - `series.column(name)` returns a schema-narrowed typed column view, with
+    public re-exports of the `Float64Column` / `BooleanColumn` /
+    `StringColumn` / `KeyColumn` (time / timeRange / interval) variants
+    ([#154](https://github.com/pjm17971/pond-ts/pull/154),
+    [#155](https://github.com/pjm17971/pond-ts/pull/155)).
+  - `Float64Column`: scalar reductions (`min` / `max` / `sum` / `mean` /
+    `count` / …) and `scan`
+    ([#155](https://github.com/pjm17971/pond-ts/pull/155)); `bin(...)` for
+    histogram / downsample bucketing
+    ([#156](https://github.com/pjm17971/pond-ts/pull/156)); and
+    `toFloat64Array()` for a storage-agnostic gather into a dense array
+    ([#165](https://github.com/pjm17971/pond-ts/pull/165)).
+  - `KeyColumn.at(i)` and `.slice(start, end)`
+    ([#159](https://github.com/pjm17971/pond-ts/pull/159)).
+- **Columnar substrate (Phase 4.7 step 1, framework layer).** All
+  eight sub-steps (1a–1h) shipped to main as PRs #132 / #133 /
+  #134 / #135 / #136 / #147 / #148 / #149. See `PLAN.md` and
+  [`packages/core/src/columnar/README.md`](packages/core/src/columnar/README.md)
+  for the full inventory. Framework-internal — surfaced behind the existing
+  `TimeSeries` API at step 2 (below) and the public column API at step 8
+  (above).
+
+### Changed
+
+- **Chunked columnar live backing for strict time-keyed `LiveSeries`**
+  ([#170](https://github.com/pjm17971/pond-ts/pull/170)). A top-level
+  `LiveSeries` with `ordering: 'strict'` and a time key now backs its
+  retained window with batch-granular columnar chunks instead of an
+  `Event[]` window — each `pushMany` validates straight into typed columns,
+  retaining **zero `Event` objects** (~4.7× less retained heap in-pond; the
+  high-partition-count OOM fix). Two consequences:
+  - **`pushMany` commit semantics** on the chunked path: the batch is
+    appended atomically _before_ any `'event'` fires, so a listener observes
+    the full post-batch `length` (not a row-by-row `1, 2, 3`), and a listener
+    that throws mid-batch leaves the whole batch committed. The per-row
+    `Event[]` backing (`reorder` / `drop` / interval-keyed /
+    internally-created series) keeps per-row commit. Listener _values_ and
+    `event → batch → evict` ordering are unchanged.
+  - **`LiveReduce` eviction** resolves by event identity (primary) with a
+    FIFO-frontier fallback for the chunked backing's materialized evictions —
+    correct for both `reorder` and the chunked backing. `min` / `max` /
+    `first` / `last` / `samples` over a `reorder` source **with retention**
+    remain a documented limitation (see `LiveReduce` JSDoc and PLAN
+    "Deferred") — pre-existing, not introduced here.
+- **Internal, behavior-preserving performance work.** Column-native intake
+  bypasses per-row `Event` allocation at `TimeSeries` construction
+  ([#151](https://github.com/pjm17971/pond-ts/pull/151)); numeric reducers
+  (`min` / `max` / `sum` / `avg` / …) compute over typed-array columns where
+  available, with NaN parity preserved
+  ([#153](https://github.com/pjm17971/pond-ts/pull/153)); the live storage
+  strategy was extracted behind an internal interface
+  ([#168](https://github.com/pjm17971/pond-ts/pull/168)).
+
+### Changed (BREAKING)
+
+- **Interval-keyed series must use one label type throughout**
+  ([#150](https://github.com/pjm17971/pond-ts/pull/150)). Pre-2a,
+  TimeSeries silently tolerated mixed-kind interval labels —
+  rows with `value: 'row-1'` (string) and `value: 2` (number) could
+  coexist in a single series because events were stored as a raw
+  array with no per-column type alignment. The columnar substrate
+  introduced at Phase 4.7 enforces one label kind per column via
+  `IntervalKeyColumn`, so mixed-kind labels now throw at series
+  construction with a row-pointed error message.
+  - **Affected:** Any series built via `new TimeSeries(...)`,
+    `TimeSeries.fromJSON(...)`, `TimeSeries.fromEvents(...)`, or
+    any transform that produces interval-keyed events, where the
+    `value` field of `IntervalInput` rows or `Interval` keys
+    mixes `string` and `number` types.
+  - **Migration:** Choose one label kind for the whole series.
+    Numeric labels can be stringified at intake (`String(label)`)
+    if the downstream consumer accepts string equality; string
+    labels parseable as integers can be converted to numbers at
+    intake. The error message names the first offending row so
+    the offending data is easy to find.
+  - **Rationale:** Aligns the row-API contract with the columnar
+    substrate's per-column kind discipline (matching Polars /
+    Arrow / Parquet). The previous behavior produced type-broken
+    events that worked only because TimeSeries didn't enforce
+    per-column alignment; downstream columnar operators (the
+    upcoming reducer adaptation in steps 3+) require it.
+  - **Affected types:** `IntervalValue` remains `string | number`
+    per the `Interval` class contract. The runtime restriction
+    is at the **series** level (all intervals within one series
+    must share a kind), not the per-interval level. Type-level
+    narrowing of `IntervalKeyedSchema<S>` over label kind is a
+    follow-up deferred to a later sub-step.
+
+## [0.17.1] — 2026-05-11
+
+Bug fix: `live.partitionBy()` now default-inherits `ordering`,
+`graceWindow`, and `retention` from the source `LiveSeries`. Surfaced
+by the gRPC experiment's
+[M4 late-data friction note](https://github.com/pjm17971/pond-grpc-experiment/blob/main/friction-notes/M4.md),
+which measured `99.5%` of late events crashing the partition router
+under `source = LiveSeries({ ordering: 'reorder', graceWindow })`
+followed by bare `partitionBy('host')`.
+
+### Fixed
+
+- **`LiveSeries.partitionBy(by)` default-inherits source config**
+  ([#TBD](https://github.com/pjm17971/pond-ts/pull/TBD)). Pre-fix,
+  per-partition sub-series were constructed with default
+  `ordering: 'strict'` regardless of source mode. Under a `'reorder'`
+  source, late events that the source accepted via its reorder path
+  were routed into the partition's `#insert` and threw with a
+  strict-mode error; the throw propagated back through the source's
+  listener fan-out into `live.push()`.
+
+  Post-fix, `partitionBy(by)` defaults each per-partition sub-series'
+  `ordering`, `graceWindow`, and `retention` to the source's values.
+  Explicit options on `partitionBy(by, { ordering, ... })` override
+  per-field. `graceWindow` inheritance is gated on effective ordering
+  being `'reorder'` (LiveSeries rejects strict + graceWindow combos).
+
+  ```ts
+  // Pre-0.17.1: crashed the partition router
+  const live = new LiveSeries({
+    name: 'metrics',
+    schema,
+    ordering: 'reorder',
+    graceWindow: '30s',
+  });
+  live.partitionBy('host'); // ← partition was strict regardless
+
+  // Post-0.17.1: partitions inherit reorder + 30s grace; late events
+  // accept correctly via the reorder path.
+  ```
+
+  Existing callers with explicit `partitionBy(by, { ordering, ... })`:
+  unchanged. Existing callers on `'strict'` sources: unchanged.
+  Existing callers on `'reorder'` sources with bare `partitionBy`:
+  the previously-thrown late events now accept correctly — bug fix,
+  not a behavior change anyone could rely on.
+
+- **`collect()` and `apply()` on `LivePartitionedSeries` default-
+  inherit `ordering` and `graceWindow`** from the partitioned series
+  (which inherits from source). Pre-fix, the unified buffer defaulted
+  to `'strict'`, so partition fan-in on a `'reorder'` source could
+  deliver events out-of-order to a strict unified buffer and throw.
+  Retention stays caller-explicit on these per the existing append-
+  only fan-in semantics.
+
+### Notes
+
+- **Six regression tests pin the new defaults** in
+  `LivePartitionedSeries.test.ts`: inherited ordering, inherited
+  graceWindow within reorder, inherited retention on partitions,
+  explicit override of inheritance, strict-source no-change, and the
+  edge case where overriding ordering to strict suppresses graceWindow
+  inheritance. `collect()` inheritance pinned separately.
+- The gRPC experiment's M4 friction note also surfaced milestone B
+  (capability-based late repair) as **driver-light by empirical test**
+  after Codex's adversarial pass caught simulator RNG leakage across
+  A/B legs. Drift signal collapsed to within noise on every host once
+  all randomness sources were seeded — milestone B's library design
+  stays sound, but the gRPC experiment's measurement style (last-tick
+  `.value()` reads) doesn't surface its payoff. Milestone B sequencing
+  updated in PLAN.md to reflect this finding.
+
 ## [0.17.0] — 2026-05-08
 
 `sample({...})` operator wave: bounded-memory stream thinning, surfaced
@@ -385,6 +562,7 @@ compaction); any downstream code reading `#entries` directly would
 break, but those fields are private. Public APIs and types are
 unchanged.
 
+[0.17.1]: https://github.com/pjm17971/pond-ts/compare/v0.17.0...v0.17.1
 [0.17.0]: https://github.com/pjm17971/pond-ts/compare/v0.16.1...v0.17.0
 [0.16.1]: https://github.com/pjm17971/pond-ts/compare/v0.16.0...v0.16.1
 [0.16.0]: https://github.com/pjm17971/pond-ts/compare/v0.15.2...v0.16.0