@clickhouse/client 1.22.0 → 1.23.0-head.287977a.1

package/skills/clickhouse-js-node-rowbinary-parser/SKILL.md

@@ -0,0 +1,206 @@
+---
+name: clickhouse-js-node-rowbinary-parser
+description: >
+  Generate TypeScript/JavaScript code that reads and decodes ClickHouse
+  RowBinary streams from the ClickHouse HTTP server.
+  Use this skill whenever a user wants to parse `RowBinary`,
+  `RowBinaryWithNames`, or `RowBinaryWithNamesAndTypes`.
+  Node.js only, doesn't cover browsers.
+---
+
+# ClickHouse JS RowBinary Parser Generator for Node.js
+
+## First: is RowBinary even the right format?
+
+RowBinary exists for throughput, but it is **not automatically the fastest
+path** — match the format to the shape of the data before committing to a
+bespoke parser.
+
+**Prefer a `JSON*` format (e.g. `JSONEachRow`) when** the result is mostly
+strings / JSON-like values that you consume wholesale — randomly accessing
+essentially every field, running string/regexp methods on them, treating values
+as text. V8's native `JSON.parse` is heavily optimized C++ and builds JS strings
+and objects faster than a JS-level RowBinary decoder can; pair it with HTTP
+response compression (`gzip` / `zstd`, which crushes JSON's repetitive keys) and
+the wire cost shrinks too.
+
+**RowBinary clearly wins when** the result is dominated by:
+
+- **Wide numerics** — `Int128`/`Int256`/`UInt128`/`UInt256`,
+  `Decimal128`/`Decimal256`.
+- **Binary / fixed-width blobs** — `IPv4`, `IPv6`, `UUID`, `FixedString`.
+- **High-volume fixed-width numeric columns** generally, where each value is a
+  single `DataView` read.
+
+**Prefer the `Native` format when** columnar load and client-side analytics are
+the main goal (fold/scan/filter columns, feed typed arrays to a Worker or WASM).
+`Native` is column-major, so it loads straight into one typed array per column
+with no transpose.
+
+For help choosing and consuming a `JSON*` format (or CSV / TSV) instead, use the
+**`clickhouse-js-node-coding`** skill.
+
+## Second: complete buffer, or incremental stream?
+
+Decide this before writing the reader — it changes the shape of the code and is
+a real performance fork.
+
+- **Incremental / streaming (the default here).** You consume the HTTP response
+  chunk by chunk as it arrives — low latency to the first row, bounded memory.
+  It is generally the best choice for large results, but slower per-row.
+
+- **Whole buffer in memory (faster, when it fits).** If you already hold the
+  entire response as one `Buffer`, the bounds check never fires — so you can drop
+  `advance()` entirely and read at a running offset in one monolithic loop.
+  This is 2-3x faster but introduces latency and unbounded memory use.
+
+The exposed API is streaming by default and requires an optimisation pass.
+
+## Third: row objects, or columnar (typed arrays)?
+
+The default output is one object per row (array-of-structs). For a **numeric,
+fixed-width result that the consumer reads column-wise**, decode instead into one
+typed array per column (struct-of-arrays) — it is **~4x faster and several times
+smaller** because it removes the per-row object, `Date`, and number-boxing
+allocations that dominate a numeric decode (the byte reads are already at memory
+bandwidth). Measured in `tests/iot.columnar.bench.ts`; rationale in
+`case-studies/wasm-vs-js.md`.
+
+- **Use columnar when** columns are numeric/fixed-width and the consumer
+  aggregates / filters / scans / plots them, or hands the buffers to a Worker or
+  WASM kernel (typed-array `ArrayBuffer`s are transferable — zero-copy).
+- **The preallocation trick:** if EVERY column is fixed-width the row stride is
+  known, so the exact count is `buf.length / stride` — allocate each column once,
+  write at `[i]`, no growth, no per-row bounds check.
+- **Streaming columnar is just that arithmetic per chunk.** Fixed width means
+  honoring a partial buffer needs no `advance()`/`NeedMoreData`/restart: the
+  complete-row count is `(chunk.length / stride) | 0`, and the leftover bytes
+  carry to the next chunk. Yield one typed-array batch per chunk, each owning a
+  fresh transferable `ArrayBuffer` (see `streamSensorColumns` in
+  `src/columnar.ts`).
+- **Stay row-oriented when** downstream code is row-shaped, the row is
+  string-dominated (columnar's win is numeric — a JS string allocates either
+  way), or the schema is nested/heterogeneous (`Array`/`Map`/`Tuple`).
+- **Hybrid:** store columnar, expose a lazy `rowAt(i)` accessor that builds an
+  object only for rows actually touched (see `iotRowAt` in `src/examples/iot.ts`).
+
+## Fourth: are the column types known ahead of time?
+
+- **Known (the default).** Generate a straight-line reader specialized to those
+  types — everything below.
+- **Only at runtime** (the schema varies, or you just want to decode an arbitrary
+  `RowBinaryWithNamesAndTypes` stream). Call
+  `compileRowBinaryWithNamesAndTypes(cursor)` (`src/rowBinaryWithNamesAndTypes.ts`):
+  it reads the header, folds each column type's AST into a `Reader`
+  (`astToReader`, `src/compile.ts`; type strings parsed by
+  `@clickhouse/datatype-parser`), and returns a `readRows` driver for the rest of
+  the stream. Generic and unoptimized (no codegen), so prefer the specialized
+  path whenever the types are fixed.
+
+## Core guidance
+
+When generating a parser, follow these:
+
+- **Little-endian only.** RowBinary is little-endian; target x86/ARM. Read every
+  multi-byte number with `DataView` accessors passing a **literal** `true` for
+  the `littleEndian` flag.
+
+- **Correct first, then optimize.** First emit a correct reader built from the
+  plain per-type API. Only after it's correct (and tested) specialize it. Don't
+  bake performance assumptions in before correctness.
+
+- **Monomorphize generic/composite types.** Emit specialized, inlined code per
+  type combination instead of passing functions as arguments where the type
+  is known ahead of time.
+
+- **Streaming: throw + restart, not generators.** To signal "need more bytes",
+  a synchronous reader that throws a sentinel (`NeedMoreData`) and restarts the
+  row beats generators for realistic chunk sizes;
+
+- **Keep an eye on chunk sizes.** Partial trailing rows, small chunks are a silent
+  throughput killer: `streamRowBatches` warns once when
+  rows-per-chunk falls too low, and `coalesceChunks(source, { minSize, timeoutMs })`
+  merges small chunks in front of it when the source size isn't yours to raise.
+
+- **Shared scratch is not reentrant.** Some hot methods reuse a module-level
+  scratch buffer as a write-then-read pair — correct only because reads are fully
+  synchronous. An `async`/`yield` boundary between populating and reading it
+  corrupts the value.
+
+- **Hoist the cursor into locals.** Prefer the working buffer and view declared
+  once at the top of the generated reader, and keep the read offset in a **local variable**,
+  operating on it directly instead of re-reading from an object.
+
+- **Coalesce `advance()` across adjacent fixed-width columns.** A run of
+  neighbouring fixed-width columns has a known combined size, so bounds-check it
+  ONCE.
+
+- **Inline the leaf reads.** The per-type `readX` functions are the correct,
+  composable reference; the generated parser should INLINE their bodies, not call
+  them, so the row reader is straight-line with no per-field indirection (and so
+  the two points above can fold the offset arithmetic together).
+
+- **Annotate the decoded type per column.** Inlining erases the type structure,
+  so put a short comment above each column's decode block naming the ClickHouse
+  type it reads.
+
+- **Pre-allocate small result arrays.** RowBinary gives every array/map its
+  element count up front (the LEB128 prefix), so DEFAULT is to `new Array(n)`.
+  NOTE: for **large** arrays the application will iterate or compute over repeatedly,
+  prefer `[]` + `push` (faster to traverse in V8) — or a typed array (`Float64Array`…)
+  for numeric elements.
+
+- **TypeScript by default.** Generate TypeScript parsers and helpers unless the
+  user explicitly asks for plain JavaScript.
+
+## Type family references
+
+The readers live as real code under `src/`, split by type family.
+
+| Result contains (trigger)                                                                                                                      | Open                                                                                                                                                                  |
+| ---------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| **Always** — cursor state, `advance()`, `NeedMoreData`, `Reader<T>`                                                                            | `src/core.ts`                                                                                                                                                         |
+| LEB128 length/count prefixes for `String`/`Array`/`Map` (`readUVarint`)                                                                        | `src/varint.ts`                                                                                                                                                       |
+| `Int8`–`Int256`, `UInt8`–`UInt256`                                                                                                             | `src/integers.ts`                                                                                                                                                     |
+| `Bool`                                                                                                                                         | `src/bool.ts`                                                                                                                                                         |
+| `Enum8`, `Enum16`                                                                                                                              | `src/enums.ts`                                                                                                                                                        |
+| `Float32`, `Float64`, `BFloat16`                                                                                                               | `src/floats.ts`                                                                                                                                                       |
+| `Decimal32/64/128/256`, `Decimal(P, S)`                                                                                                        | `src/decimals.ts`                                                                                                                                                     |
+| `String`, `FixedString(N)`                                                                                                                     | `src/strings.ts`                                                                                                                                                      |
+| `UUID`                                                                                                                                         | `src/uuid.ts`                                                                                                                                                         |
+| `IPv4`, `IPv6`                                                                                                                                 | `src/ip.ts`                                                                                                                                                           |
+| `Date`, `Date32`, `DateTime`, `DateTime(tz)`, `DateTime64(P[, tz])`                                                                            | `src/datetime.ts`                                                                                                                                                     |
+| `Time`, `Time64(P)`                                                                                                                            | `src/time.ts`                                                                                                                                                         |
+| `IntervalNanosecond` … `IntervalYear`                                                                                                          | `src/interval.ts`                                                                                                                                                     |
+| `Array(T)`, `Map(K, V)`, `Tuple(...)`, `Nullable(T)`, `Variant(...)`, `QBit(...)`                                                              | `src/composite.ts`                                                                                                                                                    |
+| `Point`, `Ring`, `LineString`, `MultiLineString`, `Polygon`, `MultiPolygon`, `Geometry`                                                        | `src/geo.ts`                                                                                                                                                          |
+| `Dynamic` (and `Variant`/`Interval`/`Nested`/`Dynamic` nested inside it)                                                                       | `src/dynamic.ts`                                                                                                                                                      |
+| `JSON`                                                                                                                                         | `src/json.ts`                                                                                                                                                         |
+| The whole result — loop rows to EOF (`readRows`)                                                                                               | `src/rows.ts`                                                                                                                                                         |
+| A chunked HTTP response — `streamRowBatches`, `coalesceChunks`                                                                                 | `src/stream.ts`                                                                                                                                                       |
+| The `RowBinaryWithNamesAndTypes` header — column names + type strings (`readHeader`)                                                           | `src/header.ts`                                                                                                                                                       |
+| Fold one parsed type AST into a `Reader` (`astToReader`) — AST in, reader out                                                                  | `src/compile.ts`                                                                                                                                                      |
+| **Types known only at runtime** — compile a whole header into a row reader (`compileRowBinaryWithNamesAndTypes`, `typeStringToReader`)         | `src/rowBinaryWithNamesAndTypes.ts`                                                                                                                                   |
+| **Numeric/fixed-width result read column-wise** (aggregate/scan/plot, hand to a Worker/WASM) → decode into typed arrays, not row objects (~4x) | `src/columnar.ts` (`streamSensorColumns` — streaming, yields transferable typed-array batches); `decodeIotColumnar` in `src/examples/iot.ts` is the whole-buffer form |
+| `LowCardinality(T)` — transparent, decode as `T`                                                                                               | `src/lowCardinality.ts`                                                                                                                                               |
+| `SimpleAggregateFunction(f, T)` — transparent, decode as `T`                                                                                   | `src/simpleAggregateFunction.ts`                                                                                                                                      |
+| `Nested(...)` — no wire of its own; `Array(Tuple(...))`                                                                                        | `src/nested.ts`                                                                                                                                                       |
+| `Nothing` — zero-width, never decoded (only wrapped)                                                                                           | `src/nothing.ts`                                                                                                                                                      |
+| `AggregateFunction(...)` — opaque state; finalize server-side                                                                                  | `src/aggregateFunction.ts`                                                                                                                                            |
+
+## Worked examples
+
+Six end-to-end examples with real speedup are catalogued in [EXAMPLES.md](EXAMPLES.md).
+
+## Out of scope
+
+- **JSON / CSV / TSV / Parquet parsing** → use `clickhouse-js-node-coding`.
+- **Connection errors, hangs, type mismatches** → use
+  `clickhouse-js-node-troubleshooting`.
+- **Browser / Web Worker / Edge** → `@clickhouse/client-web`.
+
+## Still Stuck?
+
+- [ClickHouse RowBinary format](https://clickhouse.com/docs/interfaces/formats#rowbinary)
+- [ClickHouse data types](https://clickhouse.com/docs/sql-reference/data-types)
+- [ClickHouse JS client docs](https://clickhouse.com/docs/integrations/javascript)

package/skills/clickhouse-js-node-rowbinary-parser/case-studies/iot-rowbinary-vs-json.md

@@ -0,0 +1,83 @@
+# Case study: RowBinary vs JSON on a table of IoT readings
+
+**TL;DR** — On a dense fixed-width numeric row, the skill's optimized RowBinary
+reader decodes **3.5x faster than the best JSON format** (`JSONCompactEachRow`)
+and **5.4x faster than `JSONEachRow`**, over a wire that is **1.6–3.3x smaller**.
+This is the workload shape the [SKILL's format-choice
+guidance](../SKILL.md#first-is-rowbinary-even-the-right-format) points at
+RowBinary for — and the numbers below are _measured_, not assumed.
+
+Reproduce: `npx vitest bench --run tests/iot.bench.ts` (against a live
+ClickHouse server). Source: [`tests/iot.bench.ts`](../tests/iot.bench.ts),
+reader: [`src/examples/iot.ts`](../src/examples/iot.ts).
+
+## The data
+
+A table of IoT sensor readings — every column fixed-width, not a string in the
+row, so the whole record is a flat 41-byte run:
+
+```sql
+sensor_id   UInt32         -- 4 bytes
+ts          DateTime64(3)  -- 8 bytes
+temperature Float64        -- 8 bytes
+humidity    Float64        -- 8 bytes
+pressure    Float64        -- 8 bytes
+battery     Float32        -- 4 bytes
+status      UInt8          -- 1 byte
+```
+
+50,000 rows, fetched from a live server in three formats and decoded into
+equivalent JS objects. A cross-format check asserts the RowBinary (binary
+float) and JSON (decimal-text → float) decodes agree on every numeric column
+before any timing is taken — so this measures the same work three ways, not
+three different results.
+
+## What was compared
+
+- **RowBinary — optimized.** The skill's monomorphized reader: the seven column
+  bounds checks coalesce into one `advance(s, 41)`, every field read at a
+  constant offset off that base.
+- **RowBinary — API combinators.** The same logic written with the plain
+  per-type readers (`readUInt32`, `readFloat64`, …) — the clear default.
+- **JSONCompactEachRow — `JSON.parse`.** Newline-delimited _arrays_ (no repeated
+  keys). The strongest JSON contender a knowledgeable user would pick.
+- **JSONEachRow — `JSON.parse`.** Newline-delimited _objects_ (keys repeated
+  every row) — the naive idiomatic choice.
+
+Both JSON paths use the fastest idiomatic decode: splice the rows into one
+`[...]` document and hand it to V8's native `JSON.parse` in a single call.
+
+## Wire size (HTTP response bytes)
+
+| Format             | Size    | B/row | vs RowBinary |
+| ------------------ | ------- | ----- | ------------ |
+| RowBinary          | 2.05 MB | 41.0  | 1.0x         |
+| JSONCompactEachRow | 3.38 MB | 67.6  | 1.6x         |
+| JSONEachRow        | 6.68 MB | 133.6 | 3.3x         |
+
+## Decode throughput (full 50k-row decode; higher = faster)
+
+| Decoder                           | ops/s | ms/decode | ≈ rows/s | speedup  |
+| --------------------------------- | ----- | --------- | -------- | -------- |
+| **RowBinary — optimized**         | 399   | 2.50      | ~20.0 M  | **1.0x** |
+| RowBinary — API combinators       | 159   | 6.31      | ~7.9 M   | 0.40x    |
+| JSONCompactEachRow — `JSON.parse` | 114   | 8.76      | ~5.7 M   | 0.29x    |
+| JSONEachRow — `JSON.parse`        | 74    | 13.47     | ~3.7 M   | 0.19x    |
+
+_Node 24 / V8. Your numbers will vary; run `npm run bench` on your own hardware._
+
+## Takeaways
+
+- **This is the textbook RowBinary win.** High-volume fixed-width numerics where
+  each field is one `DataView` read and there is no text to tokenize or numbers
+  to parse from decimal strings. The monomorphization win (2.5x over the
+  combinator API) is unusually large here because the whole row coalesces into a
+  _single_ bounds check with constant-offset reads.
+- **Format choice matters more than the optimization.** Even the plain
+  combinator-API RowBinary reader (~7.9 M rows/s) beats the best JSON option —
+  before any monomorphization.
+- **The flip side still holds.** Had this been a string-heavy result (logs, JSON
+  blobs, text consumed wholesale), `JSON.parse`'s optimized C++ would likely
+  _win_, and the skill would steer you to `JSONEachRow` + compression instead.
+  For IoT telemetry, RowBinary is clearly right — match the format to the shape
+  of the data.

package/skills/clickhouse-js-node-rowbinary-parser/case-studies/ledger-rowbinary-vs-json.md

@@ -0,0 +1,103 @@
+# Case study: RowBinary vs JSON on a financial ledger (wide ints & decimals)
+
+**TL;DR** — When every column is wider than a JS `number` can hold (`UInt128`,
+`Int64`, `Decimal128(18)`, `UInt256`), RowBinary wins _twice over_. Stock
+`JSON.parse` is not merely slow here — it is **silently wrong**, rounding every
+value to a float64. The only correct JSON path quotes the values server-side and
+re-parses each string into a `bigint`/decimal pair by hand, which is **~5x
+slower** than the optimized RowBinary reader over a **2.1–2.6x larger** wire.
+RowBinary reads each value exactly, straight off the wire.
+
+This is the workload the [SKILL's format-choice
+guidance](../SKILL.md#first-is-rowbinary-even-the-right-format) calls out
+explicitly: "RowBinary clearly wins when the result is dominated by **wide
+numerics** — `Int128`/`Int256`/`UInt128`/`UInt256`, `Decimal128`/`Decimal256`."
+
+Reproduce: `npx vitest bench --run tests/ledger.bench.ts` (against a live
+ClickHouse server). Source: [`tests/ledger.bench.ts`](../tests/ledger.bench.ts),
+reader: [`src/examples/ledger.ts`](../src/examples/ledger.ts).
+
+## The data
+
+A financial ledger — every column exceeds IEEE-754 double's 53-bit exact range:
+
+```sql
+txn_id   UInt128         -- 16 bytes
+account  Int64           --  8 bytes  (values past 2^53)
+amount   Decimal128(18)  -- 16 bytes  (~32 significant digits)
+balance  Decimal128(18)  -- 16 bytes
+fee      Decimal64(4)    --  8 bytes
+volume   UInt256         -- 32 bytes
+```
+
+50,000 rows, fixed-width (96 bytes/row), fetched from a live server.
+
+## The correctness trap
+
+ClickHouse emits these types as **bare, unquoted JSON numbers**. So stock
+`JSON.parse` parses them as float64 and silently corrupts every one — measured
+on row 0 of the live result:
+
+| Column    | Exact value (RowBinary)                   | `JSON.parse` of bare JSON                 |                  |
+| --------- | ----------------------------------------- | ----------------------------------------- | ---------------- |
+| `txn_id`  | `340282366920938463463374607431768200000` | `340282366920938463463374607431768211456` | ✗ off by 11 456  |
+| `account` | `9007199254740993`                        | `9007199254740992`                        | ✗ off by 1       |
+| `amount`  | `98765432109876.123456789012345678`       | `98765432109876.12`                       | ✗ lost 16 digits |
+
+No exception, no warning — just wrong numbers. For money and IDs, that is a
+correctness bug, not a performance footnote.
+
+### Making JSON correct costs extra work
+
+The only way to get exact values through JSON is to **quote them server-side** so
+they arrive as strings, then re-parse each one:
+
+```sql
+... SETTINGS output_format_json_quote_64bit_integers = 1,
+             output_format_json_quote_decimals = 1
+```
+
+```ts
+txn_id:  BigInt(r.txn_id),                 // string -> bigint
+amount:  parseDecimal(r.amount, 18),       // string -> [unscaled, scale]
+```
+
+That per-field `BigInt(...)` / decimal parse is work RowBinary doesn't do — it
+reads the exact `bigint` directly with two `DataView` reads — and it lands on
+top of a larger wire (strings are longer than the binary words).
+
+## Wire size (correct paths quote wide values as strings)
+
+| Format                      | Size     | vs RowBinary |
+| --------------------------- | -------- | ------------ |
+| RowBinary                   | 4.80 MB  | 1.0x         |
+| JSONCompactEachRow (quoted) | 9.88 MB  | 2.1x         |
+| JSONEachRow (quoted)        | 12.28 MB | 2.6x         |
+
+## Decode throughput (full 50k-row decode; higher = faster)
+
+| Decoder                                            | ops/s | ms/decode | ≈ rows/s | speedup  | correct?       |
+| -------------------------------------------------- | ----- | --------- | -------- | -------- | -------------- |
+| **RowBinary — optimized**                          | 130   | 7.71      | ~6.5 M   | **1.0x** | ✅             |
+| RowBinary — API combinators                        | 80    | 12.50     | ~4.0 M   | 0.62x    | ✅             |
+| JSONEachRow bare — `JSON.parse` only               | 44    | 22.74     | ~2.2 M   | 0.34x    | ❌ **corrupt** |
+| JSONCompactEachRow quoted — parse + BigInt/decimal | 26    | 37.78     | ~1.3 M   | 0.20x    | ✅             |
+| JSONEachRow quoted — parse + BigInt/decimal        | 25    | 40.70     | ~1.2 M   | 0.19x    | ✅             |
+
+_Node 24 / V8. Your numbers will vary; run `npm run bench` on your own hardware._
+
+## Takeaways
+
+- **The fast JSON path is the wrong one.** Bare `JSON.parse` is JSON's quickest
+  option and it is still 2.95x slower than RowBinary — _and_ it silently
+  corrupts every wide value. There is no "fast and correct" JSON here.
+- **The correct JSON path is ~5x slower.** Quote + per-field `BigInt`/decimal
+  parsing is the price of correctness, on top of a 2.1–2.6x larger wire.
+- **RowBinary is correct by construction.** Each value is composed from 64-bit
+  words read at constant offsets (high word signed for the signed types),
+  yielding an exact `bigint` or `[unscaled, scale]` pair — no rounding, no
+  string re-parsing.
+- **Contrast with the [IoT case study](iot-rowbinary-vs-json.md):** there the
+  numbers fit a float64 and the win was purely throughput (3.5x). Here the values
+  don't fit, so the win is _correctness first_, throughput second. Match the
+  format to the shape of the data.

package/skills/clickhouse-js-node-rowbinary-parser/case-studies/logs-json-wins.md

@@ -0,0 +1,86 @@
+# Case study: JSON beats RowBinary on a string-heavy log table
+
+**TL;DR** — This is the honest counter-case. When the result is mostly **text
+consumed wholesale** (an application log table), `JSONCompactEachRow` +
+`JSON.parse` decodes **1.4x faster** than the optimized RowBinary reader — and
+once you turn on HTTP compression, RowBinary's raw-wire size advantage
+**disappears**: gzip ties the two, and with **zstd the JSON response is actually
+slightly smaller**. For this shape the skill steers you _away_ from RowBinary —
+and proving that is what makes its "use RowBinary here" advice (see the
+[IoT](iot-rowbinary-vs-json.md) and [ledger](ledger-rowbinary-vs-json.md)
+studies) trustworthy.
+
+This is exactly what the [SKILL's format-choice
+guidance](../SKILL.md#first-is-rowbinary-even-the-right-format) says: prefer a
+`JSON*` format when the result is "mostly strings / JSON-like values that you
+consume wholesale," because V8's native `JSON.parse` is heavily optimized C++
+and "pair it with HTTP response compression (`gzip` / `zstd`, which crushes
+JSON's repetitive keys)."
+
+Reproduce: `npx vitest bench --run tests/logs.bench.ts` (against a live
+ClickHouse server). Source: [`tests/logs.bench.ts`](../tests/logs.bench.ts),
+reader: [`src/examples/logs.ts`](../src/examples/logs.ts).
+
+## The data
+
+An application log table — four of five columns are text consumed as text:
+
+```sql
+ts        DateTime
+level     LowCardinality(String)   -- transparent in RowBinary -> plain String
+service   LowCardinality(String)
+message   String                   -- templated log line, varying values
+trace_id  String                   -- high-cardinality 32-char hex
+```
+
+50,000 rows, fetched from a live server. The two `LowCardinality` columns carry
+no dictionary on the RowBinary wire — they decode as plain `String`.
+
+## Decode throughput (full 50k-row decode; higher = faster)
+
+| Decoder                               | ops/s | ms/decode | ≈ rows/s | speedup  |
+| ------------------------------------- | ----- | --------- | -------- | -------- |
+| **JSONCompactEachRow — `JSON.parse`** | 93    | 10.73     | ~4.7 M   | **1.0x** |
+| JSONEachRow — `JSON.parse`            | 72    | 13.89     | ~3.6 M   | 0.77x    |
+| RowBinary — optimized (monomorphized) | 66    | 15.07     | ~3.3 M   | 0.71x    |
+| RowBinary — API combinators           | 54    | 18.68     | ~2.7 M   | 0.57x    |
+
+`JSONCompactEachRow` (arrays, no repeated keys) is the fastest JSON option and
+beats even the optimized RowBinary reader by ~1.4x. A RowBinary string is a
+varint length + `buf.toString("utf8", …)` decoded one field at a time in JS;
+`JSON.parse` builds the same JS strings in one optimized C++ pass.
+
+## Wire size — raw, and compressed (gzip / zstd)
+
+| Format             | raw     | gzip    | zstd    |
+| ------------------ | ------- | ------- | ------- |
+| RowBinary          | 5.04 MB | 1.46 MB | 1.35 MB |
+| JSONCompactEachRow | 6.84 MB | 1.51 MB | 1.32 MB |
+| JSONEachRow        | 8.84 MB | 1.52 MB | 1.33 MB |
+
+RowBinary is 1.4–1.8x smaller **raw**, which is the usual argument for it. But
+that edge is mostly JSON's repeated structure (keys, punctuation) — exactly what
+a compressor removes. With `gzip` the three are within ~4% of each other, and
+with `zstd` the JSON responses are _slightly smaller_ than RowBinary. Any
+production HTTP path should have compression on, so the wire-size case for
+RowBinary on this data effectively vanishes.
+
+_Node 24 / V8. Your numbers will vary; run `npm run bench` on your own hardware._
+
+## Takeaways
+
+- **JSON wins both axes here.** Faster to decode (~1.4x) _and_, once compressed,
+  no larger on the wire. There is no reason to hand-write a RowBinary parser for
+  this shape.
+- **`JSONCompactEachRow` is the one to reach for** — it drops the per-row
+  repeated keys, so it parses faster than `JSONEachRow` and compresses about the
+  same.
+- **Compression erases RowBinary's raw-size advantage on text.** RowBinary's
+  smaller raw wire comes largely from not repeating keys; a compressor already
+  does that for JSON. Always compare _compressed_ sizes when the data is
+  string-heavy.
+- **This is the boundary of the skill.** RowBinary earns its keep on
+  numeric/wide/binary data ([IoT](iot-rowbinary-vs-json.md),
+  [ledger](ledger-rowbinary-vs-json.md)); on string-heavy results read as text,
+  the right answer is `JSONCompactEachRow` + compression. Match the format to the
+  shape of the data — and measure.

package/skills/clickhouse-js-node-rowbinary-parser/case-studies/wasm-vs-js.md

@@ -0,0 +1,172 @@
+# Case study: why JS, not WASM, for RowBinary parsing (and the one place WASM wins)
+
+**TL;DR** — A JIT-compiled JS RowBinary reader already streams bytes at **memory
+bandwidth** (~16 GB/s), and the dominant cost of decoding is allocating the JS
+values themselves (objects, `Date`, strings, `BigInt`) — which **WASM cannot do
+and therefore cannot remove**. So for the skill's actual job, turning a RowBinary
+response into usable JS data, WASM buys ~nothing (a wash, or a loss after the
+copy-in tax). WASM wins decisively in exactly **one** different problem:
+**in-place aggregation of wide integers / decimals** (and hash group-by), where
+JS is forced onto heap `BigInt`/`Map`. There we measured a hand-written WASM
+kernel at **27–38x** over JS. But that is _compute_, not parsing — and it is
+usually pushable to ClickHouse anyway. And if you genuinely need heavier
+**client-side analytics**, the lever isn't WASM-over-RowBinary at all — it's a
+**columnar wire format (`Native`)**, coming soon to the JS client out of the
+Python-client collaboration; RowBinary is row-major and fights every analytical
+pass.
+
+Reproduce:
+
+- `npx vitest bench --run tests/iot.wasm-headroom.bench.ts` (the parsing headroom)
+- `node tests/wasm-int128.experiment.mjs` (the hand-emitted WASM kernel)
+
+All numbers Node 24 / V8; yours will vary.
+
+## The idea under test
+
+A tempting architecture: a _dynamic WASM JIT inside the JS runtime_. A type
+builder (`t.Int32()`, `t.Map(t.FixedString, t.Int32())`) plus a query DSL
+(`q.sum(q.column(1))`) compile **on the fly** to a WASM module that parses the
+raw network chunk sitting at address 0 in linear memory, computes the answer,
+writes it to a result region, and returns the offset where the incomplete
+trailing row begins (streaming resume). Elegant. The question is _what it would
+win_ — and the honest answer needs three measurements.
+
+## Proof 1 — JIT-compiled JS reads at memory speed
+
+V8 compiles `DataView` accessors to native loads. Folding a 32 MB column of
+native-width values (`Float64`) in a plain JS loop:
+
+| Read                   | ms / 32 MB | throughput    |
+| ---------------------- | ---------- | ------------- |
+| JS `DataView` f64 fold | 1.94 ms    | **16.5 GB/s** |
+
+That is essentially RAM bandwidth. **There is no headroom for a "faster
+language" to read these bytes** — JS is already at the metal. A WASM parser
+reading the same bytes lands in the same place (see Proof 3, where the WASM
+kernel reads at 28 GB/s doing _integer_ loads — same order, also bandwidth-bound,
+not 10x).
+
+## Proof 2 — the parsing bottleneck is allocation, which WASM can't touch
+
+On the best case for RowBinary (IoT, every column fixed-width numeric), three
+decoders over the same buffer (`tests/iot.wasm-headroom.bench.ts`):
+
+| Decode                                               | ms   | vs current | what it isolates     |
+| ---------------------------------------------------- | ---- | ---------- | -------------------- |
+| **rows** — current fast reader (objects + `Date`)    | 3.48 | 1.0x       | full materialization |
+| **columnar** — into typed arrays, no per-row objects | 0.86 | 4.0x       | drop the objects     |
+| **parseOnly** — reads only, zero allocation          | 0.61 | 5.8x       | the pure-read floor  |
+
+**~83% of decode time is JS-side object/`Date` allocation**, not byte reading.
+A WASM parser still has to produce those JS values across the boundary, so it
+_cannot_ remove that 83%. Even if WASM made the parse slice instantaneous and the
+copy-in free, the row-object decode would drop only `3.48 → 2.88 ms` — a **max
+~1.2x**, and realistically a wash once you add the copy into linear memory.
+
+The 4.0x that _is_ on the table comes from the **output contract** (columnar
+typed arrays), and it's available in **plain JS** — no WASM. (That columnar path
+is worth shipping; it's the real win this whole investigation surfaced.)
+
+## Proof 3 — the one place WASM wins: wide-int / decimal aggregation
+
+Summing an `Int128` column forces JS onto heap `BigInt` (one allocation per
+row). A hand-emitted WASM kernel (94 bytes; native `i64` add-with-carry) does it
+in registers. Same 32 MB buffer, result verified equal to the BigInt sum
+(`tests/wasm-int128.experiment.mjs`):
+
+| Sum of an `Int128` column               | ms / 32 MB | throughput |                |
+| --------------------------------------- | ---------- | ---------- | -------------- |
+| **JS BigInt-128 sum** (what JS must do) | 42.93 ms   | 0.7 GB/s   | correct        |
+| WASM `i64` add-carry — kernel only      | 1.14 ms    | 28.2 GB/s  | correct        |
+| WASM + copy-in boundary tax             | 1.62 ms    | 19.7 GB/s  | (copy 0.49 ms) |
+
+**WASM is 37.8x faster than JS (26.5x including the copy into linear memory).**
+Note _why_: the win is escaping `BigInt`, not reading bytes faster — the WASM
+kernel (28 GB/s) is the same order as the JS f64 floor (16.5 GB/s). JS pays a
+**22x `BigInt` tax** purely to add 128-bit integers; WASM's native `i64` reclaims
+it. The same logic applies to `Decimal128/256` accumulation and to hash group-by
+(WASM open-addressing table in linear memory vs JS `Map` + GC).
+
+## Verdict on the dynamic-WASM-JIT
+
+The architecture is **sound for the aggregation regime and only that regime**.
+It targets the one quadrant where WASM beats well-written JS: _parse and compute
+in place, return a small result, never cross the boundary per value._ The design
+answers its own open questions well:
+
+- **Where does the answer go?** Scalars return directly (`i128` via multi-value
+  or two `i64`s); group-by results go to a reserved linear-memory region that JS
+  reads as a typed-array view — only the small final result crosses.
+- **Streaming.** Returning the resume offset (vs throwing across the FFI) is
+  clean, and accumulator state lives in linear memory across chunks — the module
+  _is_ the streaming aggregation state.
+
+But three caveats bound where it's worth building:
+
+1. **For parsing → JS values, use generated JS, not WASM.** Proofs 1–2: JS is
+   already at memory speed and the cost is materialization WASM can't remove. A
+   `DSL → new Function(generatedJS)` backend captures the parse + native-numeric
+   aggregation case with **zero toolchain**, debuggable. This is the skill's
+   existing monomorphization thesis.
+2. **Reserve a WASM backend for the wide-int/decimal + group-by kernels only** —
+   gate it on the presence of `Int128/256`, `Decimal128/256`, or a `GROUP BY`,
+   where Proof 3's 27–38x is real. For `Float64` sums it would tie JS.
+3. **SIMD won't help much** — RowBinary is row-major (AoS); strided columns
+   defeat Wasm SIMD (no gather) without a transpose pass. The WASM win here is
+   native `i64` + no GC, not vectorization.
+4. **The elephant: push it down.** `q.sum(col)` is `SELECT sum(col)` — ClickHouse
+   will beat any client. Client-side aggregation only justifies itself when you
+   _can't_ push down: folding a stream you already receive for another reason,
+   combining across queries/sources, or compute SQL can't express.
+
+## If you need more client-side analytical strength: reach for Native columnar
+
+Step back from WASM and look at _why_ the wins above are so narrow. RowBinary is
+**row-major (AoS)**: every row interleaves all columns, so any analytical pass —
+fold a column, vectorize, build a column-at-a-time accumulator — has to stride
+over the bytes it doesn't want and re-materialize a value at a time. That is the
+same row-major tax that defeats SIMD (caveat 3) and that makes the free **4x in
+Proof 2 cost a transpose** today (you decode rows, _then_ pack into typed
+arrays).
+
+So the honest answer to _"I need real client-side analytical strength"_ is **not
+a smarter parser over RowBinary, and not WASM** — it is a **columnar wire
+format**. ClickHouse's **`Native`** format is **column-major (SoA)**: each block
+arrives as contiguous per-column runs. That flips every constraint in this study:
+
+- The Proof-2 columnar typed-array path stops needing a transpose — the wire
+  _is_ already `Float64Array`-shaped, so you `subarray`/`set` a column in one
+  move instead of decoding rows first.
+- Vectorization becomes real: a contiguous column is exactly what `v128.load` /
+  SIMD (and even auto-vectorized JS) want — the gather problem disappears.
+- The wide-int/decimal aggregation win (Proof 3) keeps applying, now over
+  contiguous input, which is the friendliest possible layout for it.
+
+A columnar reader is **coming to the JS client soon**, out of the **collaboration
+with the Python client** (which already ships a mature `Native`/columnar path —
+the format and lessons port directly). When it lands, the order of preference for
+client-side analytics becomes: **push down to ClickHouse → if you can't, decode
+`Native` columnar → reserve WASM for the wide-int/decimal/group-by kernel on top
+of those columns.** RowBinary stays the right tool for what this skill targets —
+turning a result into JS _rows/values_ — not for analytics over them.
+
+## Takeaways
+
+- **Generated JS is the right engine for the parser.** It reads at memory
+  bandwidth; the remaining cost is JS-value materialization that no language
+  swap removes. WASM for parsing is a wash-to-loss.
+- **The free 4x is a columnar (typed-array) output contract — in pure JS.** Worth
+  capturing as a first-class option for numeric results.
+- **WASM earns its complexity in one place: in-place wide-int/decimal/group-by
+  aggregation** (27–38x measured), where JS is trapped in `BigInt`/`Map`. And
+  even then, prefer pushing the aggregation to ClickHouse unless you genuinely
+  can't.
+- **For real client-side analytical strength, the answer is columnar, not WASM.**
+  RowBinary is row-major and taxes every analytical pass; a `Native` (SoA)
+  columnar reader — coming to the JS client soon via the Python-client
+  collaboration — removes the transpose, unlocks SIMD, and is the natural
+  substrate for the aggregation kernels above.
+- Matches the rest of the studies' through-line: pick the tool for the shape of
+  the work, and **measure** — the 94-byte WASM kernel exists precisely so this
+  claim isn't hand-waved.