questdb 0.1.0-arm64-darwin

data/docs/arrow.md

@@ -0,0 +1,89 @@
+# Optional Apache Arrow integration
+
+The core client has no Ruby dataframe dependency. Load the optional adapter explicitly:
+
+```ruby
+require "questdb/arrow"
+```
+
+This adds Arrow C Stream ingestion and query export while loading none of `nanoarrow`, `polars-df`, or `red-arrow`. Install only the consumer you use.
+
+## Ingestion
+
+`Client#ingest_arrow` accepts:
+
+- any object responding to `arrow_c_stream`, including `Nanoarrow::Array`, `Nanoarrow::ArrayStream`, and `Polars::DataFrame`;
+- `Arrow::Table`, `Arrow::RecordBatch`, and `Arrow::RecordBatchReader` from `red-arrow`.
+
+```ruby
+require "nanoarrow"
+require "questdb/arrow"
+
+batch = Nanoarrow::Array.new(
+  [
+    {"symbol" => "ETH-USD", "price" => 2615.54},
+    {"symbol" => "BTC-USD", "price" => 60310.0}
+  ],
+  Nanoarrow.struct({
+    "symbol" => Nanoarrow.string,
+    "price" => Nanoarrow.double
+  })
+)
+
+client.ingest_arrow(
+  "trades",
+  batch,
+  at: QuestDB::TimestampNanos.now,
+  symbols: ["symbol"]
+)
+```
+
+`at:` accepts a timestamp column name, `TimestampNanos`, `TimestampMicros`, `Time`, or `SERVER_TIMESTAMP`. Arrow dictionary-encoded UTF-8 defaults to QuestDB SYMBOL and plain UTF-8 defaults to VARCHAR. `symbols:` forces named plain UTF-8 columns to SYMBOL.
+
+Use `schema_overrides:` for QuestDB-specific logical types:
+
+```ruby
+client.ingest_arrow(
+  "events",
+  batch,
+  at: :timestamp,
+  schema_overrides: {
+    venue: :symbol,
+    address: :ipv4,
+    grade: :char,
+    location: [:geohash, 30],
+    category: :varchar # force an Arrow dictionary column to VARCHAR
+  }
+)
+```
+
+The source stream is consumed exactly once. Arrow producer callbacks and network operations run without the GVL. As required by the Arrow C Data Interface, use a trusted producer whose declared buffer sizes match its allocations; the ABI cannot let a consumer prove buffer lengths. Producer batches are pipelined through the c-client's direct sender and committed at the requested `:ok` or `:durable` boundary. To bound deferred native memory and the replay window, `checkpoint_batches:` defaults to 100; streaming producers retain their own batch sizes. `PooledSender#ingest_arrow` is a convenience forwarding to its client; like `ingest_columns`, it uses a separate direct connection and is not ordered against that sender's row buffer. Standalone WebSocket `QuestDB::Sender#ingest_arrow` opens a poolless direct connection with the same authentication/TLS configuration and likewise has no ordering relationship with its row buffer. If an error reports `in_doubt?`, do not replay the whole stream without suitable QuestDB deduplication keys.
+
+## Query results
+
+The neutral integration point is `QueryResult#arrow_c_stream`:
+
+```ruby
+result = client.query("SELECT * FROM trades")
+frame = Polars::DataFrame.new(result)
+```
+
+Convenience methods lazy-load their respective optional gems:
+
+```ruby
+nanoarrow_array = client.query(sql).to_nanoarrow
+polars_frame = client.query(sql).to_polars
+red_arrow_table = client.query(sql).to_red_arrow_table
+```
+
+The current query adapter fully drains the QWP cursor into native Arrow-owned record batches before returning the C Stream capsule. This keeps consumer callbacks nonblocking, permits safe whole-result failover replay, and avoids per-cell Ruby objects, but peak memory is proportional to the result size. Use ordinary `each_batch` or `each_values` when bounded-memory streaming is required.
+
+Calling an Arrow materializer consumes and closes the single-use `QueryResult`. SYMBOL columns are exported as compact Arrow dictionaries through the official c-client Arrow path.
+
+## Packaging
+
+Arrow support uses the stable Arrow C Data and C Stream interfaces implemented by the pinned `c-questdb-client`. It does not link Arrow C++ or GLib. Arrow Rust sources are included in the existing offline Cargo vendor tree, so the source gem remains fully offline-buildable.
+
+Official precompiled extensions include the c-client Arrow feature. This is necessary because Cargo features are selected when the native extension is built, while `require "questdb/arrow"` happens later at runtime; omitting it would make the add-on fail for users of precompiled gems. The module remains optional at the Ruby layer and adds no dataframe dependency. A separate Arrow native gem would duplicate the statically linked c-client and complicate shared ownership, so it is not used.
+
+`red-arrow` itself requires Arrow GLib system integration; `nanoarrow` has no runtime gem dependencies. Neither is required by `questdb`.

data/docs/configuration.md

@@ -0,0 +1,61 @@
+# QWP configuration
+
+`QuestDB.connect` accepts either a native `ws::` / `wss::` connection string or `host:` keywords:
+
+```ruby
+QuestDB.connect("ws::addr=localhost:9000;")
+QuestDB.connect(host: "localhost", port: 9000)
+QuestDB.connect(host: "db.example.com", port: 443, tls: true,
+                username: ENV.fetch("QDB_USER"),
+                password: ENV.fetch("QDB_PASSWORD"))
+```
+
+The forms are mutually exclusive. IPv6 literals may be bare (`::1`) or bracketed (`[::1]`) with `host:`; the builder inserts brackets. Do not include a port in `host:`. Additional keyword settings are converted to the native connect-string grammar. Semicolons are escaped as `;;`; control characters are rejected; a keyword cannot duplicate a key already present in a supplied string.
+
+`QuestDB::Client.from_env` reads `QDB_CLIENT_CONF`.
+
+`QuestDB::Sender` is the standalone QWP/WebSocket API. It accepts `ws::` or `wss::` through `.from_conf`/`.from_env`, or `protocol: :ws/:wss` with `host:` and `port:`. The default port is 9000. UDP and ILP schemes are rejected.
+
+The Ruby-only keywords `connection_listener`, `error_handler`, `connection_event_inbox_capacity`, and `error_event_inbox_capacity` are consumed by the wrapper and are never serialized into the native connect string. They apply to pooled clients and standalone WebSocket senders. Handlers must respond to `call`; capacities must be nonnegative integers. Capacity `0` selects the default 64. A callback-enabled native pool or standalone sender is opened only when the corresponding handler exists. Without `error_handler`, the extension leaves the c-client default rejection logger active.
+
+## Native settings
+
+Settings are passed directly to `c-questdb-client` 7.1.0 so new native settings do not require a Ruby release. Important groups include:
+
+- pool: `sender_pool_min`, `sender_pool_max`, `query_pool_min`, `query_pool_max`, `acquire_timeout_ms`, `idle_timeout_ms`, `pool_reap`, `lazy_connect`;
+- reconnect/failover: `addr` (comma-separated endpoints), `target`, `zone`, `reconnect_initial_backoff_millis`, `reconnect_max_backoff_millis`, `reconnect_max_duration_millis`, `initial_connect_retry`;
+- store-and-forward: `sf_dir`, `sender_id`, `sf_max_segment_bytes`, `sf_max_total_bytes`, `sf_append_deadline_millis`, `sf_durability`, `sf_sync_interval_millis`, `drain_orphans`, `max_background_drainers`;
+- delivery: `request_durable_ack`, `close_flush_timeout_millis`, `max_buf_size`, `max_name_len`, `error_inbox_capacity`;
+- authentication/TLS: `username`, `password`, `token`, `auth_timeout_ms`, `tls_verify`, `tls_ca`, `tls_roots`, `tls_roots_password`.
+
+Consult the matching upstream 7.1.0 connect-string reference for exact constraints. Unknown or invalid settings are rejected by the native parser.
+
+Direct column ingestion borrows from the native direct sender pool and uses `acquire_timeout_ms`; initial direct connection retry is bounded by `reconnect_max_duration_millis`. `max_rows_per_batch` is a method argument rather than a connection setting. Direct writes do not use `sf_dir`, do not produce FSNs, and are unaffected by row auto-publish thresholds. `ack: :durable` still requires both `request_durable_ack=on` and server support.
+
+## Auto-publish
+
+Pooled row senders default to the Python QWP policy:
+
+- 1,000 rows;
+- 100 milliseconds (checked after row appends; no Ruby timer thread);
+- 90% of the native effective frame cap, limited to 8 MiB while no server cap is known.
+
+Configure `auto_flush_rows`, `auto_flush_interval`, or `auto_flush_bytes`; use `off` to disable one threshold or `auto_flush=off` to disable all. Auto-publish never waits for an ACK. Explicit `flush(wait: true)` or `wait` provides the barrier.
+
+## Authentication and TLS
+
+QWP uses HTTP credentials during the WebSocket upgrade:
+
+```ruby
+QuestDB.connect(
+  "wss::addr=db.example.com:443;" \
+  "token=#{ENV.fetch('QDB_TOKEN')};" \
+  "tls_roots=#{ENV.fetch('QDB_TLS_ROOTS')};"
+)
+```
+
+Use either `username` plus `password` (HTTP Basic) or `token` (bearer), not both. Bearer tokens, multi-host failover, and durable ACKs require the applicable Enterprise server features, but this repository contains no Enterprise server code or credentials.
+
+Certificate and hostname verification are enabled by default. `tls_ca` selects `webpki_roots`, `os_roots`, or `webpki_and_os_roots`. For a custom PEM bundle, set `tls_roots=/path/to/ca.pem` and omit `tls_ca`; the native QWP configuration infers PEM mode. With `tls_roots_password`, WSS also accepts JKS/PKCS#12 trust stores. Never commit credentials or log full configuration strings.
+
+`tls_verify=unsafe_off` (or `tls_verify: false`) is compiled for isolated diagnostics only. It disables certificate and hostname verification and must not be used in production; install the correct CA instead.

data/docs/design.md

@@ -0,0 +1,46 @@
+# Design
+
+## Native boundary
+
+The gem links statically to official `c-questdb-client` 7.1.0. Ruby owns thin typed-data wrappers for:
+
+- one ref-counted `questdb_db` pool state;
+- short-lived borrowed `qwp_sender` leases;
+- caller-owned pool-created `line_sender_buffer` row buffers;
+- native error and structured diagnostic values copied into Ruby objects;
+- synchronous, pool-accounted `qwp_direct_sender` operations and caller-owned `qwp_chunk` values backed by native staging;
+- pooled `qwp_reader`/cursor handles and a standalone `line_sender` for QWP WebSocket.
+
+No WebSocket, QWP encoding, TLS, authentication, retry, failover, store-and-forward, or ACK logic is reimplemented in Ruby.
+
+## Ownership
+
+The Ruby client and every native sender lease hold a reference to shared pool state. Explicit `Client#close` refuses active leases. If unreachable objects are collected in any order, the pool remains alive until the final lease is returned/dropped, preventing use-after-free. Explicit close/return is idempotent at the Ruby layer and every native pointer is consumed exactly once.
+
+Errors that make delivery or connection health uncertain mark the lease for drop rather than recycling it. Client-side validation/configuration errors do not retire a healthy connection. A QWP no-progress timeout from `wait` or the wait phase of `flush(wait: true)` also leaves the lease healthy: queued frames remain owned by store-and-forward and continue delivery. Buffers own their native pointers independently and free them once.
+
+Explicit close is the primary lifecycle. As a misuse fallback, an unreachable client or sender enqueues its native handles onto a lock-free deferred cleanup queue. Cleanup records are preallocated with their wrappers, so GC does not allocate them. A postponed Ruby VM job performs sender drop and pool close without the GVL; it never recycles an abandoned sender. The v7 FFI handle wraps Rust's `OwnedSender` (`Send`), so the unreachable handle is moved with a happens-before edge and never used concurrently, despite the stricter public recommendation to return a borrow on its borrowing thread.
+
+## Threading and the GVL
+
+The pool supports concurrent borrows, protected against close races by the Ruby client's mutex. Each lease records the borrowing `Thread` and rejects use elsewhere, even without overlapping calls. Native busy flags reject reentrant/concurrent operations.
+
+Connect, borrow, return/drop, flush, wait, reap, close, callback-queue waits, and deferred GC cleanup release the GVL. Connect strings are copied before release; row values are encoded into native-owned buffers while the GVL is held. Shared pool references, active-lease counts, closure state, and the pool pointer use C11 atomics so a return/drop cannot race a concurrent close or inspection. Cleanup paths restore busy state after exceptions.
+
+Native connection/rejection callbacks never enter Ruby. For pools and standalone WebSocket senders, they deep-copy every borrowed field with plain native allocation and perform a constant-time push into portable mutex/condition-variable queues. The bounded queues use drop-oldest overflow, so slow Ruby handlers never block c-client dispatch, producer, or I/O threads. Dedicated Ruby threads drain the native queues, create immutable values, retain bounded polling copies, and isolate handler exceptions. Delivered/dropped counters add the applicable c-client inbox, native-to-Ruby bridge, and Ruby polling-copy stages; an event can therefore be counted once per stage it successfully exits.
+
+Close first atomically rejects work, closes the pool without the GVL (fencing c-client dispatchers), then closes/wakes bridge queues. Ruby dispatcher joins happen outside the client mutex and skip the calling dispatcher when a callback closes its own client. Failed connect tears down its fenced callback context. Deferred native cleanup performs the same fence without calling Ruby.
+
+## Row atomicity
+
+Every row starts with a native marker. Any Ruby exception or native validation error rewinds to it before propagating. This prevents partially encoded rows from contaminating a later publish.
+
+## Direct column ownership
+
+Ruby synchronously normalizes direct columns into private immutable packed values. The extension then deep-copies all packed data, names, offsets, symbol dictionaries, validity maps, tensor shapes, and designated timestamps into native allocations before releasing the GVL. Callers must not mutate inputs concurrently just because the method has started; mutation is safe after the call returns or after a separately constructed `ColumnVector` has copied its values. A `qwp_chunk` borrows native allocations only while the direct flush is in flight. All batches share one borrowed direct sender and are committed once; successful commit returns it, while any uncertain or partial path drops it. A temporary pool-state reference and active-lease count prevent concurrent client close and are released exactly once, including asynchronous Ruby exceptions.
+
+Direct ingestion does not enter the row sender's store-and-forward queue and cannot be ordered relative to it. Its only success boundary is the checked direct commit. After one or more flushes, errors are conservatively marked in-doubt and include a partial-completion warning.
+
+## Current boundary
+
+The client exposes pooled QWP row ingress, native-staged direct columnar ingress, QWP egress queries, pool-wide Ruby callback dispatch, and standalone QWP WebSocket ingestion. Reader batches are copied into immutable Ruby objects before advancing the native cursor, and partial-result cleanup drops rather than recycles the connection. Whole-result materialization clears a partial prefix after a failover reset; streaming raises before replay can duplicate yielded rows. ILP compatibility is not retained. The c-client Arrow feature is linked, but Ruby-facing Arrow methods are installed only by `require "questdb/arrow"`; no dataframe gem is a core dependency.

data/docs/ingestion.md

@@ -0,0 +1,96 @@
+# Pooled QWP row ingestion
+
+Borrow a sender for one logical unit of work and return it promptly:
+
+```ruby
+client.sender do |sender|
+  sender.row("telemetry",
+             symbols: { device: "sensor-7" },
+             columns: { temperature: 21.5, active: true },
+             at: QuestDB::TimestampNanos.now)
+  sender.flush(wait: true)
+end
+```
+
+`symbols` become QuestDB `SYMBOL` columns; `columns` carry scalar, timestamp, decimal, or `Float64Array` values. `nil` values are omitted. Plain Ruby arrays are rejected to avoid ambiguous type/shape inference. A row containing only omitted values is a no-op. Timestamp wrappers represent the full signed 64-bit QWP domain for query values, but row/designated timestamps still must satisfy the native QWP ingestion rules; invalid pre-epoch row timestamps are rejected atomically by the native buffer API.
+
+Every row append uses a native bookmark. If Ruby conversion or native validation fails, the entire row is rewound and earlier completed rows remain usable.
+
+## Publishing and acknowledgement
+
+- `flush` publishes and clears buffered rows locally.
+- `flush(wait: true)` additionally waits for the `OK` acknowledgement level.
+- `wait(timeout_millis: 0)` waits for all frames published through this lease; zero means no no-progress deadline.
+- `flush_and_get_fsn` returns the frame sequence number assigned to a publication.
+- `flush_and_keep_and_get_fsn` publishes without clearing the row buffer.
+- `published_fsn` and `acked_fsn` expose connection watermarks.
+- `await_acked_fsn(fsn, timeout_millis:)` returns false on no-progress timeout.
+- `ack: :durable` requires native `request_durable_ack=on` and server support.
+
+FSNs belong to the borrowed connection and are meaningful only while that lease is held. `acked_fsn` can advance for a frame dropped after a non-terminal server rejection; inspect diagnostics as well.
+
+## Diagnostics
+
+`poll_error` returns an immutable `QuestDB::SenderError` with category, applied policy, optional status, message, optional message sequence, and affected FSN range. `error_events_dropped` reports overflow of that connection's bounded diagnostic ring.
+
+Pool-wide server rejections use the native non-silent default logger. `Client#error_events_delivered` and `#error_events_dropped` expose pool counters.
+
+## Standalone QWP senders
+
+`QuestDB::Sender` owns one QWP WebSocket connection. Its row API matches pooled senders, but it is not part of a deployment pool. WebSocket senders provide `flush`, ACK waits, FSNs, diagnostics, manual `drive_once`, checked close-drain, direct columnar ingestion, and safely queued `connection_listener` / `error_handler` callbacks. Standalone handlers run on dispatcher threads and must not call the thread-affine sender; communicate back to its owner thread instead. Configure `qwp_ws_progress=manual` when the application will drive progress itself. Standalone auto-flush defaults to 600 rows and 1,000 ms, matching the Python WebSocket client. Standalone senders are thread-affine and should be explicitly closed or used with `open`.
+
+## Concurrency
+
+`QuestDB::Client` can serve concurrent Ruby threads. A `PooledSender` and its internal buffer are single-threaded and bound to the borrowing Ruby thread. Use one lease per thread:
+
+```ruby
+threads = 4.times.map do |worker|
+  Thread.new do
+    client.sender do |sender|
+      sender.row("jobs", symbols: { worker: worker.to_s },
+                 columns: { done: true }, at: QuestDB::SERVER_TIMESTAMP)
+    end
+  end
+end
+threads.each(&:join)
+```
+
+Blocking native connect, borrow, publish/wait, return/drop, reap, and close calls run without the GVL. Ruby strings and arrays are converted before any no-GVL operation.
+
+## Direct columnar ingestion
+
+`Client#ingest_columns` (alias `write_columns`) accepts equal-length column arrays and uses the pool's direct QWP sender:
+
+```ruby
+client.ingest_columns(
+  "telemetry",
+  columns: {
+    temperature: [21.5, nil, 19.75],
+    sequence: QuestDB::ColumnVector.int([1, 2, 3])
+  },
+  symbols: { device: ["a", "a", "b"] },
+  at: [
+    QuestDB::TimestampNanos.new(1_700_000_000_000_000_001),
+    QuestDB::TimestampNanos.new(1_700_000_000_000_000_002),
+    QuestDB::TimestampNanos.new(1_700_000_000_000_000_003)
+  ],
+  max_rows_per_batch: 16_384,
+  ack: :ok
+)
+```
+
+The same API is available as `PooledSender#ingest_columns` and on standalone `ws`/`wss` senders. The pooled form borrows a separate direct connection from its client; the standalone form opens a poolless direct connection from the sender's auth/TLS configuration. Neither form orders the direct commit against that sender's row buffer.
+
+Homogeneous Arrays infer conservative types (`Integer`→LONG, `Float`→DOUBLE, `String`→VARCHAR, plus the explicit QuestDB wrapper classes). All-null arrays and ambiguous or heterogeneous values require `ColumnVector`. Explicit kinds are `byte`, `short`, `int`, `long`, `float`, `double`, `boolean`, `varchar`, `binary`, `uuid`, `long256`, `ipv4`, `timestamp`, `timestamp_nanos`, `date`, `char`, `geohash`, `decimal64`, `decimal128`, `decimal256`, and `double_array`. GEOHASH requires `precision_bits:`, decimals require `scale:`, and DOUBLE_ARRAY requires a common `shape:`. SYMBOL arrays go in `symbols:`.
+
+`at:` is required and accepts a non-null Array of `TimestampNanos`, `TimestampMicros`, or `Time`, a fixed timestamp value, or `SERVER_TIMESTAMP`. A fixed value is repeatable and suitable for QuestDB deduplication. Server timestamps are assigned on arrival and therefore change on retry. Empty columns are a no-op. Names, UTF-8, lengths, ranges, shapes, timestamp nullability, ACK level, and batch size are validated before a native sender is borrowed.
+
+The implementation synchronously normalizes inputs and then copies packed values, offsets, dictionaries, validity bitmaps, names, and timestamps into native-owned staging before releasing the GVL. Do not mutate input Arrays or strings concurrently merely because the method has started; they are safe to mutate after `ingest_columns` returns (or after a separately constructed `ColumnVector` has copied them). Each chunk borrows staging only until its direct flush returns.
+
+### Direct commit semantics
+
+Direct ingestion pipelines every batch on one direct connection and performs one checked commit at `ack: :ok` or `:durable`. It is intentionally **not** store-and-forward: there are no FSNs, no background replay, and returning a direct connection is not used as an unchecked commit. On failure the connection and uncommitted tail are dropped. If any batch was flushed before an error, the raised `QuestDB::Error` says that partial completion is possible and `in_doubt?` is true; never retry the whole input blindly. Use stable designated timestamps plus QuestDB deduplication when application retries are required.
+
+Row and direct-column writes use different delivery paths. They have no ordering relationship, even when issued through the same `Client`; establish ordering with application-level acknowledgement/query barriers rather than call order.
+
+QuestDB BYTE, SHORT, BOOLEAN, and CHAR have no distinct SQL NULL representation. Their direct vectors reject `nil`; nullable kinds preserve `nil` with validity bitmaps. INT/LONG minimum sentinels and floating NaN are likewise rejected as non-null values. Native Arrow C Stream ingestion is available through the optional [`questdb/arrow`](arrow.md) adapter.

data/docs/performance.md

@@ -0,0 +1,36 @@
+# Performance
+
+The hot row path crosses the Ruby/C boundary once per row and calls the official native buffer API per field. Blocking pool/network operations release the GVL. For throughput:
+
+- keep one `QuestDB::Client` for the process/service;
+- borrow one sender per worker thread or logical batch;
+- keep auto-publish enabled unless the application owns batching;
+- avoid `flush(wait: true)` per row—publish batches, then use one ACK barrier;
+- use `sf_dir` when reconnect/replay durability matters;
+- size `sender_pool_max` to application concurrency and server connection capacity;
+- use `ingest_columns` for already column-oriented batches and tune `max_rows_per_batch` to balance staging/frame size against per-chunk overhead;
+- for allocation-critical row producers, reuse the table name and mutable `symbols`/`columns` Hashes between calls—the normal keyword API remains unchanged;
+- use `QueryResult#each_values` when positional Array access is sufficient; ordinary `each` retains ergonomic indexed `Row` objects and name lookup.
+
+`Buffer#bytesize` and `#capacity` are QWP implementation hints, not serialized wire byte counts. Benchmark labels deliberately distinguish row buffering from QWP publication and ACK latency.
+
+Run:
+
+```sh
+bundle exec ruby -Ilib benchmark/serialize_rows.rb
+bundle exec ruby -Ilib benchmark/preparation.rb
+QDB_BENCH_CONF='ws::addr=localhost:9000;auto_flush=off;' \
+  bundle exec ruby -Ilib benchmark/network.rb
+QDB_BENCH_CONF='ws::addr=localhost:9000;' \
+  bundle exec ruby -Ilib benchmark/query_rows.rb
+QDB_BENCH_CONF='ws::addr=localhost:9000;' \
+  bundle exec ruby -Ilib benchmark/query_access.rb
+QDB_BENCH_CONF='ws::addr=localhost:9000;' \
+  ruby -Ilib benchmark/arrow.rb # optional nanoarrow gem
+QDB_BENCH_CONF='ws::addr=localhost:9000;' \
+  bundle exec ruby -Ilib benchmark/columnar.rb
+QDB_BENCH_CONF='ws::addr=localhost:9000;auto_flush=off;' \
+  bundle exec ruby -Ilib benchmark/ingestion.rb
+```
+
+Reports contain medians, median absolute deviation, allocation counts, raw samples, and environment metadata. The `ingestion` benchmark times complete row build/publish/ACK and direct columnar build/commit paths, while `network` deliberately excludes row construction. Run `rake benchmark:smoke` for a broad catastrophic-regression guard. Set `BASELINE=baseline.json` when running `ingestion.rb`, or use `benchmark/check.rb`, to enforce median-throughput and allocation budgets against a like-for-like dedicated-runner baseline. Shared-host results are not defensible regression thresholds.

data/docs/python-parity.md

@@ -0,0 +1,34 @@
+# Python QWP parity
+
+This client tracks the QWP surface of the QuestDB Python 5.0 client at the `c-questdb-client` 7.1.0 ABI boundary. Parity means equivalent delivery, lifecycle, type, and error semantics expressed idiomatically in Ruby; it does not mean retaining Python's legacy ILP APIs.
+
+| Python QWP capability | Ruby API | Status |
+| --- | --- | --- |
+| Deployment connection and shared sender/reader pools | `QuestDB.connect`, `Client` | Equivalent |
+| Thread-affine pooled row sender, auto-publish, ACK barriers | `Client#sender`, `PooledSender` | Equivalent |
+| FSNs, published/acked watermarks, durable ACK | `flush_and_get_fsn`, `await_acked_fsn`, `ack: :durable` | Equivalent |
+| Store-and-forward, failover, TLS and authentication | Official connect-string settings | Native pass-through |
+| Direct columnar/dataframe load | `Client#ingest_columns`, `ColumnVector`; optional `Client#ingest_arrow` | Ruby-native arrays plus Arrow C Stream integration |
+| SQL query/execute and reusable reader lease | `Client#query`, `#execute`, `#reader` | Equivalent |
+| Typed positional binds and scalar/`DOUBLE[]` results | Ruby value wrappers, `Row`, `ResultBatch` | Equivalent within the v7 typed reader ABI |
+| Arrow query export | Optional `QueryResult#arrow_c_stream` and dataframe conveniences | Equivalent native C Stream boundary; currently whole-result materialized |
+| Query cancellation, single-use results, failover reset | `QueryResult` | Equivalent; streaming refuses duplicate replay |
+| Pool connection/rejection callbacks | `connection_listener`, `error_handler` | Equivalent with safer queued Ruby dispatch |
+| Standalone WebSocket sender and manual progress | `QuestDB::Sender` with `ws::`/`wss::` | Equivalent |
+| Standalone callback diagnostics | `Sender` callback keywords and polling copies | Equivalent with queued Ruby dispatch |
+| QWP/UDP datagrams | Not exposed | Intentional scope reduction; use WebSocket |
+| Structured native errors and delivery ambiguity | `Error#code`, `#sender_error`, `#in_doubt?` | Equivalent |
+| Standalone WebSocket auto-flush defaults | 600 rows, 1,000 ms | Matched |
+
+## Intentional Ruby-native differences
+
+- Homogeneous Ruby arrays and immutable `ColumnVector` descriptors are the dependency-free columnar API. `require "questdb/arrow"` adds native Arrow C Stream support for Nanoarrow, Polars, and Red Arrow without requiring any of them.
+- Arrow query export currently drains into native-owned Arrow batches before returning the stream capsule; use Ruby `each_batch`/`each_values` for bounded-memory streaming. A call targets one table; split multi-table input explicitly.
+- Python's ILP TCP/HTTP compatibility, protocol v1, HTTP transactions, and best-effort QWP/UDP transport are outside this WebSocket-only client's scope.
+- Ruby callback handlers run on dedicated Ruby threads. Native callback threads only deep-copy into bounded queues and never invoke Ruby.
+- Standalone `length` is the number of buffered QWP rows, matching pooled senders, rather than the legacy ILP byte-count convention. `Buffer#bytesize` remains a native memory hint.
+- `LONG_ARRAY` query projection is rejected explicitly because c-client 7.0 does not expose a typed LONG-array projection. `DOUBLE_ARRAY` is fully supported.
+
+## Parity guardrails
+
+Unit and live tests cover row and direct ingestion, standalone WebSocket, callbacks, typed binds/results, failover replay policy, native lifecycle/GC, ACK barriers, error diagnostics, and thread affinity. The benchmark suite covers local row buffering, complete row ingestion, direct columnar commits, query decoding, allocations, buffer growth, and shared-pool concurrency. See [performance](performance.md) for regression-baseline usage.

data/docs/querying.md

@@ -0,0 +1,54 @@
+# QWP queries
+
+`QuestDB::Client#query` borrows a QWP reader from the shared native pool and returns a single-use `QuestDB::QueryResult`. Use the block form or close the result explicitly.
+
+```ruby
+client.query("SELECT symbol, price FROM trades WHERE price > $1", binds: [100.0]) do |result|
+  result.each do |row|
+    puts [row["symbol"], row["price"]].inspect
+  end
+end
+```
+
+`QueryResult#each_batch` yields immutable `ResultBatch` objects. Every batch, column name, and value has been copied into Ruby-owned memory before the native cursor can advance. `each` yields immutable `Row` objects. Rows retain the complete ordered name list, including duplicates; name lookup returns the first match and `to_h` necessarily keeps only the final value for duplicate names. Name lookup uses a batch-shared index, so `row[:symbol]` remains ergonomic without a per-access String allocation.
+
+For allocation-sensitive scans that only need positional values, `each_values` yields frozen, Ruby-owned Arrays directly and avoids allocating a `Row` wrapper:
+
+```ruby
+result.each_values { |values| consume(values[0], values[1]) }
+```
+
+Whole-result `to_a` safely handles a mid-query failover by discarding its partial prefix before accepting replay. Streaming `each`, `each_values`, and `each_batch` cannot retract rows already yielded, so they raise `:failover_would_duplicate` before yielding replayed rows.
+
+For native interchange with Nanoarrow, Polars, or Red Arrow, load `questdb/arrow`. Its C Stream query path avoids per-cell Ruby objects; see [optional Arrow integration](arrow.md).
+
+For several sequential queries on one connection, borrow a reader:
+
+```ruby
+client.reader do |reader|
+  reader.execute("CREATE TABLE IF NOT EXISTS audit (ts TIMESTAMP, n LONG) TIMESTAMP(ts) WAL")
+  rows = reader.query("SELECT n FROM audit ORDER BY ts").to_a
+end
+```
+
+Only one result may be active on a reader. Fully draining a result returns a healthy connection to the pool. `cancel` synchronously requests cancellation and drains to a terminal boundary; call `close` afterward. Closing an undrained result marks the connection unsafe to recycle and terminates that reader lease. Results and readers are thread-affine.
+
+## Binds
+
+Binds are positional and correspond to `$1`, `$2`, and so on. The container must be an `Array`. Supported values:
+
+- `nil` (NULL VARCHAR), `true`/`false`, signed 64-bit `Integer`, `Float`, and UTF-8 `String`
+- `Time`, `TimestampMicros`, `TimestampNanos`, and `DateMillis`; timestamp wrappers use the full signed 64-bit range, so pre-epoch query values are preserved exactly
+- `BigDecimal` or `DecimalValue`
+- `UUID`, `Binary`, `IPv4`/`IPAddr`, `Geohash`, `Long256`, and `Char`
+- `TypedNull` when SQL requires a non-VARCHAR NULL type
+
+Values are validated and converted before query execution. Decimal mantissas retain arbitrary precision up to QWP DECIMAL256. The client can encode BINARY and IPv4 values (including `TypedNull.new(:binary)` and `TypedNull.new(:ipv4)`), but current Phase-1 QuestDB servers reject those bind kinds; this is a server limitation rather than local coercion.
+
+## Result types
+
+Numeric and boolean columns become Ruby numeric/boolean values. TIMESTAMP and TIMESTAMP_NS use the exact integer timestamp wrappers; DATE uses `DateMillis`. Decimal values use `DecimalValue`, UUID uses `UUID`, IPv4 uses `IPv4`, LONG256 uses `Long256`, GEOHASH uses `Geohash`, CHAR uses `Char`, and BINARY is an `ASCII-8BIT` String. SYMBOL and VARCHAR are UTF-8 strings. DOUBLE arrays become `Float64Array` with their per-row shape. SQL NULL always becomes `nil`.
+
+A zero-row QWP batch still establishes `QueryResult#columns`, preserving the result schema. `LONG_ARRAY` cannot be projected by the c-questdb-client 7.0 reader API, so materialization raises an explicit unsupported-column error instead of fabricating cell values (including for NULL cells). Future unknown wire kinds likewise raise rather than being guessed or silently corrupted; `OpaqueValue` remains available for a future native projection that can provide the original bytes.
+
+QWP query execution does not make a preceding WAL write immediately visible. An ingestion `OK` acknowledgement means QuestDB accepted the frame; WAL application remains asynchronous.

data/docs/release-infrastructure.md

@@ -0,0 +1,70 @@
+# Infrastructure for the initial 0.1.0 release
+
+This is an inventory for the early-adopter release, not authorization to run builds or publish. Existing approved hosts are sufficient; dedicated Azure/Hetzner automation is not technically required. `docs/releasing.md` currently specifies Azure/Hetzner agents, so using another provider requires a release-owner-approved policy change. No release jobs have been started by preparing this inventory.
+
+## Required for the advertised eight-gem release
+
+### Build and install-test hosts
+
+Five host classes cover the seven native platforms. They can be reused sequentially; 28 simultaneously provisioned machines are not needed.
+
+| Host | Native gem targets | Environment |
+| --- | --- | --- |
+| Linux x86-64 | `x86_64-linux-gnu`, `x86_64-linux-musl` | Docker; manylinux_2_28_x86_64 for GNU builds; Ruby Alpine containers for musl |
+| Linux ARM64 | `aarch64-linux-gnu`, `aarch64-linux-musl` | Docker; manylinux_2_28_aarch64 for GNU builds; ARM64 Ruby Alpine containers for musl |
+| macOS Intel | `x86_64-darwin` | Apple command-line tools; deployment target macOS 12.0 |
+| macOS Apple Silicon | `arm64-darwin` | Apple command-line tools; deployment target macOS 12.0 |
+| Windows x64 | `x64-mingw-ucrt` | RubyInstaller UCRT/MSYS2 development tools; Rust `x86_64-pc-windows-gnu` target |
+
+On each target:
+
+- MRI Ruby 3.1, 3.2, 3.3, and 3.4, appropriate Ruby development headers, Bundler, and Make/C compiler.
+- Rust 1.91.1 or newer, with the release toolchain recorded.
+- Binary stripping/linkage inspection tools; GNU builds must enforce the glibc 2.28 symbol ceiling.
+- Clean install-test environments for all four Ruby minors. Test the final gem outside the checkout so local extensions cannot mask missing package contents.
+- RubyGems 3.3.22+ for flavor-specific Linux packages; Bundler 2.5.6+ for musl selection.
+- Network access to GitHub, RubyGems, crates.io, and required image/toolchain registries during preparation. Source-gem validation must also run with Cargo networking disabled.
+
+This produces 28 versioned extensions, packaged into seven platform gems, plus one source gem. One Linux host can assemble and verify all eight packages and perform the source-gem offline install test. Handle the source gem's Ruby runtime dependencies explicitly; offline Cargo compilation does not imply offline Ruby dependency resolution. Native install tests should preserve installed runtime gems rather than accidentally demanding a compiler to reinstall `bigdecimal`.
+
+### Qualification environment
+
+The x86-64 Linux host can also provide:
+
+- Docker and the pinned QuestDB 10.0.1 OSS image for live QWP ingestion, queries, callbacks, and Arrow tests.
+- Two independently reachable QuestDB endpoints for the existing query-failover qualification script.
+- Disposable Docker volumes and local store-and-forward directories; permission to kill and restart only the qualification containers.
+- Nanoarrow and Red Arrow test dependencies, including Arrow GLib/GObject development packages.
+- Valgrind for the checked-in native lifecycle tests.
+- `bundler-audit`, `cargo-audit`, and an SPDX-capable SBOM generator.
+- Repository/package secret and path scanning, plus the checked-in license-header and package-verification scripts.
+
+No production database, Grafana deployment, or dedicated benchmark host is needed. Enterprise durable-ACK qualification is optional under the current initial-release policy. Shared-host benchmark results are informational.
+
+### Artifact retention
+
+An access-controlled artifact directory/store is sufficient; it need not be a new service. Retain:
+
+- The eight final gem files and sorted `SHA256SUMS`.
+- SPDX SBOM and build provenance: candidate SHA, recursive submodule SHAs, toolchains, runner/container identities, build commands, and artifact hashes.
+- Dependency-audit, unit/live/Arrow, native-memory, fault, and install-test results for the same candidate.
+- Release-owner approval and any explicit security exceptions.
+
+Publish these exact approved files without rebuilding them. Do not include credentials, Enterprise sources, private keys, or proprietary/local paths.
+
+### Publishing access
+
+- An approved RubyGems maintainer account with MFA and permission to claim/publish `questdb`; configure official QuestDB ownership.
+- A least-privilege RubyGems API key held only in the publisher's protected user-level credentials file (owner-only permissions) for manual publication. Never put it in source, logs, commands, or release artifacts.
+- An approved secure publishing workstation with RubyGems, checksum tools, Git, and a signing identity for the `v0.1.0` tag.
+- GitHub permissions to push the approved signed tag and create a release with the final artifacts and notes.
+- Representative GNU, musl, macOS, and Windows environments for post-publication installation checks; reuse the hosts above.
+
+## Can be deferred
+
+- Dedicated Azure/Hetzner pipeline migration, if an alternative is explicitly approved.
+- Automated publishing and RubyGems trusted-publishing integration.
+- Dedicated benchmark hardware and performance dashboards.
+- Enterprise-only durable-ACK test infrastructure.
+
+If all five host classes are unavailable, a source-only or reduced-platform initial release is possible only after explicitly narrowing the release policy and advertised precompiled-platform support. Do not silently omit promised packages.

data/docs/releasing.md

@@ -0,0 +1,47 @@
+# Releasing
+
+Releases contain one checksum-complete source gem and precompiled native gems for every supported platform. Release builds run only on approved QuestDB Azure Pipelines and Hetzner agents; preparing a candidate never authorizes running or publishing it.
+
+## Supported native gem matrix
+
+Each platform gem carries a versioned extension for MRI 3.1, 3.2, 3.3, and 3.4:
+
+- `x86_64-linux-gnu` and `aarch64-linux-gnu` (built and symbol-gated on glibc 2.28);
+- `x86_64-linux-musl` and `aarch64-linux-musl`;
+- `x86_64-darwin` and `arm64-darwin`;
+- `x64-mingw-ucrt`.
+
+The versioned layout is required because MRI's native ABI and `libruby` soname may change between minor releases. `lib/questdb/ingress/native.rb` selects the extension matching the running Ruby minor. Flavor-specific Linux packages require RubyGems 3.3.22 or newer, and musl users should use Bundler 2.5.6 or newer so the correct libc variant is selected.
+
+## Candidate validation
+
+1. Update `QuestDB::Ingress::VERSION` and change the changelog heading from `unreleased` to the intended release date.
+2. Build from a clean candidate commit with recursively initialized submodules on approved QuestDB infrastructure.
+3. Require Ruby and Rust dependency audits to pass without ignored advisories. Temporary exception for 0.1.0: the rustls advisories listed in `.cargo/audit.toml` are ignored until a patch release moves `c-questdb-client` to rustls 0.23.46 or newer; remove each ignore as soon as its fix ships. Enterprise-only durable-ACK qualification remains optional, matching the Python client's policy.
+4. Run unit, integration, Arrow, native-memory, package, and bounded fault qualification. Shared-host benchmark results remain informational.
+5. Produce all eight gems, then run `script/verify-gem` on each final file and inspect native linkage. Clean-install each native gem on MRI 3.1, 3.2, 3.3, and 3.4 on its matching platform.
+6. Confirm the source gem installs with `CARGO_NET_OFFLINE=true` in a clean `GEM_HOME` and contains no generated extension or nested QuestDB server source.
+7. Generate sorted `SHA256SUMS`, an SPDX SBOM, and build provenance from the final files.
+8. Confirm no credentials, private keys, Enterprise sources, proprietary paths, or local paths are present.
+9. Have a release owner approve the immutable artifact set before creating a tag or pushing any gem.
+
+## Initial 0.1 release policy
+
+`0.1.0` is a normal RubyGems version so users can install it without `--prerelease`. It is described as an initial pre-1.0 release in the README, changelog, and release notes. Use a version such as `0.1.0.beta.1` instead only if RubyGems should treat the package as a formal prerelease.
+
+## Publishing
+
+1. For the initial release, an approved QuestDB maintainer account with MFA enabled may claim the unowned `questdb` name. Add or transfer ownership to the official QuestDB maintainers as soon as that account is available.
+2. Create a signed `vVERSION` tag pointing to the exact candidate commit and verify the tag before publishing.
+3. Promote the already validated artifacts; do not rebuild different files in the publishing stage.
+4. Manual publication from an approved secure workstation is acceptable. Keep the least-privilege RubyGems key only in RubyGems' user-level credentials file, with owner-only filesystem permissions; do not pass it through an environment variable or command-line argument.
+5. From a directory outside the checkout, verify `SHA256SUMS`, then push the source gem followed by each of the seven platform gems. Let `gem push` prompt interactively for the MFA code.
+6. Push all platform variants for the same version only after the complete set has passed validation. RubyGems does not permit replacing a published file in place.
+7. Create the GitHub release with release notes, `SHA256SUMS`, SPDX SBOM, provenance, and the eight gem files.
+8. Install from RubyGems on representative GNU, musl, macOS, and Windows hosts and add official owners with `gem owner questdb --add OWNER` as required by QuestDB policy.
+
+Publishing credentials must never be written to the repository, logs, artifacts, shell history, connection strings, or release bundles. Revoke or rotate the initial key after ownership and long-term release access are configured.
+
+## Arrow binary policy
+
+Official extensions always compile the pinned c-client's `arrow` feature. `require "questdb/arrow"` is optional and no Arrow/dataframe gem is a runtime dependency, but Cargo features cannot be enabled after a precompiled extension is installed. Splitting Arrow into another native gem would statically link a second c-client and complicate ownership across extensions, so the modestly larger official binary is preferred.

data/docs/troubleshooting.md

@@ -0,0 +1,41 @@
+# Troubleshooting
+
+## `config_error`
+
+Only `ws::` and `wss::` are accepted. Check duplicate keys, escaped semicolons (`;;`), pool limits, TLS options, and store-and-forward paths. Configuration is parsed by the pinned native client.
+
+## `auth_error`, `tls_error`, or `handshake_error`
+
+QWP authenticates during the WebSocket HTTP upgrade. Use either Basic (`username`/`password`) or bearer `token`. For private PKI, configure `tls_roots`; do not disable verification in production. QuestDB QWP requires server 10.0+.
+
+## `failover_retry` and `in_doubt?`
+
+A `failover_retry` from `wait` or the acknowledgement phase of `flush(wait: true)` can be a no-progress timeout. It does not discard the queued frame or retire the sender; the background store-and-forward runner keeps delivering, so call `wait`/`await_acked_fsn` again rather than publishing the same rows again. Other failures follow the native reconnect/failover policy. If `error.in_doubt?` is true, replaying the same rows may duplicate data. Use designated client timestamps and appropriate QuestDB deduplication keys.
+
+## Direct column write failed or may be partial
+
+Direct column writes are not store-and-forward. Batches are pipelined and then committed at one checked ACK boundary. If an error follows one or more successful direct flushes, its message warns about partial completion and `in_doubt?` is true. Do not retry the complete Array set blindly; use stable designated timestamps with QuestDB deduplication, or reconcile the written timestamp/key range first. A pool/acquire/connect failure before the first flush is safe to retry only when `in_doubt?` is false.
+
+Length, type, UTF-8, range, shape, or `at:` errors are local and occur before borrowing. All-null columns need an explicit `ColumnVector`. BYTE, SHORT, BOOLEAN, and CHAR reject `nil` because QuestDB cannot represent a distinct NULL for those types. Row and direct ingestion have no cross-path ordering guarantee.
+
+## A flush succeeded but rows are not queryable
+
+Publish-only `flush` confirms only local store-and-forward acceptance. `flush(wait: true)` confirms server acceptance, not WAL application. Poll query visibility with a bounded deadline when read-after-write is required.
+
+## Server rejection appeared in logs
+
+Without `error_handler`, pool-wide diagnostics use the native default logger so rejections are never silent. With a handler, it receives an immutable `SenderError` on a dedicated Ruby thread; `Client#poll_error` returns a nonblocking polling copy. A pooled sender can already have been returned or closed when its pool-wide diagnostic arrives. Check `error_events_dropped` and the pool delivered/dropped counters if diagnostics arrive faster than they are consumed.
+
+Callback exceptions are reported with `warn` and do not stop dispatch or poison the client. Connection events and rejections are diagnostic signals, not ACK or WAL-visibility barriers. Slow handlers can overflow bounded drop-oldest queues; increase the matching Ruby-only inbox capacity or keep handlers lightweight. If a callback closes its client, dispatch shuts down without joining itself. A handler that captures the client keeps it alive, so use explicit `close`.
+
+## Partial or abandoned query results
+
+Fully iterate, cancel, or close every `QueryResult`. Closing an undrained result intentionally terminates its reader lease; the connection is not returned to the pool because unread protocol frames would contaminate the next query. Use `Client#query` with a block for automatic cleanup. A result is single-use.
+
+## `client is busy`, pooled lease errors, or thread errors
+
+Do not share sender, reader, or query-result leases across threads. Borrow one lease per thread. Client pool methods may be called concurrently through the Ruby wrapper, but closing a client with active leases is rejected. Always use block forms or explicit `close`; GC cleanup is a deferred emergency path that drops abandoned handles instead of recycling them.
+
+## Offline native build fails
+
+Use Rust 1.91.1 or newer. Confirm `vendor/c-questdb-client` is at the revision in `vendor/c-questdb-client.version`, initialize submodules, and keep `.cargo/config.toml` pointing to `vendor/rust-dependencies`. Cargo checksum errors indicate a modified or incompletely packaged vendored source tree.

data/examples/basic.rb

@@ -0,0 +1,32 @@
+#      ___                  _   ____  ____
+#     / _ \ _   _  ___  ___| |_|  _ \| __ )
+#    | | | | | | |/ _ \/ __| __| | | |  _ \
+#    | |_| | |_| |  __/\__ \ |_| |_| | |_) |
+#     \__\_\\__,_|\___||___/\__|____/|____/
+#
+#   Copyright (c) 2014-2019 Appsicle
+#   Copyright (c) 2019-2026 QuestDB
+#
+#   Licensed under the Apache License, Version 2.0 (the "License");
+#   you may not use this file except in compliance with the License.
+#   You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+#   Unless required by applicable law or agreed to in writing, software
+#   distributed under the License is distributed on an "AS IS" BASIS,
+#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#   See the License for the specific language governing permissions and
+#   limitations under the License.
+#
+require "questdb"
+
+QuestDB.connect(host: "localhost") do |client|
+  client.sender do |sender|
+    sender.row("trades",
+               symbols: { symbol: "ETH-USD", side: "sell" },
+               columns: { price: 2615.54, amount: 0.00044 },
+               at: QuestDB::TimestampNanos.now)
+    sender.flush(wait: true)
+  end
+end