skaidb 1.1.0 → 1.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +47 -0
- data/README.md +55 -16
- data/docs/api.md +44 -10
- data/docs/pooling.md +131 -9
- data/docs/streaming.md +37 -2
- data/docs/types.md +3 -2
- data/lib/skaidb.rb +753 -97
- metadata +2 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: a452610f682221a59cfbbc48e42c9a37f1c0e880d2191037f1c9c7151a3034fa
|
|
4
|
+
data.tar.gz: 1a73f7f552091bfaed9b4072dfd684b57daac637ff12d163a34bf1c278bef784
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: ce44a1e28e0add382aa9fde578eb09350e1b5306d22c3cf5d272faa5d606c8ec05be08b77967d014a33ceddba5fefd0ed449503c24d4997680c37af156ad2e5f
|
|
7
|
+
data.tar.gz: 3d34175d862982988e593fb42c631bdbb9cd6f23682908ec40fc53a7d25fe3868105201dd50988108508783bfb8bf076065737864d71c6cc5337c8b6c371e6c3
|
data/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,50 @@ All notable changes to the skaidb Ruby driver. The format follows
|
|
|
4
4
|
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versions follow
|
|
5
5
|
[Semantic Versioning](https://semver.org/).
|
|
6
6
|
|
|
7
|
+
## [1.3.0] - 2026-10-29
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
- **Parameterised streaming**: `conn.stream(sql, params)` binds `$1`, `$2`,
|
|
11
|
+
… exactly as `exec_params(sql, params)` does — typed, with the same values
|
|
12
|
+
(Arrays, Hashes, `Time`, `BigDecimal`, `Skaidb::Uuid`, …) and the same
|
|
13
|
+
placeholder checks — over `OP_QUERY_STREAM_PARAMS` (server ≥ 0.330), so a
|
|
14
|
+
range-bounded export no longer needs its values written into the SQL. It
|
|
15
|
+
takes `consistency:`, a block or the Enumerator form like any stream.
|
|
16
|
+
Without parameters (`nil` or `[]`) the request is the original
|
|
17
|
+
`OP_QUERY_STREAM`, byte for byte, so it works against every server; a
|
|
18
|
+
parameterised stream against an older server raises `QueryError`
|
|
19
|
+
(`server does not support parameterised streaming (skaidb >= 0.330): …`)
|
|
20
|
+
and leaves the connection usable. `params` that are not an Array raise
|
|
21
|
+
`ArgumentError`.
|
|
22
|
+
- The vendored conformance suite's `stream_params` case (a streamed
|
|
23
|
+
statement with bound parameters) runs through `stream(sql, params)`.
|
|
24
|
+
|
|
25
|
+
## [1.2.0] - 2026-10-25
|
|
26
|
+
|
|
27
|
+
### Added
|
|
28
|
+
- **Pool load balancing across members** (the rules every skaidb driver
|
|
29
|
+
shares, PROTOCOL.md §6.1): `load_balancing: :round_robin` /
|
|
30
|
+
`:least_latency` (power of two choices on the round-trip average ×
|
|
31
|
+
checked-out connections, re-measuring a member not used in the last 2 s;
|
|
32
|
+
`:random` stays the default), `idle_order: :fifo`, health ejection
|
|
33
|
+
(consecutive transport failures, or a latency average far above the other
|
|
34
|
+
members') with a background probe that re-admits the member,
|
|
35
|
+
`discover: true` (members from `SHOW ENDPOINTS`, server ≥ 0.327: joiners
|
|
36
|
+
added, leavers dropped, resyncing/unreachable members skipped and
|
|
37
|
+
re-checked every 5 s), and `read_your_writes: <seconds>` with
|
|
38
|
+
`checkout(affinity: key)` / `with(affinity: key)` — reads follow the member
|
|
39
|
+
that took the key's last write. `pool.members` reports each member's
|
|
40
|
+
state, latency and connections; `logger:` is told of ejections and
|
|
41
|
+
re-admissions. Options take Symbols or Strings.
|
|
42
|
+
- `Connection#endpoint`: the `"host:port"` of the member a connection reached.
|
|
43
|
+
|
|
44
|
+
### Changed
|
|
45
|
+
- A pooled connection that re-dials does so in the pool's order (usable
|
|
46
|
+
members first, the one it lost last) instead of shuffling the seeds.
|
|
47
|
+
- `Skaidb::Pool.new` checks its `Skaidb.connect` keywords when it is created
|
|
48
|
+
instead of at the first checkout, and a connection checked in to a pool
|
|
49
|
+
that did not create it is closed instead of kept.
|
|
50
|
+
|
|
7
51
|
## [1.1.0] - 2026-09-26
|
|
8
52
|
|
|
9
53
|
### Added
|
|
@@ -148,6 +192,9 @@ batches, a 2 500-row stream, error handling, and the `drivers` table showing
|
|
|
148
192
|
because it is a bundled gem since Ruby 3.4.
|
|
149
193
|
- The example moved to `examples/basic.rb`.
|
|
150
194
|
|
|
195
|
+
[1.3.0]: https://github.com/porcupin26/skaidb-ruby/compare/v1.2.0...v1.3.0
|
|
196
|
+
[1.2.0]: https://github.com/porcupin26/skaidb-ruby/compare/v1.1.0...v1.2.0
|
|
197
|
+
[1.1.0]: https://github.com/porcupin26/skaidb-ruby/compare/v1.0.3...v1.1.0
|
|
151
198
|
[1.0.3]: https://github.com/porcupin26/skaidb-ruby/compare/v1.0.2...v1.0.3
|
|
152
199
|
[1.0.2]: https://github.com/porcupin26/skaidb-ruby/compare/v1.0.1...v1.0.2
|
|
153
200
|
[1.0.1]: https://github.com/porcupin26/skaidb-ruby/compare/v1.0.0...v1.0.1
|
data/README.md
CHANGED
|
@@ -13,7 +13,8 @@ and the result behaves like `PG::Result`. **Pure standard library** (`socket`,
|
|
|
13
13
|
Ruby 2.7 through 3.4. It speaks skaidb's binary protocol directly:
|
|
14
14
|
SCRAM-SHA-256 authentication, server-side prepared statements with typed
|
|
15
15
|
parameters, one-round-trip batches, streamed result sets, multi-seed failover,
|
|
16
|
-
transparent reconnect, TLS and connection
|
|
16
|
+
transparent reconnect, TLS and a connection pool that balances across the
|
|
17
|
+
cluster's members.
|
|
17
18
|
|
|
18
19
|
- Repository: <https://github.com/porcupin26/skaidb-ruby>
|
|
19
20
|
- RubyGems: <https://rubygems.org/gems/skaidb>
|
|
@@ -54,13 +55,13 @@ Xcode command-line tools on macOS). `gem install skaidb` and vendoring reuse
|
|
|
54
55
|
the `bigdecimal` already installed.
|
|
55
56
|
|
|
56
57
|
Or vendor the single file: copy
|
|
57
|
-
[`lib/skaidb.rb`](https://github.com/porcupin26/skaidb-ruby/blob/v1.
|
|
58
|
+
[`lib/skaidb.rb`](https://github.com/porcupin26/skaidb-ruby/blob/v1.3.0/lib/skaidb.rb)
|
|
58
59
|
into your project and `require_relative` it. It has no dependencies beyond
|
|
59
60
|
Ruby's standard library.
|
|
60
61
|
|
|
61
62
|
Every release is also attached as a `.gem` to its
|
|
62
63
|
[GitHub release](https://github.com/porcupin26/skaidb-ruby/releases) for
|
|
63
|
-
installs without registry access: `gem install ./skaidb-1.
|
|
64
|
+
installs without registry access: `gem install ./skaidb-1.3.0.gem`.
|
|
64
65
|
|
|
65
66
|
## Quick start
|
|
66
67
|
|
|
@@ -98,7 +99,8 @@ end
|
|
|
98
99
|
prepared statements, `exec_batch`
|
|
99
100
|
- [Results](#results)
|
|
100
101
|
- [Streaming large results](#streaming-large-results)
|
|
101
|
-
- [Connection pool](#connection-pool)
|
|
102
|
+
- [Connection pool](#connection-pool) — balancing across members,
|
|
103
|
+
health ejection, discovery, read-your-writes
|
|
102
104
|
- [Streams (`CREATE STREAM`)](#streams-create-stream) — `subscribe`
|
|
103
105
|
- [Type mapping](#type-mapping)
|
|
104
106
|
- [Errors](#errors)
|
|
@@ -278,8 +280,9 @@ A result holds all its rows in memory; for results that do not fit, stream.
|
|
|
278
280
|
|
|
279
281
|
## Streaming large results
|
|
280
282
|
|
|
281
|
-
`conn.stream(sql, consistency: nil)` runs the statement over the
|
|
282
|
-
opcode and yields one row Hash at a time, holding **one chunk** in
|
|
283
|
+
`conn.stream(sql, params = [], consistency: nil)` runs the statement over the
|
|
284
|
+
streaming opcode and yields one row Hash at a time, holding **one chunk** in
|
|
285
|
+
memory:
|
|
283
286
|
|
|
284
287
|
```ruby
|
|
285
288
|
conn.stream("SELECT id, pad FROM big ORDER BY id") do |row|
|
|
@@ -287,12 +290,23 @@ conn.stream("SELECT id, pad FROM big ORDER BY id") do |row|
|
|
|
287
290
|
end
|
|
288
291
|
|
|
289
292
|
first = conn.stream("SELECT id FROM big ORDER BY id").first(10) # Enumerator form
|
|
293
|
+
|
|
294
|
+
conn.stream("SELECT id, ts, body FROM events WHERE ts >= $1 AND ts < $2", [from, to]) do |row|
|
|
295
|
+
export(row) # bound parameters, typed
|
|
296
|
+
end
|
|
290
297
|
```
|
|
291
298
|
|
|
292
|
-
`
|
|
293
|
-
|
|
294
|
-
(
|
|
295
|
-
|
|
299
|
+
`params` bind `$1`, `$2`, … exactly as `exec_params` binds them — typed, with
|
|
300
|
+
the same values accepted — and need a server ≥ 0.330: an older one raises
|
|
301
|
+
`QueryError` (`server does not support parameterised streaming (skaidb >=
|
|
302
|
+
0.330): …`), and there is no client-side text fallback. A stream without
|
|
303
|
+
parameters sends the original streaming opcode, which every server
|
|
304
|
+
understands. The server pages a plain `SELECT` (bare columns from one table,
|
|
305
|
+
unordered or in storage-key order), and no scan budget applies to a streamed
|
|
306
|
+
page (server ≥ 0.330). With a block `stream` returns a `Result` whose `kind`
|
|
307
|
+
is `:stream` and whose `fields` are the column names (the rows went to the
|
|
308
|
+
block); a non-row statement returns its `:mutation` / `:ddl` Result without
|
|
309
|
+
yielding.
|
|
296
310
|
|
|
297
311
|
**The abandon/drain rule.** The connection is busy for the whole stream:
|
|
298
312
|
a statement from another thread waits for it. Leaving the block early —
|
|
@@ -321,7 +335,29 @@ pool.close # closes idle connections
|
|
|
321
335
|
blocks — when no idle connection is available a new one is dialed, and a
|
|
322
336
|
returned connection beyond `maxsize` is closed. Connections are validated with
|
|
323
337
|
`usable?` on checkout and check-in, so one broken by a transport error or an
|
|
324
|
-
undrained stream is closed and replaced.
|
|
338
|
+
undrained stream is closed and replaced.
|
|
339
|
+
|
|
340
|
+
The pool spreads its connections across the members by the rules every skaidb
|
|
341
|
+
driver shares
|
|
342
|
+
([PROTOCOL.md §6.1](https://skaidb.org/docs/PROTOCOL.html#61-balancing-a-pool-across-members)):
|
|
343
|
+
|
|
344
|
+
```ruby
|
|
345
|
+
pool = Skaidb::Pool.new(seeds: ["db1", "db2", "db3"], database: "app",
|
|
346
|
+
load_balancing: :least_latency, # or :round_robin; :random is the default
|
|
347
|
+
discover: true, # members from SHOW ENDPOINTS, resyncing ones skipped
|
|
348
|
+
read_your_writes: 2.0) # seconds; see affinity below
|
|
349
|
+
|
|
350
|
+
pool.with(affinity: user_id) { |conn| conn.exec_params("INSERT INTO notes (id, owner) VALUES ($1, $2)", [nid, user_id]) }
|
|
351
|
+
pool.with(affinity: user_id) { |conn| conn.exec_params("SELECT * FROM notes WHERE id = $1", [nid]) } # same member
|
|
352
|
+
|
|
353
|
+
pool.members # => [{ endpoint: "db1:7000", canonical: "10.0.0.1:7000", state: :ready,
|
|
354
|
+
# latency_ms: 0.8, in_use: 2, idle: 3, seed: true }, …]
|
|
355
|
+
```
|
|
356
|
+
|
|
357
|
+
A member with consecutive transport failures, or a round-trip average far
|
|
358
|
+
above the others', is ejected and re-admitted by a background probe (the last
|
|
359
|
+
usable member never is); `idle_order: :fifo` keeps every idle connection in
|
|
360
|
+
use. Details: [docs/pooling.md](https://github.com/porcupin26/skaidb-ruby/blob/main/docs/pooling.md).
|
|
325
361
|
|
|
326
362
|
## Streams (`CREATE STREAM`)
|
|
327
363
|
|
|
@@ -393,8 +429,9 @@ StandardError
|
|
|
393
429
|
connection stays usable.
|
|
394
430
|
- A transport failure (`ConnectionError`) marks the connection broken; the
|
|
395
431
|
next statement re-dials, or a pool replaces it.
|
|
396
|
-
- `ArgumentError` for `Pool.new(maxsize: 0)`
|
|
397
|
-
|
|
432
|
+
- `ArgumentError` for `Pool.new(maxsize: 0)` or an unknown
|
|
433
|
+
`load_balancing`/`idle_order`, for `Skaidb::Uuid.new` of a string that
|
|
434
|
+
is not a UUID, and for `stream` parameters that are not an Array.
|
|
398
435
|
|
|
399
436
|
## Transactions
|
|
400
437
|
|
|
@@ -421,7 +458,7 @@ frame after every handshake. It shows up as `client_name = 'ruby'` /
|
|
|
421
458
|
|
|
422
459
|
```sql
|
|
423
460
|
SELECT client_name, client_version FROM drivers;
|
|
424
|
-
-- ruby | 1.
|
|
461
|
+
-- ruby | 1.3.0
|
|
425
462
|
```
|
|
426
463
|
|
|
427
464
|
The server records the row asynchronously, so a `SELECT` immediately after
|
|
@@ -434,8 +471,10 @@ know the opcode answers with an error the driver ignores.
|
|
|
434
471
|
`bigdecimal` is declared because it is a bundled gem since Ruby 3.4.
|
|
435
472
|
- Works with any skaidb server. Where a server predates an opcode the driver
|
|
436
473
|
falls back: prepared statements → client-side text binding, `Hello` →
|
|
437
|
-
ignored
|
|
438
|
-
|
|
474
|
+
ignored, pool discovery (`SHOW ENDPOINTS`, server ≥ 0.327) → the
|
|
475
|
+
configured seeds. `stream` and `exec_batch` need a server with those
|
|
476
|
+
opcodes, and `stream` with parameters a server ≥ 0.330; they raise
|
|
477
|
+
`QueryError` otherwise.
|
|
439
478
|
- The wire protocol (framing, SCRAM handshake, value encoding, opcodes) is
|
|
440
479
|
specified at <https://skaidb.org/docs/PROTOCOL.html>. The driver speaks the
|
|
441
480
|
binary protocol on port 7000; the server's REST/JSON gateway (port 7080)
|
data/docs/api.md
CHANGED
|
@@ -55,8 +55,16 @@ Raises `Skaidb::Error` before dialing for an unknown `auth_mechanism`, a
|
|
|
55
55
|
- `prepare_server(sql) → [id, nparams] | nil` — prepare a statement with `?`
|
|
56
56
|
placeholders on the server; `nil` when the server refuses. Cached per
|
|
57
57
|
connection (240 entries); the cache is cleared on re-dial.
|
|
58
|
-
- `stream(sql, consistency: nil) { |row| … } → Result` — see
|
|
59
|
-
[streaming.md](streaming.md).
|
|
58
|
+
- `stream(sql, params = [], consistency: nil) { |row| … } → Result` — see
|
|
59
|
+
[streaming.md](streaming.md). `params` (an Array) bind `$1`, `$2`, …
|
|
60
|
+
exactly as `exec_params` binds them, typed, over `OP_QUERY_STREAM_PARAMS`
|
|
61
|
+
(server ≥ 0.330; an older one raises `QueryError` `server does not support
|
|
62
|
+
parameterised streaming (skaidb >= 0.330): …`, with no text fallback);
|
|
63
|
+
without parameters (`nil` or `[]`) the request is the original
|
|
64
|
+
`OP_QUERY_STREAM`, which every server understands. A placeholder/parameter
|
|
65
|
+
mismatch or a value that cannot be bound raises `QueryError` before
|
|
66
|
+
anything is sent, and `params` that are not an Array raise
|
|
67
|
+
`ArgumentError`. For a row stream the returned Result has
|
|
60
68
|
`kind == :stream`, the column names in `fields` and no `rows` (they went to
|
|
61
69
|
the block); a statement without rows returns its `:mutation` / `:ddl`
|
|
62
70
|
Result without yielding. Without a block returns an Enumerator.
|
|
@@ -73,6 +81,8 @@ Raises `Skaidb::Error` before dialing for an unknown `auth_mechanism`, a
|
|
|
73
81
|
- `usable?` — `false` once closed, once a transport error broke the socket,
|
|
74
82
|
or while a stream is in flight (including one abandoned through external
|
|
75
83
|
iteration). No round-trip.
|
|
84
|
+
- `endpoint` — the member the connection reached, as the `"host:port"` it
|
|
85
|
+
dialed (it changes when a re-dial lands on another member).
|
|
76
86
|
- `closed`, `finished?` — `true` after `close`.
|
|
77
87
|
- `close` — closes the socket; idempotent. Does not take the connection's
|
|
78
88
|
lock, so another thread can use it to break a blocked read.
|
|
@@ -81,10 +91,11 @@ Raises `Skaidb::Error` before dialing for an unknown `auth_mechanism`, a
|
|
|
81
91
|
|
|
82
92
|
A transport failure raises `Skaidb::ConnectionError` and marks the
|
|
83
93
|
connection broken. The next `exec`/`exec_params`/`exec_batch`/`stream`
|
|
84
|
-
re-dials across the original seeds
|
|
85
|
-
|
|
86
|
-
re-
|
|
87
|
-
|
|
94
|
+
re-dials across the original seeds (a pooled connection: in the pool's
|
|
95
|
+
order, usable members first and the one it lost last), re-authenticates,
|
|
96
|
+
re-sends `Hello`, re-runs `USE`, and starts with an empty prepared-statement
|
|
97
|
+
cache. If the re-dial fails, that statement raises `ConnectionError` and the
|
|
98
|
+
one after it tries again.
|
|
88
99
|
|
|
89
100
|
## `Skaidb::Result`
|
|
90
101
|
|
|
@@ -105,10 +116,33 @@ Shaped like `PG::Result`; `Enumerable` over row Hashes.
|
|
|
105
116
|
|
|
106
117
|
## `Skaidb::Pool`
|
|
107
118
|
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
119
|
+
```ruby
|
|
120
|
+
Skaidb::Pool.new(maxsize: 10, load_balancing: :random, idle_order: :lifo, discover: false,
|
|
121
|
+
refresh_interval: 60.0, read_your_writes: 0.0, eject_errors: 3,
|
|
122
|
+
eject_latency_factor: 3.0, eject_latency_floor: 0.05, eject_backoff: 5.0,
|
|
123
|
+
eject_backoff_max: 60.0, logger: nil, **connect_options)
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
A thread-safe pool balanced across the cluster's members — see
|
|
127
|
+
[pooling.md](pooling.md) for every option.
|
|
128
|
+
|
|
129
|
+
- `with(affinity: nil) { |conn| … }` — check out, yield, check in; returns
|
|
130
|
+
the block's value.
|
|
131
|
+
- `checkout(affinity: nil) → Connection` — an idle connection to the member
|
|
132
|
+
the policy picks, or a new one. Never blocks. `affinity` (any Hash key;
|
|
133
|
+
`nil` = pool-wide) scopes `read_your_writes`.
|
|
134
|
+
- `checkin(conn)` — return it; closed instead when broken, when its member
|
|
135
|
+
is ejected or gone, or when `maxsize` are already idle.
|
|
136
|
+
- `members → Array<Hash>` — per member: `:endpoint`, `:canonical`, `:state`
|
|
137
|
+
(`:ready`, `:ejected`, `:resyncing`, `:unreachable`, `:unlisted`),
|
|
138
|
+
`:latency_ms` (`nil` until measured), `:in_use`, `:idle`, `:seed`.
|
|
139
|
+
- `close` — closes the idle connections and stops the background thread.
|
|
140
|
+
|
|
141
|
+
`load_balancing` (`:random`, `:round_robin`, `:least_latency`) and
|
|
142
|
+
`idle_order` (`:lifo`, `:fifo`) take a Symbol or a String. Raises
|
|
143
|
+
`ArgumentError` for `maxsize < 1`, `eject_errors < 1` or an unknown
|
|
144
|
+
`load_balancing`/`idle_order`, and `Skaidb::Error` on checkout from a closed
|
|
145
|
+
pool.
|
|
112
146
|
|
|
113
147
|
## `Skaidb::Uuid`
|
|
114
148
|
|
data/docs/pooling.md
CHANGED
|
@@ -2,7 +2,9 @@
|
|
|
2
2
|
|
|
3
3
|
`Skaidb::Pool` keeps a bounded set of idle connections and hands one to each
|
|
4
4
|
caller, so a threaded server or job runner can use one connection per
|
|
5
|
-
thread without dialing and authenticating on every request.
|
|
5
|
+
thread without dialing and authenticating on every request. It spreads its
|
|
6
|
+
connections across the cluster's members by the rules every skaidb driver
|
|
7
|
+
shares.
|
|
6
8
|
|
|
7
9
|
```ruby
|
|
8
10
|
pool = Skaidb::Pool.new(seeds: ["db1", "db2", "db3"], user: "app", password: "secret",
|
|
@@ -22,11 +24,126 @@ end
|
|
|
22
24
|
pool.close # closes every idle connection
|
|
23
25
|
```
|
|
24
26
|
|
|
27
|
+
## API
|
|
28
|
+
|
|
29
|
+
```ruby
|
|
30
|
+
Skaidb::Pool.new(
|
|
31
|
+
maxsize: 10,
|
|
32
|
+
load_balancing: :random, # :random | :round_robin | :least_latency
|
|
33
|
+
idle_order: :lifo, # :lifo | :fifo
|
|
34
|
+
discover: false, # read the members from SHOW ENDPOINTS
|
|
35
|
+
refresh_interval: 60.0, # seconds between discovery refreshes
|
|
36
|
+
read_your_writes: 0.0, # seconds; 0 = off
|
|
37
|
+
eject_errors: 3, # consecutive transport failures
|
|
38
|
+
eject_latency_factor: 3.0, # x the other members' median; 0 = off
|
|
39
|
+
eject_latency_floor: 0.05, # seconds above the median, at least
|
|
40
|
+
eject_backoff: 5.0, # first ejection, doubling per repeat
|
|
41
|
+
eject_backoff_max: 60.0,
|
|
42
|
+
logger: nil, # anything with #warn: ejections, re-admissions
|
|
43
|
+
**connect_options # every Skaidb.connect keyword
|
|
44
|
+
)
|
|
45
|
+
|
|
46
|
+
pool.with(affinity: nil) { |conn| … } # the block's value
|
|
47
|
+
pool.checkout(affinity: nil) -> Skaidb::Connection
|
|
48
|
+
pool.checkin(conn)
|
|
49
|
+
pool.members -> Array of Hashes (see "Watching it")
|
|
50
|
+
pool.close
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
`load_balancing` and `idle_order` take a Symbol or a String. `host`/`port`/
|
|
54
|
+
`seeds` name the starting members; the other `Skaidb.connect` keywords
|
|
55
|
+
(`database`, TLS, `consistency`, `timeout`) apply to every connection, so
|
|
56
|
+
pooled connections run `USE` and authenticate exactly like single ones. A
|
|
57
|
+
pooled connection that re-dials after a transport error does so in the
|
|
58
|
+
pool's order: usable members first, the one it lost last.
|
|
59
|
+
`Connection#endpoint` names the member a connection reached.
|
|
60
|
+
|
|
61
|
+
## Balancing across members
|
|
62
|
+
|
|
63
|
+
With the default `load_balancing: :random` each new connection dials the
|
|
64
|
+
members in random order and stays on the member it reached — fine for many
|
|
65
|
+
clients, but one process can end up with most of its connections on one
|
|
66
|
+
member. Pick a policy to spread a single process's load:
|
|
67
|
+
|
|
68
|
+
```ruby
|
|
69
|
+
pool = Skaidb::Pool.new(
|
|
70
|
+
seeds: ["db1", "db2", "db3"], database: "app", user: "app", password: pw,
|
|
71
|
+
load_balancing: :least_latency, # or :round_robin
|
|
72
|
+
discover: true, # learn the members from the cluster
|
|
73
|
+
read_your_writes: 2.0, # seconds; see below
|
|
74
|
+
idle_order: :fifo
|
|
75
|
+
)
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
| Option | Default | What it does |
|
|
79
|
+
|---|---|---|
|
|
80
|
+
| `load_balancing` | `:random` | `:round_robin`: checkouts rotate over the usable members in endpoint order. `:least_latency`: of two members picked at random, the one with the lower round-trip average × (checked-out connections + 1), so near and idle members win; a member not measured yet, or not in the last 2 s, scores 0 and is tried, so a lightly loaded pool keeps re-measuring every member. |
|
|
81
|
+
| `idle_order` | `:lifo` | `:lifo` hands out the most recently returned connection first, so a quiet pool keeps few sockets warm. `:fifo` hands out the longest-idle one first, so every idle connection keeps being used and a server-side idle timeout never finds one. |
|
|
82
|
+
| `discover` | `false` | Reads the members with `SHOW ENDPOINTS` (server ≥ 0.327; no privilege needed) when a member is first reached and every `refresh_interval` (60 s) after — every 5 s while a member is `resyncing` or `unreachable`. New members are added, members that left are dropped (seeds are kept, marked `unlisted`), and members the server reports `resyncing` or `unreachable` get no new connections until they are `ready` again, when they start with a fresh latency average. An older or standalone server leaves the seeds as they are, and the pool stops asking. |
|
|
83
|
+
| `eject_errors` | `3` | Consecutive transport failures (a failed dial, an I/O error mid-request, a capacity refusal) that eject a member. Statement errors and refused logins do not count. |
|
|
84
|
+
| `eject_latency_factor`, `eject_latency_floor` | `3.0`, `0.05` | A member whose round-trip average is above factor × the other usable members' median **and** more than floor seconds above it is ejected (once it and another member have 10 samples each). A factor of `0` disables it. |
|
|
85
|
+
| `eject_backoff`, `eject_backoff_max` | `5.0`, `60.0` | How long an ejection lasts, doubling for each repeat up to the max; a member that stays healthy for `eject_backoff_max` after a re-admission starts over at the base. When the time is up a background thread dials the member and runs one statement: success re-admits it (with a fresh latency average), failure ejects it again. The last usable member is never ejected — a slow or flaky member beats none. |
|
|
86
|
+
| `read_your_writes` | `0.0` | Seconds after a write during which checkouts with the same `affinity` go to the member that took the write. |
|
|
87
|
+
| `logger` | `nil` | A `Logger` (or anything with `warn`) told of each ejection and re-admission. |
|
|
88
|
+
|
|
89
|
+
An ejected member gets no new connections and its idle ones are closed; a
|
|
90
|
+
connection to it that is returned to the pool is closed instead of kept.
|
|
91
|
+
With a target member (round-robin, least-latency, an affinity) the pool
|
|
92
|
+
hands out an idle connection to that member or dials it; a failed dial falls
|
|
93
|
+
through to the other usable members, then to the unusable ones, so a
|
|
94
|
+
checkout never has nowhere to go while any member is configured.
|
|
95
|
+
|
|
96
|
+
**Latency.** Per member the pool keeps an exponentially weighted average of
|
|
97
|
+
round trips (request sent to the first response frame): each sample moves it
|
|
98
|
+
5 % of the way, and no sample counts for more than 10 × the larger of the
|
|
99
|
+
current average and `eject_latency_floor`. One slow statement barely moves
|
|
100
|
+
it; a member that stays slow is noticed within a few dozen statements.
|
|
101
|
+
|
|
102
|
+
**Read-your-writes.** The member that coordinated a write reads it back at
|
|
103
|
+
any consistency, `ONE` included; another member may not have applied it yet.
|
|
104
|
+
With `read_your_writes` set, pass an affinity key — a user or session id —
|
|
105
|
+
when you check out:
|
|
106
|
+
|
|
107
|
+
```ruby
|
|
108
|
+
pool.with(affinity: user_id) do |conn|
|
|
109
|
+
conn.exec_params("INSERT INTO notes (id, owner, body) VALUES ($1, $2, $3)", [nid, user_id, body])
|
|
110
|
+
end
|
|
111
|
+
# … the next request for the same user, within the window:
|
|
112
|
+
pool.with(affinity: user_id) do |conn| # same member
|
|
113
|
+
conn.exec_params("SELECT body FROM notes WHERE id = $1", [nid])
|
|
114
|
+
end
|
|
115
|
+
```
|
|
116
|
+
|
|
117
|
+
The window starts when a write succeeds on a connection checked out under
|
|
118
|
+
the key, and checkouts under it go to that member until the window ends
|
|
119
|
+
(unless the member is ejected or not `ready`). Checkouts without a key
|
|
120
|
+
share one pool-wide key, so with constant writes everything follows the
|
|
121
|
+
latest writer — pass a key when you balance. A statement is a write unless
|
|
122
|
+
its first keyword (after whitespace, comments and opening parentheses) is
|
|
123
|
+
`SELECT`, `SHOW`, `EXPLAIN`, `DESCRIBE`, `DESC`, `VALUES`, `WITH` or `USE`.
|
|
124
|
+
|
|
125
|
+
Derive the key without touching the pool: if it comes from something that
|
|
126
|
+
itself queries the database through the same pool (a web framework's
|
|
127
|
+
"current user" loader), a helper that checks out a connection and computes
|
|
128
|
+
the key inside it re-enters itself. Read the raw id from the session or
|
|
129
|
+
token instead.
|
|
130
|
+
|
|
131
|
+
**Watching it.** `pool.members` returns one Hash per member, sorted by
|
|
132
|
+
endpoint:
|
|
133
|
+
|
|
134
|
+
```ruby
|
|
135
|
+
[{ endpoint: "db1:7000", canonical: "10.0.0.1:7000", state: :ready,
|
|
136
|
+
latency_ms: 0.8, in_use: 2, idle: 3, seed: true }, …]
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
`endpoint` is what the pool dials, `canonical` the name the member reports
|
|
140
|
+
for itself (once discovered), `state` one of `:ready`, `:ejected`,
|
|
141
|
+
`:resyncing`, `:unreachable`, `:unlisted`, and `latency_ms` is `nil` until
|
|
142
|
+
the member has been measured. The rules are the same in every skaidb driver
|
|
143
|
+
([PROTOCOL.md §6.1](https://skaidb.org/docs/PROTOCOL.html#61-balancing-a-pool-across-members)).
|
|
144
|
+
|
|
25
145
|
## Behaviour
|
|
26
146
|
|
|
27
|
-
- Every `Skaidb.connect` keyword passes through (`seeds`, `database`, TLS,
|
|
28
|
-
`consistency`, `timeout`), so pooled connections fail over across seeds
|
|
29
|
-
and run `USE` exactly like single ones.
|
|
30
147
|
- `maxsize` (default 10) bounds the connections kept **idle**, not the number
|
|
31
148
|
checked out. `checkout` never blocks: with no idle connection available it
|
|
32
149
|
dials a new one, and a connection returned while `maxsize` are already idle
|
|
@@ -38,10 +155,15 @@ pool.close # closes every idle connection
|
|
|
38
155
|
and will fail its first statement with `Skaidb::ConnectionError`; that
|
|
39
156
|
failure marks it broken, so `checkin` discards it and the next checkout
|
|
40
157
|
gets a fresh one. Retry the statement in that case if it is safe to.
|
|
41
|
-
- `close` marks the pool closed
|
|
42
|
-
checked out at the time are closed when
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
158
|
+
- `close` marks the pool closed, closes the idle connections and stops the
|
|
159
|
+
background thread; connections checked out at the time are closed when
|
|
160
|
+
they are returned. `checkout` on a closed pool raises `Skaidb::Error`.
|
|
161
|
+
`maxsize < 1`, `eject_errors < 1` and an unknown `load_balancing` or
|
|
162
|
+
`idle_order` raise `ArgumentError`.
|
|
163
|
+
- The pool is thread-safe: the idle list and the member table are guarded
|
|
164
|
+
by one mutex. The background thread (probes and discovery refreshes) is
|
|
165
|
+
started on first need; Ruby never waits for it at exit. A single
|
|
166
|
+
connection should still be used by one thread at a time; the pool is what
|
|
167
|
+
gives each thread its own.
|
|
46
168
|
|
|
47
169
|
See also `examples/pool.rb`.
|
data/docs/streaming.md
CHANGED
|
@@ -4,7 +4,12 @@
|
|
|
4
4
|
a result that should not be held in memory at once, `stream` runs the
|
|
5
5
|
statement over the protocol's streaming opcode: the server answers with a
|
|
6
6
|
header (the column names), then row chunks, then an end marker, and the
|
|
7
|
-
driver yields rows as each chunk arrives, holding one chunk at a time.
|
|
7
|
+
driver yields rows as each chunk arrives, holding one chunk at a time. The
|
|
8
|
+
server executes a plain `SELECT` (bare columns from one table, unordered or
|
|
9
|
+
in storage-key order) page by page, and no scan budget applies to a
|
|
10
|
+
streamed page (server ≥ 0.330; `EXPLAIN` says whether a statement pages, and
|
|
11
|
+
the rules are under "Streamed queries" in the server's
|
|
12
|
+
[SQL syntax reference](https://skaidb.org/docs/query_syntax.html)).
|
|
8
13
|
|
|
9
14
|
```ruby
|
|
10
15
|
conn.stream("SELECT id, payload FROM events ORDER BY id") do |row|
|
|
@@ -12,7 +17,9 @@ conn.stream("SELECT id, payload FROM events ORDER BY id") do |row|
|
|
|
12
17
|
end
|
|
13
18
|
```
|
|
14
19
|
|
|
15
|
-
- `stream(sql, consistency: nil)`
|
|
20
|
+
- `stream(sql, params = [], consistency: nil)`: `params` bind `$1`, `$2`, …
|
|
21
|
+
exactly as `exec_params` binds them ([Parameters](#parameters), below);
|
|
22
|
+
`consistency:` overrides the connection's level for this statement.
|
|
16
23
|
- With a block it returns a `Skaidb::Result` after the last row: `kind` is
|
|
17
24
|
`:stream`, `fields` holds the column names (known even when no row
|
|
18
25
|
arrived), and `rows` is empty because the rows went to the block.
|
|
@@ -24,6 +31,34 @@ end
|
|
|
24
31
|
- A server without the streaming opcode raises `Skaidb::QueryError`
|
|
25
32
|
(`server does not support streaming: …`).
|
|
26
33
|
|
|
34
|
+
## Parameters
|
|
35
|
+
|
|
36
|
+
`params` bind the statement's `$1`, `$2`, … placeholders exactly as
|
|
37
|
+
`exec_params` binds them: typed, with every value [types.md](types.md) lists
|
|
38
|
+
(Arrays, Hashes, `Time`, `BigDecimal`, `Skaidb::Uuid`, …), the same `$N` as
|
|
39
|
+
often as you like, and the same checks before anything is sent (a
|
|
40
|
+
placeholder without a parameter, a parameter no placeholder uses, a bare
|
|
41
|
+
`?`).
|
|
42
|
+
|
|
43
|
+
```ruby
|
|
44
|
+
conn.stream("SELECT id, ts, body FROM events WHERE ts >= $1 AND ts < $2", [from, to]) do |row|
|
|
45
|
+
export(row)
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
conn.stream("SELECT id, v FROM t WHERE tenant = $1", [tenant], consistency: :one).each_slice(1000) do |batch|
|
|
49
|
+
load(batch)
|
|
50
|
+
end
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
They travel in `OP_QUERY_STREAM_PARAMS`, which needs a server ≥ 0.330. An
|
|
54
|
+
older server refuses it, and `stream` raises `Skaidb::QueryError`
|
|
55
|
+
(`server does not support parameterised streaming (skaidb >= 0.330): …`);
|
|
56
|
+
the connection stays usable. Unlike `exec_params` there is no client-side
|
|
57
|
+
text fallback. A stream without parameters (`nil` or `[]`) sends the
|
|
58
|
+
original `OP_QUERY_STREAM` with the SQL untouched, so it works against every
|
|
59
|
+
server. `params` must be an Array — anything else raises `ArgumentError`;
|
|
60
|
+
the consistency is only ever the `consistency:` keyword.
|
|
61
|
+
|
|
27
62
|
## Errors mid-stream
|
|
28
63
|
|
|
29
64
|
The server can fail a statement after it has started sending rows (a node
|
data/docs/types.md
CHANGED
|
@@ -1,8 +1,9 @@
|
|
|
1
1
|
# Types
|
|
2
2
|
|
|
3
3
|
Results decode from skaidb's typed wire values; parameters encode back to
|
|
4
|
-
them when a statement is prepared, which is the normal path whenever you
|
|
5
|
-
values to `exec_params` or `exec_batch
|
|
4
|
+
them — when a statement is prepared, which is the normal path whenever you
|
|
5
|
+
pass values to `exec_params` or `exec_batch`, and always for the values you
|
|
6
|
+
pass to `stream`.
|
|
6
7
|
|
|
7
8
|
| skaidb type | Result value | Accepted as a parameter |
|
|
8
9
|
|---|---|---|
|