skaidb 1.0.3 → 1.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +60 -0
- data/README.md +82 -15
- data/docs/api.md +47 -11
- data/docs/pooling.md +125 -9
- data/docs/streaming.md +5 -2
- data/docs/tls.md +31 -3
- data/lib/skaidb.rb +825 -97
- metadata +2 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 7a9f39f07d8b6d8abb6edf43d48b4c70da568b4a66f62044fb6f36d7b7b0e1ae
|
|
4
|
+
data.tar.gz: bb205511e213fe77efcfbaff5c17b82c453fcc2135cee1c362e154b9ad169e6f
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 9ce5348688c5bd6cd480006b2fa567ed7af74cc0ffe557f387d55705effd49dcbd754b8c52b32a30ae3361234226e4a82828bff7dc0d3d43f9a03e086849c53b
|
|
7
|
+
data.tar.gz: e7d6fd76c93392bbea0e2583b5741b418188c4d045d0226d7e8d0dcf2828fbe4cd03e34a32da58598d0173d81875cf8b1fe5b405c64e0ac658c108be67adf1da
|
data/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,64 @@ All notable changes to the skaidb Ruby driver. The format follows
|
|
|
4
4
|
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versions follow
|
|
5
5
|
[Semantic Versioning](https://semver.org/).
|
|
6
6
|
|
|
7
|
+
## [1.2.0] - 2026-10-25
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
- **Pool load balancing across members** (the rules every skaidb driver
|
|
11
|
+
shares, PROTOCOL.md §6.1): `load_balancing: :round_robin` /
|
|
12
|
+
`:least_latency` (power of two choices on the round-trip average ×
|
|
13
|
+
checked-out connections, re-measuring a member not used in the last 2 s;
|
|
14
|
+
`:random` stays the default), `idle_order: :fifo`, health ejection
|
|
15
|
+
(consecutive transport failures, or a latency average far above the other
|
|
16
|
+
members') with a background probe that re-admits the member,
|
|
17
|
+
`discover: true` (members from `SHOW ENDPOINTS`, server ≥ 0.327: joiners
|
|
18
|
+
added, leavers dropped, resyncing/unreachable members skipped and
|
|
19
|
+
re-checked every 5 s), and `read_your_writes: <seconds>` with
|
|
20
|
+
`checkout(affinity: key)` / `with(affinity: key)` — reads follow the member
|
|
21
|
+
that took the key's last write. `pool.members` reports each member's
|
|
22
|
+
state, latency and connections; `logger:` is told of ejections and
|
|
23
|
+
re-admissions. Options take Symbols or Strings.
|
|
24
|
+
- `Connection#endpoint`: the `"host:port"` of the member a connection reached.
|
|
25
|
+
|
|
26
|
+
### Changed
|
|
27
|
+
- A pooled connection that re-dials does so in the pool's order (usable
|
|
28
|
+
members first, the one it lost last) instead of shuffling the seeds.
|
|
29
|
+
- `Skaidb::Pool.new` checks its `Skaidb.connect` keywords when it is created
|
|
30
|
+
instead of at the first checkout, and a connection checked in to a pool
|
|
31
|
+
that did not create it is closed instead of kept.
|
|
32
|
+
|
|
33
|
+
## [1.1.0] - 2026-09-26
|
|
34
|
+
|
|
35
|
+
### Added
|
|
36
|
+
- Certificate login (wire mechanism EXTERNAL): `auth_mechanism: :certificate`
|
|
37
|
+
with `tls_client_cert:` / `tls_client_key:` (PEM files). AuthStart carries
|
|
38
|
+
the optional user, an empty client nonce and mechanism byte 2; the
|
|
39
|
+
outcome's signature is not verified, since TLS authenticated the server.
|
|
40
|
+
`user:` is optional under it and, when given, must equal the certificate's
|
|
41
|
+
Common Name. `Skaidb::AuthMechanism` resolves the option.
|
|
42
|
+
- `tls_client_cert:` / `tls_client_key:` also present a client certificate
|
|
43
|
+
under SCRAM, for servers that verify client certificates; either turns TLS
|
|
44
|
+
on, and they must be given together.
|
|
45
|
+
- The shared wire-protocol conformance suite: `conformance/vectors.json` and
|
|
46
|
+
its contract, vendored byte-identical, run by `test/conformance_test.rb`
|
|
47
|
+
(values, SCRAM, auth outcomes, every case through the public API; nothing
|
|
48
|
+
skipped). CI fails when the vendored vectors differ from
|
|
49
|
+
https://skaidb.org/conformance/vectors.json.
|
|
50
|
+
- `Result#kind` (`:rows`, `:mutation`, `:ddl`, `:stream`): a DDL
|
|
51
|
+
acknowledgement was indistinguishable from "0 rows affected".
|
|
52
|
+
- `Skaidb.salted_password` and `Skaidb.auth_message`, the SCRAM pieces the
|
|
53
|
+
handshake uses, so they can be checked against the reference vectors.
|
|
54
|
+
|
|
55
|
+
### Changed
|
|
56
|
+
- `Connection#stream` with a block returns a `Result` instead of `nil`. Found
|
|
57
|
+
by the conformance suite: a stream with no rows had no way to report its
|
|
58
|
+
column names, and a non-row statement sent over the streaming opcode
|
|
59
|
+
dropped its affected-row count. A row stream now returns `kind: :stream`
|
|
60
|
+
with the column names in `fields`; a non-row statement returns its
|
|
61
|
+
`:mutation` (with `cmd_tuples`) or `:ddl` Result.
|
|
62
|
+
- `Skaidb.connect`'s `user:` defaults to `nil`, which still means
|
|
63
|
+
`"anonymous"` under SCRAM and an empty user under certificate login.
|
|
64
|
+
|
|
7
65
|
## [1.0.3] - 2026-09-20
|
|
8
66
|
|
|
9
67
|
Release automation: published from GitHub Actions. No code change.
|
|
@@ -116,6 +174,8 @@ batches, a 2 500-row stream, error handling, and the `drivers` table showing
|
|
|
116
174
|
because it is a bundled gem since Ruby 3.4.
|
|
117
175
|
- The example moved to `examples/basic.rb`.
|
|
118
176
|
|
|
177
|
+
[1.2.0]: https://github.com/porcupin26/skaidb-ruby/compare/v1.1.0...v1.2.0
|
|
178
|
+
[1.1.0]: https://github.com/porcupin26/skaidb-ruby/compare/v1.0.3...v1.1.0
|
|
119
179
|
[1.0.3]: https://github.com/porcupin26/skaidb-ruby/compare/v1.0.2...v1.0.3
|
|
120
180
|
[1.0.2]: https://github.com/porcupin26/skaidb-ruby/compare/v1.0.1...v1.0.2
|
|
121
181
|
[1.0.1]: https://github.com/porcupin26/skaidb-ruby/compare/v1.0.0...v1.0.1
|
data/README.md
CHANGED
|
@@ -13,7 +13,8 @@ and the result behaves like `PG::Result`. **Pure standard library** (`socket`,
|
|
|
13
13
|
Ruby 2.7 through 3.4. It speaks skaidb's binary protocol directly:
|
|
14
14
|
SCRAM-SHA-256 authentication, server-side prepared statements with typed
|
|
15
15
|
parameters, one-round-trip batches, streamed result sets, multi-seed failover,
|
|
16
|
-
transparent reconnect, TLS and connection
|
|
16
|
+
transparent reconnect, TLS and a connection pool that balances across the
|
|
17
|
+
cluster's members.
|
|
17
18
|
|
|
18
19
|
- Repository: <https://github.com/porcupin26/skaidb-ruby>
|
|
19
20
|
- RubyGems: <https://rubygems.org/gems/skaidb>
|
|
@@ -54,13 +55,13 @@ Xcode command-line tools on macOS). `gem install skaidb` and vendoring reuse
|
|
|
54
55
|
the `bigdecimal` already installed.
|
|
55
56
|
|
|
56
57
|
Or vendor the single file: copy
|
|
57
|
-
[`lib/skaidb.rb`](https://github.com/porcupin26/skaidb-ruby/blob/v1.0
|
|
58
|
+
[`lib/skaidb.rb`](https://github.com/porcupin26/skaidb-ruby/blob/v1.2.0/lib/skaidb.rb)
|
|
58
59
|
into your project and `require_relative` it. It has no dependencies beyond
|
|
59
60
|
Ruby's standard library.
|
|
60
61
|
|
|
61
62
|
Every release is also attached as a `.gem` to its
|
|
62
63
|
[GitHub release](https://github.com/porcupin26/skaidb-ruby/releases) for
|
|
63
|
-
installs without registry access: `gem install ./skaidb-1.0.
|
|
64
|
+
installs without registry access: `gem install ./skaidb-1.2.0.gem`.
|
|
64
65
|
|
|
65
66
|
## Quick start
|
|
66
67
|
|
|
@@ -98,7 +99,8 @@ end
|
|
|
98
99
|
prepared statements, `exec_batch`
|
|
99
100
|
- [Results](#results)
|
|
100
101
|
- [Streaming large results](#streaming-large-results)
|
|
101
|
-
- [Connection pool](#connection-pool)
|
|
102
|
+
- [Connection pool](#connection-pool) — balancing across members,
|
|
103
|
+
health ejection, discovery, read-your-writes
|
|
102
104
|
- [Streams (`CREATE STREAM`)](#streams-create-stream) — `subscribe`
|
|
103
105
|
- [Type mapping](#type-mapping)
|
|
104
106
|
- [Errors](#errors)
|
|
@@ -117,7 +119,9 @@ Skaidb.connect(
|
|
|
117
119
|
timeout: 10.0, # TCP connect timeout in seconds; nil = none
|
|
118
120
|
database: nil, # run USE <database> as part of connecting
|
|
119
121
|
seeds: nil, # ["db1", "db2:7000", …] tried in random order
|
|
120
|
-
tls: false, tls_ca: nil, tls_insecure: false, tls_server_name: "skaidb"
|
|
122
|
+
tls: false, tls_ca: nil, tls_insecure: false, tls_server_name: "skaidb",
|
|
123
|
+
tls_client_cert: nil, tls_client_key: nil, # PEM files: present a client certificate
|
|
124
|
+
auth_mechanism: :scram # or :certificate: the client certificate is the login
|
|
121
125
|
) # => Skaidb::Connection (or the block's value when a block is given)
|
|
122
126
|
```
|
|
123
127
|
|
|
@@ -163,8 +167,19 @@ conn = Skaidb.connect(host: "db1", tls_insecure: true) # dev only: no
|
|
|
163
167
|
|
|
164
168
|
Any of the three enables TLS. The server name sent as SNI and checked against
|
|
165
169
|
the certificate is `tls_server_name` (default `"skaidb"`, the SAN skaidb's own
|
|
166
|
-
certificates carry). SCRAM runs inside the TLS session.
|
|
167
|
-
|
|
170
|
+
certificates carry). SCRAM runs inside the TLS session.
|
|
171
|
+
|
|
172
|
+
`tls_client_cert:` / `tls_client_key:` present a client certificate; with
|
|
173
|
+
`auth_mechanism: :certificate` that certificate is the login (its Common Name
|
|
174
|
+
is the user, no password is sent):
|
|
175
|
+
|
|
176
|
+
```ruby
|
|
177
|
+
conn = Skaidb.connect(host: "db1", tls_ca: "/etc/skaidb/ca.crt",
|
|
178
|
+
tls_client_cert: "app.crt", tls_client_key: "app.key",
|
|
179
|
+
auth_mechanism: :certificate)
|
|
180
|
+
```
|
|
181
|
+
|
|
182
|
+
Details: [docs/tls.md](https://github.com/porcupin26/skaidb-ruby/blob/main/docs/tls.md).
|
|
168
183
|
|
|
169
184
|
### Consistency
|
|
170
185
|
|
|
@@ -255,6 +270,8 @@ shaped like `PG::Result`:
|
|
|
255
270
|
cell by index or column name.
|
|
256
271
|
- `cmd_tuples`: rows affected by `INSERT`/`UPDATE`/`DELETE`; `0` for a
|
|
257
272
|
`SELECT` and for DDL.
|
|
273
|
+
- `kind`: `:rows`, `:mutation` (a count in `cmd_tuples`) or `:ddl` (a bare
|
|
274
|
+
acknowledgement), which tells "DDL succeeded" from "0 rows affected".
|
|
258
275
|
- `result_sets`: for a `CALL` whose procedure `EMIT`s several result sets,
|
|
259
276
|
every set in order (each a `Result`); the last set is the result's own
|
|
260
277
|
`rows`/`fields`. Empty for an ordinary reply.
|
|
@@ -274,8 +291,10 @@ end
|
|
|
274
291
|
first = conn.stream("SELECT id FROM big ORDER BY id").first(10) # Enumerator form
|
|
275
292
|
```
|
|
276
293
|
|
|
277
|
-
`stream` takes SQL text only (no parameters).
|
|
278
|
-
`
|
|
294
|
+
`stream` takes SQL text only (no parameters). With a block it returns a
|
|
295
|
+
`Result` whose `kind` is `:stream` and whose `fields` are the column names
|
|
296
|
+
(the rows went to the block); a non-row statement returns its `:mutation` /
|
|
297
|
+
`:ddl` Result without yielding.
|
|
279
298
|
|
|
280
299
|
**The abandon/drain rule.** The connection is busy for the whole stream:
|
|
281
300
|
a statement from another thread waits for it. Leaving the block early —
|
|
@@ -304,7 +323,29 @@ pool.close # closes idle connections
|
|
|
304
323
|
blocks — when no idle connection is available a new one is dialed, and a
|
|
305
324
|
returned connection beyond `maxsize` is closed. Connections are validated with
|
|
306
325
|
`usable?` on checkout and check-in, so one broken by a transport error or an
|
|
307
|
-
undrained stream is closed and replaced.
|
|
326
|
+
undrained stream is closed and replaced.
|
|
327
|
+
|
|
328
|
+
The pool spreads its connections across the members by the rules every skaidb
|
|
329
|
+
driver shares
|
|
330
|
+
([PROTOCOL.md §6.1](https://skaidb.org/docs/PROTOCOL.html#61-balancing-a-pool-across-members)):
|
|
331
|
+
|
|
332
|
+
```ruby
|
|
333
|
+
pool = Skaidb::Pool.new(seeds: ["db1", "db2", "db3"], database: "app",
|
|
334
|
+
load_balancing: :least_latency, # or :round_robin; :random is the default
|
|
335
|
+
discover: true, # members from SHOW ENDPOINTS, resyncing ones skipped
|
|
336
|
+
read_your_writes: 2.0) # seconds; see affinity below
|
|
337
|
+
|
|
338
|
+
pool.with(affinity: user_id) { |conn| conn.exec_params("INSERT INTO notes (id, owner) VALUES ($1, $2)", [nid, user_id]) }
|
|
339
|
+
pool.with(affinity: user_id) { |conn| conn.exec_params("SELECT * FROM notes WHERE id = $1", [nid]) } # same member
|
|
340
|
+
|
|
341
|
+
pool.members # => [{ endpoint: "db1:7000", canonical: "10.0.0.1:7000", state: :ready,
|
|
342
|
+
# latency_ms: 0.8, in_use: 2, idle: 3, seed: true }, …]
|
|
343
|
+
```
|
|
344
|
+
|
|
345
|
+
A member with consecutive transport failures, or a round-trip average far
|
|
346
|
+
above the others', is ejected and re-admitted by a background probe (the last
|
|
347
|
+
usable member never is); `idle_order: :fifo` keeps every idle connection in
|
|
348
|
+
use. Details: [docs/pooling.md](https://github.com/porcupin26/skaidb-ruby/blob/main/docs/pooling.md).
|
|
308
349
|
|
|
309
350
|
## Streams (`CREATE STREAM`)
|
|
310
351
|
|
|
@@ -376,8 +417,9 @@ StandardError
|
|
|
376
417
|
connection stays usable.
|
|
377
418
|
- A transport failure (`ConnectionError`) marks the connection broken; the
|
|
378
419
|
next statement re-dials, or a pool replaces it.
|
|
379
|
-
- `ArgumentError` for `Pool.new(maxsize: 0)`
|
|
380
|
-
|
|
420
|
+
- `ArgumentError` for `Pool.new(maxsize: 0)` or an unknown
|
|
421
|
+
`load_balancing`/`idle_order`, and for `Skaidb::Uuid.new` of a string that
|
|
422
|
+
is not a UUID.
|
|
381
423
|
|
|
382
424
|
## Transactions
|
|
383
425
|
|
|
@@ -404,7 +446,7 @@ frame after every handshake. It shows up as `client_name = 'ruby'` /
|
|
|
404
446
|
|
|
405
447
|
```sql
|
|
406
448
|
SELECT client_name, client_version FROM drivers;
|
|
407
|
-
-- ruby | 1.0
|
|
449
|
+
-- ruby | 1.2.0
|
|
408
450
|
```
|
|
409
451
|
|
|
410
452
|
The server records the row asynchronously, so a `SELECT` immediately after
|
|
@@ -417,13 +459,38 @@ know the opcode answers with an error the driver ignores.
|
|
|
417
459
|
`bigdecimal` is declared because it is a bundled gem since Ruby 3.4.
|
|
418
460
|
- Works with any skaidb server. Where a server predates an opcode the driver
|
|
419
461
|
falls back: prepared statements → client-side text binding, `Hello` →
|
|
420
|
-
ignored
|
|
421
|
-
|
|
462
|
+
ignored, pool discovery (`SHOW ENDPOINTS`, server ≥ 0.327) → the
|
|
463
|
+
configured seeds. `stream` and `exec_batch` need a server with those
|
|
464
|
+
opcodes and raise `QueryError` otherwise.
|
|
422
465
|
- The wire protocol (framing, SCRAM handshake, value encoding, opcodes) is
|
|
423
466
|
specified at <https://skaidb.org/docs/PROTOCOL.html>. The driver speaks the
|
|
424
467
|
binary protocol on port 7000; the server's REST/JSON gateway (port 7080)
|
|
425
468
|
is a dependency-free alternative for other clients.
|
|
426
469
|
|
|
470
|
+
## Conformance
|
|
471
|
+
|
|
472
|
+
`test/conformance_test.rb` runs skaidb's shared wire-protocol conformance
|
|
473
|
+
suite, vendored byte-identical from the server repository in
|
|
474
|
+
[`conformance/`](https://github.com/porcupin26/skaidb-ruby/tree/main/conformance)
|
|
475
|
+
(`vectors.json` plus the contract in its `README.md`), as part of `rake test`:
|
|
476
|
+
|
|
477
|
+
- every value vector decodes to the expected Ruby value and encodes back to
|
|
478
|
+
the reference bytes (a Uuid cell is compared as its canonical String, the
|
|
479
|
+
form the driver returns);
|
|
480
|
+
- every SCRAM vector (auth message, salted password, client proof, server
|
|
481
|
+
signature) through the driver's own SCRAM code;
|
|
482
|
+
- the auth outcomes against a fake server that verifies the client proof
|
|
483
|
+
independently: `ok` connects, a server signature that does not verify
|
|
484
|
+
fails the connect, `denied` fails with the server's reason;
|
|
485
|
+
- every case through the public API (`exec`, `stream`, `exec_params`,
|
|
486
|
+
`exec_batch`, and sequences of them on one connection), checking the exact
|
|
487
|
+
request bytes and the decoded result. The vectors' `?` placeholders are
|
|
488
|
+
passed as this driver's `$1`, `$2`, ….
|
|
489
|
+
|
|
490
|
+
Nothing is skipped: the test prints `skipped call methods: none`. CI also
|
|
491
|
+
fails when the vendored `vectors.json` differs from the published
|
|
492
|
+
<https://skaidb.org/conformance/vectors.json>.
|
|
493
|
+
|
|
427
494
|
## Examples
|
|
428
495
|
|
|
429
496
|
[`examples/`](https://github.com/porcupin26/skaidb-ruby/tree/main/examples)
|
data/docs/api.md
CHANGED
|
@@ -8,7 +8,7 @@ Everything lives under the `Skaidb` module in `lib/skaidb.rb`.
|
|
|
8
8
|
|---|---|---|
|
|
9
9
|
| `host` | `"localhost"` | Node to dial (ignored when `seeds` is given). |
|
|
10
10
|
| `port` | `7000` | Binary-protocol port; also the port for a seed without one. |
|
|
11
|
-
| `user` | `"anonymous"` |
|
|
11
|
+
| `user` | `"anonymous"` (SCRAM), `""` (certificate) | User name. Under `auth_mechanism: :certificate` it is optional and, when given, must equal the certificate's Common Name. |
|
|
12
12
|
| `password` | `""` | SCRAM password. Empty skips server-signature verification (anonymous connect). |
|
|
13
13
|
| `consistency` | `:quorum` | Default level for every statement: `:one`, `:quorum`, `:all`, `"ONE"`…, or `0`/`1`/`2`. |
|
|
14
14
|
| `timeout` | `10.0` | TCP connect timeout in seconds; `nil` for none. Reads are not bounded. |
|
|
@@ -18,12 +18,18 @@ Everything lives under the `Skaidb` module in `lib/skaidb.rb`.
|
|
|
18
18
|
| `tls_ca` | `nil` | PEM file to trust instead of the system store; implies `tls`. |
|
|
19
19
|
| `tls_insecure` | `false` | TLS without any certificate verification; implies `tls`. Development only. |
|
|
20
20
|
| `tls_server_name` | `"skaidb"` | SNI name, also checked against the certificate's SANs. |
|
|
21
|
+
| `tls_client_cert` | `nil` | PEM file of a client certificate to present in the TLS handshake; implies `tls`. Needs `tls_client_key`. |
|
|
22
|
+
| `tls_client_key` | `nil` | PEM file of that certificate's private key. |
|
|
23
|
+
| `auth_mechanism` | `:scram` | `:scram` (user + password) or `:certificate` (the client certificate is the login, wire mechanism EXTERNAL; needs `tls_client_cert`/`tls_client_key`). Strings and the aliases `"password"`, `"external"`, `"x509"` are accepted. |
|
|
21
24
|
|
|
22
25
|
With a block, yields the connection and closes it when the block returns
|
|
23
26
|
(the block's value is returned). Raises `Skaidb::ConnectionError` when no
|
|
24
27
|
endpoint accepts the handshake (`no reachable endpoint in …: <last error>`),
|
|
25
28
|
`authentication denied: …` on a wrong password, and
|
|
26
29
|
`server signature mismatch` when the server fails mutual authentication.
|
|
30
|
+
Raises `Skaidb::Error` before dialing for an unknown `auth_mechanism`, a
|
|
31
|
+
`tls_client_cert` without its `tls_client_key` (or the reverse), and
|
|
32
|
+
`:certificate` without a client certificate.
|
|
27
33
|
|
|
28
34
|
## `Skaidb::Connection`
|
|
29
35
|
|
|
@@ -49,8 +55,11 @@ endpoint accepts the handshake (`no reachable endpoint in …: <last error>`),
|
|
|
49
55
|
- `prepare_server(sql) → [id, nparams] | nil` — prepare a statement with `?`
|
|
50
56
|
placeholders on the server; `nil` when the server refuses. Cached per
|
|
51
57
|
connection (240 entries); the cache is cleared on re-dial.
|
|
52
|
-
- `stream(sql, consistency: nil) { |row| … } →
|
|
53
|
-
[streaming.md](streaming.md).
|
|
58
|
+
- `stream(sql, consistency: nil) { |row| … } → Result` — see
|
|
59
|
+
[streaming.md](streaming.md). For a row stream the returned Result has
|
|
60
|
+
`kind == :stream`, the column names in `fields` and no `rows` (they went to
|
|
61
|
+
the block); a statement without rows returns its `:mutation` / `:ddl`
|
|
62
|
+
Result without yielding. Without a block returns an Enumerator.
|
|
54
63
|
- `subscribe(stream_name, after: nil, poll: 0.5) { |event| … }` — yields a
|
|
55
64
|
`CREATE STREAM` log's events forever, each a Hash with `"id"`, `"op"`,
|
|
56
65
|
`"k"`, `"ts"`, `"doc"`. Polls `_stream_<name>` 500 rows at a time with a
|
|
@@ -64,6 +73,8 @@ endpoint accepts the handshake (`no reachable endpoint in …: <last error>`),
|
|
|
64
73
|
- `usable?` — `false` once closed, once a transport error broke the socket,
|
|
65
74
|
or while a stream is in flight (including one abandoned through external
|
|
66
75
|
iteration). No round-trip.
|
|
76
|
+
- `endpoint` — the member the connection reached, as the `"host:port"` it
|
|
77
|
+
dialed (it changes when a re-dial lands on another member).
|
|
67
78
|
- `closed`, `finished?` — `true` after `close`.
|
|
68
79
|
- `close` — closes the socket; idempotent. Does not take the connection's
|
|
69
80
|
lock, so another thread can use it to break a blocked read.
|
|
@@ -72,10 +83,11 @@ endpoint accepts the handshake (`no reachable endpoint in …: <last error>`),
|
|
|
72
83
|
|
|
73
84
|
A transport failure raises `Skaidb::ConnectionError` and marks the
|
|
74
85
|
connection broken. The next `exec`/`exec_params`/`exec_batch`/`stream`
|
|
75
|
-
re-dials across the original seeds
|
|
76
|
-
|
|
77
|
-
re-
|
|
78
|
-
|
|
86
|
+
re-dials across the original seeds (a pooled connection: in the pool's
|
|
87
|
+
order, usable members first and the one it lost last), re-authenticates,
|
|
88
|
+
re-sends `Hello`, re-runs `USE`, and starts with an empty prepared-statement
|
|
89
|
+
cache. If the re-dial fails, that statement raises `ConnectionError` and the
|
|
90
|
+
one after it tries again.
|
|
79
91
|
|
|
80
92
|
## `Skaidb::Result`
|
|
81
93
|
|
|
@@ -91,14 +103,38 @@ Shaped like `PG::Result`; `Enumerable` over row Hashes.
|
|
|
91
103
|
| `[i]` | Row `i` as a Hash, or `nil`. |
|
|
92
104
|
| `getvalue(row, col)` | One cell; `col` is an index or a column name. |
|
|
93
105
|
| `cmd_tuples` | Rows affected by `INSERT`/`UPDATE`/`DELETE`; `0` for `SELECT` and DDL. |
|
|
106
|
+
| `kind` | What the server answered: `:rows` (a result set, or several in `result_sets`), `:mutation` (a count in `cmd_tuples`), `:ddl` (a bare acknowledgement — no rows and no count), `:stream` (the summary `stream` returns). |
|
|
94
107
|
| `result_sets` | For a `CALL` whose body `EMIT`s, every result set in order (each a `Result`); the last one is this result's own `rows`/`fields`. `[]` otherwise. |
|
|
95
108
|
|
|
96
109
|
## `Skaidb::Pool`
|
|
97
110
|
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
111
|
+
```ruby
|
|
112
|
+
Skaidb::Pool.new(maxsize: 10, load_balancing: :random, idle_order: :lifo, discover: false,
|
|
113
|
+
refresh_interval: 60.0, read_your_writes: 0.0, eject_errors: 3,
|
|
114
|
+
eject_latency_factor: 3.0, eject_latency_floor: 0.05, eject_backoff: 5.0,
|
|
115
|
+
eject_backoff_max: 60.0, logger: nil, **connect_options)
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
A thread-safe pool balanced across the cluster's members — see
|
|
119
|
+
[pooling.md](pooling.md) for every option.
|
|
120
|
+
|
|
121
|
+
- `with(affinity: nil) { |conn| … }` — check out, yield, check in; returns
|
|
122
|
+
the block's value.
|
|
123
|
+
- `checkout(affinity: nil) → Connection` — an idle connection to the member
|
|
124
|
+
the policy picks, or a new one. Never blocks. `affinity` (any Hash key;
|
|
125
|
+
`nil` = pool-wide) scopes `read_your_writes`.
|
|
126
|
+
- `checkin(conn)` — return it; closed instead when broken, when its member
|
|
127
|
+
is ejected or gone, or when `maxsize` are already idle.
|
|
128
|
+
- `members → Array<Hash>` — per member: `:endpoint`, `:canonical`, `:state`
|
|
129
|
+
(`:ready`, `:ejected`, `:resyncing`, `:unreachable`, `:unlisted`),
|
|
130
|
+
`:latency_ms` (`nil` until measured), `:in_use`, `:idle`, `:seed`.
|
|
131
|
+
- `close` — closes the idle connections and stops the background thread.
|
|
132
|
+
|
|
133
|
+
`load_balancing` (`:random`, `:round_robin`, `:least_latency`) and
|
|
134
|
+
`idle_order` (`:lifo`, `:fifo`) take a Symbol or a String. Raises
|
|
135
|
+
`ArgumentError` for `maxsize < 1`, `eject_errors < 1` or an unknown
|
|
136
|
+
`load_balancing`/`idle_order`, and `Skaidb::Error` on checkout from a closed
|
|
137
|
+
pool.
|
|
102
138
|
|
|
103
139
|
## `Skaidb::Uuid`
|
|
104
140
|
|
data/docs/pooling.md
CHANGED
|
@@ -2,7 +2,9 @@
|
|
|
2
2
|
|
|
3
3
|
`Skaidb::Pool` keeps a bounded set of idle connections and hands one to each
|
|
4
4
|
caller, so a threaded server or job runner can use one connection per
|
|
5
|
-
thread without dialing and authenticating on every request.
|
|
5
|
+
thread without dialing and authenticating on every request. It spreads its
|
|
6
|
+
connections across the cluster's members by the rules every skaidb driver
|
|
7
|
+
shares.
|
|
6
8
|
|
|
7
9
|
```ruby
|
|
8
10
|
pool = Skaidb::Pool.new(seeds: ["db1", "db2", "db3"], user: "app", password: "secret",
|
|
@@ -22,11 +24,120 @@ end
|
|
|
22
24
|
pool.close # closes every idle connection
|
|
23
25
|
```
|
|
24
26
|
|
|
27
|
+
## API
|
|
28
|
+
|
|
29
|
+
```ruby
|
|
30
|
+
Skaidb::Pool.new(
|
|
31
|
+
maxsize: 10,
|
|
32
|
+
load_balancing: :random, # :random | :round_robin | :least_latency
|
|
33
|
+
idle_order: :lifo, # :lifo | :fifo
|
|
34
|
+
discover: false, # read the members from SHOW ENDPOINTS
|
|
35
|
+
refresh_interval: 60.0, # seconds between discovery refreshes
|
|
36
|
+
read_your_writes: 0.0, # seconds; 0 = off
|
|
37
|
+
eject_errors: 3, # consecutive transport failures
|
|
38
|
+
eject_latency_factor: 3.0, # x the other members' median; 0 = off
|
|
39
|
+
eject_latency_floor: 0.05, # seconds above the median, at least
|
|
40
|
+
eject_backoff: 5.0, # first ejection, doubling per repeat
|
|
41
|
+
eject_backoff_max: 60.0,
|
|
42
|
+
logger: nil, # anything with #warn: ejections, re-admissions
|
|
43
|
+
**connect_options # every Skaidb.connect keyword
|
|
44
|
+
)
|
|
45
|
+
|
|
46
|
+
pool.with(affinity: nil) { |conn| … } # the block's value
|
|
47
|
+
pool.checkout(affinity: nil) -> Skaidb::Connection
|
|
48
|
+
pool.checkin(conn)
|
|
49
|
+
pool.members -> Array of Hashes (see "Watching it")
|
|
50
|
+
pool.close
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
`load_balancing` and `idle_order` take a Symbol or a String. `host`/`port`/
|
|
54
|
+
`seeds` name the starting members; the other `Skaidb.connect` keywords
|
|
55
|
+
(`database`, TLS, `consistency`, `timeout`) apply to every connection, so
|
|
56
|
+
pooled connections run `USE` and authenticate exactly like single ones. A
|
|
57
|
+
pooled connection that re-dials after a transport error does so in the
|
|
58
|
+
pool's order: usable members first, the one it lost last.
|
|
59
|
+
`Connection#endpoint` names the member a connection reached.
|
|
60
|
+
|
|
61
|
+
## Balancing across members
|
|
62
|
+
|
|
63
|
+
With the default `load_balancing: :random` each new connection dials the
|
|
64
|
+
members in random order and stays on the member it reached — fine for many
|
|
65
|
+
clients, but one process can end up with most of its connections on one
|
|
66
|
+
member. Pick a policy to spread a single process's load:
|
|
67
|
+
|
|
68
|
+
```ruby
|
|
69
|
+
pool = Skaidb::Pool.new(
|
|
70
|
+
seeds: ["db1", "db2", "db3"], database: "app", user: "app", password: pw,
|
|
71
|
+
load_balancing: :least_latency, # or :round_robin
|
|
72
|
+
discover: true, # learn the members from the cluster
|
|
73
|
+
read_your_writes: 2.0, # seconds; see below
|
|
74
|
+
idle_order: :fifo
|
|
75
|
+
)
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
| Option | Default | What it does |
|
|
79
|
+
|---|---|---|
|
|
80
|
+
| `load_balancing` | `:random` | `:round_robin`: checkouts rotate over the usable members in endpoint order. `:least_latency`: of two members picked at random, the one with the lower round-trip average × (checked-out connections + 1), so near and idle members win; a member not measured yet, or not in the last 2 s, scores 0 and is tried, so a lightly loaded pool keeps re-measuring every member. |
|
|
81
|
+
| `idle_order` | `:lifo` | `:lifo` hands out the most recently returned connection first, so a quiet pool keeps few sockets warm. `:fifo` hands out the longest-idle one first, so every idle connection keeps being used and a server-side idle timeout never finds one. |
|
|
82
|
+
| `discover` | `false` | Reads the members with `SHOW ENDPOINTS` (server ≥ 0.327; no privilege needed) when a member is first reached and every `refresh_interval` (60 s) after — every 5 s while a member is `resyncing` or `unreachable`. New members are added, members that left are dropped (seeds are kept, marked `unlisted`), and members the server reports `resyncing` or `unreachable` get no new connections until they are `ready` again, when they start with a fresh latency average. An older or standalone server leaves the seeds as they are, and the pool stops asking. |
|
|
83
|
+
| `eject_errors` | `3` | Consecutive transport failures (a failed dial, an I/O error mid-request, a capacity refusal) that eject a member. Statement errors and refused logins do not count. |
|
|
84
|
+
| `eject_latency_factor`, `eject_latency_floor` | `3.0`, `0.05` | A member whose round-trip average is above factor × the other usable members' median **and** more than floor seconds above it is ejected (once it and another member have 10 samples each). A factor of `0` disables it. |
|
|
85
|
+
| `eject_backoff`, `eject_backoff_max` | `5.0`, `60.0` | How long an ejection lasts, doubling for each repeat up to the max; a member that stays healthy for `eject_backoff_max` after a re-admission starts over at the base. When the time is up a background thread dials the member and runs one statement: success re-admits it (with a fresh latency average), failure ejects it again. The last usable member is never ejected — a slow or flaky member beats none. |
|
|
86
|
+
| `read_your_writes` | `0.0` | Seconds after a write during which checkouts with the same `affinity` go to the member that took the write. |
|
|
87
|
+
| `logger` | `nil` | A `Logger` (or anything with `warn`) told of each ejection and re-admission. |
|
|
88
|
+
|
|
89
|
+
An ejected member gets no new connections and its idle ones are closed; a
|
|
90
|
+
connection to it that is returned to the pool is closed instead of kept.
|
|
91
|
+
With a target member (round-robin, least-latency, an affinity) the pool
|
|
92
|
+
hands out an idle connection to that member or dials it; a failed dial falls
|
|
93
|
+
through to the other usable members, then to the unusable ones, so a
|
|
94
|
+
checkout never has nowhere to go while any member is configured.
|
|
95
|
+
|
|
96
|
+
**Latency.** Per member the pool keeps an exponentially weighted average of
|
|
97
|
+
round trips (request sent to the first response frame): each sample moves it
|
|
98
|
+
5 % of the way, and no sample counts for more than 10 × the larger of the
|
|
99
|
+
current average and `eject_latency_floor`. One slow statement barely moves
|
|
100
|
+
it; a member that stays slow is noticed within a few dozen statements.
|
|
101
|
+
|
|
102
|
+
**Read-your-writes.** The member that coordinated a write reads it back at
|
|
103
|
+
any consistency, `ONE` included; another member may not have applied it yet.
|
|
104
|
+
With `read_your_writes` set, pass an affinity key — a user or session id —
|
|
105
|
+
when you check out:
|
|
106
|
+
|
|
107
|
+
```ruby
|
|
108
|
+
pool.with(affinity: user_id) do |conn|
|
|
109
|
+
conn.exec_params("INSERT INTO notes (id, owner, body) VALUES ($1, $2, $3)", [nid, user_id, body])
|
|
110
|
+
end
|
|
111
|
+
# … the next request for the same user, within the window:
|
|
112
|
+
pool.with(affinity: user_id) do |conn| # same member
|
|
113
|
+
conn.exec_params("SELECT body FROM notes WHERE id = $1", [nid])
|
|
114
|
+
end
|
|
115
|
+
```
|
|
116
|
+
|
|
117
|
+
The window starts when a write succeeds on a connection checked out under
|
|
118
|
+
the key, and checkouts under it go to that member until the window ends
|
|
119
|
+
(unless the member is ejected or not `ready`). Checkouts without a key
|
|
120
|
+
share one pool-wide key, so with constant writes everything follows the
|
|
121
|
+
latest writer — pass a key when you balance. A statement is a write unless
|
|
122
|
+
its first keyword (after whitespace, comments and opening parentheses) is
|
|
123
|
+
`SELECT`, `SHOW`, `EXPLAIN`, `DESCRIBE`, `DESC`, `VALUES`, `WITH` or `USE`.
|
|
124
|
+
|
|
125
|
+
**Watching it.** `pool.members` returns one Hash per member, sorted by
|
|
126
|
+
endpoint:
|
|
127
|
+
|
|
128
|
+
```ruby
|
|
129
|
+
[{ endpoint: "db1:7000", canonical: "10.0.0.1:7000", state: :ready,
|
|
130
|
+
latency_ms: 0.8, in_use: 2, idle: 3, seed: true }, …]
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
`endpoint` is what the pool dials, `canonical` the name the member reports
|
|
134
|
+
for itself (once discovered), `state` one of `:ready`, `:ejected`,
|
|
135
|
+
`:resyncing`, `:unreachable`, `:unlisted`, and `latency_ms` is `nil` until
|
|
136
|
+
the member has been measured. The rules are the same in every skaidb driver
|
|
137
|
+
([PROTOCOL.md §6.1](https://skaidb.org/docs/PROTOCOL.html#61-balancing-a-pool-across-members)).
|
|
138
|
+
|
|
25
139
|
## Behaviour
|
|
26
140
|
|
|
27
|
-
- Every `Skaidb.connect` keyword passes through (`seeds`, `database`, TLS,
|
|
28
|
-
`consistency`, `timeout`), so pooled connections fail over across seeds
|
|
29
|
-
and run `USE` exactly like single ones.
|
|
30
141
|
- `maxsize` (default 10) bounds the connections kept **idle**, not the number
|
|
31
142
|
checked out. `checkout` never blocks: with no idle connection available it
|
|
32
143
|
dials a new one, and a connection returned while `maxsize` are already idle
|
|
@@ -38,10 +149,15 @@ pool.close # closes every idle connection
|
|
|
38
149
|
and will fail its first statement with `Skaidb::ConnectionError`; that
|
|
39
150
|
failure marks it broken, so `checkin` discards it and the next checkout
|
|
40
151
|
gets a fresh one. Retry the statement in that case if it is safe to.
|
|
41
|
-
- `close` marks the pool closed
|
|
42
|
-
checked out at the time are closed when
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
152
|
+
- `close` marks the pool closed, closes the idle connections and stops the
|
|
153
|
+
background thread; connections checked out at the time are closed when
|
|
154
|
+
they are returned. `checkout` on a closed pool raises `Skaidb::Error`.
|
|
155
|
+
`maxsize < 1`, `eject_errors < 1` and an unknown `load_balancing` or
|
|
156
|
+
`idle_order` raise `ArgumentError`.
|
|
157
|
+
- The pool is thread-safe: the idle list and the member table are guarded
|
|
158
|
+
by one mutex. The background thread (probes and discovery refreshes) is
|
|
159
|
+
started on first need; Ruby never waits for it at exit. A single
|
|
160
|
+
connection should still be used by one thread at a time; the pool is what
|
|
161
|
+
gives each thread its own.
|
|
46
162
|
|
|
47
163
|
See also `examples/pool.rb`.
|
data/docs/streaming.md
CHANGED
|
@@ -13,11 +13,14 @@ end
|
|
|
13
13
|
```
|
|
14
14
|
|
|
15
15
|
- `stream(sql, consistency: nil)` takes SQL text only — no parameters.
|
|
16
|
-
- With a block it returns `
|
|
16
|
+
- With a block it returns a `Skaidb::Result` after the last row: `kind` is
|
|
17
|
+
`:stream`, `fields` holds the column names (known even when no row
|
|
18
|
+
arrived), and `rows` is empty because the rows went to the block.
|
|
17
19
|
- Without a block it returns an Enumerator: `conn.stream(sql).first(10)`,
|
|
18
20
|
`.each_slice(1000)`, `.lazy.map { … }`, and the rest of Enumerable work.
|
|
19
21
|
- A non-row statement (an `INSERT`, DDL, `USE`) answers with its ordinary
|
|
20
|
-
result over the same opcode; `stream` returns `
|
|
22
|
+
result over the same opcode; `stream` returns that Result (`kind`
|
|
23
|
+
`:mutation` with `cmd_tuples`, or `:ddl`) without yielding.
|
|
21
24
|
- A server without the streaming opcode raises `Skaidb::QueryError`
|
|
22
25
|
(`server does not support streaming: …`).
|
|
23
26
|
|
data/docs/tls.md
CHANGED
|
@@ -42,16 +42,44 @@ OpenSSL's message.
|
|
|
42
42
|
|
|
43
43
|
## What runs inside the session
|
|
44
44
|
|
|
45
|
-
The TCP connection is upgraded to TLS *before* the
|
|
45
|
+
The TCP connection is upgraded to TLS *before* the login handshake, so the
|
|
46
46
|
user name, nonces and proof all travel encrypted; with `seeds:`, each
|
|
47
|
-
endpoint is upgraded and authenticated in turn until one succeeds.
|
|
48
|
-
|
|
47
|
+
endpoint is upgraded and authenticated in turn until one succeeds.
|
|
48
|
+
|
|
49
|
+
## Client certificates
|
|
50
|
+
|
|
51
|
+
`tls_client_cert:` / `tls_client_key:` (PEM files) make the driver present a
|
|
52
|
+
client certificate in the TLS handshake, which a server with
|
|
53
|
+
`client_tls = "required"` and client verification needs. Either one turns TLS
|
|
54
|
+
on, and they go together. On its own the certificate only opens the TLS
|
|
55
|
+
session; the login is still the SCRAM user.
|
|
56
|
+
|
|
57
|
+
With `auth_mechanism: :certificate` the certificate IS the login (wire
|
|
58
|
+
mechanism EXTERNAL): the server maps its Common Name to a role and no
|
|
59
|
+
password is sent. The server needs `auth.x509_enabled`, and its client CA
|
|
60
|
+
must have signed the certificate.
|
|
61
|
+
|
|
62
|
+
```ruby
|
|
63
|
+
conn = Skaidb.connect(host: "db1", tls_ca: "/etc/skaidb/ca.crt",
|
|
64
|
+
tls_client_cert: "/etc/app/app.crt",
|
|
65
|
+
tls_client_key: "/etc/app/app.key",
|
|
66
|
+
auth_mechanism: :certificate, database: "app")
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
Pass `user:` only to assert the expected identity: a certificate mapped to a
|
|
70
|
+
different role fails the connect. The server's answer carries no signature
|
|
71
|
+
to check — TLS already authenticated the server, so keep `tls_ca:` (or the
|
|
72
|
+
system store) and do not combine a certificate login with `tls_insecure`.
|
|
73
|
+
`Skaidb::Pool` passes these keywords through like the others.
|
|
49
74
|
|
|
50
75
|
## Failures
|
|
51
76
|
|
|
52
77
|
- Wrong CA, expired certificate, name mismatch: `Skaidb::ConnectionError`
|
|
53
78
|
with the OpenSSL message, from `connect` (or from the statement that
|
|
54
79
|
triggered a re-dial).
|
|
80
|
+
- A certificate login the server refuses (no role for the CN, `user:` not
|
|
81
|
+
matching it, x509 login disabled): `Skaidb::ConnectionError` with
|
|
82
|
+
`authentication denied: <reason>`.
|
|
55
83
|
- Plain TCP against a `client_tls = required` server: the handshake cannot
|
|
56
84
|
complete, so `connect` raises `Skaidb::ConnectionError`
|
|
57
85
|
(`no reachable endpoint in …`) naming the underlying failure.
|