skaidb 1.1.0 → 1.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +28 -0
- data/README.md +36 -10
- data/docs/api.md +34 -8
- data/docs/pooling.md +125 -9
- data/lib/skaidb.rb +699 -88
- metadata +2 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 7a9f39f07d8b6d8abb6edf43d48b4c70da568b4a66f62044fb6f36d7b7b0e1ae
|
|
4
|
+
data.tar.gz: bb205511e213fe77efcfbaff5c17b82c453fcc2135cee1c362e154b9ad169e6f
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 9ce5348688c5bd6cd480006b2fa567ed7af74cc0ffe557f387d55705effd49dcbd754b8c52b32a30ae3361234226e4a82828bff7dc0d3d43f9a03e086849c53b
|
|
7
|
+
data.tar.gz: e7d6fd76c93392bbea0e2583b5741b418188c4d045d0226d7e8d0dcf2828fbe4cd03e34a32da58598d0173d81875cf8b1fe5b405c64e0ac658c108be67adf1da
|
data/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,32 @@ All notable changes to the skaidb Ruby driver. The format follows
|
|
|
4
4
|
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versions follow
|
|
5
5
|
[Semantic Versioning](https://semver.org/).
|
|
6
6
|
|
|
7
|
+
## [1.2.0] - 2026-10-25
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
- **Pool load balancing across members** (the rules every skaidb driver
|
|
11
|
+
shares, PROTOCOL.md §6.1): `load_balancing: :round_robin` /
|
|
12
|
+
`:least_latency` (power of two choices on the round-trip average ×
|
|
13
|
+
checked-out connections, re-measuring a member not used in the last 2 s;
|
|
14
|
+
`:random` stays the default), `idle_order: :fifo`, health ejection
|
|
15
|
+
(consecutive transport failures, or a latency average far above the other
|
|
16
|
+
members') with a background probe that re-admits the member,
|
|
17
|
+
`discover: true` (members from `SHOW ENDPOINTS`, server ≥ 0.327: joiners
|
|
18
|
+
added, leavers dropped, resyncing/unreachable members skipped and
|
|
19
|
+
re-checked every 5 s), and `read_your_writes: <seconds>` with
|
|
20
|
+
`checkout(affinity: key)` / `with(affinity: key)` — reads follow the member
|
|
21
|
+
that took the key's last write. `pool.members` reports each member's
|
|
22
|
+
state, latency and connections; `logger:` is told of ejections and
|
|
23
|
+
re-admissions. Options take Symbols or Strings.
|
|
24
|
+
- `Connection#endpoint`: the `"host:port"` of the member a connection reached.
|
|
25
|
+
|
|
26
|
+
### Changed
|
|
27
|
+
- A pooled connection that re-dials does so in the pool's order (usable
|
|
28
|
+
members first, the one it lost last) instead of shuffling the seeds.
|
|
29
|
+
- `Skaidb::Pool.new` checks its `Skaidb.connect` keywords when it is created
|
|
30
|
+
instead of at the first checkout, and a connection checked in to a pool
|
|
31
|
+
that did not create it is closed instead of kept.
|
|
32
|
+
|
|
7
33
|
## [1.1.0] - 2026-09-26
|
|
8
34
|
|
|
9
35
|
### Added
|
|
@@ -148,6 +174,8 @@ batches, a 2 500-row stream, error handling, and the `drivers` table showing
|
|
|
148
174
|
because it is a bundled gem since Ruby 3.4.
|
|
149
175
|
- The example moved to `examples/basic.rb`.
|
|
150
176
|
|
|
177
|
+
[1.2.0]: https://github.com/porcupin26/skaidb-ruby/compare/v1.1.0...v1.2.0
|
|
178
|
+
[1.1.0]: https://github.com/porcupin26/skaidb-ruby/compare/v1.0.3...v1.1.0
|
|
151
179
|
[1.0.3]: https://github.com/porcupin26/skaidb-ruby/compare/v1.0.2...v1.0.3
|
|
152
180
|
[1.0.2]: https://github.com/porcupin26/skaidb-ruby/compare/v1.0.1...v1.0.2
|
|
153
181
|
[1.0.1]: https://github.com/porcupin26/skaidb-ruby/compare/v1.0.0...v1.0.1
|
data/README.md
CHANGED
|
@@ -13,7 +13,8 @@ and the result behaves like `PG::Result`. **Pure standard library** (`socket`,
|
|
|
13
13
|
Ruby 2.7 through 3.4. It speaks skaidb's binary protocol directly:
|
|
14
14
|
SCRAM-SHA-256 authentication, server-side prepared statements with typed
|
|
15
15
|
parameters, one-round-trip batches, streamed result sets, multi-seed failover,
|
|
16
|
-
transparent reconnect, TLS and connection
|
|
16
|
+
transparent reconnect, TLS and a connection pool that balances across the
|
|
17
|
+
cluster's members.
|
|
17
18
|
|
|
18
19
|
- Repository: <https://github.com/porcupin26/skaidb-ruby>
|
|
19
20
|
- RubyGems: <https://rubygems.org/gems/skaidb>
|
|
@@ -54,13 +55,13 @@ Xcode command-line tools on macOS). `gem install skaidb` and vendoring reuse
|
|
|
54
55
|
the `bigdecimal` already installed.
|
|
55
56
|
|
|
56
57
|
Or vendor the single file: copy
|
|
57
|
-
[`lib/skaidb.rb`](https://github.com/porcupin26/skaidb-ruby/blob/v1.
|
|
58
|
+
[`lib/skaidb.rb`](https://github.com/porcupin26/skaidb-ruby/blob/v1.2.0/lib/skaidb.rb)
|
|
58
59
|
into your project and `require_relative` it. It has no dependencies beyond
|
|
59
60
|
Ruby's standard library.
|
|
60
61
|
|
|
61
62
|
Every release is also attached as a `.gem` to its
|
|
62
63
|
[GitHub release](https://github.com/porcupin26/skaidb-ruby/releases) for
|
|
63
|
-
installs without registry access: `gem install ./skaidb-1.
|
|
64
|
+
installs without registry access: `gem install ./skaidb-1.2.0.gem`.
|
|
64
65
|
|
|
65
66
|
## Quick start
|
|
66
67
|
|
|
@@ -98,7 +99,8 @@ end
|
|
|
98
99
|
prepared statements, `exec_batch`
|
|
99
100
|
- [Results](#results)
|
|
100
101
|
- [Streaming large results](#streaming-large-results)
|
|
101
|
-
- [Connection pool](#connection-pool)
|
|
102
|
+
- [Connection pool](#connection-pool) — balancing across members,
|
|
103
|
+
health ejection, discovery, read-your-writes
|
|
102
104
|
- [Streams (`CREATE STREAM`)](#streams-create-stream) — `subscribe`
|
|
103
105
|
- [Type mapping](#type-mapping)
|
|
104
106
|
- [Errors](#errors)
|
|
@@ -321,7 +323,29 @@ pool.close # closes idle connections
|
|
|
321
323
|
blocks — when no idle connection is available a new one is dialed, and a
|
|
322
324
|
returned connection beyond `maxsize` is closed. Connections are validated with
|
|
323
325
|
`usable?` on checkout and check-in, so one broken by a transport error or an
|
|
324
|
-
undrained stream is closed and replaced.
|
|
326
|
+
undrained stream is closed and replaced.
|
|
327
|
+
|
|
328
|
+
The pool spreads its connections across the members by the rules every skaidb
|
|
329
|
+
driver shares
|
|
330
|
+
([PROTOCOL.md §6.1](https://skaidb.org/docs/PROTOCOL.html#61-balancing-a-pool-across-members)):
|
|
331
|
+
|
|
332
|
+
```ruby
|
|
333
|
+
pool = Skaidb::Pool.new(seeds: ["db1", "db2", "db3"], database: "app",
|
|
334
|
+
load_balancing: :least_latency, # or :round_robin; :random is the default
|
|
335
|
+
discover: true, # members from SHOW ENDPOINTS, resyncing ones skipped
|
|
336
|
+
read_your_writes: 2.0) # seconds; see affinity below
|
|
337
|
+
|
|
338
|
+
pool.with(affinity: user_id) { |conn| conn.exec_params("INSERT INTO notes (id, owner) VALUES ($1, $2)", [nid, user_id]) }
|
|
339
|
+
pool.with(affinity: user_id) { |conn| conn.exec_params("SELECT * FROM notes WHERE id = $1", [nid]) } # same member
|
|
340
|
+
|
|
341
|
+
pool.members # => [{ endpoint: "db1:7000", canonical: "10.0.0.1:7000", state: :ready,
|
|
342
|
+
# latency_ms: 0.8, in_use: 2, idle: 3, seed: true }, …]
|
|
343
|
+
```
|
|
344
|
+
|
|
345
|
+
A member with consecutive transport failures, or a round-trip average far
|
|
346
|
+
above the others', is ejected and re-admitted by a background probe (the last
|
|
347
|
+
usable member never is); `idle_order: :fifo` keeps every idle connection in
|
|
348
|
+
use. Details: [docs/pooling.md](https://github.com/porcupin26/skaidb-ruby/blob/main/docs/pooling.md).
|
|
325
349
|
|
|
326
350
|
## Streams (`CREATE STREAM`)
|
|
327
351
|
|
|
@@ -393,8 +417,9 @@ StandardError
|
|
|
393
417
|
connection stays usable.
|
|
394
418
|
- A transport failure (`ConnectionError`) marks the connection broken; the
|
|
395
419
|
next statement re-dials, or a pool replaces it.
|
|
396
|
-
- `ArgumentError` for `Pool.new(maxsize: 0)`
|
|
397
|
-
|
|
420
|
+
- `ArgumentError` for `Pool.new(maxsize: 0)` or an unknown
|
|
421
|
+
`load_balancing`/`idle_order`, and for `Skaidb::Uuid.new` of a string that
|
|
422
|
+
is not a UUID.
|
|
398
423
|
|
|
399
424
|
## Transactions
|
|
400
425
|
|
|
@@ -421,7 +446,7 @@ frame after every handshake. It shows up as `client_name = 'ruby'` /
|
|
|
421
446
|
|
|
422
447
|
```sql
|
|
423
448
|
SELECT client_name, client_version FROM drivers;
|
|
424
|
-
-- ruby | 1.
|
|
449
|
+
-- ruby | 1.2.0
|
|
425
450
|
```
|
|
426
451
|
|
|
427
452
|
The server records the row asynchronously, so a `SELECT` immediately after
|
|
@@ -434,8 +459,9 @@ know the opcode answers with an error the driver ignores.
|
|
|
434
459
|
`bigdecimal` is declared because it is a bundled gem since Ruby 3.4.
|
|
435
460
|
- Works with any skaidb server. Where a server predates an opcode the driver
|
|
436
461
|
falls back: prepared statements → client-side text binding, `Hello` →
|
|
437
|
-
ignored
|
|
438
|
-
|
|
462
|
+
ignored, pool discovery (`SHOW ENDPOINTS`, server ≥ 0.327) → the
|
|
463
|
+
configured seeds. `stream` and `exec_batch` need a server with those
|
|
464
|
+
opcodes and raise `QueryError` otherwise.
|
|
439
465
|
- The wire protocol (framing, SCRAM handshake, value encoding, opcodes) is
|
|
440
466
|
specified at <https://skaidb.org/docs/PROTOCOL.html>. The driver speaks the
|
|
441
467
|
binary protocol on port 7000; the server's REST/JSON gateway (port 7080)
|
data/docs/api.md
CHANGED
|
@@ -73,6 +73,8 @@ Raises `Skaidb::Error` before dialing for an unknown `auth_mechanism`, a
|
|
|
73
73
|
- `usable?` — `false` once closed, once a transport error broke the socket,
|
|
74
74
|
or while a stream is in flight (including one abandoned through external
|
|
75
75
|
iteration). No round-trip.
|
|
76
|
+
- `endpoint` — the member the connection reached, as the `"host:port"` it
|
|
77
|
+
dialed (it changes when a re-dial lands on another member).
|
|
76
78
|
- `closed`, `finished?` — `true` after `close`.
|
|
77
79
|
- `close` — closes the socket; idempotent. Does not take the connection's
|
|
78
80
|
lock, so another thread can use it to break a blocked read.
|
|
@@ -81,10 +83,11 @@ Raises `Skaidb::Error` before dialing for an unknown `auth_mechanism`, a
|
|
|
81
83
|
|
|
82
84
|
A transport failure raises `Skaidb::ConnectionError` and marks the
|
|
83
85
|
connection broken. The next `exec`/`exec_params`/`exec_batch`/`stream`
|
|
84
|
-
re-dials across the original seeds
|
|
85
|
-
|
|
86
|
-
re-
|
|
87
|
-
|
|
86
|
+
re-dials across the original seeds (a pooled connection: in the pool's
|
|
87
|
+
order, usable members first and the one it lost last), re-authenticates,
|
|
88
|
+
re-sends `Hello`, re-runs `USE`, and starts with an empty prepared-statement
|
|
89
|
+
cache. If the re-dial fails, that statement raises `ConnectionError` and the
|
|
90
|
+
one after it tries again.
|
|
88
91
|
|
|
89
92
|
## `Skaidb::Result`
|
|
90
93
|
|
|
@@ -105,10 +108,33 @@ Shaped like `PG::Result`; `Enumerable` over row Hashes.
|
|
|
105
108
|
|
|
106
109
|
## `Skaidb::Pool`
|
|
107
110
|
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
111
|
+
```ruby
|
|
112
|
+
Skaidb::Pool.new(maxsize: 10, load_balancing: :random, idle_order: :lifo, discover: false,
|
|
113
|
+
refresh_interval: 60.0, read_your_writes: 0.0, eject_errors: 3,
|
|
114
|
+
eject_latency_factor: 3.0, eject_latency_floor: 0.05, eject_backoff: 5.0,
|
|
115
|
+
eject_backoff_max: 60.0, logger: nil, **connect_options)
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
A thread-safe pool balanced across the cluster's members — see
|
|
119
|
+
[pooling.md](pooling.md) for every option.
|
|
120
|
+
|
|
121
|
+
- `with(affinity: nil) { |conn| … }` — check out, yield, check in; returns
|
|
122
|
+
the block's value.
|
|
123
|
+
- `checkout(affinity: nil) → Connection` — an idle connection to the member
|
|
124
|
+
the policy picks, or a new one. Never blocks. `affinity` (any Hash key;
|
|
125
|
+
`nil` = pool-wide) scopes `read_your_writes`.
|
|
126
|
+
- `checkin(conn)` — return it; closed instead when broken, when its member
|
|
127
|
+
is ejected or gone, or when `maxsize` are already idle.
|
|
128
|
+
- `members → Array<Hash>` — per member: `:endpoint`, `:canonical`, `:state`
|
|
129
|
+
(`:ready`, `:ejected`, `:resyncing`, `:unreachable`, `:unlisted`),
|
|
130
|
+
`:latency_ms` (`nil` until measured), `:in_use`, `:idle`, `:seed`.
|
|
131
|
+
- `close` — closes the idle connections and stops the background thread.
|
|
132
|
+
|
|
133
|
+
`load_balancing` (`:random`, `:round_robin`, `:least_latency`) and
|
|
134
|
+
`idle_order` (`:lifo`, `:fifo`) take a Symbol or a String. Raises
|
|
135
|
+
`ArgumentError` for `maxsize < 1`, `eject_errors < 1` or an unknown
|
|
136
|
+
`load_balancing`/`idle_order`, and `Skaidb::Error` on checkout from a closed
|
|
137
|
+
pool.
|
|
112
138
|
|
|
113
139
|
## `Skaidb::Uuid`
|
|
114
140
|
|
data/docs/pooling.md
CHANGED
|
@@ -2,7 +2,9 @@
|
|
|
2
2
|
|
|
3
3
|
`Skaidb::Pool` keeps a bounded set of idle connections and hands one to each
|
|
4
4
|
caller, so a threaded server or job runner can use one connection per
|
|
5
|
-
thread without dialing and authenticating on every request.
|
|
5
|
+
thread without dialing and authenticating on every request. It spreads its
|
|
6
|
+
connections across the cluster's members by the rules every skaidb driver
|
|
7
|
+
shares.
|
|
6
8
|
|
|
7
9
|
```ruby
|
|
8
10
|
pool = Skaidb::Pool.new(seeds: ["db1", "db2", "db3"], user: "app", password: "secret",
|
|
@@ -22,11 +24,120 @@ end
|
|
|
22
24
|
pool.close # closes every idle connection
|
|
23
25
|
```
|
|
24
26
|
|
|
27
|
+
## API
|
|
28
|
+
|
|
29
|
+
```ruby
|
|
30
|
+
Skaidb::Pool.new(
|
|
31
|
+
maxsize: 10,
|
|
32
|
+
load_balancing: :random, # :random | :round_robin | :least_latency
|
|
33
|
+
idle_order: :lifo, # :lifo | :fifo
|
|
34
|
+
discover: false, # read the members from SHOW ENDPOINTS
|
|
35
|
+
refresh_interval: 60.0, # seconds between discovery refreshes
|
|
36
|
+
read_your_writes: 0.0, # seconds; 0 = off
|
|
37
|
+
eject_errors: 3, # consecutive transport failures
|
|
38
|
+
eject_latency_factor: 3.0, # x the other members' median; 0 = off
|
|
39
|
+
eject_latency_floor: 0.05, # seconds above the median, at least
|
|
40
|
+
eject_backoff: 5.0, # first ejection, doubling per repeat
|
|
41
|
+
eject_backoff_max: 60.0,
|
|
42
|
+
logger: nil, # anything with #warn: ejections, re-admissions
|
|
43
|
+
**connect_options # every Skaidb.connect keyword
|
|
44
|
+
)
|
|
45
|
+
|
|
46
|
+
pool.with(affinity: nil) { |conn| … } # the block's value
|
|
47
|
+
pool.checkout(affinity: nil) -> Skaidb::Connection
|
|
48
|
+
pool.checkin(conn)
|
|
49
|
+
pool.members -> Array of Hashes (see "Watching it")
|
|
50
|
+
pool.close
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
`load_balancing` and `idle_order` take a Symbol or a String. `host`/`port`/
|
|
54
|
+
`seeds` name the starting members; the other `Skaidb.connect` keywords
|
|
55
|
+
(`database`, TLS, `consistency`, `timeout`) apply to every connection, so
|
|
56
|
+
pooled connections run `USE` and authenticate exactly like single ones. A
|
|
57
|
+
pooled connection that re-dials after a transport error does so in the
|
|
58
|
+
pool's order: usable members first, the one it lost last.
|
|
59
|
+
`Connection#endpoint` names the member a connection reached.
|
|
60
|
+
|
|
61
|
+
## Balancing across members
|
|
62
|
+
|
|
63
|
+
With the default `load_balancing: :random` each new connection dials the
|
|
64
|
+
members in random order and stays on the member it reached — fine for many
|
|
65
|
+
clients, but one process can end up with most of its connections on one
|
|
66
|
+
member. Pick a policy to spread a single process's load:
|
|
67
|
+
|
|
68
|
+
```ruby
|
|
69
|
+
pool = Skaidb::Pool.new(
|
|
70
|
+
seeds: ["db1", "db2", "db3"], database: "app", user: "app", password: pw,
|
|
71
|
+
load_balancing: :least_latency, # or :round_robin
|
|
72
|
+
discover: true, # learn the members from the cluster
|
|
73
|
+
read_your_writes: 2.0, # seconds; see below
|
|
74
|
+
idle_order: :fifo
|
|
75
|
+
)
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
| Option | Default | What it does |
|
|
79
|
+
|---|---|---|
|
|
80
|
+
| `load_balancing` | `:random` | `:round_robin`: checkouts rotate over the usable members in endpoint order. `:least_latency`: of two members picked at random, the one with the lower round-trip average × (checked-out connections + 1), so near and idle members win; a member not measured yet, or not in the last 2 s, scores 0 and is tried, so a lightly loaded pool keeps re-measuring every member. |
|
|
81
|
+
| `idle_order` | `:lifo` | `:lifo` hands out the most recently returned connection first, so a quiet pool keeps few sockets warm. `:fifo` hands out the longest-idle one first, so every idle connection keeps being used and a server-side idle timeout never finds one. |
|
|
82
|
+
| `discover` | `false` | Reads the members with `SHOW ENDPOINTS` (server ≥ 0.327; no privilege needed) when a member is first reached and every `refresh_interval` (60 s) after — every 5 s while a member is `resyncing` or `unreachable`. New members are added, members that left are dropped (seeds are kept, marked `unlisted`), and members the server reports `resyncing` or `unreachable` get no new connections until they are `ready` again, when they start with a fresh latency average. An older or standalone server leaves the seeds as they are, and the pool stops asking. |
|
|
83
|
+
| `eject_errors` | `3` | Consecutive transport failures (a failed dial, an I/O error mid-request, a capacity refusal) that eject a member. Statement errors and refused logins do not count. |
|
|
84
|
+
| `eject_latency_factor`, `eject_latency_floor` | `3.0`, `0.05` | A member whose round-trip average is above factor × the other usable members' median **and** more than floor seconds above it is ejected (once it and another member have 10 samples each). A factor of `0` disables it. |
|
|
85
|
+
| `eject_backoff`, `eject_backoff_max` | `5.0`, `60.0` | How long an ejection lasts, doubling for each repeat up to the max; a member that stays healthy for `eject_backoff_max` after a re-admission starts over at the base. When the time is up a background thread dials the member and runs one statement: success re-admits it (with a fresh latency average), failure ejects it again. The last usable member is never ejected — a slow or flaky member beats none. |
|
|
86
|
+
| `read_your_writes` | `0.0` | Seconds after a write during which checkouts with the same `affinity` go to the member that took the write. |
|
|
87
|
+
| `logger` | `nil` | A `Logger` (or anything with `warn`) told of each ejection and re-admission. |
|
|
88
|
+
|
|
89
|
+
An ejected member gets no new connections and its idle ones are closed; a
|
|
90
|
+
connection to it that is returned to the pool is closed instead of kept.
|
|
91
|
+
With a target member (round-robin, least-latency, an affinity) the pool
|
|
92
|
+
hands out an idle connection to that member or dials it; a failed dial falls
|
|
93
|
+
through to the other usable members, then to the unusable ones, so a
|
|
94
|
+
checkout never has nowhere to go while any member is configured.
|
|
95
|
+
|
|
96
|
+
**Latency.** Per member the pool keeps an exponentially weighted average of
|
|
97
|
+
round trips (request sent to the first response frame): each sample moves it
|
|
98
|
+
5 % of the way, and no sample counts for more than 10 × the larger of the
|
|
99
|
+
current average and `eject_latency_floor`. One slow statement barely moves
|
|
100
|
+
it; a member that stays slow is noticed within a few dozen statements.
|
|
101
|
+
|
|
102
|
+
**Read-your-writes.** The member that coordinated a write reads it back at
|
|
103
|
+
any consistency, `ONE` included; another member may not have applied it yet.
|
|
104
|
+
With `read_your_writes` set, pass an affinity key — a user or session id —
|
|
105
|
+
when you check out:
|
|
106
|
+
|
|
107
|
+
```ruby
|
|
108
|
+
pool.with(affinity: user_id) do |conn|
|
|
109
|
+
conn.exec_params("INSERT INTO notes (id, owner, body) VALUES ($1, $2, $3)", [nid, user_id, body])
|
|
110
|
+
end
|
|
111
|
+
# … the next request for the same user, within the window:
|
|
112
|
+
pool.with(affinity: user_id) do |conn| # same member
|
|
113
|
+
conn.exec_params("SELECT body FROM notes WHERE id = $1", [nid])
|
|
114
|
+
end
|
|
115
|
+
```
|
|
116
|
+
|
|
117
|
+
The window starts when a write succeeds on a connection checked out under
|
|
118
|
+
the key, and checkouts under it go to that member until the window ends
|
|
119
|
+
(unless the member is ejected or not `ready`). Checkouts without a key
|
|
120
|
+
share one pool-wide key, so with constant writes everything follows the
|
|
121
|
+
latest writer — pass a key when you balance. A statement is a write unless
|
|
122
|
+
its first keyword (after whitespace, comments and opening parentheses) is
|
|
123
|
+
`SELECT`, `SHOW`, `EXPLAIN`, `DESCRIBE`, `DESC`, `VALUES`, `WITH` or `USE`.
|
|
124
|
+
|
|
125
|
+
**Watching it.** `pool.members` returns one Hash per member, sorted by
|
|
126
|
+
endpoint:
|
|
127
|
+
|
|
128
|
+
```ruby
|
|
129
|
+
[{ endpoint: "db1:7000", canonical: "10.0.0.1:7000", state: :ready,
|
|
130
|
+
latency_ms: 0.8, in_use: 2, idle: 3, seed: true }, …]
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
`endpoint` is what the pool dials, `canonical` the name the member reports
|
|
134
|
+
for itself (once discovered), `state` one of `:ready`, `:ejected`,
|
|
135
|
+
`:resyncing`, `:unreachable`, `:unlisted`, and `latency_ms` is `nil` until
|
|
136
|
+
the member has been measured. The rules are the same in every skaidb driver
|
|
137
|
+
([PROTOCOL.md §6.1](https://skaidb.org/docs/PROTOCOL.html#61-balancing-a-pool-across-members)).
|
|
138
|
+
|
|
25
139
|
## Behaviour
|
|
26
140
|
|
|
27
|
-
- Every `Skaidb.connect` keyword passes through (`seeds`, `database`, TLS,
|
|
28
|
-
`consistency`, `timeout`), so pooled connections fail over across seeds
|
|
29
|
-
and run `USE` exactly like single ones.
|
|
30
141
|
- `maxsize` (default 10) bounds the connections kept **idle**, not the number
|
|
31
142
|
checked out. `checkout` never blocks: with no idle connection available it
|
|
32
143
|
dials a new one, and a connection returned while `maxsize` are already idle
|
|
@@ -38,10 +149,15 @@ pool.close # closes every idle connection
|
|
|
38
149
|
and will fail its first statement with `Skaidb::ConnectionError`; that
|
|
39
150
|
failure marks it broken, so `checkin` discards it and the next checkout
|
|
40
151
|
gets a fresh one. Retry the statement in that case if it is safe to.
|
|
41
|
-
- `close` marks the pool closed
|
|
42
|
-
checked out at the time are closed when
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
152
|
+
- `close` marks the pool closed, closes the idle connections and stops the
|
|
153
|
+
background thread; connections checked out at the time are closed when
|
|
154
|
+
they are returned. `checkout` on a closed pool raises `Skaidb::Error`.
|
|
155
|
+
`maxsize < 1`, `eject_errors < 1` and an unknown `load_balancing` or
|
|
156
|
+
`idle_order` raise `ArgumentError`.
|
|
157
|
+
- The pool is thread-safe: the idle list and the member table are guarded
|
|
158
|
+
by one mutex. The background thread (probes and discovery refreshes) is
|
|
159
|
+
started on first need; Ruby never waits for it at exit. A single
|
|
160
|
+
connection should still be used by one thread at a time; the pool is what
|
|
161
|
+
gives each thread its own.
|
|
46
162
|
|
|
47
163
|
See also `examples/pool.rb`.
|
data/lib/skaidb.rb
CHANGED
|
@@ -34,7 +34,7 @@ require "bigdecimal"
|
|
|
34
34
|
module Skaidb
|
|
35
35
|
# The package version — the single source of truth. The gemspec reads it,
|
|
36
36
|
# and it is what the driver reports to the server in the Hello frame.
|
|
37
|
-
VERSION = "1.
|
|
37
|
+
VERSION = "1.2.0"
|
|
38
38
|
|
|
39
39
|
# Base class for every error raised by this driver.
|
|
40
40
|
class Error < StandardError; end
|
|
@@ -677,11 +677,28 @@ module Skaidb
|
|
|
677
677
|
class Connection
|
|
678
678
|
# @return [Boolean] whether the connection has been closed
|
|
679
679
|
attr_reader :closed
|
|
680
|
-
|
|
680
|
+
# @return [String, nil] the member this connection reached, as the
|
|
681
|
+
# "host:port" it dialed (nil until it connects)
|
|
682
|
+
attr_reader :endpoint
|
|
683
|
+
# Internal, for Pool: the pool told of this connection's round trips,
|
|
684
|
+
# transport failures and writes, and asked for the dial order when it
|
|
685
|
+
# re-dials (nil outside a pool); the member the pool counts it against;
|
|
686
|
+
# the affinity key it was checked out under; whether it is checked out.
|
|
687
|
+
attr_accessor :observer, :pool_member, :pool_affinity, :pool_out
|
|
688
|
+
|
|
689
|
+
# +endpoints+ and +observer+ are internal, for Pool: an explicit dial
|
|
690
|
+
# order of [host, port] pairs (no shuffle) and the pool that owns the
|
|
691
|
+
# connection.
|
|
681
692
|
def initialize(host:, port:, user:, password:, consistency:, timeout:,
|
|
682
693
|
database: nil, tls: false, tls_ca: nil, tls_insecure: false,
|
|
683
694
|
tls_server_name: "skaidb", seeds: nil, tls_client_cert: nil,
|
|
684
|
-
tls_client_key: nil, auth_mechanism: :scram
|
|
695
|
+
tls_client_key: nil, auth_mechanism: :scram, endpoints: nil,
|
|
696
|
+
observer: nil)
|
|
697
|
+
@observer = observer
|
|
698
|
+
@endpoint = nil
|
|
699
|
+
@pool_member = nil
|
|
700
|
+
@pool_affinity = nil
|
|
701
|
+
@pool_out = false
|
|
685
702
|
@consistency = Consistency.resolve(consistency)
|
|
686
703
|
@auth_mechanism = AuthMechanism.resolve(auth_mechanism)
|
|
687
704
|
if tls_client_cert.to_s.empty? != tls_client_key.to_s.empty?
|
|
@@ -704,7 +721,7 @@ module Skaidb
|
|
|
704
721
|
timeout: timeout, database: database, tls: tls, tls_ca: tls_ca,
|
|
705
722
|
tls_insecure: tls_insecure, tls_server_name: tls_server_name,
|
|
706
723
|
seeds: seeds, tls_client_cert: tls_client_cert,
|
|
707
|
-
tls_client_key: tls_client_key }
|
|
724
|
+
tls_client_key: tls_client_key, endpoints: endpoints }
|
|
708
725
|
dial!
|
|
709
726
|
end
|
|
710
727
|
|
|
@@ -726,11 +743,19 @@ module Skaidb
|
|
|
726
743
|
# Seeds: try each until one connects AND authenticates — a node that
|
|
727
744
|
# accepts TCP while unhealthy must not swallow the attempt. skaidb is
|
|
728
745
|
# leaderless, so any node serves; the order is shuffled so many
|
|
729
|
-
# clients spread instead of stampeding the first entry.
|
|
730
|
-
|
|
731
|
-
|
|
732
|
-
|
|
733
|
-
|
|
746
|
+
# clients spread instead of stampeding the first entry. A pooled
|
|
747
|
+
# connection dials in the pool's order instead: the member the pool
|
|
748
|
+
# picked first, and on a re-dial the usable members with the one it
|
|
749
|
+
# lost last.
|
|
750
|
+
endpoints = if @observer && @endpoint
|
|
751
|
+
@observer.dial_order(@endpoint)
|
|
752
|
+
elsif @dial_args[:endpoints]
|
|
753
|
+
@dial_args[:endpoints]
|
|
754
|
+
else
|
|
755
|
+
(seeds && !seeds.empty? ? seeds : ["#{host}:#{port}"]).map do |sd|
|
|
756
|
+
Skaidb.parse_endpoint(sd, port)
|
|
757
|
+
end.shuffle
|
|
758
|
+
end
|
|
734
759
|
last = nil
|
|
735
760
|
endpoints.each do |(h, prt)|
|
|
736
761
|
begin
|
|
@@ -745,10 +770,12 @@ module Skaidb
|
|
|
745
770
|
else
|
|
746
771
|
handshake(user.to_s, password.to_s)
|
|
747
772
|
end
|
|
773
|
+
@endpoint = "#{h}:#{prt}"
|
|
748
774
|
last = nil
|
|
749
775
|
break
|
|
750
776
|
rescue StandardError => e
|
|
751
777
|
last = e
|
|
778
|
+
@observer.note_failure("#{h}:#{prt}") if @observer && Skaidb.member_fault?(e)
|
|
752
779
|
begin
|
|
753
780
|
@sock&.close
|
|
754
781
|
rescue StandardError
|
|
@@ -848,7 +875,9 @@ module Skaidb
|
|
|
848
875
|
raise QueryError, "statement expects #{nparams} parameters, got #{order.length}"
|
|
849
876
|
end
|
|
850
877
|
|
|
851
|
-
|
|
878
|
+
res = exec_prepared(id, order, level)
|
|
879
|
+
note_statement(sql)
|
|
880
|
+
return res
|
|
852
881
|
end
|
|
853
882
|
end
|
|
854
883
|
text = begin
|
|
@@ -890,12 +919,10 @@ module Skaidb
|
|
|
890
919
|
req << [b.bytesize].pack("V") << b
|
|
891
920
|
end
|
|
892
921
|
end
|
|
893
|
-
reader =
|
|
894
|
-
|
|
895
|
-
|
|
896
|
-
|
|
897
|
-
end
|
|
898
|
-
parse_response(reader).cmd_tuples
|
|
922
|
+
reader = request(req)
|
|
923
|
+
n = parse_response(reader).cmd_tuples
|
|
924
|
+
note_statement(sql)
|
|
925
|
+
n
|
|
899
926
|
end
|
|
900
927
|
|
|
901
928
|
# Stream a result set: yields one row Hash at a time while holding a
|
|
@@ -948,9 +975,17 @@ module Skaidb
|
|
|
948
975
|
live = false
|
|
949
976
|
summary = nil
|
|
950
977
|
@mutex.synchronize do
|
|
951
|
-
|
|
952
|
-
|
|
978
|
+
started = Process.clock_gettime(Process::CLOCK_MONOTONIC)
|
|
979
|
+
begin
|
|
980
|
+
write_frame(req)
|
|
981
|
+
r = Reader.new(read_frame)
|
|
982
|
+
rescue ConnectionError
|
|
983
|
+
observe_failure
|
|
984
|
+
raise
|
|
985
|
+
end
|
|
986
|
+
@observer&.note_round_trip(self, Process.clock_gettime(Process::CLOCK_MONOTONIC) - started)
|
|
953
987
|
tag = r.u8
|
|
988
|
+
note_statement(sql) unless tag == 3
|
|
954
989
|
case tag
|
|
955
990
|
when 3
|
|
956
991
|
msg = r.text
|
|
@@ -1023,11 +1058,7 @@ module Skaidb
|
|
|
1023
1058
|
|
|
1024
1059
|
sql_bytes = sql.dup.force_encoding("UTF-8").b
|
|
1025
1060
|
req = [2].pack("C") + [sql_bytes.bytesize].pack("V") + sql_bytes
|
|
1026
|
-
reader =
|
|
1027
|
-
@mutex.synchronize do
|
|
1028
|
-
write_frame(req)
|
|
1029
|
-
reader = Reader.new(read_frame)
|
|
1030
|
-
end
|
|
1061
|
+
reader = request(req)
|
|
1031
1062
|
tag = reader.u8
|
|
1032
1063
|
case tag
|
|
1033
1064
|
when 4
|
|
@@ -1055,11 +1086,7 @@ module Skaidb
|
|
|
1055
1086
|
v = Skaidb.encode_value(p)
|
|
1056
1087
|
req << [v.bytesize].pack("V") << v
|
|
1057
1088
|
end
|
|
1058
|
-
reader =
|
|
1059
|
-
@mutex.synchronize do
|
|
1060
|
-
write_frame(req)
|
|
1061
|
-
reader = Reader.new(read_frame)
|
|
1062
|
-
end
|
|
1089
|
+
reader = request(req)
|
|
1063
1090
|
parse_response(reader)
|
|
1064
1091
|
end
|
|
1065
1092
|
|
|
@@ -1113,6 +1140,9 @@ module Skaidb
|
|
|
1113
1140
|
rescue StandardError
|
|
1114
1141
|
# ignore — socket already gone
|
|
1115
1142
|
end
|
|
1143
|
+
# Closed while checked out instead of returned: the pool stops
|
|
1144
|
+
# counting it against its member.
|
|
1145
|
+
@observer.note_closed(self) if @observer && @pool_out
|
|
1116
1146
|
nil
|
|
1117
1147
|
end
|
|
1118
1148
|
|
|
@@ -1275,12 +1305,38 @@ module Skaidb
|
|
|
1275
1305
|
|
|
1276
1306
|
sql_bytes = sql.dup.force_encoding("UTF-8").b
|
|
1277
1307
|
req = [1, consistency].pack("CC") + [sql_bytes.bytesize].pack("V") + sql_bytes
|
|
1278
|
-
reader =
|
|
1279
|
-
|
|
1280
|
-
|
|
1281
|
-
|
|
1308
|
+
reader = request(req)
|
|
1309
|
+
res = parse_response(reader)
|
|
1310
|
+
note_statement(sql)
|
|
1311
|
+
res
|
|
1312
|
+
end
|
|
1313
|
+
|
|
1314
|
+
# One request and its response under the connection mutex. In a pool the
|
|
1315
|
+
# round trip (send to the first response frame) feeds the member's
|
|
1316
|
+
# latency average, and a transport failure counts against the member.
|
|
1317
|
+
def request(req)
|
|
1318
|
+
started = Process.clock_gettime(Process::CLOCK_MONOTONIC)
|
|
1319
|
+
frame = begin
|
|
1320
|
+
@mutex.synchronize do
|
|
1321
|
+
write_frame(req)
|
|
1322
|
+
read_frame
|
|
1323
|
+
end
|
|
1324
|
+
rescue ConnectionError
|
|
1325
|
+
observe_failure
|
|
1326
|
+
raise
|
|
1282
1327
|
end
|
|
1283
|
-
|
|
1328
|
+
@observer&.note_round_trip(self, Process.clock_gettime(Process::CLOCK_MONOTONIC) - started)
|
|
1329
|
+
Reader.new(frame)
|
|
1330
|
+
end
|
|
1331
|
+
|
|
1332
|
+
def observe_failure
|
|
1333
|
+
@observer.note_failure(@endpoint) if @observer && @endpoint
|
|
1334
|
+
end
|
|
1335
|
+
|
|
1336
|
+
# A statement succeeded: a pool with read-your-writes on remembers which
|
|
1337
|
+
# member took a write.
|
|
1338
|
+
def note_statement(sql)
|
|
1339
|
+
@observer&.note_statement(self, sql)
|
|
1284
1340
|
end
|
|
1285
1341
|
|
|
1286
1342
|
def parse_response(r)
|
|
@@ -1338,23 +1394,8 @@ module Skaidb
|
|
|
1338
1394
|
# is optional and, when given, must equal the certificate's Common Name)
|
|
1339
1395
|
# @yield [conn] optional block; the connection is closed when it returns
|
|
1340
1396
|
# @return [Connection] (or the block's value when a block is given)
|
|
1341
|
-
def self.connect(
|
|
1342
|
-
|
|
1343
|
-
database: nil, tls: false, tls_ca: nil, tls_insecure: false,
|
|
1344
|
-
tls_server_name: "skaidb", seeds: nil, tls_client_cert: nil,
|
|
1345
|
-
tls_client_key: nil, auth_mechanism: :scram)
|
|
1346
|
-
tls = true if tls_ca || tls_insecure || tls_client_cert
|
|
1347
|
-
# A SCRAM login with no user is "anonymous"; a certificate login with no
|
|
1348
|
-
# user sends an empty one and lets the certificate's CN decide.
|
|
1349
|
-
user = "anonymous" if user.nil? && AuthMechanism.resolve(auth_mechanism) == AuthMechanism::SCRAM
|
|
1350
|
-
conn = Connection.new(host: host, port: port, user: user,
|
|
1351
|
-
password: password, consistency: consistency,
|
|
1352
|
-
timeout: timeout, database: database, tls: tls,
|
|
1353
|
-
tls_ca: tls_ca, tls_insecure: tls_insecure,
|
|
1354
|
-
tls_server_name: tls_server_name, seeds: seeds,
|
|
1355
|
-
tls_client_cert: tls_client_cert,
|
|
1356
|
-
tls_client_key: tls_client_key,
|
|
1357
|
-
auth_mechanism: auth_mechanism)
|
|
1397
|
+
def self.connect(**options)
|
|
1398
|
+
conn = Connection.new(**connection_options(**options))
|
|
1358
1399
|
return conn unless block_given?
|
|
1359
1400
|
|
|
1360
1401
|
begin
|
|
@@ -1363,67 +1404,257 @@ module Skaidb
|
|
|
1363
1404
|
conn.close
|
|
1364
1405
|
end
|
|
1365
1406
|
end
|
|
1366
|
-
|
|
1407
|
+
|
|
1408
|
+
# Internal: Skaidb.connect's keywords with their defaults applied, as
|
|
1409
|
+
# Connection.new takes them (Pool dials every connection with these).
|
|
1410
|
+
def self.connection_options(host: "localhost", port: 7000, user: nil,
|
|
1411
|
+
password: "", consistency: :quorum, timeout: 10.0,
|
|
1412
|
+
database: nil, tls: false, tls_ca: nil, tls_insecure: false,
|
|
1413
|
+
tls_server_name: "skaidb", seeds: nil, tls_client_cert: nil,
|
|
1414
|
+
tls_client_key: nil, auth_mechanism: :scram)
|
|
1415
|
+
tls = true if tls_ca || tls_insecure || tls_client_cert
|
|
1416
|
+
# A SCRAM login with no user is "anonymous"; a certificate login with no
|
|
1417
|
+
# user sends an empty one and lets the certificate's CN decide.
|
|
1418
|
+
user = "anonymous" if user.nil? && AuthMechanism.resolve(auth_mechanism) == AuthMechanism::SCRAM
|
|
1419
|
+
{ host: host, port: port, user: user, password: password, consistency: consistency,
|
|
1420
|
+
timeout: timeout, database: database, tls: tls, tls_ca: tls_ca,
|
|
1421
|
+
tls_insecure: tls_insecure, tls_server_name: tls_server_name, seeds: seeds,
|
|
1422
|
+
tls_client_cert: tls_client_cert, tls_client_key: tls_client_key,
|
|
1423
|
+
auth_mechanism: auth_mechanism }
|
|
1424
|
+
end
|
|
1425
|
+
|
|
1426
|
+
# Internal: "host:port", or a bare host on +default_port+, as [host, port].
|
|
1427
|
+
def self.parse_endpoint(text, default_port)
|
|
1428
|
+
h, _, p = text.to_s.rpartition(":")
|
|
1429
|
+
h.empty? ? [text.to_s, default_port] : [h, p.to_i]
|
|
1430
|
+
end
|
|
1431
|
+
|
|
1432
|
+
# Leading keywords of statements that never write: everything else counts
|
|
1433
|
+
# as a write for read-your-writes affinity (a read taken for a write only
|
|
1434
|
+
# keeps the caller on its member a little longer).
|
|
1435
|
+
READ_KEYWORDS = %w[select show explain describe desc values with use].freeze
|
|
1436
|
+
# Whitespace, comments and opening parentheses before the first keyword.
|
|
1437
|
+
STATEMENT_LEAD = %r{\A(?:\s+|\(|--[^\n]*\n?|/\*.*?(?:\*/|\z))*}m.freeze
|
|
1438
|
+
private_constant :READ_KEYWORDS, :STATEMENT_LEAD
|
|
1439
|
+
|
|
1440
|
+
# Internal: whether +sql+ counts as a write for read-your-writes — its
|
|
1441
|
+
# first keyword, after whitespace, comments and opening parentheses, is
|
|
1442
|
+
# none of SELECT, SHOW, EXPLAIN, DESCRIBE, DESC, VALUES, WITH, USE.
|
|
1443
|
+
def self.write_statement?(sql)
|
|
1444
|
+
rest = sql.to_s.b.sub(STATEMENT_LEAD, "")
|
|
1445
|
+
!READ_KEYWORDS.include?(rest[/\A[A-Za-z_]*/].downcase)
|
|
1446
|
+
end
|
|
1447
|
+
|
|
1448
|
+
# Internal: whether a failed dial says something about the member's
|
|
1449
|
+
# health. A refused login is the caller's problem, not the member's —
|
|
1450
|
+
# except a capacity refusal, which is the member being full.
|
|
1451
|
+
def self.member_fault?(error)
|
|
1452
|
+
return true unless error.is_a?(Error) # socket, TLS, timeout
|
|
1453
|
+
|
|
1454
|
+
msg = error.message
|
|
1455
|
+
return msg.include?("too many connections:") if msg.start_with?("authentication denied")
|
|
1456
|
+
|
|
1457
|
+
!msg.start_with?("server signature mismatch")
|
|
1458
|
+
end
|
|
1459
|
+
|
|
1460
|
+
# A thread-safe pool of connections, balanced across the cluster's members.
|
|
1367
1461
|
#
|
|
1368
1462
|
# +maxsize+ bounds the connections kept IDLE, not the number checked out: a
|
|
1369
1463
|
# burst creates extras and the surplus is closed on return. Every keyword
|
|
1370
|
-
# accepted by Skaidb.connect passes through
|
|
1371
|
-
#
|
|
1464
|
+
# accepted by Skaidb.connect passes through: +host+/+port+/+seeds+ name the
|
|
1465
|
+
# starting members, and the rest (TLS, the session database, ...) apply to
|
|
1466
|
+
# every connection.
|
|
1372
1467
|
#
|
|
1373
|
-
#
|
|
1468
|
+
# Load balancing (PROTOCOL.md §6.1, the same in every skaidb driver; see
|
|
1469
|
+
# docs/pooling.md):
|
|
1470
|
+
#
|
|
1471
|
+
# - +load_balancing+ — +:random+ (default: each new connection dials the
|
|
1472
|
+
# members in random order), +:round_robin+ (checkouts rotate over the
|
|
1473
|
+
# members) or +:least_latency+ (of two members picked at random, the one
|
|
1474
|
+
# with the lower latency average x checked-out connections).
|
|
1475
|
+
# - +idle_order+ — +:lifo+ (default: the most recently returned idle
|
|
1476
|
+
# connection first) or +:fifo+ (the longest idle first, so every idle
|
|
1477
|
+
# connection keeps getting used).
|
|
1478
|
+
# - +discover+ — read the members from SHOW ENDPOINTS (server >= 0.327)
|
|
1479
|
+
# when a member is first reached and every +refresh_interval+ seconds
|
|
1480
|
+
# after (every 5 s while one is not ready): joiners are added, leavers
|
|
1481
|
+
# dropped, and members reported +resyncing+ or +unreachable+ get no new
|
|
1482
|
+
# connections until they are +ready+ again.
|
|
1483
|
+
# - Health ejection: a member with +eject_errors+ consecutive transport
|
|
1484
|
+
# failures, or whose latency average is above +eject_latency_factor+
|
|
1485
|
+
# times the other members' median and more than +eject_latency_floor+
|
|
1486
|
+
# seconds above it, gets no connections for +eject_backoff+ seconds
|
|
1487
|
+
# (doubling per repeat, up to +eject_backoff_max+); then a background
|
|
1488
|
+
# thread probes it and re-admits it. The last usable member is never
|
|
1489
|
+
# ejected.
|
|
1490
|
+
# - +read_your_writes+ — seconds after a write during which checkouts with
|
|
1491
|
+
# the same +affinity:+ key go to the member that took it (the coordinator
|
|
1492
|
+
# that acknowledged a write reads it back even at ONE). 0 (default) = off.
|
|
1493
|
+
# - +logger+ — a Logger (or anything with +warn+) told of ejections and
|
|
1494
|
+
# re-admissions.
|
|
1495
|
+
#
|
|
1496
|
+
# pool = Skaidb::Pool.new(seeds: ["h1:7000", "h2:7000"], database: "app", maxsize: 8,
|
|
1497
|
+
# load_balancing: :least_latency, discover: true)
|
|
1374
1498
|
# pool.with { |conn| conn.exec("SELECT 1") }
|
|
1375
1499
|
# pool.close
|
|
1376
1500
|
class Pool
|
|
1377
|
-
|
|
1501
|
+
POLICIES = %i[random round_robin least_latency].freeze
|
|
1502
|
+
IDLE_ORDERS = %i[lifo fifo].freeze
|
|
1503
|
+
# Latency EWMA: each round trip moves a member's average 5% toward it,
|
|
1504
|
+
# and no sample counts for more than 10x the larger of the current
|
|
1505
|
+
# average and the ejection floor — one slow query does not eject a
|
|
1506
|
+
# healthy member, a member that stays slow is ejected within a few dozen
|
|
1507
|
+
# statements.
|
|
1508
|
+
EWMA_WEIGHT = 0.05
|
|
1509
|
+
EWMA_CLAMP = 10.0
|
|
1510
|
+
# Round trips a member (and at least one other) needs before its average
|
|
1511
|
+
# can eject it.
|
|
1512
|
+
EWMA_MIN_SAMPLES = 10
|
|
1513
|
+
# least_latency: a member whose last round trip is older than this scores
|
|
1514
|
+
# 0, so a lightly loaded pool keeps re-measuring every member instead of
|
|
1515
|
+
# settling forever on whichever answered fastest once.
|
|
1516
|
+
STALE_SAMPLE = 2.0
|
|
1517
|
+
# Discovery refresh while a listed member is not ready: a member is often
|
|
1518
|
+
# +unreachable+ or +resyncing+ only for seconds (a peer's circuit breaker
|
|
1519
|
+
# right after it restarts), and waiting a whole refresh interval to let
|
|
1520
|
+
# it back in leaves the load on the others.
|
|
1521
|
+
REFRESH_NOT_READY = 5.0
|
|
1522
|
+
# How often the background thread checks for members due a re-probe.
|
|
1523
|
+
MAINTAIN_TICK = 0.25
|
|
1524
|
+
# Read-your-writes keys kept before the expired ones are swept.
|
|
1525
|
+
AFFINITY_SWEEP = 4096
|
|
1526
|
+
private_constant :EWMA_WEIGHT, :EWMA_CLAMP, :EWMA_MIN_SAMPLES, :STALE_SAMPLE,
|
|
1527
|
+
:REFRESH_NOT_READY, :MAINTAIN_TICK, :AFFINITY_SWEEP
|
|
1528
|
+
|
|
1529
|
+
# One member as the pool sees it.
|
|
1530
|
+
class Member
|
|
1531
|
+
attr_reader :endpoint, :host, :port, :seed
|
|
1532
|
+
attr_accessor :canonical, :listed, :ewma, :samples, :last_sample, :errors,
|
|
1533
|
+
:ejected_until, :ejections, :readmitted_at, :in_use, :probing
|
|
1534
|
+
|
|
1535
|
+
def initialize(host, port, seed)
|
|
1536
|
+
@host = host
|
|
1537
|
+
@port = port
|
|
1538
|
+
@endpoint = "#{host}:#{port}" # what the pool dials, and its key
|
|
1539
|
+
@seed = seed # configured: discovery never drops it
|
|
1540
|
+
@canonical = nil # its own SHOW ENDPOINTS name
|
|
1541
|
+
@listed = :ready # :ready, :resyncing, :unreachable, :unlisted
|
|
1542
|
+
@ewma = nil # seconds
|
|
1543
|
+
@samples = 0
|
|
1544
|
+
@last_sample = 0.0
|
|
1545
|
+
@errors = 0 # consecutive transport failures
|
|
1546
|
+
@ejected_until = nil # nil = admitted
|
|
1547
|
+
@ejections = 0
|
|
1548
|
+
@readmitted_at = 0.0
|
|
1549
|
+
@in_use = 0
|
|
1550
|
+
@probing = false
|
|
1551
|
+
end
|
|
1552
|
+
end
|
|
1553
|
+
private_constant :Member
|
|
1554
|
+
|
|
1555
|
+
def initialize(maxsize: 10, load_balancing: :random, idle_order: :lifo, discover: false,
|
|
1556
|
+
refresh_interval: 60.0, read_your_writes: 0.0, eject_errors: 3,
|
|
1557
|
+
eject_latency_factor: 3.0, eject_latency_floor: 0.05, eject_backoff: 5.0,
|
|
1558
|
+
eject_backoff_max: 60.0, logger: nil, **connect_kwargs)
|
|
1378
1559
|
raise ArgumentError, "maxsize must be >= 1" if maxsize < 1
|
|
1560
|
+
raise ArgumentError, "eject_errors must be >= 1" if eject_errors < 1
|
|
1379
1561
|
|
|
1562
|
+
@policy = choice(load_balancing, POLICIES, "load_balancing")
|
|
1563
|
+
@fifo = choice(idle_order, IDLE_ORDERS, "idle_order") == :fifo
|
|
1380
1564
|
@maxsize = maxsize
|
|
1381
|
-
@
|
|
1565
|
+
@discover = discover ? true : false
|
|
1566
|
+
@refresh_interval = refresh_interval.to_f
|
|
1567
|
+
@ryw = read_your_writes.to_f
|
|
1568
|
+
@eject_errors = eject_errors
|
|
1569
|
+
@eject_factor = eject_latency_factor.to_f
|
|
1570
|
+
@eject_floor = eject_latency_floor.to_f
|
|
1571
|
+
@backoff = eject_backoff.to_f
|
|
1572
|
+
@backoff_max = eject_backoff_max.to_f
|
|
1573
|
+
@logger = logger
|
|
1574
|
+
@conn_opts = Skaidb.connection_options(**connect_kwargs)
|
|
1575
|
+
@port = @conn_opts[:port]
|
|
1576
|
+
seeds = @conn_opts[:seeds]
|
|
1577
|
+
@members = {}
|
|
1578
|
+
(seeds && !seeds.empty? ? seeds : ["#{@conn_opts[:host]}:#{@port}"]).each do |s|
|
|
1579
|
+
host, port = Skaidb.parse_endpoint(s, @port)
|
|
1580
|
+
@members["#{host}:#{port}"] ||= Member.new(host, port, true)
|
|
1581
|
+
end
|
|
1382
1582
|
@idle = []
|
|
1383
1583
|
@mutex = Mutex.new
|
|
1584
|
+
@wake = ConditionVariable.new
|
|
1585
|
+
@rr = -1
|
|
1586
|
+
@affinity = {}
|
|
1587
|
+
@discovery_off = false
|
|
1588
|
+
@next_refresh = 0.0
|
|
1589
|
+
@maintainer = nil
|
|
1384
1590
|
@closed = false
|
|
1385
1591
|
end
|
|
1386
1592
|
|
|
1387
|
-
#
|
|
1388
|
-
|
|
1593
|
+
# -- checkout / checkin --
|
|
1594
|
+
|
|
1595
|
+
# Check out a usable connection: an idle one to the member the policy
|
|
1596
|
+
# picks, or a new one dialed to it (falling back to the other members).
|
|
1597
|
+
# +affinity+ scopes read-your-writes (nil = pool-wide).
|
|
1598
|
+
def checkout(affinity: nil)
|
|
1389
1599
|
loop do
|
|
1390
|
-
conn =
|
|
1600
|
+
conn = order = nil
|
|
1601
|
+
stale = []
|
|
1602
|
+
@mutex.synchronize do
|
|
1391
1603
|
raise Error, "pool is closed" if @closed
|
|
1392
1604
|
|
|
1393
|
-
|
|
1605
|
+
target = pick(affinity)
|
|
1606
|
+
conn, stale = take_idle(target)
|
|
1607
|
+
order = order_for(target) if conn.nil?
|
|
1394
1608
|
end
|
|
1395
|
-
|
|
1396
|
-
|
|
1397
|
-
|
|
1398
|
-
|
|
1399
|
-
|
|
1400
|
-
|
|
1401
|
-
|
|
1402
|
-
|
|
1403
|
-
|
|
1609
|
+
stale.each { |c| close_quietly(c) }
|
|
1610
|
+
if conn.nil?
|
|
1611
|
+
conn = Connection.new(**@conn_opts, endpoints: order, observer: self)
|
|
1612
|
+
conn.pool_member = conn.endpoint
|
|
1613
|
+
discover_through(conn)
|
|
1614
|
+
unless conn.usable?
|
|
1615
|
+
close_quietly(conn)
|
|
1616
|
+
next
|
|
1617
|
+
end
|
|
1618
|
+
elsif !conn.usable?
|
|
1619
|
+
# A connection the server closed while it sat idle still looks
|
|
1620
|
+
# fine locally, so check before handing it out.
|
|
1621
|
+
close_quietly(conn)
|
|
1622
|
+
next
|
|
1404
1623
|
end
|
|
1624
|
+
@mutex.synchronize do
|
|
1625
|
+
m = @members[conn.pool_member]
|
|
1626
|
+
m.in_use += 1 if m
|
|
1627
|
+
conn.pool_affinity = affinity
|
|
1628
|
+
conn.pool_out = true
|
|
1629
|
+
end
|
|
1630
|
+
return conn
|
|
1405
1631
|
end
|
|
1406
1632
|
end
|
|
1407
1633
|
|
|
1408
|
-
# Return a connection, closing it if broken
|
|
1634
|
+
# Return a connection, closing it if it is broken, its member is ejected
|
|
1635
|
+
# or gone, or the pool is full.
|
|
1409
1636
|
def checkin(conn)
|
|
1410
1637
|
keep = @mutex.synchronize do
|
|
1411
|
-
|
|
1412
|
-
|
|
1413
|
-
|
|
1414
|
-
|
|
1415
|
-
|
|
1416
|
-
|
|
1417
|
-
|
|
1418
|
-
|
|
1419
|
-
|
|
1420
|
-
|
|
1638
|
+
if conn.pool_out
|
|
1639
|
+
conn.pool_out = false
|
|
1640
|
+
counted = @members[conn.pool_member]
|
|
1641
|
+
counted.in_use -= 1 if counted && counted.in_use.positive?
|
|
1642
|
+
end
|
|
1643
|
+
m = @members[conn.pool_member]
|
|
1644
|
+
if !@closed && conn.usable? && m && usable?(m) && @idle.length < @maxsize
|
|
1645
|
+
@idle.push(conn)
|
|
1646
|
+
true
|
|
1647
|
+
else
|
|
1648
|
+
false
|
|
1649
|
+
end
|
|
1421
1650
|
end
|
|
1651
|
+
close_quietly(conn) unless keep
|
|
1422
1652
|
end
|
|
1423
1653
|
|
|
1424
1654
|
# Run the block with a checked-out connection, returning it afterwards.
|
|
1425
|
-
|
|
1426
|
-
|
|
1655
|
+
# +affinity+ scopes read-your-writes (nil = pool-wide).
|
|
1656
|
+
def with(affinity: nil)
|
|
1657
|
+
conn = checkout(affinity: affinity)
|
|
1427
1658
|
begin
|
|
1428
1659
|
yield conn
|
|
1429
1660
|
ensure
|
|
@@ -1431,20 +1662,400 @@ module Skaidb
|
|
|
1431
1662
|
end
|
|
1432
1663
|
end
|
|
1433
1664
|
|
|
1434
|
-
#
|
|
1665
|
+
# The members as the pool sees them, for monitoring: one Hash each with
|
|
1666
|
+
# +:endpoint+ (what the pool dials), +:canonical+ (the member's own name,
|
|
1667
|
+
# once discovered), +:state+ (+:ready+, +:ejected+, +:resyncing+,
|
|
1668
|
+
# +:unreachable+, +:unlisted+), +:latency_ms+ (the round-trip average,
|
|
1669
|
+
# nil until measured), +:in_use+, +:idle+ and +:seed+.
|
|
1670
|
+
def members
|
|
1671
|
+
@mutex.synchronize do
|
|
1672
|
+
idle = Hash.new(0)
|
|
1673
|
+
@idle.each { |c| idle[c.pool_member] += 1 }
|
|
1674
|
+
@members.values.sort_by { |m| [m.host, m.port] }.map do |m|
|
|
1675
|
+
{ endpoint: m.endpoint, canonical: m.canonical,
|
|
1676
|
+
state: m.ejected_until ? :ejected : m.listed,
|
|
1677
|
+
latency_ms: m.ewma && m.ewma * 1000.0, in_use: m.in_use,
|
|
1678
|
+
idle: idle[m.endpoint], seed: m.seed }
|
|
1679
|
+
end
|
|
1680
|
+
end
|
|
1681
|
+
end
|
|
1682
|
+
|
|
1683
|
+
# Close the pool and every idle connection, and stop the background
|
|
1684
|
+
# thread. Checked-out connections are closed when they are returned.
|
|
1435
1685
|
def close
|
|
1436
1686
|
drained = @mutex.synchronize do
|
|
1437
1687
|
@closed = true
|
|
1688
|
+
@wake.broadcast
|
|
1438
1689
|
d = @idle
|
|
1439
1690
|
@idle = []
|
|
1440
1691
|
d
|
|
1441
1692
|
end
|
|
1442
|
-
drained.each
|
|
1443
|
-
|
|
1444
|
-
|
|
1445
|
-
|
|
1446
|
-
|
|
1693
|
+
drained.each { |c| close_quietly(c) }
|
|
1694
|
+
end
|
|
1695
|
+
|
|
1696
|
+
# -- what connections report (internal: called by Connection, any thread) --
|
|
1697
|
+
|
|
1698
|
+
# Internal: the order a pooled connection re-dials in — the usable
|
|
1699
|
+
# members first, +exclude+ (the member it lost) last.
|
|
1700
|
+
def dial_order(exclude)
|
|
1701
|
+
@mutex.synchronize { order_for(nil, exclude) }
|
|
1702
|
+
end
|
|
1703
|
+
|
|
1704
|
+
# Internal: a round trip on +conn+ took +elapsed+ seconds.
|
|
1705
|
+
def note_round_trip(conn, elapsed)
|
|
1706
|
+
@mutex.synchronize do
|
|
1707
|
+
ep = conn.endpoint
|
|
1708
|
+
if ep != conn.pool_member
|
|
1709
|
+
# The connection re-dialed to another member mid-checkout.
|
|
1710
|
+
if conn.pool_out
|
|
1711
|
+
old = @members[conn.pool_member]
|
|
1712
|
+
old.in_use -= 1 if old && old.in_use.positive?
|
|
1713
|
+
moved = @members[ep]
|
|
1714
|
+
moved.in_use += 1 if moved
|
|
1715
|
+
end
|
|
1716
|
+
conn.pool_member = ep
|
|
1717
|
+
end
|
|
1718
|
+
m = @members[ep]
|
|
1719
|
+
next if m.nil?
|
|
1720
|
+
|
|
1721
|
+
m.errors = 0
|
|
1722
|
+
if m.ewma.nil?
|
|
1723
|
+
m.ewma = elapsed
|
|
1724
|
+
else
|
|
1725
|
+
cap = EWMA_CLAMP * [m.ewma, @eject_floor].max
|
|
1726
|
+
m.ewma += EWMA_WEIGHT * ([elapsed, cap].min - m.ewma)
|
|
1447
1727
|
end
|
|
1728
|
+
m.samples += 1
|
|
1729
|
+
m.last_sample = now
|
|
1730
|
+
next unless m.samples >= EWMA_MIN_SAMPLES && @eject_factor.positive? && m.ejected_until.nil?
|
|
1731
|
+
|
|
1732
|
+
others = @members.each_value.select do |o|
|
|
1733
|
+
!o.equal?(m) && usable?(o) && o.ewma && o.samples >= EWMA_MIN_SAMPLES
|
|
1734
|
+
end.map(&:ewma).sort
|
|
1735
|
+
next if others.empty?
|
|
1736
|
+
|
|
1737
|
+
median = others[(others.length - 1) / 2]
|
|
1738
|
+
if m.ewma > @eject_factor * median && m.ewma - median > @eject_floor
|
|
1739
|
+
eject(m, format("latency %.1f ms vs median %.1f ms", m.ewma * 1000, median * 1000))
|
|
1740
|
+
end
|
|
1741
|
+
end
|
|
1742
|
+
end
|
|
1743
|
+
|
|
1744
|
+
# Internal: a dial or a request to +endpoint+ failed in transport.
|
|
1745
|
+
def note_failure(endpoint)
|
|
1746
|
+
@mutex.synchronize do
|
|
1747
|
+
m = @members[endpoint]
|
|
1748
|
+
next if m.nil?
|
|
1749
|
+
|
|
1750
|
+
m.errors += 1
|
|
1751
|
+
eject(m, "#{m.errors} consecutive connection failures") if m.errors >= @eject_errors && m.ejected_until.nil?
|
|
1752
|
+
end
|
|
1753
|
+
end
|
|
1754
|
+
|
|
1755
|
+
# Internal: +sql+ succeeded on +conn+; a write pins the connection's
|
|
1756
|
+
# affinity key to its member for +read_your_writes+ seconds.
|
|
1757
|
+
def note_statement(conn, sql)
|
|
1758
|
+
return if @ryw <= 0 || !conn.pool_out || !Skaidb.write_statement?(sql)
|
|
1759
|
+
|
|
1760
|
+
t = now
|
|
1761
|
+
@mutex.synchronize do
|
|
1762
|
+
@affinity.select! { |_, (_, untl)| untl > t } if @affinity.length >= AFFINITY_SWEEP
|
|
1763
|
+
@affinity[conn.pool_affinity] = [conn.endpoint, t + @ryw]
|
|
1764
|
+
end
|
|
1765
|
+
end
|
|
1766
|
+
|
|
1767
|
+
# Internal: +conn+ was closed while checked out instead of returned.
|
|
1768
|
+
def note_closed(conn)
|
|
1769
|
+
@mutex.synchronize do
|
|
1770
|
+
next unless conn.pool_out
|
|
1771
|
+
|
|
1772
|
+
conn.pool_out = false
|
|
1773
|
+
m = @members[conn.pool_member]
|
|
1774
|
+
m.in_use -= 1 if m && m.in_use.positive?
|
|
1775
|
+
end
|
|
1776
|
+
end
|
|
1777
|
+
|
|
1778
|
+
private
|
|
1779
|
+
|
|
1780
|
+
def choice(value, allowed, name)
|
|
1781
|
+
v = value.to_s.downcase.to_sym if value.is_a?(String) || value.is_a?(Symbol)
|
|
1782
|
+
return v if allowed.include?(v)
|
|
1783
|
+
|
|
1784
|
+
names = allowed.map(&:inspect)
|
|
1785
|
+
raise ArgumentError, "unknown #{name} #{value.inspect} (use #{names[0..-2].join(', ')} or #{names[-1]})"
|
|
1786
|
+
end
|
|
1787
|
+
|
|
1788
|
+
def now
|
|
1789
|
+
Process.clock_gettime(Process::CLOCK_MONOTONIC)
|
|
1790
|
+
end
|
|
1791
|
+
|
|
1792
|
+
def close_quietly(conn)
|
|
1793
|
+
conn.close
|
|
1794
|
+
rescue StandardError
|
|
1795
|
+
nil
|
|
1796
|
+
end
|
|
1797
|
+
|
|
1798
|
+
def log(message)
|
|
1799
|
+
@logger&.warn("skaidb pool: #{message}")
|
|
1800
|
+
end
|
|
1801
|
+
|
|
1802
|
+
# -- choosing a member (callers hold @mutex) --
|
|
1803
|
+
|
|
1804
|
+
def usable?(m)
|
|
1805
|
+
m.ejected_until.nil? && m.listed == :ready
|
|
1806
|
+
end
|
|
1807
|
+
|
|
1808
|
+
# The member this checkout should go to, or nil for "any" (the random
|
|
1809
|
+
# policy, or no usable member at all).
|
|
1810
|
+
def pick(affinity)
|
|
1811
|
+
if @ryw.positive? && (hit = @affinity[affinity])
|
|
1812
|
+
ep, untl = hit
|
|
1813
|
+
m = @members[ep]
|
|
1814
|
+
if now >= untl || m.nil?
|
|
1815
|
+
@affinity.delete(affinity)
|
|
1816
|
+
elsif usable?(m)
|
|
1817
|
+
return m
|
|
1818
|
+
end
|
|
1819
|
+
end
|
|
1820
|
+
usable = @members.values.select { |m| usable?(m) }
|
|
1821
|
+
return nil if usable.empty? || @policy == :random
|
|
1822
|
+
|
|
1823
|
+
if @policy == :round_robin
|
|
1824
|
+
usable.sort_by! { |m| [m.host, m.port] }
|
|
1825
|
+
@rr = (@rr + 1) % usable.length
|
|
1826
|
+
return usable[@rr]
|
|
1827
|
+
end
|
|
1828
|
+
return usable[0] if usable.length == 1
|
|
1829
|
+
|
|
1830
|
+
a, b = usable.sample(2)
|
|
1831
|
+
score(a) <= score(b) ? a : b
|
|
1832
|
+
end
|
|
1833
|
+
|
|
1834
|
+
# A member never measured, or not lately, scores 0: it is tried and
|
|
1835
|
+
# measured.
|
|
1836
|
+
def score(m)
|
|
1837
|
+
return 0.0 if m.ewma.nil? || now - m.last_sample > STALE_SAMPLE
|
|
1838
|
+
|
|
1839
|
+
m.ewma * (m.in_use + 1)
|
|
1840
|
+
end
|
|
1841
|
+
|
|
1842
|
+
# Remove and return an idle connection (to +target+, or to any usable
|
|
1843
|
+
# member), plus the idle connections found to belong to members that are
|
|
1844
|
+
# no longer usable (to close outside the lock).
|
|
1845
|
+
def take_idle(target)
|
|
1846
|
+
stale = []
|
|
1847
|
+
found = nil
|
|
1848
|
+
indexes = @fifo ? (0...@idle.length) : (@idle.length - 1).downto(0)
|
|
1849
|
+
indexes.each do |i|
|
|
1850
|
+
m = @members[@idle[i].pool_member]
|
|
1851
|
+
if m.nil? || !usable?(m)
|
|
1852
|
+
stale << i
|
|
1853
|
+
elsif target.nil? || m.equal?(target)
|
|
1854
|
+
found = i
|
|
1855
|
+
break
|
|
1856
|
+
end
|
|
1857
|
+
end
|
|
1858
|
+
picked = found && @idle[found]
|
|
1859
|
+
stale_conns = stale.map { |i| @idle[i] }
|
|
1860
|
+
(found ? stale + [found] : stale).sort.reverse_each { |i| @idle.delete_at(i) }
|
|
1861
|
+
[picked, stale_conns]
|
|
1862
|
+
end
|
|
1863
|
+
|
|
1864
|
+
# [host, port] pairs to dial, best first: +first+, the other usable
|
|
1865
|
+
# members in random order, then the rest (resyncing/unreachable before
|
|
1866
|
+
# ejected, soonest re-admission first) — a dial never has nowhere to go
|
|
1867
|
+
# while any member is configured. +exclude+ goes last.
|
|
1868
|
+
def order_for(first, exclude = nil)
|
|
1869
|
+
all = @members.values
|
|
1870
|
+
usable = all.select { |m| !m.equal?(first) && usable?(m) }.shuffle
|
|
1871
|
+
rest = all.select { |m| !m.equal?(first) && !usable?(m) }
|
|
1872
|
+
rest = rest.sort_by { |m| m.ejected_until ? [1, m.ejected_until] : [0, 0.0] }
|
|
1873
|
+
order = (first ? [first] : []) + usable + rest
|
|
1874
|
+
lost = exclude && order.length > 1 && order.find { |m| m.endpoint == exclude }
|
|
1875
|
+
order = order.reject { |m| m.equal?(lost) } + [lost] if lost
|
|
1876
|
+
order.map { |m| [m.host, m.port] }
|
|
1877
|
+
end
|
|
1878
|
+
|
|
1879
|
+
# -- ejection and re-admission --
|
|
1880
|
+
|
|
1881
|
+
# Take +m+ out of rotation (caller holds @mutex) — unless it is the last
|
|
1882
|
+
# usable member: a slow or flaky member beats none.
|
|
1883
|
+
def eject(m, why)
|
|
1884
|
+
return unless @members.each_value.any? { |o| !o.equal?(m) && usable?(o) }
|
|
1885
|
+
|
|
1886
|
+
t = now
|
|
1887
|
+
m.ejections = 0 if t - m.readmitted_at > @backoff_max # healthy for a while: start over
|
|
1888
|
+
m.ejections += 1
|
|
1889
|
+
m.ejected_until = t + backoff_for(m)
|
|
1890
|
+
log("ejecting #{m.endpoint} (#{why})")
|
|
1891
|
+
ensure_maintainer
|
|
1892
|
+
end
|
|
1893
|
+
|
|
1894
|
+
def backoff_for(m)
|
|
1895
|
+
[@backoff * (2**[m.ejections - 1, 0].max), @backoff_max].min
|
|
1896
|
+
end
|
|
1897
|
+
|
|
1898
|
+
# Start the background thread (caller holds @mutex): it re-probes ejected
|
|
1899
|
+
# members and refreshes discovery, and ends when the pool closes. Ruby
|
|
1900
|
+
# never waits for it at exit.
|
|
1901
|
+
def ensure_maintainer
|
|
1902
|
+
return if @maintainer || @closed
|
|
1903
|
+
|
|
1904
|
+
@maintainer = Thread.new { maintain }
|
|
1905
|
+
@maintainer.name = "skaidb-pool"
|
|
1906
|
+
end
|
|
1907
|
+
|
|
1908
|
+
def maintain
|
|
1909
|
+
loop do
|
|
1910
|
+
due = []
|
|
1911
|
+
refresh = false
|
|
1912
|
+
@mutex.synchronize do
|
|
1913
|
+
@wake.wait(@mutex, MAINTAIN_TICK) unless @closed
|
|
1914
|
+
return if @closed
|
|
1915
|
+
|
|
1916
|
+
t = now
|
|
1917
|
+
due = @members.each_value.select { |m| m.ejected_until && t >= m.ejected_until && !m.probing }
|
|
1918
|
+
due.each { |m| m.probing = true }
|
|
1919
|
+
refresh = @discover && !@discovery_off && t >= @next_refresh
|
|
1920
|
+
end
|
|
1921
|
+
due.each { |m| probe(m) }
|
|
1922
|
+
refresh_members if refresh
|
|
1923
|
+
end
|
|
1924
|
+
end
|
|
1925
|
+
|
|
1926
|
+
# Dial +m+ alone and run one statement: success re-admits it (the probe
|
|
1927
|
+
# connection joins the idle set), failure ejects it again for twice as
|
|
1928
|
+
# long.
|
|
1929
|
+
def probe(m)
|
|
1930
|
+
conn = nil
|
|
1931
|
+
ok = begin
|
|
1932
|
+
conn = Connection.new(**@conn_opts, endpoints: [[m.host, m.port]])
|
|
1933
|
+
conn.exec("SHOW DATABASES")
|
|
1934
|
+
true
|
|
1935
|
+
rescue StandardError
|
|
1936
|
+
false
|
|
1937
|
+
end
|
|
1938
|
+
keep = false
|
|
1939
|
+
@mutex.synchronize do
|
|
1940
|
+
m.probing = false
|
|
1941
|
+
if ok && !@closed
|
|
1942
|
+
m.ejected_until = nil
|
|
1943
|
+
m.errors = 0
|
|
1944
|
+
m.ewma = nil
|
|
1945
|
+
m.samples = 0
|
|
1946
|
+
m.readmitted_at = now
|
|
1947
|
+
log("re-admitted #{m.endpoint}")
|
|
1948
|
+
if @idle.length < @maxsize
|
|
1949
|
+
conn.observer = self
|
|
1950
|
+
conn.pool_member = m.endpoint
|
|
1951
|
+
@idle.push(conn)
|
|
1952
|
+
keep = true
|
|
1953
|
+
end
|
|
1954
|
+
else
|
|
1955
|
+
m.ejections += 1
|
|
1956
|
+
m.ejected_until = now + backoff_for(m)
|
|
1957
|
+
end
|
|
1958
|
+
end
|
|
1959
|
+
close_quietly(conn) if conn && !keep
|
|
1960
|
+
end
|
|
1961
|
+
|
|
1962
|
+
# -- discovery --
|
|
1963
|
+
|
|
1964
|
+
# Run discovery on a freshly dialed connection when the member it reached
|
|
1965
|
+
# has not named itself yet (once per member, so seeds given as host names
|
|
1966
|
+
# are matched to the endpoints the server reports).
|
|
1967
|
+
def discover_through(conn)
|
|
1968
|
+
return if !@discover || @discovery_off
|
|
1969
|
+
|
|
1970
|
+
ask = @mutex.synchronize do
|
|
1971
|
+
m = @members[conn.pool_member]
|
|
1972
|
+
next false if m.nil? || m.canonical
|
|
1973
|
+
|
|
1974
|
+
ensure_maintainer
|
|
1975
|
+
true
|
|
1976
|
+
end
|
|
1977
|
+
list_through(conn) if ask
|
|
1978
|
+
end
|
|
1979
|
+
|
|
1980
|
+
def refresh_members
|
|
1981
|
+
conn = begin
|
|
1982
|
+
checkout
|
|
1983
|
+
rescue StandardError
|
|
1984
|
+
@mutex.synchronize { @next_refresh = now + @refresh_interval }
|
|
1985
|
+
return
|
|
1986
|
+
end
|
|
1987
|
+
begin
|
|
1988
|
+
list_through(conn)
|
|
1989
|
+
ensure
|
|
1990
|
+
checkin(conn)
|
|
1991
|
+
end
|
|
1992
|
+
end
|
|
1993
|
+
|
|
1994
|
+
def list_through(conn)
|
|
1995
|
+
rows = begin
|
|
1996
|
+
conn.exec("SHOW ENDPOINTS").rows
|
|
1997
|
+
rescue QueryError
|
|
1998
|
+
# A server that predates the statement: keep the seeds.
|
|
1999
|
+
@mutex.synchronize { @discovery_off = true }
|
|
2000
|
+
return
|
|
2001
|
+
rescue Error
|
|
2002
|
+
return
|
|
2003
|
+
end
|
|
2004
|
+
if rows.empty?
|
|
2005
|
+
# A standalone server: nothing to discover, ever.
|
|
2006
|
+
@mutex.synchronize { @discovery_off = true }
|
|
2007
|
+
return
|
|
2008
|
+
end
|
|
2009
|
+
@mutex.synchronize do
|
|
2010
|
+
apply_listing(conn.pool_member, rows)
|
|
2011
|
+
interval = @refresh_interval
|
|
2012
|
+
if @members.each_value.any? { |m| m.listed == :resyncing || m.listed == :unreachable }
|
|
2013
|
+
interval = [interval, REFRESH_NOT_READY].min
|
|
2014
|
+
end
|
|
2015
|
+
@next_refresh = now + interval
|
|
2016
|
+
end
|
|
2017
|
+
end
|
|
2018
|
+
|
|
2019
|
+
# Fold a non-empty SHOW ENDPOINTS answer (caller holds @mutex).
|
|
2020
|
+
def apply_listing(dialed, rows)
|
|
2021
|
+
listed = {}
|
|
2022
|
+
rows.each do |endpoint, state, local|
|
|
2023
|
+
listed[endpoint] = state
|
|
2024
|
+
next unless local
|
|
2025
|
+
|
|
2026
|
+
me = @members[dialed]
|
|
2027
|
+
me.canonical = endpoint if me
|
|
2028
|
+
end
|
|
2029
|
+
# A member reported under the name of one already known (a seed given
|
|
2030
|
+
# as a host name) is that member, not a new one.
|
|
2031
|
+
by_name = {}
|
|
2032
|
+
@members.each_value do |m|
|
|
2033
|
+
by_name[m.canonical] = m if m.canonical && (!by_name.key?(m.canonical) || m.seed)
|
|
2034
|
+
end
|
|
2035
|
+
@members.delete_if { |_, m| m.canonical && !by_name[m.canonical].equal?(m) } # a duplicate of a seed
|
|
2036
|
+
seen = {}.compare_by_identity
|
|
2037
|
+
listed.each do |endpoint, state|
|
|
2038
|
+
m = by_name[endpoint]
|
|
2039
|
+
if m.nil?
|
|
2040
|
+
host, port = Skaidb.parse_endpoint(endpoint, @port)
|
|
2041
|
+
m = (@members["#{host}:#{port}"] ||= Member.new(host, port, false))
|
|
2042
|
+
m.canonical = endpoint
|
|
2043
|
+
end
|
|
2044
|
+
state = %w[ready resyncing unreachable].include?(state) ? state.to_sym : :ready
|
|
2045
|
+
if state == :ready && m.listed != :ready
|
|
2046
|
+
# Back in rotation: measured afresh, like a re-admission.
|
|
2047
|
+
m.ewma = nil
|
|
2048
|
+
m.samples = 0
|
|
2049
|
+
m.errors = 0
|
|
2050
|
+
end
|
|
2051
|
+
m.listed = state
|
|
2052
|
+
seen[m] = true
|
|
2053
|
+
end
|
|
2054
|
+
@members.delete_if do |_, m|
|
|
2055
|
+
next false if seen[m] || (m.seed && m.canonical.nil?) # listed, or a seed not identified yet
|
|
2056
|
+
|
|
2057
|
+
m.listed = :unlisted if m.seed # configured, but no longer a member
|
|
2058
|
+
!m.seed
|
|
1448
2059
|
end
|
|
1449
2060
|
end
|
|
1450
2061
|
end
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: skaidb
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 1.
|
|
4
|
+
version: 1.2.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- skaidb contributors
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2026-
|
|
11
|
+
date: 2026-10-25 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: bigdecimal
|