tuber 0.6.0 → 0.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 28e2397d8368c54a6711fa1db82b5c54ea649a4113e4da808b9dbc6db9cec1cb
4
- data.tar.gz: dee769c5e37ff64726e1563150af3a25e2dc3f65405844f58b76bafbf4f7148f
3
+ metadata.gz: a1d30e48b1dd0a0e43c0d5b028605ba85481353fcfcd0c8b4429cee30f55ae5a
4
+ data.tar.gz: 21e6f5e9e165c513b1b55d482b828d2f0d1e03f545667466f4a153fc131972f6
5
5
  SHA512:
6
- metadata.gz: 48a99c28b12b3f83cd1f24f743fabda6f06a97802c3e2633bb805f7309a3574508d45b3ba7c2ce1cfd7842b1f92159115e19f7331362c79d662c687f0b60af75
7
- data.tar.gz: 60fa0ea0b9d9440ba798bf953f640f9a6c654bf3988bdca58bc51e037bfa347fefc606964c81a236335467cd5441ce4344ef9382a725e2920d69d8a871ba996f
6
+ metadata.gz: 4160816477011c902e40ef12b601ffda5d155612e4b1a98c3cb36bbab335b1f9b28e7cfefb8d7b8b74278c54c54eeace1d7f75fe1e5a0f3bc700af83646251eb
7
+ data.tar.gz: 5b299b53bc6cab47d96d2afc4dd532517000025d35659480977b871c912e013a4f464e8c718efda9b2f50b9e51435a9381b98f180f729e2f4433052271927352
data/CHANGELOG.md CHANGED
@@ -4,6 +4,31 @@ The tuber gem is a fork of [beaneater](https://github.com/beanstalkd/beaneater),
4
4
  beaneater 1.1.4. Entries from 1.1.4 down are beaneater's history, kept for reference.
5
5
  (0.0.1 was an empty placeholder published to claim the gem name.)
6
6
 
7
+ ## 0.8.0 (November 22 2026)
8
+
9
+ Surviving a server whose host vanishes. A consumer blocked in `reserve` used to
10
+ wait forever on a connection whose server had gone without closing it, as after
11
+ a host reboot. It now notices in about a minute, reconnects and carries on.
12
+
13
+ * Turn on TCP keepalive by default, so a consumer blocked in `reserve` notices a server whose host vanished without closing the connection (a reboot that takes the network down first, a power cut, a partition) in about 60 seconds, instead of waiting forever on a half-open socket. Tune it with `keepalive_idle` (30), `keepalive_interval` (10) and `keepalive_count` (3), or turn it off with `keepalive = false`. (#2)
14
+ * Heal the connection after `Errno::ETIMEDOUT` (a failed keepalive) and `IO::TimeoutError` (a read or write timeout), as for a dropped socket: an idempotent command such as `reserve` reconnects and is re-sent, and the caller never sees the error. A timeout is re-sent at most once. A second one on a fresh socket means the server is up but not answering, and is raised.
15
+ * Fix `read_timeout` and `write_timeout`, which did nothing on Ruby 3.0 and newer: Ruby's sockets are non-blocking from 3.0 on, and ignore `SO_RCVTIMEO`/`SO_SNDTIMEO`. They now use `IO#timeout` on Ruby 3.2 and newer. Older Rubies keep the `setsockopt` fallback. Behaviour change: with `read_timeout` set, a reply that takes longer now raises `IO::TimeoutError` after one re-send. `reserve-with-timeout` and `reserve_batch` with a timeout get their own timeout added to `read_timeout`, and a plain `reserve` has no read deadline.
16
+
17
+ ## 0.7.0 (October 11 2026)
18
+
19
+ Recurring schedules. A Tuber with the scheduler runs an app's recurring jobs
20
+ itself, so the app can drop its scheduler process. This release adds the client
21
+ side. Nothing changes for code that doesn't call it.
22
+
23
+ * Add `Tuber#schedules`: `sync(owner) { |s| s.add ... }` replaces an owner's whole set of entries in one step, and `list` and `stats` show what the server holds. `s.add` takes `cron:` (with `zone:`), `every:` (with `offset:`) or a raw `spec:`, the put options with their usual defaults and value shapes, and `catchup:`. See "Recurring Schedules" in the README.
24
+ * `sync` refuses an empty set unless given `allow_empty: true`. An empty sync removes every entry the owner has, and an app that loaded an empty config by mistake shouldn't wipe its schedule.
25
+ * `sync` runs on a short-lived connection of its own, closed afterwards. A worker blocked in `reserve` on the client's connection can't hold it up, and an error partway can't leave that connection mid-sync. If the socket drops, the whole sync is re-run from the start, never resumed partway.
26
+ * Add `Tuber::ScheduleSyncError` for a sync tuber refuses. It names the owner, the refused entry (`#entry`) and tuber's reply (`#status`). The gem aborts the sync before raising, so the previous schedule keeps running. `#applied?` is true for the one exception: a commit that took effect but couldn't be written to disk.
27
+ * `schedules.stats` returns a `StatStruct`, like tube and job stats: `stats.next_fire` or `stats["next-fire"]`. Times come back as `Time` in the entry's own zone, counters as Integers, and empty fields as nil.
28
+ * Add `Connection#transmit_raw`, which returns an `OK <bytes>` body unparsed. `transmit`'s YAML handling rewrites `:`, `-` and `~` inside values, and schedule stats are full of them.
29
+ * Fix a put refused with `OUT_OF_STORAGE` coming back as a result hash, as though it had succeeded. Tuber sends that reply once a persistent server's `--max-storage-bytes` budget is spent. It now raises the new `Tuber::OutOfStorageError`, as `OUT_OF_MEMORY` raises `OutOfMemoryError`.
30
+ * `Tuber::UnexpectedResponse.new` takes an optional message as a third argument.
31
+
7
32
  ## 0.6.0 (September 16 2026)
8
33
 
9
34
  Reconnection release. The gem retried reconnects but not connects, so a
data/README.md CHANGED
@@ -72,6 +72,10 @@ Tuber.configure do |config|
72
72
  # config.resolv_timeout = nil
73
73
  # config.read_timeout = nil
74
74
  # config.write_timeout = nil
75
+ # config.keepalive = true
76
+ # config.keepalive_idle = 30
77
+ # config.keepalive_interval = 10
78
+ # config.keepalive_count = 3
75
79
  # config.connect_retries = 0
76
80
  # config.connect_retry_interval = 1
77
81
  end
@@ -79,7 +83,11 @@ end
79
83
 
80
84
  The above options are all defaults, so only include a configuration block if you need to make changes.
81
85
 
82
- `connect_timeout` and `resolv_timeout` are passed through to `TCPSocket.new` on Ruby 3.0 and newer (ignored on Ruby < 3.0 for compatibility). `read_timeout` and `write_timeout` apply socket read and write timeouts via `setsockopt`.
86
+ `connect_timeout` and `resolv_timeout` are passed through to `TCPSocket.new` on Ruby 3.0 and newer (ignored on Ruby < 3.0 for compatibility).
87
+
88
+ `read_timeout` is how long to wait for a reply before raising `IO::TimeoutError`, *beyond* any wait the command asks for: `reserve-with-timeout 30` and `reserve_batch(10, 30)` get their 30 seconds added on, and a plain `reserve` has no client deadline at all. `write_timeout` is how long a write may block on a server that has stopped reading. Both need Ruby 3.2 or newer (`IO#timeout`). On older Rubies they fall back to `SO_RCVTIMEO`/`SO_SNDTIMEO`, which Ruby's non-blocking sockets ignore from 3.0 on.
89
+
90
+ `keepalive` turns on TCP keepalive (see "Surviving a server restart" below). The `keepalive_*` timings are in seconds, and each is set only where the platform has the option.
83
91
 
84
92
  ### Connection
85
93
 
@@ -115,6 +123,16 @@ end
115
123
  down) and reconnects on a client you already hold. It defaults to `0` — a
116
124
  single attempt — which is the historical behaviour.
117
125
 
126
+ A consumer blocked in `reserve` is waiting on the server, so it only notices
127
+ the socket has died if the server's host says so. A clean restart closes the
128
+ connection, but when the host vanishes without a word — a reboot that takes the
129
+ network down before the server, a power cut, a partition — the old connection
130
+ is left half-open. TCP keepalive (on by default) probes an idle connection after
131
+ `keepalive_idle` seconds, every `keepalive_interval` seconds, and gives up after
132
+ `keepalive_count` unanswered probes. The blocked read then fails (about 60
133
+ seconds with the defaults) and the connection heals as above: the `reserve` is
134
+ re-sent on a fresh socket, and the caller never sees the error.
135
+
118
136
  When a command does surface `Tuber::NotConnected`, heal the client you have
119
137
  rather than building a new one:
120
138
 
@@ -371,6 +389,98 @@ lower than expected means jobs hit their TTR and went back to the queue while th
371
389
  worker was busy — otherwise invisible, since nothing notifies a worker that it
372
390
  lost a job.
373
391
 
392
+ ### Recurring Schedules (Tuber only)
393
+
394
+ Tuber can run an app's recurring jobs itself, so the app needs no scheduler
395
+ process (whenever, rufus-scheduler, sidekiq-cron, a `bin/scheduler` container).
396
+ The app sends its complete schedule under a name of its choosing, the *owner*,
397
+ and tuber puts each entry's body into its tube at the times the entry names. A
398
+ fire is an ordinary put: workers reserve, delete and bury it like any other job.
399
+
400
+ ```ruby
401
+ tuber = Tuber.new
402
+
403
+ result = tuber.schedules.sync("splat") do |s|
404
+ s.add "monitor_evaluation", tube: "splat.checkins", every: 60,
405
+ body: { class: "Monitors::EvaluateJob", args: [] }.to_json,
406
+ idp: "monitor_evaluation", pri: 1024
407
+ s.add "histogram_rollup", tube: "splat.maintenance", cron: "5 * * * *",
408
+ body: { class: "Analytics::HistogramRollupJob", args: [] }.to_json,
409
+ idp: "histogram_rollup", con: "maintenance", pri: 1024
410
+ s.add "data_retention", tube: "splat.maintenance",
411
+ cron: "0 2 * * *", zone: "Australia/Melbourne",
412
+ body: { class: "Maintenance::RetentionJob", args: [] }.to_json,
413
+ idp: "data_retention", con: "maintenance", pri: 1024
414
+ end
415
+ result.to_h # => {added: 3, changed: 0, removed: 0, unchanged: 0}
416
+ ```
417
+
418
+ - **Send the complete set every time.** An entry you leave out is removed. An
419
+ edited entry keeps its name and is updated in place. An identical sync changes
420
+ nothing, not even when entries next fire, so every replica may run it.
421
+ - **An empty set is refused** with `ArgumentError`, since it would remove every
422
+ entry the owner has. Pass `allow_empty: true` when that is what you mean.
423
+ - **Sync when a release starts, not on reconnect.** Use a release step (a Kamal
424
+ `pre-deploy` hook, a Heroku `release` phase) or sync once at process boot.
425
+ Tuber keeps schedules across its own restarts (with `-b`), and during a
426
+ rolling deploy an old process re-syncing after a reconnect would put back
427
+ entries the new release removed. The gem never syncs on its own.
428
+ - A sync uses a short-lived connection of its own, so a worker blocked in
429
+ `reserve` on the client's connection doesn't hold it up.
430
+
431
+ `s.add` takes the entry's name and:
432
+
433
+ | Option | |
434
+ | --- | --- |
435
+ | `tube:`, `body:` | Required. The body goes through `job_serializer`, as for `put` |
436
+ | `cron:` | 5-field cron or a macro (`"@daily"`); `zone:` adds an IANA zone (default UTC) |
437
+ | `every:` | Seconds, or `"10m"`, `"1h"`, `"1d"`, aligned to the Unix epoch; `offset:` shifts it |
438
+ | `spec:` | The spec as tuber reads it, instead of `cron:` or `every:` |
439
+ | `pri:`, `delay:`, `ttr:` | As for `put`, with the same defaults |
440
+ | `idp:`, `con:`, `grp:`, `aft:` | As for `put`: `idp: "key"` or `["key", ttl]`, `con: "key"` or `["key", limit]` |
441
+ | `catchup:` | For ticks missed while tuber was down: `:once` (default) fires once for the gap, an Integer fires it only if at most that many seconds late, `:never` skips it |
442
+
443
+ A `con:` value is a key, not a number: every job with `con: "maintenance"`
444
+ shares one slot, so entries that share it run one after another.
445
+
446
+ Anything tuber would refuse that the gem can see — a bad name, two specs, a body
447
+ that isn't a String — raises `ArgumentError` before anything is sent. When tuber
448
+ refuses the sync, it raises `Tuber::ScheduleSyncError`, and nothing changed: the
449
+ previous schedule keeps running. `#entry` names the refused entry (nil when tuber
450
+ refused the sync as a whole) and `#status` gives tuber's reply, such as
451
+ `BAD_SCHEDULE` for a zone typo like `Australia/Melborne`. The one exception is
452
+ `#applied?`: the commit took effect but tuber couldn't write it to disk
453
+ (`INTERNAL_ERROR`). It runs until tuber restarts, so sync again once the disk is
454
+ fixed.
455
+
456
+ ```ruby
457
+ begin
458
+ tuber.schedules.sync("splat") { |s| ... }
459
+ rescue Tuber::ScheduleSyncError => e
460
+ Rails.logger.error(e.message) # names the entry and the reason
461
+ end
462
+ ```
463
+
464
+ To see what tuber holds:
465
+
466
+ ```ruby
467
+ tuber.schedules.list # => [["splat", "data_retention"], ...] across owners
468
+ tuber.schedules.list("splat")
469
+ stats = tuber.schedules.stats("splat", "data_retention") # nil if there's no such entry
470
+ stats.next_fire # => 2026-10-12 02:00:00 +1100 (next_fire_2, next_fire_3 too)
471
+ stats.last_result # => "INSERTED 4711"
472
+ stats.buried_streak # => 0
473
+ ```
474
+
475
+ **`idp:` and buried jobs.** With `idp:`, a fire is skipped while the previous
476
+ fire's job is still in the tube, which is what you want when a run outlasts its
477
+ interval. But a *buried* job keeps its key too, so every later fire dedups
478
+ against it and queues nothing until it is kicked or deleted. `buried_streak`
479
+ counts those fires: alert when it is above 0.
480
+
481
+ Tuber's [scheduling guide](https://github.com/tuberq/tuber/blob/main/docs/scheduling.md)
482
+ covers spec syntax, daylight saving, catch-up and monitoring.
483
+
374
484
  ### Stats
375
485
 
376
486
  ```ruby
@@ -393,6 +503,13 @@ Tuber.configure do |config|
393
503
  # Connect attempts; see "Surviving a server restart" above.
394
504
  config.connect_retries = 0
395
505
  config.connect_retry_interval = 1
506
+
507
+ # TCP keepalive: notice a server whose host vanished in about
508
+ # idle + interval * count seconds. See "Surviving a server restart".
509
+ config.keepalive = true
510
+ config.keepalive_idle = 30
511
+ config.keepalive_interval = 10
512
+ config.keepalive_count = 3
396
513
  end
397
514
  ```
398
515
 
@@ -413,6 +530,8 @@ end
413
530
  | Tuber::NotFoundError | Job or tube not found. |
414
531
  | Tuber::TimedOutError | Reserve timed out. |
415
532
  | Tuber::JobNotReserved | Action requires a reserved job. |
533
+ | Tuber::OutOfStorageError | A persistent Tuber's disk budget (`--max-storage-bytes`) is spent. |
534
+ | Tuber::ScheduleSyncError | Tuber refused a schedule sync; see Recurring Schedules. |
416
535
 
417
536
  See the [Tuber protocol](https://github.com/tuberq/tuber/blob/main/docs/protocol.md) (a superset of the [beanstalk protocol](https://github.com/beanstalkd/beanstalkd/blob/master/doc/protocol.txt)) for additional error types.
418
537
 
@@ -12,8 +12,30 @@ class Tuber
12
12
  alias_method :beanstalkd_url=, :tuber_url=
13
13
  attr_accessor :connect_timeout # TCP connect timeout in seconds
14
14
  attr_accessor :resolv_timeout # DNS resolve timeout in seconds
15
- attr_accessor :read_timeout # socket read timeout in seconds
16
- attr_accessor :write_timeout # socket write timeout in seconds
15
+
16
+ # Seconds to wait for a reply before raising IO::TimeoutError, beyond any
17
+ # wait the command itself asks for: reserve-with-timeout and reserve-batch
18
+ # get their own timeout added on, and a plain reserve has no deadline at
19
+ # all. Needs Ruby >= 3.2 (IO#timeout); older Rubies fall back to
20
+ # SO_RCVTIMEO, which Ruby's non-blocking sockets ignore from 3.0 on.
21
+ attr_accessor :read_timeout
22
+
23
+ # Seconds a write may block (the server not reading) before raising
24
+ # IO::TimeoutError. Same Ruby caveat as read_timeout.
25
+ attr_accessor :write_timeout
26
+
27
+ # TCP keepalive, on by default. A blocked read waits on the server, so it
28
+ # never learns the server is gone if the server's host vanished without
29
+ # closing the connection: a reboot that takes the network down first, a
30
+ # power cut, a partition. Keepalive probes an idle connection after
31
+ # keepalive_idle seconds, every keepalive_interval seconds, and gives up
32
+ # after keepalive_count unanswered probes, so the read fails (and the
33
+ # connection heals) after about idle + interval * count seconds: 60 with
34
+ # the defaults.
35
+ attr_accessor :keepalive
36
+ attr_accessor :keepalive_idle # idle seconds before the first probe
37
+ attr_accessor :keepalive_interval # seconds between probes
38
+ attr_accessor :keepalive_count # unanswered probes before giving up
17
39
 
18
40
  # How hard a client tries to connect — "how long should this ride out a
19
41
  # server restart", in one knob.
@@ -41,6 +63,10 @@ class Tuber
41
63
  @resolv_timeout = nil
42
64
  @read_timeout = nil
43
65
  @write_timeout = nil
66
+ @keepalive = true
67
+ @keepalive_idle = 30
68
+ @keepalive_interval = 10
69
+ @keepalive_count = 3
44
70
  @connect_retries = 0
45
71
  # Matches Tuber::Connection::DEFAULT_RETRY_INTERVAL, which is not loaded yet.
46
72
  @connect_retry_interval = 1
@@ -78,6 +78,21 @@ class Tuber
78
78
  (defined?(IO::TimeoutError) ? [IO::TimeoutError] : [])
79
79
  ).freeze
80
80
 
81
+ # What a blocked read or write raises once the server is judged gone
82
+ # without having said so: Errno::ETIMEDOUT when keepalive probes go
83
+ # unanswered, IO::TimeoutError (Ruby >= 3.2) when read_timeout or
84
+ # write_timeout runs out. The socket is useless afterwards either way: a
85
+ # reply still in flight could arrive later and be read as the answer to
86
+ # the next command.
87
+ TIMEOUT_ERRORS = (
88
+ [Errno::ETIMEDOUT] + (defined?(IO::TimeoutError) ? [IO::TimeoutError] : [])
89
+ ).freeze
90
+
91
+ # Whether this Ruby has IO#timeout (3.2 and newer), the only read and
92
+ # write timeout its non-blocking sockets honour. SO_RCVTIMEO and
93
+ # SO_SNDTIMEO are kept for older Rubies but have no effect from 3.0 on.
94
+ IO_TIMEOUTS = IO.method_defined?(:timeout=)
95
+
81
96
  # Initializes new connection.
82
97
  #
83
98
  # @param [String] address beanstalkd instance address.
@@ -127,7 +142,7 @@ class Tuber
127
142
  _raise_not_connected! unless connection
128
143
 
129
144
  command = command.dup.force_encoding('ASCII-8BIT') if command.respond_to?(:force_encoding)
130
- connection.write(command.to_s + "\r\n")
145
+ _write(command.to_s + "\r\n", read_timeout: _read_timeout_for(command))
131
146
  res = connection.readline
132
147
  parse_response(command, res)
133
148
  end
@@ -152,7 +167,7 @@ class Tuber
152
167
  _raise_not_connected! unless connection
153
168
 
154
169
  cmd = timeout ? "reserve-batch #{count} #{timeout}" : "reserve-batch #{count}"
155
- connection.write(cmd + "\r\n")
170
+ _write(cmd + "\r\n", read_timeout: _read_timeout_for(cmd))
156
171
 
157
172
  header = connection.readline.chomp
158
173
  status, actual_count_str = header.split(/\s/, 2)
@@ -188,7 +203,7 @@ class Tuber
188
203
  _raise_not_connected! unless connection
189
204
 
190
205
  cmd = "delete-batch #{ids.join(' ')}"
191
- connection.write(cmd + "\r\n")
206
+ _write(cmd + "\r\n")
192
207
 
193
208
  res = connection.readline.chomp
194
209
  status, deleted, not_found = res.split(/\s/)
@@ -200,6 +215,81 @@ class Tuber
200
215
  end
201
216
  end
202
217
 
218
+ # Sends a command answered with `OK <bytes>` and returns the body
219
+ # unparsed. For replies whose values #parse_response would mangle:
220
+ # its Psych workaround rewrites `:`, `-` and `~` inside every value, and
221
+ # stats-schedule's times, specs and tags are full of them.
222
+ #
223
+ # @param [String] command A command whose success reply is `OK <bytes>`
224
+ # @return [String] The reply body (the YAML document), without its CRLF
225
+ # @raise [Tuber::UnexpectedResponse] Any other reply, e.g. NotFoundError
226
+ #
227
+ def transmit_raw(command)
228
+ _with_retry do
229
+ @mutex.synchronize do
230
+ _raise_not_connected! unless connection
231
+
232
+ _write(command + "\r\n")
233
+ status, bytes = connection.readline.chomp.split(/\s/, 2)
234
+ raise UnexpectedResponse.from_status(status, command) unless status == 'OK'
235
+
236
+ body = connection.read(bytes.to_i)
237
+ crlf = connection.read(2)
238
+ raise ExpectedCrlfError.new('EXPECTED_CRLF', command) unless crlf == "\r\n"
239
+ body.force_encoding(Encoding::UTF_8)
240
+ end
241
+ end
242
+ end
243
+
244
+ # Replaces +owner+'s recurring schedule with +entries+ in one exchange:
245
+ # schedule-begin, a schedule-put per entry, schedule-commit. The mutex is
246
+ # held throughout, so no other command interleaves with it on the socket.
247
+ #
248
+ # A dropped socket makes the server discard what was staged, so the whole
249
+ # exchange is re-run on a fresh connection — never resumed partway. That
250
+ # is safe because a sync is idempotent; if the drop came after the commit
251
+ # landed, the re-run reports every entry as unchanged.
252
+ #
253
+ # Run it on a connection of its own, and close that afterwards, as
254
+ # Tuber::Schedules#sync does: any other error partway can leave the server
255
+ # mid-sync on this socket with a reply unread.
256
+ #
257
+ # @param [String] owner The owner whose entries are replaced
258
+ # @param [Array<Tuber::Schedules::Entry>] entries The complete new set
259
+ # @return [Tuber::Schedules::SyncResult]
260
+ # @raise [Tuber::ScheduleSyncError] The server refused the sync
261
+ # @api private Use Tuber::Schedules#sync, which validates the entries
262
+ #
263
+ def schedule_sync(owner, entries)
264
+ _with_retry do
265
+ @mutex.synchronize do
266
+ _raise_not_connected! unless connection
267
+
268
+ cmd = "schedule-begin #{owner}"
269
+ reply = _round_trip(cmd)
270
+ raise ScheduleSyncError.new(reply[/\A\S*/], cmd, owner: owner) unless reply == "SYNCING #{owner}"
271
+
272
+ entries.each do |entry|
273
+ cmd = "schedule-put #{entry.line}"
274
+ reply = _round_trip(cmd, entry.body)
275
+ next if reply == 'STAGED'
276
+
277
+ # The refusal has already doomed the sync; the commit would only
278
+ # say ABORTED. Abort now so the error names the entry, and so the
279
+ # connection leaves sync mode before the caller sees it.
280
+ _round_trip('schedule-abort')
281
+ raise ScheduleSyncError.new(reply, cmd, owner: owner, entry: entry.name)
282
+ end
283
+
284
+ status, *counts = _round_trip('schedule-commit').split(' ')
285
+ unless status == 'SYNCED' && counts.size == 4
286
+ raise ScheduleSyncError.new(status.to_s, 'schedule-commit', owner: owner)
287
+ end
288
+ Schedules::SyncResult.new(*counts.map(&:to_i))
289
+ end
290
+ end
291
+ end
292
+
203
293
  # Close connection with beanstalkd server.
204
294
  #
205
295
  # @example
@@ -312,8 +402,11 @@ class Tuber
312
402
  end
313
403
 
314
404
  begin
315
- socket.setsockopt(Socket::SOL_SOCKET, Socket::SO_RCVTIMEO, _timeval_for(config.read_timeout)) if config.read_timeout
316
- socket.setsockopt(Socket::SOL_SOCKET, Socket::SO_SNDTIMEO, _timeval_for(config.write_timeout)) if config.write_timeout
405
+ _enable_keepalive(socket) if config.keepalive
406
+ unless IO_TIMEOUTS # IO#timeout is set per exchange instead, in #_write
407
+ socket.setsockopt(Socket::SOL_SOCKET, Socket::SO_RCVTIMEO, _timeval_for(config.read_timeout)) if config.read_timeout
408
+ socket.setsockopt(Socket::SOL_SOCKET, Socket::SO_SNDTIMEO, _timeval_for(config.write_timeout)) if config.write_timeout
409
+ end
317
410
  @connection = socket
318
411
  rescue
319
412
  socket.close rescue nil
@@ -393,6 +486,17 @@ class Tuber
393
486
  transmit("reserve-mode #{@reserve_mode}", init: false) if @reserve_mode && @reserve_mode.to_s != 'fifo'
394
487
  end
395
488
 
489
+ # Writes one command, and a body framed as for put when given, and reads
490
+ # the one-line reply. Callers hold the mutex.
491
+ #
492
+ # @return [String] The reply line without its CRLF
493
+ def _round_trip(command, body = nil)
494
+ frame = "#{command}\r\n".b
495
+ frame << body.b << "\r\n" if body
496
+ _write(frame)
497
+ connection.readline.chomp
498
+ end
499
+
396
500
  # Wrapper method for capturing certain failures and retry the payload block
397
501
  #
398
502
  # @param [Proc] block The command to execute.
@@ -403,22 +507,88 @@ class Tuber
403
507
  # @param [Integer] tries The maximum number of tries in draining mode
404
508
  # @return [Object] Result of the block passed
405
509
  #
510
+ # A timeout (see TIMEOUT_ERRORS) is retransmitted at most once. One fresh
511
+ # socket heals a connection whose server vanished; a second timeout on it
512
+ # means the server is up but not answering, and the caller hears so
513
+ # rather than waiting on a loop that never ends.
514
+ #
406
515
  def _with_retry(retry_interval: DEFAULT_RETRY_INTERVAL, init: true, tries: MAX_RETRIES, retransmit: true, &block)
407
- yield
408
- rescue EOFError, Errno::ECONNRESET, Errno::EPIPE,
409
- Errno::ECONNREFUSED => ex
410
- _reconnect(tries: _connect_tries, retry_interval: retry_interval)
411
- _initialize_tubes if init
412
- raise ex unless retransmit
413
- retry
414
- rescue Tuber::DrainingError
415
- tries -= 1
416
- if tries.zero?
417
- close
418
- raise
516
+ timed_out = false
517
+ begin
518
+ yield
519
+ rescue EOFError, Errno::ECONNRESET, Errno::EPIPE,
520
+ Errno::ECONNREFUSED, *TIMEOUT_ERRORS => ex
521
+ _reconnect(tries: _connect_tries, retry_interval: retry_interval)
522
+ _initialize_tubes if init
523
+ raise ex unless retransmit
524
+ if TIMEOUT_ERRORS.any? { |klass| ex.is_a?(klass) }
525
+ raise ex if timed_out
526
+ timed_out = true
527
+ end
528
+ retry
529
+ rescue Tuber::DrainingError
530
+ tries -= 1
531
+ if tries.zero?
532
+ close
533
+ raise
534
+ end
535
+ sleep(retry_interval)
536
+ retry
537
+ end
538
+ end
539
+
540
+ # Writes a command, and any body framed with it, then sets how long the
541
+ # reads of its reply may block. Callers hold the mutex.
542
+ #
543
+ # On Ruby >= 3.2 both limits are IO#timeout, swapped between the write and
544
+ # the reads; older Rubies get the SO_*TIMEO set at connect instead.
545
+ #
546
+ # @param [String] data The bytes to send
547
+ # @param [Numeric, nil] read_timeout Seconds each read of the reply may
548
+ # block, nil for no limit (see #_read_timeout_for)
549
+ def _write(data, read_timeout: config.read_timeout)
550
+ connection.timeout = config.write_timeout if IO_TIMEOUTS
551
+ connection.write(data)
552
+ connection.timeout = read_timeout if IO_TIMEOUTS
553
+ end
554
+
555
+ # How long the reply to +command+ may take: read_timeout, plus however
556
+ # long the command asks the server to hold it before answering. A plain
557
+ # reserve waits as long as it takes by design, so it has no deadline;
558
+ # keepalive still notices a server that has vanished under it.
559
+ #
560
+ # @param [String] command The command line sent
561
+ # @return [Numeric, nil] Seconds, or nil for no limit
562
+ def _read_timeout_for(command)
563
+ limit = config.read_timeout
564
+ return limit unless limit
565
+
566
+ words = command.to_s[/\A[^\r\n]*/].split
567
+ case words[0]
568
+ when 'reserve' then nil
569
+ when 'reserve-with-timeout' then limit + words[1].to_i
570
+ when 'reserve-batch' then limit + words[2].to_i # count, then timeout
571
+ else limit
572
+ end
573
+ end
574
+
575
+ # Turns on TCP keepalive with the configured probe timings, so a read
576
+ # blocked on a server that vanished without closing the connection fails
577
+ # with Errno::ETIMEDOUT (or ECONNRESET, if the host is back and refuses
578
+ # the probe) instead of waiting forever. Each timing is set only where the
579
+ # platform has the option; macOS calls the idle time TCP_KEEPALIVE.
580
+ #
581
+ # @param [TCPSocket] socket A freshly connected socket
582
+ def _enable_keepalive(socket)
583
+ socket.setsockopt(Socket::SOL_SOCKET, Socket::SO_KEEPALIVE, true)
584
+ {
585
+ (Socket.const_defined?(:TCP_KEEPIDLE) ? :TCP_KEEPIDLE : :TCP_KEEPALIVE) => config.keepalive_idle,
586
+ :TCP_KEEPINTVL => config.keepalive_interval,
587
+ :TCP_KEEPCNT => config.keepalive_count,
588
+ }.each do |name, value|
589
+ next unless value && Socket.const_defined?(name)
590
+ socket.setsockopt(Socket::IPPROTO_TCP, Socket.const_get(name), value.ceil)
419
591
  end
420
- sleep(retry_interval)
421
- retry
422
592
  end
423
593
 
424
594
  # Connect attempts to allow when healing a connection that was already
data/lib/tuber/errors.rb CHANGED
@@ -11,7 +11,7 @@ class Tuber
11
11
  # Abstract class for errors that occur when a command does not complete successfully.
12
12
  class UnexpectedResponse < RuntimeError
13
13
  # Set of status states that are considered errors
14
- ERROR_STATES = %w(OUT_OF_MEMORY INTERNAL_ERROR
14
+ ERROR_STATES = %w(OUT_OF_MEMORY OUT_OF_STORAGE INTERNAL_ERROR
15
15
  BAD_FORMAT UNKNOWN_COMMAND JOB_TOO_BIG DRAINING
16
16
  TIMED_OUT DEADLINE_SOON NOT_FOUND NOT_IGNORED EXPECTED_CRLF)
17
17
 
@@ -27,13 +27,14 @@ class Tuber
27
27
  #
28
28
  # @param [Tuber::UnexpectedResponse] status Unexpected response object
29
29
  # @param [String] cmd Beanstalkd request command
30
+ # @param [String, nil] message Overrides the default "Response failed with: <status>"
30
31
  #
31
32
  # @example
32
33
  # Tuber::UnexpectedResponse.new(NotFoundError, 'bury 123')
33
34
  #
34
- def initialize(status, cmd)
35
+ def initialize(status, cmd, message = nil)
35
36
  @status, @cmd = status, cmd
36
- super("Response failed with: #{status}")
37
+ super(message || "Response failed with: #{status}")
37
38
  end
38
39
 
39
40
  # Translate beanstalkd error status to ruby Exeception
@@ -55,6 +56,8 @@ class Tuber
55
56
 
56
57
  # Raises when the beanstalkd instance runs out of memory
57
58
  class OutOfMemoryError < UnexpectedResponse; end
59
+ # Raises when a persistent Tuber server's disk budget (--max-storage-bytes) is spent
60
+ class OutOfStorageError < UnexpectedResponse; end
58
61
  # Raises when the beanstalkd instance is draining and new jobs cannot be inserted
59
62
  class DrainingError < UnexpectedResponse; end
60
63
  # Raises when the job or tube cannot be found
@@ -75,4 +78,55 @@ class Tuber
75
78
  class TimedOutError < UnexpectedResponse; end
76
79
  # Raises when a tube could not be ignored because it is the last watched tube.
77
80
  class NotIgnoredError < UnexpectedResponse; end
81
+
82
+ # Raises when a Tuber server refuses a schedule sync (see Tuber::Schedules#sync).
83
+ #
84
+ # Unless #applied? is true, nothing changed: the owner's previous schedule
85
+ # is still the one in effect.
86
+ class ScheduleSyncError < UnexpectedResponse
87
+ # @!attribute owner
88
+ # @return [String] the owner whose sync was refused
89
+ # @!attribute entry
90
+ # @return [String, nil] the entry tuber refused, nil when it refused the
91
+ # sync as a whole (on begin or commit)
92
+ attr_reader :owner, :entry
93
+
94
+ # @param [String] status The server's reply, e.g. "BAD_SCHEDULE"
95
+ # @param [String] cmd The command it refused
96
+ # @param [String] owner The owner being synced
97
+ # @param [String, nil] entry The refused entry's name
98
+ def initialize(status, cmd, owner:, entry: nil)
99
+ @owner, @entry = owner, entry
100
+ super(status, cmd, describe(status, cmd, owner, entry))
101
+ end
102
+
103
+ # True when the new set is live despite the error: the commit took effect
104
+ # in memory but tuber couldn't write it to disk (INTERNAL_ERROR). A tuber
105
+ # restart before the next successful sync brings back the previous set.
106
+ #
107
+ # @return [Boolean]
108
+ def applied?
109
+ cmd == 'schedule-commit' && status == 'INTERNAL_ERROR'
110
+ end
111
+
112
+ private
113
+
114
+ def describe(status, cmd, owner, entry)
115
+ if entry
116
+ "tuber refused schedule #{entry.inspect} (#{status}): #{cmd}. " \
117
+ "Nothing changed: #{owner.inspect}'s previous schedule is still in effect."
118
+ elsif cmd == 'schedule-commit' && status == 'INTERNAL_ERROR'
119
+ "#{owner.inspect}'s new schedule is live, but tuber couldn't write it to disk " \
120
+ "(INTERNAL_ERROR). A tuber restart before the next successful sync brings back the previous one."
121
+ elsif cmd == 'schedule-commit'
122
+ "tuber refused #{owner.inspect}'s schedule on commit (#{status}). " \
123
+ "Nothing changed: the previous schedule is still in effect."
124
+ elsif status == 'UNKNOWN_COMMAND'
125
+ "this server has no recurring schedules (UNKNOWN_COMMAND to #{cmd.inspect}): " \
126
+ "it is beanstalkd, or a tuber older than the scheduler."
127
+ else
128
+ "tuber refused to begin a schedule sync for #{owner.inspect} (#{status})."
129
+ end
130
+ end
131
+ end
78
132
  end