tuber 0.6.0 → 0.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 28e2397d8368c54a6711fa1db82b5c54ea649a4113e4da808b9dbc6db9cec1cb
4
- data.tar.gz: dee769c5e37ff64726e1563150af3a25e2dc3f65405844f58b76bafbf4f7148f
3
+ metadata.gz: 3dad07a18da118a0c3f50cdb00de169fbc73c4691e7484c422d98bccd12b750e
4
+ data.tar.gz: eeb2247e67cf72d407c578f9a0e2e61820ebc3b3594ce6da39f90eb312ef23d4
5
5
  SHA512:
6
- metadata.gz: 48a99c28b12b3f83cd1f24f743fabda6f06a97802c3e2633bb805f7309a3574508d45b3ba7c2ce1cfd7842b1f92159115e19f7331362c79d662c687f0b60af75
7
- data.tar.gz: 60fa0ea0b9d9440ba798bf953f640f9a6c654bf3988bdca58bc51e037bfa347fefc606964c81a236335467cd5441ce4344ef9382a725e2920d69d8a871ba996f
6
+ metadata.gz: 476561b12a94b1017ec600d249236ad1296a3760c19c8fae126bace4ae26de1b9d448c54b74413dbf7805eb3ed91d8bf4ee53c17a148fe194705c43783e44e71
7
+ data.tar.gz: d3e34a35f30ee2ae578bdfb1f88c243585a2bbfcb9de4e0d76241d1839e35cb51715fcd5a9921ff5e002557c1d2f53f66aad1daea18c6114ae62687d43b74e58
data/CHANGELOG.md CHANGED
@@ -4,6 +4,21 @@ The tuber gem is a fork of [beaneater](https://github.com/beanstalkd/beaneater),
4
4
  beaneater 1.1.4. Entries from 1.1.4 down are beaneater's history, kept for reference.
5
5
  (0.0.1 was an empty placeholder published to claim the gem name.)
6
6
 
7
+ ## 0.7.0 (October 11 2026)
8
+
9
+ Recurring schedules. A Tuber with the scheduler runs an app's recurring jobs
10
+ itself, so the app can drop its scheduler process. This release adds the client
11
+ side. Nothing changes for code that doesn't call it.
12
+
13
+ * Add `Tuber#schedules`: `sync(owner) { |s| s.add ... }` replaces an owner's whole set of entries in one step, and `list` and `stats` show what the server holds. `s.add` takes `cron:` (with `zone:`), `every:` (with `offset:`) or a raw `spec:`, the put options with their usual defaults and value shapes, and `catchup:`. See "Recurring Schedules" in the README.
14
+ * `sync` refuses an empty set unless given `allow_empty: true`. An empty sync removes every entry the owner has, and an app that loaded an empty config by mistake shouldn't wipe its schedule.
15
+ * `sync` runs on a short-lived connection of its own, closed afterwards. A worker blocked in `reserve` on the client's connection can't hold it up, and an error partway can't leave that connection mid-sync. If the socket drops, the whole sync is re-run from the start, never resumed partway.
16
+ * Add `Tuber::ScheduleSyncError` for a sync tuber refuses. It names the owner, the refused entry (`#entry`) and tuber's reply (`#status`). The gem aborts the sync before raising, so the previous schedule keeps running. `#applied?` is true for the one exception: a commit that took effect but couldn't be written to disk.
17
+ * `schedules.stats` returns a `StatStruct`, like tube and job stats: `stats.next_fire` or `stats["next-fire"]`. Times come back as `Time` in the entry's own zone, counters as Integers, and empty fields as nil.
18
+ * Add `Connection#transmit_raw`, which returns an `OK <bytes>` body unparsed. `transmit`'s YAML handling rewrites `:`, `-` and `~` inside values, and schedule stats are full of them.
19
+ * Fix a put refused with `OUT_OF_STORAGE` coming back as a result hash, as though it had succeeded. Tuber sends that reply once a persistent server's `--max-storage-bytes` budget is spent. It now raises the new `Tuber::OutOfStorageError`, as `OUT_OF_MEMORY` raises `OutOfMemoryError`.
20
+ * `Tuber::UnexpectedResponse.new` takes an optional message as a third argument.
21
+
7
22
  ## 0.6.0 (September 16 2026)
8
23
 
9
24
  Reconnection release. The gem retried reconnects but not connects, so a
data/README.md CHANGED
@@ -371,6 +371,98 @@ lower than expected means jobs hit their TTR and went back to the queue while th
371
371
  worker was busy — otherwise invisible, since nothing notifies a worker that it
372
372
  lost a job.
373
373
 
374
+ ### Recurring Schedules (Tuber only)
375
+
376
+ Tuber can run an app's recurring jobs itself, so the app needs no scheduler
377
+ process (whenever, rufus-scheduler, sidekiq-cron, a `bin/scheduler` container).
378
+ The app sends its complete schedule under a name of its choosing, the *owner*,
379
+ and tuber puts each entry's body into its tube at the times the entry names. A
380
+ fire is an ordinary put: workers reserve, delete and bury it like any other job.
381
+
382
+ ```ruby
383
+ tuber = Tuber.new
384
+
385
+ result = tuber.schedules.sync("splat") do |s|
386
+ s.add "monitor_evaluation", tube: "splat.checkins", every: 60,
387
+ body: { class: "Monitors::EvaluateJob", args: [] }.to_json,
388
+ idp: "monitor_evaluation", pri: 1024
389
+ s.add "histogram_rollup", tube: "splat.maintenance", cron: "5 * * * *",
390
+ body: { class: "Analytics::HistogramRollupJob", args: [] }.to_json,
391
+ idp: "histogram_rollup", con: "maintenance", pri: 1024
392
+ s.add "data_retention", tube: "splat.maintenance",
393
+ cron: "0 2 * * *", zone: "Australia/Melbourne",
394
+ body: { class: "Maintenance::RetentionJob", args: [] }.to_json,
395
+ idp: "data_retention", con: "maintenance", pri: 1024
396
+ end
397
+ result.to_h # => {added: 3, changed: 0, removed: 0, unchanged: 0}
398
+ ```
399
+
400
+ - **Send the complete set every time.** An entry you leave out is removed. An
401
+ edited entry keeps its name and is updated in place. An identical sync changes
402
+ nothing, not even when entries next fire, so every replica may run it.
403
+ - **An empty set is refused** with `ArgumentError`, since it would remove every
404
+ entry the owner has. Pass `allow_empty: true` when that is what you mean.
405
+ - **Sync when a release starts, not on reconnect.** Use a release step (a Kamal
406
+ `pre-deploy` hook, a Heroku `release` phase) or sync once at process boot.
407
+ Tuber keeps schedules across its own restarts (with `-b`), and during a
408
+ rolling deploy an old process re-syncing after a reconnect would put back
409
+ entries the new release removed. The gem never syncs on its own.
410
+ - A sync uses a short-lived connection of its own, so a worker blocked in
411
+ `reserve` on the client's connection doesn't hold it up.
412
+
413
+ `s.add` takes the entry's name and:
414
+
415
+ | Option | |
416
+ | --- | --- |
417
+ | `tube:`, `body:` | Required. The body goes through `job_serializer`, as for `put` |
418
+ | `cron:` | 5-field cron or a macro (`"@daily"`); `zone:` adds an IANA zone (default UTC) |
419
+ | `every:` | Seconds, or `"10m"`, `"1h"`, `"1d"`, aligned to the Unix epoch; `offset:` shifts it |
420
+ | `spec:` | The spec as tuber reads it, instead of `cron:` or `every:` |
421
+ | `pri:`, `delay:`, `ttr:` | As for `put`, with the same defaults |
422
+ | `idp:`, `con:`, `grp:`, `aft:` | As for `put`: `idp: "key"` or `["key", ttl]`, `con: "key"` or `["key", limit]` |
423
+ | `catchup:` | For ticks missed while tuber was down: `:once` (default) fires once for the gap, an Integer fires it only if at most that many seconds late, `:never` skips it |
424
+
425
+ A `con:` value is a key, not a number: every job with `con: "maintenance"`
426
+ shares one slot, so entries that share it run one after another.
427
+
428
+ Anything tuber would refuse that the gem can see — a bad name, two specs, a body
429
+ that isn't a String — raises `ArgumentError` before anything is sent. When tuber
430
+ refuses the sync, it raises `Tuber::ScheduleSyncError`, and nothing changed: the
431
+ previous schedule keeps running. `#entry` names the refused entry (nil when tuber
432
+ refused the sync as a whole) and `#status` gives tuber's reply, such as
433
+ `BAD_SCHEDULE` for a zone typo like `Australia/Melborne`. The one exception is
434
+ `#applied?`: the commit took effect but tuber couldn't write it to disk
435
+ (`INTERNAL_ERROR`). It runs until tuber restarts, so sync again once the disk is
436
+ fixed.
437
+
438
+ ```ruby
439
+ begin
440
+ tuber.schedules.sync("splat") { |s| ... }
441
+ rescue Tuber::ScheduleSyncError => e
442
+ Rails.logger.error(e.message) # names the entry and the reason
443
+ end
444
+ ```
445
+
446
+ To see what tuber holds:
447
+
448
+ ```ruby
449
+ tuber.schedules.list # => [["splat", "data_retention"], ...] across owners
450
+ tuber.schedules.list("splat")
451
+ stats = tuber.schedules.stats("splat", "data_retention") # nil if there's no such entry
452
+ stats.next_fire # => 2026-10-12 02:00:00 +1100 (next_fire_2, next_fire_3 too)
453
+ stats.last_result # => "INSERTED 4711"
454
+ stats.buried_streak # => 0
455
+ ```
456
+
457
+ **`idp:` and buried jobs.** With `idp:`, a fire is skipped while the previous
458
+ fire's job is still in the tube, which is what you want when a run outlasts its
459
+ interval. But a *buried* job keeps its key too, so every later fire dedups
460
+ against it and queues nothing until it is kicked or deleted. `buried_streak`
461
+ counts those fires: alert when it is above 0.
462
+
463
+ Tuber's [scheduling guide](https://github.com/tuberq/tuber/blob/main/docs/scheduling.md)
464
+ covers spec syntax, daylight saving, catch-up and monitoring.
465
+
374
466
  ### Stats
375
467
 
376
468
  ```ruby
@@ -413,6 +505,8 @@ end
413
505
  | Tuber::NotFoundError | Job or tube not found. |
414
506
  | Tuber::TimedOutError | Reserve timed out. |
415
507
  | Tuber::JobNotReserved | Action requires a reserved job. |
508
+ | Tuber::OutOfStorageError | A persistent Tuber's disk budget (`--max-storage-bytes`) is spent. |
509
+ | Tuber::ScheduleSyncError | Tuber refused a schedule sync; see Recurring Schedules. |
416
510
 
417
511
  See the [Tuber protocol](https://github.com/tuberq/tuber/blob/main/docs/protocol.md) (a superset of the [beanstalk protocol](https://github.com/beanstalkd/beanstalkd/blob/master/doc/protocol.txt)) for additional error types.
418
512
 
@@ -200,6 +200,81 @@ class Tuber
200
200
  end
201
201
  end
202
202
 
203
+ # Sends a command answered with `OK <bytes>` and returns the body
204
+ # unparsed. For replies whose values #parse_response would mangle:
205
+ # its Psych workaround rewrites `:`, `-` and `~` inside every value, and
206
+ # stats-schedule's times, specs and tags are full of them.
207
+ #
208
+ # @param [String] command A command whose success reply is `OK <bytes>`
209
+ # @return [String] The reply body (the YAML document), without its CRLF
210
+ # @raise [Tuber::UnexpectedResponse] Any other reply, e.g. NotFoundError
211
+ #
212
+ def transmit_raw(command)
213
+ _with_retry do
214
+ @mutex.synchronize do
215
+ _raise_not_connected! unless connection
216
+
217
+ connection.write(command + "\r\n")
218
+ status, bytes = connection.readline.chomp.split(/\s/, 2)
219
+ raise UnexpectedResponse.from_status(status, command) unless status == 'OK'
220
+
221
+ body = connection.read(bytes.to_i)
222
+ crlf = connection.read(2)
223
+ raise ExpectedCrlfError.new('EXPECTED_CRLF', command) unless crlf == "\r\n"
224
+ body.force_encoding(Encoding::UTF_8)
225
+ end
226
+ end
227
+ end
228
+
229
+ # Replaces +owner+'s recurring schedule with +entries+ in one exchange:
230
+ # schedule-begin, a schedule-put per entry, schedule-commit. The mutex is
231
+ # held throughout, so no other command interleaves with it on the socket.
232
+ #
233
+ # A dropped socket makes the server discard what was staged, so the whole
234
+ # exchange is re-run on a fresh connection — never resumed partway. That
235
+ # is safe because a sync is idempotent; if the drop came after the commit
236
+ # landed, the re-run reports every entry as unchanged.
237
+ #
238
+ # Run it on a connection of its own, and close that afterwards, as
239
+ # Tuber::Schedules#sync does: any other error partway can leave the server
240
+ # mid-sync on this socket with a reply unread.
241
+ #
242
+ # @param [String] owner The owner whose entries are replaced
243
+ # @param [Array<Tuber::Schedules::Entry>] entries The complete new set
244
+ # @return [Tuber::Schedules::SyncResult]
245
+ # @raise [Tuber::ScheduleSyncError] The server refused the sync
246
+ # @api private Use Tuber::Schedules#sync, which validates the entries
247
+ #
248
+ def schedule_sync(owner, entries)
249
+ _with_retry do
250
+ @mutex.synchronize do
251
+ _raise_not_connected! unless connection
252
+
253
+ cmd = "schedule-begin #{owner}"
254
+ reply = _round_trip(cmd)
255
+ raise ScheduleSyncError.new(reply[/\A\S*/], cmd, owner: owner) unless reply == "SYNCING #{owner}"
256
+
257
+ entries.each do |entry|
258
+ cmd = "schedule-put #{entry.line}"
259
+ reply = _round_trip(cmd, entry.body)
260
+ next if reply == 'STAGED'
261
+
262
+ # The refusal has already doomed the sync; the commit would only
263
+ # say ABORTED. Abort now so the error names the entry, and so the
264
+ # connection leaves sync mode before the caller sees it.
265
+ _round_trip('schedule-abort')
266
+ raise ScheduleSyncError.new(reply, cmd, owner: owner, entry: entry.name)
267
+ end
268
+
269
+ status, *counts = _round_trip('schedule-commit').split(' ')
270
+ unless status == 'SYNCED' && counts.size == 4
271
+ raise ScheduleSyncError.new(status.to_s, 'schedule-commit', owner: owner)
272
+ end
273
+ Schedules::SyncResult.new(*counts.map(&:to_i))
274
+ end
275
+ end
276
+ end
277
+
203
278
  # Close connection with beanstalkd server.
204
279
  #
205
280
  # @example
@@ -393,6 +468,17 @@ class Tuber
393
468
  transmit("reserve-mode #{@reserve_mode}", init: false) if @reserve_mode && @reserve_mode.to_s != 'fifo'
394
469
  end
395
470
 
471
+ # Writes one command, and a body framed as for put when given, and reads
472
+ # the one-line reply. Callers hold the mutex.
473
+ #
474
+ # @return [String] The reply line without its CRLF
475
+ def _round_trip(command, body = nil)
476
+ frame = "#{command}\r\n".b
477
+ frame << body.b << "\r\n" if body
478
+ connection.write(frame)
479
+ connection.readline.chomp
480
+ end
481
+
396
482
  # Wrapper method for capturing certain failures and retry the payload block
397
483
  #
398
484
  # @param [Proc] block The command to execute.
data/lib/tuber/errors.rb CHANGED
@@ -11,7 +11,7 @@ class Tuber
11
11
  # Abstract class for errors that occur when a command does not complete successfully.
12
12
  class UnexpectedResponse < RuntimeError
13
13
  # Set of status states that are considered errors
14
- ERROR_STATES = %w(OUT_OF_MEMORY INTERNAL_ERROR
14
+ ERROR_STATES = %w(OUT_OF_MEMORY OUT_OF_STORAGE INTERNAL_ERROR
15
15
  BAD_FORMAT UNKNOWN_COMMAND JOB_TOO_BIG DRAINING
16
16
  TIMED_OUT DEADLINE_SOON NOT_FOUND NOT_IGNORED EXPECTED_CRLF)
17
17
 
@@ -27,13 +27,14 @@ class Tuber
27
27
  #
28
28
  # @param [Tuber::UnexpectedResponse] status Unexpected response object
29
29
  # @param [String] cmd Beanstalkd request command
30
+ # @param [String, nil] message Overrides the default "Response failed with: <status>"
30
31
  #
31
32
  # @example
32
33
  # Tuber::UnexpectedResponse.new(NotFoundError, 'bury 123')
33
34
  #
34
- def initialize(status, cmd)
35
+ def initialize(status, cmd, message = nil)
35
36
  @status, @cmd = status, cmd
36
- super("Response failed with: #{status}")
37
+ super(message || "Response failed with: #{status}")
37
38
  end
38
39
 
39
40
  # Translate beanstalkd error status to ruby Exeception
@@ -55,6 +56,8 @@ class Tuber
55
56
 
56
57
  # Raises when the beanstalkd instance runs out of memory
57
58
  class OutOfMemoryError < UnexpectedResponse; end
59
+ # Raises when a persistent Tuber server's disk budget (--max-storage-bytes) is spent
60
+ class OutOfStorageError < UnexpectedResponse; end
58
61
  # Raises when the beanstalkd instance is draining and new jobs cannot be inserted
59
62
  class DrainingError < UnexpectedResponse; end
60
63
  # Raises when the job or tube cannot be found
@@ -75,4 +78,55 @@ class Tuber
75
78
  class TimedOutError < UnexpectedResponse; end
76
79
  # Raises when a tube could not be ignored because it is the last watched tube.
77
80
  class NotIgnoredError < UnexpectedResponse; end
81
+
82
+ # Raises when a Tuber server refuses a schedule sync (see Tuber::Schedules#sync).
83
+ #
84
+ # Unless #applied? is true, nothing changed: the owner's previous schedule
85
+ # is still the one in effect.
86
+ class ScheduleSyncError < UnexpectedResponse
87
+ # @!attribute owner
88
+ # @return [String] the owner whose sync was refused
89
+ # @!attribute entry
90
+ # @return [String, nil] the entry tuber refused, nil when it refused the
91
+ # sync as a whole (on begin or commit)
92
+ attr_reader :owner, :entry
93
+
94
+ # @param [String] status The server's reply, e.g. "BAD_SCHEDULE"
95
+ # @param [String] cmd The command it refused
96
+ # @param [String] owner The owner being synced
97
+ # @param [String, nil] entry The refused entry's name
98
+ def initialize(status, cmd, owner:, entry: nil)
99
+ @owner, @entry = owner, entry
100
+ super(status, cmd, describe(status, cmd, owner, entry))
101
+ end
102
+
103
+ # True when the new set is live despite the error: the commit took effect
104
+ # in memory but tuber couldn't write it to disk (INTERNAL_ERROR). A tuber
105
+ # restart before the next successful sync brings back the previous set.
106
+ #
107
+ # @return [Boolean]
108
+ def applied?
109
+ cmd == 'schedule-commit' && status == 'INTERNAL_ERROR'
110
+ end
111
+
112
+ private
113
+
114
+ def describe(status, cmd, owner, entry)
115
+ if entry
116
+ "tuber refused schedule #{entry.inspect} (#{status}): #{cmd}. " \
117
+ "Nothing changed: #{owner.inspect}'s previous schedule is still in effect."
118
+ elsif cmd == 'schedule-commit' && status == 'INTERNAL_ERROR'
119
+ "#{owner.inspect}'s new schedule is live, but tuber couldn't write it to disk " \
120
+ "(INTERNAL_ERROR). A tuber restart before the next successful sync brings back the previous one."
121
+ elsif cmd == 'schedule-commit'
122
+ "tuber refused #{owner.inspect}'s schedule on commit (#{status}). " \
123
+ "Nothing changed: the previous schedule is still in effect."
124
+ elsif status == 'UNKNOWN_COMMAND'
125
+ "this server has no recurring schedules (UNKNOWN_COMMAND to #{cmd.inspect}): " \
126
+ "it is beanstalkd, or a tuber older than the scheduler."
127
+ else
128
+ "tuber refused to begin a schedule sync for #{owner.inspect} (#{status})."
129
+ end
130
+ end
131
+ end
78
132
  end
@@ -0,0 +1,288 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'time'
4
+
5
+ class Tuber
6
+ # Recurring schedules held by a Tuber server (beanstalkd has none): entries
7
+ # the server puts into a tube at the times a spec names, so an app needs no
8
+ # scheduler process of its own. A fire is an ordinary put — workers reserve
9
+ # the job as they would any other.
10
+ #
11
+ # An app owns its entries under a name of its choosing, the *owner*, and
12
+ # replaces the whole set with one #sync: entries it no longer sends are
13
+ # removed, an edited one is updated in place, and an unchanged one keeps its
14
+ # next fire. Every replica may sync; an identical set changes nothing.
15
+ #
16
+ # @example
17
+ # tuber.schedules.sync("splat") do |s|
18
+ # s.add "data_retention", tube: "splat.maintenance",
19
+ # cron: "0 2 * * *", zone: "Australia/Melbourne",
20
+ # body: { class: "DataRetentionJob", args: [] }.to_json, con: "maintenance"
21
+ # s.add "monitor_evaluation", tube: "splat.checkins", every: 60,
22
+ # idp: "monitor_evaluation", body: { class: "MonitorEvaluationJob", args: [] }.to_json
23
+ # end
24
+ # # => #<struct Tuber::Schedules::SyncResult added=2, changed=0, removed=0, unchanged=0>
25
+ #
26
+ class Schedules
27
+ # What a committed sync did to the owner's entries.
28
+ #
29
+ # If the connection dropped and the sync was re-run, the counts describe
30
+ # the last attempt only: a drop after the first attempt's commit landed
31
+ # makes the re-run report every entry as unchanged.
32
+ SyncResult = Struct.new(:added, :changed, :removed, :unchanged)
33
+
34
+ # One staged entry: its name, the `schedule-put` arguments, and the
35
+ # serialized body.
36
+ Entry = Struct.new(:name, :line, :body)
37
+
38
+ # Owners, entry names and tubes follow the server's tube-name rules.
39
+ NAME = %r{\A[A-Za-z0-9+/;.$_()][A-Za-z0-9\-+/;.$_()]*\z}
40
+
41
+ # @!attribute client
42
+ # @return [Tuber] returns the client instance
43
+ attr_reader :client
44
+
45
+ # @param [Tuber] client The tuber client instance.
46
+ def initialize(client)
47
+ @client = client
48
+ end
49
+
50
+ # Replaces +owner+'s entries with the ones the block adds, in one step.
51
+ #
52
+ # Send the complete schedule every time: an entry left out is deleted.
53
+ #
54
+ # Sync when a new release starts — a release step, or once at process
55
+ # boot — not on reconnect. Tuber keeps schedules across its own restarts
56
+ # (with -b), and during a rolling deploy an old process re-syncing would
57
+ # put back what the new release removed.
58
+ #
59
+ # @param [String] owner The app's name for its set of entries
60
+ # @param [Boolean] allow_empty A sync with no entries removes all of the
61
+ # owner's entries. Refused unless this is true, since an app that loaded
62
+ # an empty config by mistake would otherwise wipe its schedule.
63
+ # @yieldparam [Tuber::Schedules::Builder] s Call +s.add+ once per entry
64
+ # @return [Tuber::Schedules::SyncResult]
65
+ # @raise [Tuber::ScheduleSyncError] The server refused an entry or the
66
+ # sync. Nothing changed unless the error's #applied? says otherwise.
67
+ # @raise [ArgumentError] An entry is malformed, or the set is empty
68
+ # without +allow_empty+. Nothing is sent.
69
+ # @raise [Tuber::NotConnected] The sync's own connection couldn't be
70
+ # made (see Tuber::Configuration#connect_retries)
71
+ # @api public
72
+ def sync(owner, allow_empty: false)
73
+ owner = Schedules.name!("owner", owner)
74
+ builder = Builder.new
75
+ yield builder if block_given?
76
+ if builder.entries.empty? && !allow_empty
77
+ raise ArgumentError, "refusing to sync no entries for #{owner.inspect}: that removes " \
78
+ "all of its schedules. Pass allow_empty: true if that's meant."
79
+ end
80
+
81
+ # A connection of its own: a blocking reserve on the client's connection
82
+ # can't hold the sync up, and an error partway, which can leave the
83
+ # server mid-sync with a reply unread, is cleaned up by closing it.
84
+ connection = Connection.new(client.connection.address)
85
+ begin
86
+ connection.schedule_sync(owner, builder.entries)
87
+ ensure
88
+ connection.close
89
+ end
90
+ end
91
+
92
+ # The entries the server holds, as [owner, name] pairs.
93
+ #
94
+ # @param [String, nil] owner Only this owner's entries
95
+ # @return [Array<Array(String, String)>]
96
+ # @example
97
+ # tuber.schedules.list("splat") # => [["splat", "data_retention"], ["splat", "monitor_evaluation"]]
98
+ # @api public
99
+ def list(owner = nil)
100
+ owner = Schedules.name!("owner", owner) if owner
101
+ yaml = client.connection.transmit_raw(owner ? "list-schedules #{owner}" : "list-schedules")
102
+ yaml.each_line.grep(/\A- /).map { |line| line.chomp[2..-1].split(' ', 2) }
103
+ end
104
+
105
+ # One entry's definition, next three fires, last fire and counters.
106
+ #
107
+ # Times are Time objects at the entry's own zone offset, nil when there is
108
+ # none yet; counters are Integers. +buried_streak+ above 0 means fires are
109
+ # deduplicating against a buried job (through +idp:+) and queueing nothing.
110
+ #
111
+ # @param [String] owner The entry's owner
112
+ # @param [String] name The entry's name
113
+ # @return [Tuber::StatStruct, nil] nil if there is no such entry
114
+ # @example
115
+ # stats = tuber.schedules.stats("splat", "data_retention")
116
+ # stats.next_fire # => 2026-10-12 02:00:00 +1100
117
+ # stats["next-fire-2"] # => 2026-10-13 02:00:00 +1100
118
+ # stats.buried_streak # => 0
119
+ # @api public
120
+ def stats(owner, name)
121
+ owner = Schedules.name!("owner", owner)
122
+ name = Schedules.name!("name", name)
123
+ yaml = client.connection.transmit_raw("stats-schedule #{owner} #{name}")
124
+ stats = yaml.each_line.with_object({}) do |line, hash|
125
+ key, value = line.chomp.split(': ', 2)
126
+ hash[key] = Schedules.stat_value(value) if value
127
+ end
128
+ StatStruct.from_hash(stats)
129
+ rescue Tuber::NotFoundError
130
+ nil
131
+ end
132
+
133
+ # String representation of the schedules collection.
134
+ def to_s
135
+ "#<Tuber::Schedules>"
136
+ end
137
+ alias :inspect :to_s
138
+
139
+ # Checks an owner, entry name or tube against tube-name rules.
140
+ #
141
+ # @return [String] The name, a Symbol converted
142
+ # @raise [ArgumentError] It breaks them
143
+ # @api private
144
+ def self.name!(what, value)
145
+ value = value.to_s if value.is_a?(Symbol)
146
+ return value if value.is_a?(String) && value.bytesize <= 200 && value.match?(NAME)
147
+ raise ArgumentError, "invalid schedule #{what} #{value.inspect}: up to 200 bytes of letters, " \
148
+ "digits and - + / ; . $ _ ( ), not starting with -"
149
+ end
150
+
151
+ # Converts one stats-schedule value by how tuber writes it: a quoted value
152
+ # is a String, `""` meaning none (nil); bare digits are a counter; any
153
+ # other bare value is a time.
154
+ #
155
+ # @api private
156
+ def self.stat_value(value)
157
+ if value.start_with?('"')
158
+ string = value.delete_prefix('"').delete_suffix('"')
159
+ string.empty? ? nil : string
160
+ elsif value.match?(/\A\d+\z/)
161
+ value.to_i
162
+ else
163
+ begin
164
+ Time.iso8601(value)
165
+ rescue ArgumentError
166
+ value # a field newer than this gem; better raw than an error
167
+ end
168
+ end
169
+ end
170
+
171
+ # Collects the entries of one sync. Nothing is sent until the block ends,
172
+ # and nothing at all if an #add raises.
173
+ class Builder
174
+ # @!attribute entries
175
+ # @return [Array<Tuber::Schedules::Entry>] the entries added so far
176
+ attr_reader :entries
177
+
178
+ def initialize
179
+ @entries = []
180
+ end
181
+
182
+ # Adds one entry. Give exactly one of +cron:+, +every:+ or +spec:+.
183
+ #
184
+ # @param [String] name The entry's name, unique within the owner. Keep
185
+ # it stable: a renamed entry is a removal plus an addition.
186
+ # @param [String] tube The tube each fire puts into
187
+ # @param [String] body The job body each fire puts, passed through
188
+ # Tuber.configuration.job_serializer as for Tube#put
189
+ # @param [String] cron 5-field cron (`"0 2 * * *"`) or a macro
190
+ # (`"@daily"`), optionally followed by an IANA zone
191
+ # @param [String] zone IANA zone for +cron+ (default UTC)
192
+ # @param [Integer, String] every Interval, aligned to the Unix epoch:
193
+ # seconds, or `"10m"`, `"1h"`, `"1d"`
194
+ # @param [Integer, String] offset Shift for +every+, e.g. `"5m"`
195
+ # @param [String] spec The spec as tuber reads it, instead of the above
196
+ # @param [Integer] pri Priority; defaults as for a put
197
+ # @param [Integer] delay Delay of each fire's job; defaults as for a put
198
+ # @param [Integer] ttr TTR of each fire's job; defaults as for a put
199
+ # @param [String, Array] idp Idempotency key, or [key, ttl], as for a
200
+ # put. A fire is skipped while the previous fire's job is still in
201
+ # the tube — buried included, so a buried job stops the entry until
202
+ # it is kicked or deleted.
203
+ # @param [String, Array] con Concurrency key, or [key, limit], as for a put
204
+ # @param [String] grp Group key, as for a put
205
+ # @param [String] aft After-group key, as for a put
206
+ # @param [Symbol, Integer, false] catchup What a tick missed while
207
+ # tuber was down does: `:once` (the default) fires once for the gap;
208
+ # an Integer fires that once only if it is at most that many seconds
209
+ # stale; `:never`, `false` or 0 never fires it.
210
+ # @return [self]
211
+ # @raise [ArgumentError] Something tuber would refuse, or that would
212
+ # break the line
213
+ def add(name, tube:, body:, cron: nil, zone: nil, every: nil, offset: nil, spec: nil,
214
+ pri: nil, delay: nil, ttr: nil, idp: nil, con: nil, grp: nil, aft: nil, catchup: :once)
215
+ name = Schedules.name!("name", name)
216
+ tube = Schedules.name!("tube", tube)
217
+ raise ArgumentError, "schedule #{name.inspect} added twice" if @entries.any? { |e| e.name == name }
218
+
219
+ config = Tuber.configuration
220
+ serialized = config.job_serializer.call(body)
221
+ unless serialized.is_a?(String)
222
+ raise ArgumentError, "the body of schedule #{name.inspect} is a #{serialized.class}, not a String: " \
223
+ "serialize it, or set Tuber.configuration.job_serializer"
224
+ end
225
+ serialized = serialized.b
226
+
227
+ words = [name, tube,
228
+ Integer(pri || config.default_put_pri),
229
+ Integer(delay || config.default_put_delay),
230
+ Integer(ttr || config.default_put_ttr),
231
+ serialized.bytesize]
232
+ words.concat(tags(name, idp: idp, con: con, grp: grp, aft: aft, catchup: catchup))
233
+ words << spec_text(name, cron: cron, zone: zone, every: every, offset: offset, spec: spec)
234
+ @entries << Entry.new(name, words.join(' '), serialized)
235
+ self
236
+ end
237
+
238
+ private
239
+
240
+ def spec_text(name, cron:, zone:, every:, offset:, spec:)
241
+ given = { spec: spec, cron: cron, every: every }.compact
242
+ unless given.size == 1
243
+ raise ArgumentError, "schedule #{name.inspect} needs exactly one of cron:, every: or spec: " \
244
+ "(got #{given.empty? ? 'none' : given.keys.join(', ')})"
245
+ end
246
+ raise ArgumentError, "schedule #{name.inspect}: zone: goes with cron:" if zone && !cron
247
+ raise ArgumentError, "schedule #{name.inspect}: offset: goes with every:" if offset && !every
248
+
249
+ text = if every
250
+ ["every #{duration(every)}", (offset && "offset #{duration(offset)}")].compact.join(' ')
251
+ elsif cron
252
+ [cron, zone].compact.join(' ')
253
+ else
254
+ spec
255
+ end
256
+ # Tags end at the first word without a colon, so a spec can't have one.
257
+ if !text.is_a?(String) || text.strip.empty? || text.match?(/[\r\n:]/)
258
+ raise ArgumentError, "invalid spec #{text.inspect} for schedule #{name.inspect}"
259
+ end
260
+ text.strip
261
+ end
262
+
263
+ def duration(value)
264
+ value.is_a?(Integer) ? "#{value.to_i}s" : value.to_s
265
+ end
266
+
267
+ def tags(name, catchup:, **options)
268
+ tags = Tube.tags(options)
269
+ case catchup
270
+ when :once, nil then nil
271
+ when :never, false then tags << "catchup:0"
272
+ when Integer
273
+ raise ArgumentError, "catchup: can't be negative (schedule #{name.inspect})" if catchup.negative?
274
+ tags << "catchup:#{catchup.to_i}"
275
+ else
276
+ raise ArgumentError, "invalid catchup: #{catchup.inspect} for schedule #{name.inspect} " \
277
+ "(:once, :never, or a number of seconds)"
278
+ end
279
+ tags.each do |tag|
280
+ next if tag.match?(/\A[a-z]+:\S+\z/)
281
+ raise ArgumentError, "invalid tag #{tag.inspect} for schedule #{name.inspect}: " \
282
+ "keys can't be empty or contain whitespace"
283
+ end
284
+ tags
285
+ end
286
+ end
287
+ end # Schedules
288
+ end # Tuber
@@ -62,21 +62,32 @@ class Tuber
62
62
 
63
63
  cmd_options = "#{options[:pri]} #{options[:delay]} #{options[:ttr]} #{serialized_body.bytesize}"
64
64
 
65
- tags = []
66
- if options[:idp]
67
- tags << (options[:idp].is_a?(Array) ? "idp:#{options[:idp].join(':')}" : "idp:#{options[:idp]}")
68
- end
69
- if options[:con]
70
- tags << (options[:con].is_a?(Array) ? "con:#{options[:con].join(':')}" : "con:#{options[:con]}")
71
- end
72
- tags << "grp:#{options[:grp]}" if options[:grp]
73
- tags << "aft:#{options[:aft]}" if options[:aft]
74
-
65
+ tags = Tube.tags(options)
75
66
  cmd_options = "#{cmd_options} #{tags.join(' ')}" if tags.any?
76
67
  transmit("put #{cmd_options}\r\n#{serialized_body}")
77
68
  end
78
69
  end
79
70
 
71
+ # The tags a put carries for its :idp, :con, :grp and :aft options.
72
+ # Tuber::Schedules builds its entries' tags here too: every fire is a put
73
+ # carrying them.
74
+ #
75
+ # @param [Hash] options Put options, as for #put
76
+ # @return [Array<String>] e.g. ["idp:report:300", "con:api"]
77
+ # @api private
78
+ def self.tags(options)
79
+ tags = []
80
+ if options[:idp]
81
+ tags << (options[:idp].is_a?(Array) ? "idp:#{options[:idp].join(':')}" : "idp:#{options[:idp]}")
82
+ end
83
+ if options[:con]
84
+ tags << (options[:con].is_a?(Array) ? "con:#{options[:con].join(':')}" : "con:#{options[:con]}")
85
+ end
86
+ tags << "grp:#{options[:grp]}" if options[:grp]
87
+ tags << "aft:#{options[:aft]}" if options[:aft]
88
+ tags
89
+ end
90
+
80
91
  # Peek at next job within this tube in given `state`.
81
92
  #
82
93
  # @param [String] state The job state to peek at (`ready`, `buried`, `delayed`)
data/lib/tuber/version.rb CHANGED
@@ -2,5 +2,5 @@
2
2
 
3
3
  class Tuber
4
4
  # Current version of gem.
5
- VERSION = "0.6.0"
5
+ VERSION = "0.7.0"
6
6
  end
data/lib/tuber.rb CHANGED
@@ -2,7 +2,7 @@
2
2
 
3
3
  require 'thread' unless defined?(Mutex)
4
4
 
5
- %w(version configuration errors connection tube job stats).each do |f|
5
+ %w(version configuration errors connection tube job stats schedules).each do |f|
6
6
  require "tuber/#{f}"
7
7
  end
8
8
 
@@ -51,6 +51,15 @@ class Tuber
51
51
  @stats ||= Stats.new(self)
52
52
  end
53
53
 
54
+ # Returns Tuber::Schedules object for the server's recurring schedules
55
+ # (Tuber only).
56
+ #
57
+ # @return [Tuber::Schedules] schedules object
58
+ # @api public
59
+ def schedules
60
+ @schedules ||= Schedules.new(self)
61
+ end
62
+
54
63
  # Puts the server into drain mode, rejecting new puts.
55
64
  #
56
65
  # @return [Hash] Response from beanstalkd
@@ -624,4 +624,24 @@ describe Tuber::Connection do
624
624
  assert_raises(Tuber::DrainingError) { @bc.transmit("put 0 0 100 4\r\ntest") }
625
625
  end
626
626
  end # drain and undrain
627
+
628
+ describe 'for a put refused with OUT_OF_STORAGE' do
629
+ # Tuber answers OUT_OF_STORAGE once a persistent server's disk budget
630
+ # (--max-storage-bytes) is spent. It must raise like OUT_OF_MEMORY does,
631
+ # not come back as a hash a caller would read as a successful put.
632
+ it "should raise" do
633
+ @bc = Tuber::Connection.new('localhost')
634
+ TCPSocket.any_instance.stubs(:write)
635
+ TCPSocket.any_instance.expects(:readline).returns("OUT_OF_STORAGE\r\n")
636
+ err = begin
637
+ @bc.transmit("put 0 0 100 4\r\ntest")
638
+ nil
639
+ rescue Tuber::UnexpectedResponse => e
640
+ e
641
+ end
642
+ refute_nil err, "OUT_OF_STORAGE came back as a result instead of raising"
643
+ assert_equal 'OUT_OF_STORAGE', err.status
644
+ assert_equal 'OutOfStorageError', err.class.name.split('::').last
645
+ end
646
+ end # OUT_OF_STORAGE
627
647
  end # Tuber::Connection
data/test/errors_test.rb CHANGED
@@ -32,4 +32,11 @@ describe "Tuber::Errors" do
32
32
  assert_equal 'reserve 0', @klazz.cmd
33
33
  assert_equal 'EXPECTED_CRLF', @klazz.status
34
34
  end
35
+
36
+ it 'should raise proper exception for invalid status OUT_OF_STORAGE' do
37
+ @klazz = Tuber::UnexpectedResponse.from_status("OUT_OF_STORAGE", "put 0 0 120 4")
38
+ assert_kind_of(Tuber::UnexpectedResponse, @klazz)
39
+ assert_equal 'OutOfStorageError', @klazz.class.name.split('::').last
40
+ assert_equal 'OUT_OF_STORAGE', @klazz.status
41
+ end
35
42
  end
@@ -0,0 +1,382 @@
1
+ # frozen_string_literal: true
2
+
3
+ # test/schedules_test.rb
4
+
5
+ require File.expand_path('../test_helper', __FILE__)
6
+
7
+ describe Tuber::Schedules do
8
+ before do
9
+ @beanstalk = Tuber.new(tuber_address)
10
+ @schedules = @beanstalk.schedules
11
+ # Owners are namespaces, so a unique one keeps tests apart.
12
+ @owner = "gemtest.#{Process.pid}.#{rand(1 << 30)}"
13
+ end
14
+
15
+ after do
16
+ begin
17
+ @schedules.sync(@owner, allow_empty: true) if $tuber_schedules
18
+ rescue Tuber::UnexpectedResponse, Tuber::NotConnected
19
+ # the test already failed; don't bury its error under this one
20
+ end
21
+ end
22
+
23
+ def add_two(sync)
24
+ sync.add "rollup", tube: "gemtest.sched", cron: "5 * * * *", body: "rollup"
25
+ sync.add "checkin", tube: "gemtest.sched", every: 60, body: "checkin"
26
+ end
27
+
28
+ describe "for #sync" do
29
+ before { require_schedules! }
30
+
31
+ it "adds, keeps, changes and removes entries" do
32
+ assert_equal [2, 0, 0, 0], @schedules.sync(@owner) { |s| add_two(s) }.to_a
33
+ assert_equal [0, 0, 0, 2], @schedules.sync(@owner) { |s| add_two(s) }.to_a
34
+ assert_equal [[@owner, "checkin"], [@owner, "rollup"]], @schedules.list(@owner).sort
35
+ assert_includes @schedules.list, [@owner, "rollup"]
36
+
37
+ result = @schedules.sync(@owner) do |s|
38
+ s.add "rollup", tube: "gemtest.sched", cron: "15 * * * *", body: "rollup"
39
+ end
40
+ assert_equal [0, 1, 1, 0], result.to_a
41
+ assert_equal [[@owner, "rollup"]], @schedules.list(@owner)
42
+ end
43
+
44
+ it "lands each option in the entry tuber stores" do
45
+ @schedules.sync(@owner) do |s|
46
+ s.add "nightly", tube: "gemtest.sched", cron: "0 2 * * *", zone: "Australia/Melbourne",
47
+ body: "nightly", idp: ["nightly", 300], con: ["maintenance", 2], catchup: 300,
48
+ pri: 1024, delay: 5, ttr: 60
49
+ s.add "hourly", tube: "gemtest.sched", every: "1h", offset: "5m", body: "hourly", catchup: false
50
+ s.add "daily", tube: "gemtest.sched", spec: "@daily", body: "daily"
51
+ end
52
+
53
+ nightly = @schedules.stats(@owner, "nightly")
54
+ assert_equal "0 2 * * * Australia/Melbourne", nightly.spec
55
+ assert_equal "gemtest.sched", nightly.tube
56
+ assert_equal [1024, 5, 60], [nightly.pri, nightly.delay, nightly.ttr]
57
+ assert_equal "idp:nightly:300 con:maintenance:2", nightly.tags
58
+ assert_equal "once within 300s", nightly.catchup
59
+
60
+ hourly = @schedules.stats(@owner, "hourly")
61
+ assert_equal "every 1h offset 5m", hourly.spec
62
+ assert_equal "never", hourly.catchup
63
+ assert_nil hourly.tags
64
+
65
+ daily = @schedules.stats(@owner, "daily")
66
+ assert_equal "once", daily.catchup
67
+ config = Tuber.configuration
68
+ assert_equal [config.default_put_pri, config.default_put_delay, config.default_put_ttr],
69
+ [daily.pri, daily.delay, daily.ttr]
70
+ end
71
+
72
+ it "raises naming the refused entry, and leaves the previous set in place" do
73
+ nightly = lambda do |s|
74
+ s.add "nightly", tube: "gemtest.sched", cron: "0 2 * * *", zone: "Australia/Melbourne", body: "n"
75
+ end
76
+ @schedules.sync(@owner, &nightly)
77
+
78
+ err = assert_raises(Tuber::ScheduleSyncError) do
79
+ @schedules.sync(@owner) do |s|
80
+ s.add "nightly", tube: "gemtest.sched", cron: "0 3 * * *", zone: "Australia/Melbourne", body: "n"
81
+ s.add "typo", tube: "gemtest.sched", cron: "0 2 * * *", zone: "Australia/Melborne", body: "t"
82
+ end
83
+ end
84
+ assert_equal "BAD_SCHEDULE", err.status
85
+ assert_equal "typo", err.entry
86
+ assert_equal @owner, err.owner
87
+ refute err.applied?
88
+ assert_match %r{Australia/Melborne}, err.message
89
+
90
+ assert_equal [[@owner, "nightly"]], @schedules.list(@owner)
91
+ assert_equal "0 2 * * * Australia/Melbourne", @schedules.stats(@owner, "nightly").spec
92
+ # The connection left sync mode: the next sync goes through.
93
+ assert_equal [0, 0, 0, 1], @schedules.sync(@owner, &nightly).to_a
94
+ end
95
+
96
+ it "refuses an empty set unless allow_empty" do
97
+ @schedules.sync(@owner) { |s| add_two(s) }
98
+
99
+ assert_raises(ArgumentError) { @schedules.sync(@owner) { |_s| } }
100
+ assert_raises(ArgumentError) { @schedules.sync(@owner) }
101
+ assert_equal 2, @schedules.list(@owner).size
102
+
103
+ assert_equal [0, 0, 2, 0], @schedules.sync(@owner, allow_empty: true).to_a
104
+ assert_equal [], @schedules.list(@owner)
105
+ end
106
+
107
+ it "sends nothing when an entry is invalid" do
108
+ @schedules.sync(@owner) { |s| add_two(s) }
109
+
110
+ assert_raises(ArgumentError) do
111
+ @schedules.sync(@owner) do |s|
112
+ s.add "rollup", tube: "gemtest.sched", cron: "5 * * * *", body: "rollup"
113
+ s.add "broken", tube: "gemtest.sched", cron: "5 * * * *", every: 60, body: "broken"
114
+ end
115
+ end
116
+ assert_equal 2, @schedules.list(@owner).size
117
+ end
118
+
119
+ it "puts the body into the tube when an entry fires" do
120
+ serializer, parser = Tuber.configuration.job_serializer, Tuber.configuration.job_parser
121
+ Tuber.configure do |config|
122
+ config.job_serializer = lambda { |body| JSON.dump(body) }
123
+ config.job_parser = lambda { |body| JSON.parse(body) }
124
+ end
125
+ tube = "gemtest.fire.#{rand(1 << 30)}"
126
+ @schedules.sync(@owner) do |s|
127
+ s.add "tick", tube: tube, every: 1, body: { "class" => "TickJob", "args" => [1] }
128
+ end
129
+
130
+ job = @beanstalk.tubes.find(tube).reserve(3)
131
+ assert_equal({ "class" => "TickJob", "args" => [1] }, job.body)
132
+ job.delete
133
+ ensure
134
+ Tuber.configure do |config|
135
+ config.job_serializer = serializer
136
+ config.job_parser = parser
137
+ end
138
+ end
139
+
140
+ it "re-runs the whole sync when the connection drops partway" do
141
+ $sync_writes = []
142
+ $drop_mid_sync = true
143
+ TCPSocket.prepend Module.new {
144
+ def write(data)
145
+ $sync_writes << data.to_s if $sync_writes
146
+ super
147
+ end
148
+
149
+ def readline
150
+ if $drop_mid_sync && $sync_writes.last.to_s.start_with?("schedule-put")
151
+ $drop_mid_sync = false
152
+ close # the server sees the connection go and discards what was staged
153
+ raise EOFError
154
+ end
155
+ super
156
+ end
157
+ }
158
+
159
+ result = @schedules.sync(@owner) { |s| add_two(s) }
160
+
161
+ assert_equal 2, $sync_writes.count { |w| w.start_with?("schedule-begin") }
162
+ assert_equal [2, 0, 0, 0], result.to_a
163
+ assert_equal [[@owner, "checkin"], [@owner, "rollup"]], @schedules.list(@owner).sort
164
+ ensure
165
+ # Leave the prepended module inert for the rest of the suite.
166
+ $drop_mid_sync = false
167
+ $sync_writes = nil
168
+ end
169
+
170
+ it "isn't held up by a blocking reserve on the client's connection" do
171
+ tube = "gemtest.empty.#{rand(1 << 30)}"
172
+ reserver = Thread.new do
173
+ @beanstalk.tubes.find(tube).reserve(3)
174
+ rescue Tuber::TimedOutError
175
+ nil
176
+ end
177
+ sleep 0.3 # let the reserve take the connection
178
+
179
+ started = Time.now
180
+ @schedules.sync(@owner) { |s| add_two(s) }
181
+ assert_operator Time.now - started, :<, 1.5, "the sync waited for the reserve to time out"
182
+ ensure
183
+ reserver.join if reserver
184
+ end
185
+
186
+ it "closes the connection it syncs on, refused or not" do
187
+ # A leaked socket closes only when it is garbage collected, which would
188
+ # hide the leak; and earlier tests' unclosed clients must go first.
189
+ GC.start
190
+ GC.disable
191
+ sleep 0.2 # let the server notice anything just closed
192
+ before = @beanstalk.stats["current-connections"]
193
+
194
+ @schedules.sync(@owner) { |s| add_two(s) }
195
+ assert_raises(Tuber::ScheduleSyncError) do
196
+ @schedules.sync(@owner) do |s|
197
+ s.add "typo", tube: "gemtest.sched", cron: "0 2 * * *", zone: "Australia/Melborne", body: "t"
198
+ end
199
+ end
200
+
201
+ after = nil
202
+ 10.times do # the server notices a close a moment later
203
+ after = @beanstalk.stats["current-connections"]
204
+ break if after <= before
205
+ sleep 0.1
206
+ end
207
+ assert_operator after, :<=, before
208
+ ensure
209
+ GC.enable
210
+ end
211
+
212
+ it "leaves nothing behind when an unexpected error interrupts it" do
213
+ $fail_mid_sync = true
214
+ $sync_last_write = nil
215
+ TCPSocket.prepend Module.new {
216
+ def write(data)
217
+ $sync_last_write = data.to_s if $fail_mid_sync
218
+ super
219
+ end
220
+
221
+ def readline
222
+ if $fail_mid_sync && $sync_last_write.to_s.start_with?("schedule-put")
223
+ $fail_mid_sync = false
224
+ raise IOError, "simulated read failure" # e.g. a read timeout: not a dropped socket
225
+ end
226
+ super
227
+ end
228
+ }
229
+
230
+ assert_raises(IOError) { @schedules.sync(@owner) { |s| add_two(s) } }
231
+
232
+ # The client's connection still answers its own commands in step...
233
+ assert_equal "default", @beanstalk.connection.transmit("list-tube-used")[:id]
234
+ # ...nothing of the interrupted sync was committed, and the next sync starts clean.
235
+ assert_equal [], @schedules.list(@owner)
236
+ assert_equal [2, 0, 0, 0], @schedules.sync(@owner) { |s| add_two(s) }.to_a
237
+ ensure
238
+ $fail_mid_sync = false
239
+ end
240
+ end # sync
241
+
242
+ describe "for #stats" do
243
+ before { require_schedules! }
244
+
245
+ it "returns times as Time in the entry's zone, and counters as Integers" do
246
+ @schedules.sync(@owner) do |s|
247
+ s.add "nightly", tube: "gemtest.sched", cron: "0 2 * * *", zone: "Australia/Melbourne", body: "n"
248
+ end
249
+
250
+ stats = @schedules.stats(@owner, "nightly")
251
+ assert_equal @owner, stats.owner
252
+ assert_kind_of Time, stats.synced_at
253
+ assert_kind_of Time, stats.next_fire
254
+ assert_includes [10 * 3600, 11 * 3600], stats.next_fire.utc_offset
255
+ assert_includes [2, 3], stats.next_fire.hour # 3 on a spring-forward day
256
+ assert stats.next_fire < stats.next_fire_2
257
+ assert stats.next_fire_2 < stats.next_fire_3
258
+ assert_equal stats.next_fire_2, stats["next-fire-2"]
259
+ assert_nil stats.last_fire
260
+ assert_nil stats.last_result
261
+ assert_equal 0, stats.total_fires
262
+ assert_equal 0, stats.total_deduped_buried
263
+ assert_equal 0, stats.buried_streak
264
+ end
265
+
266
+ it "returns nil for an entry it doesn't hold" do
267
+ assert_nil @schedules.stats(@owner, "nope")
268
+ end
269
+
270
+ it "keeps a name and tube made of digits as Strings" do
271
+ @schedules.sync(@owner) { |s| s.add "2026", tube: "123", every: 60, body: "b" }
272
+ stats = @schedules.stats(@owner, "2026")
273
+ assert_equal "2026", stats.name
274
+ assert_equal "123", stats.tube
275
+ end
276
+ end # stats
277
+ end # Tuber::Schedules
278
+
279
+ # Replies a healthy server won't produce on demand. No @beanstalk here, so the
280
+ # suite's teardown doesn't talk through the stubbed socket.
281
+ describe "Tuber::Schedules sync refusals" do
282
+ before do
283
+ @client = Tuber.new(tuber_address)
284
+ @owner = "gemtest.stubbed"
285
+ TCPSocket.any_instance.stubs(:write)
286
+ end
287
+
288
+ after do
289
+ @client.close
290
+ end
291
+
292
+ def sync_one
293
+ @client.schedules.sync(@owner) { |s| s.add "a", tube: "gemtest.sched", every: 60, body: "a" }
294
+ end
295
+
296
+ it "says the set is live when the commit couldn't be written" do
297
+ TCPSocket.any_instance.stubs(:readline).returns("SYNCING #{@owner}\r\n", "STAGED\r\n", "INTERNAL_ERROR\r\n")
298
+ err = assert_raises(Tuber::ScheduleSyncError) { sync_one }
299
+ assert_equal "INTERNAL_ERROR", err.status
300
+ assert_nil err.entry
301
+ assert err.applied?
302
+ assert_match(/is live/, err.message)
303
+ end
304
+
305
+ it "says nothing changed when the commit is refused" do
306
+ TCPSocket.any_instance.stubs(:readline).returns("SYNCING #{@owner}\r\n", "STAGED\r\n", "OUT_OF_STORAGE\r\n")
307
+ err = assert_raises(Tuber::ScheduleSyncError) { sync_one }
308
+ assert_equal "OUT_OF_STORAGE", err.status
309
+ refute err.applied?
310
+ assert_match(/Nothing changed/, err.message)
311
+ end
312
+
313
+ it "aborts the sync before raising for a refused entry" do
314
+ TCPSocket.any_instance.expects(:write).with("schedule-abort\r\n")
315
+ TCPSocket.any_instance.stubs(:readline).returns("SYNCING #{@owner}\r\n", "JOB_TOO_BIG\r\n", "ABORTED\r\n")
316
+ err = assert_raises(Tuber::ScheduleSyncError) { sync_one }
317
+ assert_equal "JOB_TOO_BIG", err.status
318
+ assert_equal "a", err.entry
319
+ end
320
+
321
+ it "explains a server without schedules" do
322
+ TCPSocket.any_instance.stubs(:readline).returns("UNKNOWN_COMMAND\r\n")
323
+ err = assert_raises(Tuber::ScheduleSyncError) { sync_one }
324
+ assert_equal "UNKNOWN_COMMAND", err.status
325
+ assert_match(/no recurring schedules/, err.message)
326
+ end
327
+ end
328
+
329
+ describe Tuber::Schedules::Builder do
330
+ before do
331
+ @builder = Tuber::Schedules::Builder.new
332
+ end
333
+
334
+ it "writes the schedule-put arguments: tags before the spec" do
335
+ @builder.add "nightly", tube: "splat.maintenance", cron: "0 2 * * *", zone: "Australia/Melbourne",
336
+ body: "{}", idp: ["nightly", 300], con: "maintenance", catchup: 300, pri: 1024
337
+ entry = @builder.entries.first
338
+ assert_equal "nightly", entry.name
339
+ assert_equal "nightly splat.maintenance 1024 0 120 2 idp:nightly:300 con:maintenance catchup:300 " \
340
+ "0 2 * * * Australia/Melbourne", entry.line
341
+ assert_equal Encoding::BINARY, entry.body.encoding
342
+ end
343
+
344
+ it "counts the body in bytes" do
345
+ @builder.add "utf8", tube: "t", every: 60, body: "café"
346
+ assert_equal "utf8 t 65536 0 120 5 every 60s", @builder.entries.first.line
347
+ end
348
+
349
+ it "writes every: and offset:, and catchup: :never" do
350
+ @builder.add :hourly, tube: "t", every: 3600, offset: 300, body: "b", catchup: :never
351
+ assert_equal "hourly t 65536 0 120 1 catchup:0 every 3600s offset 300s", @builder.entries.first.line
352
+ end
353
+
354
+ it "refuses an entry tuber would refuse, or that would break the line" do
355
+ bad = [
356
+ { tube: "t", body: "b" }, # no spec
357
+ { tube: "t", body: "b", cron: "5 * * * *", every: 60 }, # two specs
358
+ { tube: "t", body: "b", every: 60, zone: "UTC" }, # zone without cron
359
+ { tube: "t", body: "b", cron: "5 * * * *", offset: "5m" }, # offset without every
360
+ { tube: "t", body: "b", spec: "5 * * * * UTC+10:00" }, # a colon reads as a tag
361
+ { tube: "t", body: "b", spec: "5 * * * *\r\nflush-tube t" }, # a second command
362
+ { tube: "-t", body: "b", every: 60 }, # bad tube name
363
+ { tube: "t", body: { "a" => 1 }, every: 60 }, # body isn't a String
364
+ { tube: "t", body: "b", every: 60, idp: "two words" }, # tag with a space
365
+ { tube: "t", body: "b", every: 60, con: "" }, # empty tag key
366
+ { tube: "t", body: "b", every: 60, catchup: -1 },
367
+ { tube: "t", body: "b", every: 60, catchup: "soon" },
368
+ ]
369
+ bad.each do |options|
370
+ assert_raises(ArgumentError, options.inspect) { Tuber::Schedules::Builder.new.add("name", **options) }
371
+ end
372
+
373
+ ["", "-name", "has space", "x" * 201, nil].each do |name|
374
+ assert_raises(ArgumentError, name.inspect) { @builder.add(name, tube: "t", every: 60, body: "b") }
375
+ end
376
+ end
377
+
378
+ it "refuses a name added twice" do
379
+ @builder.add "once", tube: "t", every: 60, body: "b"
380
+ assert_raises(ArgumentError) { @builder.add "once", tube: "t", every: 120, body: "b" }
381
+ end
382
+ end
data/test/test_helper.rb CHANGED
@@ -43,6 +43,24 @@ class MiniTest::Unit::TestCase
43
43
  skip "requires a Tuber server, not stock beanstalkd" unless tuber_server?
44
44
  end
45
45
 
46
+ # Skips the current test unless the server runs recurring schedules. A
47
+ # Tuber older than the scheduler answers UNKNOWN_COMMAND to list-schedules.
48
+ def require_schedules!
49
+ require_tuber!
50
+ if $tuber_schedules.nil?
51
+ client = Tuber.new(tuber_address)
52
+ begin
53
+ client.connection.transmit_raw('list-schedules')
54
+ $tuber_schedules = true
55
+ rescue Tuber::UnknownCommandError
56
+ $tuber_schedules = false
57
+ ensure
58
+ client.close
59
+ end
60
+ end
61
+ skip "requires a Tuber server with recurring schedules" unless $tuber_schedules
62
+ end
63
+
46
64
  # Cleans up all jobs from specific tubes
47
65
  #
48
66
  # @example
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: tuber
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.6.0
4
+ version: 0.7.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Dan Milne
@@ -105,6 +105,7 @@ files:
105
105
  - lib/tuber/job.rb
106
106
  - lib/tuber/job/collection.rb
107
107
  - lib/tuber/job/record.rb
108
+ - lib/tuber/schedules.rb
108
109
  - lib/tuber/stats.rb
109
110
  - lib/tuber/stats/fast_struct.rb
110
111
  - lib/tuber/stats/stat_struct.rb
@@ -117,6 +118,7 @@ files:
117
118
  - test/job_test.rb
118
119
  - test/jobs_test.rb
119
120
  - test/prompt_regexp_test.rb
121
+ - test/schedules_test.rb
120
122
  - test/stat_struct_test.rb
121
123
  - test/stats_test.rb
122
124
  - test/test_helper.rb
@@ -145,7 +147,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
145
147
  - !ruby/object:Gem::Version
146
148
  version: '0'
147
149
  requirements: []
148
- rubygems_version: 4.0.16
150
+ rubygems_version: 4.0.20
149
151
  specification_version: 4
150
152
  summary: Ruby client for the Tuber job queue server.
151
153
  test_files: []