tobox 0.4.5 → 0.5.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +16 -0
- data/README.md +100 -47
- data/lib/tobox/configuration.rb +10 -8
- data/lib/tobox/fetcher.rb +87 -79
- data/lib/tobox/plugins/event_grouping.rb +47 -0
- data/lib/tobox/plugins/inbox.rb +46 -0
- data/lib/tobox/plugins/progress.rb +33 -0
- data/lib/tobox/plugins/stats.rb +23 -4
- data/lib/tobox/pool/threaded_pool.rb +2 -0
- data/lib/tobox/pool.rb +2 -2
- data/lib/tobox/version.rb +1 -1
- data/lib/tobox/worker.rb +1 -1
- data/lib/tobox.rb +3 -1
- metadata +7 -4
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: '0481835fffcdbaf85fb53ebf116a6b117ad267cfb4a4efbcbae05b2f87058951'
|
4
|
+
data.tar.gz: 39589407d92d28fc538aebb59a039a606d833aeb0eb26f82b7cf96228e73d5b3
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 8a1137598cc58cb22573ccbec80f448aa55db3505a97b741cc146eaef6f16392ba3159b9db9f61498cea530dba33c431bba4f9c847325064a770a96553d9ec20
|
7
|
+
data.tar.gz: 274be354130a8ae5ab59188ad98c4de208adac78cd7c9720b569db98de75d2ec0514a2560db05c080b9118e6967b573b7ea1fd11b0297edf9c07f05478acbefe
|
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,21 @@
|
|
1
1
|
## [Unreleased]
|
2
2
|
|
3
|
+
## [0.5.0] - 2024-09-16
|
4
|
+
|
5
|
+
### Features
|
6
|
+
|
7
|
+
A new `:progress` plugin can be used in order to release database transactions before handling events (useful for when event handling times vary and may cause transaction bookkeeping overhead in the database).
|
8
|
+
|
9
|
+
**Note**: This may become the default behaviour in a future release.
|
10
|
+
|
11
|
+
### Improvements
|
12
|
+
|
13
|
+
* The event grouping and inbox capabilities were converted into plugins (respectively, `:event_grouping` and `:inbox`).
|
14
|
+
|
15
|
+
### Bugfixes
|
16
|
+
|
17
|
+
* exponential backoff calculation was broken.
|
18
|
+
* behaviour fixed for databases which do not support `ON CONFLICT` or `UPDATE ... RETURNING` (like MySQL).
|
3
19
|
|
4
20
|
## [0.4.5] - 2024-02-28
|
5
21
|
|
data/README.md
CHANGED
@@ -13,10 +13,10 @@ Simple, data-first events processing framework based on the [transactional outbo
|
|
13
13
|
- [Usage](#usage)
|
14
14
|
- [Configuration](#configuration)
|
15
15
|
- [Event](#event)
|
16
|
-
- [Features](#features)
|
17
|
-
- [Ordered event processing](#ordered-event-processing)
|
18
|
-
- [Inbox](#inbox)
|
19
16
|
- [Plugins](#plugins)
|
17
|
+
- [Progress](#progress)
|
18
|
+
- [Event Grouping](#event-grouping)
|
19
|
+
- [Inbox](#inbox)
|
20
20
|
- [Zeitwerk](#zeitwerk)
|
21
21
|
- [Sentry](#sentry)
|
22
22
|
- [Datadog](#datadog)
|
@@ -204,7 +204,17 @@ table :outbox
|
|
204
204
|
Maximum number of times a failed attempt to process an event will be retried (`10` by default).
|
205
205
|
|
206
206
|
```ruby
|
207
|
-
|
207
|
+
max_attempts 4
|
208
|
+
```
|
209
|
+
|
210
|
+
**Note**: the new attempt will be retried in `n ** 4`, where `n` is the number of past attempts for that event.
|
211
|
+
|
212
|
+
### `exponential_retry_factor`
|
213
|
+
|
214
|
+
Factor by which the number of seconds until an event can be retried will be exponentially calculated, i.e. 2 seconds on first attempt, then 4, then 8, then 16 (`2` by default).
|
215
|
+
|
216
|
+
```ruby
|
217
|
+
exponential_retry_factor 2
|
208
218
|
```
|
209
219
|
|
210
220
|
**Note**: the new attempt will be retried in `n ** 4`, where `n` is the number of past attempts for that event.
|
@@ -320,17 +330,6 @@ Overrides the internal logger (an instance of `Logger`).
|
|
320
330
|
|
321
331
|
Overrides the default log level ("info" when in "production" environment, "debug" otherwise).
|
322
332
|
|
323
|
-
### group_column
|
324
|
-
|
325
|
-
Defines the column to be used for event grouping, when [ordered processing of events is a requirement](#ordered-event-processing).
|
326
|
-
|
327
|
-
### inbox table
|
328
|
-
|
329
|
-
Defines the name of the table to be used for inbox, when [inbox usage is a requirement](#inbox).
|
330
|
-
|
331
|
-
### inbox column
|
332
|
-
|
333
|
-
Defines the column in the outbox table which references the inbox table, when one is set.
|
334
333
|
|
335
334
|
<a id="markdown-event" name="event"></a>
|
336
335
|
## Event
|
@@ -345,36 +344,85 @@ The event is composed of the following properties:
|
|
345
344
|
|
346
345
|
(*NOTE*: The event is also composed of other properties which are only relevant for `tobox`.)
|
347
346
|
|
348
|
-
<a id="markdown-features" name="features"></a>
|
349
|
-
## Features
|
350
347
|
|
351
|
-
|
348
|
+
<a id="markdown-plugins" name="plugins"></a>
|
349
|
+
## Plugins
|
350
|
+
|
351
|
+
`tobox` ships with a very simple plugin system. (TODO: add docs).
|
352
|
+
|
353
|
+
Plugins can be loaded in the config via `plugin`:
|
354
|
+
|
355
|
+
```ruby
|
356
|
+
# tobox.rb
|
357
|
+
plugin(:plugin_name)
|
358
|
+
```
|
359
|
+
|
360
|
+
<a id="markdown-progress" name="progress"></a>
|
361
|
+
### Progress
|
362
|
+
|
363
|
+
By default, the database transaction used to consume the event is kept open while the event is handled. While this ensures atomic event consumption, it may also cause overhead related to transaction management given enough load, particularly in cases where event handling time varies (i.e. throttled HTTP requests).
|
364
|
+
|
365
|
+
The `:progress` plugin fixes this by releasing the databaase transaction after fetching the event. It does so by making the fetched event "invisible" for a certain period, during which the event must be successfully handled.
|
366
|
+
|
367
|
+
Here's how to use it:
|
368
|
+
|
369
|
+
```ruby
|
370
|
+
# in your tobox.rb
|
371
|
+
plugin :progress
|
372
|
+
|
373
|
+
visibility_timeout 90 # default: 30
|
374
|
+
```
|
375
|
+
|
376
|
+
3. insert related outbox events with the same group id
|
377
|
+
|
378
|
+
```ruby
|
379
|
+
order = Order.new(
|
380
|
+
item_id: item.id,
|
381
|
+
price: 20_20,
|
382
|
+
currency: "EUR"
|
383
|
+
)
|
384
|
+
DB.transaction do
|
385
|
+
order.save
|
386
|
+
DB[:outbox].insert(event_type: "order_created", group_id: order.id, data_after: order.to_hash)
|
387
|
+
DB[:outbox].insert(event_type: "billing_event_started", group_id: order.id, data_after: order.to_hash)
|
388
|
+
end
|
389
|
+
|
390
|
+
# "order_created" will be processed first
|
391
|
+
# "billing_event_created" will only start processing once "order_created" finishes
|
392
|
+
```
|
393
|
+
|
394
|
+
#### Configuration
|
352
395
|
|
353
|
-
|
354
|
-
|
396
|
+
##### `visibility_timeout`
|
397
|
+
|
398
|
+
Timeout (in seconds) after which a previously marked-for-consumption event can be retried (default: `30`)
|
399
|
+
|
400
|
+
<a id="markdown-event-grouping" name="event-grouping"></a>
|
401
|
+
### Event grouping
|
355
402
|
|
356
403
|
By default, events are taken and processed from the "outbox" table concurrently by workers, which means that, while worker A may process the most recent event, and worker B takes the following, worker B may process it faster than worker A. This may be an issue if the consumer expects events from a certain context to arrive in a certain order.
|
357
404
|
|
358
|
-
One solution is to have a single worker processing the "outbox" events. Another is to use the `
|
405
|
+
One solution is to have a single worker processing the "outbox" events. Another is to use the `:event_grouping` plugin.
|
359
406
|
|
360
|
-
|
407
|
+
All you have to do is:
|
361
408
|
|
362
409
|
1. add a "group id" column to the "outbox" table
|
363
410
|
|
364
411
|
```ruby
|
365
412
|
create_table(:outbox) do
|
366
413
|
primary_key :id
|
367
|
-
column :group_id, :integer
|
368
|
-
# The type is irrelevant, could also be :string, :uuid...
|
414
|
+
column :group_id, :integer # The type is irrelevant, could also be :string, :uuid...
|
369
415
|
# ..
|
416
|
+
index :group_id
|
370
417
|
```
|
371
418
|
|
372
|
-
2.
|
419
|
+
2. Enable the plugin
|
373
420
|
|
374
421
|
```ruby
|
375
422
|
# in your tobox.rb
|
376
|
-
|
377
|
-
|
423
|
+
plugin :event_grouping
|
424
|
+
|
425
|
+
group_column :group_id # by default already `:group_id`
|
378
426
|
```
|
379
427
|
|
380
428
|
3. insert related outbox events with the same group id
|
@@ -394,25 +442,28 @@ end
|
|
394
442
|
# "order_created" will be processed first
|
395
443
|
# "billing_event_created" will only start processing once "order_created" finishes
|
396
444
|
```
|
445
|
+
|
446
|
+
#### Configuration
|
447
|
+
|
448
|
+
##### `group_column`
|
449
|
+
|
450
|
+
Defines the database column to be used for event grouping (`:group_id` by default).
|
451
|
+
|
397
452
|
<a id="inbox" name="inbox"></a>
|
398
453
|
### Inbox
|
399
454
|
|
400
|
-
`tobox` also supports the [inbox pattern](https://event-driven.io/en/outbox_inbox_patterns_and_delivery_guarantees_explained/), to ensure "exactly-once" processing of events. This is achieved by "tagging" events with a unique identifier, and registering them in the inbox before processing (and if they're there, ignoring it altogether).
|
455
|
+
Via the `:inbox` plugin, `tobox` also supports the [inbox pattern](https://event-driven.io/en/outbox_inbox_patterns_and_delivery_guarantees_explained/), to ensure "exactly-once" processing of events. This is achieved by "tagging" events with a unique identifier, and registering them in the inbox before processing (and if they're there, ignoring it altogether).
|
401
456
|
|
402
457
|
In order to do so, you'll have to:
|
403
458
|
|
404
|
-
1. add an "inbox" table in the database
|
459
|
+
1. add an "inbox" table in the database and the unique id reference in the outbox table:
|
405
460
|
|
406
461
|
```ruby
|
407
462
|
create_table(:inbox) do
|
408
463
|
column :inbox_id, :varchar, null: true, primary_key: true # it can also be a uuid, you decide
|
409
464
|
column :created_at, "timestamp without time zone", null: false, default: Sequel::CURRENT_TIMESTAMP
|
410
465
|
end
|
411
|
-
```
|
412
466
|
|
413
|
-
2. add the unique id reference in the outbox table:
|
414
|
-
|
415
|
-
```ruby
|
416
467
|
create_table(:outbox) do
|
417
468
|
primary_key :id
|
418
469
|
column :type, :varchar, null: false
|
@@ -421,15 +472,17 @@ create_table(:outbox) do
|
|
421
472
|
foreign_key :inbox_id, :inbox
|
422
473
|
```
|
423
474
|
|
424
|
-
|
475
|
+
2. Load the plugin and reference them in the configuration
|
425
476
|
|
426
477
|
```ruby
|
427
478
|
# tobox.rb
|
428
|
-
|
429
|
-
|
479
|
+
plugin :inbox
|
480
|
+
|
481
|
+
inbox_table :inbox # :inbox by default already
|
482
|
+
inbox_column :inbox_id # :inbox_id by default already
|
430
483
|
```
|
431
484
|
|
432
|
-
|
485
|
+
3. insert related outbox events with an inbox id
|
433
486
|
|
434
487
|
```ruby
|
435
488
|
order = Order.new(
|
@@ -446,19 +499,19 @@ end
|
|
446
499
|
# assuming this bit above runs two times in two separate workers, each will be processed by tobox only once.
|
447
500
|
```
|
448
501
|
|
449
|
-
|
502
|
+
#### Configuration
|
450
503
|
|
451
|
-
<a id="markdown-plugins" name="plugins"></a>
|
452
|
-
## Plugins
|
453
504
|
|
454
|
-
|
505
|
+
##### inbox table
|
455
506
|
|
456
|
-
|
507
|
+
Defines the name of the table to be used for inbox (`:inbox` by default).
|
457
508
|
|
458
|
-
|
459
|
-
|
460
|
-
|
461
|
-
|
509
|
+
##### inbox column
|
510
|
+
|
511
|
+
Defines the column in the outbox table which references the inbox table (`:inbox_id` by default).
|
512
|
+
|
513
|
+
|
514
|
+
**NOTE**: make sure you keep cleaning the inbox periodically from older messages, once there's no more danger of receiving them again.
|
462
515
|
|
463
516
|
It ships with the following integrations.
|
464
517
|
|
@@ -576,7 +629,7 @@ end
|
|
576
629
|
<a id="markdown-supported-rubies" name="supported-rubies"></a>
|
577
630
|
## Supported Rubies
|
578
631
|
|
579
|
-
All Rubies greater or equal to 2.
|
632
|
+
All Rubies greater or equal to 2.7, and always latest JRuby and Truffleruby.
|
580
633
|
|
581
634
|
|
582
635
|
<a id="markdown-rails-support" name="rails-support"></a>
|
data/lib/tobox/configuration.rb
CHANGED
@@ -7,7 +7,8 @@ module Tobox
|
|
7
7
|
class Configuration
|
8
8
|
extend Forwardable
|
9
9
|
|
10
|
-
attr_reader :handlers, :lifecycle_events, :arguments_handler, :default_logger, :database
|
10
|
+
attr_reader :plugins, :handlers, :lifecycle_events, :arguments_handler, :default_logger, :database, :fetcher_class,
|
11
|
+
:config
|
11
12
|
|
12
13
|
def_delegator :@config, :[]
|
13
14
|
|
@@ -18,11 +19,9 @@ module Tobox
|
|
18
19
|
database_uri: nil,
|
19
20
|
database_options: nil,
|
20
21
|
table: :outbox,
|
21
|
-
|
22
|
-
inbox_table: nil,
|
23
|
-
inbox_column: nil,
|
22
|
+
created_at_column: nil,
|
24
23
|
max_attempts: 10,
|
25
|
-
exponential_retry_factor:
|
24
|
+
exponential_retry_factor: 2,
|
26
25
|
wait_for_events_delay: 5,
|
27
26
|
shutdown_timeout: 10,
|
28
27
|
concurrency: 4, # TODO: CPU count
|
@@ -45,6 +44,7 @@ module Tobox
|
|
45
44
|
@handlers = {}
|
46
45
|
@message_to_arguments = nil
|
47
46
|
@plugins = []
|
47
|
+
@fetcher_class = Class.new(Fetcher)
|
48
48
|
|
49
49
|
if block
|
50
50
|
case block.arity
|
@@ -141,6 +141,8 @@ module Tobox
|
|
141
141
|
|
142
142
|
extend(plugin::ConfigurationMethods) if defined?(plugin::ConfigurationMethods)
|
143
143
|
|
144
|
+
@fetcher_class.__send__(:include, plugin::FetcherMethods) if defined?(plugin::FetcherMethods)
|
145
|
+
|
144
146
|
plugin.configure(self, **options, &block) if plugin.respond_to?(:configure)
|
145
147
|
end
|
146
148
|
|
@@ -157,7 +159,7 @@ module Tobox
|
|
157
159
|
private
|
158
160
|
|
159
161
|
def method_missing(meth, *args, &block)
|
160
|
-
if
|
162
|
+
if @config.key?(meth) && args.size == 1
|
161
163
|
@config[meth] = args.first
|
162
164
|
elsif /\Aon_(.*)\z/.match(meth) && args.empty?
|
163
165
|
on(Regexp.last_match(1).to_sym, &block)
|
@@ -167,8 +169,8 @@ module Tobox
|
|
167
169
|
end
|
168
170
|
|
169
171
|
def respond_to_missing?(meth, *args)
|
170
|
-
super
|
171
|
-
|
172
|
+
super ||
|
173
|
+
@config.key?(meth) ||
|
172
174
|
/\Aon_(.*)\z/.match(meth)
|
173
175
|
end
|
174
176
|
end
|
data/lib/tobox/fetcher.rb
CHANGED
@@ -13,14 +13,10 @@ module Tobox
|
|
13
13
|
@db = configuration.database
|
14
14
|
|
15
15
|
@table = configuration[:table]
|
16
|
-
@group_column = configuration[:group_column]
|
17
16
|
@exponential_retry_factor = configuration[:exponential_retry_factor]
|
18
17
|
|
19
18
|
max_attempts = configuration[:max_attempts]
|
20
19
|
|
21
|
-
@inbox_table = configuration[:inbox_table]
|
22
|
-
@inbox_column = configuration[:inbox_column]
|
23
|
-
|
24
20
|
@ds = @db[@table]
|
25
21
|
|
26
22
|
run_at_conds = [
|
@@ -32,6 +28,8 @@ module Tobox
|
|
32
28
|
.where(run_at_conds)
|
33
29
|
.order(Sequel.desc(:run_at, nulls: :first), :id)
|
34
30
|
|
31
|
+
@mark_as_fetched_params = { attempts: Sequel[@table][:attempts] + 1, last_error: nil }
|
32
|
+
|
35
33
|
@before_event_handlers = Array(@configuration.lifecycle_events[:before_event])
|
36
34
|
@after_event_handlers = Array(@configuration.lifecycle_events[:after_event])
|
37
35
|
@error_event_handlers = Array(@configuration.lifecycle_events[:error_event])
|
@@ -39,75 +37,28 @@ module Tobox
|
|
39
37
|
|
40
38
|
def fetch_events(&blk)
|
41
39
|
num_events = 0
|
42
|
-
|
43
|
-
|
44
|
-
group = @pick_next_sql.for_update
|
45
|
-
.skip_locked
|
46
|
-
.limit(1)
|
47
|
-
.select(@group_column)
|
48
|
-
|
49
|
-
# get total from a group, to compare to the number of future locked rows.
|
50
|
-
total_from_group = @ds.where(@group_column => group).count
|
51
|
-
|
52
|
-
event_ids = @ds.where(@group_column => group)
|
53
|
-
.order(Sequel.desc(:run_at, nulls: :first), :id)
|
54
|
-
.for_update.skip_locked.select_map(:id)
|
55
|
-
|
56
|
-
if event_ids.size != total_from_group
|
57
|
-
# this happens if concurrent workers locked different rows from the same group,
|
58
|
-
# or when new rows from a given group have been inserted after the lock has been
|
59
|
-
# acquired
|
60
|
-
event_ids = []
|
61
|
-
end
|
62
|
-
|
63
|
-
# lock all, process 1
|
64
|
-
event_ids = event_ids[0, 1]
|
65
|
-
else
|
66
|
-
event_ids = @pick_next_sql.for_update
|
67
|
-
.skip_locked
|
68
|
-
.limit(1).select_map(:id) # lock starts here
|
69
|
-
end
|
40
|
+
events_tr do
|
41
|
+
event_ids = EMPTY
|
70
42
|
|
71
|
-
|
72
|
-
|
73
|
-
|
74
|
-
@db.transaction(savepoint: true) do
|
75
|
-
events = @ds.where(id: event_ids).returning.delete
|
76
|
-
|
77
|
-
if blk
|
78
|
-
num_events = events.size
|
79
|
-
|
80
|
-
events.map! do |ev|
|
81
|
-
try_insert_inbox(ev) do
|
82
|
-
ev[:metadata] = try_json_parse(ev[:metadata])
|
83
|
-
handle_before_event(ev)
|
84
|
-
yield(to_message(ev))
|
85
|
-
ev
|
86
|
-
end
|
87
|
-
rescue StandardError => e
|
88
|
-
error = e
|
89
|
-
raise Sequel::Rollback
|
90
|
-
end.compact!
|
91
|
-
else
|
92
|
-
events.map!(&method(:to_message))
|
93
|
-
end
|
94
|
-
end
|
43
|
+
event_ids_tr do
|
44
|
+
event_ids = fetch_event_ids
|
45
|
+
mark_as_fetched(event_ids)
|
95
46
|
end
|
96
47
|
|
97
|
-
|
48
|
+
evts = nil
|
98
49
|
|
99
|
-
|
50
|
+
unless event_ids.empty?
|
51
|
+
with_events(event_ids) do |events|
|
52
|
+
evts = events
|
53
|
+
num_events = events.count
|
100
54
|
|
101
|
-
|
102
|
-
|
103
|
-
if error
|
104
|
-
event.merge!(mark_as_error(event, error))
|
105
|
-
handle_error_event(event, error)
|
106
|
-
else
|
107
|
-
handle_after_event(event)
|
55
|
+
evts = events.filter_map do |ev|
|
56
|
+
prepare_event(ev, &blk)
|
108
57
|
end
|
109
58
|
end
|
110
59
|
end
|
60
|
+
|
61
|
+
return 0 if evts.nil?
|
111
62
|
end
|
112
63
|
|
113
64
|
num_events
|
@@ -115,19 +66,86 @@ module Tobox
|
|
115
66
|
|
116
67
|
private
|
117
68
|
|
69
|
+
def prepare_event(event)
|
70
|
+
event[:metadata] = try_json_parse(event[:metadata])
|
71
|
+
handle_before_event(event)
|
72
|
+
yield(to_message(event))
|
73
|
+
event
|
74
|
+
end
|
75
|
+
|
76
|
+
def fetch_event_ids
|
77
|
+
@pick_next_sql.for_update
|
78
|
+
.skip_locked
|
79
|
+
.limit(1).select_map(:id) # lock starts here
|
80
|
+
end
|
81
|
+
|
82
|
+
def mark_as_fetched(event_ids)
|
83
|
+
@ds.where(id: event_ids).update(@mark_as_fetched_params) unless event_ids.empty?
|
84
|
+
end
|
85
|
+
|
86
|
+
def events_tr(&block)
|
87
|
+
@db.transaction(savepoint: false, &block)
|
88
|
+
end
|
89
|
+
|
90
|
+
def event_ids_tr
|
91
|
+
yield
|
92
|
+
end
|
93
|
+
|
94
|
+
def with_events(event_ids, &blk)
|
95
|
+
events, error = yield_events(event_ids, &blk)
|
96
|
+
|
97
|
+
events.each do |event|
|
98
|
+
if error
|
99
|
+
event.merge!(mark_as_error(event, error))
|
100
|
+
handle_error_event(event, error)
|
101
|
+
else
|
102
|
+
handle_after_event(event)
|
103
|
+
end
|
104
|
+
end
|
105
|
+
end
|
106
|
+
|
107
|
+
def yield_events(event_ids)
|
108
|
+
events_ds = @ds.where(id: event_ids)
|
109
|
+
events = EMPTY
|
110
|
+
error = nil
|
111
|
+
|
112
|
+
begin
|
113
|
+
events = events_ds.all
|
114
|
+
|
115
|
+
yield events
|
116
|
+
|
117
|
+
events_ds.delete
|
118
|
+
rescue StandardError => e
|
119
|
+
error = e
|
120
|
+
end
|
121
|
+
|
122
|
+
[events, error]
|
123
|
+
end
|
124
|
+
|
118
125
|
def log_message(msg)
|
119
126
|
"(worker: #{@label}) -> #{msg}"
|
120
127
|
end
|
121
128
|
|
122
129
|
def mark_as_error(event, error)
|
123
|
-
|
124
|
-
attempts: Sequel[@table][:attempts] + 1,
|
130
|
+
update_params = {
|
125
131
|
run_at: Sequel.date_add(Sequel::CURRENT_TIMESTAMP,
|
126
|
-
seconds: event[:attempts]
|
132
|
+
seconds: @exponential_retry_factor**(event[:attempts] - 1)),
|
127
133
|
# run_at: Sequel.date_add(Sequel::CURRENT_TIMESTAMP,
|
128
134
|
# seconds: Sequel.function(:POWER, Sequel[@table][:attempts] + 1, 4)),
|
129
135
|
last_error: "#{error.message}\n#{error.backtrace.join("\n")}"
|
130
|
-
|
136
|
+
}
|
137
|
+
|
138
|
+
set_event_retry_attempts(event, update_params)
|
139
|
+
end
|
140
|
+
|
141
|
+
def set_event_retry_attempts(event, update_params)
|
142
|
+
ds = @ds.where(id: event[:id])
|
143
|
+
if @ds.supports_returning?(:update)
|
144
|
+
ds.returning.update(update_params).first
|
145
|
+
else
|
146
|
+
ds.update(update_params)
|
147
|
+
ds.first
|
148
|
+
end
|
131
149
|
end
|
132
150
|
|
133
151
|
def to_message(event)
|
@@ -148,16 +166,6 @@ module Tobox
|
|
148
166
|
data
|
149
167
|
end
|
150
168
|
|
151
|
-
def try_insert_inbox(event)
|
152
|
-
return yield unless @inbox_table && @inbox_column
|
153
|
-
|
154
|
-
ret = @db[@inbox_table].insert_conflict.insert(@inbox_column => event[@inbox_column])
|
155
|
-
|
156
|
-
return unless ret
|
157
|
-
|
158
|
-
yield
|
159
|
-
end
|
160
|
-
|
161
169
|
def handle_before_event(event)
|
162
170
|
@logger.debug do
|
163
171
|
log_message("outbox event (type: \"#{event[:type]}\", attempts: #{event[:attempts]}) starting...")
|
@@ -0,0 +1,47 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module Tobox
|
4
|
+
module Plugins
|
5
|
+
module EventGrouping
|
6
|
+
def self.configure(conf)
|
7
|
+
conf.config[:group_column] = :group_id
|
8
|
+
end
|
9
|
+
|
10
|
+
module FetcherMethods
|
11
|
+
def initialize(_, configuration)
|
12
|
+
super
|
13
|
+
|
14
|
+
@group_column = configuration[:group_column]
|
15
|
+
end
|
16
|
+
|
17
|
+
private
|
18
|
+
|
19
|
+
def fetch_event_ids
|
20
|
+
group = @pick_next_sql.for_update
|
21
|
+
.skip_locked
|
22
|
+
.limit(1)
|
23
|
+
.select(@group_column)
|
24
|
+
|
25
|
+
# get total from a group, to compare to the number of future locked rows.
|
26
|
+
total_from_group = @ds.where(@group_column => group).count
|
27
|
+
|
28
|
+
event_ids = @ds.where(@group_column => group)
|
29
|
+
.order(Sequel.desc(:run_at, nulls: :first), :id)
|
30
|
+
.for_update.skip_locked.select_map(:id)
|
31
|
+
|
32
|
+
if event_ids.size != total_from_group
|
33
|
+
# this happens if concurrent workers locked different rows from the same group,
|
34
|
+
# or when new rows from a given group have been inserted after the lock has been
|
35
|
+
# acquired
|
36
|
+
event_ids = []
|
37
|
+
end
|
38
|
+
|
39
|
+
# lock all, process 1
|
40
|
+
event_ids[0, 1]
|
41
|
+
end
|
42
|
+
end
|
43
|
+
end
|
44
|
+
|
45
|
+
register_plugin :event_grouping, EventGrouping
|
46
|
+
end
|
47
|
+
end
|
@@ -0,0 +1,46 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module Tobox
|
4
|
+
module Plugins
|
5
|
+
module Inbox
|
6
|
+
def self.configure(conf)
|
7
|
+
conf.config[:inbox_table] = :inbox
|
8
|
+
conf.config[:inbox_column] = :unique_id
|
9
|
+
end
|
10
|
+
|
11
|
+
module FetcherMethods
|
12
|
+
def initialize(_, configuration)
|
13
|
+
super
|
14
|
+
|
15
|
+
inbox_table = configuration[:inbox_table]
|
16
|
+
|
17
|
+
@inbox_ds = @db[inbox_table]
|
18
|
+
@inbox_column = configuration[:inbox_column]
|
19
|
+
end
|
20
|
+
|
21
|
+
private
|
22
|
+
|
23
|
+
def prepare_event(event, &blk)
|
24
|
+
try_insert_inbox(event) { super }
|
25
|
+
end
|
26
|
+
|
27
|
+
def try_insert_inbox(event)
|
28
|
+
if @inbox_ds.respond_to?(:supports_insert_conflict?) && @inbox_ds.supports_insert_conflict?
|
29
|
+
ret = @inbox_ds.insert_conflict.insert(@inbox_column => event[@inbox_column])
|
30
|
+
|
31
|
+
return event unless ret
|
32
|
+
else
|
33
|
+
begin
|
34
|
+
@inbox_ds.insert(@inbox_column => event[@inbox_column])
|
35
|
+
rescue Sequel::UniqueConstraintViolation
|
36
|
+
return event
|
37
|
+
end
|
38
|
+
end
|
39
|
+
|
40
|
+
yield
|
41
|
+
end
|
42
|
+
end
|
43
|
+
end
|
44
|
+
register_plugin :inbox, Inbox
|
45
|
+
end
|
46
|
+
end
|
@@ -0,0 +1,33 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module Tobox
|
4
|
+
module Plugins
|
5
|
+
module Progress
|
6
|
+
def self.configure(conf)
|
7
|
+
conf.config[:visibility_timeout] = 30
|
8
|
+
end
|
9
|
+
|
10
|
+
module FetcherMethods
|
11
|
+
private
|
12
|
+
|
13
|
+
def initialize(_, configuration)
|
14
|
+
super
|
15
|
+
|
16
|
+
@mark_as_fetched_params[:run_at] = Sequel.date_add(
|
17
|
+
Sequel::CURRENT_TIMESTAMP,
|
18
|
+
seconds: configuration[:visibility_timeout]
|
19
|
+
)
|
20
|
+
end
|
21
|
+
|
22
|
+
def events_tr
|
23
|
+
yield
|
24
|
+
end
|
25
|
+
|
26
|
+
def event_ids_tr(&block)
|
27
|
+
@db.transaction(savepoint: false, &block)
|
28
|
+
end
|
29
|
+
end
|
30
|
+
end
|
31
|
+
register_plugin :progress, Progress
|
32
|
+
end
|
33
|
+
end
|
data/lib/tobox/plugins/stats.rb
CHANGED
@@ -25,6 +25,8 @@ module Tobox
|
|
25
25
|
|
26
26
|
config = @config
|
27
27
|
|
28
|
+
plugins = config.plugins.map(&:name)
|
29
|
+
|
28
30
|
interval = config.stats_interval_seconds
|
29
31
|
@stats_handlers = Array(config.lifecycle_events[:stats])
|
30
32
|
|
@@ -34,14 +36,24 @@ module Tobox
|
|
34
36
|
|
35
37
|
@max_attempts = config[:max_attempts]
|
36
38
|
|
39
|
+
@created_at_column = config[:created_at_column]
|
40
|
+
|
37
41
|
@db = Sequel.connect(config.database.opts.merge(max_connections: 1))
|
42
|
+
@db.loggers = config.database.loggers
|
38
43
|
Array(config.lifecycle_events[:database_connect]).each { |cb| cb.call(@db) }
|
39
44
|
|
40
|
-
|
41
|
-
@outbox_ds = @db[
|
45
|
+
outbox_table = config[:table]
|
46
|
+
@outbox_ds = @db[outbox_table]
|
47
|
+
|
48
|
+
if plugins.include?("Tobox::Plugins::Inbox")
|
49
|
+
inbox_table = config[:inbox_table]
|
50
|
+
@inbox_ds = @db[inbox_table]
|
51
|
+
end
|
42
52
|
|
43
|
-
|
44
|
-
|
53
|
+
if @created_at_column
|
54
|
+
# discard already handled events
|
55
|
+
@oldest_event_age_ds = @outbox_ds.where(last_error: nil, run_at: nil).order(Sequel.asc(:id))
|
56
|
+
end
|
45
57
|
|
46
58
|
logger = config.default_logger
|
47
59
|
|
@@ -104,6 +116,13 @@ module Tobox
|
|
104
116
|
stats[:failed_count] ||= 0
|
105
117
|
|
106
118
|
stats[:inbox_count] = @inbox_ds.count if @inbox_ds
|
119
|
+
|
120
|
+
if @oldest_event_age_ds
|
121
|
+
created_at = @oldest_event_age_ds.get(@created_at_column)
|
122
|
+
age = created_at ? (Time.now - created_at).to_i : 0
|
123
|
+
stats[:oldest_event_age_in_seconds] = age
|
124
|
+
end
|
125
|
+
|
107
126
|
stats
|
108
127
|
end
|
109
128
|
end
|
@@ -56,6 +56,8 @@ module Tobox
|
|
56
56
|
if worker.finished? && @running
|
57
57
|
idx = @workers.index(worker)
|
58
58
|
|
59
|
+
raise Error, "worker not found" unless idx
|
60
|
+
|
59
61
|
subst_worker = Worker.new(worker.label, @configuration)
|
60
62
|
@workers[idx] = subst_worker
|
61
63
|
subst_thread = start_thread_worker(subst_worker)
|
data/lib/tobox/pool.rb
CHANGED
data/lib/tobox/version.rb
CHANGED
data/lib/tobox/worker.rb
CHANGED
@@ -8,7 +8,7 @@ module Tobox
|
|
8
8
|
@label = label
|
9
9
|
@wait_for_events_delay = configuration[:wait_for_events_delay]
|
10
10
|
@handlers = configuration.handlers || {}
|
11
|
-
@fetcher =
|
11
|
+
@fetcher = configuration.fetcher_class.new(label, configuration)
|
12
12
|
@finished = false
|
13
13
|
|
14
14
|
return unless (message_to_arguments = configuration.arguments_handler)
|
data/lib/tobox.rb
CHANGED
@@ -9,6 +9,8 @@ require "mutex_m"
|
|
9
9
|
module Tobox
|
10
10
|
class Error < StandardError; end
|
11
11
|
|
12
|
+
EMPTY = [].freeze
|
13
|
+
|
12
14
|
module Plugins
|
13
15
|
@plugins = {}
|
14
16
|
@plugins.extend(Mutex_m)
|
@@ -34,8 +36,8 @@ module Tobox
|
|
34
36
|
end
|
35
37
|
end
|
36
38
|
|
37
|
-
require_relative "tobox/configuration"
|
38
39
|
require_relative "tobox/fetcher"
|
39
40
|
require_relative "tobox/worker"
|
40
41
|
require_relative "tobox/pool"
|
41
42
|
require_relative "tobox/application"
|
43
|
+
require_relative "tobox/configuration"
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: tobox
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.5.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- HoneyryderChuck
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2024-
|
11
|
+
date: 2024-09-16 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: sequel
|
@@ -45,6 +45,9 @@ files:
|
|
45
45
|
- lib/tobox/plugins/datadog/configuration.rb
|
46
46
|
- lib/tobox/plugins/datadog/integration.rb
|
47
47
|
- lib/tobox/plugins/datadog/patcher.rb
|
48
|
+
- lib/tobox/plugins/event_grouping.rb
|
49
|
+
- lib/tobox/plugins/inbox.rb
|
50
|
+
- lib/tobox/plugins/progress.rb
|
48
51
|
- lib/tobox/plugins/sentry.rb
|
49
52
|
- lib/tobox/plugins/stats.rb
|
50
53
|
- lib/tobox/plugins/zeitwerk.rb
|
@@ -71,14 +74,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
71
74
|
requirements:
|
72
75
|
- - ">="
|
73
76
|
- !ruby/object:Gem::Version
|
74
|
-
version: 2.
|
77
|
+
version: 2.7.0
|
75
78
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
76
79
|
requirements:
|
77
80
|
- - ">="
|
78
81
|
- !ruby/object:Gem::Version
|
79
82
|
version: '0'
|
80
83
|
requirements: []
|
81
|
-
rubygems_version: 3.
|
84
|
+
rubygems_version: 3.5.3
|
82
85
|
signing_key:
|
83
86
|
specification_version: 4
|
84
87
|
summary: Transactional outbox pattern implementation in ruby
|