hutch-schedule 0.4.5 → 0.5.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +19 -10
- data/lib/hutch/enqueue.rb +62 -3
- data/lib/hutch/error_handlers/max_retry.rb +2 -0
- data/lib/hutch/schedule/core.rb +17 -21
- data/lib/hutch/schedule/version.rb +1 -1
- data/lib/hutch/schedule.rb +29 -12
- data/lib/hutch-schedule.rb +1 -1
- metadata +3 -3
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: c39d9fb10e3c0a56ec9b631c740a24b377ed7c58
|
4
|
+
data.tar.gz: c1954bbbe01856982cd21e7e522ebbc30b6c086d
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: eded10acdb13ed5bfc9b620cc686c6f82c36ac083d0b03b38362cc96e46d91942e45a863992e317a3b609bbd9ec5437ccb43f16186ce2b2f0fc1f9f7ab5b5dc2
|
7
|
+
data.tar.gz: 66c135271fc10dd710ec7ac4dcc8b220d6de476254f4216c9071f8f85bbd198abf0424b40a374dbec1b7d3bc0961296d7410497fd328c9bac4839f6d19da6d63
|
data/README.md
CHANGED
@@ -1,6 +1,6 @@
|
|
1
1
|
# Hutch::Schedule
|
2
2
|
|
3
|
-
|
3
|
+
Add the schedule message function to [Hutch](https://github.com/gocardless/hutch).
|
4
4
|
|
5
5
|
See [hutch-schedule-demo](https://github.com/wppurking/hutch-schedule-demo) how to integration with rails.
|
6
6
|
|
@@ -25,18 +25,33 @@ Or install it yourself as:
|
|
25
25
|
Use the code below to initialize the Hutch::Schedule
|
26
26
|
|
27
27
|
```ruby
|
28
|
-
Hutch.connect
|
29
28
|
Hutch::Schedule.connect
|
30
29
|
```
|
31
30
|
|
32
31
|
They will do something below:
|
33
32
|
|
34
|
-
1. Declear an topic exchange called `<hutch>.schedule` just for routing message to
|
35
|
-
2. Declear an queue named `<hutch>
|
33
|
+
1. Declear an topic exchange called `<hutch>.schedule` just for routing message to <hutch>_delay_queue_<5s>.
|
34
|
+
2. Declear an queue named `<hutch>_delay_queue_<5s>` and with some params:
|
36
35
|
- Set `x-dead-letter-exchange: <hutch>`: let queue republish message to default <hutch> exchange.
|
37
36
|
- Set `x-message-ttl: <30.days>`: to avoid the queue is to large, because there is no consumer with this queue.
|
38
37
|
3. If ActiveJob is loaded. it will use `ActiveJob::Base.descendants` to register all ActiveJob class to one-job-per-consumer to Hutch::Consumer
|
39
38
|
|
39
|
+
### Hutch::Enqueue
|
40
|
+
Let consumer to include `Hutch::Enqueue` then it has the ability of publishing message to RabbitMQ with the `consume '<routing_key>'`
|
41
|
+
|
42
|
+
* enqueue: just publish one message
|
43
|
+
* enqueue_in: publish one message and delay <interval> seconds
|
44
|
+
* enqueue_at: publish one message and auto calculate the <interval> seconds need to delay
|
45
|
+
|
46
|
+
According to the RabbitMQ [TTL Message design limits](http://www.rabbitmq.com/ttl.html#per-message-ttl-caveats) ([discus](https://github.com/rebus-org/Rebus/issues/594#issuecomment-289961537)),
|
47
|
+
We design the fixed delay level from seconds to hours, below is the details:
|
48
|
+
|
49
|
+
* seconds(4): 5s, 10s, 20s, 30s
|
50
|
+
* minutes(14): 1m, 2m, 3m, 4m, 5m, 6m, 7m, 8m, 9m, 10m, 20m, 30m, 40m, 50m
|
51
|
+
* hours(3): 1h, 2h, 3h
|
52
|
+
|
53
|
+
RabbitMQ is not fit for storage lot`s of delay message so if you want delay an message beyand 3 hours so you need to storage it
|
54
|
+
into database or some place.
|
40
55
|
|
41
56
|
### Error Retry
|
42
57
|
If you want use error retry, then:
|
@@ -101,12 +116,6 @@ EmailJob.perform_later(user.id)
|
|
101
116
|
EmailJob.set(wait: 5.seconds).perform_later(user.id)
|
102
117
|
```
|
103
118
|
|
104
|
-
## Development
|
105
|
-
|
106
|
-
After checking out the repo, run `rake spec` to run the tests.
|
107
|
-
|
108
|
-
To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
|
109
|
-
|
110
119
|
## Contributing
|
111
120
|
|
112
121
|
Bug reports and pull requests are welcome on GitHub at https://github.com/wppurking/hutch-schedule. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
|
data/lib/hutch/enqueue.rb
CHANGED
@@ -15,11 +15,17 @@ module Hutch
|
|
15
15
|
end
|
16
16
|
|
17
17
|
# publish message at a delay times
|
18
|
-
# interval: delay interval
|
18
|
+
# interval: delay interval seconds
|
19
19
|
# message: publish message
|
20
20
|
def enqueue_in(interval, message, props = {})
|
21
|
-
|
22
|
-
|
21
|
+
# TODO: 超过 3h 的延迟也会接收, 但是不会延迟那么长时间, 但给予 warn
|
22
|
+
delay_seconds = delay_seconds_level(interval)
|
23
|
+
|
24
|
+
# 设置固定的延迟, 利用 headers 中的 CC, 以及区分的 topic, 将消息重新投递进入队列
|
25
|
+
properties = props.merge(expiration: (delay_seconds * 1000).to_i, headers: { :'CC' => [enqueue_routing_key] })
|
26
|
+
delay_routing_key = Hutch::Schedule.delay_routing_key("#{delay_seconds}s")
|
27
|
+
|
28
|
+
Hutch::Schedule.publish(delay_routing_key, message, properties)
|
23
29
|
end
|
24
30
|
|
25
31
|
# delay at exatly time point
|
@@ -43,6 +49,59 @@ module Hutch
|
|
43
49
|
def max_attempts
|
44
50
|
@max_retries || 0
|
45
51
|
end
|
52
|
+
|
53
|
+
# 计算 delay 的 level
|
54
|
+
# 5s 10s 20s 30s
|
55
|
+
# 60s 120s 180s 240s 300s 360s 420s 480s 540s 600s 1200s 1800s 2400s
|
56
|
+
# 3600s 7200s 10800s
|
57
|
+
def delay_seconds_level(delay_seconds)
|
58
|
+
case delay_seconds
|
59
|
+
when 0..5 # 5s
|
60
|
+
5
|
61
|
+
when 5..10 # 10s
|
62
|
+
10
|
63
|
+
when 10..20 # 20s
|
64
|
+
20
|
65
|
+
when 20..30 # 30s
|
66
|
+
30
|
67
|
+
when 30..60 # 60s
|
68
|
+
60
|
69
|
+
when 60..120 # 120s
|
70
|
+
120
|
71
|
+
when 120..180 # 180s
|
72
|
+
180
|
73
|
+
when 180..240 # 240s
|
74
|
+
240
|
75
|
+
when 240..300 # 300s
|
76
|
+
300
|
77
|
+
when 300..360 # 360s
|
78
|
+
360
|
79
|
+
when 360..420 # 420s
|
80
|
+
420
|
81
|
+
when 420..480 # 480s
|
82
|
+
480
|
83
|
+
when 480..540 # 540s
|
84
|
+
540
|
85
|
+
when 540..600 # 600s
|
86
|
+
600
|
87
|
+
when 600..1200 # 1200s
|
88
|
+
1200
|
89
|
+
when 1200..1800 # 1800s
|
90
|
+
1800
|
91
|
+
when 1800..2400 # 2400s
|
92
|
+
2400
|
93
|
+
when 2400..3000 # 3000s
|
94
|
+
3000
|
95
|
+
when 3000..3600 # 3600s
|
96
|
+
3600
|
97
|
+
when 3600..7200 # 7200s
|
98
|
+
7200
|
99
|
+
when 7200..10800 # 10800s
|
100
|
+
10800
|
101
|
+
else
|
102
|
+
10800
|
103
|
+
end
|
104
|
+
end
|
46
105
|
end
|
47
106
|
end
|
48
107
|
end
|
@@ -52,6 +52,8 @@ module Hutch
|
|
52
52
|
end
|
53
53
|
end
|
54
54
|
|
55
|
+
# becareful with the RabbitMQ fixed delay level, this retry_dealy seconds will fit to one fixed delay level.
|
56
|
+
# so the max delay time is limit to 3 hours(10800s, error times 11: 14643)
|
55
57
|
def retry_delay(executes)
|
56
58
|
(executes**4) + 2
|
57
59
|
end
|
data/lib/hutch/schedule/core.rb
CHANGED
@@ -6,9 +6,7 @@ require 'active_support/core_ext/module/delegation'
|
|
6
6
|
module Hutch
|
7
7
|
module Schedule
|
8
8
|
class Core
|
9
|
-
|
10
9
|
attr_reader :broker, :exchange
|
11
|
-
|
12
10
|
delegate :channel, :connection, :logger, to: :broker
|
13
11
|
|
14
12
|
def initialize(broker)
|
@@ -16,30 +14,25 @@ module Hutch
|
|
16
14
|
@broker = broker
|
17
15
|
end
|
18
16
|
|
19
|
-
# Use the config with Hutch::Broker instance
|
20
|
-
def config
|
21
|
-
broker.instance_variable_get(:@config)
|
22
|
-
end
|
23
|
-
|
24
17
|
# Becareful with the sequence of initialize
|
25
18
|
def connect!
|
26
|
-
|
19
|
+
declare_delay_exchange!
|
27
20
|
declare_publisher!
|
28
|
-
|
21
|
+
setup_delay_queues!
|
29
22
|
end
|
30
23
|
|
31
24
|
def declare_publisher!
|
32
|
-
@publisher = Hutch::Publisher.new(connection, channel, exchange
|
25
|
+
@publisher = Hutch::Publisher.new(connection, channel, exchange)
|
33
26
|
end
|
34
27
|
|
35
28
|
# The exchange used by Hutch::Schedule
|
36
|
-
def
|
37
|
-
@exchange =
|
29
|
+
def declare_delay_exchange!
|
30
|
+
@exchange = declare_delay_exchange
|
38
31
|
end
|
39
32
|
|
40
|
-
def
|
41
|
-
exchange_name
|
42
|
-
exchange_options = { durable: true }.merge(
|
33
|
+
def declare_delay_exchange(ch = channel)
|
34
|
+
exchange_name = "#{Hutch::Config.get(:mq_exchange)}.schedule"
|
35
|
+
exchange_options = { durable: true }.merge(Hutch::Config.get(:mq_exchange_options))
|
43
36
|
logger.info "using topic exchange(schedule) '#{exchange_name}'"
|
44
37
|
|
45
38
|
broker.send(:with_bunny_precondition_handler, 'schedule exchange') do
|
@@ -48,14 +41,17 @@ module Hutch
|
|
48
41
|
end
|
49
42
|
|
50
43
|
# The queue used by Hutch::Schedule
|
51
|
-
def
|
44
|
+
def setup_delay_queues!
|
45
|
+
DELAY_QUEUES.map { |suffix| setup_delay_queue!(suffix) }
|
46
|
+
end
|
47
|
+
|
48
|
+
def setup_delay_queue!(suffix)
|
52
49
|
# TODO: extract the ttl to config params
|
53
|
-
props = { 'x-message-ttl'
|
54
|
-
queue = broker.queue(
|
50
|
+
props = { :'x-message-ttl' => 30.days.in_milliseconds, :'x-dead-letter-exchange' => Hutch::Config.get(:mq_exchange) }
|
51
|
+
queue = broker.queue(Hutch::Schedule.delay_queue_name(suffix), props)
|
55
52
|
|
56
|
-
#
|
57
|
-
queue.
|
58
|
-
queue.bind(exchange, routing_key: '#')
|
53
|
+
# bind routing_key to schedule exchange
|
54
|
+
queue.bind(exchange, routing_key: Hutch::Schedule.delay_routing_key(suffix))
|
59
55
|
end
|
60
56
|
|
61
57
|
# Schedule`s publisher, publish the message to schedule topic exchange
|
data/lib/hutch/schedule.rb
CHANGED
@@ -16,20 +16,37 @@ module Hutch
|
|
16
16
|
# and then just use like Hutch to publish message `Hutch::Schedule.publish`
|
17
17
|
module Schedule
|
18
18
|
|
19
|
-
|
20
|
-
|
21
|
-
|
22
|
-
|
23
|
-
|
24
|
-
ActiveJob::QueueAdapters::HutchAdapter.register_actice_job_classes if defined?(ActiveJob::QueueAdapters::HutchAdapter)
|
25
|
-
end
|
19
|
+
# fixed delay levels
|
20
|
+
# seconds(4): 5s, 10s, 20s, 30s
|
21
|
+
# minutes(14): 1m, 2m, 3m, 4m, 5m, 6m, 7m, 8m, 9m, 10m, 20m, 30m, 40m, 50m
|
22
|
+
# hours(3): 1h, 2h, 3h
|
23
|
+
DELAY_QUEUES = %w(5s 10s 20s 30s 60s 120s 180s 240s 300s 360s 420s 480s 540s 600s 1200s 1800s 2400s 3000s 3600s 7200s 10800s)
|
26
24
|
|
27
|
-
|
28
|
-
|
29
|
-
|
25
|
+
class << self
|
26
|
+
def connect
|
27
|
+
return if core.present?
|
28
|
+
Hutch.connect unless Hutch.connected?
|
29
|
+
@core = Hutch::Schedule::Core.new(Hutch.broker)
|
30
|
+
@core.connect!
|
31
|
+
ActiveJob::QueueAdapters::HutchAdapter.register_actice_job_classes if defined?(ActiveJob::QueueAdapters::HutchAdapter)
|
32
|
+
end
|
33
|
+
|
34
|
+
def core
|
35
|
+
@core
|
36
|
+
end
|
37
|
+
|
38
|
+
def publish(*args)
|
39
|
+
core.publish(*args)
|
40
|
+
end
|
41
|
+
|
42
|
+
# fixed delay level queue's routing_key
|
43
|
+
def delay_routing_key(suffix)
|
44
|
+
"#{Hutch::Config.get(:mq_exchange)}.schedule.#{suffix}"
|
45
|
+
end
|
30
46
|
|
31
|
-
|
32
|
-
|
47
|
+
def delay_queue_name(suffix)
|
48
|
+
"#{Hutch::Config.get(:mq_exchange)}_delay_queue_#{suffix}"
|
49
|
+
end
|
33
50
|
end
|
34
51
|
end
|
35
52
|
end
|
data/lib/hutch-schedule.rb
CHANGED
@@ -1,2 +1,2 @@
|
|
1
1
|
# requrie entry point
|
2
|
-
require 'hutch/schedule'
|
2
|
+
require 'hutch/schedule'
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: hutch-schedule
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.5.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Wyatt pan
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2018-04-15 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: hutch
|
@@ -153,7 +153,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
153
153
|
version: '0'
|
154
154
|
requirements: []
|
155
155
|
rubyforge_project:
|
156
|
-
rubygems_version: 2.6.
|
156
|
+
rubygems_version: 2.6.13
|
157
157
|
signing_key:
|
158
158
|
specification_version: 4
|
159
159
|
summary: Add Schedule and Error Retry To Hutch.
|