hutch-schedule 0.4.5 → 0.5.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 8203089901a4cac7bc031bf572526f764d3b642b
-  data.tar.gz: 294f22cefb67d4efac6a4dd368a65b424a4daa36
+  metadata.gz: c39d9fb10e3c0a56ec9b631c740a24b377ed7c58
+  data.tar.gz: c1954bbbe01856982cd21e7e522ebbc30b6c086d
 SHA512:
-  metadata.gz: 3f474472b5e1a70806094d7d65fcb9e81fcf1ffdf996ecde04d55bd3f20e7ea6dd0809c30fe7642800269d2656b5e12fda2290fb8a65087bafc3dd6e59372a32
-  data.tar.gz: 1bd7e1485d3f0ca40a98e5fee9529f09cc2611c902e3caeec9aee176fc29238f5b927170bac69005cf59f152dde85b118517d42217f9ee10d57015d90227058a
+  metadata.gz: eded10acdb13ed5bfc9b620cc686c6f82c36ac083d0b03b38362cc96e46d91942e45a863992e317a3b609bbd9ec5437ccb43f16186ce2b2f0fc1f9f7ab5b5dc2
+  data.tar.gz: 66c135271fc10dd710ec7ac4dcc8b220d6de476254f4216c9071f8f85bbd198abf0424b40a374dbec1b7d3bc0961296d7410497fd328c9bac4839f6d19da6d63

data/README.md

@@ -1,6 +1,6 @@
 # Hutch::Schedule
 
-Welcome to your new gem! In this directory, you'll find the files you need to be able to package up your Ruby library into a gem. Put your Ruby code in the file `lib/hutch/schedule`. To experiment with that code, run `bin/console` for an interactive prompt.
+Add the schedule message function to [Hutch](https://github.com/gocardless/hutch).
 
 See [hutch-schedule-demo](https://github.com/wppurking/hutch-schedule-demo) how to integration with rails.
 
@@ -25,18 +25,33 @@ Or install it yourself as:
 Use the code below to initialize the Hutch::Schedule
 
 ```ruby
-Hutch.connect
 Hutch::Schedule.connect
 ```
 
 They will do something below:
 
-1. Declear an topic exchange called `<hutch>.schedule` just for routing message to schedule_queue.
-2. Declear an queue named `<hutch>_schedule_queue` and with some params:
+1. Declear an topic exchange called `<hutch>.schedule` just for routing message to <hutch>_delay_queue_<5s>.
+2. Declear an queue named `<hutch>_delay_queue_<5s>` and with some params:
   - Set `x-dead-letter-exchange: <hutch>`: let queue republish message to default <hutch> exchange.
   - Set `x-message-ttl: <30.days>`: to avoid the queue is to large, because there is no consumer with this queue.
 3. If ActiveJob is loaded. it will use `ActiveJob::Base.descendants` to register all ActiveJob class to one-job-per-consumer to Hutch::Consumer 
 
+### Hutch::Enqueue
+Let consumer to include `Hutch::Enqueue` then it has the ability of publishing message to RabbitMQ with the `consume '<routing_key>'`
+
+* enqueue: just publish one message
+* enqueue_in: publish one message and delay <interval> seconds
+* enqueue_at: publish one message and auto calculate the <interval> seconds need to delay
+
+According to the RabbitMQ [TTL Message design limits](http://www.rabbitmq.com/ttl.html#per-message-ttl-caveats) ([discus](https://github.com/rebus-org/Rebus/issues/594#issuecomment-289961537)),
+We design the fixed delay level from seconds to hours, below is the details:
+
+* seconds(4): 5s, 10s, 20s, 30s
+* minutes(14): 1m, 2m, 3m, 4m, 5m, 6m, 7m, 8m, 9m, 10m, 20m, 30m, 40m, 50m
+* hours(3): 1h, 2h, 3h
+
+RabbitMQ is not fit for storage lot`s of delay message so if you want delay an message beyand 3 hours so you need to storage it
+into database or some place.
 
 ### Error Retry
 If you want use error retry, then:
@@ -101,12 +116,6 @@ EmailJob.perform_later(user.id)
 EmailJob.set(wait: 5.seconds).perform_later(user.id)
 ```
 
-## Development
-
-After checking out the repo, run `rake spec` to run the tests.
-
-To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
-
 ## Contributing
 
 Bug reports and pull requests are welcome on GitHub at https://github.com/wppurking/hutch-schedule. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.

data/lib/hutch/enqueue.rb

@@ -15,11 +15,17 @@ module Hutch
       end
 
       # publish message at a delay times
-      # interval: delay interval
+      # interval: delay interval seconds
       # message: publish message
       def enqueue_in(interval, message, props = {})
-        properties = props.merge(expiration: interval.in_milliseconds.to_i)
-        Hutch::Schedule.publish(enqueue_routing_key, message, properties)
+        # TODO: 超过 3h 的延迟也会接收, 但是不会延迟那么长时间, 但给予 warn
+        delay_seconds = delay_seconds_level(interval)
+
+        # 设置固定的延迟, 利用 headers 中的 CC, 以及区分的 topic, 将消息重新投递进入队列
+        properties = props.merge(expiration: (delay_seconds * 1000).to_i, headers: { :'CC' => [enqueue_routing_key] })
+        delay_routing_key = Hutch::Schedule.delay_routing_key("#{delay_seconds}s")
+
+        Hutch::Schedule.publish(delay_routing_key, message, properties)
       end
 
       # delay at exatly time point
@@ -43,6 +49,59 @@ module Hutch
       def max_attempts
         @max_retries || 0
       end
+
+      # 计算 delay 的 level
+      # 5s 10s 20s 30s
+      # 60s 120s 180s 240s 300s 360s 420s 480s 540s 600s 1200s 1800s 2400s
+      # 3600s 7200s 10800s
+      def delay_seconds_level(delay_seconds)
+        case delay_seconds
+        when 0..5 # 5s
+          5
+        when 5..10 # 10s
+          10
+        when 10..20 # 20s
+          20
+        when 20..30 # 30s
+          30
+        when 30..60 # 60s
+          60
+        when 60..120 # 120s
+          120
+        when 120..180 # 180s
+          180
+        when 180..240 # 240s
+          240
+        when 240..300 # 300s
+          300
+        when 300..360 # 360s
+          360
+        when 360..420 # 420s
+          420
+        when 420..480 # 480s
+          480
+        when 480..540 # 540s
+          540
+        when 540..600 # 600s
+          600
+        when 600..1200 # 1200s
+          1200
+        when 1200..1800 # 1800s
+          1800
+        when 1800..2400 # 2400s
+          2400
+        when 2400..3000 # 3000s
+          3000
+        when 3000..3600 # 3600s
+          3600
+        when 3600..7200 # 7200s
+          7200
+        when 7200..10800 # 10800s
+          10800
+        else
+          10800
+        end
+      end
     end
   end
 end

data/lib/hutch/error_handlers/max_retry.rb

@@ -52,6 +52,8 @@ module Hutch
         end
       end
 
+      # becareful with the RabbitMQ fixed delay level, this retry_dealy seconds will fit to one fixed delay level.
+      # so the max delay time is limit to 3 hours(10800s, error times 11: 14643)
       def retry_delay(executes)
         (executes**4) + 2
       end

data/lib/hutch/schedule/core.rb

@@ -6,9 +6,7 @@ require 'active_support/core_ext/module/delegation'
 module Hutch
   module Schedule
     class Core
-
       attr_reader :broker, :exchange
-
       delegate :channel, :connection, :logger, to: :broker
 
       def initialize(broker)
@@ -16,30 +14,25 @@ module Hutch
         @broker = broker
       end
 
-      # Use the config with Hutch::Broker instance
-      def config
-        broker.instance_variable_get(:@config)
-      end
-
       # Becareful with the sequence of initialize
       def connect!
-        declare_exchange!
+        declare_delay_exchange!
         declare_publisher!
-        setup_queue!
+        setup_delay_queues!
       end
 
       def declare_publisher!
-        @publisher = Hutch::Publisher.new(connection, channel, exchange, config)
+        @publisher = Hutch::Publisher.new(connection, channel, exchange)
       end
 
       # The exchange used by Hutch::Schedule
-      def declare_exchange!
-        @exchange = declare_exchange
+      def declare_delay_exchange!
+        @exchange = declare_delay_exchange
       end
 
-      def declare_exchange(ch = channel)
-        exchange_name    = "#{config[:mq_exchange]}.schedule"
-        exchange_options = { durable: true }.merge(config[:mq_exchange_options])
+      def declare_delay_exchange(ch = channel)
+        exchange_name = "#{Hutch::Config.get(:mq_exchange)}.schedule"
+        exchange_options = { durable: true }.merge(Hutch::Config.get(:mq_exchange_options))
         logger.info "using topic exchange(schedule) '#{exchange_name}'"
 
         broker.send(:with_bunny_precondition_handler, 'schedule exchange') do
@@ -48,14 +41,17 @@ module Hutch
       end
 
       # The queue used by Hutch::Schedule
-      def setup_queue!
+      def setup_delay_queues!
+        DELAY_QUEUES.map { |suffix| setup_delay_queue!(suffix) }
+      end
+
+      def setup_delay_queue!(suffix)
         # TODO: extract the ttl to config params
-        props = { 'x-message-ttl': 30.days.in_milliseconds, 'x-dead-letter-exchange': config[:mq_exchange] }
-        queue = broker.queue("#{config[:mq_exchange]}_schedule_queue", props)
+        props = { :'x-message-ttl' => 30.days.in_milliseconds, :'x-dead-letter-exchange' => Hutch::Config.get(:mq_exchange) }
+        queue = broker.queue(Hutch::Schedule.delay_queue_name(suffix), props)
 
-        # routing all to this queue
-        queue.unbind(exchange, routing_key: '#')
-        queue.bind(exchange, routing_key: '#')
+        # bind routing_key to schedule exchange
+        queue.bind(exchange, routing_key: Hutch::Schedule.delay_routing_key(suffix))
       end
 
       # Schedule`s publisher, publish the message to schedule topic exchange

data/lib/hutch/schedule/version.rb

@@ -1,5 +1,5 @@
 module Hutch
   module Schedule
-    VERSION = "0.4.5"
+    VERSION = "0.5.1"
   end
 end

data/lib/hutch/schedule.rb

@@ -16,20 +16,37 @@ module Hutch
   # and then just use like Hutch to publish message `Hutch::Schedule.publish`
   module Schedule
 
-    def self.connect
-      return if core.present?
-      Hutch.connect unless Hutch.connected?
-      @core = Hutch::Schedule::Core.new(Hutch.broker)
-      @core.connect!
-      ActiveJob::QueueAdapters::HutchAdapter.register_actice_job_classes if defined?(ActiveJob::QueueAdapters::HutchAdapter)
-    end
+    # fixed delay levels
+    # seconds(4): 5s, 10s, 20s, 30s
+    # minutes(14): 1m, 2m, 3m, 4m, 5m, 6m, 7m, 8m, 9m, 10m, 20m, 30m, 40m, 50m
+    # hours(3): 1h, 2h, 3h
+    DELAY_QUEUES = %w(5s 10s 20s 30s 60s 120s 180s 240s 300s 360s 420s 480s 540s 600s 1200s 1800s 2400s 3000s 3600s 7200s 10800s)
 
-    def self.core
-      @core
-    end
+    class << self
+      def connect
+        return if core.present?
+        Hutch.connect unless Hutch.connected?
+        @core = Hutch::Schedule::Core.new(Hutch.broker)
+        @core.connect!
+        ActiveJob::QueueAdapters::HutchAdapter.register_actice_job_classes if defined?(ActiveJob::QueueAdapters::HutchAdapter)
+      end
+
+      def core
+        @core
+      end
+
+      def publish(*args)
+        core.publish(*args)
+      end
+
+      # fixed delay level queue's routing_key
+      def delay_routing_key(suffix)
+        "#{Hutch::Config.get(:mq_exchange)}.schedule.#{suffix}"
+      end
 
-    def self.publish(*args)
-      core.publish(*args)
+      def delay_queue_name(suffix)
+        "#{Hutch::Config.get(:mq_exchange)}_delay_queue_#{suffix}"
+      end
     end
   end
 end

data/lib/hutch-schedule.rb

@@ -1,2 +1,2 @@
 # requrie entry point
-require 'hutch/schedule'
+require 'hutch/schedule'

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: hutch-schedule
 version: !ruby/object:Gem::Version
-  version: 0.4.5
+  version: 0.5.1
 platform: ruby
 authors:
 - Wyatt pan
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2017-07-31 00:00:00.000000000 Z
+date: 2018-04-15 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: hutch
@@ -153,7 +153,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 2.6.11
+rubygems_version: 2.6.13
 signing_key: 
 specification_version: 4
 summary: Add Schedule and Error Retry To Hutch.