good_job 0.5.0 → 0.6.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: c11065689d3442c14ab8afb08ad17a96c45414287e9c24440d26be766eff3566
-  data.tar.gz: 316168ca2ecf9aa9a239d007b85b589c34ec68064ad3ec376665bb893aceae4c
+  metadata.gz: c20439024a46084c22e46bb42e38589687c2ab2cea47d90705abc4fa042ad449
+  data.tar.gz: e5ea8e7fa607a10fd936c62623a39d32b891ef629636e0b04fb3d36aa7ab1f3f
 SHA512:
-  metadata.gz: 9cb0b54f3e2a7189684fb5f3ea98ea4febd6e1651ff7f9f47b78be19e9a4948c731c9e990e89921b3763236a33cbf45905d8c05c417d5044caaf844b67600510
-  data.tar.gz: 8f45cc0b5ccd1c61c3e55ac748359156d3f4081646537b327476bd964e15e45e6a953c01ae0f237b0c08fd7bcbd3bd94bf9d7bb484c45d26600790f7870b9ccf
+  metadata.gz: 4af34ce400c409a61892c9d35693f69e023b5dbd3d54cacfd6d9b044aaf481c616e3fdb9da53470f79919830a916a08f67550ce807eda1c59a43fa1da4092f07
+  data.tar.gz: 81304be6bc9e6925b16fa030d2031377bd90bac894b4c8792cfe5e02ff81677bf04d476b1a3435e951a804e6d284e4d8aa7452c6090969f841dbd6e59a9c301c

data/CHANGELOG.md

@@ -1,14 +1,30 @@
 # Changelog
 
-## [0.5.0](https://github.com/bensheldon/good_job/tree/0.5.0) (2020-07-13)
+## [v0.6.0](https://github.com/bensheldon/good_job/tree/v0.6.0) (2020-07-15)
 
-[Full Changelog](https://github.com/bensheldon/good_job/compare/v0.4.0...0.5.0)
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v0.5.0...v0.6.0)
+
+**Closed issues:**
+
+- Improve the command line options [\#32](https://github.com/bensheldon/good_job/issues/32)
+- Allow config.active\_job.queue\_adapter = :good\_job to work [\#5](https://github.com/bensheldon/good_job/issues/5)
 
 **Merged pull requests:**
 
-- Update development Ruby to 2.6.6 and gems [\#29](https://github.com/bensheldon/good_job/pull/29) ([bensheldon](https://github.com/bensheldon))
+- Improve generation of changelog [\#36](https://github.com/bensheldon/good_job/pull/36) ([bensheldon](https://github.com/bensheldon))
+- Update Github Action Workflow for Backlog Project Board [\#35](https://github.com/bensheldon/good_job/pull/35) ([bensheldon](https://github.com/bensheldon))
+- Add configuration options to good\_job executable [\#33](https://github.com/bensheldon/good_job/pull/33) ([bensheldon](https://github.com/bensheldon))
+- Extract Job querying behavior out of Scheduler [\#31](https://github.com/bensheldon/good_job/pull/31) ([bensheldon](https://github.com/bensheldon))
 - Allow configuration of Rails queue adapter with `:good\_job` [\#28](https://github.com/bensheldon/good_job/pull/28) ([bensheldon](https://github.com/bensheldon))
 
+## [v0.5.0](https://github.com/bensheldon/good_job/tree/v0.5.0) (2020-07-13)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v0.4.0...v0.5.0)
+
+**Merged pull requests:**
+
+- Update development Ruby to 2.6.6 and gems [\#29](https://github.com/bensheldon/good_job/pull/29) ([bensheldon](https://github.com/bensheldon))
+
 ## [v0.4.0](https://github.com/bensheldon/good_job/tree/v0.4.0) (2020-03-31)
 
 [Full Changelog](https://github.com/bensheldon/good_job/compare/v0.3.0...v0.4.0)
@@ -46,6 +62,7 @@
 
 - Clean up Gemspec [\#15](https://github.com/bensheldon/good_job/pull/15) ([bensheldon](https://github.com/bensheldon))
 - Set up Rubocop [\#14](https://github.com/bensheldon/good_job/pull/14) ([bensheldon](https://github.com/bensheldon))
+- Add pg gem as explicit dependency [\#13](https://github.com/bensheldon/good_job/pull/13) ([bensheldon](https://github.com/bensheldon))
 - Bump nokogiri from 1.10.7 to 1.10.9 [\#12](https://github.com/bensheldon/good_job/pull/12) ([dependabot[bot]](https://github.com/apps/dependabot))
 - Add Appraisal with tests for Rails 5.1, 5.2, 6.0 [\#11](https://github.com/bensheldon/good_job/pull/11) ([bensheldon](https://github.com/bensheldon))
 
@@ -55,7 +72,6 @@
 
 **Merged pull requests:**
 
-- Add pg gem as explicit dependency [\#13](https://github.com/bensheldon/good_job/pull/13) ([bensheldon](https://github.com/bensheldon))
 - Use Rails.logger and ActiveSupport::Notifications for logging instead of puts [\#10](https://github.com/bensheldon/good_job/pull/10) ([bensheldon](https://github.com/bensheldon))
 - Remove minitest files [\#9](https://github.com/bensheldon/good_job/pull/9) ([bensheldon](https://github.com/bensheldon))
 - Use scheduled\_at and priority for scheduling [\#8](https://github.com/bensheldon/good_job/pull/8) ([bensheldon](https://github.com/bensheldon))

data/README.md

@@ -78,6 +78,19 @@ $ bundle install
     $ bundle exec good_job
     ```
    
+   Configuration options available with `help`:
+   ```bash
+   $ bundle exec good_job help start
+   
+   # Usage:
+   #  good_job start
+   #
+   # Options:
+   #   [--max-threads=N]         # Maximum number of threads to use for working jobs (default: ActiveRecord::Base.connection_pool.size)
+   #   [--queues=queue1,queue2]  # Queues to work from. Separate multiple queues with commas (default: *)
+   #   [--poll-interval=N]       # Interval between polls for available jobs in seconds (default: 1)
+   ```
+   
 ### Configuring Job Execution Threads
     
 GoodJob executes enqueued jobs using threads. There is a lot than can be said about [multithreaded behavior in Ruby on Rails](https://guides.rubyonrails.org/threading_and_code_execution.html), but briefly:
@@ -136,7 +149,7 @@ Package maintainers can release this gem with the following [gem-release](https:
 $ gem signin
 
 # Update version number, changelog, and create git commit:
-$ bundle exec rake commit_version
+$ bundle exec rake commit_version[minor] # major,minor,patch
 
 # ..and follow subsequent directions. 
 ```

data/lib/good_job/adapter.rb

@@ -16,8 +16,11 @@ module GoodJob
       )
 
       if inline?
-        good_job.perform
-        good_job.advisory_unlock
+        begin
+          good_job.perform
+        ensure
+          good_job.advisory_unlock
+        end
       end
 
       good_job

data/lib/good_job/cli.rb

@@ -4,23 +4,59 @@ module GoodJob
   class CLI < Thor
     RAILS_ENVIRONMENT_RB = File.expand_path("config/environment.rb")
 
-    desc :start, "Start jobs"
-    method_option :max_threads, type: :numeric
+    desc :start, "Start job worker"
+    method_option :max_threads,
+                  type: :numeric,
+                  desc: "Maximum number of threads to use for working jobs (default: ActiveRecord::Base.connection_pool.size)"
+    method_option :queues,
+                  type: :string,
+                  banner: "queue1,queue2",
+                  desc: "Queues to work from. Separate multiple queues with commas (default: *)"
+    method_option :poll_interval,
+                  type: :numeric,
+                  desc: "Interval between polls for available jobs in seconds (default: 1)"
     def start
       require RAILS_ENVIRONMENT_RB
 
-      max_threads = options[:max_threads] ||
-                    ENV['GOOD_JOB_MAX_THREADS'] ||
-                    ENV['RAILS_MAX_THREADS'] ||
-                    ActiveRecord::Base.connection_pool.size
+      max_threads = (
+        options[:max_threads] ||
+        ENV['GOOD_JOB_MAX_THREADS'] ||
+        ENV['RAILS_MAX_THREADS'] ||
+        ActiveRecord::Base.connection_pool.size
+      ).to_i
 
-      $stdout.puts "GoodJob starting with max_threads=#{max_threads}"
-      scheduler = GoodJob::Scheduler.new(pool_options: { max_threads: max_threads })
+      queue_names = (
+        options[:queues] ||
+        ENV['GOOD_JOB_QUEUES'] ||
+        '*'
+      ).split(',').map(&:strip)
 
+      poll_interval = (
+        options[:poll_interval] ||
+        ENV['GOOD_JOB_POLL_INTERVAL']
+      ).to_i
+
+      job_query = GoodJob::Job.all
+      queue_names_without_all = queue_names.reject { |q| q == '*' }
+      job_query = job_query.where(queue_name: queue_names_without_all) unless queue_names_without_all.size.zero?
+
+      job_performer = job_query.only_scheduled.priority_ordered.to_performer
+
+      $stdout.puts "GoodJob worker starting with max_threads=#{max_threads} on queues=#{queue_names.join(',')}"
+
+      timer_options = {}
+      timer_options[:execution_interval] = poll_interval if poll_interval.positive?
+
+      pool_options = {
+        max_threads: max_threads,
+      }
+
+      scheduler = GoodJob::Scheduler.new(job_performer, timer_options: timer_options, pool_options: pool_options)
+
+      @stop_good_job_executable = false
       %w[INT TERM].each do |signal|
         trap(signal) { @stop_good_job_executable = true }
       end
-      @stop_good_job_executable = false
 
       Kernel.loop do
         sleep 0.1

data/lib/good_job/job.rb

@@ -4,6 +4,29 @@ module GoodJob
 
     self.table_name = 'good_jobs'
 
+    scope :only_scheduled, -> { where("scheduled_at < ?", Time.current).or(where(scheduled_at: nil)) }
+    scope :priority_ordered, -> { order(priority: :desc) }
+    scope :to_performer, -> { Performer.new(self) }
+
+    class Performer
+      def initialize(query)
+        @query = query
+      end
+
+      def next
+        good_job = nil
+
+        @query.limit(1).with_advisory_lock do |good_jobs|
+          good_job = good_jobs.first
+          break unless good_job
+
+          good_job.perform
+        end
+
+        good_job
+      end
+    end
+
     def self.enqueue(active_job, scheduled_at: nil, create_with_advisory_lock: false)
       good_job = nil
       ActiveSupport::Notifications.instrument("enqueue_job.good_job", { active_job: active_job, scheduled_at: scheduled_at, create_with_advisory_lock: create_with_advisory_lock }) do |instrument_payload|

data/lib/good_job/scheduler.rb

@@ -20,9 +20,10 @@ module GoodJob
       fallback_policy: :abort, # shouldn't matter -- 0 max queue
     }.freeze
 
-    def initialize(query = GoodJob::Job.all, timer_options: {}, pool_options: {})
-      @query = query
+    def initialize(performer, timer_options: {}, pool_options: {})
+      raise ArgumentError, "Performer argument must implement #next" unless performer.respond_to?(:next)
 
+      @performer = performer
       @pool = Concurrent::ThreadPoolExecutor.new(DEFAULT_POOL_OPTIONS.merge(pool_options))
       @timer = Concurrent::TimerTask.new(DEFAULT_TIMER_OPTIONS.merge(timer_options)) do
         idle_threads = @pool.max_length - @pool.length
@@ -32,10 +33,6 @@ module GoodJob
       @timer.execute
     end
 
-    def ordered_query
-      @query.where("scheduled_at < ?", Time.current).or(@query.where(scheduled_at: nil)).order(priority: :desc)
-    end
-
     def execute
     end
 
@@ -61,19 +58,10 @@ module GoodJob
     end
 
     def create_thread
-      future = Concurrent::Future.new(args: [ordered_query], executor: @pool) do |query|
-        good_job = nil
-
-        Rails.application.executor.wrap do
-          query.limit(1).with_advisory_lock do |good_jobs|
-            good_job = good_jobs.first
-            break unless good_job
-
-            good_job.perform
-          end
-        end
-
-        good_job
+      future = Concurrent::Future.new(args: [@performer], executor: @pool) do |performer|
+        result = nil
+        Rails.application.executor.wrap { result = performer.next }
+        result
       end
       future.add_observer(self, :task_observer)
       future.execute
@@ -83,9 +71,9 @@ module GoodJob
       ActiveSupport::Notifications.instrument("finished_timer_task.good_job", { result: executed_task, error: error, time: time })
     end
 
-    def task_observer(time, performed_job, error)
-      ActiveSupport::Notifications.instrument("finished_job_task.good_job", { good_job: performed_job, error: error, time: time })
-      create_thread if performed_job
+    def task_observer(time, result, error)
+      ActiveSupport::Notifications.instrument("finished_job_task.good_job", { result: result, error: error, time: time })
+      create_thread if result
     end
   end
 end

data/lib/good_job/version.rb

@@ -1,3 +1,3 @@
 module GoodJob
-  VERSION = '0.5.0'.freeze
+  VERSION = '0.6.0'.freeze
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: good_job
 version: !ruby/object:Gem::Version
-  version: 0.5.0
+  version: 0.6.0
 platform: ruby
 authors:
 - Ben Sheldon
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2020-07-13 00:00:00.000000000 Z
+date: 2020-07-15 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: concurrent-ruby