good_job 1.99.1 → 2.0.3

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 97662bae9e8ba65e5ed3e9ba52b8b987d77974866885a548a8a3b13878045c7a
4
- data.tar.gz: 415d36696796cfb4c0ff1737c7c75e2b66bbfceea15f3a90aba55b49ee5da715
3
+ metadata.gz: d4c1686f53ec7019417a342069c499c829dccb4f8cfcbd7396623b96e9edd2fe
4
+ data.tar.gz: de4e3d7a20e47c63aad25dd167a640b83a9349f0b8933d4692afc6f8b12d16d3
5
5
  SHA512:
6
- metadata.gz: 2d293eabb7b926386b30a1348c529de6a2252c1c734b7a92a81abad2fdd791a28abe5067850c9a4f7595182afd72d52eccc0c640d7596aaddcba467a995449ca
7
- data.tar.gz: 4b981bda934dc7e58aab80bd998c16b9c8261309cf8f5ffce09253c45821f0106ef0db0249934a3649e735715d865ff6dd62232876afb7ffca76cfb4fe473179
6
+ metadata.gz: dfe017b7ea652b134c009622e5a7b1d83e7d502a4f066b49538ffca138555f4395d6279f7256a10c872e1967aa127bb82eba17a72fbf8930adcd12da9650dbf4
7
+ data.tar.gz: 1e5178b1d9ddf492347e6f25b10bae710dc25454974436fad322b306698a25bdf29d777f54de5c988479b56a21d2b0a8cfdc7b7f87799ae5cf6f7df4c76f4b6b
data/CHANGELOG.md CHANGED
@@ -1,18 +1,39 @@
1
1
  # Changelog
2
2
 
3
- ## [v1.99.1](https://github.com/bensheldon/good_job/tree/v1.99.1) (2021-08-27)
3
+ ## [v2.0.3](https://github.com/bensheldon/good_job/tree/v2.0.3) (2021-08-31)
4
4
 
5
- [Full Changelog](https://github.com/bensheldon/good_job/compare/v2.0.1...v1.99.1)
5
+ [Full Changelog](https://github.com/bensheldon/good_job/compare/v2.0.2...v2.0.3)
6
6
 
7
7
  **Closed issues:**
8
8
 
9
- - Does Good job support delay method? [\#344](https://github.com/bensheldon/good_job/issues/344)
9
+ - Expose CLI `cleanup_preserved_jobs` functionality via `GoodJob`? [\#351](https://github.com/bensheldon/good_job/issues/351)
10
10
 
11
11
  **Merged pull requests:**
12
12
 
13
+ - Implement `GoodJob.cleanup_preserved_jobs`, fixes \#351 [\#356](https://github.com/bensheldon/good_job/pull/356) ([aried3r](https://github.com/aried3r))
14
+
15
+ ## [v2.0.2](https://github.com/bensheldon/good_job/tree/v2.0.2) (2021-08-27)
16
+
17
+ [Full Changelog](https://github.com/bensheldon/good_job/compare/v1.99.1...v2.0.2)
18
+
19
+ **Closed issues:**
20
+
21
+ - Migrations generator assumes migrations are in db/migrate [\#352](https://github.com/bensheldon/good_job/issues/352)
22
+
23
+ **Merged pull requests:**
24
+
25
+ - v2.0: Generators support multiple databases: `--database` option, `migrations_paths`, custom `GoodJob.active_record_parent_class` [\#354](https://github.com/bensheldon/good_job/pull/354) ([bensheldon](https://github.com/bensheldon))
13
26
  - README style/typo fixes: "web server" and possessive "Rails'" [\#350](https://github.com/bensheldon/good_job/pull/350) ([aried3r](https://github.com/aried3r))
14
27
  - Add examples of setting config.good\_job.queues [\#349](https://github.com/bensheldon/good_job/pull/349) ([zachmargolis](https://github.com/zachmargolis))
15
28
 
29
+ ## [v1.99.1](https://github.com/bensheldon/good_job/tree/v1.99.1) (2021-08-27)
30
+
31
+ [Full Changelog](https://github.com/bensheldon/good_job/compare/v2.0.1...v1.99.1)
32
+
33
+ **Closed issues:**
34
+
35
+ - Does Good job support delay method? [\#344](https://github.com/bensheldon/good_job/issues/344)
36
+
16
37
  ## [v2.0.1](https://github.com/bensheldon/good_job/tree/v2.0.1) (2021-08-24)
17
38
 
18
39
  [Full Changelog](https://github.com/bensheldon/good_job/compare/v2.0.0...v2.0.1)
data/README.md CHANGED
@@ -132,10 +132,10 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
132
132
  - GoodJob can also be configured to execute jobs within the web server process to save on resources. This is useful for low-workloads when economy is paramount.
133
133
 
134
134
  ```
135
- $ GOOD_JOB_EXECUTION_MODE=async_server rails server
135
+ $ GOOD_JOB_EXECUTION_MODE=async rails server
136
136
  ```
137
137
 
138
- Additional configuration is likely necessary, see the reference below for f configuration.
138
+ Additional configuration is likely necessary, see the reference below for configuration.
139
139
 
140
140
  ## Compatibility
141
141
 
@@ -184,7 +184,7 @@ separate isolated execution pools with semicolons and threads with colons.
184
184
 
185
185
  #### `good_job cleanup_preserved_jobs`
186
186
 
187
- `good_job cleanup_preserved_jobs` deletes preserved job records. See [`GoodJob.preserve_job_records` for when this command is useful.
187
+ `good_job cleanup_preserved_jobs` deletes preserved job records. See `GoodJob.preserve_job_records` for when this command is useful.
188
188
 
189
189
  ```bash
190
190
  $ bundle exec good_job help cleanup_preserved_jobs
@@ -220,16 +220,17 @@ Additional configuration can be provided via `config.good_job.OPTION = ...` for
220
220
  config.active_job.queue_adapter = :good_job
221
221
 
222
222
  # Configure options individually...
223
- config.good_job.execution_mode = :async_server
223
+ config.good_job.execution_mode = :async
224
224
  config.good_job.max_threads = 5
225
225
  config.good_job.poll_interval = 30 # seconds
226
226
  config.good_job.shutdown_timeout = 25 # seconds
227
227
  config.good_job.enable_cron = true
228
228
  config.good_job.cron = { example: { cron: '0 * * * *', class: 'ExampleJob' } }
229
+ config.good_job.queues = '*'
229
230
 
230
231
  # ...or all at once.
231
232
  config.good_job = {
232
- execution_mode: :async_server,
233
+ execution_mode: :async,
233
234
  max_threads: 5,
234
235
  poll_interval: 30,
235
236
  shutdown_timeout: 25,
@@ -240,6 +241,7 @@ config.good_job = {
240
241
  class: 'ExampleJob'
241
242
  },
242
243
  },
244
+ queues: '*',
243
245
  }
244
246
  ```
245
247
 
@@ -248,11 +250,11 @@ Available configuration options are:
248
250
  - `execution_mode` (symbol) specifies how and where jobs should be executed. You can also set this with the environment variable `GOOD_JOB_EXECUTION_MODE`. It can be any one of:
249
251
  - `:inline` executes jobs immediately in whatever process queued them (usually the web server process). This should only be used in test and development environments.
250
252
  - `:external` causes the adapter to enqueue jobs, but not execute them. When using this option (the default for production environments), you’ll need to use the command-line tool to actually execute your jobs.
251
- - `:async_server` executes jobs in separate threads within the Rails webserver process (`bundle exec rails server`). It can be more economical for small workloads because you don’t need a separate machine or environment for running your jobs, but if your web server is under heavy load or your jobs require a lot of resources, you should choose `:external` instead. When not in the Rails webserver, jobs will execute in `:external` mode to ensure jobs are not executed within `rails console`, `rails db:migrate`, `rails assets:prepare`, etc.
252
- - `:async` executes jobs in separate threads in _any_ Rails process.
253
- - `max_threads` (integer) sets the maximum number of threads to use when `execution_mode` is set to `:async` or `:async_server`. You can also set this with the environment variable `GOOD_JOB_MAX_THREADS`.
254
- - `queues` (string) determines which queues to execute jobs from when `execution_mode` is set to `:async` or `:async_server`. See the description of `good_job start` for more details on the format of this string. You can also set this with the environment variable `GOOD_JOB_QUEUES`.
255
- - `poll_interval` (integer) sets the number of seconds between polls for jobs when `execution_mode` is set to `:async` or `:async_server`. You can also set this with the environment variable `GOOD_JOB_POLL_INTERVAL`.
253
+ - `:async` (or `:async_server`) executes jobs in separate threads within the Rails web server process (`bundle exec rails server`). It can be more economical for small workloads because you don’t need a separate machine or environment for running your jobs, but if your web server is under heavy load or your jobs require a lot of resources, you should choose `:external` instead. When not in the Rails web server, jobs will execute in `:external` mode to ensure jobs are not executed within `rails console`, `rails db:migrate`, `rails assets:prepare`, etc.
254
+ - `:async_all` executes jobs in separate threads in _any_ Rails process.
255
+ - `max_threads` (integer) sets the maximum number of threads to use when `execution_mode` is set to `:async`. You can also set this with the environment variable `GOOD_JOB_MAX_THREADS`.
256
+ - `queues` (string) determines which queues to execute jobs from when `execution_mode` is set to `:async`. See the description of `good_job start` for more details on the format of this string. You can also set this with the environment variable `GOOD_JOB_QUEUES`.
257
+ - `poll_interval` (integer) sets the number of seconds between polls for jobs when `execution_mode` is set to `:async`. You can also set this with the environment variable `GOOD_JOB_POLL_INTERVAL`.
256
258
  - `max_cache` (integer) sets the maximum number of scheduled jobs that will be stored in memory to reduce execution latency when also polling for scheduled jobs. Caching 10,000 scheduled jobs uses approximately 20MB of memory. You can also set this with the environment variable `GOOD_JOB_MAX_CACHE`.
257
259
  - `shutdown_timeout` (float) number of seconds to wait for jobs to finish when shutting down before stopping the thread. Defaults to forever: `-1`. You can also set this with the environment variable `GOOD_JOB_SHUTDOWN_TIMEOUT`.
258
260
  - `enable_cron` (boolean) whether to run cron process. Defaults to `false`. You can also set this with the environment variable `GOOD_JOB_ENABLE_CRON`.
@@ -264,7 +266,7 @@ By default, GoodJob configures the following execution modes per environment:
264
266
 
265
267
  # config/environments/development.rb
266
268
  config.active_job.queue_adapter = :good_job
267
- config.good_job.execution_mode = :inline
269
+ config.good_job.execution_mode = :async
268
270
 
269
271
  # config/environments/test.rb
270
272
  config.active_job.queue_adapter = :good_job
@@ -394,7 +396,7 @@ config.good_job.enable_cron = ENV['DYNO'] == 'worker.1' # or `true` or via $GOOD
394
396
 
395
397
  # Configure cron with a hash that has a unique key for each recurring job
396
398
  config.good_job.cron = {
397
- # Every 15 minutes, enqueue `ExampleJob.set(priority: -10).perform_later(52, name: "Alice")`
399
+ # Every 15 minutes, enqueue `ExampleJob.set(priority: -10).perform_later(42, name: "Alice")`
398
400
  frequent_task: { # each recurring job must have a unique key
399
401
  cron: "*/15 * * * *", # cron-style scheduling format by fugit gem
400
402
  class: "ExampleJob", # reference the Job class with a string
@@ -443,8 +445,7 @@ To perform upgrades to the GoodJob database tables:
443
445
 
444
446
  #### Upgrading v1 to v2
445
447
 
446
- GoodJob v2 introduces a new Advisory Lock key format that is different than the v1 advisory lock key format; it's therefore necessary to perform a simple, but staged production upgrade. If you are already using `>= v1.12+` no other changes are likely44
447
- necessary.
448
+ GoodJob v2 introduces a new Advisory Lock key format that is different than the v1 advisory lock key format; it's therefore necessary to perform a simple, but staged production upgrade. If you are already using `>= v1.12+` no other changes are necessary.
448
449
 
449
450
  1. Upgrade your production environment to `v1.99.x` following the minor version upgrade process, including database migrations. `v1.99` is a transitional release that is safely compatible with both `v1.x` and `v2.0.0` because it uses both `v1`- and `v2`-formatted advisory locks.
450
451
  1. Address any deprecation warnings generated by `v1.99`.
@@ -454,6 +455,7 @@ Notable changes:
454
455
 
455
456
  - Renames `:async_server` execution mode to `:async`; renames prior `:async` execution mode to `:async_all`.
456
457
  - Sets default Development environment's execution mode to `:async` with disabled polling.
458
+ - Excludes performing jobs from `enqueue_limit`'s count in `GoodJob::ActiveJobExtensions::Concurrency`.
457
459
  - Triggers `GoodJob.on_thread_error` for unhandled ActiveJob exceptions.
458
460
  - Renames `GoodJob.reperform_jobs_on_standard_error` accessor to `GoodJob.retry_on_unhandled_error`.
459
461
  - Renames `GoodJob::Adapter.shutdown(wait:)` argument to `GoodJob::Adapter.shutdown(timeout:)`.
@@ -612,7 +614,7 @@ Keep in mind, queue operations and management is an advanced discipline. This st
612
614
 
613
615
  ### Database connections
614
616
 
615
- Each GoodJob execution thread requires its own database connection that is automatically checked out from Rails’s connection pool. _Allowing GoodJob to create more threads than available database connections can lead to timeouts and is not recommended._ For example:
617
+ Each GoodJob execution thread requires its own database connection that is automatically checked out from Rails’ connection pool. _Allowing GoodJob to create more threads than available database connections can lead to timeouts and is not recommended._ For example:
616
618
 
617
619
  ```yaml
618
620
  # config/database.yml
@@ -621,7 +623,7 @@ pool: <%= [ENV.fetch("RAILS_MAX_THREADS", 5).to_i, ENV.fetch("GOOD_JOB_MAX_THREA
621
623
 
622
624
  ### Execute jobs async / in-process
623
625
 
624
- GoodJob can execute jobs "async" in the same process as the webserver (e.g. `bin/rail s`). GoodJob's async execution mode offers benefits of economy by not requiring a separate job worker process, but with the tradeoff of increased complexity. Async mode can be configured in two ways:
626
+ GoodJob can execute jobs "async" in the same process as the web server (e.g. `bin/rails s`). GoodJob's async execution mode offers benefits of economy by not requiring a separate job worker process, but with the tradeoff of increased complexity. Async mode can be configured in two ways:
625
627
 
626
628
  - Via Rails configuration:
627
629
 
@@ -630,11 +632,11 @@ GoodJob can execute jobs "async" in the same process as the webserver (e.g. `bin
630
632
  config.active_job.queue_adapter = :good_job
631
633
 
632
634
  # To change the execution mode
633
- config.good_job.execution_mode = :async_server
635
+ config.good_job.execution_mode = :async
634
636
 
635
637
  # Or with more configuration
636
638
  config.good_job = {
637
- execution_mode: :async_server,
639
+ execution_mode: :async,
638
640
  max_threads: 4,
639
641
  poll_interval: 30
640
642
  }
@@ -643,7 +645,7 @@ GoodJob can execute jobs "async" in the same process as the webserver (e.g. `bin
643
645
  - Or, with environment variables:
644
646
 
645
647
  ```bash
646
- $ GOOD_JOB_EXECUTION_MODE=async_server GOOD_JOB_MAX_THREADS=4 GOOD_JOB_POLL_INTERVAL=30 bin/rails server
648
+ $ GOOD_JOB_EXECUTION_MODE=async GOOD_JOB_MAX_THREADS=4 GOOD_JOB_POLL_INTERVAL=30 bin/rails server
647
649
  ```
648
650
 
649
651
  Depending on your application configuration, you may need to take additional steps:
@@ -655,7 +657,7 @@ Depending on your application configuration, you may need to take additional ste
655
657
  pool: <%= ENV.fetch("RAILS_MAX_THREADS", 5).to_i + ENV.fetch("GOOD_JOB_MAX_THREADS", 4).to_i %>
656
658
  ```
657
659
 
658
- - When running Puma with workers (`WEB_CONCURRENCY > 0`) or another process-forking webserver, GoodJob's threadpool schedulers should be stopped before forking, restarted after fork, and cleanly shut down on exit. Stopping GoodJob's scheduler pre-fork is recommended to ensure that GoodJob does not continue executing jobs in the parent/controller process. For example, with Puma:
660
+ - When running Puma with workers (`WEB_CONCURRENCY > 0`) or another process-forking web server, GoodJob's threadpool schedulers should be stopped before forking, restarted after fork, and cleanly shut down on exit. Stopping GoodJob's scheduler pre-fork is recommended to ensure that GoodJob does not continue executing jobs in the parent/controller process. For example, with Puma:
659
661
 
660
662
  ```ruby
661
663
  # config/puma.rb
@@ -722,7 +724,8 @@ It is also necessary to delete these preserved jobs from the database after a ce
722
724
  - For example, in a Rake task:
723
725
 
724
726
  ```ruby
725
- GoodJob::Job.finished(1.day.ago).delete_all
727
+ GoodJob.cleanup_preserved_jobs # Will keep 1 day of job records by default.
728
+ GoodJob.cleanup_preserved_jobs(older_than: 7.days) # It also takes custom arguments.
726
729
  ```
727
730
 
728
731
  - For example, using the `good_job` command-line utility:
@@ -1,3 +1,4 @@
1
+ # frozen_string_literal: true
1
2
  class CreateGoodJobs < ActiveRecord::Migration[5.2]
2
3
  def change
3
4
  enable_extension 'pgcrypto'
@@ -13,9 +13,17 @@ class CreateGoodJobs < ActiveRecord::Migration[5.2]
13
13
  t.text :error
14
14
 
15
15
  t.timestamps
16
+
17
+ t.uuid :active_job_id
18
+ t.text :concurrency_key
19
+ t.text :cron_key
20
+ t.uuid :retried_good_job_id
16
21
  end
17
22
 
18
23
  add_index :good_jobs, :scheduled_at, where: "(finished_at IS NULL)", name: "index_good_jobs_on_scheduled_at"
19
24
  add_index :good_jobs, [:queue_name, :scheduled_at], where: "(finished_at IS NULL)", name: :index_good_jobs_on_queue_name_and_scheduled_at
25
+ add_index :good_jobs, [:active_job_id, :created_at], name: :index_good_jobs_on_active_job_id_and_created_at
26
+ add_index :good_jobs, :concurrency_key, where: "(finished_at IS NULL)", name: :index_good_jobs_on_concurrency_key_when_unfinished
27
+ add_index :good_jobs, [:cron_key, :created_at], name: :index_good_jobs_on_cron_key_and_created_at
20
28
  end
21
29
  end
@@ -4,7 +4,11 @@ module GoodJob
4
4
  module Concurrency
5
5
  extend ActiveSupport::Concern
6
6
 
7
- ConcurrencyExceededError = Class.new(StandardError)
7
+ class ConcurrencyExceededError < StandardError
8
+ def backtrace
9
+ [] # suppress backtrace
10
+ end
11
+ end
8
12
 
9
13
  included do
10
14
  class_attribute :good_job_concurrency_config, instance_accessor: false, default: {}
@@ -24,7 +28,7 @@ module GoodJob
24
28
 
25
29
  GoodJob::Job.new.with_advisory_lock(key: key, function: "pg_advisory_lock") do
26
30
  # TODO: Why is `unscoped` necessary? Nested scope is bleeding into subsequent query?
27
- enqueue_concurrency = GoodJob::Job.unscoped.where(concurrency_key: key).unfinished.count
31
+ enqueue_concurrency = GoodJob::Job.unscoped.where(concurrency_key: key).unfinished.advisory_unlocked.count
28
32
  # The job has not yet been enqueued, so check if adding it will go over the limit
29
33
  block.call unless enqueue_concurrency + 1 > limit
30
34
  end
@@ -4,16 +4,13 @@ module GoodJob
4
4
  # ActiveJob Adapter.
5
5
  #
6
6
  class Adapter
7
- # Valid execution modes.
8
- EXECUTION_MODES = [:async, :async_server, :external, :inline].freeze
9
-
10
7
  # @param execution_mode [Symbol, nil] specifies how and where jobs should be executed. You can also set this with the environment variable +GOOD_JOB_EXECUTION_MODE+.
11
8
  #
12
9
  # - +:inline+ executes jobs immediately in whatever process queued them (usually the web server process). This should only be used in test and development environments.
13
10
  # - +:external+ causes the adapter to enqueue jobs, but not execute them. When using this option (the default for production environments), you'll need to use the command-line tool to actually execute your jobs.
14
- # - +:async_server+ executes jobs in separate threads within the Rails webserver process (`bundle exec rails server`). It can be more economical for small workloads because you don't need a separate machine or environment for running your jobs, but if your web server is under heavy load or your jobs require a lot of resources, you should choose +:external+ instead.
15
- # When not in the Rails webserver, jobs will execute in +:external+ mode to ensure jobs are not executed within `rails console`, `rails db:migrate`, `rails assets:prepare`, etc.
16
- # - +:async+ executes jobs in any Rails process.
11
+ # - +:async+ (or +:async_server+) executes jobs in separate threads within the Rails web server process (`bundle exec rails server`). It can be more economical for small workloads because you don't need a separate machine or environment for running your jobs, but if your web server is under heavy load or your jobs require a lot of resources, you should choose +:external+ instead.
12
+ # When not in the Rails web server, jobs will execute in +:external+ mode to ensure jobs are not executed within `rails console`, `rails db:migrate`, `rails assets:prepare`, etc.
13
+ # - +:async_all+ executes jobs in any Rails process.
17
14
  #
18
15
  # The default value depends on the Rails environment:
19
16
  #
@@ -24,23 +21,6 @@ module GoodJob
24
21
  # @param queues [String, nil] determines which queues to execute jobs from when +execution_mode+ is set to +:async+. See {file:README.md#optimize-queues-threads-and-processes} for more details on the format of this string. You can also set this with the environment variable +GOOD_JOB_QUEUES+. Defaults to +"*"+.
25
22
  # @param poll_interval [Integer, nil] sets the number of seconds between polls for jobs when +execution_mode+ is set to +:async+. You can also set this with the environment variable +GOOD_JOB_POLL_INTERVAL+. Defaults to +1+.
26
23
  def initialize(execution_mode: nil, queues: nil, max_threads: nil, poll_interval: nil)
27
- if caller[0..4].find { |c| c.include?("/config/application.rb") || c.include?("/config/environments/") }
28
- ActiveSupport::Deprecation.warn(<<~DEPRECATION)
29
- GoodJob no longer recommends creating a GoodJob::Adapter instance:
30
-
31
- config.active_job.queue_adapter = GoodJob::Adapter.new...
32
-
33
- Instead, configure GoodJob through configuration:
34
-
35
- config.active_job.queue_adapter = :good_job
36
- config.good_job.execution_mode = :#{execution_mode}
37
- config.good_job.max_threads = #{max_threads}
38
- config.good_job.poll_interval = #{poll_interval}
39
- # etc...
40
-
41
- DEPRECATION
42
- end
43
-
44
24
  @configuration = GoodJob::Configuration.new(
45
25
  {
46
26
  execution_mode: execution_mode,
@@ -105,18 +85,8 @@ module GoodJob
105
85
  # * +-1+, the scheduler will wait until the shutdown is complete.
106
86
  # * +0+, the scheduler will immediately shutdown and stop any threads.
107
87
  # * A positive number will wait that many seconds before stopping any remaining active threads.
108
- # @param wait [Boolean, nil] Deprecated. Use +timeout:+ instead.
109
88
  # @return [void]
110
- def shutdown(timeout: :default, wait: nil)
111
- timeout = if wait.nil?
112
- timeout
113
- else
114
- ActiveSupport::Deprecation.warn(
115
- "Using `GoodJob::Adapter.shutdown` with `wait:` kwarg is deprecated; use `timeout:` kwarg instead e.g. GoodJob::Adapter.shutdown(timeout: #{wait ? '-1' : 'nil'})"
116
- )
117
- wait ? -1 : nil
118
- end
119
-
89
+ def shutdown(timeout: :default)
120
90
  timeout = if timeout == :default
121
91
  @configuration.shutdown_timeout
122
92
  else
@@ -130,15 +100,15 @@ module GoodJob
130
100
  # Whether in +:async+ execution mode.
131
101
  # @return [Boolean]
132
102
  def execute_async?
133
- @configuration.execution_mode.in?([:async, :async_all]) ||
134
- @configuration.execution_mode == :async_server && in_server_process?
103
+ @configuration.execution_mode == :async_all ||
104
+ @configuration.execution_mode.in?([:async, :async_server]) && in_server_process?
135
105
  end
136
106
 
137
107
  # Whether in +:external+ execution mode.
138
108
  # @return [Boolean]
139
109
  def execute_externally?
140
110
  @configuration.execution_mode == :external ||
141
- @configuration.execution_mode == :async_server && !in_server_process?
111
+ @configuration.execution_mode.in?([:async, :async_server]) && !in_server_process?
142
112
  end
143
113
 
144
114
  # Whether in +:inline+ execution mode.
data/lib/good_job/cli.rb CHANGED
@@ -134,16 +134,7 @@ module GoodJob
134
134
 
135
135
  configuration = GoodJob::Configuration.new(options)
136
136
 
137
- timestamp = Time.current - configuration.cleanup_preserved_jobs_before_seconds_ago
138
-
139
- ActiveSupport::Notifications.instrument(
140
- "cleanup_preserved_jobs.good_job",
141
- { before_seconds_ago: configuration.cleanup_preserved_jobs_before_seconds_ago, timestamp: timestamp }
142
- ) do |payload|
143
- deleted_records_count = GoodJob::Job.finished(timestamp).delete_all
144
-
145
- payload[:deleted_records_count] = deleted_records_count
146
- end
137
+ GoodJob.cleanup_preserved_jobs(older_than: configuration.cleanup_preserved_jobs_before_seconds_ago)
147
138
  end
148
139
 
149
140
  no_commands do
@@ -12,9 +12,11 @@ module GoodJob
12
12
  DEFAULT_MAX_THREADS = 5
13
13
  # Default number of seconds between polls for jobs
14
14
  DEFAULT_POLL_INTERVAL = 10
15
+ # Default poll interval for async in development environment
16
+ DEFAULT_DEVELOPMENT_ASYNC_POLL_INTERVAL = -1
15
17
  # Default number of threads to use per {Scheduler}
16
18
  DEFAULT_MAX_CACHE = 10000
17
- # Default number of seconds to preserve jobs for {CLI#cleanup_preserved_jobs}
19
+ # Default number of seconds to preserve jobs for {CLI#cleanup_preserved_jobs} and {GoodJob.cleanup_preserved_jobs}
18
20
  DEFAULT_CLEANUP_PRESERVED_JOBS_BEFORE_SECONDS_AGO = 24 * 60 * 60
19
21
  # Default to always wait for jobs to finish for {Adapter#shutdown}
20
22
  DEFAULT_SHUTDOWN_TIMEOUT = -1
@@ -58,19 +60,10 @@ module GoodJob
58
60
  end
59
61
 
60
62
  if mode
61
- mode_sym = mode.to_sym
62
- if mode_sym == :async
63
- ActiveSupport::Deprecation.warn <<~DEPRECATION
64
- The next major version of GoodJob will redefine the meaning of 'async'
65
- execution mode to be equivalent to 'async_server' and only execute
66
- within the webserver process.
67
-
68
- To continue using the v1.0 semantics of 'async', use `async_all` instead.
69
-
70
- DEPRECATION
71
- end
72
- mode_sym
73
- elsif Rails.env.development? || Rails.env.test?
63
+ mode.to_sym
64
+ elsif Rails.env.development?
65
+ :async
66
+ elsif Rails.env.test?
74
67
  :inline
75
68
  else
76
69
  :external
@@ -109,12 +102,19 @@ module GoodJob
109
102
  # poll (using this interval) for new queued jobs to execute.
110
103
  # @return [Integer]
111
104
  def poll_interval
112
- (
105
+ interval = (
113
106
  options[:poll_interval] ||
114
107
  rails_config[:poll_interval] ||
115
- env['GOOD_JOB_POLL_INTERVAL'] ||
116
- DEFAULT_POLL_INTERVAL
117
- ).to_i
108
+ env['GOOD_JOB_POLL_INTERVAL']
109
+ )
110
+
111
+ if interval
112
+ interval.to_i
113
+ elsif Rails.env.development? && execution_mode.in?([:async, :async_all, :async_server])
114
+ DEFAULT_DEVELOPMENT_ASYNC_POLL_INTERVAL
115
+ else
116
+ DEFAULT_POLL_INTERVAL
117
+ end
118
118
  end
119
119
 
120
120
  # The maximum number of future-scheduled jobs to store in memory.
@@ -147,12 +147,13 @@ module GoodJob
147
147
  def enable_cron
148
148
  value = ActiveModel::Type::Boolean.new.cast(
149
149
  options[:enable_cron] ||
150
- rails_config[:enable_cron] ||
151
- env['GOOD_JOB_ENABLE_CRON'] ||
152
- false
150
+ rails_config[:enable_cron] ||
151
+ env['GOOD_JOB_ENABLE_CRON'] ||
152
+ false
153
153
  )
154
154
  value && cron.size.positive?
155
155
  end
156
+
156
157
  alias enable_cron? enable_cron
157
158
 
158
159
  def cron
data/lib/good_job/job.rb CHANGED
@@ -16,7 +16,7 @@ module GoodJob
16
16
  DEFAULT_PRIORITY = 0
17
17
 
18
18
  self.table_name = 'good_jobs'
19
- self.advisory_lockable_column = 'id'
19
+ self.advisory_lockable_column = 'active_job_id'
20
20
 
21
21
  attr_readonly :serialized_params
22
22
 
@@ -52,20 +52,6 @@ module GoodJob
52
52
  end
53
53
  end
54
54
 
55
- def self._migration_pending_warning
56
- ActiveSupport::Deprecation.warn(<<~DEPRECATION)
57
- GoodJob has pending database migrations. To create the migration files, run:
58
-
59
- rails generate good_job:update
60
-
61
- To apply the migration files, run:
62
-
63
- rails db:migrate
64
-
65
- DEPRECATION
66
- nil
67
- end
68
-
69
55
  # Get Jobs with given class name
70
56
  # @!method with_job_class
71
57
  # @!scope class
@@ -78,14 +64,7 @@ module GoodJob
78
64
  # @!method unfinished
79
65
  # @!scope class
80
66
  # @return [ActiveRecord::Relation]
81
- scope :unfinished, (lambda do
82
- if column_names.include?('finished_at')
83
- where(finished_at: nil)
84
- else
85
- ActiveSupport::Deprecation.warn('GoodJob expects a good_jobs.finished_at column to exist. Please see the GoodJob README.md for migration instructions.')
86
- nil
87
- end
88
- end)
67
+ scope :unfinished, -> { where(finished_at: nil) }
89
68
 
90
69
  # Get Jobs that are not scheduled for a later time than now (i.e. jobs that
91
70
  # are not scheduled or scheduled for earlier than the current time).
@@ -231,6 +210,7 @@ module GoodJob
231
210
  def self.enqueue(active_job, scheduled_at: nil, create_with_advisory_lock: false)
232
211
  ActiveSupport::Notifications.instrument("enqueue_job.good_job", { active_job: active_job, scheduled_at: scheduled_at, create_with_advisory_lock: create_with_advisory_lock }) do |instrument_payload|
233
212
  good_job_args = {
213
+ active_job_id: active_job.job_id,
234
214
  queue_name: active_job.queue_name.presence || DEFAULT_QUEUE_NAME,
235
215
  priority: active_job.priority || DEFAULT_PRIORITY,
236
216
  serialized_params: active_job.serialize,
@@ -238,26 +218,12 @@ module GoodJob
238
218
  create_with_advisory_lock: create_with_advisory_lock,
239
219
  }
240
220
 
241
- if column_names.include?('active_job_id')
242
- good_job_args[:active_job_id] = active_job.job_id
243
- else
244
- _migration_pending_warning
245
- end
246
-
247
- if column_names.include?('concurrency_key')
248
- good_job_args[:concurrency_key] = active_job.good_job_concurrency_key if active_job.respond_to?(:good_job_concurrency_key)
249
- else
250
- _migration_pending_warning
251
- end
221
+ good_job_args[:concurrency_key] = active_job.good_job_concurrency_key if active_job.respond_to?(:good_job_concurrency_key)
252
222
 
253
- if column_names.include?('cron_key')
254
- if CurrentExecution.cron_key
255
- good_job_args[:cron_key] = CurrentExecution.cron_key
256
- elsif CurrentExecution.active_job_id == active_job.job_id
257
- good_job_args[:cron_key] = CurrentExecution.good_job.cron_key
258
- end
259
- else
260
- _migration_pending_warning
223
+ if CurrentExecution.cron_key
224
+ good_job_args[:cron_key] = CurrentExecution.cron_key
225
+ elsif CurrentExecution.active_job_id == active_job.job_id
226
+ good_job_args[:cron_key] = CurrentExecution.good_job.cron_key
261
227
  end
262
228
 
263
229
  good_job = GoodJob::Job.new(**good_job_args)
@@ -267,11 +233,7 @@ module GoodJob
267
233
  good_job.save!
268
234
  active_job.provider_job_id = good_job.id
269
235
 
270
- if column_names.include?('retried_good_job_id')
271
- CurrentExecution.good_job.retried_good_job_id = good_job.id if CurrentExecution.good_job && CurrentExecution.good_job.active_job_id == active_job.job_id
272
- else
273
- _migration_pending_warning
274
- end
236
+ CurrentExecution.good_job.retried_good_job_id = good_job.id if CurrentExecution.good_job && CurrentExecution.good_job.active_job_id == active_job.job_id
275
237
 
276
238
  good_job
277
239
  end
@@ -311,24 +273,6 @@ module GoodJob
311
273
  self.class.unscoped.unfinished.owns_advisory_locked.exists?(id: id)
312
274
  end
313
275
 
314
- def active_job_id
315
- if self.class.column_names.include?('active_job_id')
316
- super
317
- else
318
- self.class._migration_pending_warning
319
- serialized_params['job_id']
320
- end
321
- end
322
-
323
- def cron_key
324
- if self.class.column_names.include?('cron_key')
325
- super
326
- else
327
- self.class._migration_pending_warning
328
- nil
329
- end
330
- end
331
-
332
276
  private
333
277
 
334
278
  # @return [ExecutionResult]
@@ -51,7 +51,7 @@ module GoodJob
51
51
  composed_cte = Arel::Nodes::As.new(cte_table, Arel::Nodes::SqlLiteral.new([cte_type, "(", cte_query.to_sql, ")"].join(' ')))
52
52
  query = cte_table.project(cte_table[:id])
53
53
  .with(composed_cte)
54
- .where(Arel.sql(sanitize_sql_for_conditions(["#{function}(('x' || substr(md5(:table_name || #{connection.quote_table_name(cte_table.name)}.#{connection.quote_column_name(column)}::text), 1, 16))::bit(64)::bigint)", { table_name: table_name }])))
54
+ .where(Arel.sql(sanitize_sql_for_conditions(["#{function}(('x' || substr(md5(:table_name || '-' || #{connection.quote_table_name(cte_table.name)}.#{connection.quote_column_name(column)}::text), 1, 16))::bit(64)::bigint)", { table_name: table_name }])))
55
55
 
56
56
  limit = original_query.arel.ast.limit
57
57
  query.limit = limit.value if limit.present?
@@ -75,8 +75,8 @@ module GoodJob
75
75
  join_sql = <<~SQL.squish
76
76
  LEFT JOIN pg_locks ON pg_locks.locktype = 'advisory'
77
77
  AND pg_locks.objsubid = 1
78
- AND pg_locks.classid = ('x' || substr(md5(:table_name || #{quoted_table_name}.#{connection.quote_column_name(column)}::text), 1, 16))::bit(32)::int
79
- AND pg_locks.objid = (('x' || substr(md5(:table_name || #{quoted_table_name}.#{connection.quote_column_name(column)}::text), 1, 16))::bit(64) << 32)::bit(32)::int
78
+ AND pg_locks.classid = ('x' || substr(md5(:table_name || '-' || #{quoted_table_name}.#{connection.quote_column_name(column)}::text), 1, 16))::bit(32)::int
79
+ AND pg_locks.objid = (('x' || substr(md5(:table_name || '-' || #{quoted_table_name}.#{connection.quote_column_name(column)}::text), 1, 16))::bit(64) << 32)::bit(32)::int
80
80
  SQL
81
81
 
82
82
  joins(sanitize_sql_for_conditions([join_sql, { table_name: table_name }]))
@@ -315,7 +315,7 @@ module GoodJob
315
315
  # Default Advisory Lock key
316
316
  # @return [String]
317
317
  def lockable_key
318
- [self.class.table_name, self[self.class._advisory_lockable_column]].join
318
+ "#{self.class.table_name}-#{self[self.class._advisory_lockable_column]}"
319
319
  end
320
320
 
321
321
  delegate :pg_or_jdbc_query, to: :class
@@ -168,7 +168,9 @@ module GoodJob # :nodoc:
168
168
  # @!visibility private
169
169
  # @return [void]
170
170
  def task_observer(time, output, thread_error)
171
- GoodJob.on_thread_error.call(thread_error) if thread_error && GoodJob.on_thread_error.respond_to?(:call)
171
+ error = thread_error || (output.is_a?(GoodJob::ExecutionResult) ? output.unhandled_error : nil)
172
+ GoodJob.on_thread_error.call(error) if error && GoodJob.on_thread_error.respond_to?(:call)
173
+
172
174
  instrument("finished_job_task", { result: output, error: thread_error, time: time })
173
175
  create_task if output
174
176
  end
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
  module GoodJob
3
3
  # GoodJob gem version.
4
- VERSION = '1.99.1'
4
+ VERSION = '2.0.3'
5
5
  end
data/lib/good_job.rb CHANGED
@@ -42,7 +42,8 @@ module GoodJob
42
42
  # By default, GoodJob deletes job records after the job is completed successfully.
43
43
  # If you want to preserve jobs for latter inspection, set this to +true+.
44
44
  # If you want to preserve only jobs that finished with error for latter inspection, set this to +:on_unhandled_error+.
45
- # If +true+, you will need to clean out jobs using the +good_job cleanup_preserved_jobs+ CLI command.
45
+ # If +true+, you will need to clean out jobs using the +good_job cleanup_preserved_jobs+ CLI command or
46
+ # by using +Goodjob.cleanup_preserved_jobs+.
46
47
  # @return [Boolean, nil]
47
48
  mattr_accessor :preserve_job_records, default: false
48
49
 
@@ -55,25 +56,6 @@ module GoodJob
55
56
  # @return [Boolean, nil]
56
57
  mattr_accessor :retry_on_unhandled_error, default: true
57
58
 
58
- # @deprecated Use {GoodJob#retry_on_unhandled_error} instead.
59
- # @return [Boolean, nil]
60
- def self.reperform_jobs_on_standard_error
61
- ActiveSupport::Deprecation.warn(
62
- "Calling 'GoodJob.reperform_jobs_on_standard_error' is deprecated. Please use 'retry_on_unhandled_error'"
63
- )
64
- retry_on_unhandled_error
65
- end
66
-
67
- # @deprecated Use {GoodJob#retry_on_unhandled_error=} instead.
68
- # @param value [Boolean]
69
- # @return [Boolean]
70
- def self.reperform_jobs_on_standard_error=(value)
71
- ActiveSupport::Deprecation.warn(
72
- "Setting 'GoodJob.reperform_jobs_on_standard_error=' is deprecated. Please use 'retry_on_unhandled_error='"
73
- )
74
- self.retry_on_unhandled_error = value
75
- end
76
-
77
59
  # @!attribute [rw] on_thread_error
78
60
  # @!scope class
79
61
  # This callable will be called when an exception reaches GoodJob (default: +nil+).
@@ -96,16 +78,7 @@ module GoodJob
96
78
  # * +1..+, the scheduler will wait that many seconds before stopping any remaining active tasks.
97
79
  # @param wait [Boolean] whether to wait for shutdown
98
80
  # @return [void]
99
- def self.shutdown(timeout: -1, wait: nil)
100
- timeout = if wait.nil?
101
- timeout
102
- else
103
- ActiveSupport::Deprecation.warn(
104
- "Using `GoodJob.shutdown` with `wait:` kwarg is deprecated; use `timeout:` kwarg instead e.g. GoodJob.shutdown(timeout: #{wait ? '-1' : 'nil'})"
105
- )
106
- wait ? -1 : nil
107
- end
108
-
81
+ def self.shutdown(timeout: -1)
109
82
  _shutdown_all(_executables, timeout: timeout)
110
83
  end
111
84
 
@@ -142,6 +115,26 @@ module GoodJob
142
115
  end
143
116
  end
144
117
 
118
+ # Deletes preserved job records.
119
+ # By default, GoodJob deletes job records when the job is performed and this
120
+ # method is not necessary. However, when `GoodJob.preserve_job_records = true`,
121
+ # the jobs will be preserved in the database. This is useful when wanting to
122
+ # analyze or inspect job performance.
123
+ # If you are preserving job records this way, use this method regularly to
124
+ # delete old records and preserve space in your database.
125
+ # @params older_than [nil,Numeric,ActiveSupport::Duration] Jobs olders than this will be deleted (default: +86400+).
126
+ # @return [Integer] Number of jobs that were deleted.
127
+ def self.cleanup_preserved_jobs(older_than: nil)
128
+ older_than ||= GoodJob::Configuration.new({}).cleanup_preserved_jobs_before_seconds_ago
129
+ timestamp = Time.current - older_than
130
+
131
+ ActiveSupport::Notifications.instrument("cleanup_preserved_jobs.good_job", { older_than: older_than, timestamp: timestamp }) do |payload|
132
+ deleted_records_count = GoodJob::Job.finished(timestamp).delete_all
133
+
134
+ payload[:deleted_records_count] = deleted_records_count
135
+ end
136
+ end
137
+
145
138
  def self._executables
146
139
  [].concat(
147
140
  CronManager.instances,
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: good_job
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.99.1
4
+ version: 2.0.3
5
5
  platform: ruby
6
6
  authors:
7
7
  - Ben Sheldon
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2021-08-27 00:00:00.000000000 Z
11
+ date: 2021-08-31 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: activejob
@@ -372,9 +372,6 @@ files:
372
372
  - lib/generators/good_job/install_generator.rb
373
373
  - lib/generators/good_job/templates/install/migrations/create_good_jobs.rb.erb
374
374
  - lib/generators/good_job/templates/update/migrations/01_create_good_jobs.rb
375
- - lib/generators/good_job/templates/update/migrations/02_add_active_job_id_concurrency_key_cron_key_to_good_jobs.rb
376
- - lib/generators/good_job/templates/update/migrations/03_add_active_job_id_index_and_concurrency_key_index_to_good_jobs.rb
377
- - lib/generators/good_job/templates/update/migrations/04_add_retried_good_job_id_to_good_jobs.rb
378
375
  - lib/generators/good_job/update_generator.rb
379
376
  - lib/good_job.rb
380
377
  - lib/good_job/active_job_extensions.rb
@@ -1,16 +0,0 @@
1
- # frozen_string_literal: true
2
- class AddActiveJobIdConcurrencyKeyCronKeyToGoodJobs < ActiveRecord::Migration[5.2]
3
- def change
4
- reversible do |dir|
5
- dir.up do
6
- # Ensure this incremental update migration is idempotent
7
- # with monolithic install migration.
8
- return if connection.column_exists?(:good_jobs, :active_job_id)
9
- end
10
- end
11
-
12
- add_column :good_jobs, :active_job_id, :uuid
13
- add_column :good_jobs, :concurrency_key, :text
14
- add_column :good_jobs, :cron_key, :text
15
- end
16
- end
@@ -1,32 +0,0 @@
1
- # frozen_string_literal: true
2
- class AddActiveJobIdIndexAndConcurrencyKeyIndexToGoodJobs < ActiveRecord::Migration[5.2]
3
- disable_ddl_transaction!
4
-
5
- UPDATE_BATCH_SIZE = 1_000
6
-
7
- def change
8
- reversible do |dir|
9
- dir.up do
10
- # Ensure this incremental update migration is idempotent
11
- # with monolithic install migration.
12
- return if connection.index_name_exists?(:good_jobs, :index_good_jobs_on_active_job_id_and_created_at)
13
- end
14
- end
15
-
16
- add_index :good_jobs, [:active_job_id, :created_at], algorithm: :concurrently, name: :index_good_jobs_on_active_job_id_and_created_at
17
- add_index :good_jobs, :concurrency_key, where: "(finished_at IS NULL)", algorithm: :concurrently, name: :index_good_jobs_on_concurrency_key_when_unfinished
18
- add_index :good_jobs, [:cron_key, :created_at], algorithm: :concurrently, name: :index_good_jobs_on_cron_key_and_created_at
19
-
20
- return unless defined? GoodJob::Job
21
-
22
- reversible do |dir|
23
- dir.up do
24
- # Ensure that all `good_jobs` records have an active_job_id value
25
- start_time = Time.current
26
- loop do
27
- break if GoodJob::Job.where(active_job_id: nil, finished_at: nil).where("created_at < ?", start_time).limit(UPDATE_BATCH_SIZE).update_all("active_job_id = (serialized_params->>'job_id')::uuid").zero?
28
- end
29
- end
30
- end
31
- end
32
- end
@@ -1,14 +0,0 @@
1
- # frozen_string_literal: true
2
- class AddRetriedGoodJobIdToGoodJobs < ActiveRecord::Migration[5.2]
3
- def change
4
- reversible do |dir|
5
- dir.up do
6
- # Ensure this incremental update migration is idempotent
7
- # with monolithic install migration.
8
- return if connection.column_exists?(:good_jobs, :retried_good_job_id)
9
- end
10
- end
11
-
12
- add_column :good_jobs, :retried_good_job_id, :uuid
13
- end
14
- end