cloudtasker 0.12.rc10 → 0.12.rc11
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +3 -2
- data/README.md +54 -4
- data/lib/cloudtasker/config.rb +26 -2
- data/lib/cloudtasker/version.rb +1 -1
- data/lib/cloudtasker/worker_handler.rb +2 -0
- metadata +5 -5
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 1c7677889e31eccaadb3cf019f168e2d7133f10682379d0c2ccf1987da450105
|
4
|
+
data.tar.gz: 66ea1098cfbfdc9e5261dba0d6d033e44c8d84ee7623c464fc3bccd3999e708e
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: b7ccd9a1e2950d57f51dad86c847373c29caf753b092711cc9be167a8574cdaaec7eeff99dfe08fc3c0e29c4af1a7a21b04e92f36fcc6769b9e3b8f1561c6231
|
7
|
+
data.tar.gz: 7fc85d30cab6c80e8e64afa3285a50dfafb47f66dc214d228c758b7e19481239cf5900fad3bbbd29c3220658529eeeb106f12daf9e10f52a10f4d1bb10864354
|
data/CHANGELOG.md
CHANGED
@@ -1,8 +1,8 @@
|
|
1
1
|
# Changelog
|
2
2
|
|
3
|
-
## Latest RC [v0.12.
|
3
|
+
## Latest RC [v0.12.rc11](https://github.com/keypup-io/cloudtasker/tree/v0.12.rc11) (2021-06-26)
|
4
4
|
|
5
|
-
[Full Changelog](https://github.com/keypup-io/cloudtasker/compare/v0.11.0...v0.12.
|
5
|
+
[Full Changelog](https://github.com/keypup-io/cloudtasker/compare/v0.11.0...v0.12.rc11)
|
6
6
|
|
7
7
|
**Improvements:**
|
8
8
|
- ActiveJob: do not double log errors (ActiveJob has its own error logging)
|
@@ -12,6 +12,7 @@
|
|
12
12
|
- Batch redis usage: cleanup batches as they get completed or become dead to avoid excessive redis usage with large batches.
|
13
13
|
- Batch expansion: Inject `parent_batch` in jobs. Can be used to expand the parent batch the job is in.
|
14
14
|
- Configuration: allow configuration of Cloud Tasks `dispatch deadline` at global and worker level
|
15
|
+
- Configuration: allow specifying global `on_error` and `on_dead` callbacks for error reporting
|
15
16
|
- Cron jobs: Use Redis Sets instead of key pattern matching for resource listing
|
16
17
|
- Error logging: Use worker logger so as to include context (job args etc.)
|
17
18
|
- Error logging: Do not log exception and stack trace separately, combine them instead.
|
data/README.md
CHANGED
@@ -35,9 +35,10 @@ A local processing server is also available for development. This local server p
|
|
35
35
|
2. [Logging context](#logging-context)
|
36
36
|
9. [Error Handling](#error-handling)
|
37
37
|
1. [HTTP Error codes](#http-error-codes)
|
38
|
-
2. [
|
39
|
-
3. [
|
40
|
-
4. [
|
38
|
+
2. [Worker callbacks](#worker-callbacks)
|
39
|
+
3. [Global callbacks](#global-callbacks)
|
40
|
+
4. [Max retries](#max-retries)
|
41
|
+
5. [Dispatch deadline](#dispatch-deadline)
|
41
42
|
10. [Testing](#testing)
|
42
43
|
1. [Test helper setup](#test-helper-setup)
|
43
44
|
2. [In-memory queues](#in-memory-queues)
|
@@ -369,6 +370,36 @@ Cloudtasker.configure do |config|
|
|
369
370
|
# Default: 600 seconds (10 minutes)
|
370
371
|
#
|
371
372
|
# config.dispatch_deadline = 600
|
373
|
+
|
374
|
+
#
|
375
|
+
# Specify a proc to be invoked every time a job fails due to a runtime
|
376
|
+
# error.
|
377
|
+
#
|
378
|
+
# This hook is not invoked for DeadWorkerError. See on_dead instead.
|
379
|
+
#
|
380
|
+
# This is useful when you need to apply general exception handling, such
|
381
|
+
# as reporting errors to a third-party service like Rollbar or Bugsnag.
|
382
|
+
#
|
383
|
+
# Note: the worker argument might be nil, such as when InvalidWorkerError is raised.
|
384
|
+
#
|
385
|
+
# Supported since: v0.12.rc11
|
386
|
+
#
|
387
|
+
# Default: no operation
|
388
|
+
#
|
389
|
+
# config.on_error = ->(error, worker) { Rollbar.error(error) }
|
390
|
+
|
391
|
+
#
|
392
|
+
# Specify a proc to be invoked every time a job dies due to too many
|
393
|
+
# retries.
|
394
|
+
#
|
395
|
+
# This is useful when you need to apply general exception handling, such
|
396
|
+
# logging specific messages/context when a job dies.
|
397
|
+
#
|
398
|
+
# Supported since: v0.12.rc11
|
399
|
+
#
|
400
|
+
# Default: no operation
|
401
|
+
#
|
402
|
+
# config.on_dead = ->(error, worker) { Rollbar.error(error) }
|
372
403
|
end
|
373
404
|
```
|
374
405
|
|
@@ -649,7 +680,7 @@ Jobs failing will automatically return the following HTTP error code to Cloud Ta
|
|
649
680
|
| 404 | The job has specified an incorrect worker class. |
|
650
681
|
| 422 | An error happened during the execution of the worker (`perform` method) |
|
651
682
|
|
652
|
-
###
|
683
|
+
### Worker callbacks
|
653
684
|
|
654
685
|
Workers can implement the `on_error(error)` and `on_dead(error)` callbacks to do things when a job fails during its execution:
|
655
686
|
|
@@ -677,6 +708,25 @@ class HandleErrorWorker
|
|
677
708
|
end
|
678
709
|
```
|
679
710
|
|
711
|
+
### Global callbacks
|
712
|
+
**Supported since**: `0.12.rc11`
|
713
|
+
|
714
|
+
If you need to apply general exception handling logic to your workers you can specify `on_error` and `on_dead` hooks in the Cloudtasker configuration.
|
715
|
+
|
716
|
+
This is useful if you need to report errors to third-party services such as Rollbar or Bugsnag.
|
717
|
+
|
718
|
+
```ruby
|
719
|
+
# config/initializers/cloudtasker.rb
|
720
|
+
|
721
|
+
Cloudtasker.configure do |config|
|
722
|
+
#
|
723
|
+
# Report runtime and dead worker errors to Rollbar
|
724
|
+
#
|
725
|
+
config.on_error = -> (error, _worker) { Rollbar.error(error) }
|
726
|
+
config.on_dead = -> (error, _worker) { Rollbar.error(error) }
|
727
|
+
end
|
728
|
+
```
|
729
|
+
|
680
730
|
### Max retries
|
681
731
|
|
682
732
|
By default jobs are retried 25 times - using an exponential backoff - before being declared dead. This number of retries can be customized locally on workers and/or globally via the Cloudtasker initializer.
|
data/lib/cloudtasker/config.rb
CHANGED
@@ -7,7 +7,8 @@ module Cloudtasker
|
|
7
7
|
class Config
|
8
8
|
attr_accessor :redis, :store_payloads_in_redis
|
9
9
|
attr_writer :secret, :gcp_location_id, :gcp_project_id,
|
10
|
-
:gcp_queue_prefix, :processor_path, :logger, :mode, :max_retries,
|
10
|
+
:gcp_queue_prefix, :processor_path, :logger, :mode, :max_retries,
|
11
|
+
:dispatch_deadline, :on_error, :on_dead
|
11
12
|
|
12
13
|
# Max Cloud Task size in bytes
|
13
14
|
MAX_TASK_SIZE = 100 * 1024 # 100 KB
|
@@ -51,6 +52,9 @@ module Cloudtasker
|
|
51
52
|
MIN_DISPATCH_DEADLINE = 15 # seconds
|
52
53
|
MAX_DISPATCH_DEADLINE = 30 * 60 # 30 minutes
|
53
54
|
|
55
|
+
# Default on_error Proc
|
56
|
+
DEFAULT_ON_ERROR = ->(error, worker) {}
|
57
|
+
|
54
58
|
# The number of times jobs will be attempted before declaring them dead.
|
55
59
|
#
|
56
60
|
# With the default retry configuration (maxDoublings = 16 and minBackoff = 0.100s)
|
@@ -229,11 +233,31 @@ module Cloudtasker
|
|
229
233
|
# @return [String] The cloudtasker secret
|
230
234
|
#
|
231
235
|
def secret
|
232
|
-
@secret
|
236
|
+
@secret ||= (
|
233
237
|
defined?(Rails) && Rails.application.credentials&.dig(:secret_key_base)
|
234
238
|
) || raise(StandardError, SECRET_MISSING_ERROR)
|
235
239
|
end
|
236
240
|
|
241
|
+
#
|
242
|
+
# Return a Proc invoked whenever a worker runtime error is raised.
|
243
|
+
# See Cloudtasker::WorkerHandler.with_worker_handling
|
244
|
+
#
|
245
|
+
# @return [Proc] A Proc handler
|
246
|
+
#
|
247
|
+
def on_error
|
248
|
+
@on_error || DEFAULT_ON_ERROR
|
249
|
+
end
|
250
|
+
|
251
|
+
#
|
252
|
+
# Return a Proc invoked whenever a worker DeadWorkerError is raised.
|
253
|
+
# See Cloudtasker::WorkerHandler.with_worker_handling
|
254
|
+
#
|
255
|
+
# @return [Proc] A Proc handler
|
256
|
+
#
|
257
|
+
def on_dead
|
258
|
+
@on_dead || DEFAULT_ON_ERROR
|
259
|
+
end
|
260
|
+
|
237
261
|
#
|
238
262
|
# Return the chain of client middlewares.
|
239
263
|
#
|
data/lib/cloudtasker/version.rb
CHANGED
@@ -102,9 +102,11 @@ module Cloudtasker
|
|
102
102
|
# Delete stored args payload if job is dead
|
103
103
|
redis.del(args_payload_key) if args_payload_key
|
104
104
|
log_execution_error(worker, e)
|
105
|
+
Cloudtasker.config.on_dead.call(e, worker)
|
105
106
|
raise(e)
|
106
107
|
rescue StandardError => e
|
107
108
|
log_execution_error(worker, e)
|
109
|
+
Cloudtasker.config.on_error.call(e, worker)
|
108
110
|
raise(e)
|
109
111
|
end
|
110
112
|
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: cloudtasker
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.12.
|
4
|
+
version: 0.12.rc11
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Arnaud Lachaume
|
8
|
-
autorequire:
|
8
|
+
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2021-
|
11
|
+
date: 2021-06-26 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activesupport
|
@@ -416,7 +416,7 @@ metadata:
|
|
416
416
|
homepage_uri: https://github.com/keypup-io/cloudtasker
|
417
417
|
source_code_uri: https://github.com/keypup-io/cloudtasker
|
418
418
|
changelog_uri: https://github.com/keypup-io/cloudtasker/master/tree/CHANGELOG.md
|
419
|
-
post_install_message:
|
419
|
+
post_install_message:
|
420
420
|
rdoc_options: []
|
421
421
|
require_paths:
|
422
422
|
- lib
|
@@ -432,7 +432,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
432
432
|
version: 1.3.1
|
433
433
|
requirements: []
|
434
434
|
rubygems_version: 3.0.0
|
435
|
-
signing_key:
|
435
|
+
signing_key:
|
436
436
|
specification_version: 4
|
437
437
|
summary: Background jobs for Ruby using Google Cloud Tasks (beta)
|
438
438
|
test_files: []
|