cloudtasker 0.10.rc5 → 0.10.1

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: db7ce2488df7c451c0ed08a2674c32e6e5964a926ad8bf158fee3e76ec71ee82
4
- data.tar.gz: 354177692074ddf3aa74fb1dba70dd56933211b460f035884fbf34bc87dc89fc
3
+ metadata.gz: c0f76715e1b778b035aba4e5bd6175b7d632bb114dcaa471f79e6be29013933e
4
+ data.tar.gz: 8d059110a3e90b04375410b7030d4773ea34d2cc8da0ac89cad20ba146939080
5
5
  SHA512:
6
- metadata.gz: 627dc29f6318bc4f0f676deea5ea932f6ec6414cf7f98c3f986132de59c44d30db5d645c518a04bf98ca99925719efce7a430a48a85d83be1cd90551a5f8206a
7
- data.tar.gz: e9806327d16e5ac6700577732cc9347f403eb21c6debd52cbacb36157af110a3ccab917e01fab9d6cd87d98ae0cfa1558fe7ebf0d1a938d4a0b40a13a492e87d
6
+ metadata.gz: b6e33bd5cd5587baf20336445833938fcc01e12fa6f43f0236bcb90cf4f8fb8463545f89bca85f3511d95db8b0c05d4bcfe83f94b9a7da3ec2dca51f5f64e660
7
+ data.tar.gz: 7ac3e128edfd635b543160391b38bcb7acc840aa306cc6713e7f9aa5cd76ea01da3616d293f7f9222496449419392ac2d644bcd643315cd75661a93b1577ac0f
@@ -2,9 +2,9 @@ name: Test
2
2
 
3
3
  on:
4
4
  push:
5
- branches: [ master ]
5
+ branches: [ master, 0.9-stable ]
6
6
  pull_request:
7
- branches: [ master ]
7
+ branches: [ master, 0.9-stable ]
8
8
 
9
9
  jobs:
10
10
  build:
@@ -21,6 +21,10 @@ jobs:
21
21
  - 'google-cloud-tasks-1.3'
22
22
  - 'rails-5.2'
23
23
  - 'rails-6.0'
24
+ - 'semantic_logger-3.4'
25
+ - 'semantic_logger-4.6'
26
+ - 'semantic_logger-4.7.0'
27
+ - 'semantic_logger-4.7.2'
24
28
  steps:
25
29
  - name: Setup System
26
30
  run: sudo apt-get install libsqlite3-dev
@@ -38,4 +42,4 @@ jobs:
38
42
  bundle install --jobs 4 --retry 3
39
43
  bundle exec rubocop
40
44
  bundle exec appraisal ${APPRAISAL_CONTEXT} bundle
41
- bundle exec appraisal ${APPRAISAL_CONTEXT} rspec
45
+ bundle exec appraisal ${APPRAISAL_CONTEXT} rspec
@@ -37,4 +37,10 @@ Metrics/BlockLength:
37
37
  Style/Documentation:
38
38
  Exclude:
39
39
  - 'examples/**/*'
40
- - 'spec/**/*'
40
+ - 'spec/**/*'
41
+
42
+ Metrics/ParameterLists:
43
+ CountKeywordArgs: false
44
+
45
+ RSpec/MessageSpies:
46
+ Enabled: false
data/Appraisals CHANGED
@@ -23,3 +23,19 @@ end
23
23
  appraise 'rails-6.0' do
24
24
  gem 'rails', '6.0'
25
25
  end
26
+
27
+ appraise 'semantic_logger-3.4' do
28
+ gem 'semantic_logger', '3.4.1'
29
+ end
30
+
31
+ appraise 'semantic_logger-4.6' do
32
+ gem 'semantic_logger', '4.6.1'
33
+ end
34
+
35
+ appraise 'semantic_logger-4.7.0' do
36
+ gem 'semantic_logger', '4.7.0'
37
+ end
38
+
39
+ appraise 'semantic_logger-4.7.2' do
40
+ gem 'semantic_logger', '4.7.2'
41
+ end
@@ -1,5 +1,37 @@
1
1
  # Changelog
2
2
 
3
+ ## [v0.10.1](https://github.com/keypup-io/cloudtasker/tree/v0.10.1) (2020-10-05)
4
+
5
+ [Full Changelog](https://github.com/keypup-io/cloudtasker/compare/v0.10.0...v0.10.1)
6
+
7
+ **Fixed bugs:**
8
+ - Logging: fix log processing with `semantic_logger` `v4.7.2`. Accept any args on block passed to the logger.
9
+
10
+ ## [v0.10.0](https://github.com/keypup-io/cloudtasker/tree/v0.10.0) (2020-09-02)
11
+
12
+ [Full Changelog](https://github.com/keypup-io/cloudtasker/compare/v0.9.3...v0.10.0)
13
+
14
+ **Improvements:**
15
+ - Logging: Add worker name in log messages
16
+ - Logging: Add job duration in log messages
17
+ - Logging: Add Cloud Cloud Task ID in log messages
18
+ - Unique Job: Support TTL for lock keys. This feature prevents queues from being dead-locked when a critical crash occurs while processing a unique job.
19
+ - Worker: support payload storage in Redis instead of sending the payload to Google Cloud Tasks. This is useful when job arguments are expected to exceed 100kb, which is the limit set by Google Cloud Tasks
20
+
21
+ **Fixed bugs:**
22
+ - Local processing error: improve error handling and retries around network interruptions
23
+ - Redis client: prevent deadlocks in high concurrency scenario by slowing down poll time and enforcing lock expiration
24
+ - Redis client: use connecion pool with Redis to prevent race conditions
25
+ - Google API: improve error handling on job creation
26
+ - Google API: use the `X-CloudTasks-TaskRetryCount` instead of `X-CloudTasks-TaskExecutionCount` to detect how many retries Google Cloud Tasks has performed. Using `X-CloudTasks-TaskRetryCount` is theoretically less accurate than using `X-CloudTasks-TaskExecutionCount` because it includes the number of "app unreachable" retries but `X-CloudTasks-TaskExecutionCount` is currently bugged and remains at zero all the time. See [this issue](https://github.com/keypup-io/cloudtasker/issues/6)
27
+
28
+ ## [v0.9.3](https://github.com/keypup-io/cloudtasker/tree/v0.9.3) (2020-06-25)
29
+
30
+ [Full Changelog](https://github.com/keypup-io/cloudtasker/compare/v0.9.2...v0.9.3)
31
+
32
+ **Fixed bugs:**
33
+ - Google Cloud Tasks: lock version to `~> 1.0` (Google recently released a v2 which changes its bindings completely). An [issue](https://github.com/keypup-io/cloudtasker/issues/11) has been raised to upgrade Cloudtasker to `google-cloud-tasks` `v2`.
34
+
3
35
  ## [v0.9.2](https://github.com/keypup-io/cloudtasker/tree/v0.9.2) (2020-03-04)
4
36
 
5
37
  [Full Changelog](https://github.com/keypup-io/cloudtasker/compare/v0.9.1...v0.9.2)
@@ -71,7 +103,3 @@ For Sinatra applications please update your Cloudtasker controller according to
71
103
  ## [v0.1.0](https://github.com/keypup-io/cloudtasker/tree/v0.1.0) (2019-11-17)
72
104
 
73
105
  [Full Changelog](https://github.com/keypup-io/cloudtasker/compare/c137feb1ceaaaa4e2fecac0d1f0b4c73151ae002...v0.1.0)
74
-
75
-
76
-
77
- \* *This Changelog was automatically generated by [github_changelog_generator](https://github.com/github-changelog-generator/github-changelog-generator)*
data/README.md CHANGED
@@ -6,11 +6,11 @@ Background jobs for Ruby using Google Cloud Tasks.
6
6
 
7
7
  Cloudtasker provides an easy to manage interface to Google Cloud Tasks for background job processing. Workers can be defined programmatically using the Cloudtasker DSL and enqueued for processing using a simple to use API.
8
8
 
9
- Cloudtasker is particularly suited for serverless applications only responding to HTTP requests and where running a dedicated job processing is not an option (e.g. deploy via [Cloud Run](https://cloud.google.com/run)). All jobs enqueued in Cloud Tasks via Cloudtasker eventually get processed by your application via HTTP requests.
9
+ Cloudtasker is particularly suited for serverless applications only responding to HTTP requests and where running a dedicated job processing server is not an option (e.g. deploy via [Cloud Run](https://cloud.google.com/run)). All jobs enqueued in Cloud Tasks via Cloudtasker eventually get processed by your application via HTTP requests.
10
10
 
11
11
  Cloudtasker also provides optional modules for running [cron jobs](docs/CRON_JOBS.md), [batch jobs](docs/BATCH_JOBS.md) and [unique jobs](docs/UNIQUE_JOBS.md).
12
12
 
13
- A local processing server is also available in development. This local server processes jobs in lieu of Cloud Tasks and allows you to work offline.
13
+ A local processing server is also available for development. This local server processes jobs in lieu of Cloud Tasks and allows you to work offline.
14
14
 
15
15
  ## Summary
16
16
 
@@ -34,7 +34,11 @@ A local processing server is also available in development. This local server pr
34
34
  1. [HTTP Error codes](#http-error-codes)
35
35
  2. [Error callbacks](#error-callbacks)
36
36
  3. [Max retries](#max-retries)
37
- 10. [Best practices building workers](#best-practices-building-workers)
37
+ 10. [Testing](#testing)
38
+ 1. [Test helper setup](#test-helper-setup)
39
+ 2. [In-memory queues](#in-memory-queues)
40
+ 3. [Unit tests](#unit-tests)
41
+ 11. [Best practices building workers](#best-practices-building-workers)
38
42
 
39
43
  ## Installation
40
44
 
@@ -48,7 +52,7 @@ And then execute:
48
52
 
49
53
  $ bundle
50
54
 
51
- Or install it yourself as:
55
+ Or install it yourself with:
52
56
 
53
57
  $ gem install cloudtasker
54
58
 
@@ -71,7 +75,7 @@ Cloudtasker.configure do |config|
71
75
  # Adapt the server port to be the one used by your Rails web process
72
76
  #
73
77
  config.processor_host = 'http://localhost:3000'
74
-
78
+
75
79
  #
76
80
  # If you do not have any Rails secret_key_base defined, uncomment the following
77
81
  # This secret is used to authenticate jobs sent to the processing endpoint
@@ -154,14 +158,14 @@ The gem can be configured through an initializer. See below all the available co
154
158
  Cloudtasker.configure do |config|
155
159
  #
156
160
  # If you do not have any Rails secret_key_base defined, uncomment the following.
157
- # This secret is used to authenticate jobs sent to the processing endpoint
161
+ # This secret is used to authenticate jobs sent to the processing endpoint
158
162
  # of your application.
159
163
  #
160
164
  # Default with Rails: Rails.application.credentials.secret_key_base
161
165
  #
162
166
  # config.secret = 'some-long-token'
163
167
 
164
- #
168
+ #
165
169
  # Specify the details of your Google Cloud Task location.
166
170
  #
167
171
  # This not required in development using the Cloudtasker local server.
@@ -185,21 +189,21 @@ Cloudtasker.configure do |config|
185
189
  #
186
190
  # Specific queues can be created in Cloud Tasks using the gcloud SDK or
187
191
  # via the `rake cloudtasker:setup_queue name=<queue_name>` task.
188
- #
192
+ #
189
193
  config.gcp_queue_prefix = 'my-app'
190
194
 
191
- #
195
+ #
192
196
  # Specify the publicly accessible host for your application
193
197
  #
194
198
  # > E.g. in development, using the cloudtasker local server
195
199
  # config.processor_host = 'http://localhost:3000'
196
- #
200
+ #
197
201
  # > E.g. in development, using `config.mode = :production` and ngrok
198
202
  # config.processor_host = 'https://111111.ngrok.io'
199
203
  #
200
204
  config.processor_host = 'https://app.mydomain.com'
201
205
 
202
- #
206
+ #
203
207
  # Specify the mode of operation:
204
208
  # - :development => jobs will be pushed to Redis and picked up by the Cloudtasker local server
205
209
  # - :production => jobs will be pushed to Google Cloud Tasks. Requires a publicly accessible domain.
@@ -208,20 +212,20 @@ Cloudtasker.configure do |config|
208
212
  #
209
213
  # config.mode = Rails.env.production? || Rails.env.my_other_env? ? :production : :development
210
214
 
211
- #
215
+ #
212
216
  # Specify the logger to use
213
- #
217
+ #
214
218
  # Default with Rails: Rails.logger
215
219
  # Default without Rails: Logger.new(STDOUT)
216
- #
220
+ #
217
221
  # config.logger = MyLogger.new(STDOUT)
218
222
 
219
- #
223
+ #
220
224
  # Specify how many retries are allowed on jobs. This number of retries excludes any
221
- # connectivity error that would be due to the application being down or unreachable.
222
- #
225
+ # connectivity error due to the application being down or unreachable.
226
+ #
223
227
  # Default: 25
224
- #
228
+ #
225
229
  # config.max_retries = 10
226
230
 
227
231
  #
@@ -246,7 +250,7 @@ Cloudtasker.configure do |config|
246
250
  # You can set this configuration parameter to a KB value if you want to store jobs
247
251
  # args in redis only if the JSONified arguments payload exceeds that threshold.
248
252
  #
249
- # Supported since: v0.10.rc1
253
+ # Supported since: v0.10.0
250
254
  #
251
255
  # Default: false
252
256
  #
@@ -258,7 +262,7 @@ Cloudtasker.configure do |config|
258
262
  end
259
263
  ```
260
264
 
261
- If the default queue `<gcp_queue_prefix>-default` does not exist in Cloud Tasks you should [create it using the gcloud sdk](https://cloud.google.com/tasks/docs/creating-queues).
265
+ If the default queue `<gcp_queue_prefix>-default` does not exist in Cloud Tasks you should [create it using the gcloud sdk](https://cloud.google.com/tasks/docs/creating-queues).
262
266
 
263
267
  Alternatively with Rails you can simply run the following rake task if you have queue admin permissions (`cloudtasks.queues.get` and `cloudtasks.queues.create`).
264
268
  ```bash
@@ -289,7 +293,7 @@ MyWorker.schedule(args: [arg1, arg2], time_at: Time.parse('2025-01-01 00:50:00Z'
289
293
  MyWorker.schedule(args: [arg1, arg2], time_in: 5 * 60, queue: 'critical')
290
294
  ```
291
295
 
292
- Cloudtasker also provides a helper for re-enqueuing jobs. Re-enqueued jobs keep the same worker id. Some middlewares may rely on this to track the fact that that a job didn't actually complete (e.g. Cloustasker batch). This is optional and you can always fallback to using exception management (raise an error) to retry/re-enqueue jobs.
296
+ Cloudtasker also provides a helper for re-enqueuing jobs. Re-enqueued jobs keep the same job id. Some middlewares may rely on this to track the fact that that a job didn't actually complete (e.g. Cloustasker batch). This is optional and you can always fallback to using exception management (raise an error) to retry/re-enqueue jobs.
293
297
 
294
298
  E.g.
295
299
  ```ruby
@@ -360,7 +364,7 @@ CriticalWorker.schedule(args: [1], queue: :important)
360
364
  Cloudtasker comes with three optional features:
361
365
  - Cron Jobs [[docs](docs/CRON_JOBS.md)]: Run jobs at fixed intervals.
362
366
  - Batch Jobs [[docs](docs/BATCH_JOBS.md)]: Run jobs in jobs and track completion of the overall batch.
363
- - Unique Jobs [[docs](docs/UNIQUE_JOBS.md)]: Ensure uniqueness of jobs based on job arguments.
367
+ - Unique Jobs [[docs](docs/UNIQUE_JOBS.md)]: Ensure uniqueness of jobs based on job arguments.
364
368
 
365
369
  ## Working locally
366
370
 
@@ -377,9 +381,9 @@ You can configure your application to use the Cloudtasker local server using the
377
381
 
378
382
  Cloudtasker.configure do |config|
379
383
  # ... other options
380
-
384
+
381
385
  # Push jobs to redis and let the Cloudtasker local server collect them
382
- # This is the default mode unless CLOUDTASKER_ENV or RAILS_ENV or RACK_ENV is set
386
+ # This is the default mode unless CLOUDTASKER_ENV or RAILS_ENV or RACK_ENV is set
383
387
  # to a non-development environment
384
388
  config.mode = :development
385
389
  end
@@ -423,7 +427,7 @@ Cloudtasker.configure do |config|
423
427
 
424
428
  # Use your ngrok domain as the processor host
425
429
  config.processor_host = 'https://your-tunnel-id.ngrok.io'
426
-
430
+
427
431
  # Force Cloudtasker to use Google Cloud Tasks in development
428
432
  config.mode = :production
429
433
  end
@@ -467,14 +471,14 @@ end
467
471
 
468
472
  Will generate the following log with context `{:worker=> ..., :job_id=> ..., :job_meta=> ...}`
469
473
  ```log
470
- [Cloudtasker][d76040a1-367e-4e3b-854e-e05a74d5f773] Job run with foo. This is working!: {:worker=>"DummyWorker", :job_id=>"d76040a1-367e-4e3b-854e-e05a74d5f773", :job_meta=>{}}
474
+ [Cloudtasker][d76040a1-367e-4e3b-854e-e05a74d5f773] Job run with foo. This is working!: {:worker=>"DummyWorker", :job_id=>"d76040a1-367e-4e3b-854e-e05a74d5f773", :job_meta=>{}, :task_id => "4e755d3f-6de0-426c-b4ac-51edd445c045"}
471
475
  ```
472
476
 
473
477
  The way contextual information is displayed depends on the logger itself. For example with [semantic_logger](http://rocketjob.github.io/semantic_logger) contextual information might not appear in the log message but show up as payload data on the log entry itself (e.g. using the fluentd adapter).
474
478
 
475
479
  Contextual information can be customised globally and locally using a log context_processor. By default the `Cloudtasker::WorkerLogger` is configured the following way:
476
480
  ```ruby
477
- Cloudtasker::WorkerLogger.log_context_processor = ->(worker) { worker.to_h.slice(:worker, :job_id, :job_meta) }
481
+ Cloudtasker::WorkerLogger.log_context_processor = ->(worker) { worker.to_h.slice(:worker, :job_id, :job_meta, :job_queue, :task_id) }
478
482
  ```
479
483
 
480
484
  You can decide to add a global identifier for your worker logs using the following:
@@ -482,7 +486,7 @@ You can decide to add a global identifier for your worker logs using the followi
482
486
  # config/initializers/cloudtasker.rb
483
487
 
484
488
  Cloudtasker::WorkerLogger.log_context_processor = lambda { |worker|
485
- worker.to_h.slice(:worker, :job_id, :job_meta).merge(app: 'my-app')
489
+ worker.to_h.slice(:worker, :job_id, :job_meta, :job_queue, :task_id).merge(app: 'my-app')
486
490
  }
487
491
  ```
488
492
 
@@ -503,9 +507,24 @@ end
503
507
 
504
508
  See the [Cloudtasker::Worker class](lib/cloudtasker/worker.rb) for more information on attributes available to be logged in your `log_context_processor` proc.
505
509
 
510
+ ### Searching logs: Job ID vs Task ID
511
+ **Note**: `task_id` field is available in logs starting with `0.10.0`
512
+
513
+ Job instances are assigned two different different IDs for tracking and logging purpose: `job_id` and `task_id`. These IDs are found in each log entry to facilitate search.
514
+
515
+ | Field | Definition |
516
+ |------|-------------|
517
+ | `job_id` | This ID is generated by Cloudtasker. It identifies the job along its entire lifecyle. It is persistent across retries and reschedules. |
518
+ | `task_id` | This ID is generated by Google Cloud Tasks. It identifies a job instance on the Google Cloud Task side. It is persistent across retries but NOT across reschedules. |
519
+
520
+ The Google Cloud Task UI (GCP console) lists all the tasks pending/retrying and their associated task id (also called "Task name"). From there you can:
521
+ 1. Use a task ID to lookup the logs of a specific job instance in Stackdriver Logging (or any other logging solution).
522
+ 2. From (1) you can retrieve the `job_id` attribute of the job.
523
+ 3. From (2) you can use the `job_id` to lookup the job logs along its entire lifecycle.
524
+
506
525
  ## Error Handling
507
526
 
508
- Jobs failing will automatically return an HTTP error to Cloud Task and trigger a retry at a later time. The number of Cloud Task retries Cloud Task will depend on the configuration of your queue in Cloud Tasks.
527
+ Jobs failures will return an HTTP error to Cloud Task and trigger a retry at a later time. The number of Cloud Task retries depends on the configuration of your queue in Cloud Tasks.
509
528
 
510
529
  ### HTTP Error codes
511
530
 
@@ -513,6 +532,7 @@ Jobs failing will automatically return the following HTTP error code to Cloud Ta
513
532
 
514
533
  | Code | Description |
515
534
  |------|-------------|
535
+ | 204 | The job was processed successfully |
516
536
  | 205 | The job is dead and has been removed from the queue |
517
537
  | 404 | The job has specified an incorrect worker class. |
518
538
  | 422 | An error happened during the execution of the worker (`perform` method) |
@@ -551,26 +571,24 @@ By default jobs are retried 25 times - using an exponential backoff - before bei
551
571
 
552
572
  Note that the number of retries set on your Cloud Task queue should be many times higher than the number of retries configured in Cloudtasker because Cloud Task also includes failures to connect to your application. Ideally set the number of retries to `unlimited` in Cloud Tasks.
553
573
 
554
- **Note**: The `X-CloudTasks-TaskExecutionCount` header sent by Google Cloud Tasks and providing the number of retries outside of `HTTP 503` (instance not reachable) is currently bugged and remains at `0` all the time. Starting with `0.10.rc3` Cloudtasker uses the `X-CloudTasks-TaskRetryCount` header to detect the number of retries. This header includes `HTTP 503` errors which means that if your application is down at some point, jobs will fail and these failures will be counted toward the maximum number of retries. A [bug report](https://issuetracker.google.com/issues/154532072) has been raised with GCP to address this issue. Once fixed we will revert to using `X-CloudTasks-TaskExecutionCount` to avoid counting `HTTP 503` as job failures.
574
+ **Note**: The `X-CloudTasks-TaskExecutionCount` header sent by Google Cloud Tasks and providing the number of retries outside of `HTTP 503` (instance not reachable) is currently bugged and remains at `0` all the time. Starting with `v0.10.0` Cloudtasker uses the `X-CloudTasks-TaskRetryCount` header to detect the number of retries. This header includes `HTTP 503` errors which means that if your application is down at some point, jobs will fail and these failures will be counted toward the maximum number of retries. A [bug report](https://issuetracker.google.com/issues/154532072) has been raised with GCP to address this issue. Once fixed we will revert to using `X-CloudTasks-TaskExecutionCount` to avoid counting `HTTP 503` as job failures.
555
575
 
556
576
  E.g. Set max number of retries globally via the cloudtasker initializer.
557
577
  ```ruby
558
578
  # config/initializers/cloudtasker.rb
559
579
 
560
580
  Cloudtasker.configure do |config|
561
- #
581
+ #
562
582
  # Specify how many retries are allowed on jobs. This number of retries excludes any
563
583
  # connectivity error that would be due to the application being down or unreachable.
564
- #
584
+ #
565
585
  # Default: 25
566
- #
586
+ #
567
587
  config.max_retries = 10
568
588
  end
569
589
  ```
570
590
 
571
591
  E.g. Set max number of retries to 3 on a given worker
572
-
573
- E.g.
574
592
  ```ruby
575
593
  # app/workers/some_error_worker.rb
576
594
 
@@ -580,12 +598,126 @@ class SomeErrorWorker
580
598
  # This will override the global setting
581
599
  cloudtasker_options max_retries: 3
582
600
 
583
- def perform()
601
+ def perform
602
+ raise(ArgumentError)
603
+ end
604
+ end
605
+ ```
606
+
607
+ E.g. Evaluate the number of max retries at runtime (target: v0.11.0)
608
+ ```ruby
609
+ # app/workers/some_error_worker.rb
610
+
611
+ class SomeErrorWorker
612
+ include Cloudtasker::Worker
613
+
614
+ # Return the number of max retries based on
615
+ # worker arguments.
616
+ #
617
+ # If this method returns nil then max_retries
618
+ # will delegate to the class `max_retries` setting or Cloudtasker
619
+ # `max_retries` configuration otion.
620
+ def max_retries(arg1, arg2)
621
+ arg1 == 'foo' ? 13 : nil
622
+ end
623
+
624
+ def perform(arg1, arg2)
584
625
  raise(ArgumentError)
585
626
  end
586
627
  end
587
628
  ```
588
629
 
630
+ ## Testing
631
+ Cloudtasker provides several options to test your workers.
632
+
633
+ ### Test helper setup
634
+ Require `cloudtasker/testing` in your `rails_helper.rb` (Rspec Rails) or `spec_helper.rb` (Rspec) or test unit helper file then enable one of the three modes:
635
+
636
+ ```ruby
637
+ require 'cloudtasker/testing'
638
+
639
+ # Mode 1 (default): Push jobs to Google Cloud Tasks (env != development) or Redis (env == development)
640
+ Cloudtasker::Testing.enable!
641
+
642
+ # Mode 2: Push jobs to an in-memory queue. Jobs will not be processed until you call
643
+ # Cloudtasker::Worker.drain_all (process all jobs) or MyWorker.drain (process jobs for specific worker)
644
+ Cloudtasker::Testing.fake!
645
+
646
+ # Mode 3: Push jobs to an in-memory queue. Jobs will be processed immediately.
647
+ Cloudtasker::Testing.inline!
648
+ ```
649
+
650
+ You can query the current testing mode with:
651
+ ```ruby
652
+ Cloudtasker::Testing.enabled?
653
+ Cloudtasker::Testing.fake?
654
+ Cloudtasker::Testing.inline?
655
+ ```
656
+
657
+ Each testing mode accepts a block argument to temporarily switch to it:
658
+ ```ruby
659
+ # Enable fake mode for all tests
660
+ Cloudtasker::Testing.fake!
661
+
662
+ # Enable inline! mode temporarily for a given test
663
+ Cloudtasker.inline! do
664
+ MyWorker.perform_async(1,2)
665
+ end
666
+ ```
667
+
668
+ Note that extension middlewares - e.g. unique job, batch job etc. - run in test mode. You can disable middlewares in your tests by adding the following to your test helper:
669
+ ```ruby
670
+ # Remove all middlewares
671
+ Cloudtasker.configure do |c|
672
+ c.client_middleware.clear
673
+ c.server_middleware.clear
674
+ end
675
+
676
+ # Remove all unique job middlewares
677
+ Cloudtasker.configure do |c|
678
+ c.client_middleware.remove(Cloudtasker::UniqueJob::Middleware::Client)
679
+ c.server_middleware.remove(Cloudtasker::UniqueJob::Middleware::Server)
680
+ end
681
+ ```
682
+
683
+ ### In-memory queues
684
+ The `fake!` or `inline!` modes use in-memory queues, which can be queried and controlled using the following methods:
685
+
686
+ ```ruby
687
+ # Perform all jobs in queue
688
+ Cloudtasker::Worker.drain_all
689
+
690
+ # Remove all jobs in queue
691
+ Cloudtasker::Worker.clear_all
692
+
693
+ # Perform all jobs in queue for a specific worker type
694
+ MyWorker.drain
695
+
696
+ # Return the list of jobs in queue for a specific worker type
697
+ MyWorker.jobs
698
+ ```
699
+
700
+ ### Unit tests
701
+ Below are examples of rspec tests. It is assumed that `Cloudtasker::Testing.fake!` has been set in the test helper.
702
+
703
+ **Example 1**: Testing that a job is scheduled
704
+ ```ruby
705
+ describe 'worker scheduling'
706
+ subject(:enqueue_job) { MyWorker.perform_async(1,2) }
707
+
708
+ it { expect { enqueue_job }.to change(MyWorker.jobs, :size).by(1) }
709
+ end
710
+ ```
711
+
712
+ **Example 2**: Testing job execution logic
713
+ ```ruby
714
+ describe 'worker calls api'
715
+ subject { Cloudtasker::Testing.inline! { MyApiWorker.perform_async(1,2) } }
716
+
717
+ before { expect(MyApi).to receive(:fetch).and_return([]) }
718
+ it { is_expected.to be_truthy }
719
+ end
720
+ ```
589
721
 
590
722
  ## Best practices building workers
591
723
 
@@ -661,7 +793,7 @@ Google Cloud Tasks enforces a limit of 100 KB for job payloads. Taking into acco
661
793
  Any excessive job payload (> 100 KB) will raise a `Cloudtasker::MaxTaskSizeExceededError`, both in production and development mode.
662
794
 
663
795
  #### Option 1: Use Cloudtasker optional support for payload storage in Redis
664
- **Supported since**: `0.10.rc1`
796
+ **Supported since**: `0.10.0`
665
797
 
666
798
  Cloudtasker provides optional support for storing argument payloads in Redis instead of sending them to Google Cloud Tasks.
667
799