cloudtasker 0.11.rc1 → 0.11.rc2
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.rubocop.yml +4 -0
- data/CHANGELOG.md +8 -0
- data/README.md +92 -0
- data/cloudtasker.gemspec +1 -0
- data/lib/active_job/queue_adapters/cloudtasker_adapter.rb +82 -0
- data/lib/cloudtasker/cli.rb +1 -0
- data/lib/cloudtasker/engine.rb +6 -0
- data/lib/cloudtasker/version.rb +1 -1
- metadata +17 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: c5d91132bd3363bf3d7c08cd207b385f502eae3056be4734f73f60ef220f94c5
|
4
|
+
data.tar.gz: fe935947d1ece340202d32eb54927336840ca236510a5044c2b1996c2161080d
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: ed918a89e19dca6068f6c4d7b2aa0e55f838829f19741b1509edcbf3e03117e102454791efd808c5521da30e862d8cfc1d5969444cfb6ee250589e5b7dd281de
|
7
|
+
data.tar.gz: d4fbb9afebcf84efb8bbe3f11b126aefc93a14c62d9d09b1a2f27186defb9c30ee0d28c13688b6bcc38411a3cbe8a736010fbea1a75e7dfa6334870e0522f2ff
|
data/.rubocop.yml
CHANGED
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,13 @@
|
|
1
1
|
# Changelog
|
2
2
|
|
3
|
+
## Latest RC: [v0.11.rc2](https://github.com/keypup-io/cloudtasker/tree/v0.11.rc2) (2020-11-23)
|
4
|
+
|
5
|
+
[Full Changelog](https://github.com/keypup-io/cloudtasker/compare/v0.10.0...v0.11.rc2)
|
6
|
+
|
7
|
+
**Improvements:**
|
8
|
+
- Worker: drop job (return 205 response) when worker arguments are not available (e.g. arguments were stored in Redis and the latter was flushed)
|
9
|
+
- Rails: add ActiveJob adapter (thanks @vovimayhem)
|
10
|
+
|
3
11
|
## [v0.10.1](https://github.com/keypup-io/cloudtasker/tree/v0.10.1) (2020-10-05)
|
4
12
|
|
5
13
|
[Full Changelog](https://github.com/keypup-io/cloudtasker/compare/v0.10.0...v0.10.1)
|
data/README.md
CHANGED
@@ -16,6 +16,7 @@ A local processing server is also available for development. This local server p
|
|
16
16
|
|
17
17
|
1. [Installation](#installation)
|
18
18
|
2. [Get started with Rails](#get-started-with-rails)
|
19
|
+
2. [Get started with Rails & ActiveJob](#get-started-with-rails--activejob)
|
19
20
|
3. [Configuring Cloudtasker](#configuring-cloudtasker)
|
20
21
|
1. [Cloud Tasks authentication & permissions](#cloud-tasks-authentication--permissions)
|
21
22
|
2. [Cloudtasker initializer](#cloudtasker-initializer)
|
@@ -132,6 +133,95 @@ That's it! Your job was picked up by the Cloudtasker local server and sent for p
|
|
132
133
|
|
133
134
|
Now jump to the next section to configure your app to use Google Cloud Tasks as a backend.
|
134
135
|
|
136
|
+
## Get started with Rails & ActiveJob
|
137
|
+
**Note**: ActiveJob is supported since `0.11.rc2`
|
138
|
+
**Note**: Cloudtasker extensions (cron, batch and unique jobs) are not available when using cloudtasker via ActiveJob.
|
139
|
+
|
140
|
+
Cloudtasker is pre-integrated with ActiveJob. Follow the steps below to get started.
|
141
|
+
|
142
|
+
Install redis on your machine (this is required by the Cloudtasker local processing server)
|
143
|
+
```bash
|
144
|
+
# E.g. using brew
|
145
|
+
brew install redis
|
146
|
+
```
|
147
|
+
|
148
|
+
Add the following initializer
|
149
|
+
```ruby
|
150
|
+
# config/initializers/cloudtasker.rb
|
151
|
+
|
152
|
+
Cloudtasker.configure do |config|
|
153
|
+
#
|
154
|
+
# Adapt the server port to be the one used by your Rails web process
|
155
|
+
#
|
156
|
+
config.processor_host = 'http://localhost:3000'
|
157
|
+
|
158
|
+
#
|
159
|
+
# If you do not have any Rails secret_key_base defined, uncomment the following
|
160
|
+
# This secret is used to authenticate jobs sent to the processing endpoint
|
161
|
+
# of your application.
|
162
|
+
#
|
163
|
+
# config.secret = 'some-long-token'
|
164
|
+
end
|
165
|
+
```
|
166
|
+
|
167
|
+
Configure ActiveJob to use Cloudtasker. You can also configure ActiveJob per environment via the config/environments/:env.rb files
|
168
|
+
```ruby
|
169
|
+
# config/application.rb
|
170
|
+
|
171
|
+
require_relative 'boot'
|
172
|
+
require 'rails/all'
|
173
|
+
|
174
|
+
Bundler.require(*Rails.groups)
|
175
|
+
|
176
|
+
module Dummy
|
177
|
+
class Application < Rails::Application
|
178
|
+
# Initialize configuration defaults for originally generated Rails version.
|
179
|
+
config.load_defaults 6.0
|
180
|
+
|
181
|
+
# Settings in config/environments/* take precedence over those specified here.
|
182
|
+
# Application configuration can go into files in config/initializers
|
183
|
+
# -- all .rb files in that directory are automatically loaded after loading
|
184
|
+
# the framework and any gems in your application.
|
185
|
+
|
186
|
+
# Use cloudtasker as the ActiveJob backend:
|
187
|
+
config.active_job.queue_adapter = :cloudtasker
|
188
|
+
end
|
189
|
+
end
|
190
|
+
|
191
|
+
```
|
192
|
+
|
193
|
+
Define your first job:
|
194
|
+
```ruby
|
195
|
+
# app/jobs/example_job.rb
|
196
|
+
|
197
|
+
class ExampleJob < ApplicationJob
|
198
|
+
queue_as :default
|
199
|
+
|
200
|
+
def perform(some_arg)
|
201
|
+
logger.info("Job run with #{some_arg}. This is working!")
|
202
|
+
end
|
203
|
+
end
|
204
|
+
```
|
205
|
+
|
206
|
+
Launch Rails and the local Cloudtasker processing server (or add `cloudtasker` to your foreman config as a `worker` process)
|
207
|
+
```bash
|
208
|
+
# In one terminal
|
209
|
+
> rails s -p 3000
|
210
|
+
|
211
|
+
# In another terminal
|
212
|
+
> cloudtasker
|
213
|
+
```
|
214
|
+
|
215
|
+
Open a Rails console and enqueue some jobs
|
216
|
+
```ruby
|
217
|
+
# Process job as soon as possible
|
218
|
+
ExampleJob.perform_later('foo')
|
219
|
+
|
220
|
+
# Process job in 60 seconds
|
221
|
+
ExampleJob.set(wait: 60).perform_later('foo')
|
222
|
+
```
|
223
|
+
|
224
|
+
|
135
225
|
## Configuring Cloudtasker
|
136
226
|
|
137
227
|
### Cloud Tasks authentication & permissions
|
@@ -361,6 +451,8 @@ CriticalWorker.schedule(args: [1], queue: :important)
|
|
361
451
|
```
|
362
452
|
|
363
453
|
## Extensions
|
454
|
+
**Note**: Extensions are not available when using cloudtasker via ActiveJob.
|
455
|
+
|
364
456
|
Cloudtasker comes with three optional features:
|
365
457
|
- Cron Jobs [[docs](docs/CRON_JOBS.md)]: Run jobs at fixed intervals.
|
366
458
|
- Batch Jobs [[docs](docs/BATCH_JOBS.md)]: Run jobs in jobs and track completion of the overall batch.
|
data/cloudtasker.gemspec
CHANGED
@@ -41,6 +41,7 @@ Gem::Specification.new do |spec|
|
|
41
41
|
spec.add_development_dependency 'github_changelog_generator'
|
42
42
|
spec.add_development_dependency 'rake', '>= 12.3.3'
|
43
43
|
spec.add_development_dependency 'rspec', '~> 3.0'
|
44
|
+
spec.add_development_dependency 'rspec-json_expectations', '~> 2.2'
|
44
45
|
spec.add_development_dependency 'rubocop', '0.76.0'
|
45
46
|
spec.add_development_dependency 'rubocop-rspec', '1.37.0'
|
46
47
|
spec.add_development_dependency 'semantic_logger'
|
@@ -0,0 +1,82 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
# ActiveJob docs: http://guides.rubyonrails.org/active_job_basics.html
|
4
|
+
# Example adapters ref: https://github.com/rails/rails/tree/master/activejob/lib/active_job/queue_adapters
|
5
|
+
|
6
|
+
module ActiveJob
|
7
|
+
module QueueAdapters
|
8
|
+
# == Cloudtasker adapter for Active Job
|
9
|
+
#
|
10
|
+
# To use Cloudtasker set the queue_adapter config to +:cloudtasker+.
|
11
|
+
#
|
12
|
+
# Rails.application.config.active_job.queue_adapter = :cloudtasker
|
13
|
+
class CloudtaskerAdapter
|
14
|
+
SERIALIZATION_FILTERED_KEYS = [
|
15
|
+
'executions', # Given by the worker at processing
|
16
|
+
'provider_job_id', # Also given by the worker at processing
|
17
|
+
'priority' # Not used
|
18
|
+
].freeze
|
19
|
+
|
20
|
+
# Enqueues the given ActiveJob instance for execution
|
21
|
+
#
|
22
|
+
# @param job [ActiveJob::Base] The ActiveJob instance
|
23
|
+
#
|
24
|
+
# @return [Cloudtasker::CloudTask] The Google Task response
|
25
|
+
#
|
26
|
+
def enqueue(job)
|
27
|
+
build_worker(job).schedule
|
28
|
+
end
|
29
|
+
|
30
|
+
# Enqueues the given ActiveJob instance for execution at a given time
|
31
|
+
#
|
32
|
+
# @param job [ActiveJob::Base] The ActiveJob instance
|
33
|
+
# @param precise_timestamp [Integer] The timestamp at which the job must be executed
|
34
|
+
#
|
35
|
+
# @return [Cloudtasker::CloudTask] The Google Task response
|
36
|
+
#
|
37
|
+
def enqueue_at(job, precise_timestamp)
|
38
|
+
build_worker(job).schedule(time_at: Time.at(precise_timestamp))
|
39
|
+
end
|
40
|
+
|
41
|
+
private
|
42
|
+
|
43
|
+
def build_worker(job)
|
44
|
+
job_serialization = job.serialize.except(*SERIALIZATION_FILTERED_KEYS)
|
45
|
+
|
46
|
+
JobWrapper.new(
|
47
|
+
job_id: job_serialization.delete('job_id'),
|
48
|
+
job_queue: job_serialization.delete('queue_name'),
|
49
|
+
job_args: [job_serialization]
|
50
|
+
)
|
51
|
+
end
|
52
|
+
|
53
|
+
# == Job Wrapper for the Cloudtasker adapter
|
54
|
+
#
|
55
|
+
# Executes jobs scheduled by the Cloudtasker ActiveJob adapter
|
56
|
+
class JobWrapper #:nodoc:
|
57
|
+
include Cloudtasker::Worker
|
58
|
+
|
59
|
+
# Executes the given serialized ActiveJob call.
|
60
|
+
# - See https://api.rubyonrails.org/classes/ActiveJob/Core.html#method-i-serialize
|
61
|
+
#
|
62
|
+
# @param [Hash] job_serialization The serialized ActiveJob call
|
63
|
+
#
|
64
|
+
# @return [any] The execution of the ActiveJob call
|
65
|
+
#
|
66
|
+
def perform(job_serialization, *_extra_options)
|
67
|
+
job_executions = job_retries < 1 ? 0 : (job_retries + 1)
|
68
|
+
|
69
|
+
job_serialization.merge!(
|
70
|
+
'job_id' => job_id,
|
71
|
+
'queue_name' => job_queue,
|
72
|
+
'provider_job_id' => task_id,
|
73
|
+
'executions' => job_executions,
|
74
|
+
'priority' => nil
|
75
|
+
)
|
76
|
+
|
77
|
+
Base.execute job_serialization
|
78
|
+
end
|
79
|
+
end
|
80
|
+
end
|
81
|
+
end
|
82
|
+
end
|
data/lib/cloudtasker/cli.rb
CHANGED
data/lib/cloudtasker/engine.rb
CHANGED
@@ -5,12 +5,18 @@ module Cloudtasker
|
|
5
5
|
class Engine < ::Rails::Engine
|
6
6
|
isolate_namespace Cloudtasker
|
7
7
|
|
8
|
+
# Setup cloudtasker processing route
|
8
9
|
initializer 'cloudtasker', before: :load_config_initializers do
|
9
10
|
Rails.application.routes.append do
|
10
11
|
mount Cloudtasker::Engine, at: '/cloudtasker'
|
11
12
|
end
|
12
13
|
end
|
13
14
|
|
15
|
+
# Setup active job adapter
|
16
|
+
initializer 'cloudtasker.active_job', after: :load_config_initializers do
|
17
|
+
require 'active_job/queue_adapters/cloudtasker_adapter' if defined?(::ActiveJob::Railtie)
|
18
|
+
end
|
19
|
+
|
14
20
|
config.generators do |g|
|
15
21
|
g.test_framework :rspec, fixture: false
|
16
22
|
g.assets false
|
data/lib/cloudtasker/version.rb
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: cloudtasker
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.11.
|
4
|
+
version: 0.11.rc2
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Arnaud Lachaume
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2020-
|
11
|
+
date: 2020-11-23 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activesupport
|
@@ -178,6 +178,20 @@ dependencies:
|
|
178
178
|
- - "~>"
|
179
179
|
- !ruby/object:Gem::Version
|
180
180
|
version: '3.0'
|
181
|
+
- !ruby/object:Gem::Dependency
|
182
|
+
name: rspec-json_expectations
|
183
|
+
requirement: !ruby/object:Gem::Requirement
|
184
|
+
requirements:
|
185
|
+
- - "~>"
|
186
|
+
- !ruby/object:Gem::Version
|
187
|
+
version: '2.2'
|
188
|
+
type: :development
|
189
|
+
prerelease: false
|
190
|
+
version_requirements: !ruby/object:Gem::Requirement
|
191
|
+
requirements:
|
192
|
+
- - "~>"
|
193
|
+
- !ruby/object:Gem::Version
|
194
|
+
version: '2.2'
|
181
195
|
- !ruby/object:Gem::Dependency
|
182
196
|
name: rubocop
|
183
197
|
requirement: !ruby/object:Gem::Requirement
|
@@ -343,6 +357,7 @@ files:
|
|
343
357
|
- gemfiles/semantic_logger_4.7.0.gemfile
|
344
358
|
- gemfiles/semantic_logger_4.7.2.gemfile
|
345
359
|
- gemfiles/semantic_logger_4.7.gemfile
|
360
|
+
- lib/active_job/queue_adapters/cloudtasker_adapter.rb
|
346
361
|
- lib/cloudtasker.rb
|
347
362
|
- lib/cloudtasker/authentication_error.rb
|
348
363
|
- lib/cloudtasker/authenticator.rb
|