lev 4.3.2 → 5.0.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +8 -8
- data/README.md +13 -13
- data/lib/lev.rb +6 -6
- data/lib/lev/active_job.rb +8 -8
- data/lib/lev/background_job.rb +173 -0
- data/lib/lev/errors.rb +5 -5
- data/lib/lev/memory_store.rb +4 -0
- data/lib/lev/no_background_job.rb +26 -0
- data/lib/lev/routine.rb +11 -14
- data/lib/lev/version.rb +1 -1
- data/spec/active_job_routines_spec.rb +12 -0
- data/spec/lev/status_spec.rb +44 -0
- data/spec/statused_routines_spec.rb +69 -58
- metadata +6 -4
- data/lib/lev/black_hole_status.rb +0 -26
- data/lib/lev/status.rb +0 -125
checksums.yaml
CHANGED
@@ -1,15 +1,15 @@
|
|
1
1
|
---
|
2
2
|
!binary "U0hBMQ==":
|
3
3
|
metadata.gz: !binary |-
|
4
|
-
|
4
|
+
MGNhMzZiZWRhNDllMzMwYjBhMmFkZTQ5YzJiYWNkNjYxMzQ4NmMzMg==
|
5
5
|
data.tar.gz: !binary |-
|
6
|
-
|
6
|
+
NTdmM2U2NWY0YjlkNDM3N2ViZWU5NzQ3NmYyYjkyZDVkZDUwMzMyNw==
|
7
7
|
SHA512:
|
8
8
|
metadata.gz: !binary |-
|
9
|
-
|
10
|
-
|
11
|
-
|
9
|
+
NDMyM2UzNzY2MDQxODlkNmNlMWJhOTRjYjJkZmI2OWE3ZWM1ZDAyZDU1OWZk
|
10
|
+
ZmRhOWVjZWI5MGNhYWJmNWViMzJlNzlmMTQ0ZjVlMDNjODA1YzJkMDYwMzNj
|
11
|
+
OTc4NGVmYzI2ZDUzOWUwNTVmM2E5NzMyOGQ0MjQ5MWMzMTJhNjM=
|
12
12
|
data.tar.gz: !binary |-
|
13
|
-
|
14
|
-
|
15
|
-
|
13
|
+
M2I4ODI3Mjg0MWJiYzYyZGM0ZDA0OTVmMmJhOTZmMjk0NGIxMDA5ZjUxNTlm
|
14
|
+
NmI2ZmZhMTFmMWUyMDU0NzUwMjFlOGRkYjVhNWE3YjBhNTBkYmM0MmIxYTI0
|
15
|
+
ZDAzYzMyNWYyOTExNjA1NTJjMTJhM2RlZjY3YzNhNzU4NzdkZGE=
|
data/README.md
CHANGED
@@ -435,23 +435,23 @@ end
|
|
435
435
|
|
436
436
|
Routines run as ActiveJobs can also publish their status somewhere it can be listened to (e.g. to Redis).
|
437
437
|
|
438
|
-
Routines have a `
|
438
|
+
Routines have a `job` object and can call the following methods:
|
439
439
|
|
440
440
|
* `set_progress(at, out_of = nil)` sets the current progress; can either pass a float between 0.0 and 1.0 or
|
441
441
|
a counter towards a total, e.g. `set_progress(67,212)`.
|
442
|
-
* `queued!` Sets the status to 'queued'
|
443
|
-
* `working!` Sets the status to 'working'
|
444
|
-
* `completed!` Sets the status to 'completed'
|
445
|
-
* `failed!` Sets the status to 'failed'
|
446
|
-
* `killed!` Sets the status to 'killed'
|
447
|
-
* `save(hash)` Takes a hash of key value pairs and writes those keys and values to the status; there are several reserved keys which cannot be used (and which will blow up if you try to use them)
|
448
|
-
* `add_error(is_fatal, error)` takes a boolean and a Lev `Error` object and adds its data to an array of `errors` in the status hash.
|
442
|
+
* `queued!` Sets the job status to 'queued'
|
443
|
+
* `working!` Sets the job status to 'working'
|
444
|
+
* `completed!` Sets the job status to 'completed'
|
445
|
+
* `failed!` Sets the job status to 'failed'
|
446
|
+
* `killed!` Sets the job status to 'killed'
|
447
|
+
* `save(hash)` Takes a hash of key value pairs and writes those keys and values to the job status; there are several reserved keys which cannot be used (and which will blow up if you try to use them)
|
448
|
+
* `add_error(is_fatal, error)` takes a boolean and a Lev `Error` object and adds its data to an array of `errors` in the job status hash.
|
449
449
|
|
450
|
-
All routines have such a
|
450
|
+
All routines have such a job object. For plain vanilla routines not run as an active job, the job calls are no-ops. When a routine is invoked with `perform_later`, the job object actually records the jobs to a store of your choice. The store is configured in the Lev configuration block, e.g.:
|
451
451
|
|
452
452
|
```ruby
|
453
453
|
Lev.configure do |config|
|
454
|
-
config.
|
454
|
+
config.job_store = whatever
|
455
455
|
end
|
456
456
|
```
|
457
457
|
|
@@ -462,18 +462,18 @@ The store needs to respond to the following methods:
|
|
462
462
|
|
463
463
|
The default store is essentially a hash (implemented in `Lev::MemoryStore`). Any `ActiveSupport::Cache::Store` will work.
|
464
464
|
|
465
|
-
A routine's
|
465
|
+
A routine's job can be retrieved with `Lev::BackgroundJob.get(uuid_here)`. This just returns a simple hash. Notable keys are
|
466
466
|
|
467
467
|
* `'progress'` which returns the progress as a number between 0.0 and 1.0
|
468
468
|
* `'status'` which returns one of the status strings shown above
|
469
469
|
* `'errors'` which (if present) is an error of error hashes
|
470
|
-
* `'
|
470
|
+
* `'id'` the UUID of the routine / job
|
471
471
|
|
472
472
|
Other routine-specific keys (set with a `save` call) are also present.
|
473
473
|
|
474
474
|
**Notes:**
|
475
475
|
|
476
|
-
1. Don't try to write a
|
476
|
+
1. Don't try to write a job store that uses the ActiveRecord database, as the database changes would only be seen when the routine completes and its transaction is committed.
|
477
477
|
2. Job killing hasn't been implemented yet, but shouldn't be too bad. For routines run in a transaction (which are frankly the only ones you'd want to kill), we can likely kill them by raising an exception or similar to cause a rollback (will need to have good tests to prove that).
|
478
478
|
|
479
479
|
## Handlers
|
data/lib/lev.rb
CHANGED
@@ -26,8 +26,8 @@ require "lev/transaction_isolation"
|
|
26
26
|
|
27
27
|
require 'lev/active_job'
|
28
28
|
require 'lev/memory_store'
|
29
|
-
require 'lev/
|
30
|
-
require 'lev/
|
29
|
+
require 'lev/background_job'
|
30
|
+
require 'lev/no_background_job'
|
31
31
|
|
32
32
|
module Lev
|
33
33
|
class << self
|
@@ -59,16 +59,16 @@ module Lev
|
|
59
59
|
attr_accessor :security_transgression_error
|
60
60
|
attr_accessor :illegal_argument_error
|
61
61
|
attr_accessor :raise_fatal_errors
|
62
|
-
attr_accessor :
|
63
|
-
attr_accessor :
|
62
|
+
attr_accessor :job_store
|
63
|
+
attr_accessor :job_store_namespace
|
64
64
|
|
65
65
|
def initialize
|
66
66
|
@form_error_class = 'error'
|
67
67
|
@security_transgression_error = Lev::SecurityTransgression
|
68
68
|
@illegal_argument_error = Lev::IllegalArgument
|
69
69
|
@raise_fatal_errors = false
|
70
|
-
@
|
71
|
-
@
|
70
|
+
@job_store = Lev::MemoryStore.new
|
71
|
+
@job_store_namespace = "lev_job"
|
72
72
|
super
|
73
73
|
end
|
74
74
|
end
|
data/lib/lev/active_job.rb
CHANGED
@@ -6,25 +6,25 @@ if defined?(::ActiveJob)
|
|
6
6
|
queue_as routine_class.active_job_queue
|
7
7
|
args.push(routine_class.to_s)
|
8
8
|
|
9
|
-
# To enable tracking of this job's status, create a new
|
9
|
+
# To enable tracking of this job's status, create a new BackgroundJob object
|
10
10
|
# and push it on to the arguments so that in `perform` it can be peeled
|
11
|
-
# off and handed to the routine instance. The
|
11
|
+
# off and handed to the routine instance. The BackgroundJob UUID is returned
|
12
12
|
# so that callers can track the status.
|
13
|
-
|
14
|
-
|
15
|
-
args.push(
|
13
|
+
job = Lev::BackgroundJob.new
|
14
|
+
job.queued!
|
15
|
+
args.push(job.id)
|
16
16
|
|
17
17
|
super(*args, &block)
|
18
18
|
|
19
|
-
|
19
|
+
job.id
|
20
20
|
end
|
21
21
|
|
22
22
|
def perform(*args, &block)
|
23
23
|
# Pop arguments added by perform_later
|
24
|
-
|
24
|
+
id = args.pop
|
25
25
|
routine_class = Kernel.const_get(args.pop)
|
26
26
|
|
27
|
-
routine_instance = routine_class.new(Lev::
|
27
|
+
routine_instance = routine_class.new(Lev::BackgroundJob.new(id: id))
|
28
28
|
routine_instance.call(*args, &block)
|
29
29
|
end
|
30
30
|
end
|
@@ -0,0 +1,173 @@
|
|
1
|
+
require 'json'
|
2
|
+
|
3
|
+
module Lev
|
4
|
+
class BackgroundJob
|
5
|
+
attr_reader :id, :status, :progress, :errors
|
6
|
+
|
7
|
+
STATE_QUEUED = 'queued'
|
8
|
+
STATE_WORKING = 'working'
|
9
|
+
STATE_COMPLETED = 'completed'
|
10
|
+
STATE_FAILED = 'failed'
|
11
|
+
STATE_KILLED = 'killed'
|
12
|
+
STATE_UNKNOWN = 'unknown'
|
13
|
+
|
14
|
+
STATES = [
|
15
|
+
STATE_QUEUED,
|
16
|
+
STATE_WORKING,
|
17
|
+
STATE_COMPLETED,
|
18
|
+
STATE_FAILED,
|
19
|
+
STATE_KILLED,
|
20
|
+
STATE_UNKNOWN
|
21
|
+
].freeze
|
22
|
+
|
23
|
+
def initialize(attrs = {})
|
24
|
+
@id = attrs[:id] || attrs['id'] || SecureRandom.uuid
|
25
|
+
@status = attrs[:status] || attrs['status'] || STATE_UNKNOWN
|
26
|
+
@progress = attrs[:progress] || attrs['progress'] || set_progress(0)
|
27
|
+
@errors = attrs[:errors] || attrs['errors'] || []
|
28
|
+
|
29
|
+
set({ id: id,
|
30
|
+
status: status,
|
31
|
+
progress: progress,
|
32
|
+
errors: errors })
|
33
|
+
end
|
34
|
+
|
35
|
+
def self.find(id)
|
36
|
+
attrs = { id: id }
|
37
|
+
|
38
|
+
if job = store.fetch(job_key(id))
|
39
|
+
attrs.merge!(JSON.parse(job))
|
40
|
+
else
|
41
|
+
attrs.merge!(status: STATE_UNKNOWN)
|
42
|
+
end
|
43
|
+
|
44
|
+
new(attrs)
|
45
|
+
end
|
46
|
+
|
47
|
+
def self.all
|
48
|
+
job_ids.map { |id| find(id) }
|
49
|
+
end
|
50
|
+
|
51
|
+
def set_progress(at, out_of = nil)
|
52
|
+
progress = compute_fractional_progress(at, out_of)
|
53
|
+
|
54
|
+
data_to_set = { progress: progress }
|
55
|
+
data_to_set[:status] = STATE_COMPLETED if 1.0 == progress
|
56
|
+
|
57
|
+
set(data_to_set)
|
58
|
+
|
59
|
+
progress
|
60
|
+
end
|
61
|
+
|
62
|
+
STATES.each do |state|
|
63
|
+
define_method("#{state}!") do
|
64
|
+
set(status: state)
|
65
|
+
end
|
66
|
+
end
|
67
|
+
|
68
|
+
def add_error(error, options = { })
|
69
|
+
options = { is_fatal: false }.merge(options)
|
70
|
+
@errors << { is_fatal: options[:is_fatal],
|
71
|
+
code: error.code,
|
72
|
+
message: error.message }
|
73
|
+
set(errors: @errors)
|
74
|
+
end
|
75
|
+
|
76
|
+
# Rails compatibility
|
77
|
+
# returns a Hash of all key-value pairs that have been #set()
|
78
|
+
def as_json(options = {})
|
79
|
+
stored
|
80
|
+
end
|
81
|
+
|
82
|
+
def save(incoming_hash)
|
83
|
+
if reserved = incoming_hash.select { |k, _| RESERVED_KEYS.include?(k) }.first
|
84
|
+
raise IllegalArgument, "Cannot set reserved key: #{reserved[0]}"
|
85
|
+
else
|
86
|
+
set(incoming_hash)
|
87
|
+
end
|
88
|
+
end
|
89
|
+
|
90
|
+
def method_missing(method_name, *args)
|
91
|
+
instance_variable_get("@#{method_name}") || super
|
92
|
+
end
|
93
|
+
|
94
|
+
def respond_to?(method_name)
|
95
|
+
if method_name.match /\?$/
|
96
|
+
super
|
97
|
+
else
|
98
|
+
instance_variable_get("@#{method_name}").present? || super
|
99
|
+
end
|
100
|
+
end
|
101
|
+
|
102
|
+
protected
|
103
|
+
RESERVED_KEYS = [:id, :status, :progress, :errors]
|
104
|
+
|
105
|
+
def set(incoming_hash)
|
106
|
+
incoming_hash = stored.merge(incoming_hash)
|
107
|
+
incoming_hash.each { |k, v| instance_variable_set("@#{k}", v) }
|
108
|
+
self.class.store.write(job_key, incoming_hash.to_json)
|
109
|
+
track_job_id
|
110
|
+
end
|
111
|
+
|
112
|
+
def self.store
|
113
|
+
Lev.configuration.job_store
|
114
|
+
end
|
115
|
+
|
116
|
+
def self.job_ids
|
117
|
+
store.fetch(job_key('lev_job_ids')) || []
|
118
|
+
end
|
119
|
+
|
120
|
+
def stored
|
121
|
+
if found = self.class.store.fetch(job_key)
|
122
|
+
JSON.parse(found)
|
123
|
+
else
|
124
|
+
{}
|
125
|
+
end
|
126
|
+
end
|
127
|
+
|
128
|
+
def track_job_id
|
129
|
+
ids = self.class.job_ids
|
130
|
+
ids << @id
|
131
|
+
self.class.store.write(self.class.job_key('lev_job_ids'), ids.uniq)
|
132
|
+
end
|
133
|
+
|
134
|
+
def job_key
|
135
|
+
self.class.job_key(@id)
|
136
|
+
end
|
137
|
+
|
138
|
+
def self.job_key(id)
|
139
|
+
"#{Lev.configuration.job_store_namespace}:#{id}"
|
140
|
+
end
|
141
|
+
|
142
|
+
def has_reserved_keys?(hash)
|
143
|
+
(hash.keys.collect(&:to_sym) & RESERVED_KEYS).any?
|
144
|
+
end
|
145
|
+
|
146
|
+
def push(key, new_item)
|
147
|
+
new_value = (send(key) || []).push(new_item)
|
148
|
+
set(key => new_value)
|
149
|
+
end
|
150
|
+
|
151
|
+
STATES.each do |state|
|
152
|
+
define_method("#{state}?") do
|
153
|
+
status == state
|
154
|
+
end
|
155
|
+
end
|
156
|
+
|
157
|
+
def compute_fractional_progress(at, out_of)
|
158
|
+
if at.nil?
|
159
|
+
raise IllegalArgument, "Must specify at least `at` argument to `progress` call"
|
160
|
+
elsif at < 0
|
161
|
+
raise IllegalArgument, "progress cannot be negative (at=#{at})"
|
162
|
+
elsif out_of && out_of < at
|
163
|
+
raise IllegalArgument, "`out_of` must be greater than `at` in `progress` calls"
|
164
|
+
elsif out_of.nil? && (at < 0 || at > 1)
|
165
|
+
raise IllegalArgument, "If `out_of` not specified, `at` must be in the range [0.0, 1.0]"
|
166
|
+
end
|
167
|
+
|
168
|
+
at.to_f / (out_of || 1).to_f
|
169
|
+
end
|
170
|
+
|
171
|
+
end
|
172
|
+
end
|
173
|
+
|
data/lib/lev/errors.rb
CHANGED
@@ -4,8 +4,8 @@ module Lev
|
|
4
4
|
#
|
5
5
|
class Errors < Array
|
6
6
|
|
7
|
-
def initialize(
|
8
|
-
@
|
7
|
+
def initialize(routine_job = nil, raise_fatal_errors = false)
|
8
|
+
@routine_job = routine_job || NoBackgroundJob.new
|
9
9
|
@raise_fatal_errors = raise_fatal_errors
|
10
10
|
end
|
11
11
|
|
@@ -16,10 +16,10 @@ module Lev
|
|
16
16
|
return if ignored_error_procs.any?{|proc| proc.call(error)}
|
17
17
|
self.push(error)
|
18
18
|
|
19
|
-
|
19
|
+
routine_job.add_error(error, is_fatal: fail)
|
20
20
|
|
21
21
|
if fail
|
22
|
-
|
22
|
+
routine_job.failed!
|
23
23
|
|
24
24
|
if raise_fatal_errors
|
25
25
|
raise StandardError, args.to_a.map { |i| i.join(' ') }.join(' - ')
|
@@ -54,7 +54,7 @@ module Lev
|
|
54
54
|
|
55
55
|
protected
|
56
56
|
|
57
|
-
attr_reader :
|
57
|
+
attr_reader :routine_job
|
58
58
|
attr_reader :raise_fatal_errors
|
59
59
|
|
60
60
|
def ignored_error_procs
|
data/lib/lev/memory_store.rb
CHANGED
@@ -0,0 +1,26 @@
|
|
1
|
+
module Lev
|
2
|
+
class NoBackgroundJob
|
3
|
+
|
4
|
+
# Provide null object pattern methods for background jobs; routines should
|
5
|
+
# not be checking their own status (they should know it), and outside callers
|
6
|
+
# should not be checking status unless the background job is a real one.
|
7
|
+
|
8
|
+
def set_progress(*); end
|
9
|
+
def save(*); end
|
10
|
+
def add_error(*); end
|
11
|
+
|
12
|
+
Lev::BackgroundJob::STATES.each do |state|
|
13
|
+
define_method("#{state}!") do; end
|
14
|
+
end
|
15
|
+
|
16
|
+
def self.method_missing(method_sym, *args, &block)
|
17
|
+
if Lev::BackgroundJob.new.respond_to?(method_sym)
|
18
|
+
raise NameError,
|
19
|
+
"'#{method_sym}' is Lev::BackgroundJob query method, and those cannot be called on NoBackgroundJob"
|
20
|
+
else
|
21
|
+
super
|
22
|
+
end
|
23
|
+
end
|
24
|
+
|
25
|
+
end
|
26
|
+
end
|
data/lib/lev/routine.rb
CHANGED
@@ -191,7 +191,7 @@ module Lev
|
|
191
191
|
end
|
192
192
|
end
|
193
193
|
|
194
|
-
attr_reader :
|
194
|
+
attr_reader :id
|
195
195
|
|
196
196
|
def self.included(base)
|
197
197
|
base.extend(ClassMethods)
|
@@ -267,7 +267,7 @@ module Lev
|
|
267
267
|
def call(*args, &block)
|
268
268
|
@after_transaction_blocks = []
|
269
269
|
|
270
|
-
|
270
|
+
job.working!
|
271
271
|
|
272
272
|
in_transaction do
|
273
273
|
catch :fatal_errors_encountered do
|
@@ -285,7 +285,7 @@ module Lev
|
|
285
285
|
block.call
|
286
286
|
end
|
287
287
|
|
288
|
-
|
288
|
+
job.completed! if !errors?
|
289
289
|
|
290
290
|
result
|
291
291
|
end
|
@@ -418,26 +418,23 @@ module Lev
|
|
418
418
|
|
419
419
|
# Note that the parent may neglect to call super, leading to this method never being called.
|
420
420
|
# Do not perform any initialization here that cannot be safely skipped
|
421
|
-
def initialize(
|
422
|
-
# If someone cares about the
|
423
|
-
#
|
424
|
-
@
|
421
|
+
def initialize(job = nil)
|
422
|
+
# If someone cares about the job, they'll pass it in; otherwise all
|
423
|
+
# job updates go into the bit bucket.
|
424
|
+
@job = job
|
425
425
|
end
|
426
426
|
|
427
427
|
protected
|
428
428
|
|
429
429
|
attr_writer :runner
|
430
430
|
|
431
|
-
def
|
432
|
-
@
|
431
|
+
def job
|
432
|
+
@job ||= Lev::NoBackgroundJob.new
|
433
433
|
end
|
434
434
|
|
435
435
|
def result
|
436
|
-
@result ||= Result.new(
|
437
|
-
|
438
|
-
Errors.new(status,
|
439
|
-
topmost_runner.class.raise_fatal_errors?)
|
440
|
-
)
|
436
|
+
@result ||= Result.new(Outputs.new,
|
437
|
+
Errors.new(job, topmost_runner.class.raise_fatal_errors?))
|
441
438
|
end
|
442
439
|
|
443
440
|
def outputs
|
data/lib/lev/version.rb
CHANGED
@@ -16,9 +16,21 @@ RSpec.describe 'ActiveJob routines' do
|
|
16
16
|
end
|
17
17
|
|
18
18
|
it 'can have the default queue overridden' do
|
19
|
+
ActiveJob::Base.queue_adapter.enqueued_jobs.clear
|
20
|
+
|
19
21
|
LaterRoutine.perform_later
|
22
|
+
|
20
23
|
queue_name = ActiveJob::Base.queue_adapter.enqueued_jobs.first[:queue]
|
24
|
+
|
21
25
|
expect(queue_name).to eq('something_else')
|
22
26
|
end
|
27
|
+
|
28
|
+
it 'stores all the UUIDs of queued jobs' do
|
29
|
+
Lev.configuration.job_store.clear
|
30
|
+
|
31
|
+
job_id1 = LaterRoutine.perform_later
|
32
|
+
|
33
|
+
expect(Lev::BackgroundJob.send(:job_ids)).to eq([job_id1])
|
34
|
+
end
|
23
35
|
end
|
24
36
|
end
|
@@ -0,0 +1,44 @@
|
|
1
|
+
require 'spec_helper'
|
2
|
+
|
3
|
+
RSpec.describe Lev::BackgroundJob do
|
4
|
+
class DelayedRoutine
|
5
|
+
lev_routine
|
6
|
+
protected
|
7
|
+
def exec; end
|
8
|
+
end
|
9
|
+
|
10
|
+
subject(:job) { described_class.all.last }
|
11
|
+
|
12
|
+
before do
|
13
|
+
Lev.configuration.job_store.clear
|
14
|
+
allow(SecureRandom).to receive(:uuid) { '123abc' }
|
15
|
+
DelayedRoutine.perform_later
|
16
|
+
end
|
17
|
+
|
18
|
+
it 'behaves as a nice ruby object' do
|
19
|
+
expect(job.id).to eq('123abc')
|
20
|
+
expect(job.status).to eq(Lev::BackgroundJob::STATE_QUEUED)
|
21
|
+
expect(job.progress).to eq(0.0)
|
22
|
+
end
|
23
|
+
|
24
|
+
it 'is unknown when not found' do
|
25
|
+
foo = described_class.find('noooooo')
|
26
|
+
expect(foo.status).to eq(Lev::BackgroundJob::STATE_UNKNOWN)
|
27
|
+
end
|
28
|
+
|
29
|
+
it 'uses as_json' do
|
30
|
+
json = job.as_json
|
31
|
+
|
32
|
+
expect(json).to eq({
|
33
|
+
'id' => '123abc',
|
34
|
+
'status' => Lev::BackgroundJob::STATE_QUEUED,
|
35
|
+
'progress' => 0.0,
|
36
|
+
'errors' => []
|
37
|
+
})
|
38
|
+
|
39
|
+
job.save(foo: :bar)
|
40
|
+
json = job.as_json
|
41
|
+
|
42
|
+
expect(json['foo']).to eq('bar')
|
43
|
+
end
|
44
|
+
end
|
@@ -5,135 +5,146 @@ class StatusedRoutine
|
|
5
5
|
|
6
6
|
protected
|
7
7
|
def exec
|
8
|
-
|
8
|
+
job.set_progress(9, 10)
|
9
9
|
end
|
10
10
|
end
|
11
11
|
|
12
12
|
RSpec.describe 'Statused Routines' do
|
13
|
-
subject(:
|
13
|
+
subject(:job) { Lev::BackgroundJob.new }
|
14
14
|
|
15
15
|
context 'in a routine' do
|
16
|
-
it 'queues the
|
17
|
-
|
18
|
-
|
16
|
+
it 'queues the job object on queue' do
|
17
|
+
id = StatusedRoutine.perform_later
|
18
|
+
job = Lev::BackgroundJob.find(id)
|
19
19
|
|
20
|
-
expect(status
|
20
|
+
expect(job.status).to eq(Lev::BackgroundJob::STATE_QUEUED)
|
21
21
|
end
|
22
22
|
|
23
23
|
context 'inline activejob mode' do
|
24
24
|
before { ::ActiveJob::Base.queue_adapter = :inline }
|
25
25
|
after { ::ActiveJob::Base.queue_adapter = :test }
|
26
26
|
|
27
|
-
it 'sets
|
28
|
-
expect_any_instance_of(Lev::
|
27
|
+
it 'sets job to working when called' do
|
28
|
+
expect_any_instance_of(Lev::BackgroundJob).to receive(:working!)
|
29
29
|
StatusedRoutine.perform_later
|
30
30
|
end
|
31
31
|
|
32
|
-
it 'completes the
|
33
|
-
|
34
|
-
|
35
|
-
expect(status
|
36
|
-
expect(
|
32
|
+
it 'completes the job object on completion, returning other data' do
|
33
|
+
id = StatusedRoutine.perform_later
|
34
|
+
job = Lev::BackgroundJob.find(id)
|
35
|
+
expect(job.status).to eq(Lev::BackgroundJob::STATE_COMPLETED)
|
36
|
+
expect(job.progress).to eq(0.9)
|
37
37
|
end
|
38
38
|
end
|
39
39
|
end
|
40
40
|
|
41
|
+
describe '#save' do
|
42
|
+
it 'saves the hash given and writes them to the job' do
|
43
|
+
job.save(something: 'else')
|
44
|
+
expect(job.something).to eq('else')
|
45
|
+
end
|
46
|
+
end
|
47
|
+
|
48
|
+
describe '#add_error' do
|
49
|
+
it 'adds the error object data to the job object' do
|
50
|
+
errors = Lev::Error.new(code: 'bad', message: 'awful')
|
51
|
+
job.add_error(errors)
|
52
|
+
expect(job.errors).to eq([{ is_fatal: false,
|
53
|
+
code: 'bad',
|
54
|
+
message: 'awful' }])
|
55
|
+
end
|
56
|
+
end
|
57
|
+
|
41
58
|
describe '#save' do
|
42
59
|
it 'prevents the use of reserved keys' do
|
43
60
|
expect {
|
44
|
-
|
61
|
+
job.save(progress: 'blocked')
|
45
62
|
}.to raise_error(Lev::IllegalArgument)
|
46
63
|
|
47
64
|
expect {
|
48
|
-
|
65
|
+
job.save(id: 'blocked')
|
49
66
|
}.to raise_error(Lev::IllegalArgument)
|
50
67
|
|
51
68
|
expect {
|
52
|
-
|
69
|
+
job.save(status: 'blocked')
|
53
70
|
}.to raise_error(Lev::IllegalArgument)
|
54
71
|
|
55
72
|
expect {
|
56
|
-
|
73
|
+
job.save(errors: 'blocked')
|
57
74
|
}.to raise_error(Lev::IllegalArgument)
|
58
75
|
end
|
59
76
|
|
60
|
-
it 'saves the hash given and writes them to the
|
61
|
-
|
62
|
-
expect(
|
63
|
-
|
64
|
-
end
|
65
|
-
|
66
|
-
describe '#add_error' do
|
67
|
-
it 'adds the error object data to the status object' do
|
68
|
-
errors = Lev::Error.new(code: 'bad', message: 'awful')
|
69
|
-
status.add_error(errors)
|
70
|
-
expect(status.get('errors')).to eq([{ 'is_fatal' => false,
|
71
|
-
'code' => 'bad',
|
72
|
-
'message' => 'awful' }])
|
77
|
+
it 'saves the hash given and writes them to the job' do
|
78
|
+
job.save(something: 'else')
|
79
|
+
expect(job).to respond_to(:something)
|
80
|
+
expect(job.something).to eq('else')
|
73
81
|
end
|
74
82
|
end
|
75
83
|
|
76
|
-
describe 'dynamic
|
84
|
+
describe 'dynamic job setters/getters' do
|
77
85
|
it 'is queued' do
|
78
|
-
expect(
|
79
|
-
|
80
|
-
expect(
|
86
|
+
expect(job).not_to be_queued
|
87
|
+
job.queued!
|
88
|
+
expect(job).to be_queued
|
81
89
|
end
|
82
90
|
|
83
91
|
it 'is working' do
|
84
|
-
expect(
|
85
|
-
|
86
|
-
expect(
|
92
|
+
expect(job).not_to be_working
|
93
|
+
job.working!
|
94
|
+
expect(job).to be_working
|
87
95
|
end
|
88
96
|
|
89
97
|
it 'is completed' do
|
90
|
-
expect(
|
91
|
-
|
92
|
-
expect(
|
98
|
+
expect(job).not_to be_completed
|
99
|
+
job.completed!
|
100
|
+
expect(job).to be_completed
|
93
101
|
end
|
94
102
|
|
95
103
|
it 'is failed' do
|
96
|
-
expect(
|
97
|
-
|
98
|
-
expect(
|
104
|
+
expect(job).not_to be_failed
|
105
|
+
job.failed!
|
106
|
+
expect(job).to be_failed
|
99
107
|
end
|
100
108
|
|
101
109
|
it 'is killed' do
|
102
|
-
expect(
|
103
|
-
|
104
|
-
expect(
|
110
|
+
expect(job).not_to be_killed
|
111
|
+
job.killed!
|
112
|
+
expect(job).to be_killed
|
113
|
+
end
|
114
|
+
|
115
|
+
it 'is unknown' do
|
116
|
+
expect(job).to be_unknown
|
105
117
|
end
|
106
118
|
end
|
107
119
|
|
108
120
|
describe '#set_progress' do
|
109
|
-
it 'sets the progress key on the
|
110
|
-
|
111
|
-
progress
|
112
|
-
expect(progress).to eq(0.8)
|
121
|
+
it 'sets the progress key on the job object' do
|
122
|
+
job.set_progress(8, 10)
|
123
|
+
expect(job.progress).to eq(0.8)
|
113
124
|
end
|
114
125
|
|
115
126
|
context 'when `out_of` is supplied' do
|
116
127
|
it 'requires a positive `at` float or integer' do
|
117
128
|
expect {
|
118
|
-
|
129
|
+
job.set_progress(nil, 1)
|
119
130
|
}.to raise_error(Lev::IllegalArgument)
|
120
131
|
|
121
132
|
expect {
|
122
|
-
|
133
|
+
job.set_progress(-1, 1)
|
123
134
|
}.to raise_error(Lev::IllegalArgument)
|
124
135
|
|
125
136
|
expect {
|
126
|
-
|
137
|
+
job.set_progress(2, 5)
|
127
138
|
}.not_to raise_error
|
128
139
|
end
|
129
140
|
|
130
141
|
it 'requires `out_of` to be greater than `at`' do
|
131
142
|
expect {
|
132
|
-
|
143
|
+
job.set_progress(15, 8)
|
133
144
|
}.to raise_error(Lev::IllegalArgument)
|
134
145
|
|
135
146
|
expect {
|
136
|
-
|
147
|
+
job.set_progress(5, 10)
|
137
148
|
}.not_to raise_error
|
138
149
|
end
|
139
150
|
end
|
@@ -141,15 +152,15 @@ RSpec.describe 'Statused Routines' do
|
|
141
152
|
context 'without out_of specified' do
|
142
153
|
it 'requires `at` to be a float between 0.0 and 1.0' do
|
143
154
|
expect {
|
144
|
-
|
155
|
+
job.set_progress(1.1)
|
145
156
|
}.to raise_error(Lev::IllegalArgument)
|
146
157
|
|
147
158
|
expect {
|
148
|
-
|
159
|
+
job.set_progress(-1)
|
149
160
|
}.to raise_error(Lev::IllegalArgument)
|
150
161
|
|
151
162
|
expect {
|
152
|
-
|
163
|
+
job.set_progress(0.78)
|
153
164
|
}.not_to raise_error
|
154
165
|
end
|
155
166
|
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: lev
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version:
|
4
|
+
version: 5.0.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- JP Slavinsky
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2015-07-
|
11
|
+
date: 2015-07-30 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activemodel
|
@@ -218,8 +218,8 @@ files:
|
|
218
218
|
- Rakefile
|
219
219
|
- lib/lev.rb
|
220
220
|
- lib/lev/active_job.rb
|
221
|
+
- lib/lev/background_job.rb
|
221
222
|
- lib/lev/better_active_model_errors.rb
|
222
|
-
- lib/lev/black_hole_status.rb
|
223
223
|
- lib/lev/delegate_to_routine.rb
|
224
224
|
- lib/lev/error.rb
|
225
225
|
- lib/lev/error_transferer.rb
|
@@ -231,10 +231,10 @@ files:
|
|
231
231
|
- lib/lev/handler.rb
|
232
232
|
- lib/lev/handler_helper.rb
|
233
233
|
- lib/lev/memory_store.rb
|
234
|
+
- lib/lev/no_background_job.rb
|
234
235
|
- lib/lev/object.rb
|
235
236
|
- lib/lev/outputs.rb
|
236
237
|
- lib/lev/routine.rb
|
237
|
-
- lib/lev/status.rb
|
238
238
|
- lib/lev/term_mapper.rb
|
239
239
|
- lib/lev/transaction_isolation.rb
|
240
240
|
- lib/lev/utilities.rb
|
@@ -244,6 +244,7 @@ files:
|
|
244
244
|
- spec/deep_merge_spec.rb
|
245
245
|
- spec/delegates_to_spec.rb
|
246
246
|
- spec/errors_spec.rb
|
247
|
+
- spec/lev/status_spec.rb
|
247
248
|
- spec/outputs_spec.rb
|
248
249
|
- spec/paramify_handler_spec.rb
|
249
250
|
- spec/routine_spec.rb
|
@@ -289,6 +290,7 @@ test_files:
|
|
289
290
|
- spec/deep_merge_spec.rb
|
290
291
|
- spec/delegates_to_spec.rb
|
291
292
|
- spec/errors_spec.rb
|
293
|
+
- spec/lev/status_spec.rb
|
292
294
|
- spec/outputs_spec.rb
|
293
295
|
- spec/paramify_handler_spec.rb
|
294
296
|
- spec/routine_spec.rb
|
@@ -1,26 +0,0 @@
|
|
1
|
-
module Lev
|
2
|
-
class BlackHoleStatus
|
3
|
-
|
4
|
-
# Provide null object pattern methods for status setters; routines should
|
5
|
-
# not be checking their own status (they should know it), and outside callers
|
6
|
-
# should not be checking status unless the status object is a real one.
|
7
|
-
|
8
|
-
def set_progress(*); end
|
9
|
-
def save(*); end
|
10
|
-
def add_error(*); end
|
11
|
-
|
12
|
-
Lev::Status::STATES.each do |state|
|
13
|
-
define_method("#{state}!") do; end
|
14
|
-
end
|
15
|
-
|
16
|
-
def self.method_missing(method_sym, *args, &block)
|
17
|
-
if Lev::Status.new.respond_to?(method_sym)
|
18
|
-
raise NameError,
|
19
|
-
"'#{method_sym}' is Lev::Status query method, and those cannot be called on BlackHoleStatus"
|
20
|
-
else
|
21
|
-
super
|
22
|
-
end
|
23
|
-
end
|
24
|
-
|
25
|
-
end
|
26
|
-
end
|
data/lib/lev/status.rb
DELETED
@@ -1,125 +0,0 @@
|
|
1
|
-
require 'json'
|
2
|
-
|
3
|
-
module Lev
|
4
|
-
class Status
|
5
|
-
STATE_QUEUED = 'queued'
|
6
|
-
STATE_WORKING = 'working'
|
7
|
-
STATE_COMPLETED = 'completed'
|
8
|
-
STATE_FAILED = 'failed'
|
9
|
-
STATE_KILLED = 'killed'
|
10
|
-
|
11
|
-
STATES = [
|
12
|
-
STATE_QUEUED,
|
13
|
-
STATE_WORKING,
|
14
|
-
STATE_COMPLETED,
|
15
|
-
STATE_FAILED,
|
16
|
-
STATE_KILLED
|
17
|
-
].freeze
|
18
|
-
|
19
|
-
attr_reader :uuid
|
20
|
-
|
21
|
-
def initialize(uuid = nil)
|
22
|
-
@uuid = uuid || SecureRandom.uuid
|
23
|
-
save
|
24
|
-
end
|
25
|
-
|
26
|
-
def self.find(uuid)
|
27
|
-
if status = store.fetch(status_key(uuid))
|
28
|
-
JSON.parse(status)
|
29
|
-
else
|
30
|
-
nil
|
31
|
-
end
|
32
|
-
end
|
33
|
-
|
34
|
-
def set_progress(at, out_of = nil)
|
35
|
-
progress = compute_fractional_progress(at, out_of)
|
36
|
-
|
37
|
-
data_to_set = { progress: progress }
|
38
|
-
data_to_set[:state] = STATE_COMPLETED if 1.0 == progress
|
39
|
-
|
40
|
-
set(data_to_set)
|
41
|
-
end
|
42
|
-
|
43
|
-
STATES.each do |state|
|
44
|
-
define_method("#{state}!") do
|
45
|
-
set(state: state)
|
46
|
-
end
|
47
|
-
end
|
48
|
-
|
49
|
-
def save(hash = {})
|
50
|
-
if has_reserved_keys?(hash)
|
51
|
-
raise IllegalArgument,
|
52
|
-
"Caller cannot specify any reserved keys (#{RESERVED_KEYS})"
|
53
|
-
else
|
54
|
-
set(hash)
|
55
|
-
end
|
56
|
-
end
|
57
|
-
|
58
|
-
def add_error(error, options = { })
|
59
|
-
options = { is_fatal: false }.merge(options)
|
60
|
-
push('errors', { is_fatal: options[:is_fatal],
|
61
|
-
code: error.code,
|
62
|
-
message: error.message })
|
63
|
-
end
|
64
|
-
|
65
|
-
def get(key)
|
66
|
-
self.class.find(uuid)[key]
|
67
|
-
end
|
68
|
-
|
69
|
-
protected
|
70
|
-
RESERVED_KEYS = [:progress, :uuid, :state, :errors]
|
71
|
-
|
72
|
-
def self.store
|
73
|
-
# Nice to get the store from lev config each time so it isn't serialized
|
74
|
-
# when activejobs are sent off to places like redis
|
75
|
-
Lev.configuration.status_store
|
76
|
-
end
|
77
|
-
|
78
|
-
def set(incoming_hash)
|
79
|
-
if existing_settings = self.class.find(uuid)
|
80
|
-
incoming_hash = existing_settings.merge(incoming_hash)
|
81
|
-
end
|
82
|
-
|
83
|
-
self.class.store.write(status_key, incoming_hash.to_json)
|
84
|
-
end
|
85
|
-
|
86
|
-
def status_key
|
87
|
-
self.class.status_key(uuid)
|
88
|
-
end
|
89
|
-
|
90
|
-
def self.status_key(uuid)
|
91
|
-
"#{Lev.configuration.status_store_namespace}:#{uuid}"
|
92
|
-
end
|
93
|
-
|
94
|
-
def has_reserved_keys?(hash)
|
95
|
-
(hash.keys.collect(&:to_sym) & RESERVED_KEYS).any?
|
96
|
-
end
|
97
|
-
|
98
|
-
def push(key, new_item)
|
99
|
-
new_value = (get(key) || []).push(new_item)
|
100
|
-
set(key => new_value)
|
101
|
-
end
|
102
|
-
|
103
|
-
STATES.each do |state|
|
104
|
-
define_method("#{state}?") do
|
105
|
-
get('state') == state
|
106
|
-
end
|
107
|
-
end
|
108
|
-
|
109
|
-
def compute_fractional_progress(at, out_of)
|
110
|
-
if at.nil?
|
111
|
-
raise IllegalArgument, "Must specify at least `at` argument to `progress` call"
|
112
|
-
elsif at < 0
|
113
|
-
raise IllegalArgument, "progress cannot be negative (at=#{at})"
|
114
|
-
elsif out_of && out_of < at
|
115
|
-
raise IllegalArgument, "`out_of` must be greater than `at` in `progress` calls"
|
116
|
-
elsif out_of.nil? && (at < 0 || at > 1)
|
117
|
-
raise IllegalArgument, "If `out_of` not specified, `at` must be in the range [0.0, 1.0]"
|
118
|
-
end
|
119
|
-
|
120
|
-
at.to_f / (out_of || 1).to_f
|
121
|
-
end
|
122
|
-
|
123
|
-
end
|
124
|
-
end
|
125
|
-
|