lev 4.3.2 → 5.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +8 -8
- data/README.md +13 -13
- data/lib/lev.rb +6 -6
- data/lib/lev/active_job.rb +8 -8
- data/lib/lev/background_job.rb +173 -0
- data/lib/lev/errors.rb +5 -5
- data/lib/lev/memory_store.rb +4 -0
- data/lib/lev/no_background_job.rb +26 -0
- data/lib/lev/routine.rb +11 -14
- data/lib/lev/version.rb +1 -1
- data/spec/active_job_routines_spec.rb +12 -0
- data/spec/lev/status_spec.rb +44 -0
- data/spec/statused_routines_spec.rb +69 -58
- metadata +6 -4
- data/lib/lev/black_hole_status.rb +0 -26
- data/lib/lev/status.rb +0 -125
checksums.yaml
CHANGED
@@ -1,15 +1,15 @@
|
|
1
1
|
---
|
2
2
|
!binary "U0hBMQ==":
|
3
3
|
metadata.gz: !binary |-
|
4
|
-
|
4
|
+
MGNhMzZiZWRhNDllMzMwYjBhMmFkZTQ5YzJiYWNkNjYxMzQ4NmMzMg==
|
5
5
|
data.tar.gz: !binary |-
|
6
|
-
|
6
|
+
NTdmM2U2NWY0YjlkNDM3N2ViZWU5NzQ3NmYyYjkyZDVkZDUwMzMyNw==
|
7
7
|
SHA512:
|
8
8
|
metadata.gz: !binary |-
|
9
|
-
|
10
|
-
|
11
|
-
|
9
|
+
NDMyM2UzNzY2MDQxODlkNmNlMWJhOTRjYjJkZmI2OWE3ZWM1ZDAyZDU1OWZk
|
10
|
+
ZmRhOWVjZWI5MGNhYWJmNWViMzJlNzlmMTQ0ZjVlMDNjODA1YzJkMDYwMzNj
|
11
|
+
OTc4NGVmYzI2ZDUzOWUwNTVmM2E5NzMyOGQ0MjQ5MWMzMTJhNjM=
|
12
12
|
data.tar.gz: !binary |-
|
13
|
-
|
14
|
-
|
15
|
-
|
13
|
+
M2I4ODI3Mjg0MWJiYzYyZGM0ZDA0OTVmMmJhOTZmMjk0NGIxMDA5ZjUxNTlm
|
14
|
+
NmI2ZmZhMTFmMWUyMDU0NzUwMjFlOGRkYjVhNWE3YjBhNTBkYmM0MmIxYTI0
|
15
|
+
ZDAzYzMyNWYyOTExNjA1NTJjMTJhM2RlZjY3YzNhNzU4NzdkZGE=
|
data/README.md
CHANGED
@@ -435,23 +435,23 @@ end
|
|
435
435
|
|
436
436
|
Routines run as ActiveJobs can also publish their status somewhere it can be listened to (e.g. to Redis).
|
437
437
|
|
438
|
-
Routines have a `
|
438
|
+
Routines have a `job` object and can call the following methods:
|
439
439
|
|
440
440
|
* `set_progress(at, out_of = nil)` sets the current progress; can either pass a float between 0.0 and 1.0 or
|
441
441
|
a counter towards a total, e.g. `set_progress(67,212)`.
|
442
|
-
* `queued!` Sets the status to 'queued'
|
443
|
-
* `working!` Sets the status to 'working'
|
444
|
-
* `completed!` Sets the status to 'completed'
|
445
|
-
* `failed!` Sets the status to 'failed'
|
446
|
-
* `killed!` Sets the status to 'killed'
|
447
|
-
* `save(hash)` Takes a hash of key value pairs and writes those keys and values to the status; there are several reserved keys which cannot be used (and which will blow up if you try to use them)
|
448
|
-
* `add_error(is_fatal, error)` takes a boolean and a Lev `Error` object and adds its data to an array of `errors` in the status hash.
|
442
|
+
* `queued!` Sets the job status to 'queued'
|
443
|
+
* `working!` Sets the job status to 'working'
|
444
|
+
* `completed!` Sets the job status to 'completed'
|
445
|
+
* `failed!` Sets the job status to 'failed'
|
446
|
+
* `killed!` Sets the job status to 'killed'
|
447
|
+
* `save(hash)` Takes a hash of key value pairs and writes those keys and values to the job status; there are several reserved keys which cannot be used (and which will blow up if you try to use them)
|
448
|
+
* `add_error(is_fatal, error)` takes a boolean and a Lev `Error` object and adds its data to an array of `errors` in the job status hash.
|
449
449
|
|
450
|
-
All routines have such a
|
450
|
+
All routines have such a job object. For plain vanilla routines not run as an active job, the job calls are no-ops. When a routine is invoked with `perform_later`, the job object actually records the jobs to a store of your choice. The store is configured in the Lev configuration block, e.g.:
|
451
451
|
|
452
452
|
```ruby
|
453
453
|
Lev.configure do |config|
|
454
|
-
config.
|
454
|
+
config.job_store = whatever
|
455
455
|
end
|
456
456
|
```
|
457
457
|
|
@@ -462,18 +462,18 @@ The store needs to respond to the following methods:
|
|
462
462
|
|
463
463
|
The default store is essentially a hash (implemented in `Lev::MemoryStore`). Any `ActiveSupport::Cache::Store` will work.
|
464
464
|
|
465
|
-
A routine's
|
465
|
+
A routine's job can be retrieved with `Lev::BackgroundJob.get(uuid_here)`. This just returns a simple hash. Notable keys are
|
466
466
|
|
467
467
|
* `'progress'` which returns the progress as a number between 0.0 and 1.0
|
468
468
|
* `'status'` which returns one of the status strings shown above
|
469
469
|
* `'errors'` which (if present) is an error of error hashes
|
470
|
-
* `'
|
470
|
+
* `'id'` the UUID of the routine / job
|
471
471
|
|
472
472
|
Other routine-specific keys (set with a `save` call) are also present.
|
473
473
|
|
474
474
|
**Notes:**
|
475
475
|
|
476
|
-
1. Don't try to write a
|
476
|
+
1. Don't try to write a job store that uses the ActiveRecord database, as the database changes would only be seen when the routine completes and its transaction is committed.
|
477
477
|
2. Job killing hasn't been implemented yet, but shouldn't be too bad. For routines run in a transaction (which are frankly the only ones you'd want to kill), we can likely kill them by raising an exception or similar to cause a rollback (will need to have good tests to prove that).
|
478
478
|
|
479
479
|
## Handlers
|
data/lib/lev.rb
CHANGED
@@ -26,8 +26,8 @@ require "lev/transaction_isolation"
|
|
26
26
|
|
27
27
|
require 'lev/active_job'
|
28
28
|
require 'lev/memory_store'
|
29
|
-
require 'lev/
|
30
|
-
require 'lev/
|
29
|
+
require 'lev/background_job'
|
30
|
+
require 'lev/no_background_job'
|
31
31
|
|
32
32
|
module Lev
|
33
33
|
class << self
|
@@ -59,16 +59,16 @@ module Lev
|
|
59
59
|
attr_accessor :security_transgression_error
|
60
60
|
attr_accessor :illegal_argument_error
|
61
61
|
attr_accessor :raise_fatal_errors
|
62
|
-
attr_accessor :
|
63
|
-
attr_accessor :
|
62
|
+
attr_accessor :job_store
|
63
|
+
attr_accessor :job_store_namespace
|
64
64
|
|
65
65
|
def initialize
|
66
66
|
@form_error_class = 'error'
|
67
67
|
@security_transgression_error = Lev::SecurityTransgression
|
68
68
|
@illegal_argument_error = Lev::IllegalArgument
|
69
69
|
@raise_fatal_errors = false
|
70
|
-
@
|
71
|
-
@
|
70
|
+
@job_store = Lev::MemoryStore.new
|
71
|
+
@job_store_namespace = "lev_job"
|
72
72
|
super
|
73
73
|
end
|
74
74
|
end
|
data/lib/lev/active_job.rb
CHANGED
@@ -6,25 +6,25 @@ if defined?(::ActiveJob)
|
|
6
6
|
queue_as routine_class.active_job_queue
|
7
7
|
args.push(routine_class.to_s)
|
8
8
|
|
9
|
-
# To enable tracking of this job's status, create a new
|
9
|
+
# To enable tracking of this job's status, create a new BackgroundJob object
|
10
10
|
# and push it on to the arguments so that in `perform` it can be peeled
|
11
|
-
# off and handed to the routine instance. The
|
11
|
+
# off and handed to the routine instance. The BackgroundJob UUID is returned
|
12
12
|
# so that callers can track the status.
|
13
|
-
|
14
|
-
|
15
|
-
args.push(
|
13
|
+
job = Lev::BackgroundJob.new
|
14
|
+
job.queued!
|
15
|
+
args.push(job.id)
|
16
16
|
|
17
17
|
super(*args, &block)
|
18
18
|
|
19
|
-
|
19
|
+
job.id
|
20
20
|
end
|
21
21
|
|
22
22
|
def perform(*args, &block)
|
23
23
|
# Pop arguments added by perform_later
|
24
|
-
|
24
|
+
id = args.pop
|
25
25
|
routine_class = Kernel.const_get(args.pop)
|
26
26
|
|
27
|
-
routine_instance = routine_class.new(Lev::
|
27
|
+
routine_instance = routine_class.new(Lev::BackgroundJob.new(id: id))
|
28
28
|
routine_instance.call(*args, &block)
|
29
29
|
end
|
30
30
|
end
|
@@ -0,0 +1,173 @@
|
|
1
|
+
require 'json'
|
2
|
+
|
3
|
+
module Lev
|
4
|
+
class BackgroundJob
|
5
|
+
attr_reader :id, :status, :progress, :errors
|
6
|
+
|
7
|
+
STATE_QUEUED = 'queued'
|
8
|
+
STATE_WORKING = 'working'
|
9
|
+
STATE_COMPLETED = 'completed'
|
10
|
+
STATE_FAILED = 'failed'
|
11
|
+
STATE_KILLED = 'killed'
|
12
|
+
STATE_UNKNOWN = 'unknown'
|
13
|
+
|
14
|
+
STATES = [
|
15
|
+
STATE_QUEUED,
|
16
|
+
STATE_WORKING,
|
17
|
+
STATE_COMPLETED,
|
18
|
+
STATE_FAILED,
|
19
|
+
STATE_KILLED,
|
20
|
+
STATE_UNKNOWN
|
21
|
+
].freeze
|
22
|
+
|
23
|
+
def initialize(attrs = {})
|
24
|
+
@id = attrs[:id] || attrs['id'] || SecureRandom.uuid
|
25
|
+
@status = attrs[:status] || attrs['status'] || STATE_UNKNOWN
|
26
|
+
@progress = attrs[:progress] || attrs['progress'] || set_progress(0)
|
27
|
+
@errors = attrs[:errors] || attrs['errors'] || []
|
28
|
+
|
29
|
+
set({ id: id,
|
30
|
+
status: status,
|
31
|
+
progress: progress,
|
32
|
+
errors: errors })
|
33
|
+
end
|
34
|
+
|
35
|
+
def self.find(id)
|
36
|
+
attrs = { id: id }
|
37
|
+
|
38
|
+
if job = store.fetch(job_key(id))
|
39
|
+
attrs.merge!(JSON.parse(job))
|
40
|
+
else
|
41
|
+
attrs.merge!(status: STATE_UNKNOWN)
|
42
|
+
end
|
43
|
+
|
44
|
+
new(attrs)
|
45
|
+
end
|
46
|
+
|
47
|
+
def self.all
|
48
|
+
job_ids.map { |id| find(id) }
|
49
|
+
end
|
50
|
+
|
51
|
+
def set_progress(at, out_of = nil)
|
52
|
+
progress = compute_fractional_progress(at, out_of)
|
53
|
+
|
54
|
+
data_to_set = { progress: progress }
|
55
|
+
data_to_set[:status] = STATE_COMPLETED if 1.0 == progress
|
56
|
+
|
57
|
+
set(data_to_set)
|
58
|
+
|
59
|
+
progress
|
60
|
+
end
|
61
|
+
|
62
|
+
STATES.each do |state|
|
63
|
+
define_method("#{state}!") do
|
64
|
+
set(status: state)
|
65
|
+
end
|
66
|
+
end
|
67
|
+
|
68
|
+
def add_error(error, options = { })
|
69
|
+
options = { is_fatal: false }.merge(options)
|
70
|
+
@errors << { is_fatal: options[:is_fatal],
|
71
|
+
code: error.code,
|
72
|
+
message: error.message }
|
73
|
+
set(errors: @errors)
|
74
|
+
end
|
75
|
+
|
76
|
+
# Rails compatibility
|
77
|
+
# returns a Hash of all key-value pairs that have been #set()
|
78
|
+
def as_json(options = {})
|
79
|
+
stored
|
80
|
+
end
|
81
|
+
|
82
|
+
def save(incoming_hash)
|
83
|
+
if reserved = incoming_hash.select { |k, _| RESERVED_KEYS.include?(k) }.first
|
84
|
+
raise IllegalArgument, "Cannot set reserved key: #{reserved[0]}"
|
85
|
+
else
|
86
|
+
set(incoming_hash)
|
87
|
+
end
|
88
|
+
end
|
89
|
+
|
90
|
+
def method_missing(method_name, *args)
|
91
|
+
instance_variable_get("@#{method_name}") || super
|
92
|
+
end
|
93
|
+
|
94
|
+
def respond_to?(method_name)
|
95
|
+
if method_name.match /\?$/
|
96
|
+
super
|
97
|
+
else
|
98
|
+
instance_variable_get("@#{method_name}").present? || super
|
99
|
+
end
|
100
|
+
end
|
101
|
+
|
102
|
+
protected
|
103
|
+
RESERVED_KEYS = [:id, :status, :progress, :errors]
|
104
|
+
|
105
|
+
def set(incoming_hash)
|
106
|
+
incoming_hash = stored.merge(incoming_hash)
|
107
|
+
incoming_hash.each { |k, v| instance_variable_set("@#{k}", v) }
|
108
|
+
self.class.store.write(job_key, incoming_hash.to_json)
|
109
|
+
track_job_id
|
110
|
+
end
|
111
|
+
|
112
|
+
def self.store
|
113
|
+
Lev.configuration.job_store
|
114
|
+
end
|
115
|
+
|
116
|
+
def self.job_ids
|
117
|
+
store.fetch(job_key('lev_job_ids')) || []
|
118
|
+
end
|
119
|
+
|
120
|
+
def stored
|
121
|
+
if found = self.class.store.fetch(job_key)
|
122
|
+
JSON.parse(found)
|
123
|
+
else
|
124
|
+
{}
|
125
|
+
end
|
126
|
+
end
|
127
|
+
|
128
|
+
def track_job_id
|
129
|
+
ids = self.class.job_ids
|
130
|
+
ids << @id
|
131
|
+
self.class.store.write(self.class.job_key('lev_job_ids'), ids.uniq)
|
132
|
+
end
|
133
|
+
|
134
|
+
def job_key
|
135
|
+
self.class.job_key(@id)
|
136
|
+
end
|
137
|
+
|
138
|
+
def self.job_key(id)
|
139
|
+
"#{Lev.configuration.job_store_namespace}:#{id}"
|
140
|
+
end
|
141
|
+
|
142
|
+
def has_reserved_keys?(hash)
|
143
|
+
(hash.keys.collect(&:to_sym) & RESERVED_KEYS).any?
|
144
|
+
end
|
145
|
+
|
146
|
+
def push(key, new_item)
|
147
|
+
new_value = (send(key) || []).push(new_item)
|
148
|
+
set(key => new_value)
|
149
|
+
end
|
150
|
+
|
151
|
+
STATES.each do |state|
|
152
|
+
define_method("#{state}?") do
|
153
|
+
status == state
|
154
|
+
end
|
155
|
+
end
|
156
|
+
|
157
|
+
def compute_fractional_progress(at, out_of)
|
158
|
+
if at.nil?
|
159
|
+
raise IllegalArgument, "Must specify at least `at` argument to `progress` call"
|
160
|
+
elsif at < 0
|
161
|
+
raise IllegalArgument, "progress cannot be negative (at=#{at})"
|
162
|
+
elsif out_of && out_of < at
|
163
|
+
raise IllegalArgument, "`out_of` must be greater than `at` in `progress` calls"
|
164
|
+
elsif out_of.nil? && (at < 0 || at > 1)
|
165
|
+
raise IllegalArgument, "If `out_of` not specified, `at` must be in the range [0.0, 1.0]"
|
166
|
+
end
|
167
|
+
|
168
|
+
at.to_f / (out_of || 1).to_f
|
169
|
+
end
|
170
|
+
|
171
|
+
end
|
172
|
+
end
|
173
|
+
|
data/lib/lev/errors.rb
CHANGED
@@ -4,8 +4,8 @@ module Lev
|
|
4
4
|
#
|
5
5
|
class Errors < Array
|
6
6
|
|
7
|
-
def initialize(
|
8
|
-
@
|
7
|
+
def initialize(routine_job = nil, raise_fatal_errors = false)
|
8
|
+
@routine_job = routine_job || NoBackgroundJob.new
|
9
9
|
@raise_fatal_errors = raise_fatal_errors
|
10
10
|
end
|
11
11
|
|
@@ -16,10 +16,10 @@ module Lev
|
|
16
16
|
return if ignored_error_procs.any?{|proc| proc.call(error)}
|
17
17
|
self.push(error)
|
18
18
|
|
19
|
-
|
19
|
+
routine_job.add_error(error, is_fatal: fail)
|
20
20
|
|
21
21
|
if fail
|
22
|
-
|
22
|
+
routine_job.failed!
|
23
23
|
|
24
24
|
if raise_fatal_errors
|
25
25
|
raise StandardError, args.to_a.map { |i| i.join(' ') }.join(' - ')
|
@@ -54,7 +54,7 @@ module Lev
|
|
54
54
|
|
55
55
|
protected
|
56
56
|
|
57
|
-
attr_reader :
|
57
|
+
attr_reader :routine_job
|
58
58
|
attr_reader :raise_fatal_errors
|
59
59
|
|
60
60
|
def ignored_error_procs
|
data/lib/lev/memory_store.rb
CHANGED
@@ -0,0 +1,26 @@
|
|
1
|
+
module Lev
|
2
|
+
class NoBackgroundJob
|
3
|
+
|
4
|
+
# Provide null object pattern methods for background jobs; routines should
|
5
|
+
# not be checking their own status (they should know it), and outside callers
|
6
|
+
# should not be checking status unless the background job is a real one.
|
7
|
+
|
8
|
+
def set_progress(*); end
|
9
|
+
def save(*); end
|
10
|
+
def add_error(*); end
|
11
|
+
|
12
|
+
Lev::BackgroundJob::STATES.each do |state|
|
13
|
+
define_method("#{state}!") do; end
|
14
|
+
end
|
15
|
+
|
16
|
+
def self.method_missing(method_sym, *args, &block)
|
17
|
+
if Lev::BackgroundJob.new.respond_to?(method_sym)
|
18
|
+
raise NameError,
|
19
|
+
"'#{method_sym}' is Lev::BackgroundJob query method, and those cannot be called on NoBackgroundJob"
|
20
|
+
else
|
21
|
+
super
|
22
|
+
end
|
23
|
+
end
|
24
|
+
|
25
|
+
end
|
26
|
+
end
|
data/lib/lev/routine.rb
CHANGED
@@ -191,7 +191,7 @@ module Lev
|
|
191
191
|
end
|
192
192
|
end
|
193
193
|
|
194
|
-
attr_reader :
|
194
|
+
attr_reader :id
|
195
195
|
|
196
196
|
def self.included(base)
|
197
197
|
base.extend(ClassMethods)
|
@@ -267,7 +267,7 @@ module Lev
|
|
267
267
|
def call(*args, &block)
|
268
268
|
@after_transaction_blocks = []
|
269
269
|
|
270
|
-
|
270
|
+
job.working!
|
271
271
|
|
272
272
|
in_transaction do
|
273
273
|
catch :fatal_errors_encountered do
|
@@ -285,7 +285,7 @@ module Lev
|
|
285
285
|
block.call
|
286
286
|
end
|
287
287
|
|
288
|
-
|
288
|
+
job.completed! if !errors?
|
289
289
|
|
290
290
|
result
|
291
291
|
end
|
@@ -418,26 +418,23 @@ module Lev
|
|
418
418
|
|
419
419
|
# Note that the parent may neglect to call super, leading to this method never being called.
|
420
420
|
# Do not perform any initialization here that cannot be safely skipped
|
421
|
-
def initialize(
|
422
|
-
# If someone cares about the
|
423
|
-
#
|
424
|
-
@
|
421
|
+
def initialize(job = nil)
|
422
|
+
# If someone cares about the job, they'll pass it in; otherwise all
|
423
|
+
# job updates go into the bit bucket.
|
424
|
+
@job = job
|
425
425
|
end
|
426
426
|
|
427
427
|
protected
|
428
428
|
|
429
429
|
attr_writer :runner
|
430
430
|
|
431
|
-
def
|
432
|
-
@
|
431
|
+
def job
|
432
|
+
@job ||= Lev::NoBackgroundJob.new
|
433
433
|
end
|
434
434
|
|
435
435
|
def result
|
436
|
-
@result ||= Result.new(
|
437
|
-
|
438
|
-
Errors.new(status,
|
439
|
-
topmost_runner.class.raise_fatal_errors?)
|
440
|
-
)
|
436
|
+
@result ||= Result.new(Outputs.new,
|
437
|
+
Errors.new(job, topmost_runner.class.raise_fatal_errors?))
|
441
438
|
end
|
442
439
|
|
443
440
|
def outputs
|
data/lib/lev/version.rb
CHANGED
@@ -16,9 +16,21 @@ RSpec.describe 'ActiveJob routines' do
|
|
16
16
|
end
|
17
17
|
|
18
18
|
it 'can have the default queue overridden' do
|
19
|
+
ActiveJob::Base.queue_adapter.enqueued_jobs.clear
|
20
|
+
|
19
21
|
LaterRoutine.perform_later
|
22
|
+
|
20
23
|
queue_name = ActiveJob::Base.queue_adapter.enqueued_jobs.first[:queue]
|
24
|
+
|
21
25
|
expect(queue_name).to eq('something_else')
|
22
26
|
end
|
27
|
+
|
28
|
+
it 'stores all the UUIDs of queued jobs' do
|
29
|
+
Lev.configuration.job_store.clear
|
30
|
+
|
31
|
+
job_id1 = LaterRoutine.perform_later
|
32
|
+
|
33
|
+
expect(Lev::BackgroundJob.send(:job_ids)).to eq([job_id1])
|
34
|
+
end
|
23
35
|
end
|
24
36
|
end
|
@@ -0,0 +1,44 @@
|
|
1
|
+
require 'spec_helper'
|
2
|
+
|
3
|
+
RSpec.describe Lev::BackgroundJob do
|
4
|
+
class DelayedRoutine
|
5
|
+
lev_routine
|
6
|
+
protected
|
7
|
+
def exec; end
|
8
|
+
end
|
9
|
+
|
10
|
+
subject(:job) { described_class.all.last }
|
11
|
+
|
12
|
+
before do
|
13
|
+
Lev.configuration.job_store.clear
|
14
|
+
allow(SecureRandom).to receive(:uuid) { '123abc' }
|
15
|
+
DelayedRoutine.perform_later
|
16
|
+
end
|
17
|
+
|
18
|
+
it 'behaves as a nice ruby object' do
|
19
|
+
expect(job.id).to eq('123abc')
|
20
|
+
expect(job.status).to eq(Lev::BackgroundJob::STATE_QUEUED)
|
21
|
+
expect(job.progress).to eq(0.0)
|
22
|
+
end
|
23
|
+
|
24
|
+
it 'is unknown when not found' do
|
25
|
+
foo = described_class.find('noooooo')
|
26
|
+
expect(foo.status).to eq(Lev::BackgroundJob::STATE_UNKNOWN)
|
27
|
+
end
|
28
|
+
|
29
|
+
it 'uses as_json' do
|
30
|
+
json = job.as_json
|
31
|
+
|
32
|
+
expect(json).to eq({
|
33
|
+
'id' => '123abc',
|
34
|
+
'status' => Lev::BackgroundJob::STATE_QUEUED,
|
35
|
+
'progress' => 0.0,
|
36
|
+
'errors' => []
|
37
|
+
})
|
38
|
+
|
39
|
+
job.save(foo: :bar)
|
40
|
+
json = job.as_json
|
41
|
+
|
42
|
+
expect(json['foo']).to eq('bar')
|
43
|
+
end
|
44
|
+
end
|
@@ -5,135 +5,146 @@ class StatusedRoutine
|
|
5
5
|
|
6
6
|
protected
|
7
7
|
def exec
|
8
|
-
|
8
|
+
job.set_progress(9, 10)
|
9
9
|
end
|
10
10
|
end
|
11
11
|
|
12
12
|
RSpec.describe 'Statused Routines' do
|
13
|
-
subject(:
|
13
|
+
subject(:job) { Lev::BackgroundJob.new }
|
14
14
|
|
15
15
|
context 'in a routine' do
|
16
|
-
it 'queues the
|
17
|
-
|
18
|
-
|
16
|
+
it 'queues the job object on queue' do
|
17
|
+
id = StatusedRoutine.perform_later
|
18
|
+
job = Lev::BackgroundJob.find(id)
|
19
19
|
|
20
|
-
expect(status
|
20
|
+
expect(job.status).to eq(Lev::BackgroundJob::STATE_QUEUED)
|
21
21
|
end
|
22
22
|
|
23
23
|
context 'inline activejob mode' do
|
24
24
|
before { ::ActiveJob::Base.queue_adapter = :inline }
|
25
25
|
after { ::ActiveJob::Base.queue_adapter = :test }
|
26
26
|
|
27
|
-
it 'sets
|
28
|
-
expect_any_instance_of(Lev::
|
27
|
+
it 'sets job to working when called' do
|
28
|
+
expect_any_instance_of(Lev::BackgroundJob).to receive(:working!)
|
29
29
|
StatusedRoutine.perform_later
|
30
30
|
end
|
31
31
|
|
32
|
-
it 'completes the
|
33
|
-
|
34
|
-
|
35
|
-
expect(status
|
36
|
-
expect(
|
32
|
+
it 'completes the job object on completion, returning other data' do
|
33
|
+
id = StatusedRoutine.perform_later
|
34
|
+
job = Lev::BackgroundJob.find(id)
|
35
|
+
expect(job.status).to eq(Lev::BackgroundJob::STATE_COMPLETED)
|
36
|
+
expect(job.progress).to eq(0.9)
|
37
37
|
end
|
38
38
|
end
|
39
39
|
end
|
40
40
|
|
41
|
+
describe '#save' do
|
42
|
+
it 'saves the hash given and writes them to the job' do
|
43
|
+
job.save(something: 'else')
|
44
|
+
expect(job.something).to eq('else')
|
45
|
+
end
|
46
|
+
end
|
47
|
+
|
48
|
+
describe '#add_error' do
|
49
|
+
it 'adds the error object data to the job object' do
|
50
|
+
errors = Lev::Error.new(code: 'bad', message: 'awful')
|
51
|
+
job.add_error(errors)
|
52
|
+
expect(job.errors).to eq([{ is_fatal: false,
|
53
|
+
code: 'bad',
|
54
|
+
message: 'awful' }])
|
55
|
+
end
|
56
|
+
end
|
57
|
+
|
41
58
|
describe '#save' do
|
42
59
|
it 'prevents the use of reserved keys' do
|
43
60
|
expect {
|
44
|
-
|
61
|
+
job.save(progress: 'blocked')
|
45
62
|
}.to raise_error(Lev::IllegalArgument)
|
46
63
|
|
47
64
|
expect {
|
48
|
-
|
65
|
+
job.save(id: 'blocked')
|
49
66
|
}.to raise_error(Lev::IllegalArgument)
|
50
67
|
|
51
68
|
expect {
|
52
|
-
|
69
|
+
job.save(status: 'blocked')
|
53
70
|
}.to raise_error(Lev::IllegalArgument)
|
54
71
|
|
55
72
|
expect {
|
56
|
-
|
73
|
+
job.save(errors: 'blocked')
|
57
74
|
}.to raise_error(Lev::IllegalArgument)
|
58
75
|
end
|
59
76
|
|
60
|
-
it 'saves the hash given and writes them to the
|
61
|
-
|
62
|
-
expect(
|
63
|
-
|
64
|
-
end
|
65
|
-
|
66
|
-
describe '#add_error' do
|
67
|
-
it 'adds the error object data to the status object' do
|
68
|
-
errors = Lev::Error.new(code: 'bad', message: 'awful')
|
69
|
-
status.add_error(errors)
|
70
|
-
expect(status.get('errors')).to eq([{ 'is_fatal' => false,
|
71
|
-
'code' => 'bad',
|
72
|
-
'message' => 'awful' }])
|
77
|
+
it 'saves the hash given and writes them to the job' do
|
78
|
+
job.save(something: 'else')
|
79
|
+
expect(job).to respond_to(:something)
|
80
|
+
expect(job.something).to eq('else')
|
73
81
|
end
|
74
82
|
end
|
75
83
|
|
76
|
-
describe 'dynamic
|
84
|
+
describe 'dynamic job setters/getters' do
|
77
85
|
it 'is queued' do
|
78
|
-
expect(
|
79
|
-
|
80
|
-
expect(
|
86
|
+
expect(job).not_to be_queued
|
87
|
+
job.queued!
|
88
|
+
expect(job).to be_queued
|
81
89
|
end
|
82
90
|
|
83
91
|
it 'is working' do
|
84
|
-
expect(
|
85
|
-
|
86
|
-
expect(
|
92
|
+
expect(job).not_to be_working
|
93
|
+
job.working!
|
94
|
+
expect(job).to be_working
|
87
95
|
end
|
88
96
|
|
89
97
|
it 'is completed' do
|
90
|
-
expect(
|
91
|
-
|
92
|
-
expect(
|
98
|
+
expect(job).not_to be_completed
|
99
|
+
job.completed!
|
100
|
+
expect(job).to be_completed
|
93
101
|
end
|
94
102
|
|
95
103
|
it 'is failed' do
|
96
|
-
expect(
|
97
|
-
|
98
|
-
expect(
|
104
|
+
expect(job).not_to be_failed
|
105
|
+
job.failed!
|
106
|
+
expect(job).to be_failed
|
99
107
|
end
|
100
108
|
|
101
109
|
it 'is killed' do
|
102
|
-
expect(
|
103
|
-
|
104
|
-
expect(
|
110
|
+
expect(job).not_to be_killed
|
111
|
+
job.killed!
|
112
|
+
expect(job).to be_killed
|
113
|
+
end
|
114
|
+
|
115
|
+
it 'is unknown' do
|
116
|
+
expect(job).to be_unknown
|
105
117
|
end
|
106
118
|
end
|
107
119
|
|
108
120
|
describe '#set_progress' do
|
109
|
-
it 'sets the progress key on the
|
110
|
-
|
111
|
-
progress
|
112
|
-
expect(progress).to eq(0.8)
|
121
|
+
it 'sets the progress key on the job object' do
|
122
|
+
job.set_progress(8, 10)
|
123
|
+
expect(job.progress).to eq(0.8)
|
113
124
|
end
|
114
125
|
|
115
126
|
context 'when `out_of` is supplied' do
|
116
127
|
it 'requires a positive `at` float or integer' do
|
117
128
|
expect {
|
118
|
-
|
129
|
+
job.set_progress(nil, 1)
|
119
130
|
}.to raise_error(Lev::IllegalArgument)
|
120
131
|
|
121
132
|
expect {
|
122
|
-
|
133
|
+
job.set_progress(-1, 1)
|
123
134
|
}.to raise_error(Lev::IllegalArgument)
|
124
135
|
|
125
136
|
expect {
|
126
|
-
|
137
|
+
job.set_progress(2, 5)
|
127
138
|
}.not_to raise_error
|
128
139
|
end
|
129
140
|
|
130
141
|
it 'requires `out_of` to be greater than `at`' do
|
131
142
|
expect {
|
132
|
-
|
143
|
+
job.set_progress(15, 8)
|
133
144
|
}.to raise_error(Lev::IllegalArgument)
|
134
145
|
|
135
146
|
expect {
|
136
|
-
|
147
|
+
job.set_progress(5, 10)
|
137
148
|
}.not_to raise_error
|
138
149
|
end
|
139
150
|
end
|
@@ -141,15 +152,15 @@ RSpec.describe 'Statused Routines' do
|
|
141
152
|
context 'without out_of specified' do
|
142
153
|
it 'requires `at` to be a float between 0.0 and 1.0' do
|
143
154
|
expect {
|
144
|
-
|
155
|
+
job.set_progress(1.1)
|
145
156
|
}.to raise_error(Lev::IllegalArgument)
|
146
157
|
|
147
158
|
expect {
|
148
|
-
|
159
|
+
job.set_progress(-1)
|
149
160
|
}.to raise_error(Lev::IllegalArgument)
|
150
161
|
|
151
162
|
expect {
|
152
|
-
|
163
|
+
job.set_progress(0.78)
|
153
164
|
}.not_to raise_error
|
154
165
|
end
|
155
166
|
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: lev
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version:
|
4
|
+
version: 5.0.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- JP Slavinsky
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2015-07-
|
11
|
+
date: 2015-07-30 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activemodel
|
@@ -218,8 +218,8 @@ files:
|
|
218
218
|
- Rakefile
|
219
219
|
- lib/lev.rb
|
220
220
|
- lib/lev/active_job.rb
|
221
|
+
- lib/lev/background_job.rb
|
221
222
|
- lib/lev/better_active_model_errors.rb
|
222
|
-
- lib/lev/black_hole_status.rb
|
223
223
|
- lib/lev/delegate_to_routine.rb
|
224
224
|
- lib/lev/error.rb
|
225
225
|
- lib/lev/error_transferer.rb
|
@@ -231,10 +231,10 @@ files:
|
|
231
231
|
- lib/lev/handler.rb
|
232
232
|
- lib/lev/handler_helper.rb
|
233
233
|
- lib/lev/memory_store.rb
|
234
|
+
- lib/lev/no_background_job.rb
|
234
235
|
- lib/lev/object.rb
|
235
236
|
- lib/lev/outputs.rb
|
236
237
|
- lib/lev/routine.rb
|
237
|
-
- lib/lev/status.rb
|
238
238
|
- lib/lev/term_mapper.rb
|
239
239
|
- lib/lev/transaction_isolation.rb
|
240
240
|
- lib/lev/utilities.rb
|
@@ -244,6 +244,7 @@ files:
|
|
244
244
|
- spec/deep_merge_spec.rb
|
245
245
|
- spec/delegates_to_spec.rb
|
246
246
|
- spec/errors_spec.rb
|
247
|
+
- spec/lev/status_spec.rb
|
247
248
|
- spec/outputs_spec.rb
|
248
249
|
- spec/paramify_handler_spec.rb
|
249
250
|
- spec/routine_spec.rb
|
@@ -289,6 +290,7 @@ test_files:
|
|
289
290
|
- spec/deep_merge_spec.rb
|
290
291
|
- spec/delegates_to_spec.rb
|
291
292
|
- spec/errors_spec.rb
|
293
|
+
- spec/lev/status_spec.rb
|
292
294
|
- spec/outputs_spec.rb
|
293
295
|
- spec/paramify_handler_spec.rb
|
294
296
|
- spec/routine_spec.rb
|
@@ -1,26 +0,0 @@
|
|
1
|
-
module Lev
|
2
|
-
class BlackHoleStatus
|
3
|
-
|
4
|
-
# Provide null object pattern methods for status setters; routines should
|
5
|
-
# not be checking their own status (they should know it), and outside callers
|
6
|
-
# should not be checking status unless the status object is a real one.
|
7
|
-
|
8
|
-
def set_progress(*); end
|
9
|
-
def save(*); end
|
10
|
-
def add_error(*); end
|
11
|
-
|
12
|
-
Lev::Status::STATES.each do |state|
|
13
|
-
define_method("#{state}!") do; end
|
14
|
-
end
|
15
|
-
|
16
|
-
def self.method_missing(method_sym, *args, &block)
|
17
|
-
if Lev::Status.new.respond_to?(method_sym)
|
18
|
-
raise NameError,
|
19
|
-
"'#{method_sym}' is Lev::Status query method, and those cannot be called on BlackHoleStatus"
|
20
|
-
else
|
21
|
-
super
|
22
|
-
end
|
23
|
-
end
|
24
|
-
|
25
|
-
end
|
26
|
-
end
|
data/lib/lev/status.rb
DELETED
@@ -1,125 +0,0 @@
|
|
1
|
-
require 'json'
|
2
|
-
|
3
|
-
module Lev
|
4
|
-
class Status
|
5
|
-
STATE_QUEUED = 'queued'
|
6
|
-
STATE_WORKING = 'working'
|
7
|
-
STATE_COMPLETED = 'completed'
|
8
|
-
STATE_FAILED = 'failed'
|
9
|
-
STATE_KILLED = 'killed'
|
10
|
-
|
11
|
-
STATES = [
|
12
|
-
STATE_QUEUED,
|
13
|
-
STATE_WORKING,
|
14
|
-
STATE_COMPLETED,
|
15
|
-
STATE_FAILED,
|
16
|
-
STATE_KILLED
|
17
|
-
].freeze
|
18
|
-
|
19
|
-
attr_reader :uuid
|
20
|
-
|
21
|
-
def initialize(uuid = nil)
|
22
|
-
@uuid = uuid || SecureRandom.uuid
|
23
|
-
save
|
24
|
-
end
|
25
|
-
|
26
|
-
def self.find(uuid)
|
27
|
-
if status = store.fetch(status_key(uuid))
|
28
|
-
JSON.parse(status)
|
29
|
-
else
|
30
|
-
nil
|
31
|
-
end
|
32
|
-
end
|
33
|
-
|
34
|
-
def set_progress(at, out_of = nil)
|
35
|
-
progress = compute_fractional_progress(at, out_of)
|
36
|
-
|
37
|
-
data_to_set = { progress: progress }
|
38
|
-
data_to_set[:state] = STATE_COMPLETED if 1.0 == progress
|
39
|
-
|
40
|
-
set(data_to_set)
|
41
|
-
end
|
42
|
-
|
43
|
-
STATES.each do |state|
|
44
|
-
define_method("#{state}!") do
|
45
|
-
set(state: state)
|
46
|
-
end
|
47
|
-
end
|
48
|
-
|
49
|
-
def save(hash = {})
|
50
|
-
if has_reserved_keys?(hash)
|
51
|
-
raise IllegalArgument,
|
52
|
-
"Caller cannot specify any reserved keys (#{RESERVED_KEYS})"
|
53
|
-
else
|
54
|
-
set(hash)
|
55
|
-
end
|
56
|
-
end
|
57
|
-
|
58
|
-
def add_error(error, options = { })
|
59
|
-
options = { is_fatal: false }.merge(options)
|
60
|
-
push('errors', { is_fatal: options[:is_fatal],
|
61
|
-
code: error.code,
|
62
|
-
message: error.message })
|
63
|
-
end
|
64
|
-
|
65
|
-
def get(key)
|
66
|
-
self.class.find(uuid)[key]
|
67
|
-
end
|
68
|
-
|
69
|
-
protected
|
70
|
-
RESERVED_KEYS = [:progress, :uuid, :state, :errors]
|
71
|
-
|
72
|
-
def self.store
|
73
|
-
# Nice to get the store from lev config each time so it isn't serialized
|
74
|
-
# when activejobs are sent off to places like redis
|
75
|
-
Lev.configuration.status_store
|
76
|
-
end
|
77
|
-
|
78
|
-
def set(incoming_hash)
|
79
|
-
if existing_settings = self.class.find(uuid)
|
80
|
-
incoming_hash = existing_settings.merge(incoming_hash)
|
81
|
-
end
|
82
|
-
|
83
|
-
self.class.store.write(status_key, incoming_hash.to_json)
|
84
|
-
end
|
85
|
-
|
86
|
-
def status_key
|
87
|
-
self.class.status_key(uuid)
|
88
|
-
end
|
89
|
-
|
90
|
-
def self.status_key(uuid)
|
91
|
-
"#{Lev.configuration.status_store_namespace}:#{uuid}"
|
92
|
-
end
|
93
|
-
|
94
|
-
def has_reserved_keys?(hash)
|
95
|
-
(hash.keys.collect(&:to_sym) & RESERVED_KEYS).any?
|
96
|
-
end
|
97
|
-
|
98
|
-
def push(key, new_item)
|
99
|
-
new_value = (get(key) || []).push(new_item)
|
100
|
-
set(key => new_value)
|
101
|
-
end
|
102
|
-
|
103
|
-
STATES.each do |state|
|
104
|
-
define_method("#{state}?") do
|
105
|
-
get('state') == state
|
106
|
-
end
|
107
|
-
end
|
108
|
-
|
109
|
-
def compute_fractional_progress(at, out_of)
|
110
|
-
if at.nil?
|
111
|
-
raise IllegalArgument, "Must specify at least `at` argument to `progress` call"
|
112
|
-
elsif at < 0
|
113
|
-
raise IllegalArgument, "progress cannot be negative (at=#{at})"
|
114
|
-
elsif out_of && out_of < at
|
115
|
-
raise IllegalArgument, "`out_of` must be greater than `at` in `progress` calls"
|
116
|
-
elsif out_of.nil? && (at < 0 || at > 1)
|
117
|
-
raise IllegalArgument, "If `out_of` not specified, `at` must be in the range [0.0, 1.0]"
|
118
|
-
end
|
119
|
-
|
120
|
-
at.to_f / (out_of || 1).to_f
|
121
|
-
end
|
122
|
-
|
123
|
-
end
|
124
|
-
end
|
125
|
-
|