job-iteration 1.1.14 → 1.2.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.github/workflows/ci.yml +1 -3
- data/.rubocop.yml +1 -1
- data/CHANGELOG.md +5 -0
- data/Gemfile +0 -1
- data/README.md +2 -2
- data/bin/test +32 -0
- data/dev.yml +38 -2
- data/job-iteration.gemspec +3 -2
- data/lib/job-iteration/version.rb +1 -1
- data/lib/job-iteration.rb +7 -9
- metadata +5 -4
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: eff37be02274e3c0db10de1cf3e49a10eea5b5b838b19800d143853aee94c08f
|
4
|
+
data.tar.gz: '085dc8b7eec8897393461e43fb4ece8acaddc6705165a2b6de0856cef54b1215'
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 22352326237ca16a23fc4177611187d4c6d426c07397caecd09165303726ce9ba00acaa882e35fb580b19bbe53f056ddde642298268a329dd02c88f61f82b5b7
|
7
|
+
data.tar.gz: 583406e5180f2721895673aa0410a390dce378c9c30e625151b5455fa8c1331867e94cc1cf2518b8b29cb0fbe908af517ee88a88429fd5f71f34a80617f5d8e6
|
data/.github/workflows/ci.yml
CHANGED
@@ -13,11 +13,9 @@ jobs:
|
|
13
13
|
- 6379:6379
|
14
14
|
strategy:
|
15
15
|
matrix:
|
16
|
-
ruby: [2.
|
16
|
+
ruby: [2.6, 2.7, 3.0]
|
17
17
|
gemfile: [rails_5_2, rails_6_0, rails_edge]
|
18
18
|
exclude:
|
19
|
-
- ruby: 2.5
|
20
|
-
gemfile: rails_edge
|
21
19
|
- ruby: 2.6
|
22
20
|
gemfile: rails_edge
|
23
21
|
- ruby: 3.0
|
data/.rubocop.yml
CHANGED
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,10 @@
|
|
1
1
|
### Master (unreleased)
|
2
2
|
|
3
|
+
## v1.2.0 (Sept 21, 2021)
|
4
|
+
- [107](https://github.com/Shopify/job-iteration/pull/107) - Remove broken links from README
|
5
|
+
- [108](https://github.com/Shopify/job-iteration/pull/108) - Drop support for ruby 2.5
|
6
|
+
- [110](https://github.com/Shopify/job-iteration/pull/110) - Update rubocop TargetRubyVersion
|
7
|
+
|
3
8
|
## v1.1.14 (May 28, 2021)
|
4
9
|
|
5
10
|
#### Bug fix
|
data/Gemfile
CHANGED
data/README.md
CHANGED
@@ -1,6 +1,6 @@
|
|
1
1
|
# Job Iteration API
|
2
2
|
|
3
|
-
[![
|
3
|
+
[![CI](https://github.com/Shopify/job-iteration/actions/workflows/ci.yml/badge.svg)](https://github.com/Shopify/job-iteration/actions/workflows/ci.yml)
|
4
4
|
|
5
5
|
Meet Iteration, an extension for [ActiveJob](https://github.com/rails/rails/tree/master/activejob) that makes your jobs interruptible and resumable, saving all progress that the job has made (aka checkpoint for jobs).
|
6
6
|
|
@@ -171,7 +171,7 @@ There a few configuration assumptions that are required for Iteration to work wi
|
|
171
171
|
|
172
172
|
**Why is it important that `each_iteration` takes less than 30 seconds?** When the job worker is scheduled for restart or shutdown, it gets a notice to finish remaining unit of work. To guarantee that no progress is lost we need to make sure that `each_iteration` completes within a reasonable amount of time.
|
173
173
|
|
174
|
-
**What do I do if each iteration takes a long time, because it's doing nested operations?** If your `each_iteration` is complex, we recommend enqueuing another job, which will run your nested business logic. We may expose primitives in the future to do this more effectively, but this is not terribly common today.
|
174
|
+
**What do I do if each iteration takes a long time, because it's doing nested operations?** If your `each_iteration` is complex, we recommend enqueuing another job, which will run your nested business logic. We may expose primitives in the future to do this more effectively, but this is not terribly common today.
|
175
175
|
|
176
176
|
**Why do I use have to use this ugly helper in `build_enumerator`? Why can't you automatically infer it?** This is how the first version of the API worked. We checked the type of object returned by `build_enumerable`, and whether it was ActiveRecord Relation or an Array, we used the matching adapter. This caused opaque type branching in Iteration internals and it didn’t allow developers to craft their own Enumerators and control the cursor value. We made a decision to _always_ return Enumerator instance from `build_enumerator`. Now we provide explicit helpers to convert ActiveRecord Relation or an Array to Enumerator, and for more complex iteration flows developers can build their own `Enumerator` objects.
|
177
177
|
|
data/bin/test
ADDED
@@ -0,0 +1,32 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
# frozen_string_literal: true
|
3
|
+
|
4
|
+
def main
|
5
|
+
begin
|
6
|
+
command = create_command
|
7
|
+
rescue ArgumentError => e
|
8
|
+
abort(e.message)
|
9
|
+
end
|
10
|
+
puts "Running #{command.join(" ")}"
|
11
|
+
system(*command)
|
12
|
+
end
|
13
|
+
|
14
|
+
def create_command
|
15
|
+
case ARGV.length
|
16
|
+
when 0
|
17
|
+
["bundle", "exec", "rake", "test"]
|
18
|
+
when 1
|
19
|
+
filename = ARGV[0]
|
20
|
+
["bundle", "exec", "rake", "test", "TEST=#{filename}"]
|
21
|
+
when 2
|
22
|
+
filename = ARGV[0]
|
23
|
+
test_name = ARGV[1]
|
24
|
+
test_name_with_underscores = test_name.tr(" ", "_")
|
25
|
+
test_name_pattern = "/#{Regexp.escape(test_name_with_underscores)}/"
|
26
|
+
["bundle", "exec", "rake", "test", "TEST=#{filename}", "TESTOPTS=\"--name=#{test_name_pattern} -v\""]
|
27
|
+
else
|
28
|
+
raise ArgumentError, "Too many arguments. Did you forget to put the test name in quotes?"
|
29
|
+
end
|
30
|
+
end
|
31
|
+
|
32
|
+
main
|
data/dev.yml
CHANGED
@@ -13,7 +13,43 @@ up:
|
|
13
13
|
- custom:
|
14
14
|
name: Create Job Iteration database
|
15
15
|
meet: mysql -uroot -h job-iteration.railgun -e "CREATE DATABASE job_iteration_test"
|
16
|
-
met?: mysql -uroot -h job-iteration.railgun job_iteration_test -e "SELECT 1"
|
16
|
+
met?: mysql -uroot -h job-iteration.railgun job_iteration_test -e "SELECT 1" &> /dev/null
|
17
17
|
|
18
18
|
commands:
|
19
|
-
test:
|
19
|
+
test:
|
20
|
+
run: bin/test "$@"
|
21
|
+
syntax:
|
22
|
+
optional: filename testnamepattern
|
23
|
+
aliases: [t]
|
24
|
+
desc: run tests
|
25
|
+
long_desc: |
|
26
|
+
{{bold:Default}}
|
27
|
+
=======
|
28
|
+
Run the entire test suite.
|
29
|
+
|
30
|
+
Examples:
|
31
|
+
{{command:dev test}}
|
32
|
+
{{command:dev t}}
|
33
|
+
|
34
|
+
{{bold:Run all tests in a file}}
|
35
|
+
========================
|
36
|
+
Include the file path.
|
37
|
+
|
38
|
+
Example:
|
39
|
+
{{command:dev test test/unit/iteration_test.rb}}
|
40
|
+
|
41
|
+
{{bold:Run a single test in a given file}}
|
42
|
+
========================
|
43
|
+
Include the file path and the name of the test you'd like to run.
|
44
|
+
|
45
|
+
Example:
|
46
|
+
{{command:dev test test/unit/iteration_test.rb test_that_it_has_a_version_number}}
|
47
|
+
|
48
|
+
{{bold:Run all tests in a given file whose name contains a string}}
|
49
|
+
========================
|
50
|
+
Include the file path and the string that the test names should contain.
|
51
|
+
|
52
|
+
Example:
|
53
|
+
{{command:dev test test/unit/iteration_test.rb version_number}}
|
54
|
+
style:
|
55
|
+
run: bundle exec rubocop -a
|
data/job-iteration.gemspec
CHANGED
@@ -5,9 +5,10 @@ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
|
|
5
5
|
require "job-iteration/version"
|
6
6
|
|
7
7
|
Gem::Specification.new do |spec|
|
8
|
+
spec.required_ruby_version = ">= 2.6"
|
8
9
|
spec.name = "job-iteration"
|
9
10
|
spec.version = JobIteration::VERSION
|
10
|
-
spec.authors =
|
11
|
+
spec.authors = ["Shopify"]
|
11
12
|
spec.email = ["ops-accounts+shipit@shopify.com"]
|
12
13
|
|
13
14
|
spec.summary = "Makes your background jobs interruptible and resumable."
|
@@ -20,7 +21,7 @@ Gem::Specification.new do |spec|
|
|
20
21
|
end
|
21
22
|
spec.bindir = "exe"
|
22
23
|
spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
|
23
|
-
spec.require_paths =
|
24
|
+
spec.require_paths = ["lib"]
|
24
25
|
|
25
26
|
spec.metadata["changelog_uri"] = "https://github.com/Shopify/job-iteration/blob/master/CHANGELOG.md"
|
26
27
|
spec.metadata["allowed_push_host"] = "https://rubygems.org"
|
data/lib/job-iteration.rb
CHANGED
@@ -38,16 +38,14 @@ module JobIteration
|
|
38
38
|
def load_integrations
|
39
39
|
loaded = nil
|
40
40
|
INTEGRATIONS.each do |integration|
|
41
|
-
|
42
|
-
|
43
|
-
|
44
|
-
|
45
|
-
|
46
|
-
"Iteration will only work with one integration."
|
47
|
-
end
|
48
|
-
loaded = integration
|
49
|
-
rescue LoadError
|
41
|
+
load_integration(integration)
|
42
|
+
if loaded
|
43
|
+
raise IntegrationLoadError,
|
44
|
+
"#{loaded} integration has already been loaded, but #{integration} is also available. " \
|
45
|
+
"Iteration will only work with one integration."
|
50
46
|
end
|
47
|
+
loaded = integration
|
48
|
+
rescue LoadError
|
51
49
|
end
|
52
50
|
end
|
53
51
|
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: job-iteration
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.
|
4
|
+
version: 1.2.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Shopify
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2021-
|
11
|
+
date: 2021-09-22 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activerecord
|
@@ -56,6 +56,7 @@ files:
|
|
56
56
|
- README.md
|
57
57
|
- Rakefile
|
58
58
|
- bin/setup
|
59
|
+
- bin/test
|
59
60
|
- dev.yml
|
60
61
|
- gemfiles/rails_5_2.gemfile
|
61
62
|
- gemfiles/rails_6_0.gemfile
|
@@ -92,14 +93,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
92
93
|
requirements:
|
93
94
|
- - ">="
|
94
95
|
- !ruby/object:Gem::Version
|
95
|
-
version: '
|
96
|
+
version: '2.6'
|
96
97
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
97
98
|
requirements:
|
98
99
|
- - ">="
|
99
100
|
- !ruby/object:Gem::Version
|
100
101
|
version: '0'
|
101
102
|
requirements: []
|
102
|
-
rubygems_version: 3.2.
|
103
|
+
rubygems_version: 3.2.20
|
103
104
|
signing_key:
|
104
105
|
specification_version: 4
|
105
106
|
summary: Makes your background jobs interruptible and resumable.
|