gush 2.1.0 → 3.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: dc70498c39ad506749bed1f55c139d29691e540b849dd39e2ba132b54b511d66
4
- data.tar.gz: 700d093574e0ff4772c7d36bfeb7792cda44550dd162a48afe17bec5551250cd
3
+ metadata.gz: e588202fe13ded7c99f192bda5bc33d3d6e8994deb3fb17f5a823d4882c4552f
4
+ data.tar.gz: 759593f344caf579cce496b1da9c64c9431d6f2cbb92cbcced181d49421fdefb
5
5
  SHA512:
6
- metadata.gz: ac4c1647f57a87600466445a8d24660e3d95016f17adf9e2fe8f99260ba82a2b3e22a339b9456f653834c89c6e4b9ed6bcb50ebc337b804e07355f9a513ce8ca
7
- data.tar.gz: 1ae01511acfc2e23bb0a671328a7b3e9b650b8589b9620461c2b9667ff9d80a47f03b8a156f249746b4f69e613863db0833be8d6221acdabd211d7eba2daee15
6
+ metadata.gz: 5d6068f23d178beb5dbfaa9cf317ae086542dedf33aeab82f82c8cab5a6ea569d932b2dc0a787978629e892a39f25270827c55fa4a6b0a6bb349f9ab87fca1db
7
+ data.tar.gz: 69ddc27452586d4b188969ece9ab73aefd90377fd93503f3e83c9fd074722d808a09cc620f98953756d7c684bab808db14550c15bada3e93111815613e52b78c
@@ -26,35 +26,10 @@ jobs:
26
26
  runs-on: ubuntu-latest
27
27
  strategy:
28
28
  matrix:
29
- rails_version: ['4.2.7', '5.1.0', '5.2.0', '6.0.0', '6.1.0', '7.0']
30
- ruby-version: ['2.6', '2.7', '3.0', '3.1']
31
- exclude:
32
- - ruby-version: '3.0'
33
- rails_version: '4.2.7'
34
- - ruby-version: '3.1'
35
- rails_version: '4.2.7'
36
- - ruby-version: '3.0'
37
- rails_version: '5.0'
38
- - ruby-version: '3.1'
39
- rails_version: '5.0'
40
- - ruby-version: '3.0'
41
- rails_version: '5.1'
42
- - ruby-version: '3.1'
43
- rails_version: '5.1'
44
- - ruby-version: '3.0'
45
- rails_version: '5.2'
46
- - ruby-version: '3.1'
47
- rails_version: '5.2'
48
- - ruby-version: '3.0'
49
- rails_version: '6.0'
50
- - ruby-version: '3.1'
51
- rails_version: '6.0'
52
- - ruby-version: '3.1'
53
- rails_version: '6.1'
54
- - ruby-version: '2.6'
55
- rails_version: '7.0'
29
+ rails_version: ['6.1.0', '7.0', '7.1.0']
30
+ ruby-version: ['3.0', '3.1', '3.2', '3.3']
56
31
  steps:
57
- - uses: actions/checkout@v2
32
+ - uses: actions/checkout@v4
58
33
  - name: Set up Ruby
59
34
  uses: ruby/setup-ruby@v1
60
35
  with:
data/README.md CHANGED
@@ -1,5 +1,9 @@
1
1
  # Gush
2
2
 
3
+ ![Gem Version](https://img.shields.io/gem/v/gush)
4
+ ![GitHub Actions Workflow Status](https://img.shields.io/github/actions/workflow/status/chaps-io/gush/ruby.yml)
5
+
6
+
3
7
  Gush is a parallel workflow runner using only Redis as storage and [ActiveJob](http://guides.rubyonrails.org/v4.2/active_job_basics.html#introduction) for scheduling and executing jobs.
4
8
 
5
9
  ## Theory
@@ -15,7 +19,7 @@ This README is about the latest `master` code, which might differ from what is r
15
19
  ### 1. Add `gush` to Gemfile
16
20
 
17
21
  ```ruby
18
- gem 'gush', '~> 2.0'
22
+ gem 'gush', '~> 3.0'
19
23
  ```
20
24
 
21
25
  ### 2. Create `Gushfile`
@@ -74,7 +78,17 @@ end
74
78
 
75
79
  and this is how the graph will look like:
76
80
 
77
- ![SampleWorkflow](https://i.imgur.com/DFh6j51.png)
81
+ ```mermaid
82
+ graph TD
83
+ A{Start} --> B[FetchJob1]
84
+ A --> C[FetchJob2]
85
+ B --> D[PersistJob1]
86
+ C --> E[PersistJob2]
87
+ D --> F[NormalizeJob]
88
+ E --> F
89
+ F --> G[IndexJob]
90
+ G --> H{Finish}
91
+ ```
78
92
 
79
93
 
80
94
  ## Defining workflows
@@ -211,7 +225,7 @@ For example, in case of Sidekiq this would be:
211
225
  bundle exec sidekiq -q gush
212
226
  ```
213
227
 
214
- **[Click here to see backends section in official ActiveJob documentation about configuring backends](http://guides.rubyonrails.org/v4.2/active_job_basics.html#backends)**
228
+ **[Click here to see backends section in official ActiveJob documentation about configuring backends](http://guides.rubyonrails.org/active_job_basics.html#backends)**
215
229
 
216
230
  **Hint**: gush uses `gush` queue name by default. Keep that in mind, because some backends (like Sidekiq) will only run jobs from explicitly stated queues.
217
231
 
@@ -319,7 +333,21 @@ flow = NotifyWorkflow.create([54, 21, 24, 154, 65]) # 5 user ids as an argument
319
333
 
320
334
  it will generate a workflow with 5 `NotificationJob`s and one `AdminNotificationJob` which will depend on all of them:
321
335
 
322
- ![DynamicWorkflow](https://i.imgur.com/HOI3fjc.png)
336
+
337
+ ```mermaid
338
+ graph TD
339
+ A{Start} --> B[NotificationJob]
340
+ A --> C[NotificationJob]
341
+ A --> D[NotificationJob]
342
+ A --> E[NotificationJob]
343
+ A --> F[NotificationJob]
344
+ B --> G[AdminNotificationJob]
345
+ C --> G
346
+ D --> G
347
+ E --> G
348
+ F --> G
349
+ G --> H{Finish}
350
+ ```
323
351
 
324
352
  ### Dynamic queue for jobs
325
353
 
@@ -338,6 +366,23 @@ class NotifyWorkflow < Gush::Workflow
338
366
  end
339
367
  ```
340
368
 
369
+ ### Dynamic waitable time for jobs
370
+
371
+ There might be a case you want to configure a job to be executed after a time. Based on above example, we want to configure `AdminNotificationJob` to be executed after 5 seconds.
372
+
373
+ ```ruby
374
+
375
+ class NotifyWorkflow < Gush::Workflow
376
+ def configure(user_ids)
377
+ notification_jobs = user_ids.map do |user_id|
378
+ run NotificationJob, params: {user_id: user_id}, queue: 'user'
379
+ end
380
+
381
+ run AdminNotificationJob, after: notification_jobs, queue: 'admin', wait: 5.seconds
382
+ end
383
+ end
384
+ ```
385
+
341
386
  ## Command line interface (CLI)
342
387
 
343
388
  ### Checking status
data/gush.gemspec CHANGED
@@ -2,11 +2,13 @@
2
2
  lib = File.expand_path('../lib', __FILE__)
3
3
  $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
4
4
 
5
+ require_relative 'lib/gush/version'
6
+
5
7
  Gem::Specification.new do |spec|
6
8
  spec.name = "gush"
7
- spec.version = "2.1.0"
8
- spec.authors = ["Piotrek Okoński"]
9
- spec.email = ["piotrek@okonski.org"]
9
+ spec.version = Gush::VERSION
10
+ spec.authors = ["Piotrek Okoński", "Michał Krzyżanowski"]
11
+ spec.email = ["piotrek@okonski.org", "michal.krzyzanowski+github@gmail.com"]
10
12
  spec.summary = "Fast and distributed workflow runner based on ActiveJob and Redis"
11
13
  spec.description = "Gush is a parallel workflow runner using Redis as storage and ActiveJob for executing jobs."
12
14
  spec.homepage = "https://github.com/chaps-io/gush"
@@ -16,11 +18,12 @@ Gem::Specification.new do |spec|
16
18
  spec.executables = "gush"
17
19
  spec.test_files = spec.files.grep(%r{^(test|spec|features)/})
18
20
  spec.require_paths = ["lib"]
21
+ spec.required_ruby_version = '>= 3.0.0'
19
22
 
20
- spec.add_dependency "activejob", ">= 4.2.7", "< 7.1"
23
+ spec.add_dependency "activejob", ">= 6.1.0", "< 7.2"
21
24
  spec.add_dependency "concurrent-ruby", "~> 1.0"
22
25
  spec.add_dependency "multi_json", "~> 1.11"
23
- spec.add_dependency "redis", ">= 3.2", "< 5"
26
+ spec.add_dependency "redis", ">= 3.2", "< 6"
24
27
  spec.add_dependency "redis-mutex", "~> 4.0.1"
25
28
  spec.add_dependency "hiredis", "~> 0.6"
26
29
  spec.add_dependency "graphviz", "~> 1.2"
@@ -29,7 +32,7 @@ Gem::Specification.new do |spec|
29
32
  spec.add_dependency "thor", ">= 0.19", "< 1.3"
30
33
  spec.add_dependency "launchy", "~> 2.4"
31
34
  spec.add_development_dependency "bundler"
32
- spec.add_development_dependency "rake", "~> 10.4"
35
+ spec.add_development_dependency "rake", "~> 12"
33
36
  spec.add_development_dependency "rspec", '~> 3.0'
34
37
  spec.add_development_dependency "pry", '~> 0.10'
35
38
  end
data/lib/gush/client.rb CHANGED
@@ -156,8 +156,13 @@ module Gush
156
156
  job.enqueue!
157
157
  persist_job(workflow_id, job)
158
158
  queue = job.queue || configuration.namespace
159
-
160
- Gush::Worker.set(queue: queue).perform_later(*[workflow_id, job.name])
159
+ wait = job.wait
160
+
161
+ if wait.present?
162
+ Gush::Worker.set(queue: queue, wait: wait).perform_later(*[workflow_id, job.name])
163
+ else
164
+ Gush::Worker.set(queue: queue).perform_later(*[workflow_id, job.name])
165
+ end
161
166
  end
162
167
 
163
168
  private
data/lib/gush/job.rb CHANGED
@@ -1,7 +1,8 @@
1
1
  module Gush
2
2
  class Job
3
3
  attr_accessor :workflow_id, :incoming, :outgoing, :params,
4
- :finished_at, :failed_at, :started_at, :enqueued_at, :payloads, :klass, :queue
4
+ :finished_at, :failed_at, :started_at, :enqueued_at, :payloads,
5
+ :klass, :queue, :wait
5
6
  attr_reader :id, :klass, :output_payload, :params
6
7
 
7
8
  def initialize(opts = {})
@@ -126,6 +127,7 @@ module Gush
126
127
  @output_payload = opts[:output_payload]
127
128
  @workflow_id = opts[:workflow_id]
128
129
  @queue = opts[:queue]
130
+ @wait = opts[:wait]
129
131
  end
130
132
  end
131
133
  end
@@ -0,0 +1,3 @@
1
+ module Gush
2
+ VERSION = '3.0.0'
3
+ end
data/lib/gush/workflow.rb CHANGED
@@ -112,7 +112,8 @@ module Gush
112
112
  workflow_id: id,
113
113
  id: client.next_free_job_id(id, klass.to_s),
114
114
  params: opts.fetch(:params, {}),
115
- queue: opts[:queue]
115
+ queue: opts[:queue],
116
+ wait: opts[:wait]
116
117
  })
117
118
 
118
119
  jobs << node
@@ -5,9 +5,10 @@ describe "Workflows" do
5
5
  context "when all jobs finish successfuly" do
6
6
  it "marks workflow as completed" do
7
7
  flow = TestWorkflow.create
8
- perform_enqueued_jobs do
9
- flow.start!
10
- end
8
+
9
+ ActiveJob::Base.queue_adapter.perform_enqueued_jobs = true
10
+ flow.start!
11
+ ActiveJob::Base.queue_adapter.perform_enqueued_jobs = false
11
12
 
12
13
  flow = flow.reload
13
14
  expect(flow).to be_finished
@@ -37,6 +37,18 @@ describe Gush::Client do
37
37
  end
38
38
 
39
39
  describe "#start_workflow" do
40
+ context "when there is wait parameter configured" do
41
+ let(:freeze_time) { Time.utc(2023, 01, 21, 14, 36, 0) }
42
+
43
+ it "schedules job execution" do
44
+ travel_to freeze_time do
45
+ workflow = WaitableTestWorkflow.create
46
+ client.start_workflow(workflow)
47
+ expect(Gush::Worker).to have_a_job_enqueued_at(workflow.id, job_with_id("Prepare"), 5.minutes)
48
+ end
49
+ end
50
+ end
51
+
40
52
  it "enqueues next jobs from the workflow" do
41
53
  workflow = TestWorkflow.create
42
54
  expect {
@@ -113,7 +125,7 @@ describe Gush::Client do
113
125
  describe "#persist_job" do
114
126
  it "persists JSON dump of the job in Redis" do
115
127
 
116
- job = BobJob.new(name: 'bob')
128
+ job = BobJob.new(name: 'bob', id: 'abcd123')
117
129
 
118
130
  client.persist_job('deadbeef', job)
119
131
  expect(redis.keys("gush.jobs.deadbeef.*").length).to eq(1)
@@ -121,6 +121,13 @@ describe Gush::Workflow do
121
121
  expect(flow.jobs.first.params).to eq ({ something: 1 })
122
122
  end
123
123
 
124
+ it "allows passing wait param to the job" do
125
+ flow = Gush::Workflow.new
126
+ flow.run(Gush::Job, wait: 5.seconds)
127
+ flow.save
128
+ expect(flow.jobs.first.wait).to eq (5.seconds)
129
+ end
130
+
124
131
  context "when graph is empty" do
125
132
  it "adds new job with the given class as a node" do
126
133
  flow = Gush::Workflow.new
data/spec/spec_helper.rb CHANGED
@@ -1,3 +1,5 @@
1
+ require 'active_support'
2
+ require 'active_support/testing/time_helpers'
1
3
  require 'gush'
2
4
  require 'json'
3
5
  require 'pry'
@@ -34,6 +36,11 @@ class ParameterTestWorkflow < Gush::Workflow
34
36
  end
35
37
  end
36
38
 
39
+ class WaitableTestWorkflow < Gush::Workflow
40
+ def configure
41
+ run Prepare, wait: 5.minutes
42
+ end
43
+ end
37
44
 
38
45
  REDIS_URL = ENV["REDIS_URL"] || "redis://localhost:6379/12"
39
46
 
@@ -86,7 +93,22 @@ RSpec::Matchers.define :have_no_jobs do |flow, jobs|
86
93
  end
87
94
  end
88
95
 
96
+ RSpec::Matchers.define :have_a_job_enqueued_at do |flow, job, at|
97
+ expected_execution_timestamp = (Time.current.utc + at).to_i
98
+
99
+ match do |actual|
100
+ expected = hash_including(args: include(flow, job), at: expected_execution_timestamp)
101
+
102
+ expect(ActiveJob::Base.queue_adapter.enqueued_jobs).to match_array(expected)
103
+ end
104
+
105
+ failure_message do |actual|
106
+ "expected to have enqueued job #{job} to be executed at #{Time.current.utc + at}, but instead has: #{Time.at(enqueued_jobs.first[:at]).to_datetime.utc}"
107
+ end
108
+ end
109
+
89
110
  RSpec.configure do |config|
111
+ config.include ActiveSupport::Testing::TimeHelpers
90
112
  config.include ActiveJob::TestHelper
91
113
  config.include GushHelpers
92
114
 
metadata CHANGED
@@ -1,14 +1,15 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: gush
3
3
  version: !ruby/object:Gem::Version
4
- version: 2.1.0
4
+ version: 3.0.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Piotrek Okoński
8
+ - Michał Krzyżanowski
8
9
  autorequire:
9
10
  bindir: bin
10
11
  cert_chain: []
11
- date: 2022-09-19 00:00:00.000000000 Z
12
+ date: 2024-02-29 00:00:00.000000000 Z
12
13
  dependencies:
13
14
  - !ruby/object:Gem::Dependency
14
15
  name: activejob
@@ -16,20 +17,20 @@ dependencies:
16
17
  requirements:
17
18
  - - ">="
18
19
  - !ruby/object:Gem::Version
19
- version: 4.2.7
20
+ version: 6.1.0
20
21
  - - "<"
21
22
  - !ruby/object:Gem::Version
22
- version: '7.1'
23
+ version: '7.2'
23
24
  type: :runtime
24
25
  prerelease: false
25
26
  version_requirements: !ruby/object:Gem::Requirement
26
27
  requirements:
27
28
  - - ">="
28
29
  - !ruby/object:Gem::Version
29
- version: 4.2.7
30
+ version: 6.1.0
30
31
  - - "<"
31
32
  - !ruby/object:Gem::Version
32
- version: '7.1'
33
+ version: '7.2'
33
34
  - !ruby/object:Gem::Dependency
34
35
  name: concurrent-ruby
35
36
  requirement: !ruby/object:Gem::Requirement
@@ -67,7 +68,7 @@ dependencies:
67
68
  version: '3.2'
68
69
  - - "<"
69
70
  - !ruby/object:Gem::Version
70
- version: '5'
71
+ version: '6'
71
72
  type: :runtime
72
73
  prerelease: false
73
74
  version_requirements: !ruby/object:Gem::Requirement
@@ -77,7 +78,7 @@ dependencies:
77
78
  version: '3.2'
78
79
  - - "<"
79
80
  - !ruby/object:Gem::Version
80
- version: '5'
81
+ version: '6'
81
82
  - !ruby/object:Gem::Dependency
82
83
  name: redis-mutex
83
84
  requirement: !ruby/object:Gem::Requirement
@@ -208,14 +209,14 @@ dependencies:
208
209
  requirements:
209
210
  - - "~>"
210
211
  - !ruby/object:Gem::Version
211
- version: '10.4'
212
+ version: '12'
212
213
  type: :development
213
214
  prerelease: false
214
215
  version_requirements: !ruby/object:Gem::Requirement
215
216
  requirements:
216
217
  - - "~>"
217
218
  - !ruby/object:Gem::Version
218
- version: '10.4'
219
+ version: '12'
219
220
  - !ruby/object:Gem::Dependency
220
221
  name: rspec
221
222
  requirement: !ruby/object:Gem::Requirement
@@ -248,6 +249,7 @@ description: Gush is a parallel workflow runner using Redis as storage and Activ
248
249
  for executing jobs.
249
250
  email:
250
251
  - piotrek@okonski.org
252
+ - michal.krzyzanowski+github@gmail.com
251
253
  executables:
252
254
  - gush
253
255
  extensions: []
@@ -272,6 +274,7 @@ files:
272
274
  - lib/gush/graph.rb
273
275
  - lib/gush/job.rb
274
276
  - lib/gush/json.rb
277
+ - lib/gush/version.rb
275
278
  - lib/gush/worker.rb
276
279
  - lib/gush/workflow.rb
277
280
  - spec/Gushfile
@@ -298,14 +301,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
298
301
  requirements:
299
302
  - - ">="
300
303
  - !ruby/object:Gem::Version
301
- version: '0'
304
+ version: 3.0.0
302
305
  required_rubygems_version: !ruby/object:Gem::Requirement
303
306
  requirements:
304
307
  - - ">="
305
308
  - !ruby/object:Gem::Version
306
309
  version: '0'
307
310
  requirements: []
308
- rubygems_version: 3.3.3
311
+ rubygems_version: 3.4.22
309
312
  signing_key:
310
313
  specification_version: 4
311
314
  summary: Fast and distributed workflow runner based on ActiveJob and Redis