taskinator 0.4.0 → 0.4.4

Sign up to get free protection for your applications and to get access to all the features.
data/README.md CHANGED
@@ -1,25 +1,29 @@
1
1
  # Taskinator
2
2
 
3
3
  [![Gem Version](https://badge.fury.io/rb/taskinator.svg)](http://badge.fury.io/rb/taskinator)
4
- [![Build Status](https://secure.travis-ci.org/virtualstaticvoid/taskinator.png?branch=master)](http://travis-ci.org/virtualstaticvoid/taskinator)
4
+ [![Build Status](https://img.shields.io/github/workflow/status/virtualstaticvoid/taskinator/Taskinator?style=flat-square)](https://github.com/virtualstaticvoid/taskinator/actions)
5
5
  [![Code Climate](https://codeclimate.com/github/virtualstaticvoid/taskinator.png)](https://codeclimate.com/github/virtualstaticvoid/taskinator)
6
- [![Coverage Status](https://coveralls.io/repos/virtualstaticvoid/taskinator/badge.png)](https://coveralls.io/r/virtualstaticvoid/taskinator)
7
6
 
8
- A simple orchestration library for running complex processes or workflows in Ruby. Processes are defined using a simple DSL, where the sequences and
9
- tasks are defined. Processes can then be queued for execution. Sequences can be synchronous or asynchronous, and the overall process can be monitored
10
- for completion or failure.
7
+ A simple orchestration library for running complex processes or workflows in Ruby.
8
+ Processes are defined using a simple DSL, where the sequences and tasks are defined.
9
+ Processes can then be queued for execution. Sequences can be synchronous or asynchronous,
10
+ and the overall process can be monitored for completion or failure.
11
11
 
12
- Processes and tasks are executed by background workers and you can use any one of the following gems:
12
+ Processes and tasks are executed by background workers and you can use any one of the
13
+ following gems:
13
14
 
15
+ * [active_job](https://github.com/rails/rails/tree/main/activejob)
14
16
  * [resque](https://github.com/resque/resque)
15
17
  * [sidekiq](https://github.com/mperham/sidekiq)
16
18
  * [delayed_job](https://github.com/collectiveidea/delayed_job)
17
19
 
18
- The configuration and state of each process and their respective tasks is stored using Redis key/values.
20
+ The configuration and state of each process and their respective tasks is stored using
21
+ Redis key/values.
19
22
 
20
23
  ## Requirements
21
24
 
22
- The latest MRI (2.1, 2.0) version. Other versions/VMs are untested but might work fine. MRI 1.9 is not supported.
25
+ The latest MRI 2.x or 3.x version. Other versions/VMs are untested, but might work fine.
26
+ MRI 1.x is not supported.
23
27
 
24
28
  Redis 2.4 or greater is required.
25
29
 
@@ -35,7 +39,7 @@ Add this line to your application's Gemfile:
35
39
 
36
40
  And then execute:
37
41
 
38
- $ bundle
42
+ $ bundle install
39
43
 
40
44
  Or install it yourself as:
41
45
 
@@ -49,6 +53,7 @@ Start by creating a "process" module and extending `Taskinator::Definition`.
49
53
 
50
54
  ```ruby
51
55
  require 'taskinator'
56
+
52
57
  module MyProcess
53
58
  extend Taskinator::Definition
54
59
 
@@ -68,7 +73,8 @@ module MyProcess
68
73
  end
69
74
  ```
70
75
 
71
- The `define_process` method optionally takes the list of expected arguments which are used to validate the arguments supplied when creating a new process.
76
+ The `define_process` method optionally takes the list of expected arguments which are used
77
+ to validate the arguments supplied when creating a new process.
72
78
  These should be specified with symbols.
73
79
 
74
80
  ```ruby
@@ -87,7 +93,8 @@ process = MyProcess.create_process Date.today, :option_1 => true
87
93
 
88
94
  _NOTE:_ The current implementation performs a naive check on the count of arguments.
89
95
 
90
- Next, specify the tasks with their corresponding implementation methods, that make up the process, using the `task` method and providing the `method` to execute for the task.
96
+ Next, specify the tasks with their corresponding implementation methods, that make up the
97
+ process, using the `task` method and providing the `method` to execute for the task.
91
98
 
92
99
  ```ruby
93
100
  module MyProcess
@@ -108,7 +115,8 @@ module MyProcess
108
115
  end
109
116
  ```
110
117
 
111
- More complex processes may define sequential or concurrent steps, using the `sequential` and `concurrent` methods respectively.
118
+ More complex processes may define sequential or concurrent steps, using the `sequential`
119
+ and `concurrent` methods respectively.
112
120
 
113
121
  ```ruby
114
122
  module MyProcess
@@ -141,10 +149,16 @@ module MyProcess
141
149
  end
142
150
  ```
143
151
 
144
- It is likely that you already have worker classes for one of the queueing libraries, such as resque or delayed_job, and wish to reuse them for executing them in the sequence defined by the process definition.
152
+ #### Reusing ActiveJob jobs
153
+
154
+ It is likely that you already have one or more [jobs](https://guides.rubyonrails.org/active_job_basics.html)
155
+ and want to reuse them within the process definition.
156
+
157
+ Define a `job` step, providing the class of the Active Job to run and then taskinator will
158
+ invoke that job as part of the process.
145
159
 
146
- Define a `job` step, providing the class of the worker, and then taskinator will execute that worker as part of the process definition.
147
- The `job` step will be queued and executed on same queue as configured by `delayed_job`, or that of the worker for `resque` and `sidekiq`.
160
+ The `job` step will be queued and executed on same queue as
161
+ [configured by the job](https://guides.rubyonrails.org/active_job_basics.html#queues).
148
162
 
149
163
  ```ruby
150
164
  # E.g. A resque worker
@@ -168,8 +182,13 @@ module MyProcess
168
182
  end
169
183
  ```
170
184
 
171
- You can also define data driven tasks using the `for_each` method, which takes an iterator method name as an argument.
172
- The iterator method yields the parameters necessary for the task or job. Notice that the task method takes a parameter in this case, which will be the return values provided by the iterator.
185
+ #### Data Driven Process Definitions
186
+
187
+ You can also define data driven tasks using the `for_each` method, which takes an iterator method
188
+ name as an argument.
189
+
190
+ The iterator method yields the parameters necessary for the task or job. Notice that the task
191
+ method takes a parameter in this case, which will be the return values provided by the iterator.
173
192
 
174
193
  ```ruby
175
194
  module MyProcess
@@ -192,8 +211,11 @@ module MyProcess
192
211
  end
193
212
  ```
194
213
 
195
- It is possible to branch the process logic based on the options hash passed in when creating a process.
196
- The `options?` method takes the options key as an argument and calls the supplied block if the option is present and it's value is truthy.
214
+ #### Branching
215
+
216
+ It is possible to branch the process logic based on the options hash passed in when creating
217
+ a process. The `options?` method takes the options key as an argument and calls the supplied
218
+ block if the option is present and it's value is _truthy_.
197
219
 
198
220
  ```ruby
199
221
  module MyProcess
@@ -227,8 +249,13 @@ process2 = MyProcess.create_process
227
249
  process2.tasks.count #=> 1
228
250
  ```
229
251
 
230
- In addition, it is possible to transform the arguments used by a task or job, by including a `transform` step in the definition.
231
- Similarly for the `for_each` method, `transform` takes a method name as an argument. The transformer method must yield the new arguments as required.
252
+ #### Transformations
253
+
254
+ In addition, it is possible to transform the arguments used by a task or job, by including
255
+ a `transform` step in the definition.
256
+
257
+ Similarly for the `for_each` method, `transform` takes a method name as an argument.
258
+ The transformer method must yield the new arguments as required.
232
259
 
233
260
  ```ruby
234
261
  module MyProcess
@@ -252,6 +279,8 @@ module MyProcess
252
279
  end
253
280
  ```
254
281
 
282
+ #### Subprocesses
283
+
255
284
  Processes can be composed of other processes too:
256
285
 
257
286
  ```ruby
@@ -273,7 +302,10 @@ module MyProcess
273
302
  end
274
303
  ```
275
304
 
276
- Any combination or nesting of `task`, `sequential`, `concurrent` and `for_each` steps are possible. E.g.
305
+ #### Complex Process Definitions
306
+
307
+ Any combination or nesting of `task`, `sequential`, `concurrent` and `for_each` steps are
308
+ possible. E.g.
277
309
 
278
310
  ```ruby
279
311
  module MyProcess
@@ -306,13 +338,17 @@ module MyProcess
306
338
  end
307
339
  ```
308
340
 
309
- In this example, the `work_step_begin` is executed, followed by the `work_step_all_at_once` steps which are executed concurrently, then
310
- the sub process `MySubProcess` is created and executed, followed by the `work_step_one_by_one` tasks which are executed sequentially and
341
+ In this example, the `work_step_begin` is executed, followed by the `work_step_all_at_once`
342
+ steps which are executed concurrently, then the sub process `MySubProcess` is created and
343
+ executed, followed by the `work_step_one_by_one` tasks which are executed sequentially and
311
344
  finally the `work_step_end` is executed.
312
345
 
313
- It is also possible to embed conditional logic within the process definition stages in order to produce steps based on the required logic.
314
- All builder methods are available within the scope of the `define_process` block. These methods include `args` and `options`
315
- which are passed into the `create_process` method of the definition.
346
+ It is also possible to embed conditional logic within the process definition stages in
347
+ order to produce steps based on the required logic.
348
+
349
+ All builder methods are available within the scope of the `define_process` block. These
350
+ methods include `args` and `options` which are passed into the `create_process` method
351
+ of the definition.
316
352
 
317
353
  E.g.
318
354
 
@@ -332,7 +368,8 @@ module MyProcess
332
368
  end
333
369
 
334
370
  # when creating this proces, you supply to option when calling `create_process`
335
- # in this example, 'args' will be an array [1,2,3] and options will be a Hash {:send_notification => true}
371
+ # in this example, 'args' will be an array [1,2,3]
372
+ # and options will be a Hash {:send_notification => true}
336
373
  MyProcess.create_process(1, 2, 3, :send_notification => true)
337
374
 
338
375
  ```
@@ -364,8 +401,12 @@ To best understand how arguments are handled, you need to break it down into 3 p
364
401
  * Creation and
365
402
  * Execution
366
403
 
367
- Firstly, a process definition is declarative in that the `define_process` and a mix of `sequential`, `concurrent`, `for_each`, `task` and `job` directives provide the way to specify the sequencing of the steps for the process.
368
- Taskinator will interprete this definition and execute each step in the desired sequence or concurrency.
404
+ Firstly, a process definition is declarative in that the `define_process` and a mix of
405
+ `sequential`, `concurrent`, `for_each`, `task` and `job` directives provide the way to
406
+ specify the sequencing of the steps for the process.
407
+
408
+ Taskinator will interprete this definition and execute each step in the desired sequence
409
+ or concurrency.
369
410
 
370
411
  Consider the following process definition:
371
412
 
@@ -411,11 +452,17 @@ end
411
452
 
412
453
  There are three tasks; namely `:work_step_1`, `:work_step_2` and `:work_step_3`.
413
454
 
414
- The third task, `:work_step_3`, is built up using the `for_each` iterator, which means that the number of `:work_step_3` tasks will depend on how many times the `additional_step` iterator method yields to the definition.
455
+ The third task, `:work_step_3`, is built up using the `for_each` iterator, which means that
456
+ the number of `:work_step_3` tasks will depend on how many times the `additional_step`
457
+ iterator method yields to the definition.
458
+
459
+ This brings us to the creation part. When `create_process` is called on the given module,
460
+ you provide arguments to it, which will get passed onto the respective `task` and
461
+ `for_each` iterator methods.
415
462
 
416
- This brings us to the creation part. When `create_process` is called on the given module, you provide arguments to it, which will get passed onto the respective `task` and `for_each` iterator methods.
463
+ So, considering the `MySimpleProcess` module shown above, `work_step_1`, `work_step_2`
464
+ and `work_step_3` methods each expect arguments.
417
465
 
418
- So, considering the `MySimpleProcess` module shown above, `work_step_1`, `work_step_2` and `work_step_3` methods each expect arguments.
419
466
  These will ultimately come from the arguments passed into the `create_process` method.
420
467
 
421
468
  E.g.
@@ -438,7 +485,8 @@ process = MySimpleProcess.create_process(options)
438
485
 
439
486
  ```
440
487
 
441
- To best understand how the process is created, consider the following "procedural" code for how it could work.
488
+ To best understand how the process is created, consider the following "procedural" code
489
+ for how it could work.
442
490
 
443
491
  ```ruby
444
492
  # A process, which maps the target and a list of steps
@@ -535,11 +583,17 @@ process.execute
535
583
 
536
584
  ```
537
585
 
538
- In reality, each task is executed by a worker process, possibly on another host, so the execution process isn't as simple, but this example should help you to understand conceptually how the process is executed, and how the arguments are propagated through.
586
+ In reality, each task is executed by a worker process, possibly on another host, so the
587
+ execution process isn't as simple, but this example should help you to understand
588
+ conceptually how the process is executed, and how the arguments are propagated through.
539
589
 
540
590
  ### Monitoring
541
591
 
542
- To monitor the state of the processes, use the `Taskinator::Api::Processes` class. This is still a work in progress.
592
+ NOTE: This aspect of the library is still a work in progress.
593
+
594
+ #### Processes
595
+
596
+ To monitor the state of the processes, use the `Taskinator::Api::Processes` class.
543
597
 
544
598
  ```ruby
545
599
  processes = Taskinator::Api::Processes.new
@@ -549,11 +603,57 @@ processes.each do |process|
549
603
  end
550
604
  ```
551
605
 
606
+ #### Debugging
607
+
608
+ To aid debugging specific processes and tasks, where the process or task identifier is
609
+ known, it is possible to retrieve the specific task or process using `Taskinator::Api`.
610
+
611
+ To retrieve a specific process, given the process identifier:
612
+
613
+ ```ruby
614
+ process_id = "SUPPLY-PROCESS-IDENTIFIER"
615
+ process = Taskinator::Api.find_process(process_id)
616
+
617
+ puts process.inspect
618
+ puts process.current_state
619
+ puts process.tasks
620
+ # etc...
621
+ ```
622
+
623
+ The type of process may be one of the following:
624
+
625
+ * `Taskinator::Process::Sequential`
626
+ * `Taskinator::Process::Concurrent`
627
+
628
+ Then, to retrieve a specific task, given the task identifier:
629
+
630
+ ```ruby
631
+ task_id = "SUPPLY-TASK-IDENTIFIER"
632
+ task = Taskinator::Api.find_task(task_id)
633
+
634
+ puts task.inspect
635
+ puts task.class
636
+ puts task.args # for Step and Job types
637
+ puts task.sub_process.tasks # for SubProcess type
638
+ # etc...
639
+ ```
640
+
641
+ Depending on the type of task, different attributes will be available for inspection.
642
+
643
+ The types include:
644
+
645
+ * `Taskinator::Task::Step`
646
+ * `Taskinator::Task::Job`
647
+ * `Taskinator::Task::SubProcess`
648
+
552
649
  ## Configuration
553
650
 
554
651
  ### Redis
555
652
 
556
- By default Taskinator assumes Redis is located at `localhost:6397`. This is fine for development, but for many production environments you will need to point to an external Redis server. You may also what to use a namespace for the Redis keys.
653
+ By default Taskinator assumes Redis is located at `localhost:6397`. This is fine for development,
654
+ but for many production environments you will need to point to an external Redis server.
655
+ You may also what to use a namespace for the Redis keys.
656
+
557
657
  _NOTE:_ The configuration hash _must_ have symbolized keys.
558
658
 
559
659
  ```ruby
@@ -568,15 +668,19 @@ end
568
668
  Or, alternatively, via an `ENV` variable
569
669
 
570
670
  Set the `REDIS_PROVIDER` environment variable to the Redis server url.
571
- E.g. On Heroku, with RedisGreen: set REDIS_PROVIDER=REDISGREEN_URL and Taskinator will use the value of the `REDISGREEN_URL` environment variable when connecting to Redis.
671
+ E.g. On Heroku, with RedisGreen: set REDIS_PROVIDER=REDISGREEN_URL and Taskinator will use the
672
+ value of the `REDISGREEN_URL` environment variable when connecting to Redis.
572
673
 
573
674
  You may also use the generic `REDIS_URL` which may be set to your own private Redis server.
574
675
 
575
- The Redis configuration leverages the same setup as `sidekiq`. For advanced options, checkout the [Sidekiq Advanced Options](https://github.com/mperham/sidekiq/wiki/Advanced-Options#complete-control) wiki for more information.
676
+ The Redis configuration leverages the same setup as `sidekiq`. For advanced options, checkout the
677
+ [Sidekiq Advanced Options](https://github.com/mperham/sidekiq/wiki/Advanced-Options#complete-control)
678
+ wiki page for more information.
576
679
 
577
680
  ### Queues
578
681
 
579
- By default the queue names for process and task workers is `default`, however, you can specify the queue names as follows:
682
+ By default the queue names for process and task workers is `default`, however, you can specify
683
+ the queue names as follows:
580
684
 
581
685
  ```ruby
582
686
  Taskinator.configure do |config|
@@ -589,7 +693,8 @@ end
589
693
 
590
694
  ### Instrumentation
591
695
 
592
- It is possible to instrument processes, tasks and jobs by providing an instrumeter such as `ActiveSupport::Notifications`.
696
+ It is possible to instrument processes, tasks and jobs by providing an instrumeter such
697
+ as `ActiveSupport::Notifications`.
593
698
 
594
699
  ```ruby
595
700
  Taskinator.configure do |config|
@@ -607,40 +712,41 @@ end
607
712
 
608
713
  The following instrumentation events are issued:
609
714
 
610
- | Event | When |
611
- |------------------------------------|-----------------------------------------------------------|
612
- | `taskinator.process.created` | After a root process gets created |
613
- | `taskinator.process.saved` | After a root process has been persisted to Redis |
614
- | `taskinator.process.enqueued` | After a process or subprocess is enqueued for processing |
615
- | `taskinator.process.processing` | When a process or subprocess is processing |
616
- | `taskinator.process.paused` | When a process or subprocess is paused |
617
- | `taskinator.process.resumed` | When a process or subprocess is resumed |
618
- | `taskinator.process.completed` | After a process or subprocess has completed processing |
619
- | `taskinator.process.cancelled` | After a process or subprocess has been cancelled |
620
- | `taskinator.process.failed` | After a process or subprocess has failed |
621
- | `taskinator.task.enqueued` | After a task has been enqueued |
622
- | `taskinator.task.processing` | When a task is processing |
623
- | `taskinator.task.completed` | After a task has completed |
624
- | `taskinator.task.cancelled` | After a task has been cancelled |
625
- | `taskinator.task.failed` | After a task has failed |
715
+ | Event | When |
716
+ |---------------------------------|----------------------------------------------------------|
717
+ | `taskinator.process.created` | After a root process gets created |
718
+ | `taskinator.process.saved` | After a root process has been persisted to Redis |
719
+ | `taskinator.process.enqueued` | After a process or subprocess is enqueued for processing |
720
+ | `taskinator.process.processing` | When a process or subprocess is processing |
721
+ | `taskinator.process.paused` | When a process or subprocess is paused |
722
+ | `taskinator.process.resumed` | When a process or subprocess is resumed |
723
+ | `taskinator.process.completed` | After a process or subprocess has completed processing |
724
+ | `taskinator.process.cancelled` | After a process or subprocess has been cancelled |
725
+ | `taskinator.process.failed` | After a process or subprocess has failed |
726
+ | `taskinator.task.enqueued` | After a task has been enqueued |
727
+ | `taskinator.task.processing` | When a task is processing |
728
+ | `taskinator.task.completed` | After a task has completed |
729
+ | `taskinator.task.cancelled` | After a task has been cancelled |
730
+ | `taskinator.task.failed` | After a task has failed |
626
731
 
627
732
  For all events, the data included contains the following information:
628
733
 
629
- | Key | Value |
630
- |--------------------------|-------------------------------------------------------|
631
- | `:type` | The type name of the component reporting the event |
632
- | `:process_uuid` | The UUID of the root process |
633
- | `:process_options` | Options hash of the root process |
634
- | `:uuid` | The UUID of the respective task, job or sub process |
635
- | `:options` | Options hash of the component |
636
- | `:state` | State of the component |
637
- | `:percentage_completed` | The percentage of completed tasks |
638
- | `:percentage_failed` | The percentage of failed tasks |
639
- | `:percentage_cancelled` | The percentage of cancelled tasks |
734
+ | Key | Value |
735
+ |---------------------------------|----------------------------------------------------------|
736
+ | `:type` | The type name of the component reporting the event |
737
+ | `:process_uuid` | The UUID of the root process |
738
+ | `:process_options` | Options hash of the root process |
739
+ | `:uuid` | The UUID of the respective task, job or sub process |
740
+ | `:options` | Options hash of the component |
741
+ | `:state` | State of the component |
742
+ | `:percentage_completed` | The percentage of completed tasks |
743
+ | `:percentage_failed` | The percentage of failed tasks |
744
+ | `:percentage_cancelled` | The percentage of cancelled tasks |
640
745
 
641
746
  ## Notes
642
747
 
643
- The persistence logic is decoupled from the implementation, so it is possible to implement another backing store if required.
748
+ The persistence logic is decoupled from the implementation, so it is possible to implement
749
+ another backing store if required.
644
750
 
645
751
  ## Contributing
646
752
 
@@ -651,12 +757,19 @@ The persistence logic is decoupled from the implementation, so it is possible to
651
757
  5. Create new Pull Request
652
758
 
653
759
  ## License
760
+
654
761
  MIT Copyright (c) 2014 Chris Stefano
655
762
 
656
763
  Portions of code are from the Sidekiq project, Copyright (c) Contributed Systems LLC.
657
764
 
658
765
  ## Inspiration
659
766
 
660
- Inspired by the [sidekiq](https://github.com/mperham/sidekiq) and [workflow](https://github.com/geekq/workflow) gems.
767
+ Inspired by the [sidekiq](https://github.com/mperham/sidekiq) and
768
+ [workflow](https://github.com/geekq/workflow) gems.
769
+
770
+ For other workflow solutions, checkout [Stonepath](https://github.com/bokmann/stonepath),
771
+ the now deprecated [ruote](https://github.com/jmettraux/ruote) gem and
772
+ [workflow](https://github.com/geekq/workflow).
661
773
 
662
- For other workflow solutions, checkout [Stonepath](https://github.com/bokmann/stonepath), the now deprecated [ruote](https://github.com/jmettraux/ruote) gem and [workflow](https://github.com/geekq/workflow). Alternatively, for a robust enterprise ready solution checkout the [AWS Flow Framework for Ruby](http://docs.aws.amazon.com/amazonswf/latest/awsrbflowguide/welcome.html).
774
+ Alternatively, for a robust enterprise ready solution checkout the
775
+ [AWS Flow Framework for Ruby](http://docs.aws.amazon.com/amazonswf/latest/awsrbflowguide/welcome.html).
@@ -13,12 +13,13 @@ module Taskinator
13
13
  def each(&block)
14
14
  return to_enum(__method__) unless block_given?
15
15
 
16
+ identifiers = Taskinator.redis do |conn|
17
+ conn.smembers(@processes_list_key)
18
+ end
19
+
16
20
  instance_cache = {}
17
- Taskinator.redis do |conn|
18
- uuids = conn.smembers(@processes_list_key)
19
- uuids.each do |uuid|
20
- yield Process.fetch(uuid, instance_cache)
21
- end
21
+ identifiers.each do |identifier|
22
+ yield Process.fetch(identifier, instance_cache)
22
23
  end
23
24
  end
24
25
 
@@ -28,5 +29,13 @@ module Taskinator
28
29
  end
29
30
  end
30
31
  end
32
+
33
+ def self.find_process(identifier)
34
+ Process.fetch(identifier)
35
+ end
36
+
37
+ def self.find_task(identifier)
38
+ Task.fetch(identifier)
39
+ end
31
40
  end
32
41
  end
@@ -24,7 +24,10 @@ module Taskinator
24
24
  raise ArgumentError, 'block' unless block_given?
25
25
 
26
26
  sub_process = Process.define_sequential_process_for(@definition, options)
27
- Builder.new(define_sub_process_task(@process, sub_process, options), @definition, *@args).instance_eval(&block)
27
+ task = define_sub_process_task(@process, sub_process, options)
28
+ Builder.new(sub_process, @definition, *@args).instance_eval(&block)
29
+ @process.tasks << task if sub_process.tasks.any?
30
+ nil
28
31
  end
29
32
 
30
33
  # defines a sub process of tasks which are executed concurrently
@@ -32,7 +35,10 @@ module Taskinator
32
35
  raise ArgumentError, 'block' unless block_given?
33
36
 
34
37
  sub_process = Process.define_concurrent_process_for(@definition, complete_on, options)
35
- Builder.new(define_sub_process_task(@process, sub_process, options), @definition, *@args).instance_eval(&block)
38
+ task = define_sub_process_task(@process, sub_process, options)
39
+ Builder.new(sub_process, @definition, *@args).instance_eval(&block)
40
+ @process.tasks << task if sub_process.tasks.any?
41
+ nil
36
42
  end
37
43
 
38
44
  # dynamically defines tasks, using the given @iterator method
@@ -51,6 +57,7 @@ module Taskinator
51
57
  @executor.send(method, *method_args) do |*args|
52
58
  Builder.new(@process, @definition, *args).instance_eval(&block)
53
59
  end
60
+ nil
54
61
  end
55
62
 
56
63
  alias_method :transform, :for_each
@@ -61,6 +68,7 @@ module Taskinator
61
68
  raise NoMethodError, method unless @executor.respond_to?(method)
62
69
 
63
70
  define_step_task(@process, method, @args, options)
71
+ nil
64
72
  end
65
73
 
66
74
  # defines a task which executes the given @job
@@ -70,6 +78,7 @@ module Taskinator
70
78
  raise ArgumentError, 'job' unless job.methods.include?(:perform) || job.instance_methods.include?(:perform)
71
79
 
72
80
  define_job_task(@process, job, @args, options)
81
+ nil
73
82
  end
74
83
 
75
84
  # defines a sub process task, for the given @definition
@@ -82,7 +91,10 @@ module Taskinator
82
91
  # TODO: decide whether the sub process to dynamically receive arguments
83
92
 
84
93
  sub_process = definition.create_sub_process(*@args, combine_options(options))
85
- Builder.new(define_sub_process_task(@process, sub_process, options), definition, *@args)
94
+ task = define_sub_process_task(@process, sub_process, options)
95
+ Builder.new(sub_process, definition, *@args)
96
+ @process.tasks << task if sub_process.tasks.any?
97
+ nil
86
98
  end
87
99
 
88
100
  private
@@ -100,10 +112,7 @@ module Taskinator
100
112
  end
101
113
 
102
114
  def define_sub_process_task(process, sub_process, options={})
103
- define_task(process) {
104
- Task.define_sub_process_task(process, sub_process, combine_options(options))
105
- }
106
- sub_process
115
+ Task.define_sub_process_task(process, sub_process, combine_options(options))
107
116
  end
108
117
 
109
118
  def define_task(process)
@@ -604,6 +604,9 @@ module Taskinator
604
604
  end
605
605
 
606
606
  def visit_tasks(tasks)
607
+ @conn.expire "#{@key}:tasks", expire_in
608
+ @conn.expire "#{@key}.count", expire_in
609
+ @conn.expire "#{@key}.pending", expire_in
607
610
  tasks.each do |task|
608
611
  RedisCleanupVisitor.new(@conn, task, expire_in).visit
609
612
  end
@@ -0,0 +1,53 @@
1
+ module Taskinator
2
+ module Queues
3
+
4
+ # https://guides.rubyonrails.org/active_job_basics.html
5
+
6
+ def self.create_active_job_adapter(config={})
7
+ ActiveJobAdapter.new(config)
8
+ end
9
+
10
+ class ActiveJobAdapter
11
+ def initialize(config={})
12
+ @config = Taskinator::Queues::DefaultConfig.merge(config)
13
+ end
14
+
15
+ def enqueue_create_process(definition, uuid, args)
16
+ queue = definition.queue || @config[:definition_queue]
17
+ CreateProcessWorker.set(:queue => queue)
18
+ .perform_later(definition.name, uuid, Taskinator::Persistence.serialize(args))
19
+ end
20
+
21
+ def enqueue_process(process)
22
+ queue = process.queue || @config[:process_queue]
23
+ ProcessWorker.set(:queue => queue)
24
+ .perform_later(process.uuid)
25
+ end
26
+
27
+ def enqueue_task(task)
28
+ queue = task.queue || @config[:task_queue]
29
+ TaskWorker.set(:queue => queue)
30
+ .perform_later(task.uuid)
31
+ end
32
+
33
+ class CreateProcessWorker < ApplicationJob
34
+ def perform(definition_name, uuid, args)
35
+ Taskinator::CreateProcessWorker.new(definition_name, uuid, args).perform
36
+ end
37
+ end
38
+
39
+ class ProcessWorker < ApplicationJob
40
+ def perform(process_uuid)
41
+ Taskinator::ProcessWorker.new(process_uuid).perform
42
+ end
43
+ end
44
+
45
+ class TaskWorker < ApplicationJob
46
+ def perform(task_uuid)
47
+ Taskinator::TaskWorker.new(task_uuid).perform
48
+ end
49
+ end
50
+
51
+ end
52
+ end
53
+ end
@@ -48,6 +48,7 @@ module Taskinator
48
48
  end
49
49
  end
50
50
 
51
+ require 'taskinator/queues/active_job' if defined?(ApplicationJob)
51
52
  require 'taskinator/queues/delayed_job' if defined?(Delayed)
52
53
  require 'taskinator/queues/resque' if defined?(Resque)
53
54
  require 'taskinator/queues/sidekiq' if defined?(Sidekiq)
@@ -1,3 +1,3 @@
1
1
  module Taskinator
2
- VERSION = "0.4.0"
2
+ VERSION = "0.4.4"
3
3
  end
data/lib/taskinator.rb CHANGED
@@ -98,7 +98,7 @@ module Taskinator
98
98
 
99
99
  # the queue adapter to use
100
100
  # supported adapters include
101
- # :delayed_job, :redis and :sidekiq
101
+ # :active_job, :delayed_job, :redis and :sidekiq
102
102
  # NOTE: ensure that the respective gem is included
103
103
  attr_reader :queue_adapter
104
104
 
@@ -118,7 +118,7 @@ module Taskinator
118
118
 
119
119
  def queue
120
120
  @queue ||= begin
121
- adapter = self.queue_adapter || :resque
121
+ adapter = self.queue_adapter || :resque # TODO: change default to :active_job
122
122
  config = queue_config || {}
123
123
  Taskinator::Queues.create_adapter(adapter, config)
124
124
  end