standard_procedure_operations 0.2.6 → 0.3.5

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: abbdad16738becd8b42eefb407d07e199bd4ab02b3590cdc5e14a605a8e9a8a2
-  data.tar.gz: f78887c11e55d0542d0b6f32435de1b07132879248dfccc9349f7b87e2746000
+  metadata.gz: 71a0ca8e718085505be26fcb316ed4a1e2f9338a12ea55901626696fe5e15269
+  data.tar.gz: 3b94981a2c8b8193bc01ee4f414db2c34161ac111c85a59f68eb3c88dd3b7aa6
 SHA512:
-  metadata.gz: e1d7e6702dce6caaeaae519e72dbad953b9a597e70863d8dc222f9db08c9292c8736b9e3f16059567d6b972754272aa7e11dffd9ab73f4168ed178ce286030d2
-  data.tar.gz: d1c4e2642723b131264dc6421755cd781f40ec6a1da78b385ac4b53bd11ad52901f1cbeacfd5b76620076ed3f3adc33a56088c5c764d9ba1cf1a05c310c03615
+  metadata.gz: 970184f8761aa729ceebf0cc5fa00a0c8c24a8ef9b1afaa3203b990d92c7c992441a51ef07579efcc93f4948f94d4da4b21c5df07df2adea700523cdb290d13d
+  data.tar.gz: 970e42aecaf9a622c259e27eb830a283ff1be70a4545ab01a4a42debe65ccf2cfcf68aa817a3778c2cfee503957cbf0c96de59e0ace254e2d773e1bb41d5c96e

data/README.md

@@ -116,6 +116,20 @@ end
 ```
 (In theory the block used in the `fail_with` case can do anything within the [DataCarrier context](#data-and-results) - so you could set internal state or call methods on the containing task - but I've not tried this yet).
 
+Alternatively, you can evaluate multiple conditions in your decision handler.  
+
+```ruby 
+decision :is_the_weather_good? do 
+  condition { weather_forecast.sunny? }
+  go_to :the_beach 
+  condition { weather_forecast.rainy? }
+  go_to :grab_an_umbrella 
+  condition { weather_forecast.snowing? }
+  go_to :build_a_snowman 
+end
+```
+If no conditions are matched then the task fails with a `NoDecision` exception.
+
 You can specify the data that is required for a decision handler to run by specifying `inputs` and `optionals`:
 ```ruby
 decision :authorised? do 
@@ -157,8 +171,24 @@ end
 ```
 Do not forget to call `go_to` from your action handler, otherwise the operation will just stop whilst still being marked as in progress.  (TODO: don't let this happen).
 
+### Waiting
+Wait handlers are very similar to decision handlers but only work within [background tasks](#background-operations-and-pauses).  
+
+```ruby 
+wait_until :weather_forecast_available? do 
+  condition { weather_forecast.sunny? }
+  go_to :the_beach 
+  condition { weather_forecast.rainy? }
+  go_to :grab_an_umbrella 
+  condition { weather_forecast.snowing? }
+  go_to :build_a_snowman 
+end
+```
+
+If no conditions are met, then, unlike a decision handler, the task continues waiting in the same state.  
+
 ### Results
-A result handler marks the end of an operation, optionally returning some results.  You need to copy your desired results from your [data](#data-and-results) to your results object.  This is so only the information that matters to you is stored as the results (and your data, effectively your "working memory", can be safely discarded).  
+A result handler marks the end of an operation, optionally returning some results.  You need to copy your desired results from your [data](#data-and-results) to your results object.  This is so only the information that matters to you is stored as the results.  
 
 ```ruby
 action :send_invitations do 
@@ -207,22 +237,6 @@ end
 ### Calling an operation
 You would use the earlier [PrepareDocumentForDownload](spec/examples/prepare_document_for_download_spec.rb) operation in a controller like this:
 
-```ruby
-class DownloadsController < ApplicationController 
-  def show 
-    @document = Document.includes(:account).find(params[:id])
-    @task = PrepareDocumentForDownload.call(user: Current.user, document: @document, use_filename_scrambler: @document.account.use_filename_scrambler?)
-    if @task.completed?
-      @filename = @task.results[:filename]
-      send_data @document.contents, filename: @filename, disposition: "attachment"
-    else
-      render action: "error", message: @task.results[:failure_message], status: 401
-    end
-  end
-end
-```
-
-Future dev: I'm going to change this so it raises an exception on failure so it will become:
 ```ruby
 class DownloadsController < ApplicationController 
   def show 
@@ -237,12 +251,16 @@ class DownloadsController < ApplicationController
 end
 ```
 
-OK - so that's a pretty longwinded way of performing a simple task.  But, in Collabor8Online, the actual operation for handling downloads has over twenty states, with half of them being decisions (as there are a number of feature flags and per-account configuration options).  Originally these were spread across multiple controllers, models and other objects.  Now they are centralised in a single "operations map" that describes the flowchart used to prepare a document for download - invaluable for comprehension of complex logic.  
+OK - so that's a pretty longwinded way of performing a simple task.  
+
+But many operations end up as complex flows of conditionals and actions, often spread across multiple classes and objects.  This means that someone trying to understand the rules for an operation can spend a lot of time tracing through code, understanding that flow.  
+
+In Collabor8Online, the actual operation for handling downloads has over twenty states, with half of them being decisions (there are a number of feature flags and per-account configuration options which need to be considered).  Now they are defined as four ruby classes (using [composition](#sub-tasks) to split the workload) and the logic is extremely easy to follow.  
 
 ### Data and results
 Each operation carries its own, mutable, [data](/app/models/operations/task/data_carrier.rb) for the duration of the operation.  
 
-This is provided when you `call` the operation to start it and is passed through to each decision, action and result.  This data is transient and not stored in the database. If you modify the data then that modification is passed on to the next handler.  (Note - when background tasks are implemented, we may end up storing the data in the database).
+This is provided when you `call` the operation to start it and is passed through to each decision, action and result.  If you modify the data then that modification is passed on to the next handler.  
 
 Within handlers you can read the data directly (the implementation uses `instance_eval`/`instance_exec`).  Here the `build_name` action knows the `first_name` and `last_name` provided and adds in a new property of `name`.  
 
@@ -265,18 +283,22 @@ task = CombineNames.call first_name: "Alice", last_name: "Aardvark"
 task.results[:name] # => Alice Aardvark
 ```
 
-Because handlers are run in the context of the data carrier, this means you do not have direct access to methods or properties on your task object.  So you need to use `task` to access it - `task.do_something` or `task.some_attribute`.  The exceptions are the `go_to` and `fail_with` methods which the data carrier forwards to the task (and the `TestResultCarrier` intercepts when you are testing your operation).  
+Because handlers are run in the context of the data carrier, you do not have direct access to methods or properties on your task object.  However, the data carrier holds a reference to your task; use `task.do_something` or `task.some_attribute` to access it.  The exceptions are the `go_to`, `fail_with`, `call` and `start` methods which the data carrier understands (and are intercepted when you are [testing](#testing)).  
+
+Both your task's `data` and its final `results` are stored in the database, so they can be examined later.  The `results` because that's what you're interested in, the `data` as it can be useful for debugging or auditing purposes.  
+
+They are both stored as hashes that are encoded into JSON.  
 
-The final `results` data from any `result` handlers is stored, along with the task, in the database, so it can be examined later.  It is a Hash that is encoded into JSON with any ActiveRecord models translated using a [GlobalID](https://github.com/rails/globalid) (this uses [ActiveJob::Arguments](https://guides.rubyonrails.org/active_job_basics.html#supported-types-for-arguments) so works the same way as passing data to ActiveJob).  
+Instead of using the standard [JSON coder](https://api.rubyonrails.org/v4.2/classes/ActiveModel/Serializers/JSON.html), we use a [GlobalIDSerialiser](/lib/operations/global_id_serialiser.rb).  This uses [ActiveJob::Arguments](https://guides.rubyonrails.org/active_job_basics.html#supported-types-for-arguments) to transform any models into [GlobalIDs](https://github.com/rails/globalid) before storage and convert them back to models upon retrieval.  
 
-Be aware that if you do store an ActiveRecord model into your `results` and that model is later deleted from the database, your task's `results` will be unavailable (as `GlobalID::Locator` will fail when it tries to load the record).  The data is not lost though - if the deserialisation fails, the routine will return the JSON string as `results[:raw_data]`.
+If the original database record was deleted between the time the hash was serialised and when it was retrieved, the `GlobalID::Locator` will fail.  With ActiveJob, this means that the job cannot run and is discarded.  For Operations, we attempt to deserialise a second time, returning the GlobalID string instead of the model.  So be aware that when you access `data` or `results` you may receive a string (similar to `"gid://test-app/User/1"`) instead of the models you were expecting.  And the error handling deserialiser is very simple so you may get format changes in some of the data as well.  If serialisation fails you can access the original JSON string as `data.raw_data` or `results[:raw_data]`.  
+
+TODO: Replace the ActiveJob::Arguments deserialiser with the [transporter](https://github.com/standard-procedure/plumbing/blob/main/lib/plumbing/actor/transporter.rb) from [plumbing](https://github.com/standard-procedure/plumbing)
 
 ### Failures and exceptions
 If any handlers raise an exception, the task will be terminated. It will be marked as `failed?` and the `results` hash will contain `results[:failure_message]`, `results[:exception_class]` and `results[:exception_backtrace]` for the exception's message, class name and backtrace respectively.  
 
-You can also stop a task at any point by calling `fail_with message`.  This will mark the task as `failed?` and the `results` has will contain `results[:failure_message]`.
-
-(Future dev - going to change this so it raises an exception (or passes it on to the caller) so you don't need to test for `failed?` every time - this will simplify managing sub-tasks).
+You can also stop a task at any point by calling `fail_with message`.  This will raise an `Operations::Failure` exception, marking the task as `failed?` and the `results` has will contain `results[:failure_message]`.
 
 ### Task life-cycle and the database
 There is an ActiveRecord migration that creates the `operations_tasks` table.  Use `bin/rails operations:install:migrations` to copy it to your application, then run `bin/rails db:migrate` to add the table to your application's database.  
@@ -332,8 +354,122 @@ class PrepareDownload < Operations::Task
 end
 ```
 
+If you want the sub-task to be a [background task](#background-operations-and-pauses), use `start` instead of `call`.  This will return the newly created sub-task immediately.  As the sub-task will not have completed, you cannot access its results unless you [wait](#waiting-for-sub-tasks-to-complete) (which is only possible if the parent task is a background task as well).  
+
 ### Background operations and pauses
-Coming soon.  
+If you have ActiveJob configured, you can run your operations in the background.  
+
+Instead of using `call`, use `start` to initiate the operation.  This takes the same data parameters and returns a task object that you can refer back to.  But it will be `waiting?` instead of `in_progress?` or `completed?`.  An `Operations::TaskRunnerJob` will be queued and it will mark the task as `in_progress?`, then call a _single_ state handler.  Instead of handling the next state immediately another `Operations::TaskRunnerJob` is queued.  And so on, until the task either fails or is completed.  
+
+By itself, this is not particularly useful - it just makes your operation take even longer to complete.  
+
+But if your operation may need to wait for something else to happen, background tasks are perfect.  
+
+#### Waiting for user input
+For example, maybe you're handling a user registration process and you need to wait until the verification link has been clicked.  The verification link goes to another controller and updates the user record in question.  Once clicked, you can then notify the administrator.  
+
+```ruby
+class UserRegistration < Operations::Task 
+  inputs :email
+  starts_with :create_user 
+  
+  action :create_user do 
+    inputs :email 
+
+    self.user = User.create! email: email
+    go_to :send_verification_email 
+  end 
+
+  action :send_verification_email do 
+    inputs :user 
+
+    UserMailer.with(user: user).verification_email.deliver_later 
+    go_to :verified? 
+  end 
+
+  wait_until :verified? do 
+    condition { user.verified? }
+    go_to :notify_administrator 
+  end 
+
+  action :notify_administrator do 
+    inputs :user 
+
+    AdminMailer.with(user: user).verification_completed.deliver_later
+  end
+end
+
+@task = UserRegistration.start email: "someone@example.com"
+```
+Because background tasks use ActiveJobs, every time the `verified?` condition is evaluated, the task object (and hence its data) will be reloaded from the database.  So the `user.verified?` property will be refreshed on each evaluation.  
+
+#### Waiting for sub-tasks to complete
+Alternatively, you may have a number of sub-tasks that you want to run in parallel then continue once they have all completed.  This allows you to spread their execution across multiple processes or even servers (depending upon how your job queue processes are configured).
+
+```ruby
+class ParallelTasks < Operations::Task 
+  inputs :number_of_sub_tasks
+  starts_with :start_sub_tasks 
+  
+  action :start_sub_tasks do 
+    inputs :number_of_sub_tasks
+    self.sub_tasks = (1..number_of_sub_tasks).collect { |i| start LongRunningTask, number: i }
+    go_to :do_something_else 
+  end 
+
+  action :do_something_else do 
+    # do something else while the sub-tasks do their thing
+    go_to :sub_tasks_completed? 
+  end 
+
+  wait_until :sub_tasks_completed? do 
+    condition { sub_tasks.all? { |t| t.completed? } }
+    go_to :done
+  end 
+
+  result :done 
+end
+
+@task = ParallelTasks.start number_of_sub_tasks: 5
+```
+The principle is the same as above; we store the newly created sub-tasks in our own `data` property.  As they are ActiveRecord models, they get reloaded each time `sub_tasks_completed?` is evaluated - and we check to see that they have all completed before moving on.  
+
+#### Delays and Timeouts
+When you run an operation in the background, it schedules an [ActiveJob](app/jobs/operations/task_runner_job.rb) which performs the individual state handler, scheduling a follow-up job for subsequent states.  By default, these jobs are scheduled to run after 1 second - so a five state operation will take a minimum of five seconds to complete (depending upon the backlog in your job queues).  This is to prevent a single operation from starving the job process of resources and allow other ActiveJobs the time to execute.  
+
+However, if you know that your `wait_until` condition may take a while you can change the default delay to something longer.  In your operations definition, declare the required delay: 
+
+```ruby
+class ParallelTasks < Operations::Tasks 
+  delay 1.minute
+  ...
+end
+```
+
+Likewise, it's possible for a background task to get stuck.  In the sub-tasks example above, if one of the sub-tasks fails, waiting for them _all_ to complete will never happen.  Every operation has a default timeout of 5 minutes - if the operation has not completed or failed 5 minutes after it was initially started, it will fail with an `Operations::Timeout` exception.  
+
+If you need to change this (such as the user verification example above), you can declare the timeout when defining the operation.  Long timeouts fit well with longer delays, so you're not filling the job queues with jobs that are meaninglessly evaluating your conditions.
+
+```ruby
+class UserRegistration < Operations::Task 
+  timeout 24.hours 
+  delay 15.minutes
+  ...
+end
+```
+
+Instead of failing with an `Operations::Timeout` exception, you define an `on_timeout` handler for any special processing should the time-out occur.  
+
+```ruby 
+class WaitForSomething < Operations::Task 
+  timeout 10.minutes 
+  delay 1.minute 
+
+  on_timeout do 
+    Notifier.send_timeout_notification
+  end
+end>
+```
 
 ## Testing
 Because operations are intended to model long, complex, flowcharts of decisions and actions, it can be a pain coming up with the combinations of inputs to test every path through the sequence.  
@@ -407,13 +543,9 @@ end
 ### Testing failures 
 To test if a handler has failed:
 ```ruby
-MyOperation.handling(:a_failure, some: "data") do |test|
-  assert_equal test.failure_message, "oh dear"
-  # or
-  expect(test).to have_failed_with "oh dear"
-end
+
+expect { MyOperation.handling(:a_failure, some: "data") }.to raise_error(SomeException)
 ```
-(Future dev: exceptions will not be silently caught so this will probably become `expect { MyOperation.handling(:a_failure, some: "data") }.to raise_exception(Whatever)).
 
 If you are using RSpec, you must `require "operations/matchers"` to make the matchers available to your specs.  
 
@@ -457,7 +589,10 @@ The gem is available as open source under the terms of the [LGPL License](/LICEN
 - [ ] Figure out how to stub calling sub-tasks with known results data 
 - [ ] Figure out how to test the parameters passed to sub-tasks when they are called
 - [ ] Split out the state-management definition stuff from the task class (so you can use it without subclassing Operations::Task)
-- [ ] Make Operations::Task work in the background using ActiveJob
-- [ ] Add pause/resume capabilities (for example, when a task needs to wait for user input)
-- [ ] Add wait for sub-tasks capabilities
+- [x] Make Operations::Task work in the background using ActiveJob
+- [x] Add pause/resume capabilities (for example, when a task needs to wait for user input)
+- [x] Add wait for sub-tasks capabilities
+- [ ] Add ActiveModel validations support for task parameters
+- [ ] Option to change background job queue and priority settings
+- [ ] Replace the ActiveJob::Arguments deserialiser with the [transporter](https://github.com/standard-procedure/plumbing/blob/main/lib/plumbing/actor/transporter.rb) from [plumbing](https://github.com/standard-procedure/plumbing)
 - [ ] Maybe? Split this out into two gems - one defining an Operation (pure ruby) and another defining the Task (using ActiveJob as part of a Rails Engine)

data/app/jobs/operations/application_job.rb

@@ -0,0 +1,9 @@
+module Operations
+  class ApplicationJob < ActiveJob::Base
+    # Automatically retry jobs that encountered a deadlock
+    # retry_on ActiveRecord::Deadlocked
+
+    # Most jobs are safe to ignore if the underlying records are no longer available
+    # discard_on ActiveJob::DeserializationError
+  end
+end

data/app/jobs/operations/task_runner_job.rb

@@ -0,0 +1,11 @@
+module Operations
+  class TaskRunnerJob < ApplicationJob
+    queue_as :default
+
+    def perform task
+      task.perform if task.waiting?
+    rescue => ex
+      Rails.logger.error "TaskRunnerJob failed: #{ex.message} for #{task.inspect}"
+    end
+  end
+end

data/app/models/operations/task/background.rb

@@ -0,0 +1,32 @@
+module Operations::Task::Background
+  extend ActiveSupport::Concern
+
+  class_methods do
+    def delay(value)
+      @background_delay = value
+    end
+
+    def timeout(value)
+      @execution_timeout = value
+    end
+
+    def on_timeout(&handler) = @on_timeout = handler
+
+    def background_delay = @background_delay ||= 1.second
+
+    def execution_timeout = @execution_timeout ||= 5.minutes
+
+    def timeout_handler = @on_timeout
+
+    def with_timeout(data) = data.merge(_execution_timeout: execution_timeout.from_now.utc)
+  end
+
+  private def background_delay = self.class.background_delay
+  private def execution_timeout = self.class.execution_timeout
+  private def timeout_handler = self.class.timeout_handler
+  private def timeout!
+    return unless timeout_expired?
+    timeout_handler.nil? ? raise(Operations::Timeout.new("Timeout expired", self)) : timeout_handler.call
+  end
+  private def timeout_expired? = data[:_execution_timeout].present? && data[:_execution_timeout] < Time.now.utc
+end

data/app/models/operations/task/data_carrier.rb

@@ -1,10 +1,12 @@
 class Operations::Task::DataCarrier < OpenStruct
-  def go_to(state, message = nil) = task.go_to(state, self, message)
+  def go_to(state, message: nil) = task.go_to(state, self, message: message)
 
   def fail_with(message) = task.fail_with(message)
 
   def call(sub_task_class, **data, &result_handler) = task.call(sub_task_class, **data, &result_handler)
 
+  def start(sub_task_class, **data, &result_handler) = task.start(sub_task_class, **data, &result_handler)
+
   def complete(results) = task.complete(results)
 
   def inputs(*names)

data/app/models/operations/task/state_management/action_handler.rb

@@ -6,7 +6,5 @@ class Operations::Task::StateManagement::ActionHandler
     @action = action
   end
 
-  def call(task, data)
-    @action.nil? ? task.send(@name, data) : data.instance_exec(&@action)
-  end
+  def call(task, data) = data.instance_exec(&@action)
 end

data/app/models/operations/task/state_management/decision_handler.rb

@@ -3,13 +3,16 @@ class Operations::Task::StateManagement::DecisionHandler
 
   def initialize name, &config
     @name = name.to_sym
-    @condition = nil
+    @conditions = []
+    @destinations = []
     @true_state = nil
     @false_state = nil
     instance_eval(&config)
   end
 
-  def condition(&condition) = @condition = condition
+  def condition(&condition) = @conditions << condition
+
+  def go_to(destination) = @destinations << destination
 
   def if_true(state = nil, &handler) = @true_state = state || handler
 
@@ -17,8 +20,20 @@ class Operations::Task::StateManagement::DecisionHandler
 
   def call(task, data)
     validate_inputs! data.to_h
-    result = @condition.nil? ? task.send(@name, data) : data.instance_exec(&@condition)
-    next_state = result ? @true_state : @false_state
-    next_state.respond_to?(:call) ? data.instance_eval(&next_state) : data.go_to(next_state, data)
+    has_true_false_handlers? ? handle_single_condition(task, data) : handle_multiple_conditions(task, data)
+  end
+
+  private def has_true_false_handlers? = !@true_state.nil? || !@false_state.nil?
+
+  private def handle_single_condition(task, data)
+    next_state = data.instance_eval(&@conditions.first) ? @true_state : @false_state
+    next_state.respond_to?(:call) ? data.instance_eval(&next_state) : data.go_to(next_state)
+  end
+
+  private def handle_multiple_conditions(task, data)
+    condition = @conditions.find { |condition| data.instance_eval(&condition) }
+    raise Operations::NoDecision.new("No conditions matched #{@name}") if condition.nil?
+    index = @conditions.index condition
+    data.go_to @destinations[index]
   end
 end

data/app/models/operations/task/state_management/wait_handler.rb

@@ -0,0 +1,27 @@
+class Operations::Task::StateManagement::WaitHandler
+  def initialize name, &config
+    @name = name.to_sym
+    @conditions = []
+    @destinations = []
+    instance_eval(&config)
+    puts "Configured"
+  end
+
+  def condition(&condition) = @conditions << condition
+
+  def go_to(state) = @destinations << state
+
+  def call(task, data)
+    raise Operations::CannotWaitInForeground.new("#{task.class} cannot wait in the foreground", task) unless task.background?
+    puts "Searching"
+    condition = @conditions.find { |condition| data.instance_eval(&condition) }
+    if condition.nil?
+      puts "None"
+      data.go_to task.state
+    else
+      index = @conditions.index condition
+      puts "Found #{@destinations[index]}"
+      data.go_to @destinations[index]
+    end
+  end
+end

data/app/models/operations/task/state_management.rb

@@ -15,6 +15,8 @@ module Operations::Task::StateManagement
 
     def action(name, inputs: [], optional: [], &handler) = state_handlers[name.to_sym] = ActionHandler.new(name, inputs, optional, &handler)
 
+    def wait_until(name, &config) = state_handlers[name.to_sym] = WaitHandler.new(name, &config)
+
     def result(name, inputs: [], optional: [], &results) = state_handlers[name.to_sym] = CompletionHandler.new(name, inputs, optional, &results)
 
     def state_handlers = @state_handlers ||= {}
@@ -23,12 +25,6 @@ module Operations::Task::StateManagement
   end
 
   private def handler_for(state) = self.class.handler_for(state.to_sym)
-  private def process_current_state(data)
-    handler_for(state).call(self, data)
-  rescue => ex
-    update! status: "failed", status_message: ex.message.to_s.truncate(240), results: {failure_message: ex.message, exception_class: ex.class.name, exception_backtrace: ex.backtrace}
-    raise ex
-  end
   private def state_is_valid
     errors.add :state, :invalid if state.blank? || handler_for(state.to_sym).nil?
   end

data/app/models/operations/task/testing.rb

@@ -21,13 +21,22 @@ module Operations::Task::Testing
     end
 
     def call(sub_task_class, **data, &result_handler)
-      self.sub_tasks ||= []
-      self.sub_tasks << sub_task_class
+      record_sub_task sub_task_class
+      super
+    end
+
+    def start(sub_task_class, **data, &result_handler)
+      record_sub_task sub_task_class
       super
     end
 
     def complete(results)
       self.completion_results = results
     end
+
+    private def record_sub_task sub_task_class
+      self.sub_tasks ||= []
+      self.sub_tasks << sub_task_class
+    end
   end
 end

data/app/models/operations/task.rb

@@ -1,24 +1,54 @@
 module Operations
   class Task < ApplicationRecord
     include StateManagement
-    include SubTasks
     include Deletion
     include Testing
+    include Background
     extend InputValidation
 
-    enum :status, in_progress: 0, completed: 1, failed: -1
+    enum :status, in_progress: 0, waiting: 10, completed: 100, failed: -1
+    serialize :data, coder: Operations::GlobalIDSerialiser, type: Hash, default: {}
     serialize :results, coder: Operations::GlobalIDSerialiser, type: Hash, default: {}
 
-    def self.call(data = {})
-      validate_inputs! data
-      create!(state: initial_state, status_message: "").tap do |task|
-        task.send(:process_current_state, DataCarrier.new(data.merge(task: task)))
+    def call sub_task_class, **data, &result_handler
+      sub_task = sub_task_class.call(**data)
+      result_handler&.call(sub_task.results)
+      sub_task.results
+    end
+
+    def start sub_task_class, **data, &result_handler
+      sub_task_class.start(**data)
+    end
+
+    def perform
+      timeout!
+      in_progress!
+      handler_for(state).call(self, carrier_for(data))
+    rescue => ex
+      update! status: "failed", status_message: ex.message.to_s.truncate(240), results: {failure_message: ex.message, exception_class: ex.class.name, exception_backtrace: ex.backtrace}
+      raise ex
+    end
+
+    def perform_later
+      waiting!
+      TaskRunnerJob.set(wait_until: background_delay.from_now).perform_later self
+    end
+
+    def self.call(**)
+      build(background: false, **).tap do |task|
+        task.perform
+      end
+    end
+
+    def self.start(**data)
+      build(background: true, **with_timeout(data)).tap do |task|
+        task.perform_later
       end
     end
 
-    def go_to(state, data = {}, message = nil)
-      update!(state: state, status_message: (message || state).to_s.truncate(240))
-      process_current_state(data)
+    def go_to(state, data = {}, message: nil)
+      update!(state: state, data: data.to_h, status_message: (message || state).to_s.truncate(240))
+      background? ? perform_later : perform
     end
 
     def fail_with(message)
@@ -27,5 +57,12 @@ module Operations
     end
 
     def complete(results) = update!(status: "completed", status_message: "completed", results: results.to_h)
+
+    private def carrier_for(data) = data.is_a?(DataCarrier) ? data : DataCarrier.new(data.merge(task: self))
+
+    def self.build(background:, **data)
+      validate_inputs! data
+      create!(state: initial_state, status: background ? "waiting" : "in_progress", data: data, status_message: "", background: background)
+    end
   end
 end

data/lib/operations/cannot_wait_in_foreground.rb

@@ -0,0 +1,2 @@
+class Operations::CannotWaitInForeground < Operations::Error
+end

data/lib/operations/global_id_serialiser.rb

@@ -15,7 +15,15 @@ module Operations
     def self.load(json)
       ActiveJob::Arguments.deserialize(ActiveSupport::JSON.decode(json)).first
     rescue => ex
-      {exception_message: ex.message, exception_class: ex.class.name, raw_data: json.to_s}
+      _load_without_global_ids(json).merge exception_message: ex.message, exception_class: ex.class.name, raw_data: json.to_s
+    end
+
+    def self._load_without_global_ids(json)
+      ActiveSupport::JSON.decode(json).first.tap do |hash|
+        hash.delete("_aj_symbol_keys")
+      end.transform_values do |value|
+        (value.is_a?(Hash) && value.key?("_aj_globalid")) ? value["_aj_globalid"] : value
+      end.transform_keys(&:to_sym)
     end
   end
 end

data/lib/operations/no_decision.rb

@@ -0,0 +1,2 @@
+class Operations::NoDecision < Operations::Error
+end

data/lib/operations/timeout.rb

@@ -0,0 +1,2 @@
+class Operations::Timeout < Operations::Error
+end

data/lib/operations/version.rb

@@ -1,3 +1,3 @@
 module Operations
-  VERSION = "0.2.6"
+  VERSION = "0.3.5"
 end

data/lib/operations.rb

@@ -12,4 +12,7 @@ module Operations
   require "operations/engine"
   require "operations/global_id_serialiser"
   require "operations/failure"
+  require "operations/cannot_wait_in_foreground"
+  require "operations/timeout"
+  require "operations/no_decision"
 end

metadata

@@ -1,13 +1,13 @@
 --- !ruby/object:Gem::Specification
 name: standard_procedure_operations
 version: !ruby/object:Gem::Version
-  version: 0.2.6
+  version: 0.3.5
 platform: ruby
 authors:
 - Rahoul Baruah
 bindir: bin
 cert_chain: []
-date: 2025-02-04 00:00:00.000000000 Z
+date: 2025-03-04 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rails
@@ -33,7 +33,10 @@ files:
 - LICENSE
 - README.md
 - Rakefile
+- app/jobs/operations/application_job.rb
+- app/jobs/operations/task_runner_job.rb
 - app/models/operations/task.rb
+- app/models/operations/task/background.rb
 - app/models/operations/task/data_carrier.rb
 - app/models/operations/task/deletion.rb
 - app/models/operations/task/input_validation.rb
@@ -41,15 +44,18 @@ files:
 - app/models/operations/task/state_management/action_handler.rb
 - app/models/operations/task/state_management/completion_handler.rb
 - app/models/operations/task/state_management/decision_handler.rb
-- app/models/operations/task/sub_tasks.rb
+- app/models/operations/task/state_management/wait_handler.rb
 - app/models/operations/task/testing.rb
 - config/routes.rb
 - db/migrate/20250127160616_create_operations_tasks.rb
 - lib/operations.rb
+- lib/operations/cannot_wait_in_foreground.rb
 - lib/operations/engine.rb
 - lib/operations/failure.rb
 - lib/operations/global_id_serialiser.rb
 - lib/operations/matchers.rb
+- lib/operations/no_decision.rb
+- lib/operations/timeout.rb
 - lib/operations/version.rb
 - lib/standard_procedure_operations.rb
 - lib/tasks/operations_tasks.rake

data/app/models/operations/task/sub_tasks.rb

@@ -1,7 +0,0 @@
-module Operations::Task::SubTasks
-  def call sub_task_class, **data, &result_handler
-    sub_task = sub_task_class.call(data)
-    result_handler&.call(sub_task.results)
-    sub_task.results
-  end
-end