standard_procedure_operations 0.5.0 → 0.5.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a14742a79230259d8cb55462cac7682b6adfe8993cacc0ec4f35ed3832910858
-  data.tar.gz: 939c54f7cb755794ecd71ee920bfc1ff65f8e925187f6ccd9c1e98515d5a1b3d
+  metadata.gz: c254fd322931c448af32ef7aef5bfba2f09293b5d2ab2b85d069783be09cb8e3
+  data.tar.gz: e329a253576b4c4756793a068eedbb275c9ddab4ce0af4059285c0d13ad5dffe
 SHA512:
-  metadata.gz: 6c3ec1e16dada2b58d48aa72a11e64aaa7701b7a481836cc9b35ac268f752257b9dabf619eb43ecc6e4541fbc6ed50ae1b51d6ee4b4ed60fe4ad2ec302b1f8b8
-  data.tar.gz: 67d54e8449c264dc2fb1dd4883b64a810cabd47a2b976e931f1060d1573d0579c4964336159f6f8acf1d6d599ae3edf4de64b818f08c4c46c963035534264500
+  metadata.gz: 920f49215031fce81eabe5dd4aec50df392d1dfd93cc497eb47aafbb4544ab9defd378faf0bb99d88b0c5c5fbf326e2dd2e77df6f2d8c737589944e174c0e740
+  data.tar.gz: 2c5c2cc235ed70113c7881fb75f46a9962548bd333d589cc73cbcbc5fbeb50f030b5212e2e2bbd16b7b4b3509284aa1b8e04d151b84cfde07780c4da5f9a9e5c

data/README.md

@@ -48,7 +48,6 @@ class PrepareDocumentForDownload < Operations::Task
   starts_with :authorised?
 
   decision :authorised? do
-    inputs :user
     condition { user.can?(:read, data.document) }
 
     if_true :within_download_limits?
@@ -56,7 +55,6 @@ class PrepareDocumentForDownload < Operations::Task
   end
 
   decision :within_download_limits? do
-    inputs :user
     condition { user.within_download_limits? }
 
     if_true :use_filename_scrambler?
@@ -64,7 +62,6 @@ class PrepareDocumentForDownload < Operations::Task
   end
 
   decision :use_filename_scrambler? do
-    inputs :use_filename_scrambler
     condition { use_filename_scrambler }
 
     if_true :scramble_filename
@@ -72,16 +69,11 @@ class PrepareDocumentForDownload < Operations::Task
   end
 
   action :scramble_filename do
-    inputs :document
-
     self.filename = "#{Faker::Lorem.word}#{File.extname(document.filename.to_s)}"
-end
+  end
   go_to :return_filename
 
   result :return_filename do |results|
-    inputs :document
-    optional :filename
-
     results.filename = filename || document.filename.to_s
   end
 end
@@ -158,7 +150,9 @@ go_to :send_invitations
 You can also specify the required and optional data for your action handler using parameters or within the block. `optional` is decorative and helps with documentation. When using the block form, ensure you call `inputs` at the start of the block so that the task fails before doing any meaningful work.
 
 ```ruby 
-action :have_a_party, inputs: [:number_of_guests], optional: [:music] do
+action :have_a_party do 
+  inputs :number_of_guests 
+  optional :music
   self.food = task.buy_some_food_for(number_of_guests)
   self.beer = task.buy_some_beer_for(number_of_guests)
   self.music ||= task.plan_a_party_playlist
@@ -166,8 +160,6 @@ end
 go_to :send_invitations
 ```
 
-Defining state transitions statically with `go_to` ensures that all transitions are known when the operation class is loaded, making the flow easier to understand and analyze. The `go_to` method will automatically associate the transition with the most recently defined action handler.
-
 ### Waiting
 Wait handlers are very similar to decision handlers but only work within [background tasks](#background-operations-and-pauses).  
 
@@ -280,15 +272,17 @@ task = CombineNames.call first_name: "Alice", last_name: "Aardvark"
 task.results[:name] # => Alice Aardvark
 ```
 
-Because handlers are run in the context of the data carrier, you do not have direct access to methods or properties on your task object.  However, the data carrier holds a reference to your task; use `task.do_something` or `task.some_attribute` to access it.  The exception is the `fail_with`, `call` and `start` methods which the data carrier understands (and are intercepted when you are [testing](#testing)). Note that `go_to` has been removed from the data carrier to enforce static state transitions with the `go_to` method.  
+Because handlers are run in the context of the data carrier, you do not have direct access to methods or properties on your task object.  However, the data carrier holds a reference to your task; use `task.do_something` or `task.some_attribute` to access it.  The exception is the `fail_with`, `call` and `start` methods which the data carrier understands (and are intercepted when you are [testing](#testing)). 
 
 Both your task's `data` and its final `results` are stored in the database, so they can be examined later.  The `results` because that's what you're interested in, the `data` as it can be useful for debugging or auditing purposes.  
 
 They are both stored as hashes that are encoded into JSON.  
 
-Instead of using the standard [JSON coder](https://api.rubyonrails.org/v4.2/classes/ActiveModel/Serializers/JSON.html), we use a [GlobalIDSerialiser](/lib/operations/global_id_serialiser.rb).  This uses [ActiveJob::Arguments](https://guides.rubyonrails.org/active_job_basics.html#supported-types-for-arguments) to transform any models into [GlobalIDs](https://github.com/rails/globalid) before storage and convert them back to models upon retrieval.  
+Instead of using the standard [JSON coder](https://api.rubyonrails.org/v4.2/classes/ActiveModel/Serializers/JSON.html), we use a [GlobalIdSerialiser](https://github.com/standard-procedure/global_id_serialiser).  This serialises most data into standard JSON types, as you would expect, but it also takes any [GlobalID::Identification](https://github.com/rails/globalid) objects (which includes all ActiveRecord models) and converts them to a GlobalID string.  Then when the data is deserialised from the database, the GlobalID is converted back into the appropriate model.  
+
+If the original database record was deleted between the time the hash was serialised and when it was retrieved, the `GlobalID::Locator` will fail.  In this case, the deserialised data will contain a `nil` for the value in question.  
 
-If the original database record was deleted between the time the hash was serialised and when it was retrieved, the `GlobalID::Locator` will fail.  With ActiveJob, this means that the job cannot run and is discarded.  For Operations, we attempt to deserialise a second time, returning the GlobalID string instead of the model.  So be aware that when you access `data` or `results` you may receive a string (similar to `"gid://test-app/User/1"`) instead of the models you were expecting.  And the error handling deserialiser is very simple so you may get format changes in some of the data as well.  If serialisation fails you can access the original JSON string as `data.raw_data` or `results[:raw_data]`.  
+Also note that the GlobalIdSerialiser automatically converts all hash keys into symbols (unlike the standard JSON coder which uses strings).  
 
 #### Indexing data and results
 
@@ -301,7 +295,7 @@ For example, you create your task as:
 @alice = User.find 123
 @task = DoSomethingImportant.call user: @alice 
 ```
-There will not be a `TaskParticipant` record with a `context` of "data", `role` of "user" and `participant` of `@alice`.  
+There will be a `TaskParticipant` record with a `context` of "data", `role` of "user" and `participant` of `@alice`.  
 
 Likewise, you can see all the tasks that Alice was involved with using: 
 ```ruby
@@ -386,15 +380,11 @@ class UserRegistration < Operations::Task
   starts_with :create_user 
   
   action :create_user do 
-    inputs :email 
-
     self.user = User.create! email: email
   end
   go_to :send_verification_email
 
   action :send_verification_email do 
-    inputs :user 
-
     UserMailer.with(user: user).verification_email.deliver_later 
   end
   go_to :verified?
@@ -405,8 +395,6 @@ class UserRegistration < Operations::Task
   end 
 
   action :notify_administrator do 
-    inputs :user 
-
     AdminMailer.with(user: user).verification_completed.deliver_later
   end
 end
@@ -424,7 +412,6 @@ class ParallelTasks < Operations::Task
   starts_with :start_sub_tasks 
   
   action :start_sub_tasks do 
-    inputs :number_of_sub_tasks
     self.sub_tasks = (1..number_of_sub_tasks).collect { |i| start LongRunningTask, number: i }
   end
   go_to :do_something_else
@@ -483,6 +470,16 @@ class WaitForSomething < Operations::Task
 end
 ```
 
+#### Zombie tasks
+
+There's a chance that the `Operations::TaskRunnerJob` might get lost - maybe there's a crash in some process and the job does not restart correctly.  As the process for handling background tasks relies on the task "waking up", performing the next action, then queuing up the next task-runner, if the background job does not queue as expected, the task will sit there, waiting forever.  
+
+To monitor for this, every task can be checked to see if it is a `zombie?`.  This means that the current time is more than 3 times the expected delay, compared to the `updated_at` field.  So if the `delay` is set to 1 minute and the task last woke up more than 3 minutes ago, it is classed as a zombie.  
+
+There are two ways to handle zombies.  
+- Manually; add a user interface listing your tasks with a "Restart" button.  The "Restart" button calls `restart` on the task (which internally schedules a new task runner job).
+- Automatically; set up a cron job which calls the `operations:restart_zombie_tasks` rake task.  This rake task searches for zombie jobs and calls `restart` on them.  Note that cron jobs have a minimum resolution of 1 minute so this will cause pauses in tasks with a delay measured in seconds.   Also be aware that a cron job that calls a rake task will load the entire Rails stack as a new process, so be sure that your server has sufficient memory to cope.  If you're using [SolidQueue](https://github.com/rails/solid_queue/), the job runner already sets up a separate "supervisor" process and allows you to define [recurring jobs](https://github.com/rails/solid_queue/#recurring-tasks) with a resolution of 1 second.  This may be a suitable solution, but I've not tried it yet.  
+
 ## Testing
 Because operations are intended to model long, complex, flowcharts of decisions and actions, it can be a pain coming up with the combinations of inputs to test every path through the sequence.  
 
@@ -598,7 +595,6 @@ end
 
 The visualization includes:
 - Color-coded nodes by state type (decisions, actions, wait states, results)
-- Required and optional inputs for each state
 - Transition conditions between states with custom labels when provided
 - Special handling for custom transition blocks
 

data/app/models/operations/task/background.rb

@@ -1,6 +1,11 @@
 module Operations::Task::Background
   extend ActiveSupport::Concern
 
+  included do
+    scope :zombies, -> { zombies_at(Time.now) }
+    scope :zombies_at, ->(time) { where(becomes_zombie_at: ..time) }
+  end
+
   class_methods do
     def delay(value) = @background_delay = value
 
@@ -15,9 +20,15 @@ module Operations::Task::Background
     def timeout_handler = @on_timeout
 
     def with_timeout(data) = data.merge(_execution_timeout: execution_timeout.from_now.utc)
+
+    def restart_zombie_tasks = zombies.find_each { |t| t.restart! }
   end
 
+  def zombie? = Time.now > (updated_at + zombie_delay)
+
   private def background_delay = self.class.background_delay
+  private def zombie_delay = background_delay * 3
+  private def zombie_time = becomes_zombie_at || Time.now
   private def execution_timeout = self.class.execution_timeout
   private def timeout_handler = self.class.timeout_handler
   private def timeout!

data/app/models/operations/task.rb

@@ -9,8 +9,8 @@ module Operations
 
     enum :status, in_progress: 0, waiting: 10, completed: 100, failed: -1
 
-    serialize :data, coder: Operations::GlobalIDSerialiser, type: Hash, default: {}
-    serialize :results, coder: Operations::GlobalIDSerialiser, type: Hash, default: {}
+    serialize :data, coder: GlobalIdSerialiser, type: Hash, default: {}
+    serialize :results, coder: GlobalIdSerialiser, type: Hash, default: {}
 
     has_many :task_participants, class_name: "Operations::TaskParticipant", dependent: :destroy
     after_save :record_participants
@@ -35,9 +35,10 @@ module Operations
     end
 
     def perform_later
-      waiting!
+      update! status: "waiting", becomes_zombie_at: Time.now + zombie_delay
       TaskRunnerJob.set(wait_until: background_delay.from_now).perform_later self
     end
+    alias_method :restart!, :perform_later
 
     def self.call(**)
       build(background: false, **).tap do |task|

data/db/migrate/20250403075414_add_becomes_zombie_at_field.rb

@@ -0,0 +1,6 @@
+class AddBecomesZombieAtField < ActiveRecord::Migration[8.0]
+  def change
+    add_column :operations_tasks, :becomes_zombie_at, :datetime, null: true
+    add_index :operations_tasks, :becomes_zombie_at
+  end
+end

data/lib/operations/version.rb

@@ -1,3 +1,3 @@
 module Operations
-  VERSION = "0.5.0"
+  VERSION = "0.5.2"
 end

data/lib/operations.rb

@@ -1,4 +1,5 @@
 require "ostruct"
+require "global_id_serialiser"
 
 module Operations
   class Error < StandardError
@@ -10,7 +11,6 @@ module Operations
   end
   require "operations/version"
   require "operations/engine"
-  require "operations/global_id_serialiser"
   require "operations/failure"
   require "operations/cannot_wait_in_foreground"
   require "operations/timeout"

data/lib/tasks/operations_tasks.rake

@@ -1,4 +1,4 @@
-# desc "Explaining what the task does"
-# task :operations do
-#   # Task goes here
-# end
+desc "Restart any zombie tasks"
+task :restart_zombie_tasks do
+  Operations::Task.restart_zombie_tasks
+end

metadata

@@ -1,13 +1,13 @@
 --- !ruby/object:Gem::Specification
 name: standard_procedure_operations
 version: !ruby/object:Gem::Version
-  version: 0.5.0
+  version: 0.5.2
 platform: ruby
 authors:
 - Rahoul Baruah
 bindir: bin
 cert_chain: []
-date: 2025-03-10 00:00:00.000000000 Z
+date: 2025-04-03 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rails
@@ -23,6 +23,20 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: 7.1.3
+- !ruby/object:Gem::Dependency
+  name: standard_procedure_global_id_serialiser
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 description: Pipelines and State Machines for composable, trackable business logic
 email:
 - rahoulb@echodek.co
@@ -51,13 +65,13 @@ files:
 - app/models/operations/task_participant.rb
 - config/routes.rb
 - db/migrate/20250127160616_create_operations_tasks.rb
-- db/migrate/20250309_create_operations_task_participants.rb
+- db/migrate/20250309160616_create_operations_task_participants.rb
+- db/migrate/20250403075414_add_becomes_zombie_at_field.rb
 - lib/operations.rb
 - lib/operations/cannot_wait_in_foreground.rb
 - lib/operations/engine.rb
 - lib/operations/exporters/graphviz.rb
 - lib/operations/failure.rb
-- lib/operations/global_id_serialiser.rb
 - lib/operations/matchers.rb
 - lib/operations/no_decision.rb
 - lib/operations/timeout.rb

data/lib/operations/global_id_serialiser.rb

@@ -1,29 +0,0 @@
-module Operations
-  # Serialise and deserialise data to and from JSON
-  # Unlike the standard JSON coder, this coder uses the ActiveJob::Arguments serializer.
-  # This means that if the data contains an ActiveRecord model, it will be serialised as a GlobalID string
-  #
-  # Usage:
-  # class MyModel < ApplicationRecord
-  #   serialize :data, coder: GlobalIDSerialiser, type: Hash, default: {}
-  # end
-  # @my_model = MyModel.create! data: {hello: "world", user: User.first}
-  # puts @my_model[:data] # => {hello: "world", user: #<User id: 1>}
-  class GlobalIDSerialiser
-    def self.dump(data) = ActiveSupport::JSON.dump(ActiveJob::Arguments.serialize([data]))
-
-    def self.load(json)
-      ActiveJob::Arguments.deserialize(ActiveSupport::JSON.decode(json)).first
-    rescue => ex
-      _load_without_global_ids(json).merge exception_message: ex.message, exception_class: ex.class.name, raw_data: json.to_s
-    end
-
-    def self._load_without_global_ids(json)
-      ActiveSupport::JSON.decode(json).first.tap do |hash|
-        hash.delete("_aj_symbol_keys")
-      end.transform_values do |value|
-        (value.is_a?(Hash) && value.key?("_aj_globalid")) ? value["_aj_globalid"] : value
-      end.transform_keys(&:to_sym)
-    end
-  end
-end