acidic_job 0.4.0 → 0.5.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 82e7abf5e57a4a6b68691e3c6c69ee8a6bbc2eabcedb69a3ef29e91ac0e99e8c
-  data.tar.gz: b73584561b8a558529b2e057e449896dedfff2c21ab859dacf82b070f75c7910
+  metadata.gz: e4041a344bd53943f281696cafdd0b0a7fa05969b33ac849475f39efd9333018
+  data.tar.gz: 73a7b1175df2b8e8b7ce6e89cbc13bce208e575b662a31ea03f11ed66fbd0961
 SHA512:
-  metadata.gz: 6e42251e87faff4790ecfba572eca0dd70251fc58a4293b906560b526efa4bc14db00e01ee0e90c43a0d3f3dc355b36b8db3a4be5129529c071711f3a0b2506f
-  data.tar.gz: 167aa9f07dd033a5a3646edcff171a8dffea485d9cbb943c230526214c6d4eaa68761418321cc4e7795a0326361699181bd142377516ceadaa430d286628aea5
+  metadata.gz: bb042126f12c372d0d3767d785d095aa68b48bb71faa10866fa7de6216b47ff7e885ef55611b10cd1d65f9d53aff515a7723fc429ee3095dc3b6ac749e0f761d
+  data.tar.gz: 8cbf5b168781b8ea1125382b998492e6f6d38cf1b46dc8c766b432e93f19e786547dc5dfc8f5437b5623d33970d67d090857abdcd58a5eb5e80804fc682efd36

data/Gemfile

@@ -26,3 +26,5 @@ gem "database_cleaner"
 gem "simplecov"
 
 gem "pry"
+
+gem "sidekiq"

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    acidic_job (0.4.0)
+    acidic_job (0.5.3)
       activerecord (>= 4.0.0)
       activesupport
 
@@ -39,6 +39,7 @@ GEM
     builder (3.2.4)
     coderay (1.1.3)
     concurrent-ruby (1.1.9)
+    connection_pool (2.2.5)
     crass (1.0.6)
     database_cleaner (2.0.1)
       database_cleaner-active_record (~> 2.0.0)
@@ -86,6 +87,7 @@ GEM
       thor (~> 1.0)
     rainbow (3.0.0)
     rake (13.0.4)
+    redis (4.4.0)
     regexp_parser (2.1.1)
     rexml (3.2.5)
     rubocop (1.18.3)
@@ -104,6 +106,10 @@ GEM
     rubocop-rake (0.6.0)
       rubocop (~> 1.0)
     ruby-progressbar (1.11.0)
+    sidekiq (6.2.2)
+      connection_pool (>= 2.2.2)
+      rack (~> 2.0)
+      redis (>= 4.2.0)
     simplecov (0.21.2)
       docile (~> 1.1)
       simplecov-html (~> 0.11)
@@ -133,6 +139,7 @@ DEPENDENCIES
   rubocop (~> 1.7)
   rubocop-minitest
   rubocop-rake
+  sidekiq
   simplecov
   sqlite3
 

data/blog_post.md

@@ -0,0 +1,28 @@
+# ACIDic Operations in Rails
+
+At the conceptual heart of basically any software are "operations"—the discrete actions the software performs. At the horizon of basically any software is the goal to make that sofware _robust_. Typically, one makes a software system robust by making each of its operations robust. Moreover, typically, robustness in software is considered as the software being "ACIDic"—atomic, consistent, isolated, durable.
+
+In a loosely collected series of articles, Brandur Leach lays out the core techniques and principles required to make an HTTP API properly ACIDic:
+
+1. https://brandur.org/acid
+2. https://brandur.org/http-transactions
+3. https://brandur.org/job-drain
+4. https://brandur.org/idempotency-keys
+
+With these techniques and principles in mind, our challenge is bring them into the world of a standard Rails application. This will require us to conceptually map the concepts of an HTTP request, an API server action, and an HTTP response into the world of a running Rails process.
+
+We can begin to make this mapping by observing that an API server action is a specific instantiation of the general concept of an "operation". Like all operations, it has a "trigger" (the HTTP request) and a "response" (the HTTP response). So, what we need is a foundation upon which to build our Rails "operations".
+
+In order to help us find that tool, let us consider the necessary characteristics we need. We need something that we can easily trigger from other Ruby code throughout our Rails application (controller actions, model methods, model callbacks, etc.). It should also be able to be run both synchronously (blocking execution and then returning its response to the caller) and asychronously (non-blocking and the caller doesn't know its response). It should then also be able to retry a specific operation (in much the way that an API consumer can "retry an operation" by hitting the same endpoint with the same request). 
+
+As we lay out these characteristics, I imagine your mind is going where mine went—`ActiveJob` gives us a solid foundation upon which we can build "ACIDic" operations.
+
+So, our challenge to build tooling which will allow us to make "operational" jobs _robust_.
+
+What we need primarily is to be able to make our jobs *idempotent*, and one of the simplest yet still most powerful tools for making an operation idempotent is the idempotency key. As laid out in the article linked above, an idempotency key is a record that we store in our database to uniquely identify a particular execution of an operation and a related "recovery point" for where we are in the process of that operation.
+
+
+
+
+
+

data/lib/acidic_job/errors.rb

@@ -0,0 +1,17 @@
+# frozen_string_literal: true
+
+module AcidicJob
+  class Error < StandardError; end
+
+  class MismatchedIdempotencyKeyAndJobArguments < Error; end
+
+  class LockedIdempotencyKey < Error; end
+
+  class UnknownRecoveryPoint < Error; end
+
+  class UnknownAtomicPhaseType < Error; end
+
+  class SerializedTransactionConflict < Error; end
+
+  class UnknownJobAdapter < Error; end
+end

data/lib/acidic_job/key.rb

@@ -1,5 +1,7 @@
 # frozen_string_literal: true
 
+require "active_record"
+
 module AcidicJob
   class Key < ActiveRecord::Base
     RECOVERY_POINT_FINISHED = "FINISHED"
@@ -8,6 +10,7 @@ module AcidicJob
 
     serialize :error_object
     serialize :job_args
+    store :attr_accessors
 
     validates :idempotency_key, presence: true, uniqueness: { scope: %i[job_name job_args] }
     validates :job_name, presence: true

data/lib/acidic_job/perform_transactionally_extension.rb

@@ -0,0 +1,33 @@
+# frozen_string_literal: true
+
+require "active_support/concern"
+
+module AcidicJob
+  module PerformTransactionallyExtension
+    extend ActiveSupport::Concern
+
+    class_methods do
+      # rubocop:disable Metrics/MethodLength
+      def perform_transactionally(*args)
+        attributes = if self < ActiveJob::Base
+                       {
+                         adapter: "activejob",
+                         job_name: name,
+                         job_args: job_or_instantiate(*args).serialize
+                       }
+                     elsif include? Sidekiq::Worker
+                       {
+                         adapter: "sidekiq",
+                         job_name: name,
+                         job_args: args
+                       }
+                     else
+                       raise UnknownJobAdapter
+                     end
+
+        AcidicJob::Staged.create!(attributes)
+      end
+      # rubocop:enable Metrics/MethodLength
+    end
+  end
+end

data/lib/acidic_job/perform_wrapper.rb

@@ -0,0 +1,21 @@
+# frozen_string_literal: true
+
+module AcidicJob
+  module PerformWrapper
+    def perform(*args, **kwargs)
+      # store arguments passed into `perform` so that we can later persist
+      # them to `AcidicJob::Key#job_args` for both ActiveJob and Sidekiq::Worker
+      @arguments_for_perform = if args.any? && kwargs.any?
+                                 args + [kwargs]
+                               elsif args.any? && kwargs.none?
+                                 args
+                               elsif args.none? && kwargs.any?
+                                 [kwargs]
+                               else
+                                 []
+                               end
+
+      super
+    end
+  end
+end

data/lib/acidic_job/recovery_point.rb

@@ -11,6 +11,7 @@ module AcidicJob
     end
 
     def call(key:)
+      # Skip AR callbacks as there are none on the model
       key.update_column(:recovery_point, name)
     end
   end

data/lib/acidic_job/response.rb

@@ -6,7 +6,8 @@
 module AcidicJob
   class Response
     def call(key:)
-      key.update!(
+      # Skip AR callbacks as there are none on the model
+      key.update_columns(
         locked_at: nil,
         recovery_point: Key::RECOVERY_POINT_FINISHED
       )

data/lib/acidic_job/staged.rb

@@ -0,0 +1,32 @@
+# frozen_string_literal: true
+
+require "active_record"
+
+module AcidicJob
+  class Staged < ActiveRecord::Base
+    self.table_name = "staged_acidic_jobs"
+
+    validates :adapter, presence: true
+    validates :job_name, presence: true
+    validates :job_args, presence: true
+
+    serialize :job_args
+
+    after_create_commit :enqueue_job
+
+    def enqueue_job
+      case adapter
+      when "activejob"
+        job = ActiveJob::Base.deserialize(job_args)
+        job.enqueue
+      when "sidekiq"
+        Sidekiq::Client.push("class" => job_name, "args" => job_args)
+      else
+        raise UnknownJobAdapter.new(adapter: adapter)
+      end
+
+      # TODO: ensure successful enqueuing before deletion
+      delete
+    end
+  end
+end

data/lib/acidic_job/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module AcidicJob
-  VERSION = "0.4.0"
+  VERSION = "0.5.3"
 end

data/lib/acidic_job.rb

@@ -1,45 +1,39 @@
 # frozen_string_literal: true
 
 require_relative "acidic_job/version"
+require_relative "acidic_job/errors"
 require_relative "acidic_job/no_op"
 require_relative "acidic_job/recovery_point"
 require_relative "acidic_job/response"
 require_relative "acidic_job/key"
-require_relative "acidic_job/staging"
+require_relative "acidic_job/staged"
+require_relative "acidic_job/perform_wrapper"
+require_relative "acidic_job/perform_transactionally_extension"
 require "active_support/concern"
 
 # rubocop:disable Metrics/ModuleLength, Metrics/AbcSize, Metrics/MethodLength
 module AcidicJob
-  class MismatchedIdempotencyKeyAndJobArguments < StandardError; end
-
-  class LockedIdempotencyKey < StandardError; end
-
-  class UnknownRecoveryPoint < StandardError; end
-
-  class UnknownAtomicPhaseType < StandardError; end
-
-  class SerializedTransactionConflict < StandardError; end
-
   extend ActiveSupport::Concern
 
-  module ActiveJobExtension
-    extend ActiveSupport::Concern
+  def self.wire_everything_up(klass)
+    klass.attr_reader :key
+    klass.attr_accessor :arguments_for_perform
 
-    class_methods do
-      def perform_transactionally(*args)
-        AcidicJob::Staging.create!(
-          serialized_params: job_or_instantiate(*args).serialize
-        )
-      end
-    end
+    # Extend ActiveJob with `perform_transactionally` class method
+    klass.include PerformTransactionallyExtension
+
+    # Ensure our `perform` method always runs first to gather parameters
+    klass.prepend PerformWrapper
   end
 
   included do
-    attr_reader :key
+    AcidicJob.wire_everything_up(self)
+  end
 
-    # Extend ActiveJob only once it has been loaded
-    ActiveSupport.on_load(:active_job) do
-      send(:include, ActiveJobExtension)
+  class_methods do
+    def inherited(subclass)
+      AcidicJob.wire_everything_up(subclass)
+      super
     end
   end
 
@@ -49,13 +43,8 @@ module AcidicJob
   # this might not happen 100% of the time, so this is a hedge against it.
   IDEMPOTENCY_KEY_LOCK_TIMEOUT = 90
 
-  # &block
-  # &block
+  # takes a block
   def idempotently(with:)
-    # set accessors for each argument passed in to ensure they are available
-    # to the step methods the job will have written
-    define_accessors_for_passed_arguments(with)
-
     # execute the block to gather the info on what phases are defined for this job
     defined_steps = yield
     # [:create_ride_and_audit_record, :create_stripe_charge, :send_receipt]
@@ -71,13 +60,17 @@ module AcidicJob
     # close proximity, one of the two will be aborted by Postgres because we're
     # using a transaction with SERIALIZABLE isolation level. It may not look
     # it, but this code is safe from races.
-    ensure_idempotency_key_record(job_id, defined_steps.first)
+    ensure_idempotency_key_record(idempotency_key_value, defined_steps.first)
 
     # if the key record is already marked as finished, immediately return its result
     return @key.succeeded? if @key.finished?
 
+    # set accessors for each argument passed in to ensure they are available
+    # to the step methods the job will have written
+    define_accessors_for_passed_arguments(with, @key)
+
     # otherwise, we will enter a loop to process each required step of the job
-    100.times do
+    phases.size.times do
       # our `phases` hash uses Symbols for keys
       recovery_point = @key.recovery_point.to_sym
 
@@ -101,10 +94,16 @@ module AcidicJob
     @_steps
   end
 
+  def safely_finish_acidic_job
+    # Short circuits execution by sending execution right to 'finished'.
+    # So, ends the job "successfully"
+    AcidicJob::Response.new
+  end
+
   private
 
   def atomic_phase(key, proc = nil, &block)
-    error = false
+    rescued_error = false
     phase_callable = (proc || block)
 
     begin
@@ -114,17 +113,19 @@ module AcidicJob
         phase_result.call(key: key)
       end
     rescue StandardError => e
-      error = e
+      rescued_error = e
       raise e
     ensure
-      # If we're leaving under an error condition, try to unlock the idempotency
-      # key right away so that another request can try again.
-      begin
-        key.update_columns(locked_at: nil, error_object: error) if error.present?
-      rescue StandardError => e
-        # We're already inside an error condition, so swallow any additional
-        # errors from here and just send them to logs.
-        puts "Failed to unlock key #{key.id} because of #{e}."
+      if rescued_error
+        # If we're leaving under an error condition, try to unlock the idempotency
+        # key right away so that another request can try again.3
+        begin
+          key.update_columns(locked_at: nil, error_object: rescued_error)
+        rescue StandardError => e
+          # We're already inside an error condition, so swallow any additional
+          # errors from here and just send them to logs.
+          puts "Failed to unlock key #{key.id} because of #{e}."
+        end
       end
     end
   end
@@ -136,7 +137,6 @@ module AcidicJob
                       else
                         :serializable
                       end
-    serialized_job_info = serialize
 
     ActiveRecord::Base.transaction(isolation: isolation_level) do
       @key = Key.find_by(idempotency_key: key_val)
@@ -144,7 +144,7 @@ module AcidicJob
       if @key
         # Programs enqueuing multiple jobs with different parameters but the
         # same idempotency key is a bug.
-        raise MismatchedIdempotencyKeyAndJobArguments if @key.job_args != serialized_job_info["arguments"]
+        raise MismatchedIdempotencyKeyAndJobArguments if @key.job_args != @arguments_for_perform
 
         # Only acquire a lock if the key is unlocked or its lock has expired
         # because the original job was long enough ago.
@@ -158,19 +158,31 @@ module AcidicJob
           locked_at: Time.current,
           last_run_at: Time.current,
           recovery_point: first_step,
-          job_name: serialized_job_info["job_class"],
-          job_args: serialized_job_info["arguments"]
+          job_name: self.class.name,
+          job_args: @arguments_for_perform
         )
       end
     end
   end
 
-  def define_accessors_for_passed_arguments(passed_arguments)
+  def define_accessors_for_passed_arguments(passed_arguments, key)
+    # first, ensure that `Key#attr_accessors` is populated with initial values
+    key.update_column(:attr_accessors, passed_arguments)
+
     passed_arguments.each do |accessor, value|
       # the reader method may already be defined
       self.class.attr_reader accessor unless respond_to?(accessor)
       # but we should always update the value to match the current value
       instance_variable_set("@#{accessor}", value)
+      # and we overwrite the setter to ensure any updates to an accessor update the `Key` stored value
+      # Note: we must define the singleton method on the instance to avoid overwriting setters on other
+      # instances of the same class
+      define_singleton_method("#{accessor}=") do |current_value|
+        instance_variable_set("@#{accessor}", current_value)
+        key.attr_accessors[accessor] = current_value
+        key.save!(validate: false)
+        current_value
+      end
     end
 
     true
@@ -193,5 +205,13 @@ module AcidicJob
       end
     end
   end
+
+  def idempotency_key_value
+    return job_id if defined?(job_id) && !job_id.nil?
+    return jid if defined?(jid) && !jid.nil?
+
+    require "securerandom"
+    SecureRandom.hex
+  end
 end
 # rubocop:enable Metrics/ModuleLength, Metrics/AbcSize, Metrics/MethodLength

data/lib/generators/acidic_job_generator.rb

@@ -3,11 +3,10 @@
 require "rails/generators"
 require "rails/generators/active_record"
 
-# This generator adds a migration for the {FriendlyId::History
-# FriendlyId::History} addon.
 class AcidicJobGenerator < ActiveRecord::Generators::Base
-  # ActiveRecord::Generators::Base inherits from Rails::Generators::NamedBase which requires a NAME parameter for the
-  # new table name. Our generator always uses 'acidic_job_keys', so we just set a random name here.
+  # ActiveRecord::Generators::Base inherits from Rails::Generators::NamedBase
+  # which requires a NAME parameter for the new table name.
+  # Our generator always uses "acidic_job_keys", so we just set a random name here.
   argument :name, type: :string, default: "random_name"
 
   source_root File.expand_path("templates", __dir__)
@@ -23,11 +22,16 @@ class AcidicJobGenerator < ActiveRecord::Generators::Base
   end
 
   # Copies the migration template to db/migrate.
-  def copy_files
-    migration_template "migration.rb.erb",
+  def copy_acidic_job_keys_migration_files
+    migration_template "create_acidic_job_keys_migration.rb.erb",
                        "db/migrate/create_acidic_job_keys.rb"
   end
 
+  def copy_staged_acidic_jobs_migration_files
+    migration_template "create_staged_acidic_jobs_migration.rb.erb",
+                       "db/migrate/create_staged_acidic_jobs.rb"
+  end
+
   protected
 
   def migration_class

data/lib/generators/templates/{migration.rb.erb → create_acidic_job_keys_migration.rb.erb}

@@ -8,14 +8,12 @@ class CreateAcidicJobKeys < <%= migration_class %>
       t.datetime :locked_at, null: true
       t.string :recovery_point, null: false
       t.text :error_object
+      t.text :attr_accessors
       t.timestamps
 
       t.index %i[idempotency_key job_name job_args],
         unique: true,
         name: "idx_acidic_job_keys_on_idempotency_key_n_job_name_n_job_args"
-
-    create_table :acidic_job_stagings do |t|
-      t.text :serialized_params, null: false
     end
   end
 end

data/lib/generators/templates/create_staged_acidic_jobs_migration.rb.erb

@@ -0,0 +1,9 @@
+class CreateStagedAcidicJobs < <%= migration_class %>
+  def change
+    create_table :staged_acidic_jobs do |t|
+      t.string :adapter, null: false
+      t.string :job_name, null: false
+      t.text :job_args, null: true
+    end
+  end
+end

data/slides.md

@@ -0,0 +1,65 @@
+# ACIDic Jobs
+
+## A bit about me
+
+- programming in Ruby for 6 years
+- working for test IO / EPAM
+- consulting for RCRDSHP
+- building Smokestack QA on the side
+
+## Jobs are essential
+
+- job / operation / work
+- in every company, with every app, jobs are essential. Why?
+- jobs are what your app *does*, expressed as a distinct unit
+- jobs can be called from anywhere, run sync or async, and have retry mechanisms built-in
+
+## Jobs are internal API endpoints
+
+- Like API endpoints, both are discrete units of work
+- Like API endpoints, we should expect failure
+- Like API endpoints, we should expect retries
+- Like API endpoints, we should expect concurrency
+- this symmetry allows us to port much of the wisdom built up over decades of building robust APIs to our app job infrastructure
+
+## ACIDic APIs
+
+In a loosely collected series of articles, Brandur Leach lays out the core techniques and principles required to make an HTTP API properly ACIDic:
+
+1. https://brandur.org/acid
+2. https://brandur.org/http-transactions
+3. https://brandur.org/job-drain
+4. https://brandur.org/idempotency-keys
+
+His central points can be summarized as follows:
+
+- "ACID databases are one of the most important tools in existence for ensuring maintainability and data correctness in big production systems"
+- "for a common idempotent HTTP request, requests should map to backend transactions at 1:1"
+- "We can dequeue jobs gracefully by using a transactionally-staged job drain."
+- "Implementations that need to make synchronous changes in foreign state (i.e. outside of a local ACID store) are somewhat more difficult to design. ... To guarantee idempotency on this type of endpoint we’ll need to introduce idempotency keys."
+
+Key concepts:
+
+- foreign state mutations
+  - The reason that the local vs. foreign distinction matters is that unlike a local set of operations where we can leverage an ACID store to roll back a result that we didn’t like, once we make our first foreign state mutation, we’re committed one way or another
+- "An atomic phase is a set of local state mutations that occur in transactions between foreign state mutations."
+- "A recovery point is a name of a check point that we get to after having successfully executed any atomic phase or foreign state mutation"
+- "transactionally-staged job drain"
+  - "With this pattern, jobs aren’t immediately sent to the job queue. Instead, they’re staged in a table within the relational database itself, and the ACID properties of the running transaction keep them invisible until they’re ready to be worked. A secondary enqueuer process reads the table and sends any jobs it finds to the job queue before removing their rows."
+
+
+https://github.com/mperham/sidekiq/wiki/Best-Practices#2-make-your-job-idempotent-and-transactional
+
+2. Make your job idempotent and transactional
+
+Idempotency means that your job can safely execute multiple times. For instance, with the error retry functionality, your job might be half-processed, throw an error, and then be re-executed over and over until it successfully completes. Let's say you have a job which voids a credit card transaction and emails the user to let them know the charge has been refunded:
+
+```ruby
+def perform(card_charge_id)
+  charge = CardCharge.find(card_charge_id)
+  charge.void_transaction
+  Emailer.charge_refunded(charge).deliver
+end
+```
+
+What happens when the email fails to render due to a bug? Will the void_transaction method handle the case where a charge has already been refunded? You can use a database transaction to ensure data changes are rolled back if there is an error or you can write your code to be resilient in the face of errors. Just remember that Sidekiq will execute your job at least once, not exactly once.

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: acidic_job
 version: !ruby/object:Gem::Version
-  version: 0.4.0
+  version: 0.5.3
 platform: ruby
 authors:
 - fractaledmind
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2021-09-06 00:00:00.000000000 Z
+date: 2021-10-07 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activerecord
@@ -71,15 +71,21 @@ files:
 - acidic_job.gemspec
 - bin/console
 - bin/setup
+- blog_post.md
 - lib/acidic_job.rb
+- lib/acidic_job/errors.rb
 - lib/acidic_job/key.rb
 - lib/acidic_job/no_op.rb
+- lib/acidic_job/perform_transactionally_extension.rb
+- lib/acidic_job/perform_wrapper.rb
 - lib/acidic_job/recovery_point.rb
 - lib/acidic_job/response.rb
-- lib/acidic_job/staging.rb
+- lib/acidic_job/staged.rb
 - lib/acidic_job/version.rb
 - lib/generators/acidic_job_generator.rb
-- lib/generators/templates/migration.rb.erb
+- lib/generators/templates/create_acidic_job_keys_migration.rb.erb
+- lib/generators/templates/create_staged_acidic_jobs_migration.rb.erb
+- slides.md
 homepage: https://github.com/fractaledmind/acidic_job
 licenses:
 - MIT

data/lib/acidic_job/staging.rb

@@ -1,19 +0,0 @@
-# frozen_string_literal: true
-
-module AcidicJob
-  class Staging < ActiveRecord::Base
-    self.table_name = "acidic_job_stagings"
-
-    validates :serialized_params, presence: true
-
-    serialize :serialized_params
-
-    after_create_commit :enqueue_job
-
-    def enqueue_job
-      job = ActiveJob::Base.deserialize(serialized_params)
-      job.enqueue
-      delete
-    end
-  end
-end