active_job_store 0.1.1 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: df269c0cadf76b1981c02b87f77ac89c81b4dc13896c5162956942cfe274855f
-  data.tar.gz: 2a864cb0aa3573e77f83d8050cc40ce93988c7100838bce05de73e4db0b99f3b
+  metadata.gz: a9fd49a85e50473450a09a741acaf01d5e6565116cb6f945f314bd6e4dd74f6d
+  data.tar.gz: d5655ec674b4a86d99b267c8a851e2a8d1cd687e188c58c8a292bf5120cffc3b
 SHA512:
-  metadata.gz: 4881d10048838fecb729eb48309aacebd77c4071636f7077fe2e3f2f1f3a1f4a8cce0b90ccad6ae59446790c635880c9ede070b245eb5354b2bb9d8e2dedd02d
-  data.tar.gz: 8245277be9b2e65d7b9479a2f9c9740a3167cca1be20e680530d9dedc9ea6fabb8f6cb597f7a7ecabfb633421ce88362118908faafc10e7f889af2f428d30e49
+  metadata.gz: f9e1a166e181d32afb0265ac062b954930b8d748af36b21e837d2ee6fe28a7d57edc2f9ad590339e49fefbc3256e6b43317d19ab4e9ff1de36e7f2aa67d0dac4
+  data.tar.gz: a407a4446dec032e4bf6f7b5f55536dadd703c0d1d1a75d61cdf37ace3388e09c836c520ffde6c0b31043867bd92c82ad04c57b54d7398ca126baa080715e0a5

data/README.md

@@ -7,14 +7,11 @@
 Persist job execution information on a support model `ActiveJobStore::Record`.
 
 It can be useful to:
-- improve jobs logging capabilities;
-- query historical data about job executions;
-- extract job's statistical data;
-- track a job's state or add custom data to the jobs.
+- store the job's state / set progress value / add custom data to the jobs;
+- query historical data about job executions / extract job's statistical data;
+- improve jobs' logging capabilities.
 
-Support some customizations:
-- set custom data attributes (via `active_job_store_custom_data` accessor);
-- format the job result to store (overriding `active_job_store_format_result` method).
+Please ⭐ if you like it.
 
 ## Installation
 
@@ -24,6 +21,19 @@ Support some customizations:
 - Add to your job `include ActiveJobStore` (or to your `ApplicationJob` class if you prefer)
 - Access to the job executions data using the class method `job_executions` on your job (ex. `YourJob.job_executions`)
 
+## API
+
+attr_accessor on the jobs:
+- `active_job_store_custom_data`: to set / manipulate job's custom data
+
+Instance methods on the jobs:
+- `active_job_store_format_result(result) => result2`: to format / manipulate / serialize the job result
+- `active_job_store_record => store record`: returns the store's record
+- `save_job_custom_data(custom_data = nil)`: to persist custom data while the job is performing
+
+Class methods on the jobs:
+- `job_executions => relation`: query the list of job executions for the specific job class (returns an ActiveRecord Relation)
+
 ## Usage examples
 
 ```rb
@@ -48,6 +58,23 @@ SomeJob.job_executions.first
 #  created_at: Wed, 09 Nov 2022 21:09:50.611900000 UTC +00:00>
 ```
 
+Query jobs in a specific range of time:
+
+```rb
+SomeJob.job_executions.where(started_at: 16.minutes.ago...).pluck(:job_id, :result, :started_at)
+# => [["02beb3d6-a4eb-442c-8d78-29103ab894dc", "some_result", Wed, 09 Nov 2022 21:20:57.576018000 UTC +00:00],
+#  ["267e087e-cfa7-4c88-8d3b-9d40f912733f", "some_result", Wed, 09 Nov 2022 21:13:18.011484000 UTC +00:00]]
+```
+
+Some statistics on completed jobs:
+
+```rb
+SomeJob.job_executions.completed.map { |job| { id: job.id, execution_time: job.completed_at - job.started_at, started_at: job.started_at } }
+# => [{:id=>6, :execution_time=>1.005239, :started_at=>Wed, 09 Nov 2022 21:20:57.576018000 UTC +00:00},
+#  {:id=>4, :execution_time=>1.004485, :started_at=>Wed, 09 Nov 2022 21:13:18.011484000 UTC +00:00},
+#  {:id=>1, :execution_time=>0.011442, :started_at=>Wed, 09 Nov 2022 21:09:50.611355000 UTC +00:00}]
+```
+
 Extract some logs:
 
 ```rb
@@ -60,26 +87,45 @@ puts ::ActiveJobStore::Record.order(id: :desc).pluck(:created_at, :job_class, :a
 # 2022-11-09 21:09:50 UTC, SomeJob, Some test, completed, 2022-11-09 21:09:50 UTC
 ```
 
-Query jobs in a specific range of time:
+Query information from a job (even when it's performing):
 
 ```rb
-SomeJob.job_executions.where(started_at: 16.minutes.ago...).pluck(:job_id, :result, :started_at)
-# => [["02beb3d6-a4eb-442c-8d78-29103ab894dc", "some_result", Wed, 09 Nov 2022 21:20:57.576018000 UTC +00:00],
-#  ["267e087e-cfa7-4c88-8d3b-9d40f912733f", "some_result", Wed, 09 Nov 2022 21:13:18.011484000 UTC +00:00]]
+job = SomeJob.perform_later 123
+job.active_job_store_record.slice(:job_id, :job_class, :arguments)
+# => {"job_id"=>"b009f7c7-a264-4fb5-a1f8-68a8141f323b", "job_class"=>"SomeJob", "arguments"=>[123]}
+
+job = AnotherJob.perform_later 456
+job.active_job_store_record.custom_data
+# => {"progress"=>0.5}
+### After a while:
+job.active_job_store_record.reload.custom_data
+# => {"progress"=>1.0}
 ```
 
-Some statistics on completed jobs:
+## Features' details
+
+To store the custom data (ex. a progress value):
 
 ```rb
-SomeJob.job_executions.completed.map { |job| { id: job.id, execution_time: job.completed_at - job.started_at, started_at: job.started_at } }
-# => [{:id=>6, :execution_time=>1.005239, :started_at=>Wed, 09 Nov 2022 21:20:57.576018000 UTC +00:00},
-#  {:id=>4, :execution_time=>1.004485, :started_at=>Wed, 09 Nov 2022 21:13:18.011484000 UTC +00:00},
-#  {:id=>1, :execution_time=>0.011442, :started_at=>Wed, 09 Nov 2022 21:09:50.611355000 UTC +00:00}]
-```
+class AnotherJob < ApplicationJob
+  include ActiveJobStore
+
+  def perform
+    # do something...
+    save_job_custom_data(progress: 0.5)
+    # do something else...
+    save_job_custom_data(progress: 1.0)
 
-## Customizations
+    'some_result'
+  end
+end
+
+# Usage example:
+AnotherJob.perform_later(456)
+AnotherJob.job_executions.last.custom_data['progress'] # 1.0 (at the end)
+```
 
-If you need to store custom data, use `active_job_store_custom_data` accessor:
+To manipulate the custom data, there is the `active_job_store_custom_data` accessor:
 
 ```rb
 class AnotherJob < ApplicationJob
@@ -96,12 +142,13 @@ class AnotherJob < ApplicationJob
   end
 end
 
+# Usage example:
 AnotherJob.perform_now(123)
 AnotherJob.job_executions.last.custom_data
 # => [{"time"=>"2022-11-09T21:20:57.580Z", "message"=>"SomeJob step 1"}, {"time"=>"2022-11-09T21:20:58.581Z", "message"=>"SomeJob step 2"}]
 ```
 
-If for any reason it's needed to process the result before storing it, just override `active_job_store_format_result`:
+To process the result before storing it (ex. for serialization), override `active_job_store_format_result`:
 
 ```rb
 class AnotherJob < ApplicationJob
@@ -116,6 +163,7 @@ class AnotherJob < ApplicationJob
   end
 end
 
+# Usage example:
 AnotherJob.perform_now(123)
 AnotherJob.job_executions.last.result
 # => 84

data/lib/active_job_store/store.rb

@@ -0,0 +1,76 @@
+# frozen_string_literal: true
+
+module ActiveJobStore
+  class Store
+    DETAILS_ATTRS = %w[exception_executions executions priority queue_name scheduled_at timezone].freeze
+
+    attr_reader :record
+
+    def around_enqueue(job)
+      prepare_record_on_enqueue(job)
+      record.lock! # NOTE: needed to avoid update conflicts with perform when setting the state to enqueued
+      yield
+      job_enqueued!
+    end
+
+    def around_perform(job)
+      prepare_record_on_perform(job)
+      job_started!
+      result = yield
+      formatted_result = job.active_job_store_format_result(result)
+      job_competed!(custom_data: job.active_job_store_custom_data, result: formatted_result)
+    rescue StandardError => e
+      job_failed!(exception: e, custom_data: job.active_job_store_custom_data)
+      raise
+    end
+
+    def update_job_custom_data(custom_data)
+      record.update!(custom_data: custom_data)
+    end
+
+    private
+
+    def job_competed!(result:, custom_data:)
+      record.update!(state: :completed, completed_at: Time.current, result: result, custom_data: custom_data)
+      record
+    end
+
+    def job_enqueued!
+      record.update!(state: :enqueued, enqueued_at: Time.current)
+      record
+    end
+
+    def job_failed!(exception:, custom_data:)
+      record.update!(state: :error, exception: exception.inspect, custom_data: custom_data)
+      record
+    end
+
+    def job_started!
+      record.update!(state: :started, started_at: Time.current)
+      record
+    end
+
+    def prepare_record_on_enqueue(job)
+      @record = ::ActiveJobStore::Record.find_or_create_by!(record_reference(job)) do |record|
+        record.arguments = job.arguments
+        record.details = DETAILS_ATTRS.zip(DETAILS_ATTRS.map { job.send(_1) }).to_h
+        record.state = :initialized
+      end
+    end
+
+    def prepare_record_on_perform(job)
+      @record = ::ActiveJobStore::Record.find_or_initialize_by(record_reference(job)) do |record|
+        record.arguments = job.arguments
+      end
+      record.details = DETAILS_ATTRS.zip(DETAILS_ATTRS.map { job.send(_1) }).to_h
+      record
+    end
+
+    def record_reference(job)
+      {
+        job_id: job.job_id,
+        job_class: job.class.to_s
+      }
+    end
+  end
+end

data/lib/active_job_store/version.rb

@@ -2,6 +2,6 @@
 
 # :nocov:
 module ActiveJobStore
-  VERSION = '0.1.1'
+  VERSION = '0.3.0'
 end
 # :nocov:

data/lib/active_job_store.rb

@@ -1,58 +1,62 @@
 # frozen_string_literal: true
 
 require_relative 'active_job_store/engine'
+require_relative 'active_job_store/store'
 
 module ActiveJobStore
-  IGNORE_ATTRS = %w[arguments job_id successfully_enqueued].freeze
-
+  # Set / manipulate job's custom data
   attr_accessor :active_job_store_custom_data
 
+  # Format / manipulate / serialize the job result
+  #
+  # @param result [any] Job's return value
+  #
+  # @return [any] Processed job's return value
+  def active_job_store_format_result(result)
+    result
+  end
+
+  # Persist custom data while the job is performing
+  #
+  # @param custom_data [any] Attributes to serialize (it must be serializable in JSON)
+  def save_job_custom_data(custom_data = nil)
+    self.active_job_store_custom_data = custom_data if custom_data
+    store.update_job_custom_data(active_job_store_custom_data)
+  end
+
+  # Return the associated Active Job Store record
+  #
+  # @return [ActiveJobStore::Record] the corresponding record
+  def active_job_store_record
+    store.record
+  end
+
+  module ClassMethods
+    # Query the list of job executions for the current job class
+    #
+    # @return [ActiveRecord Relation] query result
+    def job_executions
+      ::ActiveJobStore::Record.where(job_class: to_s)
+    end
+  end
+
   class << self
     def included(base)
       base.extend(ClassMethods)
 
       base.around_enqueue do |job, block|
-        store_record = ::ActiveJobStore::Record.find_or_create_by!(job.active_job_store_reference) do |record|
-          record.arguments = job.arguments
-          record.details = job.as_json.except(*IGNORE_ATTRS)
-          record.state = :initialized
-        end
-        store_record.lock! # NOTE: needed to avoid update conflicts with perform when setting the state to enqueued
-        block.call
-        store_record.update!(state: :enqueued, enqueued_at: Time.current)
+        store.around_enqueue(job) { block.call }
       end
 
       base.around_perform do |job, block|
-        store_record = ::ActiveJobStore::Record.find_or_initialize_by(job.active_job_store_reference) do |record|
-          record.arguments = job.arguments
-        end
-        store_record.update!(details: job.as_json.except(*IGNORE_ATTRS), state: :started, started_at: Time.current)
-        result = block.call
-        formatted_result = job.active_job_store_format_result(result)
-        store_record.update!(
-          state: :completed,
-          completed_at: Time.current,
-          result: formatted_result,
-          custom_data: active_job_store_custom_data
-        )
-      rescue StandardError => e
-        store_record.update!(state: :error, exception: e.inspect, custom_data: active_job_store_custom_data)
-        raise
+        store.around_perform(job) { block.call }
       end
     end
   end
 
-  def active_job_store_format_result(result)
-    result
-  end
-
-  def active_job_store_reference
-    { job_id: job_id, job_class: self.class.to_s }
-  end
+  private
 
-  module ClassMethods
-    def job_executions
-      ::ActiveJobStore::Record.where(job_class: to_s)
-    end
+  def store
+    @store ||= ::ActiveJobStore::Store.new
   end
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: active_job_store
 version: !ruby/object:Gem::Version
-  version: 0.1.1
+  version: 0.3.0
 platform: ruby
 authors:
 - Mattia Roccoberton
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2022-11-11 00:00:00.000000000 Z
+date: 2022-11-15 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activejob
@@ -24,6 +24,20 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '6.0'
+- !ruby/object:Gem::Dependency
+  name: activerecord
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '6.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '6.0'
 description: ActiveJob Store permits to store jobs state and custom data on a database
 email: mat@blocknot.es
 executables: []
@@ -36,6 +50,7 @@ files:
 - db/migrate/20221101010101_create_active_job_store.rb
 - lib/active_job_store.rb
 - lib/active_job_store/engine.rb
+- lib/active_job_store/store.rb
 - lib/active_job_store/version.rb
 homepage: https://github.com/blocknotes/active_job_store
 licenses: