job-workflow 0.2.0 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 9ac9f67bfd9ba71b60f98643a5f5ed720423239650bdf80a8bb07de594d9b502
-  data.tar.gz: 6902b370e54d0285721390a40af5ff0a7b459c6ce0a67af6c538dc1877f0df85
+  metadata.gz: 2a4a82b69372293b7bf689c33125b60b4cbdbcdf4f2e1f468e205cf9bf0c1d85
+  data.tar.gz: e2e6e91167d6127e6cee7e2af0339f79ffee244fe6e768d73e50ebb6c2e7b372
 SHA512:
-  metadata.gz: 6c24f513d3c1192513fef902a92d844050a25e095b2739e9f40e774639cde3fb75c4df1a6e2878a50f13583e5df0e665faa6ef803a434e9444d020a81ec2a71b
-  data.tar.gz: 1c7218d7a1c2d45444a111956bd26d152f16b2e1c6f86fb2959fd255c2ab844444d7929863149f3c7e4322073d520ff40ec3ea02ec29a1595135350723ed4a6a
+  metadata.gz: 84356b092c371e103d601576f9fc6ed5cdee9025340948b0d1fe8222b5b1f49f301342f8254a0d858fc89da58e8f828241d7d8ec706103349e63d6f68a426945
+  data.tar.gz: b146df16edb3112fa10379437e03ebd6a7f74c46e3a83526bb195320e141e9493319caa96d601622cf1e2f5d8cf18b071cc56c93aac1ad39492b89cfa8e08392

data/CHANGELOG.md

@@ -1,5 +1,25 @@
 ## [Unreleased]
 
+## [0.3.0] - 2026-03-13
+
+### Added
+
+- Add `fetch_job_contexts(job_ids)` to queue adapter interface (`Abstract`, `SolidQueueAdapter`, `NullAdapter`) for fetching sub-job context data without direct `SolidQueue::Job` dependency from domain classes
+- Add `persist_job_context(job)` to queue adapter interface for persisting task outputs back to SolidQueue job records after execution
+- Add `without_query_cache` private helper to `SolidQueueAdapter` to bypass ActiveRecord query cache during polling queries
+- Add `"job_workflow_context"` key to `find_job` return hash for direct access to workflow context data
+- Add `AcceptanceNoDependencyWaitJob` and acceptance tests for `depends_on` without `dependency_wait` (polling-only mode)
+- Add acceptance test for output aggregation verification in async workflows
+
+### Changed
+
+- **Breaking (internal):** Replace `Output#update_task_outputs_from_db` and `Output#update_task_outputs_from_jobs` with `Output#update_task_outputs_from_contexts` — callers now pass context data hashes instead of `SolidQueue::Job` objects
+- `Runner#update_task_outputs` now routes through `QueueAdapter.current.fetch_job_contexts` instead of directly querying `SolidQueue::Job`
+- `Runner#run` now calls `QueueAdapter.current.persist_job_context(job)` after both sub-job and workflow execution
+- `WorkflowStatus.from_job_data` now reads `job_workflow_context` from top-level data first, falling back to `arguments.first.dig("job_workflow_context")`
+- `reschedule_solid_queue_job` now saves full serialized job hash (`active_job.serialize.deep_stringify_keys`) instead of only `["arguments"]`
+- Wrap `find_job`, `fetch_job_statuses`, `job_status`, `reschedule_job`, and `fetch_job_contexts` with `without_query_cache` to prevent stale reads under SolidQueue executor
+
 ## [0.2.0] - 2026-03-12
 
 ### Added

data/README.md

@@ -1,6 +1,6 @@
 # JobWorkflow
 
-> ⚠️ **Early Stage (v0.1.3):** This library is in active development. APIs and features may change in breaking ways without notice. Use in production at your own risk and expect potential breaking changes in future releases.
+> ⚠️ **Early Stage (v0.3.0):** This library is in active development. APIs and features may change in breaking ways without notice. Use in production at your own risk and expect potential breaking changes in future releases.
 
 ## Overview
 

data/guides/PRODUCTION_DEPLOYMENT.md

@@ -1,6 +1,6 @@
 # Production Deployment
 
-> ⚠️ **Early Stage (v0.1.3):** JobWorkflow is still in early development. While this section outlines potential deployment patterns, please thoroughly test in your specific environment and monitor for any issues before relying on JobWorkflow in critical production systems.
+> ⚠️ **Early Stage (v0.3.0):** JobWorkflow is still in early development. While this section outlines potential deployment patterns, please thoroughly test in your specific environment and monitor for any issues before relying on JobWorkflow in critical production systems.
 
 This section covers suggested settings and patterns for running JobWorkflow in production-like environments.
 

data/guides/README.md

@@ -1,6 +1,6 @@
 # JobWorkflow Guides
 
-> ⚠️ **Early Stage (v0.1.3):** JobWorkflow is in active development. APIs and features may change. The following guides provide patterns and examples for building workflows, but be aware that implementations may need adjustment as the library evolves.
+> ⚠️ **Early Stage (v0.3.0):** JobWorkflow is in active development. APIs and features may change. The following guides provide patterns and examples for building workflows, but be aware that implementations may need adjustment as the library evolves.
 
 Welcome to the JobWorkflow documentation! This directory contains comprehensive guides to help you build robust workflows with JobWorkflow.
 

data/lib/job_workflow/output.rb

@@ -50,18 +50,10 @@ module JobWorkflow
       task_outputs[task_output.task_name][task_output.each_index] = task_output
     end
 
-    #:  (Array[String], Workflow) -> void
-    def update_task_outputs_from_db(job_ids, workflow)
-      jobs = SolidQueue::Job.where(active_job_id: job_ids)
-      return if jobs.empty?
-
-      update_task_outputs_from_jobs(jobs.to_a, workflow)
-    end
-
-    #:  (Array[SolidQueue::Job], Workflow) -> void
-    def update_task_outputs_from_jobs(jobs, workflow)
-      jobs.each do |job|
-        context = Context.deserialize(job.arguments["job_workflow_context"].merge("workflow" => workflow))
+    #:  (Array[Hash[String, untyped]], Workflow) -> void
+    def update_task_outputs_from_contexts(context_data_list, workflow)
+      context_data_list.each do |context_data|
+        context = Context.deserialize(context_data.merge("workflow" => workflow))
         task_output = context.each_task_output
         next if task_output.nil?
 

data/lib/job_workflow/queue_adapters/abstract.rb

@@ -77,10 +77,18 @@ module JobWorkflow
         raise NotImplementedError, "#{self.class}#find_job must be implemented"
       end
 
+      #:  (Array[String]) -> Array[Hash[String, untyped]]
+      def fetch_job_contexts(_job_ids)
+        raise NotImplementedError, "#{self.class}#fetch_job_contexts must be implemented"
+      end
+
       #:  (DSL, Numeric) -> bool
       def reschedule_job(_job, _wait)
         false
       end
+
+      #:  (DSL) -> void
+      def persist_job_context(_job); end
     end
     # rubocop:enable Naming/PredicateMethod
   end

data/lib/job_workflow/queue_adapters/null_adapter.rb

@@ -90,6 +90,11 @@ module JobWorkflow
         @stored_jobs[job_id]
       end
 
+      #:  (Array[String]) -> Array[Hash[String, untyped]]
+      def fetch_job_contexts(_job_ids)
+        []
+      end
+
       #:  (DSL, Numeric) -> bool
       def reschedule_job(_job, _wait)
         false

data/lib/job_workflow/queue_adapters/solid_queue_adapter.rb

@@ -2,7 +2,7 @@
 
 module JobWorkflow
   module QueueAdapters
-    # rubocop:disable Naming/PredicateMethod
+    # rubocop:disable Naming/PredicateMethod, Metrics/ClassLength
     class SolidQueueAdapter < Abstract
       # @note
       #   - Registry scope: @semaphore_registry is process-scoped (shared across fibers/threads
@@ -74,16 +74,20 @@ module JobWorkflow
       def fetch_job_statuses(job_ids)
         return {} unless defined?(SolidQueue::Job)
 
-        SolidQueue::Job.where(active_job_id: job_ids).index_by(&:active_job_id)
+        without_query_cache do
+          SolidQueue::Job.where(active_job_id: job_ids).index_by(&:active_job_id)
+        end
       end
 
       #:  (untyped) -> Symbol
       def job_status(job)
-        return :failed if job.failed?
-        return :succeeded if job.finished?
-        return :running if job.claimed?
+        without_query_cache do
+          return :failed if job.failed?
+          return :succeeded if job.finished?
+          return :running if job.claimed?
 
-        :pending
+          :pending
+        end
       end
 
       #:  () -> bool
@@ -153,23 +157,40 @@ module JobWorkflow
       def find_job(job_id)
         return unless defined?(SolidQueue::Job)
 
-        job = SolidQueue::Job.find_by(active_job_id: job_id)
+        job = without_query_cache { SolidQueue::Job.find_by(active_job_id: job_id) }
         return if job.nil?
 
+        args = job.arguments
         {
           "job_id" => job.active_job_id,
           "class_name" => job.class_name,
           "queue_name" => job.queue_name,
-          "arguments" => job.arguments.is_a?(Hash) ? job.arguments["arguments"] : job.arguments,
+          "arguments" => args.is_a?(Hash) ? args["arguments"] : args,
+          "job_workflow_context" => args.is_a?(Hash) ? args["job_workflow_context"] : nil,
           "status" => job_status(job)
         }
       end
 
+      # @note
+      #   - Fetches job_workflow_context hashes for the given job IDs.
+      #
+      #:  (Array[String]) -> Array[Hash[String, untyped]]
+      def fetch_job_contexts(job_ids)
+        return [] unless defined?(SolidQueue::Job)
+        return [] if job_ids.empty?
+
+        jobs = without_query_cache { SolidQueue::Job.where(active_job_id: job_ids).to_a }
+        jobs.filter_map do |job|
+          args = job.arguments
+          args.is_a?(Hash) ? args["job_workflow_context"] : nil
+        end
+      end
+
       #:  (DSL, Numeric) -> bool
       def reschedule_job(job, wait)
         return false unless defined?(SolidQueue::Job)
 
-        solid_queue_job = SolidQueue::Job.find_by(active_job_id: job.job_id)
+        solid_queue_job = without_query_cache { SolidQueue::Job.find_by(active_job_id: job.job_id) }
         return false unless solid_queue_job&.claimed?
 
         reschedule_solid_queue_job(solid_queue_job, job, wait)
@@ -177,17 +198,44 @@ module JobWorkflow
         false
       end
 
+      # @note
+      #   - Persists the job's updated context (including task outputs) back
+      #     to the SolidQueue job record after execution completes. Without this,
+      #     outputs computed during job execution would be lost because
+      #     SolidQueue does not re-serialize job arguments after perform.
+      #
+      #:  (DSL) -> void
+      def persist_job_context(job)
+        return unless defined?(SolidQueue::Job)
+
+        solid_queue_job = SolidQueue::Job.find_by(active_job_id: job.job_id)
+        return if solid_queue_job.nil?
+
+        solid_queue_job.update!(arguments: job.serialize.deep_stringify_keys)
+      end
+
       private
 
       attr_reader :semaphore_registry #: Hash[Object, ^(SolidQueue::Worker) -> void]
 
+      # @note
+      #   - Bypasses ActiveRecord query cache for the given block.
+      #   - When running under SolidQueue's executor, SELECT queries are cached
+      #     for the entire job execution. Polling queries must bypass this cache
+      #     to observe status changes made by other threads/processes.
+      #
+      #:  [T] () { () -> T } -> T
+      def without_query_cache(&)
+        defined?(SolidQueue::Job) ? SolidQueue::Job.uncached(&) : yield
+      end
+
       #:  (SolidQueue::Job, DSL, Numeric) -> bool
       def reschedule_solid_queue_job(solid_queue_job, active_job, wait)
         solid_queue_job.with_lock do
           solid_queue_job.claimed_execution&.destroy!
           solid_queue_job.update!(
             scheduled_at: wait.seconds.from_now,
-            arguments: active_job.serialize.deep_stringify_keys["arguments"]
+            arguments: active_job.serialize.deep_stringify_keys
           )
           solid_queue_job.prepare_for_execution
         end
@@ -219,6 +267,6 @@ module JobWorkflow
         end
       end
     end
-    # rubocop:enable Naming/PredicateMethod
+    # rubocop:enable Naming/PredicateMethod, Metrics/ClassLength
   end
 end

data/lib/job_workflow/runner.rb

@@ -13,9 +13,14 @@ module JobWorkflow
     #:  () -> void
     def run
       task = context._task_context.task
-      return run_task(task) if !task.nil? && context.sub_job?
+      if !task.nil? && context.sub_job?
+        run_task(task)
+        QueueAdapter.current.persist_job_context(job)
+        return
+      end
 
       catch(:rescheduled) { run_workflow }
+      QueueAdapter.current.persist_job_context(job)
     end
 
     private
@@ -167,7 +172,8 @@ module JobWorkflow
     #:  (Task) -> void
     def update_task_outputs(task)
       finished_job_ids = context.job_status.finished_job_ids(task_name: task.task_name)
-      context.output.update_task_outputs_from_db(finished_job_ids, context.workflow)
+      context_data_list = QueueAdapter.current.fetch_job_contexts(finished_job_ids)
+      context.output.update_task_outputs_from_contexts(context_data_list, context.workflow)
     end
   end
 end

data/lib/job_workflow/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module JobWorkflow
-  VERSION = "0.2.0" # : String
+  VERSION = "0.3.0" # : String
 end

data/lib/job_workflow/workflow_status.rb

@@ -34,7 +34,7 @@ module JobWorkflow
         job_class = job_class_name.constantize
         workflow = job_class._workflow
 
-        context_data = data["arguments"].first["job_workflow_context"]
+        context_data = data["job_workflow_context"] || data["arguments"]&.first&.dig("job_workflow_context")
         context = if context_data
                     Context.deserialize(context_data.merge("workflow" => workflow))
                   else

data/rbs_collection.lock.yaml

@@ -6,7 +6,7 @@ gems:
   source:
     type: git
     name: ruby/gem_rbs_collection
-    revision: 2af46cde7fec5e4130f6c84c5b691c0538af8789
+    revision: 9bf2eebb1c54b5d6f23f2acb65d4c36f195b4783
     remote: https://github.com/ruby/gem_rbs_collection.git
     repo_dir: gems
 - name: activerecord
@@ -14,7 +14,7 @@ gems:
   source:
     type: git
     name: ruby/gem_rbs_collection
-    revision: 2af46cde7fec5e4130f6c84c5b691c0538af8789
+    revision: 9bf2eebb1c54b5d6f23f2acb65d4c36f195b4783
     remote: https://github.com/ruby/gem_rbs_collection.git
     repo_dir: gems
 - name: activesupport
@@ -22,7 +22,7 @@ gems:
   source:
     type: git
     name: ruby/gem_rbs_collection
-    revision: 2af46cde7fec5e4130f6c84c5b691c0538af8789
+    revision: 9bf2eebb1c54b5d6f23f2acb65d4c36f195b4783
     remote: https://github.com/ruby/gem_rbs_collection.git
     repo_dir: gems
 - name: base64
@@ -34,7 +34,7 @@ gems:
   source:
     type: git
     name: ruby/gem_rbs_collection
-    revision: 2af46cde7fec5e4130f6c84c5b691c0538af8789
+    revision: 9bf2eebb1c54b5d6f23f2acb65d4c36f195b4783
     remote: https://github.com/ruby/gem_rbs_collection.git
     repo_dir: gems
 - name: concurrent-ruby
@@ -42,7 +42,7 @@ gems:
   source:
     type: git
     name: ruby/gem_rbs_collection
-    revision: 2af46cde7fec5e4130f6c84c5b691c0538af8789
+    revision: 9bf2eebb1c54b5d6f23f2acb65d4c36f195b4783
     remote: https://github.com/ruby/gem_rbs_collection.git
     repo_dir: gems
 - name: connection_pool
@@ -50,7 +50,7 @@ gems:
   source:
     type: git
     name: ruby/gem_rbs_collection
-    revision: 2af46cde7fec5e4130f6c84c5b691c0538af8789
+    revision: 9bf2eebb1c54b5d6f23f2acb65d4c36f195b4783
     remote: https://github.com/ruby/gem_rbs_collection.git
     repo_dir: gems
 - name: date
@@ -74,7 +74,7 @@ gems:
   source:
     type: git
     name: ruby/gem_rbs_collection
-    revision: 2af46cde7fec5e4130f6c84c5b691c0538af8789
+    revision: 9bf2eebb1c54b5d6f23f2acb65d4c36f195b4783
     remote: https://github.com/ruby/gem_rbs_collection.git
     repo_dir: gems
 - name: i18n
@@ -82,7 +82,7 @@ gems:
   source:
     type: git
     name: ruby/gem_rbs_collection
-    revision: 2af46cde7fec5e4130f6c84c5b691c0538af8789
+    revision: 9bf2eebb1c54b5d6f23f2acb65d4c36f195b4783
     remote: https://github.com/ruby/gem_rbs_collection.git
     repo_dir: gems
 - name: json
@@ -98,7 +98,7 @@ gems:
   source:
     type: git
     name: ruby/gem_rbs_collection
-    revision: 2af46cde7fec5e4130f6c84c5b691c0538af8789
+    revision: 9bf2eebb1c54b5d6f23f2acb65d4c36f195b4783
     remote: https://github.com/ruby/gem_rbs_collection.git
     repo_dir: gems
 - name: monitor
@@ -138,7 +138,7 @@ gems:
   source:
     type: git
     name: ruby/gem_rbs_collection
-    revision: 2af46cde7fec5e4130f6c84c5b691c0538af8789
+    revision: 9bf2eebb1c54b5d6f23f2acb65d4c36f195b4783
     remote: https://github.com/ruby/gem_rbs_collection.git
     repo_dir: gems
 - name: stringio
@@ -162,7 +162,7 @@ gems:
   source:
     type: git
     name: ruby/gem_rbs_collection
-    revision: 2af46cde7fec5e4130f6c84c5b691c0538af8789
+    revision: 9bf2eebb1c54b5d6f23f2acb65d4c36f195b4783
     remote: https://github.com/ruby/gem_rbs_collection.git
     repo_dir: gems
 - name: uri

data/sig/generated/job_workflow/output.rbs

@@ -23,11 +23,8 @@ module JobWorkflow
     # :  (TaskOutput) -> void
     def add_task_output: (TaskOutput) -> void
 
-    # :  (Array[String], Workflow) -> void
-    def update_task_outputs_from_db: (Array[String], Workflow) -> void
-
-    # :  (Array[SolidQueue::Job], Workflow) -> void
-    def update_task_outputs_from_jobs: (Array[SolidQueue::Job], Workflow) -> void
+    # :  (Array[Hash[String, untyped]], Workflow) -> void
+    def update_task_outputs_from_contexts: (Array[Hash[String, untyped]], Workflow) -> void
 
     # :  () -> Array[TaskOutput]
     def flat_task_outputs: () -> Array[TaskOutput]

data/sig/generated/job_workflow/queue_adapters/abstract.rbs

@@ -49,8 +49,14 @@ module JobWorkflow
       # :  (String) -> Hash[String, untyped]?
       def find_job: (String) -> Hash[String, untyped]?
 
+      # :  (Array[String]) -> Array[Hash[String, untyped]]
+      def fetch_job_contexts: (Array[String]) -> Array[Hash[String, untyped]]
+
       # :  (DSL, Numeric) -> bool
       def reschedule_job: (DSL, Numeric) -> bool
+
+      # :  (DSL) -> void
+      def persist_job_context: (DSL) -> void
     end
   end
 end

data/sig/generated/job_workflow/queue_adapters/null_adapter.rbs

@@ -51,6 +51,9 @@ module JobWorkflow
       # :  (String) -> Hash[String, untyped]?
       def find_job: (String) -> Hash[String, untyped]?
 
+      # :  (Array[String]) -> Array[Hash[String, untyped]]
+      def fetch_job_contexts: (Array[String]) -> Array[Hash[String, untyped]]
+
       # :  (DSL, Numeric) -> bool
       def reschedule_job: (DSL, Numeric) -> bool
 

data/sig/generated/job_workflow/queue_adapters/solid_queue_adapter.rbs

@@ -2,7 +2,7 @@
 
 module JobWorkflow
   module QueueAdapters
-    # rubocop:disable Naming/PredicateMethod
+    # rubocop:disable Naming/PredicateMethod, Metrics/ClassLength
     class SolidQueueAdapter < Abstract
       # @note
       #   - Registry scope: @semaphore_registry is process-scoped (shared across fibers/threads
@@ -82,13 +82,37 @@ module JobWorkflow
       # :  (String) -> Hash[String, untyped]?
       def find_job: (String) -> Hash[String, untyped]?
 
+      # @note
+      #   - Fetches job_workflow_context hashes for the given job IDs.
+      #
+      # :  (Array[String]) -> Array[Hash[String, untyped]]
+      def fetch_job_contexts: (Array[String]) -> Array[Hash[String, untyped]]
+
       # :  (DSL, Numeric) -> bool
       def reschedule_job: (DSL, Numeric) -> bool
 
+      # @note
+      #   - Persists the job's updated context (including task outputs) back
+      #     to the SolidQueue job record after execution completes. Without this,
+      #     outputs computed during job execution would be lost because
+      #     SolidQueue does not re-serialize job arguments after perform.
+      #
+      # :  (DSL) -> void
+      def persist_job_context: (DSL) -> void
+
       private
 
       attr_reader semaphore_registry: Hash[Object, ^(SolidQueue::Worker) -> void]
 
+      # @note
+      #   - Bypasses ActiveRecord query cache for the given block.
+      #   - When running under SolidQueue's executor, SELECT queries are cached
+      #     for the entire job execution. Polling queries must bypass this cache
+      #     to observe status changes made by other threads/processes.
+      #
+      # :  [T] () { () -> T } -> T
+      def without_query_cache: [T] () { () -> T } -> T
+
       # :  (SolidQueue::Job, DSL, Numeric) -> bool
       def reschedule_solid_queue_job: (SolidQueue::Job, DSL, Numeric) -> bool
 

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: job-workflow
 version: !ruby/object:Gem::Version
-  version: 0.2.0
+  version: 0.3.0
 platform: ruby
 authors:
 - shoma07