job-workflow 0.3.0 → 0.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 2a4a82b69372293b7bf689c33125b60b4cbdbcdf4f2e1f468e205cf9bf0c1d85
-  data.tar.gz: e2e6e91167d6127e6cee7e2af0339f79ffee244fe6e768d73e50ebb6c2e7b372
+  metadata.gz: 9474c65583cf21d485c3468675d4f23124e60e88f98c954bd38256b916d98ffe
+  data.tar.gz: 59d5e0b60a2b52e87ea4da4aab1834d02c63f88cba36f661acd8c74f89a4e6a9
 SHA512:
-  metadata.gz: 84356b092c371e103d601576f9fc6ed5cdee9025340948b0d1fe8222b5b1f49f301342f8254a0d858fc89da58e8f828241d7d8ec706103349e63d6f68a426945
-  data.tar.gz: b146df16edb3112fa10379437e03ebd6a7f74c46e3a83526bb195320e141e9493319caa96d601622cf1e2f5d8cf18b071cc56c93aac1ad39492b89cfa8e08392
+  metadata.gz: 4ac034e1b373b28aa52dcffd059bc5b64e5c45fc04d9b07460c24491c2514eabddf839b2dcb2c4a2cfbf36b578e2836e153e9bd6f34cd24f8af543cf06242ce9
+  data.tar.gz: 455b6913137103b6a5871562a6adc0ffca11e9e99543b94ec7e0bda9f2b78c6c76c5cee73baec30ca5b5b5e2fd388d6a4b119c08d7cf1039c20e115e255f58d7

data/CHANGELOG.md

@@ -1,5 +1,26 @@
 ## [Unreleased]
 
+## [0.5.0] - 2026-05-13
+
+### Added
+
+- Add task-scoped cursor helpers on `Context` so tasks can persist progress, restore task-local continuation state, and delegate explicit checkpoints without exposing ActiveJob continuation internals
+
+### Fixed
+
+- Initialize the SolidQueue adapter from a Railtie after Rails boot so the `ClaimedExecution` patch is applied reliably, rescheduled `dependency_wait` jobs are not marked finished early, and Rails apps no longer need a manual initializer
+
+## [0.4.0] - 2026-05-12
+
+### Fixed
+
+- Fix `enqueue: true` consumer tasks that read `depends_on` outputs by persisting the parent context before `perform_all_later`, so sub-jobs load the latest dependency outputs instead of stale persisted state
+- Fix sequential `each` task resumption to persist continuation cursor progress at the workflow step level, resume from `step.cursor`, and keep the failed iteration retrying from the same `each_index` without skipping previously collected outputs
+
+### Removed
+
+- Remove the `namespace` DSL and `JobWorkflow::Namespace`, flatten task naming to a single task space, and delete the dedicated namespace guide/example coverage that existed for the removed feature
+
 ## [0.3.0] - 2026-03-13
 
 ### Added

data/README.md

@@ -1,6 +1,6 @@
 # JobWorkflow
 
-> ⚠️ **Early Stage (v0.3.0):** This library is in active development. APIs and features may change in breaking ways without notice. Use in production at your own risk and expect potential breaking changes in future releases.
+> ⚠️ **Early Stage (v0.5.0):** This library is in active development. APIs and features may change in breaking ways without notice. Use in production at your own risk and expect potential breaking changes in future releases.
 
 ## Overview
 

data/guides/API_REFERENCE.md

@@ -84,6 +84,76 @@ end
 
 **Map Task Output**: When `each:` is specified, outputs are automatically collected as an array.
 
+### Task continuation helpers
+
+Inside a task body, you can read the current task cursor, store a new cursor, and create interruption points through the task context.
+
+#### Regular task example
+
+```ruby
+task :sync_pages, output: { processed: "Integer" } do |ctx|
+  page = ctx.cursor || 1
+  result = ExternalAPI.fetch(page:)
+
+  ctx.set_cursor!(page + 1) if result.next_page?
+
+  { processed: result.items.size }
+end
+```
+
+- `ctx.cursor` returns the current task cursor, or `nil` when no cursor has been stored
+- `ctx.set_cursor!(value)` validates that `value` is ActiveJob-serializable, stores it in the current continuation step, and checkpoints the job through Active Job continuation
+- `ctx.checkpoint!` creates a checkpoint without changing the public cursor value
+- Call `ctx.set_cursor!` when you want to change the public cursor value and create a checkpoint at the same time
+- Call `ctx.checkpoint!` when you want the current task execution to become interruptible without changing the public cursor value
+- Outside task execution, `ctx.cursor` returns `nil`, and `ctx.set_cursor!` / `ctx.checkpoint!` raise an error
+
+For regular tasks, a cursor is only persisted when you call `ctx.set_cursor!(value)` explicitly.
+
+#### Checkpoint without changing the cursor
+
+```ruby
+task :publish_report do |ctx|
+  report = build_report
+  ctx.checkpoint!
+  deliver_report(report)
+end
+```
+
+#### Repeating work inside a task
+
+```ruby
+task :sync_users do |ctx|
+  start_index = ctx.cursor || 0
+
+  ctx.arguments.user_ids.drop(start_index).each_with_index do |user_id, offset|
+    sync_user(user_id)
+    ctx.set_cursor!(start_index + offset + 1)
+  end
+end
+```
+
+This pattern is useful when a single task iterates over an Enumerable internally and you want to resume from the last completed item after an interruption.
+
+#### `each:` task example
+
+```ruby
+task :sync_users, each: ->(ctx) { ctx.arguments.user_ids } do |ctx|
+  next_cursor = ExternalAPI.sync_user(
+    user_id: ctx.each_value,
+    cursor: ctx.cursor
+  )
+
+  ctx.set_cursor!(next_cursor) unless next_cursor.nil?
+end
+```
+
+For `each:` tasks, JobWorkflow keeps the existing integer resume behavior for completed iterations.
+
+- If an `each:` task is interrupted after calling `ctx.set_cursor!`, JobWorkflow resumes with both the current iteration index and the saved task cursor
+- If an iteration completes normally, the resume state advances to the next integer index
+- In other words, a custom cursor in an `each:` task is meant for resuming work inside the current iteration, not for replacing the completion index of finished iterations
+
 ### workflow_concurrency
 
 Configure job-level concurrency limits with workflow-aware context.

data/guides/PRODUCTION_DEPLOYMENT.md

@@ -1,6 +1,6 @@
 # Production Deployment
 
-> ⚠️ **Early Stage (v0.3.0):** JobWorkflow is still in early development. While this section outlines potential deployment patterns, please thoroughly test in your specific environment and monitor for any issues before relying on JobWorkflow in critical production systems.
+> ⚠️ **Early Stage (v0.5.0):** JobWorkflow is still in early development. While this section outlines potential deployment patterns, please thoroughly test in your specific environment and monitor for any issues before relying on JobWorkflow in critical production systems.
 
 This section covers suggested settings and patterns for running JobWorkflow in production-like environments.
 

data/guides/README.md

@@ -1,6 +1,6 @@
 # JobWorkflow Guides
 
-> ⚠️ **Early Stage (v0.3.0):** JobWorkflow is in active development. APIs and features may change. The following guides provide patterns and examples for building workflows, but be aware that implementations may need adjustment as the library evolves.
+> ⚠️ **Early Stage (v0.5.0):** JobWorkflow is in active development. APIs and features may change. The following guides provide patterns and examples for building workflows, but be aware that implementations may need adjustment as the library evolves.
 
 Welcome to the JobWorkflow documentation! This directory contains comprehensive guides to help you build robust workflows with JobWorkflow.
 
@@ -67,11 +67,6 @@ Power features for complex workflows:
   - Configuration options (poll_timeout, poll_interval, reschedule_delay)
   - SolidQueue integration
 
-- **[NAMESPACES.md](NAMESPACES.md)** - Organizing large workflows
-  - Basic namespace usage
-  - Nested namespaces
-  - Cross-namespace dependencies
-
 - **[THROTTLING.md](THROTTLING.md)** - Rate limiting and resource control
   - Task-level throttling
   - Runtime throttling

data/lib/job_workflow/context.rb

@@ -2,6 +2,8 @@
 
 module JobWorkflow
   class Context # rubocop:disable Metrics/ClassLength
+    EACH_TASK_CURSOR_MARKER = "__job_workflow_each_cursor__"
+
     attr_reader :workflow #: Workflow
     attr_reader :arguments #: Arguments
     attr_reader :output #: Output
@@ -52,7 +54,7 @@ module JobWorkflow
     #     job_status: JobStatus,
     #     ?job: DSL?
     #   ) -> void
-    def initialize(workflow:, arguments:, task_context:, output:, job_status:, job: nil) # rubocop:disable Metrics/ParameterLists
+    def initialize(workflow:, arguments:, task_context:, output:, job_status:, job: nil) # rubocop:disable Metrics/ParameterLists, Metrics/AbcSize, Metrics/MethodLength
       raise "job does not match the provided workflow" if job&.then { |j| j.class._workflow != workflow }
 
       self.job = job
@@ -64,6 +66,8 @@ module JobWorkflow
       self.enabled_with_each_value = false
       self.throttle_index = 0
       self.skip_in_dry_run_index = 0
+      self.current_step = nil
+      self.current_cursor = nil
     end
 
     #:  () -> Hash[String, untyped]
@@ -77,6 +81,31 @@ module JobWorkflow
       self
     end
 
+    #:  () -> untyped
+    def cursor
+      return if current_step.nil?
+
+      current_cursor
+    end
+
+    #:  (untyped) -> void
+    def set_cursor!(value)
+      step = current_step || (raise "set_cursor! can be called only in task")
+
+      ActiveJob::Arguments.serialize([value])
+      self.current_cursor = value
+      step.set!(build_step_cursor(value))
+    end
+
+    #:  () -> void
+    def checkpoint!
+      step = current_step || (raise "checkpoint! can be called only in task")
+
+      return step.checkpoint! unless each_task?
+
+      step.set!(build_step_cursor(current_cursor))
+    end
+
     #:  (DSL) -> void
     def _job=(job)
       self.job = job
@@ -108,13 +137,13 @@ module JobWorkflow
       [task_context.parent_job_id, task.task_name].compact.join("/")
     end
 
-    #:  (Task) -> Enumerator[Context]
-    def _with_each_value(task)
+    #:  (Task, ?start_index: Integer?) -> Enumerator[Context]
+    def _with_each_value(task, start_index: nil)
       raise "Nested _with_each_value calls are not allowed" if enabled_with_each_value
 
       self.enabled_with_each_value = true
       Enumerator.new do |y|
-        with_task_context(task, y)
+        with_task_context(task, y, start_index:)
       ensure
         self.enabled_with_each_value = false
       end
@@ -220,6 +249,18 @@ module JobWorkflow
       task_context
     end
 
+    #:  (ActiveJob::Continuation::Step, ?cursor: untyped) { () -> void } -> void
+    def _with_current_step(step, cursor: nil)
+      previous_step = current_step
+      previous_cursor = current_cursor
+      self.current_step = step
+      self.current_cursor = cursor
+      yield
+    ensure
+      self.current_step = previous_step
+      self.current_cursor = previous_cursor
+    end
+
     #:  (TaskOutput) -> void
     def _add_task_output(task_output)
       output.add_task_output(task_output)
@@ -245,12 +286,31 @@ module JobWorkflow
     attr_accessor :enabled_with_each_value #: bool
     attr_accessor :throttle_index #: Integer
     attr_accessor :skip_in_dry_run_index #: Integer
+    attr_accessor :current_step #: ActiveJob::Continuation::Step?
+    attr_accessor :current_cursor #: untyped
 
     #:  () -> String
     def parent_job_id
       _task_context.parent_job_id || job_id
     end
 
+    #:  () -> bool
+    def each_task?
+      task_context.task.each?
+    end
+
+    #:  (untyped) -> untyped
+    def build_step_cursor(value)
+      return value unless each_task?
+      return task_context.index if value.nil?
+
+      {
+        EACH_TASK_CURSOR_MARKER => true,
+        "index" => task_context.index,
+        "cursor" => value
+      }
+    end
+
     #:  () -> Hash[String, untyped]
     def serialize_for_job
       {
@@ -270,11 +330,11 @@ module JobWorkflow
       }
     end
 
-    #:  (Task, Enumerator::Yielder) -> void
-    def with_task_context(task, yielder) # rubocop:disable Metrics/MethodLength
+    #:  (Task, Enumerator::Yielder, ?start_index: Integer?) -> void
+    def with_task_context(task, yielder, start_index: nil) # rubocop:disable Metrics/MethodLength
       reset_task_context_if_task_changed(task)
 
-      with_each_index_and_value(task) do |value, index|
+      with_each_index_and_value(task, start_index:) do |value, index|
         dry_run = calculate_dry_run(task)
         with_retry(task) do |retry_count|
           self.task_context = TaskContext.new(task:, parent_job_id:, index:, value:, retry_count:, dry_run:)
@@ -294,10 +354,12 @@ module JobWorkflow
       self.task_context = TaskContext.new if task_context.task&.task_name != task.task_name
     end
 
-    #:  (Task) { (untyped, Integer) -> void } -> void
-    def with_each_index_and_value(task)
+    #:  (Task, ?start_index: Integer?) { (untyped, Integer) -> void } -> void
+    def with_each_index_and_value(task, start_index: nil)
+      resume_index = start_index || task_context.index
+
       task.each.call(self).each.with_index do |value, index|
-        next if index < task_context.index
+        next if index < resume_index
 
         yield value, index
 

data/lib/job_workflow/dsl.rb

@@ -108,15 +108,9 @@ module JobWorkflow
 
       #:  (Symbol argument_name, String type, ?default: untyped) -> void
       def argument(argument_name, type, default: nil)
-        validate_namespace!
         _workflow.add_argument(ArgumentDef.new(name: argument_name, type:, default:))
       end
 
-      #:  (Symbol) { () -> void } -> void
-      def namespace(namespace_name, &)
-        _workflow.add_namespace(Namespace.new(name: namespace_name), &)
-      end
-
       # rubocop:disable Metrics/ParameterLists
       #
       #:  (
@@ -134,7 +128,7 @@ module JobWorkflow
       #   ) { (untyped) -> void } -> void
       def task(
         task_name,
-        each: ->(_ctx) { [TaskContext::NULL_VALUE] },
+        each: Task::DEFAULT_EACH,
         enqueue: nil,
         retry: 0,
         output: {},
@@ -149,7 +143,6 @@ module JobWorkflow
         new_task = Task.new(
           job_name: name,
           name: task_name,
-          namespace: _workflow.namespace,
           block: block,
           enqueue:,
           each:,
@@ -172,25 +165,21 @@ module JobWorkflow
 
       #:  (*Symbol) { (Context) -> void } -> void
       def before(*task_names, &block)
-        validate_namespace!
         _workflow.add_hook(:before, task_names:, block:)
       end
 
       #:  (*Symbol) { (Context) -> void } -> void
       def after(*task_names, &block)
-        validate_namespace!
         _workflow.add_hook(:after, task_names:, block:)
       end
 
       #:  (*Symbol) { (Context, TaskCallable) -> void } -> void
       def around(*task_names, &block)
-        validate_namespace!
         _workflow.add_hook(:around, task_names:, block:)
       end
 
       #:  (*Symbol) { (Context, StandardError, Task) -> void } -> void
       def on_error(*task_names, &block)
-        validate_namespace!
         _workflow.add_hook(:error, task_names:, block:)
       end
 
@@ -240,7 +229,6 @@ module JobWorkflow
 
       #:  (?bool) ?{ (Context) -> bool } -> void
       def dry_run(value = nil, &block)
-        validate_namespace!
         _workflow.dry_run_config = block || value
       end
 
@@ -254,7 +242,6 @@ module JobWorkflow
       #     ?description: String?
       #   ) -> void
       def schedule(expression, key: nil, queue: nil, priority: nil, args: {}, description: nil)
-        validate_namespace!
         _workflow.add_schedule(
           Schedule.new(
             expression:,
@@ -268,13 +255,6 @@ module JobWorkflow
         )
       end
       # rubocop:enable Metrics/ParameterLists
-
-      private
-
-      #:  () -> void
-      def validate_namespace!
-        raise "cannot be defined within a namespace." unless _workflow.namespace.default?
-      end
     end
   end
 end

data/lib/job_workflow/railtie.rb

@@ -0,0 +1,10 @@
+# frozen_string_literal: true
+
+module JobWorkflow
+  class Railtie < Rails::Railtie
+    config.after_initialize do
+      JobWorkflow::QueueAdapter.reset!
+      JobWorkflow::QueueAdapter.current.initialize_adapter!
+    end
+  end
+end

data/lib/job_workflow/runner.rb

@@ -14,13 +14,13 @@ module JobWorkflow
     def run
       task = context._task_context.task
       if !task.nil? && context.sub_job?
-        run_task(task)
-        QueueAdapter.current.persist_job_context(job)
+        job.step(task.task_name) { |step| run_task(task, step:) }
+        persist_current_job_context
         return
       end
 
       catch(:rescheduled) { run_workflow }
-      QueueAdapter.current.persist_job_context(job)
+      persist_current_job_context
     end
 
     private
@@ -50,7 +50,7 @@ module JobWorkflow
 
           job.step(task.task_name) do |step|
             wait_for_dependent_tasks(task, step)
-            task.enqueue.should_enqueue?(context) ? enqueue_task(task) : run_task(task)
+            task.enqueue.should_enqueue?(context) ? enqueue_task(task) : run_task(task, step:)
           end
         end
       end
@@ -63,24 +63,46 @@ module JobWorkflow
       result
     end
 
-    #:  (Task) -> void
-    def run_task(task)
+    #:  (Task, step: ActiveJob::Continuation::Step) -> void
+    def run_task(task, step:) # rubocop:disable Metrics/AbcSize, Metrics/MethodLength
       context._load_parent_task_output
-      context._with_each_value(task).each do |ctx|
-        run_each_task(task, ctx)
+      start_index, task_cursor = decode_task_cursor(task, step.cursor)
+
+      context._with_each_value(task, start_index:).each do |ctx|
+        iteration_cursor = task_cursor
+        iteration_cursor = nil if task.each? && (task_cursor.nil? || start_index != ctx._task_context.index)
+
+        run_each_task(task, ctx, step:, cursor: iteration_cursor)
+        step.set!(ctx._task_context.index + 1) if task.each?
       rescue StandardError => e
         run_error_hooks(task, ctx, e)
         raise
       end
     end
 
-    #:  (Task, Context) -> void
-    def run_each_task(task, ctx)
+    #:  (Task, untyped) -> [Integer?, untyped]
+    def decode_task_cursor(task, task_cursor)
+      return [nil, task_cursor] unless task.each?
+      return [task_cursor, nil] if task_cursor.is_a?(Integer)
+
+      if task_cursor.is_a?(Hash) && task_cursor[Context::EACH_TASK_CURSOR_MARKER]
+        return [task_cursor.fetch("index"), task_cursor.fetch("cursor")]
+      end
+
+      raise "invalid each task cursor: #{task_cursor.inspect}" unless task_cursor.nil?
+
+      [nil, nil]
+    end
+
+    #:  (Task, Context, step: ActiveJob::Continuation::Step, ?cursor: untyped) -> void
+    def run_each_task(task, ctx, step:, cursor: nil)
       Instrumentation.instrument_task(job, task, ctx) do
-        ctx._with_task_throttle do
-          run_hooks(task, ctx) do
-            data = task.block.call(ctx)
-            add_task_output(ctx:, task:, each_index: ctx._task_context.index, data:)
+        ctx._with_current_step(step, cursor:) do
+          ctx._with_task_throttle do
+            run_hooks(task, ctx) do
+              data = task.block.call(ctx)
+              add_task_output(ctx:, task:, each_index: ctx._task_context.index, data:)
+            end
           end
         end
       end
@@ -112,6 +134,7 @@ module JobWorkflow
     #:  (Task) -> void
     def enqueue_task(task)
       sub_jobs = context._with_each_value(task).map { |ctx| job.class.from_context(ctx) }
+      persist_current_job_context
       ActiveJob.perform_all_later(sub_jobs)
       context.job_status.update_task_job_statuses_from_jobs(task_name: task.task_name, jobs: sub_jobs)
       Instrumentation.notify_task_enqueue(job, task, sub_jobs.size)
@@ -175,5 +198,10 @@ module JobWorkflow
       context_data_list = QueueAdapter.current.fetch_job_contexts(finished_job_ids)
       context.output.update_task_outputs_from_contexts(context_data_list, context.workflow)
     end
+
+    #:  () -> void
+    def persist_current_job_context
+      QueueAdapter.current.persist_job_context(job)
+    end
   end
 end

data/lib/job_workflow/task.rb

@@ -2,8 +2,9 @@
 
 module JobWorkflow
   class Task
+    DEFAULT_EACH = ->(_ctx) { [nil] }
+
     attr_reader :job_name #: String
-    attr_reader :namespace #: Namespace
     attr_reader :block #: ^(untyped) -> void
     attr_reader :each #: ^(Context) -> untyped
     attr_reader :enqueue #: TaskEnqueue
@@ -16,11 +17,10 @@ module JobWorkflow
     attr_reader :dependency_wait #: TaskDependencyWait
     attr_reader :dry_run_config #: DryRunConfig
 
-    # rubocop:disable Metrics/ParameterLists, Metrics/MethodLength, Metrics/AbcSize
+    # rubocop:disable Metrics/ParameterLists, Metrics/MethodLength
     #:  (
     #     job_name: String,
     #     name: Symbol,
-    #     namespace: Namespace,
     #     block: ^(untyped) -> void,
     #     ?each: ^(Context) -> untyped,
     #     ?enqueue: true | false | ^(Context) -> bool | Hash[Symbol, untyped],
@@ -36,7 +36,6 @@ module JobWorkflow
     def initialize(
       job_name:,
       name:,
-      namespace:,
       block:,
       each: nil,
       enqueue: nil,
@@ -51,7 +50,6 @@ module JobWorkflow
     )
       @job_name = job_name
       @name = name
-      @namespace = namespace #: Namespace
       @block = block
       @each = each
       @enqueue = TaskEnqueue.from_primitive_value(enqueue)
@@ -64,11 +62,11 @@ module JobWorkflow
       @dependency_wait = TaskDependencyWait.from_primitive_value(dependency_wait)
       @dry_run_config = DryRunConfig.from_primitive_value(dry_run)
     end
-    # rubocop:enable Metrics/ParameterLists, Metrics/MethodLength, Metrics/AbcSize
+    # rubocop:enable Metrics/ParameterLists, Metrics/MethodLength
 
     #:  () -> Symbol
     def task_name
-      [namespace.full_name.to_s, name.to_s].reject(&:empty?).join(":").to_sym
+      name
     end
 
     #:  () -> String
@@ -76,6 +74,11 @@ module JobWorkflow
       "#{job_name}:#{task_name}"
     end
 
+    #:  () -> bool
+    def each?
+      !each.nil? && !each.equal?(DEFAULT_EACH)
+    end
+
     private
 
     attr_reader :name #: Symbol

data/lib/job_workflow/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module JobWorkflow
-  VERSION = "0.3.0" # : String
+  VERSION = "0.5.0" # : String
 end

data/lib/job_workflow/workflow.rb

@@ -2,7 +2,6 @@
 
 module JobWorkflow
   class Workflow
-    attr_reader :namespace #: Namespace
     attr_reader :dry_run_config #: DryRunConfig
 
     #:  () -> void
@@ -10,7 +9,6 @@ module JobWorkflow
       @task_graph = TaskGraph.new
       @argument_defs = {} #: Hash[Symbol, ArgumentDef]
       @hook_registry = HookRegistry.new
-      @namespace = Namespace.default #: Namespace
       @schedules = {} #: Hash[Symbol, Schedule]
       @dry_run_config = DryRunConfig.new
     end
@@ -20,15 +18,6 @@ module JobWorkflow
       @dry_run_config = DryRunConfig.from_primitive_value(value)
     end
 
-    #:  (Namespace) { () -> void } -> void
-    def add_namespace(namespace)
-      original_namespace = @namespace
-      @namespace = namespace.update_parent(original_namespace)
-      yield
-    ensure
-      @namespace = original_namespace
-    end
-
     #:  (Task) -> void
     def add_task(task)
       @task_graph.add(task)

data/lib/job_workflow.rb

@@ -23,7 +23,6 @@ require_relative "job_workflow/task_throttle"
 require_relative "job_workflow/task_enqueue"
 require_relative "job_workflow/task_dependency_wait"
 require_relative "job_workflow/semaphore"
-require_relative "job_workflow/namespace"
 require_relative "job_workflow/hook"
 require_relative "job_workflow/error_hook"
 require_relative "job_workflow/hook_registry"
@@ -46,6 +45,9 @@ require_relative "job_workflow/task_output"
 require_relative "job_workflow/output"
 require_relative "job_workflow/queue"
 require_relative "job_workflow/auto_scaling"
+# :nocov:
+require_relative "job_workflow/railtie" if defined?(Rails::Railtie)
+# :nocov:
 
 module JobWorkflow
   class Error < StandardError; end
@@ -54,6 +56,5 @@ module JobWorkflow
 
   Instrumentation::LogSubscriber.attach!
 
-  ActiveSupport.on_load(:solid_queue) { QueueAdapter.current.initialize_adapter! }
   QueueAdapter.current.initialize_adapter!
 end

data/rbs_collection.lock.yaml

@@ -6,7 +6,7 @@ gems:
   source:
     type: git
     name: ruby/gem_rbs_collection
-    revision: 9bf2eebb1c54b5d6f23f2acb65d4c36f195b4783
+    revision: 81fa8bd0617286078617a62b6a3229cebfd4af23
     remote: https://github.com/ruby/gem_rbs_collection.git
     repo_dir: gems
 - name: activerecord
@@ -14,7 +14,7 @@ gems:
   source:
     type: git
     name: ruby/gem_rbs_collection
-    revision: 9bf2eebb1c54b5d6f23f2acb65d4c36f195b4783
+    revision: 81fa8bd0617286078617a62b6a3229cebfd4af23
     remote: https://github.com/ruby/gem_rbs_collection.git
     repo_dir: gems
 - name: activesupport
@@ -22,7 +22,7 @@ gems:
   source:
     type: git
     name: ruby/gem_rbs_collection
-    revision: 9bf2eebb1c54b5d6f23f2acb65d4c36f195b4783
+    revision: 81fa8bd0617286078617a62b6a3229cebfd4af23
     remote: https://github.com/ruby/gem_rbs_collection.git
     repo_dir: gems
 - name: base64
@@ -34,7 +34,7 @@ gems:
   source:
     type: git
     name: ruby/gem_rbs_collection
-    revision: 9bf2eebb1c54b5d6f23f2acb65d4c36f195b4783
+    revision: 81fa8bd0617286078617a62b6a3229cebfd4af23
     remote: https://github.com/ruby/gem_rbs_collection.git
     repo_dir: gems
 - name: concurrent-ruby
@@ -42,7 +42,7 @@ gems:
   source:
     type: git
     name: ruby/gem_rbs_collection
-    revision: 9bf2eebb1c54b5d6f23f2acb65d4c36f195b4783
+    revision: 81fa8bd0617286078617a62b6a3229cebfd4af23
     remote: https://github.com/ruby/gem_rbs_collection.git
     repo_dir: gems
 - name: connection_pool
@@ -50,7 +50,7 @@ gems:
   source:
     type: git
     name: ruby/gem_rbs_collection
-    revision: 9bf2eebb1c54b5d6f23f2acb65d4c36f195b4783
+    revision: 81fa8bd0617286078617a62b6a3229cebfd4af23
     remote: https://github.com/ruby/gem_rbs_collection.git
     repo_dir: gems
 - name: date
@@ -74,7 +74,7 @@ gems:
   source:
     type: git
     name: ruby/gem_rbs_collection
-    revision: 9bf2eebb1c54b5d6f23f2acb65d4c36f195b4783
+    revision: 81fa8bd0617286078617a62b6a3229cebfd4af23
     remote: https://github.com/ruby/gem_rbs_collection.git
     repo_dir: gems
 - name: i18n
@@ -82,7 +82,7 @@ gems:
   source:
     type: git
     name: ruby/gem_rbs_collection
-    revision: 9bf2eebb1c54b5d6f23f2acb65d4c36f195b4783
+    revision: 81fa8bd0617286078617a62b6a3229cebfd4af23
     remote: https://github.com/ruby/gem_rbs_collection.git
     repo_dir: gems
 - name: json
@@ -98,7 +98,7 @@ gems:
   source:
     type: git
     name: ruby/gem_rbs_collection
-    revision: 9bf2eebb1c54b5d6f23f2acb65d4c36f195b4783
+    revision: 81fa8bd0617286078617a62b6a3229cebfd4af23
     remote: https://github.com/ruby/gem_rbs_collection.git
     repo_dir: gems
 - name: monitor
@@ -138,7 +138,7 @@ gems:
   source:
     type: git
     name: ruby/gem_rbs_collection
-    revision: 9bf2eebb1c54b5d6f23f2acb65d4c36f195b4783
+    revision: 81fa8bd0617286078617a62b6a3229cebfd4af23
     remote: https://github.com/ruby/gem_rbs_collection.git
     repo_dir: gems
 - name: stringio
@@ -162,7 +162,7 @@ gems:
   source:
     type: git
     name: ruby/gem_rbs_collection
-    revision: 9bf2eebb1c54b5d6f23f2acb65d4c36f195b4783
+    revision: 81fa8bd0617286078617a62b6a3229cebfd4af23
     remote: https://github.com/ruby/gem_rbs_collection.git
     repo_dir: gems
 - name: uri

data/sig/generated/job_workflow/context.rbs

@@ -3,6 +3,8 @@
 module JobWorkflow
   class Context
     # rubocop:disable Metrics/ClassLength
+    EACH_TASK_CURSOR_MARKER: ::String
+
     attr_reader workflow: Workflow
 
     attr_reader arguments: Arguments
@@ -33,6 +35,15 @@ module JobWorkflow
     # :  (Hash[Symbol, untyped]) -> Context
     def _update_arguments: (Hash[Symbol, untyped]) -> Context
 
+    # :  () -> untyped
+    def cursor: () -> untyped
+
+    # :  (untyped) -> void
+    def set_cursor!: (untyped) -> void
+
+    # :  () -> void
+    def checkpoint!: () -> void
+
     # :  (DSL) -> void
     def _job=: (DSL) -> void
 
@@ -48,8 +59,8 @@ module JobWorkflow
     # :  () -> String?
     def concurrency_key: () -> String?
 
-    # :  (Task) -> Enumerator[Context]
-    def _with_each_value: (Task) -> Enumerator[Context]
+    # :  (Task, ?start_index: Integer?) -> Enumerator[Context]
+    def _with_each_value: (Task, ?start_index: Integer?) -> Enumerator[Context]
 
     # :  () { () -> void } -> void
     def _with_task_throttle: () { () -> void } -> void
@@ -93,6 +104,9 @@ module JobWorkflow
     # :  () -> TaskContext
     def _task_context: () -> TaskContext
 
+    # :  (ActiveJob::Continuation::Step, ?cursor: untyped) { () -> void } -> void
+    def _with_current_step: (ActiveJob::Continuation::Step, ?cursor: untyped) { () -> void } -> void
+
     # :  (TaskOutput) -> void
     def _add_task_output: (TaskOutput) -> void
 
@@ -119,23 +133,33 @@ module JobWorkflow
 
     attr_accessor skip_in_dry_run_index: Integer
 
+    attr_accessor current_step: ActiveJob::Continuation::Step?
+
+    attr_accessor current_cursor: untyped
+
     # :  () -> String
     def parent_job_id: () -> String
 
+    # :  () -> bool
+    def each_task?: () -> bool
+
+    # :  (untyped) -> untyped
+    def build_step_cursor: (untyped) -> untyped
+
     # :  () -> Hash[String, untyped]
     def serialize_for_job: () -> Hash[String, untyped]
 
     # :  () -> Hash[String, untyped]
     def serialize_for_sub_job: () -> Hash[String, untyped]
 
-    # :  (Task, Enumerator::Yielder) -> void
-    def with_task_context: (Task, Enumerator::Yielder) -> void
+    # :  (Task, Enumerator::Yielder, ?start_index: Integer?) -> void
+    def with_task_context: (Task, Enumerator::Yielder, ?start_index: Integer?) -> void
 
     # :  (Task) -> void
     def reset_task_context_if_task_changed: (Task) -> void
 
-    # :  (Task) { (untyped, Integer) -> void } -> void
-    def with_each_index_and_value: (Task) { (untyped, Integer) -> void } -> void
+    # :  (Task, ?start_index: Integer?) { (untyped, Integer) -> void } -> void
+    def with_each_index_and_value: (Task, ?start_index: Integer?) { (untyped, Integer) -> void } -> void
 
     # :  () -> void
     def clear_after_each_index_and_value: () -> void

data/sig/generated/job_workflow/dsl.rbs

@@ -62,9 +62,6 @@ module JobWorkflow
       # :  (Symbol argument_name, String type, ?default: untyped) -> void
       def argument: (Symbol argument_name, String type, ?default: untyped) -> void
 
-      # :  (Symbol) { () -> void } -> void
-      def namespace: (Symbol) { () -> void } -> void
-
       # rubocop:disable Metrics/ParameterLists
       #
       # :  (
@@ -139,11 +136,6 @@ module JobWorkflow
       #     ?description: String?
       #   ) -> void
       def schedule: (String expression, ?key: (String | Symbol)?, ?queue: String?, ?priority: Integer?, ?args: Hash[Symbol, untyped], ?description: String?) -> void
-
-      private
-
-      # :  () -> void
-      def validate_namespace!: () -> void
     end
   end
 end

data/sig/generated/job_workflow/railtie.rbs

@@ -0,0 +1,6 @@
+# Generated from lib/job_workflow/railtie.rb with RBS::Inline
+
+module JobWorkflow
+  class Railtie < Rails::Railtie
+  end
+end

data/sig/generated/job_workflow/runner.rbs

@@ -30,11 +30,14 @@ module JobWorkflow
     # :  (Task) -> bool
     def skip_task?: (Task) -> bool
 
-    # :  (Task) -> void
-    def run_task: (Task) -> void
+    # :  (Task, step: ActiveJob::Continuation::Step) -> void
+    def run_task: (Task, step: ActiveJob::Continuation::Step) -> void
+
+    # :  (Task, untyped) -> [Integer?, untyped]
+    def decode_task_cursor: (Task, untyped) -> [ Integer?, untyped ]
 
-    # :  (Task, Context) -> void
-    def run_each_task: (Task, Context) -> void
+    # :  (Task, Context, step: ActiveJob::Continuation::Step, ?cursor: untyped) -> void
+    def run_each_task: (Task, Context, step: ActiveJob::Continuation::Step, ?cursor: untyped) -> void
 
     # :  (Task, Context) { () -> void } -> void
     def run_hooks: (Task, Context) { () -> void } -> void
@@ -62,5 +65,8 @@ module JobWorkflow
 
     # :  (Task) -> void
     def update_task_outputs: (Task) -> void
+
+    # :  () -> void
+    def persist_current_job_context: () -> void
   end
 end

data/sig/generated/job_workflow/task.rbs

@@ -2,9 +2,9 @@
 
 module JobWorkflow
   class Task
-    attr_reader job_name: String
+    DEFAULT_EACH: untyped
 
-    attr_reader namespace: Namespace
+    attr_reader job_name: String
 
     attr_reader block: ^(untyped) -> void
 
@@ -28,11 +28,10 @@ module JobWorkflow
 
     attr_reader dry_run_config: DryRunConfig
 
-    # rubocop:disable Metrics/ParameterLists, Metrics/MethodLength, Metrics/AbcSize
+    # rubocop:disable Metrics/ParameterLists, Metrics/MethodLength
     # :  (
     #     job_name: String,
     #     name: Symbol,
-    #     namespace: Namespace,
     #     block: ^(untyped) -> void,
     #     ?each: ^(Context) -> untyped,
     #     ?enqueue: true | false | ^(Context) -> bool | Hash[Symbol, untyped],
@@ -45,7 +44,7 @@ module JobWorkflow
     #     ?dependency_wait: Hash[Symbol, untyped],
     #     ?dry_run: bool | ^(Context) -> bool
     #   ) -> void
-    def initialize: (job_name: String, name: Symbol, namespace: Namespace, block: ^(untyped) -> void, condition: ^(Context) -> bool, ?each: ^(Context) -> untyped, ?enqueue: true | false | ^(Context) -> bool | Hash[Symbol, untyped], ?output: Hash[Symbol, String], ?depends_on: Array[Symbol], ?task_retry: Integer | Hash[Symbol, untyped], ?throttle: Integer | Hash[Symbol, untyped], ?timeout: Numeric?, ?dependency_wait: Hash[Symbol, untyped], ?dry_run: bool | ^(Context) -> bool) -> void
+    def initialize: (job_name: String, name: Symbol, block: ^(untyped) -> void, condition: ^(Context) -> bool, ?each: ^(Context) -> untyped, ?enqueue: true | false | ^(Context) -> bool | Hash[Symbol, untyped], ?output: Hash[Symbol, String], ?depends_on: Array[Symbol], ?task_retry: Integer | Hash[Symbol, untyped], ?throttle: Integer | Hash[Symbol, untyped], ?timeout: Numeric?, ?dependency_wait: Hash[Symbol, untyped], ?dry_run: bool | ^(Context) -> bool) -> void
 
     # :  () -> Symbol
     def task_name: () -> Symbol
@@ -53,6 +52,9 @@ module JobWorkflow
     # :  () -> String
     def throttle_prefix_key: () -> String
 
+    # :  () -> bool
+    def each?: () -> bool
+
     private
 
     attr_reader name: Symbol

data/sig/generated/job_workflow/workflow.rbs

@@ -2,8 +2,6 @@
 
 module JobWorkflow
   class Workflow
-    attr_reader namespace: Namespace
-
     attr_reader dry_run_config: DryRunConfig
 
     # :  () -> void
@@ -12,9 +10,6 @@ module JobWorkflow
     # :  (bool | ^(Context) -> bool) -> void
     def dry_run_config=: (bool | ^(Context) -> bool) -> void
 
-    # :  (Namespace) { () -> void } -> void
-    def add_namespace: (Namespace) { () -> void } -> void
-
     # :  (Task) -> void
     def add_task: (Task) -> void
 

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: job-workflow
 version: !ruby/object:Gem::Version
-  version: 0.3.0
+  version: 0.5.0
 platform: ruby
 authors:
 - shoma07
@@ -49,7 +49,6 @@ files:
 - guides/GETTING_STARTED.md
 - guides/INSTRUMENTATION.md
 - guides/LIFECYCLE_HOOKS.md
-- guides/NAMESPACES.md
 - guides/OPENTELEMETRY_INTEGRATION.md
 - guides/PARALLEL_PROCESSING.md
 - guides/PRODUCTION_DEPLOYMENT.md
@@ -84,7 +83,6 @@ files:
 - lib/job_workflow/instrumentation/opentelemetry_subscriber.rb
 - lib/job_workflow/job_status.rb
 - lib/job_workflow/logger.rb
-- lib/job_workflow/namespace.rb
 - lib/job_workflow/output.rb
 - lib/job_workflow/output_def.rb
 - lib/job_workflow/queue.rb
@@ -92,6 +90,7 @@ files:
 - lib/job_workflow/queue_adapters/abstract.rb
 - lib/job_workflow/queue_adapters/null_adapter.rb
 - lib/job_workflow/queue_adapters/solid_queue_adapter.rb
+- lib/job_workflow/railtie.rb
 - lib/job_workflow/runner.rb
 - lib/job_workflow/schedule.rb
 - lib/job_workflow/semaphore.rb
@@ -136,7 +135,6 @@ files:
 - sig/generated/job_workflow/instrumentation/opentelemetry_subscriber.rbs
 - sig/generated/job_workflow/job_status.rbs
 - sig/generated/job_workflow/logger.rbs
-- sig/generated/job_workflow/namespace.rbs
 - sig/generated/job_workflow/output.rbs
 - sig/generated/job_workflow/output_def.rbs
 - sig/generated/job_workflow/queue.rbs
@@ -144,6 +142,7 @@ files:
 - sig/generated/job_workflow/queue_adapters/abstract.rbs
 - sig/generated/job_workflow/queue_adapters/null_adapter.rbs
 - sig/generated/job_workflow/queue_adapters/solid_queue_adapter.rbs
+- sig/generated/job_workflow/railtie.rbs
 - sig/generated/job_workflow/runner.rbs
 - sig/generated/job_workflow/schedule.rbs
 - sig/generated/job_workflow/semaphore.rbs

data/guides/NAMESPACES.md

@@ -1,75 +0,0 @@
-# Namespaces
-
-Logically grouping tasks improves readability and maintainability of complex workflows. JobWorkflow provides namespace functionality.
-
-## Basic Namespaces
-
-### namespace DSL
-
-Group related tasks.
-
-```ruby
-class ECommerceOrderJob < ApplicationJob
-  include JobWorkflow::DSL
-  
-  argument :order, "Order"
-  
-  # Payment-related tasks
-  namespace :payment do
-    task :validate do |ctx|
-      order = ctx.arguments.order
-      PaymentValidator.validate(order)
-    end
-    
-    task :charge, depends_on: [:"payment:validate"], output: { payment_result: "Hash" } do |ctx|
-      order = ctx.arguments.order
-      { payment_result: PaymentProcessor.charge(order) }
-    end
-    
-    task :send_receipt, depends_on: [:"payment:charge"] do |ctx|
-      order = ctx.arguments.order
-      payment_result = ctx.output[:"payment:charge"].first.payment_result
-      ReceiptMailer.send(order, payment_result)
-    end
-  end
-  
-  # Inventory-related tasks
-  namespace :inventory do
-    task :check_availability do |ctx|
-      order = ctx.arguments.order
-      InventoryService.check(order.items)
-    end
-    
-    task :reserve, depends_on: [:"inventory:check_availability"], output: { reserved: "bool" } do |ctx|
-      order = ctx.arguments.order
-      { reserved: InventoryService.reserve(order.items) }
-    end
-  end
-  
-  # Shipping-related tasks
-  namespace :shipping do
-    task :calculate_cost, output: { shipping_cost: "Float" } do |ctx|
-      order = ctx.arguments.order
-      { shipping_cost: ShippingCalculator.calculate(order) }
-    end
-    
-    task :create_label, depends_on: [:"shipping:calculate_cost"], output: { shipping_label: "String" } do |ctx|
-      order = ctx.arguments.order
-      { shipping_label: ShippingService.create_label(order) }
-    end
-  end
-end
-```
-
-Tasks in namespaces are identified as `:namespace:task_name` at runtime:
-
-```ruby
-# Executed tasks:
-# - :payment:validate
-# - :payment:charge
-# - :payment:send_receipt
-# - :inventory:check_availability
-# - :inventory:reserve
-# - :shipping:calculate_cost
-# - :shipping:create_label
-```

data/lib/job_workflow/namespace.rb

@@ -1,36 +0,0 @@
-# frozen_string_literal: true
-
-module JobWorkflow
-  class Namespace
-    attr_reader :name #: Symbol
-    attr_reader :parent #: Namespace?
-
-    class << self
-      #:  () -> Namespace
-      def default
-        new(name: :"")
-      end
-    end
-
-    #:  (name: Symbol, ?parent: Namespace?) -> void
-    def initialize(name:, parent: nil)
-      @name = name #: Symbol
-      @parent = parent #: Namespace?
-    end
-
-    #:  () -> bool
-    def default?
-      name.empty?
-    end
-
-    #:  (Namespace) -> Namespace
-    def update_parent(parent)
-      self.class.new(name:, parent:)
-    end
-
-    #:  () -> Symbol
-    def full_name
-      [parent&.full_name, name.to_s].compact.reject(&:empty?).join(":").to_sym
-    end
-  end
-end

data/sig/generated/job_workflow/namespace.rbs

@@ -1,24 +0,0 @@
-# Generated from lib/job_workflow/namespace.rb with RBS::Inline
-
-module JobWorkflow
-  class Namespace
-    attr_reader name: Symbol
-
-    attr_reader parent: Namespace?
-
-    # :  () -> Namespace
-    def self.default: () -> Namespace
-
-    # :  (name: Symbol, ?parent: Namespace?) -> void
-    def initialize: (name: Symbol, ?parent: Namespace?) -> void
-
-    # :  () -> bool
-    def default?: () -> bool
-
-    # :  (Namespace) -> Namespace
-    def update_parent: (Namespace) -> Namespace
-
-    # :  () -> Symbol
-    def full_name: () -> Symbol
-  end
-end