nexo_ai 0.7.0 → 0.8.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 34123b61080f43662220abb5a92686ac3123ef5ae1255a1318f311985a08e304
-  data.tar.gz: 46fdf315414f80c500cc7fa33794c5a576839c6b67fcbb0e282d2ad327d354fd
+  metadata.gz: 7e3ef801100c09633b3bc0eba7f1c5444abc3a28d39b093b3f556a6e6a193304
+  data.tar.gz: 770aa194ebef30adc05049890df4611914d9af49ce5bc37460d142dce0a5277d
 SHA512:
-  metadata.gz: 6df53f1442c2332977b40a2c3d1024699b2fc7ec9f464714e8dd0f0364dfa906c13c423af438523e7c054f93b61391c2810e44e4bab97a5a087361a82635f993
-  data.tar.gz: 961b1ce3a8975f2127a2d0a207d93742e674ea233cb549942fe18c39b3cc4b742f61bf7b41fe6eadca815ee863cae956362a18acf9731450089e55bcfbf57204
+  metadata.gz: 201531d77a85255355cf77d34d0dd676843f20dee2149627226a1f9068c5b6dfcc7e9e4d92e33b357536326f35e861d9fc1d1d6230bac1e5eac5db9314e3ec75
+  data.tar.gz: aff13284d24b6465a10a34b60847d6ceece755b8d8791b688777a6ec7538d7506db82922a9e83d4382a66d3045ee45ee3a4c9eede4d2b8a37e4021e9e134a934

data/CHANGELOG.md

@@ -1,5 +1,28 @@
 ## [Unreleased]
 
+## [0.8.0] - 2026-07-16
+
+Durability enhancements for workflows: a suspended run can now wake itself on a
+timer, and independent checkpoints run concurrently, each persisting as it
+completes. Both build on the existing ActiveJob and `Nexo.concurrent` seams — no
+schema, run-status, or store change.
+
+### Added
+
+- **Scheduled enqueue/resume.** `Workflow.run_later` and `Workflow.resume_later`
+  accept `wait:` (a duration) or `wait_until:` (an absolute time), forwarded to the
+  installed ActiveJob's own `.set(...)` scheduler — so a suspended run can wake
+  itself on a timer and an initial enqueue can be deferred, without Nexo adding a
+  scheduler. Passing both raises `ArgumentError`; with neither the enqueue is
+  unchanged. Status stays `"queued"`/`"suspended"` (no new status). No retry
+  semantics are added.
+- **Parallel checkpoints.** `Workflow#checkpoint_all(name => callable, …)` runs
+  independent checkpoints concurrently through the existing `Nexo.concurrent`
+  driver, persisting **each step as it completes** so a resume after a partial
+  failure re-runs only the still-missing steps. Each newly-completed step emits a
+  `"checkpoint"`-typed event (name-only) on the existing `nexo.workflow.event`
+  seam. Reuses `state`/`save_state!` — no schema, run-status, or store change.
+
 ## [0.7.0] - 2026-07-10
 
 The harness fills out: MCP data sources, a web-content capability, durable

data/docs/concurrency.md

@@ -41,6 +41,12 @@ most important knob for staying under provider rate limits.
 Using `Nexo.concurrent` with `async` not installed raises
 `Nexo::MissingDependencyError` with install guidance.
 
+Inside a durable workflow, `Workflow#checkpoint_all` is the workflow-durability
+flavored sibling of `Nexo.concurrent`: it drives this same bounded fan-out but
+persists each step to the run's `state` as it lands, so a resume only re-runs what
+never completed. See [Parallel checkpoints](durable-workflows.md#parallel-checkpoints--checkpoint_all)
+in the durable-workflows guide.
+
 ## `Sandboxes::Local` offload
 
 Under a reactor, blocking file/subprocess I/O would stall every other fiber. Flip

data/docs/durable-workflows.md

@@ -64,6 +64,52 @@ DocumentApproval.resume_later(run.id, { approved: true }, queue: :nexo)
 See [`examples/approval_workflow.rb`](../examples/approval_workflow.rb) for the full
 offline flow (`ruby -Ilib examples/approval_workflow.rb`).
 
+## Parallel checkpoints — `checkpoint_all`
+
+When several checkpoints are **independent** (no step depends on another's
+result), run them concurrently with `checkpoint_all(name => callable, …)` instead
+of a sequence of `checkpoint` calls. It fans the pending steps out through
+[`Nexo.concurrent`](concurrency.md) — all in flight at once — and persists **each
+step as it completes** (not the batch as a whole), so a resume after a partial
+failure only re-runs the steps that never landed:
+
+```ruby
+class BuildDashboard < Nexo::Workflow
+  def call(payload)
+    data = checkpoint_all(
+      account: -> { fetch_account(payload[:id]) },   # these two run
+      usage:   -> { fetch_usage(payload[:id]) }      # concurrently
+    )
+    { report: render(data[:account], data[:usage]) }
+  end
+end
+```
+
+`checkpoint_all` returns a Hash keyed by the **original** names you passed
+(`data[:account]`), with values read back from `state` — the same shape whether a
+value came from this pass or a prior one. Each newly-completed step also surfaces a
+`"checkpoint"`-typed event on the run's event log and the `nexo.workflow.event`
+notification (data is the step **name** only, never the value — so a dashboard can
+show batch progress without the event log carrying large or sensitive results).
+Steps already present in `state` are skipped silently and emit nothing.
+
+Bound the batch by how many keys you pass — there is no separate rate knob; every
+pending step goes in flight. Because it drives `Nexo.concurrent`, `checkpoint_all`
+needs the `async` gem **only when something is actually pending** — an
+all-persisted pass (every step already done) returns the prior values directly
+without touching concurrency. The same restrictions as `checkpoint` apply: values
+must be json-serializable, a step must **not** be named after a reserved state key
+(`__suspend__`/`__approval__`/`__buffer_events__` — raises `Nexo::Error` before any
+step runs), and do **not** call `suspend!` inside a step (undefined — unsupported).
+
+> **⚠️ Known trade-off: per-step persistence, not an atomic batch.** `checkpoint_all`
+> is **not** transactional. If step B raises after step A persisted, A stays in
+> `state`, B is absent, the run goes `"failed"`, and the exception propagates through
+> the workflow's normal failure path (`Nexo.concurrent`'s "first failure re-raises,
+> the rest stop" — it is not rescued away). A subsequent `execute` of the **same** run
+> re-submits only the still-missing names — A is skipped, B re-runs. Do **not** treat
+> a batch as all-or-nothing.
+
 ## Durable **agent** approval — `:approve` (bridge a mid-run gate to a suspend)
 
 The example above suspends at an **explicit** `suspend!` the workflow author placed.

data/docs/rails.md

@@ -35,6 +35,39 @@ GenerateReport.run_later(account_id: 42, queue: :nexo)  # per-call
 Nexo.configure { |c| c.job_queue = :nexo }              # or a global default
 ```
 
+### Scheduling a future run or resume
+
+`run_later` and `resume_later` accept `wait:` (a duration) or `wait_until:` (an
+absolute time), forwarded straight to **the installed ActiveJob's own**
+`.set(...)` scheduler — Nexo adds no scheduler of its own. Use them to defer an
+initial enqueue ("send this digest at 9am") or to let a suspended run wake itself
+on a timer, symmetrically:
+
+```ruby
+# Defer the initial enqueue until tomorrow morning.
+DailyDigest.run_later({account_id: 42}, wait_until: Date.tomorrow.noon)
+
+# Let a suspended run wake itself up in an hour (no separate scheduled job).
+MyWorkflow.resume_later(run.id, {reminder: true}, wait: 1.hour)
+```
+
+The run's status is unchanged — a scheduled `run_later` is still `"queued"` (no
+`"scheduled"` status is invented), and a scheduled `resume_later` leaves the run
+`"suspended"` until the job fires. Passing **both** `wait:` and `wait_until:` in
+one call raises `ArgumentError` (checked before any run is created or job
+enqueued) — the installed ActiveJob would otherwise silently keep just one. With
+neither given, the enqueue is byte-for-byte the immediate one above.
+
+> **⚠️ `wait:`/`wait_until:`/`queue:` are scheduling options, not payload.** Like
+> `queue:` already was, a bare-keyword call consumes them as options: `run_later(wait:
+> 60)` schedules the job **60 seconds out** and leaves the payload `{}` — it does
+> **not** store `"wait" => 60` as data. A payload that legitimately needs a key named
+> `"wait"` must be passed as an explicit positional Hash: `run_later({wait: "value"})`.
+
+This is still "no scheduler, no cron" — `wait:`/`wait_until:` schedule a **single**
+future run/resume via ActiveJob; recurring schedules stay the host's (see the "no
+queue and no scheduler" note below).
+
 Nexo ships **no queue and no scheduler** — ActiveJob uses whatever adapter your
 app configured (Sidekiq, GoodJob, Solid Queue, …), and scheduling (cron / GoodJob
 / `whenever`) stays the host's. Without ActiveJob, `run_later` raises

data/lib/nexo/version.rb

@@ -2,5 +2,5 @@
 
 module Nexo
   # The gem version (also used as +spec.version+ in the gemspec).
-  VERSION = "0.7.0"
+  VERSION = "0.8.0"
 end

data/lib/nexo/workflow.rb

@@ -40,6 +40,13 @@ module Nexo
   # expensive/side-effectful steps so resume skips already-paid-for work — see the
   # "Durable workflows" README section for the honest resume semantics.
   class Workflow
+    # State keys Nexo reserves for lifecycle metadata, never a caller's data:
+    # +"__suspend__"+ (suspend reason/resume_key — Spec 13), +"__approval__"+
+    # (pending approval call — Spec 16), and +"__buffer_events__"+ (the persisted
+    # buffering choice — Spec 5). ::cleared_state strips these when a run reaches
+    # +"done"+; #checkpoint_all refuses a step named after any of them (Spec 21).
+    RESERVED_STATE_KEYS = %w[__suspend__ __approval__ __buffer_events__].freeze
+
     # Control-flow signal raised by #suspend! and caught by ::execute to pause
     # a run durably — NOT a failure. It transitions the run to +"suspended"+
     # (never +"failed"+) and returns the run to the caller rather than re-raising.
@@ -165,11 +172,30 @@ module Nexo
       # also reachable. Not resumable: a crashed or retried job re-runs +#call+ from
       # scratch (Nexo adds no +retry_on+); pair with ::reconcile_interrupted! to
       # catch runs orphaned in +"running"+.
-      def run_later(payload = nil, queue: Nexo.config.job_queue, **kwargs)
+      #
+      # +wait:+ / +wait_until:+ (Spec 21) defer the enqueue via the installed
+      # ActiveJob's own +.set(...)+ scheduler — +wait:+ takes a duration
+      # (+wait: 1.hour+), +wait_until:+ an absolute time (+wait_until: tomorrow_9am+).
+      # Nexo adds no scheduler of its own; it just forwards these to +.set+. The run
+      # is still +"queued"+ (no +"scheduled"+ status is invented). Passing **both** in
+      # one call raises ArgumentError (the installed ActiveJob would silently keep one)
+      # — checked before any run is created. With neither given the enqueue is
+      # byte-for-byte the pre-Spec-21 immediate one (no +:at+ on the job).
+      #
+      # +wait:+/+wait_until:+/+queue:+ share the bare-keyword/positional ambiguity
+      # that +queue:+ already carried: a bare-keyword call like +run_later(wait: 60)+
+      # consumes +wait+ as the scheduling option (the payload stays +{}+). A payload
+      # that legitimately needs a key literally named +"wait"+ must be passed as an
+      # explicit positional Hash — +run_later({wait: "value"})+.
+      def run_later(payload = nil, queue: Nexo.config.job_queue, wait: nil, wait_until: nil, **kwargs)
         unless defined?(::ActiveJob)
           raise Nexo::MissingDependencyError,
             "run_later requires ActiveJob (Rails). Use `run` for synchronous execution."
         end
+        if wait && wait_until
+          raise ArgumentError,
+            "pass either `wait:` or `wait_until:`, not both (got both)"
+        end
         if payload && !kwargs.empty?
           raise ArgumentError,
             "pass the payload either as a positional Hash or as keywords, not both (got both)"
@@ -178,9 +204,7 @@ module Nexo
         run = Nexo::RunStore.default.create(workflow_class: name, payload: stringify(payload))
         run.update!(status: "queued")
         notify_status(run) # let a dashboard learn the run was enqueued
-        job = Nexo::WorkflowJob
-        job = job.set(queue: queue) if queue
-        job.perform_later(run.id)
+        enqueue_job(Nexo::WorkflowJob, queue: queue, wait: wait, wait_until: wait_until).perform_later(run.id)
         run
       end
 
@@ -229,15 +253,27 @@ module Nexo
       # Requires ActiveJob (Rails): without it this raises Nexo::MissingDependencyError
       # pointing at ::resume for synchronous execution. +queue:+ (default
       # Nexo.config.job_queue) routes the job exactly like ::run_later.
-      def resume_later(run_id, input = {}, queue: Nexo.config.job_queue)
+      #
+      # +wait:+ / +wait_until:+ (Spec 21) defer the resume via the installed
+      # ActiveJob's own +.set(...)+ — +wait:+ a duration (+resume_later(id, input,
+      # wait: 1.hour)+ so a suspended run wakes itself on a timer), +wait_until:+ an
+      # absolute time. Nexo adds no scheduler and no retry; a crashed scheduled-resume
+      # job remains the host's +reconcile_interrupted!+ / +retry_on+ story. Passing
+      # **both** raises ArgumentError (the installed ActiveJob would silently keep one),
+      # checked before the job is enqueued. With neither given the enqueue is
+      # byte-for-byte the pre-Spec-21 immediate one. The return value is unchanged: the
+      # run stays +"suspended"+ until the job fires and re-enters ::resume's atomic claim.
+      def resume_later(run_id, input = {}, queue: Nexo.config.job_queue, wait: nil, wait_until: nil)
         unless defined?(::ActiveJob)
           raise Nexo::MissingDependencyError,
             "resume_later requires ActiveJob (Rails). Use `resume` for synchronous execution."
         end
+        if wait && wait_until
+          raise ArgumentError,
+            "pass either `wait:` or `wait_until:`, not both (got both)"
+        end
         run = Nexo::RunStore.default.find(run_id)
-        job = Nexo::WorkflowJob
-        job = job.set(queue: queue) if queue
-        job.perform_later(run.id, input)
+        enqueue_job(Nexo::WorkflowJob, queue: queue, wait: wait, wait_until: wait_until).perform_later(run.id, input)
         run
       end
 
@@ -341,6 +377,22 @@ module Nexo
         )
       end
 
+      # Builds the ActiveJob dispatch for ::run_later/::resume_later, forwarding
+      # only the non-nil scheduling options to the installed ActiveJob's own
+      # +.set(...)+ (Spec 21). The options Hash is accumulated conditionally so that
+      # with no +queue+/+wait+/+wait_until+ there is no +.set+ call at all — keeping
+      # the no-scheduling-option enqueue byte-for-byte identical to the pre-Spec-21
+      # path (the job carries no +:at+). +wait:+ and +wait_until:+ are never both
+      # present here (the callers raise ArgumentError first), so ActiveJob never has
+      # to silently pick between them.
+      def enqueue_job(job, queue: nil, wait: nil, wait_until: nil)
+        options = {}
+        options[:queue] = queue if queue
+        options[:wait] = wait if wait
+        options[:wait_until] = wait_until if wait_until
+        options.empty? ? job : job.set(**options)
+      end
+
       # Persists a +true+ buffering choice under the reserved "__buffer_events__"
       # state key (idempotent — written once) so ::resume can honor it. The
       # unbuffered default writes nothing, keeping the Spec 2 hot path untouched.
@@ -358,11 +410,10 @@ module Nexo
       def cleared_state(run)
         return {} unless run.respond_to?(:state)
 
-        reserved = %w[__suspend__ __approval__ __buffer_events__]
         state = run.state || {}
-        return {} unless reserved.any? { |k| state.key?(k) }
+        return {} unless RESERVED_STATE_KEYS.any? { |k| state.key?(k) }
 
-        {state: state.except(*reserved)}
+        {state: state.except(*RESERVED_STATE_KEYS)}
       end
 
       def stringify(hash) = hash.transform_keys(&:to_s)
@@ -383,6 +434,11 @@ module Nexo
       @run = run
       @buffer_events = buffer_events
       @event_buffer = []
+      # Serializes the read-current → merge → assign → save_state! sequence across
+      # the concurrent fibers #checkpoint_all runs each step on (Spec 21). A plain
+      # Mutex is fiber-safe under the async reactor (Group 0 verified it serializes
+      # without stalling); #checkpoint (singular, no concurrency) does not use it.
+      @checkpoint_mutex = Mutex.new
     end
 
     # Subclasses implement the work here. The +payload+ is symbol-keyed; the
@@ -424,6 +480,62 @@ module Nexo
       value
     end
 
+    # Runs several independent checkpoints **concurrently** on the first pass and,
+    # crucially, persists **each step as it completes** — so a resume after a
+    # partial failure only re-runs the steps that never landed (Spec 21):
+    #
+    #   fetched = checkpoint_all(
+    #     account: -> { fetch_account(payload[:id]) },
+    #     usage:   -> { fetch_usage(payload[:id]) }
+    #   )
+    #   fetched[:account] # => the account value (this pass or a prior one)
+    #
+    # +steps+ is a Hash of +name => callable+ (each value a Proc/lambda); names may
+    # be symbols or strings and are stringified for storage exactly like #checkpoint
+    # (+name.to_s+). The returned Hash is keyed by the **original** (un-stringified)
+    # names with values read back from +@run.state+, so a caller gets the same shape
+    # whether a value came from this call or a prior pass.
+    #
+    # The pending steps (those not already in +@run.state+) run through the existing
+    # Nexo.concurrent driver, all in flight at once — callers bound the batch by how
+    # many keys they pass; there is no separate rate knob. Each step persists on its
+    # own through the same read-current → merge → assign → +save_state!+ sequence as
+    # #checkpoint, serialized across the concurrent fibers by an internal Mutex, and
+    # emits a +:checkpoint+ event naming the step (Spec 21 R3). Concurrency (and the
+    # +async+ gem) is touched **only** when something is pending — an all-persisted
+    # pass returns the prior-pass values directly without requiring +async+.
+    #
+    # Known trade-off: this is per-step persistence, **not** an atomic batch. If
+    # step B raises after step A persisted, A stays in +run.state+, B is absent, the
+    # run goes +"failed"+, and the exception propagates through the workflow's normal
+    # failure path (Nexo.concurrent's "first failure re-raises, the rest stop" — not
+    # rescued away). A subsequent ::execute of the SAME run re-submits only the
+    # still-missing names — A is skipped, B re-runs. Do NOT treat a batch as
+    # all-or-nothing.
+    #
+    # Same restrictions as #checkpoint: values must be json-serializable (they
+    # round-trip the store), a step must NOT be named after a RESERVED_STATE_KEYS
+    # entry (raises Nexo::Error before any step runs), and do NOT call #suspend!
+    # inside a step (undefined — v1 unsupported, documented not enforced).
+    def checkpoint_all(steps)
+      if (reserved = steps.keys.find { |name| RESERVED_STATE_KEYS.include?(name.to_s) })
+        raise Nexo::Error,
+          "checkpoint name #{reserved.to_s.inspect} is reserved (#{RESERVED_STATE_KEYS.join(", ")})"
+      end
+
+      state = @run.state || {}
+      pending = steps.reject { |name, _| state.key?(name.to_s) }
+
+      unless pending.empty?
+        Nexo.concurrent(max_in_flight: pending.size) do |c|
+          pending.each { |name, callable| c.add { persist_checkpoint(name, callable.call) } }
+        end
+      end
+
+      current = @run.state || {}
+      steps.keys.to_h { |name| [name, current[name.to_s]] }
+    end
+
     # Pauses the run durably (Spec 13): raises Suspended, which ::execute catches
     # to mark the run +"suspended"+ (a non-failure outcome) and return it to the
     # caller. Call this **outside** a checkpoint block. +reason+ is surfaced to a
@@ -615,6 +727,28 @@ module Nexo
       d.key?(:approved) ? {decision: {approved: !!d[:approved]}} : {}
     end
 
+    # Persists ONE completed #checkpoint_all step (Spec 21) — json_normalizes the
+    # raw value, then, inside +@checkpoint_mutex.synchronize+, re-reads the current
+    # +@run.state+, merges the single key, reassigns, and +save_state!+s it: the
+    # identical read-current → merge → assign → save sequence as #checkpoint, but
+    # serialized across the concurrent fibers so two steps landing at once can't
+    # clobber each other's merge. The +:checkpoint+ event is emitted **inside** the
+    # same synchronized block, alongside the state write, so the event-log append
+    # (which mutates the shared buffer / run) serializes with it too. Only reached
+    # for genuinely pending steps, so a skipped (already-persisted) step emits
+    # nothing. Returns the normalized value.
+    def persist_checkpoint(name, raw_value)
+      key = name.to_s
+      value = json_normalize(raw_value)
+      @checkpoint_mutex.synchronize do
+        store = @run.state || {}
+        @run.state = store.merge(key => value)
+        @run.save_state! if @run.respond_to?(:save_state!)
+        emit(:checkpoint, name: key)
+      end
+      value
+    end
+
     # Coerces a checkpoint value into exactly what it would become after a round
     # trip through the AR json column (string keys, symbol values → strings), so
     # the Memory store and the AR store return identical data on resume. A

data/sig/nexo/workflow.rbs

@@ -0,0 +1,14 @@
+# Partial RBS for the Spec 21 surface only (RBS reopening) — the scheduled
+# enqueue/resume keywords on the class methods and the new instance
+# #checkpoint_all. The rest of Workflow stays unsigned, matching today's
+# partial RBS coverage.
+module Nexo
+  class Workflow
+    RESERVED_STATE_KEYS: Array[String]
+
+    def self.run_later: (?untyped payload, ?queue: untyped, ?wait: untyped, ?wait_until: untyped, **untyped) -> untyped
+    def self.resume_later: (untyped run_id, ?untyped input, ?queue: untyped, ?wait: untyped, ?wait_until: untyped) -> untyped
+
+    def checkpoint_all: (Hash[untyped, untyped] steps) -> Hash[untyped, untyped]
+  end
+end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: nexo_ai
 version: !ruby/object:Gem::Version
-  version: 0.7.0
+  version: 0.8.0
 platform: ruby
 authors:
 - Mario Alberto Chávez
@@ -222,6 +222,7 @@ files:
 - sig/nexo/sandboxes/local.rbs
 - sig/nexo/sandboxes/virtual.rbs
 - sig/nexo/tools.rbs
+- sig/nexo/workflow.rbs
 - sig/nexo_ai.rbs
 homepage: https://maquina.app
 licenses: