nexo_ai 0.7.0 → 0.8.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 34123b61080f43662220abb5a92686ac3123ef5ae1255a1318f311985a08e304
-  data.tar.gz: 46fdf315414f80c500cc7fa33794c5a576839c6b67fcbb0e282d2ad327d354fd
+  metadata.gz: af80f73909a7207f4414d45ae28de58bd36555e8aa186f27a7d76b93b6d6ac85
+  data.tar.gz: 91efdbb90334708bc98ddfa099bf1d74f87424fdb6a29bdb0dd83ec6a326e844
 SHA512:
-  metadata.gz: 6df53f1442c2332977b40a2c3d1024699b2fc7ec9f464714e8dd0f0364dfa906c13c423af438523e7c054f93b61391c2810e44e4bab97a5a087361a82635f993
-  data.tar.gz: 961b1ce3a8975f2127a2d0a207d93742e674ea233cb549942fe18c39b3cc4b742f61bf7b41fe6eadca815ee863cae956362a18acf9731450089e55bcfbf57204
+  metadata.gz: 985b074e6c59e6e5aa2353beef2fd14890efceb5f8cd20d7c3611ced5861f3fdf7a086a12b67a90ba2fbd8358c5288ac3c1e774bb3c2bb512a0868f2cc053b75
+  data.tar.gz: '091eff209c74596c4acb2b767182c21fd7f717416fc8a8ef9b3150e245b65d11ab185d8eb546d63dd508d36f7e81faced7c182fcd26fcbe3ad2998561e06965f'

data/CHANGELOG.md

@@ -1,5 +1,105 @@
 ## [Unreleased]
 
+## [0.8.1] - 2026-08-19
+
+Sandbox tiers made honest and skill resources made usable. A skill's `scripts/` and
+`assets/` live outside every sandbox, so an agent could not reach them; `Skills.materialize`
+stages them in through the sandbox's own `#write`, which is the one route that works on
+`:local`, `:container` and `:remote` alike. Along the way the `:apple` runtime turned out
+to be unable to start a container at all, and `Container#write` was reporting success
+while writing nothing. Verified end to end against Apple `container` 1.2.2 and Docker
+29.4.0.
+
+### Fixed
+
+- **`Container#write` creates parent directories and raises on failure.** It ran
+  `sh -c 'cat > "$0"'` and discarded the exit status, so writing a nested path into a
+  fresh workspace failed with `Directory nonexistent` while reporting success to the
+  caller. `Local#write` has always done `mkdir_p` and raised; the two now match. Staging
+  anything with a directory in its path — a skill's `scripts/render.rb` — silently
+  produced nothing on a container before this.
+- **The `:apple` runtime can start a container.** `#run_argv` passed
+  `--security-opt no-new-privileges` and `--pids-limit`, which Apple's `container` CLI
+  rejects with `Unknown option`, so `runtime: :apple` failed before any tool ran. Both
+  come from defaults, so no configuration avoided it. They are now omitted for `:apple`
+  and reported through `#hardening_gaps`.
+
+### Added
+
+- **`Nexo::Skills.materialize(name, into:)`** — copies a skill's `scripts/`, `assets/`
+  and `references/` into a sandbox so an agent can actually reach them, through the
+  sandbox's own `#write`. Returns **sandbox-relative** paths so callers can build a
+  command without knowing which tier they are on. `kinds:` narrows the copy;
+  `overwrite: false` skips files already present, for images that bake the skill in.
+- **`Container#hardening_gaps`** — hardening the caller asked for that the runtime
+  cannot honor, as human-readable strings; empty on `:docker`. Running with weaker
+  isolation than requested is now visible rather than silent. It also reports that
+  Apple's `--tmpfs` is not writable under `--read-only`, which makes the default
+  `readonly_rootfs: true` leave an `:apple` sandbox with no writable workspace.
+- **`Agent#wrap_mcp_tool(tool)`** — a hook called once per MCP tool, after gating and
+  before attachment, for decorating every MCP tool an agent gets (capping an oversized
+  reply, timings, redaction). Default returns the tool unchanged; previously the only
+  way in was overriding the private `#apply_mcp`. Gating happens underneath, so a
+  wrapper cannot widen what the agent may do.
+- **`Nexo.config.tool_concurrency`** — surfaces RubyLLM's concurrent execution of
+  multiple tool calls from one assistant turn (`:fibers` / `:threads` / `false`).
+  Defaults to `nil`, leaving RubyLLM's own setting alone, so behaviour is unchanged
+  until you opt in. Applied after every tool is attached, since each `with_tools`
+  resets it.
+- **`Sandboxes::Remote#instructions`**, with an optional `instructions:` on the
+  constructor. `Local` and `Container` describe their environment to the agent;
+  `Remote` returned `nil`, leaving the tier most likely to surprise a tool-caller the
+  one it knew least about.
+
+### Changed
+
+- **`max_turns` is now counted and reported** on `Loops::RubyLLM` instead of being
+  accepted and ignored. It still cannot halt a run — ruby_llm executes the whole tool
+  loop inside `Chat#ask` and its callbacks are observation-only — but exceeding the
+  budget now emits `:turn_limit_exceeded` with `{turns:, max_turns:}`, once per prompt.
+  Previously the parameter read as a safety bound it has never been.
+
+### Documentation
+
+- **`docs/skills.md`: bundled files are not reachable until staged.** The docs stated
+  that a skill's `scripts/`/`references/` are reached through Nexo's sandbox-backed
+  tools. They are not — a skill lives under `skills_path`, outside every sandbox, and
+  nothing bridged the two. Now documents `Skills.materialize`, the relative-path rule,
+  the ambient-environment pitfall, and the writable-space tradeoff.
+- **Apple `container` parity table filled from live runs**, replacing the previous
+  all-`unverified` placeholder. Records that `--network none` *does* work on Apple,
+  that `ps -aqf` does not exist there (Apple has `list`), and the
+  `--tmpfs`-under-`--read-only` divergence.
+- **`docs/sandboxes.md`: path confinement on `:remote` is the client's job.** `Local`
+  and `Container` raise `SecurityError` on escape; `Remote` passes paths through
+  untouched. Deliberate, but it moves a guarantee callers may rely on.
+- `docs/mcp.md` documents `wrap_mcp_tool`; `docs/concurrency.md` documents
+  `tool_concurrency` and that tools must be concurrency-safe before enabling it;
+  `docs/loops.md` documents what `max_turns` does and does not do.
+
+## [0.8.0] - 2026-07-16
+
+Durability enhancements for workflows: a suspended run can now wake itself on a
+timer, and independent checkpoints run concurrently, each persisting as it
+completes. Both build on the existing ActiveJob and `Nexo.concurrent` seams — no
+schema, run-status, or store change.
+
+### Added
+
+- **Scheduled enqueue/resume.** `Workflow.run_later` and `Workflow.resume_later`
+  accept `wait:` (a duration) or `wait_until:` (an absolute time), forwarded to the
+  installed ActiveJob's own `.set(...)` scheduler — so a suspended run can wake
+  itself on a timer and an initial enqueue can be deferred, without Nexo adding a
+  scheduler. Passing both raises `ArgumentError`; with neither the enqueue is
+  unchanged. Status stays `"queued"`/`"suspended"` (no new status). No retry
+  semantics are added.
+- **Parallel checkpoints.** `Workflow#checkpoint_all(name => callable, …)` runs
+  independent checkpoints concurrently through the existing `Nexo.concurrent`
+  driver, persisting **each step as it completes** so a resume after a partial
+  failure re-runs only the still-missing steps. Each newly-completed step emits a
+  `"checkpoint"`-typed event (name-only) on the existing `nexo.workflow.event`
+  seam. Reuses `state`/`save_state!` — no schema, run-status, or store change.
+
 ## [0.7.0] - 2026-07-10
 
 The harness fills out: MCP data sources, a web-content capability, durable

data/docs/concurrency.md

@@ -41,6 +41,12 @@ most important knob for staying under provider rate limits.
 Using `Nexo.concurrent` with `async` not installed raises
 `Nexo::MissingDependencyError` with install guidance.
 
+Inside a durable workflow, `Workflow#checkpoint_all` is the workflow-durability
+flavored sibling of `Nexo.concurrent`: it drives this same bounded fan-out but
+persists each step to the run's `state` as it lands, so a resume only re-runs what
+never completed. See [Parallel checkpoints](durable-workflows.md#parallel-checkpoints--checkpoint_all)
+in the durable-workflows guide.
+
 ## `Sandboxes::Local` offload
 
 Under a reactor, blocking file/subprocess I/O would stall every other fiber. Flip
@@ -91,4 +97,32 @@ Note that DB work under a reactor is *offloaded/pooled*, not truly fiber-async
 Nexo does not ship a fiber-native DB driver. For server setup (Falcon, the fiber
 scheduler), see the [`async` guide](https://socketry.github.io/async/).
 
+
+## Tool concurrency — several tool calls in one turn
+
+`Nexo.config.concurrency` governs how **Nexo** runs blocking work. A separate setting
+governs how **RubyLLM** executes multiple tool calls returned in a *single* assistant
+turn:
+
+```ruby
+Nexo.configure { |c| c.tool_concurrency = :fibers }   # :fibers | :threads | false | nil
+```
+
+- `:fibers` — via `async`, the same driver `Nexo.concurrent` uses.
+- `:threads` — OS threads.
+- `false` — one at a time.
+- `nil` (**default**) — leave RubyLLM's own setting alone, so behaviour is unchanged
+  until you opt in.
+
+It is applied to the chat after every tool is attached, because each `with_tools` call
+resets the chat's concurrency.
+
+This only matters when a model returns several tool calls in one turn. Tools that batch
+their own work — one call that reads twenty messages rather than twenty calls — make it
+largely moot, and are the better optimization where you control the tool.
+
+**Your tools must be safe to run concurrently before you turn this on.** Anything sharing
+a process-wide resource — a CLI that serializes on a keychain, a single connection, a
+non-reentrant client — needs its own lock regardless of this setting.
+
 ← Back to the [README](../README.md)

data/docs/durable-workflows.md

@@ -64,6 +64,52 @@ DocumentApproval.resume_later(run.id, { approved: true }, queue: :nexo)
 See [`examples/approval_workflow.rb`](../examples/approval_workflow.rb) for the full
 offline flow (`ruby -Ilib examples/approval_workflow.rb`).
 
+## Parallel checkpoints — `checkpoint_all`
+
+When several checkpoints are **independent** (no step depends on another's
+result), run them concurrently with `checkpoint_all(name => callable, …)` instead
+of a sequence of `checkpoint` calls. It fans the pending steps out through
+[`Nexo.concurrent`](concurrency.md) — all in flight at once — and persists **each
+step as it completes** (not the batch as a whole), so a resume after a partial
+failure only re-runs the steps that never landed:
+
+```ruby
+class BuildDashboard < Nexo::Workflow
+  def call(payload)
+    data = checkpoint_all(
+      account: -> { fetch_account(payload[:id]) },   # these two run
+      usage:   -> { fetch_usage(payload[:id]) }      # concurrently
+    )
+    { report: render(data[:account], data[:usage]) }
+  end
+end
+```
+
+`checkpoint_all` returns a Hash keyed by the **original** names you passed
+(`data[:account]`), with values read back from `state` — the same shape whether a
+value came from this pass or a prior one. Each newly-completed step also surfaces a
+`"checkpoint"`-typed event on the run's event log and the `nexo.workflow.event`
+notification (data is the step **name** only, never the value — so a dashboard can
+show batch progress without the event log carrying large or sensitive results).
+Steps already present in `state` are skipped silently and emit nothing.
+
+Bound the batch by how many keys you pass — there is no separate rate knob; every
+pending step goes in flight. Because it drives `Nexo.concurrent`, `checkpoint_all`
+needs the `async` gem **only when something is actually pending** — an
+all-persisted pass (every step already done) returns the prior values directly
+without touching concurrency. The same restrictions as `checkpoint` apply: values
+must be json-serializable, a step must **not** be named after a reserved state key
+(`__suspend__`/`__approval__`/`__buffer_events__` — raises `Nexo::Error` before any
+step runs), and do **not** call `suspend!` inside a step (undefined — unsupported).
+
+> **⚠️ Known trade-off: per-step persistence, not an atomic batch.** `checkpoint_all`
+> is **not** transactional. If step B raises after step A persisted, A stays in
+> `state`, B is absent, the run goes `"failed"`, and the exception propagates through
+> the workflow's normal failure path (`Nexo.concurrent`'s "first failure re-raises,
+> the rest stop" — it is not rescued away). A subsequent `execute` of the **same** run
+> re-submits only the still-missing names — A is skipped, B re-runs. Do **not** treat
+> a batch as all-or-nothing.
+
 ## Durable **agent** approval — `:approve` (bridge a mid-run gate to a suspend)
 
 The example above suspends at an **explicit** `suspend!` the workflow author placed.

data/docs/loops.md

@@ -90,4 +90,31 @@ NEXO_LIVE=1 NEXO_MODEL=gemma3:12b bundle exec rake test TEST=test/live_smoke_tes
 If Gemma's tool-calling proves too weak, point `NEXO_MODEL` at a stronger model — the gem
 stays provider-neutral; only the smoke target changes.
 
+
+## `max_turns` is a budget, not a hard stop
+
+`Agent#prompt`, `Session#prompt` and `Workflow#run_agent` all take `max_turns:`. On the
+default `Loops::RubyLLM` it **cannot halt a run**: ruby_llm executes the entire tool loop
+inside `Chat#ask`, and the callbacks Nexo wires are observation-only.
+
+What it does do is count tool calls and tell you when the budget is passed:
+
+```ruby
+agent.prompt("Triage the inbox", max_turns: 10) do |type, payload|
+  warn "over budget: #{payload}" if type == :turn_limit_exceeded
+end
+# => :turn_limit_exceeded, { turns: 11, max_turns: 10 }
+```
+
+Emitted once per prompt, not per call.
+
+If you need a **hard** ceiling, the options are:
+
+- bound the work inside your own tools (a read tool that accepts at most N ids);
+- use `Loops::AgentSDK`, whose engine enforces `max_turns` natively;
+- stop the run yourself from the event block.
+
+Treat `max_turns` on the default loop as telemetry. It was previously accepted and never
+read, which made it look like a safety bound it has never been.
+
 ← Back to the [README](../README.md)

data/docs/mcp.md

@@ -141,4 +141,30 @@ deny-all). Attaching an authenticated server adds no permission surface.
 server. Without it installed, `require "nexo"` still loads; building a server raises a clear
 `Nexo::MissingDependencyError` telling you to add `gem "ruby_llm-mcp"`.
 
+
+## Decorating MCP tools
+
+`wrap_mcp_tool` is called once per MCP tool, after it is wrapped in a permission-gating
+`MCP::GatedTool` and before it is attached. Override it to decorate every MCP tool an
+agent gets — a third-party server's response shape is not yours to change, and its tools
+are attached by the harness rather than by you:
+
+```ruby
+class MailAgent < Nexo::Agent
+  mcp :mail, transport: :stdio, command: "apple-mail-mcp"
+
+  # This server returns an entire message body with no cap; bound it.
+  def wrap_mcp_tool(tool)
+    CappedTool.new(tool: tool, max_chars: 4_000)
+  end
+end
+```
+
+A wrapper must keep the duck type the chat relies on — `#name`, `#description`,
+`#params_schema` and `#call`. `GatedTool` delegates the rest through `method_missing`, so
+wrappers compose.
+
+Gating happens **underneath** the wrapper, so it only ever sees an already-authorized
+call. Wrapping cannot widen what an agent may do.
+
 ← Back to the [README](../README.md)

data/docs/rails.md

@@ -35,6 +35,39 @@ GenerateReport.run_later(account_id: 42, queue: :nexo)  # per-call
 Nexo.configure { |c| c.job_queue = :nexo }              # or a global default
 ```
 
+### Scheduling a future run or resume
+
+`run_later` and `resume_later` accept `wait:` (a duration) or `wait_until:` (an
+absolute time), forwarded straight to **the installed ActiveJob's own**
+`.set(...)` scheduler — Nexo adds no scheduler of its own. Use them to defer an
+initial enqueue ("send this digest at 9am") or to let a suspended run wake itself
+on a timer, symmetrically:
+
+```ruby
+# Defer the initial enqueue until tomorrow morning.
+DailyDigest.run_later({account_id: 42}, wait_until: Date.tomorrow.noon)
+
+# Let a suspended run wake itself up in an hour (no separate scheduled job).
+MyWorkflow.resume_later(run.id, {reminder: true}, wait: 1.hour)
+```
+
+The run's status is unchanged — a scheduled `run_later` is still `"queued"` (no
+`"scheduled"` status is invented), and a scheduled `resume_later` leaves the run
+`"suspended"` until the job fires. Passing **both** `wait:` and `wait_until:` in
+one call raises `ArgumentError` (checked before any run is created or job
+enqueued) — the installed ActiveJob would otherwise silently keep just one. With
+neither given, the enqueue is byte-for-byte the immediate one above.
+
+> **⚠️ `wait:`/`wait_until:`/`queue:` are scheduling options, not payload.** Like
+> `queue:` already was, a bare-keyword call consumes them as options: `run_later(wait:
+> 60)` schedules the job **60 seconds out** and leaves the payload `{}` — it does
+> **not** store `"wait" => 60` as data. A payload that legitimately needs a key named
+> `"wait"` must be passed as an explicit positional Hash: `run_later({wait: "value"})`.
+
+This is still "no scheduler, no cron" — `wait:`/`wait_until:` schedule a **single**
+future run/resume via ActiveJob; recurring schedules stay the host's (see the "no
+queue and no scheduler" note below).
+
 Nexo ships **no queue and no scheduler** — ActiveJob uses whatever adapter your
 app configured (Sidekiq, GoodJob, Solid Queue, …), and scheduling (cron / GoodJob
 / `whenever`) stays the host's. Without ActiveJob, `run_later` raises

data/docs/sandboxes.md

@@ -138,6 +138,30 @@ Nexo ships **only** `Remote` plus this documented pattern — purpose-built
 `Sandboxes::E2B` / `Sandboxes::Daytona` classes are a possible future addition, deliberately
 left out of v1 because their vendor client APIs aren't pinned yet.
 
+### Path confinement is the client's job
+
+`Local` and `Container` confine every `read`/`write`/`glob` to their working directory and
+raise `SecurityError` on escape. **`Remote` does not.** It passes each path to the injected
+client untouched, because only the provider knows what its own boundary is.
+
+That is deliberate — Nexo re-implementing a confinement it cannot enforce would be
+theatre — but it moves a guarantee you may be relying on. A shim is responsible for:
+
+- rejecting paths that escape the session's working directory;
+- deciding what `write` does with a missing parent directory (`Local` creates it,
+  `Container` creates it — a shim should too, or say that it does not);
+- raising on failure rather than returning a non-zero status the caller may ignore.
+
+Give the shim an `instructions:` describing the environment, too — `Remote` cannot derive
+one, and an agent told nothing about where it runs will assume it is on the host:
+
+```ruby
+Nexo::Sandboxes::Remote.new(
+  client: my_client,
+  instructions: "You run in an E2B sandbox, cwd /home/user. Ruby 3.4 is available."
+)
+```
+
 ## Container sandbox — Docker / Apple Container
 
 `Sandboxes::Container` runs an agent's tools inside a **throwaway OCI container** via the
@@ -161,12 +185,50 @@ the host dir enters only through a `binds:` entry.
 
 `sandbox :docker` (or `runtime: :docker`) shells out to `docker`; `sandbox :apple`
 (`runtime: :apple`) shells out to Apple's `container` binary. The `run`/`exec` surface is
-largely shared; where the CLIs diverge (networking especially) the class branches on the
-runtime. **Apple `container` parity is NOT yet verified** — the flags are encoded from the
-reference mapping, not confirmed against a live daemon, so every Apple flag must be verified
-before trust (the networking flag in particular; see the parity table below). An unknown
+largely shared; where the CLIs diverge the class branches on the runtime. An unknown
 runtime raises `Nexo::ConfigurationError`.
 
+**Apple `container` parity is now verified** against `container` 1.2.2 on macOS. Three
+flags docker accepts are rejected outright, so they are omitted for `:apple`:
+
+| Knob | `:docker` | `:apple` |
+|---|---|---|
+| `--security-opt no-new-privileges` | applied | **rejected** (`Unknown option`) — omitted |
+| `--pids-limit` | applied (`512`) | **rejected** (`Unknown option`) — omitted |
+| `--read-only` + `--tmpfs <cwd>:rw` | scratch is writable | flags accepted, but **the scratch is NOT writable** |
+
+Everything else — `--network` (including `none`), `--cap-drop ALL`, `--memory`,
+`--cpus`, `--user`, `-v` binds, `-e` env, `-w`, `exec -i` — is honored identically.
+
+> **`readonly_rootfs: true` is the default, and on `:apple` it leaves you with no
+> writable workspace.** Apple accepts `--tmpfs /workspace:rw` but under `--read-only`
+> the mount is still read-only, so every `write` fails. Pass `readonly_rootfs: false`,
+> or mount a `:rw` bind, when using the `:apple` runtime.
+
+Because omitting a hardening flag means *weaker isolation than you asked for*, the
+difference is reported rather than hidden:
+
+```ruby
+sandbox = Nexo::Sandboxes::Container.new(image: "…", runtime: :apple, network: :none)
+sandbox.hardening_gaps
+# => ["--security-opt no-new-privileges is not supported by apple",
+#     "--pids-limit is not supported by apple",
+#     "apple has no 'none' network; using 'default' instead"]
+```
+
+`hardening_gaps` is empty on `:docker`. If your threat model requires one of those knobs,
+check it and refuse rather than assuming every runtime honors every flag.
+
+Two practical notes for Apple `container`:
+
+- It resolves its **init image through the keychain** and fails in a non-interactive
+  shell (`errSecAuthFailed`, status `-25293`). Pinning an already-pulled tag avoids the
+  lookup: `container run --init-image ghcr.io/apple/containerization/vminit:<tag> …`.
+- Containers are **VM-backed**, so infrastructure failures surface differently — e.g.
+  `no available interface strategy for network default, plugin=container-network-vmnet`
+  when the network service needs restarting (`container system stop && container system
+  start`).
+
 ### Hardened by default — every knob an explicit opt-out
 
 All of the following are applied to the `run` argv by default and individually invertible:
@@ -239,9 +301,9 @@ The container starts **lazily** on first tool use and its id is memoized.
   `:rw` bind (this is where staged files and artifacts land).
 - **Non-root is recommended, not forced.** The default hardening holds regardless of uid; set
   `user:` for defense-in-depth.
-- **Apple `container` parity is NOT yet verified** — especially networking. Every Apple flag
-  in the parity table below is UNVERIFIED; confirm the flag/subcommand against Apple's CLI
-  before trusting the `:apple` runtime in production.
+- **Apple `container` needs `readonly_rootfs: false`.** The default read-only rootfs makes the
+  `--tmpfs` scratch unwritable on Apple, so an agent cannot write to its own workspace. The
+  parity table below records what was verified.
 - **Reconnect is Docker-only today.** `reconnect: true` combined with `runtime: :apple` raises
   `Nexo::ConfigurationError` at the point reconnect would run. Apple's `container` CLI has no
   **live-verified** exact `label=` filter, and a name-substring match is unsafe (it can attach
@@ -251,40 +313,35 @@ The container starts **lazily** on first tool use and its id is memoized.
 
 #### Apple `container` parity table (Group 0)
 
-This table is filled **only** from live Group 0 runs against a real Apple `container` runtime —
-never from assumption. Apple's runtime is macOS-only and is **not** present in CI (or in the
-environment this spec shipped from), so the divergences below are **UNVERIFIED** and marked as
-such. Until a maintainer runs Group 0 on Apple hardware and records the results here, treat the
-`:apple` runtime as **Docker-flag-shaped but unconfirmed** and reconnect as unsupported (it
-raises). Do not silently trust any Apple hardening flag until its row reads `same`.
+Filled from live Group 0 runs against **Apple `container` 1.2.2 on macOS (arm64)** and
+**Docker 29.4.0**, image `ubuntu:24.04`. Rows marked _unverified_ have not been run.
 
 | Subcommand / flag | Docker | Apple `container` | Divergence → action |
 |---|---|---|---|
-| `run -d` | ✓ | _unverified_ | verify before trust |
-| `exec -i` | ✓ | _unverified_ | verify before trust |
-| `ps -aqf` | ✓ | _unverified_ | verify before trust |
-| `start <id>` | ✓ | _unverified_ | verify before trust |
-| `rm -f <id>` | ✓ | _unverified_ | verify before trust |
-| `--label` / `label=` filter | ✓ (exact) | _unverified_ | **reconnect raises `ConfigurationError` until confirmed** |
-| `--network` | ✓ | _unverified_ | expected to diverge — verify before trust |
-| `--tmpfs` | ✓ | _unverified_ | verify before trust |
-| `--read-only` | ✓ | _unverified_ | verify before trust |
-| `--cap-drop` / `--cap-add` | ✓ | _unverified_ | verify before trust |
-| `--security-opt no-new-privileges` | ✓ | _unverified_ | verify before trust |
-| `--pids-limit` | ✓ | _unverified_ | verify before trust |
-| `-w` / `-v` / `-e` / `--user` / `--memory` / `--cpus` | ✓ | _unverified_ | verify before trust |
-
-Keep-alive (Docker, Group 0): `tail -f /dev/null` holds Alpine / Debian-slim / ruby-slim open —
-**to be confirmed on the maintainer's live daemon** (busybox-portable by construction; `sleep
-infinity` is not and was replaced for exactly this reason).
-
-> **Reduced-guarantee posture.** Where a hardening flag turns out to have no Apple equivalent,
-> the container stays functional but the guarantee is **reduced** — and that reduction is
-> documented here, never silently dropped. Until the table above is filled from a live run, a
-> maintainer must not assume any given `:apple` hardening flag is honored.
-
-Live container runs are exercised by `NEXO_LIVE`-gated smoke
-(`test/sandboxes/container_live_test.rb`); the core suite asserts argv construction with no
-daemon. See `examples/container_review.rb` for a runnable end-to-end example.
+| `run -d` | ✓ | ✓ | same |
+| `exec -i` (stdin reaches the process) | ✓ | ✓ | same — this is what `#write` uses |
+| `stop` / `start <id>` | ✓ | ✓ | same |
+| `rm -f <id>` | ✓ | ✓ | same |
+| `ps -aqf label=` | ✓ (exact) | ✗ `Plugin 'container-ps' not found` | Apple has `list`, not `ps` → **reconnect raises `ConfigurationError`** |
+| `--network` (incl. `none`) | ✓ | ✓ | same — `--network none` runs |
+| `--cap-drop ALL` | ✓ | ✓ | same |
+| `--read-only` | ✓ | ✓ | rootfs is read-only on both |
+| `--tmpfs <cwd>:rw` | ✓ writable | ✗ **not writable under `--read-only`** | → omit `--read-only` (`readonly_rootfs: false`) or use a `:rw` bind |
+| `--security-opt no-new-privileges` | ✓ | ✗ `Unknown option` | → omitted for `:apple`, reported in `#hardening_gaps` |
+| `--pids-limit` | ✓ | ✗ `Unknown option` | → omitted for `:apple`, reported in `#hardening_gaps` |
+| `-v host:ctr:ro` | ✓ | ✓ | same |
+| `-e KEY=val` | ✓ | ✓ | same |
+| `-w <dir>` | ✓ | ✓ | same |
+| `--user` | ✓ | ✓ | same |
+| `--memory` / `--cpus` | ✓ | ✓ | same |
+
+Two operational notes for Apple, both hit during verification:
+
+- It resolves its **init image via the keychain** and fails in a non-interactive shell
+  (`errSecAuthFailed`, `-25293`). Pin an already-pulled tag to skip the lookup:
+  `--init-image ghcr.io/apple/containerization/vminit:<tag>`.
+- Containers are **VM-backed**, so infrastructure failures look unfamiliar — e.g.
+  `no available interface strategy for network default, plugin=container-network-vmnet`,
+  cleared by `container system stop && container system start`.
 
 ← Back to the [README](../README.md)

data/docs/skills.md

@@ -61,8 +61,58 @@ skill that does not exist raises `Nexo::Error` naming the missing `SKILL.md` pat
 
 A skill contributes **instructions only**. A loaded skill ships no independent tools, and
 Nexo deliberately does not attach `ruby_llm-skills`' progressive-disclosure tool (which
-reads files outside the sandbox). The model reaches a skill's `references/`/`scripts/`
-files through Nexo's own permission-gated, sandbox-backed tools — so **attaching a skill
-never widens what an agent can do** beyond its configured sandbox/permission mode.
+reads files outside the sandbox) — so **attaching a skill never widens what an agent can
+do** beyond its configured sandbox/permission mode.
+
+## Bundled files are not reachable until you stage them
+
+A skill's `scripts/`, `assets/` and `references/` live under `skills_path`, which is
+**outside every sandbox**. Sandboxes confine file access to their own working directory
+(`Local` and `Container` raise `SecurityError` on escape), so an agent cannot read or run
+a skill's bundled files just because the skill is attached.
+
+`Skills.materialize` copies them in, through the sandbox's own `#write` — the only route
+that works on every tier:
+
+```ruby
+staged = Nexo::Skills.materialize(:dashboard_designer, into: agent.sandbox)
+# => { scripts: ["scripts/render_dashboard.rb"],
+#      assets:  ["assets/dashboard-template.html"] }
+```
+
+They are then ordinary workspace files, reached through the permission-gated `read`,
+`glob` and `shell` tools like anything else in the workspace — and the returned paths are
+**sandbox-relative**, so you can build a command without knowing which tier you are on:
+
+```ruby
+agent.prompt("Run: ruby #{staged[:scripts].first} digest.json #{staged[:assets].first} out.html")
+```
+
+`Local` writes to the filesystem, `Container` streams over `docker exec` / `container
+exec`, and `Remote` hands off to the injected client. The call is identical.
+
+| Option | Meaning |
+|---|---|
+| `kinds:` | which of `scripts` / `assets` / `references` to copy (default: all three) |
+| `overwrite:` | `false` skips files already present — what you want against an image that bakes the skill in |
+
+Kinds the skill ships nothing for are omitted from the result.
+
+Two things to get right:
+
+- **Use relative paths** (`"scripts/render.rb"`), in the write and in any command you
+  hand the agent. A host-absolute path is meaningless inside a container and on a remote
+  sandbox.
+- **A bundled script must not depend on ambient environment.** It runs wherever the
+  sandbox is, which may have no locale (Ruby then defaults to `US-ASCII`, and reading a
+  UTF-8 file raises `Encoding::InvalidByteSequenceError`), a different `PATH`, or no
+  interpreter at all. Be explicit — `File.read(path, encoding: "UTF-8")` — and document
+  which interpreter your skill needs.
+
+Staged files land in **writable** space. If the agent also holds `:shell` it can rewrite a
+script before running it — where the same file, left outside the sandbox, could not be
+touched at all. `materialize` overwrites by default, so re-materializing at the start of
+every run bounds tampering to a single turn. If that is not enough, keep the resource out
+of the sandbox and feed the agent its contents another way.
 
 ← Back to the [README](../README.md)

data/lib/nexo/agent.rb

@@ -246,6 +246,7 @@ module Nexo
       apply_mcp(c)
       apply_fetch(c)
       apply_search(c)
+      apply_tool_concurrency(c)
       c
     end
 
@@ -360,11 +361,49 @@ module Nexo
 
       @mcp_clients ||= self.class.mcp.map { |cfg| Nexo::MCP.build(**cfg) }
       gated = @mcp_clients.flat_map(&:tools).map do |tool|
-        Nexo::MCP::GatedTool.new(tool: tool, permissions: @permissions)
+        wrap_mcp_tool(Nexo::MCP::GatedTool.new(tool: tool, permissions: @permissions))
       end
       chat.with_tools(*gated) unless gated.empty?
     end
 
+    # Applies Nexo.config.tool_concurrency to the chat, LAST — every +with_tools+ call
+    # resets the chat's concurrency to its current value, so setting it before the
+    # tools are attached would be undone. +nil+ leaves RubyLLM's own setting alone.
+    def apply_tool_concurrency(chat)
+      mode = Nexo.config.tool_concurrency
+      return if mode.nil?
+      return unless chat.respond_to?(:with_tools)
+
+      chat.with_tools(concurrency: mode)
+    end
+
+    # Hook: called once per MCP tool, AFTER it is wrapped in a permission-gating
+    # MCP::GatedTool and before it is attached. Returns the tool to attach; the
+    # default returns it unchanged.
+    #
+    # Override to decorate every MCP tool an agent gets — capping an oversized reply,
+    # recording timings, redacting a field. A third-party MCP server's response shape
+    # is not yours to change, and its tools are attached by the harness rather than by
+    # you, so without this seam the only way in was to override the private
+    # +#apply_mcp+.
+    #
+    #   class MailAgent < Nexo::Agent
+    #     mcp :mail, transport: :stdio, command: "apple-mail-mcp"
+    #
+    #     def wrap_mcp_tool(tool)
+    #       CappedTool.new(tool: tool, max_chars: 4_000)
+    #     end
+    #   end
+    #
+    # A wrapper must keep the duck type the chat relies on — +#name+, +#description+,
+    # +#params_schema+ and +#call+. GatedTool delegates the rest through
+    # +method_missing+, so wrappers compose. Gating happens underneath, so a wrapper
+    # only ever sees an already-authorized call: wrapping cannot widen what the agent
+    # may do.
+    def wrap_mcp_tool(tool)
+      tool
+    end
+
     # Attaches a single Nexo::Tools::Fetch scoped to the agent's +fetch_allow+
     # hosts (Spec 9). Returns early when no host is declared, so an agent that never
     # calls +fetch_allow+ gets no fetch tool. Attached right after +apply_mcp+ so

data/lib/nexo/configuration.rb

@@ -25,6 +25,14 @@ module Nexo
     # which always runs its own reactor when called.
     attr_accessor :concurrency
 
+    # How RubyLLM executes MULTIPLE tool calls returned in a single assistant turn:
+    # +:fibers+ (via +async+, the same driver Nexo.concurrent uses), +:threads+, or
+    # +false+ for one at a time. +nil+ (the default) leaves RubyLLM's own setting
+    # alone, so existing behaviour is unchanged until you opt in.
+    #
+    # Distinct from #concurrency, which governs how Nexo itself runs blocking work.
+    attr_accessor :tool_concurrency
+
     # Max in-flight tasks for Nexo.concurrent fan-out (Spec 5), default +8+.
     # The single most important knob for staying under provider rate limits.
     attr_accessor :max_in_flight

data/lib/nexo/loops/ruby_llm.rb

@@ -22,10 +22,19 @@ module Nexo
       # +chat:+ lets a Nexo::Session inject the hydrated, continuing chat so the
       # loop runs over the persisted thread; left nil it builds the agent's own
       # fresh chat exactly as before — the default (no-session) path is unchanged.
+      # +max_turns+ is a BUDGET, not a hard stop. ruby_llm runs the whole tool loop
+      # inside #ask and its callbacks cannot halt it, so this loop cannot cut a run
+      # short. What it can do is COUNT the tool calls and say so: exceeding the budget
+      # emits a +:turn_limit_exceeded+ event (once) carrying the count and the limit.
+      #
+      # Previously the parameter was accepted and never read, which made it look like
+      # a safety bound it has never been. Treat it as telemetry: if you need a hard
+      # ceiling, bound the work inside your tools, or use Loops::AgentSDK whose engine
+      # enforces its own max_turns natively.
       def run(agent:, prompt:, max_turns: 25, chat: nil, &on_event)
         chat ||= agent.chat
 
-        wire_observability(chat, &on_event)
+        wire_observability(chat, max_turns: max_turns, &on_event)
 
         response = chat.ask(prompt)
         on_event&.call(:done, response)
@@ -47,20 +56,39 @@ module Nexo
       # prompt observes with its own block rather than the first one. A fresh chat
       # (the default per-prompt path) simply wires once — byte-for-byte the prior
       # behavior.
-      def wire_observability(chat, &on_event)
+      def wire_observability(chat, max_turns: nil, &on_event)
         return unless chat.respond_to?(:before_tool_call) && chat.respond_to?(:after_tool_result)
 
         chat.instance_variable_set(:@nexo_on_event, on_event)
+        chat.instance_variable_set(:@nexo_max_turns, max_turns)
+        # The count belongs to the PROMPT, not the chat: a continuing Session runs the
+        # loop repeatedly over one chat, and each prompt gets its own budget.
+        chat.instance_variable_set(:@nexo_turns, 0)
+        chat.instance_variable_set(:@nexo_turns_reported, false)
         return if chat.instance_variable_get(:@nexo_observed)
 
         chat.instance_variable_set(:@nexo_observed, true)
         chat.before_tool_call do |tc|
+          count_turn(chat)
           chat.instance_variable_get(:@nexo_on_event)&.call(:tool_call, tc)
         end
         chat.after_tool_result do |r|
           chat.instance_variable_get(:@nexo_on_event)&.call(:tool_result, r)
         end
       end
+
+      # Counts one tool call and reports the first time the budget is passed. Reported
+      # once per prompt, not per call, so a long run does not drown its own event log.
+      def count_turn(chat)
+        limit = chat.instance_variable_get(:@nexo_max_turns)
+        turns = chat.instance_variable_get(:@nexo_turns).to_i + 1
+        chat.instance_variable_set(:@nexo_turns, turns)
+        return unless limit && turns > limit
+        return if chat.instance_variable_get(:@nexo_turns_reported)
+
+        chat.instance_variable_set(:@nexo_turns_reported, true)
+        chat.instance_variable_get(:@nexo_on_event)&.call(:turn_limit_exceeded, {turns: turns, max_turns: limit})
+      end
     end
   end
 end

data/lib/nexo/sandboxes/container.rb

@@ -39,6 +39,23 @@ module Nexo
       # supported local runtimes in v1.
       RUNTIMES = {docker: "docker", apple: "container"}.freeze
 
+      # What each runtime's CLI can actually express. Verified live against
+      # +container+ 1.2.2 on macOS and Docker 29.4: Apple's CLI rejects
+      # +--security-opt+ and +--pids-limit+ with "Unknown option", which aborts
+      # +container run+ before the sandbox ever starts.
+      #
+      # +writable_tmpfs+ is not about the flag being accepted — Apple accepts
+      # +--tmpfs+ — but about what it does: combined with +--read-only+ the mount is
+      # NOT writable there, so a read-only rootfs leaves no usable scratch.
+      #
+      # Anything a runtime cannot honor is reported through #hardening_gaps rather
+      # than dropped quietly: running with weaker isolation, or with a workspace you
+      # cannot write to, must be visible.
+      CAPABILITIES = {
+        docker: {security_opt: true, pids_limit: true, writable_tmpfs: true},
+        apple: {security_opt: false, pids_limit: false, writable_tmpfs: false}
+      }.freeze
+
       # The container working directory (default +/workspace+, a container path),
       # the selected +runtime+ (+:docker+ / +:apple+), and the required +image+.
       attr_reader :cwd, :runtime, :image
@@ -81,10 +98,20 @@ module Nexo
         out[:stdout]
       end
 
-      # Writes +content+ to +path+ (guarded) inside the container. Content travels
-      # on stdin, never interpolated into the argv, so arbitrary bytes are safe.
+      # Writes +content+ to +path+ (guarded) inside the container, creating parent
+      # directories first. Content travels on stdin, never interpolated into the argv,
+      # so arbitrary bytes are safe.
+      #
+      # Matches Local#write on both counts, which it previously did not: without the
+      # +mkdir+ a nested path failed with "Directory nonexistent", and because the
+      # exit status was discarded the caller was told the write had succeeded. Raises
+      # +IOError+ on failure, mirroring how #read raises +Errno::ENOENT+.
       def write(path, content)
-        exec_stdin!(content, "sh", "-c", 'cat > "$0"', guard_path(path))
+        full = guard_path(path)
+        out = exec_stdin!(content, "sh", "-c", 'mkdir -p "$(dirname "$0")" && cat > "$0"', full)
+        raise IOError, "container write failed: #{path}: #{out[:stderr].strip}" unless out[:status].zero?
+
+        full
       end
 
       # Returns the container paths matching the glob +pattern+ (guarded). Empty
@@ -120,6 +147,20 @@ module Nexo
         @cid = nil
       end
 
+      # Hardening the caller asked for that this runtime cannot express, as short
+      # human-readable strings. Empty on +:docker+. Callers that require a guarantee
+      # should check this rather than assume every runtime honors every knob.
+      def hardening_gaps
+        gaps = []
+        gaps << "--security-opt no-new-privileges is not supported by #{@runtime}" unless capable?(:security_opt)
+        gaps << "--pids-limit is not supported by #{@runtime}" if @pids_limit && !capable?(:pids_limit)
+        if @readonly_rootfs && !capable?(:writable_tmpfs)
+          gaps << "#{@runtime} --tmpfs is not writable under --read-only; " \
+                  "set readonly_rootfs: false or add a :rw bind for a writable workspace"
+        end
+        gaps
+      end
+
       # A short, human-readable description of the execution environment for the
       # agent to inject later (consumed by the Refinements spec; until then the
       # method simply exists and is correct).
@@ -160,11 +201,11 @@ module Nexo
         argv = [@bin, "run", "-d", "--name", @name,
           "--label", "nexo.sandbox.id=#{@name}",
           "--network", @network.to_s,
-          "--cap-drop", "ALL",
-          "--security-opt", "no-new-privileges"]
+          "--cap-drop", "ALL"]
+        argv += ["--security-opt", "no-new-privileges"] if capable?(:security_opt)
         @cap_add.each { |cap| argv += ["--cap-add", cap.to_s] }
         argv += ["--read-only", "--tmpfs", "#{@cwd}:rw"] if @readonly_rootfs
-        argv += ["--pids-limit", @pids_limit.to_s] unless @pids_limit.nil?
+        argv += ["--pids-limit", @pids_limit.to_s] if @pids_limit && capable?(:pids_limit)
         argv += ["--memory", @memory.to_s] unless @memory.nil?
         argv += ["--cpus", @cpus.to_s] unless @cpus.nil?
         argv += ["--user", @user.to_s] unless @user.nil?
@@ -177,6 +218,11 @@ module Nexo
         argv
       end
 
+      # Whether this runtime's CLI can express +knob+.
+      def capable?(knob)
+        CAPABILITIES.fetch(@runtime, CAPABILITIES[:docker]).fetch(knob, true)
+      end
+
       # Builds one +-v host:ctr:mode+ spec. Binds default to +:ro+; a Hash form
       # +{ to:, mode: :rw }+ makes a single bind writable.
       def bind_spec(host, dst)

data/lib/nexo/sandboxes/remote.rb

@@ -25,8 +25,29 @@ module Nexo
     # default sandbox stays +:virtual+.
     class Remote < Sandbox
       # Stores any object responding to +read+/+write+/+exec+/+close+.
-      def initialize(client:)
+      #
+      # +instructions:+ describes the remote environment for the agent's system
+      # prompt — working directory, available tooling, what is writable. Local and
+      # Container derive that themselves; a remote sandbox cannot, because only the
+      # shim knows where it points. Supplying it is strongly recommended: the tier
+      # most likely to surprise a weak tool-caller is the one it knows least about.
+      def initialize(client:, instructions: nil)
         @client = client
+        @instructions = instructions
+      end
+
+      # A short, plain-text description of the execution environment. Falls back to an
+      # honest generic statement rather than +nil+, so an agent is never left assuming
+      # it runs on the host.
+      #
+      # NOTE: unlike Local and Container, +Remote+ performs NO path confinement — every
+      # path is passed to the client untouched. Confining the agent to a working
+      # directory is the client's responsibility, not Nexo's.
+      def instructions
+        @instructions ||
+          "You run inside a remote sandbox managed by an external provider. The " \
+          "working directory, available tooling, and writable paths are defined by " \
+          "that provider."
       end
 
       # Reads +path+ via the client.

data/lib/nexo/skills.rb

@@ -11,9 +11,15 @@ module Nexo
   # MissingDependencyError with install guidance.
   #
   # A loaded skill contributes its **instructions** (the SKILL.md body) to a chat.
-  # It ships no independent tools — its +scripts/+ and +references/+ files are
-  # reached by the model through Nexo's own sandbox-backed, permission-gated tools,
-  # so attaching a skill never widens the agent's effective capabilities. See the
+  # It ships no independent tools, so attaching a skill never widens the agent's
+  # effective capabilities.
+  #
+  # Its +scripts/+, +assets/+ and +references/+ files are NOT reachable on their
+  # own: a skill lives under +skills_path+, every sandbox confines file access to
+  # its own working directory, and nothing bridges the two. To let an agent read or
+  # run a skill's bundled files, copy them into the sandbox first — through the
+  # sandbox's own +#write+, so it works on every tier. They are then reached through
+  # Nexo's permission-gated tools like any other workspace file. See the
   # spec's "Verified APIs" / safety resolution for why the gem's progressive
   # disclosure +SkillTool+ (which does ungated +File.read+) is deliberately not
   # attached.
@@ -49,6 +55,75 @@ module Nexo
 
         RubyLLM::Skills.load(dir)
       end
+
+      # Copies a skill's bundled files INTO a sandbox, so an agent can read or run
+      # them through its permission-gated tools.
+      #
+      # A skill lives under +skills_path+, outside every sandbox, and each sandbox
+      # confines file access to its own working directory — so +scripts/+, +assets/+
+      # and +references/+ are unreachable until they are staged. This is that step.
+      #
+      # It goes through the sandbox's own +#write+, which is the ONLY route that works
+      # on every tier: +Local+ writes to the filesystem, +Container+ streams over
+      # +docker exec+ / +container exec+, and +Remote+ hands off to the injected
+      # client. Nothing here knows or cares which one it is.
+      #
+      #   Nexo::Skills.materialize(:dashboard_designer, into: agent.sandbox)
+      #   # => { scripts: ["scripts/render_dashboard.rb"],
+      #   #      assets:  ["assets/dashboard-template.html"] }
+      #
+      # Returns the SANDBOX-RELATIVE path of every file written, grouped by kind, so a
+      # caller can build a command without knowing where the sandbox lives. Build
+      # commands from these relative paths — a host-absolute path is meaningless
+      # inside a container and on another machine.
+      #
+      # Kinds a skill ships nothing for are omitted. +kinds:+ narrows the copy;
+      # +overwrite: false+ skips files already present, which is what you want against
+      # an image that already bakes the skill in.
+      #
+      # SECURITY: staged files land in WRITABLE space. An agent holding +:write+ and
+      # +:shell+ can rewrite a script before executing it — where the same file, left
+      # outside the sandbox, could not be touched at all. Re-materializing on every run
+      # bounds tampering to a single turn; if that is not enough, keep the resource out
+      # of the sandbox and feed the agent its contents another way.
+      def materialize(name, into:, kinds: %i[scripts assets references], overwrite: true)
+        skill = name.respond_to?(:scripts) ? name : find(name)
+
+        Array(kinds).each_with_object({}) do |kind, staged|
+          files = resources(skill, kind)
+          next if files.empty?
+
+          staged[kind] = files.map { |host_path| copy_in(host_path, kind, into, overwrite) }
+        end
+      end
+
+      private
+
+      # A skill's files of one kind, as host paths. Virtual (database-backed) skills
+      # have no filesystem resources and answer with an empty list.
+      def resources(skill, kind)
+        return [] unless %i[scripts assets references].include?(kind.to_sym)
+        return [] unless skill.respond_to?(kind)
+
+        Array(skill.public_send(kind))
+      end
+
+      # Writes one host file into the sandbox under +<kind>/<basename>+, preserving the
+      # layout a skill's own prose refers to ("run scripts/render.rb").
+      def copy_in(host_path, kind, sandbox, overwrite)
+        relative = File.join(kind.to_s, File.basename(host_path))
+        return relative if !overwrite && present?(sandbox, relative)
+
+        sandbox.write(relative, File.binread(host_path))
+        relative
+      end
+
+      def present?(sandbox, relative)
+        sandbox.read(relative)
+        true
+      rescue
+        false
+      end
     end
   end
 end

data/lib/nexo/version.rb

@@ -2,5 +2,5 @@
 
 module Nexo
   # The gem version (also used as +spec.version+ in the gemspec).
-  VERSION = "0.7.0"
+  VERSION = "0.8.1"
 end

data/lib/nexo/workflow.rb

@@ -40,6 +40,13 @@ module Nexo
   # expensive/side-effectful steps so resume skips already-paid-for work — see the
   # "Durable workflows" README section for the honest resume semantics.
   class Workflow
+    # State keys Nexo reserves for lifecycle metadata, never a caller's data:
+    # +"__suspend__"+ (suspend reason/resume_key — Spec 13), +"__approval__"+
+    # (pending approval call — Spec 16), and +"__buffer_events__"+ (the persisted
+    # buffering choice — Spec 5). ::cleared_state strips these when a run reaches
+    # +"done"+; #checkpoint_all refuses a step named after any of them (Spec 21).
+    RESERVED_STATE_KEYS = %w[__suspend__ __approval__ __buffer_events__].freeze
+
     # Control-flow signal raised by #suspend! and caught by ::execute to pause
     # a run durably — NOT a failure. It transitions the run to +"suspended"+
     # (never +"failed"+) and returns the run to the caller rather than re-raising.
@@ -165,11 +172,30 @@ module Nexo
       # also reachable. Not resumable: a crashed or retried job re-runs +#call+ from
       # scratch (Nexo adds no +retry_on+); pair with ::reconcile_interrupted! to
       # catch runs orphaned in +"running"+.
-      def run_later(payload = nil, queue: Nexo.config.job_queue, **kwargs)
+      #
+      # +wait:+ / +wait_until:+ (Spec 21) defer the enqueue via the installed
+      # ActiveJob's own +.set(...)+ scheduler — +wait:+ takes a duration
+      # (+wait: 1.hour+), +wait_until:+ an absolute time (+wait_until: tomorrow_9am+).
+      # Nexo adds no scheduler of its own; it just forwards these to +.set+. The run
+      # is still +"queued"+ (no +"scheduled"+ status is invented). Passing **both** in
+      # one call raises ArgumentError (the installed ActiveJob would silently keep one)
+      # — checked before any run is created. With neither given the enqueue is
+      # byte-for-byte the pre-Spec-21 immediate one (no +:at+ on the job).
+      #
+      # +wait:+/+wait_until:+/+queue:+ share the bare-keyword/positional ambiguity
+      # that +queue:+ already carried: a bare-keyword call like +run_later(wait: 60)+
+      # consumes +wait+ as the scheduling option (the payload stays +{}+). A payload
+      # that legitimately needs a key literally named +"wait"+ must be passed as an
+      # explicit positional Hash — +run_later({wait: "value"})+.
+      def run_later(payload = nil, queue: Nexo.config.job_queue, wait: nil, wait_until: nil, **kwargs)
         unless defined?(::ActiveJob)
           raise Nexo::MissingDependencyError,
             "run_later requires ActiveJob (Rails). Use `run` for synchronous execution."
         end
+        if wait && wait_until
+          raise ArgumentError,
+            "pass either `wait:` or `wait_until:`, not both (got both)"
+        end
         if payload && !kwargs.empty?
           raise ArgumentError,
             "pass the payload either as a positional Hash or as keywords, not both (got both)"
@@ -178,9 +204,7 @@ module Nexo
         run = Nexo::RunStore.default.create(workflow_class: name, payload: stringify(payload))
         run.update!(status: "queued")
         notify_status(run) # let a dashboard learn the run was enqueued
-        job = Nexo::WorkflowJob
-        job = job.set(queue: queue) if queue
-        job.perform_later(run.id)
+        enqueue_job(Nexo::WorkflowJob, queue: queue, wait: wait, wait_until: wait_until).perform_later(run.id)
         run
       end
 
@@ -229,15 +253,27 @@ module Nexo
       # Requires ActiveJob (Rails): without it this raises Nexo::MissingDependencyError
       # pointing at ::resume for synchronous execution. +queue:+ (default
       # Nexo.config.job_queue) routes the job exactly like ::run_later.
-      def resume_later(run_id, input = {}, queue: Nexo.config.job_queue)
+      #
+      # +wait:+ / +wait_until:+ (Spec 21) defer the resume via the installed
+      # ActiveJob's own +.set(...)+ — +wait:+ a duration (+resume_later(id, input,
+      # wait: 1.hour)+ so a suspended run wakes itself on a timer), +wait_until:+ an
+      # absolute time. Nexo adds no scheduler and no retry; a crashed scheduled-resume
+      # job remains the host's +reconcile_interrupted!+ / +retry_on+ story. Passing
+      # **both** raises ArgumentError (the installed ActiveJob would silently keep one),
+      # checked before the job is enqueued. With neither given the enqueue is
+      # byte-for-byte the pre-Spec-21 immediate one. The return value is unchanged: the
+      # run stays +"suspended"+ until the job fires and re-enters ::resume's atomic claim.
+      def resume_later(run_id, input = {}, queue: Nexo.config.job_queue, wait: nil, wait_until: nil)
         unless defined?(::ActiveJob)
           raise Nexo::MissingDependencyError,
             "resume_later requires ActiveJob (Rails). Use `resume` for synchronous execution."
         end
+        if wait && wait_until
+          raise ArgumentError,
+            "pass either `wait:` or `wait_until:`, not both (got both)"
+        end
         run = Nexo::RunStore.default.find(run_id)
-        job = Nexo::WorkflowJob
-        job = job.set(queue: queue) if queue
-        job.perform_later(run.id, input)
+        enqueue_job(Nexo::WorkflowJob, queue: queue, wait: wait, wait_until: wait_until).perform_later(run.id, input)
         run
       end
 
@@ -341,6 +377,22 @@ module Nexo
         )
       end
 
+      # Builds the ActiveJob dispatch for ::run_later/::resume_later, forwarding
+      # only the non-nil scheduling options to the installed ActiveJob's own
+      # +.set(...)+ (Spec 21). The options Hash is accumulated conditionally so that
+      # with no +queue+/+wait+/+wait_until+ there is no +.set+ call at all — keeping
+      # the no-scheduling-option enqueue byte-for-byte identical to the pre-Spec-21
+      # path (the job carries no +:at+). +wait:+ and +wait_until:+ are never both
+      # present here (the callers raise ArgumentError first), so ActiveJob never has
+      # to silently pick between them.
+      def enqueue_job(job, queue: nil, wait: nil, wait_until: nil)
+        options = {}
+        options[:queue] = queue if queue
+        options[:wait] = wait if wait
+        options[:wait_until] = wait_until if wait_until
+        options.empty? ? job : job.set(**options)
+      end
+
       # Persists a +true+ buffering choice under the reserved "__buffer_events__"
       # state key (idempotent — written once) so ::resume can honor it. The
       # unbuffered default writes nothing, keeping the Spec 2 hot path untouched.
@@ -358,11 +410,10 @@ module Nexo
       def cleared_state(run)
         return {} unless run.respond_to?(:state)
 
-        reserved = %w[__suspend__ __approval__ __buffer_events__]
         state = run.state || {}
-        return {} unless reserved.any? { |k| state.key?(k) }
+        return {} unless RESERVED_STATE_KEYS.any? { |k| state.key?(k) }
 
-        {state: state.except(*reserved)}
+        {state: state.except(*RESERVED_STATE_KEYS)}
       end
 
       def stringify(hash) = hash.transform_keys(&:to_s)
@@ -383,6 +434,11 @@ module Nexo
       @run = run
       @buffer_events = buffer_events
       @event_buffer = []
+      # Serializes the read-current → merge → assign → save_state! sequence across
+      # the concurrent fibers #checkpoint_all runs each step on (Spec 21). A plain
+      # Mutex is fiber-safe under the async reactor (Group 0 verified it serializes
+      # without stalling); #checkpoint (singular, no concurrency) does not use it.
+      @checkpoint_mutex = Mutex.new
     end
 
     # Subclasses implement the work here. The +payload+ is symbol-keyed; the
@@ -424,6 +480,62 @@ module Nexo
       value
     end
 
+    # Runs several independent checkpoints **concurrently** on the first pass and,
+    # crucially, persists **each step as it completes** — so a resume after a
+    # partial failure only re-runs the steps that never landed (Spec 21):
+    #
+    #   fetched = checkpoint_all(
+    #     account: -> { fetch_account(payload[:id]) },
+    #     usage:   -> { fetch_usage(payload[:id]) }
+    #   )
+    #   fetched[:account] # => the account value (this pass or a prior one)
+    #
+    # +steps+ is a Hash of +name => callable+ (each value a Proc/lambda); names may
+    # be symbols or strings and are stringified for storage exactly like #checkpoint
+    # (+name.to_s+). The returned Hash is keyed by the **original** (un-stringified)
+    # names with values read back from +@run.state+, so a caller gets the same shape
+    # whether a value came from this call or a prior pass.
+    #
+    # The pending steps (those not already in +@run.state+) run through the existing
+    # Nexo.concurrent driver, all in flight at once — callers bound the batch by how
+    # many keys they pass; there is no separate rate knob. Each step persists on its
+    # own through the same read-current → merge → assign → +save_state!+ sequence as
+    # #checkpoint, serialized across the concurrent fibers by an internal Mutex, and
+    # emits a +:checkpoint+ event naming the step (Spec 21 R3). Concurrency (and the
+    # +async+ gem) is touched **only** when something is pending — an all-persisted
+    # pass returns the prior-pass values directly without requiring +async+.
+    #
+    # Known trade-off: this is per-step persistence, **not** an atomic batch. If
+    # step B raises after step A persisted, A stays in +run.state+, B is absent, the
+    # run goes +"failed"+, and the exception propagates through the workflow's normal
+    # failure path (Nexo.concurrent's "first failure re-raises, the rest stop" — not
+    # rescued away). A subsequent ::execute of the SAME run re-submits only the
+    # still-missing names — A is skipped, B re-runs. Do NOT treat a batch as
+    # all-or-nothing.
+    #
+    # Same restrictions as #checkpoint: values must be json-serializable (they
+    # round-trip the store), a step must NOT be named after a RESERVED_STATE_KEYS
+    # entry (raises Nexo::Error before any step runs), and do NOT call #suspend!
+    # inside a step (undefined — v1 unsupported, documented not enforced).
+    def checkpoint_all(steps)
+      if (reserved = steps.keys.find { |name| RESERVED_STATE_KEYS.include?(name.to_s) })
+        raise Nexo::Error,
+          "checkpoint name #{reserved.to_s.inspect} is reserved (#{RESERVED_STATE_KEYS.join(", ")})"
+      end
+
+      state = @run.state || {}
+      pending = steps.reject { |name, _| state.key?(name.to_s) }
+
+      unless pending.empty?
+        Nexo.concurrent(max_in_flight: pending.size) do |c|
+          pending.each { |name, callable| c.add { persist_checkpoint(name, callable.call) } }
+        end
+      end
+
+      current = @run.state || {}
+      steps.keys.to_h { |name| [name, current[name.to_s]] }
+    end
+
     # Pauses the run durably (Spec 13): raises Suspended, which ::execute catches
     # to mark the run +"suspended"+ (a non-failure outcome) and return it to the
     # caller. Call this **outside** a checkpoint block. +reason+ is surfaced to a
@@ -615,6 +727,28 @@ module Nexo
       d.key?(:approved) ? {decision: {approved: !!d[:approved]}} : {}
     end
 
+    # Persists ONE completed #checkpoint_all step (Spec 21) — json_normalizes the
+    # raw value, then, inside +@checkpoint_mutex.synchronize+, re-reads the current
+    # +@run.state+, merges the single key, reassigns, and +save_state!+s it: the
+    # identical read-current → merge → assign → save sequence as #checkpoint, but
+    # serialized across the concurrent fibers so two steps landing at once can't
+    # clobber each other's merge. The +:checkpoint+ event is emitted **inside** the
+    # same synchronized block, alongside the state write, so the event-log append
+    # (which mutates the shared buffer / run) serializes with it too. Only reached
+    # for genuinely pending steps, so a skipped (already-persisted) step emits
+    # nothing. Returns the normalized value.
+    def persist_checkpoint(name, raw_value)
+      key = name.to_s
+      value = json_normalize(raw_value)
+      @checkpoint_mutex.synchronize do
+        store = @run.state || {}
+        @run.state = store.merge(key => value)
+        @run.save_state! if @run.respond_to?(:save_state!)
+        emit(:checkpoint, name: key)
+      end
+      value
+    end
+
     # Coerces a checkpoint value into exactly what it would become after a round
     # trip through the AR json column (string keys, symbol values → strings), so
     # the Memory store and the AR store return identical data on resume. A

data/sig/nexo/workflow.rbs

@@ -0,0 +1,14 @@
+# Partial RBS for the Spec 21 surface only (RBS reopening) — the scheduled
+# enqueue/resume keywords on the class methods and the new instance
+# #checkpoint_all. The rest of Workflow stays unsigned, matching today's
+# partial RBS coverage.
+module Nexo
+  class Workflow
+    RESERVED_STATE_KEYS: Array[String]
+
+    def self.run_later: (?untyped payload, ?queue: untyped, ?wait: untyped, ?wait_until: untyped, **untyped) -> untyped
+    def self.resume_later: (untyped run_id, ?untyped input, ?queue: untyped, ?wait: untyped, ?wait_until: untyped) -> untyped
+
+    def checkpoint_all: (Hash[untyped, untyped] steps) -> Hash[untyped, untyped]
+  end
+end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: nexo_ai
 version: !ruby/object:Gem::Version
-  version: 0.7.0
+  version: 0.8.1
 platform: ruby
 authors:
 - Mario Alberto Chávez
@@ -222,6 +222,7 @@ files:
 - sig/nexo/sandboxes/local.rbs
 - sig/nexo/sandboxes/virtual.rbs
 - sig/nexo/tools.rbs
+- sig/nexo/workflow.rbs
 - sig/nexo_ai.rbs
 homepage: https://maquina.app
 licenses:
@@ -245,7 +246,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 4.0.14
+rubygems_version: 4.0.17
 specification_version: 4
 summary: Agent = Model + Harness. Nexo is the connective tissue linking RubyLLM to
   tools, sandboxes, skills, and runs.