nexo_ai 0.9.0 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +51 -0
- data/docs/permissions.md +17 -2
- data/docs/sandboxes.md +13 -4
- data/docs/tools.md +16 -6
- data/docs/workflows.md +28 -0
- data/lib/nexo/agent.rb +18 -8
- data/lib/nexo/configuration.rb +9 -0
- data/lib/nexo/permissions.rb +26 -3
- data/lib/nexo/run_store.rb +147 -4
- data/lib/nexo/version.rb +1 -1
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 2a372c4ac5c253a76cbf930d99b134bbdc62c596dc8bde4414da26f08c99ad39
|
|
4
|
+
data.tar.gz: 8329003fd163b8b4131f4608b18ecb5604717b7dd6995699328216cb22143320
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 3e3de8747793d6c7b052d9272f6e23345e33d0749e66cd17a4247429eec1999f1329830c1c21841ac399bc04743f2d6595126496d8a24d340b537ad5ee4de026
|
|
7
|
+
data.tar.gz: 271699e28b6c236f618250b06f75ed8c4fc5d0dec269d24d0f4bac145366e1cedc0f6fee37926117b9fbf2446a8d4d7b54054f2c73562f3d6a1bfd6ae41e21d5
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,56 @@
|
|
|
1
1
|
## [Unreleased]
|
|
2
2
|
|
|
3
|
+
## [0.11.0] - 2026-08-20
|
|
4
|
+
|
|
5
|
+
An agent's tool schema now tells the truth about what it may do.
|
|
6
|
+
|
|
7
|
+
### Changed
|
|
8
|
+
|
|
9
|
+
- **An agent no longer advertises a tool its permission mode can never authorize.**
|
|
10
|
+
Tool attach was gated on the sandbox (`Sandbox#supports?`) but not on the gate, so a
|
|
11
|
+
`:read_only` agent — the default — put `WriteFile` and `Shell` in its schema on every
|
|
12
|
+
turn even though `#authorize!` was guaranteed to deny them. The same held for `Fetch`
|
|
13
|
+
and `WebSearch`, which were gated on `fetch_allow` / `search_backend` being declared
|
|
14
|
+
but not on the capability being permitted. Models do try these tools, and each attempt
|
|
15
|
+
costs a full round trip.
|
|
16
|
+
|
|
17
|
+
`Agent#chat`, `#apply_fetch` and `#apply_search` now also consult
|
|
18
|
+
`Permissions#never_allows?`. Only `:read_only` is decidable ahead of time; `:auto`,
|
|
19
|
+
`:ask` and `:approve` decide per call and still attach — `:approve` in particular must
|
|
20
|
+
reach the gate so it can raise `ApprovalRequired` and suspend the run.
|
|
21
|
+
|
|
22
|
+
This is a cost and description-accuracy measure, **not** a security change:
|
|
23
|
+
`#authorize!` remains the boundary and still denies at call time.
|
|
24
|
+
|
|
25
|
+
**Upgrading:** an agent that is supposed to write, shell out, fetch or search needs the
|
|
26
|
+
capability in `allow:` (or a non-`:read_only` mode). If it does not have it today the
|
|
27
|
+
tool was already failing on every call — the tool now disappears from the schema instead
|
|
28
|
+
of erroring. Note that `fetch_allow` scopes hosts and `search_backend` names a backend;
|
|
29
|
+
neither is a capability grant.
|
|
30
|
+
|
|
31
|
+
### Added
|
|
32
|
+
|
|
33
|
+
- `Nexo::Permissions#never_allows?(capability)` — true when a capability can never be
|
|
34
|
+
authorized for this gate, for any call. Derived from the new `Permissions::PRIVILEGED`
|
|
35
|
+
constant, which `#authorize!` also reads, so the predicate cannot drift from the gate.
|
|
36
|
+
|
|
37
|
+
## [0.10.0] - 2026-08-20
|
|
38
|
+
|
|
39
|
+
Durable workflows without a database.
|
|
40
|
+
|
|
41
|
+
### Added
|
|
42
|
+
|
|
43
|
+
- `Nexo.config.run_store` — set a store instance and it wins over the automatic
|
|
44
|
+
choice (ActiveRecord under Rails, in-memory otherwise).
|
|
45
|
+
- `Nexo::RunStore::Disk` — a file-backed store, one JSON document per run under a
|
|
46
|
+
directory, written atomically (temp file + rename). This is what makes `checkpoint`,
|
|
47
|
+
`suspend!`/`resume` and a run's recorded artifacts available to a host with no
|
|
48
|
+
database: all of them read state back from the store, and the in-memory one dies
|
|
49
|
+
with the process. `Disk::Run` subclasses `Memory::Run`, so the run shape and every
|
|
50
|
+
read helper are shared and host code behaves identically on either store. `#all`
|
|
51
|
+
lists the directory newest-first. `claim_for_resume!` re-reads from disk so a stale
|
|
52
|
+
copy cannot win a claim; it is atomic within a process, not across machines.
|
|
53
|
+
|
|
3
54
|
## [0.9.0] - 2026-08-20
|
|
4
55
|
|
|
5
56
|
Skills and sandboxes learn to talk about the environment, and an agent's output finally
|
data/docs/permissions.md
CHANGED
|
@@ -8,7 +8,7 @@ and the agent loop continues — it does not raise. A path that escapes the work
|
|
|
8
8
|
|
|
9
9
|
| | `:read` | `:glob` | `:write` | `:shell` | `:fetch` | `:search` |
|
|
10
10
|
| ------------------------ | ------- | ------- | ------------------- | --------------------------------- | ---------- | ---------- |
|
|
11
|
-
| `:read_only` (default) | ✅ | ✅ | ❌
|
|
11
|
+
| `:read_only` (default) | ✅ | ✅ | ❌ not attached ‡ | ❌ not attached ‡ | ❌ not attached ‡ | ❌ not attached ‡ |
|
|
12
12
|
| `:auto` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
|
|
13
13
|
| `:ask` | ✅ | ✅ | per `on_ask` | per `on_ask` | per `on_ask` | per `on_ask` |
|
|
14
14
|
| `:approve` | ✅ | ✅ | per `decision` | per `decision` | per `decision` | per `decision` |
|
|
@@ -18,7 +18,22 @@ and the agent loop continues — it does not raise. A path that escapes the work
|
|
|
18
18
|
|
|
19
19
|
`:read`/`:glob` are auto-allowed under **every** mode (they sit in the default
|
|
20
20
|
`allow` list), so `:ask`/`:approve` never prompt for them — only
|
|
21
|
-
`:write`/`:shell`/`:fetch`/`:search` reach the gate.
|
|
21
|
+
`:write`/`:shell`/`:fetch`/`:search` reach the gate.
|
|
22
|
+
|
|
23
|
+
**‡** Under `:read_only` these four capabilities can *never* be authorized, so their tools are
|
|
24
|
+
not attached at all — the model never sees `WriteFile`, `Shell`, `Fetch` or `WebSearch` in its
|
|
25
|
+
schema. Anything named in `allow:` is exempt and attaches normally, and `:auto`/`:ask`/`:approve`
|
|
26
|
+
always attach because they decide per call. Leaving a guaranteed failure in the schema is not
|
|
27
|
+
free: a model that reads the schema tries the tool, and each attempt costs a full round trip.
|
|
28
|
+
`Permissions#never_allows?` is the predicate, and it is derived from the same `PRIVILEGED` list
|
|
29
|
+
`#authorize!` uses so the two cannot disagree. This is a cost and description-accuracy measure,
|
|
30
|
+
**not** a security boundary — `#authorize!` is still the gate and still denies at call time.
|
|
31
|
+
|
|
32
|
+
Note that declaring `fetch_allow` or a `search_backend` is *not* a capability grant: `fetch_allow`
|
|
33
|
+
scopes which hosts are reachable, and both still need `:fetch` / `:search` permitted before the
|
|
34
|
+
tool is attached.
|
|
35
|
+
|
|
36
|
+
**†** `:fetch` and `:search`
|
|
22
37
|
run in the **host process** (stdlib `net/http` / a host-injected backend), so **no
|
|
23
38
|
sandbox constrains them** — not even a `--network none` container. They are bounded
|
|
24
39
|
only by the capability gate above plus `fetch_allow` / the injected backend.
|
data/docs/sandboxes.md
CHANGED
|
@@ -54,10 +54,19 @@ scope; none widens authority silently.
|
|
|
54
54
|
instructions → sandbox instructions → skill instructions**. Provider-neutral, injected through
|
|
55
55
|
the existing `with_instructions` path.
|
|
56
56
|
|
|
57
|
-
- **
|
|
58
|
-
a
|
|
59
|
-
|
|
60
|
-
|
|
57
|
+
- **Gated tool attach, on two axes.** An agent does not advertise a tool it could never
|
|
58
|
+
successfully call, because a guaranteed failure in the schema costs a round trip every time a
|
|
59
|
+
model tries it. `Agent#chat` attaches a tool only when **both** hold:
|
|
60
|
+
|
|
61
|
+
1. *The sandbox supports the capability* (`Sandbox#supports?`). `Local`/`Container` support all
|
|
62
|
+
four; `Virtual` supports everything but `:shell`, so a `:virtual` agent has no `Shell`.
|
|
63
|
+
2. *The permission gate does not deny it statically* (`Permissions#never_allows?`). Under
|
|
64
|
+
`:read_only`, `:write` and `:shell` can never be authorized unless listed in `allow:`, so
|
|
65
|
+
neither `WriteFile` nor `Shell` is attached. `:auto`, `:ask` and `:approve` decide per call
|
|
66
|
+
and always attach — `:approve` in particular *must* reach the gate so it can suspend the run.
|
|
67
|
+
|
|
68
|
+
`ReadFile` and `Glob` are always attached. This is a cost and description-accuracy measure, not
|
|
69
|
+
a security boundary: `Permissions#authorize!` remains the gate and still denies at call time.
|
|
61
70
|
|
|
62
71
|
- **Shell output truncation (`Nexo::OutputTruncator`).** Unbounded command output (`npm install`,
|
|
63
72
|
`git log`) is truncated before it reaches the model, so a single command can't blow a small
|
data/docs/tools.md
CHANGED
|
@@ -1,12 +1,22 @@
|
|
|
1
1
|
# Tools
|
|
2
2
|
Nexo attaches four sandbox-backed tools — `ReadFile`, `WriteFile`, `Shell`, and `Glob` — each gated by the [sandbox](sandboxes.md) and [permission](permissions.md) seams.
|
|
3
3
|
|
|
4
|
-
Which tools attach depends on what the sandbox supports
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
4
|
+
Which tools attach depends on what the sandbox supports *and* on what the permission mode could
|
|
5
|
+
ever allow:
|
|
6
|
+
|
|
7
|
+
- **Gated tool attach, on two axes.** An agent does not advertise a tool it could never
|
|
8
|
+
successfully call, because a guaranteed failure in the schema costs a round trip every time a
|
|
9
|
+
model tries it. `Agent#chat` attaches a tool only when **both** hold:
|
|
10
|
+
|
|
11
|
+
1. *The sandbox supports the capability* (`Sandbox#supports?`). `Local`/`Container` support all
|
|
12
|
+
four; `Virtual` supports everything but `:shell`, so a `:virtual` agent has no `Shell`.
|
|
13
|
+
2. *The permission gate does not deny it statically* (`Permissions#never_allows?`). Under
|
|
14
|
+
`:read_only`, `:write` and `:shell` can never be authorized unless listed in `allow:`, so
|
|
15
|
+
neither `WriteFile` nor `Shell` is attached. `:auto`, `:ask` and `:approve` decide per call
|
|
16
|
+
and always attach — `:approve` in particular *must* reach the gate so it can suspend the run.
|
|
17
|
+
|
|
18
|
+
`ReadFile` and `Glob` are always attached. This is a cost and description-accuracy measure, not
|
|
19
|
+
a security boundary: `Permissions#authorize!` remains the gate and still denies at call time.
|
|
10
20
|
|
|
11
21
|
`Shell` truncates unbounded command output before it reaches the model:
|
|
12
22
|
|
data/docs/workflows.md
CHANGED
|
@@ -82,6 +82,34 @@ ActiveRecord when it is available and the in-memory store otherwise. The schema
|
|
|
82
82
|
uses portable `json` columns (SQLite and PostgreSQL alike) and a UUID string
|
|
83
83
|
primary key.
|
|
84
84
|
|
|
85
|
+
### Durable without a database — `RunStore::Disk`
|
|
86
|
+
|
|
87
|
+
The in-memory store dies with the process, and **everything durable reads state
|
|
88
|
+
back from the store**: `checkpoint`, `suspend!`/`resume`, and a run's recorded
|
|
89
|
+
artifacts. So for a CLI or a script — no Rails, no database, but a process that
|
|
90
|
+
ends — point the store at a directory:
|
|
91
|
+
|
|
92
|
+
```ruby
|
|
93
|
+
Nexo.config.run_store = Nexo::RunStore::Disk.new(dir: "~/.local/state/myapp/runs")
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
One JSON document per run, rewritten whenever the run changes, written to a temp
|
|
97
|
+
file and renamed so a crash cannot leave a truncated document behind. The run shape
|
|
98
|
+
and every read helper are Memory's — `Disk::Run` subclasses it — so host code
|
|
99
|
+
written against one store behaves identically on the other. `#all` lists the
|
|
100
|
+
directory newest-first, which is what a host wants for *"what did the last run
|
|
101
|
+
produce?"*.
|
|
102
|
+
|
|
103
|
+
A configured store wins over the automatic choice, including under Rails.
|
|
104
|
+
|
|
105
|
+
**Scope, honestly.** Rewriting the whole document per change suits the run sizes this
|
|
106
|
+
is meant for (a CLI's own history) and is wrong for a busy multi-worker queue — that
|
|
107
|
+
is what the ActiveRecord backend is for. `claim_for_resume!` re-reads from disk, so a
|
|
108
|
+
stale in-memory copy cannot win a claim, but it is atomic **within** a process only.
|
|
109
|
+
Runs are never pruned: retention is the host's policy and `dir` is a plain directory
|
|
110
|
+
it can manage. `Workflow.reconcile_interrupted!` sweeps the Memory and ActiveRecord
|
|
111
|
+
stores only.
|
|
112
|
+
|
|
85
113
|
## Input staging and artifacts
|
|
86
114
|
|
|
87
115
|
A run owns a **sandbox** — declared with the `sandbox` class macro (default
|
data/lib/nexo/agent.rb
CHANGED
|
@@ -285,14 +285,19 @@ module Nexo
|
|
|
285
285
|
# One ReadTracker per chat, shared by ReadFile (records) and WriteFile
|
|
286
286
|
# (enforces the read-before-write + stale guard) — R4.
|
|
287
287
|
tracker = ReadTracker.new
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
|
|
291
|
-
|
|
292
|
-
|
|
293
|
-
#
|
|
294
|
-
#
|
|
295
|
-
|
|
288
|
+
# A sandbox tool is attached only when the agent could actually use it, on
|
|
289
|
+
# BOTH axes: the sandbox has to support the capability (R2 — a :virtual
|
|
290
|
+
# sandbox has no shell) and the gate must not deny it statically (a
|
|
291
|
+
# :read_only agent can never be authorized for :write or :shell). Otherwise
|
|
292
|
+
# a guaranteed failure sits in the tool schema on every turn and models do
|
|
293
|
+
# try it. This is the same attach-time gating apply_fetch/apply_search
|
|
294
|
+
# already apply; #authorize! stays the actual boundary either way.
|
|
295
|
+
tools = [Tools::ReadFile.new(sandbox: @sandbox, permissions: @permissions, tracker: tracker)]
|
|
296
|
+
unless @permissions.never_allows?(:write)
|
|
297
|
+
tools << Tools::WriteFile.new(sandbox: @sandbox, permissions: @permissions, tracker: tracker)
|
|
298
|
+
end
|
|
299
|
+
tools << Tools::Glob.new(sandbox: @sandbox, permissions: @permissions)
|
|
300
|
+
if @sandbox.supports?(:shell) && !@permissions.never_allows?(:shell)
|
|
296
301
|
tools << Tools::Shell.new(sandbox: @sandbox, permissions: @permissions)
|
|
297
302
|
end
|
|
298
303
|
c.with_tools(*tools)
|
|
@@ -570,6 +575,10 @@ module Nexo
|
|
|
570
575
|
# allow-list only scopes hosts, it is not the capability grant.
|
|
571
576
|
def apply_fetch(chat)
|
|
572
577
|
return if self.class.fetch_allow.empty?
|
|
578
|
+
# The allow-list scopes hosts; it is not the capability grant. A gate that
|
|
579
|
+
# can never authorize :fetch gets no tool rather than a guaranteed failure
|
|
580
|
+
# in its schema (see Permissions#never_allows?).
|
|
581
|
+
return if @permissions.never_allows?(:fetch)
|
|
573
582
|
|
|
574
583
|
chat.with_tools(
|
|
575
584
|
Tools::Fetch.new(sandbox: @sandbox, permissions: @permissions, allow_hosts: self.class.fetch_allow)
|
|
@@ -585,6 +594,7 @@ module Nexo
|
|
|
585
594
|
# through Permissions#authorize! at call time.
|
|
586
595
|
def apply_search(chat)
|
|
587
596
|
backend = self.class.search_backend or return
|
|
597
|
+
return if @permissions.never_allows?(:search)
|
|
588
598
|
|
|
589
599
|
chat.with_tools(
|
|
590
600
|
Tools::WebSearch.new(sandbox: @sandbox, permissions: @permissions, backend: backend)
|
data/lib/nexo/configuration.rb
CHANGED
|
@@ -57,6 +57,14 @@ module Nexo
|
|
|
57
57
|
# this only gates the opt-in Turbo mirror.
|
|
58
58
|
attr_accessor :broadcast_events
|
|
59
59
|
|
|
60
|
+
# The run store Workflow persists runs through, default +nil+ → the automatic
|
|
61
|
+
# choice (ActiveRecord under Rails, in-memory otherwise). Set an instance to
|
|
62
|
+
# override, which is what a host outside Rails needs for anything durable:
|
|
63
|
+
# checkpoint/suspend/resume all read state back from the store, and the
|
|
64
|
+
# in-memory one dies with the process. +Nexo::RunStore::Disk.new(dir:)+ is the
|
|
65
|
+
# shipped answer for a CLI or a script.
|
|
66
|
+
attr_accessor :run_store
|
|
67
|
+
|
|
60
68
|
# The ActiveJob queue Workflow.run_later enqueues onto (Spec 11 R1), default
|
|
61
69
|
# +nil+ → ActiveJob's default queue. Set to a symbol (e.g. +:nexo+) to route
|
|
62
70
|
# workflow jobs to a dedicated queue.
|
|
@@ -74,6 +82,7 @@ module Nexo
|
|
|
74
82
|
@buffer_workflow_events = false
|
|
75
83
|
@session_chat_model = "Chat" # ruby_llm's own default acts_as_chat host
|
|
76
84
|
@broadcast_events = false # safe default: opt in to the Turbo mirror
|
|
85
|
+
@run_store = nil # nil → the automatic choice (ActiveRecord under Rails, else Memory)
|
|
77
86
|
@job_queue = nil # nil → ActiveJob's default queue
|
|
78
87
|
end
|
|
79
88
|
|
data/lib/nexo/permissions.rb
CHANGED
|
@@ -20,6 +20,11 @@ module Nexo
|
|
|
20
20
|
# The recognized permission modes: +:auto+, +:read_only+, +:ask+, +:approve+.
|
|
21
21
|
MODES = %i[auto read_only ask approve].freeze
|
|
22
22
|
|
|
23
|
+
# The capabilities +:read_only+ refuses. Named once so #authorize! and
|
|
24
|
+
# #never_allows? cannot drift apart: the whole value of the predicate is that
|
|
25
|
+
# it reports what the gate will actually do, so both must read the same list.
|
|
26
|
+
PRIVILEGED = %i[write shell fetch search].freeze
|
|
27
|
+
|
|
23
28
|
# Raised when a capability is not authorized. Tools rescue this and return
|
|
24
29
|
# +{ error: ... }+ so the agent loop continues.
|
|
25
30
|
class Denied < StandardError; end
|
|
@@ -76,9 +81,8 @@ module Nexo
|
|
|
76
81
|
when :auto
|
|
77
82
|
true
|
|
78
83
|
when :read_only
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
end
|
|
84
|
+
raise Denied, "#{capability} denied in read_only mode" if PRIVILEGED.include?(capability)
|
|
85
|
+
|
|
82
86
|
true
|
|
83
87
|
when :ask
|
|
84
88
|
# Scoped-ask: when ask_when says this action doesn't need a prompt,
|
|
@@ -109,6 +113,25 @@ module Nexo
|
|
|
109
113
|
end
|
|
110
114
|
end
|
|
111
115
|
|
|
116
|
+
# Whether +capability+ can NEVER be authorized by this gate, for any call.
|
|
117
|
+
#
|
|
118
|
+
# True only under +:read_only+, for a PRIVILEGED capability absent from
|
|
119
|
+
# +allow:+ — that is the one case knowable ahead of time. Every other mode
|
|
120
|
+
# decides per call and must be reported as *possible*: +:auto+ allows, +:ask+
|
|
121
|
+
# consults its hook, and +:approve+ has to reach the gate so it can raise
|
|
122
|
+
# ApprovalRequired and suspend the run.
|
|
123
|
+
#
|
|
124
|
+
# Agent#chat uses this to skip ATTACHING a tool the model could never
|
|
125
|
+
# successfully call, so a guaranteed failure stops occupying the tool schema
|
|
126
|
+
# on every turn. That makes this a cost and description-accuracy measure, not
|
|
127
|
+
# a security boundary: #authorize! remains the gate and still denies at call
|
|
128
|
+
# time whether or not the tool was advertised.
|
|
129
|
+
def never_allows?(capability)
|
|
130
|
+
return false if @allow.include?(capability)
|
|
131
|
+
|
|
132
|
+
@mode == :read_only && PRIVILEGED.include?(capability)
|
|
133
|
+
end
|
|
134
|
+
|
|
112
135
|
# Authorizes an MCP tool *call* by name. A deliberate sibling of #authorize!
|
|
113
136
|
# on a separate capability axis: an MCP tool runs inside the MCP server,
|
|
114
137
|
# outside the sandbox, so this gates the authority to *invoke* it — a different
|
data/lib/nexo/run_store.rb
CHANGED
|
@@ -1,5 +1,8 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require "json"
|
|
4
|
+
require "fileutils"
|
|
5
|
+
|
|
3
6
|
module Nexo
|
|
4
7
|
# Storage seam for Workflow runs. Two interchangeable backends expose an
|
|
5
8
|
# identical create/find interface and return run objects responding to the
|
|
@@ -11,11 +14,19 @@ module Nexo
|
|
|
11
14
|
# +push_event(event)+, +save_events!+, +push_artifact(artifact)+,
|
|
12
15
|
# +save_artifacts!+, and +save_state!+ (Spec 13).
|
|
13
16
|
module RunStore
|
|
14
|
-
# Selects a backend:
|
|
15
|
-
# and Nexo::WorkflowRun are
|
|
16
|
-
#
|
|
17
|
-
# path never references
|
|
17
|
+
# Selects a backend: +Nexo.config.run_store+ when a host set one, else the
|
|
18
|
+
# ActiveRecord store when both ::ActiveRecord::Base and Nexo::WorkflowRun are
|
|
19
|
+
# defined (the Rails path), otherwise the in-memory store. With no Rails loaded
|
|
20
|
+
# the AR check short-circuits, so the plain-Ruby path never references
|
|
21
|
+
# ActiveRecord.
|
|
22
|
+
#
|
|
23
|
+
# The configured store wins over both, which is the only way a non-Rails host
|
|
24
|
+
# gets durability: everything that survives a process — checkpoint, suspend,
|
|
25
|
+
# resume, a run's recorded artifacts — is read back from the store, and Memory
|
|
26
|
+
# dies with the process. See Disk.
|
|
18
27
|
def self.default
|
|
28
|
+
return Nexo.config.run_store if Nexo.config.run_store
|
|
29
|
+
|
|
19
30
|
if defined?(::ActiveRecord::Base) && defined?(Nexo::WorkflowRun)
|
|
20
31
|
ActiveRecord.new
|
|
21
32
|
else
|
|
@@ -158,5 +169,137 @@ module Nexo
|
|
|
158
169
|
.update_all(status: "running") == 1
|
|
159
170
|
end
|
|
160
171
|
end
|
|
172
|
+
|
|
173
|
+
# File-backed backend for a host with no database: one JSON document per run
|
|
174
|
+
# under +dir+, rewritten whenever the run changes. This is what makes durability
|
|
175
|
+
# available outside Rails — checkpoint, suspend/resume and a run's recorded
|
|
176
|
+
# artifacts all read state back from the store, and Memory dies with the process.
|
|
177
|
+
#
|
|
178
|
+
# Nexo.config.run_store = Nexo::RunStore::Disk.new(dir: "~/.local/state/myapp/runs")
|
|
179
|
+
#
|
|
180
|
+
# The Run is Memory's, subclassed: the shape and every read helper are shared, so
|
|
181
|
+
# a host written against one store behaves identically on the other. Only the
|
|
182
|
+
# +save_*+ hooks differ — they write the document. +push_event+/+push_artifact+
|
|
183
|
+
# deliberately do NOT, because Workflow always calls the matching +save_*+ right
|
|
184
|
+
# after, and persisting twice would double the writes for nothing.
|
|
185
|
+
#
|
|
186
|
+
# Scope, honestly: rewriting the whole document per change is fine for the run
|
|
187
|
+
# sizes this is meant for (a CLI's own history) and wrong for a busy multi-worker
|
|
188
|
+
# queue — that is what the ActiveRecord backend is for. +claim_for_resume!+ is
|
|
189
|
+
# atomic within a process but NOT across processes; two machines resuming the
|
|
190
|
+
# same run concurrently is out of scope. Runs are never pruned here: retention is
|
|
191
|
+
# the host's policy, and +dir+ is a plain directory it can manage.
|
|
192
|
+
class Disk
|
|
193
|
+
# A Memory::Run that writes itself to +path+ whenever Workflow saves it.
|
|
194
|
+
class Run < Memory::Run
|
|
195
|
+
# Absolute path of this run's JSON document. Assigned by the store.
|
|
196
|
+
attr_accessor :path
|
|
197
|
+
|
|
198
|
+
# The three save hooks Workflow calls after mutating a run. Memory's are
|
|
199
|
+
# no-ops because the Struct already holds everything; here they are the
|
|
200
|
+
# whole point.
|
|
201
|
+
def save_events! = persist!
|
|
202
|
+
|
|
203
|
+
# Writes the run after Workflow appended an artifact.
|
|
204
|
+
def save_artifacts! = persist!
|
|
205
|
+
|
|
206
|
+
# Writes the run after Workflow stored checkpoint/suspend state.
|
|
207
|
+
def save_state! = persist!
|
|
208
|
+
|
|
209
|
+
# Assigns attributes (Memory's behaviour) and writes the run.
|
|
210
|
+
def update!(attrs)
|
|
211
|
+
super
|
|
212
|
+
persist!
|
|
213
|
+
end
|
|
214
|
+
|
|
215
|
+
# Writes the document atomically — a temp file in the same directory, then
|
|
216
|
+
# a rename — so a crash mid-write cannot leave a truncated run behind.
|
|
217
|
+
def persist!
|
|
218
|
+
return if path.nil?
|
|
219
|
+
|
|
220
|
+
tmp = "#{path}.#{Process.pid}.tmp"
|
|
221
|
+
::File.write(tmp, JSON.generate(to_h.except(:path)))
|
|
222
|
+
::File.rename(tmp, path)
|
|
223
|
+
end
|
|
224
|
+
end
|
|
225
|
+
|
|
226
|
+
# Serializes writes within the process, mirroring Memory's store-wide lock.
|
|
227
|
+
MUTEX = Mutex.new
|
|
228
|
+
|
|
229
|
+
# +dir+ is created on demand; +~+ is expanded.
|
|
230
|
+
def initialize(dir:)
|
|
231
|
+
@dir = ::File.expand_path(dir)
|
|
232
|
+
end
|
|
233
|
+
|
|
234
|
+
# The directory runs are written to.
|
|
235
|
+
attr_reader :dir
|
|
236
|
+
|
|
237
|
+
# Builds a fresh +"pending"+ Run, writes it, and returns it.
|
|
238
|
+
def create(workflow_class:, payload:)
|
|
239
|
+
run = Run.new(
|
|
240
|
+
id: Nexo.generate_run_id, workflow_class: workflow_class, status: "pending",
|
|
241
|
+
payload: payload, result: nil, error: nil, events: [], artifacts: [], state: {}
|
|
242
|
+
)
|
|
243
|
+
::FileUtils.mkdir_p(@dir)
|
|
244
|
+
run.path = path_for(run.id)
|
|
245
|
+
run.persist!
|
|
246
|
+
run
|
|
247
|
+
end
|
|
248
|
+
|
|
249
|
+
# Reads a run back by id. A miss raises KeyError, matching Memory.
|
|
250
|
+
def find(id)
|
|
251
|
+
path = path_for(id)
|
|
252
|
+
raise KeyError, "run not found: #{id}" unless ::File.exist?(path)
|
|
253
|
+
|
|
254
|
+
hydrate(JSON.parse(::File.read(path)), path)
|
|
255
|
+
end
|
|
256
|
+
|
|
257
|
+
# Every run in the directory, newest-written first. Not part of the store
|
|
258
|
+
# contract Workflow calls — it is here because a host with a run directory
|
|
259
|
+
# inevitably wants to list it ("what did the last run produce?").
|
|
260
|
+
def all
|
|
261
|
+
return [] unless ::File.directory?(@dir)
|
|
262
|
+
|
|
263
|
+
Dir.glob(::File.join(@dir, "*.json"))
|
|
264
|
+
.sort_by { |p| -::File.mtime(p).to_f }
|
|
265
|
+
.filter_map do |p|
|
|
266
|
+
hydrate(JSON.parse(::File.read(p)), p)
|
|
267
|
+
rescue JSON::ParserError, SystemCallError
|
|
268
|
+
# A half-written or hand-edited document must not break the listing.
|
|
269
|
+
nil
|
|
270
|
+
end
|
|
271
|
+
end
|
|
272
|
+
|
|
273
|
+
# Atomically claims a +"suspended"+ run for resume, re-reading from disk so a
|
|
274
|
+
# stale in-memory copy cannot win. Within one process only — see the class
|
|
275
|
+
# note.
|
|
276
|
+
def claim_for_resume!(run)
|
|
277
|
+
MUTEX.synchronize do
|
|
278
|
+
current = find(run.id)
|
|
279
|
+
return false unless current.status == "suspended"
|
|
280
|
+
|
|
281
|
+
run.status = "running"
|
|
282
|
+
run.path ||= path_for(run.id)
|
|
283
|
+
run.persist!
|
|
284
|
+
true
|
|
285
|
+
end
|
|
286
|
+
end
|
|
287
|
+
|
|
288
|
+
private
|
|
289
|
+
|
|
290
|
+
def path_for(id) = ::File.join(@dir, "#{id}.json")
|
|
291
|
+
|
|
292
|
+
# Rebuilds a Run from its document. Missing members read as nil/empty rather
|
|
293
|
+
# than raising, so a document written by an older version still loads.
|
|
294
|
+
def hydrate(data, path)
|
|
295
|
+
run = Run.new(
|
|
296
|
+
id: data["id"], workflow_class: data["workflow_class"], status: data["status"],
|
|
297
|
+
payload: data["payload"], result: data["result"], error: data["error"],
|
|
298
|
+
events: data["events"] || [], artifacts: data["artifacts"] || [], state: data["state"] || {}
|
|
299
|
+
)
|
|
300
|
+
run.path = path
|
|
301
|
+
run
|
|
302
|
+
end
|
|
303
|
+
end
|
|
161
304
|
end
|
|
162
305
|
end
|
data/lib/nexo/version.rb
CHANGED