omakase-agents 0.3.0 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +41 -0
- data/README.md +47 -19
- data/lib/omakase/agent.rb +40 -7
- data/lib/omakase/capabilities.rb +7 -2
- data/lib/omakase/executor.rb +145 -8
- data/lib/omakase/fake_chat.rb +28 -3
- data/lib/omakase/mcp.rb +90 -1
- data/lib/omakase/request.rb +15 -3
- data/lib/omakase/schema.rb +56 -6
- data/lib/omakase/skills/how_to_act/SKILL.md +40 -0
- data/lib/omakase/skills.rb +18 -4
- data/lib/omakase/strategies/code_act.rb +25 -8
- data/lib/omakase/strategies/predict.rb +8 -2
- data/lib/omakase/tools/ruby.rb +11 -3
- data/lib/omakase/trace.rb +8 -3
- data/lib/omakase/version.rb +1 -1
- data/lib/omakase.rb +25 -2
- metadata +5 -4
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: bdeb9ba4fb16ca770bcedd8b127b76cc925af49cfa73a9257d9e3d27e060de97
|
|
4
|
+
data.tar.gz: a7b72e4946ab8add9c5baac75bcd9bf5bf0419d70bfe79c54fbd843a280f78ed
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: baa837f849d8a646edd9a9777660d236defb2c676f274cb9c2f5cade9c74e70145d8dcde3ab6c17b260f622bed61fa025489e465e8e854e45b0cf679c1099a02
|
|
7
|
+
data.tar.gz: b0dfbe030b3f6668a8ad2faea141a6c7fc0754475371bb6e3630584770b3b91e0e5e9e33e839d9de46d4dd2222b17b17ec4665b00ad7e93899c32d4cca78af08
|
data/CHANGELOG.md
CHANGED
|
@@ -3,6 +3,47 @@
|
|
|
3
3
|
One entry per released version, written when the gem is pushed. Until `1.0`, a minor version may
|
|
4
4
|
move the API — what breaks is listed first, so an upgrade is a decision rather than a surprise.
|
|
5
5
|
|
|
6
|
+
## 0.4.0
|
|
7
|
+
|
|
8
|
+
RubyLLM 2.0 underneath, and MCP waits for it. Generated code gets its inputs as locals, can crash in
|
|
9
|
+
a child process instead of yours, and its answer is checked all the way down.
|
|
10
|
+
|
|
11
|
+
### Breaking
|
|
12
|
+
|
|
13
|
+
- RubyLLM `~> 2.0`. `ruby_llm-mcp` has no release for it yet, so `mcp` servers do not connect until
|
|
14
|
+
it does; the `mcp` declaration itself still loads.
|
|
15
|
+
- Prompt caching is on for every chat Omakase builds. `model "…", caching: false` turns it off.
|
|
16
|
+
- A text reply under `:code_act` gets one more turn in the same chat — "call finish" — before the
|
|
17
|
+
fallback to `:predict`. One more model call when a model forgets how to answer.
|
|
18
|
+
- `finish` is held to the whole schema: nested objects, array items, enums. An answer 0.3 took can
|
|
19
|
+
now be refused, with the path of the mistake (`tags[0]: expected <string>, got 1`).
|
|
20
|
+
- Chat options other than the model are the chat's own `with_*` calls — `temperature: 0.2`,
|
|
21
|
+
`thinking: {effort: :high}` — and one RubyLLM::Chat has no `with_*` for raises. 0.3 passed them
|
|
22
|
+
to `RubyLLM.chat`, which refused them anyway.
|
|
23
|
+
- Generations nest ten deep at most.
|
|
24
|
+
|
|
25
|
+
### Added
|
|
26
|
+
|
|
27
|
+
- `Omakase::Executor::Subprocess`: generated code in a forked child, so a timeout or a crash takes
|
|
28
|
+
the child and not you. Ivars come back; the child still reaches what this process reaches.
|
|
29
|
+
- Under `:code_act` the inputs are local variables in the generated code, and locals last for the
|
|
30
|
+
rest of the generation. The prompt shows only the first 500 characters of each input.
|
|
31
|
+
- Every agent has `how_to_act`, a built-in skill: how to write the Ruby — `finish`, prints, `doc`,
|
|
32
|
+
locals — with examples.
|
|
33
|
+
- The instructions go first and are marked as a cache boundary; `context` follows them.
|
|
34
|
+
- `FakeChat.replies(a, b)`: one reply per model turn, and a turn past the last one raises.
|
|
35
|
+
- A `:error` event; `Omakase::Trace` prints it and indents nested generations.
|
|
36
|
+
|
|
37
|
+
### Fixed
|
|
38
|
+
|
|
39
|
+
- `exit` in generated code no longer ends your process; the model is told to `finish` instead.
|
|
40
|
+
- An observation past 4KB keeps its end as well as its start, so the error is not the part cut off.
|
|
41
|
+
- MCP servers connect on the first instance, not at class load: an unreachable sidecar does not
|
|
42
|
+
fail boot, and a reload does not reconnect.
|
|
43
|
+
- A ```ruby fence around the whole code is stripped before it runs.
|
|
44
|
+
- SKILL.md is read as UTF-8, CRLF front matter is read, and front matter that is not YAML falls back
|
|
45
|
+
to plain `key: value` lines.
|
|
46
|
+
|
|
6
47
|
## 0.3.0
|
|
7
48
|
|
|
8
49
|
One thing changes under you: a generation may no longer call itself. The rest is additions — a real
|
data/README.md
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
# Omakase
|
|
1
|
+
# Omakase Agents
|
|
2
2
|
|
|
3
3
|
A light agent framework — about 800 lines of library. *Omakase* (お任せ): you name what you want,
|
|
4
4
|
the rest is left to the chef.
|
|
@@ -57,6 +57,12 @@ Against **RubyLLM alone**: the tool loop, the schema plumbing, and the correctio
|
|
|
57
57
|
answer are what these 800 lines are. Everything else — providers, keys, models, streaming, tracing —
|
|
58
58
|
is still RubyLLM's, and stays reachable.
|
|
59
59
|
|
|
60
|
+
Against **`RubyLLM::Agent`**: that is a chat preset — model, tools, instructions and a schema kept
|
|
61
|
+
on a class, handing back a `RubyLLM::Chat`, with Rails persistence of the conversation. Reach for it
|
|
62
|
+
when the product is the chat. Here the product is a method: `generates` defines one the model
|
|
63
|
+
implements, the agent's own methods and fields are its tools and state, and the call returns a typed
|
|
64
|
+
value. Both sit on RubyLLM and do not overlap.
|
|
65
|
+
|
|
60
66
|
Against **a framework with a tool registry**: there is nothing to register and nothing to keep in
|
|
61
67
|
sync. The model gets one tool, `ruby`, and reaches the rest through the object. A tool's description
|
|
62
68
|
is `describe`, a line above the method, instead of a JSON schema that drifts from the code it
|
|
@@ -185,7 +191,9 @@ FeedbackAgent.new.analyze(text: "…")
|
|
|
185
191
|
|
|
186
192
|
By default they take whatever you pass. Name them with `takes:` and they become a real Ruby
|
|
187
193
|
signature, so a missing or misspelled argument is an `ArgumentError` at the call rather than noise
|
|
188
|
-
in a prompt — and the model reads the names instead of `**inputs`:
|
|
194
|
+
in a prompt — and the model reads the names instead of `**inputs`. Under `:code_act` the inputs are
|
|
195
|
+
also local variables in the generated code, so a long list or a record is used, not retyped; the
|
|
196
|
+
prompt shows only the first 500 characters of each:
|
|
189
197
|
|
|
190
198
|
```ruby
|
|
191
199
|
generates :decide, "Decide this refund.", takes: %i[email complaint], returns: Refund
|
|
@@ -258,7 +266,10 @@ class ApplicationAgent < Omakase::Agent
|
|
|
258
266
|
end
|
|
259
267
|
```
|
|
260
268
|
|
|
261
|
-
Naming a provider implies `assume_model_exists: true
|
|
269
|
+
Naming a provider implies `assume_model_exists: true`. Any other option is the chat's own `with_*`
|
|
270
|
+
call — `temperature: 0.2`, `thinking: {effort: :high}`, `max_output_tokens: 2_000` — and an option
|
|
271
|
+
RubyLLM::Chat has no `with_*` for raises. Prompt caching is on unless you say `caching: false`: the
|
|
272
|
+
tool loop resends the whole chat on every step, and the instructions come first so they cache.
|
|
262
273
|
Subclasses inherit the setting and can override it, so one `ApplicationAgent` configures the lot.
|
|
263
274
|
|
|
264
275
|
A generation method can name its own model — a cheap one for classification beside a strong one
|
|
@@ -288,7 +299,8 @@ end
|
|
|
288
299
|
|
|
289
300
|
An MCP server's tools become methods on the agent, listed among its capabilities like any other —
|
|
290
301
|
so generated code calls a remote tool and the agent's own methods in the same expression. Add the
|
|
291
|
-
`ruby_llm-mcp` gem; options are passed to it verbatim.
|
|
302
|
+
`ruby_llm-mcp` gem; options are passed to it verbatim. Its 1.0 release still pins RubyLLM 1.x, so
|
|
303
|
+
this waits on a `ruby_llm-mcp` that supports RubyLLM 2.
|
|
292
304
|
|
|
293
305
|
```ruby
|
|
294
306
|
class DocsAgent < ApplicationAgent
|
|
@@ -300,13 +312,13 @@ class DocsAgent < ApplicationAgent
|
|
|
300
312
|
end
|
|
301
313
|
```
|
|
302
314
|
|
|
303
|
-
The connection opens
|
|
315
|
+
The connection opens on the first instance and the tools are read from the server then, so a
|
|
304
316
|
tool's arguments reach the model as documentation. A failed call raises, which the model sees and
|
|
305
317
|
can correct. Only text comes back: an image or audio result is dropped.
|
|
306
318
|
|
|
307
|
-
|
|
308
|
-
|
|
309
|
-
|
|
319
|
+
Class load does not talk to the server, so an unreachable sidecar does not fail boot, and a reload
|
|
320
|
+
does not reconnect. The first instance pays that cost; if the server is down, its tools are absent
|
|
321
|
+
and the generate still runs. The next instance tries that server again.
|
|
310
322
|
|
|
311
323
|
### Skills
|
|
312
324
|
|
|
@@ -331,6 +343,10 @@ reaches the model if the generated code calls `commit_style`. Anything else the
|
|
|
331
343
|
scripts, templates — sits in the same directory, and the body ends with its path, so generated Ruby
|
|
332
344
|
can read or run it.
|
|
333
345
|
|
|
346
|
+
Every agent already has one: `how_to_act`. It is how to write the Ruby that implements a
|
|
347
|
+
generation — `finish`, prints, `doc`, ivars — with examples. The description sits in the capability
|
|
348
|
+
list; the body arrives when the model calls it.
|
|
349
|
+
|
|
334
350
|
### Remembering
|
|
335
351
|
|
|
336
352
|
The chat is fresh on every call — two threads calling one agent must not share a mutable
|
|
@@ -440,7 +456,8 @@ end
|
|
|
440
456
|
A leaf is its own summary, so the model is asked only where there is something to fold. A fresh
|
|
441
457
|
agent per branch is not ceremony either: siblings then share no state, and one object may not
|
|
442
458
|
re-enter a generation it is already inside. That is refused, because a nested run opens its own chat
|
|
443
|
-
with its own tool budget — nothing would bound the spend.
|
|
459
|
+
with its own tool budget — nothing would bound the spend. For the same reason generations nest ten
|
|
460
|
+
deep at most. Generated code can start a sub-agent the
|
|
444
461
|
same way. [`examples/recursive_agent.rb`](examples/recursive_agent.rb) is the runnable version: four
|
|
445
462
|
comments, two of them leaves, two generations.
|
|
446
463
|
|
|
@@ -455,6 +472,9 @@ assert_equal "high", SupportAgent.new(chat:).triage(message: "broken")[:severity
|
|
|
455
472
|
|
|
456
473
|
# drive the tool the way a model would
|
|
457
474
|
chat = Omakase::FakeChat.new { |fake| fake.run("finish(stock_of(:apple))") }
|
|
475
|
+
|
|
476
|
+
# one reply per model turn, in order; one turn too many raises
|
|
477
|
+
chat = Omakase::FakeChat.replies("prose, not JSON", {"severity" => "high", "summary" => "…"})
|
|
458
478
|
```
|
|
459
479
|
|
|
460
480
|
It records `instructions`, `schema`, `tools` and `tasks`, so the prompt is assertable too.
|
|
@@ -484,7 +504,7 @@ Omakase.listener = ->(event, **payload) { Rails.logger.info("#{event} #{payload.
|
|
|
484
504
|
```
|
|
485
505
|
|
|
486
506
|
`:generation` carries `agent:, name:, inputs:` · `:ruby` carries `agent:, code:, outcome:` ·
|
|
487
|
-
`:answer` carries `agent:, name:, value:`.
|
|
507
|
+
`:answer` carries `agent:, name:, value:` · `:error` carries `agent:, name:, error:`.
|
|
488
508
|
|
|
489
509
|
One listener is included, for reading a run rather than storing it: it prints each step to stderr,
|
|
490
510
|
in colour when stderr is a terminal.
|
|
@@ -534,7 +554,7 @@ logs and APM without any code of ours:
|
|
|
534
554
|
|
|
535
555
|
```ruby
|
|
536
556
|
ActiveSupport::Notifications.subscribe("chat.ruby_llm") do |*, payload|
|
|
537
|
-
Rails.logger.info(model: payload[:model], input: payload[:
|
|
557
|
+
Rails.logger.info(model: payload[:model], input: payload[:tokens].input, output: payload[:tokens].output)
|
|
538
558
|
end
|
|
539
559
|
```
|
|
540
560
|
|
|
@@ -739,7 +759,7 @@ generates :plan, strategy: CriticStrategy
|
|
|
739
759
|
lib/omakase/type.rb return types that are a Ruby class
|
|
740
760
|
lib/omakase/capabilities.rb the agent’s own methods, listed for the model
|
|
741
761
|
lib/omakase/doc.rb what an unfamiliar object offers, for generated code
|
|
742
|
-
lib/omakase/executor.rb
|
|
762
|
+
lib/omakase/executor.rb in-process; Executor::Subprocess isolates a crash
|
|
743
763
|
lib/omakase/tools/ruby.rb that executor, as a RubyLLM tool, with a call budget
|
|
744
764
|
lib/omakase/mcp.rb an MCP server’s tools, as methods on the agent
|
|
745
765
|
lib/omakase/skills.rb a SKILL.md directory, as one described method
|
|
@@ -767,6 +787,7 @@ Copy `.env.example` to `.env` and fill in a key; `MODEL` and `PROVIDER` there pi
|
|
|
767
787
|
| [`interview_agent.rb`](examples/interview_agent.rb) | remembering across calls, without a shared chat |
|
|
768
788
|
| [`memory_agent.rb`](examples/memory_agent.rb) | recall by meaning, kept across a marshalled run |
|
|
769
789
|
| [`recursive_agent.rb`](examples/recursive_agent.rb) | a comment thread folded from the leaves up, one agent per node |
|
|
790
|
+
| [`subprocess_agent.rb`](examples/subprocess_agent.rb) | generated code in a child process: a crash there is an observation here |
|
|
770
791
|
|
|
771
792
|
```bash
|
|
772
793
|
bundle exec rake # tests and Standard, no network
|
|
@@ -785,14 +806,20 @@ too. Two rules follow:
|
|
|
785
806
|
write is remote code execution, resumed run or not.
|
|
786
807
|
|
|
787
808
|
What is bounded: ten tool calls per generation, one run of a generation at a time, a 30-second
|
|
788
|
-
timeout per execution, and 4KB of observation
|
|
789
|
-
|
|
790
|
-
|
|
791
|
-
|
|
792
|
-
|
|
809
|
+
timeout per execution, and 4KB of observation — its start and its end, where the error is. `exit` in
|
|
810
|
+
generated code is refused rather than ending your process; `exit!` cannot be caught, so only
|
|
811
|
+
Subprocess survives it. The default executor uses Ruby's `Timeout` in this
|
|
812
|
+
process — inside a database driver it can leave the connection unusable.
|
|
813
|
+
`Omakase::Executor::Subprocess` is the reference swap: the same `instance_eval`, in a child process,
|
|
814
|
+
so a timeout or a crash takes the child and not you. Ivars come back one at a time, so one Proc
|
|
815
|
+
does not drop the rest; methods the model defined on the object die with the child. The child is a
|
|
816
|
+
copy of this process, so it can still reach ActiveRecord, ENV, and the disk — that is isolation of
|
|
817
|
+
fate, not of capability. Untrusted input still belongs to `:predict`. Do not use it from a threaded
|
|
818
|
+
server; run the generation in a job. Do not call MCP tools under it: the child would share the
|
|
819
|
+
parent's stdio client.
|
|
793
820
|
|
|
794
821
|
```ruby
|
|
795
|
-
Omakase.executor =
|
|
822
|
+
Omakase.executor = Omakase::Executor::Subprocess
|
|
796
823
|
```
|
|
797
824
|
|
|
798
825
|
## Not here, on purpose
|
|
@@ -801,7 +828,8 @@ Omakase.executor = MySubprocessExecutor # returns an observation String or Ex
|
|
|
801
828
|
RubyLLM, and making it resumable would be a different library.
|
|
802
829
|
- **Reflection and forgetting in memory.** No decay, no consolidation pass: `Memory` grows until you
|
|
803
830
|
prune it, and past a few hundred entries the answer is pgvector, not more code here.
|
|
804
|
-
- **A sandbox.**
|
|
831
|
+
- **A sandbox.** The child can still reach what this process can. Subprocess isolates a crash and a
|
|
832
|
+
timeout, not `File` or ActiveRecord.
|
|
805
833
|
- **Multi-agent orchestration.** An agent is an object, so one agent calling another is a method
|
|
806
834
|
call. There is nothing to add.
|
|
807
835
|
- **Streaming.** A generation method returns a value, not tokens. RubyLLM streams if you need that.
|
data/lib/omakase/agent.rb
CHANGED
|
@@ -6,6 +6,9 @@ module Omakase
|
|
|
6
6
|
class Agent
|
|
7
7
|
# The generations this thread is inside, so one cannot re-enter itself.
|
|
8
8
|
RUNNING = :omakase_running
|
|
9
|
+
# Each level opens a chat with a fresh tool budget, so depth is what bounds the cost.
|
|
10
|
+
# ponytail: a constant; a setting when a real tree needs more.
|
|
11
|
+
MAX_DEPTH = 10
|
|
9
12
|
class << self
|
|
10
13
|
# The model and any RubyLLM chat option. Naming a provider takes the model
|
|
11
14
|
# id on trust, since providers like OpenRouter or Ollama serve ids that are
|
|
@@ -31,9 +34,14 @@ module Omakase
|
|
|
31
34
|
|
|
32
35
|
# An MCP server's tools, as methods on the agent. Options are passed to
|
|
33
36
|
# `ruby_llm-mcp` verbatim: `mcp :files, transport_type: :stdio, config: {command: "npx", …}`.
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
+
# The server opens on the first instance, not at class load.
|
|
38
|
+
def mcp(name, **options) = MCP.defer(self, name, options)
|
|
39
|
+
|
|
40
|
+
# Here rather than in `initialize`, which a subclass may override and
|
|
41
|
+
# never call super from — and then generate with none of its tools.
|
|
42
|
+
def new(...)
|
|
43
|
+
MCP.ensure(self)
|
|
44
|
+
super
|
|
37
45
|
end
|
|
38
46
|
|
|
39
47
|
# A skill directory — a SKILL.md with YAML front matter. Its description
|
|
@@ -124,6 +132,7 @@ module Omakase
|
|
|
124
132
|
subclass.strategy(@strategy) if @strategy
|
|
125
133
|
subclass.generations.merge!(generations)
|
|
126
134
|
subclass.descriptions.merge!(descriptions)
|
|
135
|
+
Skills.attach_core(subclass)
|
|
127
136
|
end
|
|
128
137
|
end
|
|
129
138
|
|
|
@@ -177,19 +186,43 @@ module Omakase
|
|
|
177
186
|
|
|
178
187
|
# Generated code can see this method and call it. Each nested call opens its
|
|
179
188
|
# own chat with its own tool budget, so the budget would bound nothing.
|
|
180
|
-
raise Error, "#{self.class}##{name} is already running — it cannot call itself" if running.
|
|
189
|
+
raise Error, "#{self.class}##{name} is already running — it cannot call itself" if running.any? { |frame| frame[:key] == key }
|
|
190
|
+
raise Error, "#{self.class}##{name}: generations nest deeper than #{MAX_DEPTH}" if running.size >= MAX_DEPTH
|
|
181
191
|
|
|
182
|
-
|
|
192
|
+
frame = {key:, inputs:}
|
|
193
|
+
running.push(frame)
|
|
183
194
|
begin
|
|
184
195
|
Omakase.emit(:generation, agent: self, name:, inputs:)
|
|
185
196
|
value = generation.strategy.call(Request.new(agent: self, generation:, inputs:))
|
|
186
197
|
Omakase.emit(:answer, agent: self, name:, value:)
|
|
187
198
|
value
|
|
188
199
|
rescue RubyLLM::Error, RubyLLM::ConfigurationError, RubyLLM::ModelNotFoundError => e
|
|
189
|
-
|
|
200
|
+
error = ProviderError.new("#{self.class}##{name}: #{e.message}")
|
|
201
|
+
Omakase.emit(:error, agent: self, name:, error:)
|
|
202
|
+
raise error
|
|
203
|
+
rescue => e
|
|
204
|
+
Omakase.emit(:error, agent: self, name:, error: e)
|
|
205
|
+
raise
|
|
190
206
|
ensure
|
|
191
|
-
running.delete(
|
|
207
|
+
running.delete(frame)
|
|
208
|
+
end
|
|
209
|
+
end
|
|
210
|
+
|
|
211
|
+
# Where generated code runs during a generation: self is the agent and the
|
|
212
|
+
# inputs are locals, which then last for the rest of that generation.
|
|
213
|
+
def omakase_scope
|
|
214
|
+
frame = Thread.current[RUNNING]&.reverse_each&.find { |entry| entry[:key].first == object_id }
|
|
215
|
+
return unless frame
|
|
216
|
+
|
|
217
|
+
frame[:scope] ||= omakase_blank_binding.tap do |scope|
|
|
218
|
+
frame[:inputs].except(:with).each do |name, value|
|
|
219
|
+
scope.local_variable_set(name, value) if /\A[a-z_]\w*\z/.match?(name.to_s)
|
|
220
|
+
end
|
|
192
221
|
end
|
|
193
222
|
end
|
|
223
|
+
|
|
224
|
+
# A string instance_eval, so a `def` in generated code lands on this object
|
|
225
|
+
# alone — and no local of the caller leaks in.
|
|
226
|
+
def omakase_blank_binding = instance_eval("binding", __FILE__, __LINE__)
|
|
194
227
|
end
|
|
195
228
|
end
|
data/lib/omakase/capabilities.rb
CHANGED
|
@@ -18,10 +18,15 @@ module Omakase
|
|
|
18
18
|
end
|
|
19
19
|
|
|
20
20
|
def entry(agent_class, name)
|
|
21
|
-
|
|
21
|
+
method = agent_class.instance_method(name)
|
|
22
|
+
signature = "#{name}(#{parameters(method)})"
|
|
22
23
|
# A prompt written as a block needs an instance to read; `describe` it instead.
|
|
23
24
|
prompt = agent_class.generations[name]&.prompt
|
|
24
|
-
|
|
25
|
+
# Look up on the method's owner so a late attach on a parent still
|
|
26
|
+
# documents the tool for subclasses created before that generate.
|
|
27
|
+
owner = method.owner
|
|
28
|
+
description = (owner.descriptions[name] if owner.respond_to?(:descriptions)) ||
|
|
29
|
+
(prompt unless prompt.is_a?(Proc))
|
|
25
30
|
description ? "#{signature} — #{description}" : signature
|
|
26
31
|
end
|
|
27
32
|
|
data/lib/omakase/executor.rb
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
module Omakase
|
|
4
4
|
# Runs model-written Ruby in the agent's own context.
|
|
5
|
-
# ponytail: instance_eval is not a sandbox —
|
|
5
|
+
# ponytail: instance_eval is not a sandbox — Subprocess isolates a crash, not File.
|
|
6
6
|
module Executor
|
|
7
7
|
SOURCE = "(generated)"
|
|
8
8
|
RESULT = :omakase_result
|
|
@@ -23,25 +23,40 @@ module Omakase
|
|
|
23
23
|
def call(agent, code, timeout: TIMEOUT)
|
|
24
24
|
printed = StringIO.new
|
|
25
25
|
answer = catch(RESULT) do
|
|
26
|
-
value = capturing(printed) { Timeout.timeout(timeout) { agent
|
|
26
|
+
value = capturing(printed) { Timeout.timeout(timeout) { evaluate(agent, code) } }
|
|
27
27
|
return observation([printed.string.chomp, "=> #{value.inspect}"])
|
|
28
28
|
end
|
|
29
29
|
Answer.new(value: answer, printed: printed.string.chomp)
|
|
30
30
|
rescue ScriptError, StandardError => e
|
|
31
31
|
observation([printed.string.chomp, failure(e, code)])
|
|
32
|
+
rescue SystemExit
|
|
33
|
+
# In process, exit would end the host. exit! cannot be caught: Subprocess covers that.
|
|
34
|
+
observation([printed.string.chomp, "exit is not allowed — answer with finish(value)"])
|
|
32
35
|
end
|
|
33
36
|
|
|
34
|
-
#
|
|
37
|
+
# Inside a generation the inputs are locals; outside one, plain instance_eval.
|
|
38
|
+
def evaluate(agent, code)
|
|
39
|
+
scope = agent.respond_to?(:omakase_scope, true) && agent.send(:omakase_scope)
|
|
40
|
+
scope ? scope.eval(code, SOURCE, 1) : agent.instance_eval(code, SOURCE, 1)
|
|
41
|
+
end
|
|
42
|
+
|
|
43
|
+
# The model can only fix what it can locate, so point at the line. A
|
|
44
|
+
# SyntaxError has no generated frame to point from.
|
|
35
45
|
def failure(error, code)
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
46
|
+
message = "#{error.class}: #{error.message}"
|
|
47
|
+
line = error.backtrace.grep(TRACE).first&.slice(/:(\d+)/, 1)&.to_i
|
|
48
|
+
return message unless line
|
|
49
|
+
|
|
50
|
+
"#{message}\nline #{line}: #{code.lines[line - 1].to_s.strip}"
|
|
39
51
|
end
|
|
40
52
|
|
|
53
|
+
# Head and tail: the error or the value comes last, and it is what the model needs next.
|
|
41
54
|
def observation(parts)
|
|
42
55
|
text = parts.reject(&:empty?).join("\n")
|
|
43
|
-
|
|
44
|
-
|
|
56
|
+
return text if text.length <= MAX_OUTPUT
|
|
57
|
+
|
|
58
|
+
half = MAX_OUTPUT / 2
|
|
59
|
+
"#{text[0, half]}\n… (#{text.length - MAX_OUTPUT} characters truncated)\n#{text[-half..]}"
|
|
45
60
|
end
|
|
46
61
|
|
|
47
62
|
# Thread-local, so concurrent agents never share a buffer. Agent#puts reads it.
|
|
@@ -52,5 +67,127 @@ module Omakase
|
|
|
52
67
|
ensure
|
|
53
68
|
Thread.current[OUTPUT] = previous
|
|
54
69
|
end
|
|
70
|
+
|
|
71
|
+
# Generated code runs in a child process so a timeout, a crash, or a
|
|
72
|
+
# runaway loop cannot take the parent with it. The child is a copy of
|
|
73
|
+
# this process — it can still reach ActiveRecord, ENV, and the disk.
|
|
74
|
+
# That is isolation of fate, not of capability. Untrusted input still
|
|
75
|
+
# belongs to :predict.
|
|
76
|
+
#
|
|
77
|
+
# Ivars written in the child are marshalled back one at a time, so a
|
|
78
|
+
# generation's second tool call sees what the first one set. Methods
|
|
79
|
+
# the model defined on the object die with the child.
|
|
80
|
+
module Subprocess
|
|
81
|
+
module_function
|
|
82
|
+
|
|
83
|
+
def call(agent, code, timeout: TIMEOUT)
|
|
84
|
+
IO.pipe(binmode: true) do |reader, writer|
|
|
85
|
+
pid = fork do
|
|
86
|
+
reader.close
|
|
87
|
+
# Own process group, so a timeout can kill grandchildren too.
|
|
88
|
+
Process.setsid
|
|
89
|
+
# Parent owns the deadline; Timeout here would race it.
|
|
90
|
+
payload = pack(agent, Executor.call(agent, code, timeout: nil))
|
|
91
|
+
writer.write([payload.bytesize].pack("N"), payload)
|
|
92
|
+
ensure
|
|
93
|
+
exit! 0
|
|
94
|
+
end
|
|
95
|
+
writer.close
|
|
96
|
+
collect(reader, agent, pid, clock + timeout)
|
|
97
|
+
end
|
|
98
|
+
end
|
|
99
|
+
|
|
100
|
+
def collect(reader, agent, pid, deadline)
|
|
101
|
+
payload = read_packet(reader, deadline)
|
|
102
|
+
stop(pid) if payload == :timeout
|
|
103
|
+
status = reap(pid)
|
|
104
|
+
case payload
|
|
105
|
+
when :timeout then "execution timed out"
|
|
106
|
+
when :eof then "child process #{fate(status)}"
|
|
107
|
+
else unpack(agent, payload)
|
|
108
|
+
end
|
|
109
|
+
end
|
|
110
|
+
|
|
111
|
+
def pack(agent, result)
|
|
112
|
+
kept, dropped = agent.marshal_dump.partition { |_, value| marshalable?(value) }
|
|
113
|
+
result = note_dropped(result, dropped.map(&:first))
|
|
114
|
+
Marshal.dump({result: carry(result), state: kept.to_h})
|
|
115
|
+
end
|
|
116
|
+
|
|
117
|
+
def marshalable?(value)
|
|
118
|
+
Marshal.dump(value)
|
|
119
|
+
true
|
|
120
|
+
rescue TypeError
|
|
121
|
+
false
|
|
122
|
+
end
|
|
123
|
+
|
|
124
|
+
# A dropped ivar turns the answer into an observation: silent state loss
|
|
125
|
+
# would leave the next tool call reasoning about a value that is gone.
|
|
126
|
+
def note_dropped(result, dropped)
|
|
127
|
+
return result if dropped.empty?
|
|
128
|
+
|
|
129
|
+
prior = result.is_a?(Answer) ? [result.printed, "finish #{result.value.inspect}"].reject(&:empty?).join("\n") : result
|
|
130
|
+
Executor.observation([prior, "cannot keep #{dropped.join(", ")} across the process boundary"])
|
|
131
|
+
end
|
|
132
|
+
|
|
133
|
+
# Only an Answer can fail here — an observation is a String.
|
|
134
|
+
def carry(result)
|
|
135
|
+
return result if marshalable?(result)
|
|
136
|
+
|
|
137
|
+
"cannot return #{result.value.class} across the process boundary"
|
|
138
|
+
end
|
|
139
|
+
|
|
140
|
+
def unpack(agent, payload)
|
|
141
|
+
packet = Marshal.load(payload)
|
|
142
|
+
agent.marshal_load(packet[:state])
|
|
143
|
+
packet[:result]
|
|
144
|
+
rescue ArgumentError, TypeError => e
|
|
145
|
+
"#{e.message}: a class defined in generated code does not survive the process boundary"
|
|
146
|
+
end
|
|
147
|
+
|
|
148
|
+
# Length-prefixed, so a leftover write-end cannot hang the parent.
|
|
149
|
+
def read_packet(io, deadline)
|
|
150
|
+
header = read_exactly(io, 4, deadline)
|
|
151
|
+
return header if header.is_a?(Symbol)
|
|
152
|
+
|
|
153
|
+
read_exactly(io, header.unpack1("N"), deadline)
|
|
154
|
+
end
|
|
155
|
+
|
|
156
|
+
# select is exact for a pipe, so readpartial cannot block past the deadline.
|
|
157
|
+
def read_exactly(io, n, deadline)
|
|
158
|
+
buf = "".b
|
|
159
|
+
while buf.bytesize < n
|
|
160
|
+
return :timeout unless IO.select([io], nil, nil, [deadline - clock, 0].max)
|
|
161
|
+
|
|
162
|
+
buf << io.readpartial(n - buf.bytesize)
|
|
163
|
+
end
|
|
164
|
+
buf
|
|
165
|
+
rescue EOFError
|
|
166
|
+
:eof
|
|
167
|
+
end
|
|
168
|
+
|
|
169
|
+
def clock = Process.clock_gettime(Process::CLOCK_MONOTONIC)
|
|
170
|
+
|
|
171
|
+
def fate(status)
|
|
172
|
+
return "was killed" if status.nil? || status.signaled?
|
|
173
|
+
return "ended without an answer" if status.success?
|
|
174
|
+
|
|
175
|
+
"exited #{status.exitstatus}"
|
|
176
|
+
end
|
|
177
|
+
|
|
178
|
+
# The child led its own group unless the deadline beat it to setsid.
|
|
179
|
+
def stop(pid)
|
|
180
|
+
Process.kill("KILL", (Process.getpgid(pid) == pid) ? -pid : pid)
|
|
181
|
+
rescue Errno::ESRCH
|
|
182
|
+
nil
|
|
183
|
+
end
|
|
184
|
+
|
|
185
|
+
# ECHILD: the host reaps children itself, with a CHLD trap.
|
|
186
|
+
def reap(pid)
|
|
187
|
+
Process.wait2(pid).last
|
|
188
|
+
rescue Errno::ECHILD
|
|
189
|
+
nil
|
|
190
|
+
end
|
|
191
|
+
end
|
|
55
192
|
end
|
|
56
193
|
end
|
data/lib/omakase/fake_chat.rb
CHANGED
|
@@ -9,9 +9,22 @@ module Omakase
|
|
|
9
9
|
# The script receives the chat, so it can drive the tool the way a model would:
|
|
10
10
|
#
|
|
11
11
|
# Omakase::FakeChat.new { |chat| chat.run("finish(42)") }
|
|
12
|
+
#
|
|
13
|
+
# Or one reply per model turn, in order; a turn past the last one raises:
|
|
14
|
+
#
|
|
15
|
+
# Omakase::FakeChat.replies("prose, not JSON", {"result" => 42})
|
|
12
16
|
class FakeChat
|
|
13
17
|
Response = Struct.new(:content)
|
|
14
18
|
|
|
19
|
+
def self.replies(*replies)
|
|
20
|
+
new do |chat|
|
|
21
|
+
raise Error, "FakeChat: no scripted reply left for turn #{chat.tasks.size}" if replies.empty?
|
|
22
|
+
|
|
23
|
+
reply = replies.shift
|
|
24
|
+
reply.respond_to?(:call) ? reply.call(chat) : reply
|
|
25
|
+
end
|
|
26
|
+
end
|
|
27
|
+
|
|
15
28
|
attr_reader :instructions, :schema, :tools, :tasks, :attachments
|
|
16
29
|
|
|
17
30
|
def initialize(&script)
|
|
@@ -20,20 +33,32 @@ module Omakase
|
|
|
20
33
|
@tools = []
|
|
21
34
|
@tasks = []
|
|
22
35
|
@attachments = []
|
|
36
|
+
@complete = true
|
|
23
37
|
end
|
|
24
38
|
|
|
25
|
-
|
|
39
|
+
# A log of every call — the real chat replaces unless `append:`.
|
|
40
|
+
def with_instructions(text, **) = tap { @instructions << text }
|
|
26
41
|
|
|
27
42
|
def with_schema(schema) = tap { @schema = schema }
|
|
28
43
|
|
|
29
|
-
def
|
|
44
|
+
def with_tools(*tools) = tap { @tools.concat(tools) }
|
|
30
45
|
|
|
31
|
-
def ask(task, with: nil)
|
|
46
|
+
def ask(task, with: nil) = ask_later(task, with:).step
|
|
47
|
+
|
|
48
|
+
def ask_later(task, with: nil)
|
|
32
49
|
@tasks << task
|
|
33
50
|
@attachments << with if with
|
|
51
|
+
tap { @complete = false }
|
|
52
|
+
end
|
|
53
|
+
|
|
54
|
+
# The whole script is one step: it answers, so the chat is then complete.
|
|
55
|
+
def step
|
|
56
|
+
@complete = true
|
|
34
57
|
Response.new(@script.call(self))
|
|
35
58
|
end
|
|
36
59
|
|
|
60
|
+
def complete? = @complete
|
|
61
|
+
|
|
37
62
|
# Run code the way the model would, through the agent's one tool.
|
|
38
63
|
def run(code) = tools.fetch(0).call(code:)
|
|
39
64
|
end
|
data/lib/omakase/mcp.rb
CHANGED
|
@@ -4,19 +4,91 @@ module Omakase
|
|
|
4
4
|
# An MCP server's tools, as methods on the agent — so generated code calls a
|
|
5
5
|
# remote tool the same way it calls anything else the agent exposes.
|
|
6
6
|
module MCP
|
|
7
|
+
LOCKS = Mutex.new
|
|
8
|
+
# A tool name landing on a method is your bug, not a sidecar being down —
|
|
9
|
+
# and both arrive as an Error out of the same call.
|
|
10
|
+
Clash = Class.new(Error)
|
|
11
|
+
|
|
12
|
+
class << self
|
|
13
|
+
def client_factory=(factory)
|
|
14
|
+
unless factory.nil? || factory.respond_to?(:call)
|
|
15
|
+
raise Error, "client_factory must answer call, got #{factory.class}"
|
|
16
|
+
end
|
|
17
|
+
|
|
18
|
+
@client_factory = factory
|
|
19
|
+
end
|
|
20
|
+
|
|
21
|
+
def client_factory
|
|
22
|
+
@client_factory || ->(name, **options) {
|
|
23
|
+
require "ruby_llm/mcp"
|
|
24
|
+
RubyLLM::MCP.add_client(name: name.to_s, **options)
|
|
25
|
+
}
|
|
26
|
+
end
|
|
27
|
+
end
|
|
28
|
+
|
|
7
29
|
module_function
|
|
8
30
|
|
|
31
|
+
def defer(agent_class, name, options)
|
|
32
|
+
servers(agent_class)[name.to_sym] = options
|
|
33
|
+
end
|
|
34
|
+
|
|
35
|
+
def ensure(agent_class)
|
|
36
|
+
return unless pending?(agent_class)
|
|
37
|
+
|
|
38
|
+
agent_class.ancestors.take_while { |mod| mod != Agent }.reverse_each do |klass|
|
|
39
|
+
next unless klass.is_a?(Class)
|
|
40
|
+
|
|
41
|
+
mutex_for(klass).synchronize { attach_pending(klass) }
|
|
42
|
+
end
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
def pending?(agent_class)
|
|
46
|
+
agent_class.ancestors.take_while { |mod| mod != Agent }.any? do |mod|
|
|
47
|
+
mod.is_a?(Class) && (servers(mod).keys - attached(mod)).any?
|
|
48
|
+
end
|
|
49
|
+
end
|
|
50
|
+
|
|
51
|
+
def attach_pending(klass)
|
|
52
|
+
servers(klass).each do |name, options|
|
|
53
|
+
next if attached(klass).include?(name)
|
|
54
|
+
|
|
55
|
+
# A down sidecar stays unattached — connecting and listing its tools are
|
|
56
|
+
# the same trip. The generate still runs. Next ensure retries. A name
|
|
57
|
+
# clash is not the sidecar being down, it is your bug, so it raises.
|
|
58
|
+
begin
|
|
59
|
+
attach(klass, client_factory.call(name, **options))
|
|
60
|
+
rescue Clash
|
|
61
|
+
raise
|
|
62
|
+
rescue => e
|
|
63
|
+
Omakase.emit(:mcp, agent: klass, name:, error: e)
|
|
64
|
+
next
|
|
65
|
+
end
|
|
66
|
+
attached(klass) << name
|
|
67
|
+
end
|
|
68
|
+
end
|
|
69
|
+
|
|
9
70
|
def attach(agent_class, client)
|
|
71
|
+
defined = []
|
|
10
72
|
client.tools.each do |tool|
|
|
11
73
|
name = method_name(tool)
|
|
12
74
|
# A remote tool list must not quietly shadow a capability the agent already has.
|
|
13
|
-
raise
|
|
75
|
+
raise Clash, "#{agent_class} already has ##{name}" if Capabilities.names(agent_class).include?(name)
|
|
14
76
|
|
|
15
77
|
agent_class.describe(description(tool))
|
|
16
78
|
# nil is how a model leaves an argument out; MCP servers reject it.
|
|
17
79
|
agent_class.define_method(name) { |**arguments| MCP.result(tool.execute(**arguments.compact)) }
|
|
80
|
+
defined << name
|
|
18
81
|
end
|
|
19
82
|
client
|
|
83
|
+
rescue
|
|
84
|
+
# Else the next ensure dies on "already has #name".
|
|
85
|
+
defined.each do |name|
|
|
86
|
+
agent_class.send(:remove_method, name)
|
|
87
|
+
agent_class.descriptions.delete(name)
|
|
88
|
+
end
|
|
89
|
+
agent_class.instance_variable_set(:@pending_description, nil)
|
|
90
|
+
client.close if client.respond_to?(:close)
|
|
91
|
+
raise
|
|
20
92
|
end
|
|
21
93
|
|
|
22
94
|
# Tool names may hold characters a Ruby method name cannot.
|
|
@@ -39,5 +111,22 @@ module Omakase
|
|
|
39
111
|
text = tool.description.to_s.gsub(/\s+/, " ").strip
|
|
40
112
|
[text, ("Arguments — #{arguments.join(", ")}" if arguments.any?)].compact.join(" ")
|
|
41
113
|
end
|
|
114
|
+
|
|
115
|
+
def mutex_for(klass)
|
|
116
|
+
mutex = klass.instance_variable_get(:@mcp_mutex)
|
|
117
|
+
return mutex if mutex
|
|
118
|
+
|
|
119
|
+
LOCKS.synchronize do
|
|
120
|
+
klass.instance_variable_get(:@mcp_mutex) || klass.instance_variable_set(:@mcp_mutex, Mutex.new)
|
|
121
|
+
end
|
|
122
|
+
end
|
|
123
|
+
|
|
124
|
+
def servers(klass)
|
|
125
|
+
klass.instance_variable_get(:@mcp_servers) || klass.instance_variable_set(:@mcp_servers, {})
|
|
126
|
+
end
|
|
127
|
+
|
|
128
|
+
def attached(klass)
|
|
129
|
+
klass.instance_variable_get(:@mcp_attached) || klass.instance_variable_set(:@mcp_attached, [])
|
|
130
|
+
end
|
|
42
131
|
end
|
|
43
132
|
end
|
data/lib/omakase/request.rb
CHANGED
|
@@ -7,17 +7,21 @@ module Omakase
|
|
|
7
7
|
|
|
8
8
|
def schema = generation.schema
|
|
9
9
|
|
|
10
|
-
def instructions =
|
|
10
|
+
def instructions = agent.class.instructions
|
|
11
|
+
|
|
12
|
+
# Rebuilt on every call, so it goes after what a provider can cache.
|
|
13
|
+
def context = agent.context.to_s
|
|
11
14
|
|
|
12
15
|
# `with:` is reserved: files for the model to look at, passed through to
|
|
13
16
|
# RubyLLM's `ask(with:)` as attachments rather than rendered into the text.
|
|
14
17
|
def attachments = inputs[:with]
|
|
15
18
|
|
|
16
|
-
|
|
19
|
+
# With `preview:`, the model's code holds the inputs, so the prompt only shows them.
|
|
20
|
+
def task(preview: false)
|
|
17
21
|
arguments = inputs.except(:with)
|
|
18
22
|
return prompt if arguments.empty?
|
|
19
23
|
|
|
20
|
-
lines = arguments.map { |name, value| "- #{name}: #{value.inspect}" }
|
|
24
|
+
lines = arguments.map { |name, value| "- #{name}: #{preview ? shorten(value.inspect) : value.inspect}" }
|
|
21
25
|
"#{prompt}\n\nInputs:\n#{lines.join("\n")}"
|
|
22
26
|
end
|
|
23
27
|
|
|
@@ -27,5 +31,13 @@ module Omakase
|
|
|
27
31
|
text = generation.prompt
|
|
28
32
|
text.is_a?(Proc) ? agent.instance_exec(&text) : text
|
|
29
33
|
end
|
|
34
|
+
|
|
35
|
+
# Past the limit, code_act shows the start of an input: the whole value is a local.
|
|
36
|
+
# ponytail: inspects the whole value, then cuts; a bounded printer if a huge input shows up in a profile.
|
|
37
|
+
def shorten(text, limit = 500)
|
|
38
|
+
return text if text.length <= limit
|
|
39
|
+
|
|
40
|
+
"#{text[0, limit]}… (#{text.length} characters — the whole value is in the local)"
|
|
41
|
+
end
|
|
30
42
|
end
|
|
31
43
|
end
|
data/lib/omakase/schema.rb
CHANGED
|
@@ -43,9 +43,10 @@ module Omakase
|
|
|
43
43
|
|
|
44
44
|
# From the provider's JSON: unwrap first, then hold it to the contract.
|
|
45
45
|
def cast(content)
|
|
46
|
+
content = parse(content) if content.is_a?(String)
|
|
46
47
|
raise ContractError, "expected JSON matching #{JSON.generate(json)}, got #{content.inspect}" unless content.is_a?(Hash)
|
|
47
48
|
|
|
48
|
-
data =
|
|
49
|
+
data = symbolize(content)
|
|
49
50
|
take(wrapped? ? data.fetch(RESULT) { raise ContractError, %(missing "result" in #{data.inspect}) } : data)
|
|
50
51
|
end
|
|
51
52
|
|
|
@@ -54,9 +55,9 @@ module Omakase
|
|
|
54
55
|
return demand(value, properties.fetch("result")["type"]) if wrapped?
|
|
55
56
|
raise ContractError, "expected #{describe}, got #{value.inspect}" unless value.is_a?(Hash)
|
|
56
57
|
|
|
57
|
-
data =
|
|
58
|
-
|
|
59
|
-
raise ContractError, "
|
|
58
|
+
data = symbolize(value)
|
|
59
|
+
problem = object_mismatch(data, json, nil)
|
|
60
|
+
raise ContractError, "#{problem} — expected #{describe}" if problem
|
|
60
61
|
|
|
61
62
|
data
|
|
62
63
|
end
|
|
@@ -67,11 +68,60 @@ module Omakase
|
|
|
67
68
|
|
|
68
69
|
def properties = json.fetch("properties")
|
|
69
70
|
|
|
71
|
+
# RubyLLM 2 hands structured output back as a JSON string. Not JSON stays a
|
|
72
|
+
# String, so cast reports what the model actually said.
|
|
73
|
+
def parse(content)
|
|
74
|
+
JSON.parse(content)
|
|
75
|
+
rescue JSON::ParserError
|
|
76
|
+
content
|
|
77
|
+
end
|
|
78
|
+
|
|
79
|
+
def symbolize(value)
|
|
80
|
+
case value
|
|
81
|
+
when Hash then value.to_h { |key, item| [key.respond_to?(:to_sym) ? key.to_sym : key, symbolize(item)] }
|
|
82
|
+
when Array then value.map { |item| symbolize(item) }
|
|
83
|
+
else value
|
|
84
|
+
end
|
|
85
|
+
end
|
|
86
|
+
|
|
70
87
|
def demand(value, type)
|
|
71
|
-
|
|
72
|
-
raise ContractError, "expected <#{type}>, got #{value.inspect}" unless matched
|
|
88
|
+
raise ContractError, "expected <#{type}>, got #{value.inspect}" unless type?(value, type)
|
|
73
89
|
|
|
74
90
|
value
|
|
75
91
|
end
|
|
92
|
+
|
|
93
|
+
def type?(value, type)
|
|
94
|
+
case type
|
|
95
|
+
when "boolean" then [true, false].include?(value)
|
|
96
|
+
when "null" then value.nil?
|
|
97
|
+
when Array then type.any? { |each| type?(value, each) }
|
|
98
|
+
else value.is_a?(RUBY_TYPES.fetch(type, BasicObject))
|
|
99
|
+
end
|
|
100
|
+
end
|
|
101
|
+
|
|
102
|
+
# The first place a nested value breaks its schema, as a path the model can fix.
|
|
103
|
+
def mismatch(value, spec, path)
|
|
104
|
+
return "#{path}: expected <#{Array(spec["type"]).join("|")}>, got #{value.inspect}" if spec["type"] && !type?(value, spec["type"])
|
|
105
|
+
return "#{path}: expected one of #{spec["enum"].inspect}, got #{value.inspect}" if spec["enum"] && !spec["enum"].include?(value)
|
|
106
|
+
|
|
107
|
+
case value
|
|
108
|
+
when Array then value.each_with_index.filter_map { |item, i| mismatch(item, spec["items"], "#{path}[#{i}]") if spec["items"] }.first
|
|
109
|
+
when Hash then object_mismatch(value, spec, path)
|
|
110
|
+
end
|
|
111
|
+
end
|
|
112
|
+
|
|
113
|
+
# An optional field left nil is a field left out, which is allowed.
|
|
114
|
+
def object_mismatch(value, spec, path)
|
|
115
|
+
required = Array(spec["required"])
|
|
116
|
+
missing = required.map(&:to_sym) - value.keys
|
|
117
|
+
return [path, "missing #{missing.join(", ")}"].compact.join(": ") if missing.any?
|
|
118
|
+
|
|
119
|
+
Hash(spec["properties"]).filter_map do |name, child|
|
|
120
|
+
field = value[name.to_sym]
|
|
121
|
+
next if field.nil? && !required.include?(name)
|
|
122
|
+
|
|
123
|
+
mismatch(field, child, [path, name].compact.join(".")) if value.key?(name.to_sym)
|
|
124
|
+
end.first
|
|
125
|
+
end
|
|
76
126
|
end
|
|
77
127
|
end
|
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: how-to-act
|
|
3
|
+
description: How to write the Ruby that implements a generation — finish, prints, doc, ivars. Call once before you act.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
You write Ruby. It runs on the agent: its methods and ivars are yours, on self.
|
|
7
|
+
|
|
8
|
+
Call a method. Print what you need to see. `finish` with the answer — the value, not a sentence about it.
|
|
9
|
+
|
|
10
|
+
```ruby
|
|
11
|
+
orders = orders_for("ada@example.com")
|
|
12
|
+
orders.each { |o| puts "total: #{o.total}" }
|
|
13
|
+
puts policy_on(:damage)
|
|
14
|
+
finish(Refund.new(order_id: 1, amount: 39.9, reason: "cracked mug, policy :damage"))
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
Without `finish`, the last expression comes back as `=> …` and you keep going.
|
|
18
|
+
|
|
19
|
+
```ruby
|
|
20
|
+
stock_of("apple") + stock_of("pear")
|
|
21
|
+
# => 7
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
The inputs are local variables. Locals and ivars last for the rest of this generation:
|
|
25
|
+
|
|
26
|
+
```ruby
|
|
27
|
+
n = items.sum { |item| stock_of(item) }
|
|
28
|
+
# later:
|
|
29
|
+
finish(n + 1)
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
An object you do not know:
|
|
33
|
+
|
|
34
|
+
```ruby
|
|
35
|
+
doc(orders.first) # a class works too
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
For meaning — classify, judge, summarize — call a generation method, not a regex. Never type out large data by hand: compute it or take it from an input.
|
|
39
|
+
|
|
40
|
+
Prints come back to you, not to the process. If `finish` is refused, the message says why — fix it in the next call. Do not retype a value you already computed. Work in as few calls as you can.
|
data/lib/omakase/skills.rb
CHANGED
|
@@ -6,11 +6,18 @@ module Omakase
|
|
|
6
6
|
# capabilities; the body only arrives when the model calls the method — which
|
|
7
7
|
# is all "loaded on demand" has to mean.
|
|
8
8
|
module Skills
|
|
9
|
+
CORE = File.expand_path("skills/how_to_act", __dir__)
|
|
10
|
+
|
|
9
11
|
module_function
|
|
10
12
|
|
|
13
|
+
# Every agent gets the how-to. Skip when a parent already defined it.
|
|
14
|
+
def attach_core(agent_class)
|
|
15
|
+
attach(agent_class, CORE) unless Capabilities.names(agent_class).include?(:how_to_act)
|
|
16
|
+
end
|
|
17
|
+
|
|
11
18
|
def attach(agent_class, path)
|
|
12
19
|
directory = File.expand_path(path)
|
|
13
|
-
front_matter, body = parse(File.read(File.join(directory, "SKILL.md")))
|
|
20
|
+
front_matter, body = parse(File.read(File.join(directory, "SKILL.md"), encoding: "UTF-8"))
|
|
14
21
|
name = (front_matter["name"] || File.basename(directory)).tr("-", "_").to_sym
|
|
15
22
|
raise Error, "#{agent_class} already has ##{name}" if Capabilities.names(agent_class).include?(name)
|
|
16
23
|
|
|
@@ -19,12 +26,19 @@ module Omakase
|
|
|
19
26
|
name
|
|
20
27
|
end
|
|
21
28
|
|
|
22
|
-
# The front matter every SKILL.md in the wild is written with.
|
|
29
|
+
# The front matter every SKILL.md in the wild is written with — CRLF too.
|
|
23
30
|
def parse(text)
|
|
24
|
-
match = text.match(/\A---\n(.*?)\n---\n(.*)\z/m)
|
|
31
|
+
match = text.match(/\A---\r?\n(.*?)\r?\n---\r?\n(.*)\z/m)
|
|
25
32
|
return [{}, text.strip] unless match
|
|
26
33
|
|
|
27
|
-
[
|
|
34
|
+
[front_matter(match[1]), match[2].strip]
|
|
35
|
+
end
|
|
36
|
+
|
|
37
|
+
# Claude Code style hints like `argument-hint: "<x>" [-p]` are not YAML; read those line by line.
|
|
38
|
+
def front_matter(text)
|
|
39
|
+
YAML.safe_load(text)
|
|
40
|
+
rescue Psych::SyntaxError
|
|
41
|
+
text.scan(/^([\w-]+):[ \t]*(.*?)\r?$/).to_h
|
|
28
42
|
end
|
|
29
43
|
end
|
|
30
44
|
end
|
|
@@ -9,14 +9,16 @@ module Omakase
|
|
|
9
9
|
|
|
10
10
|
def call(request)
|
|
11
11
|
tool = Tools::Ruby.new(request.agent, request.schema)
|
|
12
|
-
|
|
13
|
-
.
|
|
14
|
-
.
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
12
|
+
chat = Predict.instruct(request.chat, instructions(request), request.context)
|
|
13
|
+
.with_tools(tool)
|
|
14
|
+
.ask_later(request.task(preview: true), with: request.attachments)
|
|
15
|
+
response = run(chat, tool)
|
|
16
|
+
# A text reply is usually a model that forgot how to answer, not one that is done.
|
|
17
|
+
response = run(chat.ask_later(nudge(request)), tool) unless tool.done?
|
|
18
18
|
return tool.answer.value if tool.answer
|
|
19
19
|
|
|
20
|
+
notes = response&.content
|
|
21
|
+
|
|
20
22
|
# It never called finish. A JSON answer can still be given in a tool-free turn.
|
|
21
23
|
return Predict.call(request, task: "#{request.task}\n\nWork done:\n#{notes}") unless request.schema.code_only?
|
|
22
24
|
|
|
@@ -24,18 +26,33 @@ module Omakase
|
|
|
24
26
|
raise ContractError, "#{request.generation.name}: the model never called finish(#{request.schema.describe})"
|
|
25
27
|
end
|
|
26
28
|
|
|
29
|
+
def run(chat, tool)
|
|
30
|
+
response = nil
|
|
31
|
+
response = chat.step until chat.complete? || tool.done?
|
|
32
|
+
response
|
|
33
|
+
end
|
|
34
|
+
|
|
35
|
+
def nudge(request)
|
|
36
|
+
"Your reply was text with no tool call, so the task is not done. " \
|
|
37
|
+
"Call the `ruby` tool and end with finish(#{request.schema.describe})."
|
|
38
|
+
end
|
|
39
|
+
|
|
40
|
+
# Nothing here changes between calls of one generation, so providers can cache it.
|
|
27
41
|
def instructions(request)
|
|
28
42
|
<<~TEXT
|
|
29
43
|
#{request.instructions}
|
|
30
44
|
|
|
31
45
|
You act by writing Ruby: call the `ruby` tool with code that is evaluated on the
|
|
32
|
-
agent object, so its methods and state are available on self.
|
|
46
|
+
agent object, so its methods and state are available on self. The inputs are
|
|
47
|
+
local variables in that code — use them, do not retype them.
|
|
33
48
|
|
|
34
49
|
#{capabilities(request).join("\n")}
|
|
35
50
|
|
|
36
51
|
`doc(object)` prints what an object of an unfamiliar type offers; a class works too.
|
|
52
|
+
`how_to_act` is the rest, with examples — call it once before you write code.
|
|
37
53
|
|
|
38
|
-
|
|
54
|
+
Run the task; do not define a method for it. Return the answer from inside the
|
|
55
|
+
code, never as a message — the last thing you run is:
|
|
39
56
|
|
|
40
57
|
finish(#{request.schema.describe})
|
|
41
58
|
|
|
@@ -9,8 +9,7 @@ module Omakase
|
|
|
9
9
|
module_function
|
|
10
10
|
|
|
11
11
|
def call(request, task: request.task)
|
|
12
|
-
chat = request.chat
|
|
13
|
-
.with_instructions(instructions(request))
|
|
12
|
+
chat = instruct(request.chat, instructions(request), request.context)
|
|
14
13
|
.with_schema(request.schema.definition)
|
|
15
14
|
|
|
16
15
|
request.schema.cast(chat.ask(task, with: request.attachments).content)
|
|
@@ -19,6 +18,13 @@ module Omakase
|
|
|
19
18
|
request.schema.cast(chat.ask("#{e.message}\n\n#{CORRECTION}").content)
|
|
20
19
|
end
|
|
21
20
|
|
|
21
|
+
# The stable text first and marked for the cache; the context, which
|
|
22
|
+
# changes from call to call, after it.
|
|
23
|
+
def instruct(chat, stable, context)
|
|
24
|
+
chat = chat.with_instructions(stable, cache_until_here: true)
|
|
25
|
+
context.empty? ? chat : chat.with_instructions(context, append: true)
|
|
26
|
+
end
|
|
27
|
+
|
|
22
28
|
# Weaker providers treat the schema as a hint, so it goes in the prompt too.
|
|
23
29
|
def instructions(request)
|
|
24
30
|
"#{request.instructions}\n\nAnswer as JSON matching this schema:\n#{JSON.generate(request.schema.json)}"
|
data/lib/omakase/tools/ruby.rb
CHANGED
|
@@ -6,6 +6,8 @@ module Omakase
|
|
|
6
6
|
# through it, so the model composes calls in code instead of one per turn.
|
|
7
7
|
class Ruby < RubyLLM::Tool
|
|
8
8
|
BUDGET = 10
|
|
9
|
+
# Only a fence around the whole code: one inside it is part of a string.
|
|
10
|
+
FENCE = /\A```(?:ruby|rb)?[ \t]*\r?\n(.*?)\r?\n?```\z/m
|
|
9
11
|
|
|
10
12
|
description <<~TEXT
|
|
11
13
|
Evaluate Ruby in the context of the agent object: its methods and state are
|
|
@@ -13,7 +15,7 @@ module Omakase
|
|
|
13
15
|
expression, is returned to you. Call finish(value) to answer.
|
|
14
16
|
TEXT
|
|
15
17
|
|
|
16
|
-
|
|
18
|
+
parameter :code, description: "Ruby source to evaluate."
|
|
17
19
|
|
|
18
20
|
attr_reader :answer
|
|
19
21
|
|
|
@@ -29,11 +31,17 @@ module Omakase
|
|
|
29
31
|
|
|
30
32
|
def name = "ruby"
|
|
31
33
|
|
|
34
|
+
# RubyLLM 2 tools cannot end the loop, so the strategy asks this between steps.
|
|
35
|
+
def done? = !@answer.nil? || @calls > @budget + 1
|
|
36
|
+
|
|
32
37
|
def execute(code:)
|
|
38
|
+
code = code.strip[FENCE, 1] || code
|
|
33
39
|
# Nothing bounds the provider's tool loop, so the budget does.
|
|
34
40
|
@calls += 1
|
|
35
41
|
return "No tool calls left — answer with what you have." if @calls == @budget + 1
|
|
36
|
-
return
|
|
42
|
+
return "Tool budget spent." if @calls > @budget + 1
|
|
43
|
+
# A later call in the same round must not act on an agent that has answered.
|
|
44
|
+
return "Answer already accepted." if @answer
|
|
37
45
|
|
|
38
46
|
outcome = @executor.call(@agent, code, timeout: @timeout)
|
|
39
47
|
Omakase.emit(:ruby, agent: @agent, code:, outcome:)
|
|
@@ -42,7 +50,7 @@ module Omakase
|
|
|
42
50
|
raise Error, "executor must return a String or Executor::Answer, got #{outcome.class}" unless outcome.is_a?(Executor::Answer)
|
|
43
51
|
|
|
44
52
|
@answer = Executor::Answer.new(value: @schema.take(outcome.value), printed: outcome.printed)
|
|
45
|
-
|
|
53
|
+
"Answer accepted."
|
|
46
54
|
rescue ContractError => e
|
|
47
55
|
# Off-contract answers are corrected inside the same loop, not by another request.
|
|
48
56
|
# Anything else — a broken executor, a bad configuration — is not the model's to fix.
|
data/lib/omakase/trace.rb
CHANGED
|
@@ -5,7 +5,7 @@ module Omakase
|
|
|
5
5
|
# A run reads top to bottom — the call, the code the model wrote, the answer.
|
|
6
6
|
# Colour when the stream is a terminal, plain when it is a log.
|
|
7
7
|
class Trace
|
|
8
|
-
COLOURS = {generation: 36, ruby: 33, answer: 32}.freeze
|
|
8
|
+
COLOURS = {generation: 36, ruby: 33, answer: 32, error: 31, mcp: 31}.freeze
|
|
9
9
|
LIMIT = 800
|
|
10
10
|
|
|
11
11
|
def initialize(io: $stderr)
|
|
@@ -18,11 +18,16 @@ module Omakase
|
|
|
18
18
|
when :generation then ["→ #{agent.class}##{payload[:name]}", inputs(payload[:inputs])]
|
|
19
19
|
when :ruby then ["· ruby", "#{payload[:code].strip}\n#{outcome(payload[:outcome])}"]
|
|
20
20
|
when :answer then ["← #{agent.class}##{payload[:name]}", truncate(payload[:value].inspect)]
|
|
21
|
+
when :error then ["✗ #{agent.class}##{payload[:name]}", truncate("#{payload[:error].class}: #{payload[:error].message}")]
|
|
22
|
+
# A down sidecar is not an error the run raises, so nothing else would say it.
|
|
23
|
+
when :mcp then ["! #{agent} mcp #{payload[:name]}", truncate(payload[:error].message)]
|
|
21
24
|
else return # a listener that raises takes the run down with it
|
|
22
25
|
end
|
|
23
26
|
|
|
24
|
-
|
|
25
|
-
|
|
27
|
+
# A generation called from generated code sits inside its caller's.
|
|
28
|
+
indent = " " * [(Thread.current[Agent::RUNNING]&.size || 1) - 1, 0].max
|
|
29
|
+
@io.puts(indent + paint(event, head))
|
|
30
|
+
@io.puts(body.gsub(/^/, "#{indent} ")) unless body.empty?
|
|
26
31
|
end
|
|
27
32
|
|
|
28
33
|
private
|
data/lib/omakase/version.rb
CHANGED
data/lib/omakase.rb
CHANGED
|
@@ -21,6 +21,8 @@ module Omakase
|
|
|
21
21
|
ContractError = Class.new(Error)
|
|
22
22
|
# The model or its provider failed. RubyLLM has already retried what it retries.
|
|
23
23
|
ProviderError = Class.new(Error)
|
|
24
|
+
# What RubyLLM.chat itself takes; every other option is a `with_*` call on the chat.
|
|
25
|
+
CHAT_ARGUMENTS = %i[model provider protocol assume_model_exists context].freeze
|
|
24
26
|
|
|
25
27
|
class << self
|
|
26
28
|
# Providers, keys, default model, timeouts, logging — all of it is RubyLLM's.
|
|
@@ -45,7 +47,8 @@ module Omakase
|
|
|
45
47
|
end
|
|
46
48
|
|
|
47
49
|
# Where generated code runs. Anything answering `call(agent, code, timeout:)`
|
|
48
|
-
# will do —
|
|
50
|
+
# will do — Executor::Subprocess is the reference: a child process, so a
|
|
51
|
+
# timeout cannot take this one with it.
|
|
49
52
|
def executor=(executor)
|
|
50
53
|
@executor = callable!(executor, "executor")
|
|
51
54
|
end
|
|
@@ -67,7 +70,27 @@ module Omakase
|
|
|
67
70
|
@chat_factory = callable!(factory, "chat_factory")
|
|
68
71
|
end
|
|
69
72
|
|
|
70
|
-
def chat_factory = @chat_factory ||=
|
|
73
|
+
def chat_factory = @chat_factory ||= method(:build_chat)
|
|
74
|
+
|
|
75
|
+
# `caching: true`, `thinking: {effort: :high}`, `temperature: 0.2` become the
|
|
76
|
+
# chat's own with_* calls. Caching is on unless you say `caching: false`: the
|
|
77
|
+
# tool loop resends the whole chat on every step.
|
|
78
|
+
def build_chat(**options)
|
|
79
|
+
chat = RubyLLM.chat(**options.slice(*CHAT_ARGUMENTS))
|
|
80
|
+
{caching: true, **options.except(*CHAT_ARGUMENTS)}.each do |name, value|
|
|
81
|
+
setter = :"with_#{name}"
|
|
82
|
+
raise Error, "unknown chat option #{name}: RubyLLM::Chat has no ##{setter}" unless chat.respond_to?(setter)
|
|
83
|
+
|
|
84
|
+
keywords = chat.method(setter).parameters.any? { |kind, _| kind == :keyrest }
|
|
85
|
+
case value
|
|
86
|
+
when true then chat.public_send(setter)
|
|
87
|
+
when Array then chat.public_send(setter, *value)
|
|
88
|
+
when Hash then keywords ? chat.public_send(setter, **value) : chat.public_send(setter, value)
|
|
89
|
+
else chat.public_send(setter, value)
|
|
90
|
+
end
|
|
91
|
+
end
|
|
92
|
+
chat
|
|
93
|
+
end
|
|
71
94
|
|
|
72
95
|
# Every step, as it happens: a generation starts, model-written code runs,
|
|
73
96
|
# an answer lands. Anything answering `call(event, **payload)` will do —
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: omakase-agents
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.4.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- eugeny
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2026-
|
|
11
|
+
date: 2026-09-25 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: ruby_llm
|
|
@@ -16,14 +16,14 @@ dependencies:
|
|
|
16
16
|
requirements:
|
|
17
17
|
- - "~>"
|
|
18
18
|
- !ruby/object:Gem::Version
|
|
19
|
-
version: '
|
|
19
|
+
version: '2.0'
|
|
20
20
|
type: :runtime
|
|
21
21
|
prerelease: false
|
|
22
22
|
version_requirements: !ruby/object:Gem::Requirement
|
|
23
23
|
requirements:
|
|
24
24
|
- - "~>"
|
|
25
25
|
- !ruby/object:Gem::Version
|
|
26
|
-
version: '
|
|
26
|
+
version: '2.0'
|
|
27
27
|
- !ruby/object:Gem::Dependency
|
|
28
28
|
name: schematist
|
|
29
29
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -79,6 +79,7 @@ files:
|
|
|
79
79
|
- lib/omakase/request.rb
|
|
80
80
|
- lib/omakase/schema.rb
|
|
81
81
|
- lib/omakase/skills.rb
|
|
82
|
+
- lib/omakase/skills/how_to_act/SKILL.md
|
|
82
83
|
- lib/omakase/strategies.rb
|
|
83
84
|
- lib/omakase/strategies/code_act.rb
|
|
84
85
|
- lib/omakase/strategies/predict.rb
|