llm.rb 16.0.0 → 17.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +535 -2
- data/README.md +857 -501
- data/data/alibaba.json +2 -2
- data/data/anthropic.json +66 -3
- data/data/bedrock.json +781 -12
- data/data/deepinfra.json +157 -1
- data/data/google.json +48 -0
- data/data/mistral.json +326 -16
- data/data/moonshot.json +2 -1
- data/data/openai.json +13 -0
- data/data/openrouter.json +1078 -1235
- data/data/xai.json +4 -2
- data/docs/deepdive/advanced/cancellation.md +35 -7
- data/docs/deepdive/advanced/compaction.md +5 -5
- data/docs/deepdive/advanced/context.md +45 -24
- data/docs/deepdive/advanced/{guard.md → intercept.md} +87 -83
- data/docs/deepdive/features/builtin_tools.md +1 -1
- data/docs/deepdive/features/concurrency.md +15 -10
- data/docs/deepdive/features/database.md +19 -19
- data/docs/deepdive/features/embeddings.md +4 -4
- data/docs/deepdive/features/files.md +7 -5
- data/docs/deepdive/fundamentals/agents.md +13 -8
- data/docs/deepdive/fundamentals/providers.md +1 -1
- data/docs/deepdive/fundamentals/schema.md +2 -2
- data/docs/deepdive/fundamentals/stream.md +8 -8
- data/docs/deepdive/fundamentals/tools.md +14 -14
- data/docs/deepdive/media/ocr.md +1 -1
- data/docs/deepdive/protocols/a2a.md +3 -3
- data/docs/deepdive/protocols/mcp.md +116 -7
- data/docs/deepdive/reference/cost.md +1 -1
- data/docs/deepdive/reference/response.md +1 -1
- data/docs/deepdive/reference/tracer.md +90 -22
- data/docs/deepdive.md +1 -1
- data/lib/llm/agent/interrupt.rb +35 -46
- data/lib/llm/agent/registry.rb +106 -0
- data/lib/llm/agent.rb +242 -268
- data/lib/llm/context.rb +63 -55
- data/lib/llm/function/array.rb +2 -1
- data/lib/llm/function/async/group.rb +2 -2
- data/lib/llm/function/async/task.rb +25 -48
- data/lib/llm/function/fiber/group.rb +2 -2
- data/lib/llm/function/fiber/task.rb +16 -30
- data/lib/llm/function/fork/group.rb +2 -2
- data/lib/llm/function/fork/job.rb +8 -4
- data/lib/llm/function/fork/task.rb +25 -30
- data/lib/llm/function/group.rb +33 -5
- data/lib/llm/function/ractor/group.rb +2 -2
- data/lib/llm/function/ractor/job.rb +8 -6
- data/lib/llm/function/ractor/task.rb +32 -9
- data/lib/llm/function/sequential/group.rb +2 -1
- data/lib/llm/function/sequential/task.rb +8 -1
- data/lib/llm/function/task.rb +52 -4
- data/lib/llm/function/thread/group.rb +2 -2
- data/lib/llm/function/thread/task.rb +36 -10
- data/lib/llm/function/window.rb +16 -0
- data/lib/llm/function.rb +14 -43
- data/lib/llm/intercept/multi.rb +40 -0
- data/lib/llm/intercept/null.rb +18 -0
- data/lib/llm/intercept.rb +96 -0
- data/lib/llm/message.rb +1 -1
- data/lib/llm/provider.rb +16 -1
- data/lib/llm/providers/anthropic/stream_parser.rb +1 -1
- data/lib/llm/providers/bedrock/stream_parser.rb +1 -1
- data/lib/llm/providers/deepseek/files.rb +49 -0
- data/lib/llm/providers/deepseek/request_adapter/completion.rb +15 -13
- data/lib/llm/providers/deepseek.rb +16 -2
- data/lib/llm/providers/google/stream_parser.rb +1 -1
- data/lib/llm/providers/openai/responses/stream_parser.rb +1 -1
- data/lib/llm/providers/openai/stream_parser.rb +1 -1
- data/lib/llm/stream.rb +8 -7
- data/lib/llm/tool.rb +1 -1
- data/lib/llm/tools/utils.rb +1 -1
- data/lib/llm/tracer/logger.rb +42 -1
- data/lib/llm/tracer/multi.rb +144 -0
- data/lib/llm/tracer/null.rb +20 -0
- data/lib/llm/tracer/pretty_logger.rb +33 -2
- data/lib/llm/tracer/rescue.rb +110 -0
- data/lib/llm/tracer/telemetry.rb +69 -4
- data/lib/llm/tracer.rb +125 -48
- data/lib/llm/utils.rb +12 -2
- data/lib/llm/version.rb +1 -1
- data/lib/llm.rb +36 -1
- data/llm.gemspec +9 -4
- metadata +16 -35
- data/lib/llm/function/tracing.rb +0 -21
- data/lib/llm/guard/null.rb +0 -19
- data/lib/llm/guard.rb +0 -55
|
@@ -201,7 +201,7 @@ from stalling the conversation.
|
|
|
201
201
|
|
|
202
202
|
[`LLM::Tool::Exec`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Exec.html)
|
|
203
203
|
can be dangerous given a low-quality model. Gate it behind
|
|
204
|
-
[`LLM::Agent#confirm`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#confirm)
|
|
204
|
+
[`LLM::Agent#confirm`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#confirm-instance_method)
|
|
205
205
|
or manage the tool loop manually through
|
|
206
206
|
[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html).
|
|
207
207
|
On interrupt, the running child process is killed. The `ruby` tool
|
|
@@ -8,23 +8,23 @@
|
|
|
8
8
|
llm.rb supports six concurrency strategies for tool execution:
|
|
9
9
|
`:sequential`, `:thread`, `:fiber`, `:async`, `:fork`, and
|
|
10
10
|
`:ractor`. Each implements the same interface
|
|
11
|
-
([`LLM::Function::Task#spawn`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#spawn),
|
|
12
|
-
[`LLM::Function::Task#wait`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#wait),
|
|
13
|
-
[`LLM::Function::Task#alive?`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#alive
|
|
14
|
-
[`LLM::Function::Task#interrupt!`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#interrupt
|
|
11
|
+
([`LLM::Function::Task#spawn`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#spawn-instance_method),
|
|
12
|
+
[`LLM::Function::Task#wait`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#wait-instance_method),
|
|
13
|
+
[`LLM::Function::Task#alive?`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#alive?-instance_method),
|
|
14
|
+
[`LLM::Function::Task#interrupt!`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#interrupt!-instance_method)),
|
|
15
15
|
so the caller never has to care which
|
|
16
16
|
strategy is behind a given task.
|
|
17
17
|
|
|
18
18
|
#### How it works
|
|
19
19
|
|
|
20
20
|
Each strategy creates a task object that wraps a function call.
|
|
21
|
-
[`LLM::Function::Task#spawn`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#spawn)
|
|
21
|
+
[`LLM::Function::Task#spawn`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#spawn-instance_method)
|
|
22
22
|
starts execution,
|
|
23
|
-
[`LLM::Function::Task#wait`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#wait)
|
|
23
|
+
[`LLM::Function::Task#wait`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#wait-instance_method)
|
|
24
24
|
collects the result,
|
|
25
|
-
[`LLM::Function::Task#alive?`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#alive
|
|
25
|
+
[`LLM::Function::Task#alive?`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#alive?-instance_method)
|
|
26
26
|
checks whether the task is still running, and
|
|
27
|
-
[`LLM::Function::Task#interrupt!`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#interrupt
|
|
27
|
+
[`LLM::Function::Task#interrupt!`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#interrupt!-instance_method)
|
|
28
28
|
signals the task to stop.
|
|
29
29
|
|
|
30
30
|
Interruption is reliable across all six.
|
|
@@ -83,8 +83,13 @@ during blocking IO. Best for IO-bound tools like HTTP calls and
|
|
|
83
83
|
database queries.
|
|
84
84
|
|
|
85
85
|
**fiber**: Each tool runs in a scheduler-backed Fiber. Requires
|
|
86
|
-
`Fiber.scheduler
|
|
87
|
-
|
|
86
|
+
`Fiber.scheduler`, and that scheduler has to implement
|
|
87
|
+
`fiber_interrupt`, which is how a cancel is asked of a fiber rather than
|
|
88
|
+
raised into it. A scheduler without it is refused with
|
|
89
|
+
[`LLM::FiberError`](https://r.uby.dev/api-docs/llm.rb/LLM/FiberError.html)
|
|
90
|
+
before anything is scheduled, since a cancel that cannot be asked for is
|
|
91
|
+
a cancel that cannot be delivered. Best for IO-bound tools inside an
|
|
92
|
+
async framework. Much lighter than threads.
|
|
88
93
|
|
|
89
94
|
**async**: Each tool runs as an `Async::Task` inside a managed
|
|
90
95
|
background reactor. Best for IO-bound tools when you want Async's
|
|
@@ -28,11 +28,11 @@ All four use the same serialization mechanism under the hood.
|
|
|
28
28
|
|
|
29
29
|
[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
|
|
30
30
|
implements
|
|
31
|
-
[`LLM::Context#to_h`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#to_h)
|
|
31
|
+
[`LLM::Context#to_h`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#to_h-instance_method)
|
|
32
32
|
and
|
|
33
|
-
[`LLM::Context#to_json`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#to_json)
|
|
33
|
+
[`LLM::Context#to_json`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#to_json-instance_method)
|
|
34
34
|
for serialization and
|
|
35
|
-
[`LLM::Context#restore`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#restore)
|
|
35
|
+
[`LLM::Context#restore`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#restore-instance_method)
|
|
36
36
|
for deserialization. Save writes the current state, restore loads
|
|
37
37
|
it back and picks up where the conversation left off.
|
|
38
38
|
|
|
@@ -70,16 +70,16 @@ underlying serialization as filesystem persistence.
|
|
|
70
70
|
|
|
71
71
|
#### Overview
|
|
72
72
|
|
|
73
|
-
The [`LLM::Agent#path`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#path)
|
|
73
|
+
The [`LLM::Agent#path`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#path-instance_method)
|
|
74
74
|
attribute provides transparent auto-persistence. Set a file path once
|
|
75
75
|
and the agent restores conversation history from that file on startup
|
|
76
76
|
and saves it back after every
|
|
77
|
-
[`LLM::Agent#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#talk)
|
|
77
|
+
[`LLM::Agent#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#talk-instance_method)
|
|
78
78
|
or
|
|
79
|
-
[`LLM::Agent#ask`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#ask)
|
|
79
|
+
[`LLM::Agent#ask`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#ask-instance_method)
|
|
80
80
|
turn. No manual
|
|
81
|
-
[`LLM::Agent#save`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#save)/
|
|
82
|
-
[`LLM::Agent#restore`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#restore)
|
|
81
|
+
[`LLM::Agent#save`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#save-instance_method)/
|
|
82
|
+
[`LLM::Agent#restore`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#restore-instance_method)
|
|
83
83
|
calls needed.
|
|
84
84
|
|
|
85
85
|
The feature is available on subclasses via the class DSL and on
|
|
@@ -89,9 +89,9 @@ direct instances via the `path:` keyword argument.
|
|
|
89
89
|
|
|
90
90
|
When a `path` is set, the agent loads existing state from the file
|
|
91
91
|
during initialization. After each turn
|
|
92
|
-
([`LLM::Agent#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#talk)
|
|
92
|
+
([`LLM::Agent#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#talk-instance_method)
|
|
93
93
|
or
|
|
94
|
-
[`LLM::Agent#ask`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#ask)),
|
|
94
|
+
[`LLM::Agent#ask`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#ask-instance_method)),
|
|
95
95
|
the updated
|
|
96
96
|
state is written back automatically. If the file does not exist yet,
|
|
97
97
|
the agent starts with a blank conversation and creates the file on
|
|
@@ -140,9 +140,9 @@ Auto-persistence eliminates boilerplate. You set the path once and
|
|
|
140
140
|
forget about serialization entirely. The agent picks up where it
|
|
141
141
|
left off across process restarts, console sessions, or debugging runs
|
|
142
142
|
without a single
|
|
143
|
-
[`LLM::Agent#save`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#save)
|
|
143
|
+
[`LLM::Agent#save`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#save-instance_method)
|
|
144
144
|
or
|
|
145
|
-
[`LLM::Agent#restore`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#restore)
|
|
145
|
+
[`LLM::Agent#restore`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#restore-instance_method)
|
|
146
146
|
call in your code.
|
|
147
147
|
|
|
148
148
|
#### Notes
|
|
@@ -150,9 +150,9 @@ call in your code.
|
|
|
150
150
|
The auto-path feature is a strict superset of manual filesystem
|
|
151
151
|
persistence. If you need fine-grained control over when state is
|
|
152
152
|
saved (e.g. batch several turns before persisting), use the manual
|
|
153
|
-
[`LLM::Agent#save`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#save)
|
|
153
|
+
[`LLM::Agent#save`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#save-instance_method)
|
|
154
154
|
and
|
|
155
|
-
[`LLM::Agent#restore`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#restore)
|
|
155
|
+
[`LLM::Agent#restore`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#restore-instance_method)
|
|
156
156
|
methods described in the Filesystem section below. The `path`
|
|
157
157
|
attribute delegates to the same underlying serialization.
|
|
158
158
|
|
|
@@ -170,12 +170,12 @@ process or on a different machine, and pick up where you left off.
|
|
|
170
170
|
|
|
171
171
|
[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
|
|
172
172
|
implements
|
|
173
|
-
[`LLM::Context#to_h`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#to_h)
|
|
173
|
+
[`LLM::Context#to_h`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#to_h-instance_method)
|
|
174
174
|
and
|
|
175
|
-
[`LLM::Context#to_json`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#to_json)
|
|
175
|
+
[`LLM::Context#to_json`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#to_json-instance_method)
|
|
176
176
|
for serialization
|
|
177
177
|
and
|
|
178
|
-
[`LLM::Context#restore`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#restore)
|
|
178
|
+
[`LLM::Context#restore`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#restore-instance_method)
|
|
179
179
|
for deserialization. The serialized state includes the context id,
|
|
180
180
|
the message history, the model name, the compaction status, and a
|
|
181
181
|
snapshot of token usage. Save and
|
|
@@ -358,7 +358,7 @@ parameters. Those two are the only callbacks `acts_as_llm` and
|
|
|
358
358
|
|
|
359
359
|
An agent built by `acts_as_agent` is bound to the record it was
|
|
360
360
|
loaded from, and
|
|
361
|
-
[`LLM::Agent#record`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#record)
|
|
361
|
+
[`LLM::Agent#record`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#record-instance_method)
|
|
362
362
|
returns it. To persist a context instead of an agent, use
|
|
363
363
|
`acts_as_llm`: the model gains `#llm` (the provider) and `#ctx` (the
|
|
364
364
|
context), and persists the same state without the automatic tool
|
|
@@ -547,6 +547,6 @@ To persist a context instead of an agent, use `plugin :llm`. The
|
|
|
547
547
|
model gains `#llm` (the provider) and `#ctx` (the context), and
|
|
548
548
|
persists the same state without the automatic tool loop. An agent
|
|
549
549
|
built by `plugin :agent` is bound to its record, which
|
|
550
|
-
[`LLM::Agent#record`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#record)
|
|
550
|
+
[`LLM::Agent#record`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#record-instance_method)
|
|
551
551
|
returns.
|
|
552
552
|
|
|
@@ -9,7 +9,7 @@ Embeddings convert text into numerical vectors that capture semantic
|
|
|
9
9
|
meaning. Similar texts produce vectors that are close together in the
|
|
10
10
|
vector space, enabling semantic search, clustering, and classification.
|
|
11
11
|
Most providers offer embedding models through an
|
|
12
|
-
[`LLM::Provider#embed`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#embed)
|
|
12
|
+
[`LLM::Provider#embed`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#embed-instance_method)
|
|
13
13
|
method.
|
|
14
14
|
|
|
15
15
|
llm.rb also includes support for OpenAI's vector store API, which
|
|
@@ -19,7 +19,7 @@ provides a managed vector database as an HTTP service.
|
|
|
19
19
|
|
|
20
20
|
When you want to generate an embedding vector for a piece of text,
|
|
21
21
|
call
|
|
22
|
-
[`LLM::Provider#embed`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#embed)
|
|
22
|
+
[`LLM::Provider#embed`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#embed-instance_method)
|
|
23
23
|
on any provider that supports it. The method accepts an input string
|
|
24
24
|
or array of strings and returns a response with the resulting
|
|
25
25
|
embeddings, which can be stored in a vector database and used for
|
|
@@ -113,7 +113,7 @@ The trade-off is vendor lock-in and per-request pricing.
|
|
|
113
113
|
Vector stores require the `openai` provider. Other providers such as
|
|
114
114
|
DeepInfra, Google, and Mistral expose an embedding method but no
|
|
115
115
|
vector store API. For a self-hosted vector database, pair
|
|
116
|
-
[`LLM::Provider#embed`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#embed)
|
|
116
|
+
[`LLM::Provider#embed`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#embed-instance_method)
|
|
117
117
|
with sqlite-vec or pgvector.
|
|
118
118
|
|
|
119
119
|
### Local vector search
|
|
@@ -121,7 +121,7 @@ with sqlite-vec or pgvector.
|
|
|
121
121
|
#### Overview
|
|
122
122
|
|
|
123
123
|
Local vector search combines
|
|
124
|
-
[`LLM::Provider#embed`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#embed)
|
|
124
|
+
[`LLM::Provider#embed`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#embed-instance_method)
|
|
125
125
|
with a local vector database such as
|
|
126
126
|
[sqlite-vec](https://github.com/asg017/sqlite-vec) or
|
|
127
127
|
[pgvector](https://github.com/pgvector/pgvector). This approach keeps
|
|
@@ -47,8 +47,10 @@ assistants that read them, get their content.
|
|
|
47
47
|
|
|
48
48
|
Anthropic, Google, and OpenAI implement the Files API, and so do the
|
|
49
49
|
providers built on OpenAI's - Moonshot and Alibaba among them, which
|
|
50
|
-
inherit it.
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
50
|
+
inherit it. DeepSeek implements the Files API too, scoped to images that
|
|
51
|
+
a chat request references by `file_id`. A provider that has no Files API
|
|
52
|
+
of its own, such as Mistral or xAI, raises `NotImplementedError`.
|
|
53
|
+
`download` is OpenAI-only, so the OpenAI-compatible providers inherit
|
|
54
|
+
that too. A file id belongs to the provider that minted it: a file id minted by
|
|
55
|
+
another provider is passed through as-is, and fails when DeepSeek's API
|
|
56
|
+
tries to resolve it.
|
|
@@ -16,7 +16,7 @@ serialization, compaction, and concurrency.
|
|
|
16
16
|
#### How it works
|
|
17
17
|
|
|
18
18
|
An agent holds a conversation with a model. You send input with
|
|
19
|
-
[`LLM::Agent#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#talk),
|
|
19
|
+
[`LLM::Agent#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#talk-instance_method),
|
|
20
20
|
the model responds, and if it requests tools the agent
|
|
21
21
|
executes them automatically and feeds the results back. The tool
|
|
22
22
|
loop can be bounded with
|
|
@@ -30,7 +30,7 @@ prompt already carries a system message of the caller's own.
|
|
|
30
30
|
Agents manage the tool loop for you. They keep conversation state
|
|
31
31
|
across turns and let you define reusable configurations at the
|
|
32
32
|
class level. The loop runs until the model stops requesting tools
|
|
33
|
-
unless you bound it with `tool_budget` or
|
|
33
|
+
unless you bound it with `tool_budget` or an intercept. If you need
|
|
34
34
|
manual control over the tool loop, use
|
|
35
35
|
[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
|
|
36
36
|
directly instead.
|
|
@@ -100,8 +100,8 @@ Set `path:` on a subclass or instance for automatic filesystem
|
|
|
100
100
|
persistence; the agent restores conversation history from the
|
|
101
101
|
file on startup and saves it back after every turn with no
|
|
102
102
|
manual
|
|
103
|
-
[`LLM::Agent#save`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#save)/
|
|
104
|
-
[`LLM::Agent#restore`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#restore)
|
|
103
|
+
[`LLM::Agent#save`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#save-instance_method)/
|
|
104
|
+
[`LLM::Agent#restore`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#restore-instance_method)
|
|
105
105
|
calls. See the
|
|
106
106
|
[database deepdive](../features/database.md) for details.
|
|
107
107
|
|
|
@@ -244,10 +244,15 @@ of hanging.
|
|
|
244
244
|
|
|
245
245
|
#### Notes
|
|
246
246
|
|
|
247
|
-
The retry budget applies to
|
|
248
|
-
|
|
249
|
-
|
|
250
|
-
|
|
247
|
+
The retry budget applies to the errors the provider lists in
|
|
248
|
+
[`LLM::Provider#retry_errors`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#retry_errors-instance_method):
|
|
249
|
+
rate limits (`LLM::RateLimitError`), an exhausted quota, and timeouts
|
|
250
|
+
(`Net::OpenTimeout`, `Net::ReadTimeout`, `Net::WriteTimeout`). A
|
|
251
|
+
provider extends that list rather than replacing it, so
|
|
252
|
+
[`LLM::DeepSeek`](https://r.uby.dev/api-docs/llm.rb/LLM/DeepSeek.html)
|
|
253
|
+
also retries `LLM::ServerError`, the error it returns when its servers
|
|
254
|
+
are overloaded; anything not on the list is never retried. The budget
|
|
255
|
+
lives on the provider, where it defaults to five
|
|
251
256
|
(eight on Alibaba, which rate limits more often), and an agent that
|
|
252
257
|
sets none of its own takes the provider's. A raw
|
|
253
258
|
[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
|
|
@@ -37,7 +37,7 @@ as `LLM.aliyun`.
|
|
|
37
37
|
Providers are the first building block of every llm.rb program.
|
|
38
38
|
Because the whole runtime shares one provider interface, you can
|
|
39
39
|
swap one provider for another with a single line change, and the
|
|
40
|
-
same tools, schemas, streaming, and
|
|
40
|
+
same tools, schemas, streaming, and intercepts work unchanged.
|
|
41
41
|
|
|
42
42
|
#### Notes
|
|
43
43
|
|
|
@@ -16,9 +16,9 @@ When you want a structured response from the model, subclass
|
|
|
16
16
|
[`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html)
|
|
17
17
|
and declare properties with types and constraints. When you pass
|
|
18
18
|
the schema to
|
|
19
|
-
[`LLM::Agent#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#talk)
|
|
19
|
+
[`LLM::Agent#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#talk-instance_method)
|
|
20
20
|
or
|
|
21
|
-
[`LLM::Context#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#talk),
|
|
21
|
+
[`LLM::Context#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#talk-instance_method),
|
|
22
22
|
the runtime includes it in the request parameters. The provider returns a
|
|
23
23
|
JSON object matching the schema, which is coerced into an
|
|
24
24
|
[`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html) with typed accessors.
|
|
@@ -37,7 +37,7 @@ single blocking response.
|
|
|
37
37
|
#### Notes
|
|
38
38
|
|
|
39
39
|
The IO-like form is equivalent to
|
|
40
|
-
[`LLM::Stream#on_content`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_content)
|
|
40
|
+
[`LLM::Stream#on_content`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_content-instance_method)
|
|
41
41
|
and does not include the other hooks. It covers content output
|
|
42
42
|
(piping to stdout or a log file) without the overhead of a full
|
|
43
43
|
subclass. A
|
|
@@ -61,11 +61,11 @@ When you want to react to specific runtime events, override the
|
|
|
61
61
|
hooks on a
|
|
62
62
|
[`LLM::Stream`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html)
|
|
63
63
|
subclass.
|
|
64
|
-
[`LLM::Stream#on_content`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_content)
|
|
64
|
+
[`LLM::Stream#on_content`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_content-instance_method)
|
|
65
65
|
receives tokens as they arrive.
|
|
66
|
-
[`LLM::Stream#on_tool_call`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_tool_call)
|
|
66
|
+
[`LLM::Stream#on_tool_call`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_tool_call-instance_method)
|
|
67
67
|
fires when the model requests a tool.
|
|
68
|
-
[`LLM::Stream#on_tool_return`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_tool_return)
|
|
68
|
+
[`LLM::Stream#on_tool_return`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_tool_return-instance_method)
|
|
69
69
|
fires when the tool completes.
|
|
70
70
|
[`LLM::Stream#on_step`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_step-instance_method)
|
|
71
71
|
fires when a request completes, which is the boundary between one
|
|
@@ -78,13 +78,13 @@ written against it - it is prepended onto a stream, and saves the
|
|
|
78
78
|
conversation through the record the context is bound to, so a turn
|
|
79
79
|
that is interrupted can be continued from its last completed request
|
|
80
80
|
rather than started over.
|
|
81
|
-
[`LLM::Stream#on_retry`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_retry)
|
|
81
|
+
[`LLM::Stream#on_retry`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_retry-instance_method)
|
|
82
82
|
fires each time a failed request is retried. Compaction hooks
|
|
83
83
|
let you show progress or log what was trimmed. Skill hooks bracket a
|
|
84
84
|
skill's subagent execution:
|
|
85
|
-
[`LLM::Stream#on_skill_call`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_skill_call)
|
|
85
|
+
[`LLM::Stream#on_skill_call`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_skill_call-instance_method)
|
|
86
86
|
fires before a skill's subagent runs, and
|
|
87
|
-
[`LLM::Stream#on_skill_return`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_skill_return)
|
|
87
|
+
[`LLM::Stream#on_skill_return`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_skill_return-instance_method)
|
|
88
88
|
fires after it finishes.
|
|
89
89
|
|
|
90
90
|
```ruby
|
|
@@ -164,7 +164,7 @@ instead, and hand the result to the turn when it is asked for.
|
|
|
164
164
|
#### How it works
|
|
165
165
|
|
|
166
166
|
Start the tool from
|
|
167
|
-
[`LLM::Stream#on_tool_call`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_tool_call)
|
|
167
|
+
[`LLM::Stream#on_tool_call`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_tool_call-instance_method)
|
|
168
168
|
and enqueue its task on
|
|
169
169
|
[`LLM::Stream#queue`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#queue-instance_method).
|
|
170
170
|
[`LLM::Context#wait`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#wait-instance_method)
|
|
@@ -12,10 +12,10 @@ when a tool fits the request; the runtime calls it and sends the
|
|
|
12
12
|
result back.
|
|
13
13
|
|
|
14
14
|
A tool is a Ruby class with a name, a description, and a
|
|
15
|
-
[`LLM::Tool#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#call)
|
|
15
|
+
[`LLM::Tool#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#call-instance_method)
|
|
16
16
|
method. The name tells the model what the tool is called. The
|
|
17
17
|
description tells the model when to use it. The
|
|
18
|
-
[`LLM::Tool#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#call)
|
|
18
|
+
[`LLM::Tool#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#call-instance_method)
|
|
19
19
|
method receives the arguments and returns the result. That is the entire
|
|
20
20
|
contract: name, description, parameters, and a method that runs.
|
|
21
21
|
|
|
@@ -26,7 +26,7 @@ A tool is a subclass of
|
|
|
26
26
|
with a name, description, and optional typed parameters. The model sees the name and
|
|
27
27
|
description and decides whether to call it. When it does, the
|
|
28
28
|
runtime serializes the arguments and passes them to
|
|
29
|
-
[`LLM::Tool#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#call).
|
|
29
|
+
[`LLM::Tool#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#call-instance_method).
|
|
30
30
|
|
|
31
31
|
Tools that spawn subprocesses can include
|
|
32
32
|
[`LLM::Tool::Utils`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Utils.html)
|
|
@@ -36,11 +36,11 @@ and `now` help. The built-in `Exec`, `Git`, `Rg`, `Mkdir`, and `Ruby`
|
|
|
36
36
|
tools use it to kill a command that exceeds its `timeout`.
|
|
37
37
|
|
|
38
38
|
If
|
|
39
|
-
[`LLM::Tool#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#call)
|
|
39
|
+
[`LLM::Tool#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#call-instance_method)
|
|
40
40
|
raises, the runtime rescues it and returns a structured
|
|
41
41
|
error to the model instead. The conversation stays valid. You can
|
|
42
42
|
also handle errors yourself inside
|
|
43
|
-
[`LLM::Tool#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#call)
|
|
43
|
+
[`LLM::Tool#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#call-instance_method)
|
|
44
44
|
by rescuing and returning a domain-specific error hash.
|
|
45
45
|
|
|
46
46
|
```ruby
|
|
@@ -96,9 +96,9 @@ API call at once, then return a synthesized answer. See
|
|
|
96
96
|
|
|
97
97
|
Confirmation gates tools behind explicit approval. List tool names
|
|
98
98
|
in
|
|
99
|
-
[`LLM::Agent#confirm`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#confirm)
|
|
99
|
+
[`LLM::Agent#confirm`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#confirm-instance_method)
|
|
100
100
|
to block execution until you override
|
|
101
|
-
[`LLM::Agent#on_tool_confirmation`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#on_tool_confirmation).
|
|
101
|
+
[`LLM::Agent#on_tool_confirmation`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#on_tool_confirmation-instance_method).
|
|
102
102
|
Confirmation also accepts a Symbol that resolves to an instance
|
|
103
103
|
method, letting the confirmed set change per-instance based on
|
|
104
104
|
runtime conditions.
|
|
@@ -122,10 +122,10 @@ keywords in full.
|
|
|
122
122
|
|
|
123
123
|
Tools that perform destructive actions can be gated behind explicit
|
|
124
124
|
approval. List their names in
|
|
125
|
-
[`LLM::Agent#confirm`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#confirm)
|
|
125
|
+
[`LLM::Agent#confirm`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#confirm-instance_method)
|
|
126
126
|
to block execution
|
|
127
127
|
until you override
|
|
128
|
-
[`LLM::Agent#on_tool_confirmation`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#on_tool_confirmation).
|
|
128
|
+
[`LLM::Agent#on_tool_confirmation`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#on_tool_confirmation-instance_method).
|
|
129
129
|
The default handler cancels
|
|
130
130
|
the tool. Override it per-agent to prompt the user, log the decision,
|
|
131
131
|
or auto-approve certain tools.
|
|
@@ -134,12 +134,12 @@ or auto-approve certain tools.
|
|
|
134
134
|
|
|
135
135
|
When you want to override the default approval flow for a gated
|
|
136
136
|
tool, override
|
|
137
|
-
[`LLM::Agent#on_tool_confirmation`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#on_tool_confirmation)
|
|
137
|
+
[`LLM::Agent#on_tool_confirmation`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#on_tool_confirmation-instance_method)
|
|
138
138
|
on the subclass. The method receives the pending function and the
|
|
139
139
|
execution strategy. Call
|
|
140
|
-
[`LLM::Function#task`](https://r.uby.dev/api-docs/llm.rb/LLM/Function.html#task)
|
|
140
|
+
[`LLM::Function#task`](https://r.uby.dev/api-docs/llm.rb/LLM/Function.html#task-instance_method)
|
|
141
141
|
to execute the tool or
|
|
142
|
-
[`LLM::Function#cancel`](https://r.uby.dev/api-docs/llm.rb/LLM/Function.html#cancel)
|
|
142
|
+
[`LLM::Function#cancel`](https://r.uby.dev/api-docs/llm.rb/LLM/Function.html#cancel-instance_method)
|
|
143
143
|
to block it. The default handler cancels the call.
|
|
144
144
|
|
|
145
145
|
```ruby
|
|
@@ -178,7 +178,7 @@ matter what.
|
|
|
178
178
|
#### How it works
|
|
179
179
|
|
|
180
180
|
If
|
|
181
|
-
[`LLM::Tool#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#call)
|
|
181
|
+
[`LLM::Tool#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#call-instance_method)
|
|
182
182
|
raises, the runtime returns `{error: true, type: "RuntimeError",
|
|
183
183
|
message: "boom"}` to the model. A tool can also check for a known
|
|
184
184
|
failure and return its own error shape, which gives the model
|
|
@@ -445,7 +445,7 @@ gem for process management and interrupt handling.
|
|
|
445
445
|
|
|
446
446
|
`LLM::Tool.subclasses` lists the direct subclasses of
|
|
447
447
|
[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html), so a
|
|
448
|
-
tool defined under an intermediate
|
|
448
|
+
tool defined under an intermediate superclass is not in the list.
|
|
449
449
|
[`LLM::Tool.registry`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#registry-class_method)
|
|
450
450
|
lists every tool that has a name, wherever it sits in the hierarchy.
|
|
451
451
|
|
data/docs/deepdive/media/ocr.md
CHANGED
|
@@ -13,7 +13,7 @@ document URLs.
|
|
|
13
13
|
#### How it works
|
|
14
14
|
|
|
15
15
|
Mistral is the only provider with a dedicated OCR endpoint. The
|
|
16
|
-
[`LLM::Provider#ocr`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#ocr)
|
|
16
|
+
[`LLM::Provider#ocr`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#ocr-instance_method)
|
|
17
17
|
method accepts an `image_url:` or `document_url:` parameter.
|
|
18
18
|
Document URLs can point to PDFs.
|
|
19
19
|
|
|
@@ -38,7 +38,7 @@ the network.
|
|
|
38
38
|
#### Notes
|
|
39
39
|
|
|
40
40
|
An agent's capabilities are advertised through a card. The
|
|
41
|
-
[`LLM::A2A::Card#interfaces`](https://r.uby.dev/api-docs/llm.rb/LLM/A2A/Card.html#interfaces)
|
|
41
|
+
[`LLM::A2A::Card#interfaces`](https://r.uby.dev/api-docs/llm.rb/LLM/A2A/Card.html#interfaces-instance_method)
|
|
42
42
|
method lists which transports a remote agent supports.
|
|
43
43
|
|
|
44
44
|
##### Persistent connections
|
|
@@ -69,7 +69,7 @@ JSON-RPC is an alternative transport for A2A agents. It uses a
|
|
|
69
69
|
more structured protocol than REST, with request and response
|
|
70
70
|
objects that follow the JSON-RPC 2.0 spec. Some agents advertise
|
|
71
71
|
only JSON-RPC, others advertise both. The
|
|
72
|
-
[`LLM::A2A::Card#interfaces`](https://r.uby.dev/api-docs/llm.rb/LLM/A2A/Card.html#interfaces)
|
|
72
|
+
[`LLM::A2A::Card#interfaces`](https://r.uby.dev/api-docs/llm.rb/LLM/A2A/Card.html#interfaces-instance_method)
|
|
73
73
|
method lists which transports a remote agent supports.
|
|
74
74
|
|
|
75
75
|
#### How it works
|
|
@@ -93,7 +93,7 @@ agent.talk "What's happening, fellow agent?"
|
|
|
93
93
|
JSON-RPC provides typed request and response objects that follow
|
|
94
94
|
the 2.0 spec, offering a more structured protocol than REST. Some
|
|
95
95
|
agents advertise only JSON-RPC, while others advertise both. The
|
|
96
|
-
[`LLM::A2A::Card#interfaces`](https://r.uby.dev/api-docs/llm.rb/LLM/A2A/Card.html#interfaces)
|
|
96
|
+
[`LLM::A2A::Card#interfaces`](https://r.uby.dev/api-docs/llm.rb/LLM/A2A/Card.html#interfaces-instance_method)
|
|
97
97
|
method lists which transports a remote agent supports.
|
|
98
98
|
|
|
99
99
|
#### Notes
|
|
@@ -17,7 +17,7 @@ subclasses the model can call.
|
|
|
17
17
|
|
|
18
18
|
The stdio transport runs the server as a child process and
|
|
19
19
|
communicates over stdin/stdout. Use
|
|
20
|
-
[`LLM::MCP#session`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html#session)
|
|
20
|
+
[`LLM::MCP#session`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html#session-instance_method)
|
|
21
21
|
to avoid launching the same process multiple times.
|
|
22
22
|
|
|
23
23
|
```ruby
|
|
@@ -41,11 +41,62 @@ one. Remove them without touching the agent's code.
|
|
|
41
41
|
#### Notes
|
|
42
42
|
|
|
43
43
|
For stdio, use
|
|
44
|
-
[`LLM::MCP#session`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html#session)
|
|
44
|
+
[`LLM::MCP#session`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html#session-instance_method)
|
|
45
45
|
to avoid launching the same process multiple times. For HTTP,
|
|
46
|
-
[`LLM::MCP#session`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html#session)
|
|
46
|
+
[`LLM::MCP#session`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html#session-instance_method)
|
|
47
47
|
carries little benefit.
|
|
48
48
|
|
|
49
|
+
### Local repository lookup with Sem
|
|
50
|
+
|
|
51
|
+
[Sem](https://github.com/Ataraxy-Labs/sem) provides symbol lookup and
|
|
52
|
+
source search through a local stdio MCP server. Install Sem following
|
|
53
|
+
its upstream instructions and verify `sem --version`. This example
|
|
54
|
+
uses the existing stdio client; no model credentials are needed.
|
|
55
|
+
|
|
56
|
+
Save the following as `sem_code.rb`, then run it with an existing
|
|
57
|
+
Git checkout and a symbol in that checkout:
|
|
58
|
+
|
|
59
|
+
```bash
|
|
60
|
+
SEM_REPO=/absolute/path/to/repo ruby sem_code.rb parse_config
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
```ruby
|
|
64
|
+
require "llm"
|
|
65
|
+
|
|
66
|
+
repo = File.realpath(ENV.fetch("SEM_REPO"))
|
|
67
|
+
query = ARGV.fetch(0)
|
|
68
|
+
mcp = LLM::MCP.stdio(
|
|
69
|
+
argv: [ENV.fetch("SEM_BIN", "sem"), "mcp"],
|
|
70
|
+
env: {"SEM_REPO" => repo},
|
|
71
|
+
cwd: repo
|
|
72
|
+
)
|
|
73
|
+
|
|
74
|
+
mcp.session do
|
|
75
|
+
lookup = mcp.tools.find { |tool| tool.name == "sem_find" }
|
|
76
|
+
raise "Server did not expose sem_find" unless lookup
|
|
77
|
+
|
|
78
|
+
puts LLM.json.dump(lookup.new.call(
|
|
79
|
+
query: query,
|
|
80
|
+
mode: "context",
|
|
81
|
+
token_budget: 1200
|
|
82
|
+
))
|
|
83
|
+
end
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
Use `SEM_BIN=/absolute/path/to/sem` if Sem is not on PATH. Keeping
|
|
87
|
+
discovery and lookup inside `session` reuses the child process. The
|
|
88
|
+
first query may build a repository index; an empty result can mean
|
|
89
|
+
the symbol was not found. Consult the discovered tool schema for
|
|
90
|
+
the installed Sem version.
|
|
91
|
+
|
|
92
|
+
For an agent conversation, pass only the tool classes you intend
|
|
93
|
+
to expose, for example `mcp.tools.select { |t| t.name == "sem_find" }`,
|
|
94
|
+
inside the same session. This filters the model's available tools,
|
|
95
|
+
not the server's permissions. The full Sem server also exposes
|
|
96
|
+
validation tools that can execute commands: use trusted checkouts
|
|
97
|
+
and host permissions. `SEM_REPO` is not a sandbox. The lookup does
|
|
98
|
+
not edit source files, but Sem may write its local index.
|
|
99
|
+
|
|
49
100
|
### HTTP
|
|
50
101
|
|
|
51
102
|
#### Overview
|
|
@@ -90,7 +141,7 @@ by a different team and exposed as a service.
|
|
|
90
141
|
#### Notes
|
|
91
142
|
|
|
92
143
|
For HTTP,
|
|
93
|
-
[`LLM::MCP#session`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html#session)
|
|
144
|
+
[`LLM::MCP#session`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html#session-instance_method)
|
|
94
145
|
carries little benefit.
|
|
95
146
|
|
|
96
147
|
##### Persistent connections
|
|
@@ -109,15 +160,73 @@ mcp = LLM::MCP.http(
|
|
|
109
160
|
)
|
|
110
161
|
```
|
|
111
162
|
|
|
163
|
+
### Parallel web search and fetch
|
|
164
|
+
|
|
165
|
+
[Parallel Search MCP](https://docs.parallel.ai/integrations/mcp/search-mcp)
|
|
166
|
+
provides `web_search` and `web_fetch` over HTTP without a Parallel
|
|
167
|
+
API key. The anonymous tier is free for exploration and light use,
|
|
168
|
+
with rate limits and server-managed search settings.
|
|
169
|
+
|
|
170
|
+
This example discovers the tools through the HTTP client, searches
|
|
171
|
+
for Ruby documentation, then extracts excerpts from a specific page.
|
|
172
|
+
It uses the default `net/http` transport and needs no model or provider
|
|
173
|
+
credentials. Save it as `parallel_search.rb` and run
|
|
174
|
+
`ruby parallel_search.rb` after installing `llm.rb`.
|
|
175
|
+
|
|
176
|
+
```ruby
|
|
177
|
+
require "llm"
|
|
178
|
+
|
|
179
|
+
mcp = LLM::MCP.http(
|
|
180
|
+
url: "https://search.parallel.ai/mcp",
|
|
181
|
+
headers: {
|
|
182
|
+
"User-Agent" => "llm.rb/#{LLM::VERSION}",
|
|
183
|
+
"Accept" => "application/json, text/event-stream"
|
|
184
|
+
}
|
|
185
|
+
)
|
|
186
|
+
|
|
187
|
+
mcp.session do
|
|
188
|
+
tools = mcp.tools
|
|
189
|
+
search = tools.find { _1.name == "web_search" }.new
|
|
190
|
+
fetch = tools.find { _1.name == "web_fetch" }.new
|
|
191
|
+
session_id = SecureRandom.uuid
|
|
192
|
+
queries = ["Ruby Fiber scheduler documentation"]
|
|
193
|
+
|
|
194
|
+
puts LLM.json.dump(search.call(
|
|
195
|
+
objective: "Find the official Ruby Fiber scheduler documentation.",
|
|
196
|
+
search_queries: queries,
|
|
197
|
+
session_id:
|
|
198
|
+
))
|
|
199
|
+
|
|
200
|
+
puts LLM.json.dump(fetch.call(
|
|
201
|
+
urls: ["https://docs.ruby-lang.org/en/master/Fiber.html"],
|
|
202
|
+
objective: "Explain how Ruby uses a Fiber scheduler.",
|
|
203
|
+
search_queries: queries,
|
|
204
|
+
session_id:
|
|
205
|
+
))
|
|
206
|
+
end
|
|
207
|
+
```
|
|
208
|
+
|
|
209
|
+
To let an agent choose when to search or fetch, pass the discovered
|
|
210
|
+
tools to `talk` instead. Model inference is separate from the free
|
|
211
|
+
MCP service and uses your chosen provider's credentials and pricing.
|
|
212
|
+
Using the same `mcp` configuration above:
|
|
213
|
+
|
|
214
|
+
```ruby
|
|
215
|
+
llm = LLM.deepseek(key: ENV.fetch("DEEPSEEK_API_KEY"))
|
|
216
|
+
agent = LLM::Agent.new(llm)
|
|
217
|
+
agent.talk "Find the official Ruby Fiber scheduler docs and explain them.",
|
|
218
|
+
tools: mcp.tools
|
|
219
|
+
```
|
|
220
|
+
|
|
112
221
|
### Prompts
|
|
113
222
|
|
|
114
223
|
#### Overview
|
|
115
224
|
|
|
116
225
|
An MCP server can offer prompts as well as tools: reusable,
|
|
117
226
|
parameterized message templates a client fetches and sends to a model.
|
|
118
|
-
[`LLM::MCP#prompts`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html#prompts)
|
|
227
|
+
[`LLM::MCP#prompts`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html#prompts-instance_method)
|
|
119
228
|
lists them, and
|
|
120
|
-
[`LLM::MCP#find_prompt`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html#find_prompt)
|
|
229
|
+
[`LLM::MCP#find_prompt`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html#find_prompt-instance_method)
|
|
121
230
|
fetches one and returns its messages.
|
|
122
231
|
|
|
123
232
|
#### How it works
|
|
@@ -155,7 +264,7 @@ it, so it can change without a client release.
|
|
|
155
264
|
`find_prompt` adapts each message's content to the runtime's shape: a
|
|
156
265
|
text item becomes a string, and anything else stays an `LLM::Object`
|
|
157
266
|
under `original_content`.
|
|
158
|
-
[`LLM::MCP#get_prompt`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html#get_prompt)
|
|
267
|
+
[`LLM::MCP#get_prompt`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html#get_prompt-instance_method)
|
|
159
268
|
is an alias. Like `tools`, both methods borrow a session when one is
|
|
160
269
|
not already running.
|
|
161
270
|
|