llm.rb 11.2.0 → 11.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +85 -11
- data/README.md +137 -144
- data/lib/llm/a2a.rb +2 -2
- data/lib/llm/active_record/acts_as_agent.rb +7 -0
- data/lib/llm/active_record/acts_as_llm.rb +7 -0
- data/lib/llm/agent.rb +26 -5
- data/lib/llm/cost.rb +1 -1
- data/lib/llm/error.rb +4 -0
- data/lib/llm/function/fiber_group.rb +2 -2
- data/lib/llm/function/task_group.rb +2 -2
- data/lib/llm/function/thread_group.rb +3 -3
- data/lib/llm/pipe.rb +1 -1
- data/lib/llm/providers/anthropic/error_handler.rb +2 -0
- data/lib/llm/providers/anthropic/request_adapter.rb +1 -1
- data/lib/llm/providers/bedrock/error_handler.rb +1 -1
- data/lib/llm/providers/bedrock/request_adapter/completion.rb +5 -5
- data/lib/llm/providers/bedrock/request_adapter.rb +3 -3
- data/lib/llm/providers/bedrock/response_adapter/completion.rb +2 -2
- data/lib/llm/providers/bedrock/response_adapter.rb +2 -2
- data/lib/llm/providers/deepseek/request_adapter.rb +1 -1
- data/lib/llm/providers/google/error_handler.rb +2 -0
- data/lib/llm/providers/google/request_adapter.rb +1 -1
- data/lib/llm/providers/ollama/error_handler.rb +2 -0
- data/lib/llm/providers/ollama/request_adapter.rb +1 -1
- data/lib/llm/providers/openai/error_handler.rb +2 -0
- data/lib/llm/providers/openai/request_adapter.rb +1 -1
- data/lib/llm/registry.rb +2 -2
- data/lib/llm/response.rb +1 -1
- data/lib/llm/schema/object.rb +1 -1
- data/lib/llm/stream.rb +1 -1
- data/lib/llm/tool.rb +2 -2
- data/lib/llm/transport/http.rb +2 -2
- data/lib/llm/transport/persistent_http.rb +1 -1
- data/lib/llm/transport/response/http.rb +1 -1
- data/lib/llm/utils.rb +1 -1
- data/lib/llm/version.rb +1 -1
- data/lib/llm.rb +11 -8
- data/llm.gemspec +11 -9
- data/resources/deepdive.md +432 -0
- metadata +28 -15
data/README.md
CHANGED
|
@@ -1,52 +1,58 @@
|
|
|
1
1
|
<p align="center">
|
|
2
|
-
<a href="https://
|
|
3
|
-
<img
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
<a href="https://opensource.org/license/0bsd">
|
|
11
|
-
<img src="https://img.shields.io/badge/License-0BSD-orange.svg?" alt="License">
|
|
12
|
-
</a>
|
|
13
|
-
<a href="https://github.com/llmrb/llm.rb/tags">
|
|
14
|
-
<img src="https://img.shields.io/badge/version-11.2.0-green.svg?" alt="Version">
|
|
2
|
+
<a href="https://r.uby.dev">
|
|
3
|
+
<img
|
|
4
|
+
src="https://github.com/r-uby-dev/llm.rb/raw/main/rubydev.svg"
|
|
5
|
+
width="400"
|
|
6
|
+
height="200"
|
|
7
|
+
border="0"
|
|
8
|
+
alt="a r.uby.dev project"
|
|
9
|
+
>
|
|
15
10
|
</a>
|
|
16
11
|
</p>
|
|
17
12
|
|
|
18
|
-
|
|
13
|
+
> A [r.uby.dev](https://r.uby.dev) project.
|
|
14
|
+
|
|
15
|
+
Ruby's capable AI runtime.
|
|
16
|
+
|
|
17
|
+
It provides one Ruby interface for building with large language models:
|
|
18
|
+
providers, agents, tools, skills, MCP, A2A, RAG, streaming, files, and persisted conversation state all share the same runtime.
|
|
19
|
+
|
|
20
|
+
The gem runs on Ruby's standard library by default and loads optional
|
|
21
|
+
integrations only when needed. It supports OpenAI, OpenAI-compatible
|
|
22
|
+
endpoints, Anthropic, Google Gemini, DeepSeek, xAI, Z.ai, AWS Bedrock,
|
|
23
|
+
Ollama, and llama.cpp, with built-in ActiveRecord and Sequel support.
|
|
24
|
+
|
|
25
|
+
## Services
|
|
19
26
|
|
|
20
|
-
llm.rb is
|
|
27
|
+
llm.rb is a [r.uby.dev](https://r.uby.dev) project
|
|
28
|
+
that is part of a growing family of AI-related
|
|
29
|
+
projects that also includes publically accessible
|
|
30
|
+
SSH services.
|
|
21
31
|
|
|
22
|
-
|
|
23
|
-
only when needed, and offers a single runtime for providers, agents,
|
|
24
|
-
tools, skills, MCP, A2A (Agent2Agent), RAG (vector stores & embeddings),
|
|
25
|
-
streaming, files, and persisted state.
|
|
32
|
+
#### matz - the mruby expert
|
|
26
33
|
|
|
27
|
-
|
|
28
|
-
Gemini, DeepSeek, xAI, Z.ai, AWS Bedrock, Ollama, and llama.cpp. It
|
|
29
|
-
also includes built-in ActiveRecord and Sequel support, plus concurrent
|
|
30
|
-
tool execution through threads, tasks (via async gem), fibers, ractors,
|
|
31
|
-
and fork (via xchan.rb gem).
|
|
34
|
+
> ssh matz@r.uby.dev
|
|
32
35
|
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
36
|
+
See [https://r.uby.dev/matz](https://r.uby.dev/matz) for more information.
|
|
37
|
+
|
|
38
|
+
#### robert - the freebsd expert
|
|
39
|
+
|
|
40
|
+
> ssh robert@4.4bsd.dev
|
|
41
|
+
|
|
42
|
+
See [https://4.4bsd.dev/robert](https://4.4bsd.dev/robert) for more information.
|
|
37
43
|
|
|
38
44
|
## Quick start
|
|
39
45
|
|
|
40
46
|
#### LLM::Context
|
|
41
47
|
|
|
42
48
|
The
|
|
43
|
-
[LLM::Context](https://
|
|
49
|
+
[LLM::Context](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
|
|
44
50
|
object is at the heart of the runtime. Almost all other features build
|
|
45
51
|
on top of it. It is a low-level interface to a model, and requires tool
|
|
46
52
|
execution to be managed manually. The
|
|
47
|
-
[LLM::Agent](https://
|
|
53
|
+
[LLM::Agent](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
|
|
48
54
|
class is almost the same as
|
|
49
|
-
[LLM::Context](https://
|
|
55
|
+
[LLM::Context](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
|
|
50
56
|
but it manages tool execution for you - we'll cover agents next:
|
|
51
57
|
|
|
52
58
|
```ruby
|
|
@@ -60,9 +66,9 @@ ctx.talk "Hello world"
|
|
|
60
66
|
#### LLM::Agent
|
|
61
67
|
|
|
62
68
|
The
|
|
63
|
-
[LLM::Agent](https://
|
|
69
|
+
[LLM::Agent](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
|
|
64
70
|
object is implemented on top of
|
|
65
|
-
[LLM::Context](https://
|
|
71
|
+
[LLM::Context](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html).
|
|
66
72
|
It provides the same interface, but manages tool execution for you. It
|
|
67
73
|
also has builtin features such as a loop guard that detects repeated
|
|
68
74
|
tool call patterns, and another guard that detects infinite tool call
|
|
@@ -84,7 +90,7 @@ executed. When a matching tool is called, llm.rb runs
|
|
|
84
90
|
`on_tool_confirmation`. That callback must decide whether to cancel the
|
|
85
91
|
tool call or approve it and execute it by calling
|
|
86
92
|
`fn.spawn(strategy).wait`, and it must always return an instance of
|
|
87
|
-
[`LLM::Function::Return`](https://
|
|
93
|
+
[`LLM::Function::Return`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Return.html):
|
|
88
94
|
|
|
89
95
|
```ruby
|
|
90
96
|
require "llm"
|
|
@@ -110,7 +116,7 @@ Agent.new(llm, stream: $stdout).talk("Delete /tmp/example.txt.")
|
|
|
110
116
|
#### Tools
|
|
111
117
|
|
|
112
118
|
The
|
|
113
|
-
[LLM::Tool](https://
|
|
119
|
+
[LLM::Tool](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html)
|
|
114
120
|
class can be subclassed to implement your own tools that can extend the
|
|
115
121
|
abilities of a model:
|
|
116
122
|
|
|
@@ -122,7 +128,7 @@ class ReadFile < LLM::Tool
|
|
|
122
128
|
required %i[path]
|
|
123
129
|
|
|
124
130
|
def call(path:)
|
|
125
|
-
{contents: File.read(path)}
|
|
131
|
+
{ contents: File.read(path) }
|
|
126
132
|
end
|
|
127
133
|
end
|
|
128
134
|
```
|
|
@@ -130,18 +136,18 @@ end
|
|
|
130
136
|
#### MCP
|
|
131
137
|
|
|
132
138
|
The
|
|
133
|
-
[LLM::MCP](https://
|
|
139
|
+
[LLM::MCP](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html)
|
|
134
140
|
object lets llm.rb use tools provided by an MCP server. Those tools are
|
|
135
141
|
exposed through the same runtime as local tools, so you can pass them
|
|
136
142
|
to either
|
|
137
|
-
[LLM::Context](https://
|
|
143
|
+
[LLM::Context](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
|
|
138
144
|
or
|
|
139
|
-
[LLM::Agent](https://
|
|
145
|
+
[LLM::Agent](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html).
|
|
140
146
|
In this example, the MCP server runs over stdio and
|
|
141
|
-
[LLM::
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
147
|
+
[LLM::Agent](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
|
|
148
|
+
manages the tool loop. For **stdio**, `mcp.session` is the preferred
|
|
149
|
+
pattern because it keeps one MCP session alive across discovery and
|
|
150
|
+
tool calls:
|
|
145
151
|
|
|
146
152
|
```ruby
|
|
147
153
|
require "llm"
|
|
@@ -150,9 +156,8 @@ llm = LLM.openai(key: ENV["KEY"])
|
|
|
150
156
|
mcp = LLM::MCP.stdio(argv: ["ruby", "server.rb"])
|
|
151
157
|
|
|
152
158
|
mcp.session do
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
ctx.talk(ctx.wait(:call)) while ctx.functions?
|
|
159
|
+
agent = LLM::Agent.new(llm, stream: $stdout, tools: mcp.tools)
|
|
160
|
+
agent.talk "Use the available tools to inspect the environment."
|
|
156
161
|
end
|
|
157
162
|
```
|
|
158
163
|
|
|
@@ -167,16 +172,15 @@ require "llm"
|
|
|
167
172
|
llm = LLM.openai(key: ENV["KEY"])
|
|
168
173
|
mcp = LLM::MCP.stdio(argv: ["ruby", "server.rb"])
|
|
169
174
|
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
ctx.talk(ctx.wait(:call)) while ctx.functions?
|
|
175
|
+
agent = LLM::Agent.new(llm, tools: mcp.tools)
|
|
176
|
+
agent.talk("Use the available tools to inspect the environment.")
|
|
173
177
|
```
|
|
174
178
|
|
|
175
179
|
The HTTP transport can be used with or without the `session` method,
|
|
176
180
|
and unlike the stdio transport it can remain efficient without the
|
|
177
181
|
`session` method through a persistent connection pool that is available
|
|
178
182
|
through the
|
|
179
|
-
[LLM::Transport.net_http_persistent](https://
|
|
183
|
+
[LLM::Transport.net_http_persistent](https://r.uby.dev/api-docs/llm.rb/LLM/Transport.html#method-c-net_http_persistent)
|
|
180
184
|
transport:
|
|
181
185
|
|
|
182
186
|
```ruby
|
|
@@ -188,21 +192,20 @@ mcp = LLM::MCP.http(
|
|
|
188
192
|
transport: :net_http_persistent
|
|
189
193
|
)
|
|
190
194
|
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
ctx.talk(ctx.wait(:call)) while ctx.functions?
|
|
195
|
+
agent = LLM::Agent.new(llm, tools: mcp.tools)
|
|
196
|
+
agent.talk("Use the available tools to inspect the environment.")
|
|
194
197
|
```
|
|
195
198
|
|
|
196
199
|
#### A2A (Agent 2 Agent)
|
|
197
200
|
|
|
198
201
|
The
|
|
199
|
-
[LLM::A2A](https://
|
|
202
|
+
[LLM::A2A](https://r.uby.dev/api-docs/llm.rb/LLM/A2A.html)
|
|
200
203
|
object lets llm.rb use skills provided by a remote A2A agent. Those
|
|
201
204
|
skills are exposed through the same runtime as local tools, so you can
|
|
202
205
|
pass them to either
|
|
203
|
-
[LLM::Context](https://
|
|
206
|
+
[LLM::Context](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
|
|
204
207
|
or
|
|
205
|
-
[LLM::Agent](https://
|
|
208
|
+
[LLM::Agent](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html).
|
|
206
209
|
|
|
207
210
|
Use remote skills as local tools:
|
|
208
211
|
|
|
@@ -211,12 +214,11 @@ require "llm"
|
|
|
211
214
|
|
|
212
215
|
a2a = LLM::A2A.rest(
|
|
213
216
|
url: "https://remote-agent.example.com",
|
|
214
|
-
headers: {"Authorization" => "Bearer token"}
|
|
217
|
+
headers: { "Authorization" => "Bearer token" }
|
|
215
218
|
)
|
|
216
219
|
llm = LLM.openai(key: ENV["KEY"])
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
ctx.talk(ctx.wait(:call)) while ctx.functions?
|
|
220
|
+
agent = LLM::Agent.new(llm, tools: a2a.skills)
|
|
221
|
+
agent.talk "Analyze this CSV and summarize the trends."
|
|
220
222
|
```
|
|
221
223
|
|
|
222
224
|
Use persistent HTTP connections:
|
|
@@ -232,7 +234,7 @@ a2a = LLM::A2A.rest(
|
|
|
232
234
|
|
|
233
235
|
For more on direct messaging, task operations, push notification
|
|
234
236
|
configs, and JSON-RPC, see the
|
|
235
|
-
[LLM::A2A API docs](https://
|
|
237
|
+
[LLM::A2A API docs](https://r.uby.dev/api-docs/llm.rb/LLM/A2A.html).
|
|
236
238
|
|
|
237
239
|
#### Transports
|
|
238
240
|
|
|
@@ -242,9 +244,9 @@ or provide a transport shortcut when you want a different backend.
|
|
|
242
244
|
`transport: :curb` uses libcurl through the optional `curb` gem.
|
|
243
245
|
|
|
244
246
|
Custom transports can implement the
|
|
245
|
-
[LLM::Transport](https://
|
|
247
|
+
[LLM::Transport](https://r.uby.dev/api-docs/llm.rb/LLM/Transport.html)
|
|
246
248
|
interface and receive transport-agnostic
|
|
247
|
-
[LLM::Transport::Request](https://
|
|
249
|
+
[LLM::Transport::Request](https://r.uby.dev/api-docs/llm.rb/LLM/Transport/Request.html)
|
|
248
250
|
objects from providers.
|
|
249
251
|
|
|
250
252
|
```ruby
|
|
@@ -289,7 +291,7 @@ ReleaseAgent.new(llm, stream: $stdout).talk("Prepare the next release.")
|
|
|
289
291
|
|
|
290
292
|
A skill can also have its sub-agent inherit the parents tools through the
|
|
291
293
|
`inherit` directive. The `inherit` directive has coverage for the "classic"
|
|
292
|
-
tools (a subclass of [LLM::Tool](https://
|
|
294
|
+
tools (a subclass of [LLM::Tool](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html)),
|
|
293
295
|
MCP tools, and A2A tools that a parent context or agent has access to:
|
|
294
296
|
|
|
295
297
|
```yaml
|
|
@@ -303,7 +305,7 @@ MCP tools, and A2A tools that a parent context or agent has access to:
|
|
|
303
305
|
#### LLM::Stream
|
|
304
306
|
|
|
305
307
|
The
|
|
306
|
-
[LLM::Stream](https://
|
|
308
|
+
[LLM::Stream](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html)
|
|
307
309
|
object lets you observe output and runtime events as they happen. You
|
|
308
310
|
can subclass it to handle streamed content in your own application:
|
|
309
311
|
|
|
@@ -317,14 +319,14 @@ class Stream < LLM::Stream
|
|
|
317
319
|
end
|
|
318
320
|
|
|
319
321
|
llm = LLM.openai(key: ENV["KEY"])
|
|
320
|
-
|
|
321
|
-
|
|
322
|
+
agent = LLM::Agent.new(llm, stream: Stream.new)
|
|
323
|
+
agent.talk "Write a haiku about Ruby."
|
|
322
324
|
```
|
|
323
325
|
|
|
324
326
|
#### LLM::Stream (advanced)
|
|
325
327
|
|
|
326
328
|
The
|
|
327
|
-
[LLM::Stream](https://
|
|
329
|
+
[LLM::Stream](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html)
|
|
328
330
|
object can also resolve tool calls while output is still streaming. In
|
|
329
331
|
`on_tool_call`, you can spawn the tool, push the work onto the stream
|
|
330
332
|
queue, and later drain it with `wait`:
|
|
@@ -354,7 +356,7 @@ ctx.talk(ctx.wait) while ctx.functions?
|
|
|
354
356
|
llm.rb can run tool work concurrently. This is useful when a model calls
|
|
355
357
|
multiple tools and you want to resolve them in parallel instead of one
|
|
356
358
|
at a time. On
|
|
357
|
-
[LLM::Agent](https://
|
|
359
|
+
[LLM::Agent](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html),
|
|
358
360
|
you can enable this with `concurrency`. Common options are `:call` for
|
|
359
361
|
sequential execution, `:thread`, or `:task` for concurrent IO-bound work, and
|
|
360
362
|
`:ractor` or `:fork` for more isolated CPU-bound work:
|
|
@@ -375,46 +377,47 @@ agent.talk "Read README.md and CHANGELOG.md and compare them."
|
|
|
375
377
|
|
|
376
378
|
#### Serialization
|
|
377
379
|
|
|
378
|
-
The [`LLM::
|
|
380
|
+
The [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
|
|
379
381
|
object can be serialized to JSON, which makes it suitable for storing
|
|
380
382
|
in a file, a database column, or a Redis queue. The built-in
|
|
381
|
-
ActiveRecord and Sequel plugins are built on top of
|
|
383
|
+
ActiveRecord and Sequel plugins are built on top of the same underlying
|
|
384
|
+
serialization feature:
|
|
382
385
|
|
|
383
386
|
```ruby
|
|
384
387
|
require "llm"
|
|
385
388
|
|
|
386
389
|
llm = LLM.openai(key: ENV["KEY"])
|
|
387
390
|
|
|
388
|
-
# Serialize
|
|
389
|
-
|
|
390
|
-
|
|
391
|
-
string =
|
|
391
|
+
# Serialize an agent
|
|
392
|
+
agent1 = LLM::Agent.new(llm)
|
|
393
|
+
agent1.talk "Remember that my favorite language is Ruby"
|
|
394
|
+
string = agent1.to_json
|
|
392
395
|
|
|
393
|
-
# Restore
|
|
394
|
-
|
|
395
|
-
|
|
396
|
-
|
|
396
|
+
# Restore an agent (from JSON)
|
|
397
|
+
agent2 = LLM::Agent.new(llm, stream: $stdout)
|
|
398
|
+
agent2.restore(string:)
|
|
399
|
+
agent2.talk "What is my favorite language?"
|
|
397
400
|
```
|
|
398
401
|
|
|
399
402
|
#### ask
|
|
400
403
|
|
|
401
|
-
[`LLM::
|
|
404
|
+
[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
|
|
402
405
|
also provides `ask`, a convenience interface that is compatible with
|
|
403
406
|
RubyLLM's `ask` method. It accepts a prompt, an optional `with:`
|
|
404
407
|
attachment path or paths, an optional `stream:` target, and an optional
|
|
405
408
|
block that chunks are yielded to. It returns an
|
|
406
|
-
[`LLM::Response`](https://
|
|
409
|
+
[`LLM::Response`](https://r.uby.dev/api-docs/llm.rb/LLM/Response.html),
|
|
407
410
|
so use `.content` when you want the text directly:
|
|
408
411
|
|
|
409
412
|
```ruby
|
|
410
413
|
require "llm"
|
|
411
414
|
|
|
412
415
|
llm = LLM.openai(key: ENV["KEY"])
|
|
413
|
-
|
|
416
|
+
agent = LLM::Agent.new(llm)
|
|
414
417
|
|
|
415
|
-
puts
|
|
416
|
-
puts
|
|
417
|
-
|
|
418
|
+
puts agent.ask("Hello world").content
|
|
419
|
+
puts agent.ask("Summarize this document.", with: "README.md").content
|
|
420
|
+
agent.ask("Stream this reply.") { $stdout << _1 }
|
|
418
421
|
```
|
|
419
422
|
|
|
420
423
|
## Installation
|
|
@@ -427,62 +430,62 @@ gem install llm.rb
|
|
|
427
430
|
|
|
428
431
|
#### REPL
|
|
429
432
|
|
|
430
|
-
This example uses [`LLM::
|
|
431
|
-
|
|
432
|
-
[deepdive (web)](https://
|
|
433
|
+
This example uses [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
|
|
434
|
+
for an interactive REPL. <br> See the
|
|
435
|
+
[deepdive (web)](https://r.uby.dev/llm/) or
|
|
433
436
|
[deepdive (markdown)](resources/deepdive.md) for more examples.
|
|
434
437
|
|
|
435
438
|
```ruby
|
|
436
439
|
require "llm"
|
|
437
440
|
|
|
438
441
|
llm = LLM.openai(key: ENV["KEY"])
|
|
439
|
-
|
|
442
|
+
agent = LLM::Agent.new(llm, stream: $stdout)
|
|
440
443
|
|
|
441
444
|
loop do
|
|
442
445
|
print "> "
|
|
443
|
-
|
|
446
|
+
agent.talk(STDIN.gets || break)
|
|
444
447
|
puts
|
|
445
448
|
end
|
|
446
449
|
```
|
|
447
450
|
|
|
448
451
|
#### Multimodal: Local Files
|
|
449
452
|
|
|
450
|
-
In llm.rb, a prompt can be a string, an [`LLM::Prompt`](https://
|
|
453
|
+
In llm.rb, a prompt can be a string, an [`LLM::Prompt`](https://r.uby.dev/api-docs/llm.rb/LLM/Prompt.html), or an array.
|
|
451
454
|
When you use an array, each element can be plain text or a tagged object such as
|
|
452
|
-
[`
|
|
453
|
-
[`
|
|
454
|
-
or [`
|
|
455
|
+
[`agent.image_url(...)`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#image_url-instance_method),
|
|
456
|
+
[`agent.local_file(...)`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#local_file-instance_method),
|
|
457
|
+
or [`agent.remote_file(...)`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#remote_file-instance_method).
|
|
455
458
|
Those tagged objects carry the metadata the provider adapter needs to turn one
|
|
456
459
|
Ruby prompt into the provider-specific multimodal request schema.
|
|
457
460
|
|
|
458
461
|
If the model understands that file type, you can attach a local file directly
|
|
459
|
-
with `
|
|
462
|
+
with `agent.ask(..., with: path)` instead of uploading it first through a
|
|
460
463
|
provider Files API. Under the hood, llm.rb tags the path as a
|
|
461
|
-
[`
|
|
464
|
+
[`agent.local_file(...)`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#local_file-instance_method)
|
|
462
465
|
object:
|
|
463
466
|
|
|
464
467
|
```ruby
|
|
465
468
|
require "llm"
|
|
466
469
|
|
|
467
470
|
llm = LLM.openai(key: ENV["KEY"])
|
|
468
|
-
|
|
469
|
-
puts
|
|
471
|
+
agent = LLM::Agent.new(llm)
|
|
472
|
+
puts agent.ask("Summarize this document.", with: "README.md").content
|
|
470
473
|
```
|
|
471
474
|
|
|
472
475
|
#### Context Compaction
|
|
473
476
|
|
|
474
|
-
This example uses [`LLM::
|
|
475
|
-
[`LLM::Compactor`](https://
|
|
476
|
-
[`LLM::Stream`](https://
|
|
477
|
-
long-lived
|
|
477
|
+
This example uses [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html),
|
|
478
|
+
[`LLM::Compactor`](https://r.uby.dev/api-docs/llm.rb/LLM/Compactor.html), and
|
|
479
|
+
[`LLM::Stream`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html) together so
|
|
480
|
+
long-lived conversations can summarize older history and expose the lifecycle
|
|
478
481
|
through stream hooks. This approach is inspired by General Intelligence
|
|
479
482
|
Systems. The
|
|
480
483
|
compactor can also use its own `model:` if you want summarization to run on a
|
|
481
|
-
different model from the main
|
|
484
|
+
different model from the main conversation. `token_threshold:` accepts either a
|
|
482
485
|
fixed token count or a percentage string like `"90%"`, which resolves
|
|
483
486
|
against the active model context window and triggers compaction once total
|
|
484
487
|
token usage goes over that percentage. See the
|
|
485
|
-
[deepdive (web)](https://
|
|
488
|
+
[deepdive (web)](https://r.uby.dev/llm/) or
|
|
486
489
|
[deepdive (markdown)](resources/deepdive.md) for more examples.
|
|
487
490
|
|
|
488
491
|
```ruby
|
|
@@ -499,7 +502,7 @@ class Stream < LLM::Stream
|
|
|
499
502
|
end
|
|
500
503
|
|
|
501
504
|
llm = LLM.openai(key: ENV["KEY"])
|
|
502
|
-
|
|
505
|
+
agent = LLM::Agent.new(
|
|
503
506
|
llm,
|
|
504
507
|
stream: Stream.new,
|
|
505
508
|
compactor: {
|
|
@@ -512,15 +515,14 @@ ctx = LLM::Context.new(
|
|
|
512
515
|
|
|
513
516
|
#### Reasoning
|
|
514
517
|
|
|
515
|
-
This example uses [`LLM::Stream`](https://
|
|
518
|
+
This example uses [`LLM::Stream`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html)
|
|
516
519
|
with the OpenAI Responses API so reasoning output is streamed separately from
|
|
517
520
|
visible assistant output. See the
|
|
518
|
-
[deepdive (web)](https://
|
|
521
|
+
[deepdive (web)](https://r.uby.dev/llm/) or
|
|
519
522
|
[deepdive (markdown)](resources/deepdive.md) for more examples.
|
|
520
523
|
|
|
521
|
-
To use the Responses API (OpenAI-specific), initialize
|
|
522
|
-
|
|
523
|
-
`talk` for turns.
|
|
524
|
+
To use the Responses API (OpenAI-specific), initialize an agent with
|
|
525
|
+
`mode: :responses` and keep using `talk` for turns.
|
|
524
526
|
|
|
525
527
|
```ruby
|
|
526
528
|
require "llm"
|
|
@@ -536,21 +538,21 @@ class Stream < LLM::Stream
|
|
|
536
538
|
end
|
|
537
539
|
|
|
538
540
|
llm = LLM.openai(key: ENV["KEY"])
|
|
539
|
-
|
|
541
|
+
agent = LLM::Agent.new(
|
|
540
542
|
llm,
|
|
541
543
|
model: "gpt-5.4-mini",
|
|
542
544
|
mode: :responses,
|
|
543
|
-
reasoning: {effort: "medium"},
|
|
545
|
+
reasoning: { effort: "medium" },
|
|
544
546
|
stream: Stream.new
|
|
545
547
|
)
|
|
546
|
-
|
|
548
|
+
agent.talk("Solve 17 * 19 and show your work.")
|
|
547
549
|
```
|
|
548
550
|
|
|
549
551
|
#### Request Cancellation
|
|
550
552
|
|
|
551
553
|
Need to cancel a stream? llm.rb has you covered through
|
|
552
|
-
[`LLM::
|
|
553
|
-
<br> See the [deepdive (web)](https://
|
|
554
|
+
[`LLM::Agent#interrupt!`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#interrupt-21-instance_method).
|
|
555
|
+
<br> See the [deepdive (web)](https://r.uby.dev/llm/)
|
|
554
556
|
or [deepdive (markdown)](resources/deepdive.md) for more examples.
|
|
555
557
|
|
|
556
558
|
```ruby
|
|
@@ -558,27 +560,27 @@ require "llm"
|
|
|
558
560
|
require "io/console"
|
|
559
561
|
|
|
560
562
|
llm = LLM.openai(key: ENV["KEY"])
|
|
561
|
-
|
|
563
|
+
agent = LLM::Agent.new(llm, stream: $stdout)
|
|
562
564
|
worker = Thread.new do
|
|
563
|
-
|
|
565
|
+
agent.talk("Write a very long essay about network protocols.")
|
|
564
566
|
rescue LLM::Interrupt
|
|
565
567
|
puts "Request was interrupted!"
|
|
566
568
|
end
|
|
567
569
|
|
|
568
570
|
STDIN.getch
|
|
569
|
-
|
|
571
|
+
agent.interrupt!
|
|
570
572
|
worker.join
|
|
571
573
|
```
|
|
572
574
|
|
|
573
575
|
#### Sequel (ORM)
|
|
574
576
|
|
|
575
577
|
The `plugin :llm` integration wraps
|
|
576
|
-
[`LLM::Context`](https://
|
|
578
|
+
[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html) on a
|
|
577
579
|
`Sequel::Model` and keeps tool execution explicit. Like the ActiveRecord
|
|
578
580
|
wrappers, its built-in persistence contract is the serialized `data` column,
|
|
579
581
|
while `provider:` resolves a real `LLM::Provider` instance and `context:`
|
|
580
582
|
injects defaults such as `model:`. <br> See the
|
|
581
|
-
[deepdive (web)](https://
|
|
583
|
+
[deepdive (web)](https://r.uby.dev/llm/) or
|
|
582
584
|
[deepdive (markdown)](resources/deepdive.md) for more examples.
|
|
583
585
|
|
|
584
586
|
```ruby
|
|
@@ -597,7 +599,7 @@ class Context < Sequel::Model
|
|
|
597
599
|
end
|
|
598
600
|
|
|
599
601
|
def set_context
|
|
600
|
-
{model: "gpt-5.4-mini", mode: :responses, store: false}
|
|
602
|
+
{ model: "gpt-5.4-mini", mode: :responses, store: false }
|
|
601
603
|
end
|
|
602
604
|
end
|
|
603
605
|
|
|
@@ -608,13 +610,13 @@ puts ctx.talk("What is my favorite language?").content
|
|
|
608
610
|
|
|
609
611
|
#### ActiveRecord (ORM): acts_as_llm
|
|
610
612
|
|
|
611
|
-
The `acts_as_llm` method wraps [`LLM::Context`](https://
|
|
613
|
+
The `acts_as_llm` method wraps [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html) and
|
|
612
614
|
provides full control over tool execution. Its built-in persistence contract is
|
|
613
615
|
one serialized `data` column. If your app has provider, model, or usage
|
|
614
616
|
columns, provide them to llm.rb through `provider:` and `context:` instead of
|
|
615
617
|
relying on reserved wrapper columns.
|
|
616
618
|
|
|
617
|
-
See the [deepdive (web)](https://
|
|
619
|
+
See the [deepdive (web)](https://r.uby.dev/llm/)
|
|
618
620
|
or [deepdive (markdown)](resources/deepdive.md) for more examples.
|
|
619
621
|
|
|
620
622
|
```ruby
|
|
@@ -632,7 +634,7 @@ class Context < ApplicationRecord
|
|
|
632
634
|
end
|
|
633
635
|
|
|
634
636
|
def set_context
|
|
635
|
-
{model: "gpt-5.4-mini", mode: :responses, store: false}
|
|
637
|
+
{ model: "gpt-5.4-mini", mode: :responses, store: false }
|
|
636
638
|
end
|
|
637
639
|
end
|
|
638
640
|
|
|
@@ -659,19 +661,19 @@ class Context < ApplicationRecord
|
|
|
659
661
|
end
|
|
660
662
|
|
|
661
663
|
def set_context
|
|
662
|
-
{model: model_name, mode: :responses, store: false}
|
|
664
|
+
{ model: model_name, mode: :responses, store: false }
|
|
663
665
|
end
|
|
664
666
|
end
|
|
665
667
|
```
|
|
666
668
|
|
|
667
669
|
#### ActiveRecord (ORM): acts_as_agent
|
|
668
670
|
|
|
669
|
-
The `acts_as_agent` method wraps [`LLM::Agent`](https://
|
|
671
|
+
The `acts_as_agent` method wraps [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html) and
|
|
670
672
|
manages tool execution for you. Like `acts_as_llm`, its built-in persistence
|
|
671
673
|
contract is one serialized `data` column. If your app has provider or model
|
|
672
674
|
columns, provide them to llm.rb through your hooks and agent DSL.
|
|
673
675
|
|
|
674
|
-
See the [deepdive (web)](https://
|
|
676
|
+
See the [deepdive (web)](https://r.uby.dev/llm/)
|
|
675
677
|
or [deepdive (markdown)](resources/deepdive.md) for more examples.
|
|
676
678
|
|
|
677
679
|
```ruby
|
|
@@ -693,7 +695,7 @@ class Ticket < ApplicationRecord
|
|
|
693
695
|
end
|
|
694
696
|
|
|
695
697
|
def set_context
|
|
696
|
-
{mode: :responses, store: false}
|
|
698
|
+
{ mode: :responses, store: false }
|
|
697
699
|
end
|
|
698
700
|
end
|
|
699
701
|
|
|
@@ -718,18 +720,18 @@ class Ticket < ApplicationRecord
|
|
|
718
720
|
end
|
|
719
721
|
|
|
720
722
|
def set_context
|
|
721
|
-
{mode: :responses, store: false}
|
|
723
|
+
{ mode: :responses, store: false }
|
|
722
724
|
end
|
|
723
725
|
end
|
|
724
726
|
```
|
|
725
727
|
|
|
726
728
|
#### MCP
|
|
727
729
|
|
|
728
|
-
This example uses [`LLM::MCP`](https://
|
|
730
|
+
This example uses [`LLM::MCP`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html)
|
|
729
731
|
over HTTP so remote GitHub MCP tools run through the same
|
|
730
|
-
`LLM::
|
|
732
|
+
`LLM::Agent` tool path as local tools. It expects a GitHub token in
|
|
731
733
|
`ENV["GITHUB_PAT"]`. See the
|
|
732
|
-
[deepdive (web)](https://
|
|
734
|
+
[deepdive (web)](https://r.uby.dev/llm/) or
|
|
733
735
|
[deepdive (markdown)](resources/deepdive.md) for more examples.
|
|
734
736
|
|
|
735
737
|
```ruby
|
|
@@ -739,23 +741,14 @@ require "net/http/persistent"
|
|
|
739
741
|
llm = LLM.openai(key: ENV["KEY"], persistent: true)
|
|
740
742
|
mcp = LLM::MCP.http(
|
|
741
743
|
url: "https://api.githubcopilot.com/mcp/",
|
|
742
|
-
headers: {"Authorization" => "Bearer
|
|
744
|
+
headers: { "Authorization" => "Bearer " + ENV["GITHUB_PAT"].to_s },
|
|
743
745
|
persistent: true
|
|
744
746
|
)
|
|
745
747
|
|
|
746
|
-
|
|
747
|
-
|
|
748
|
-
ctx.talk(ctx.wait(:call)) while ctx.functions?
|
|
748
|
+
agent = LLM::Agent.new(llm, stream: $stdout, tools: mcp.tools)
|
|
749
|
+
agent.talk("Pull information about my GitHub account.")
|
|
749
750
|
```
|
|
750
751
|
|
|
751
|
-
## Resources
|
|
752
|
-
|
|
753
|
-
- [deepdive (web)](https://llmrb.github.io/llm.rb/) and
|
|
754
|
-
[deepdive (markdown)](resources/deepdive.md) are the examples guide.
|
|
755
|
-
- [relay](https://github.com/llmrb/relay) shows a real application built on
|
|
756
|
-
top of llm.rb.
|
|
757
|
-
- [doc site](https://0x1eef.github.io/x/llm.rb?rebuild=1) has the API docs.
|
|
758
|
-
|
|
759
752
|
## License
|
|
760
753
|
|
|
761
754
|
[BSD Zero Clause](https://choosealicense.com/licenses/0bsd/)
|
data/lib/llm/a2a.rb
CHANGED
|
@@ -177,7 +177,7 @@ class LLM::A2A
|
|
|
177
177
|
##
|
|
178
178
|
# Sends a message to the agent and returns the response.
|
|
179
179
|
# @param [String] text The message text to send
|
|
180
|
-
# @param [Hash]
|
|
180
|
+
# @param [Hash] configuration
|
|
181
181
|
# Optional configuration (accepted_output_modes, return_immediately)
|
|
182
182
|
# @param [Hash, nil] metadata
|
|
183
183
|
# Optional metadata to attach to the request
|
|
@@ -198,7 +198,7 @@ class LLM::A2A
|
|
|
198
198
|
# The block is called for each {LLM::Object} event in the stream
|
|
199
199
|
# (Task, Message, TaskStatusUpdateEvent, TaskArtifactUpdateEvent).
|
|
200
200
|
# @param [String] text The message text to send
|
|
201
|
-
# @param [Hash]
|
|
201
|
+
# @param [Hash] configuration Optional configuration
|
|
202
202
|
# @yieldparam [LLM::Object] event A stream event
|
|
203
203
|
# @return [void]
|
|
204
204
|
def send_streaming_message(text, configuration = {}, &on_event)
|