llm.rb 12.6.0 → 13.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +571 -13
- data/LICENSE +21 -93
- data/README.md +183 -167
- data/bin/llm.rb +124 -0
- data/data/deepinfra.json +3 -0
- data/data/xai.json +1 -1
- data/lib/llm/a2a.rb +1 -1
- data/lib/llm/active_record/acts_as_llm.rb +6 -6
- data/lib/llm/agent.rb +136 -27
- data/lib/llm/buffer.rb +85 -3
- data/lib/llm/compactor/null.rb +19 -0
- data/lib/llm/compactor/truncate.rb +80 -0
- data/lib/llm/compactor.rb +42 -124
- data/lib/llm/context.rb +31 -37
- data/lib/llm/contract.rb +4 -25
- data/lib/llm/function/array.rb +18 -17
- data/lib/llm/function/async/group.rb +54 -0
- data/lib/llm/function/async/reactor.rb +48 -0
- data/lib/llm/function/async/task.rb +83 -0
- data/lib/llm/function/fiber/group.rb +46 -0
- data/lib/llm/function/fiber/task.rb +62 -0
- data/lib/llm/function/{fork_group.rb → fork/group.rb} +14 -5
- data/lib/llm/function/fork/job.rb +2 -2
- data/lib/llm/function/fork/task.rb +19 -10
- data/lib/llm/function/group.rb +40 -0
- data/lib/llm/function/{ractor_group.rb → ractor/group.rb} +13 -5
- data/lib/llm/function/ractor/job.rb +9 -3
- data/lib/llm/function/ractor/mailbox.rb +2 -0
- data/lib/llm/function/ractor/task.rb +23 -15
- data/lib/llm/function/{call_group.rb → sequential/group.rb} +12 -8
- data/lib/llm/function/sequential/task.rb +49 -0
- data/lib/llm/function/task.rb +25 -48
- data/lib/llm/function/thread/group.rb +46 -0
- data/lib/llm/function/thread/task.rb +60 -0
- data/lib/llm/function.rb +54 -65
- data/lib/llm/loop_guard.rb +1 -2
- data/lib/llm/mcp.rb +22 -0
- data/lib/llm/object.rb +2 -1
- data/lib/llm/provider.rb +6 -3
- data/lib/llm/providers/anthropic.rb +1 -1
- data/lib/llm/providers/bedrock/request_adapter.rb +1 -1
- data/lib/llm/providers/google.rb +2 -2
- data/lib/llm/providers/mistral.rb +1 -1
- data/lib/llm/providers/ollama.rb +1 -1
- data/lib/llm/providers/openai/responses.rb +1 -1
- data/lib/llm/providers/openai.rb +1 -1
- data/lib/llm/repl/{transcript.rb → buffer.rb} +34 -21
- data/lib/llm/repl/command.rb +47 -13
- data/lib/llm/repl/commands/compact.rb +33 -0
- data/lib/llm/repl/commands/help.rb +3 -5
- data/lib/llm/repl/input.rb +80 -15
- data/lib/llm/repl/markdown/table.rb +80 -0
- data/lib/llm/repl/markdown.rb +33 -3
- data/lib/llm/repl/node.rb +37 -0
- data/lib/llm/repl/status.rb +4 -4
- data/lib/llm/repl/stream.rb +12 -5
- data/lib/llm/repl/walker.rb +46 -0
- data/lib/llm/repl/window.rb +31 -32
- data/lib/llm/repl.rb +70 -38
- data/lib/llm/response.rb +10 -0
- data/lib/llm/schema/leaf.rb +5 -0
- data/lib/llm/schema/object.rb +11 -5
- data/lib/llm/sequel/plugin.rb +6 -6
- data/lib/llm/skill.rb +20 -4
- data/lib/llm/stream.rb +24 -17
- data/lib/llm/tool.rb +20 -4
- data/lib/llm/tools/chdir.rb +0 -2
- data/lib/llm/tools/{swap_text.rb → edit-file.rb} +3 -3
- data/lib/llm/tools/git.rb +11 -4
- data/lib/llm/tools/mkdir.rb +4 -1
- data/lib/llm/tools/pwd.rb +0 -2
- data/lib/llm/tools/read_file.rb +0 -2
- data/lib/llm/tools/rg.rb +11 -4
- data/lib/llm/tools/ruby.rb +46 -0
- data/lib/llm/tools/shell.rb +11 -4
- data/lib/llm/tools/utils.rb +31 -0
- data/lib/llm/tracer/pretty_logger.rb +127 -0
- data/lib/llm/tracer.rb +1 -0
- data/lib/llm/version.rb +1 -1
- data/lib/llm.rb +25 -5
- data/llm.gemspec +11 -5
- data/resources/deepdive.md +45 -1198
- metadata +39 -17
- data/lib/llm/function/call_task.rb +0 -46
- data/lib/llm/function/fiber_group.rb +0 -105
- data/lib/llm/function/task_group.rb +0 -97
- data/lib/llm/function/thread_group.rb +0 -102
data/LICENSE
CHANGED
|
@@ -1,93 +1,21 @@
|
|
|
1
|
-
|
|
2
|
-
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
Change License:
|
|
24
|
-
BSD Zero Clause License (0BSD)
|
|
25
|
-
|
|
26
|
-
Terms
|
|
27
|
-
|
|
28
|
-
The Licensor hereby grants you the right to copy, modify, create derivative
|
|
29
|
-
works, redistribute, and make non-production use of the Licensed Work. The
|
|
30
|
-
Licensor may make an Additional Use Grant, above, permitting limited
|
|
31
|
-
production use.
|
|
32
|
-
|
|
33
|
-
Effective on the Change Date, or the fourth anniversary of the first publicly
|
|
34
|
-
available distribution of a specific version of the Licensed Work under this
|
|
35
|
-
License, whichever comes first, the Licensor hereby grants you rights under
|
|
36
|
-
the terms of the Change License, and the rights granted in the paragraph
|
|
37
|
-
above terminate.
|
|
38
|
-
|
|
39
|
-
If your use of the Licensed Work does not comply with the requirements
|
|
40
|
-
currently in effect as described in this License, you must purchase a
|
|
41
|
-
commercial license from the Licensor, its affiliated entities, or authorized
|
|
42
|
-
resellers, or you must refrain from using the Licensed Work.
|
|
43
|
-
|
|
44
|
-
All copies of the original and modified Licensed Work, and derivative works
|
|
45
|
-
of the Licensed Work, are subject to this License. This License applies
|
|
46
|
-
separately for each version of the Licensed Work and the Change Date may vary
|
|
47
|
-
for each version of the Licensed Work released by Licensor.
|
|
48
|
-
|
|
49
|
-
You must conspicuously display this License on each original or modified copy
|
|
50
|
-
of the Licensed Work. If you receive the Licensed Work in original or
|
|
51
|
-
modified form from a third party, the terms and conditions set forth in this
|
|
52
|
-
License apply to your use of that work.
|
|
53
|
-
|
|
54
|
-
Any use of the Licensed Work in violation of this License will automatically
|
|
55
|
-
terminate your rights under this License for the current and all other
|
|
56
|
-
versions of the Licensed Work.
|
|
57
|
-
|
|
58
|
-
This License does not grant you any right in any trademark or logo of
|
|
59
|
-
Licensor or its affiliates (provided that you may use a trademark or logo of
|
|
60
|
-
Licensor as expressly required by this License).
|
|
61
|
-
|
|
62
|
-
TO THE EXTENT PERMITTED BY APPLICABLE LAW, THE LICENSED WORK IS PROVIDED ON
|
|
63
|
-
AN "AS IS" BASIS. LICENSOR HEREBY DISCLAIMS ALL WARRANTIES AND CONDITIONS,
|
|
64
|
-
EXPRESS OR IMPLIED, INCLUDING (WITHOUT LIMITATION) WARRANTIES OF
|
|
65
|
-
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT, AND
|
|
66
|
-
TITLE.
|
|
67
|
-
|
|
68
|
-
MariaDB hereby grants you permission to use this License's text to license
|
|
69
|
-
your works, and to refer to it using the trademark "Business Source
|
|
70
|
-
License", as long as you comply with the Covenants of Licensor below.
|
|
71
|
-
|
|
72
|
-
Covenants of Licensor
|
|
73
|
-
|
|
74
|
-
In consideration of the right to use this License's text and the "Business
|
|
75
|
-
Source License" name and trademark, Licensor covenants to MariaDB, and to all
|
|
76
|
-
other recipients of the licensed work to be provided by Licensor:
|
|
77
|
-
|
|
78
|
-
To specify as the Change License the GPL Version 2.0 or any later version, or
|
|
79
|
-
a license that is compatible with GPL Version 2.0 or a later version, where
|
|
80
|
-
"compatible" means that software provided under the Change License can be
|
|
81
|
-
included in a program with software provided under GPL Version 2.0 or a later
|
|
82
|
-
version. Licensor may specify additional Change Licenses without limitation.
|
|
83
|
-
|
|
84
|
-
To either: (a) specify an additional grant of rights to use that does not
|
|
85
|
-
impose any additional restriction on the right granted in this License, as
|
|
86
|
-
the Additional Use Grant; or (b) insert the text "None" to specify a Change
|
|
87
|
-
Date. Not to modify this License in any other way.
|
|
88
|
-
|
|
89
|
-
Notice
|
|
90
|
-
|
|
91
|
-
The Business Source License (this document, or the "License") is not an Open
|
|
92
|
-
Source license. However, the Licensed Work will eventually be made available
|
|
93
|
-
under an Open Source License, as stated in this License.
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Robert Gleeson
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
data/README.md
CHANGED
|
@@ -16,15 +16,19 @@ Welcome to the canonical llm.rb repository.
|
|
|
16
16
|
|
|
17
17
|
llm.rb is an advanced runtime for building capable AI applications
|
|
18
18
|
on CRuby. By default it has zero runtime dependencies although certain
|
|
19
|
-
functionality
|
|
19
|
+
functionality (such as ActiveRecord support) require
|
|
20
20
|
optional dependencies that are opt-in.
|
|
21
21
|
|
|
22
|
+
When you want to learn more than what the README covers, checkout
|
|
23
|
+
the [deepdive.md](https://r.uby.dev/llm/deepdive/).
|
|
24
|
+
|
|
22
25
|
## Features
|
|
23
26
|
|
|
24
27
|
The runtime supports OpenAI, OpenAI-compatible endpoints, Anthropic, Google
|
|
25
28
|
Gemini, Mistral, DeepSeek, DeepInfra, xAI, Z.ai, AWS Bedrock, Ollama, and llama.cpp.
|
|
26
29
|
It has first-class support for streaming, tool calls, MCP
|
|
27
|
-
and A2A, embeddings, vector stores
|
|
30
|
+
and A2A, embeddings, vector stores, OCR, context compaction,
|
|
31
|
+
and the RAG pattern.
|
|
28
32
|
|
|
29
33
|
There are multiple HTTP backends to choose from, tools can be run concurrently
|
|
30
34
|
or in parallel via threads, async tasks, fibers, ractors, and fork, and it is
|
|
@@ -32,7 +36,8 @@ also possible to make a tool call while the model is still streaming.
|
|
|
32
36
|
|
|
33
37
|
The runtime builds on top of three core concepts: providers, contexts, and agents,
|
|
34
38
|
so once you learn the fundamentals, everything else falls into place naturally. And once
|
|
35
|
-
you learn llm.rb, you will also be able to use
|
|
39
|
+
you learn llm.rb, you will also be able to use
|
|
40
|
+
<a href="https://r.uby.dev/mruby-llm">mruby-llm</a> and
|
|
36
41
|
<a href="https://r.uby.dev/wasm-llm">wasm-llm</a> because the API is pretty much identical.
|
|
37
42
|
|
|
38
43
|
## Install
|
|
@@ -45,7 +50,9 @@ gem install llm.rb
|
|
|
45
50
|
|
|
46
51
|
#### LLM::Agent
|
|
47
52
|
|
|
48
|
-
The
|
|
53
|
+
The
|
|
54
|
+
[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
|
|
55
|
+
class is the default high-level interface,
|
|
49
56
|
and it is recommended for most use-cases. It manages tool execution
|
|
50
57
|
automatically, guards against infinite loops, manages conversation
|
|
51
58
|
state, and much more.
|
|
@@ -58,13 +65,63 @@ agent = LLM::Agent.new(llm, stream: $stdout)
|
|
|
58
65
|
agent.talk "Hello world"
|
|
59
66
|
```
|
|
60
67
|
|
|
68
|
+
##### set
|
|
69
|
+
|
|
70
|
+
[`LLM::Agent.set`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#set-class_method)
|
|
71
|
+
is a class-level DSL that accepts a Hash of properties. Each key resolves to a
|
|
72
|
+
corresponding class accessor: `name`, `description`, `model`, `tools`,
|
|
73
|
+
`instructions`, `schema`, `stream`, `tracer`, `concurrency`, `confirm`,
|
|
74
|
+
`path`, and `skills`. All options are optional; zero or more can be set.
|
|
75
|
+
An error is raised for unknown keys so that typos are caught early.
|
|
76
|
+
|
|
77
|
+
```ruby
|
|
78
|
+
class SystemAdmin < LLM::Agent
|
|
79
|
+
set name: "sysadmin",
|
|
80
|
+
description: "system administration agent",
|
|
81
|
+
model: "deepseek-v4-pro",
|
|
82
|
+
tools: [Shell]
|
|
83
|
+
end
|
|
84
|
+
|
|
85
|
+
llm = LLM.deepseek(key: ENV["KEY"])
|
|
86
|
+
agent = SystemAdmin.new(llm)
|
|
87
|
+
agent.talk "Run 'date'"
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
##### Persistence
|
|
91
|
+
|
|
92
|
+
Set `path:` on an agent for automatic filesystem persistence;
|
|
93
|
+
the agent restores conversation history from the file on startup
|
|
94
|
+
and saves it back after every turn, with no manual serialization
|
|
95
|
+
code. For database-backed persistence, ActiveRecord and Sequel
|
|
96
|
+
integrations are also available (see the
|
|
97
|
+
[database deepdive](https://r.uby.dev/llm/deepdive/advanced/database)
|
|
98
|
+
for details). All persistence options use the same underlying
|
|
99
|
+
serialization.
|
|
100
|
+
|
|
101
|
+
```ruby
|
|
102
|
+
require "llm"
|
|
103
|
+
|
|
104
|
+
llm = LLM.deepseek(key: ENV["KEY"])
|
|
105
|
+
agent = LLM::Agent.new(llm, path: "session.json")
|
|
106
|
+
agent.talk "remember my name is robert"
|
|
107
|
+
|
|
108
|
+
# Next time, the conversation is restored automatically:
|
|
109
|
+
agent = LLM::Agent.new(llm, path: "session.json")
|
|
110
|
+
agent.talk "what's my name?"
|
|
111
|
+
```
|
|
112
|
+
|
|
61
113
|
#### LLM::Context
|
|
62
114
|
|
|
63
|
-
The
|
|
64
|
-
|
|
115
|
+
The
|
|
116
|
+
[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
|
|
117
|
+
class is at the heart of the runtime
|
|
118
|
+
and it is what
|
|
119
|
+
[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
|
|
120
|
+
uses under the hood.
|
|
65
121
|
It requires that the tool call loop be managed manually -
|
|
66
122
|
sometimes that can be useful, but usually for advanced use-cases.
|
|
67
|
-
If you're new to llm.rb, try
|
|
123
|
+
If you're new to llm.rb, try
|
|
124
|
+
[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html) first.
|
|
68
125
|
|
|
69
126
|
```ruby
|
|
70
127
|
require "llm"
|
|
@@ -76,7 +133,9 @@ ctx.talk "Hello world"
|
|
|
76
133
|
|
|
77
134
|
#### LLM::Tool
|
|
78
135
|
|
|
79
|
-
Subclasses of
|
|
136
|
+
Subclasses of
|
|
137
|
+
[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html)
|
|
138
|
+
are plain Ruby classes with
|
|
80
139
|
an optional set of typed parameters. <br> The model can choose to
|
|
81
140
|
call them on your behalf, and they're one of the most powerful features
|
|
82
141
|
for extending the feature set or abilities of a model.
|
|
@@ -97,7 +156,8 @@ end
|
|
|
97
156
|
#### LLM::Stream
|
|
98
157
|
|
|
99
158
|
Streams can be simple IO objects or subclasses of
|
|
100
|
-
[`LLM::Stream`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html)
|
|
159
|
+
[`LLM::Stream`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html)
|
|
160
|
+
with structured callbacks for content,
|
|
101
161
|
reasoning, tool calls, tool returns, and compaction.
|
|
102
162
|
|
|
103
163
|
```ruby
|
|
@@ -116,160 +176,88 @@ agent = LLM::Agent.new(llm, stream: MyStream.new)
|
|
|
116
176
|
agent.talk "Explain Ruby fibers."
|
|
117
177
|
```
|
|
118
178
|
|
|
119
|
-
#### LLM::
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
and
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
skills, and instructions.
|
|
142
|
-
|
|
143
|
-
```ruby
|
|
144
|
-
require "llm"
|
|
145
|
-
|
|
146
|
-
llm = LLM.deepseek(key: ENV["KEY"])
|
|
147
|
-
agent = LLM::Agent.new(llm)
|
|
148
|
-
agent.repl
|
|
149
|
-
```
|
|
150
|
-
|
|
151
|
-
##### REPL: State
|
|
152
|
-
|
|
153
|
-
The `path:` option accepts a file path where runtime state
|
|
154
|
-
is read from and written to. This lets you resume a
|
|
155
|
-
conversation across REPL sessions.
|
|
179
|
+
#### LLM::Schema
|
|
180
|
+
|
|
181
|
+
[`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html)
|
|
182
|
+
subclasses produce typed, structured
|
|
183
|
+
output from any model call. Pass a schema to `LLM::Context#talk`,
|
|
184
|
+
`LLM::Agent#talk`, or `LLM::Provider#complete` to receive validated
|
|
185
|
+
JSON instead of free text. Schemas work alongside tools and streams.
|
|
186
|
+
|
|
187
|
+
[`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html)
|
|
188
|
+
can define objects, arrays, enums, nested schemas,
|
|
189
|
+
and more. It is also used internally by
|
|
190
|
+
[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html) for parameter
|
|
191
|
+
definitions, so you already benefit from it when you declare tool
|
|
192
|
+
parameters.
|
|
193
|
+
|
|
194
|
+
The
|
|
195
|
+
[`LLM::DeepSeek`](https://r.uby.dev/api-docs/llm.rb/LLM/DeepSeek.html)
|
|
196
|
+
provider includes runtime-level optimisations such as structured
|
|
197
|
+
output support (despite no official structured outputs API) and
|
|
198
|
+
SVG image generation. This example uses
|
|
199
|
+
[`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html) with
|
|
200
|
+
DeepSeek:
|
|
156
201
|
|
|
157
202
|
```ruby
|
|
158
|
-
|
|
203
|
+
class Weather < LLM::Schema
|
|
204
|
+
property :city, String, "The city name"
|
|
205
|
+
property :temperature, Float, "Current temperature"
|
|
206
|
+
property :conditions, String, "Weather conditions"
|
|
207
|
+
required %i[city temperature conditions]
|
|
208
|
+
end
|
|
159
209
|
|
|
160
210
|
llm = LLM.deepseek(key: ENV["KEY"])
|
|
161
|
-
agent = LLM::Agent.new(llm)
|
|
162
|
-
agent.
|
|
211
|
+
agent = LLM::Agent.new(llm, schema: Weather)
|
|
212
|
+
res = agent.talk "Weather in Paris?"
|
|
213
|
+
res.content! # => {city: "Paris", temperature: 15.0, conditions: "Cloudy"}
|
|
163
214
|
```
|
|
164
215
|
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
The `tools` option lets you attach additional tools
|
|
168
|
-
for the duration of the session. This is in addition to
|
|
169
|
-
any tools that might already be associated with an agent.
|
|
170
|
-
|
|
171
|
-
A number of optional tools are distributed as part of
|
|
172
|
-
llm.rb. They power the agents that can be found in the
|
|
173
|
-
[agents/](agents/) directory.
|
|
174
|
-
|
|
175
|
-
```ruby
|
|
176
|
-
require "llm"
|
|
177
|
-
|
|
178
|
-
llm = LLM.deepseek(key: ENV["KEY"])
|
|
179
|
-
agent = LLM::Agent.new(llm)
|
|
180
|
-
agent.repl(tools: [Debugger])
|
|
181
|
-
```
|
|
216
|
+
#### LLM::REPL
|
|
182
217
|
|
|
183
|
-
The
|
|
184
|
-
|
|
218
|
+
The [LLM::Agent#repl](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#repl-instance_method)
|
|
219
|
+
method drops you into a curses-based TUI for talking to an
|
|
220
|
+
agent interactively. Set `path:` on the agent for automatic
|
|
221
|
+
persistence across REPL sessions. The `tools:` option attaches
|
|
222
|
+
extra tools for the duration of the session. It is like
|
|
223
|
+
`binding.pry` but for agents. For the full reference see the
|
|
224
|
+
[REPL section](https://r.uby.dev/llm/deepdive/fundamentals/repl) in the
|
|
225
|
+
deepdive.
|
|
185
226
|
|
|
186
227
|
```ruby
|
|
187
228
|
require "llm"
|
|
188
229
|
require "llm/tools"
|
|
189
230
|
|
|
190
231
|
llm = LLM.deepseek(key: ENV["KEY"])
|
|
191
|
-
agent = LLM::Agent.new(llm)
|
|
232
|
+
agent = LLM::Agent.new(llm, name: "my-agent", path: "agent.json")
|
|
192
233
|
agent.repl(tools: LLM::Tool.subclasses)
|
|
193
234
|
```
|
|
194
235
|
|
|
195
|
-
#####
|
|
236
|
+
##### CLI
|
|
196
237
|
|
|
197
|
-
The `
|
|
198
|
-
|
|
238
|
+
The `llm.rb` executable is available on your PATH after installation.
|
|
239
|
+
It starts a REPL session from any directory:
|
|
199
240
|
|
|
200
|
-
```
|
|
201
|
-
|
|
202
|
-
|
|
203
|
-
llm
|
|
204
|
-
agent = LLM::Agent.new(llm)
|
|
205
|
-
agent.repl(skills: [__dir__])
|
|
206
|
-
```
|
|
207
|
-
|
|
208
|
-
##### REPL: Tracer
|
|
209
|
-
|
|
210
|
-
By default the tracer is disabled for the duration of the
|
|
211
|
-
session. Setting `tracer: true` configures the REPL to use
|
|
212
|
-
the tracer associated with an instance of
|
|
213
|
-
[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html).
|
|
214
|
-
|
|
215
|
-
```ruby
|
|
216
|
-
require "llm"
|
|
217
|
-
|
|
218
|
-
llm = LLM.deepseek(key: ENV["KEY"])
|
|
219
|
-
tracer = LLM.logger(llm, path: "agent.log")
|
|
220
|
-
agent = LLM::Agent.new(llm, tracer:)
|
|
221
|
-
agent.repl(tracer: true, tools: [Debugger])
|
|
222
|
-
```
|
|
223
|
-
|
|
224
|
-
##### REPL: Commands
|
|
225
|
-
|
|
226
|
-
Commands are recognized by a `/` prefix and are backed by the
|
|
227
|
-
[`LLM::Repl::Command`](https://r.uby.dev/api-docs/llm.rb/LLM/Repl/Command.html)
|
|
228
|
-
class, which can be subclassed to add custom commands. Once you
|
|
229
|
-
create a subclass, it is automatically added to the repl. A command
|
|
230
|
-
can have zero or more parameters, and all parameters are presumed
|
|
231
|
-
to be a String (at least for now).
|
|
232
|
-
|
|
233
|
-
```ruby
|
|
234
|
-
require "llm"
|
|
235
|
-
require "llm/repl"
|
|
236
|
-
|
|
237
|
-
class Greeter < LLM::Command
|
|
238
|
-
name "greet"
|
|
239
|
-
description "Greets the given name"
|
|
240
|
-
parameter :name, String, "The person's name"
|
|
241
|
-
required %i[name]
|
|
242
|
-
|
|
243
|
-
def call(name:)
|
|
244
|
-
write("Welcome #{name}!\n")
|
|
245
|
-
end
|
|
246
|
-
end
|
|
241
|
+
```bash
|
|
242
|
+
llm.rb # auto-detect from $DEEPSEEK_API_KEY
|
|
243
|
+
llm.rb -p openai # use OpenAI explicitly
|
|
244
|
+
llm.rb -t # temporary session, no persistence
|
|
247
245
|
```
|
|
248
246
|
|
|
249
|
-
|
|
250
|
-
|
|
251
|
-
|
|
252
|
-
|
|
253
|
-
| Key | Action |
|
|
254
|
-
|---|---|
|
|
255
|
-
| `Enter` | Submit the current prompt |
|
|
256
|
-
| `Ctrl+A` | Jump to the start of the line |
|
|
257
|
-
| `Ctrl+E` | Jump to the end of the line |
|
|
258
|
-
| `Ctrl+F` | Move the cursor forward |
|
|
259
|
-
| `Ctrl+K` | Erase from cursor to the end of the line |
|
|
260
|
-
| `Ctrl+Y` | Paste previously killed text |
|
|
261
|
-
| `Ctrl+D` | Delete the character at the cursor |
|
|
262
|
-
| `Left / Right` | Move the cursor |
|
|
263
|
-
| `Up / Down` | Scroll the transcript |
|
|
264
|
-
| `/exit` | Leave the REPL |
|
|
247
|
+
The CLI auto-detects your provider from standard environment variables
|
|
248
|
+
(`DEEPSEEK_API_KEY`, `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, etc.).
|
|
249
|
+
Persistent sessions are stored under `~/.llm.rb/` and restored
|
|
250
|
+
automatically on your next visit.
|
|
265
251
|
|
|
266
252
|
#### LLM::MCP
|
|
267
253
|
|
|
268
254
|
The Model Context Protocol (MCP) has first-class support
|
|
269
255
|
in llm.rb. The stdio and http transports work out of the
|
|
270
256
|
box. MCP tools are translated into subclasses of
|
|
271
|
-
[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html) that can be
|
|
272
|
-
|
|
257
|
+
[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html) that can be
|
|
258
|
+
used with
|
|
259
|
+
[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html) or
|
|
260
|
+
[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html).
|
|
273
261
|
|
|
274
262
|
```ruby
|
|
275
263
|
require "llm"
|
|
@@ -285,8 +273,10 @@ agent.talk "Run the tool"
|
|
|
285
273
|
The Agent 2 Agent (A2A) protocol has first-class support
|
|
286
274
|
in llm.rb. The http and jsonrpc transports work out of the
|
|
287
275
|
box. A2A skills are translated into subclasses of
|
|
288
|
-
[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html) that can be
|
|
289
|
-
|
|
276
|
+
[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html) that can be
|
|
277
|
+
used with
|
|
278
|
+
[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html) or
|
|
279
|
+
[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html).
|
|
290
280
|
|
|
291
281
|
```ruby
|
|
292
282
|
require "llm"
|
|
@@ -297,6 +287,38 @@ agent = LLM::Agent.new(llm, stream: $stdout, tools: a2a.skills)
|
|
|
297
287
|
agent.talk "Run the skill"
|
|
298
288
|
```
|
|
299
289
|
|
|
290
|
+
#### LLM::Skill
|
|
291
|
+
|
|
292
|
+
A skill turns a markdown file into a callable tool. When the model
|
|
293
|
+
calls it, the runtime spawns a subagent with the skill's instructions
|
|
294
|
+
as its system prompt and the skill's own tool set. The subagent runs
|
|
295
|
+
one turn and returns the result, then is discarded. Each call
|
|
296
|
+
is fresh and stateless. For a deeper explanation see the
|
|
297
|
+
[deepdive.md](https://r.uby.dev/llm/deepdive/fundamentals/skills).
|
|
298
|
+
|
|
299
|
+
**SKILL.md**
|
|
300
|
+
|
|
301
|
+
```markdown
|
|
302
|
+
---
|
|
303
|
+
name: summary
|
|
304
|
+
description: Reads recent git history and writes a summary
|
|
305
|
+
tools: all
|
|
306
|
+
---
|
|
307
|
+
|
|
308
|
+
Collect the recent git log, analyze each commit,
|
|
309
|
+
and write a summary to summary.txt.
|
|
310
|
+
```
|
|
311
|
+
|
|
312
|
+
**agent.rb**
|
|
313
|
+
|
|
314
|
+
```ruby
|
|
315
|
+
require "llm"
|
|
316
|
+
|
|
317
|
+
llm = LLM.deepseek(key: ENV["KEY"])
|
|
318
|
+
agent = LLM::Agent.new(llm, skills: ["./skills/summary"])
|
|
319
|
+
agent.talk "Summarize the last week of work"
|
|
320
|
+
```
|
|
321
|
+
|
|
300
322
|
#### RAG
|
|
301
323
|
|
|
302
324
|
Most providers offer an embedding model that can be
|
|
@@ -317,6 +339,8 @@ llm = LLM.openai(key: ENV["KEY"])
|
|
|
317
339
|
body = "llm.rb is Ruby's capable AI runtime."
|
|
318
340
|
embedding = llm.embed([body]).embeddings.first
|
|
319
341
|
|
|
342
|
+
# Document is your ActiveRecord or Sequel model
|
|
343
|
+
# with a vector column (e.g. sqlite-vec or pgvector)
|
|
320
344
|
Document.create!(
|
|
321
345
|
title: "llm.rb",
|
|
322
346
|
body:,
|
|
@@ -326,16 +350,20 @@ Document.create!(
|
|
|
326
350
|
|
|
327
351
|
#### Concurrency
|
|
328
352
|
|
|
329
|
-
The runtime supports
|
|
353
|
+
The runtime supports six different concurrency strategies that have
|
|
330
354
|
different attributes. The choice between all of them often depends
|
|
331
355
|
on the requirements of your application.
|
|
332
356
|
|
|
333
|
-
IO-bound tools are a good fit for the `:
|
|
357
|
+
IO-bound tools are a good fit for the `:async`, `:thread`,
|
|
334
358
|
and `:fiber` strategies while true parallelism can be achieved
|
|
335
359
|
with the `:fork` and `:ractor` strategies. The
|
|
336
|
-
`:
|
|
360
|
+
`:sequential` strategy runs tools one at a time and is the default.
|
|
361
|
+
The `:fork` strategy also provides a separate process that offers
|
|
337
362
|
isolation from its parent.
|
|
338
363
|
|
|
364
|
+
You can learn more about the llm.rb concurrency model in the
|
|
365
|
+
[deepdive.md](https://r.uby.dev/llm/deepdive/fundamentals/concurrency).
|
|
366
|
+
|
|
339
367
|
```ruby
|
|
340
368
|
require "llm"
|
|
341
369
|
|
|
@@ -347,7 +375,9 @@ agent.talk "Run the tools in parallel"
|
|
|
347
375
|
|
|
348
376
|
#### ORM
|
|
349
377
|
|
|
350
|
-
Because both
|
|
378
|
+
Because both
|
|
379
|
+
[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html) and
|
|
380
|
+
[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
|
|
351
381
|
can be serialized to JSON and stored in a simple string, both ActiveRecord
|
|
352
382
|
and Sequel support can be implemented within a single column on a single row.
|
|
353
383
|
|
|
@@ -363,7 +393,8 @@ require "llm/active_record"
|
|
|
363
393
|
|
|
364
394
|
class Agent < ApplicationRecord
|
|
365
395
|
acts_as_agent
|
|
366
|
-
set
|
|
396
|
+
set name: "my-agent",
|
|
397
|
+
instructions: "solve the user's query",
|
|
367
398
|
model: "deepseek-v4-pro",
|
|
368
399
|
tools: [Research, FinalizeResearch, ActOnResearch]
|
|
369
400
|
|
|
@@ -433,9 +464,9 @@ In no particular order:
|
|
|
433
464
|
<summary>I have a limited budget. What should I do?</summary>
|
|
434
465
|
<br>
|
|
435
466
|
<p>
|
|
436
|
-
There a few options. The first option is to host
|
|
467
|
+
There are a few options. The first option is to host
|
|
437
468
|
your own model, and use the ollama or llamacpp
|
|
438
|
-
providers. This can be
|
|
469
|
+
providers. This can be difficult though because
|
|
439
470
|
a capable model requires hardware that can
|
|
440
471
|
match it. If you have the ability to self-host,
|
|
441
472
|
this would be my first option.
|
|
@@ -460,7 +491,7 @@ If you're on a budget, DeepSeek is hard to beat.
|
|
|
460
491
|
<details>
|
|
461
492
|
<summary>Can I download llm.rb via a decentralized network?</summary>
|
|
462
493
|
<br>
|
|
463
|
-
|
|
494
|
+
Yes.
|
|
464
495
|
<br>
|
|
465
496
|
We are on the <a href="https://radicle.network">radicle.network</a>
|
|
466
497
|
<br>
|
|
@@ -468,7 +499,9 @@ Every commit that lands on GitHub also lands on Radicle.
|
|
|
468
499
|
<br>
|
|
469
500
|
Our repository ID is z2PtfQ6dYwyYaW2aGrztG1sMyDmCE.
|
|
470
501
|
<br>
|
|
471
|
-
Browse on <a
|
|
502
|
+
Browse on <a
|
|
503
|
+
href="https://radicle.network/nodes/iris.radicle.network/z2PtfQ6dYwyYaW2aGrztG1sMyDmCE">the
|
|
504
|
+
web</a>.
|
|
472
505
|
</details>
|
|
473
506
|
|
|
474
507
|
## Resources
|
|
@@ -481,22 +514,5 @@ and resources.
|
|
|
481
514
|
|
|
482
515
|
## License
|
|
483
516
|
|
|
484
|
-
|
|
485
|
-
|
|
486
|
-
Commercial production use requires a commercial license.
|
|
487
|
-
<br>
|
|
488
|
-
Each version converts to the [BSD Zero Clause](https://choosealicense.com/licenses/0bsd/)
|
|
489
|
-
four years after its first public release.
|
|
490
|
-
<br>
|
|
491
|
-
Contact [robert@r.uby.dev](mailto:robert@r.uby.dev) for a commercial license.
|
|
492
|
-
|
|
493
|
-
### Waivers
|
|
494
|
-
|
|
495
|
-
Waivers are automatically granted for: <br>
|
|
496
|
-
|
|
497
|
-
* Personal use
|
|
498
|
-
* Students
|
|
499
|
-
* Teachers
|
|
500
|
-
* Evaluation, development, and testing
|
|
501
|
-
* Non-profits and charities
|
|
502
|
-
* Companies with less than or equal to 50 employees
|
|
517
|
+
This software is released under the terms of the MIT license. <br>
|
|
518
|
+
See [LICENSE](./LICENSE) for details.
|