llm.rb 12.6.0 → 13.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (88) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +571 -13
  3. data/LICENSE +21 -93
  4. data/README.md +183 -167
  5. data/bin/llm.rb +124 -0
  6. data/data/deepinfra.json +3 -0
  7. data/data/xai.json +1 -1
  8. data/lib/llm/a2a.rb +1 -1
  9. data/lib/llm/active_record/acts_as_llm.rb +6 -6
  10. data/lib/llm/agent.rb +136 -27
  11. data/lib/llm/buffer.rb +85 -3
  12. data/lib/llm/compactor/null.rb +19 -0
  13. data/lib/llm/compactor/truncate.rb +80 -0
  14. data/lib/llm/compactor.rb +42 -124
  15. data/lib/llm/context.rb +31 -37
  16. data/lib/llm/contract.rb +4 -25
  17. data/lib/llm/function/array.rb +18 -17
  18. data/lib/llm/function/async/group.rb +54 -0
  19. data/lib/llm/function/async/reactor.rb +48 -0
  20. data/lib/llm/function/async/task.rb +83 -0
  21. data/lib/llm/function/fiber/group.rb +46 -0
  22. data/lib/llm/function/fiber/task.rb +62 -0
  23. data/lib/llm/function/{fork_group.rb → fork/group.rb} +14 -5
  24. data/lib/llm/function/fork/job.rb +2 -2
  25. data/lib/llm/function/fork/task.rb +19 -10
  26. data/lib/llm/function/group.rb +40 -0
  27. data/lib/llm/function/{ractor_group.rb → ractor/group.rb} +13 -5
  28. data/lib/llm/function/ractor/job.rb +9 -3
  29. data/lib/llm/function/ractor/mailbox.rb +2 -0
  30. data/lib/llm/function/ractor/task.rb +23 -15
  31. data/lib/llm/function/{call_group.rb → sequential/group.rb} +12 -8
  32. data/lib/llm/function/sequential/task.rb +49 -0
  33. data/lib/llm/function/task.rb +25 -48
  34. data/lib/llm/function/thread/group.rb +46 -0
  35. data/lib/llm/function/thread/task.rb +60 -0
  36. data/lib/llm/function.rb +54 -65
  37. data/lib/llm/loop_guard.rb +1 -2
  38. data/lib/llm/mcp.rb +22 -0
  39. data/lib/llm/object.rb +2 -1
  40. data/lib/llm/provider.rb +6 -3
  41. data/lib/llm/providers/anthropic.rb +1 -1
  42. data/lib/llm/providers/bedrock/request_adapter.rb +1 -1
  43. data/lib/llm/providers/google.rb +2 -2
  44. data/lib/llm/providers/mistral.rb +1 -1
  45. data/lib/llm/providers/ollama.rb +1 -1
  46. data/lib/llm/providers/openai/responses.rb +1 -1
  47. data/lib/llm/providers/openai.rb +1 -1
  48. data/lib/llm/repl/{transcript.rb → buffer.rb} +34 -21
  49. data/lib/llm/repl/command.rb +47 -13
  50. data/lib/llm/repl/commands/compact.rb +33 -0
  51. data/lib/llm/repl/commands/help.rb +3 -5
  52. data/lib/llm/repl/input.rb +80 -15
  53. data/lib/llm/repl/markdown/table.rb +80 -0
  54. data/lib/llm/repl/markdown.rb +33 -3
  55. data/lib/llm/repl/node.rb +37 -0
  56. data/lib/llm/repl/status.rb +4 -4
  57. data/lib/llm/repl/stream.rb +12 -5
  58. data/lib/llm/repl/walker.rb +46 -0
  59. data/lib/llm/repl/window.rb +31 -32
  60. data/lib/llm/repl.rb +70 -38
  61. data/lib/llm/response.rb +10 -0
  62. data/lib/llm/schema/leaf.rb +5 -0
  63. data/lib/llm/schema/object.rb +11 -5
  64. data/lib/llm/sequel/plugin.rb +6 -6
  65. data/lib/llm/skill.rb +20 -4
  66. data/lib/llm/stream.rb +24 -17
  67. data/lib/llm/tool.rb +20 -4
  68. data/lib/llm/tools/chdir.rb +0 -2
  69. data/lib/llm/tools/{swap_text.rb → edit-file.rb} +3 -3
  70. data/lib/llm/tools/git.rb +11 -4
  71. data/lib/llm/tools/mkdir.rb +4 -1
  72. data/lib/llm/tools/pwd.rb +0 -2
  73. data/lib/llm/tools/read_file.rb +0 -2
  74. data/lib/llm/tools/rg.rb +11 -4
  75. data/lib/llm/tools/ruby.rb +46 -0
  76. data/lib/llm/tools/shell.rb +11 -4
  77. data/lib/llm/tools/utils.rb +31 -0
  78. data/lib/llm/tracer/pretty_logger.rb +127 -0
  79. data/lib/llm/tracer.rb +1 -0
  80. data/lib/llm/version.rb +1 -1
  81. data/lib/llm.rb +25 -5
  82. data/llm.gemspec +11 -5
  83. data/resources/deepdive.md +45 -1198
  84. metadata +39 -17
  85. data/lib/llm/function/call_task.rb +0 -46
  86. data/lib/llm/function/fiber_group.rb +0 -105
  87. data/lib/llm/function/task_group.rb +0 -97
  88. data/lib/llm/function/thread_group.rb +0 -102
data/LICENSE CHANGED
@@ -1,93 +1,21 @@
1
- Business Source License 1.1
2
-
3
- License text copyright (c) 2024 MariaDB plc, All Rights Reserved.
4
- "Business Source License" is a trademark of MariaDB plc.
5
-
6
- Parameters
7
-
8
- Licensor: Robert Gleeson
9
-
10
- Licensed Work: llm.rb
11
-
12
- Additional Use Grant:
13
- - Free for personal use
14
- - Free for students and their teachers
15
- - Free for evaluation, development, and testing
16
- - Free for non-profits and charities
17
- - Free for companies with less than or equal to 50 employees
18
-
19
- Change Date:
20
- Four years after the first public release of each specific version of the
21
- Licensed Work.
22
-
23
- Change License:
24
- BSD Zero Clause License (0BSD)
25
-
26
- Terms
27
-
28
- The Licensor hereby grants you the right to copy, modify, create derivative
29
- works, redistribute, and make non-production use of the Licensed Work. The
30
- Licensor may make an Additional Use Grant, above, permitting limited
31
- production use.
32
-
33
- Effective on the Change Date, or the fourth anniversary of the first publicly
34
- available distribution of a specific version of the Licensed Work under this
35
- License, whichever comes first, the Licensor hereby grants you rights under
36
- the terms of the Change License, and the rights granted in the paragraph
37
- above terminate.
38
-
39
- If your use of the Licensed Work does not comply with the requirements
40
- currently in effect as described in this License, you must purchase a
41
- commercial license from the Licensor, its affiliated entities, or authorized
42
- resellers, or you must refrain from using the Licensed Work.
43
-
44
- All copies of the original and modified Licensed Work, and derivative works
45
- of the Licensed Work, are subject to this License. This License applies
46
- separately for each version of the Licensed Work and the Change Date may vary
47
- for each version of the Licensed Work released by Licensor.
48
-
49
- You must conspicuously display this License on each original or modified copy
50
- of the Licensed Work. If you receive the Licensed Work in original or
51
- modified form from a third party, the terms and conditions set forth in this
52
- License apply to your use of that work.
53
-
54
- Any use of the Licensed Work in violation of this License will automatically
55
- terminate your rights under this License for the current and all other
56
- versions of the Licensed Work.
57
-
58
- This License does not grant you any right in any trademark or logo of
59
- Licensor or its affiliates (provided that you may use a trademark or logo of
60
- Licensor as expressly required by this License).
61
-
62
- TO THE EXTENT PERMITTED BY APPLICABLE LAW, THE LICENSED WORK IS PROVIDED ON
63
- AN "AS IS" BASIS. LICENSOR HEREBY DISCLAIMS ALL WARRANTIES AND CONDITIONS,
64
- EXPRESS OR IMPLIED, INCLUDING (WITHOUT LIMITATION) WARRANTIES OF
65
- MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT, AND
66
- TITLE.
67
-
68
- MariaDB hereby grants you permission to use this License's text to license
69
- your works, and to refer to it using the trademark "Business Source
70
- License", as long as you comply with the Covenants of Licensor below.
71
-
72
- Covenants of Licensor
73
-
74
- In consideration of the right to use this License's text and the "Business
75
- Source License" name and trademark, Licensor covenants to MariaDB, and to all
76
- other recipients of the licensed work to be provided by Licensor:
77
-
78
- To specify as the Change License the GPL Version 2.0 or any later version, or
79
- a license that is compatible with GPL Version 2.0 or a later version, where
80
- "compatible" means that software provided under the Change License can be
81
- included in a program with software provided under GPL Version 2.0 or a later
82
- version. Licensor may specify additional Change Licenses without limitation.
83
-
84
- To either: (a) specify an additional grant of rights to use that does not
85
- impose any additional restriction on the right granted in this License, as
86
- the Additional Use Grant; or (b) insert the text "None" to specify a Change
87
- Date. Not to modify this License in any other way.
88
-
89
- Notice
90
-
91
- The Business Source License (this document, or the "License") is not an Open
92
- Source license. However, the Licensed Work will eventually be made available
93
- under an Open Source License, as stated in this License.
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Robert Gleeson
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
data/README.md CHANGED
@@ -16,15 +16,19 @@ Welcome to the canonical llm.rb repository.
16
16
 
17
17
  llm.rb is an advanced runtime for building capable AI applications
18
18
  on CRuby. By default it has zero runtime dependencies although certain
19
- functionality – such as ActiveRecord support – require
19
+ functionality (such as ActiveRecord support) require
20
20
  optional dependencies that are opt-in.
21
21
 
22
+ When you want to learn more than what the README covers, checkout
23
+ the [deepdive.md](https://r.uby.dev/llm/deepdive/).
24
+
22
25
  ## Features
23
26
 
24
27
  The runtime supports OpenAI, OpenAI-compatible endpoints, Anthropic, Google
25
28
  Gemini, Mistral, DeepSeek, DeepInfra, xAI, Z.ai, AWS Bedrock, Ollama, and llama.cpp.
26
29
  It has first-class support for streaming, tool calls, MCP
27
- and A2A, embeddings, vector stores and the RAG pattern.
30
+ and A2A, embeddings, vector stores, OCR, context compaction,
31
+ and the RAG pattern.
28
32
 
29
33
  There are multiple HTTP backends to choose from, tools can be run concurrently
30
34
  or in parallel via threads, async tasks, fibers, ractors, and fork, and it is
@@ -32,7 +36,8 @@ also possible to make a tool call while the model is still streaming.
32
36
 
33
37
  The runtime builds on top of three core concepts: providers, contexts, and agents,
34
38
  so once you learn the fundamentals, everything else falls into place naturally. And once
35
- you learn llm.rb, you will also be able to use <a href="https://r.uby.dev/mruby-llm">mruby-llm</a> and
39
+ you learn llm.rb, you will also be able to use
40
+ <a href="https://r.uby.dev/mruby-llm">mruby-llm</a> and
36
41
  <a href="https://r.uby.dev/wasm-llm">wasm-llm</a> because the API is pretty much identical.
37
42
 
38
43
  ## Install
@@ -45,7 +50,9 @@ gem install llm.rb
45
50
 
46
51
  #### LLM::Agent
47
52
 
48
- The [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html) class is the default high-level interface,
53
+ The
54
+ [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
55
+ class is the default high-level interface,
49
56
  and it is recommended for most use-cases. It manages tool execution
50
57
  automatically, guards against infinite loops, manages conversation
51
58
  state, and much more.
@@ -58,13 +65,63 @@ agent = LLM::Agent.new(llm, stream: $stdout)
58
65
  agent.talk "Hello world"
59
66
  ```
60
67
 
68
+ ##### set
69
+
70
+ [`LLM::Agent.set`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#set-class_method)
71
+ is a class-level DSL that accepts a Hash of properties. Each key resolves to a
72
+ corresponding class accessor: `name`, `description`, `model`, `tools`,
73
+ `instructions`, `schema`, `stream`, `tracer`, `concurrency`, `confirm`,
74
+ `path`, and `skills`. All options are optional; zero or more can be set.
75
+ An error is raised for unknown keys so that typos are caught early.
76
+
77
+ ```ruby
78
+ class SystemAdmin < LLM::Agent
79
+ set name: "sysadmin",
80
+ description: "system administration agent",
81
+ model: "deepseek-v4-pro",
82
+ tools: [Shell]
83
+ end
84
+
85
+ llm = LLM.deepseek(key: ENV["KEY"])
86
+ agent = SystemAdmin.new(llm)
87
+ agent.talk "Run 'date'"
88
+ ```
89
+
90
+ ##### Persistence
91
+
92
+ Set `path:` on an agent for automatic filesystem persistence;
93
+ the agent restores conversation history from the file on startup
94
+ and saves it back after every turn, with no manual serialization
95
+ code. For database-backed persistence, ActiveRecord and Sequel
96
+ integrations are also available (see the
97
+ [database deepdive](https://r.uby.dev/llm/deepdive/advanced/database)
98
+ for details). All persistence options use the same underlying
99
+ serialization.
100
+
101
+ ```ruby
102
+ require "llm"
103
+
104
+ llm = LLM.deepseek(key: ENV["KEY"])
105
+ agent = LLM::Agent.new(llm, path: "session.json")
106
+ agent.talk "remember my name is robert"
107
+
108
+ # Next time, the conversation is restored automatically:
109
+ agent = LLM::Agent.new(llm, path: "session.json")
110
+ agent.talk "what's my name?"
111
+ ```
112
+
61
113
  #### LLM::Context
62
114
 
63
- The [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html) class is at the heart of the runtime
64
- and it is what [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html) uses under the hood.
115
+ The
116
+ [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
117
+ class is at the heart of the runtime
118
+ and it is what
119
+ [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
120
+ uses under the hood.
65
121
  It requires that the tool call loop be managed manually -
66
122
  sometimes that can be useful, but usually for advanced use-cases.
67
- If you're new to llm.rb, try [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html) first.
123
+ If you're new to llm.rb, try
124
+ [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html) first.
68
125
 
69
126
  ```ruby
70
127
  require "llm"
@@ -76,7 +133,9 @@ ctx.talk "Hello world"
76
133
 
77
134
  #### LLM::Tool
78
135
 
79
- Subclasses of [`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html) are plain Ruby classes with
136
+ Subclasses of
137
+ [`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html)
138
+ are plain Ruby classes with
80
139
  an optional set of typed parameters. <br> The model can choose to
81
140
  call them on your behalf, and they're one of the most powerful features
82
141
  for extending the feature set or abilities of a model.
@@ -97,7 +156,8 @@ end
97
156
  #### LLM::Stream
98
157
 
99
158
  Streams can be simple IO objects or subclasses of
100
- [`LLM::Stream`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html) with structured callbacks for content,
159
+ [`LLM::Stream`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html)
160
+ with structured callbacks for content,
101
161
  reasoning, tool calls, tool returns, and compaction.
102
162
 
103
163
  ```ruby
@@ -116,160 +176,88 @@ agent = LLM::Agent.new(llm, stream: MyStream.new)
116
176
  agent.talk "Explain Ruby fibers."
117
177
  ```
118
178
 
119
- #### LLM::REPL
120
-
121
- The [LLM::Agent#repl](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#repl-instance_method)
122
- method allows an agent to spawn a read-eval-print loop
123
- that can be useful while developing or operating agents.
124
- It can be used to debug tool calls, confirm an
125
- agent has done what was expected, or improve an agent by
126
- asking questions about what it has done up to that point.
127
-
128
- This feature requires that the [curses](https://github.com/ruby/curses)
129
- and [kramdown](https://github.com/gettalong/kramdown) libraries are
130
- installed and available to require.
131
-
132
- The TUI displays a status line with a context-usage bar and cost
133
- counter, a scrollable transcript with markdown rendering, and a
134
- multi-line input area. The UI stays responsive while the model
135
- is generating a response.
136
-
137
- ##### REPL: Agent
138
-
139
- A REPL session is started by calling `repl` on any agent
140
- instance. The session inherits the agent's model, tools,
141
- skills, and instructions.
142
-
143
- ```ruby
144
- require "llm"
145
-
146
- llm = LLM.deepseek(key: ENV["KEY"])
147
- agent = LLM::Agent.new(llm)
148
- agent.repl
149
- ```
150
-
151
- ##### REPL: State
152
-
153
- The `path:` option accepts a file path where runtime state
154
- is read from and written to. This lets you resume a
155
- conversation across REPL sessions.
179
+ #### LLM::Schema
180
+
181
+ [`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html)
182
+ subclasses produce typed, structured
183
+ output from any model call. Pass a schema to `LLM::Context#talk`,
184
+ `LLM::Agent#talk`, or `LLM::Provider#complete` to receive validated
185
+ JSON instead of free text. Schemas work alongside tools and streams.
186
+
187
+ [`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html)
188
+ can define objects, arrays, enums, nested schemas,
189
+ and more. It is also used internally by
190
+ [`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html) for parameter
191
+ definitions, so you already benefit from it when you declare tool
192
+ parameters.
193
+
194
+ The
195
+ [`LLM::DeepSeek`](https://r.uby.dev/api-docs/llm.rb/LLM/DeepSeek.html)
196
+ provider includes runtime-level optimisations such as structured
197
+ output support (despite no official structured outputs API) and
198
+ SVG image generation. This example uses
199
+ [`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html) with
200
+ DeepSeek:
156
201
 
157
202
  ```ruby
158
- require "llm"
203
+ class Weather < LLM::Schema
204
+ property :city, String, "The city name"
205
+ property :temperature, Float, "Current temperature"
206
+ property :conditions, String, "Weather conditions"
207
+ required %i[city temperature conditions]
208
+ end
159
209
 
160
210
  llm = LLM.deepseek(key: ENV["KEY"])
161
- agent = LLM::Agent.new(llm)
162
- agent.repl(path: "session.json")
211
+ agent = LLM::Agent.new(llm, schema: Weather)
212
+ res = agent.talk "Weather in Paris?"
213
+ res.content! # => {city: "Paris", temperature: 15.0, conditions: "Cloudy"}
163
214
  ```
164
215
 
165
- ##### REPL: Tools
166
-
167
- The `tools` option lets you attach additional tools
168
- for the duration of the session. This is in addition to
169
- any tools that might already be associated with an agent.
170
-
171
- A number of optional tools are distributed as part of
172
- llm.rb. They power the agents that can be found in the
173
- [agents/](agents/) directory.
174
-
175
- ```ruby
176
- require "llm"
177
-
178
- llm = LLM.deepseek(key: ENV["KEY"])
179
- agent = LLM::Agent.new(llm)
180
- agent.repl(tools: [Debugger])
181
- ```
216
+ #### LLM::REPL
182
217
 
183
- The following example starts a read-eval-print loop
184
- with all of the builtin tools available.
218
+ The [LLM::Agent#repl](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#repl-instance_method)
219
+ method drops you into a curses-based TUI for talking to an
220
+ agent interactively. Set `path:` on the agent for automatic
221
+ persistence across REPL sessions. The `tools:` option attaches
222
+ extra tools for the duration of the session. It is like
223
+ `binding.pry` but for agents. For the full reference see the
224
+ [REPL section](https://r.uby.dev/llm/deepdive/fundamentals/repl) in the
225
+ deepdive.
185
226
 
186
227
  ```ruby
187
228
  require "llm"
188
229
  require "llm/tools"
189
230
 
190
231
  llm = LLM.deepseek(key: ENV["KEY"])
191
- agent = LLM::Agent.new(llm)
232
+ agent = LLM::Agent.new(llm, name: "my-agent", path: "agent.json")
192
233
  agent.repl(tools: LLM::Tool.subclasses)
193
234
  ```
194
235
 
195
- ##### REPL: Skills
236
+ ##### CLI
196
237
 
197
- The `skills` option lets you load extra skill directories
198
- without attaching them to an agent permanently.
238
+ The `llm.rb` executable is available on your PATH after installation.
239
+ It starts a REPL session from any directory:
199
240
 
200
- ```ruby
201
- require "llm"
202
-
203
- llm = LLM.deepseek(key: ENV["KEY"])
204
- agent = LLM::Agent.new(llm)
205
- agent.repl(skills: [__dir__])
206
- ```
207
-
208
- ##### REPL: Tracer
209
-
210
- By default the tracer is disabled for the duration of the
211
- session. Setting `tracer: true` configures the REPL to use
212
- the tracer associated with an instance of
213
- [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html).
214
-
215
- ```ruby
216
- require "llm"
217
-
218
- llm = LLM.deepseek(key: ENV["KEY"])
219
- tracer = LLM.logger(llm, path: "agent.log")
220
- agent = LLM::Agent.new(llm, tracer:)
221
- agent.repl(tracer: true, tools: [Debugger])
222
- ```
223
-
224
- ##### REPL: Commands
225
-
226
- Commands are recognized by a `/` prefix and are backed by the
227
- [`LLM::Repl::Command`](https://r.uby.dev/api-docs/llm.rb/LLM/Repl/Command.html)
228
- class, which can be subclassed to add custom commands. Once you
229
- create a subclass, it is automatically added to the repl. A command
230
- can have zero or more parameters, and all parameters are presumed
231
- to be a String (at least for now).
232
-
233
- ```ruby
234
- require "llm"
235
- require "llm/repl"
236
-
237
- class Greeter < LLM::Command
238
- name "greet"
239
- description "Greets the given name"
240
- parameter :name, String, "The person's name"
241
- required %i[name]
242
-
243
- def call(name:)
244
- write("Welcome #{name}!\n")
245
- end
246
- end
241
+ ```bash
242
+ llm.rb # auto-detect from $DEEPSEEK_API_KEY
243
+ llm.rb -p openai # use OpenAI explicitly
244
+ llm.rb -t # temporary session, no persistence
247
245
  ```
248
246
 
249
- ##### REPL: Input
250
-
251
- The input area supports several keyboard shortcuts:
252
-
253
- | Key | Action |
254
- |---|---|
255
- | `Enter` | Submit the current prompt |
256
- | `Ctrl+A` | Jump to the start of the line |
257
- | `Ctrl+E` | Jump to the end of the line |
258
- | `Ctrl+F` | Move the cursor forward |
259
- | `Ctrl+K` | Erase from cursor to the end of the line |
260
- | `Ctrl+Y` | Paste previously killed text |
261
- | `Ctrl+D` | Delete the character at the cursor |
262
- | `Left / Right` | Move the cursor |
263
- | `Up / Down` | Scroll the transcript |
264
- | `/exit` | Leave the REPL |
247
+ The CLI auto-detects your provider from standard environment variables
248
+ (`DEEPSEEK_API_KEY`, `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, etc.).
249
+ Persistent sessions are stored under `~/.llm.rb/` and restored
250
+ automatically on your next visit.
265
251
 
266
252
  #### LLM::MCP
267
253
 
268
254
  The Model Context Protocol (MCP) has first-class support
269
255
  in llm.rb. The stdio and http transports work out of the
270
256
  box. MCP tools are translated into subclasses of
271
- [`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html) that can be used with [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
272
- or [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html).
257
+ [`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html) that can be
258
+ used with
259
+ [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html) or
260
+ [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html).
273
261
 
274
262
  ```ruby
275
263
  require "llm"
@@ -285,8 +273,10 @@ agent.talk "Run the tool"
285
273
  The Agent 2 Agent (A2A) protocol has first-class support
286
274
  in llm.rb. The http and jsonrpc transports work out of the
287
275
  box. A2A skills are translated into subclasses of
288
- [`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html) that can be used with [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
289
- or [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html).
276
+ [`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html) that can be
277
+ used with
278
+ [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html) or
279
+ [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html).
290
280
 
291
281
  ```ruby
292
282
  require "llm"
@@ -297,6 +287,38 @@ agent = LLM::Agent.new(llm, stream: $stdout, tools: a2a.skills)
297
287
  agent.talk "Run the skill"
298
288
  ```
299
289
 
290
+ #### LLM::Skill
291
+
292
+ A skill turns a markdown file into a callable tool. When the model
293
+ calls it, the runtime spawns a subagent with the skill's instructions
294
+ as its system prompt and the skill's own tool set. The subagent runs
295
+ one turn and returns the result, then is discarded. Each call
296
+ is fresh and stateless. For a deeper explanation see the
297
+ [deepdive.md](https://r.uby.dev/llm/deepdive/fundamentals/skills).
298
+
299
+ **SKILL.md**
300
+
301
+ ```markdown
302
+ ---
303
+ name: summary
304
+ description: Reads recent git history and writes a summary
305
+ tools: all
306
+ ---
307
+
308
+ Collect the recent git log, analyze each commit,
309
+ and write a summary to summary.txt.
310
+ ```
311
+
312
+ **agent.rb**
313
+
314
+ ```ruby
315
+ require "llm"
316
+
317
+ llm = LLM.deepseek(key: ENV["KEY"])
318
+ agent = LLM::Agent.new(llm, skills: ["./skills/summary"])
319
+ agent.talk "Summarize the last week of work"
320
+ ```
321
+
300
322
  #### RAG
301
323
 
302
324
  Most providers offer an embedding model that can be
@@ -317,6 +339,8 @@ llm = LLM.openai(key: ENV["KEY"])
317
339
  body = "llm.rb is Ruby's capable AI runtime."
318
340
  embedding = llm.embed([body]).embeddings.first
319
341
 
342
+ # Document is your ActiveRecord or Sequel model
343
+ # with a vector column (e.g. sqlite-vec or pgvector)
320
344
  Document.create!(
321
345
  title: "llm.rb",
322
346
  body:,
@@ -326,16 +350,20 @@ Document.create!(
326
350
 
327
351
  #### Concurrency
328
352
 
329
- The runtime supports five different concurrency strategies that have
353
+ The runtime supports six different concurrency strategies that have
330
354
  different attributes. The choice between all of them often depends
331
355
  on the requirements of your application.
332
356
 
333
- IO-bound tools are a good fit for the `:task`, `:thread`,
357
+ IO-bound tools are a good fit for the `:async`, `:thread`,
334
358
  and `:fiber` strategies while true parallelism can be achieved
335
359
  with the `:fork` and `:ractor` strategies. The
336
- `:fork` strategy also provides a separate process that offers
360
+ `:sequential` strategy runs tools one at a time and is the default.
361
+ The `:fork` strategy also provides a separate process that offers
337
362
  isolation from its parent.
338
363
 
364
+ You can learn more about the llm.rb concurrency model in the
365
+ [deepdive.md](https://r.uby.dev/llm/deepdive/fundamentals/concurrency).
366
+
339
367
  ```ruby
340
368
  require "llm"
341
369
 
@@ -347,7 +375,9 @@ agent.talk "Run the tools in parallel"
347
375
 
348
376
  #### ORM
349
377
 
350
- Because both [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html), and [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
378
+ Because both
379
+ [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html) and
380
+ [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
351
381
  can be serialized to JSON and stored in a simple string, both ActiveRecord
352
382
  and Sequel support can be implemented within a single column on a single row.
353
383
 
@@ -363,7 +393,8 @@ require "llm/active_record"
363
393
 
364
394
  class Agent < ApplicationRecord
365
395
  acts_as_agent
366
- set instructions: "solve the user's query",
396
+ set name: "my-agent",
397
+ instructions: "solve the user's query",
367
398
  model: "deepseek-v4-pro",
368
399
  tools: [Research, FinalizeResearch, ActOnResearch]
369
400
 
@@ -433,9 +464,9 @@ In no particular order:
433
464
  <summary>I have a limited budget. What should I do?</summary>
434
465
  <br>
435
466
  <p>
436
- There a few options. The first option is to host
467
+ There are a few options. The first option is to host
437
468
  your own model, and use the ollama or llamacpp
438
- providers. This can be diffilcult though because
469
+ providers. This can be difficult though because
439
470
  a capable model requires hardware that can
440
471
  match it. If you have the ability to self-host,
441
472
  this would be my first option.
@@ -460,7 +491,7 @@ If you're on a budget, DeepSeek is hard to beat.
460
491
  <details>
461
492
  <summary>Can I download llm.rb via a decentralized network?</summary>
462
493
  <br>
463
- You can!
494
+ Yes.
464
495
  <br>
465
496
  We are on the <a href="https://radicle.network">radicle.network</a>
466
497
  <br>
@@ -468,7 +499,9 @@ Every commit that lands on GitHub also lands on Radicle.
468
499
  <br>
469
500
  Our repository ID is z2PtfQ6dYwyYaW2aGrztG1sMyDmCE.
470
501
  <br>
471
- Browse on <a href="https://radicle.network/nodes/iris.radicle.network/z2PtfQ6dYwyYaW2aGrztG1sMyDmCE">the web</a>.
502
+ Browse on <a
503
+ href="https://radicle.network/nodes/iris.radicle.network/z2PtfQ6dYwyYaW2aGrztG1sMyDmCE">the
504
+ web</a>.
472
505
  </details>
473
506
 
474
507
  ## Resources
@@ -481,22 +514,5 @@ and resources.
481
514
 
482
515
  ## License
483
516
 
484
- [Business Source License 1.1](./LICENSE)
485
- <br>
486
- Commercial production use requires a commercial license.
487
- <br>
488
- Each version converts to the [BSD Zero Clause](https://choosealicense.com/licenses/0bsd/)
489
- four years after its first public release.
490
- <br>
491
- Contact [robert@r.uby.dev](mailto:robert@r.uby.dev) for a commercial license.
492
-
493
- ### Waivers
494
-
495
- Waivers are automatically granted for: <br>
496
-
497
- * Personal use
498
- * Students
499
- * Teachers
500
- * Evaluation, development, and testing
501
- * Non-profits and charities
502
- * Companies with less than or equal to 50 employees
517
+ This software is released under the terms of the MIT license. <br>
518
+ See [LICENSE](./LICENSE) for details.