llm.rb 13.0.0 → 13.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +185 -14
- data/README.md +155 -30
- data/bin/llm.rb +124 -0
- data/lib/llm/agent.rb +78 -8
- data/lib/llm/function/array.rb +3 -3
- data/lib/llm/function.rb +1 -2
- data/lib/llm/providers/anthropic.rb +1 -1
- data/lib/llm/providers/bedrock/request_adapter.rb +1 -1
- data/lib/llm/providers/mistral.rb +1 -1
- data/lib/llm/providers/ollama.rb +1 -1
- data/lib/llm/providers/openai/responses.rb +1 -1
- data/lib/llm/providers/openai.rb +1 -1
- data/lib/llm/repl/{transcript.rb → buffer.rb} +34 -21
- data/lib/llm/repl/command.rb +12 -5
- data/lib/llm/repl/commands/compact.rb +2 -2
- data/lib/llm/repl/commands/help.rb +3 -5
- data/lib/llm/repl/markdown/table.rb +7 -3
- data/lib/llm/repl/markdown.rb +3 -3
- data/lib/llm/repl/node.rb +37 -0
- data/lib/llm/repl/status.rb +4 -4
- data/lib/llm/repl/stream.rb +2 -2
- data/lib/llm/repl/window.rb +31 -32
- data/lib/llm/repl.rb +56 -30
- data/lib/llm/skill.rb +20 -4
- data/lib/llm/tools/{swap_text.rb → edit-file.rb} +3 -3
- data/lib/llm/tools/git.rb +3 -0
- data/lib/llm/tools/mkdir.rb +3 -0
- data/lib/llm/tools/rg.rb +3 -0
- data/lib/llm/tools/ruby.rb +46 -0
- data/lib/llm/tools/shell.rb +3 -0
- data/lib/llm/tracer/pretty_logger.rb +127 -0
- data/lib/llm/tracer.rb +1 -0
- data/lib/llm/version.rb +1 -1
- data/llm.gemspec +8 -2
- data/resources/deepdive.md +45 -1785
- metadata +16 -5
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: bb5c6136080e0136309987a4e042e6f646c26de39e38d6422bb3904fc665bf5f
|
|
4
|
+
data.tar.gz: d832b9c19f4173ba6e5631ffdc57f495eb4a9a8fb0b657801de185707bc4085e
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 7ba4417a8c5b48a28b2baf04a2960374eae28c44dc2ec19e8f79da9a95f75fdf8cb5f1c749e6bfa1b1f6d1c790723e2005cb576f94f118997bb5e8b0f5d53a63
|
|
7
|
+
data.tar.gz: 0105af6bcaaec50930546e3a5a7207b8cf65e30bbea33359784061159748a3e2f3afb6e2b599eaad904e0c477d9827bb444c698315e56d43572be6ba86fe98b8
|
data/CHANGELOG.md
CHANGED
|
@@ -15,7 +15,178 @@
|
|
|
15
15
|
|
|
16
16
|
## What's next
|
|
17
17
|
|
|
18
|
-
|
|
18
|
+
*No unreleased changes yet. Check back after the next release.*
|
|
19
|
+
|
|
20
|
+
## v13.1.0
|
|
21
|
+
|
|
22
|
+
Changes since `v13.0.0`.
|
|
23
|
+
|
|
24
|
+
This release adds `LLM::Agent` class DSL attributes (`path`, `description`),
|
|
25
|
+
extends skills with file-path loading and the `tools: all` directive, adds new
|
|
26
|
+
built-in tools (`LLM::Tool::Ruby`, `LLM::Tool::EditFile`), introduces the
|
|
27
|
+
`LLM::Tracer::PrettyLogger` for human-readable tracing, renames `Transcript`
|
|
28
|
+
to `Buffer` across the REPL, ships a `bin/llm.rb` CLI entry point, and fixes
|
|
29
|
+
several agent and tool bugs around persistence, interruption, and naming.
|
|
30
|
+
|
|
31
|
+
### Core
|
|
32
|
+
|
|
33
|
+
* **add post install message with deepdive link** <br>
|
|
34
|
+
The gemspec now includes a `post_install_message` that points users to
|
|
35
|
+
the deepdive guide at `https://r.uby.dev/llm/deepdive` after installation,
|
|
36
|
+
making it easier for new users to discover the project documentation.
|
|
37
|
+
|
|
38
|
+
### Agent
|
|
39
|
+
|
|
40
|
+
* **add `description` class DSL and instance method** <br>
|
|
41
|
+
[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html) now
|
|
42
|
+
has a `description` class DSL (`description "release engineer"`) and a
|
|
43
|
+
corresponding `#description` instance method. The description is an
|
|
44
|
+
optional self-documenting string that serves as a brief summary of the
|
|
45
|
+
agent's purpose. It can be set via the class DSL,
|
|
46
|
+
`LLM::Agent.set(description: ...)`, or `LLM::Agent.new(description: ...)`.
|
|
47
|
+
|
|
48
|
+
* **add `path` class DSL and instance method** <br>
|
|
49
|
+
[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html) now
|
|
50
|
+
has a `path` class DSL (`path "contexts/admin.json"`) and a
|
|
51
|
+
corresponding `#path` instance method. When a path is set, the agent
|
|
52
|
+
automatically restores its conversation history from that file on
|
|
53
|
+
initialization and saves it back after each `talk` or `ask` turn,
|
|
54
|
+
making session persistence across process restarts transparent.
|
|
55
|
+
|
|
56
|
+
### Skills
|
|
57
|
+
|
|
58
|
+
* **accept a path to a markdown file** <br>
|
|
59
|
+
[`LLM::Skill.load`](https://r.uby.dev/api-docs/llm.rb/LLM/Skill.html#load-class_method)
|
|
60
|
+
now accepts a path to a markdown file in addition to a directory path.
|
|
61
|
+
When given a file path, the file is read directly instead of looking for
|
|
62
|
+
a `SKILL.md` inside a directory. This makes it possible to load a single
|
|
63
|
+
markdown file as a skill without placing it in a dedicated directory.
|
|
64
|
+
|
|
65
|
+
* **extend with `all` keyword for loading the full tool registry** <br>
|
|
66
|
+
`LLM::Skill` now supports `tools: all` (or `tools: "*"`) in the frontmatter
|
|
67
|
+
to load all tools from the global
|
|
68
|
+
[`LLM::Tool.registry`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#registry-class_method).
|
|
69
|
+
Previously, the `tools:` frontmatter only accepted `inherit`, an array of tool
|
|
70
|
+
names, or nothing. The new `all` keyword makes it possible to give a skill
|
|
71
|
+
access to every registered tool without listing them individually.
|
|
72
|
+
|
|
73
|
+
### Tools
|
|
74
|
+
|
|
75
|
+
* **add `LLM::Tool::Ruby` for executing Ruby code in a subprocess** <br>
|
|
76
|
+
[`LLM::Tool::Ruby`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Ruby.html)
|
|
77
|
+
is a new built-in tool that runs a string of Ruby code in a separate
|
|
78
|
+
Ruby process with a configurable timeout (default 15s). The code runs
|
|
79
|
+
in an isolated address space unaware of its parent, making it useful
|
|
80
|
+
for safe(ish) dynamic code execution. It must be required explicitly
|
|
81
|
+
with `require "llm/tools/ruby"` and requires the `test-cmd.rb` gem.
|
|
82
|
+
|
|
83
|
+
* **rename `LLM::Tool::SwapText` to `LLM::Tool::EditFile`** <br>
|
|
84
|
+
The `SwapText` tool has been renamed to
|
|
85
|
+
[`LLM::Tool::EditFile`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/EditFile.html)
|
|
86
|
+
to better match the naming of sibling tools (`ReadFile`, `WriteFile`).
|
|
87
|
+
The old `require "llm/tools/swap_text"` path no longer exists; use
|
|
88
|
+
`require "llm/tools/edit-file"` instead.
|
|
89
|
+
|
|
90
|
+
### Tracer
|
|
91
|
+
|
|
92
|
+
* **add `LLM::Tracer::PrettyLogger` for human-readable tracing** <br>
|
|
93
|
+
[`LLM::Tracer::PrettyLogger`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/PrettyLogger.html)
|
|
94
|
+
is a new tracer that writes human-readable request and tool-call logs to a
|
|
95
|
+
console or file. Unlike the structured JSON output of
|
|
96
|
+
`LLM::Tracer::Logger`, the pretty logger emits single-line entries with
|
|
97
|
+
inline context, making it easier to follow agent activity at a glance.
|
|
98
|
+
It writes to `$stderr` by default and accepts an `io:` option for file
|
|
99
|
+
output.
|
|
100
|
+
|
|
101
|
+
### Repl
|
|
102
|
+
|
|
103
|
+
* **rename `LLM::Repl::Transcript` to `LLM::Repl::Buffer`** <br>
|
|
104
|
+
`LLM::Repl::Transcript` has been renamed to
|
|
105
|
+
[`LLM::Repl::Buffer`](https://r.uby.dev/api-docs/llm.rb/LLM/Repl/Buffer.html)
|
|
106
|
+
to better reflect its role as a conversation state manager. The old
|
|
107
|
+
`start` and `finish` methods have been renamed to `open` and `close`
|
|
108
|
+
respectively. The public accessor on
|
|
109
|
+
[`LLM::Repl`](https://r.uby.dev/api-docs/llm.rb/LLM/Repl.html) has been
|
|
110
|
+
renamed from `transcript` to `buffer`.
|
|
111
|
+
|
|
112
|
+
* **add `write_message` for formatted message writing** <br>
|
|
113
|
+
[`LLM::Repl::Buffer#write_message`](https://r.uby.dev/api-docs/llm.rb/LLM/Repl/Buffer.html#write_message-instance_method)
|
|
114
|
+
and
|
|
115
|
+
[`LLM::Repl#write_message`](https://r.uby.dev/api-docs/llm.rb/LLM/Repl.html#write_message-instance_method)
|
|
116
|
+
provide a convenience method that takes a username and content string,
|
|
117
|
+
formatting the output with a bold `user:` label and a trailing newline.
|
|
118
|
+
This is simpler than the equivalent sequence of `write` calls.
|
|
119
|
+
|
|
120
|
+
* **add `Command#write_message` and refactor `Command#write`** <br>
|
|
121
|
+
[`LLM::Command#write_message`](https://r.uby.dev/api-docs/llm.rb/LLM/Repl/Command.html#write_message-instance_method)
|
|
122
|
+
provides a convenience method that takes a username and content string,
|
|
123
|
+
matching the same interface on `LLM::Repl` and `LLM::Buffer`. The
|
|
124
|
+
`Command#write` method is now implemented on top of `write_message`,
|
|
125
|
+
always prefixing output with `command(<name>): `. The `who:` keyword
|
|
126
|
+
argument previously accepted by `write` has been removed; use
|
|
127
|
+
`write_message` instead.
|
|
128
|
+
|
|
129
|
+
* **display pre-existing agent messages when the repl starts** <br>
|
|
130
|
+
When
|
|
131
|
+
[`LLM::Agent#repl`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#repl-instance_method)
|
|
132
|
+
starts, any messages already in the agent's buffer are now rendered
|
|
133
|
+
in the REPL window. Previously the REPL started with an empty
|
|
134
|
+
transcript even when the agent carried prior conversation history,
|
|
135
|
+
making it harder to resume a session. Tool-call and tool-return
|
|
136
|
+
messages are skipped to avoid visual noise.
|
|
137
|
+
|
|
138
|
+
### CLI
|
|
139
|
+
|
|
140
|
+
* **add `bin/llm.rb` for launching the REPL from the command line** <br>
|
|
141
|
+
A new executable script (`bin/llm.rb`) provides a convenient way to start
|
|
142
|
+
an interactive REPL session directly from the terminal. It auto-detects
|
|
143
|
+
the provider from environment variables like `OPENAI_API_KEY`, supports
|
|
144
|
+
a `-p PROVIDER` flag for explicit provider selection, a `-t` flag for
|
|
145
|
+
temporary (non-persistent) sessions, and `-h` for help. Sessions are
|
|
146
|
+
automatically saved to `~/.llm.rb/` by default.
|
|
147
|
+
|
|
148
|
+
### Fix
|
|
149
|
+
|
|
150
|
+
* **agent: fix `path` restore on first run** <br>
|
|
151
|
+
Fix a bug where [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
|
|
152
|
+
called `@ctx.restore(path:)` even when the path's file did not exist.
|
|
153
|
+
The fix checks `File.readable?(@path)` before attempting to restore,
|
|
154
|
+
so the agent starts with a blank conversation on first use instead of
|
|
155
|
+
failing with a file-not-found error.
|
|
156
|
+
|
|
157
|
+
* **tools: re-raise `LLM::Interrupt` to abort the turn** <br>
|
|
158
|
+
[`LLM::Tool::Git`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Git.html),
|
|
159
|
+
[`LLM::Tool::Mkdir`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Mkdir.html),
|
|
160
|
+
[`LLM::Tool::Rg`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Rg.html),
|
|
161
|
+
[`LLM::Tool::Ruby`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Ruby.html),
|
|
162
|
+
and
|
|
163
|
+
[`LLM::Tool::Shell`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Shell.html)
|
|
164
|
+
now re-raise `LLM::Interrupt` after killing their running command. The
|
|
165
|
+
previous behavior rescued the interrupt and killed the child process but
|
|
166
|
+
let the turn continue, which meant a cancelled tool call did not abort
|
|
167
|
+
the conversation turn. Re-raising ensures the entire turn is interrupted.
|
|
168
|
+
|
|
169
|
+
* **tools: rescue `LLM::Interrupt` in shell-based tools** <br>
|
|
170
|
+
[`LLM::Tool::Shell`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Shell.html),
|
|
171
|
+
[`LLM::Tool::Git`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Git.html),
|
|
172
|
+
[`LLM::Tool::Mkdir`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Mkdir.html),
|
|
173
|
+
and
|
|
174
|
+
[`LLM::Tool::Rg`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Rg.html)
|
|
175
|
+
now rescue `LLM::Interrupt` and kill their running command, preventing
|
|
176
|
+
orphaned child processes when a tool is interrupted during execution.
|
|
177
|
+
|
|
178
|
+
* **agent: fix default name derivation** <br>
|
|
179
|
+
Fix a bug where `LLM::Agent` used without a subclass derived its default
|
|
180
|
+
name as `"l-lm-agent"` instead of `"agent"`. The fix replaces the
|
|
181
|
+
regex-based parameterization with a pattern that correctly handles
|
|
182
|
+
single-word class names and multi-word namespaced names.
|
|
183
|
+
|
|
184
|
+
* **function: `#params` always returns an `LLM::Object`** <br>
|
|
185
|
+
[`LLM::Function#params`](https://r.uby.dev/api-docs/llm.rb/LLM/Function.html#params-instance_method)
|
|
186
|
+
now always returns an `LLM::Object` representing the function's parameter
|
|
187
|
+
schema. Previously it returned `nil` when a function defined no parameters,
|
|
188
|
+
forcing every caller to guard against `nil`. All provider adapters now use
|
|
189
|
+
`fn.params.to_h` instead of `fn.params || {type: "object", properties: {}}`.
|
|
19
190
|
|
|
20
191
|
## v13.0.0
|
|
21
192
|
|
|
@@ -406,7 +577,7 @@ reliable across all six concurrency backends. The `functions` and
|
|
|
406
577
|
|
|
407
578
|
Changes since `v12.5.1`.
|
|
408
579
|
|
|
409
|
-
This release adds bulk defaults for tools and agents
|
|
580
|
+
This release adds bulk defaults for tools and agents: `LLM::Tool.defaults`
|
|
410
581
|
for setting parameter defaults and `LLM::Agent.set` for mass-assigning
|
|
411
582
|
class-level defaults, both mirrored on ActiveRecord and Sequel agent models.
|
|
412
583
|
|
|
@@ -442,7 +613,7 @@ prior call.
|
|
|
442
613
|
On cancel, `LLM::Interrupt` is now raised on the thread that is
|
|
443
614
|
running a tool. The tool can rescue `LLM::Interrupt` and gracefully
|
|
444
615
|
terminate (e.g., clean up resources). The previous approach used
|
|
445
|
-
`Thread#interrupt` which was less reliable
|
|
616
|
+
`Thread#interrupt` which was less reliable. It did not interrupt a
|
|
446
617
|
sleeping thread.
|
|
447
618
|
|
|
448
619
|
* **function: suppress thread exception reporting in `:thread` concurrency** <br>
|
|
@@ -468,15 +639,15 @@ prior call.
|
|
|
468
639
|
`LLM::Interrupt` is now raised on the active fiber via `Fiber#raise`
|
|
469
640
|
when interrupting `:fiber`-concurrency tools.
|
|
470
641
|
<br><br>
|
|
471
|
-
`Task#interrupt!` now dispatches by task type
|
|
472
|
-
threads, `Fiber#raise` for fibers
|
|
642
|
+
`Task#interrupt!` now dispatches by task type: `Thread#raise` for
|
|
643
|
+
threads, `Fiber#raise` for fibers. Making interruption reliable
|
|
473
644
|
across all concurrency strategies.
|
|
474
645
|
|
|
475
646
|
* **function: raise `LLM::Interrupt` on fork-backed tool tasks** <br>
|
|
476
647
|
`LLM::Interrupt` is now raised on the main thread of a fork child
|
|
477
648
|
process via `Thread.main.raise(LLM::Interrupt)` when interrupting
|
|
478
649
|
`:fork`-concurrency tools, and the fork `Task#wait` re-raises the
|
|
479
|
-
interrupt on the parent side
|
|
650
|
+
interrupt on the parent side. Making interruption reliable across
|
|
480
651
|
all concurrency strategies including `:fork`.
|
|
481
652
|
|
|
482
653
|
* **function: raise `LLM::Interrupt` on `Async::Task`-backed tool tasks** <br>
|
|
@@ -494,7 +665,7 @@ prior call.
|
|
|
494
665
|
raises `LLM::Interrupt` on the ractor's main thread.
|
|
495
666
|
<br><br>
|
|
496
667
|
`Task#interrupt!` delegates to the mailbox to send the interrupt
|
|
497
|
-
message
|
|
668
|
+
message. Extending reliable interruption to the `:ractor`
|
|
498
669
|
concurrency strategy.
|
|
499
670
|
|
|
500
671
|
## v12.5.1
|
|
@@ -641,7 +812,7 @@ OpenAI, Google, DeepInfra, DeepSeek, and xAI model entries.
|
|
|
641
812
|
|
|
642
813
|
* **repl: display command errors in the curses UI** <br>
|
|
643
814
|
Commands invoked with too few arguments now display an error
|
|
644
|
-
message
|
|
815
|
+
message: `command(<name>): too few arguments`. Displayed directly in
|
|
645
816
|
the curses transcript area, giving immediate feedback instead
|
|
646
817
|
of silently failing.
|
|
647
818
|
|
|
@@ -765,7 +936,7 @@ a command system foundation with the `/exit` command, and several new
|
|
|
765
936
|
keybindings (Ctrl+F, Ctrl+K, Ctrl+Y). Tool calls are rendered with a
|
|
766
937
|
compact function-call syntax in the status bar.
|
|
767
938
|
|
|
768
|
-
Two new built-in tools
|
|
939
|
+
Two new built-in tools: `LLM::Tool::Ls` and `LLM::Tool::Which` are
|
|
769
940
|
available as opt-in additions for file listing and executable lookup.
|
|
770
941
|
|
|
771
942
|
Model metadata has been refreshed across providers, the REPL loop
|
|
@@ -832,7 +1003,7 @@ area.
|
|
|
832
1003
|
The curses-based REPL input now detects paste operations by tracking
|
|
833
1004
|
the rate at which characters arrive. A paste rate of ≤50ms is
|
|
834
1005
|
assumed to be a burst of characters that could only be explained by
|
|
835
|
-
a paste
|
|
1006
|
+
a paste. No human types that fast. Multiline pastes are supported
|
|
836
1007
|
through internal refactoring of the input handling logic.
|
|
837
1008
|
|
|
838
1009
|
* **repl: optimize paste mode rendering** <br>
|
|
@@ -855,7 +1026,7 @@ area.
|
|
|
855
1026
|
|
|
856
1027
|
* **repl: render tool calls in a function-call syntax** <br>
|
|
857
1028
|
The curses-based REPL status bar now renders tool calls with a
|
|
858
|
-
compact function-call syntax
|
|
1029
|
+
compact function-call syntax: `tool(key: value)` instead of
|
|
859
1030
|
`tool: name`. Strings are quoted and truncated, arrays show their
|
|
860
1031
|
first two elements, and hashes collapse to `{…}`, making it easier
|
|
861
1032
|
to see what arguments the model is passing. The `tool done` status
|
|
@@ -983,8 +1154,8 @@ tracer logger instances with less verbosity.
|
|
|
983
1154
|
streamed content via `#<<`. `LLM::Stream::Disabled` represents an explicitly
|
|
984
1155
|
disabled stream with no-op callbacks.
|
|
985
1156
|
|
|
986
|
-
This is part of an internal refactoring that lets all stream values
|
|
987
|
-
objects, `true`, `false`, `nil`, and `LLM::Stream` instances themselves
|
|
1157
|
+
This is part of an internal refactoring that lets all stream values: IO
|
|
1158
|
+
objects, `true`, `false`, `nil`, and `LLM::Stream` instances themselves
|
|
988
1159
|
be represented by the same `LLM::Stream` interface via the new
|
|
989
1160
|
`LLM::Stream.try` factory method.
|
|
990
1161
|
|
|
@@ -1053,7 +1224,7 @@ tracer logger instances with less verbosity.
|
|
|
1053
1224
|
|
|
1054
1225
|
* **repl: add extra padding between markdown nodes** <br>
|
|
1055
1226
|
The curses-based REPL markdown renderer now adds extra vertical spacing
|
|
1056
|
-
between certain markdown elements
|
|
1227
|
+
between certain markdown elements: paragraphs, headers, and codeblocks
|
|
1057
1228
|
for improved readability of model responses.
|
|
1058
1229
|
|
|
1059
1230
|
* **repl: add a visual divider between transcript and the rows below it** <br>
|
data/README.md
CHANGED
|
@@ -16,9 +16,12 @@ Welcome to the canonical llm.rb repository.
|
|
|
16
16
|
|
|
17
17
|
llm.rb is an advanced runtime for building capable AI applications
|
|
18
18
|
on CRuby. By default it has zero runtime dependencies although certain
|
|
19
|
-
functionality
|
|
19
|
+
functionality (such as ActiveRecord support) require
|
|
20
20
|
optional dependencies that are opt-in.
|
|
21
21
|
|
|
22
|
+
When you want to learn more than what the README covers, checkout
|
|
23
|
+
the [deepdive.md](https://r.uby.dev/llm/deepdive/).
|
|
24
|
+
|
|
22
25
|
## Features
|
|
23
26
|
|
|
24
27
|
The runtime supports OpenAI, OpenAI-compatible endpoints, Anthropic, Google
|
|
@@ -33,12 +36,10 @@ also possible to make a tool call while the model is still streaming.
|
|
|
33
36
|
|
|
34
37
|
The runtime builds on top of three core concepts: providers, contexts, and agents,
|
|
35
38
|
so once you learn the fundamentals, everything else falls into place naturally. And once
|
|
36
|
-
you learn llm.rb, you will also be able to use
|
|
39
|
+
you learn llm.rb, you will also be able to use
|
|
40
|
+
<a href="https://r.uby.dev/mruby-llm">mruby-llm</a> and
|
|
37
41
|
<a href="https://r.uby.dev/wasm-llm">wasm-llm</a> because the API is pretty much identical.
|
|
38
42
|
|
|
39
|
-
For detailed explanations, configuration, and advanced patterns, see the
|
|
40
|
-
[deepdive.md](https://r.uby.dev/llm/deepdive/).
|
|
41
|
-
|
|
42
43
|
## Install
|
|
43
44
|
|
|
44
45
|
```bash
|
|
@@ -49,7 +50,9 @@ gem install llm.rb
|
|
|
49
50
|
|
|
50
51
|
#### LLM::Agent
|
|
51
52
|
|
|
52
|
-
The
|
|
53
|
+
The
|
|
54
|
+
[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
|
|
55
|
+
class is the default high-level interface,
|
|
53
56
|
and it is recommended for most use-cases. It manages tool execution
|
|
54
57
|
automatically, guards against infinite loops, manages conversation
|
|
55
58
|
state, and much more.
|
|
@@ -62,13 +65,63 @@ agent = LLM::Agent.new(llm, stream: $stdout)
|
|
|
62
65
|
agent.talk "Hello world"
|
|
63
66
|
```
|
|
64
67
|
|
|
68
|
+
##### set
|
|
69
|
+
|
|
70
|
+
[`LLM::Agent.set`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#set-class_method)
|
|
71
|
+
is a class-level DSL that accepts a Hash of properties. Each key resolves to a
|
|
72
|
+
corresponding class accessor: `name`, `description`, `model`, `tools`,
|
|
73
|
+
`instructions`, `schema`, `stream`, `tracer`, `concurrency`, `confirm`,
|
|
74
|
+
`path`, and `skills`. All options are optional; zero or more can be set.
|
|
75
|
+
An error is raised for unknown keys so that typos are caught early.
|
|
76
|
+
|
|
77
|
+
```ruby
|
|
78
|
+
class SystemAdmin < LLM::Agent
|
|
79
|
+
set name: "sysadmin",
|
|
80
|
+
description: "system administration agent",
|
|
81
|
+
model: "deepseek-v4-pro",
|
|
82
|
+
tools: [Shell]
|
|
83
|
+
end
|
|
84
|
+
|
|
85
|
+
llm = LLM.deepseek(key: ENV["KEY"])
|
|
86
|
+
agent = SystemAdmin.new(llm)
|
|
87
|
+
agent.talk "Run 'date'"
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
##### Persistence
|
|
91
|
+
|
|
92
|
+
Set `path:` on an agent for automatic filesystem persistence;
|
|
93
|
+
the agent restores conversation history from the file on startup
|
|
94
|
+
and saves it back after every turn, with no manual serialization
|
|
95
|
+
code. For database-backed persistence, ActiveRecord and Sequel
|
|
96
|
+
integrations are also available (see the
|
|
97
|
+
[database deepdive](https://r.uby.dev/llm/deepdive/advanced/database)
|
|
98
|
+
for details). All persistence options use the same underlying
|
|
99
|
+
serialization.
|
|
100
|
+
|
|
101
|
+
```ruby
|
|
102
|
+
require "llm"
|
|
103
|
+
|
|
104
|
+
llm = LLM.deepseek(key: ENV["KEY"])
|
|
105
|
+
agent = LLM::Agent.new(llm, path: "session.json")
|
|
106
|
+
agent.talk "remember my name is robert"
|
|
107
|
+
|
|
108
|
+
# Next time, the conversation is restored automatically:
|
|
109
|
+
agent = LLM::Agent.new(llm, path: "session.json")
|
|
110
|
+
agent.talk "what's my name?"
|
|
111
|
+
```
|
|
112
|
+
|
|
65
113
|
#### LLM::Context
|
|
66
114
|
|
|
67
|
-
The
|
|
68
|
-
|
|
115
|
+
The
|
|
116
|
+
[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
|
|
117
|
+
class is at the heart of the runtime
|
|
118
|
+
and it is what
|
|
119
|
+
[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
|
|
120
|
+
uses under the hood.
|
|
69
121
|
It requires that the tool call loop be managed manually -
|
|
70
122
|
sometimes that can be useful, but usually for advanced use-cases.
|
|
71
|
-
If you're new to llm.rb, try
|
|
123
|
+
If you're new to llm.rb, try
|
|
124
|
+
[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html) first.
|
|
72
125
|
|
|
73
126
|
```ruby
|
|
74
127
|
require "llm"
|
|
@@ -80,7 +133,9 @@ ctx.talk "Hello world"
|
|
|
80
133
|
|
|
81
134
|
#### LLM::Tool
|
|
82
135
|
|
|
83
|
-
Subclasses of
|
|
136
|
+
Subclasses of
|
|
137
|
+
[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html)
|
|
138
|
+
are plain Ruby classes with
|
|
84
139
|
an optional set of typed parameters. <br> The model can choose to
|
|
85
140
|
call them on your behalf, and they're one of the most powerful features
|
|
86
141
|
for extending the feature set or abilities of a model.
|
|
@@ -101,7 +156,8 @@ end
|
|
|
101
156
|
#### LLM::Stream
|
|
102
157
|
|
|
103
158
|
Streams can be simple IO objects or subclasses of
|
|
104
|
-
[`LLM::Stream`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html)
|
|
159
|
+
[`LLM::Stream`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html)
|
|
160
|
+
with structured callbacks for content,
|
|
105
161
|
reasoning, tool calls, tool returns, and compaction.
|
|
106
162
|
|
|
107
163
|
```ruby
|
|
@@ -122,16 +178,27 @@ agent.talk "Explain Ruby fibers."
|
|
|
122
178
|
|
|
123
179
|
#### LLM::Schema
|
|
124
180
|
|
|
125
|
-
[`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html)
|
|
181
|
+
[`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html)
|
|
182
|
+
subclasses produce typed, structured
|
|
126
183
|
output from any model call. Pass a schema to `LLM::Context#talk`,
|
|
127
184
|
`LLM::Agent#talk`, or `LLM::Provider#complete` to receive validated
|
|
128
185
|
JSON instead of free text. Schemas work alongside tools and streams.
|
|
129
186
|
|
|
130
|
-
[`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html)
|
|
131
|
-
|
|
187
|
+
[`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html)
|
|
188
|
+
can define objects, arrays, enums, nested schemas,
|
|
189
|
+
and more. It is also used internally by
|
|
190
|
+
[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html) for parameter
|
|
132
191
|
definitions, so you already benefit from it when you declare tool
|
|
133
192
|
parameters.
|
|
134
193
|
|
|
194
|
+
The
|
|
195
|
+
[`LLM::DeepSeek`](https://r.uby.dev/api-docs/llm.rb/LLM/DeepSeek.html)
|
|
196
|
+
provider includes runtime-level optimisations such as structured
|
|
197
|
+
output support (despite no official structured outputs API) and
|
|
198
|
+
SVG image generation. This example uses
|
|
199
|
+
[`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html) with
|
|
200
|
+
DeepSeek:
|
|
201
|
+
|
|
135
202
|
```ruby
|
|
136
203
|
class Weather < LLM::Schema
|
|
137
204
|
property :city, String, "The city name"
|
|
@@ -140,7 +207,7 @@ class Weather < LLM::Schema
|
|
|
140
207
|
required %i[city temperature conditions]
|
|
141
208
|
end
|
|
142
209
|
|
|
143
|
-
llm = LLM.
|
|
210
|
+
llm = LLM.deepseek(key: ENV["KEY"])
|
|
144
211
|
agent = LLM::Agent.new(llm, schema: Weather)
|
|
145
212
|
res = agent.talk "Weather in Paris?"
|
|
146
213
|
res.content! # => {city: "Paris", temperature: 15.0, conditions: "Cloudy"}
|
|
@@ -150,11 +217,11 @@ res.content! # => {city: "Paris", temperature: 15.0, conditions: "Cloudy"}
|
|
|
150
217
|
|
|
151
218
|
The [LLM::Agent#repl](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#repl-instance_method)
|
|
152
219
|
method drops you into a curses-based TUI for talking to an
|
|
153
|
-
agent interactively.
|
|
154
|
-
|
|
220
|
+
agent interactively. Set `path:` on the agent for automatic
|
|
221
|
+
persistence across REPL sessions. The `tools:` option attaches
|
|
155
222
|
extra tools for the duration of the session. It is like
|
|
156
223
|
`binding.pry` but for agents. For the full reference see the
|
|
157
|
-
[REPL section](https://r.uby.dev/llm/deepdive
|
|
224
|
+
[REPL section](https://r.uby.dev/llm/deepdive/fundamentals/repl) in the
|
|
158
225
|
deepdive.
|
|
159
226
|
|
|
160
227
|
```ruby
|
|
@@ -162,17 +229,35 @@ require "llm"
|
|
|
162
229
|
require "llm/tools"
|
|
163
230
|
|
|
164
231
|
llm = LLM.deepseek(key: ENV["KEY"])
|
|
165
|
-
agent = LLM::Agent.new(llm, name: "my-agent")
|
|
166
|
-
agent.repl(
|
|
232
|
+
agent = LLM::Agent.new(llm, name: "my-agent", path: "agent.json")
|
|
233
|
+
agent.repl(tools: LLM::Tool.subclasses)
|
|
234
|
+
```
|
|
235
|
+
|
|
236
|
+
##### CLI
|
|
237
|
+
|
|
238
|
+
The `llm.rb` executable is available on your PATH after installation.
|
|
239
|
+
It starts a REPL session from any directory:
|
|
240
|
+
|
|
241
|
+
```bash
|
|
242
|
+
llm.rb # auto-detect from $DEEPSEEK_API_KEY
|
|
243
|
+
llm.rb -p openai # use OpenAI explicitly
|
|
244
|
+
llm.rb -t # temporary session, no persistence
|
|
167
245
|
```
|
|
168
246
|
|
|
247
|
+
The CLI auto-detects your provider from standard environment variables
|
|
248
|
+
(`DEEPSEEK_API_KEY`, `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, etc.).
|
|
249
|
+
Persistent sessions are stored under `~/.llm.rb/` and restored
|
|
250
|
+
automatically on your next visit.
|
|
251
|
+
|
|
169
252
|
#### LLM::MCP
|
|
170
253
|
|
|
171
254
|
The Model Context Protocol (MCP) has first-class support
|
|
172
255
|
in llm.rb. The stdio and http transports work out of the
|
|
173
256
|
box. MCP tools are translated into subclasses of
|
|
174
|
-
[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html) that can be
|
|
175
|
-
|
|
257
|
+
[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html) that can be
|
|
258
|
+
used with
|
|
259
|
+
[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html) or
|
|
260
|
+
[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html).
|
|
176
261
|
|
|
177
262
|
```ruby
|
|
178
263
|
require "llm"
|
|
@@ -188,8 +273,10 @@ agent.talk "Run the tool"
|
|
|
188
273
|
The Agent 2 Agent (A2A) protocol has first-class support
|
|
189
274
|
in llm.rb. The http and jsonrpc transports work out of the
|
|
190
275
|
box. A2A skills are translated into subclasses of
|
|
191
|
-
[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html) that can be
|
|
192
|
-
|
|
276
|
+
[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html) that can be
|
|
277
|
+
used with
|
|
278
|
+
[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html) or
|
|
279
|
+
[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html).
|
|
193
280
|
|
|
194
281
|
```ruby
|
|
195
282
|
require "llm"
|
|
@@ -200,6 +287,38 @@ agent = LLM::Agent.new(llm, stream: $stdout, tools: a2a.skills)
|
|
|
200
287
|
agent.talk "Run the skill"
|
|
201
288
|
```
|
|
202
289
|
|
|
290
|
+
#### LLM::Skill
|
|
291
|
+
|
|
292
|
+
A skill turns a markdown file into a callable tool. When the model
|
|
293
|
+
calls it, the runtime spawns a subagent with the skill's instructions
|
|
294
|
+
as its system prompt and the skill's own tool set. The subagent runs
|
|
295
|
+
one turn and returns the result, then is discarded. Each call
|
|
296
|
+
is fresh and stateless. For a deeper explanation see the
|
|
297
|
+
[deepdive.md](https://r.uby.dev/llm/deepdive/fundamentals/skills).
|
|
298
|
+
|
|
299
|
+
**SKILL.md**
|
|
300
|
+
|
|
301
|
+
```markdown
|
|
302
|
+
---
|
|
303
|
+
name: summary
|
|
304
|
+
description: Reads recent git history and writes a summary
|
|
305
|
+
tools: all
|
|
306
|
+
---
|
|
307
|
+
|
|
308
|
+
Collect the recent git log, analyze each commit,
|
|
309
|
+
and write a summary to summary.txt.
|
|
310
|
+
```
|
|
311
|
+
|
|
312
|
+
**agent.rb**
|
|
313
|
+
|
|
314
|
+
```ruby
|
|
315
|
+
require "llm"
|
|
316
|
+
|
|
317
|
+
llm = LLM.deepseek(key: ENV["KEY"])
|
|
318
|
+
agent = LLM::Agent.new(llm, skills: ["./skills/summary"])
|
|
319
|
+
agent.talk "Summarize the last week of work"
|
|
320
|
+
```
|
|
321
|
+
|
|
203
322
|
#### RAG
|
|
204
323
|
|
|
205
324
|
Most providers offer an embedding model that can be
|
|
@@ -220,6 +339,8 @@ llm = LLM.openai(key: ENV["KEY"])
|
|
|
220
339
|
body = "llm.rb is Ruby's capable AI runtime."
|
|
221
340
|
embedding = llm.embed([body]).embeddings.first
|
|
222
341
|
|
|
342
|
+
# Document is your ActiveRecord or Sequel model
|
|
343
|
+
# with a vector column (e.g. sqlite-vec or pgvector)
|
|
223
344
|
Document.create!(
|
|
224
345
|
title: "llm.rb",
|
|
225
346
|
body:,
|
|
@@ -241,7 +362,7 @@ The `:fork` strategy also provides a separate process that offers
|
|
|
241
362
|
isolation from its parent.
|
|
242
363
|
|
|
243
364
|
You can learn more about the llm.rb concurrency model in the
|
|
244
|
-
[deepdive.md](https://r.uby.dev/llm/deepdive
|
|
365
|
+
[deepdive.md](https://r.uby.dev/llm/deepdive/fundamentals/concurrency).
|
|
245
366
|
|
|
246
367
|
```ruby
|
|
247
368
|
require "llm"
|
|
@@ -254,7 +375,9 @@ agent.talk "Run the tools in parallel"
|
|
|
254
375
|
|
|
255
376
|
#### ORM
|
|
256
377
|
|
|
257
|
-
Because both
|
|
378
|
+
Because both
|
|
379
|
+
[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html) and
|
|
380
|
+
[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
|
|
258
381
|
can be serialized to JSON and stored in a simple string, both ActiveRecord
|
|
259
382
|
and Sequel support can be implemented within a single column on a single row.
|
|
260
383
|
|
|
@@ -341,9 +464,9 @@ In no particular order:
|
|
|
341
464
|
<summary>I have a limited budget. What should I do?</summary>
|
|
342
465
|
<br>
|
|
343
466
|
<p>
|
|
344
|
-
There a few options. The first option is to host
|
|
467
|
+
There are a few options. The first option is to host
|
|
345
468
|
your own model, and use the ollama or llamacpp
|
|
346
|
-
providers. This can be
|
|
469
|
+
providers. This can be difficult though because
|
|
347
470
|
a capable model requires hardware that can
|
|
348
471
|
match it. If you have the ability to self-host,
|
|
349
472
|
this would be my first option.
|
|
@@ -368,7 +491,7 @@ If you're on a budget, DeepSeek is hard to beat.
|
|
|
368
491
|
<details>
|
|
369
492
|
<summary>Can I download llm.rb via a decentralized network?</summary>
|
|
370
493
|
<br>
|
|
371
|
-
|
|
494
|
+
Yes.
|
|
372
495
|
<br>
|
|
373
496
|
We are on the <a href="https://radicle.network">radicle.network</a>
|
|
374
497
|
<br>
|
|
@@ -376,7 +499,9 @@ Every commit that lands on GitHub also lands on Radicle.
|
|
|
376
499
|
<br>
|
|
377
500
|
Our repository ID is z2PtfQ6dYwyYaW2aGrztG1sMyDmCE.
|
|
378
501
|
<br>
|
|
379
|
-
Browse on <a
|
|
502
|
+
Browse on <a
|
|
503
|
+
href="https://radicle.network/nodes/iris.radicle.network/z2PtfQ6dYwyYaW2aGrztG1sMyDmCE">the
|
|
504
|
+
web</a>.
|
|
380
505
|
</details>
|
|
381
506
|
|
|
382
507
|
## Resources
|