llm.rb 13.0.0 → 14.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +505 -14
- data/README.md +484 -50
- data/bin/llm.rb +148 -0
- data/data/anthropic.json +206 -263
- data/data/bedrock.json +2138 -1860
- data/data/deepinfra.json +1003 -624
- data/data/deepseek.json +38 -34
- data/data/google.json +1079 -371
- data/data/mistral.json +448 -368
- data/data/moonshot.json +384 -0
- data/data/openai.json +974 -1343
- data/data/xai.json +154 -126
- data/data/zai.json +191 -191
- data/lib/llm/agent.rb +123 -20
- data/lib/llm/context.rb +71 -88
- data/lib/llm/cost.rb +23 -17
- data/lib/llm/error.rb +0 -8
- data/lib/llm/function/array.rb +3 -3
- data/lib/llm/function/async/task.rb +2 -0
- data/lib/llm/function/fiber/task.rb +2 -0
- data/lib/llm/function/fork/task.rb +2 -0
- data/lib/llm/function/ractor/task.rb +2 -0
- data/lib/llm/function/sequential/group.rb +4 -1
- data/lib/llm/function/sequential/task.rb +1 -1
- data/lib/llm/function/task.rb +4 -0
- data/lib/llm/function/thread/task.rb +2 -0
- data/lib/llm/function.rb +33 -6
- data/lib/llm/guard/loop.rb +89 -0
- data/lib/llm/guard/null.rb +19 -0
- data/lib/llm/guard.rb +61 -0
- data/lib/llm/provider.rb +36 -0
- data/lib/llm/providers/anthropic/stream_parser.rb +1 -0
- data/lib/llm/providers/anthropic.rb +2 -9
- data/lib/llm/providers/bedrock/request_adapter.rb +1 -1
- data/lib/llm/providers/bedrock/stream_parser.rb +1 -0
- data/lib/llm/providers/bedrock.rb +1 -8
- data/lib/llm/providers/google/stream_parser.rb +1 -0
- data/lib/llm/providers/google.rb +1 -8
- data/lib/llm/providers/mistral.rb +1 -1
- data/lib/llm/providers/moonshot.rb +76 -0
- data/lib/llm/providers/ollama.rb +2 -9
- data/lib/llm/providers/openai/responses/stream_parser.rb +1 -0
- data/lib/llm/providers/openai/responses.rb +7 -9
- data/lib/llm/providers/openai/stream_parser.rb +1 -0
- data/lib/llm/providers/openai.rb +4 -11
- data/lib/llm/repl/bar.rb +4 -3
- data/lib/llm/repl/{transcript.rb → buffer.rb} +69 -29
- data/lib/llm/repl/color.rb +78 -0
- data/lib/llm/repl/command.rb +12 -5
- data/lib/llm/repl/commands/compact.rb +2 -2
- data/lib/llm/repl/commands/help.rb +3 -5
- data/lib/llm/repl/input/char.rb +46 -0
- data/lib/llm/repl/input/row.rb +39 -0
- data/lib/llm/repl/input.rb +251 -66
- data/lib/llm/repl/markdown/table.rb +11 -3
- data/lib/llm/repl/markdown.rb +34 -8
- data/lib/llm/repl/node.rb +37 -0
- data/lib/llm/repl/status.rb +42 -7
- data/lib/llm/repl/stream.rb +18 -6
- data/lib/llm/repl/walker.rb +3 -2
- data/lib/llm/repl/window.rb +54 -35
- data/lib/llm/repl.rb +74 -32
- data/lib/llm/skill.rb +20 -4
- data/lib/llm/stream.rb +8 -7
- data/lib/llm/tool.rb +29 -0
- data/lib/llm/tools/{swap_text.rb → edit-file.rb} +3 -3
- data/lib/llm/tools/git.rb +3 -0
- data/lib/llm/tools/mkdir.rb +3 -0
- data/lib/llm/tools/rg.rb +3 -0
- data/lib/llm/tools/ruby.rb +46 -0
- data/lib/llm/tools/shell.rb +3 -0
- data/lib/llm/tracer/pretty_logger.rb +127 -0
- data/lib/llm/tracer.rb +1 -0
- data/lib/llm/transformer/null.rb +21 -0
- data/lib/llm/transformer.rb +55 -0
- data/lib/llm/version.rb +1 -1
- data/lib/llm.rb +12 -2
- data/llm.gemspec +9 -2
- data/resources/deepdive/advanced/cancellation.md +74 -0
- data/resources/deepdive/advanced/compaction.md +83 -0
- data/resources/deepdive/advanced/context.md +267 -0
- data/resources/deepdive/advanced/guard.md +371 -0
- data/resources/deepdive/advanced/tracer.md +180 -0
- data/resources/deepdive/advanced/transformer.md +67 -0
- data/resources/deepdive/advanced/transports.md +45 -0
- data/resources/deepdive/everything_else/audio.md +122 -0
- data/resources/deepdive/everything_else/cost.md +99 -0
- data/resources/deepdive/everything_else/images.md +89 -0
- data/resources/deepdive/everything_else/object.md +108 -0
- data/resources/deepdive/everything_else/ocr.md +48 -0
- data/resources/deepdive/fundamentals/agents.md +202 -0
- data/resources/deepdive/fundamentals/builtin_tools.md +191 -0
- data/resources/deepdive/fundamentals/concurrency.md +104 -0
- data/resources/deepdive/fundamentals/database.md +449 -0
- data/resources/deepdive/fundamentals/embeddings.md +157 -0
- data/resources/deepdive/fundamentals/repl.md +87 -0
- data/resources/deepdive/fundamentals/schema.md +61 -0
- data/resources/deepdive/fundamentals/skills.md +106 -0
- data/resources/deepdive/fundamentals/stream.md +110 -0
- data/resources/deepdive/fundamentals/tools.md +265 -0
- data/resources/deepdive/protocols/a2a.md +106 -0
- data/resources/deepdive/protocols/mcp.md +111 -0
- data/resources/deepdive.md +58 -1792
- metadata +51 -7
- data/lib/llm/loop_guard.rb +0 -107
|
@@ -0,0 +1,108 @@
|
|
|
1
|
+
|
|
2
|
+
## LLM::Object
|
|
3
|
+
|
|
4
|
+
### Introduction
|
|
5
|
+
|
|
6
|
+
#### Overview
|
|
7
|
+
|
|
8
|
+
[`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html)
|
|
9
|
+
is the hash-like object that llm.rb uses everywhere structured data
|
|
10
|
+
flows through the runtime. Response bodies, tool arguments, schema
|
|
11
|
+
results, usage and cost data, and function parameters all come back
|
|
12
|
+
as [`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html)
|
|
13
|
+
instances. It is similar in spirit to OpenStruct,
|
|
14
|
+
and it was introduced after OpenStruct became a bundled gem rather
|
|
15
|
+
than a default gem in Ruby 3.5.
|
|
16
|
+
|
|
17
|
+
#### How it works
|
|
18
|
+
|
|
19
|
+
When you want to read a value from an
|
|
20
|
+
[`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html),
|
|
21
|
+
use either method-style or bracket access. Keys are indifferent, so
|
|
22
|
+
strings and symbols work interchangeably:
|
|
23
|
+
|
|
24
|
+
```ruby
|
|
25
|
+
obj = LLM::Object.from(city: "Paris", temperature: 15.0)
|
|
26
|
+
|
|
27
|
+
obj.city # => "Paris"
|
|
28
|
+
obj["city"] # => "Paris"
|
|
29
|
+
obj[:city] # => "Paris"
|
|
30
|
+
obj[:temperature] # => 15.0
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
Nested hashes and arrays are converted recursively, so deep chains
|
|
34
|
+
read naturally:
|
|
35
|
+
|
|
36
|
+
```ruby
|
|
37
|
+
obj = LLM::Object.from(person: {name: "John"})
|
|
38
|
+
obj.person.name # => "John"
|
|
39
|
+
obj.person.class # => LLM::Object
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
#### Why would I use it?
|
|
43
|
+
|
|
44
|
+
Most of the time you do not construct
|
|
45
|
+
[`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html)
|
|
46
|
+
instances yourself. They come back from `talk`, `ask`, `embed`, and
|
|
47
|
+
every other call that returns structured data. Knowing how they
|
|
48
|
+
behave lets you read response fields, pass tool arguments, and
|
|
49
|
+
inspect usage without reaching for `to_h` on every line.
|
|
50
|
+
|
|
51
|
+
#### Notes
|
|
52
|
+
|
|
53
|
+
An
|
|
54
|
+
[`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html)
|
|
55
|
+
is enumerable and supports the usual Hash operations: `keys`,
|
|
56
|
+
`values`, `key?`, `fetch`, `dig`, `slice`, `merge`, `merge!`,
|
|
57
|
+
`delete`, `size`, and `empty?`. Use `to_h` for a plain Hash and
|
|
58
|
+
`to_hash` for one with symbol keys. A missing key returns `nil`
|
|
59
|
+
rather than raising. Because it subclasses `BasicObject`, `to_json`
|
|
60
|
+
is defined explicitly and serializes through the configured JSON
|
|
61
|
+
adapter via
|
|
62
|
+
[`LLM.json.dump`](https://r.uby.dev/api-docs/llm.rb/LLM.html#json-class_method).
|
|
63
|
+
|
|
64
|
+
### Reading and writing
|
|
65
|
+
|
|
66
|
+
#### Overview
|
|
67
|
+
|
|
68
|
+
Beyond simple reads,
|
|
69
|
+
[`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html)
|
|
70
|
+
supports assignment, mutation, and iteration, so you can treat it
|
|
71
|
+
like a Hash in place.
|
|
72
|
+
|
|
73
|
+
#### How it works
|
|
74
|
+
|
|
75
|
+
Assign values with method or bracket syntax, mutate in place, and
|
|
76
|
+
iterate like a Hash:
|
|
77
|
+
|
|
78
|
+
```ruby
|
|
79
|
+
obj = LLM::Object.from({})
|
|
80
|
+
|
|
81
|
+
obj.city = "Paris" # method-style write
|
|
82
|
+
obj["country"] = "France"
|
|
83
|
+
|
|
84
|
+
obj.key?(:city) # => true
|
|
85
|
+
obj.keys # => ["city", "country"]
|
|
86
|
+
obj.merge!(population: 2_100_000)
|
|
87
|
+
|
|
88
|
+
obj.each { |key, value| puts "#{key}: #{value}" }
|
|
89
|
+
obj.transform_values!(&:upcase) if obj.any?
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
#### Why would I use it?
|
|
93
|
+
|
|
94
|
+
Tool implementations receive their arguments as an
|
|
95
|
+
[`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html)
|
|
96
|
+
and often build a result Hash from them. Merging defaults, deleting
|
|
97
|
+
optional keys, and transforming values in place keeps that code
|
|
98
|
+
concise without converting back and forth between Hash and
|
|
99
|
+
[`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html).
|
|
100
|
+
|
|
101
|
+
#### Notes
|
|
102
|
+
|
|
103
|
+
`merge` returns a new
|
|
104
|
+
[`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html);
|
|
105
|
+
`merge!` mutates in place.
|
|
106
|
+
Assignment always stores the key as a string internally, which is
|
|
107
|
+
why string and symbol lookups both work. Equality compares against
|
|
108
|
+
anything that responds to `to_h`.
|
|
@@ -0,0 +1,48 @@
|
|
|
1
|
+
|
|
2
|
+
## OCR
|
|
3
|
+
|
|
4
|
+
### Introduction
|
|
5
|
+
|
|
6
|
+
#### Overview
|
|
7
|
+
|
|
8
|
+
OCR pulls text out of images and PDFs so you can search, index,
|
|
9
|
+
or process scanned content. Mistral is the only provider with a
|
|
10
|
+
dedicated OCR endpoint, and it accepts both image URLs and
|
|
11
|
+
document URLs.
|
|
12
|
+
|
|
13
|
+
#### How it works
|
|
14
|
+
|
|
15
|
+
Mistral is the only provider with a dedicated OCR endpoint. The
|
|
16
|
+
[`LLM::Provider#ocr`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#ocr)
|
|
17
|
+
method accepts an `image_url:` or `document_url:` parameter.
|
|
18
|
+
Document URLs can point to PDFs.
|
|
19
|
+
|
|
20
|
+
```ruby
|
|
21
|
+
require "llm"
|
|
22
|
+
|
|
23
|
+
llm = LLM.mistral(key: ENV["KEY"])
|
|
24
|
+
|
|
25
|
+
# Extract text from an image
|
|
26
|
+
res = llm.ocr(image_url: "https://example.com/photo.png")
|
|
27
|
+
res.pages.each { |page| puts page.markdown }
|
|
28
|
+
|
|
29
|
+
# Extract text from a PDF
|
|
30
|
+
res = llm.ocr(document_url: "https://example.com/report.pdf")
|
|
31
|
+
res.pages.each { |page| puts page.markdown }
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
#### Why would I use it?
|
|
35
|
+
|
|
36
|
+
OCR extracts text from scanned documents and images. The result
|
|
37
|
+
feeds into search indexes, extraction pipelines that pull out dates
|
|
38
|
+
and amounts, or archives so scanned contracts become searchable
|
|
39
|
+
records. PDFs and images both work, and the response is structured
|
|
40
|
+
per page with markdown.
|
|
41
|
+
|
|
42
|
+
#### Notes
|
|
43
|
+
|
|
44
|
+
Only Mistral currently supports OCR through the llm.rb runtime.
|
|
45
|
+
The response exposes pages through
|
|
46
|
+
[`LLM::OCR::Response#pages`](https://r.uby.dev/api-docs/llm.rb/LLM/OCR/Response.html#pages),
|
|
47
|
+
where each page
|
|
48
|
+
has a `markdown` field containing the extracted text.
|
|
@@ -0,0 +1,202 @@
|
|
|
1
|
+
|
|
2
|
+
## Agents
|
|
3
|
+
### Introduction
|
|
4
|
+
|
|
5
|
+
#### Overview
|
|
6
|
+
|
|
7
|
+
[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
|
|
8
|
+
is the recommended entry point for most use-cases. It provides a
|
|
9
|
+
class-level DSL for defining reusable, preconfigured assistants
|
|
10
|
+
with defaults for model, tools, schema, and instructions. Under
|
|
11
|
+
the hood it delegates to
|
|
12
|
+
[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html),
|
|
13
|
+
so it has the same runtime surface: message history, streaming,
|
|
14
|
+
serialization, compaction, and concurrency.
|
|
15
|
+
|
|
16
|
+
#### How it works
|
|
17
|
+
|
|
18
|
+
An agent holds a conversation with a model. You send input with
|
|
19
|
+
[`LLM::Agent#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#talk),
|
|
20
|
+
the model responds, and if it requests tools the agent
|
|
21
|
+
executes them automatically and feeds the results back. It enables
|
|
22
|
+
[a loop guard by default](https://r.uby.dev/llm/deepdive/advanced/guard)
|
|
23
|
+
that detects repeated tool-call patterns
|
|
24
|
+
and blocks stuck execution. The tool loop can also be bounded with
|
|
25
|
+
[`LLM::Agent.tool_budget`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#tool_budget-class_method)
|
|
26
|
+
(see the Tool budget section). Instructions are injected once
|
|
27
|
+
unless a system message is already present.
|
|
28
|
+
|
|
29
|
+
#### Why would I use it?
|
|
30
|
+
|
|
31
|
+
Agents manage the tool loop for you. They guard against infinite
|
|
32
|
+
loops, keep conversation state across turns, and let you define
|
|
33
|
+
reusable configurations at the class level. If you need manual
|
|
34
|
+
control over the tool loop, use
|
|
35
|
+
[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
|
|
36
|
+
directly instead.
|
|
37
|
+
|
|
38
|
+
#### Notes
|
|
39
|
+
|
|
40
|
+
Agents support the same concurrency strategies, compaction,
|
|
41
|
+
cancellation, and serialization as contexts. The trade-off between
|
|
42
|
+
a subclass and a direct instance is only in how the agent is
|
|
43
|
+
organized, not in what it can do. Tool loop execution can be
|
|
44
|
+
configured with `concurrency: :sequential`, `:thread`, `:async`,
|
|
45
|
+
`:fiber`, `:fork`, or `:ractor`.
|
|
46
|
+
|
|
47
|
+
### Class-based
|
|
48
|
+
|
|
49
|
+
#### Overview
|
|
50
|
+
|
|
51
|
+
A subclass of
|
|
52
|
+
[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
|
|
53
|
+
gives you a reusable agent with its own behavior. You define the model, tools, and other
|
|
54
|
+
attributes at the class level, and each instance picks them up
|
|
55
|
+
as defaults. Attributes can be overridden per-instance, and they
|
|
56
|
+
can be plain values, blocks, or Symbols that resolve to methods.
|
|
57
|
+
The class becomes a self-contained worker that you can instantiate
|
|
58
|
+
and talk to from anywhere.
|
|
59
|
+
|
|
60
|
+
#### How it works
|
|
61
|
+
|
|
62
|
+
A subclass declares its defaults with
|
|
63
|
+
[`LLM::Agent.set`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#set-class_method).
|
|
64
|
+
Each key is a
|
|
65
|
+
class-level accessor: `name`, `description`, `model`, `tools`,
|
|
66
|
+
`skills`, `instructions`, `stream`, `tracer`, `concurrency`,
|
|
67
|
+
`schema`, `confirm`, `path`, `tool_budget`.
|
|
68
|
+
Keyword arguments in the constructor override these defaults.
|
|
69
|
+
|
|
70
|
+
```ruby
|
|
71
|
+
class Agent < LLM::Agent
|
|
72
|
+
set model: "deepseek-v4-pro",
|
|
73
|
+
description: "system administration agent",
|
|
74
|
+
tools: [Shell]
|
|
75
|
+
end
|
|
76
|
+
|
|
77
|
+
llm = LLM.openai(key: ENV["KEY"])
|
|
78
|
+
agent = Agent.new(llm)
|
|
79
|
+
agent.talk "Run 'date'"
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
#### Why would I use it?
|
|
83
|
+
|
|
84
|
+
A subclass is useful when multiple parts of an application need to
|
|
85
|
+
call the same agent. The configuration and any helper methods live in one place. Define a
|
|
86
|
+
`research!` method that kicks off the agent's work. The subclass
|
|
87
|
+
becomes a self-contained worker.
|
|
88
|
+
|
|
89
|
+
#### Notes
|
|
90
|
+
|
|
91
|
+
Attributes passed to
|
|
92
|
+
[`LLM::Agent.set`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#set-class_method)
|
|
93
|
+
can be plain values, blocks, or
|
|
94
|
+
Symbols. A Symbol is evaluated as an instance method on the
|
|
95
|
+
subclass, so `tracer: :set_tracer` calls `set_tracer` on the
|
|
96
|
+
instance. A block like `stream: -> { $stdout }` is evaluated
|
|
97
|
+
when the attribute is first accessed.
|
|
98
|
+
|
|
99
|
+
Set `path:` on a subclass or instance for automatic filesystem
|
|
100
|
+
persistence; the agent restores conversation history from the
|
|
101
|
+
file on startup and saves it back after every turn with no
|
|
102
|
+
manual
|
|
103
|
+
[`LLM::Agent#save`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#save)/
|
|
104
|
+
[`LLM::Agent#restore`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#restore)
|
|
105
|
+
calls. See the
|
|
106
|
+
[database deepdive](../fundamentals/database.md) for details.
|
|
107
|
+
|
|
108
|
+
### Object-based
|
|
109
|
+
|
|
110
|
+
#### Overview
|
|
111
|
+
|
|
112
|
+
An
|
|
113
|
+
[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
|
|
114
|
+
instance is the simplest way to get started. You pass a provider and any configuration as keyword
|
|
115
|
+
arguments, and the agent runs the tool loop and manages state
|
|
116
|
+
just like a subclass would. This is the right choice when you
|
|
117
|
+
are prototyping, running a one-off task, or when the agent's
|
|
118
|
+
configuration is determined at runtime.
|
|
119
|
+
|
|
120
|
+
#### How it works
|
|
121
|
+
|
|
122
|
+
A direct instance takes the same attributes as keyword arguments
|
|
123
|
+
to
|
|
124
|
+
[`LLM::Agent.new`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#initialize-instance_method).
|
|
125
|
+
The first argument is always the provider.
|
|
126
|
+
Everything else is optional. The agent runs the tool loop
|
|
127
|
+
and manages state under the hood through a
|
|
128
|
+
[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html).
|
|
129
|
+
|
|
130
|
+
```ruby
|
|
131
|
+
llm = LLM.deepseek(key: ENV["KEY"])
|
|
132
|
+
agent = LLM::Agent.new(llm, stream: $stdout)
|
|
133
|
+
agent.talk "Hello world"
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
#### Why would I use it?
|
|
137
|
+
|
|
138
|
+
A direct instance is the right choice for quick experiments, one-shot
|
|
139
|
+
tasks, or when defining a class would be overkill. It is also
|
|
140
|
+
the right choice when the agent's configuration is determined at
|
|
141
|
+
runtime and a class hierarchy adds unnecessary complexity.
|
|
142
|
+
|
|
143
|
+
#### Notes
|
|
144
|
+
|
|
145
|
+
Direct instances accept all the same options as subclasses. The
|
|
146
|
+
difference is only in how the agent is organized, not in what it
|
|
147
|
+
can do. Under the hood,
|
|
148
|
+
[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
|
|
149
|
+
creates a
|
|
150
|
+
[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
|
|
151
|
+
that manages the message history.
|
|
152
|
+
|
|
153
|
+
### Tool budget
|
|
154
|
+
|
|
155
|
+
#### Overview
|
|
156
|
+
|
|
157
|
+
[`LLM::Agent.tool_budget`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#tool_budget-class_method)
|
|
158
|
+
caps the number of tool calls allowed in a single turn. Once the
|
|
159
|
+
budget is spent, the agent sends an in-band advisory message back
|
|
160
|
+
through the model instead of running more tools. By default no
|
|
161
|
+
budget is set, so the feature is disabled.
|
|
162
|
+
|
|
163
|
+
#### How it works
|
|
164
|
+
|
|
165
|
+
When you want to bound how many tools an agent can call in one
|
|
166
|
+
turn, set the budget with
|
|
167
|
+
[`LLM::Agent.set`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#set-class_method).
|
|
168
|
+
The budget can be a plain number or a block evaluated against the
|
|
169
|
+
agent instance. Once the agent has made the budgeted number of tool
|
|
170
|
+
calls, it stops and returns an in-band advisory message describing
|
|
171
|
+
the spent budget. A model will usually change course afterwards:
|
|
172
|
+
|
|
173
|
+
```ruby
|
|
174
|
+
class Researcher < LLM::Agent
|
|
175
|
+
set model: "deepseek-v4-pro",
|
|
176
|
+
tools: [FetchNews, FetchStocks],
|
|
177
|
+
tool_budget: 5
|
|
178
|
+
end
|
|
179
|
+
|
|
180
|
+
llm = LLM.deepseek(key: ENV["KEY"])
|
|
181
|
+
agent = Researcher.new(llm)
|
|
182
|
+
agent.talk "Research the market"
|
|
183
|
+
```
|
|
184
|
+
|
|
185
|
+
#### Why would I use it?
|
|
186
|
+
|
|
187
|
+
A tool budget prevents runaway tool loops. A misbehaving model can
|
|
188
|
+
otherwise keep calling tools, spending tokens on every round trip.
|
|
189
|
+
Capping the budget turns that into a bounded conversation: after
|
|
190
|
+
the cap, the model is told it has run out of tool calls and must
|
|
191
|
+
respond from what it has.
|
|
192
|
+
|
|
193
|
+
#### Notes
|
|
194
|
+
|
|
195
|
+
The budget is disabled by default (`nil`). Set it on a subclass
|
|
196
|
+
with the `tool_budget` DSL, through
|
|
197
|
+
[`LLM::Agent.set`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#set-class_method)
|
|
198
|
+
with `tool_budget:`, or per-instance with the `tool_budget:` keyword
|
|
199
|
+
argument to
|
|
200
|
+
[`LLM::Agent.new`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#initialize-instance_method).
|
|
201
|
+
This replaces the previous `tool_attempts` parameter, which is no
|
|
202
|
+
longer used.
|
|
@@ -0,0 +1,191 @@
|
|
|
1
|
+
|
|
2
|
+
## Built-in tools
|
|
3
|
+
|
|
4
|
+
### Introduction
|
|
5
|
+
|
|
6
|
+
#### Overview
|
|
7
|
+
|
|
8
|
+
llm.rb ships with twelve ready-made tools that cover the operations
|
|
9
|
+
a coding or system agent needs most: filesystem work, search, and
|
|
10
|
+
shell commands. Each tool is a subclass of
|
|
11
|
+
[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html)
|
|
12
|
+
with a name, description, and typed parameters, exactly like a tool
|
|
13
|
+
you would write yourself. Load the whole catalog with
|
|
14
|
+
`require "llm/tools"`.
|
|
15
|
+
|
|
16
|
+
#### How it works
|
|
17
|
+
|
|
18
|
+
When you want to attach every built-in tool to an agent, require
|
|
19
|
+
the catalog and pass the full set of subclasses as the `tools:`
|
|
20
|
+
option. The runtime registers each tool's schema and lets the model
|
|
21
|
+
decide when to call it.
|
|
22
|
+
|
|
23
|
+
```ruby
|
|
24
|
+
require "llm"
|
|
25
|
+
require "llm/tools"
|
|
26
|
+
|
|
27
|
+
llm = LLM.deepseek(key: ENV["KEY"])
|
|
28
|
+
agent = LLM::Agent.new(llm, tools: LLM::Tool.subclasses)
|
|
29
|
+
agent.talk "List the files in this repository"
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
#### Why would I use it?
|
|
33
|
+
|
|
34
|
+
The built-in tools cover the operations a coding or system agent
|
|
35
|
+
needs most, so you rarely need to write your own. They also handle
|
|
36
|
+
edge cases correctly: subprocesses get timeouts, interrupts kill
|
|
37
|
+
the child process, and failed calls return structured errors
|
|
38
|
+
instead of crashing the conversation.
|
|
39
|
+
|
|
40
|
+
#### Notes
|
|
41
|
+
|
|
42
|
+
The tools that spawn subprocesses use the optional `test-cmd.rb`
|
|
43
|
+
gem for process management and interrupt handling. The gem is
|
|
44
|
+
required only when those tools load. Every tool returns a Hash with
|
|
45
|
+
an `ok:` key that tells the model whether the call succeeded.
|
|
46
|
+
|
|
47
|
+
### Filesystem
|
|
48
|
+
|
|
49
|
+
#### Overview
|
|
50
|
+
|
|
51
|
+
The filesystem tools let the model read, write, and edit files,
|
|
52
|
+
list and create directories, and move around the working tree.
|
|
53
|
+
The most distinctive is
|
|
54
|
+
[`LLM::Tool::EditFile`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/EditFile.html),
|
|
55
|
+
which replaces an exact snippet and verifies the match count:
|
|
56
|
+
|
|
57
|
+
```ruby
|
|
58
|
+
LLM::Tool::EditFile.new.call(
|
|
59
|
+
path: "config.yml",
|
|
60
|
+
before: "port: 3000",
|
|
61
|
+
after: "port: 4000"
|
|
62
|
+
)
|
|
63
|
+
# => {ok: true, replaced: 1}
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
#### How it works
|
|
67
|
+
|
|
68
|
+
Each filesystem tool takes a path and returns a result Hash. The
|
|
69
|
+
`read-file` tool reads a whole file by default and accepts `start:`
|
|
70
|
+
and `stop:` to read a range of lines instead. The `edit-file` tool
|
|
71
|
+
counts occurrences of `before` and raises unless the count matches
|
|
72
|
+
`expected_count`, which defaults to 1.
|
|
73
|
+
|
|
74
|
+
| Tool | Name | Parameters | Purpose |
|
|
75
|
+
|---|---|---|---|
|
|
76
|
+
| [`LLM::Tool::Pwd`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Pwd.html) | `pwd` | none | Report the current working directory |
|
|
77
|
+
| [`LLM::Tool::Ls`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Ls.html) | `ls` | `path`, `glob` | List files and directories, optionally matching a glob |
|
|
78
|
+
| [`LLM::Tool::Chdir`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Chdir.html) | `chdir` | `path` | Change the current working directory |
|
|
79
|
+
| [`LLM::Tool::Mkdir`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Mkdir.html) | `mkdir` | `path` | Create a tree of directories |
|
|
80
|
+
| [`LLM::Tool::ReadFile`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/ReadFile.html) | `read-file` | `path`, `start`, `stop` | Read a file, optionally a range of lines |
|
|
81
|
+
| [`LLM::Tool::WriteFile`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/WriteFile.html) | `write-file` | `path`, `content` | Write a string to a file |
|
|
82
|
+
| [`LLM::Tool::EditFile`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/EditFile.html) | `edit-file` | `path`, `before`, `after`, `expected_count` | Replace an exact snippet in a file |
|
|
83
|
+
|
|
84
|
+
#### Why would I use it?
|
|
85
|
+
|
|
86
|
+
Reading source files to answer questions, writing new files, and
|
|
87
|
+
editing a snippet in place are the bread and butter of a coding
|
|
88
|
+
agent. The filesystem tools give the model all of it without you
|
|
89
|
+
writing a single tool.
|
|
90
|
+
|
|
91
|
+
#### Notes
|
|
92
|
+
|
|
93
|
+
The `chdir` tool changes the working directory for the whole
|
|
94
|
+
process, so subsequent file operations see the new directory. The
|
|
95
|
+
`mkdir` tool creates parent directories, like `mkdir -p`. The `ls`
|
|
96
|
+
tool raises when the path does not exist.
|
|
97
|
+
|
|
98
|
+
### Search
|
|
99
|
+
|
|
100
|
+
#### Overview
|
|
101
|
+
|
|
102
|
+
The search tools find things without reading the whole tree: `rg`
|
|
103
|
+
searches file contents, and `which` locates an executable on the
|
|
104
|
+
PATH. When a method name or a term is needed, the model searches
|
|
105
|
+
for it instead of guessing.
|
|
106
|
+
|
|
107
|
+
```ruby
|
|
108
|
+
LLM::Tool::Rg.new.call(patterns: ["def talk"], path: "lib")
|
|
109
|
+
# => {ok: true, stdout: "lib/llm/context.rb:42:def talk", stderr: ""}
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
#### How it works
|
|
113
|
+
|
|
114
|
+
When you want to search for one or more patterns, call the
|
|
115
|
+
[`LLM::Tool::Rg#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Rg.html#call-instance_method)
|
|
116
|
+
method with an Array of patterns, an optional `path:`, and a
|
|
117
|
+
`timeout:` in seconds. When you want to check whether a command is
|
|
118
|
+
installed, call the
|
|
119
|
+
[`LLM::Tool::Which#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Which.html#call-instance_method)
|
|
120
|
+
method with a `name:`.
|
|
121
|
+
|
|
122
|
+
| Tool | Name | Parameters | Purpose |
|
|
123
|
+
|---|---|---|---|
|
|
124
|
+
| [`LLM::Tool::Rg`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Rg.html) | `rg` | `patterns`, `path`, `timeout` | Recursively search for lines matching patterns |
|
|
125
|
+
| [`LLM::Tool::Which`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Which.html) | `which` | `name` | Locate an executable on the system PATH |
|
|
126
|
+
|
|
127
|
+
#### Why would I use it?
|
|
128
|
+
|
|
129
|
+
Search is how the model finds something without scanning files one
|
|
130
|
+
by one. Asked where a method is defined, the agent searches for it.
|
|
131
|
+
Before running a command, it checks with `which` that the binary
|
|
132
|
+
exists, and falls back to another approach when it does not.
|
|
133
|
+
|
|
134
|
+
#### Notes
|
|
135
|
+
|
|
136
|
+
The `rg` tool runs the ripgrep binary, so `rg` must be installed
|
|
137
|
+
and on the PATH. It refuses to search from the filesystem root and
|
|
138
|
+
rejects the pattern `.` to keep the model from dumping the entire
|
|
139
|
+
tree in one call. The `which` tool is pure Ruby: it scans the PATH
|
|
140
|
+
in order and returns the first directory that contains an
|
|
141
|
+
executable with the given name. When no match is found it returns
|
|
142
|
+
`{ok: false, path: nil}`.
|
|
143
|
+
|
|
144
|
+
### Shell
|
|
145
|
+
|
|
146
|
+
#### Overview
|
|
147
|
+
|
|
148
|
+
The shell tools run real subprocesses: arbitrary commands through
|
|
149
|
+
`shell`, Ruby code through `ruby`, and a fixed set of git actions
|
|
150
|
+
through `git`. All three accept a `timeout:` and kill the child
|
|
151
|
+
process when the model interrupts the turn.
|
|
152
|
+
|
|
153
|
+
```ruby
|
|
154
|
+
LLM::Tool::Shell.new.call(
|
|
155
|
+
name: "bundle",
|
|
156
|
+
arguments: ["exec", "rspec", "spec/llm"],
|
|
157
|
+
timeout: 30
|
|
158
|
+
)
|
|
159
|
+
```
|
|
160
|
+
|
|
161
|
+
#### How it works
|
|
162
|
+
|
|
163
|
+
When you want to run a command and capture its output, call the
|
|
164
|
+
[`LLM::Tool::Shell#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Shell.html#call-instance_method)
|
|
165
|
+
method with a `name:` and optional `arguments:`. The `git` tool
|
|
166
|
+
accepts an `action:` from a fixed set, and the `ruby` tool runs its
|
|
167
|
+
code in a fresh process.
|
|
168
|
+
|
|
169
|
+
| Tool | Name | Parameters | Purpose |
|
|
170
|
+
|---|---|---|---|
|
|
171
|
+
| [`LLM::Tool::Shell`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Shell.html) | `shell` | `name`, `arguments`, `timeout` | Run a shell command |
|
|
172
|
+
| [`LLM::Tool::Git`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Git.html) | `git` | `action`, `arguments`, `timeout` | Run a fixed set of git actions |
|
|
173
|
+
| [`LLM::Tool::Ruby`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Ruby.html) | `ruby` | `code`, `timeout` | Run a string of Ruby code |
|
|
174
|
+
|
|
175
|
+
#### Why would I use it?
|
|
176
|
+
|
|
177
|
+
Running tests, inspecting git history, and executing a snippet of
|
|
178
|
+
Ruby are things a coding agent needs to do. The shell tools make
|
|
179
|
+
those actions first-class, and the timeout keeps a hanging command
|
|
180
|
+
from stalling the conversation.
|
|
181
|
+
|
|
182
|
+
#### Notes
|
|
183
|
+
|
|
184
|
+
[`LLM::Tool::Shell`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Shell.html)
|
|
185
|
+
can be dangerous given a low-quality model. Gate it behind
|
|
186
|
+
[`LLM::Agent#confirm`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#confirm)
|
|
187
|
+
or manage the tool loop manually through
|
|
188
|
+
[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html).
|
|
189
|
+
On interrupt, the running child process is killed. The `ruby` tool
|
|
190
|
+
uses the same Ruby that launched llm.rb. The `git` tool wraps the
|
|
191
|
+
actions `log`, `diff`, `commit`, `checkout`, `branch`, and `show`.
|
|
@@ -0,0 +1,104 @@
|
|
|
1
|
+
|
|
2
|
+
## Concurrency
|
|
3
|
+
|
|
4
|
+
### Introduction
|
|
5
|
+
|
|
6
|
+
#### Overview
|
|
7
|
+
|
|
8
|
+
llm.rb supports six concurrency strategies for tool execution:
|
|
9
|
+
`:sequential`, `:thread`, `:fiber`, `:async`, `:fork`, and
|
|
10
|
+
`:ractor`. Each implements the same interface
|
|
11
|
+
([`LLM::Function::Task#spawn`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#spawn),
|
|
12
|
+
[`LLM::Function::Task#wait`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#wait),
|
|
13
|
+
[`LLM::Function::Task#alive?`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#alive?),
|
|
14
|
+
[`LLM::Function::Task#interrupt!`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#interrupt!)),
|
|
15
|
+
so the caller never has to care which
|
|
16
|
+
strategy is behind a given task.
|
|
17
|
+
|
|
18
|
+
#### How it works
|
|
19
|
+
|
|
20
|
+
Each strategy creates a task object that wraps a function call.
|
|
21
|
+
[`LLM::Function::Task#spawn`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#spawn)
|
|
22
|
+
starts execution,
|
|
23
|
+
[`LLM::Function::Task#wait`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#wait)
|
|
24
|
+
collects the result,
|
|
25
|
+
[`LLM::Function::Task#alive?`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#alive?)
|
|
26
|
+
checks whether the task is still running, and
|
|
27
|
+
[`LLM::Function::Task#interrupt!`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#interrupt!)
|
|
28
|
+
signals the task to stop.
|
|
29
|
+
|
|
30
|
+
Interruption is reliable across all six.
|
|
31
|
+
[`LLM::Interrupt`](https://r.uby.dev/api-docs/llm.rb/LLM/Interrupt.html)
|
|
32
|
+
reaches the tool regardless of whether it runs on a thread, fiber, process,
|
|
33
|
+
or ractor.
|
|
34
|
+
|
|
35
|
+
A tool can rescue
|
|
36
|
+
[`LLM::Interrupt`](https://r.uby.dev/api-docs/llm.rb/LLM/Interrupt.html)
|
|
37
|
+
and return a structured error to cancel the tool call, or it can
|
|
38
|
+
re-raise to cancel the turn entirely. When you re-raise
|
|
39
|
+
[`LLM::Interrupt`](https://r.uby.dev/api-docs/llm.rb/LLM/Interrupt.html)
|
|
40
|
+
or leave it unrescued the exception is raised on the caller's thread
|
|
41
|
+
and that effectively terminates the turn.
|
|
42
|
+
|
|
43
|
+
The main benefit of rescuing yourself is to let the tool clean up
|
|
44
|
+
and free lingering resources, but you usually want to re-raise so
|
|
45
|
+
the cancel request ends the turn. Otherwise the tool call is
|
|
46
|
+
intercepted but the turn continues.
|
|
47
|
+
|
|
48
|
+
```ruby
|
|
49
|
+
##
|
|
50
|
+
# At a high-level
|
|
51
|
+
agent = LLM::Agent.new(llm, concurrency: :fork, tools: [...])
|
|
52
|
+
agent.talk "Run the tools"
|
|
53
|
+
|
|
54
|
+
##
|
|
55
|
+
# At a low-level
|
|
56
|
+
fn = FetchStocks.function
|
|
57
|
+
fn.task(:thread).wait
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
#### Why would I use it?
|
|
61
|
+
|
|
62
|
+
Different tools have different execution requirements. IO-bound
|
|
63
|
+
tools benefit from thread or async concurrency. CPU-bound tools
|
|
64
|
+
benefit from ractors or forks. Tools that might crash the process
|
|
65
|
+
should run in a fork for isolation.
|
|
66
|
+
|
|
67
|
+
The common interface means you can change strategies without
|
|
68
|
+
changing your tool code.
|
|
69
|
+
|
|
70
|
+
#### Notes
|
|
71
|
+
|
|
72
|
+
**sequential**: Tools run one at a time on the calling thread. No
|
|
73
|
+
overhead. Best for simple agents or when tool order matters.
|
|
74
|
+
|
|
75
|
+
**thread**: Each tool runs in its own Thread. Releases the GVL
|
|
76
|
+
during blocking IO. Best for IO-bound tools like HTTP calls and
|
|
77
|
+
database queries.
|
|
78
|
+
|
|
79
|
+
**fiber**: Each tool runs in a scheduler-backed Fiber. Requires
|
|
80
|
+
`Fiber.scheduler`. Best for IO-bound tools inside an async
|
|
81
|
+
framework. Much lighter than threads.
|
|
82
|
+
|
|
83
|
+
**async**: Each tool runs as an `Async::Task` inside a managed
|
|
84
|
+
background reactor. Best for IO-bound tools when you want Async's
|
|
85
|
+
structured concurrency model without running your whole application
|
|
86
|
+
inside a reactor. Requires the `async` gem.
|
|
87
|
+
|
|
88
|
+
**fork**: Each tool runs in a forked child process. True parallelism
|
|
89
|
+
and process isolation. Best for shell commands, native extensions,
|
|
90
|
+
or anything you do not want touching the parent's memory. Requires
|
|
91
|
+
the `xchan` gem.
|
|
92
|
+
|
|
93
|
+
**ractor**: Each class-based tool runs in a Ruby Ractor. True
|
|
94
|
+
parallelism without the overhead of forking full processes. Only
|
|
95
|
+
class-based tools are supported. Arguments must be ractor-shareable.
|
|
96
|
+
|
|
97
|
+
| Strategy | Backing | Parallel? | Isolation? | Requires |
|
|
98
|
+
|---|---|---|---|---|
|
|
99
|
+
| `:sequential` | direct call | No | No | None |
|
|
100
|
+
| `:thread` | `Thread` | IO only (GVL) | No | None |
|
|
101
|
+
| `:fiber` | `Fiber.schedule` | Cooperative | No | `Fiber.scheduler` |
|
|
102
|
+
| `:async` | `Async::Reactor` | Cooperative | No | `async` gem |
|
|
103
|
+
| `:fork` | `Kernel.fork` | Yes (process) | Yes (memory) | `xchan` gem |
|
|
104
|
+
| `:ractor` | `Ractor` | Yes (CPU) | Limited | None |
|