llm.rb 13.0.0 → 14.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (106) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +505 -14
  3. data/README.md +484 -50
  4. data/bin/llm.rb +148 -0
  5. data/data/anthropic.json +206 -263
  6. data/data/bedrock.json +2138 -1860
  7. data/data/deepinfra.json +1003 -624
  8. data/data/deepseek.json +38 -34
  9. data/data/google.json +1079 -371
  10. data/data/mistral.json +448 -368
  11. data/data/moonshot.json +384 -0
  12. data/data/openai.json +974 -1343
  13. data/data/xai.json +154 -126
  14. data/data/zai.json +191 -191
  15. data/lib/llm/agent.rb +123 -20
  16. data/lib/llm/context.rb +71 -88
  17. data/lib/llm/cost.rb +23 -17
  18. data/lib/llm/error.rb +0 -8
  19. data/lib/llm/function/array.rb +3 -3
  20. data/lib/llm/function/async/task.rb +2 -0
  21. data/lib/llm/function/fiber/task.rb +2 -0
  22. data/lib/llm/function/fork/task.rb +2 -0
  23. data/lib/llm/function/ractor/task.rb +2 -0
  24. data/lib/llm/function/sequential/group.rb +4 -1
  25. data/lib/llm/function/sequential/task.rb +1 -1
  26. data/lib/llm/function/task.rb +4 -0
  27. data/lib/llm/function/thread/task.rb +2 -0
  28. data/lib/llm/function.rb +33 -6
  29. data/lib/llm/guard/loop.rb +89 -0
  30. data/lib/llm/guard/null.rb +19 -0
  31. data/lib/llm/guard.rb +61 -0
  32. data/lib/llm/provider.rb +36 -0
  33. data/lib/llm/providers/anthropic/stream_parser.rb +1 -0
  34. data/lib/llm/providers/anthropic.rb +2 -9
  35. data/lib/llm/providers/bedrock/request_adapter.rb +1 -1
  36. data/lib/llm/providers/bedrock/stream_parser.rb +1 -0
  37. data/lib/llm/providers/bedrock.rb +1 -8
  38. data/lib/llm/providers/google/stream_parser.rb +1 -0
  39. data/lib/llm/providers/google.rb +1 -8
  40. data/lib/llm/providers/mistral.rb +1 -1
  41. data/lib/llm/providers/moonshot.rb +76 -0
  42. data/lib/llm/providers/ollama.rb +2 -9
  43. data/lib/llm/providers/openai/responses/stream_parser.rb +1 -0
  44. data/lib/llm/providers/openai/responses.rb +7 -9
  45. data/lib/llm/providers/openai/stream_parser.rb +1 -0
  46. data/lib/llm/providers/openai.rb +4 -11
  47. data/lib/llm/repl/bar.rb +4 -3
  48. data/lib/llm/repl/{transcript.rb → buffer.rb} +69 -29
  49. data/lib/llm/repl/color.rb +78 -0
  50. data/lib/llm/repl/command.rb +12 -5
  51. data/lib/llm/repl/commands/compact.rb +2 -2
  52. data/lib/llm/repl/commands/help.rb +3 -5
  53. data/lib/llm/repl/input/char.rb +46 -0
  54. data/lib/llm/repl/input/row.rb +39 -0
  55. data/lib/llm/repl/input.rb +251 -66
  56. data/lib/llm/repl/markdown/table.rb +11 -3
  57. data/lib/llm/repl/markdown.rb +34 -8
  58. data/lib/llm/repl/node.rb +37 -0
  59. data/lib/llm/repl/status.rb +42 -7
  60. data/lib/llm/repl/stream.rb +18 -6
  61. data/lib/llm/repl/walker.rb +3 -2
  62. data/lib/llm/repl/window.rb +54 -35
  63. data/lib/llm/repl.rb +74 -32
  64. data/lib/llm/skill.rb +20 -4
  65. data/lib/llm/stream.rb +8 -7
  66. data/lib/llm/tool.rb +29 -0
  67. data/lib/llm/tools/{swap_text.rb → edit-file.rb} +3 -3
  68. data/lib/llm/tools/git.rb +3 -0
  69. data/lib/llm/tools/mkdir.rb +3 -0
  70. data/lib/llm/tools/rg.rb +3 -0
  71. data/lib/llm/tools/ruby.rb +46 -0
  72. data/lib/llm/tools/shell.rb +3 -0
  73. data/lib/llm/tracer/pretty_logger.rb +127 -0
  74. data/lib/llm/tracer.rb +1 -0
  75. data/lib/llm/transformer/null.rb +21 -0
  76. data/lib/llm/transformer.rb +55 -0
  77. data/lib/llm/version.rb +1 -1
  78. data/lib/llm.rb +12 -2
  79. data/llm.gemspec +9 -2
  80. data/resources/deepdive/advanced/cancellation.md +74 -0
  81. data/resources/deepdive/advanced/compaction.md +83 -0
  82. data/resources/deepdive/advanced/context.md +267 -0
  83. data/resources/deepdive/advanced/guard.md +371 -0
  84. data/resources/deepdive/advanced/tracer.md +180 -0
  85. data/resources/deepdive/advanced/transformer.md +67 -0
  86. data/resources/deepdive/advanced/transports.md +45 -0
  87. data/resources/deepdive/everything_else/audio.md +122 -0
  88. data/resources/deepdive/everything_else/cost.md +99 -0
  89. data/resources/deepdive/everything_else/images.md +89 -0
  90. data/resources/deepdive/everything_else/object.md +108 -0
  91. data/resources/deepdive/everything_else/ocr.md +48 -0
  92. data/resources/deepdive/fundamentals/agents.md +202 -0
  93. data/resources/deepdive/fundamentals/builtin_tools.md +191 -0
  94. data/resources/deepdive/fundamentals/concurrency.md +104 -0
  95. data/resources/deepdive/fundamentals/database.md +449 -0
  96. data/resources/deepdive/fundamentals/embeddings.md +157 -0
  97. data/resources/deepdive/fundamentals/repl.md +87 -0
  98. data/resources/deepdive/fundamentals/schema.md +61 -0
  99. data/resources/deepdive/fundamentals/skills.md +106 -0
  100. data/resources/deepdive/fundamentals/stream.md +110 -0
  101. data/resources/deepdive/fundamentals/tools.md +265 -0
  102. data/resources/deepdive/protocols/a2a.md +106 -0
  103. data/resources/deepdive/protocols/mcp.md +111 -0
  104. data/resources/deepdive.md +58 -1792
  105. metadata +51 -7
  106. data/lib/llm/loop_guard.rb +0 -107
@@ -0,0 +1,108 @@
1
+
2
+ ## LLM::Object
3
+
4
+ ### Introduction
5
+
6
+ #### Overview
7
+
8
+ [`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html)
9
+ is the hash-like object that llm.rb uses everywhere structured data
10
+ flows through the runtime. Response bodies, tool arguments, schema
11
+ results, usage and cost data, and function parameters all come back
12
+ as [`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html)
13
+ instances. It is similar in spirit to OpenStruct,
14
+ and it was introduced after OpenStruct became a bundled gem rather
15
+ than a default gem in Ruby 3.5.
16
+
17
+ #### How it works
18
+
19
+ When you want to read a value from an
20
+ [`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html),
21
+ use either method-style or bracket access. Keys are indifferent, so
22
+ strings and symbols work interchangeably:
23
+
24
+ ```ruby
25
+ obj = LLM::Object.from(city: "Paris", temperature: 15.0)
26
+
27
+ obj.city # => "Paris"
28
+ obj["city"] # => "Paris"
29
+ obj[:city] # => "Paris"
30
+ obj[:temperature] # => 15.0
31
+ ```
32
+
33
+ Nested hashes and arrays are converted recursively, so deep chains
34
+ read naturally:
35
+
36
+ ```ruby
37
+ obj = LLM::Object.from(person: {name: "John"})
38
+ obj.person.name # => "John"
39
+ obj.person.class # => LLM::Object
40
+ ```
41
+
42
+ #### Why would I use it?
43
+
44
+ Most of the time you do not construct
45
+ [`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html)
46
+ instances yourself. They come back from `talk`, `ask`, `embed`, and
47
+ every other call that returns structured data. Knowing how they
48
+ behave lets you read response fields, pass tool arguments, and
49
+ inspect usage without reaching for `to_h` on every line.
50
+
51
+ #### Notes
52
+
53
+ An
54
+ [`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html)
55
+ is enumerable and supports the usual Hash operations: `keys`,
56
+ `values`, `key?`, `fetch`, `dig`, `slice`, `merge`, `merge!`,
57
+ `delete`, `size`, and `empty?`. Use `to_h` for a plain Hash and
58
+ `to_hash` for one with symbol keys. A missing key returns `nil`
59
+ rather than raising. Because it subclasses `BasicObject`, `to_json`
60
+ is defined explicitly and serializes through the configured JSON
61
+ adapter via
62
+ [`LLM.json.dump`](https://r.uby.dev/api-docs/llm.rb/LLM.html#json-class_method).
63
+
64
+ ### Reading and writing
65
+
66
+ #### Overview
67
+
68
+ Beyond simple reads,
69
+ [`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html)
70
+ supports assignment, mutation, and iteration, so you can treat it
71
+ like a Hash in place.
72
+
73
+ #### How it works
74
+
75
+ Assign values with method or bracket syntax, mutate in place, and
76
+ iterate like a Hash:
77
+
78
+ ```ruby
79
+ obj = LLM::Object.from({})
80
+
81
+ obj.city = "Paris" # method-style write
82
+ obj["country"] = "France"
83
+
84
+ obj.key?(:city) # => true
85
+ obj.keys # => ["city", "country"]
86
+ obj.merge!(population: 2_100_000)
87
+
88
+ obj.each { |key, value| puts "#{key}: #{value}" }
89
+ obj.transform_values!(&:upcase) if obj.any?
90
+ ```
91
+
92
+ #### Why would I use it?
93
+
94
+ Tool implementations receive their arguments as an
95
+ [`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html)
96
+ and often build a result Hash from them. Merging defaults, deleting
97
+ optional keys, and transforming values in place keeps that code
98
+ concise without converting back and forth between Hash and
99
+ [`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html).
100
+
101
+ #### Notes
102
+
103
+ `merge` returns a new
104
+ [`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html);
105
+ `merge!` mutates in place.
106
+ Assignment always stores the key as a string internally, which is
107
+ why string and symbol lookups both work. Equality compares against
108
+ anything that responds to `to_h`.
@@ -0,0 +1,48 @@
1
+
2
+ ## OCR
3
+
4
+ ### Introduction
5
+
6
+ #### Overview
7
+
8
+ OCR pulls text out of images and PDFs so you can search, index,
9
+ or process scanned content. Mistral is the only provider with a
10
+ dedicated OCR endpoint, and it accepts both image URLs and
11
+ document URLs.
12
+
13
+ #### How it works
14
+
15
+ Mistral is the only provider with a dedicated OCR endpoint. The
16
+ [`LLM::Provider#ocr`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#ocr)
17
+ method accepts an `image_url:` or `document_url:` parameter.
18
+ Document URLs can point to PDFs.
19
+
20
+ ```ruby
21
+ require "llm"
22
+
23
+ llm = LLM.mistral(key: ENV["KEY"])
24
+
25
+ # Extract text from an image
26
+ res = llm.ocr(image_url: "https://example.com/photo.png")
27
+ res.pages.each { |page| puts page.markdown }
28
+
29
+ # Extract text from a PDF
30
+ res = llm.ocr(document_url: "https://example.com/report.pdf")
31
+ res.pages.each { |page| puts page.markdown }
32
+ ```
33
+
34
+ #### Why would I use it?
35
+
36
+ OCR extracts text from scanned documents and images. The result
37
+ feeds into search indexes, extraction pipelines that pull out dates
38
+ and amounts, or archives so scanned contracts become searchable
39
+ records. PDFs and images both work, and the response is structured
40
+ per page with markdown.
41
+
42
+ #### Notes
43
+
44
+ Only Mistral currently supports OCR through the llm.rb runtime.
45
+ The response exposes pages through
46
+ [`LLM::OCR::Response#pages`](https://r.uby.dev/api-docs/llm.rb/LLM/OCR/Response.html#pages),
47
+ where each page
48
+ has a `markdown` field containing the extracted text.
@@ -0,0 +1,202 @@
1
+
2
+ ## Agents
3
+ ### Introduction
4
+
5
+ #### Overview
6
+
7
+ [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
8
+ is the recommended entry point for most use-cases. It provides a
9
+ class-level DSL for defining reusable, preconfigured assistants
10
+ with defaults for model, tools, schema, and instructions. Under
11
+ the hood it delegates to
12
+ [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html),
13
+ so it has the same runtime surface: message history, streaming,
14
+ serialization, compaction, and concurrency.
15
+
16
+ #### How it works
17
+
18
+ An agent holds a conversation with a model. You send input with
19
+ [`LLM::Agent#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#talk),
20
+ the model responds, and if it requests tools the agent
21
+ executes them automatically and feeds the results back. It enables
22
+ [a loop guard by default](https://r.uby.dev/llm/deepdive/advanced/guard)
23
+ that detects repeated tool-call patterns
24
+ and blocks stuck execution. The tool loop can also be bounded with
25
+ [`LLM::Agent.tool_budget`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#tool_budget-class_method)
26
+ (see the Tool budget section). Instructions are injected once
27
+ unless a system message is already present.
28
+
29
+ #### Why would I use it?
30
+
31
+ Agents manage the tool loop for you. They guard against infinite
32
+ loops, keep conversation state across turns, and let you define
33
+ reusable configurations at the class level. If you need manual
34
+ control over the tool loop, use
35
+ [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
36
+ directly instead.
37
+
38
+ #### Notes
39
+
40
+ Agents support the same concurrency strategies, compaction,
41
+ cancellation, and serialization as contexts. The trade-off between
42
+ a subclass and a direct instance is only in how the agent is
43
+ organized, not in what it can do. Tool loop execution can be
44
+ configured with `concurrency: :sequential`, `:thread`, `:async`,
45
+ `:fiber`, `:fork`, or `:ractor`.
46
+
47
+ ### Class-based
48
+
49
+ #### Overview
50
+
51
+ A subclass of
52
+ [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
53
+ gives you a reusable agent with its own behavior. You define the model, tools, and other
54
+ attributes at the class level, and each instance picks them up
55
+ as defaults. Attributes can be overridden per-instance, and they
56
+ can be plain values, blocks, or Symbols that resolve to methods.
57
+ The class becomes a self-contained worker that you can instantiate
58
+ and talk to from anywhere.
59
+
60
+ #### How it works
61
+
62
+ A subclass declares its defaults with
63
+ [`LLM::Agent.set`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#set-class_method).
64
+ Each key is a
65
+ class-level accessor: `name`, `description`, `model`, `tools`,
66
+ `skills`, `instructions`, `stream`, `tracer`, `concurrency`,
67
+ `schema`, `confirm`, `path`, `tool_budget`.
68
+ Keyword arguments in the constructor override these defaults.
69
+
70
+ ```ruby
71
+ class Agent < LLM::Agent
72
+ set model: "deepseek-v4-pro",
73
+ description: "system administration agent",
74
+ tools: [Shell]
75
+ end
76
+
77
+ llm = LLM.openai(key: ENV["KEY"])
78
+ agent = Agent.new(llm)
79
+ agent.talk "Run 'date'"
80
+ ```
81
+
82
+ #### Why would I use it?
83
+
84
+ A subclass is useful when multiple parts of an application need to
85
+ call the same agent. The configuration and any helper methods live in one place. Define a
86
+ `research!` method that kicks off the agent's work. The subclass
87
+ becomes a self-contained worker.
88
+
89
+ #### Notes
90
+
91
+ Attributes passed to
92
+ [`LLM::Agent.set`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#set-class_method)
93
+ can be plain values, blocks, or
94
+ Symbols. A Symbol is evaluated as an instance method on the
95
+ subclass, so `tracer: :set_tracer` calls `set_tracer` on the
96
+ instance. A block like `stream: -> { $stdout }` is evaluated
97
+ when the attribute is first accessed.
98
+
99
+ Set `path:` on a subclass or instance for automatic filesystem
100
+ persistence; the agent restores conversation history from the
101
+ file on startup and saves it back after every turn with no
102
+ manual
103
+ [`LLM::Agent#save`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#save)/
104
+ [`LLM::Agent#restore`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#restore)
105
+ calls. See the
106
+ [database deepdive](../fundamentals/database.md) for details.
107
+
108
+ ### Object-based
109
+
110
+ #### Overview
111
+
112
+ An
113
+ [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
114
+ instance is the simplest way to get started. You pass a provider and any configuration as keyword
115
+ arguments, and the agent runs the tool loop and manages state
116
+ just like a subclass would. This is the right choice when you
117
+ are prototyping, running a one-off task, or when the agent's
118
+ configuration is determined at runtime.
119
+
120
+ #### How it works
121
+
122
+ A direct instance takes the same attributes as keyword arguments
123
+ to
124
+ [`LLM::Agent.new`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#initialize-instance_method).
125
+ The first argument is always the provider.
126
+ Everything else is optional. The agent runs the tool loop
127
+ and manages state under the hood through a
128
+ [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html).
129
+
130
+ ```ruby
131
+ llm = LLM.deepseek(key: ENV["KEY"])
132
+ agent = LLM::Agent.new(llm, stream: $stdout)
133
+ agent.talk "Hello world"
134
+ ```
135
+
136
+ #### Why would I use it?
137
+
138
+ A direct instance is the right choice for quick experiments, one-shot
139
+ tasks, or when defining a class would be overkill. It is also
140
+ the right choice when the agent's configuration is determined at
141
+ runtime and a class hierarchy adds unnecessary complexity.
142
+
143
+ #### Notes
144
+
145
+ Direct instances accept all the same options as subclasses. The
146
+ difference is only in how the agent is organized, not in what it
147
+ can do. Under the hood,
148
+ [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
149
+ creates a
150
+ [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
151
+ that manages the message history.
152
+
153
+ ### Tool budget
154
+
155
+ #### Overview
156
+
157
+ [`LLM::Agent.tool_budget`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#tool_budget-class_method)
158
+ caps the number of tool calls allowed in a single turn. Once the
159
+ budget is spent, the agent sends an in-band advisory message back
160
+ through the model instead of running more tools. By default no
161
+ budget is set, so the feature is disabled.
162
+
163
+ #### How it works
164
+
165
+ When you want to bound how many tools an agent can call in one
166
+ turn, set the budget with
167
+ [`LLM::Agent.set`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#set-class_method).
168
+ The budget can be a plain number or a block evaluated against the
169
+ agent instance. Once the agent has made the budgeted number of tool
170
+ calls, it stops and returns an in-band advisory message describing
171
+ the spent budget. A model will usually change course afterwards:
172
+
173
+ ```ruby
174
+ class Researcher < LLM::Agent
175
+ set model: "deepseek-v4-pro",
176
+ tools: [FetchNews, FetchStocks],
177
+ tool_budget: 5
178
+ end
179
+
180
+ llm = LLM.deepseek(key: ENV["KEY"])
181
+ agent = Researcher.new(llm)
182
+ agent.talk "Research the market"
183
+ ```
184
+
185
+ #### Why would I use it?
186
+
187
+ A tool budget prevents runaway tool loops. A misbehaving model can
188
+ otherwise keep calling tools, spending tokens on every round trip.
189
+ Capping the budget turns that into a bounded conversation: after
190
+ the cap, the model is told it has run out of tool calls and must
191
+ respond from what it has.
192
+
193
+ #### Notes
194
+
195
+ The budget is disabled by default (`nil`). Set it on a subclass
196
+ with the `tool_budget` DSL, through
197
+ [`LLM::Agent.set`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#set-class_method)
198
+ with `tool_budget:`, or per-instance with the `tool_budget:` keyword
199
+ argument to
200
+ [`LLM::Agent.new`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#initialize-instance_method).
201
+ This replaces the previous `tool_attempts` parameter, which is no
202
+ longer used.
@@ -0,0 +1,191 @@
1
+
2
+ ## Built-in tools
3
+
4
+ ### Introduction
5
+
6
+ #### Overview
7
+
8
+ llm.rb ships with twelve ready-made tools that cover the operations
9
+ a coding or system agent needs most: filesystem work, search, and
10
+ shell commands. Each tool is a subclass of
11
+ [`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html)
12
+ with a name, description, and typed parameters, exactly like a tool
13
+ you would write yourself. Load the whole catalog with
14
+ `require "llm/tools"`.
15
+
16
+ #### How it works
17
+
18
+ When you want to attach every built-in tool to an agent, require
19
+ the catalog and pass the full set of subclasses as the `tools:`
20
+ option. The runtime registers each tool's schema and lets the model
21
+ decide when to call it.
22
+
23
+ ```ruby
24
+ require "llm"
25
+ require "llm/tools"
26
+
27
+ llm = LLM.deepseek(key: ENV["KEY"])
28
+ agent = LLM::Agent.new(llm, tools: LLM::Tool.subclasses)
29
+ agent.talk "List the files in this repository"
30
+ ```
31
+
32
+ #### Why would I use it?
33
+
34
+ The built-in tools cover the operations a coding or system agent
35
+ needs most, so you rarely need to write your own. They also handle
36
+ edge cases correctly: subprocesses get timeouts, interrupts kill
37
+ the child process, and failed calls return structured errors
38
+ instead of crashing the conversation.
39
+
40
+ #### Notes
41
+
42
+ The tools that spawn subprocesses use the optional `test-cmd.rb`
43
+ gem for process management and interrupt handling. The gem is
44
+ required only when those tools load. Every tool returns a Hash with
45
+ an `ok:` key that tells the model whether the call succeeded.
46
+
47
+ ### Filesystem
48
+
49
+ #### Overview
50
+
51
+ The filesystem tools let the model read, write, and edit files,
52
+ list and create directories, and move around the working tree.
53
+ The most distinctive is
54
+ [`LLM::Tool::EditFile`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/EditFile.html),
55
+ which replaces an exact snippet and verifies the match count:
56
+
57
+ ```ruby
58
+ LLM::Tool::EditFile.new.call(
59
+ path: "config.yml",
60
+ before: "port: 3000",
61
+ after: "port: 4000"
62
+ )
63
+ # => {ok: true, replaced: 1}
64
+ ```
65
+
66
+ #### How it works
67
+
68
+ Each filesystem tool takes a path and returns a result Hash. The
69
+ `read-file` tool reads a whole file by default and accepts `start:`
70
+ and `stop:` to read a range of lines instead. The `edit-file` tool
71
+ counts occurrences of `before` and raises unless the count matches
72
+ `expected_count`, which defaults to 1.
73
+
74
+ | Tool | Name | Parameters | Purpose |
75
+ |---|---|---|---|
76
+ | [`LLM::Tool::Pwd`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Pwd.html) | `pwd` | none | Report the current working directory |
77
+ | [`LLM::Tool::Ls`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Ls.html) | `ls` | `path`, `glob` | List files and directories, optionally matching a glob |
78
+ | [`LLM::Tool::Chdir`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Chdir.html) | `chdir` | `path` | Change the current working directory |
79
+ | [`LLM::Tool::Mkdir`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Mkdir.html) | `mkdir` | `path` | Create a tree of directories |
80
+ | [`LLM::Tool::ReadFile`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/ReadFile.html) | `read-file` | `path`, `start`, `stop` | Read a file, optionally a range of lines |
81
+ | [`LLM::Tool::WriteFile`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/WriteFile.html) | `write-file` | `path`, `content` | Write a string to a file |
82
+ | [`LLM::Tool::EditFile`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/EditFile.html) | `edit-file` | `path`, `before`, `after`, `expected_count` | Replace an exact snippet in a file |
83
+
84
+ #### Why would I use it?
85
+
86
+ Reading source files to answer questions, writing new files, and
87
+ editing a snippet in place are the bread and butter of a coding
88
+ agent. The filesystem tools give the model all of it without you
89
+ writing a single tool.
90
+
91
+ #### Notes
92
+
93
+ The `chdir` tool changes the working directory for the whole
94
+ process, so subsequent file operations see the new directory. The
95
+ `mkdir` tool creates parent directories, like `mkdir -p`. The `ls`
96
+ tool raises when the path does not exist.
97
+
98
+ ### Search
99
+
100
+ #### Overview
101
+
102
+ The search tools find things without reading the whole tree: `rg`
103
+ searches file contents, and `which` locates an executable on the
104
+ PATH. When a method name or a term is needed, the model searches
105
+ for it instead of guessing.
106
+
107
+ ```ruby
108
+ LLM::Tool::Rg.new.call(patterns: ["def talk"], path: "lib")
109
+ # => {ok: true, stdout: "lib/llm/context.rb:42:def talk", stderr: ""}
110
+ ```
111
+
112
+ #### How it works
113
+
114
+ When you want to search for one or more patterns, call the
115
+ [`LLM::Tool::Rg#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Rg.html#call-instance_method)
116
+ method with an Array of patterns, an optional `path:`, and a
117
+ `timeout:` in seconds. When you want to check whether a command is
118
+ installed, call the
119
+ [`LLM::Tool::Which#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Which.html#call-instance_method)
120
+ method with a `name:`.
121
+
122
+ | Tool | Name | Parameters | Purpose |
123
+ |---|---|---|---|
124
+ | [`LLM::Tool::Rg`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Rg.html) | `rg` | `patterns`, `path`, `timeout` | Recursively search for lines matching patterns |
125
+ | [`LLM::Tool::Which`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Which.html) | `which` | `name` | Locate an executable on the system PATH |
126
+
127
+ #### Why would I use it?
128
+
129
+ Search is how the model finds something without scanning files one
130
+ by one. Asked where a method is defined, the agent searches for it.
131
+ Before running a command, it checks with `which` that the binary
132
+ exists, and falls back to another approach when it does not.
133
+
134
+ #### Notes
135
+
136
+ The `rg` tool runs the ripgrep binary, so `rg` must be installed
137
+ and on the PATH. It refuses to search from the filesystem root and
138
+ rejects the pattern `.` to keep the model from dumping the entire
139
+ tree in one call. The `which` tool is pure Ruby: it scans the PATH
140
+ in order and returns the first directory that contains an
141
+ executable with the given name. When no match is found it returns
142
+ `{ok: false, path: nil}`.
143
+
144
+ ### Shell
145
+
146
+ #### Overview
147
+
148
+ The shell tools run real subprocesses: arbitrary commands through
149
+ `shell`, Ruby code through `ruby`, and a fixed set of git actions
150
+ through `git`. All three accept a `timeout:` and kill the child
151
+ process when the model interrupts the turn.
152
+
153
+ ```ruby
154
+ LLM::Tool::Shell.new.call(
155
+ name: "bundle",
156
+ arguments: ["exec", "rspec", "spec/llm"],
157
+ timeout: 30
158
+ )
159
+ ```
160
+
161
+ #### How it works
162
+
163
+ When you want to run a command and capture its output, call the
164
+ [`LLM::Tool::Shell#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Shell.html#call-instance_method)
165
+ method with a `name:` and optional `arguments:`. The `git` tool
166
+ accepts an `action:` from a fixed set, and the `ruby` tool runs its
167
+ code in a fresh process.
168
+
169
+ | Tool | Name | Parameters | Purpose |
170
+ |---|---|---|---|
171
+ | [`LLM::Tool::Shell`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Shell.html) | `shell` | `name`, `arguments`, `timeout` | Run a shell command |
172
+ | [`LLM::Tool::Git`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Git.html) | `git` | `action`, `arguments`, `timeout` | Run a fixed set of git actions |
173
+ | [`LLM::Tool::Ruby`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Ruby.html) | `ruby` | `code`, `timeout` | Run a string of Ruby code |
174
+
175
+ #### Why would I use it?
176
+
177
+ Running tests, inspecting git history, and executing a snippet of
178
+ Ruby are things a coding agent needs to do. The shell tools make
179
+ those actions first-class, and the timeout keeps a hanging command
180
+ from stalling the conversation.
181
+
182
+ #### Notes
183
+
184
+ [`LLM::Tool::Shell`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Shell.html)
185
+ can be dangerous given a low-quality model. Gate it behind
186
+ [`LLM::Agent#confirm`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#confirm)
187
+ or manage the tool loop manually through
188
+ [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html).
189
+ On interrupt, the running child process is killed. The `ruby` tool
190
+ uses the same Ruby that launched llm.rb. The `git` tool wraps the
191
+ actions `log`, `diff`, `commit`, `checkout`, `branch`, and `show`.
@@ -0,0 +1,104 @@
1
+
2
+ ## Concurrency
3
+
4
+ ### Introduction
5
+
6
+ #### Overview
7
+
8
+ llm.rb supports six concurrency strategies for tool execution:
9
+ `:sequential`, `:thread`, `:fiber`, `:async`, `:fork`, and
10
+ `:ractor`. Each implements the same interface
11
+ ([`LLM::Function::Task#spawn`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#spawn),
12
+ [`LLM::Function::Task#wait`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#wait),
13
+ [`LLM::Function::Task#alive?`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#alive?),
14
+ [`LLM::Function::Task#interrupt!`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#interrupt!)),
15
+ so the caller never has to care which
16
+ strategy is behind a given task.
17
+
18
+ #### How it works
19
+
20
+ Each strategy creates a task object that wraps a function call.
21
+ [`LLM::Function::Task#spawn`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#spawn)
22
+ starts execution,
23
+ [`LLM::Function::Task#wait`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#wait)
24
+ collects the result,
25
+ [`LLM::Function::Task#alive?`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#alive?)
26
+ checks whether the task is still running, and
27
+ [`LLM::Function::Task#interrupt!`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#interrupt!)
28
+ signals the task to stop.
29
+
30
+ Interruption is reliable across all six.
31
+ [`LLM::Interrupt`](https://r.uby.dev/api-docs/llm.rb/LLM/Interrupt.html)
32
+ reaches the tool regardless of whether it runs on a thread, fiber, process,
33
+ or ractor.
34
+
35
+ A tool can rescue
36
+ [`LLM::Interrupt`](https://r.uby.dev/api-docs/llm.rb/LLM/Interrupt.html)
37
+ and return a structured error to cancel the tool call, or it can
38
+ re-raise to cancel the turn entirely. When you re-raise
39
+ [`LLM::Interrupt`](https://r.uby.dev/api-docs/llm.rb/LLM/Interrupt.html)
40
+ or leave it unrescued the exception is raised on the caller's thread
41
+ and that effectively terminates the turn.
42
+
43
+ The main benefit of rescuing yourself is to let the tool clean up
44
+ and free lingering resources, but you usually want to re-raise so
45
+ the cancel request ends the turn. Otherwise the tool call is
46
+ intercepted but the turn continues.
47
+
48
+ ```ruby
49
+ ##
50
+ # At a high-level
51
+ agent = LLM::Agent.new(llm, concurrency: :fork, tools: [...])
52
+ agent.talk "Run the tools"
53
+
54
+ ##
55
+ # At a low-level
56
+ fn = FetchStocks.function
57
+ fn.task(:thread).wait
58
+ ```
59
+
60
+ #### Why would I use it?
61
+
62
+ Different tools have different execution requirements. IO-bound
63
+ tools benefit from thread or async concurrency. CPU-bound tools
64
+ benefit from ractors or forks. Tools that might crash the process
65
+ should run in a fork for isolation.
66
+
67
+ The common interface means you can change strategies without
68
+ changing your tool code.
69
+
70
+ #### Notes
71
+
72
+ **sequential**: Tools run one at a time on the calling thread. No
73
+ overhead. Best for simple agents or when tool order matters.
74
+
75
+ **thread**: Each tool runs in its own Thread. Releases the GVL
76
+ during blocking IO. Best for IO-bound tools like HTTP calls and
77
+ database queries.
78
+
79
+ **fiber**: Each tool runs in a scheduler-backed Fiber. Requires
80
+ `Fiber.scheduler`. Best for IO-bound tools inside an async
81
+ framework. Much lighter than threads.
82
+
83
+ **async**: Each tool runs as an `Async::Task` inside a managed
84
+ background reactor. Best for IO-bound tools when you want Async's
85
+ structured concurrency model without running your whole application
86
+ inside a reactor. Requires the `async` gem.
87
+
88
+ **fork**: Each tool runs in a forked child process. True parallelism
89
+ and process isolation. Best for shell commands, native extensions,
90
+ or anything you do not want touching the parent's memory. Requires
91
+ the `xchan` gem.
92
+
93
+ **ractor**: Each class-based tool runs in a Ruby Ractor. True
94
+ parallelism without the overhead of forking full processes. Only
95
+ class-based tools are supported. Arguments must be ractor-shareable.
96
+
97
+ | Strategy | Backing | Parallel? | Isolation? | Requires |
98
+ |---|---|---|---|---|
99
+ | `:sequential` | direct call | No | No | None |
100
+ | `:thread` | `Thread` | IO only (GVL) | No | None |
101
+ | `:fiber` | `Fiber.schedule` | Cooperative | No | `Fiber.scheduler` |
102
+ | `:async` | `Async::Reactor` | Cooperative | No | `async` gem |
103
+ | `:fork` | `Kernel.fork` | Yes (process) | Yes (memory) | `xchan` gem |
104
+ | `:ractor` | `Ractor` | Yes (CPU) | Limited | None |