llm.rb 16.0.0 → 17.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (88) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +535 -2
  3. data/README.md +857 -501
  4. data/data/alibaba.json +2 -2
  5. data/data/anthropic.json +66 -3
  6. data/data/bedrock.json +781 -12
  7. data/data/deepinfra.json +157 -1
  8. data/data/google.json +48 -0
  9. data/data/mistral.json +326 -16
  10. data/data/moonshot.json +2 -1
  11. data/data/openai.json +13 -0
  12. data/data/openrouter.json +1078 -1235
  13. data/data/xai.json +4 -2
  14. data/docs/deepdive/advanced/cancellation.md +35 -7
  15. data/docs/deepdive/advanced/compaction.md +5 -5
  16. data/docs/deepdive/advanced/context.md +45 -24
  17. data/docs/deepdive/advanced/{guard.md → intercept.md} +87 -83
  18. data/docs/deepdive/features/builtin_tools.md +1 -1
  19. data/docs/deepdive/features/concurrency.md +15 -10
  20. data/docs/deepdive/features/database.md +19 -19
  21. data/docs/deepdive/features/embeddings.md +4 -4
  22. data/docs/deepdive/features/files.md +7 -5
  23. data/docs/deepdive/fundamentals/agents.md +13 -8
  24. data/docs/deepdive/fundamentals/providers.md +1 -1
  25. data/docs/deepdive/fundamentals/schema.md +2 -2
  26. data/docs/deepdive/fundamentals/stream.md +8 -8
  27. data/docs/deepdive/fundamentals/tools.md +14 -14
  28. data/docs/deepdive/media/ocr.md +1 -1
  29. data/docs/deepdive/protocols/a2a.md +3 -3
  30. data/docs/deepdive/protocols/mcp.md +116 -7
  31. data/docs/deepdive/reference/cost.md +1 -1
  32. data/docs/deepdive/reference/response.md +1 -1
  33. data/docs/deepdive/reference/tracer.md +90 -22
  34. data/docs/deepdive.md +1 -1
  35. data/lib/llm/agent/interrupt.rb +35 -46
  36. data/lib/llm/agent/registry.rb +106 -0
  37. data/lib/llm/agent.rb +242 -268
  38. data/lib/llm/context.rb +63 -55
  39. data/lib/llm/function/array.rb +2 -1
  40. data/lib/llm/function/async/group.rb +2 -2
  41. data/lib/llm/function/async/task.rb +25 -48
  42. data/lib/llm/function/fiber/group.rb +2 -2
  43. data/lib/llm/function/fiber/task.rb +16 -30
  44. data/lib/llm/function/fork/group.rb +2 -2
  45. data/lib/llm/function/fork/job.rb +8 -4
  46. data/lib/llm/function/fork/task.rb +25 -30
  47. data/lib/llm/function/group.rb +33 -5
  48. data/lib/llm/function/ractor/group.rb +2 -2
  49. data/lib/llm/function/ractor/job.rb +8 -6
  50. data/lib/llm/function/ractor/task.rb +32 -9
  51. data/lib/llm/function/sequential/group.rb +2 -1
  52. data/lib/llm/function/sequential/task.rb +8 -1
  53. data/lib/llm/function/task.rb +52 -4
  54. data/lib/llm/function/thread/group.rb +2 -2
  55. data/lib/llm/function/thread/task.rb +36 -10
  56. data/lib/llm/function/window.rb +16 -0
  57. data/lib/llm/function.rb +14 -43
  58. data/lib/llm/intercept/multi.rb +40 -0
  59. data/lib/llm/intercept/null.rb +18 -0
  60. data/lib/llm/intercept.rb +96 -0
  61. data/lib/llm/message.rb +1 -1
  62. data/lib/llm/provider.rb +16 -1
  63. data/lib/llm/providers/anthropic/stream_parser.rb +1 -1
  64. data/lib/llm/providers/bedrock/stream_parser.rb +1 -1
  65. data/lib/llm/providers/deepseek/files.rb +49 -0
  66. data/lib/llm/providers/deepseek/request_adapter/completion.rb +15 -13
  67. data/lib/llm/providers/deepseek.rb +16 -2
  68. data/lib/llm/providers/google/stream_parser.rb +1 -1
  69. data/lib/llm/providers/openai/responses/stream_parser.rb +1 -1
  70. data/lib/llm/providers/openai/stream_parser.rb +1 -1
  71. data/lib/llm/stream.rb +8 -7
  72. data/lib/llm/tool.rb +1 -1
  73. data/lib/llm/tools/utils.rb +1 -1
  74. data/lib/llm/tracer/logger.rb +42 -1
  75. data/lib/llm/tracer/multi.rb +144 -0
  76. data/lib/llm/tracer/null.rb +20 -0
  77. data/lib/llm/tracer/pretty_logger.rb +33 -2
  78. data/lib/llm/tracer/rescue.rb +110 -0
  79. data/lib/llm/tracer/telemetry.rb +69 -4
  80. data/lib/llm/tracer.rb +125 -48
  81. data/lib/llm/utils.rb +12 -2
  82. data/lib/llm/version.rb +1 -1
  83. data/lib/llm.rb +36 -1
  84. data/llm.gemspec +9 -4
  85. metadata +16 -35
  86. data/lib/llm/function/tracing.rb +0 -21
  87. data/lib/llm/guard/null.rb +0 -19
  88. data/lib/llm/guard.rb +0 -55
@@ -201,7 +201,7 @@ from stalling the conversation.
201
201
 
202
202
  [`LLM::Tool::Exec`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Exec.html)
203
203
  can be dangerous given a low-quality model. Gate it behind
204
- [`LLM::Agent#confirm`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#confirm)
204
+ [`LLM::Agent#confirm`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#confirm-instance_method)
205
205
  or manage the tool loop manually through
206
206
  [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html).
207
207
  On interrupt, the running child process is killed. The `ruby` tool
@@ -8,23 +8,23 @@
8
8
  llm.rb supports six concurrency strategies for tool execution:
9
9
  `:sequential`, `:thread`, `:fiber`, `:async`, `:fork`, and
10
10
  `:ractor`. Each implements the same interface
11
- ([`LLM::Function::Task#spawn`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#spawn),
12
- [`LLM::Function::Task#wait`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#wait),
13
- [`LLM::Function::Task#alive?`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#alive?),
14
- [`LLM::Function::Task#interrupt!`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#interrupt!)),
11
+ ([`LLM::Function::Task#spawn`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#spawn-instance_method),
12
+ [`LLM::Function::Task#wait`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#wait-instance_method),
13
+ [`LLM::Function::Task#alive?`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#alive?-instance_method),
14
+ [`LLM::Function::Task#interrupt!`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#interrupt!-instance_method)),
15
15
  so the caller never has to care which
16
16
  strategy is behind a given task.
17
17
 
18
18
  #### How it works
19
19
 
20
20
  Each strategy creates a task object that wraps a function call.
21
- [`LLM::Function::Task#spawn`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#spawn)
21
+ [`LLM::Function::Task#spawn`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#spawn-instance_method)
22
22
  starts execution,
23
- [`LLM::Function::Task#wait`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#wait)
23
+ [`LLM::Function::Task#wait`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#wait-instance_method)
24
24
  collects the result,
25
- [`LLM::Function::Task#alive?`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#alive?)
25
+ [`LLM::Function::Task#alive?`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#alive?-instance_method)
26
26
  checks whether the task is still running, and
27
- [`LLM::Function::Task#interrupt!`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#interrupt!)
27
+ [`LLM::Function::Task#interrupt!`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html#interrupt!-instance_method)
28
28
  signals the task to stop.
29
29
 
30
30
  Interruption is reliable across all six.
@@ -83,8 +83,13 @@ during blocking IO. Best for IO-bound tools like HTTP calls and
83
83
  database queries.
84
84
 
85
85
  **fiber**: Each tool runs in a scheduler-backed Fiber. Requires
86
- `Fiber.scheduler`. Best for IO-bound tools inside an async
87
- framework. Much lighter than threads.
86
+ `Fiber.scheduler`, and that scheduler has to implement
87
+ `fiber_interrupt`, which is how a cancel is asked of a fiber rather than
88
+ raised into it. A scheduler without it is refused with
89
+ [`LLM::FiberError`](https://r.uby.dev/api-docs/llm.rb/LLM/FiberError.html)
90
+ before anything is scheduled, since a cancel that cannot be asked for is
91
+ a cancel that cannot be delivered. Best for IO-bound tools inside an
92
+ async framework. Much lighter than threads.
88
93
 
89
94
  **async**: Each tool runs as an `Async::Task` inside a managed
90
95
  background reactor. Best for IO-bound tools when you want Async's
@@ -28,11 +28,11 @@ All four use the same serialization mechanism under the hood.
28
28
 
29
29
  [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
30
30
  implements
31
- [`LLM::Context#to_h`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#to_h)
31
+ [`LLM::Context#to_h`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#to_h-instance_method)
32
32
  and
33
- [`LLM::Context#to_json`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#to_json)
33
+ [`LLM::Context#to_json`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#to_json-instance_method)
34
34
  for serialization and
35
- [`LLM::Context#restore`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#restore)
35
+ [`LLM::Context#restore`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#restore-instance_method)
36
36
  for deserialization. Save writes the current state, restore loads
37
37
  it back and picks up where the conversation left off.
38
38
 
@@ -70,16 +70,16 @@ underlying serialization as filesystem persistence.
70
70
 
71
71
  #### Overview
72
72
 
73
- The [`LLM::Agent#path`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#path)
73
+ The [`LLM::Agent#path`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#path-instance_method)
74
74
  attribute provides transparent auto-persistence. Set a file path once
75
75
  and the agent restores conversation history from that file on startup
76
76
  and saves it back after every
77
- [`LLM::Agent#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#talk)
77
+ [`LLM::Agent#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#talk-instance_method)
78
78
  or
79
- [`LLM::Agent#ask`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#ask)
79
+ [`LLM::Agent#ask`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#ask-instance_method)
80
80
  turn. No manual
81
- [`LLM::Agent#save`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#save)/
82
- [`LLM::Agent#restore`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#restore)
81
+ [`LLM::Agent#save`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#save-instance_method)/
82
+ [`LLM::Agent#restore`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#restore-instance_method)
83
83
  calls needed.
84
84
 
85
85
  The feature is available on subclasses via the class DSL and on
@@ -89,9 +89,9 @@ direct instances via the `path:` keyword argument.
89
89
 
90
90
  When a `path` is set, the agent loads existing state from the file
91
91
  during initialization. After each turn
92
- ([`LLM::Agent#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#talk)
92
+ ([`LLM::Agent#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#talk-instance_method)
93
93
  or
94
- [`LLM::Agent#ask`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#ask)),
94
+ [`LLM::Agent#ask`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#ask-instance_method)),
95
95
  the updated
96
96
  state is written back automatically. If the file does not exist yet,
97
97
  the agent starts with a blank conversation and creates the file on
@@ -140,9 +140,9 @@ Auto-persistence eliminates boilerplate. You set the path once and
140
140
  forget about serialization entirely. The agent picks up where it
141
141
  left off across process restarts, console sessions, or debugging runs
142
142
  without a single
143
- [`LLM::Agent#save`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#save)
143
+ [`LLM::Agent#save`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#save-instance_method)
144
144
  or
145
- [`LLM::Agent#restore`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#restore)
145
+ [`LLM::Agent#restore`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#restore-instance_method)
146
146
  call in your code.
147
147
 
148
148
  #### Notes
@@ -150,9 +150,9 @@ call in your code.
150
150
  The auto-path feature is a strict superset of manual filesystem
151
151
  persistence. If you need fine-grained control over when state is
152
152
  saved (e.g. batch several turns before persisting), use the manual
153
- [`LLM::Agent#save`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#save)
153
+ [`LLM::Agent#save`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#save-instance_method)
154
154
  and
155
- [`LLM::Agent#restore`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#restore)
155
+ [`LLM::Agent#restore`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#restore-instance_method)
156
156
  methods described in the Filesystem section below. The `path`
157
157
  attribute delegates to the same underlying serialization.
158
158
 
@@ -170,12 +170,12 @@ process or on a different machine, and pick up where you left off.
170
170
 
171
171
  [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
172
172
  implements
173
- [`LLM::Context#to_h`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#to_h)
173
+ [`LLM::Context#to_h`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#to_h-instance_method)
174
174
  and
175
- [`LLM::Context#to_json`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#to_json)
175
+ [`LLM::Context#to_json`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#to_json-instance_method)
176
176
  for serialization
177
177
  and
178
- [`LLM::Context#restore`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#restore)
178
+ [`LLM::Context#restore`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#restore-instance_method)
179
179
  for deserialization. The serialized state includes the context id,
180
180
  the message history, the model name, the compaction status, and a
181
181
  snapshot of token usage. Save and
@@ -358,7 +358,7 @@ parameters. Those two are the only callbacks `acts_as_llm` and
358
358
 
359
359
  An agent built by `acts_as_agent` is bound to the record it was
360
360
  loaded from, and
361
- [`LLM::Agent#record`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#record)
361
+ [`LLM::Agent#record`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#record-instance_method)
362
362
  returns it. To persist a context instead of an agent, use
363
363
  `acts_as_llm`: the model gains `#llm` (the provider) and `#ctx` (the
364
364
  context), and persists the same state without the automatic tool
@@ -547,6 +547,6 @@ To persist a context instead of an agent, use `plugin :llm`. The
547
547
  model gains `#llm` (the provider) and `#ctx` (the context), and
548
548
  persists the same state without the automatic tool loop. An agent
549
549
  built by `plugin :agent` is bound to its record, which
550
- [`LLM::Agent#record`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#record)
550
+ [`LLM::Agent#record`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#record-instance_method)
551
551
  returns.
552
552
 
@@ -9,7 +9,7 @@ Embeddings convert text into numerical vectors that capture semantic
9
9
  meaning. Similar texts produce vectors that are close together in the
10
10
  vector space, enabling semantic search, clustering, and classification.
11
11
  Most providers offer embedding models through an
12
- [`LLM::Provider#embed`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#embed)
12
+ [`LLM::Provider#embed`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#embed-instance_method)
13
13
  method.
14
14
 
15
15
  llm.rb also includes support for OpenAI's vector store API, which
@@ -19,7 +19,7 @@ provides a managed vector database as an HTTP service.
19
19
 
20
20
  When you want to generate an embedding vector for a piece of text,
21
21
  call
22
- [`LLM::Provider#embed`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#embed)
22
+ [`LLM::Provider#embed`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#embed-instance_method)
23
23
  on any provider that supports it. The method accepts an input string
24
24
  or array of strings and returns a response with the resulting
25
25
  embeddings, which can be stored in a vector database and used for
@@ -113,7 +113,7 @@ The trade-off is vendor lock-in and per-request pricing.
113
113
  Vector stores require the `openai` provider. Other providers such as
114
114
  DeepInfra, Google, and Mistral expose an embedding method but no
115
115
  vector store API. For a self-hosted vector database, pair
116
- [`LLM::Provider#embed`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#embed)
116
+ [`LLM::Provider#embed`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#embed-instance_method)
117
117
  with sqlite-vec or pgvector.
118
118
 
119
119
  ### Local vector search
@@ -121,7 +121,7 @@ with sqlite-vec or pgvector.
121
121
  #### Overview
122
122
 
123
123
  Local vector search combines
124
- [`LLM::Provider#embed`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#embed)
124
+ [`LLM::Provider#embed`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#embed-instance_method)
125
125
  with a local vector database such as
126
126
  [sqlite-vec](https://github.com/asg017/sqlite-vec) or
127
127
  [pgvector](https://github.com/pgvector/pgvector). This approach keeps
@@ -47,8 +47,10 @@ assistants that read them, get their content.
47
47
 
48
48
  Anthropic, Google, and OpenAI implement the Files API, and so do the
49
49
  providers built on OpenAI's - Moonshot and Alibaba among them, which
50
- inherit it. A provider that has no Files API of its own, such as
51
- DeepSeek, Mistral, or xAI, raises `NotImplementedError`. `download` is
52
- OpenAI-only, so the OpenAI-compatible providers inherit that too. A file
53
- id belongs to the provider that minted it: DeepSeek, for example, raises
54
- `LLM::PromptError` when it is handed a file id from another provider.
50
+ inherit it. DeepSeek implements the Files API too, scoped to images that
51
+ a chat request references by `file_id`. A provider that has no Files API
52
+ of its own, such as Mistral or xAI, raises `NotImplementedError`.
53
+ `download` is OpenAI-only, so the OpenAI-compatible providers inherit
54
+ that too. A file id belongs to the provider that minted it: a file id minted by
55
+ another provider is passed through as-is, and fails when DeepSeek's API
56
+ tries to resolve it.
@@ -16,7 +16,7 @@ serialization, compaction, and concurrency.
16
16
  #### How it works
17
17
 
18
18
  An agent holds a conversation with a model. You send input with
19
- [`LLM::Agent#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#talk),
19
+ [`LLM::Agent#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#talk-instance_method),
20
20
  the model responds, and if it requests tools the agent
21
21
  executes them automatically and feeds the results back. The tool
22
22
  loop can be bounded with
@@ -30,7 +30,7 @@ prompt already carries a system message of the caller's own.
30
30
  Agents manage the tool loop for you. They keep conversation state
31
31
  across turns and let you define reusable configurations at the
32
32
  class level. The loop runs until the model stops requesting tools
33
- unless you bound it with `tool_budget` or a guard. If you need
33
+ unless you bound it with `tool_budget` or an intercept. If you need
34
34
  manual control over the tool loop, use
35
35
  [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
36
36
  directly instead.
@@ -100,8 +100,8 @@ Set `path:` on a subclass or instance for automatic filesystem
100
100
  persistence; the agent restores conversation history from the
101
101
  file on startup and saves it back after every turn with no
102
102
  manual
103
- [`LLM::Agent#save`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#save)/
104
- [`LLM::Agent#restore`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#restore)
103
+ [`LLM::Agent#save`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#save-instance_method)/
104
+ [`LLM::Agent#restore`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#restore-instance_method)
105
105
  calls. See the
106
106
  [database deepdive](../features/database.md) for details.
107
107
 
@@ -244,10 +244,15 @@ of hanging.
244
244
 
245
245
  #### Notes
246
246
 
247
- The retry budget applies to rate-limited requests
248
- (`LLM::RateLimitError`) and timeouts (`Timeout::Error`, covering
249
- `Net::OpenTimeout` and `Net::ReadTimeout`); other errors are never
250
- retried. The budget lives on the provider, where it defaults to five
247
+ The retry budget applies to the errors the provider lists in
248
+ [`LLM::Provider#retry_errors`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#retry_errors-instance_method):
249
+ rate limits (`LLM::RateLimitError`), an exhausted quota, and timeouts
250
+ (`Net::OpenTimeout`, `Net::ReadTimeout`, `Net::WriteTimeout`). A
251
+ provider extends that list rather than replacing it, so
252
+ [`LLM::DeepSeek`](https://r.uby.dev/api-docs/llm.rb/LLM/DeepSeek.html)
253
+ also retries `LLM::ServerError`, the error it returns when its servers
254
+ are overloaded; anything not on the list is never retried. The budget
255
+ lives on the provider, where it defaults to five
251
256
  (eight on Alibaba, which rate limits more often), and an agent that
252
257
  sets none of its own takes the provider's. A raw
253
258
  [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
@@ -37,7 +37,7 @@ as `LLM.aliyun`.
37
37
  Providers are the first building block of every llm.rb program.
38
38
  Because the whole runtime shares one provider interface, you can
39
39
  swap one provider for another with a single line change, and the
40
- same tools, schemas, streaming, and guards work unchanged.
40
+ same tools, schemas, streaming, and intercepts work unchanged.
41
41
 
42
42
  #### Notes
43
43
 
@@ -16,9 +16,9 @@ When you want a structured response from the model, subclass
16
16
  [`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html)
17
17
  and declare properties with types and constraints. When you pass
18
18
  the schema to
19
- [`LLM::Agent#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#talk)
19
+ [`LLM::Agent#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#talk-instance_method)
20
20
  or
21
- [`LLM::Context#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#talk),
21
+ [`LLM::Context#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#talk-instance_method),
22
22
  the runtime includes it in the request parameters. The provider returns a
23
23
  JSON object matching the schema, which is coerced into an
24
24
  [`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html) with typed accessors.
@@ -37,7 +37,7 @@ single blocking response.
37
37
  #### Notes
38
38
 
39
39
  The IO-like form is equivalent to
40
- [`LLM::Stream#on_content`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_content)
40
+ [`LLM::Stream#on_content`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_content-instance_method)
41
41
  and does not include the other hooks. It covers content output
42
42
  (piping to stdout or a log file) without the overhead of a full
43
43
  subclass. A
@@ -61,11 +61,11 @@ When you want to react to specific runtime events, override the
61
61
  hooks on a
62
62
  [`LLM::Stream`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html)
63
63
  subclass.
64
- [`LLM::Stream#on_content`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_content)
64
+ [`LLM::Stream#on_content`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_content-instance_method)
65
65
  receives tokens as they arrive.
66
- [`LLM::Stream#on_tool_call`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_tool_call)
66
+ [`LLM::Stream#on_tool_call`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_tool_call-instance_method)
67
67
  fires when the model requests a tool.
68
- [`LLM::Stream#on_tool_return`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_tool_return)
68
+ [`LLM::Stream#on_tool_return`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_tool_return-instance_method)
69
69
  fires when the tool completes.
70
70
  [`LLM::Stream#on_step`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_step-instance_method)
71
71
  fires when a request completes, which is the boundary between one
@@ -78,13 +78,13 @@ written against it - it is prepended onto a stream, and saves the
78
78
  conversation through the record the context is bound to, so a turn
79
79
  that is interrupted can be continued from its last completed request
80
80
  rather than started over.
81
- [`LLM::Stream#on_retry`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_retry)
81
+ [`LLM::Stream#on_retry`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_retry-instance_method)
82
82
  fires each time a failed request is retried. Compaction hooks
83
83
  let you show progress or log what was trimmed. Skill hooks bracket a
84
84
  skill's subagent execution:
85
- [`LLM::Stream#on_skill_call`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_skill_call)
85
+ [`LLM::Stream#on_skill_call`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_skill_call-instance_method)
86
86
  fires before a skill's subagent runs, and
87
- [`LLM::Stream#on_skill_return`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_skill_return)
87
+ [`LLM::Stream#on_skill_return`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_skill_return-instance_method)
88
88
  fires after it finishes.
89
89
 
90
90
  ```ruby
@@ -164,7 +164,7 @@ instead, and hand the result to the turn when it is asked for.
164
164
  #### How it works
165
165
 
166
166
  Start the tool from
167
- [`LLM::Stream#on_tool_call`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_tool_call)
167
+ [`LLM::Stream#on_tool_call`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_tool_call-instance_method)
168
168
  and enqueue its task on
169
169
  [`LLM::Stream#queue`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#queue-instance_method).
170
170
  [`LLM::Context#wait`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#wait-instance_method)
@@ -12,10 +12,10 @@ when a tool fits the request; the runtime calls it and sends the
12
12
  result back.
13
13
 
14
14
  A tool is a Ruby class with a name, a description, and a
15
- [`LLM::Tool#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#call)
15
+ [`LLM::Tool#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#call-instance_method)
16
16
  method. The name tells the model what the tool is called. The
17
17
  description tells the model when to use it. The
18
- [`LLM::Tool#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#call)
18
+ [`LLM::Tool#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#call-instance_method)
19
19
  method receives the arguments and returns the result. That is the entire
20
20
  contract: name, description, parameters, and a method that runs.
21
21
 
@@ -26,7 +26,7 @@ A tool is a subclass of
26
26
  with a name, description, and optional typed parameters. The model sees the name and
27
27
  description and decides whether to call it. When it does, the
28
28
  runtime serializes the arguments and passes them to
29
- [`LLM::Tool#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#call).
29
+ [`LLM::Tool#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#call-instance_method).
30
30
 
31
31
  Tools that spawn subprocesses can include
32
32
  [`LLM::Tool::Utils`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Utils.html)
@@ -36,11 +36,11 @@ and `now` help. The built-in `Exec`, `Git`, `Rg`, `Mkdir`, and `Ruby`
36
36
  tools use it to kill a command that exceeds its `timeout`.
37
37
 
38
38
  If
39
- [`LLM::Tool#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#call)
39
+ [`LLM::Tool#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#call-instance_method)
40
40
  raises, the runtime rescues it and returns a structured
41
41
  error to the model instead. The conversation stays valid. You can
42
42
  also handle errors yourself inside
43
- [`LLM::Tool#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#call)
43
+ [`LLM::Tool#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#call-instance_method)
44
44
  by rescuing and returning a domain-specific error hash.
45
45
 
46
46
  ```ruby
@@ -96,9 +96,9 @@ API call at once, then return a synthesized answer. See
96
96
 
97
97
  Confirmation gates tools behind explicit approval. List tool names
98
98
  in
99
- [`LLM::Agent#confirm`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#confirm)
99
+ [`LLM::Agent#confirm`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#confirm-instance_method)
100
100
  to block execution until you override
101
- [`LLM::Agent#on_tool_confirmation`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#on_tool_confirmation).
101
+ [`LLM::Agent#on_tool_confirmation`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#on_tool_confirmation-instance_method).
102
102
  Confirmation also accepts a Symbol that resolves to an instance
103
103
  method, letting the confirmed set change per-instance based on
104
104
  runtime conditions.
@@ -122,10 +122,10 @@ keywords in full.
122
122
 
123
123
  Tools that perform destructive actions can be gated behind explicit
124
124
  approval. List their names in
125
- [`LLM::Agent#confirm`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#confirm)
125
+ [`LLM::Agent#confirm`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#confirm-instance_method)
126
126
  to block execution
127
127
  until you override
128
- [`LLM::Agent#on_tool_confirmation`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#on_tool_confirmation).
128
+ [`LLM::Agent#on_tool_confirmation`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#on_tool_confirmation-instance_method).
129
129
  The default handler cancels
130
130
  the tool. Override it per-agent to prompt the user, log the decision,
131
131
  or auto-approve certain tools.
@@ -134,12 +134,12 @@ or auto-approve certain tools.
134
134
 
135
135
  When you want to override the default approval flow for a gated
136
136
  tool, override
137
- [`LLM::Agent#on_tool_confirmation`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#on_tool_confirmation)
137
+ [`LLM::Agent#on_tool_confirmation`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#on_tool_confirmation-instance_method)
138
138
  on the subclass. The method receives the pending function and the
139
139
  execution strategy. Call
140
- [`LLM::Function#task`](https://r.uby.dev/api-docs/llm.rb/LLM/Function.html#task)
140
+ [`LLM::Function#task`](https://r.uby.dev/api-docs/llm.rb/LLM/Function.html#task-instance_method)
141
141
  to execute the tool or
142
- [`LLM::Function#cancel`](https://r.uby.dev/api-docs/llm.rb/LLM/Function.html#cancel)
142
+ [`LLM::Function#cancel`](https://r.uby.dev/api-docs/llm.rb/LLM/Function.html#cancel-instance_method)
143
143
  to block it. The default handler cancels the call.
144
144
 
145
145
  ```ruby
@@ -178,7 +178,7 @@ matter what.
178
178
  #### How it works
179
179
 
180
180
  If
181
- [`LLM::Tool#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#call)
181
+ [`LLM::Tool#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#call-instance_method)
182
182
  raises, the runtime returns `{error: true, type: "RuntimeError",
183
183
  message: "boom"}` to the model. A tool can also check for a known
184
184
  failure and return its own error shape, which gives the model
@@ -445,7 +445,7 @@ gem for process management and interrupt handling.
445
445
 
446
446
  `LLM::Tool.subclasses` lists the direct subclasses of
447
447
  [`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html), so a
448
- tool defined under an intermediate base class is not in the list.
448
+ tool defined under an intermediate superclass is not in the list.
449
449
  [`LLM::Tool.registry`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#registry-class_method)
450
450
  lists every tool that has a name, wherever it sits in the hierarchy.
451
451
 
@@ -13,7 +13,7 @@ document URLs.
13
13
  #### How it works
14
14
 
15
15
  Mistral is the only provider with a dedicated OCR endpoint. The
16
- [`LLM::Provider#ocr`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#ocr)
16
+ [`LLM::Provider#ocr`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#ocr-instance_method)
17
17
  method accepts an `image_url:` or `document_url:` parameter.
18
18
  Document URLs can point to PDFs.
19
19
 
@@ -38,7 +38,7 @@ the network.
38
38
  #### Notes
39
39
 
40
40
  An agent's capabilities are advertised through a card. The
41
- [`LLM::A2A::Card#interfaces`](https://r.uby.dev/api-docs/llm.rb/LLM/A2A/Card.html#interfaces)
41
+ [`LLM::A2A::Card#interfaces`](https://r.uby.dev/api-docs/llm.rb/LLM/A2A/Card.html#interfaces-instance_method)
42
42
  method lists which transports a remote agent supports.
43
43
 
44
44
  ##### Persistent connections
@@ -69,7 +69,7 @@ JSON-RPC is an alternative transport for A2A agents. It uses a
69
69
  more structured protocol than REST, with request and response
70
70
  objects that follow the JSON-RPC 2.0 spec. Some agents advertise
71
71
  only JSON-RPC, others advertise both. The
72
- [`LLM::A2A::Card#interfaces`](https://r.uby.dev/api-docs/llm.rb/LLM/A2A/Card.html#interfaces)
72
+ [`LLM::A2A::Card#interfaces`](https://r.uby.dev/api-docs/llm.rb/LLM/A2A/Card.html#interfaces-instance_method)
73
73
  method lists which transports a remote agent supports.
74
74
 
75
75
  #### How it works
@@ -93,7 +93,7 @@ agent.talk "What's happening, fellow agent?"
93
93
  JSON-RPC provides typed request and response objects that follow
94
94
  the 2.0 spec, offering a more structured protocol than REST. Some
95
95
  agents advertise only JSON-RPC, while others advertise both. The
96
- [`LLM::A2A::Card#interfaces`](https://r.uby.dev/api-docs/llm.rb/LLM/A2A/Card.html#interfaces)
96
+ [`LLM::A2A::Card#interfaces`](https://r.uby.dev/api-docs/llm.rb/LLM/A2A/Card.html#interfaces-instance_method)
97
97
  method lists which transports a remote agent supports.
98
98
 
99
99
  #### Notes
@@ -17,7 +17,7 @@ subclasses the model can call.
17
17
 
18
18
  The stdio transport runs the server as a child process and
19
19
  communicates over stdin/stdout. Use
20
- [`LLM::MCP#session`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html#session)
20
+ [`LLM::MCP#session`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html#session-instance_method)
21
21
  to avoid launching the same process multiple times.
22
22
 
23
23
  ```ruby
@@ -41,11 +41,62 @@ one. Remove them without touching the agent's code.
41
41
  #### Notes
42
42
 
43
43
  For stdio, use
44
- [`LLM::MCP#session`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html#session)
44
+ [`LLM::MCP#session`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html#session-instance_method)
45
45
  to avoid launching the same process multiple times. For HTTP,
46
- [`LLM::MCP#session`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html#session)
46
+ [`LLM::MCP#session`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html#session-instance_method)
47
47
  carries little benefit.
48
48
 
49
+ ### Local repository lookup with Sem
50
+
51
+ [Sem](https://github.com/Ataraxy-Labs/sem) provides symbol lookup and
52
+ source search through a local stdio MCP server. Install Sem following
53
+ its upstream instructions and verify `sem --version`. This example
54
+ uses the existing stdio client; no model credentials are needed.
55
+
56
+ Save the following as `sem_code.rb`, then run it with an existing
57
+ Git checkout and a symbol in that checkout:
58
+
59
+ ```bash
60
+ SEM_REPO=/absolute/path/to/repo ruby sem_code.rb parse_config
61
+ ```
62
+
63
+ ```ruby
64
+ require "llm"
65
+
66
+ repo = File.realpath(ENV.fetch("SEM_REPO"))
67
+ query = ARGV.fetch(0)
68
+ mcp = LLM::MCP.stdio(
69
+ argv: [ENV.fetch("SEM_BIN", "sem"), "mcp"],
70
+ env: {"SEM_REPO" => repo},
71
+ cwd: repo
72
+ )
73
+
74
+ mcp.session do
75
+ lookup = mcp.tools.find { |tool| tool.name == "sem_find" }
76
+ raise "Server did not expose sem_find" unless lookup
77
+
78
+ puts LLM.json.dump(lookup.new.call(
79
+ query: query,
80
+ mode: "context",
81
+ token_budget: 1200
82
+ ))
83
+ end
84
+ ```
85
+
86
+ Use `SEM_BIN=/absolute/path/to/sem` if Sem is not on PATH. Keeping
87
+ discovery and lookup inside `session` reuses the child process. The
88
+ first query may build a repository index; an empty result can mean
89
+ the symbol was not found. Consult the discovered tool schema for
90
+ the installed Sem version.
91
+
92
+ For an agent conversation, pass only the tool classes you intend
93
+ to expose, for example `mcp.tools.select { |t| t.name == "sem_find" }`,
94
+ inside the same session. This filters the model's available tools,
95
+ not the server's permissions. The full Sem server also exposes
96
+ validation tools that can execute commands: use trusted checkouts
97
+ and host permissions. `SEM_REPO` is not a sandbox. The lookup does
98
+ not edit source files, but Sem may write its local index.
99
+
49
100
  ### HTTP
50
101
 
51
102
  #### Overview
@@ -90,7 +141,7 @@ by a different team and exposed as a service.
90
141
  #### Notes
91
142
 
92
143
  For HTTP,
93
- [`LLM::MCP#session`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html#session)
144
+ [`LLM::MCP#session`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html#session-instance_method)
94
145
  carries little benefit.
95
146
 
96
147
  ##### Persistent connections
@@ -109,15 +160,73 @@ mcp = LLM::MCP.http(
109
160
  )
110
161
  ```
111
162
 
163
+ ### Parallel web search and fetch
164
+
165
+ [Parallel Search MCP](https://docs.parallel.ai/integrations/mcp/search-mcp)
166
+ provides `web_search` and `web_fetch` over HTTP without a Parallel
167
+ API key. The anonymous tier is free for exploration and light use,
168
+ with rate limits and server-managed search settings.
169
+
170
+ This example discovers the tools through the HTTP client, searches
171
+ for Ruby documentation, then extracts excerpts from a specific page.
172
+ It uses the default `net/http` transport and needs no model or provider
173
+ credentials. Save it as `parallel_search.rb` and run
174
+ `ruby parallel_search.rb` after installing `llm.rb`.
175
+
176
+ ```ruby
177
+ require "llm"
178
+
179
+ mcp = LLM::MCP.http(
180
+ url: "https://search.parallel.ai/mcp",
181
+ headers: {
182
+ "User-Agent" => "llm.rb/#{LLM::VERSION}",
183
+ "Accept" => "application/json, text/event-stream"
184
+ }
185
+ )
186
+
187
+ mcp.session do
188
+ tools = mcp.tools
189
+ search = tools.find { _1.name == "web_search" }.new
190
+ fetch = tools.find { _1.name == "web_fetch" }.new
191
+ session_id = SecureRandom.uuid
192
+ queries = ["Ruby Fiber scheduler documentation"]
193
+
194
+ puts LLM.json.dump(search.call(
195
+ objective: "Find the official Ruby Fiber scheduler documentation.",
196
+ search_queries: queries,
197
+ session_id:
198
+ ))
199
+
200
+ puts LLM.json.dump(fetch.call(
201
+ urls: ["https://docs.ruby-lang.org/en/master/Fiber.html"],
202
+ objective: "Explain how Ruby uses a Fiber scheduler.",
203
+ search_queries: queries,
204
+ session_id:
205
+ ))
206
+ end
207
+ ```
208
+
209
+ To let an agent choose when to search or fetch, pass the discovered
210
+ tools to `talk` instead. Model inference is separate from the free
211
+ MCP service and uses your chosen provider's credentials and pricing.
212
+ Using the same `mcp` configuration above:
213
+
214
+ ```ruby
215
+ llm = LLM.deepseek(key: ENV.fetch("DEEPSEEK_API_KEY"))
216
+ agent = LLM::Agent.new(llm)
217
+ agent.talk "Find the official Ruby Fiber scheduler docs and explain them.",
218
+ tools: mcp.tools
219
+ ```
220
+
112
221
  ### Prompts
113
222
 
114
223
  #### Overview
115
224
 
116
225
  An MCP server can offer prompts as well as tools: reusable,
117
226
  parameterized message templates a client fetches and sends to a model.
118
- [`LLM::MCP#prompts`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html#prompts)
227
+ [`LLM::MCP#prompts`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html#prompts-instance_method)
119
228
  lists them, and
120
- [`LLM::MCP#find_prompt`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html#find_prompt)
229
+ [`LLM::MCP#find_prompt`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html#find_prompt-instance_method)
121
230
  fetches one and returns its messages.
122
231
 
123
232
  #### How it works
@@ -155,7 +264,7 @@ it, so it can change without a client release.
155
264
  `find_prompt` adapts each message's content to the runtime's shape: a
156
265
  text item becomes a string, and anything else stays an `LLM::Object`
157
266
  under `original_content`.
158
- [`LLM::MCP#get_prompt`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html#get_prompt)
267
+ [`LLM::MCP#get_prompt`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html#get_prompt-instance_method)
159
268
  is an alias. Like `tools`, both methods borrow a session when one is
160
269
  not already running.
161
270