ruby-pi 0.1.9 → 0.1.10
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +21 -0
- data/README.md +19 -2
- data/lib/ruby_pi/agent/core.rb +6 -0
- data/lib/ruby_pi/agent/result.rb +31 -10
- data/lib/ruby_pi/agent/state.rb +4 -0
- data/lib/ruby_pi/context/compaction.rb +14 -1
- data/lib/ruby_pi/errors.rb +16 -0
- data/lib/ruby_pi/extensions/base.rb +0 -8
- data/lib/ruby_pi/llm/anthropic.rb +33 -70
- data/lib/ruby_pi/llm/base_provider.rb +37 -8
- data/lib/ruby_pi/llm/fallback.rb +13 -4
- data/lib/ruby_pi/llm/gemini.rb +62 -80
- data/lib/ruby_pi/llm/openai.rb +69 -90
- data/lib/ruby_pi/llm/sse_parser.rb +125 -0
- data/lib/ruby_pi/tools/executor.rb +162 -76
- data/lib/ruby_pi/tools/result.rb +17 -2
- data/lib/ruby_pi/version.rb +1 -1
- data/lib/ruby_pi.rb +1 -0
- metadata +3 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: ae334e07e1a5788bfe62b1a1d721d565885ba0142dca6d9a238656b3835454a0
|
|
4
|
+
data.tar.gz: 3f68b1f787a4fa17fd6ea68ef1b4974c5bb625b76c3898786a9b23c0999c490b
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: cc0289504d07435aa8e681a102bcdc4e27d8f4f8033ab73ea73c227e345ab64be93cc0fd8d3dd27d50c640d4758508e14a91b290fed02fd089477f73e5ea3d35
|
|
7
|
+
data.tar.gz: 031ad8a0ca080a0024f596ec05e57857bd7cab68dc9236ae5ada606f09bbeedea5e59142e01e6bdea2de032008a1624bd9ad0db78fd4ed6fc1ddcddaab53331b
|
data/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,27 @@ All notable changes to this project will be documented in this file.
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
6
6
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
7
|
|
|
8
|
+
## [0.1.10] - 2026-08-28
|
|
9
|
+
|
|
10
|
+
### Security
|
|
11
|
+
|
|
12
|
+
- **Malformed or truncated HTTP-200 streams were accepted (High)**: Added a shared bounded SSE decoder with UTF-8 validation, standards-compliant event framing, final-buffer processing, and typed retryable protocol errors. OpenAI now requires `data: [DONE]`, Anthropic requires `message_stop`, and Gemini requires a finish reason or explicit prompt block before emitting `:done`
|
|
13
|
+
- **Model-controlled tool fan-out was unbounded (High)**: Parallel execution now uses a bounded pool with a default concurrency of four, rejects more than 32 calls per execution, and enforces a process-wide cap of 16 active or timed-out-but-still-running tools. Streaming output, tool arguments, tool-call count, SSE events, and retained error bodies also have hard memory limits
|
|
14
|
+
- **Malformed model arguments could enter logs (Medium)**: Provider errors no longer embed raw tool arguments, and fallback diagnostics remove control characters and cap message length
|
|
15
|
+
|
|
16
|
+
### Fixed (adversarial review round 8)
|
|
17
|
+
|
|
18
|
+
- **Tool timeout results falsely implied failure was final (High)**: Timed-out work is now reported as `status: :timeout_unknown` with `completion_unknown?`. Sequential execution does not start later calls after an uncertain timeout; those calls return `status: :skipped`. Arbitrary in-process Ruby blocks are still cooperative and cannot be force-terminated safely
|
|
19
|
+
- **Immutable agent results could crash on valid tool values (High)**: Result snapshots are cycle-aware and never call `dup` on arbitrary application objects. Non-JSON objects become bounded frozen diagnostic strings instead of raising from `initialize_copy` or recursing indefinitely
|
|
20
|
+
- **Retry/fallback reset metadata ignored tool fragments (Medium)**: Reset events now report `partial_tool_calls` and set `partial_output` whenever either text or structured tool-call output was emitted
|
|
21
|
+
- **Compaction ignored structured tool metadata (Medium)**: Token estimates and summary prompts now account for assistant tool calls, IDs, names, and arguments
|
|
22
|
+
- **Invalid safety limits failed deep in execution (Medium)**: Agent iteration limits, tool timeouts, call limits, and concurrency limits now reject non-finite, zero, negative, or wrong-type values at construction
|
|
23
|
+
- **Release gates covered tests only (Low)**: CI now runs dependency auditing, Lint/Security checks, gem build/install, and an installed-artifact load/version smoke test alongside Ruby 3.2-3.4 tests
|
|
24
|
+
|
|
25
|
+
### Known limitation
|
|
26
|
+
|
|
27
|
+
- Ruby cannot safely force-terminate an arbitrary in-process block. A timed-out tool may still finish and commit side effects, so callers must treat `completion_unknown?` as an indeterminate outcome and use idempotency keys. Enforceable cancellation requires isolated worker processes or an external job backend in a future execution API
|
|
28
|
+
|
|
8
29
|
## [0.1.9] - 2026-08-28
|
|
9
30
|
|
|
10
31
|
### Security
|
data/README.md
CHANGED
|
@@ -133,7 +133,8 @@ model.complete(messages: messages, stream: true) do |event|
|
|
|
133
133
|
when :retry_start
|
|
134
134
|
# A transient failure occurred after streaming may have begun. Discard
|
|
135
135
|
# the previous attempt before the provider retries from the beginning.
|
|
136
|
-
# Payload includes
|
|
136
|
+
# Payload includes partial_chars and partial_tool_calls. Always discard
|
|
137
|
+
# all text and structured tool state from the failed attempt.
|
|
137
138
|
clear_partial_output
|
|
138
139
|
when :fallback_start
|
|
139
140
|
# Only emitted by RubyPi::LLM::Fallback when the primary provider
|
|
@@ -240,7 +241,13 @@ registry.all # => [Definition, ...]
|
|
|
240
241
|
Run tool calls in parallel or sequentially with automatic error handling and timeouts:
|
|
241
242
|
|
|
242
243
|
```ruby
|
|
243
|
-
executor = RubyPi::Tools::Executor.new(
|
|
244
|
+
executor = RubyPi::Tools::Executor.new(
|
|
245
|
+
registry,
|
|
246
|
+
mode: :parallel,
|
|
247
|
+
timeout: 30,
|
|
248
|
+
max_calls: 32,
|
|
249
|
+
max_concurrency: 4
|
|
250
|
+
)
|
|
244
251
|
|
|
245
252
|
results = executor.execute([
|
|
246
253
|
{ name: "create_post", arguments: { content: "Hello" } },
|
|
@@ -252,6 +259,7 @@ results.each do |r|
|
|
|
252
259
|
puts "#{r.name}: #{r.value}"
|
|
253
260
|
else
|
|
254
261
|
puts "#{r.name} failed: #{r.error} (#{r.duration_ms}ms)"
|
|
262
|
+
reconcile_with_an_idempotency_key if r.completion_unknown?
|
|
255
263
|
end
|
|
256
264
|
end
|
|
257
265
|
```
|
|
@@ -263,6 +271,15 @@ end
|
|
|
263
271
|
| `value` | Return value (on success) |
|
|
264
272
|
| `error` | Error message (on failure) |
|
|
265
273
|
| `duration_ms` | Execution time in milliseconds |
|
|
274
|
+
| `status` | `:success`, `:error`, `:timeout_unknown`, or `:skipped` |
|
|
275
|
+
| `completion_unknown?` | The deadline elapsed, but the in-process tool may still finish or commit side effects |
|
|
276
|
+
| `skipped?` | Sequential execution did not start this call after an uncertain timeout |
|
|
277
|
+
|
|
278
|
+
Parallel execution is bounded. A model response containing more than `max_calls`
|
|
279
|
+
tool calls raises `RubyPi::ToolCallLimitError`, and the process refuses new work
|
|
280
|
+
after 16 tools remain active. Ruby cannot safely kill an arbitrary
|
|
281
|
+
in-process block, so use idempotency keys for side-effecting tools and treat
|
|
282
|
+
`completion_unknown?` as indeterminate rather than as a confirmed rollback.
|
|
266
283
|
|
|
267
284
|
---
|
|
268
285
|
|
data/lib/ruby_pi/agent/core.rb
CHANGED
|
@@ -101,6 +101,12 @@ module RubyPi
|
|
|
101
101
|
execution_mode: :parallel,
|
|
102
102
|
tool_timeout: 30
|
|
103
103
|
)
|
|
104
|
+
valid_timeout = tool_timeout.is_a?(Numeric) && !tool_timeout.is_a?(Complex) &&
|
|
105
|
+
tool_timeout.finite? && tool_timeout.positive?
|
|
106
|
+
unless valid_timeout
|
|
107
|
+
raise ArgumentError, "tool_timeout must be a finite positive number, got #{tool_timeout.inspect}"
|
|
108
|
+
end
|
|
109
|
+
|
|
104
110
|
@state = State.new(
|
|
105
111
|
system_prompt: system_prompt,
|
|
106
112
|
model: model,
|
data/lib/ruby_pi/agent/result.rb
CHANGED
|
@@ -143,25 +143,46 @@ module RubyPi
|
|
|
143
143
|
# arrays still allowed callers (or later State mutations) to rewrite
|
|
144
144
|
# nested messages, tool arguments, usage, and content in place.
|
|
145
145
|
def deep_frozen_copy(value)
|
|
146
|
+
snapshot(value, {}.compare_by_identity).freeze
|
|
147
|
+
end
|
|
148
|
+
|
|
149
|
+
# Copies only JSON-like containers and immutable scalars. Arbitrary tool
|
|
150
|
+
# return objects are converted to a bounded diagnostic string instead of
|
|
151
|
+
# calling #dup, which may allocate resources, raise from initialize_copy,
|
|
152
|
+
# or recurse forever through cyclic structures.
|
|
153
|
+
def snapshot(value, memo)
|
|
154
|
+
if value.is_a?(Hash) || value.is_a?(Array)
|
|
155
|
+
return "[Circular]".freeze if memo.key?(value)
|
|
156
|
+
|
|
157
|
+
memo[value] = true
|
|
158
|
+
end
|
|
159
|
+
|
|
146
160
|
copy = case value
|
|
147
161
|
when Hash
|
|
148
162
|
value.each_with_object({}) do |(key, item), result|
|
|
149
|
-
result[
|
|
163
|
+
result[snapshot(key, memo).freeze] = snapshot(item, memo).freeze
|
|
150
164
|
end
|
|
151
165
|
when Array
|
|
152
|
-
value.map { |item|
|
|
166
|
+
value.map { |item| snapshot(item, memo).freeze }
|
|
153
167
|
when String
|
|
154
168
|
value.dup
|
|
169
|
+
when NilClass, TrueClass, FalseClass, Symbol, Numeric
|
|
170
|
+
value
|
|
155
171
|
else
|
|
156
|
-
|
|
157
|
-
value.dup
|
|
158
|
-
rescue TypeError
|
|
159
|
-
# Immutable scalar values (nil, symbols, numerics, booleans)
|
|
160
|
-
# cannot be duplicated and are already safe to share.
|
|
161
|
-
value
|
|
162
|
-
end
|
|
172
|
+
safe_object_snapshot(value)
|
|
163
173
|
end
|
|
164
|
-
|
|
174
|
+
|
|
175
|
+
memo.delete(value) if value.is_a?(Hash) || value.is_a?(Array)
|
|
176
|
+
copy
|
|
177
|
+
end
|
|
178
|
+
|
|
179
|
+
def safe_object_snapshot(value)
|
|
180
|
+
rendered = value.to_s
|
|
181
|
+
rendered = rendered.encode(Encoding::UTF_8, invalid: :replace, undef: :replace, replace: "�")
|
|
182
|
+
rendered = rendered.byteslice(0, 4096).scrub if rendered.bytesize > 4096
|
|
183
|
+
"#<#{value.class}: #{rendered}>"
|
|
184
|
+
rescue StandardError
|
|
185
|
+
"#<#{value.class}: uninspectable>"
|
|
165
186
|
end
|
|
166
187
|
end
|
|
167
188
|
end
|
data/lib/ruby_pi/agent/state.rb
CHANGED
|
@@ -77,6 +77,10 @@ module RubyPi
|
|
|
77
77
|
after_tool_call: nil,
|
|
78
78
|
user_data: {}
|
|
79
79
|
)
|
|
80
|
+
unless max_iterations.is_a?(Integer) && max_iterations.positive?
|
|
81
|
+
raise ArgumentError, "max_iterations must be a positive integer, got #{max_iterations.inspect}"
|
|
82
|
+
end
|
|
83
|
+
|
|
80
84
|
@system_prompt = system_prompt
|
|
81
85
|
@model = model
|
|
82
86
|
@tools = tools
|
|
@@ -1,5 +1,7 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require "json"
|
|
4
|
+
|
|
3
5
|
# lib/ruby_pi/context/compaction.rb
|
|
4
6
|
#
|
|
5
7
|
# RubyPi::Context::Compaction — Token estimation and context window management.
|
|
@@ -28,6 +30,7 @@ module RubyPi
|
|
|
28
30
|
# Average characters per token — a rough heuristic that avoids the need
|
|
29
31
|
# for provider-specific tokenizers. Errs on the conservative side.
|
|
30
32
|
CHARS_PER_TOKEN = 4
|
|
33
|
+
SUMMARY_METADATA_LIMIT = 16_384
|
|
31
34
|
|
|
32
35
|
# @return [Integer] the token threshold above which compaction triggers
|
|
33
36
|
attr_reader :max_tokens
|
|
@@ -160,6 +163,8 @@ module RubyPi
|
|
|
160
163
|
|
|
161
164
|
messages.each do |msg|
|
|
162
165
|
total_chars += message_value(msg, :content).to_s.length
|
|
166
|
+
metadata = msg.reject { |key, _| key == :role || key == "role" || key == :content || key == "content" }
|
|
167
|
+
total_chars += serialized_message(metadata).length unless metadata.empty?
|
|
163
168
|
# Account for role and structural overhead (~10 tokens per message)
|
|
164
169
|
total_chars += 40
|
|
165
170
|
end
|
|
@@ -202,7 +207,9 @@ module RubyPi
|
|
|
202
207
|
transcript = messages.map do |msg|
|
|
203
208
|
role = message_value(msg, :role).to_s.capitalize
|
|
204
209
|
content = message_value(msg, :content).to_s
|
|
205
|
-
|
|
210
|
+
metadata = msg.reject { |key, _| key == :role || key == "role" || key == :content || key == "content" }
|
|
211
|
+
metadata_text = metadata.empty? ? nil : serialized_message(metadata).slice(0, SUMMARY_METADATA_LIMIT)
|
|
212
|
+
["#{role}: #{content}", ("Structured metadata: #{metadata_text}" if metadata_text)].compact.join("\n")
|
|
206
213
|
end.join("\n\n")
|
|
207
214
|
|
|
208
215
|
"Summarize the following conversation, preserving all key facts, " \
|
|
@@ -214,6 +221,12 @@ module RubyPi
|
|
|
214
221
|
def message_value(message, key)
|
|
215
222
|
message[key] || message[key.to_s]
|
|
216
223
|
end
|
|
224
|
+
|
|
225
|
+
def serialized_message(message)
|
|
226
|
+
JSON.generate(message)
|
|
227
|
+
rescue JSON::GeneratorError, TypeError
|
|
228
|
+
message.to_s
|
|
229
|
+
end
|
|
217
230
|
end
|
|
218
231
|
end
|
|
219
232
|
end
|
data/lib/ruby_pi/errors.rb
CHANGED
|
@@ -39,6 +39,14 @@ module RubyPi
|
|
|
39
39
|
end
|
|
40
40
|
end
|
|
41
41
|
|
|
42
|
+
# Raised when an HTTP streaming response violates the provider's protocol.
|
|
43
|
+
# This includes malformed SSE data, invalid UTF-8, oversized events, and a
|
|
44
|
+
# connection that closes before the provider's terminal event arrives.
|
|
45
|
+
#
|
|
46
|
+
# It inherits from ApiError with a nil status code so BaseProvider treats a
|
|
47
|
+
# broken HTTP-200 stream like a transient transport failure and can retry it.
|
|
48
|
+
class StreamingProtocolError < ApiError; end
|
|
49
|
+
|
|
42
50
|
# Raised when authentication fails (HTTP 401 or 403). Typically indicates
|
|
43
51
|
# an invalid, expired, or missing API key.
|
|
44
52
|
class AuthenticationError < ApiError
|
|
@@ -112,4 +120,12 @@ module RubyPi
|
|
|
112
120
|
super(message || "Model returned tool calls but no tools are registered")
|
|
113
121
|
end
|
|
114
122
|
end
|
|
123
|
+
|
|
124
|
+
# Raised when a model asks the executor to exceed its configured per-turn
|
|
125
|
+
# tool-call safety limit.
|
|
126
|
+
class ToolCallLimitError < Error; end
|
|
127
|
+
|
|
128
|
+
# Raised when too many previously timed-out in-process tools are still
|
|
129
|
+
# running and accepting more work would risk thread exhaustion.
|
|
130
|
+
class ToolExecutionCapacityError < Error; end
|
|
115
131
|
end
|
|
@@ -72,14 +72,6 @@ module RubyPi
|
|
|
72
72
|
end
|
|
73
73
|
end
|
|
74
74
|
|
|
75
|
-
# Returns the extension name. Override in subclasses to provide
|
|
76
|
-
# a human-readable identifier.
|
|
77
|
-
#
|
|
78
|
-
# @return [String] the extension name
|
|
79
|
-
def name
|
|
80
|
-
super
|
|
81
|
-
end
|
|
82
|
-
|
|
83
75
|
private
|
|
84
76
|
|
|
85
77
|
# Returns the hooks hash defined directly on this class (not
|
|
@@ -366,26 +366,28 @@ module RubyPi
|
|
|
366
366
|
usage_data = {}
|
|
367
367
|
finish_reason = nil
|
|
368
368
|
|
|
369
|
-
|
|
370
|
-
|
|
371
|
-
# which may split SSE events mid-line. We accumulate a line buffer and
|
|
372
|
-
# process complete lines incrementally so that deltas reach the caller
|
|
373
|
-
# as soon as each SSE event is fully received — not after the entire
|
|
374
|
-
# response has been buffered.
|
|
375
|
-
#
|
|
376
|
-
# The buffer is BINARY because chunks arrive as ASCII-8BIT and may end
|
|
377
|
-
# mid-way through a multi-byte UTF-8 character; appending such a chunk
|
|
378
|
-
# to a UTF-8 buffer that already holds non-ASCII text raises
|
|
379
|
-
# Encoding::CompatibilityError. Each complete line is re-encoded to
|
|
380
|
-
# UTF-8 (and scrubbed) before parsing, so deltas reach the caller as
|
|
381
|
-
# valid UTF-8 strings.
|
|
382
|
-
sse_buffer = (+"").force_encoding(Encoding::BINARY)
|
|
369
|
+
sse_parser = SSEParser.new(provider: provider_name)
|
|
370
|
+
stream_completed = false
|
|
383
371
|
response_status = nil
|
|
384
372
|
|
|
385
373
|
# Accumulate error response body separately so ApiError gets the
|
|
386
374
|
# full body even though on_data consumed the chunks.
|
|
387
375
|
error_body = (+"").force_encoding(Encoding::BINARY)
|
|
388
376
|
|
|
377
|
+
process_payload = proc do |data_str|
|
|
378
|
+
next if data_str == "[DONE]"
|
|
379
|
+
|
|
380
|
+
data = sse_parser.parse_json(data_str)
|
|
381
|
+
stream_completed = true if data["type"] == "message_stop"
|
|
382
|
+
stream_state = process_anthropic_stream_event(
|
|
383
|
+
data, accumulated_text, accumulated_tool_calls,
|
|
384
|
+
current_tool_call, current_tool_json, usage_data, finish_reason, block
|
|
385
|
+
)
|
|
386
|
+
current_tool_call = stream_state[:current_tool_call]
|
|
387
|
+
current_tool_json = stream_state[:current_tool_json]
|
|
388
|
+
finish_reason = stream_state[:finish_reason]
|
|
389
|
+
end
|
|
390
|
+
|
|
389
391
|
response = with_transport_errors do
|
|
390
392
|
conn.post("/v1/messages") do |req|
|
|
391
393
|
req.headers["Content-Type"] = "application/json"
|
|
@@ -395,7 +397,7 @@ module RubyPi
|
|
|
395
397
|
# Without this, Faraday buffers the entire response body before
|
|
396
398
|
# returning, which means no deltas reach the caller until the model
|
|
397
399
|
# finishes generating (fake streaming).
|
|
398
|
-
req.options.on_data = proc do |chunk,
|
|
400
|
+
req.options.on_data = proc do |chunk, _overall_received_bytes, env|
|
|
399
401
|
response_status ||= env&.status
|
|
400
402
|
|
|
401
403
|
# If the HTTP status indicates an error, accumulate the body for
|
|
@@ -403,41 +405,11 @@ module RubyPi
|
|
|
403
405
|
# calls on_data for error responses too, which would otherwise
|
|
404
406
|
# consume the body and leave response.body empty.
|
|
405
407
|
if response_status && response_status >= 400
|
|
406
|
-
error_body
|
|
408
|
+
append_error_body(error_body, chunk)
|
|
407
409
|
next
|
|
408
410
|
end
|
|
409
411
|
|
|
410
|
-
|
|
411
|
-
# Process all complete lines in the buffer. A complete line holds
|
|
412
|
-
# complete UTF-8 sequences (multi-byte characters split across
|
|
413
|
-
# chunks are repaired by the buffering), so re-encode it to UTF-8
|
|
414
|
-
# here; scrub guards against a server sending invalid bytes.
|
|
415
|
-
while (line_end = sse_buffer.index("\n"))
|
|
416
|
-
line = sse_buffer.slice!(0, line_end + 1).force_encoding(Encoding::UTF_8).scrub.strip
|
|
417
|
-
next if line.empty?
|
|
418
|
-
next unless line.start_with?("data: ")
|
|
419
|
-
|
|
420
|
-
data_str = line.sub(/\Adata: /, "")
|
|
421
|
-
next if data_str == "[DONE]"
|
|
422
|
-
|
|
423
|
-
begin
|
|
424
|
-
data = JSON.parse(data_str)
|
|
425
|
-
rescue JSON::ParserError
|
|
426
|
-
next
|
|
427
|
-
end
|
|
428
|
-
|
|
429
|
-
# --- process each SSE event exactly as before ---
|
|
430
|
-
# Process the SSE event and update mutable locals from the
|
|
431
|
-
# returned hash. This keeps all streaming state method-local,
|
|
432
|
-
# avoiding thread-unsafe instance variables.
|
|
433
|
-
stream_state = process_anthropic_stream_event(
|
|
434
|
-
data, accumulated_text, accumulated_tool_calls,
|
|
435
|
-
current_tool_call, current_tool_json, usage_data, finish_reason, block
|
|
436
|
-
)
|
|
437
|
-
current_tool_call = stream_state[:current_tool_call]
|
|
438
|
-
current_tool_json = stream_state[:current_tool_json]
|
|
439
|
-
finish_reason = stream_state[:finish_reason]
|
|
440
|
-
end
|
|
412
|
+
sse_parser.feed(chunk, &process_payload)
|
|
441
413
|
end
|
|
442
414
|
end # conn.post
|
|
443
415
|
end # with_transport_errors
|
|
@@ -452,25 +424,13 @@ module RubyPi
|
|
|
452
424
|
handle_error_response(error_response, override_body: error_body_str)
|
|
453
425
|
end
|
|
454
426
|
|
|
455
|
-
|
|
456
|
-
|
|
457
|
-
|
|
458
|
-
|
|
459
|
-
|
|
460
|
-
|
|
461
|
-
next if data_str == "[DONE]"
|
|
462
|
-
begin
|
|
463
|
-
data = JSON.parse(data_str)
|
|
464
|
-
rescue JSON::ParserError
|
|
465
|
-
next
|
|
466
|
-
end
|
|
467
|
-
stream_state = process_anthropic_stream_event(
|
|
468
|
-
data, accumulated_text, accumulated_tool_calls,
|
|
469
|
-
current_tool_call, current_tool_json, usage_data, finish_reason, block
|
|
427
|
+
sse_parser.finish(&process_payload)
|
|
428
|
+
|
|
429
|
+
unless stream_completed
|
|
430
|
+
raise RubyPi::StreamingProtocolError.new(
|
|
431
|
+
"anthropic stream ended before message_stop",
|
|
432
|
+
status_code: nil
|
|
470
433
|
)
|
|
471
|
-
current_tool_call = stream_state[:current_tool_call]
|
|
472
|
-
current_tool_json = stream_state[:current_tool_json]
|
|
473
|
-
finish_reason = stream_state[:finish_reason]
|
|
474
434
|
end
|
|
475
435
|
|
|
476
436
|
# (Event processing is now handled incrementally by the on_data callback
|
|
@@ -532,11 +492,14 @@ module RubyPi
|
|
|
532
492
|
delta = data["delta"] || {}
|
|
533
493
|
if delta["type"] == "text_delta"
|
|
534
494
|
text = delta["text"] || ""
|
|
535
|
-
accumulated_text
|
|
495
|
+
append_stream_data!(accumulated_text, text, limit: MAX_STREAM_OUTPUT_BYTES, label: "stream output")
|
|
536
496
|
block.call(StreamEvent.new(type: :text_delta, data: text))
|
|
537
497
|
elsif delta["type"] == "input_json_delta"
|
|
538
498
|
json_chunk = delta["partial_json"] || ""
|
|
539
|
-
|
|
499
|
+
append_stream_data!(
|
|
500
|
+
current_tool_json, json_chunk,
|
|
501
|
+
limit: MAX_STREAM_TOOL_ARGUMENT_BYTES, label: "tool arguments"
|
|
502
|
+
)
|
|
540
503
|
block.call(StreamEvent.new(type: :tool_call_delta, data: {
|
|
541
504
|
id: current_tool_call&.dig(:id),
|
|
542
505
|
partial_json: json_chunk
|
|
@@ -554,11 +517,10 @@ module RubyPi
|
|
|
554
517
|
else
|
|
555
518
|
begin
|
|
556
519
|
JSON.parse(current_tool_json)
|
|
557
|
-
rescue JSON::ParserError
|
|
520
|
+
rescue JSON::ParserError
|
|
558
521
|
raise RubyPi::ProviderError.new(
|
|
559
522
|
"Failed to parse streaming tool call arguments for " \
|
|
560
|
-
"'#{current_tool_call[:name]}'
|
|
561
|
-
"(accumulated JSON: #{current_tool_json.inspect})",
|
|
523
|
+
"'#{current_tool_call[:name]}'",
|
|
562
524
|
provider: :anthropic
|
|
563
525
|
)
|
|
564
526
|
end
|
|
@@ -568,6 +530,7 @@ module RubyPi
|
|
|
568
530
|
name: current_tool_call[:name],
|
|
569
531
|
arguments: arguments
|
|
570
532
|
)
|
|
533
|
+
enforce_stream_tool_call_limit!(accumulated_tool_calls.size)
|
|
571
534
|
current_tool_call = nil
|
|
572
535
|
current_tool_json = +""
|
|
573
536
|
end
|
|
@@ -33,6 +33,11 @@ module RubyPi
|
|
|
33
33
|
# end
|
|
34
34
|
# end
|
|
35
35
|
class BaseProvider
|
|
36
|
+
MAX_STREAM_OUTPUT_BYTES = 16 * 1024 * 1024
|
|
37
|
+
MAX_STREAM_TOOL_ARGUMENT_BYTES = 1024 * 1024
|
|
38
|
+
MAX_STREAM_TOOL_CALLS = 128
|
|
39
|
+
MAX_ERROR_BODY_BYTES = 64 * 1024
|
|
40
|
+
RETRY_AFTER_CEILING = 60.0
|
|
36
41
|
# @return [Integer] maximum number of retry attempts
|
|
37
42
|
attr_reader :max_retries
|
|
38
43
|
|
|
@@ -77,10 +82,12 @@ module RubyPi
|
|
|
77
82
|
def complete(messages:, tools: [], stream: false, &block)
|
|
78
83
|
attempt = 0
|
|
79
84
|
partial_chars = 0
|
|
85
|
+
partial_tool_calls = false
|
|
80
86
|
|
|
81
87
|
attempt_block = if stream && block
|
|
82
88
|
proc do |event|
|
|
83
89
|
partial_chars += event.data.to_s.length if event.text_delta?
|
|
90
|
+
partial_tool_calls = true if event.tool_call_delta?
|
|
84
91
|
block.call(event)
|
|
85
92
|
end
|
|
86
93
|
else
|
|
@@ -117,10 +124,12 @@ module RubyPi
|
|
|
117
124
|
provider: provider_name,
|
|
118
125
|
error: e.message,
|
|
119
126
|
attempt: attempt + 1,
|
|
120
|
-
partial_output: partial_chars.positive
|
|
121
|
-
partial_chars: partial_chars
|
|
127
|
+
partial_output: partial_chars.positive? || partial_tool_calls,
|
|
128
|
+
partial_chars: partial_chars,
|
|
129
|
+
partial_tool_calls: partial_tool_calls
|
|
122
130
|
}))
|
|
123
131
|
partial_chars = 0
|
|
132
|
+
partial_tool_calls = false
|
|
124
133
|
end
|
|
125
134
|
|
|
126
135
|
sleep(delay)
|
|
@@ -151,6 +160,31 @@ module RubyPi
|
|
|
151
160
|
|
|
152
161
|
private
|
|
153
162
|
|
|
163
|
+
def append_stream_data!(buffer, chunk, limit:, label:)
|
|
164
|
+
if buffer.bytesize + chunk.to_s.bytesize > limit
|
|
165
|
+
raise RubyPi::StreamingProtocolError.new(
|
|
166
|
+
"#{provider_name} #{label} exceeded #{limit} bytes",
|
|
167
|
+
status_code: nil
|
|
168
|
+
)
|
|
169
|
+
end
|
|
170
|
+
|
|
171
|
+
buffer << chunk
|
|
172
|
+
end
|
|
173
|
+
|
|
174
|
+
def append_error_body(error_body, chunk)
|
|
175
|
+
remaining = MAX_ERROR_BODY_BYTES - error_body.bytesize
|
|
176
|
+
error_body << chunk.to_s.b.byteslice(0, remaining) if remaining.positive?
|
|
177
|
+
end
|
|
178
|
+
|
|
179
|
+
def enforce_stream_tool_call_limit!(count)
|
|
180
|
+
return if count <= MAX_STREAM_TOOL_CALLS
|
|
181
|
+
|
|
182
|
+
raise RubyPi::StreamingProtocolError.new(
|
|
183
|
+
"#{provider_name} stream exceeded #{MAX_STREAM_TOOL_CALLS} tool calls",
|
|
184
|
+
status_code: nil
|
|
185
|
+
)
|
|
186
|
+
end
|
|
187
|
+
|
|
154
188
|
# Performs the actual completion request. Subclasses MUST implement this
|
|
155
189
|
# method with provider-specific HTTP logic.
|
|
156
190
|
#
|
|
@@ -159,15 +193,10 @@ module RubyPi
|
|
|
159
193
|
# @param stream [Boolean] streaming mode flag
|
|
160
194
|
# @yield [event] optional block for streaming events
|
|
161
195
|
# @return [RubyPi::LLM::Response]
|
|
162
|
-
def perform_complete(
|
|
196
|
+
def perform_complete(...)
|
|
163
197
|
raise RubyPi::AbstractMethodError, :perform_complete
|
|
164
198
|
end
|
|
165
199
|
|
|
166
|
-
# Maximum delay (seconds) honored from a server-provided Retry-After
|
|
167
|
-
# header. Caps pathological or misconfigured server values so a single
|
|
168
|
-
# 429 cannot stall the client indefinitely.
|
|
169
|
-
RETRY_AFTER_CEILING = 60.0
|
|
170
|
-
|
|
171
200
|
# Picks the delay before the next retry. A server-provided Retry-After
|
|
172
201
|
# on a 429 takes precedence over the local exponential backoff: the
|
|
173
202
|
# server knows its own cooldown window, and retrying earlier just burns
|
data/lib/ruby_pi/llm/fallback.rb
CHANGED
|
@@ -157,13 +157,17 @@ module RubyPi
|
|
|
157
157
|
# carries partial_output/partial_chars so consumers can deterministically
|
|
158
158
|
# truncate what they already rendered.
|
|
159
159
|
partial_chars = 0
|
|
160
|
+
partial_tool_calls = false
|
|
160
161
|
counting_block = proc do |event|
|
|
161
162
|
if event.retry_start?
|
|
162
163
|
# The provider tells consumers to discard its previous attempt,
|
|
163
164
|
# so only count text from the current attempt for a later fallback.
|
|
164
165
|
partial_chars = 0
|
|
166
|
+
partial_tool_calls = false
|
|
165
167
|
elsif event.text_delta?
|
|
166
168
|
partial_chars += event.data.to_s.length
|
|
169
|
+
elsif event.tool_call_delta?
|
|
170
|
+
partial_tool_calls = true
|
|
167
171
|
end
|
|
168
172
|
block.call(event)
|
|
169
173
|
end
|
|
@@ -193,10 +197,11 @@ module RubyPi
|
|
|
193
197
|
# this amount if appending to a shared buffer)
|
|
194
198
|
block.call(StreamEvent.new(type: :fallback_start, data: {
|
|
195
199
|
failed_provider: @primary.provider_name,
|
|
196
|
-
error: e
|
|
200
|
+
error: safe_error_message(e),
|
|
197
201
|
fallback_provider: @fallback.provider_name,
|
|
198
|
-
partial_output: partial_chars.positive
|
|
199
|
-
partial_chars: partial_chars
|
|
202
|
+
partial_output: partial_chars.positive? || partial_tool_calls,
|
|
203
|
+
partial_chars: partial_chars,
|
|
204
|
+
partial_tool_calls: partial_tool_calls
|
|
200
205
|
}))
|
|
201
206
|
|
|
202
207
|
# Stream directly from the fallback to the consumer's block.
|
|
@@ -219,10 +224,14 @@ module RubyPi
|
|
|
219
224
|
|
|
220
225
|
logger.warn(
|
|
221
226
|
"[RubyPi::Fallback] Primary provider (#{@primary.provider_name}/#{@primary.model_name}) " \
|
|
222
|
-
"failed with #{error.class}: #{error
|
|
227
|
+
"failed with #{error.class}: #{safe_error_message(error)}. " \
|
|
223
228
|
"Falling back to #{@fallback.provider_name}/#{@fallback.model_name}."
|
|
224
229
|
)
|
|
225
230
|
end
|
|
231
|
+
|
|
232
|
+
def safe_error_message(error)
|
|
233
|
+
error.message.to_s.gsub(/[[:cntrl:]]+/, " ").slice(0, 512)
|
|
234
|
+
end
|
|
226
235
|
end
|
|
227
236
|
end
|
|
228
237
|
end
|