ruby-pi 0.1.9 → 0.1.10

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: fbecc54fc18679e9f16a0a09c27944df82b0a98fff7ead1dee99ba1a659aad02
4
- data.tar.gz: 298e084a7df3b6689e628477ddbae866057a81ac472838623ce6eefccd01a431
3
+ metadata.gz: ae334e07e1a5788bfe62b1a1d721d565885ba0142dca6d9a238656b3835454a0
4
+ data.tar.gz: 3f68b1f787a4fa17fd6ea68ef1b4974c5bb625b76c3898786a9b23c0999c490b
5
5
  SHA512:
6
- metadata.gz: cbb1ae3d3469f987b4ab7652cce23626e6e1a233c08636ab24dcda3c6151d0bc728d88658e575919840e393c84ce4efabbd5fa78f302c036993a0d1b3e79b354
7
- data.tar.gz: 1132a860f026efc123813042eea18617242d036ec03ca425277cd2f9eba88a49f1cc2f461c6feee58fa021eeed8fbe42affee5c6f865ff6fad563809ede29479
6
+ metadata.gz: cc0289504d07435aa8e681a102bcdc4e27d8f4f8033ab73ea73c227e345ab64be93cc0fd8d3dd27d50c640d4758508e14a91b290fed02fd089477f73e5ea3d35
7
+ data.tar.gz: 031ad8a0ca080a0024f596ec05e57857bd7cab68dc9236ae5ada606f09bbeedea5e59142e01e6bdea2de032008a1624bd9ad0db78fd4ed6fc1ddcddaab53331b
data/CHANGELOG.md CHANGED
@@ -5,6 +5,27 @@ All notable changes to this project will be documented in this file.
5
5
  The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
6
6
  and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
7
 
8
+ ## [0.1.10] - 2026-08-28
9
+
10
+ ### Security
11
+
12
+ - **Malformed or truncated HTTP-200 streams were accepted (High)**: Added a shared bounded SSE decoder with UTF-8 validation, standards-compliant event framing, final-buffer processing, and typed retryable protocol errors. OpenAI now requires `data: [DONE]`, Anthropic requires `message_stop`, and Gemini requires a finish reason or explicit prompt block before emitting `:done`
13
+ - **Model-controlled tool fan-out was unbounded (High)**: Parallel execution now uses a bounded pool with a default concurrency of four, rejects more than 32 calls per execution, and enforces a process-wide cap of 16 active or timed-out-but-still-running tools. Streaming output, tool arguments, tool-call count, SSE events, and retained error bodies also have hard memory limits
14
+ - **Malformed model arguments could enter logs (Medium)**: Provider errors no longer embed raw tool arguments, and fallback diagnostics remove control characters and cap message length
15
+
16
+ ### Fixed (adversarial review round 8)
17
+
18
+ - **Tool timeout results falsely implied failure was final (High)**: Timed-out work is now reported as `status: :timeout_unknown` with `completion_unknown?`. Sequential execution does not start later calls after an uncertain timeout; those calls return `status: :skipped`. Arbitrary in-process Ruby blocks are still cooperative and cannot be force-terminated safely
19
+ - **Immutable agent results could crash on valid tool values (High)**: Result snapshots are cycle-aware and never call `dup` on arbitrary application objects. Non-JSON objects become bounded frozen diagnostic strings instead of raising from `initialize_copy` or recursing indefinitely
20
+ - **Retry/fallback reset metadata ignored tool fragments (Medium)**: Reset events now report `partial_tool_calls` and set `partial_output` whenever either text or structured tool-call output was emitted
21
+ - **Compaction ignored structured tool metadata (Medium)**: Token estimates and summary prompts now account for assistant tool calls, IDs, names, and arguments
22
+ - **Invalid safety limits failed deep in execution (Medium)**: Agent iteration limits, tool timeouts, call limits, and concurrency limits now reject non-finite, zero, negative, or wrong-type values at construction
23
+ - **Release gates covered tests only (Low)**: CI now runs dependency auditing, Lint/Security checks, gem build/install, and an installed-artifact load/version smoke test alongside Ruby 3.2-3.4 tests
24
+
25
+ ### Known limitation
26
+
27
+ - Ruby cannot safely force-terminate an arbitrary in-process block. A timed-out tool may still finish and commit side effects, so callers must treat `completion_unknown?` as an indeterminate outcome and use idempotency keys. Enforceable cancellation requires isolated worker processes or an external job backend in a future execution API
28
+
8
29
  ## [0.1.9] - 2026-08-28
9
30
 
10
31
  ### Security
data/README.md CHANGED
@@ -133,7 +133,8 @@ model.complete(messages: messages, stream: true) do |event|
133
133
  when :retry_start
134
134
  # A transient failure occurred after streaming may have begun. Discard
135
135
  # the previous attempt before the provider retries from the beginning.
136
- # Payload includes: { provider:, attempt:, partial_output:, partial_chars: }
136
+ # Payload includes partial_chars and partial_tool_calls. Always discard
137
+ # all text and structured tool state from the failed attempt.
137
138
  clear_partial_output
138
139
  when :fallback_start
139
140
  # Only emitted by RubyPi::LLM::Fallback when the primary provider
@@ -240,7 +241,13 @@ registry.all # => [Definition, ...]
240
241
  Run tool calls in parallel or sequentially with automatic error handling and timeouts:
241
242
 
242
243
  ```ruby
243
- executor = RubyPi::Tools::Executor.new(registry, mode: :parallel, timeout: 30)
244
+ executor = RubyPi::Tools::Executor.new(
245
+ registry,
246
+ mode: :parallel,
247
+ timeout: 30,
248
+ max_calls: 32,
249
+ max_concurrency: 4
250
+ )
244
251
 
245
252
  results = executor.execute([
246
253
  { name: "create_post", arguments: { content: "Hello" } },
@@ -252,6 +259,7 @@ results.each do |r|
252
259
  puts "#{r.name}: #{r.value}"
253
260
  else
254
261
  puts "#{r.name} failed: #{r.error} (#{r.duration_ms}ms)"
262
+ reconcile_with_an_idempotency_key if r.completion_unknown?
255
263
  end
256
264
  end
257
265
  ```
@@ -263,6 +271,15 @@ end
263
271
  | `value` | Return value (on success) |
264
272
  | `error` | Error message (on failure) |
265
273
  | `duration_ms` | Execution time in milliseconds |
274
+ | `status` | `:success`, `:error`, `:timeout_unknown`, or `:skipped` |
275
+ | `completion_unknown?` | The deadline elapsed, but the in-process tool may still finish or commit side effects |
276
+ | `skipped?` | Sequential execution did not start this call after an uncertain timeout |
277
+
278
+ Parallel execution is bounded. A model response containing more than `max_calls`
279
+ tool calls raises `RubyPi::ToolCallLimitError`, and the process refuses new work
280
+ after 16 tools remain active. Ruby cannot safely kill an arbitrary
281
+ in-process block, so use idempotency keys for side-effecting tools and treat
282
+ `completion_unknown?` as indeterminate rather than as a confirmed rollback.
266
283
 
267
284
  ---
268
285
 
@@ -101,6 +101,12 @@ module RubyPi
101
101
  execution_mode: :parallel,
102
102
  tool_timeout: 30
103
103
  )
104
+ valid_timeout = tool_timeout.is_a?(Numeric) && !tool_timeout.is_a?(Complex) &&
105
+ tool_timeout.finite? && tool_timeout.positive?
106
+ unless valid_timeout
107
+ raise ArgumentError, "tool_timeout must be a finite positive number, got #{tool_timeout.inspect}"
108
+ end
109
+
104
110
  @state = State.new(
105
111
  system_prompt: system_prompt,
106
112
  model: model,
@@ -143,25 +143,46 @@ module RubyPi
143
143
  # arrays still allowed callers (or later State mutations) to rewrite
144
144
  # nested messages, tool arguments, usage, and content in place.
145
145
  def deep_frozen_copy(value)
146
+ snapshot(value, {}.compare_by_identity).freeze
147
+ end
148
+
149
+ # Copies only JSON-like containers and immutable scalars. Arbitrary tool
150
+ # return objects are converted to a bounded diagnostic string instead of
151
+ # calling #dup, which may allocate resources, raise from initialize_copy,
152
+ # or recurse forever through cyclic structures.
153
+ def snapshot(value, memo)
154
+ if value.is_a?(Hash) || value.is_a?(Array)
155
+ return "[Circular]".freeze if memo.key?(value)
156
+
157
+ memo[value] = true
158
+ end
159
+
146
160
  copy = case value
147
161
  when Hash
148
162
  value.each_with_object({}) do |(key, item), result|
149
- result[deep_frozen_copy(key)] = deep_frozen_copy(item)
163
+ result[snapshot(key, memo).freeze] = snapshot(item, memo).freeze
150
164
  end
151
165
  when Array
152
- value.map { |item| deep_frozen_copy(item) }
166
+ value.map { |item| snapshot(item, memo).freeze }
153
167
  when String
154
168
  value.dup
169
+ when NilClass, TrueClass, FalseClass, Symbol, Numeric
170
+ value
155
171
  else
156
- begin
157
- value.dup
158
- rescue TypeError
159
- # Immutable scalar values (nil, symbols, numerics, booleans)
160
- # cannot be duplicated and are already safe to share.
161
- value
162
- end
172
+ safe_object_snapshot(value)
163
173
  end
164
- copy.freeze
174
+
175
+ memo.delete(value) if value.is_a?(Hash) || value.is_a?(Array)
176
+ copy
177
+ end
178
+
179
+ def safe_object_snapshot(value)
180
+ rendered = value.to_s
181
+ rendered = rendered.encode(Encoding::UTF_8, invalid: :replace, undef: :replace, replace: "�")
182
+ rendered = rendered.byteslice(0, 4096).scrub if rendered.bytesize > 4096
183
+ "#<#{value.class}: #{rendered}>"
184
+ rescue StandardError
185
+ "#<#{value.class}: uninspectable>"
165
186
  end
166
187
  end
167
188
  end
@@ -77,6 +77,10 @@ module RubyPi
77
77
  after_tool_call: nil,
78
78
  user_data: {}
79
79
  )
80
+ unless max_iterations.is_a?(Integer) && max_iterations.positive?
81
+ raise ArgumentError, "max_iterations must be a positive integer, got #{max_iterations.inspect}"
82
+ end
83
+
80
84
  @system_prompt = system_prompt
81
85
  @model = model
82
86
  @tools = tools
@@ -1,5 +1,7 @@
1
1
  # frozen_string_literal: true
2
2
 
3
+ require "json"
4
+
3
5
  # lib/ruby_pi/context/compaction.rb
4
6
  #
5
7
  # RubyPi::Context::Compaction — Token estimation and context window management.
@@ -28,6 +30,7 @@ module RubyPi
28
30
  # Average characters per token — a rough heuristic that avoids the need
29
31
  # for provider-specific tokenizers. Errs on the conservative side.
30
32
  CHARS_PER_TOKEN = 4
33
+ SUMMARY_METADATA_LIMIT = 16_384
31
34
 
32
35
  # @return [Integer] the token threshold above which compaction triggers
33
36
  attr_reader :max_tokens
@@ -160,6 +163,8 @@ module RubyPi
160
163
 
161
164
  messages.each do |msg|
162
165
  total_chars += message_value(msg, :content).to_s.length
166
+ metadata = msg.reject { |key, _| key == :role || key == "role" || key == :content || key == "content" }
167
+ total_chars += serialized_message(metadata).length unless metadata.empty?
163
168
  # Account for role and structural overhead (~10 tokens per message)
164
169
  total_chars += 40
165
170
  end
@@ -202,7 +207,9 @@ module RubyPi
202
207
  transcript = messages.map do |msg|
203
208
  role = message_value(msg, :role).to_s.capitalize
204
209
  content = message_value(msg, :content).to_s
205
- "#{role}: #{content}"
210
+ metadata = msg.reject { |key, _| key == :role || key == "role" || key == :content || key == "content" }
211
+ metadata_text = metadata.empty? ? nil : serialized_message(metadata).slice(0, SUMMARY_METADATA_LIMIT)
212
+ ["#{role}: #{content}", ("Structured metadata: #{metadata_text}" if metadata_text)].compact.join("\n")
206
213
  end.join("\n\n")
207
214
 
208
215
  "Summarize the following conversation, preserving all key facts, " \
@@ -214,6 +221,12 @@ module RubyPi
214
221
  def message_value(message, key)
215
222
  message[key] || message[key.to_s]
216
223
  end
224
+
225
+ def serialized_message(message)
226
+ JSON.generate(message)
227
+ rescue JSON::GeneratorError, TypeError
228
+ message.to_s
229
+ end
217
230
  end
218
231
  end
219
232
  end
@@ -39,6 +39,14 @@ module RubyPi
39
39
  end
40
40
  end
41
41
 
42
+ # Raised when an HTTP streaming response violates the provider's protocol.
43
+ # This includes malformed SSE data, invalid UTF-8, oversized events, and a
44
+ # connection that closes before the provider's terminal event arrives.
45
+ #
46
+ # It inherits from ApiError with a nil status code so BaseProvider treats a
47
+ # broken HTTP-200 stream like a transient transport failure and can retry it.
48
+ class StreamingProtocolError < ApiError; end
49
+
42
50
  # Raised when authentication fails (HTTP 401 or 403). Typically indicates
43
51
  # an invalid, expired, or missing API key.
44
52
  class AuthenticationError < ApiError
@@ -112,4 +120,12 @@ module RubyPi
112
120
  super(message || "Model returned tool calls but no tools are registered")
113
121
  end
114
122
  end
123
+
124
+ # Raised when a model asks the executor to exceed its configured per-turn
125
+ # tool-call safety limit.
126
+ class ToolCallLimitError < Error; end
127
+
128
+ # Raised when too many previously timed-out in-process tools are still
129
+ # running and accepting more work would risk thread exhaustion.
130
+ class ToolExecutionCapacityError < Error; end
115
131
  end
@@ -72,14 +72,6 @@ module RubyPi
72
72
  end
73
73
  end
74
74
 
75
- # Returns the extension name. Override in subclasses to provide
76
- # a human-readable identifier.
77
- #
78
- # @return [String] the extension name
79
- def name
80
- super
81
- end
82
-
83
75
  private
84
76
 
85
77
  # Returns the hooks hash defined directly on this class (not
@@ -366,26 +366,28 @@ module RubyPi
366
366
  usage_data = {}
367
367
  finish_reason = nil
368
368
 
369
- # Buffer for incomplete SSE lines across on_data chunks. Faraday's
370
- # on_data callback delivers raw bytes as they arrive from the network,
371
- # which may split SSE events mid-line. We accumulate a line buffer and
372
- # process complete lines incrementally so that deltas reach the caller
373
- # as soon as each SSE event is fully received — not after the entire
374
- # response has been buffered.
375
- #
376
- # The buffer is BINARY because chunks arrive as ASCII-8BIT and may end
377
- # mid-way through a multi-byte UTF-8 character; appending such a chunk
378
- # to a UTF-8 buffer that already holds non-ASCII text raises
379
- # Encoding::CompatibilityError. Each complete line is re-encoded to
380
- # UTF-8 (and scrubbed) before parsing, so deltas reach the caller as
381
- # valid UTF-8 strings.
382
- sse_buffer = (+"").force_encoding(Encoding::BINARY)
369
+ sse_parser = SSEParser.new(provider: provider_name)
370
+ stream_completed = false
383
371
  response_status = nil
384
372
 
385
373
  # Accumulate error response body separately so ApiError gets the
386
374
  # full body even though on_data consumed the chunks.
387
375
  error_body = (+"").force_encoding(Encoding::BINARY)
388
376
 
377
+ process_payload = proc do |data_str|
378
+ next if data_str == "[DONE]"
379
+
380
+ data = sse_parser.parse_json(data_str)
381
+ stream_completed = true if data["type"] == "message_stop"
382
+ stream_state = process_anthropic_stream_event(
383
+ data, accumulated_text, accumulated_tool_calls,
384
+ current_tool_call, current_tool_json, usage_data, finish_reason, block
385
+ )
386
+ current_tool_call = stream_state[:current_tool_call]
387
+ current_tool_json = stream_state[:current_tool_json]
388
+ finish_reason = stream_state[:finish_reason]
389
+ end
390
+
389
391
  response = with_transport_errors do
390
392
  conn.post("/v1/messages") do |req|
391
393
  req.headers["Content-Type"] = "application/json"
@@ -395,7 +397,7 @@ module RubyPi
395
397
  # Without this, Faraday buffers the entire response body before
396
398
  # returning, which means no deltas reach the caller until the model
397
399
  # finishes generating (fake streaming).
398
- req.options.on_data = proc do |chunk, overall_received_bytes, env|
400
+ req.options.on_data = proc do |chunk, _overall_received_bytes, env|
399
401
  response_status ||= env&.status
400
402
 
401
403
  # If the HTTP status indicates an error, accumulate the body for
@@ -403,41 +405,11 @@ module RubyPi
403
405
  # calls on_data for error responses too, which would otherwise
404
406
  # consume the body and leave response.body empty.
405
407
  if response_status && response_status >= 400
406
- error_body << chunk.b
408
+ append_error_body(error_body, chunk)
407
409
  next
408
410
  end
409
411
 
410
- sse_buffer << chunk.b
411
- # Process all complete lines in the buffer. A complete line holds
412
- # complete UTF-8 sequences (multi-byte characters split across
413
- # chunks are repaired by the buffering), so re-encode it to UTF-8
414
- # here; scrub guards against a server sending invalid bytes.
415
- while (line_end = sse_buffer.index("\n"))
416
- line = sse_buffer.slice!(0, line_end + 1).force_encoding(Encoding::UTF_8).scrub.strip
417
- next if line.empty?
418
- next unless line.start_with?("data: ")
419
-
420
- data_str = line.sub(/\Adata: /, "")
421
- next if data_str == "[DONE]"
422
-
423
- begin
424
- data = JSON.parse(data_str)
425
- rescue JSON::ParserError
426
- next
427
- end
428
-
429
- # --- process each SSE event exactly as before ---
430
- # Process the SSE event and update mutable locals from the
431
- # returned hash. This keeps all streaming state method-local,
432
- # avoiding thread-unsafe instance variables.
433
- stream_state = process_anthropic_stream_event(
434
- data, accumulated_text, accumulated_tool_calls,
435
- current_tool_call, current_tool_json, usage_data, finish_reason, block
436
- )
437
- current_tool_call = stream_state[:current_tool_call]
438
- current_tool_json = stream_state[:current_tool_json]
439
- finish_reason = stream_state[:finish_reason]
440
- end
412
+ sse_parser.feed(chunk, &process_payload)
441
413
  end
442
414
  end # conn.post
443
415
  end # with_transport_errors
@@ -452,25 +424,13 @@ module RubyPi
452
424
  handle_error_response(error_response, override_body: error_body_str)
453
425
  end
454
426
 
455
- # Process any remaining data in the buffer after the connection closes
456
- sse_buffer.force_encoding(Encoding::UTF_8).scrub.each_line do |line|
457
- line = line.strip
458
- next if line.empty?
459
- next unless line.start_with?("data: ")
460
- data_str = line.sub(/\Adata: /, "")
461
- next if data_str == "[DONE]"
462
- begin
463
- data = JSON.parse(data_str)
464
- rescue JSON::ParserError
465
- next
466
- end
467
- stream_state = process_anthropic_stream_event(
468
- data, accumulated_text, accumulated_tool_calls,
469
- current_tool_call, current_tool_json, usage_data, finish_reason, block
427
+ sse_parser.finish(&process_payload)
428
+
429
+ unless stream_completed
430
+ raise RubyPi::StreamingProtocolError.new(
431
+ "anthropic stream ended before message_stop",
432
+ status_code: nil
470
433
  )
471
- current_tool_call = stream_state[:current_tool_call]
472
- current_tool_json = stream_state[:current_tool_json]
473
- finish_reason = stream_state[:finish_reason]
474
434
  end
475
435
 
476
436
  # (Event processing is now handled incrementally by the on_data callback
@@ -532,11 +492,14 @@ module RubyPi
532
492
  delta = data["delta"] || {}
533
493
  if delta["type"] == "text_delta"
534
494
  text = delta["text"] || ""
535
- accumulated_text << text
495
+ append_stream_data!(accumulated_text, text, limit: MAX_STREAM_OUTPUT_BYTES, label: "stream output")
536
496
  block.call(StreamEvent.new(type: :text_delta, data: text))
537
497
  elsif delta["type"] == "input_json_delta"
538
498
  json_chunk = delta["partial_json"] || ""
539
- current_tool_json << json_chunk
499
+ append_stream_data!(
500
+ current_tool_json, json_chunk,
501
+ limit: MAX_STREAM_TOOL_ARGUMENT_BYTES, label: "tool arguments"
502
+ )
540
503
  block.call(StreamEvent.new(type: :tool_call_delta, data: {
541
504
  id: current_tool_call&.dig(:id),
542
505
  partial_json: json_chunk
@@ -554,11 +517,10 @@ module RubyPi
554
517
  else
555
518
  begin
556
519
  JSON.parse(current_tool_json)
557
- rescue JSON::ParserError => e
520
+ rescue JSON::ParserError
558
521
  raise RubyPi::ProviderError.new(
559
522
  "Failed to parse streaming tool call arguments for " \
560
- "'#{current_tool_call[:name]}': #{e.message} " \
561
- "(accumulated JSON: #{current_tool_json.inspect})",
523
+ "'#{current_tool_call[:name]}'",
562
524
  provider: :anthropic
563
525
  )
564
526
  end
@@ -568,6 +530,7 @@ module RubyPi
568
530
  name: current_tool_call[:name],
569
531
  arguments: arguments
570
532
  )
533
+ enforce_stream_tool_call_limit!(accumulated_tool_calls.size)
571
534
  current_tool_call = nil
572
535
  current_tool_json = +""
573
536
  end
@@ -33,6 +33,11 @@ module RubyPi
33
33
  # end
34
34
  # end
35
35
  class BaseProvider
36
+ MAX_STREAM_OUTPUT_BYTES = 16 * 1024 * 1024
37
+ MAX_STREAM_TOOL_ARGUMENT_BYTES = 1024 * 1024
38
+ MAX_STREAM_TOOL_CALLS = 128
39
+ MAX_ERROR_BODY_BYTES = 64 * 1024
40
+ RETRY_AFTER_CEILING = 60.0
36
41
  # @return [Integer] maximum number of retry attempts
37
42
  attr_reader :max_retries
38
43
 
@@ -77,10 +82,12 @@ module RubyPi
77
82
  def complete(messages:, tools: [], stream: false, &block)
78
83
  attempt = 0
79
84
  partial_chars = 0
85
+ partial_tool_calls = false
80
86
 
81
87
  attempt_block = if stream && block
82
88
  proc do |event|
83
89
  partial_chars += event.data.to_s.length if event.text_delta?
90
+ partial_tool_calls = true if event.tool_call_delta?
84
91
  block.call(event)
85
92
  end
86
93
  else
@@ -117,10 +124,12 @@ module RubyPi
117
124
  provider: provider_name,
118
125
  error: e.message,
119
126
  attempt: attempt + 1,
120
- partial_output: partial_chars.positive?,
121
- partial_chars: partial_chars
127
+ partial_output: partial_chars.positive? || partial_tool_calls,
128
+ partial_chars: partial_chars,
129
+ partial_tool_calls: partial_tool_calls
122
130
  }))
123
131
  partial_chars = 0
132
+ partial_tool_calls = false
124
133
  end
125
134
 
126
135
  sleep(delay)
@@ -151,6 +160,31 @@ module RubyPi
151
160
 
152
161
  private
153
162
 
163
+ def append_stream_data!(buffer, chunk, limit:, label:)
164
+ if buffer.bytesize + chunk.to_s.bytesize > limit
165
+ raise RubyPi::StreamingProtocolError.new(
166
+ "#{provider_name} #{label} exceeded #{limit} bytes",
167
+ status_code: nil
168
+ )
169
+ end
170
+
171
+ buffer << chunk
172
+ end
173
+
174
+ def append_error_body(error_body, chunk)
175
+ remaining = MAX_ERROR_BODY_BYTES - error_body.bytesize
176
+ error_body << chunk.to_s.b.byteslice(0, remaining) if remaining.positive?
177
+ end
178
+
179
+ def enforce_stream_tool_call_limit!(count)
180
+ return if count <= MAX_STREAM_TOOL_CALLS
181
+
182
+ raise RubyPi::StreamingProtocolError.new(
183
+ "#{provider_name} stream exceeded #{MAX_STREAM_TOOL_CALLS} tool calls",
184
+ status_code: nil
185
+ )
186
+ end
187
+
154
188
  # Performs the actual completion request. Subclasses MUST implement this
155
189
  # method with provider-specific HTTP logic.
156
190
  #
@@ -159,15 +193,10 @@ module RubyPi
159
193
  # @param stream [Boolean] streaming mode flag
160
194
  # @yield [event] optional block for streaming events
161
195
  # @return [RubyPi::LLM::Response]
162
- def perform_complete(messages:, tools:, stream:, &block)
196
+ def perform_complete(...)
163
197
  raise RubyPi::AbstractMethodError, :perform_complete
164
198
  end
165
199
 
166
- # Maximum delay (seconds) honored from a server-provided Retry-After
167
- # header. Caps pathological or misconfigured server values so a single
168
- # 429 cannot stall the client indefinitely.
169
- RETRY_AFTER_CEILING = 60.0
170
-
171
200
  # Picks the delay before the next retry. A server-provided Retry-After
172
201
  # on a 429 takes precedence over the local exponential backoff: the
173
202
  # server knows its own cooldown window, and retrying earlier just burns
@@ -157,13 +157,17 @@ module RubyPi
157
157
  # carries partial_output/partial_chars so consumers can deterministically
158
158
  # truncate what they already rendered.
159
159
  partial_chars = 0
160
+ partial_tool_calls = false
160
161
  counting_block = proc do |event|
161
162
  if event.retry_start?
162
163
  # The provider tells consumers to discard its previous attempt,
163
164
  # so only count text from the current attempt for a later fallback.
164
165
  partial_chars = 0
166
+ partial_tool_calls = false
165
167
  elsif event.text_delta?
166
168
  partial_chars += event.data.to_s.length
169
+ elsif event.tool_call_delta?
170
+ partial_tool_calls = true
167
171
  end
168
172
  block.call(event)
169
173
  end
@@ -193,10 +197,11 @@ module RubyPi
193
197
  # this amount if appending to a shared buffer)
194
198
  block.call(StreamEvent.new(type: :fallback_start, data: {
195
199
  failed_provider: @primary.provider_name,
196
- error: e.message,
200
+ error: safe_error_message(e),
197
201
  fallback_provider: @fallback.provider_name,
198
- partial_output: partial_chars.positive?,
199
- partial_chars: partial_chars
202
+ partial_output: partial_chars.positive? || partial_tool_calls,
203
+ partial_chars: partial_chars,
204
+ partial_tool_calls: partial_tool_calls
200
205
  }))
201
206
 
202
207
  # Stream directly from the fallback to the consumer's block.
@@ -219,10 +224,14 @@ module RubyPi
219
224
 
220
225
  logger.warn(
221
226
  "[RubyPi::Fallback] Primary provider (#{@primary.provider_name}/#{@primary.model_name}) " \
222
- "failed with #{error.class}: #{error.message}. " \
227
+ "failed with #{error.class}: #{safe_error_message(error)}. " \
223
228
  "Falling back to #{@fallback.provider_name}/#{@fallback.model_name}."
224
229
  )
225
230
  end
231
+
232
+ def safe_error_message(error)
233
+ error.message.to_s.gsub(/[[:cntrl:]]+/, " ").slice(0, 512)
234
+ end
226
235
  end
227
236
  end
228
237
  end