llm_logs 0.6.0 → 0.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +5 -15
- data/lib/llm_logs/engine.rb +0 -9
- data/lib/llm_logs/version.rb +1 -1
- metadata +3 -9
- data/lib/llm_logs/ruby_llm_patches/bedrock_sigv4.rb +0 -75
- data/lib/llm_logs/ruby_llm_patches/converse_claude_adaptive_thinking.rb +0 -70
- data/lib/llm_logs/ruby_llm_patches/converse_foreign_reasoning.rb +0 -80
- data/lib/llm_logs/ruby_llm_patches/converse_reasoning_config.rb +0 -78
- data/lib/llm_logs/ruby_llm_patches/retry_ssl_error.rb +0 -25
- data/lib/llm_logs/ruby_llm_patches.rb +0 -54
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 5d03f4e8b8e5607855655d0d756f0c906395e42e808d3bfe255ced558a639d5e
|
|
4
|
+
data.tar.gz: bb172cb33c5d864ec6dde523cfe64e14c9fe4783a0a4b9c3e856dc58717b0cb3
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 904aefd977972ab4bb67534972656d9abd55b3d4120c2ed99f961b661b5dfc9c1d2f7861d1afe9b74b8b6f175d86d9a5315ac2a468488ae58f23e737ef8d7c4c
|
|
7
|
+
data.tar.gz: 3e8e0f902575f2e2cbaf656965267343bf2124f781de41c036fe426d1bb6382551bbaede74f5f794ab52cdcaedba858c8adc6452554728ffbcf28786ccd0f53a
|
data/README.md
CHANGED
|
@@ -335,28 +335,18 @@ end
|
|
|
335
335
|
|
|
336
336
|
With `auto_instrument` on, LlmLogs subscribes to ruby_llm's `chat.ruby_llm` and `tool_call.ruby_llm` notifications: one `llm` span per provider round and one tool span per tool call, all siblings under the trace. Tokens and cost come from each round's usage entries. This requires `RubyLLM.config.instrumenter` to be `ActiveSupport::Notifications`; the Railtie sets that default, and `LlmLogs::Instrumentation::RubyLlmChat.install!` sets it when unset.
|
|
337
337
|
|
|
338
|
-
###
|
|
338
|
+
### Bedrock fixes in ruby_llm
|
|
339
339
|
|
|
340
|
-
|
|
340
|
+
llm_logs no longer patches ruby_llm. The Bedrock bugs it used to patch are fixed in ruby_llm 2.1: [crmne/ruby_llm#1024](https://github.com/crmne/ruby_llm/pull/1024) (re-sign each attempt, retry TLS errors), [#1025](https://github.com/crmne/ruby_llm/pull/1025) (`reasoning_config` for GPT, adaptive thinking for adaptive-only Claude) and [ab4a4f06](https://github.com/crmne/ruby_llm/commit/ab4a4f060fc4760053105b92500c042c52c3d968) (reasoning replays only to the model that produced it). llm_logs 0.7.0 requires ruby_llm 2.1 or later. Two behaviours differ from the old patches:
|
|
341
341
|
|
|
342
|
-
|
|
343
|
-
|
|
344
|
-
| BedrockSigV4, RetrySSLError | [crmne/ruby_llm#1024](https://github.com/crmne/ruby_llm/pull/1024) |
|
|
345
|
-
| ConverseReasoningConfig, ConverseClaudeAdaptiveThinking | [crmne/ruby_llm#1025](https://github.com/crmne/ruby_llm/pull/1025) |
|
|
346
|
-
| ConverseForeignReasoning | [crmne/ruby_llm#1026](https://github.com/crmne/ruby_llm/pull/1026) |
|
|
347
|
-
|
|
348
|
-
- **BedrockSigV4** ([#1024](https://github.com/crmne/ruby_llm/pull/1024), "Sign each Bedrock attempt when it is sent") re-signs every Bedrock attempt at send time, over the body being sent, and removes an `X-Amz-Security-Token` the new signature no longer carries. Upstream 2.0.0 signs once before the retry middleware, so a retry after a long timeout replays an expired signature (403 "Signature expired"). Drop when a ruby_llm release includes #1024.
|
|
349
|
-
- **RetrySSLError** ([#1024](https://github.com/crmne/ruby_llm/pull/1024), "Retry requests that fail with a TLS error") adds `Faraday::SSLError` (for example "SSL_connect ... unexpected eof") to the retried exceptions. This is the same trade-off ruby_llm already accepts for read timeouts: a request that reached the server may be sent twice. ruby_llm's `retry_if` still refuses to retry requests marked non-idempotent and streams that already delivered content. Drop when a ruby_llm release includes #1024.
|
|
350
|
-
- **ConverseReasoningConfig** ([#1025](https://github.com/crmne/ruby_llm/pull/1025), "Send reasoning_config to models that publish it") sends a reasoning effort on Bedrock Converse as `additionalModelRequestFields: {reasoning_config: "<effort>"}` (including `none`, which is also what `with_thinking(false)` resolves to for these models) when the model's Converse `additionalRequestFieldsSchema`, or that of another registry entry for the same foundation model, publishes a `reasoning_config` enum (OpenAI GPT 5.6/6 Astra, xAI Grok 4.6, ...). Upstream 2.0.0 sends `reasoning_effort`, which Bedrock rejects for GPT (400 unknown_parameter). Thinking turned off, budgets, Claude, Nova, and gpt-oss are unchanged. Beyond #1025, OpenAI GPT ids with no published schema (ruby_llm 2.0.0's registry lacks GPT-6 Sol/Luna) get the same shape, under any region prefix. Drop when a ruby_llm release includes #1025; that fallback goes with it, so first check that the registry you run publishes the schema for every GPT id you use (or add `metadata.converse.additionalRequestFieldsSchema` to your catalog entry), or those ids go back to `reasoning_effort`.
|
|
351
|
-
- **ConverseClaudeAdaptiveThinking** ([#1025](https://github.com/crmne/ruby_llm/pull/1025), "Think adaptively on adaptive-only Claude via Converse") sends adaptive thinking to adaptive-only Claude models on Bedrock Converse (the registry advertises an `effort` option and no `budget_tokens` option: Sonnet 5, Opus 4.7/4.8/5, Fable 5) as `additionalModelRequestFields: {thinking: {type: "adaptive"}, output_config: {effort: ...}}` (every advertised tier, `xhigh` and `max` included; `{thinking: {type: "adaptive"}}` alone when no effort is set, as for `with_thinking(true)` or a `display` alone; nothing for `none`; no `display` is sent). Upstream 2.0.0 sends a fixed `reasoning_config` budget, which these models reject ("thinking.type.enabled" is not supported for this model). Any budget, thinking turned off, budget-style Claude (Haiku 4.5, Sonnet 4.6), and other vendors are unchanged. An unregistered id takes its reasoning options from a registry entry for the same foundation model. Drop when a ruby_llm release includes #1025.
|
|
352
|
-
- **ConverseForeignReasoning** ([#1026](https://github.com/crmne/ruby_llm/pull/1026), "Drop Bedrock reasoning another model family produced" and "Give application inference profiles no vendor") replays a message's reasoning only to a model of the same family on Bedrock Converse. Upstream 2.0.0 replays every assistant message's reasoning to whatever model is next, and Claude and GPT are both the `bedrock` provider: a chat Claude answered and GPT continues fails with "This model doesn't support the reasoningContent.reasoningText.text field", and a chat GPT answered and Claude continues hands Claude GPT's encrypted reasoning. A model's family is `anthropic`, `openai`, or the vendor segment of any other Bedrock id (`amazon`, `meta`, ...), whatever its region prefix; an application inference profile ARN names no family, even with a dot in its id (`.../application-inference-profile/team.prod`). The producer is `RubyLLM::Message#model` (for a persisted message, the model of its last successful usage entry). Reasoning from another family is dropped. Anthropic targets otherwise get upstream's replay unchanged, and so does a target that names no family. **This patch is stricter than #1026 on purpose:** a non-Anthropic target whose producer is unknown or of the same family gets only the `redactedContent` blocks stored in `raw_reasoning["converse"]` (GPT's own encrypted reasoning across tool-call rounds), never `reasoningText` or the thinking text/signature fallback, where #1026 replays them all. Drop when a ruby_llm release includes #1026, accepting that looser replay (or keep the stricter rule as a patch of its own).
|
|
353
|
-
|
|
354
|
-
Not backported: #1025's first commit ("Recognise the in. Bedrock inference profile prefix"), which adds `in` to `Converse::REGION_PREFIXES` and also changes region rewriting and Mantle routing. The patches here match ids under any region prefix instead.
|
|
342
|
+
- GPT effort goes out as `reasoning_config` only when the model's registry entry publishes that schema. ruby_llm's registry does for the GPT-6 and GPT-5.6 ids; a catalog entry of your own for a GPT id needs `metadata.converse.additionalRequestFieldsSchema`, or the effort goes out as `reasoning_effort`, which Bedrock rejects.
|
|
343
|
+
- Reasoning replays only to the exact model that produced it, not to any model of the same family.
|
|
355
344
|
|
|
356
345
|
## Requirements
|
|
357
346
|
|
|
358
347
|
- Rails 8.0+
|
|
359
348
|
- Ruby 3.3+
|
|
349
|
+
- ruby_llm 2.1+
|
|
360
350
|
- PostgreSQL
|
|
361
351
|
|
|
362
352
|
## License
|
data/lib/llm_logs/engine.rb
CHANGED
|
@@ -11,15 +11,6 @@ module LlmLogs
|
|
|
11
11
|
end
|
|
12
12
|
end
|
|
13
13
|
|
|
14
|
-
# Bedrock fixes missing from upstream ruby_llm 2.0.0. Installed whether or not
|
|
15
|
-
# auto-instrumentation is on: they change request behaviour, not logging.
|
|
16
|
-
initializer "llm_logs.ruby_llm_patches" do
|
|
17
|
-
if defined?(RubyLLM::VERSION)
|
|
18
|
-
require "llm_logs/ruby_llm_patches"
|
|
19
|
-
LlmLogs::RubyLLMPatches.install!
|
|
20
|
-
end
|
|
21
|
-
end
|
|
22
|
-
|
|
23
14
|
rake_tasks do
|
|
24
15
|
load File.expand_path("../tasks/llm_logs.rake", __dir__)
|
|
25
16
|
end
|
data/lib/llm_logs/version.rb
CHANGED
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: llm_logs
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.7.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Anton
|
|
@@ -99,7 +99,7 @@ dependencies:
|
|
|
99
99
|
requirements:
|
|
100
100
|
- - ">="
|
|
101
101
|
- !ruby/object:Gem::Version
|
|
102
|
-
version: '2.
|
|
102
|
+
version: '2.1'
|
|
103
103
|
- - "<"
|
|
104
104
|
- !ruby/object:Gem::Version
|
|
105
105
|
version: '3'
|
|
@@ -109,7 +109,7 @@ dependencies:
|
|
|
109
109
|
requirements:
|
|
110
110
|
- - ">="
|
|
111
111
|
- !ruby/object:Gem::Version
|
|
112
|
-
version: '2.
|
|
112
|
+
version: '2.1'
|
|
113
113
|
- - "<"
|
|
114
114
|
- !ruby/object:Gem::Version
|
|
115
115
|
version: '3'
|
|
@@ -241,12 +241,6 @@ files:
|
|
|
241
241
|
- lib/llm_logs/engine.rb
|
|
242
242
|
- lib/llm_logs/instrumentation/ruby_llm_chat.rb
|
|
243
243
|
- lib/llm_logs/prompt_renderer.rb
|
|
244
|
-
- lib/llm_logs/ruby_llm_patches.rb
|
|
245
|
-
- lib/llm_logs/ruby_llm_patches/bedrock_sigv4.rb
|
|
246
|
-
- lib/llm_logs/ruby_llm_patches/converse_claude_adaptive_thinking.rb
|
|
247
|
-
- lib/llm_logs/ruby_llm_patches/converse_foreign_reasoning.rb
|
|
248
|
-
- lib/llm_logs/ruby_llm_patches/converse_reasoning_config.rb
|
|
249
|
-
- lib/llm_logs/ruby_llm_patches/retry_ssl_error.rb
|
|
250
244
|
- lib/llm_logs/tracer.rb
|
|
251
245
|
- lib/llm_logs/version.rb
|
|
252
246
|
- lib/tasks/llm_logs.rake
|
|
@@ -1,75 +0,0 @@
|
|
|
1
|
-
require "faraday"
|
|
2
|
-
|
|
3
|
-
module LlmLogs
|
|
4
|
-
module RubyLLMPatches
|
|
5
|
-
# Signs every Bedrock attempt when it is sent.
|
|
6
|
-
# Backport of crmne/ruby_llm#1024 ("Sign each Bedrock attempt when it is sent").
|
|
7
|
-
#
|
|
8
|
-
# ruby_llm 2.0.0 signs inside the request block (Protocols::Converse#signed_post,
|
|
9
|
-
# Converse::Streaming#stream_response, InvokeModel, Mantle, Guardrails, Rerank, ...),
|
|
10
|
-
# i.e. once, before Faraday's retry middleware runs. A retry after a long timeout
|
|
11
|
-
# (request_timeout 300s vs SigV4's 5-minute window) then replays an expired
|
|
12
|
-
# X-Amz-Date/Authorization and AWS answers 403 "Signature expired", which is not retried.
|
|
13
|
-
#
|
|
14
|
-
# This adds a middleware innermost in every Bedrock Transport::Connection stack (after
|
|
15
|
-
# :retry and after the :json encoder), so each attempt is re-signed over the exact body
|
|
16
|
-
# bytes being sent, at send time. It only touches requests that already carry a SigV4
|
|
17
|
-
# Authorization header and keeps the signing service named in it (bedrock,
|
|
18
|
-
# bedrock-mantle, ...). Auth#signed_get/#signed_post (model listing, batch control
|
|
19
|
-
# endpoints) use Connection.basic, which has no retry, so they need no change.
|
|
20
|
-
module BedrockSigV4
|
|
21
|
-
class Middleware < Faraday::Middleware
|
|
22
|
-
CREDENTIAL = %r{\AAWS4-HMAC-SHA256 Credential=[^/]+/\d{8}/[^/]+/(?<service>[^/]+)/aws4_request}
|
|
23
|
-
|
|
24
|
-
def initialize(app, provider:)
|
|
25
|
-
super(app)
|
|
26
|
-
@provider = provider
|
|
27
|
-
end
|
|
28
|
-
|
|
29
|
-
def call(env)
|
|
30
|
-
resign(env)
|
|
31
|
-
@app.call(env)
|
|
32
|
-
end
|
|
33
|
-
|
|
34
|
-
private
|
|
35
|
-
|
|
36
|
-
def resign(env)
|
|
37
|
-
match = CREDENTIAL.match(env.request_headers["Authorization"].to_s)
|
|
38
|
-
return unless match
|
|
39
|
-
|
|
40
|
-
body = env.body.nil? ? "" : env.body
|
|
41
|
-
return unless body.is_a?(String) # multipart/IO bodies keep the original signature
|
|
42
|
-
|
|
43
|
-
url = env.url
|
|
44
|
-
path = url.query ? "#{url.path}?#{url.query}" : url.path
|
|
45
|
-
headers = @provider.sign_headers(
|
|
46
|
-
env.method.to_s.upcase, path, body,
|
|
47
|
-
base_url: "#{url.scheme}://#{url.host}", service: match[:service]
|
|
48
|
-
)
|
|
49
|
-
env.request_headers.delete("X-Amz-Security-Token") # credentials may have rotated
|
|
50
|
-
env.request_headers.merge!(headers.except("Content-Type"))
|
|
51
|
-
end
|
|
52
|
-
end
|
|
53
|
-
|
|
54
|
-
module ConnectionPatch
|
|
55
|
-
private
|
|
56
|
-
|
|
57
|
-
def setup_middleware(faraday)
|
|
58
|
-
super
|
|
59
|
-
faraday.use(Middleware, provider: @provider) if BedrockSigV4.bedrock?(@provider)
|
|
60
|
-
end
|
|
61
|
-
end
|
|
62
|
-
|
|
63
|
-
module_function
|
|
64
|
-
|
|
65
|
-
def bedrock?(provider)
|
|
66
|
-
defined?(RubyLLM::Providers::Bedrock) && provider.is_a?(RubyLLM::Providers::Bedrock) &&
|
|
67
|
-
provider.respond_to?(:sign_headers)
|
|
68
|
-
end
|
|
69
|
-
|
|
70
|
-
def install!
|
|
71
|
-
RubyLLMPatches.prepend_checked(RubyLLM::Transport::Connection, ConnectionPatch, %i[setup_middleware])
|
|
72
|
-
end
|
|
73
|
-
end
|
|
74
|
-
end
|
|
75
|
-
end
|
|
@@ -1,70 +0,0 @@
|
|
|
1
|
-
module LlmLogs
|
|
2
|
-
module RubyLLMPatches
|
|
3
|
-
# Adaptive thinking for adaptive-only Claude models on Bedrock Converse.
|
|
4
|
-
# Backport of crmne/ruby_llm#1025 ("Think adaptively on adaptive-only Claude via Converse").
|
|
5
|
-
#
|
|
6
|
-
# ruby_llm 2.0.0 (Protocols::Converse::Chat#format_reasoning_fields) turns a Claude
|
|
7
|
-
# reasoning effort into a fixed budget, `additionalModelRequestFields: {reasoning_config:
|
|
8
|
-
# {type: "enabled", budget_tokens: N}}`, sized from the Converse budgetTokens schema.
|
|
9
|
-
# Claude models that only think adaptively (the registry advertises an effort option and
|
|
10
|
-
# no budget_tokens option: Sonnet 5, Opus 4.7/4.8/5, Fable 5) reject it with
|
|
11
|
-
# `"thinking.type.enabled" is not supported for this model. Use "thinking.type.adaptive"
|
|
12
|
-
# and "output_config.effort"`, and with_thinking(true) sends no thinking at all. They take
|
|
13
|
-
# `{thinking: {type: "adaptive"}, output_config: {effort: "low"}}`, the rule upstream's own
|
|
14
|
-
# Anthropic protocol already applies (Protocols::Anthropic::Chat#thinking_mode).
|
|
15
|
-
#
|
|
16
|
-
# For an adaptive-only Claude target with no budget set:
|
|
17
|
-
# - effort other than "none": adaptive thinking, effort in output_config (every advertised
|
|
18
|
-
# tier, xhigh and max included);
|
|
19
|
-
# - no effort (with_thinking(true), or a display alone): adaptive thinking alone;
|
|
20
|
-
# - effort "none": nothing.
|
|
21
|
-
# No `display` is sent (Converse has no field for it). Everything else goes to super: any
|
|
22
|
-
# budget, thinking turned off (reasoning_config disabled), budget-style Claude (Haiku 4.5,
|
|
23
|
-
# Sonnet 4.6), GPT, Nova and other vendors.
|
|
24
|
-
#
|
|
25
|
-
# The target is Anthropic under any region prefix (ConverseForeignReasoning::ANTHROPIC).
|
|
26
|
-
# A model with no reasoning options of its own (an unregistered id, such as an "in."
|
|
27
|
-
# inference profile) takes them from a Bedrock registry entry for the same foundation model.
|
|
28
|
-
module ConverseClaudeAdaptiveThinking
|
|
29
|
-
ANTHROPIC = ConverseForeignReasoning::ANTHROPIC
|
|
30
|
-
REGION = /\A[a-z0-9-]+\.(?=anthropic\.)/
|
|
31
|
-
|
|
32
|
-
private
|
|
33
|
-
|
|
34
|
-
def format_reasoning_fields(thinking, model, max_output_tokens = nil)
|
|
35
|
-
return super unless thinking&.enabled? && thinking.enabled != false && thinking.budget.nil?
|
|
36
|
-
return super unless llm_logs_adaptive_only_claude?(model)
|
|
37
|
-
|
|
38
|
-
effort = thinking.effort.to_s
|
|
39
|
-
return nil if effort == "none"
|
|
40
|
-
return {thinking: {type: "adaptive"}} if effort.empty?
|
|
41
|
-
|
|
42
|
-
{thinking: {type: "adaptive"}, output_config: {effort: effort}}
|
|
43
|
-
end
|
|
44
|
-
|
|
45
|
-
def llm_logs_adaptive_only_claude?(model)
|
|
46
|
-
id = foundation_model_id(model&.id)
|
|
47
|
-
return false unless ANTHROPIC.match?(id)
|
|
48
|
-
|
|
49
|
-
options = llm_logs_reasoning_options(model, id.sub(REGION, ""))
|
|
50
|
-
options.any? { |o| o[:type] == "effort" } && options.none? { |o| o[:type] == "budget_tokens" }
|
|
51
|
-
end
|
|
52
|
-
|
|
53
|
-
def llm_logs_reasoning_options(model, bare_id)
|
|
54
|
-
return model.reasoning_options if model.reasoning_options.any?
|
|
55
|
-
|
|
56
|
-
sibling = RubyLLM.models.all.find do |candidate|
|
|
57
|
-
candidate.provider == "bedrock" && candidate.reasoning_options.any? &&
|
|
58
|
-
foundation_model_id(candidate.id).sub(REGION, "") == bare_id
|
|
59
|
-
end
|
|
60
|
-
sibling ? sibling.reasoning_options : []
|
|
61
|
-
end
|
|
62
|
-
|
|
63
|
-
def self.install!
|
|
64
|
-
RubyLLMPatches.prepend_checked(
|
|
65
|
-
RubyLLM::Protocols::Converse, self, %i[format_reasoning_fields foundation_model_id]
|
|
66
|
-
)
|
|
67
|
-
end
|
|
68
|
-
end
|
|
69
|
-
end
|
|
70
|
-
end
|
|
@@ -1,80 +0,0 @@
|
|
|
1
|
-
module LlmLogs
|
|
2
|
-
module RubyLLMPatches
|
|
3
|
-
# Bedrock Converse replays a message's reasoning only to a model of the same family.
|
|
4
|
-
# Backport of crmne/ruby_llm#1026, stricter than it on purpose (see below).
|
|
5
|
-
#
|
|
6
|
-
# ruby_llm 2.0.0 (Protocols::Converse::Chat#format_thinking_blocks) replays every assistant
|
|
7
|
-
# message's reasoning (raw_reasoning["converse"] blocks, else thinking text/signature) as
|
|
8
|
-
# reasoningContent whatever the target model. Upstream only strips thinking a *different
|
|
9
|
-
# provider* produced (Protocol#foreign_thinking?), and Claude and GPT are both "bedrock".
|
|
10
|
-
# So a chat Claude answered and GPT continues fails with 400 "This model doesn't support
|
|
11
|
-
# the reasoningContent.reasoningText.text field for assistant message", and a chat GPT
|
|
12
|
-
# answered and Claude continues hands Claude GPT's encrypted reasoning (redactedContent,
|
|
13
|
-
# or a reasoningText with an empty text and GPT's blob as signature for rows migrated from
|
|
14
|
-
# 1.16), which Claude cannot validate.
|
|
15
|
-
#
|
|
16
|
-
# The family of a model id is "anthropic", "openai", or the vendor segment of any other
|
|
17
|
-
# Bedrock id ("amazon", "meta", ...), whatever region prefix it carries. The producer is
|
|
18
|
-
# RubyLLM::Message#model (the reply's model, or for a restored row the model of its last
|
|
19
|
-
# successful ruby_llm_usages entry).
|
|
20
|
-
#
|
|
21
|
-
# - Producer known and of another family: no reasoning is replayed.
|
|
22
|
-
# - Anthropic target (same family or producer unknown): super, unchanged.
|
|
23
|
-
# - Other named-vendor target (same family or producer unknown): only the redactedContent
|
|
24
|
-
# blocks in raw_reasoning["converse"] (how GPT keeps its own encrypted reasoning across
|
|
25
|
-
# tool-call rounds); reasoningText and the thinking text/signature fallback are dropped.
|
|
26
|
-
# - Target whose id names no model (application-inference-profile ARN): super, unchanged.
|
|
27
|
-
#
|
|
28
|
-
# An application inference profile id is opaque, and AWS allows dots in it
|
|
29
|
-
# (".../application-inference-profile/team.prod"), so it names no family as producer or
|
|
30
|
-
# target (backport of crmne/ruby_llm#1026, "Give application inference profiles no vendor").
|
|
31
|
-
# The rest of this module is stricter than #1026 on purpose: for a non-Anthropic target
|
|
32
|
-
# whose producer is unknown or of the same family, upstream replays all the reasoning, while
|
|
33
|
-
# this keeps only redactedContent.
|
|
34
|
-
module ConverseForeignReasoning
|
|
35
|
-
ANTHROPIC = /(?:\A|\.)anthropic\./
|
|
36
|
-
OPENAI = /(?:\A|\.)openai\./
|
|
37
|
-
VENDOR = /\A[a-z0-9-]+(?=\.)/
|
|
38
|
-
APPLICATION_INFERENCE_PROFILE = ":application-inference-profile/"
|
|
39
|
-
|
|
40
|
-
private
|
|
41
|
-
|
|
42
|
-
def format_thinking_blocks(msg)
|
|
43
|
-
target = llm_logs_model_family(model&.id)
|
|
44
|
-
return super if target.nil?
|
|
45
|
-
|
|
46
|
-
producer = llm_logs_model_family(msg.model) if msg.respond_to?(:model)
|
|
47
|
-
return [] if producer && producer != target
|
|
48
|
-
return super if target == "anthropic"
|
|
49
|
-
|
|
50
|
-
blocks = msg.raw_reasoning["converse"] || msg.raw_reasoning[:converse] if msg.raw_reasoning.is_a?(Hash)
|
|
51
|
-
kept = Array(blocks).select { |block| llm_logs_redacted_only?(block) }
|
|
52
|
-
RubyLLM::Support::Utils.deep_dup(kept)
|
|
53
|
-
end
|
|
54
|
-
|
|
55
|
-
# "anthropic", "openai", another vendor segment, or nil when the id names no model.
|
|
56
|
-
def llm_logs_model_family(model_id)
|
|
57
|
-
return if model_id.to_s.include?(APPLICATION_INFERENCE_PROFILE)
|
|
58
|
-
|
|
59
|
-
id = foundation_model_id(model_id)
|
|
60
|
-
return "anthropic" if ANTHROPIC.match?(id)
|
|
61
|
-
return "openai" if OPENAI.match?(id)
|
|
62
|
-
|
|
63
|
-
id[VENDOR]
|
|
64
|
-
end
|
|
65
|
-
|
|
66
|
-
def llm_logs_redacted_only?(block)
|
|
67
|
-
content = block["reasoningContent"] || block[:reasoningContent] if block.is_a?(Hash)
|
|
68
|
-
return false unless content.is_a?(Hash)
|
|
69
|
-
|
|
70
|
-
content.keys.map(&:to_s) == ["redactedContent"]
|
|
71
|
-
end
|
|
72
|
-
|
|
73
|
-
def self.install!
|
|
74
|
-
RubyLLMPatches.prepend_checked(
|
|
75
|
-
RubyLLM::Protocols::Converse, self, %i[format_thinking_blocks foundation_model_id model]
|
|
76
|
-
)
|
|
77
|
-
end
|
|
78
|
-
end
|
|
79
|
-
end
|
|
80
|
-
end
|
|
@@ -1,78 +0,0 @@
|
|
|
1
|
-
require "json"
|
|
2
|
-
|
|
3
|
-
module LlmLogs
|
|
4
|
-
module RubyLLMPatches
|
|
5
|
-
# Reasoning effort as the reasoning_config value on Bedrock Converse.
|
|
6
|
-
# Backport of crmne/ruby_llm#1025 ("Send reasoning_config to models that publish it").
|
|
7
|
-
#
|
|
8
|
-
# ruby_llm 2.0.0 (Protocols::Converse::Chat#format_reasoning_fields) sends the effort of a
|
|
9
|
-
# model without a reasoning-budget schema as `additionalModelRequestFields:
|
|
10
|
-
# {reasoning_effort: "low"}`, which Bedrock rejects for OpenAI GPT models (400
|
|
11
|
-
# unknown_parameter). Their Converse additionalRequestFieldsSchema publishes
|
|
12
|
-
# `{"reasoning_config": {"type": "enum", "enum": ["none", "low", ...]}}`, and they take the
|
|
13
|
-
# effort as that value: `{reasoning_config: "low"}` ("none" turns reasoning off).
|
|
14
|
-
#
|
|
15
|
-
# Upstream rule, applied as #1025 does: when the model's schema, or the schema of another
|
|
16
|
-
# Bedrock registry entry for the same foundation model, publishes that enum, an effort that
|
|
17
|
-
# would have become reasoning_effort becomes the reasoning_config value, "none" included.
|
|
18
|
-
# Thinking off (reasoning_config disabled), budgets (explicit, or from a budget schema),
|
|
19
|
-
# Nova (reasoningConfig) and an empty effort are unchanged.
|
|
20
|
-
#
|
|
21
|
-
# Fallback (not in #1025): OpenAI GPT ids with no published schema get the same shape. ruby_llm
|
|
22
|
-
# 2.0.0's packaged registry lacks GPT-6 Sol/Luna, and an app catalog that registers them
|
|
23
|
-
# without Converse metadata leaves them schema-less. The match is prefix-agnostic (upstream
|
|
24
|
-
# REGION_PREFIXES lacks "in." in 2.0.0) and skips gpt-oss, which publishes no such enum.
|
|
25
|
-
module ConverseReasoningConfig
|
|
26
|
-
OPENAI_GPT = /(?:\A|\.)openai\.gpt-(?!oss)/
|
|
27
|
-
|
|
28
|
-
private
|
|
29
|
-
|
|
30
|
-
def format_reasoning_fields(thinking, model, max_output_tokens = nil)
|
|
31
|
-
return super unless thinking&.enabled? && thinking.enabled != false
|
|
32
|
-
return super unless llm_logs_reasoning_config?(model)
|
|
33
|
-
|
|
34
|
-
effort = thinking.effort.to_s
|
|
35
|
-
return super if effort.empty? || nova_model?(model)
|
|
36
|
-
return super if reasoning_budget(thinking, effort, model, max_output_tokens)
|
|
37
|
-
|
|
38
|
-
{reasoning_config: effort}
|
|
39
|
-
end
|
|
40
|
-
|
|
41
|
-
def llm_logs_reasoning_config?(model)
|
|
42
|
-
return false unless model
|
|
43
|
-
|
|
44
|
-
llm_logs_reasoning_config_schema?(model) || OPENAI_GPT.match?(foundation_model_id(model.id))
|
|
45
|
-
end
|
|
46
|
-
|
|
47
|
-
# Bedrock publishes Converse metadata for only some regional entries of a model.
|
|
48
|
-
def llm_logs_reasoning_config_schema?(model)
|
|
49
|
-
return true if llm_logs_publishes_reasoning_config?(model)
|
|
50
|
-
|
|
51
|
-
foundation_id = foundation_model_id(model.id)
|
|
52
|
-
RubyLLM.models.all.any? do |candidate|
|
|
53
|
-
candidate.provider == "bedrock" && candidate.id != model.id &&
|
|
54
|
-
foundation_model_id(candidate.id) == foundation_id && llm_logs_publishes_reasoning_config?(candidate)
|
|
55
|
-
end
|
|
56
|
-
end
|
|
57
|
-
|
|
58
|
-
def llm_logs_publishes_reasoning_config?(model)
|
|
59
|
-
metadata = RubyLLM::Support::Utils.deep_symbolize_keys(model.metadata || {})
|
|
60
|
-
raw_schema = metadata.dig(:converse, :additionalRequestFieldsSchema)
|
|
61
|
-
return false unless raw_schema.is_a?(String)
|
|
62
|
-
|
|
63
|
-
schema = JSON.parse(raw_schema, symbolize_names: true)
|
|
64
|
-
config = schema.is_a?(Hash) ? schema[:reasoning_config] : nil
|
|
65
|
-
config.is_a?(Hash) && config[:type] == "enum"
|
|
66
|
-
rescue JSON::ParserError
|
|
67
|
-
false
|
|
68
|
-
end
|
|
69
|
-
|
|
70
|
-
def self.install!
|
|
71
|
-
RubyLLMPatches.prepend_checked(
|
|
72
|
-
RubyLLM::Protocols::Converse, self,
|
|
73
|
-
%i[format_reasoning_fields foundation_model_id nova_model? reasoning_budget]
|
|
74
|
-
)
|
|
75
|
-
end
|
|
76
|
-
end
|
|
77
|
-
end
|
|
78
|
-
end
|
|
@@ -1,25 +0,0 @@
|
|
|
1
|
-
require "faraday"
|
|
2
|
-
|
|
3
|
-
module LlmLogs
|
|
4
|
-
module RubyLLMPatches
|
|
5
|
-
# Backport of crmne/ruby_llm#1024 ("Retry requests that fail with a TLS error").
|
|
6
|
-
# Retries TLS handshake failures (Faraday::SSLError, e.g. "SSL_connect ... unexpected
|
|
7
|
-
# eof"). faraday-net_http wraps every OpenSSL::SSL::SSLError as Faraday::SSLError, including
|
|
8
|
-
# read errors after the request was written, so this is the same trade-off ruby_llm
|
|
9
|
-
# already accepts for read timeouts. ruby_llm's retry_if (transport/connection.rb) still
|
|
10
|
-
# refuses non-idempotent requests and streams that already delivered content.
|
|
11
|
-
# Faraday::SSLError is not a Faraday::ConnectionFailed, so 2.0.0's retry list misses it.
|
|
12
|
-
module RetrySSLError
|
|
13
|
-
private
|
|
14
|
-
|
|
15
|
-
def retry_exceptions
|
|
16
|
-
exceptions = super
|
|
17
|
-
exceptions.include?(Faraday::SSLError) ? exceptions : exceptions + [Faraday::SSLError]
|
|
18
|
-
end
|
|
19
|
-
|
|
20
|
-
def self.install!
|
|
21
|
-
RubyLLMPatches.prepend_checked(RubyLLM::Transport::Connection, self, %i[retry_exceptions])
|
|
22
|
-
end
|
|
23
|
-
end
|
|
24
|
-
end
|
|
25
|
-
end
|
|
@@ -1,54 +0,0 @@
|
|
|
1
|
-
require "llm_logs/ruby_llm_patches/bedrock_sigv4"
|
|
2
|
-
require "llm_logs/ruby_llm_patches/converse_reasoning_config"
|
|
3
|
-
require "llm_logs/ruby_llm_patches/converse_foreign_reasoning"
|
|
4
|
-
require "llm_logs/ruby_llm_patches/converse_claude_adaptive_thinking"
|
|
5
|
-
require "llm_logs/ruby_llm_patches/retry_ssl_error"
|
|
6
|
-
|
|
7
|
-
module LlmLogs
|
|
8
|
-
# Bedrock fixes that upstream ruby_llm 2.0.0 lacks, applied with Module#prepend. Each
|
|
9
|
-
# backports an open upstream PR (crmne/ruby_llm#1024, #1025, #1026; see the README), so it
|
|
10
|
-
# can be dropped once a ruby_llm release includes that PR.
|
|
11
|
-
# Each was verified against 2.0.x only: on 1.x they are skipped (different internals),
|
|
12
|
-
# on a newer ruby_llm they still install but log a warning so we re-check whether
|
|
13
|
-
# upstream fixed the bug (then drop the patch) or moved the method (then the patch
|
|
14
|
-
# skips itself and logs why).
|
|
15
|
-
module RubyLLMPatches
|
|
16
|
-
PATCHES = [
|
|
17
|
-
BedrockSigV4, ConverseReasoningConfig, ConverseForeignReasoning, ConverseClaudeAdaptiveThinking, RetrySSLError
|
|
18
|
-
].freeze
|
|
19
|
-
MINIMUM = Gem::Version.new("2.0.0")
|
|
20
|
-
VERIFIED = Gem::Requirement.new(">= 2.0.0", "< 2.1")
|
|
21
|
-
|
|
22
|
-
module_function
|
|
23
|
-
|
|
24
|
-
def install!
|
|
25
|
-
version = Gem::Version.new(RubyLLM::VERSION)
|
|
26
|
-
return [] if version < MINIMUM
|
|
27
|
-
|
|
28
|
-
unless VERIFIED.satisfied_by?(version)
|
|
29
|
-
warn_log("verified against ruby_llm #{VERIFIED}, running #{version}: check whether upstream " \
|
|
30
|
-
"now fixes #{PATCHES.map(&:name).join(', ')} and drop the patches it does")
|
|
31
|
-
end
|
|
32
|
-
PATCHES.select(&:install!)
|
|
33
|
-
end
|
|
34
|
-
|
|
35
|
-
def warn_log(message)
|
|
36
|
-
logger = defined?(Rails) && Rails.respond_to?(:logger) && Rails.logger
|
|
37
|
-
logger ? logger.warn("[llm_logs] #{message}") : Kernel.warn("[llm_logs] #{message}")
|
|
38
|
-
end
|
|
39
|
-
|
|
40
|
-
# Prepends +patch+ to +target+ when +target+ still defines every method in +methods+.
|
|
41
|
-
def prepend_checked(target, patch, methods)
|
|
42
|
-
return true if target <= patch
|
|
43
|
-
|
|
44
|
-
missing = methods.reject { |name| target.method_defined?(name) || target.private_method_defined?(name) }
|
|
45
|
-
if missing.any?
|
|
46
|
-
warn_log("#{patch.name} not installed: #{target} no longer defines #{missing.join(', ')}")
|
|
47
|
-
return false
|
|
48
|
-
end
|
|
49
|
-
|
|
50
|
-
target.prepend(patch)
|
|
51
|
-
true
|
|
52
|
-
end
|
|
53
|
-
end
|
|
54
|
-
end
|