llm_gateway 0.4.0 → 0.6.0

data/docs/migration_guide_0.6.0.md

@@ -0,0 +1,386 @@
+# Migration Guide: 0.5.0 to 0.6.0
+
+This guide covers user-facing changes between `v0.5.0` and the latest commit on the 0.6.0 branch.
+
+## Summary
+
+0.6.0 separates provider authentication/configuration from model selection.
+
+- Provider config now contains only provider/auth settings such as `provider`, `api_key`, `access_token`, and `account_id`.
+- `model_key` is no longer accepted in provider/client configuration.
+- Pass the model per request with `model:` when calling `chat`, `stream`, Responses/Codex methods, or embeddings.
+- Legacy provider keys such as `openai_apikey_responses` were removed. Use the shorter provider keys.
+- `LlmGateway::Prompt` now accepts/configures a provider and model separately, and uses `stream` internally.
+- The `client.model_key` reader was removed; track the selected model at the call site or read it from returned messages.
+- Streaming events now expose accumulated partial messages during the stream, while `:message_end` exposes the final message through `event.message`.
+- Non-final stream event hashes now include `partial`; normal stream consumers are unaffected, but strict `event.to_h` snapshots/comparisons may need updates.
+- Normalized usage counters were renamed to concise keys: `:input`, `:cache_write`, `:cache_read`, `:output`, and `:total`; `:reasoning_tokens` was removed.
+- Streamed assistant messages now include `timestamp` as Unix milliseconds.
+- Custom stream mappers must initialize with provider/API metadata and emit a final `:message_end` patch.
+
+## 1. Replace legacy provider keys
+
+0.6.0 removes the backward-compatible legacy provider registry entries.
+
+| 0.5.0 provider key | 0.6.0 provider key |
+|---|---|
+| `anthropic_apikey_messages` | `anthropic_messages` |
+| `anthropic_oauth_messages` | `anthropic_messages` |
+| `openai_apikey_completions` | `openai_completions` |
+| `openai_apikey_responses` | `openai_responses` |
+| `openai_oauth_codex` | `openai_codex` |
+| `groq_apikey_completions` | `groq_completions` |
+
+### Before
+
+```ruby
+adapter = LlmGateway.build_provider(
+  provider: "openai_apikey_responses",
+  api_key: ENV.fetch("OPENAI_API_KEY"),
+  model_key: "gpt-5.4"
+)
+```
+
+### After
+
+```ruby
+adapter = LlmGateway.build_provider(
+  provider: "openai_responses",
+  api_key: ENV.fetch("OPENAI_API_KEY")
+)
+```
+
+## 2. Move `model_key` from provider config to request calls
+
+`model_key` is no longer a provider option. Passing it to `LlmGateway.build_provider` raises:
+
+```text
+ArgumentError: model_key is no longer a provider option; pass model: to chat/stream instead
+```
+
+Pass `model:` on each request instead.
+
+### Streaming
+
+```ruby
+# Before
+adapter = LlmGateway.build_provider(
+  provider: "openai_apikey_responses",
+  api_key: ENV.fetch("OPENAI_API_KEY"),
+  model_key: "gpt-5.4"
+)
+result = adapter.stream("Write one short sentence about Ruby.")
+
+# After
+adapter = LlmGateway.build_provider(
+  provider: "openai_responses",
+  api_key: ENV.fetch("OPENAI_API_KEY")
+)
+result = adapter.stream("Write one short sentence about Ruby.", model: "gpt-5.4")
+```
+
+### Configure arrays
+
+```ruby
+# Before
+LlmGateway.configure([
+  {
+    name: "primary",
+    config: {
+      provider: "groq_apikey_completions",
+      api_key: ENV.fetch("GROQ_API_KEY"),
+      model_key: "openai/gpt-oss-120b"
+    }
+  }
+])
+
+# After
+LlmGateway.configure([
+  {
+    name: "primary",
+    config: {
+      provider: "groq_completions",
+      api_key: ENV.fetch("GROQ_API_KEY")
+    }
+  }
+])
+
+LlmGateway.configured_clients.fetch("primary").stream(
+  "Hello",
+  model: "openai/gpt-oss-120b"
+)
+```
+
+## 3. Update direct client usage
+
+Direct clients no longer take `model_key:` in their constructors.
+
+```ruby
+# Before
+client = LlmGateway::Clients::OpenAI.new(
+  api_key: ENV.fetch("OPENAI_API_KEY"),
+  model_key: "gpt-5.4"
+)
+client.stream(messages)
+
+# After
+client = LlmGateway::Clients::OpenAI.new(
+  api_key: ENV.fetch("OPENAI_API_KEY")
+)
+client.stream(messages, model: "gpt-5.4")
+```
+
+The same pattern applies to:
+
+- `LlmGateway::Clients::Anthropic#chat` / `#stream`
+- `LlmGateway::Clients::OpenAI#chat` / `#stream` / `#responses` / `#stream_responses`
+- `LlmGateway::Clients::OpenAI#chat_codex` / `#stream_codex`
+- `LlmGateway::Clients::Groq#chat` / `#stream`
+
+Embeddings also take a per-call model:
+
+```ruby
+client.generate_embeddings(input, model: "text-embedding-3-large")
+```
+
+If omitted, clients still provide default models.
+
+## 4. Update `LlmGateway::Prompt` classes
+
+`Prompt` no longer looks up a configured client by comparing a string to `client.model_key`. It now keeps the provider and model as separate values.
+
+If you previously called `Prompt.new("gpt-5.4")`, update that code. The first initializer argument is now a provider adapter, not a model lookup key. Configure a provider on the class or pass one to the initializer.
+
+### Class-level configuration
+
+```ruby
+class SummaryPrompt < LlmGateway::Prompt
+  self.provider = LlmGateway.build_provider(
+    provider: "openai_responses",
+    api_key: ENV.fetch("OPENAI_API_KEY")
+  )
+  self.model = "gpt-5.4"
+
+  def prompt
+    "Summarize this text."
+  end
+end
+
+SummaryPrompt.new.run
+```
+
+
+### Instance-level configuration
+
+```ruby
+provider = LlmGateway.build_provider(
+  provider: "anthropic_messages",
+  api_key: ENV.fetch("ANTHROPIC_API_KEY")
+)
+
+SummaryPrompt.new(provider, "claude-sonnet-4-20250514").run
+```
+
+### Per-call overrides
+
+```ruby
+prompt = SummaryPrompt.new(default_provider, "gpt-5.1")
+
+prompt.stream(
+  provider: other_provider,
+  model: "gpt-5.4",
+  reasoning: "high"
+)
+```
+
+If you subclassed `Prompt` and called or overrode `post`, migrate that code to `stream`. `run` now calls `stream` internally.
+
+## 5. Stop using `client.model_key`
+
+Direct clients no longer expose a `model_key` reader because model selection is no longer client/provider state.
+
+```ruby
+# Before
+client = LlmGateway::Clients::OpenAI.new(
+  api_key: ENV.fetch("OPENAI_API_KEY"),
+  model_key: "gpt-5.4"
+)
+puts client.model_key
+
+# After
+client = LlmGateway::Clients::OpenAI.new(
+  api_key: ENV.fetch("OPENAI_API_KEY")
+)
+model = "gpt-5.4"
+result = client.stream(messages, model: model)
+# Track `model` at the call site when you need it later.
+```
+
+## 6. OAuth provider names
+
+OAuth is now represented by credentials, not by separate legacy provider keys.
+
+```ruby
+# Before
+adapter = LlmGateway.build_provider(
+  provider: "openai_oauth_codex",
+  access_token: current_access_token,
+  model_key: "gpt-5.4"
+)
+
+# After
+adapter = LlmGateway.build_provider(
+  provider: "openai_codex",
+  access_token: current_access_token
+)
+
+adapter.stream("Hello from OAuth auth", model: "gpt-5.4")
+```
+
+For Anthropic OAuth, use `provider: "anthropic_messages"` with an `access_token`.
+
+## 7. Update stream callback handling
+
+The final `:message_end` stream callback event changed shape.
+
+In 0.5.x, `:message_end` was an `AssistantStreamMessageEvent` and exposed the accumulated message through `event.partial`.
+
+In 0.6.0, `:message_end` is an `AssistantStreamMessageEndEvent` and exposes the final complete `AssistantMessage` through `event.message`. It does not expose `partial`.
+
+```ruby
+response = adapter.stream("Hello", model: "gpt-5.4") do |event|
+  case event.type
+  when :text_delta
+    print event.delta
+  when :message_end
+    final_message = event.message
+    puts final_message.provider
+    puts final_message.api
+  end
+end
+
+# The stream return value is the same final AssistantMessage.
+response # => AssistantMessage
+```
+
+If you previously handled every event as if it had `partial`, branch on `event.type == :message_end` first or check `respond_to?(:partial)`.
+
+```ruby
+adapter.stream("Hello", model: "gpt-5.4") do |event|
+  if event.type == :message_end
+    persist(event.message.to_h)
+  elsif event.respond_to?(:partial)
+    update_ui(event.partial)
+  end
+end
+```
+
+## 8. Update usage accounting keys
+
+Normalized `AssistantMessage#usage` and final stream `event.usage` patches now use provider-independent concise keys plus `:raw` for the original provider usage/token payload:
+
+| 0.5.x key | 0.6.0 key |
+|---|---|
+| `:input_tokens` | `:input` |
+| `:cache_creation_input_tokens` | `:cache_write` |
+| `:cache_read_input_tokens` | `:cache_read` |
+| `:output_tokens` | `:output` |
+| computed normalized total | `:total` |
+| original provider usage payload | `:raw` |
+| `:reasoning_tokens` | removed |
+
+`reasoning_tokens` was removed because providers expose and calculate reasoning token counts inconsistently. Use the streamed/final `ReasoningContent` blocks for reasoning text, and treat usage as the normalized token buckets above.
+
+```ruby
+# Before
+result.usage[:input_tokens]
+result.usage[:cache_read_input_tokens]
+result.usage[:output_tokens]
+
+# After
+result.usage[:input]
+result.usage[:cache_read]
+result.usage[:output]
+```
+
+When checking cache behavior, use `usage[:cache_read]` and `usage[:cache_write]`. `usage[:total]` is computed as `input + cache_write + cache_read + output`. Use `usage[:raw]` when you need provider-specific token fields that are not part of the normalized counters.
+
+## 9. Account for timestamps on streamed messages
+
+`PartialAssistantMessage` and `AssistantMessage` now include a `timestamp` field in Unix milliseconds. Provider-supplied timestamps are preserved when available; otherwise the accumulator assigns one.
+
+```ruby
+response = adapter.stream("Hello", model: "gpt-5.4") do |event|
+  puts event.partial.timestamp if event.respond_to?(:partial)
+end
+
+puts response.timestamp
+puts response.to_h[:timestamp]
+```
+
+If you instantiate `PartialAssistantMessage` or `AssistantMessage` directly in tests or custom integrations, include `timestamp:`.
+
+## 10. Update custom stream mappers
+
+If you implemented a custom adapter or stream mapper, update it for the new final-message flow.
+
+`LlmGateway::Adapters::StreamMapper` now requires provider/API metadata:
+
+```ruby
+mapper = MyStreamMapper.new(provider: "openai", api: "responses")
+```
+
+`Adapter#stream` passes these values automatically when it instantiates the configured mapper, but direct mapper construction and custom initializers must accept/pass these keywords.
+
+Custom mappers must also push a final normalized end patch. Use the normalized usage keys shown above for final `usage`.
+
+```ruby
+push_patches([
+  { type: :message_delta, delta: { stop_reason: "stop" }, usage: { output: 12 } },
+  { type: :message_end }
+], &block)
+```
+
+`StreamMapper#result` now returns the final `AssistantMessage` created by the `:message_end` patch. If a custom mapper never emits `:message_end`, `adapter.stream` will not have a final message to return.
+
+## 11. Cross-provider handoff note
+
+Message sanitization for cross-provider/model handoffs now receives the target model from the request options. When replaying or handing off transcripts across providers/models, pass `model:` explicitly on the destination call so model-specific sanitizer behavior can run.
+
+```ruby
+next_response = target_adapter.stream(
+  transcript_from_another_provider,
+  model: "gpt-5.4"
+)
+```
+
+## 12. Stream event hash snapshots
+
+Non-final stream events now expose a `partial` assistant message, so `event.to_h` includes an additional `partial` field.
+
+This is additive for normal stream callback consumers:
+
+```ruby
+adapter.stream("Hello", model: "gpt-5.4") do |event|
+  puts event.type
+  puts event.delta if event.respond_to?(:delta)
+end
+```
+
+If your tests or application code compare full `event.to_h` hashes or snapshot serialized events, update those expectations to include or ignore `partial`.
+
+## Checklist
+
+- [ ] Replace all legacy provider keys with the new provider keys.
+- [ ] Remove `model_key:` from `build_provider`, `configure`, and direct client constructors.
+- [ ] Remove any direct reads of `client.model_key` / `adapter.client.model_key`.
+- [ ] Add `model:` to `chat`, `stream`, Responses/Codex, and embeddings calls where you need a specific model.
+- [ ] Update `Prompt` subclasses to configure `provider` and `model` separately.
+- [ ] Replace `Prompt.new("model-key")` model lookup usage with explicit provider/model configuration.
+- [ ] Replace custom `Prompt#post` usage with `Prompt#stream`.
+- [ ] Update stream callbacks to read `event.message` for `:message_end` and `event.partial` only for non-final events.
+- [ ] Rename normalized usage lookups to `:input`, `:cache_write`, `:cache_read`, `:output`, and `:total`; use `:raw` for provider-specific token fields; remove `:reasoning_tokens` handling.
+- [ ] Include/read `timestamp` on streamed partial and final assistant messages where you construct or persist those objects.
+- [ ] Update custom stream mappers to accept `provider:` / `api:`, emit normalized usage keys, and emit `{ type: :message_end }`.
+- [ ] For cross-provider handoffs, pass the target `model:` explicitly.
+- [ ] Update strict `event.to_h` stream event snapshots/comparisons for the new `partial` field.

data/lib/llm_gateway/adapters/adapter.rb

@@ -1,6 +1,5 @@
 # frozen_string_literal: true
 
-require_relative "stream_accumulator"
 require_relative "structs"
 
 module LlmGateway
@@ -12,34 +11,19 @@ module LlmGateway
         @client = client
       end
 
-      def chat(message, tools: nil, system: nil, **options)
-        normalized_input = map_input({
-          messages: sanitize_messages(normalize_messages(message)),
-          tools: tools,
-          system: normalize_system(system)
-        })
-
-        result = perform_chat(
-          normalized_input[:messages],
-          tools: normalized_input[:tools],
-          system: normalized_input[:system],
-          **map_options(options)
-        )
-
-        map_output(result)
-      end
-
       def stream(message, tools: nil, system: nil, **options, &block)
         raise LlmGateway::Errors::MissingMapperForProvider, "No stream_mapper configured" unless stream_mapper
 
         normalized_input = map_input({
-          messages: sanitize_messages(normalize_messages(message)),
+          messages: sanitize_messages(normalize_messages(message), target_model: options[:model]),
           tools: tools,
           system: normalize_system(system)
         })
 
-        accumulator = ::StreamAccumulator.new
-        mapper = stream_mapper.new
+        mapper = stream_mapper.new(
+          provider: LlmGateway::Client.provider_id_from_client(client),
+          api: api_name
+        )
 
         perform_stream(
           normalized_input[:messages],
@@ -47,17 +31,10 @@ module LlmGateway
           system: normalized_input[:system],
           **map_options(options)
         ) do |chunk|
-          event = mapper.map(chunk)
-          accumulator.push(event)
-          block.call(event) if block && event
+          mapper.map(chunk, &block)
         end
 
-        AssistantMessage.new(
-          accumulator.result.merge(
-            provider: LlmGateway::Client.provider_id_from_client(client),
-            api: api_name
-          )
-        )
+        mapper.result
       end
 
       def upload_file(filename:, content:, mime_type: "application/octet-stream", purpose: "assistants")
@@ -92,10 +69,6 @@ module LlmGateway
         nil
       end
 
-      def output_mapper
-        raise NotImplementedError, "#{self.class} must implement #output_mapper"
-      end
-
       def file_output_mapper
         nil
       end
@@ -108,18 +81,10 @@ module LlmGateway
         input_mapper.map(input)
       end
 
-      def map_output(output)
-        output_mapper.map(output)
-      end
-
       def map_options(options)
         option_mapper.map(options)
       end
 
-      def perform_chat(messages, tools:, system:, **options)
-        client.chat(messages, tools: tools, system: system, **options)
-      end
-
       def perform_stream(messages, tools:, system:, **options, &block)
         client.stream(messages, tools: tools, system: system, **options, &block)
       end
@@ -132,12 +97,11 @@ module LlmGateway
         nil
       end
 
-      def sanitize_messages(messages)
+      def sanitize_messages(messages, target_model: nil)
         return messages unless input_sanitizer
 
         target_provider = LlmGateway::Client.provider_id_from_client(client)
         target_api = api_name
-        target_model = client.model_key
 
         return messages if target_provider.nil? || target_api.nil? || target_model.nil?
 

data/lib/llm_gateway/adapters/anthropic/acts_like_messages.rb

@@ -11,8 +11,6 @@ module LlmGateway
 
       def input_sanitizer = InputMessageSanitizer
 
-      def output_mapper = Anthropic::OutputMapper
-
       def file_output_mapper = Anthropic::FileOutputMapper
 
       def option_mapper = AnthropicOptionMapper

data/lib/llm_gateway/adapters/anthropic/input_mapper.rb

@@ -1,7 +1,5 @@
 # frozen_string_literal: true
 
-require_relative "bidirectional_message_mapper"
-
 module LlmGateway
   module Adapters
     module Anthropic
@@ -14,42 +12,123 @@ module LlmGateway
           }
         end
 
-        private
+        def self.map_content(content)
+          content = { type: "text", text: content } unless content.is_a?(Hash)
+
+          case content[:type]
+          when "text"
+            map_text_content(content)
+          when "file"
+            map_file_content(content)
+          when "image"
+            map_image_content(content)
+          when "tool_use"
+            map_tool_use_content(content)
+          when "tool_result"
+            map_tool_result_content(content)
+          when "thinking", "reasoning"
+            map_reasoning_content(content)
+          else
+            content
+          end
+        end
+
+        class << self
+          private
 
-        def self.map_messages(messages)
-          return messages unless messages
+          def map_messages(messages)
+            return messages unless messages
 
-          message_mapper = BidirectionalMessageMapper.new(LlmGateway::DIRECTION_IN)
+            messages.map do |msg|
+              msg = msg.merge(role: "user") if msg[:role] == "developer"
 
-          messages.map do |msg|
-            msg = msg.merge(role: "user") if msg[:role] == "developer"
+              content = if msg[:content].is_a?(Array)
+                msg[:content].map { |content| map_content(content) }
+              else
+                [ map_content(msg[:content]) ]
+              end
 
-            content = if msg[:content].is_a?(Array)
-                msg[:content].map do |content|
-                  message_mapper.map_content(content)
-                end
+              {
+                role: msg[:role],
+                content: content
+              }
+            end
+          end
+
+          def map_system(system)
+            if !system || system.empty?
+              nil
+            elsif system.length == 1 && system.first[:role] == "system"
+              mapped = { type: "text", text: system.first[:content] }
+              mapped[:cache_control] = system.first[:cache_control] if system.first[:cache_control]
+              [ mapped ]
             else
-              [ message_mapper.map_content(msg[:content]) ]
+              system
             end
+          end
 
+          def map_text_content(content)
+            result = {
+              type: "text",
+              text: content[:text]
+            }
+            result[:cache_control] = content[:cache_control] if content[:cache_control]
+            result
+          end
+
+          def map_file_content(content)
             {
-              role: msg[:role],
-              content: content
+              type: "document",
+              source: {
+                data: content[:data],
+                type: "text",
+                media_type: content[:media_type]
+              }
             }
           end
-        end
 
-        def self.map_system(system)
-          if !system || system.empty?
-            nil
-          elsif system.length == 1 && system.first[:role] == "system"
-            # If we have a single system message, convert to Claude format
-            mapped = { type: "text", text: system.first[:content] }
-            mapped[:cache_control] = system.first[:cache_control] if system.first[:cache_control]
-            [ mapped ]
-          else
-            # For multiple messages or non-standard format, pass through
-            system
+          def map_image_content(content)
+            {
+              type: "image",
+              source: {
+                data: content[:data],
+                type: "base64",
+                media_type: content[:media_type]
+              }
+            }
+          end
+
+          def map_tool_use_content(content)
+            {
+              type: "tool_use",
+              id: content[:id],
+              name: content[:name],
+              input: content[:input]
+            }
+          end
+
+          def map_tool_result_content(content)
+            mapped_content = content[:content]
+            if mapped_content.is_a?(Array)
+              mapped_content = mapped_content.map do |item|
+                item.is_a?(Hash) ? map_content(item.transform_keys(&:to_sym)) : item
+              end
+            end
+
+            {
+              type: "tool_result",
+              tool_use_id: content[:tool_use_id],
+              content: mapped_content
+            }
+          end
+
+          def map_reasoning_content(content)
+            result = {
+              type: "thinking",
+              thinking: content[:reasoning]
+            }
+            result[:signature] = content[:signature] unless content[:signature].nil?
+            result
           end
         end
       end

data/lib/llm_gateway/adapters/anthropic/output_mapper.rb

@@ -12,39 +12,6 @@ module LlmGateway
           )
         end
       end
-
-      class OutputMapper
-        def self.map(data)
-          {
-            id: data[:id],
-            model: data[:model],
-            usage: data[:usage],
-            choices: map_choices(data)
-          }
-        end
-
-        private
-
-        def self.map_choices(data)
-          message_mapper = BidirectionalMessageMapper.new(LlmGateway::DIRECTION_OUT)
-
-          content = if data[:content].is_a?(Array)
-              data[:content].map do |content|
-                message_mapper.map_content(content)
-              end
-          else
-            data[:content] ? [ message_mapper.map_content(data[:content]) ] : []
-          end
-
-          # Claude returns content directly at root level, not in a choices array
-          # We need to construct the choices array from the full response data
-          [ {
-            content: content, # Use content directly from Claude response
-            finish_reason: data[:stop_reason],
-            role: "assistant"
-          } ]
-        end
-      end
     end
   end
 end