llm_gateway 0.2.0 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 829e306ff0af794ce1a301b1eac5ab52edfa9a21ce1eec479955de3a328be443
-  data.tar.gz: 6935b650d14237e48a82cdbb0614e5201c6960abf71d6dc23bd51e76d6b37ee3
+  metadata.gz: 972dac306c8d8f59b41c462f0133d0ce415e689515fbd84d5503541a9fee6f93
+  data.tar.gz: 6014254f858af1b83906b29950811e2ce84940f0ff49301a9b36dd19f592b7a8
 SHA512:
-  metadata.gz: 89572a5c0d05806fdc5fa7a06a7336f5b8b738daaff37f834595b96415a601adc50ca13e33c79cfa29abcb82766eeb6ad3cbe14c099b030e340af7d8dd943f2f
-  data.tar.gz: 0001f5413fe4b3c3b17b2bca197a37e00c0503b09da7e79b90b52a45c65a4247dfd8be60fa86cc99505eb408c4c69c3d2cd835eb700c8018faa1f64da6e0f6ab
+  metadata.gz: 388573aa9afe19a075c561f17d5e4a6b8e550de25b73c3e6d9583403096f1c26f13667055d5920408c2c06f9add888af81a0bf34434762b8ece4a58de32ee3c3
+  data.tar.gz: 42d0b37c296ee058109cb38daaeefc269ce128a39bfbb3b89745eb734653a2828ec8c54f8acd1a12242aa03d1706304507c627074a53e4e6cddc7eb27324d69d

data/CHANGELOG.md

@@ -1,5 +1,21 @@
 # Changelog
 
+## [v0.3.0](https://github.com/Hyper-Unearthing/llm_gateway/tree/v0.3.0) (2025-08-19)
+
+[Full Changelog](https://github.com/Hyper-Unearthing/llm_gateway/compare/v0.2.0...v0.3.0)
+
+**Merged pull requests:**
+
+- feat: create a method called responses to implement modern apis [\#30](https://github.com/Hyper-Unearthing/llm_gateway/pull/30) ([billybonks](https://github.com/billybonks))
+- test: Create a test that asserts the transcript format. [\#29](https://github.com/Hyper-Unearthing/llm_gateway/pull/29) ([billybonks](https://github.com/billybonks))
+- refactor: move open ai chat completions to its own folder [\#28](https://github.com/Hyper-Unearthing/llm_gateway/pull/28) ([billybonks](https://github.com/billybonks))
+- refactor: bundle all provider resources in a hash [\#27](https://github.com/Hyper-Unearthing/llm_gateway/pull/27) ([billybonks](https://github.com/billybonks))
+- refactor: message mapper to become bidirectional mapper [\#26](https://github.com/Hyper-Unearthing/llm_gateway/pull/26) ([billybonks](https://github.com/billybonks))
+- refactor: extract message mapper from input mapper [\#25](https://github.com/Hyper-Unearthing/llm_gateway/pull/25) ([billybonks](https://github.com/billybonks))
+- docs: add more information about how the library works [\#24](https://github.com/Hyper-Unearthing/llm_gateway/pull/24) ([billybonks](https://github.com/billybonks))
+- feat: enable uploading and downloading files from openai anthropic [\#23](https://github.com/Hyper-Unearthing/llm_gateway/pull/23) ([billybonks](https://github.com/billybonks))
+- ci: upload build version to github when we release [\#22](https://github.com/Hyper-Unearthing/llm_gateway/pull/22) ([billybonks](https://github.com/billybonks))
+
 ## [v0.2.0](https://github.com/Hyper-Unearthing/llm_gateway/tree/v0.2.0) (2025-08-08)
 
 [Full Changelog](https://github.com/Hyper-Unearthing/llm_gateway/compare/v0.1.6...v0.2.0)

data/README.md

@@ -1,8 +1,65 @@
 # LlmGateway
 
-Provide nuts and bolts for LLM APIs. The goal is to provide a unified interface for multiple LLM provider API's; And Enable developers to have as much control as they want.
+Provide a unified translation interface for LLM Provider API's, While allowing developers to have as much control as possible, This does make it more complicated because we dont want developers to be blocked at using something that the provider supports. As time progress the library will mature and support more responses
 
-You can use the clients directly, Or you can use the gateway to have interop between clients.
+
+## Principles:
+1. Transcription integrity is most important
+2. Input messages must have bidirectional integrity
+3. Allow developers as much control as possible
+
+## Assumptions
+things that do not support unidirectional format, probably cant be sent between providers
+
+## Mechanics
+Messages either support unidirectional or bidirectional format. (unidirectional means we can format it as an output but should not be added as an input).
+
+The result from the llm is in the format that can be sent to the provider, but if you want to consolidate complex messages like code_execution, you must run a mapper we provide manually, but dont send that format back to the provider.
+
+### bidirectional Support
+Messages
+- Text
+- Tool Use
+- Tool Response
+
+Tools
+- Server Tools
+- Tools
+
+### Unidirectional Support
+- Server Tool Use Reponse
+
+### Example flow
+
+
+```mermaid
+sequenceDiagram
+        actor developer
+        participant llm_gateway
+        participant llm_provider
+
+        developer ->> llm_gateway: Send Text Message
+        llm_gateway ->> llm_gateway: transform to provider format
+        llm_gateway ->> llm_provider: Transformed Text Message
+        llm_provider ->> llm_gateway: Response <br />(transcript in provider format)
+        llm_gateway ->> developer:  Response <br />(transcript in combination <br />of gatway and provider formats)
+        Note over llm_gateway,developer: llm_gateway will transform <br /> messages that support bi-direction
+        developer ->> developer: save the transcript
+        loop ProcessMessage
+            developer ->> llm_gateway: format message
+            llm_gateway ->> developer: return transformed message
+            Note over llm_gateway,developer: if the message: <br /> supports bidirection format returns as is <br /> otherwise will transform <br />into consolidated format
+            developer ->> developer: append earlier saved transcript
+            Note over developer, developer: for example tool use
+        end
+        developer -> llm_gateway: Transcript
+        llm_gateway ->> llm_gateway: transform to provider format
+        Note over llm_gateway,llm_gateway: non bidirectional messages are sent as is
+        llm_gateway ->> llm_provider: etc etc etc
+
+
+
+```
 
 ## Supported Providers
 Anthropic, OpenAi, Groq
@@ -116,7 +173,28 @@ it to the list of roles, when it is not supported it will be mapped to user inst
 you can assume developer and user to be interchangeable
 
 
+### Files
+
+Many providers offer the ability to upload files which can be referenced in conversations, or for other purposes like batching. Downloading files is also used for when llm generates something or batches complete.
 
+## Examples
+
+```ruby
+# Upload File
+result = LlmGateway::Client.upload_file("openai", filename: "test.txt", content: "Hello, world!", mime_type: "text/plain")
+result = LlmGateway::Client.download_file("openai", file_id: "file-Kb6X7f8YDffu7FG1NcaPVu")
+# Response Format
+{
+  id: "file-Kb6X7f8YDffu7FG1NcaPVu",
+  size_bytes: 13,  # follows anthropic naming cause clearer
+  created_at: "2025-08-08T06:03:16.000000Z", # follow anthropic style cause easier to read as human
+  filename: "test.txt",
+  mime_type: nil,
+  downloadable: true, # anthropic returns this for other providers it is infered
+  expires_at: nil,
+  purpose: "user_data" # for anthropic this is always user_data
+}
+```
 
 ### Sample Application
 

data/Rakefile

@@ -62,12 +62,18 @@ begin
     sh "git add ."
     sh "git commit -m \"Bump llm_gateway to #{new_version}\""
 
+    # Build the gem first
+    gem_file = `gem build llm_gateway.gemspec | grep 'File:' | awk '{print $2}'`.strip
+
     # Tag and push
     sh "git tag v#{new_version}"
     sh "git push origin main --tags"
 
+    # Create GitHub release with gem file
+    sh "gh release create v#{new_version} #{gem_file} --title \"Release v#{new_version}\" --generate-notes"
+
     # Release the gem
-    sh "gem push $(gem build llm_gateway.gemspec | grep 'File:' | awk '{print $2}')"
+    sh "gem push #{gem_file}"
   end
 rescue LoadError
   # gem-release not available in this environment

data/lib/llm_gateway/adapters/claude/bidirectional_message_mapper.rb

@@ -0,0 +1,83 @@
+# frozen_string_literal: true
+
+module LlmGateway
+  module Adapters
+    module Claude
+      class BidirectionalMessageMapper
+        attr_reader :direction
+
+        def initialize(direction)
+          @direction = direction
+        end
+
+        def map_content(content)
+          # Convert string content to text format
+          content = { type: "text", text: content } unless content.is_a?(Hash)
+
+          case content[:type]
+          when "text"
+            map_text_content(content)
+          when "file"
+            map_file_content(content)
+          when "image"
+            map_image_content(content)
+          when "tool_use"
+            map_tool_use_content(content)
+          when "tool_result"
+            map_tool_result_content(content)
+          else
+            content
+          end
+        end
+
+        private
+
+        def map_text_content(content)
+          {
+            type: "text",
+            text: content[:text]
+          }
+        end
+
+        def map_file_content(content)
+          {
+            type: "document",
+            source: {
+              data: content[:data],
+              type: "text",
+              media_type: content[:media_type]
+            }
+          }
+        end
+
+        def map_image_content(content)
+          {
+            type: "image",
+            source: {
+              data: content[:data],
+              type: "base64",
+              media_type: content[:media_type]
+            }
+          }
+        end
+
+        def map_tool_use_content(content)
+          {
+            type: "tool_use",
+            id: content[:id],
+            name: content[:name],
+            input: content[:input]
+          }
+        end
+
+        def map_tool_result_content(content)
+          {
+            type: "tool_result",
+            tool_use_id: content[:tool_use_id],
+            content: content[:content]
+          }
+        end
+      end
+    end
+  end
+end

data/lib/llm_gateway/adapters/claude/client.rb

@@ -28,6 +28,10 @@ module LlmGateway
           get("files/#{file_id}/content")
         end
 
+        def upload_file(filename, content, mime_type = "application/octet-stream")
+          post_file("files", content, filename, mime_type: mime_type)
+        end
+
         private
 
         def build_headers

data/lib/llm_gateway/adapters/claude/input_mapper.rb

@@ -1,5 +1,7 @@
 # frozen_string_literal: true
 
+require_relative "bidirectional_message_mapper"
+
 module LlmGateway
   module Adapters
     module Claude
@@ -18,20 +20,19 @@ module LlmGateway
         def self.map_messages(messages)
           return messages unless messages
 
+          message_mapper = BidirectionalMessageMapper.new(LlmGateway::DIRECTION_IN)
+
           messages.map do |msg|
             msg = msg.merge(role: "user") if msg[:role] == "developer"
-            msg.slice(:role, :content)
+
             content = if msg[:content].is_a?(Array)
                 msg[:content].map do |content|
-                  if content[:type] == "file"
-                    { type: "document", source: { data: content[:data], type: "text", media_type: content[:media_type] } }
-                  else
-                    content
-                  end
+                  message_mapper.map_content(content)
                 end
             else
-              msg[:content]
+              [ message_mapper.map_content(msg[:content]) ]
             end
+
             {
               role: msg[:role],
               content: content

data/lib/llm_gateway/adapters/claude/output_mapper.rb

@@ -3,6 +3,16 @@
 module LlmGateway
   module Adapters
     module Claude
+      class FileOutputMapper
+        def self.map(data)
+          data.delete(:type) # Didnt see much value in this only option is "file"
+          data.merge(
+            expires_at: nil, # came from open ai api
+            purpose: "user_data",  # came from open ai api
+          )
+        end
+      end
+
       class OutputMapper
         def self.map(data)
           {
@@ -16,10 +26,20 @@ module LlmGateway
         private
 
         def self.map_choices(data)
+          message_mapper = BidirectionalMessageMapper.new(LlmGateway::DIRECTION_OUT)
+
+          content = if data[:content].is_a?(Array)
+              data[:content].map do |content|
+                message_mapper.map_content(content)
+              end
+          else
+            data[:content] ? [ message_mapper.map_content(data[:content]) ] : []
+          end
+
           # Claude returns content directly at root level, not in a choices array
           # We need to construct the choices array from the full response data
           [ {
-            content: data[:content] || [], # Use content directly from Claude response
+            content: content, # Use content directly from Claude response
             finish_reason: data[:stop_reason],
             role: "assistant"
           } ]

data/lib/llm_gateway/adapters/groq/bidirectional_message_mapper.rb

@@ -0,0 +1,18 @@
+# frozen_string_literal: true
+
+require_relative "../open_ai/chat_completions/bidirectional_message_mapper"
+
+module LlmGateway
+  module Adapters
+    module Groq
+      class BidirectionalMessageMapper < OpenAi::ChatCompletions::BidirectionalMessageMapper
+        private
+
+        def map_file_content(content)
+          # Groq doesn't support files, return as text
+          content[:text] || "[File: #{content[:name]}]"
+        end
+      end
+    end
+  end
+end

data/lib/llm_gateway/adapters/groq/input_mapper.rb

@@ -1,104 +1,17 @@
 # frozen_string_literal: true
 
+require_relative "bidirectional_message_mapper"
+require_relative "../open_ai/chat_completions/input_mapper"
+
 module LlmGateway
   module Adapters
     module Groq
-      class InputMapper
-        def self.map(data)
-          {
-            messages: map_messages(data[:messages]),
-            response_format: map_response_format(data[:response_format]),
-            tools: map_tools(data[:tools]),
-            system: map_system(data[:system])
-          }
-        end
-
+      class InputMapper < OpenAi::ChatCompletions::InputMapper
         private
 
         def self.map_system(system)
           system
         end
-
-        def self.map_response_format(response_format)
-          response_format
-        end
-
-        def self.map_messages(messages)
-          return messages unless messages
-
-          messages.flat_map do |msg|
-            if msg[:content].is_a?(Array)
-              # Handle array content with tool calls and tool results
-              tool_calls = []
-              regular_content = []
-              tool_messages = []
-
-              msg[:content].each do |content|
-                case content[:type]
-                when "tool_result"
-                  tool_messages << map_tool_result_message(content)
-                when "tool_use"
-                  tool_calls << map_tool_usage(content)
-                else
-                  regular_content << content
-                end
-              end
-
-              result = []
-
-              # Add the main message with tool calls if any
-              if tool_calls.any? || regular_content.any?
-                main_msg = msg.dup
-                main_msg[:role] = "assistant" if !main_msg[:role]
-                main_msg[:tool_calls] = tool_calls if tool_calls.any?
-                main_msg[:content] = regular_content.any? ? regular_content : nil
-                result << main_msg
-              end
-
-              # Add separate tool result messages
-              result += tool_messages
-
-              result
-            else
-              # Regular message, return as-is
-              [ msg ]
-            end
-          end
-        end
-
-        def self.map_tools(tools)
-          return tools unless tools
-
-          tools.map do |tool|
-            {
-              type: "function",
-              function: {
-                name: tool[:name],
-                description: tool[:description],
-                parameters: tool[:input_schema]
-              }
-            }
-          end
-        end
-
-        def self.map_tool_usage(content)
-            {
-              'id': content[:id],
-              'type': "function",
-              'function': {
-                'name': content[:name],
-                'arguments': content[:input].to_json
-              }
-            }
-        end
-
-        def self.map_tool_result_message(content)
-          {
-            role: "tool",
-            tool_call_id: content[:tool_use_id],
-            content: content[:content]
-          }
-        end
       end
     end
   end

data/lib/llm_gateway/adapters/groq/output_mapper.rb

@@ -3,59 +3,7 @@
 module LlmGateway
   module Adapters
     module Groq
-      class OutputMapper
-        def self.map(data)
-          {
-            id: data[:id],
-            model: data[:model],
-            usage: data[:usage],
-            choices: map_choices(data[:choices])
-          }
-        end
-
-        private
-
-        def self.map_choices(choices)
-          return [] unless choices
-
-          choices.map do |choice|
-            message = choice[:message] || {}
-            content_item = map_content_item(message)
-            tool_calls = map_tool_calls(message[:tool_calls])
-
-            # Only include content_item if it has actual text content
-            content_array = []
-            content_array << content_item if LlmGateway::Utils.present?(content_item[:text])
-            content_array += tool_calls
-
-            { content: content_array }
-          end
-        end
-
-        def self.map_content_item(message)
-          {
-            text: message[:content],
-            type: "text"
-          }
-        end
-
-        def self.map_tool_calls(tool_calls)
-          return [] unless tool_calls
-
-          tool_calls.map do |tool_call|
-            {
-              id: tool_call[:id],
-              type: "tool_use",
-              name: tool_call.dig(:function, :name),
-              input: parse_tool_arguments(tool_call.dig(:function, :arguments))
-            }
-          end
-        end
-
-        def self.parse_tool_arguments(arguments)
-          return arguments unless arguments.is_a?(String)
-          JSON.parse(arguments, symbolize_names: true)
-        end
+      class OutputMapper < LlmGateway::Adapters::OpenAi::ChatCompletions::OutputMapper
       end
     end
   end

data/lib/llm_gateway/adapters/open_ai/chat_completions/bidirectional_message_mapper.rb

@@ -0,0 +1,103 @@
+# frozen_string_literal: true
+
+require "base64"
+
+module LlmGateway
+  module Adapters
+    module OpenAi
+      module ChatCompletions
+        class BidirectionalMessageMapper
+        attr_reader :direction
+
+        def initialize(direction)
+          @direction = direction
+        end
+
+        def map_content(content)
+          # Convert string content to text format
+          content = { type: "text", text: content } unless content.is_a?(Hash)
+          case content[:type]
+          when "text"
+            map_text_content(content)
+          when "file"
+            map_file_content(content)
+          when "image"
+            map_image_content(content)
+          when "tool_use"
+            map_tool_use_content(content)
+          when "function"
+            map_tool_use_content(content)
+          when "tool_result"
+            map_tool_result_content(content)
+          else
+            content
+          end
+        end
+
+        private
+
+        def parse_tool_arguments(arguments)
+          return arguments unless arguments.is_a?(String)
+          JSON.parse(arguments, symbolize_names: true)
+        end
+
+        def map_text_content(content)
+          {
+            type: "text",
+            text: content[:text]
+          }
+        end
+
+        def map_file_content(content)
+          # Map text/plain to application/pdf for OpenAI
+          media_type = content[:media_type] == "text/plain" ? "application/pdf" : content[:media_type]
+          {
+            type: "file",
+            file: {
+              filename: content[:name],
+              file_data: "data:#{media_type};base64,#{Base64.encode64(content[:data])}"
+            }
+          }
+        end
+
+        def map_image_content(content)
+          {
+            type: "image_url",
+            image_url: {
+              url: "data:#{content[:media_type]};base64,#{content[:data]}"
+            }
+          }
+        end
+
+        def map_tool_use_content(content)
+          if direction == LlmGateway::DIRECTION_IN
+          {
+            id: content[:id],
+            type: "function",
+            function: {
+              name: content[:name],
+              arguments: content[:input].to_json
+            }
+          }
+          else
+          {
+            id: content[:id],
+            type: "tool_use",
+            name: content[:function][:name],
+            input: parse_tool_arguments(content[:function][:arguments])
+          }
+          end
+        end
+
+        def map_tool_result_content(content)
+          {
+            role: "tool",
+            tool_call_id: content[:tool_use_id],
+            content: content[:content]
+          }
+        end
+        end
+      end
+    end
+  end
+end