ruby-openai 8.1.0 → 8.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 25124958f17722e0cc4168772bf02daf283b0d43dd2a2c143575ad23fc5539ac
-  data.tar.gz: cfaa4cb81ee668a4296774d7bce0591a2c23a3f0944794acde1db3e89586d199
+  metadata.gz: 35ebfe68ebf25eaf11f33bac4f5c537813e6f5d6f326aff65cc14f273fa93696
+  data.tar.gz: c7f452502d64d2be1d819f35412a7a102b616a9f217b35b6d111ecb7da2aa832
 SHA512:
-  metadata.gz: d2ded6da66a157a190c24b0f7748a833d13ae9a4da67b52879eae8510e74cd90d80ffa6faaf573375c3a26d74b091165af6b30ca51c4cf860b1ba6f5c633cbdf
-  data.tar.gz: d31a28fafe6cac7caa99537f97a0fa1b0b918ebea09371d730e6d425b9bf228c726d9207c7a2bd08070103d49c9aa71e34ecf4f7ea888c2321f5fa128abf4fdc
+  metadata.gz: 47bd06aacf6a5d66686526520fadcd2b0e9f0a62bd9f9840735ca4aee210714e8c48e2430af3cef295a0c7e6b356c0e63701d4d908bc2332c44d3b43c792868c
+  data.tar.gz: 24cb42e2a8ef4735b3fc07456cd308bdfe82a99cba3079d5ef642ee1118c1e2de43ca6013355de0d726210a49c19e234da1368292f9528518b9914ae089d83fb

data/.gitignore

@@ -70,3 +70,5 @@ build-iPhoneSimulator/
 
 # Mac
 .DS_Store
+
+INCIDENT_RESPONSE_PLAN.md

data/CHANGELOG.md

@@ -5,6 +5,19 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [8.2.0] - 2025-08-10
+
+### Added
+
+- Add Security.md and activate private vulnerability reporting
+- Add RealTime endpoint to create WebRTC token - thank you to [@ngelx](https://github.com/ngelx) for the PR and others for input!
+- Add multi-image upload - thank you to [@ryankon](https://github.com/ryankon) and others for requesting.
+- Refactor streaming so that Chat, Responses, Assistant Runs and any others where events are streamed now send the event to the Proc, replacing unused _bytesize. Search the README for `_event` to see how to use this. Important change implemented by [@ingemar](https://github.com/ingemar)!
+- Handle OpenAI::Files request parameters - thank you to [@okorepanov](https://github.com/okorepanov) for the PR.
+- Add Gemini docs - thanks to [@francis](https://github.com/francis).
+- Add web proxy debugging docs - thanks to [@cpb](https://github.com/cpb).
+- Add Rails / ActiveStorage transcription docs - thanks to [@AndreyAzimov](https://github.com/AndreyAzimov).
+
 ## [8.1.0] - 2025-03-30
 
 ### Added

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    ruby-openai (8.1.0)
+    ruby-openai (8.2.0)
       event_stream_parser (>= 0.3.0, < 2.0.0)
       faraday (>= 1)
       faraday-multipart (>= 1)

data/README.md

@@ -6,13 +6,30 @@
 
 Use the [OpenAI API](https://openai.com/blog/openai-api/) with Ruby! 🤖❤️
 
-Stream chats with the Responses API, transcribe and translate audio with Whisper, create images with DALL·E, and much more...
+Stream GPT-5 chats with the Responses API, initiate Realtime WebRTC conversations, and much more...
 
-💥 Click [subscribe now](https://mailchi.mp/8c7b574726a9/ruby-openai) to hear first about new releases in the Rails AI newsletter!
+**Sponsors**
 
-[![RailsAI Newsletter](https://github.com/user-attachments/assets/737cbb99-6029-42b8-9f22-a106725a4b1f)](https://mailchi.mp/8c7b574726a9/ruby-openai)
+<table>
+<tr>
+<td width="300" align="center" valign="top">
 
-[🎮 Ruby AI Builders Discord](https://discord.gg/k4Uc224xVD) | [🐦 X](https://x.com/alexrudall) | [🧠 Anthropic Gem](https://github.com/alexrudall/anthropic) | [🚂 Midjourney Gem](https://github.com/alexrudall/midjourney)
+[<img src="https://github.com/user-attachments/assets/b97e036d-3f22-4116-be97-8f8d1c432a4f" alt="InferToGo logo: man in suit falling, black and white" width="300" height="300">](https://infertogo.com/?utm_source=ruby-openai)
+
+<sub>_[InferToGo](https://infertogo.com/?utm_source=ruby-openai) - The inference addon for your PaaS application._</sub>
+
+</td>
+<td width="300" align="center" valign="top">
+
+[<img src="https://github.com/user-attachments/assets/3feb834c-2721-404c-a64d-02104ed4aba7" alt="SerpApi logo: Purple rounded square with 4 connected white holes" width="300" height="300">](https://serpapi.com/?utm_source=ruby-openai)
+
+<sub>_[SerpApi - Search API](https://serpapi.com/?utm_source=ruby-openai) - Enhance your LLM's knowledge with data from search engines like Google and Bing using our simple API._</sub>
+
+</td>
+</tr>
+</table>
+
+[🎮 Ruby AI Builders Discord](https://discord.gg/k4Uc224xVD) | [🐦 X](https://x.com/alexrudall) | [🧠 Anthropic Gem](https://github.com/alexrudall/anthropic) | [🚂 Midjourney Gem](https://github.com/alexrudall/midjourney) |  [♥️ Thanks to all sponsors!](https://github.com/sponsors/alexrudall)
 
 ## Contents
 
@@ -33,6 +50,7 @@ Stream chats with the Responses API, transcribe and translate audio with Whisper
       - [Deepseek](#deepseek)
       - [Ollama](#ollama)
       - [Groq](#groq)
+      - [Gemini](#gemini)
     - [Counting Tokens](#counting-tokens)
     - [Models](#models)
     - [Chat](#chat)
@@ -75,6 +93,7 @@ Stream chats with the Responses API, transcribe and translate audio with Whisper
       - [Translate](#translate)
       - [Transcribe](#transcribe)
       - [Speech](#speech)
+    - [Real-Time](#real-time)
     - [Usage](#usage)
     - [Errors](#errors-1)
   - [Development](#development)
@@ -216,7 +235,9 @@ client = OpenAI::Client.new(log_errors: true)
 
 ##### Faraday middleware
 
-You can pass [Faraday middleware](https://lostisland.github.io/faraday/#/middleware/index) to the client in a block, eg. to enable verbose logging with Ruby's [Logger](https://ruby-doc.org/3.2.2/stdlibs/logger/Logger.html):
+You can pass [Faraday middleware](https://lostisland.github.io/faraday/#/middleware/index) to the client in a block, eg:
+
+- To enable verbose logging with Ruby's [Logger](https://ruby-doc.org/3.2.2/stdlibs/logger/Logger.html):
 
 ```ruby
 client = OpenAI::Client.new do |f|
@@ -224,6 +245,13 @@ client = OpenAI::Client.new do |f|
 end
 ```
 
+- To add a web debugging proxy like [Charles](https://www.charlesproxy.com/documentation/welcome/):
+
+```ruby
+  client = OpenAI::Client.new do |f|
+    f.proxy = { uri: "http://localhost:8888" }
+  end
+```
 #### Azure
 
 To use the [Azure OpenAI Service](https://learn.microsoft.com/en-us/azure/cognitive-services/openai/) API, you can configure the gem like this:
@@ -254,7 +282,7 @@ client.chat(
     model: "deepseek-chat", # Required.
     messages: [{ role: "user", content: "Hello!"}], # Required.
     temperature: 0.7,
-    stream: proc do |chunk, _bytesize|
+    stream: proc do |chunk, _event|
      print chunk.dig("choices", 0, "delta", "content")
     end
   }
@@ -285,7 +313,7 @@ client.chat(
     model: "llama3", # Required.
     messages: [{ role: "user", content: "Hello!"}], # Required.
     temperature: 0.7,
-    stream: proc do |chunk, _bytesize|
+    stream: proc do |chunk, _event|
       print chunk.dig("choices", 0, "delta", "content")
     end
   }
@@ -309,11 +337,35 @@ client.chat(
     model: "llama3-8b-8192", # Required.
     messages: [{ role: "user", content: "Hello!"}], # Required.
     temperature: 0.7,
+    stream: proc do |chunk, _event|
+     print chunk.dig("choices", 0, "delta", "content")
+    end
+  }
+)
+```
+
+#### Gemini
+
+[Gemini API Chat](https://ai.google.dev/gemini-api/docs/openai) is also broadly compatible with the OpenAI API, and [currently in beta](https://ai.google.dev/gemini-api/docs/openai#current-limitations). Get an access token from [here](https://aistudio.google.com/app/apikey), then:
+
+```ruby
+client = OpenAI::Client.new(
+  access_token: "gemini_access_token_goes_here",
+  uri_base: "https://generativelanguage.googleapis.com/v1beta/openai/"
+)
+
+client.chat(
+  parameters: {
+    model: "gemini-1.5-flash", # Required.
+    messages: [{ role: "user", content: "Hello!"}], # Required.
+    temperature: 0.7,
     stream: proc do |chunk, _bytesize|
      print chunk.dig("choices", 0, "delta", "content")
     end
   }
 )
+
+# => Hello there! How can I help you today?
 ```
 
 ### Counting Tokens
@@ -371,7 +423,7 @@ client.chat(
     model: "gpt-4o", # Required.
     messages: [{ role: "user", content: "Describe a character called Anna!"}], # Required.
     temperature: 0.7,
-    stream: proc do |chunk, _bytesize|
+    stream: proc do |chunk, _event|
       print chunk.dig("choices", 0, "delta", "content")
     end
   }
@@ -457,7 +509,7 @@ You can stream it as well!
       model: "gpt-4o",
       messages: [{ role: "user", content: "Can I have some JSON please?"}],
       response_format: { type: "json_object" },
-      stream: proc do |chunk, _bytesize|
+      stream: proc do |chunk, _event|
         print chunk.dig("choices", 0, "delta", "content")
       end
     }
@@ -488,11 +540,14 @@ You can stream it as well!
 
 ```ruby
 response = client.responses.create(parameters: {
-  model: "gpt-4o",
-  input: "Hello! I'm Szymon!"
+  model: "gpt-5",
+  input: "Hello! I'm Szymon!",
+  reasoning: {
+    "effort": "minimal"
+  }
 })
 puts response.dig("output", 0, "content", 0, "text")
-# => Hello Szymon! How can I assist you today?
+# => Hi Szymon! Great to meet you. How can I help today?
 ```
 
 #### Follow-up Messages
@@ -542,7 +597,7 @@ client.responses.create(
   parameters: {
     model: "gpt-4o", # Required.
     input: "Hello!", # Required.
-    stream: proc do |chunk, _bytesize|
+    stream: proc do |chunk, _event|
       if chunk["type"] == "response.output_text.delta"
         print chunk["delta"]
         $stdout.flush  # Ensure output is displayed immediately
@@ -630,6 +685,9 @@ response =
 message = response.dig("choices", 0, "message")
 
 if message["role"] == "assistant" && message["tool_calls"]
+  # For a subsequent message with the role "tool", OpenAI requires the preceding message to have a single tool_calls argument.
+  messages << message
+
   message["tool_calls"].each do |tool_call|
     tool_call_id = tool_call.dig("id")
     function_name = tool_call.dig("function", "name")
@@ -645,9 +703,6 @@ if message["role"] == "assistant" && message["tool_calls"]
         # decide how to handle
       end
 
-    # For a subsequent message with the role "tool", OpenAI requires the preceding message to have a tool_calls argument.
-    messages << message
-
     messages << {
       tool_call_id: tool_call_id,
       role: "tool",
@@ -1163,7 +1218,7 @@ client.runs.create(
     assistant_id: assistant_id,
     max_prompt_tokens: 256,
     max_completion_tokens: 16,
-    stream: proc do |chunk, _bytesize|
+    stream: proc do |chunk, _event|
       if chunk["object"] == "thread.message.delta"
         print chunk.dig("delta", "content", 0, "text", "value")
       end
@@ -1547,6 +1602,21 @@ puts response.dig("data", 0, "url")
 
 ![Ruby](https://i.ibb.co/sWVh3BX/dalle-ruby.png)
 
+You can also upload arrays of images, eg.
+
+```ruby
+  client = OpenAI::Client.new
+  response = client.images.edit(
+    parameters: {
+      model: "gpt-image-1",
+      image: [File.open(base_image_path, "rb"), "image.png"],
+      prompt: "Take the first image as base and apply the second image as a watermark on the bottom right corner",
+      size: "1024x1024"
+      # Removed response_format parameter as it's not supported with gpt-image-1
+    }
+  )
+```
+
 ### Image Variations
 
 Create n variations of an image.
@@ -1607,6 +1677,20 @@ puts response["text"]
 # => "Transcription of the text"
 ```
 
+If you are using Ruby on Rails with Active Storage, you would need to send an audio or video file like this (User has_one_attached):
+```ruby
+user.media.blob.open do |file|
+  response = client.audio.transcribe(
+    parameters: {
+        model: "whisper-1",
+        file: File.open(file, "rb"),
+        language: "en" # Optional
+    })
+  puts response["text"]
+  # => "Transcription of the text"
+end
+```
+
 #### Speech
 
 The speech API takes as input the text and a voice and returns the content of an audio file you can listen to.
@@ -1625,6 +1709,33 @@ File.binwrite('demo.mp3', response)
 # => mp3 file that plays: "This is a speech test!"
 ```
 
+### Realtime
+
+The [Realtime API](https://platform.openai.com/docs/guides/realtime) allows you to create a live speech-to-speech session with an OpenAI model. It responds with a session object, plus a client_secret key which contains a usable ephemeral API token that can be used to [authenticate browser clients for a WebRTC connection](https://platform.openai.com/docs/guides/realtime#connect-with-webrtc).
+
+```ruby
+response = client.realtime.create(parameters: { model: "gpt-4o-realtime-preview-2024-12-17" })
+puts "ephemeral key: #{response.dig('client_secret', 'value')}"
+# => "ephemeral key: ek_abc123"
+```
+
+Then in the client-side Javascript application, make a POST request to the Real-Time API with the ephemeral key and the SDP offer.
+
+```js
+const OPENAI_REALTIME_URL = 'https://api.openai.com/v1/realtime/sessions'
+const MODEL = 'gpt-4o-realtime-preview-2024-12-17'
+
+const response = await fetch(`${OPENAI_REALTIME_URL}?model=${MODEL}`, {
+  method: 'POST',
+  headers: {
+    'Content-Type': 'application/sdp',
+    'Authorization': `Bearer ${ephemeralKey}`,
+    'OpenAI-Beta': 'realtime=v1'
+  },
+  body: offer.sdp
+})
+```
+
 ### Usage
 
 The Usage API provides information about the cost of various OpenAI services within your organization.

data/SECURITY.md

@@ -0,0 +1,9 @@
+# Security Policy
+
+Thank you for helping us keep ruby-openai and any systems it interacts with secure.
+
+## Reporting Security Issues
+
+The security of our systems and user data is our top priority. We appreciate the work of security researchers acting in good faith in identifying and reporting potential vulnerabilities.
+
+Any validated vulnerability in this functionality can be reported through Github - click on the [Security Tab](https://github.com/alexrudall/ruby-openai/security) and click "Report a vulnerability".

data/lib/openai/client.rb

@@ -1,3 +1,4 @@
+# rubocop:disable Metrics/ClassLength
 module OpenAI
   class Client
     include OpenAI::HTTP
@@ -92,6 +93,10 @@ module OpenAI
       @batches ||= OpenAI::Batches.new(client: self)
     end
 
+    def realtime
+      @realtime ||= OpenAI::Realtime.new(client: self)
+    end
+
     def moderations(parameters: {})
       json_post(path: "/moderations", parameters: parameters)
     end
@@ -132,3 +137,4 @@ module OpenAI
     end
   end
 end
+# rubocop:enable Metrics/ClassLength

data/lib/openai/files.rb

@@ -29,12 +29,12 @@ module OpenAI
       file.close if file.is_a?(File)
     end
 
-    def retrieve(id:)
-      @client.get(path: "/files/#{id}")
+    def retrieve(id:, parameters: {})
+      @client.get(path: "/files/#{id}", parameters: parameters)
     end
 
-    def content(id:)
-      @client.get(path: "/files/#{id}/content")
+    def content(id:, parameters: {})
+      @client.get(path: "/files/#{id}/content", parameters: parameters)
     end
 
     def delete(id:)

data/lib/openai/http.rb

@@ -55,27 +55,6 @@ module OpenAI
       original_response
     end
 
-    # Given a proc, returns an outer proc that can be used to iterate over a JSON stream of chunks.
-    # For each chunk, the inner user_proc is called giving it the JSON object. The JSON object could
-    # be a data object or an error object as described in the OpenAI API documentation.
-    #
-    # @param user_proc [Proc] The inner proc to call for each JSON object in the chunk.
-    # @return [Proc] An outer proc that iterates over a raw stream, converting it to JSON.
-    def to_json_stream(user_proc:)
-      parser = EventStreamParser::Parser.new
-
-      proc do |chunk, _bytes, env|
-        if env && env.status != 200
-          raise_error = Faraday::Response::RaiseError.new
-          raise_error.on_complete(env.merge(body: try_parse_json(chunk)))
-        end
-
-        parser.feed(chunk) do |_type, data|
-          user_proc.call(JSON.parse(data)) unless data == "[DONE]"
-        end
-      end
-    end
-
     def conn(multipart: false)
       connection = Faraday.new do |f|
         f.options[:timeout] = @request_timeout
@@ -120,7 +99,7 @@ module OpenAI
       req_parameters = parameters.dup
 
       if parameters[:stream].respond_to?(:call)
-        req.options.on_data = to_json_stream(user_proc: parameters[:stream])
+        req.options.on_data = Stream.new(user_proc: parameters[:stream]).to_proc
         req_parameters[:stream] = true # Necessary to tell OpenAI to stream.
       elsif parameters[:stream]
         raise ArgumentError, "The stream parameter must be a Proc or have a #call method"
@@ -129,11 +108,5 @@ module OpenAI
       req.headers = headers
       req.body = req_parameters.to_json
     end
-
-    def try_parse_json(maybe_json)
-      JSON.parse(maybe_json)
-    rescue JSON::ParserError
-      maybe_json
-    end
   end
 end

data/lib/openai/images.rb

@@ -19,9 +19,23 @@ module OpenAI
     private
 
     def open_files(parameters)
-      parameters = parameters.merge(image: File.open(parameters[:image]))
-      parameters = parameters.merge(mask: File.open(parameters[:mask])) if parameters[:mask]
-      parameters
+      params = parameters.dup
+
+      if params[:image].is_a?(Array)
+        process_image_array(params)
+      else
+        params[:image] = File.open(params[:image])
+      end
+
+      params[:mask] = File.open(params[:mask]) if params[:mask]
+      params
+    end
+
+    def process_image_array(params)
+      params[:image].each_with_index do |img_path, index|
+        params[:"image[#{index}]"] = File.open(img_path)
+      end
+      params.delete(:image)
     end
   end
 end

data/lib/openai/realtime.rb

@@ -0,0 +1,19 @@
+module OpenAI
+  class Realtime
+    def initialize(client:)
+      @client = client.beta(realtime: "v1")
+    end
+
+    # Create a new real-time session with OpenAI.
+    #
+    # This method sets up a new session for real-time voice interaction with an OpenAI model.
+    # It returns session details that can be used to establish a WebRTC connection.
+    #
+    # @param parameters [Hash] parameters for the session (see: https://platform.openai.com/docs/api-reference/realtime-sessions/create)
+    # @return [Hash] Session details including session ID, ICE servers, and other
+    # connection information
+    def create(parameters: {})
+      @client.json_post(path: "/realtime/sessions", parameters: parameters)
+    end
+  end
+end

data/lib/openai/stream.rb

@@ -0,0 +1,50 @@
+module OpenAI
+  class Stream
+    DONE = "[DONE]".freeze
+    private_constant :DONE
+
+    def initialize(user_proc:, parser: EventStreamParser::Parser.new)
+      @user_proc = user_proc
+      @parser = parser
+
+      # To be backwards compatible, we need to check how many arguments the user_proc takes.
+      @user_proc_arity =
+        case user_proc
+        when Proc
+          user_proc.arity.abs
+        else
+          user_proc.method(:call).arity.abs
+        end
+    end
+
+    def call(chunk, _bytes, env)
+      handle_http_error(chunk: chunk, env: env) if env && env.status != 200
+
+      parser.feed(chunk) do |event, data|
+        next if data == DONE
+
+        args = [JSON.parse(data), event].first(user_proc_arity)
+        user_proc.call(*args)
+      end
+    end
+
+    def to_proc
+      method(:call).to_proc
+    end
+
+    private
+
+    attr_reader :user_proc, :parser, :user_proc_arity
+
+    def handle_http_error(chunk:, env:)
+      raise_error = Faraday::Response::RaiseError.new
+      raise_error.on_complete(env.merge(body: try_parse_json(chunk)))
+    end
+
+    def try_parse_json(maybe_json)
+      JSON.parse(maybe_json)
+    rescue JSON::ParserError
+      maybe_json
+    end
+  end
+end

data/lib/openai/version.rb

@@ -1,3 +1,3 @@
 module OpenAI
-  VERSION = "8.1.0".freeze
+  VERSION = "8.2.0".freeze
 end

data/lib/openai.rb

@@ -10,8 +10,10 @@ require_relative "openai/responses"
 require_relative "openai/assistants"
 require_relative "openai/threads"
 require_relative "openai/messages"
+require_relative "openai/realtime"
 require_relative "openai/runs"
 require_relative "openai/run_steps"
+require_relative "openai/stream"
 require_relative "openai/vector_stores"
 require_relative "openai/vector_store_files"
 require_relative "openai/vector_store_file_batches"

metadata

@@ -1,14 +1,13 @@
 --- !ruby/object:Gem::Specification
 name: ruby-openai
 version: !ruby/object:Gem::Version
-  version: 8.1.0
+  version: 8.2.0
 platform: ruby
 authors:
 - Alex
-autorequire:
 bindir: exe
 cert_chain: []
-date: 2025-03-30 00:00:00.000000000 Z
+date: 1980-01-02 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: event_stream_parser
@@ -58,7 +57,6 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '1'
-description:
 email:
 - alexrudall@users.noreply.github.com
 executables: []
@@ -84,6 +82,7 @@ files:
 - LICENSE.txt
 - README.md
 - Rakefile
+- SECURITY.md
 - bin/console
 - bin/setup
 - lib/openai.rb
@@ -98,9 +97,11 @@ files:
 - lib/openai/images.rb
 - lib/openai/messages.rb
 - lib/openai/models.rb
+- lib/openai/realtime.rb
 - lib/openai/responses.rb
 - lib/openai/run_steps.rb
 - lib/openai/runs.rb
+- lib/openai/stream.rb
 - lib/openai/threads.rb
 - lib/openai/usage.rb
 - lib/openai/vector_store_file_batches.rb
@@ -119,7 +120,6 @@ metadata:
   changelog_uri: https://github.com/alexrudall/ruby-openai/blob/main/CHANGELOG.md
   rubygems_mfa_required: 'true'
   funding_uri: https://github.com/sponsors/alexrudall
-post_install_message:
 rdoc_options: []
 require_paths:
 - lib
@@ -134,8 +134,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.5.11
-signing_key:
+rubygems_version: 3.6.7
 specification_version: 4
 summary: "OpenAI API + Ruby! \U0001F916❤️"
 test_files: []