ask-llm-providers 0.10.1 → 0.12.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 15fc5c9c298a6f88b722fde13139a1c3fe2dfa795255e8d7226b9ca0b5187bd5
-  data.tar.gz: f9f1c3d677c470c2fffeac5720024ecc9c40fb58cf3b058ac2a99e1e5769100e
+  metadata.gz: 912f0553986f01615036939eb502aa1f42a35023dc0f04181b970bec4ca3fbbe
+  data.tar.gz: 71868d9efd700086b9ad8a3d4ae9f0191239bc4333eed2c889212793abeb1c71
 SHA512:
-  metadata.gz: 63e97ca4cf04aac0d2e9de99a2d4b6425bd1c95466be13a7172667729830c83b3025fb54094e4a5b20bd1b6b49374bdd1575ef8a60a9eedf3395438a559cc1cf
-  data.tar.gz: 50dfb8a0ca28288df43be9f8126530fa6bddbb6da4d1f951023b4ad7e592298b8c5ed9abb04f53494b65a049421447ba1ef4c02ea594a3af9f7b63176c707598
+  metadata.gz: 7354ea67a347577694c5cdd8ea3462719a308d8b268f90eb75022618b822f348395ec66cebb2bdb777168fc9f2359fa8d000fb6e647b947f58de84b6cc8dc495
+  data.tar.gz: e3c41f929d7118ede14d207e4c6aa480ca8bab95ab8d28a32085e0ba11ffa7ce3b87a24244d54ecaaee14323614840b57af8a312ba13ac62ab843e5fa53431aa

data/CHANGELOG.md

@@ -1,3 +1,27 @@
+## [0.11.1] — 2026-08-07
+
+### Added
+
+- **GitHub Copilot provider entry + models.** `github_copilot` in the OpenAI-compatible registry (`api.githubcopilot.com` with the `X-GitHub-Api-Version` header) plus catalog models (gpt-5.4, gpt-5.4-nano, gpt-4.1, gpt-4o — flat-rate, riding the user's Copilot subscription via ask-auth's GithubCopilot device OAuth).
+
+## [0.11.0] — 2026-08-07
+
+### Added
+
+- **OpenAI Codex provider — ChatGPT subscription access.** `Ask::Providers::OpenaiCodex` always speaks the Responses API at `chatgpt.com/backend-api/codex/responses` with a `ChatGPT-Account-Id` header; supports real **Responses SSE streaming** (`response.output_text.delta` text, `function_call` item/argument events, `response.completed` usage). Pair with ask-auth's `OpenaiCodex` OAuth provider (access token via `api_key:`, account id via `account_id:`). Registered as `:openai_codex`.
+- **Codex models in the catalog** (`openai_codex.json`): gpt-5.5, gpt-5.4, gpt-5.4-mini, gpt-5.3-codex-spark — flat-rate (pricing 0) since they ride the user's subscription.
+
+## [0.10.2] - 2026-08-06
+
+### Fixed
+
+- **Streamed token usage is now captured.** OpenAI-style streams end with a
+  chunk whose `choices` is empty but `usage` populated; `dig("choices", 0)
+  or next` silently dropped it, so every streamed call reported ~1 output
+  token (input 0) — token billing, cost accounting, and usage metrics
+  under-counted by orders of magnitude. The usage-only chunk is now added
+  to the stream (without yielding — it carries no content).
+
 ## [0.10.1] - 2026-07-31
 
 ### Fixed

data/README.md

@@ -1,20 +1,10 @@
 # ask-llm-providers
 
-All LLM providers for the ask-rb ecosystem in one gem. Implements `Ask::Provider`
-from `ask-core` with a capabilities-based interface.
-
-## Supported Providers
-
-| Provider | Auth | Implementation |
-|---|---|---|
-| **OpenAI** + all OpenAI-compatible | `Ask::Auth.resolve(:openai_api_key)` | `Ask::Providers::OpenAI` |
-| **Anthropic** (Claude) | `Ask::Auth.resolve(:anthropic_api_key)` | `Ask::Providers::Anthropic` |
-| **Google Gemini** | `Ask::Auth.resolve(:gemini_api_key)` | `Ask::Providers::Google` |
-| **Vertex AI** | GCP service account | `Ask::Providers::Google` (via Vertex) |
-| **Amazon Bedrock** | AWS credentials chain | `Ask::Providers::Bedrock` |
-| **Ollama** (local) | None needed | `Ask::Providers::Ollama` |
-| **Mistral AI** | `Ask::Auth.resolve(:mistral_api_key)` | `Ask::Providers::Mistral` |
-| **Cloudflare Workers AI** | `Ask::Auth.resolve(:cloudflare_api_key)` | `Ask::Providers::Cloudflare` |
+All LLM providers for the ask-rb ecosystem in one gem. Implements the
+`Ask::Provider` interface from `ask-core` with a capabilities-based interface:
+7 canonical provider classes plus 26 OpenAI-compatible registry entries,
+a bundled model catalog, and cost calculation. Providers auto-register and
+the model catalog auto-loads when the gem is required.
 
 ## Installation
 
@@ -22,102 +12,86 @@ from `ask-core` with a capabilities-based interface.
 gem "ask-llm-providers"
 ```
 
-## Usage
+## Quick Start
 
 ```ruby
 require "ask-llm-providers"
 
-# All providers are auto-registered with Ask::Models
-models = Ask::Models.find("gpt-4o")
-# => { provider: :openai, capabilities: [...] }
-
-# Use a provider directly
+# Use a provider directly; streaming yields chunks to the block
 provider = Ask::Providers::OpenAI.new
-provider.chat(conversation, tools: [], model: "gpt-4o") do |chunk|
+provider.chat([{ role: "user", content: "Tell me a story" }], model: "gpt-4o") do |chunk|
   print chunk.content
 end
-```
 
-## Capabilities
-
-Each provider and model exposes its capabilities:
-
-```ruby
-provider = Ask::Providers::OpenAI.new
-provider.capabilities
-# => { chat: true, streaming: true, tool_calls: true, vision: true, thinking: true,
-#     :structured_output, :embed, :transcribe, :paint, :moderate]
-
-model = Ask::Models.find("claude-sonnet-4-5")
-model[:capabilities]
-# => { chat: true, streaming: true, tool_calls: true, vision: true, thinking: true, :prompt_caching]
-
-# Unsupported capabilities raise a helpful error
-provider = Ask::Providers::Anthropic.new
-provider.embed(["text"], model: "claude-sonnet-4-5")
-# => Ask::CapabilityNotSupported: Anthropic (claude-sonnet-4-5) does not support embeddings.
+# Look up model metadata (capabilities, pricing, context window)
+model = Ask::ModelCatalog.find("gpt-4o")
+model.capabilities # => ["chat", "streaming", "tool_calls", ...]
 ```
 
+## Supported Providers
 
-
-## Streaming
+| Provider | Auth |
+|---|---|
+| OpenAI | `Ask::Auth.resolve(:openai_api_key)` (env `OPENAI_API_KEY`) |
+| Anthropic (Claude) | `Ask::Auth.resolve(:anthropic_api_key)` (env `ANTHROPIC_API_KEY`) |
+| Google Gemini | `Ask::Auth.resolve(:gemini_api_key)` (env `GEMINI_API_KEY`); Vertex AI via GCP service account |
+| Amazon Bedrock | AWS credentials chain (env, `~/.aws`, instance profile) |
+| Ollama (local) | none needed |
+| Mistral AI | `Ask::Auth.resolve(:mistral_api_key)` (env `MISTRAL_API_KEY`) |
+| Cloudflare Workers AI | `Ask::Auth.resolve(:cloudflare_api_key)` (env `CLOUDFLARE_API_KEY`) |
+| 26 OpenAI-compatible | per-provider `*_API_KEY` env var (e.g. `DEEPSEEK_API_KEY`, `GROQ_API_KEY`, `OPENROUTER_API_KEY`) |
+
+Credentials resolve through `Ask::Auth` in order: environment variables,
+`~/.ask/credentials.yml`, Rails credentials, then database and OAuth
+providers. Keys are read via `Ask::Auth.resolve(:<name>_api_key)`.
+
+The 26 OpenAI-compatible entries are registered from
+`Ask::LLM::OPENAI_COMPATIBLE`: DeepSeek, Groq, Together, Fireworks, Cerebras,
+xAI, Perplexity, DeepInfra, Anyscale, SambaNova, Nebius, Nvidia NIM, Friendli,
+Hyperbolic, Novita, Nscale, Featherless, AI/ML API, AI21, Meta, GitHub Models,
+OpenRouter, OpenCode, OpenCode Go, Mimo, and Moonshot. Each uses its own
+`*_API_KEY` env var; note that `opencode_go` uses `OPENCODE_GO_API_KEY` (its
+own key, not `OPENCODE_API_KEY`).
+
+## Model Catalog
+
+The gem bundles model metadata (capabilities, pricing, context windows,
+modalities) as JSON for 12 providers: openai, anthropic, gemini, vertex_ai,
+bedrock, deepseek, mistral, perplexity, xai, meta, moonshot, and nvidia_nim,
+with 400+ models in total.
 
 ```ruby
-stream = provider.chat(
-  [{ role: "user", content: "Tell me a story" }],
-  model: "gpt-4o",
-  stream: true
-) do |chunk|
-  print chunk.content
-end
-
-# After streaming completes, you can access the full response
-puts stream.accumulated_text
-puts stream.accumulated_usage
+Ask::ModelCatalog.find("claude-sonnet-4-5") # => Ask::ModelInfo
+Ask::ModelCatalog.chat_models               # => filtered catalog
+Ask::ModelCatalog.by_provider(:gemini)
+Ask::ModelCatalog.refresh!                  # fetch latest from models.dev
 ```
 
-## Tool Calls
+## Entry Points
 
-```ruby
-tools = [{
-  name: "get_weather",
-  description: "Get weather for a location",
-  parameters: {
-    type: "object",
-    properties: { location: { type: "string" } },
-    required: ["location"]
-  }
-}]
-
-response = provider.chat(
-  [{ role: "user", content: "What's the weather in NYC?" }],
-  model: "gpt-4o",
-  tools: tools
-)
-# response.tool_call? => true
-# response.tool_calls => [{ id: "call_1", name: "get_weather", arguments: '{"location":"NYC"}' }]
-```
+| API | Purpose |
+|---|---|
+| `Ask::Providers::OpenAI/Anthropic/Google/Bedrock/Ollama/Mistral/Cloudflare` | Canonical provider classes |
+| `Ask::LLM::OPENAI_COMPATIBLE` | Registry data for the 26 compatible providers |
+| `Ask::LLM::Catalog.load!` / `refresh!` | Load bundled + user model data into `Ask::ModelCatalog` |
+| `Ask::LLM::Aliases.resolve("claude-sonnet-4")` | Short-name to canonical model ID resolution |
+| `Ask::LLM::CostCalculator.calculate(model, input_tokens:, output_tokens:)` | USD cost from model pricing |
+| `Ask::Provider.resolve(:openai)` | Class lookup by registered slug |
 
-## Error Handling
+Providers accept a `base_url` override, so any OpenAI-compatible endpoint can
+be used through `Ask::Providers::OpenAI` or `Ask::Providers::OpenAICompatible`.
 
-Provider errors map to structured `Ask::Error` types:
+## Full documentation
 
-```ruby
-Ask::RateLimitError       # 429 — retry with backoff
-Ask::Unauthorized         # 401/403 — check your API key
-Ask::ServerError          # 500 — provider issue
-Ask::ServiceUnavailable   # 503 — temporary
-Ask::ContextLengthExceeded # context window exceeded
-Ask::ProviderError        # other provider errors
-Ask::CapabilityNotSupported # feature not available on this model
-```
+The full ask-rb documentation lives at https://ask-rb.github.io/ask-docs.
+https://ask-rb.github.io/ask-docs/core/providers covers ask-llm-providers in
+depth, including capabilities, streaming, tool calls, and error handling.
+API reference: https://ask-rb.github.io/ask-docs/reference/api.
 
 ## Development
 
-```bash
-bin/setup
+bundle install
 bundle exec rake test
-```
 
 ## License
 

data/lib/ask/llm/http.rb

@@ -1,4 +1,5 @@
 # frozen_string_literal: true
+require "time"
 
 module Ask
   module LLM

data/lib/ask/llm/models/github_copilot.json

@@ -0,0 +1,86 @@
+[
+  {
+    "id": "gpt-5.4",
+    "name": "GPT-5.4",
+    "provider": "github_copilot",
+    "family": "gpt",
+    "context_window": 400000,
+    "max_output_tokens": 128000,
+    "capabilities": ["chat", "streaming", "tool_calls"],
+    "modalities": { "input": ["text"], "output": ["text"] },
+    "pricing": {
+      "text_tokens": {
+        "standard": {
+          "input_per_million": 0.0,
+          "output_per_million": 0.0,
+          "cache_read_input_per_million": 0.0
+        }
+      }
+    },
+    "knowledge_cutoff": "2025-06-01",
+    "created_at": "2026-01-01"
+  },
+  {
+    "id": "gpt-5.4-nano",
+    "name": "GPT-5.4 nano",
+    "provider": "github_copilot",
+    "family": "gpt",
+    "context_window": 400000,
+    "max_output_tokens": 64000,
+    "capabilities": ["chat", "streaming", "tool_calls"],
+    "modalities": { "input": ["text"], "output": ["text"] },
+    "pricing": {
+      "text_tokens": {
+        "standard": {
+          "input_per_million": 0.0,
+          "output_per_million": 0.0,
+          "cache_read_input_per_million": 0.0
+        }
+      }
+    },
+    "knowledge_cutoff": "2025-06-01",
+    "created_at": "2026-01-01"
+  },
+  {
+    "id": "gpt-4.1",
+    "name": "GPT-4.1",
+    "provider": "github_copilot",
+    "family": "gpt",
+    "context_window": 1000000,
+    "max_output_tokens": 32000,
+    "capabilities": ["chat", "streaming", "tool_calls"],
+    "modalities": { "input": ["text"], "output": ["text"] },
+    "pricing": {
+      "text_tokens": {
+        "standard": {
+          "input_per_million": 0.0,
+          "output_per_million": 0.0,
+          "cache_read_input_per_million": 0.0
+        }
+      }
+    },
+    "knowledge_cutoff": "2025-06-01",
+    "created_at": "2025-05-01"
+  },
+  {
+    "id": "gpt-4o",
+    "name": "GPT-4o",
+    "provider": "github_copilot",
+    "family": "gpt",
+    "context_window": 128000,
+    "max_output_tokens": 16000,
+    "capabilities": ["chat", "streaming", "tool_calls"],
+    "modalities": { "input": ["text"], "output": ["text"] },
+    "pricing": {
+      "text_tokens": {
+        "standard": {
+          "input_per_million": 0.0,
+          "output_per_million": 0.0,
+          "cache_read_input_per_million": 0.0
+        }
+      }
+    },
+    "knowledge_cutoff": "2024-10-01",
+    "created_at": "2024-05-01"
+  }
+]

data/lib/ask/llm/models/openai_codex.json

@@ -0,0 +1,86 @@
+[
+  {
+    "id": "gpt-5.5",
+    "name": "GPT-5.5",
+    "provider": "openai_codex",
+    "family": "gpt-codex",
+    "context_window": 400000,
+    "max_output_tokens": 128000,
+    "capabilities": ["chat", "streaming", "tool_calls", "vision"],
+    "modalities": { "input": ["text"], "output": ["text"] },
+    "pricing": {
+      "text_tokens": {
+        "standard": {
+          "input_per_million": 0.0,
+          "output_per_million": 0.0,
+          "cache_read_input_per_million": 0.0
+        }
+      }
+    },
+    "knowledge_cutoff": "2025-06-01",
+    "created_at": "2026-01-01"
+  },
+  {
+    "id": "gpt-5.4",
+    "name": "GPT-5.4",
+    "provider": "openai_codex",
+    "family": "gpt-codex",
+    "context_window": 400000,
+    "max_output_tokens": 64000,
+    "capabilities": ["chat", "streaming", "tool_calls", "vision"],
+    "modalities": { "input": ["text"], "output": ["text"] },
+    "pricing": {
+      "text_tokens": {
+        "standard": {
+          "input_per_million": 0.0,
+          "output_per_million": 0.0,
+          "cache_read_input_per_million": 0.0
+        }
+      }
+    },
+    "knowledge_cutoff": "2025-06-01",
+    "created_at": "2025-11-01"
+  },
+  {
+    "id": "gpt-5.4-mini",
+    "name": "GPT-5.4 mini",
+    "provider": "openai_codex",
+    "family": "gpt-codex",
+    "context_window": 400000,
+    "max_output_tokens": 64000,
+    "capabilities": ["chat", "streaming", "tool_calls", "vision"],
+    "modalities": { "input": ["text"], "output": ["text"] },
+    "pricing": {
+      "text_tokens": {
+        "standard": {
+          "input_per_million": 0.0,
+          "output_per_million": 0.0,
+          "cache_read_input_per_million": 0.0
+        }
+      }
+    },
+    "knowledge_cutoff": "2025-06-01",
+    "created_at": "2025-11-01"
+  },
+  {
+    "id": "gpt-5.3-codex-spark",
+    "name": "GPT-5.3 Codex Spark",
+    "provider": "openai_codex",
+    "family": "gpt-codex",
+    "context_window": 400000,
+    "max_output_tokens": 64000,
+    "capabilities": ["chat", "streaming", "tool_calls", "vision"],
+    "modalities": { "input": ["text"], "output": ["text"] },
+    "pricing": {
+      "text_tokens": {
+        "standard": {
+          "input_per_million": 0.0,
+          "output_per_million": 0.0,
+          "cache_read_input_per_million": 0.0
+        }
+      }
+    },
+    "knowledge_cutoff": "2025-06-01",
+    "created_at": "2025-10-01"
+  }
+]

data/lib/ask/llm/openai_compatible.rb

@@ -75,6 +75,10 @@ module Ask
       opencode_go:   { api_base: "https://opencode.ai/zen/go/v1",                 api_key_env: "OPENCODE_GO_API_KEY",
                         capabilities: { chat: true, streaming: true, tool_calls: true } },
 
+      github_copilot: { api_base: "https://api.githubcopilot.com",                  api_key_env: "GITHUB_COPILOT_TOKEN",
+                        extra_headers: { "X-GitHub-Api-Version" => "2026-06-01" },
+                        capabilities: { chat: true, streaming: true, tool_calls: true } },
+
       openrouter:    { api_base: "https://openrouter.ai/api/v1",                  api_key_env: "OPENROUTER_API_KEY",
                        extra_headers: { "HTTP-Referer" => "https://github.com/ask-rb",
                                         "X-Title" => "ask-rb" },

data/lib/ask/llm/version.rb

@@ -2,6 +2,6 @@
 
 module Ask
   module LLM
-    VERSION = "0.10.1"
+    VERSION = "0.12.0"
   end
 end

data/lib/ask/provider/anthropic.rb

@@ -234,9 +234,27 @@ module Ask
           # Fall back to text description
           { type: "text", text: "[#{type} content not supported by Anthropic]" }
         when "file"
-          data = block[:data] || block["data"] || ""
-          filename = block[:filename] ? "[#{block[:filename]}] " : ""
-          { type: "text", text: "#{filename}#{data}" }
+          # Files become Anthropic document blocks: inline data (base64 or
+          # text source), a URL, or a provider file reference.
+          file_id = block[:file_id] || block["file_id"]
+          url = block[:url] || block["url"]
+          data = block[:data] || block["data"]
+          mime = block[:mime_type] || block["mime_type"]
+          filename = block[:filename] || "file"
+
+          if file_id
+            { type: "document", source: { type: "file", file_id: file_id }, title: filename }
+          elsif url
+            { type: "document", source: { type: "url", url: url }, title: filename }
+          elsif data
+            if mime.to_s.start_with?("text/")
+              { type: "document", source: { type: "text", media_type: (mime || "text/plain"), data: data }, title: filename }
+            else
+              { type: "document", source: { type: "base64", media_type: (mime || "application/octet-stream"), data: Base64.strict_encode64(data) }, title: filename }
+            end
+          else
+            block
+          end
         else
           block
         end

data/lib/ask/provider/google.rb

@@ -143,7 +143,13 @@ module Ask
         google_role = role == "assistant" ? "model" : role
 
         parts = []
-        parts << { text: content } if content
+        if content.is_a?(Array)
+          # Content blocks → Gemini parts (flat parts array). Previously
+          # arrays were passed through as a broken { text: [hash, …] } part.
+          content.each { |block| parts.concat(format_google_content_block(block)) }
+        elsif content
+          parts << { text: content }
+        end
 
         if msg[:tool_calls] || msg["tool_calls"]
           (msg[:tool_calls] || msg["tool_calls"]).each do |tc|
@@ -168,6 +174,57 @@ module Ask
         { role: google_role, parts: }
       end
 
+      # Gemini content block → flat parts array. Media/file blocks become
+      # inlineData (base64) or fileData (uri/file_id); text-like files are
+      # inlined as text with a filename marker; unsupported blocks are
+      # skipped rather than mangled.
+      def format_google_content_block(block)
+        block = block.transform_keys(&:to_sym) if block.respond_to?(:transform_keys)
+        type = block[:type] || block["type"]
+
+        case type
+        when "text"
+          [{ text: block[:text] || block["text"] }]
+        when "image", "audio", "video"
+          mime = block[:mime_type] || block["mime_type"]
+          file_id = block[:file_id] || block["file_id"]
+          url = block[:url] || block["url"]
+          base64 = block[:base64] || block["base64"]
+
+          if file_id
+            [{ fileData: { mimeType: mime, fileUri: file_id } }]
+          elsif url
+            [{ fileData: { mimeType: mime, fileUri: url } }]
+          elsif base64
+            [{ inlineData: { mimeType: (mime || "application/octet-stream"), data: base64 } }]
+          else
+            []
+          end
+        when "file"
+          mime = block[:mime_type] || block["mime_type"]
+          file_id = block[:file_id] || block["file_id"]
+          url = block[:url] || block["url"]
+          data = block[:data] || block["data"]
+
+          if file_id
+            [{ fileData: { mimeType: mime, fileUri: file_id } }]
+          elsif url
+            [{ fileData: { mimeType: mime, fileUri: url } }]
+          elsif data
+            if mime.to_s.start_with?("text/")
+              filename = block[:filename] ? "[#{block[:filename]}] " : ""
+              [{ text: "#{filename}#{data}" }]
+            else
+              [{ inlineData: { mimeType: (mime || "application/octet-stream"), data: Base64.strict_encode64(data) } }]
+            end
+          else
+            []
+          end
+        else
+          []
+        end
+      end
+
       def format_tools(tools)
         tools.map { |t|
           {

data/lib/ask/provider/openai.rb

@@ -124,7 +124,14 @@ module Ask
       def parse_stream(raw, stream, model, &block)
         each_sse_event(raw) do |data|
           parsed = JSON.parse(data) rescue next
-          choice = parsed.dig("choices", 0) or next
+          choice = parsed.dig("choices", 0)
+          if choice.nil?
+            # Final chunk of an OpenAI-style stream: choices is empty but
+            # usage is populated. Added to the stream (for token
+            # accounting) without yielding — it carries no content.
+            stream.add(Ask::Chunk.new(content: nil, usage: parsed["usage"])) if parsed["usage"]
+            next
+          end
           delta = choice["delta"] || {}
           thinking = extract_thinking(parsed, delta)
           chunk = Ask::Chunk.new(
@@ -272,7 +279,11 @@ module Ask
           content = msg[:content] || msg["content"] || ""
 
           entry = { role: role.to_s }
-          entry[:content] = [{ type: "input_text", text: content.to_s }]
+          entry[:content] = if content.is_a?(Array)
+            content.map { |block| format_responses_content_block(block) }
+          else
+            [{ type: "input_text", text: content.to_s }]
+          end
 
           # Handle tool calls in assistant messages
           if (tc = msg[:tool_calls] || msg["tool_calls"]) && tc.respond_to?(:any?) && tc.any?
@@ -295,6 +306,53 @@ module Ask
         end
       end
 
+      # Responses API content block → input_* part. File blocks become
+      # input_file (inline data URI, URL, or provider file id); images
+      # become input_image. Audio/video have no Responses input carrier
+      # and degrade to a text note (matching chat completions behavior).
+      def format_responses_content_block(block)
+        block = block.transform_keys(&:to_sym) if block.respond_to?(:transform_keys)
+        type = block[:type] || block["type"]
+
+        case type
+        when "text"
+          { type: "input_text", text: block[:text] || block["text"] }
+        when "image"
+          file_id = block[:file_id] || block["file_id"]
+          url = block[:url] || block["url"]
+          base64 = block[:base64] || block["base64"]
+          if url || base64
+            mime = block[:mime_type] || block["mime_type"] || "image/png"
+            url ||= Ask::DataURI.from_base64(base64, mime_type: mime)
+            { type: "input_image", image_url: url, detail: "auto" }
+          else
+            { type: "input_image", image_url: file_id, detail: "auto" }
+          end
+        when "file"
+          file_id = block[:file_id] || block["file_id"]
+          url = block[:url] || block["url"]
+          data = block[:data] || block["data"]
+          if file_id
+            { type: "input_file", file_id: file_id }
+          elsif url
+            { type: "input_file", file_url: url }
+          elsif data
+            mime = block[:mime_type] || block["mime_type"] || "application/octet-stream"
+            {
+              type: "input_file",
+              filename: block[:filename] || "file",
+              file_data: Ask::DataURI.from_base64(Base64.strict_encode64(data), mime_type: mime)
+            }
+          else
+            block
+          end
+        when "audio", "video"
+          { type: "input_text", text: "[#{type} content not supported by the Responses API]" }
+        else
+          block
+        end
+      end
+
       def extract_responses_provider_results(output, provider_tools)
         results = {}
         provider_tool_names = provider_tools.map(&:name)
@@ -386,7 +444,7 @@ module Ask
           elsif block[:base64] || block["base64"]
             mime = block[:mime_type] || block["mime_type"] || "image/png"
             data = block[:base64] || block["base64"]
-            { type: "image_url", image_url: { url: "data:#{mime};base64,#{data}" } }
+            { type: "image_url", image_url: { url: Ask::DataURI.from_base64(data, mime_type: mime) } }
           elsif block[:file_id] || block["file_id"]
             { type: "image_url", image_url: { url: block[:file_id] || block["file_id"] } }
           else

data/lib/ask/provider/openai_codex.rb

@@ -0,0 +1,114 @@
+# frozen_string_literal: true
+
+require_relative "openai"
+require_relative "../llm/sse_buffer"
+
+module Ask
+  module Providers
+    # OpenAI Codex — requests routed through a ChatGPT subscription via OAuth
+    # (the access token + account id come from ask-auth's OpenaiCodex
+    # provider, passed explicitly by the host app: api_key / account_id).
+    #
+    # Always speaks the Responses API at
+    # chatgpt.com/backend-api/codex/responses, and streams the Responses SSE
+    # event shape (response.output_text.delta, function_call item events,
+    # response.completed) — the chat/completions parsing in the base class
+    # doesn't apply here.
+    class OpenaiCodex < OpenAI
+      API_BASE = "https://chatgpt.com/backend-api/codex"
+      SLUG = "openai_codex"
+
+      def self.slug
+        SLUG
+      end
+
+      def api_base
+        @config.base_url || API_BASE
+      end
+
+      def headers
+        super.tap do |h|
+          h["ChatGPT-Account-Id"] = @config.account_id if @config.account_id
+        end
+      end
+
+      def chat(messages, model:, tools: nil, temperature: nil, stream: nil, schema: nil, **params, &block)
+        msgs = messages.is_a?(Ask::Conversation) ? messages.to_a : messages
+        regular_tools, = split_tools(tools)
+
+        payload = {
+          model: model,
+          input: format_responses_input(msgs)
+        }
+        payload[:tools] = format_tools(regular_tools) if regular_tools&.any?
+        payload[:temperature] = temperature if temperature
+        payload.merge!(params)
+
+        if stream
+          codex_stream(payload, model, &block)
+        else
+          responses_chat_nonstream(payload, model, [])
+        end
+      end
+
+      private
+
+      def codex_stream(payload, model, &block)
+        stream = Ask::Stream.new
+        init_sse_buffer
+        @http.post("responses") do |req|
+          req.body = payload.merge(stream: true)
+          req.options.on_data = proc { |data, _bytes, _env| parse_codex_stream(data, stream, model, &block) }
+        end.tap do |resp|
+          unless resp.success?
+            err_body = case resp.body
+                       when Hash then resp.body
+                       when String then (JSON.parse(resp.body) rescue { "error" => { "message" => "HTTP #{resp.status}: #{resp.body[0..200]}" } })
+                       else { "error" => { "message" => "HTTP #{resp.status}: empty response body" } }
+                       end
+            err_body["error"] ||= {}
+            err_body["error"]["_status"] = resp.status
+            raise LLM::HTTP.map_error(resp.status, err_body, provider: "OpenAI")
+          end
+        end
+        stream.finish!
+        stream
+      end
+
+      # Responses-API SSE events:
+      #   response.output_text.delta          -> text content
+      #   response.output_item.added          -> function_call id/name
+      #   response.function_call_arguments.delta -> tool-call argument fragment
+      #   response.completed                  -> usage + finish status
+      def parse_codex_stream(raw, stream, model, &block)
+        each_sse_event(raw) do |data|
+          parsed = JSON.parse(data) rescue next
+          chunk =
+            case parsed["type"]
+            when "response.output_text.delta"
+              Ask::Chunk.new(content: parsed["delta"])
+            when "response.output_item.added"
+              item = parsed["item"] || {}
+              next unless item["type"] == "function_call"
+
+              Ask::Chunk.new(tool_calls: [{index: 0, id: item["id"], name: item["name"]}])
+            when "response.function_call_arguments.delta"
+              Ask::Chunk.new(tool_calls: [{index: 0, arguments: parsed["arguments"]}])
+            when "response.completed"
+              response = parsed["response"] || {}
+              usage = response["usage"] || {}
+              Ask::Chunk.new(
+                content: nil,
+                finish_reason: response["status"],
+                usage: {input_tokens: usage["input_tokens"], output_tokens: usage["output_tokens"]}
+              )
+            end
+          next unless chunk
+
+          stream.add(chunk)
+          yield chunk if block_given?
+        end
+      end
+    end
+  end
+end

data/lib/ask-llm-providers.rb

@@ -32,6 +32,7 @@ require_relative "ask/provider/bedrock"
 require_relative "ask/provider/ollama"
 require_relative "ask/provider/mistral"
 require_relative "ask/provider/cloudflare"
+require_relative "ask/provider/openai_codex"
 
 # Register canonical providers
 Ask::Provider.register(:openai, Ask::Providers::OpenAI)
@@ -41,6 +42,7 @@ Ask::Provider.register(:bedrock, Ask::Providers::Bedrock)
 Ask::Provider.register(:ollama, Ask::Providers::Ollama)
 Ask::Provider.register(:mistral, Ask::Providers::Mistral)
 Ask::Provider.register(:cloudflare, Ask::Providers::Cloudflare)
+Ask::Provider.register(:openai_codex, Ask::Providers::OpenaiCodex)
 
 # Register OpenAI-compatible providers from the registry (data → classes)
 Ask::LLM::OPENAI_COMPATIBLE.each do |name, cfg|

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: ask-llm-providers
 version: !ruby/object:Gem::Version
-  version: 0.10.1
+  version: 0.12.0
 platform: ruby
 authors:
 - Kaka Ruto
@@ -186,11 +186,13 @@ files:
 - lib/ask/llm/models/bedrock.json
 - lib/ask/llm/models/deepseek.json
 - lib/ask/llm/models/gemini.json
+- lib/ask/llm/models/github_copilot.json
 - lib/ask/llm/models/meta.json
 - lib/ask/llm/models/mistral.json
 - lib/ask/llm/models/moonshot.json
 - lib/ask/llm/models/nvidia_nim.json
 - lib/ask/llm/models/openai.json
+- lib/ask/llm/models/openai_codex.json
 - lib/ask/llm/models/perplexity.json
 - lib/ask/llm/models/vertex_ai.json
 - lib/ask/llm/models/xai.json
@@ -208,6 +210,7 @@ files:
 - lib/ask/provider/mistral.rb
 - lib/ask/provider/ollama.rb
 - lib/ask/provider/openai.rb
+- lib/ask/provider/openai_codex.rb
 - lib/ask/provider/openai_compatible.rb
 - lib/ask/skills/providers.model_select/SKILL.md
 homepage: https://github.com/ask-rb/ask-llm-providers