ask-llm-providers 0.10.1 → 0.10.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 15fc5c9c298a6f88b722fde13139a1c3fe2dfa795255e8d7226b9ca0b5187bd5
-  data.tar.gz: f9f1c3d677c470c2fffeac5720024ecc9c40fb58cf3b058ac2a99e1e5769100e
+  metadata.gz: 3eae68598ee63c7ef7688b717687b06a434f12c740141a3b84bde44752a873d9
+  data.tar.gz: f21d623f34793e0eff9a08dcb5c1f39c8bba236c541022f1f969a5a273bcb5b7
 SHA512:
-  metadata.gz: 63e97ca4cf04aac0d2e9de99a2d4b6425bd1c95466be13a7172667729830c83b3025fb54094e4a5b20bd1b6b49374bdd1575ef8a60a9eedf3395438a559cc1cf
-  data.tar.gz: 50dfb8a0ca28288df43be9f8126530fa6bddbb6da4d1f951023b4ad7e592298b8c5ed9abb04f53494b65a049421447ba1ef4c02ea594a3af9f7b63176c707598
+  metadata.gz: 87589c9effd3da495a57912d6151d44db8c55673cd74d0e50ba6e0c107e90703905e0e94490e5ae880e5e07d27a2375556837cf7da6917bfa9764f7b1028b49d
+  data.tar.gz: 189cd4cfdfcc19cc1bece8dd6fe561f955dd1e8279a8470cb14549a2d25d05689f06b91687a60920f97237075e5027e287803628579324d98f96a3bd76cffd26

data/CHANGELOG.md

@@ -1,3 +1,14 @@
+## [0.10.2] - 2026-08-06
+
+### Fixed
+
+- **Streamed token usage is now captured.** OpenAI-style streams end with a
+  chunk whose `choices` is empty but `usage` populated; `dig("choices", 0)
+  or next` silently dropped it, so every streamed call reported ~1 output
+  token (input 0) — token billing, cost accounting, and usage metrics
+  under-counted by orders of magnitude. The usage-only chunk is now added
+  to the stream (without yielding — it carries no content).
+
 ## [0.10.1] - 2026-07-31
 
 ### Fixed

data/README.md

@@ -1,20 +1,10 @@
 # ask-llm-providers
 
-All LLM providers for the ask-rb ecosystem in one gem. Implements `Ask::Provider`
-from `ask-core` with a capabilities-based interface.
-
-## Supported Providers
-
-| Provider | Auth | Implementation |
-|---|---|---|
-| **OpenAI** + all OpenAI-compatible | `Ask::Auth.resolve(:openai_api_key)` | `Ask::Providers::OpenAI` |
-| **Anthropic** (Claude) | `Ask::Auth.resolve(:anthropic_api_key)` | `Ask::Providers::Anthropic` |
-| **Google Gemini** | `Ask::Auth.resolve(:gemini_api_key)` | `Ask::Providers::Google` |
-| **Vertex AI** | GCP service account | `Ask::Providers::Google` (via Vertex) |
-| **Amazon Bedrock** | AWS credentials chain | `Ask::Providers::Bedrock` |
-| **Ollama** (local) | None needed | `Ask::Providers::Ollama` |
-| **Mistral AI** | `Ask::Auth.resolve(:mistral_api_key)` | `Ask::Providers::Mistral` |
-| **Cloudflare Workers AI** | `Ask::Auth.resolve(:cloudflare_api_key)` | `Ask::Providers::Cloudflare` |
+All LLM providers for the ask-rb ecosystem in one gem. Implements the
+`Ask::Provider` interface from `ask-core` with a capabilities-based interface:
+7 canonical provider classes plus 26 OpenAI-compatible registry entries,
+a bundled model catalog, and cost calculation. Providers auto-register and
+the model catalog auto-loads when the gem is required.
 
 ## Installation
 
@@ -22,102 +12,86 @@ from `ask-core` with a capabilities-based interface.
 gem "ask-llm-providers"
 ```
 
-## Usage
+## Quick Start
 
 ```ruby
 require "ask-llm-providers"
 
-# All providers are auto-registered with Ask::Models
-models = Ask::Models.find("gpt-4o")
-# => { provider: :openai, capabilities: [...] }
-
-# Use a provider directly
+# Use a provider directly; streaming yields chunks to the block
 provider = Ask::Providers::OpenAI.new
-provider.chat(conversation, tools: [], model: "gpt-4o") do |chunk|
+provider.chat([{ role: "user", content: "Tell me a story" }], model: "gpt-4o") do |chunk|
   print chunk.content
 end
-```
 
-## Capabilities
-
-Each provider and model exposes its capabilities:
-
-```ruby
-provider = Ask::Providers::OpenAI.new
-provider.capabilities
-# => { chat: true, streaming: true, tool_calls: true, vision: true, thinking: true,
-#     :structured_output, :embed, :transcribe, :paint, :moderate]
-
-model = Ask::Models.find("claude-sonnet-4-5")
-model[:capabilities]
-# => { chat: true, streaming: true, tool_calls: true, vision: true, thinking: true, :prompt_caching]
-
-# Unsupported capabilities raise a helpful error
-provider = Ask::Providers::Anthropic.new
-provider.embed(["text"], model: "claude-sonnet-4-5")
-# => Ask::CapabilityNotSupported: Anthropic (claude-sonnet-4-5) does not support embeddings.
+# Look up model metadata (capabilities, pricing, context window)
+model = Ask::ModelCatalog.find("gpt-4o")
+model.capabilities # => ["chat", "streaming", "tool_calls", ...]
 ```
 
+## Supported Providers
 
-
-## Streaming
+| Provider | Auth |
+|---|---|
+| OpenAI | `Ask::Auth.resolve(:openai_api_key)` (env `OPENAI_API_KEY`) |
+| Anthropic (Claude) | `Ask::Auth.resolve(:anthropic_api_key)` (env `ANTHROPIC_API_KEY`) |
+| Google Gemini | `Ask::Auth.resolve(:gemini_api_key)` (env `GEMINI_API_KEY`); Vertex AI via GCP service account |
+| Amazon Bedrock | AWS credentials chain (env, `~/.aws`, instance profile) |
+| Ollama (local) | none needed |
+| Mistral AI | `Ask::Auth.resolve(:mistral_api_key)` (env `MISTRAL_API_KEY`) |
+| Cloudflare Workers AI | `Ask::Auth.resolve(:cloudflare_api_key)` (env `CLOUDFLARE_API_KEY`) |
+| 26 OpenAI-compatible | per-provider `*_API_KEY` env var (e.g. `DEEPSEEK_API_KEY`, `GROQ_API_KEY`, `OPENROUTER_API_KEY`) |
+
+Credentials resolve through `Ask::Auth` in order: environment variables,
+`~/.ask/credentials.yml`, Rails credentials, then database and OAuth
+providers. Keys are read via `Ask::Auth.resolve(:<name>_api_key)`.
+
+The 26 OpenAI-compatible entries are registered from
+`Ask::LLM::OPENAI_COMPATIBLE`: DeepSeek, Groq, Together, Fireworks, Cerebras,
+xAI, Perplexity, DeepInfra, Anyscale, SambaNova, Nebius, Nvidia NIM, Friendli,
+Hyperbolic, Novita, Nscale, Featherless, AI/ML API, AI21, Meta, GitHub Models,
+OpenRouter, OpenCode, OpenCode Go, Mimo, and Moonshot. Each uses its own
+`*_API_KEY` env var; note that `opencode_go` uses `OPENCODE_GO_API_KEY` (its
+own key, not `OPENCODE_API_KEY`).
+
+## Model Catalog
+
+The gem bundles model metadata (capabilities, pricing, context windows,
+modalities) as JSON for 12 providers: openai, anthropic, gemini, vertex_ai,
+bedrock, deepseek, mistral, perplexity, xai, meta, moonshot, and nvidia_nim,
+with 400+ models in total.
 
 ```ruby
-stream = provider.chat(
-  [{ role: "user", content: "Tell me a story" }],
-  model: "gpt-4o",
-  stream: true
-) do |chunk|
-  print chunk.content
-end
-
-# After streaming completes, you can access the full response
-puts stream.accumulated_text
-puts stream.accumulated_usage
+Ask::ModelCatalog.find("claude-sonnet-4-5") # => Ask::ModelInfo
+Ask::ModelCatalog.chat_models               # => filtered catalog
+Ask::ModelCatalog.by_provider(:gemini)
+Ask::ModelCatalog.refresh!                  # fetch latest from models.dev
 ```
 
-## Tool Calls
+## Entry Points
 
-```ruby
-tools = [{
-  name: "get_weather",
-  description: "Get weather for a location",
-  parameters: {
-    type: "object",
-    properties: { location: { type: "string" } },
-    required: ["location"]
-  }
-}]
-
-response = provider.chat(
-  [{ role: "user", content: "What's the weather in NYC?" }],
-  model: "gpt-4o",
-  tools: tools
-)
-# response.tool_call? => true
-# response.tool_calls => [{ id: "call_1", name: "get_weather", arguments: '{"location":"NYC"}' }]
-```
+| API | Purpose |
+|---|---|
+| `Ask::Providers::OpenAI/Anthropic/Google/Bedrock/Ollama/Mistral/Cloudflare` | Canonical provider classes |
+| `Ask::LLM::OPENAI_COMPATIBLE` | Registry data for the 26 compatible providers |
+| `Ask::LLM::Catalog.load!` / `refresh!` | Load bundled + user model data into `Ask::ModelCatalog` |
+| `Ask::LLM::Aliases.resolve("claude-sonnet-4")` | Short-name to canonical model ID resolution |
+| `Ask::LLM::CostCalculator.calculate(model, input_tokens:, output_tokens:)` | USD cost from model pricing |
+| `Ask::Provider.resolve(:openai)` | Class lookup by registered slug |
 
-## Error Handling
+Providers accept a `base_url` override, so any OpenAI-compatible endpoint can
+be used through `Ask::Providers::OpenAI` or `Ask::Providers::OpenAICompatible`.
 
-Provider errors map to structured `Ask::Error` types:
+## Full documentation
 
-```ruby
-Ask::RateLimitError       # 429 — retry with backoff
-Ask::Unauthorized         # 401/403 — check your API key
-Ask::ServerError          # 500 — provider issue
-Ask::ServiceUnavailable   # 503 — temporary
-Ask::ContextLengthExceeded # context window exceeded
-Ask::ProviderError        # other provider errors
-Ask::CapabilityNotSupported # feature not available on this model
-```
+The full ask-rb documentation lives at https://ask-rb.github.io/ask-docs.
+https://ask-rb.github.io/ask-docs/core/providers covers ask-llm-providers in
+depth, including capabilities, streaming, tool calls, and error handling.
+API reference: https://ask-rb.github.io/ask-docs/reference/api.
 
 ## Development
 
-```bash
-bin/setup
+bundle install
 bundle exec rake test
-```
 
 ## License
 

data/lib/ask/llm/http.rb

@@ -1,4 +1,5 @@
 # frozen_string_literal: true
+require "time"
 
 module Ask
   module LLM

data/lib/ask/llm/version.rb

@@ -2,6 +2,6 @@
 
 module Ask
   module LLM
-    VERSION = "0.10.1"
+    VERSION = "0.10.2"
   end
 end

data/lib/ask/provider/openai.rb

@@ -124,7 +124,14 @@ module Ask
       def parse_stream(raw, stream, model, &block)
         each_sse_event(raw) do |data|
           parsed = JSON.parse(data) rescue next
-          choice = parsed.dig("choices", 0) or next
+          choice = parsed.dig("choices", 0)
+          if choice.nil?
+            # Final chunk of an OpenAI-style stream: choices is empty but
+            # usage is populated. Added to the stream (for token
+            # accounting) without yielding — it carries no content.
+            stream.add(Ask::Chunk.new(content: nil, usage: parsed["usage"])) if parsed["usage"]
+            next
+          end
           delta = choice["delta"] || {}
           thinking = extract_thinking(parsed, delta)
           chunk = Ask::Chunk.new(

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: ask-llm-providers
 version: !ruby/object:Gem::Version
-  version: 0.10.1
+  version: 0.10.2
 platform: ruby
 authors:
 - Kaka Ruto