ollama-client 1.3.0 → 1.4.0

data/docs/ECOSYSTEM_STRATEGY.md

@@ -0,0 +1,151 @@
+# Ollama Ruby Ecosystem Strategy & Architectural Blueprint
+
+This document defines the overarching architectural strategy, dependency contracts, and governance model for the Ollama Ruby ecosystem. It establishes the foundational guardrails ensuring that as the ecosystem expands across multiple specialized gems, it maintains strict determinism, modularity, and architectural integrity.
+
+---
+
+## 1. Executive Summary & Core Philosophy
+
+The Ollama Ruby ecosystem is designed around a **deterministic kernel** (`ollama-client`) surrounded by **modular companion gems**. 
+
+### The Core Philosophy
+1. **Deterministic Kernel**: `ollama-client` is the canonical source of truth for all low-level transport, retry semantics, connection pooling, raw endpoint definitions, and error taxonomy.
+2. **Composition over Inheritance**: Companion gems (`ollama-openai`, `ollama-observability`, `ollama-stream`, `ollama-agent`, `ollama-rails`) build upon the core client through well-defined public interfaces and callback hooks rather than monkey-patching or redefining internal transport logic.
+3. **Domain Isolation**: Each companion gem encapsulates a single, cohesive domain (e.g., OpenAI compatibility, OpenTelemetry instrumentation, advanced streaming, agent orchestration, Rails integration).
+
+---
+
+## 2. The Dependency Hierarchy & Directionality
+
+To prevent architectural inversion and circular dependencies, all gems in the ecosystem must adhere strictly to the following unidirectional dependency graph:
+
+```
+┌────────────────────────────────────────────────────────┐
+│               ollama-client (Core Kernel)              │
+│    (Transport, Retries, Error Taxonomy, Raw Schemas)   │
+└───────────────────────────▲────────────────────────────┘
+                            │
+         ┌──────────────────┼──────────────────┐
+         │                  │                  │
+┌────────┴────────┐┌────────┴────────┐┌────────┴────────┐
+│  ollama-openai  ││ollama-observab. ││  ollama-stream  │
+│(OpenAI Facade)  ││ (OTel Telemetry)││ (SSE & WebSockets│
+└────────▲────────┘└─────────────────┘└────────▲────────┘
+         │                                     │
+         └──────────────────┬──────────────────┘
+                            │
+                 ┌──────────┴──────────┐
+                 │    ollama-agent     │
+                 │(Tools, Memory, Plan)│
+                 └──────────▲──────────┘
+                            │
+                 ┌──────────┴──────────┐
+                 │    ollama-rails     │
+                 │(ActiveJob, Turbo,   │
+                 │ ActionCable)        │
+                 └─────────────────────┘
+```
+
+### Critical Dependency Rules
+- **Rule 1**: `ollama-client` must NEVER depend on any companion gem or higher-level concept (e.g., agents, Rails, OpenTelemetry).
+- **Rule 2**: Companion gems must declare `ollama-client` as their primary dependency and utilize its public API or hook system.
+- **Rule 3**: Higher-level orchestration gems (`ollama-agent`, `ollama-rails`) may compose multiple lower-level companion gems (e.g., `ollama-agent` utilizing `ollama-stream` for tool streaming or `ollama-openai` for LLM interop).
+
+---
+
+## 3. Companion Gem Specifications & Responsibilities
+
+### 1. `ollama-client` (The Kernel)
+- **Role**: Canonical infrastructure layer.
+- **Responsibilities**: Faraday HTTP/Faraday WebSocket transport, connection pooling, exponential backoff retries, raw API endpoint mapping (`/api/chat`, `/api/generate`, `/api/embeddings`, `/api/pull`), configuration management (`Ollama::Config`), and canonical error taxonomy (`Ollama::Error`, `Ollama::ConnectionError`, `Ollama::TimeoutError`, `Ollama::RateLimitError`).
+- **Prohibited**: High-level workflow orchestration, third-party framework coupling.
+
+### 2. `ollama-openai` (Protocol Interop)
+- **Role**: Drop-in OpenAI compatibility layer.
+- **Responsibilities**: Translating OpenAI-style requests (`client.chat(parameters: {})`) into native Ollama payloads, mapping OpenAI function definitions to Ollama tool schemas, normalizing Ollama responses into OpenAI JSON structures (`chatcmpl-*`), and companion support for frameworks like LangChain, Vercel AI SDK, and ruby-openai consumers.
+- **Prohibited**: Custom HTTP transport implementations.
+
+### 3. `ollama-observability` (Telemetry & Logging)
+- **Role**: Enterprise-grade observability layer.
+- **Responsibilities**: Subscribing to `ollama-client` hooks (`on_response`, `on_token`, `on_error`) to generate OpenTelemetry tracing spans, tracking metrics (Time-to-First-Token, latency histograms, prompt/completion token counters), and emitting structured JSON logs. Supports payload redaction for PII/PHI compliance.
+- **Prohibited**: Modifying raw API responses or interfering with inference execution.
+
+### 4. `ollama-stream` (Advanced Streaming & Transport)
+- **Role**: High-performance streaming runtime.
+- **Responsibilities**: Encapsulating SSE streams into formal `Ollama::Stream::StreamObject` instances supporting pause, resume, and cancellation; managing persistent bidirectional WebSocket sessions; providing backpressure and queue bounding (`FlowController`); implementing incremental JSON fragment recovery (`IncrementalParser`); and offering a Rack-compatible SSE proxy adapter.
+- **Prohibited**: Re-implementing base Faraday connection logic.
+
+### 5. `ollama-agent` (Orchestration & Tooling)
+- **Role**: Autonomous agent execution framework.
+- **Responsibilities**: Defining standardized tool contracts (`Ollama::Agent::Tool`), managing tool registries, maintaining conversation memory (`WindowMemory`, `SummaryMemory`), executing autonomous ReAct/Plan-and-Solve agent loops (`Executor`), and providing structured JSON output parsing.
+- **Prohibited**: Direct HTTP transport management.
+
+### 6. `ollama-rails` (Rails Integration)
+- **Role**: Idiomatic Ruby on Rails integration.
+- **Responsibilities**: Providing Rails Railtie for zero-config initialization, wrapping async inference in ActiveJob (`Ollama::Rails::GenerateJob`, `ChatJob`), broadcasting live stream tokens over ActionCable/Turbo Streams (`Ollama::Rails::BroadcastHelpers`), offering ActiveRecord mixins (`Ollama::Rails::Embeddable` for pgvector/neighbor integration), and providing Rake tasks for model management (`ollama:pull`, `ollama:list`).
+- **Prohibited**: Modifying core Ruby runtime behavior outside the Rails application context.
+
+---
+
+## 4. Architectural Guardrails & Extension Mechanisms
+
+### The Hook System
+To enable companion gems to extend functionality without monkey-patching, `ollama-client` exposes a robust hook and middleware architecture:
+
+```ruby
+# Example of Hook Subscription in Companion Gems
+config.on_response = ->(raw_response, metadata) {
+  # Used by ollama-observability for metrics and logging
+  Telemetry.record_latency(metadata[:duration])
+}
+
+client.chat(
+  model: "llama3",
+  messages: history,
+  hooks: {
+    on_token: ->(chunk, logprobs) { # Used by ollama-stream },
+    on_tool_call: ->(tool_call) { # Used by ollama-agent },
+    on_error: ->(error) { # Used by ollama-observability }
+  }
+)
+```
+
+### Error Handling & Taxonomy
+All companion gems must catch base `Ollama::Error` exceptions from `ollama-client` and either allow them to bubble up or wrap them in domain-specific exceptions that inherit from `Ollama::Error` (e.g., `Ollama::Openai::Error < Ollama::Error`).
+
+---
+
+## 5. Development & Workspace Governance
+
+### Local Dependency Mapping
+During development, all companion gems must reference the local `ollama-client` kernel in their `Gemfile` to ensure continuous integration and contract verification across the workspace:
+
+```ruby
+# Gemfile for Companion Gems
+source "https://rubygems.org"
+gemspec
+
+# Enforce local workspace dependency
+gem "ollama-client", path: "../ollama-client"
+```
+
+### Verification & Testing
+Every companion gem must maintain an independent RSpec test suite verifying its specific domain logic against mock `Ollama::Client` instances. Contract tests must ensure that changes in `ollama-client` do not break companion gem facades.
+
+---
+
+## 6. Roadmap & Future Evolution
+
+1. **Phase 1: Foundation (Completed)**
+   - Core transport, retry semantics, error taxonomy, and configuration in `ollama-client`.
+   - OpenAI interop facade in `ollama-openai`.
+   - Enterprise telemetry in `ollama-observability`.
+   - Advanced streaming primitives in `ollama-stream`.
+
+2. **Phase 2: Orchestration & Frameworks (Current)**
+   - Autonomous agent loops and tool calling in `ollama-agent`.
+   - Idiomatic Rails integrations in `ollama-rails`.
+
+3. **Phase 3: Advanced Optimization & Ecosystem Scaling (Future)**
+   - Native C/Rust extensions for high-throughput incremental JSON parsing.
+   - Distributed multi-node Ollama cluster balancing and routing gems.

data/docs/GETTING_STARTED.md

@@ -7,11 +7,13 @@ This guide shows you step-by-step how to create a client object to use all featu
 ### Option A: Using Bundler (Recommended)
 
 Add to your `Gemfile`:
+
 ```ruby
 gem "ollama-client"
 ```
 
 Then run:
+
 ```bash
 bundle install
 ```
@@ -46,7 +48,7 @@ client = Ollama::Client.new
 
 # Defaults:
 # - base_url: "http://localhost:11434"
-# - model: "llama3.2:3b"
+# - model: "qwen3.5:4b"
 # - timeout: 20 seconds
 # - retries: 2
 # - temperature: 0.2
@@ -82,14 +84,16 @@ To use [Ollama Cloud](https://docs.ollama.com/cloud) models (hosted at ollama.co
 ```ruby
 config = Ollama::Config.new
 config.base_url = "https://ollama.com"
-config.api_key = ENV["OLLAMA_API_KEY"]  # or your API key
+config.api_keys = ENV["OLLAMA_API_KEYS"] # optional comma-separated key pool
+config.api_key = ENV["OLLAMA_API_KEY"] if config.api_keys.empty? # single-key fallback
+config.enable_multi_key_concurrency = Ollama::Config.truthy_env?(ENV["ENABLE_MULTI_KEY_CONCURRENCY"])
 client = Ollama::Client.new(config: config)
 
 # Use a cloud model (e.g. gpt-oss:120b-cloud)
 client.chat(messages: [{ role: "user", content: "Why is the sky blue?" }], model: "gpt-oss:120b-cloud")
 ```
 
-All requests will send `Authorization: Bearer <api_key>` and use HTTPS. The same client works for chat, generate, embeddings, and model listing.
+All requests will send `Authorization: Bearer <api_key>` and use HTTPS. When multiple keys are configured, HTTP 429 responses rotate to the next key; if the full pool remains rate-limited through all retry cycles, the client raises `Ollama::RateLimitExhaustedError`. The same client works for chat, generate, embeddings, and model listing.
 
 ### Option D: Client from Environment Variables
 
@@ -98,7 +102,9 @@ The gem automatically loads `.env` file. You can set these environment variables
 ```bash
 # In your .env file or shell environment
 OLLAMA_BASE_URL=http://localhost:11434
-OLLAMA_API_KEY=your_key_for_ollama_cloud   # optional, for https://ollama.com
+OLLAMA_API_KEY=your_key_for_ollama_cloud   # optional single-key fallback, for https://ollama.com
+OLLAMA_API_KEYS=key_abc123,key_xyz789      # optional multi-key pool, takes precedence
+ENABLE_MULTI_KEY_CONCURRENCY=false        # optional; true/1 round-robins initial keys across threads
 OLLAMA_MODEL=qwen2.5:14b
 OLLAMA_TEMPERATURE=0.1
 ```
@@ -111,7 +117,9 @@ require "ollama_client"
 # Create config and read from environment
 config = Ollama::Config.new
 config.base_url = ENV["OLLAMA_BASE_URL"] if ENV["OLLAMA_BASE_URL"]
-config.api_key = ENV["OLLAMA_API_KEY"] if ENV["OLLAMA_API_KEY"]
+config.api_keys = ENV["OLLAMA_API_KEYS"] if ENV["OLLAMA_API_KEYS"]
+config.api_key = ENV["OLLAMA_API_KEY"] if config.api_keys.empty? && ENV["OLLAMA_API_KEY"]
+config.enable_multi_key_concurrency = Ollama::Config.truthy_env?(ENV["ENABLE_MULTI_KEY_CONCURRENCY"])
 config.model = ENV["OLLAMA_MODEL"] if ENV["OLLAMA_MODEL"]
 config.temperature = ENV["OLLAMA_TEMPERATURE"].to_f if ENV["OLLAMA_TEMPERATURE"]
 
@@ -125,7 +133,7 @@ Create a `config.json` file:
 ```json
 {
   "base_url": "http://localhost:11434",
-  "model": "llama3.2:3b",
+  "model": "qwen3.5:4b",
   "timeout": 30,
   "retries": 3,
   "temperature": 0.2,
@@ -329,7 +337,7 @@ require "ollama_client"
 # Step 2: Create client (using environment variables from .env)
 config = Ollama::Config.new
 config.base_url = ENV["OLLAMA_BASE_URL"] || "http://localhost:11434"
-config.model = ENV["OLLAMA_MODEL"] || "llama3.2:3b"
+config.model = ENV["OLLAMA_MODEL"] || "qwen3.5:4b"
 config.temperature = ENV["OLLAMA_TEMPERATURE"].to_f if ENV["OLLAMA_TEMPERATURE"]
 
 client = Ollama::Client.new(config: config)
@@ -361,7 +369,7 @@ end
 | Option | Default | Description |
 |--------|---------|-------------|
 | `base_url` | `"http://localhost:11434"` | Ollama server URL |
-| `model` | `"llama3.2:3b"` | Default model to use |
+| `model` | `"qwen3.5:4b"` | Default model to use |
 | `timeout` | `20` | Request timeout in seconds |
 | `retries` | `2` | Number of retry attempts on failure |
 | `temperature` | `0.2` | Model temperature (0.0-2.0) |

data/docs/INTEGRATION_TESTING.md

@@ -10,7 +10,7 @@ bundle exec rspec --tag integration
 
 # Run with custom configuration
 OLLAMA_URL=http://localhost:11434 \
-OLLAMA_MODEL=llama3.2:3b \
+OLLAMA_MODEL=qwen3.5:4b \
 bundle exec rspec --tag integration
 ```
 
@@ -27,18 +27,21 @@ Environment variables are documented in the header of `script/live_branch_smoke.
 ## Prerequisites
 
 1. **Ollama server running** (default: `http://localhost:11434`)
+
    ```bash
    # Start Ollama server
    ollama serve
    ```
 
 2. **At least one model installed**
+
    ```bash
    # Install a model
-   ollama pull llama3.2:3b
+   ollama pull qwen3.5:4b
    ```
 
 3. **Optional: Embedding model** (for embedding tests)
+
    ```bash
    ollama pull nomic-embed-text:latest
    ```
@@ -48,6 +51,7 @@ Environment variables are documented in the header of `script/live_branch_smoke.
 Integration tests verify:
 
 ### ✅ Core Client Methods
+
 - `#list_models` - Lists available models
 - `#generate` - Structured JSON output with schema
 - `#generate` - Plain text output without schema
@@ -57,54 +61,62 @@ Integration tests verify:
 - `#chat_raw` - Tool calling (if model supports it)
 
 ### ✅ Embeddings
+
 - Single text embeddings
 - Multiple text embeddings
 - Error handling for missing/unsupported models
 
 ### ✅ Agent Components
+
 - `Ollama::Agent::Planner` - Planning decisions
 - `Ollama::Agent::Executor` - Tool execution loops (if model supports tools)
 
 ### ✅ Chat Session
+
 - Session management
 - Conversation state
 - Message history
 
 ### ✅ Error Handling
+
 - `NotFoundError` for non-existent models
 - Proper error propagation
 
 ## Running Tests
 
 ### Run All Integration Tests
+
 ```bash
 bundle exec rspec --tag integration
 ```
 
 ### Run Specific Test File
+
 ```bash
 bundle exec rspec spec/integration/ollama_client_integration_spec.rb --tag integration
 ```
 
 ### Run Specific Test
+
 ```bash
 bundle exec rspec spec/integration/ollama_client_integration_spec.rb:32 --tag integration
 ```
 
 ### With Environment Variables
+
 ```bash
 # Custom Ollama URL
 OLLAMA_URL=http://remote-server:11434 bundle exec rspec --tag integration
 
 # Custom model
-OLLAMA_MODEL=llama3.2:3b bundle exec rspec --tag integration
+OLLAMA_MODEL=qwen3.5:4b bundle exec rspec --tag integration
 
 # Custom embedding model
 OLLAMA_EMBEDDING_MODEL=nomic-embed-text:latest bundle exec rspec --tag integration
 
 # All together
 OLLAMA_URL=http://localhost:11434 \
-OLLAMA_MODEL=llama3.2:3b \
+OLLAMA_MODEL=qwen3.5:4b \
 OLLAMA_EMBEDDING_MODEL=nomic-embed-text:latest \
 bundle exec rspec --tag integration
 ```
@@ -112,11 +124,13 @@ bundle exec rspec --tag integration
 ## Test Behavior
 
 ### Automatic Skipping
+
 - Tests automatically skip if Ollama server is not available
 - Tests skip if required models are not installed
 - Tests skip if models don't support certain features (e.g., tool calling)
 
 ### Expected Results
+
 - **Passing**: Client correctly communicates with Ollama
 - **Pending/Skipped**: Expected when models/features unavailable
 - **Failing**: Indicates actual client issues (rare)
@@ -143,20 +157,24 @@ bundle exec rspec --tag integration
 ## Troubleshooting
 
 ### "Ollama server not available"
+
 - Start Ollama: `ollama serve`
 - Check URL: `OLLAMA_URL=http://localhost:11434`
 - Verify connection: `curl http://localhost:11434/api/tags`
 
 ### "Model not found"
-- Install model: `ollama pull llama3.2:3b`
+
+- Install model: `ollama pull qwen3.5:4b`
 - Set model: `OLLAMA_MODEL=your-model`
 
 ### "Empty embedding returned"
+
 - Install embedding model: `ollama pull nomic-embed-text:latest`
 - Verify model supports embeddings
 - Set model: `OLLAMA_EMBEDDING_MODEL=nomic-embed-text:latest`
 
 ### "HTTP 400: Bad Request" (tool calling)
+
 - Some models don't support tool calling
 - Test will skip automatically
 - Try a different model that supports tools

data/docs/PRODUCTION_FIXES.md

@@ -9,8 +9,9 @@ This document summarizes all critical fixes applied to make `ollama-client` prod
 **Issue:** LLMs can return JSON wrapped in markdown, prefixed with text, or with unicode garbage.
 
 **Fix:** Enhanced `parse_json_response()` to:
+
 - Handle JSON arrays (not just objects)
-- Strip markdown code fences (```json ... ```)
+- Strip markdown code fences (```json ...```)
 - Normalize unicode and whitespace
 - Extract nested JSON if first attempt fails
 - Better error messages with context
@@ -22,6 +23,7 @@ This document summarizes all critical fixes applied to make `ollama-client` prod
 **Issue:** Need explicit retry rules for different HTTP status codes.
 
 **Fix:** Made retry policy explicit and documented:
+
 - **Retry:** 408 (Request Timeout), 429 (Too Many Requests), 500 (Internal Server Error), 503 (Service Unavailable)
 - **Never retry:** All other 4xx and 5xx errors
 
@@ -32,11 +34,13 @@ This document summarizes all critical fixes applied to make `ollama-client` prod
 **Issue:** Global configuration is not thread-safe, but this wasn't clearly communicated.
 
 **Fix:**
+
 - Added warnings in `OllamaClient.configure` when used from multiple threads
 - Added documentation comments in `Config` class
 - Warns users to use per-client configuration for concurrent agents
 
 **Location:**
+
 - `lib/ollama_client.rb:14-16`
 - `lib/ollama/config.rb:7-15`
 
@@ -45,6 +49,7 @@ This document summarizes all critical fixes applied to make `ollama-client` prod
 **Issue:** Schemas allow extra properties by default, letting LLMs add unexpected fields.
 
 **Fix:** Enforce `additionalProperties: false` by default:
+
 - Automatically adds `additionalProperties: false` to object schemas
 - Recursively applies to nested objects and array items
 - Only if not explicitly set (allows opt-out if needed)
@@ -56,6 +61,7 @@ This document summarizes all critical fixes applied to make `ollama-client` prod
 **Issue:** `chat()` should require explicit opt-in for agent usage.
 
 **Fix:**
+
 - Added `strict:` parameter to `chat()`
 - Warns when `strict: false` (default)
 - In strict mode, doesn't retry on schema violations
@@ -68,6 +74,7 @@ This document summarizes all critical fixes applied to make `ollama-client` prod
 **Issue:** Need a variant that fails fast on schema violations without retries.
 
 **Fix:** Added `generate_strict!()` method:
+
 - No retries on schema violations
 - Immediate failure for guaranteed contract enforcement
 - Useful for strict agent contracts
@@ -79,10 +86,12 @@ This document summarizes all critical fixes applied to make `ollama-client` prod
 **Issue:** No way to track latency, attempts, or model used per request.
 
 **Fix:** Added `include_meta:` parameter to `generate()` and `chat()`:
+
 - Returns `{ "data": ..., "meta": { "latency_ms": ..., "model": ..., "attempts": ... } }`
 - Enables logging, metrics, and debugging
 
 **Location:**
+
 - `lib/ollama/client.rb:58-86` (generate)
 - `lib/ollama/client.rb:27-70` (chat)
 
@@ -91,6 +100,7 @@ This document summarizes all critical fixes applied to make `ollama-client` prod
 **Issue:** No way to verify Ollama server is reachable before making requests.
 
 **Fix:** Added `health()` method:
+
 - Returns `{ status: "healthy|unhealthy", latency_ms: ..., error: ... }`
 - Short timeout (5s) for quick health checks
 - Useful for auto-restart systems
@@ -100,6 +110,7 @@ This document summarizes all critical fixes applied to make `ollama-client` prod
 ## 📊 Impact
 
 These fixes address **~70% of real-world Ollama failures** by:
+
 - Robust JSON extraction (handles model quirks)
 - Explicit retry policies (prevents wasted retries)
 - Strict schemas (catches unexpected fields)
@@ -133,7 +144,7 @@ result = client.generate(
 
 puts result["meta"]["latency_ms"]  # 245.32
 puts result["meta"]["attempts"]     # 1
-puts result["meta"]["model"]       # "llama3.2:3b"
+puts result["meta"]["model"]       # "qwen3.5:4b"
 ```
 
 ### Health Check
@@ -162,11 +173,12 @@ result = client.generate_strict!(
 **Breaking Changes:** None - all changes are backward compatible.
 
 **New Defaults:**
+
 - Schemas now reject extra properties by default (can opt-out by setting `additionalProperties: true`)
 - `chat()` now warns unless `strict: true` is passed
 
 **Recommended Updates:**
+
 - Use `include_meta: true` for production logging
 - Use per-client config for concurrent agents
 - Use `generate_strict!` when you need guaranteed contracts
-

data/docs/QUICK_START.md

@@ -12,7 +12,7 @@ client = Ollama::Client.new
 
 # Or with custom config
 config = Ollama::Config.new
-config.model = ENV["OLLAMA_MODEL"] || "llama3.2:3b"
+config.model = ENV["OLLAMA_MODEL"] || "qwen3.5:4b"
 config.base_url = ENV["OLLAMA_BASE_URL"] || "http://localhost:11434"
 client = Ollama::Client.new(config: config)
 ```

data/docs/README.md

@@ -9,6 +9,7 @@ This directory contains internal development documentation for the ollama-client
 ## Contents
 
 ### Design Documentation
+- **[RUBYLLM_ADOPTION_MATRIX.md](RUBYLLM_ADOPTION_MATRIX.md)** - Feature-by-feature comparison against RubyLLM's public API, with copy/improve/reject/ecosystem verdicts and a prioritized build order
 - **[HANDLERS_ANALYSIS.md](HANDLERS_ANALYSIS.md)** - Analysis of handler architecture decisions (why we didn't adopt ollama-ruby's handler pattern)
 - **[FEATURES_ADDED.md](FEATURES_ADDED.md)** - Features integrated from ollama-ruby that align with our agent-first philosophy
 - **[PRODUCTION_FIXES.md](PRODUCTION_FIXES.md)** - Production-ready fixes for hybrid agents (JSON parsing, retry policy, etc.)