lex-ollama 0.1.0 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: c524d3518516ca7731280a6e84a2b354b12b47b516545a23bec97f6eda373a90
-  data.tar.gz: fbfcff5614ac931e74219661eb792f295dfb65d125c93ddb190b40eae74c6f81
+  metadata.gz: 7477574f919b18b85c79afba3a1f65c8540d9eff9ca02b9e0c807b3740fed452
+  data.tar.gz: a5c69878c8518caf02c2e238c94243fd49c320f20bfaede00252bfdc87be5cbb
 SHA512:
-  metadata.gz: 8975e31624faf65d869fcbe9c512fbf9d35b468bae3b5267726126312ccbe4520f8c5ff339550ce1d2ea44b2561c90d383ae847c67d826e538dd04a2f8a5c2ff
-  data.tar.gz: 4a01f66393c2a78bbd9cccb707ed19156338501f1de46e2afa78508b4be7d93354e90a01a7eef0acc14c18b817d39189d5fc50152da7905ce311d9d537bf0bfe
+  metadata.gz: 31566bf77244dd3cfc097531a3af1da186e8d0e7e0ec675be0b7471f8b7654649fa666d4c3d2f6bb34c46d73d29aa72a64dfa07f7beb35ae01d23c8f2bc6c797
+  data.tar.gz: f900e723d2db75dbdb266fcf33d01be56d7614b992be9e0b6d29345a85012be0d226ff9a2f42cb2d5a9f932cb1e641e6ecbfc033ccd3c6c98bbe4d2a7207ad13

data/CHANGELOG.md

@@ -1,5 +1,36 @@
 # Changelog
 
+## [0.3.0] - 2026-04-01
+
+### Added
+- S3 model distribution via new `Runners::S3Models` module
+- `list_s3_models` to discover models available in an S3 mirror
+- `import_from_s3` for direct filesystem model import (works without Ollama running)
+- `sync_from_s3` for Ollama API-based model import (push_blob + manifest write)
+- `import_default_models` convenience method for fleet provisioning
+- Runtime dependency on `lex-s3` for S3 operations
+- Streaming S3 downloads via `response_target` to avoid loading multi-GB blobs into memory
+- Error propagation in `sync_from_s3` — returns failure with error details when blob push fails
+- SHA256 digest verification for all downloaded blobs (import and sync paths)
+- Atomic blob writes via temp file + rename (prevents partial/corrupt blobs on failure)
+- Cache hits verified by SHA256 digest, not just file size — corrupted local blobs are re-downloaded
+- `DigestMismatchError` raised when S3 blob content does not match manifest digest
+
+## [0.2.0] - 2026-03-31
+
+### Added
+- `Helpers::Errors` — Faraday exception classification (TimeoutError, ConnectionFailed) with exponential backoff retry (`with_retry`, 3 retries, 0.5s base delay)
+- `Helpers::Usage` — standardized usage hash normalization from Ollama response fields (`prompt_eval_count` -> `input_tokens`, `eval_count` -> `output_tokens`, plus duration fields)
+- `Helpers::Client#streaming_client` — Faraday connection without JSON response middleware for streaming endpoints
+- `Runners::Completions#generate_stream` — streaming generate with per-chunk block callback and full text accumulation
+- `Runners::Chat#chat_stream` — streaming chat with per-chunk block callback and full text accumulation
+
+### Changed
+- All runner methods wrapped in `Helpers::Errors.with_retry` for production reliability
+- `Runners::Completions#generate` now returns a `usage:` key with standardized token/duration counts
+- `Runners::Chat#chat` now returns a `usage:` key with standardized token/duration counts
+- `Client` class now overrides `streaming_client` for host passthrough
+
 ## [0.1.0] - 2026-03-31
 
 ### Added

data/Gemfile

@@ -8,5 +8,6 @@ group :test do
   gem 'rspec'
   gem 'rspec_junit_formatter'
   gem 'rubocop'
+  gem 'rubocop-legion'
   gem 'simplecov'
 end

data/README.md

@@ -12,9 +12,11 @@ gem install lex-ollama
 
 ### Completions
 - `generate` - Generate a text completion (POST /api/generate)
+- `generate_stream` - Stream a text completion with per-chunk callbacks
 
 ### Chat
 - `chat` - Generate a chat completion with message history and tool support (POST /api/chat)
+- `chat_stream` - Stream a chat completion with per-chunk callbacks
 
 ### Models
 - `create_model` - Create a model from another model, GGUF, or safetensors (POST /api/create)
@@ -33,6 +35,12 @@ gem install lex-ollama
 - `check_blob` - Check if a blob exists on the server (HEAD /api/blobs/:digest)
 - `push_blob` - Upload a binary blob to the server (POST /api/blobs/:digest)
 
+### S3 Model Distribution
+- `list_s3_models` - List models available in an S3 mirror
+- `import_from_s3` - Download model from S3 directly to Ollama's filesystem (works before Ollama starts)
+- `sync_from_s3` - Download model from S3, push blobs through Ollama's API, write manifest to filesystem
+- `import_default_models` - Import a list of models from S3 (fleet provisioning)
+
 ### Version
 - `server_version` - Retrieve the Ollama server version (GET /api/version)
 
@@ -54,6 +62,55 @@ result = client.embed(model: 'all-minilm', input: 'Some text to embed')
 
 # List models
 result = client.list_models
+
+# Streaming generate
+client.generate_stream(model: 'llama3.2', prompt: 'Tell me a story') do |event|
+  case event[:type]
+  when :delta then print event[:text]
+  when :done  then puts "\nDone!"
+  end
+end
+
+# Streaming chat
+client.chat_stream(model: 'llama3.2', messages: [{ role: 'user', content: 'Hello!' }]) do |event|
+  print event[:text] if event[:type] == :delta
+end
+```
+
+## S3 Model Distribution
+
+Pull models from an internal S3 mirror instead of the public Ollama registry:
+
+```ruby
+client = Legion::Extensions::Ollama::Client.new
+
+# List available models in S3
+client.list_s3_models(bucket: 'legion', endpoint: 'https://mesh.s3api-core.optum.com')
+
+# Import directly to filesystem (works without Ollama running)
+client.import_from_s3(model: 'llama3:latest', bucket: 'legion',
+                      endpoint: 'https://mesh.s3api-core.optum.com')
+
+# Push through Ollama API (requires Ollama running)
+client.sync_from_s3(model: 'llama3:latest', bucket: 'legion',
+                    endpoint: 'https://mesh.s3api-core.optum.com')
+
+# Provision fleet with default models
+client.import_default_models(
+  default_models: %w[llama3:latest nomic-embed-text:latest],
+  bucket: 'legion',
+  endpoint: 'https://mesh.s3api-core.optum.com'
+)
+```
+
+S3 operations use [lex-s3](https://github.com/LegionIO/lex-s3). The S3 bucket should mirror the Ollama models directory structure (`manifests/` and `blobs/` under the configured prefix).
+
+All API calls include automatic retry with exponential backoff on connection failures and timeouts.
+
+Generate and chat responses include standardized `usage:` data:
+```ruby
+result = client.generate(model: 'llama3.2', prompt: 'Hello')
+result[:usage]  # => { input_tokens: 1, output_tokens: 5, total_duration: ..., ... }
 ```
 
 ## Requirements

data/docs/plans/2026-04-01-s3-model-distribution-design.md

@@ -0,0 +1,131 @@
+# S3 Model Distribution for lex-ollama
+
+## Problem
+
+Thousands of engineers pulling models from the public Ollama registry is wasteful and unreliable. Models should be cached in internal S3 and distributed from there. Fleet-wide model updates should be broadcast via RabbitMQ.
+
+## Design
+
+### New Runner: `Runners::S3Models`
+
+A new runner module alongside the existing `Models` runner. Three primary methods plus one convenience method.
+
+#### `import_from_s3` (filesystem write)
+
+Downloads manifest + blobs from S3, writes directly to `~/.ollama/models/`.
+
+```ruby
+import_from_s3(
+  model:,                    # e.g. "llama3:latest"
+  bucket:,                   # S3 bucket name
+  prefix: "ollama/models",   # S3 key prefix
+  models_path: nil,          # local Ollama models dir, defaults to ~/.ollama/models
+  **s3_opts                  # passed through to lex-s3 (endpoint:, region:, access_key_id:, etc.)
+)
+```
+
+Flow:
+1. Parse `model` into `name` + `tag` (default tag: `latest`)
+2. Download manifest from S3: `{prefix}/manifests/registry.ollama.ai/library/{name}/{tag}`
+3. Parse manifest JSON to get the list of blob digests
+4. For each blob, check if it already exists locally with matching SHA256 digest (skip if valid)
+5. Stream blob from S3 to `.tmp` file, verify SHA256, atomic rename to final path
+6. Raise `DigestMismatchError` if any blob fails verification (temp file cleaned up)
+7. Write the manifest file
+8. Return `{ result: true, model:, blobs_downloaded:, blobs_skipped:, status: 200 }`
+
+Best for: provisioning, bootstrapping, when Ollama is not yet running.
+
+#### `sync_from_s3` (Ollama API + filesystem manifest)
+
+Downloads from S3, pushes blobs through Ollama's API, writes manifest to filesystem.
+
+```ruby
+sync_from_s3(
+  model:,
+  bucket:,
+  prefix: "ollama/models",
+  host: nil,                   # Ollama server host
+  models_path: nil,            # local models dir for manifest write
+  **s3_opts                    # passed to lex-s3
+)
+```
+
+Flow:
+1. Parse model, download manifest from S3
+2. For each blob digest, `check_blob` via Ollama API -- skip if already present
+3. Stream blob from S3 to tempfile, verify SHA256 digest
+4. `push_blob` to Ollama API, check return value for success
+5. If any blob fails: return `{ result: false, errors: [...], status: 500 }`
+6. Write manifest to `{models_path}/manifests/registry.ollama.ai/library/{name}/{tag}`
+7. Return `{ result: true, model:, blobs_pushed:, blobs_skipped:, status: 200 }`
+
+Best for: when Ollama is running and you want blob validation through the API.
+
+#### `list_s3_models`
+
+Lists available models in the S3 mirror.
+
+```ruby
+list_s3_models(
+  bucket:,
+  prefix: "ollama/models",
+  **s3_opts
+)
+```
+
+Lists manifest keys under the prefix and parses them into model name/tag pairs.
+
+#### `import_default_models`
+
+Convenience method that reads `default_models` from settings and calls `import_from_s3` for each.
+
+### Settings
+
+```yaml
+legion:
+  ollama:
+    s3:
+      bucket: "legion"
+      prefix: "ollama/models"
+      endpoint: "https://mesh.s3api-core.optum.com"
+      region: "us-east-2"
+    default_models:
+      - "llama3:latest"
+      - "nomic-embed-text:latest"
+    models_path: null  # defaults to ~/.ollama/models, respects OLLAMA_MODELS env var
+```
+
+### Dependency
+
+`lex-ollama.gemspec` adds a runtime dependency on `lex-s3` (`>= 0.1`). The `S3Models` runner uses `Legion::Extensions::S3::Client` for all S3 operations.
+
+### Data Flow
+
+```
+S3 (mesh.s3api-core.optum.com)
+  |
+  | HTTPS (direct, no AMQP)
+  v
+Node: S3Models runner
+  |
+  |-- import_from_s3 --> filesystem write to ~/.ollama/models/
+  |-- sync_from_s3   --> Ollama HTTP API (push_blob + create_model)
+```
+
+Fleet broadcast: publish a message to the `ollama.s3_models` queue (natural LEX runner behavior). Each node picks it up and runs the download independently from S3.
+
+### File Layout
+
+```
+lib/legion/extensions/ollama/
+  runners/
+    models.rb          # existing, unchanged
+    s3_models.rb       # NEW
+  client.rb            # updated to include Runners::S3Models
+
+spec/legion/extensions/ollama/runners/
+  s3_models_spec.rb    # NEW
+```
+
+No changes to existing runner methods or the Helpers::Client module.