lex-ollama 0.3.1 → 0.3.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 28df561b00b58c7cb179b9904aed61a5aa7e278140306dadb3b4b2665eaab824
-  data.tar.gz: 446afaab9d80e6a4f62286a1f5ccc1c023bdbb178dba043cb96081412991b2d3
+  metadata.gz: '086cd0d9744893c480b69c7b9fc6a10229af627344d316cafbccbd72d01cdafe'
+  data.tar.gz: e88f7893305401d36d8a3d48a7fb057e831fb6edeb245a5691c3d1e2527e1f1f
 SHA512:
-  metadata.gz: 2915cfe6e4e959e61ee5b8ce68e7da784b4c6001cfe0c3acdb0a4e0f804da79a1e46a17b7c5297b9dd4f26e58bbae504066f5874d8cd82d6ea223b3dfc561bbb
-  data.tar.gz: cb1337292d4bb7c94603612e03dbdbcbd9a41c2a94e56f4bbfad1132d403f6d14b0316091017745ee2d282ccc426bdcdb65b137f66f07f2a505d231792e424b0
+  metadata.gz: f6154fee6005bb96262961342983f01c5d4e5f2747c2cd11a33afbebc1f5ffa00ea64e4f39ef63758620524a5c6690b1a81856c1658371acc040fdd2295c8731
+  data.tar.gz: 0dda701fb81e1f084bc3cc4404d8014d6e2b6ecc68153fcdb42009c10a337f4a078ff11890a4032d95d516a5c1bc5177f4a8e1492db4caea47e9da1641fe6669

data/CHANGELOG.md

@@ -1,5 +1,29 @@
 # Changelog
 
+## [0.3.3] - 2026-04-16
+
+### Added
+- `Actor::ModelSync` — once actor; runs 5s after extension load; reads `legion.ollama.default_models` and `legion.ollama.s3` from settings; calls `import_from_s3` for any configured model not already present on disk; no-op if either setting is absent
+
+### Fixed
+- `Transport::Queues::ModelRequest` deleted — the framework auto-discovers every file in `transport/queues/` and calls `.new` with no arguments at startup, which crashed because `ModelRequest` required `request_type:` and `model:`; the queue definition is now an anonymous class created inline by `Actor::ModelWorker#build_queue_class`
+- `Actor::ModelWorker#queue` now returns a CLASS instead of an instance — `Subscription#initialize` calls `queue.new`, so returning an instance caused a silent `NoMethodError` on `NilClass#new`; the anonymous queue class has `queue_name`, `queue_options`, `dlx_enabled`, and `initialize` (exchange bind) defined inline via `define_method`
+
+## [0.3.2] - 2026-04-08
+
+### Changed
+- `Transport::Exchanges::LlmRequest` now inherits `Legion::LLM::Fleet::Exchange` instead of declaring exchange properties independently — prevents silent divergence if the canonical exchange definition changes
+- `Transport::Queues::ModelRequest` switched from durable quorum queue to classic auto-delete with `x-max-priority: 10` — enables `basic.return` feedback when all workers disconnect; added `dlx_enabled: false` to prevent DLX provisioning on ephemeral queues
+- `Transport::Messages::LlmResponse` now inherits `Legion::LLM::Fleet::Response` instead of `Legion::Transport::Message` — gains wire protocol compliance (`type: 'llm.fleet.response'`, `message_context` propagation, default-exchange publishing, `resp_` prefixed message_id); overrides `app_id` to `'lex-ollama'`
+- `Runners::Fleet#handle_request` now accepts and propagates `message_context` verbatim from request to response; rejects `stream: true` requests with `unsupported_streaming` error; builds full wire protocol response envelope (routing, tokens, timestamps, audit, cost, stop)
+- `Runners::Fleet#publish_reply` switched from positional to keyword arguments; uses `fleet_correlation_id` instead of `correlation_id` to avoid collision with Legion task tracking
+- `Runners::Fleet#dispatch` now resolves Ollama host from `Legion::Settings` instead of using hardcoded default
+- `Actor::ModelWorker` now sets `prefetch(1)` for fair consumer dispatch; reads `consumer_priority` from `legion.ollama.fleet.consumer_priority` settings; passes `x-priority` in `subscribe_options`; injects `message_context: {}` default in `process_message`
+
+### Added
+- `Runners::Fleet#publish_error` — publishes `Legion::LLM::Fleet::Error` to caller's reply_to queue on validation failures (e.g., unsupported streaming)
+- `Runners::Fleet#build_response_body` — constructs wire protocol response body with routing, tokens, timestamps, audit, cost, and stop blocks
+
 ## [0.3.1] - 2026-04-08
 
 ### Added

data/CLAUDE.md

@@ -12,8 +12,8 @@ reporting, and **fleet queue subscription** for receiving routed LLM requests fr
 
 **GitHub**: https://github.com/LegionIO/lex-ollama
 **License**: MIT
-**Version**: 0.3.1
-**Specs**: 82 examples (12 spec files) — fleet additions add ~35 more
+**Version**: 0.3.2
+**Specs**: 166 examples (17 spec files)
 
 ---
 
@@ -38,11 +38,11 @@ Legion::Extensions::Ollama
 ├── Client             # Standalone client class (includes all runners, holds @config)
 ├── Transport/         # (loaded only when Legion::Extensions::Core is present)
 │   ├── Exchanges/
-│   │   └── LlmRequest   # topic exchange 'llm.request'
+│   │   └── LlmRequest   # references Legion::LLM::Fleet::Exchange ('llm.request')
 │   ├── Queues/
-│   │   └── ModelRequest # parametric queue — one per (type, model) pair
+│   │   └── ModelRequest # parametric queue — one per (type, model) pair, auto-delete
 │   └── Messages/
-│       └── LlmResponse  # reply message published back to reply_to
+│       └── LlmResponse  # Legion::LLM::Fleet::Response subclass, reply via default exchange
 └── Actor/
     └── ModelWorker    # subscription actor — one per registered model/type
 ```
@@ -80,9 +80,12 @@ llm.request.ollama.generate.llama3.2
 
 ### Queue Strategy
 
-Each model+type combination gets its own **durable quorum queue** with a routing key that matches
+Each model+type combination gets its own **auto-delete queue** with a routing key that matches
 its queue name exactly. Multiple nodes carrying the same model compete fairly (no SAC) — any
 subscriber can serve. The queue name is identical to the routing key for clarity in the management UI.
+RabbitMQ policies (applied externally via Terraform) set `max-length` and
+`overflow: reject-publish` on `llm.request.*` queues. Queue priority is enabled by declaring
+`x-max-priority: 10` on the queue itself (and may also be mirrored by policy for consistency).
 
 ### Configuration
 
@@ -106,20 +109,24 @@ The extension spawns one `Actor::ModelWorker` per subscription entry at boot.
 ### Data Flow
 
 ```
-Publisher (lex-llm-gateway / any fleet node)
+Publisher (legion-llm Fleet::Dispatcher / any fleet node)
   │  routing_key: "llm.request.ollama.embed.nomic-embed-text"
+  │  AMQP type: 'llm.fleet.request'
+  │  Body includes: message_context { conversation_id, message_id, parent_message_id, message_seq, request_id, exchange_id }
   ▼
 Exchange: llm.request  [topic, durable]
   │
-  └── Queue: llm.request.ollama.embed.nomic-embed-text  [quorum]
+  └── Queue: llm.request.ollama.embed.nomic-embed-text  [auto-delete]
             ▼
        Actor::ModelWorker (type=embed, model=nomic-embed-text)
             ▼
        Runners::Fleet#handle_request
+            │  copies message_context from request
             ▼
        Ollama::Client#embed(model: 'nomic-embed-text', ...)
             ▼
-       Transport::Messages::LlmResponse → reply_to queue (if present)
+       Fleet::Response (type: 'llm.fleet.response') → reply_to queue
+            Body includes: message_context (copied), response_message_id
 ```
 
 ### Standalone Mode (no Legion runtime)
@@ -152,6 +159,34 @@ The gem still works as a pure HTTP client library without AMQP, exactly as befor
 
 ---
 
+## Wire Protocol & Message Classes
+
+Fleet messages inherit from `Legion::LLM::Transport::Message` (defined in legion-llm), which
+extends `Legion::Transport::Message` with `message_context` propagation and LLM-specific headers.
+
+```
+Legion::Transport::Message                (platform base)
+  └── Legion::LLM::Transport::Message     (LLM base — message_context, llm_headers)
+       ├── Legion::LLM::Fleet::Request    (type: 'llm.fleet.request',  app_id: 'legion-llm')
+       ├── Legion::LLM::Fleet::Response   (type: 'llm.fleet.response', app_id: 'lex-ollama')
+       └── Legion::LLM::Fleet::Error      (type: 'llm.fleet.error',    app_id: 'lex-ollama')
+```
+
+Every fleet message carries `message_context` in the body for end-to-end tracing:
+```
+message_context:
+  conversation_id, message_id, parent_message_id, message_seq, request_id, exchange_id
+```
+
+A subset (`conversation_id`, `message_id`, `request_id`) is promoted to AMQP headers
+(`x-legion-llm-conversation-id`, etc.) for filtering without body parsing.
+
+The wire protocol spec (AMQP property mapping, platform-wide standard, per-message-type
+specifications) was developed during the fleet design phase and is maintained in the
+legion-llm repository alongside the implementation.
+
+---
+
 ## Dependencies
 
 | Gem | Purpose |
@@ -159,8 +194,9 @@ The gem still works as a pure HTTP client library without AMQP, exactly as befor
 | `faraday` >= 2.0 | HTTP client for Ollama REST API |
 | `lex-s3` >= 0.2 | S3 model distribution operations |
 
-Fleet transport requires Legion runtime gems (`legion-transport`, `LegionIO`) but those are *not*
-gemspec dependencies — they are expected to be present in the runtime environment.
+Fleet transport requires Legion runtime gems (`legion-transport`, `legion-llm`, `LegionIO`) but
+those are *not* gemspec dependencies — they are expected to be present in the runtime environment.
+`legion-llm` is needed for fleet message classes (`Legion::LLM::Fleet::Request`, etc.).
 
 ---
 
@@ -175,4 +211,4 @@ bundle exec rubocop
 ---
 
 **Maintained By**: Matthew Iverson (@Esity)
-**Last Updated**: 2026-04-07
+**Last Updated**: 2026-04-10

data/README.md

@@ -44,6 +44,36 @@ gem install lex-ollama
 ### Version
 - `server_version` - Retrieve the Ollama server version (GET /api/version)
 
+### Fleet Queue Subscription
+- `handle_request` - Dispatch inbound fleet AMQP messages to the appropriate runner (chat/embed/generate)
+
+When `Legion::Extensions::Core` is present, lex-ollama subscribes to model-scoped queues on the
+`llm.request` topic exchange, accepting routed LLM inference work from other Legion fleet members.
+
+Each configured `(type, model)` pair gets its own auto-delete queue with routing key
+`llm.request.ollama.<type>.<model>`. Multiple nodes serving the same model compete fairly
+via RabbitMQ round-robin with consumer priority.
+
+```yaml
+legion:
+  ollama:
+    host: "http://localhost:11434"
+    fleet:
+      consumer_priority: 10        # H100: 10, Mac Studio: 5, MacBook: 1
+    subscriptions:
+      - type: embed
+        model: nomic-embed-text
+      - type: chat
+        model: "qwen3.5:27b"
+```
+
+Fleet messages use the wire protocol defined in `legion-llm`: typed AMQP messages
+(`llm.fleet.request` / `llm.fleet.response` / `llm.fleet.error`) with `message_context`
+propagation for end-to-end tracing.
+
+Without `Legion::Extensions::Core`, the gem works as a pure HTTP client library with no
+AMQP dependency.
+
 ## Standalone Client
 
 ```ruby
@@ -85,21 +115,21 @@ Pull models from an internal S3 mirror instead of the public Ollama registry:
 client = Legion::Extensions::Ollama::Client.new
 
 # List available models in S3
-client.list_s3_models(bucket: 'legion', endpoint: 'https://mesh.s3api-core.optum.com')
+client.list_s3_models(bucket: 'legion', endpoint: 'https://s3.example.internal')
 
 # Import directly to filesystem (works without Ollama running)
 client.import_from_s3(model: 'llama3:latest', bucket: 'legion',
-                      endpoint: 'https://mesh.s3api-core.optum.com')
+                      endpoint: 'https://s3.example.internal')
 
 # Push through Ollama API (requires Ollama running)
 client.sync_from_s3(model: 'llama3:latest', bucket: 'legion',
-                    endpoint: 'https://mesh.s3api-core.optum.com')
+                    endpoint: 'https://s3.example.internal')
 
 # Provision fleet with default models
 client.import_default_models(
   default_models: %w[llama3:latest nomic-embed-text:latest],
   bucket: 'legion',
-  endpoint: 'https://mesh.s3api-core.optum.com'
+  endpoint: 'https://s3.example.internal'
 )
 ```
 
@@ -121,7 +151,7 @@ result[:usage]  # => { input_tokens: 1, output_tokens: 5, total_duration: ..., .
 
 ## Version
 
-0.3.1
+0.3.2
 
 ## License
 

data/lib/legion/extensions/ollama/actors/model_sync.rb

@@ -0,0 +1,90 @@
+# frozen_string_literal: true
+
+module Legion
+  module Extensions
+    module Ollama
+      module Actor
+        # Once actor — runs once shortly after extension load.
+        # Reads legion.ollama.s3 and legion.ollama.default_models from settings
+        # and calls import_from_s3 for any model not already present locally.
+        #
+        # Settings example:
+        #   {
+        #     "legion": {
+        #       "ollama": {
+        #         "s3": {
+        #           "bucket": "legion",
+        #           "prefix": "ollama/models",
+        #           "endpoint": "https://s3.example.internal"
+        #         },
+        #         "default_models": ["qwen3.5:4b", "nomic-embed-text:latest"]
+        #       }
+        #     }
+        #   }
+        class ModelSync < Legion::Extensions::Actors::Once
+          include Legion::Logging::Helper
+
+          # Run 5 seconds after extension load to allow the rest of startup to complete.
+          def delay
+            5.0
+          end
+
+          def use_runner?
+            false
+          end
+
+          def runner_class
+            self.class
+          end
+
+          def enabled?
+            return false unless defined?(Legion::Settings)
+
+            models = Legion::Settings.dig(:ollama, :default_models)
+            s3_cfg = Legion::Settings.dig(:ollama, :s3)
+            models.is_a?(Array) && !models.empty? && s3_cfg.is_a?(Hash) && s3_cfg[:bucket]
+          rescue StandardError => e
+            handle_exception(e, level: :warn, handled: true)
+            false
+          end
+
+          def manual
+            models = Legion::Settings.dig(:ollama, :default_models) || []
+            s3_cfg = Legion::Settings.dig(:ollama, :s3)
+            bucket = s3_cfg[:bucket]
+            s3_opts = s3_cfg.except(:bucket)
+
+            client = Object.new.extend(Legion::Extensions::Ollama::Runners::S3Models)
+            models_path = ENV.fetch('OLLAMA_MODELS', File.join(Dir.home, '.ollama', 'models'))
+
+            models.each do |model|
+              if model_present_locally?(model, models_path)
+                log.debug "[ModelSync] #{model} already present locally, skipping"
+                next
+              end
+
+              log.info "[ModelSync] importing #{model} from S3"
+              result = client.import_from_s3(model: model, bucket: bucket, models_path: models_path, **s3_opts)
+              if result[:status] == 200
+                log.info "[ModelSync] imported #{model} (blobs_downloaded=#{result[:blobs_downloaded]}, blobs_skipped=#{result[:blobs_skipped]})"
+              else
+                log.warn "[ModelSync] failed to import #{model}: #{result.inspect}"
+              end
+            rescue StandardError => e
+              handle_exception(e, level: :error, handled: true, model: model)
+            end
+          end
+
+          private
+
+          def model_present_locally?(model, models_path)
+            name, tag = model.split(':')
+            tag ||= 'latest'
+            manifest = File.join(models_path, 'manifests', 'registry.ollama.ai', 'library', name, tag)
+            File.exist?(manifest)
+          end
+        end
+      end
+    end
+  end
+end

data/lib/legion/extensions/ollama/actors/model_worker.rb

@@ -11,6 +11,8 @@ module Legion
         #
         #   legion:
         #     ollama:
+        #       fleet:
+        #         consumer_priority: 10
         #       subscriptions:
         #         - type: embed
         #           model: nomic-embed-text
@@ -43,34 +45,73 @@ module Legion
             false
           end
 
-          # Override queue to return a model-scoped queue bound with the precise
-          # routing key for this worker's (type, model) pair.
+          # prefetch(1) is required for consumer priority to work correctly:
+          # without it, a high-priority consumer can hold multiple messages while
+          # lower-priority consumers sit idle. With prefetch=1, each consumer
+          # completes one message before RabbitMQ delivers the next, and priority
+          # determines which idle consumer gets it.
+          def prefetch
+            1
+          end
+
+          # Consumer priority from settings. Tells RabbitMQ to prefer this consumer
+          # over lower-priority ones on the same queue when multiple consumers are idle.
+          # Standard scale: GPU server = 10, Mac Studio = 5, developer laptop = 1.
+          # Defaults to 0 (equal priority) if not configured.
+          def consumer_priority
+            return 0 unless defined?(Legion::Settings)
+
+            Legion::Settings.dig(:ollama, :fleet, :consumer_priority) || 0
+          end
+
+          # Subscribe options include x-priority argument so RabbitMQ can honour
+          # consumer priority when dispatching to competing consumers.
+          def subscribe_options
+            base = begin
+              super
+            rescue NoMethodError
+              {}
+            end
+            base.merge(arguments: { 'x-priority' => consumer_priority })
+          end
+
+          # Returns a queue CLASS (not instance) bound to the llm.request exchange
+          # with the routing key for this worker's (type, model) pair.
+          # The Subscription base class calls queue.new in initialize, so this must
+          # return a class, not an instance.
           def queue
-            @queue ||= build_and_bind_queue
+            @queue ||= build_queue_class
           end
 
           # Enrich every inbound message with the worker's own request_type and model
-          # so Runners::Fleet#handle_request always has them, even if the sender omitted them.
+          # so Runners::Fleet#handle_request always has them, even if the sender omitted
+          # them. Also defaults message_context to {} if absent.
           def process_message(payload, metadata, delivery_info)
             msg = super
-            msg[:request_type] ||= @request_type
-            msg[:model]        ||= @model_name
+            msg[:request_type]    ||= @request_type
+            msg[:model]           ||= @model_name
+            msg[:message_context] ||= {}
             msg
           end
 
           private
 
-          def build_and_bind_queue
+          def build_queue_class
             sanitised_model = @model_name.tr(':', '.')
             routing_key     = "llm.request.ollama.#{@request_type}.#{sanitised_model}"
+            exchange_class  = Transport::Exchanges::LlmRequest
 
-            queue_obj = Transport::Queues::ModelRequest.new(
-              request_type: @request_type,
-              model:        @model_name
-            )
-            exchange_obj = Transport::Exchanges::LlmRequest.new
-            queue_obj.bind(exchange_obj, routing_key: routing_key)
-            queue_obj
+            Class.new(Legion::Transport::Queue) do
+              define_method(:queue_name) { routing_key }
+              define_method(:queue_options) do
+                { durable: false, auto_delete: true, arguments: { 'x-max-priority' => 10 } }
+              end
+              define_method(:dlx_enabled) { false }
+              define_method(:initialize) do
+                super()
+                bind(exchange_class.new, routing_key: routing_key)
+              end
+            end
           end
         end
       end