lex-llm-vllm 0.1.7 → 0.1.9

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 3b6bccbfd1d8e01fd38459107474d9ca3853f7d847ff3b5d71a8df3ff7a66c4b
-  data.tar.gz: f2bd935851929d113f078301a08119a425a68c35907094ee66d69d10af3e5f6f
+  metadata.gz: c7c1de4a067bd42d4675c0485f5e13c4d6fe3a1a17c29a2e23c46d266588dd20
+  data.tar.gz: fe503a3a436ef92bcc88015b1d608180d4f98a226cdac4d825c7433c812ee67c
 SHA512:
-  metadata.gz: 837e7ea4d14a09dd44922cb6193e4650b92aea3c4eea8cd85ed7916d766c84b7f8887961b0fb72ab8a1578d4005742f61ed44435d181235bb4f26042aa6aecf8
-  data.tar.gz: 8c73bfdd7921d1f99d788d4a311be574fc7cb9f61c7ebb6a79bdf7ea4a68622f020ace60858134288eea85186beb3d4c32b97c5ac714515a124b8110f3253679
+  metadata.gz: 00bdc87460cf051250b56def2c2a910efe5ff058451a3eff26a7ad1254c5ec9441d3ddb592fa80fd600a83e8479d3593cd1771042e0bacadf0613bb33735ba26
+  data.tar.gz: 7d0df28f8edc25b269f64a987e63de90fe89d098e409bbd2dada20ac8f8b981caec3c046a3eeb591c7edb5c025efc80cd480837fc03b81348ab69e160bef9d2b

data/.rubocop.yml

@@ -12,7 +12,12 @@ Metrics/BlockLength:
   Exclude:
     - "*.gemspec"
     - spec/**/*
+Metrics/ClassLength:
+  Exclude:
+    - lib/legion/extensions/llm/vllm/provider.rb
 Metrics/MethodLength:
   Enabled: false
+RSpec/ExampleLength:
+  Max: 8
 RSpec/MultipleExpectations:
   Enabled: false

data/CHANGELOG.md

@@ -1,5 +1,24 @@
 # Changelog
 
+## 0.1.9 - 2026-04-30
+
+- Adopt base provider contract from lex-llm 0.1.9
+- Replace local `RegistryEventBuilder` and `RegistryPublisher` with parameterized base versions
+- Delete local `transport/` directory; base gem now ships shared exchange and message classes
+- Remove deprecated `Provider.register` call; provider options registered via `Configuration.register_provider_options`
+- Simplify `default_settings` to a flat hash (no longer delegates to `ProviderSettings.build`)
+- Override `parse_list_models_response` to populate `context_length` from vLLM `max_model_len` field
+- Require `lex-llm >= 0.1.9`
+
+## 0.1.8 - 2026-04-30
+
+- Add `Legion::Logging::Helper` to all modules and classes for structured logging
+- Replace all bare rescue blocks with `handle_exception` calls for full observability
+- Add info-level action logging to Provider key actions (health, readiness, list_models, version)
+- Add info-level logging to RegistryPublisher publish methods
+- Remove custom `log_publish_failure` method in favor of standard `handle_exception`
+- Update README to reflect registry publishing, thinking mode, and management endpoints
+
 ## 0.1.7 - 2026-04-30
 
 - Enable stream_usage_supported? for streaming token usage reporting

data/README.md

@@ -1,6 +1,6 @@
 # lex-llm-vllm
 
-LegionIO LLM provider extension for vLLM.
+LegionIO LLM provider extension for [vLLM](https://docs.vllm.ai/).
 
 This gem lives under `Legion::Extensions::Llm::Vllm` and depends on `lex-llm` for shared provider-neutral routing, fleet, and schema primitives.
 
@@ -9,14 +9,17 @@ Load it with `require 'legion/extensions/llm/vllm'`.
 ## What It Provides
 
 - `Legion::Extensions::Llm::Provider` registration as `:vllm`
-- shared `Legion::Extensions::Llm::Provider::OpenAICompatible` request and response handling
-- chat requests through `POST /v1/chat/completions`
-- streaming chat support
-- model discovery through `GET /v1/models`
-- embeddings through `POST /v1/embeddings`
-- vLLM management helpers for `/health`, `/version`, `/reset_prefix_cache`, `/reset_mm_cache`, `/sleep`, and `/wake_up`
-- normalized OpenAI-compatible capability and modality metadata for discovered models
-- shared fleet/default settings via `Legion::Extensions::Llm.provider_settings`
+- Shared `Legion::Extensions::Llm::Provider::OpenAICompatible` request and response handling
+- Chat requests through `POST /v1/chat/completions`
+- Streaming chat with `stream_usage_supported?` for token usage reporting
+- Model discovery through `GET /v1/models`
+- Embeddings through `POST /v1/embeddings`
+- vLLM thinking mode via `chat_template_kwargs` (configurable through `Legion::Settings`)
+- Best-effort `llm.registry` readiness and model availability event publishing when transport is loaded
+- vLLM management helpers: `/health`, `/version`, `/reset_prefix_cache`, `/reset_mm_cache`, `/sleep`, `/wake_up`
+- Normalized OpenAI-compatible capability and modality metadata for discovered models
+- Shared fleet/default settings via `Legion::Extensions::Llm.provider_settings`
+- Full `Legion::Logging::Helper` integration with structured `handle_exception` across all classes
 
 ## Defaults
 
@@ -47,4 +50,47 @@ Legion::Extensions::Llm.configure do |config|
 end
 ```
 
-vLLM's OpenAI-compatible server supports the chat completions, models, and embeddings APIs when the served model and task support them. Chat requests require a model with a chat template; embedding requests require an embedding-capable served model.
+### Thinking Mode
+
+Enable vLLM thinking mode globally via settings:
+
+```ruby
+# In Legion::Settings or settings JSON
+{ llm: { providers: { vllm: { enable_thinking: true } } } }
+```
+
+Or pass `thinking: { enabled: true }` per-request. When enabled, the provider adds `chat_template_kwargs: { enable_thinking: true }` to the payload and strips `reasoning_effort`.
+
+## Management Endpoints
+
+The provider exposes helpers for vLLM server management:
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| `health` | `GET /health` | Server health check |
+| `version` | `GET /version` | Server version info |
+| `reset_prefix_cache` | `POST /reset_prefix_cache` | Clear prefix cache |
+| `reset_mm_cache` | `POST /reset_mm_cache` | Clear multimodal cache |
+| `sleep(level:)` | `POST /sleep` | Put server to sleep |
+| `wake_up(tags:)` | `POST /wake_up` | Wake server up |
+
+## Registry Publishing
+
+When `lex-llm` routing and Legion transport are available, the provider publishes best-effort availability events to the `llm.registry` exchange:
+
+- **Readiness events** on `readiness(live: true)` calls
+- **Model availability events** on `list_models` discovery
+
+Publishing is async (background threads) and never blocks the caller. All failures are handled gracefully via `handle_exception`.
+
+## Development
+
+```bash
+bundle install
+bundle exec rspec
+bundle exec rubocop
+```
+
+## License
+
+MIT

data/lex-llm-vllm.gemspec

@@ -26,5 +26,5 @@ Gem::Specification.new do |spec|
   spec.add_dependency 'legion-json', '>= 1.2.1'
   spec.add_dependency 'legion-logging', '>= 1.3.2'
   spec.add_dependency 'legion-settings', '>= 1.3.14'
-  spec.add_dependency 'lex-llm', '>= 0.1.5'
+  spec.add_dependency 'lex-llm', '>= 0.1.9'
 end

data/lib/legion/extensions/llm/vllm/provider.rb

@@ -10,18 +10,17 @@ module Legion
         # vLLM provider implementation for the Legion::Extensions::Llm base provider contract.
         class Provider < Legion::Extensions::Llm::Provider
           include Legion::Extensions::Llm::Provider::OpenAICompatible
+          include Legion::Logging::Helper
 
           class << self
-            attr_writer :registry_publisher
-
             def slug = 'vllm'
-            def local? = true
+            def local? = false
             def configuration_options = %i[vllm_api_base vllm_api_key]
             def configuration_requirements = []
             def capabilities = Capabilities
 
             def registry_publisher
-              @registry_publisher ||= RegistryPublisher.new
+              Vllm.registry_publisher
             end
           end
 
@@ -66,22 +65,27 @@ module Legion
           def wake_up_url = '/wake_up'
 
           def health
+            log.info { "checking health at #{api_base}#{health_url}" }
             connection.get(health_url).body
           end
 
           def readiness(live: false)
+            log.info { "checking readiness live=#{live} at #{api_base}" }
             super.tap do |metadata|
               self.class.registry_publisher.publish_readiness_async(metadata) if live
             end
           end
 
           def list_models
+            log.info { "discovering models from #{api_base}#{models_url}" }
             super.tap do |models|
+              log.info { "discovered #{models.size} model(s) from vLLM" }
               self.class.registry_publisher.publish_models_async(models, readiness: readiness(live: false))
             end
           end
 
           def version
+            log.info { "fetching version from #{api_base}#{version_url}" }
             connection.get(version_url).body
           end
 
@@ -124,10 +128,27 @@ module Legion
 
             vllm = Legion::Settings.dig(:llm, :providers, :vllm)
             vllm.is_a?(Hash) && (vllm[:enable_thinking] == true || vllm['enable_thinking'] == true)
-          rescue StandardError
+          rescue StandardError => e
+            handle_exception(e, level: :debug, handled: true, operation: 'vllm.thinking_setting')
             false
           end
 
+          def parse_list_models_response(response, provider, capabilities)
+            response.body.fetch('data', []).map do |model|
+              critical_capabilities = critical_capabilities_for(capabilities, model)
+              Legion::Extensions::Llm::Model::Info.from_hash(
+                id: model.fetch('id'),
+                name: model['id'],
+                provider: provider,
+                created_at: model_created_at(model['created']),
+                context_length: model['max_model_len'],
+                capabilities: critical_capabilities,
+                modalities: modalities_for_capabilities(critical_capabilities),
+                metadata: model
+              )
+            end
+          end
+
           def with_query(path, positional = [], **params)
             pairs = positional + params.compact.map { |key, value| [key.to_s, value] }
             return path if pairs.empty?

data/lib/legion/extensions/llm/vllm/version.rb

@@ -4,7 +4,7 @@ module Legion
   module Extensions
     module Llm
       module Vllm
-        VERSION = '0.1.7'
+        VERSION = '0.1.9'
       end
     end
   end

data/lib/legion/extensions/llm/vllm.rb

@@ -1,9 +1,7 @@
 # frozen_string_literal: true
 
 require 'legion/extensions/llm'
-require 'legion/extensions/llm/vllm/registry_event_builder'
 require 'legion/extensions/llm/vllm/provider'
-require 'legion/extensions/llm/vllm/registry_publisher'
 require 'legion/extensions/llm/vllm/version'
 
 module Legion
@@ -12,29 +10,34 @@ module Legion
       # Vllm provider extension namespace.
       module Vllm
         extend ::Legion::Extensions::Core if ::Legion::Extensions.const_defined?(:Core, false)
+        extend Legion::Logging::Helper
 
         PROVIDER_FAMILY = :vllm
 
         def self.default_settings
-          ::Legion::Extensions::Llm.provider_settings(
-            family: PROVIDER_FAMILY,
-            instance: {
-              endpoint: 'http://localhost:8000',
-              tier: :private,
-              transport: :http,
-              usage: { inference: true, embedding: true },
-              limits: { concurrency: 8 }
-            }
-          )
+          {
+            enabled: false,
+            base_url: 'localhost:8000/v1',
+            default_model: nil,
+            enable_thinking: true,
+            model_whitelist: [],
+            model_blacklist: [],
+            model_cache_ttl: 300,
+            tls: { enabled: false, verify: :peer },
+            instances: {}
+          }
         end
 
         def self.provider_class
           Provider
         end
+
+        def self.registry_publisher
+          @registry_publisher ||= Legion::Extensions::Llm::RegistryPublisher.new(provider_family: PROVIDER_FAMILY)
+        end
+
+        Legion::Extensions::Llm::Configuration.register_provider_options(Provider.configuration_options)
       end
     end
   end
 end
-
-Legion::Extensions::Llm::Provider.register(Legion::Extensions::Llm::Vllm::PROVIDER_FAMILY,
-                                           Legion::Extensions::Llm::Vllm::Provider)

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: lex-llm-vllm
 version: !ruby/object:Gem::Version
-  version: 0.1.7
+  version: 0.1.9
 platform: ruby
 authors:
 - LegionIO
@@ -57,14 +57,14 @@ dependencies:
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: 0.1.5
+        version: 0.1.9
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: 0.1.5
+        version: 0.1.9
 description: vLLM provider integration for the LegionIO LLM routing framework.
 email:
 - matthewdiverson@gmail.com
@@ -84,10 +84,6 @@ files:
 - lex-llm-vllm.gemspec
 - lib/legion/extensions/llm/vllm.rb
 - lib/legion/extensions/llm/vllm/provider.rb
-- lib/legion/extensions/llm/vllm/registry_event_builder.rb
-- lib/legion/extensions/llm/vllm/registry_publisher.rb
-- lib/legion/extensions/llm/vllm/transport/exchanges/llm_registry.rb
-- lib/legion/extensions/llm/vllm/transport/messages/registry_event.rb
 - lib/legion/extensions/llm/vllm/version.rb
 homepage: https://github.com/LegionIO/lex-llm-vllm
 licenses:

data/lib/legion/extensions/llm/vllm/registry_event_builder.rb

@@ -1,122 +0,0 @@
-# frozen_string_literal: true
-
-module Legion
-  module Extensions
-    module Llm
-      module Vllm
-        # Builds sanitized lex-llm registry envelopes for vLLM provider state.
-        class RegistryEventBuilder
-          def readiness(readiness)
-            registry_event_class.public_send(
-              readiness[:ready] ? :available : :unavailable,
-              provider_offering(readiness),
-              runtime: runtime_metadata,
-              health: readiness_health(readiness),
-              metadata: readiness_metadata(readiness)
-            )
-          end
-
-          def model_available(model, readiness:)
-            registry_event_class.available(
-              model_offering(model),
-              runtime: runtime_metadata,
-              health: model_health(readiness),
-              metadata: model_metadata(model)
-            )
-          end
-
-          private
-
-          def provider_offering(readiness)
-            {
-              provider_family: :vllm,
-              provider_instance: provider_instance,
-              transport: :http,
-              model: 'provider-readiness',
-              usage_type: :inference,
-              capabilities: [],
-              health: readiness_health(readiness),
-              metadata: { lex: :llm_vllm, provider_readiness: true }
-            }
-          end
-
-          def model_offering(model)
-            {
-              provider_family: :vllm,
-              provider_instance: provider_instance,
-              transport: :http,
-              model: model.id,
-              usage_type: usage_type_for(model),
-              capabilities: Array(model.capabilities).map(&:to_sym),
-              limits: model_limits(model),
-              metadata: { lex: :llm_vllm, model_name: model.name }.compact
-            }
-          end
-
-          def readiness_health(readiness)
-            health = {
-              ready: readiness[:ready] == true,
-              status: readiness[:ready] ? :available : :unavailable,
-              checked: readiness.dig(:health, :checked) != false
-            }
-            add_readiness_error(health, readiness[:health])
-          end
-
-          def add_readiness_error(health, source)
-            error = source.is_a?(Hash) ? source : {}
-            error_class = error[:error] || error['error']
-            error_message = error[:message] || error['message']
-            health[:error_class] = error_class if error_class
-            health[:error] = error_message if error_message
-            health
-          end
-
-          def model_health(readiness)
-            ready = readiness.fetch(:ready, true) == true
-            { ready:, status: ready ? :available : :degraded }
-          end
-
-          def readiness_metadata(readiness)
-            {
-              extension: :lex_llm_vllm,
-              provider: :vllm,
-              configured: readiness[:configured] == true,
-              live: readiness[:live] == true
-            }
-          end
-
-          def model_metadata(model)
-            { extension: :lex_llm_vllm, provider: :vllm, model_type: model.type }
-          end
-
-          def runtime_metadata
-            { node: provider_instance }
-          end
-
-          def model_limits(model)
-            {
-              context_window: model.context_window,
-              max_output_tokens: model.max_output_tokens
-            }.compact
-          end
-
-          def usage_type_for(model)
-            model.type == 'embedding' ? :embedding : :inference
-          end
-
-          def provider_instance
-            configured_node = (::Legion::Settings.dig(:node, :canonical_name) if defined?(::Legion::Settings))
-            value = configured_node.to_s.strip
-            value.empty? ? :vllm : value.to_sym
-          rescue StandardError
-            :vllm
-          end
-
-          def registry_event_class
-            ::Legion::Extensions::Llm::Routing::RegistryEvent
-          end
-        end
-      end
-    end
-  end
-end

data/lib/legion/extensions/llm/vllm/registry_publisher.rb

@@ -1,100 +0,0 @@
-# frozen_string_literal: true
-
-module Legion
-  module Extensions
-    module Llm
-      module Vllm
-        # Best-effort publisher for vLLM provider availability events.
-        class RegistryPublisher
-          APP_ID = 'lex-llm-vllm'
-
-          def initialize(builder: RegistryEventBuilder.new)
-            @builder = builder
-          end
-
-          def publish_readiness_async(readiness)
-            schedule { publish_event(@builder.readiness(readiness)) }
-          end
-
-          def publish_models_async(models, readiness:)
-            schedule do
-              Array(models).each do |model|
-                publish_event(@builder.model_available(model, readiness:))
-              end
-            end
-          end
-
-          private
-
-          def schedule(&)
-            return false unless publishing_available?
-
-            Thread.new do
-              Thread.current.abort_on_exception = false
-              yield
-            rescue StandardError => e
-              log_publish_failure(e, level: :debug)
-            end
-          rescue StandardError => e
-            log_publish_failure(e, level: :debug)
-            false
-          end
-
-          def publish_event(event)
-            return false unless publishing_available?
-
-            message_class.new(event:, app_id: APP_ID).publish(spool: false)
-          rescue StandardError => e
-            log_publish_failure(e)
-            false
-          end
-
-          def publishing_available?
-            return false unless registry_event_available?
-            return false unless transport_message_available?
-            return true unless defined?(::Legion::Transport::Connection)
-            return true unless ::Legion::Transport::Connection.respond_to?(:session_open?)
-
-            ::Legion::Transport::Connection.session_open?
-          rescue StandardError
-            false
-          end
-
-          def registry_event_available?
-            defined?(::Legion::Extensions::Llm::Routing::RegistryEvent)
-          end
-
-          def transport_message_available?
-            return true if message_class_defined?
-            return false unless defined?(::Legion::Transport::Message) && defined?(::Legion::Transport::Exchange)
-
-            require 'legion/extensions/llm/vllm/transport/messages/registry_event'
-            message_class_defined?
-          rescue LoadError
-            false
-          end
-
-          def message_class_defined?
-            defined?(::Legion::Extensions::Llm::Vllm::Transport::Messages::RegistryEvent)
-          end
-
-          def message_class
-            ::Legion::Extensions::Llm::Vllm::Transport::Messages::RegistryEvent
-          end
-
-          def log_publish_failure(error, level: :warn)
-            message = "[lex-llm-vllm] llm.registry publish failed: #{error.class}: #{error.message}"
-            logger = ::Legion::Extensions::Llm.logger if defined?(::Legion::Extensions::Llm)
-            if logger.respond_to?(level)
-              logger.public_send(level, message)
-            elsif logger.respond_to?(:debug)
-              logger.debug(message)
-            end
-          rescue StandardError
-            nil
-          end
-        end
-      end
-    end
-  end
-end

data/lib/legion/extensions/llm/vllm/transport/exchanges/llm_registry.rb

@@ -1,24 +0,0 @@
-# frozen_string_literal: true
-
-module Legion
-  module Extensions
-    module Llm
-      module Vllm
-        module Transport
-          module Exchanges
-            # Topic exchange for vLLM provider availability events.
-            class LlmRegistry < ::Legion::Transport::Exchange
-              def exchange_name
-                'llm.registry'
-              end
-
-              def default_type
-                'topic'
-              end
-            end
-          end
-        end
-      end
-    end
-  end
-end

data/lib/legion/extensions/llm/vllm/transport/messages/registry_event.rb

@@ -1,42 +0,0 @@
-# frozen_string_literal: true
-
-require 'legion/extensions/llm/vllm/transport/exchanges/llm_registry'
-
-module Legion
-  module Extensions
-    module Llm
-      module Vllm
-        module Transport
-          module Messages
-            # Publishes lex-llm RegistryEvent envelopes to the llm.registry exchange.
-            class RegistryEvent < ::Legion::Transport::Message
-              def initialize(event:, **options)
-                super(**event.to_h.merge(options))
-              end
-
-              def exchange
-                Transport::Exchanges::LlmRegistry
-              end
-
-              def routing_key
-                @options[:routing_key] || "llm.registry.#{@options.fetch(:event_type)}"
-              end
-
-              def type
-                'llm.registry.event'
-              end
-
-              def app_id
-                @options[:app_id] || RegistryPublisher::APP_ID
-              end
-
-              def persistent # rubocop:disable Naming/PredicateMethod
-                false
-              end
-            end
-          end
-        end
-      end
-    end
-  end
-end