legion-llm 0.3.8 → 0.3.11

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b7bbfc4cc4f4d111b0407d6adc2cd070a53f139db1997dd035f423aeb8a156a6
-  data.tar.gz: 4424d53100199b49da45209cd3b240eb45a6c2a1298f142f22a1b0174bca585d
+  metadata.gz: 61c5173e4490643fbecb7977697159ffdeb082b63ec4140289128c69efd14806
+  data.tar.gz: c42e9f24e2ecc387c1076fe32f05b4636e17e9fe92b16bf7ed438198caaa3187
 SHA512:
-  metadata.gz: d9be18e398e238a82ebefe863f7eb6e29c8de94892044d9767116febe7087478f7e817d051a9dbd25fb2752062c40b341dc54ba8b55c22d0b02570e89d367e6a
-  data.tar.gz: 56e717c13a756e6f7320956faf1ea3ad6f18ce26788319a4153f5bf97fe843c939cb8a618aba838a1e5a79bfb9c0d6f6f2ff8f9121f8f3564970cd7b4e0dae4e
+  metadata.gz: b05c1c69d88184ef4ceea383523c0b6538fd363a1cee9d12bf849ab2885bfdfe7fd59170fe059458c89378f66b93851ec69803ae03af6df1a3bbb55ab1aa432c
+  data.tar.gz: 57eab8217bdbc614a8602b45836f4b38bf099d9fc2e6b85cfbd92813856b837387e0901ac06af43085cfe0cd034146d160b05cd358afeb45c03ac628280ce9d6

data/CHANGELOG.md

@@ -1,5 +1,27 @@
 # Legion LLM Changelog
 
+## [0.3.11] - 2026-03-20
+
+### Added
+- `Legion::LLM::Hooks` module with before/after chat hook registry
+- `Hooks.before_chat` and `Hooks.after_chat` for registering interceptor blocks
+- `Hooks.run_before` and `Hooks.run_after` with `:block` action support for guardrail enforcement
+- `Hooks.reset!` for test isolation
+- Before/after hook invocation wired into `_dispatch_chat` for transparent request interception
+
+## [0.3.10] - 2026-03-20
+
+### Added
+- `PrivacyModeError` raised when cloud LLM tier is used with `enterprise_data_privacy` enabled
+- `assert_cloud_allowed!` guard in `chat_single` and `ask_direct` blocks cloud-tier dispatch
+- `Router.tier_available?(:cloud)` returns false when enterprise privacy mode is active
+- Cloud provider detection covers bedrock, anthropic, openai, gemini, and azure
+
+## [0.3.9] - 2026-03-20
+
+### Added
+- OpenInference OTel span wrapping for chat, embed, and structured methods
+
 ## [0.3.8] - 2026-03-20
 
 ### Added

data/CLAUDE.md

@@ -8,7 +8,7 @@
 Core LegionIO gem providing LLM capabilities to all extensions. Wraps ruby_llm to provide a consistent interface for chat, embeddings, tool use, and agents across multiple providers (Bedrock, Anthropic, OpenAI, Gemini, Ollama). Includes a dynamic weighted routing engine that dispatches requests across local, fleet, and cloud tiers based on caller intent, priority rules, time schedules, cost multipliers, and real-time provider health.
 
 **GitHub**: https://github.com/LegionIO/legion-llm
-**Version**: 0.3.6
+**Version**: 0.3.8
 **License**: Apache-2.0
 
 ## Architecture
@@ -31,8 +31,12 @@ Legion::LLM.start
 ```
 Legion::LLM (lib/legion/llm.rb)
 ├── EscalationExhausted # Raised when all escalation attempts are exhausted
+├── DaemonDeniedError   # Raised when daemon returns HTTP 403
+├── DaemonRateLimitedError # Raised when daemon returns HTTP 429
 ├── Settings         # Default config, provider settings, routing defaults, discovery defaults
-├── Providers        # Provider configuration and Vault credential resolution
+├── Providers        # Provider configuration and Vault credential resolution (includes Azure `configure_azure`)
+├── DaemonClient     # HTTP routing to LegionIO daemon with 30s health cache
+├── ResponseCache    # Async response delivery via memcached with spool overflow
 ├── Compressor       # Deterministic prompt compression (3 levels, code-block-aware)
 ├── Discovery        # Runtime introspection for local model availability and system resources
 │   ├── Ollama       # Queries Ollama /api/tags for pulled models (TTL-cached)
@@ -128,6 +132,9 @@ Legion::LLM.shutdown                 # Cleanup
 Legion::LLM.started?                 # -> Boolean
 Legion::LLM.settings                 # -> Hash
 
+# One-shot convenience (daemon-first, direct fallback)
+Legion::LLM.ask(message, model:, provider:)                 # -> Hash with :content key; raises DaemonDeniedError/DaemonRateLimitedError
+
 # Chat (delegates to gateway when loaded, otherwise direct)
 Legion::LLM.chat(message: 'hello', model:, provider:)       # Gateway-metered if available
 Legion::LLM.chat(intent: { privacy: :strict })              # Intent-based routing
@@ -196,7 +203,8 @@ When no defaults are configured, the first enabled provider is used:
 2. Anthropic -> `claude-sonnet-4-6`
 3. OpenAI -> `gpt-4o`
 4. Gemini -> `gemini-2.0-flash`
-5. Ollama -> `llama3`
+5. Azure -> (endpoint-specific, from `api_base`)
+6. Ollama -> `llama3`
 
 ### Routing Settings
 
@@ -293,6 +301,9 @@ In-memory signal consumer with pluggable handlers. Adjusts effective priorities
 | `lib/legion/llm/settings.rb` | Default settings including routing_defaults, auto-merge into Legion::Settings |
 | `lib/legion/llm/providers.rb` | Provider config, Vault resolution, RubyLLM configuration |
 | `lib/legion/llm/bedrock_bearer_auth.rb` | Monkey-patch for Bedrock Bearer Token auth — required lazily |
+| `lib/legion/llm/claude_config_loader.rb` | Import Claude CLI config from `~/.claude/settings.json` and `~/.claude.json` |
+| `lib/legion/llm/response_cache.rb` | Async response delivery via memcached with spool overflow at 8MB |
+| `lib/legion/llm/daemon_client.rb` | HTTP routing to LegionIO daemon with health caching (30s TTL) |
 | `lib/legion/llm/compressor.rb` | Deterministic prompt compression: 3 levels, code-block-aware, stopword removal |
 | `lib/legion/llm/router.rb` | Router module: resolve, health_tracker, select_candidates pipeline |
 | `lib/legion/llm/router/resolution.rb` | Value object: tier, provider, model, rule, metadata, compress_level |
@@ -303,7 +314,7 @@ In-memory signal consumer with pluggable handlers. Adjusts effective priorities
 | `lib/legion/llm/embeddings.rb` | Embeddings module: generate, generate_batch, default_model |
 | `lib/legion/llm/shadow_eval.rb` | Shadow evaluation: enabled?, should_sample?, evaluate, compare |
 | `lib/legion/llm/structured_output.rb` | JSON schema enforcement with native response_format and prompt fallback |
-| `lib/legion/llm/version.rb` | Version constant (0.3.6) |
+| `lib/legion/llm/version.rb` | Version constant (0.3.8) |
 | `lib/legion/llm/quality_checker.rb` | QualityChecker module with QualityResult struct |
 | `lib/legion/llm/escalation_history.rb` | EscalationHistory mixin: `escalation_history`, `escalated?`, `final_resolution`, `escalation_chain` |
 | `lib/legion/llm/router/escalation_chain.rb` | EscalationChain value object |

data/README.md

@@ -2,7 +2,7 @@
 
 LLM integration for the [LegionIO](https://github.com/LegionIO/LegionIO) framework. Wraps [ruby_llm](https://github.com/crmne/ruby_llm) to provide chat, embeddings, tool use, and agent capabilities to any Legion extension.
 
-**Version**: 0.3.6
+**Version**: 0.3.8
 
 ## Installation
 
@@ -60,6 +60,7 @@ Provider-specific fields:
 | Provider | Additional Fields |
 |----------|------------------|
 | **Bedrock** | `secret_key`, `session_token`, `region` (default: `us-east-2`), `bearer_token` (alternative to SigV4 — for AWS Identity Center/SSO) |
+| **Azure** | `api_base` (Azure OpenAI endpoint URL, required), `auth_token` (bearer token alternative to `api_key`) |
 | **Ollama** | `base_url` (default: `http://localhost:11434`) |
 
 ### Credential Resolution
@@ -90,7 +91,8 @@ If no `default_model` or `default_provider` is set, legion-llm auto-detects from
 | 2 | Anthropic | `claude-sonnet-4-6` |
 | 3 | OpenAI | `gpt-4o` |
 | 4 | Gemini | `gemini-2.0-flash` |
-| 5 | Ollama | `llama3` |
+| 5 | Azure | (endpoint-specific) |
+| 6 | Ollama | `llama3` |
 
 ## Core API
 
@@ -103,6 +105,32 @@ Legion::LLM.started?     # -> Boolean
 Legion::LLM.settings     # -> Hash (current LLM settings)
 ```
 
+### One-Shot Ask
+
+`Legion::LLM.ask` is a convenience method for single-turn requests. It routes daemon-first (via the LegionIO REST API if running and configured) and falls back to direct RubyLLM:
+
+```ruby
+# Synchronous response
+response = Legion::LLM.ask("What is the capital of France?")
+puts response[:content]
+
+# The daemon path returns cached (HTTP 200), synchronous (HTTP 201), or async (HTTP 202) responses
+# HTTP 403 raises DaemonDeniedError; HTTP 429 raises DaemonRateLimitedError
+```
+
+Configure daemon routing under `llm.daemon`:
+
+```json
+{
+  "llm": {
+    "daemon": {
+      "enabled": true,
+      "url": "http://127.0.0.1:4567"
+    }
+  }
+}
+```
+
 ### Chat
 
 Returns a `RubyLLM::Chat` instance for multi-turn conversation:
@@ -266,8 +294,7 @@ legion-llm includes a dynamic weighted routing engine that dispatches requests a
 │          Zero network overhead, no Transport              │
 │                                                          │
 │  Tier 2: FLEET  → Ollama on Mac Studios / GPU servers    │
-│          Via Legion::Transport (AMQP) when local can't   │
-│          serve the model (Phase 2, not yet built)        │
+│          Via lex-llm-gateway RPC over AMQP               │
 │                                                          │
 │  Tier 3: CLOUD  → Bedrock / Anthropic / OpenAI / Gemini │
 │          Existing provider API calls                     │
@@ -277,7 +304,7 @@ legion-llm includes a dynamic weighted routing engine that dispatches requests a
 | Tier | Target | Use Case |
 |------|--------|----------|
 | `local` | Ollama on localhost | Privacy-sensitive, offline, or low-latency workloads |
-| `fleet` | Shared hardware via Legion::Transport | Larger models on dedicated GPU servers (Phase 2) |
+| `fleet` | Shared hardware via lex-llm-gateway (AMQP) | Larger models on dedicated GPU servers |
 | `cloud` | API providers (Bedrock, Anthropic, OpenAI, Gemini) | Frontier models, full-capability inference |
 
 #### Intent-Based Dispatch
@@ -566,6 +593,7 @@ end
 | Anthropic | `anthropic` | `vault://`, `env://`, or direct | Direct API access |
 | OpenAI | `openai` | `vault://`, `env://`, or direct | GPT models |
 | Google Gemini | `gemini` | `vault://`, `env://`, or direct | Gemini models |
+| Azure AI | `azure` | `vault://`, `env://`, or direct | Azure OpenAI endpoint; `api_base` + `api_key` or `auth_token` |
 | Ollama | `ollama` | Local, no credentials needed | Local inference |
 
 ## Integration with LegionIO

data/lib/legion/llm/hooks.rb

@@ -0,0 +1,45 @@
+# frozen_string_literal: true
+
+module Legion
+  module LLM
+    module Hooks
+      @before_chat = []
+      @after_chat = []
+
+      class << self
+        def before_chat(&block)
+          @before_chat << block
+        end
+
+        def after_chat(&block)
+          @after_chat << block
+        end
+
+        def run_before(messages:, model:, **)
+          @before_chat.each do |hook|
+            result = hook.call(messages: messages, model: model, **)
+            return result if result.is_a?(Hash) && result[:action] == :block
+          end
+          nil
+        rescue StandardError
+          nil
+        end
+
+        def run_after(response:, messages:, model:, **)
+          @after_chat.each do |hook|
+            result = hook.call(response: response, messages: messages, model: model, **)
+            return result if result.is_a?(Hash) && result[:action] == :block
+          end
+          nil
+        rescue StandardError
+          nil
+        end
+
+        def reset!
+          @before_chat = []
+          @after_chat = []
+        end
+      end
+    end
+  end
+end

data/lib/legion/llm/router.rb

@@ -61,7 +61,9 @@ module Legion
         # :fleet — available when Legion::Transport is loaded
         # :cloud — always available
         def tier_available?(tier)
-          return Legion.const_defined?('Transport') if tier.to_sym == :fleet
+          sym = tier.to_sym
+          return false if sym == :cloud && privacy_mode?
+          return Legion.const_defined?('Transport') if sym == :fleet
 
           true
         end
@@ -163,6 +165,14 @@ module Legion
           {}
         end
 
+        def privacy_mode?
+          if Legion.const_defined?('Settings') && Legion::Settings.respond_to?(:enterprise_privacy?)
+            Legion::Settings.enterprise_privacy?
+          else
+            ENV['LEGION_ENTERPRISE_PRIVACY'] == 'true'
+          end
+        end
+
         def pick_best(candidates)
           return nil if candidates.empty?
 

data/lib/legion/llm/version.rb

@@ -2,6 +2,6 @@
 
 module Legion
   module LLM
-    VERSION = '0.3.8'
+    VERSION = '0.3.11'
   end
 end

data/lib/legion/llm.rb

@@ -8,6 +8,7 @@ require 'legion/llm/router'
 require 'legion/llm/compressor'
 require 'legion/llm/quality_checker'
 require 'legion/llm/escalation_history'
+require 'legion/llm/hooks'
 require_relative 'llm/response_cache'
 require_relative 'llm/daemon_client'
 
@@ -22,6 +23,7 @@ module Legion
     class EscalationExhausted < StandardError; end
     class DaemonDeniedError < StandardError; end
     class DaemonRateLimitedError < StandardError; end
+    class PrivacyModeError < StandardError; end
 
     class << self
       include Legion::LLM::Providers
@@ -64,15 +66,18 @@ module Legion
       # for automatic metering and fleet dispatch
       def chat(model: nil, provider: nil, intent: nil, tier: nil, escalate: nil,
                max_escalations: nil, quality_check: nil, message: nil, **)
-        if gateway_loaded? && message
-          return gateway_chat(model: model, provider: provider, intent: intent,
-                              tier: tier, message: message, escalate: escalate,
-                              max_escalations: max_escalations, quality_check: quality_check, **)
+        if defined?(Legion::Telemetry::OpenInference)
+          Legion::Telemetry::OpenInference.llm_span(
+            model: (model || settings[:default_model]).to_s, provider: provider&.to_s, input: message
+          ) do |_span|
+            _dispatch_chat(model: model, provider: provider, intent: intent, tier: tier, escalate: escalate, max_escalations: max_escalations,
+                           quality_check: quality_check, message: message, **)
+          end
+        else
+          _dispatch_chat(model: model, provider: provider, intent: intent, tier: tier,
+                         escalate: escalate, max_escalations: max_escalations,
+                         quality_check: quality_check, message: message, **)
         end
-
-        chat_direct(model: model, provider: provider, intent: intent, tier: tier,
-                    escalate: escalate, max_escalations: max_escalations,
-                    quality_check: quality_check, message: message, **)
       end
 
       # Send a single message — daemon-first, falls through to direct on unavailability.
@@ -106,9 +111,13 @@ module Legion
 
       # Generate embeddings — delegates to gateway when available
       def embed(text, **)
-        return Legion::Extensions::LLM::Gateway::Runners::Inference.embed(text: text, **) if gateway_loaded?
-
-        embed_direct(text, **)
+        if defined?(Legion::Telemetry::OpenInference)
+          Legion::Telemetry::OpenInference.embedding_span(
+            model: (settings[:default_model] || 'unknown').to_s
+          ) { |_span| _dispatch_embed(text, **) }
+        else
+          _dispatch_embed(text, **)
+        end
       end
 
       # Direct embed bypassing gateway
@@ -127,13 +136,13 @@ module Legion
 
       # Generate structured JSON output — delegates to gateway when available
       def structured(messages:, schema:, **)
-        if gateway_loaded?
-          return Legion::Extensions::LLM::Gateway::Runners::Inference.structured(
-            messages: messages, schema: schema, **
-          )
+        if defined?(Legion::Telemetry::OpenInference)
+          Legion::Telemetry::OpenInference.llm_span(
+            model: (settings[:default_model] || 'unknown').to_s, input: messages.to_s
+          ) { |_span| _dispatch_structured(messages: messages, schema: schema, **) }
+        else
+          _dispatch_structured(messages: messages, schema: schema, **)
         end
-
-        structured_direct(messages: messages, schema: schema, **)
       end
 
       # Direct structured bypassing gateway
@@ -152,6 +161,49 @@ module Legion
 
       private
 
+      def _dispatch_chat(model:, provider:, intent:, tier:, escalate:, max_escalations:, quality_check:, message:, **)
+        messages = message.is_a?(Array) ? message : [{ role: 'user', content: message.to_s }]
+        resolved_model = model || settings[:default_model]
+
+        if defined?(Legion::LLM::Hooks)
+          blocked = Legion::LLM::Hooks.run_before(messages: messages, model: resolved_model)
+          return blocked[:response] if blocked
+        end
+
+        result = if gateway_loaded? && message
+                   gateway_chat(model: model, provider: provider, intent: intent,
+                                tier: tier, message: message, escalate: escalate,
+                                max_escalations: max_escalations, quality_check: quality_check, **)
+                 else
+                   chat_direct(model: model, provider: provider, intent: intent, tier: tier,
+                               escalate: escalate, max_escalations: max_escalations,
+                               quality_check: quality_check, message: message, **)
+                 end
+
+        if defined?(Legion::LLM::Hooks)
+          blocked = Legion::LLM::Hooks.run_after(response: result, messages: messages, model: resolved_model)
+          return blocked[:response] if blocked
+        end
+
+        result
+      end
+
+      def _dispatch_embed(text, **)
+        return Legion::Extensions::LLM::Gateway::Runners::Inference.embed(text: text, **) if gateway_loaded?
+
+        embed_direct(text, **)
+      end
+
+      def _dispatch_structured(messages:, schema:, **)
+        if gateway_loaded?
+          return Legion::Extensions::LLM::Gateway::Runners::Inference.structured(
+            messages: messages, schema: schema, **
+          )
+        end
+
+        structured_direct(messages: messages, schema: schema, **)
+      end
+
       def daemon_ask(message:, model: nil, provider: nil, context: {}, tier: nil, identity: nil) # rubocop:disable Lint/UnusedMethodArgument
         result = DaemonClient.chat(
           message: message, model: model, provider: provider,
@@ -172,6 +224,7 @@ module Legion
       end
 
       def ask_direct(message:, model: nil, provider: nil, intent: nil, tier: nil, &block)
+        assert_cloud_allowed! if effective_tier_is_cloud?(tier, provider)
         session = chat_direct(model: model, provider: provider, intent: intent, tier: tier)
         response = block ? session.ask(message, &block) : session.ask(message)
 
@@ -202,7 +255,10 @@ module Legion
             resolution = Router::GatewayInterceptor.intercept(resolution, context: kwargs.fetch(:context, {}))
             model    = resolution.model
             provider = resolution.provider
+            assert_cloud_allowed! if resolution.tier.to_sym == :cloud
           end
+        elsif tier
+          assert_cloud_allowed! if tier.to_sym == :cloud
         end
 
         model    ||= settings[:default_model]
@@ -304,6 +360,31 @@ module Legion
         esc.fetch(:quality_threshold, 50)
       end
 
+      def enterprise_privacy?
+        if Legion.const_defined?('Settings') && Legion::Settings.respond_to?(:enterprise_privacy?)
+          Legion::Settings.enterprise_privacy?
+        else
+          ENV['LEGION_ENTERPRISE_PRIVACY'] == 'true'
+        end
+      end
+
+      def assert_cloud_allowed!
+        return unless enterprise_privacy?
+
+        raise PrivacyModeError,
+              'Cloud LLM tier is disabled: enterprise_data_privacy is enabled. ' \
+              'Only Tier 0 (cache) and Tier 1 (local Ollama) are permitted.'
+      end
+
+      def effective_tier_is_cloud?(tier, provider)
+        return tier.to_sym == :cloud if tier
+        return false unless enterprise_privacy?
+
+        resolved = provider || settings[:default_provider]
+        cloud_providers = %i[anthropic bedrock openai gemini azure]
+        cloud_providers.include?(resolved&.to_sym)
+      end
+
       def set_defaults
         default_model    = settings[:default_model]
         default_provider = settings[:default_provider]

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: legion-llm
 version: !ruby/object:Gem::Version
-  version: 0.3.8
+  version: 0.3.11
 platform: ruby
 authors:
 - Esity
@@ -139,6 +139,7 @@ files:
 - lib/legion/llm/embeddings.rb
 - lib/legion/llm/escalation_history.rb
 - lib/legion/llm/helpers/llm.rb
+- lib/legion/llm/hooks.rb
 - lib/legion/llm/providers.rb
 - lib/legion/llm/quality_checker.rb
 - lib/legion/llm/response_cache.rb