legion-llm 0.8.47 → 0.9.9

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 758dae0c2bb2fba09c683bbcbf29459301ffdfe0239ff37214952e8c334422b0
-  data.tar.gz: 841c7d0ae47d825431601edb635bde28f1f7b60f938a008abe1c8d6f2ac5772d
+  metadata.gz: 6a7ea4bbe972c340ca3ccaa0cf2478724c90bb8385f1d8129a21fc74981a4380
+  data.tar.gz: a7a8250e8c84835b3bf2ffefe9cdc25494b9be7deae4cb16375bd8bfa2d8dcf2
 SHA512:
-  metadata.gz: 7a14e57a3a2bd88ac05cb83608e64461b710aed5370eb0686830dc705a80019eb7de351279204eb96429e06da167420c744d8ae79518dbeec6d7a69fcec14c7a
-  data.tar.gz: d442eeee484806be5f01dd13e212b155d8cb8d839cb50a0ddd9517d013a49675320936927ded888aa4c617ee8d282624dc93b69b814599c097ee6078a3e4409c
+  metadata.gz: be0a896d160b6824760d0361bb82e8e320f312fc23e8e9156cf9dfd924f3d264a48ff71bce0531b29f13913bf367c7eeca732cff477adc1c54310a4f4cbcce38
+  data.tar.gz: 210706b370e355606f342e506108b4ac13cb0e663c8203fe7b09afdeb643dba4e1ade3e5cb8e13b0efe4238b33656a73f7f57e74687413a6b0914821a31b0039

data/.gitignore

@@ -21,3 +21,6 @@ legion.log
 .worktrees/
 .claude/
 docs/
+bin/apollo-setup-postreboot.sh
+bin/apollo-setup-prereboot.sh
+legionio-bootstrap-uhg-v3.json

data/CHANGELOG.md

@@ -1,5 +1,140 @@
 # Legion LLM Changelog
 
+## [0.9.9] - 2026-05-07
+
+### Fixed
+- Initialized sticky-state persistence reads explicitly on cache misses to satisfy static analysis without changing runtime behavior.
+- Consume GAIA advisory `tool_hint`, `suppress`, and `context_window` data when building native tool definitions and sizing RAG retrieval, so GAIA advisory outputs affect provider calls instead of only appearing in enrichment summaries.
+
+## [0.9.8] - 2026-05-06
+
+### Fixed
+- Fixed Anthropic Messages request normalization for multi-block content.
+- Marked API-submitted client tools as non-executable server-side while preserving trusted registry/deferred tool injection.
+- Preferred explicit settings overrides over registry MCP overrides.
+- Added structured-output parse retries through alternate routes when an escalation route is available.
+- Added bounded metering spool writes, disabled-vs-dropped audit results, and warning logs for invalid settings paths and invalid inventory offerings.
+- Preserved non-text message content for audit/persistence and extracted string-keyed text blocks correctly.
+- Added trackable unknown caller identity defaults for audit/transport envelopes.
+- Used provider embedding usage tokens when formatting OpenAI-compatible embedding responses.
+- Serialized daemon-client cached state behind a shared mutex.
+- Added escalation transport exchange binding for escalation events.
+
+## [0.9.7] - 2026-05-06
+
+### Fixed
+- Enabled arbitrage by default, added zero-cost local/fleet offering coverage, and removed the dead `Quality::Checker.model_score` dependency from arbitrage eligibility.
+- Skipped debate when fewer than two distinct models are available and captured judge evaluation/confidence separately from the final answer.
+- Reused the shared context token estimator for billing preflight cost checks.
+- Logged dropped metering emissions at the inference metering call site.
+- Added resolved model metadata to routing span attributes.
+- Stopped treating standalone email addresses as PII by default while preserving contextual and opt-in email detection.
+- Allowed context curation to choose vLLM/MLX local or fleet models for LLM-assisted summarization.
+- Routed compressor summarization through standard low-cost/basic intent when no explicit compressor model is configured.
+
+## [0.9.6] - 2026-05-06
+
+### Fixed
+- Enforced privacy classification as a forced local routing constraint, even when callers request a cloud tier.
+- Made RAG faithfulness failures block caller-visible responses by default and record structured audit data.
+- Archived dropped conversation turns into Apollo with conversation-scoped tags and retrieved archived history during RAG context loading.
+- Reported escalation quality failures and low-confidence responses to `Router::HealthTracker` as quality signals.
+- Persisted conversation sticky state through the database path and restored it after in-memory LRU eviction.
+- Counted pending conversation-history and RAG enrichments in token-budget checks before provider dispatch.
+- Averaged provider health priority adjustments across instances while preserving worst-state circuit reporting.
+
+## [0.9.5] - 2026-05-06
+
+### Fixed
+- Fixed context curator cache invalidation and stored curated-summary replay so compacted messages are used on later turns.
+- Persisted curation marker records even when a pass does not rewrite individual messages, allowing structural curation to run for short-message turns.
+- Fixed compressor LLM summarization to call `Legion::LLM.chat_direct` with a prompt message instead of the obsolete session-style API.
+- Warn and omit misleading zero-dollar cost estimates when provider usage metadata collapses to zero tokens for a known model.
+
+## [0.9.4] - 2026-05-06
+
+### Changed
+- Added shared inference step logging helpers and debug-level step enter/complete/failure logs.
+- Added safe debug/info instrumentation across inference steps for routing actions, enrichment decisions, tool handling, RAG, skill injection, sticky runner state, billing, classification, debate, post-response audit, and metering emission.
+
+## [0.9.3] - 2026-05-06
+
+### Changed
+- Delegated responder-side fleet provider execution, token validation, and provider response publishing to the shared `lex-llm` fleet helpers.
+- Kept `legion-llm` as the request-side fleet dispatcher and token issuer while retaining compatibility aliases for old responder constants.
+- Bumped the `lex-llm` dependency floor to `>= 0.4.3` for shared responder execution helpers.
+
+## [0.9.2] - 2026-05-06
+
+### Fixed
+- Prefer namespaced caller ids over ambiguous display identities when publishing audit, metering, and transport identity metadata.
+
+## [0.9.1] - 2026-05-06
+
+### Changed
+- `legion-llm` now owns lex-llm provider registration by scanning loaded provider modules, constructing `LexLLMAdapter` instances, and writing `Call::Registry`.
+- Provider rediscovery now rebuilds registry entries after `Call::Registry.reset!`, supporting LegionIO reload/hot-update flows without relying on provider require-time side effects.
+- Bumped the `lex-llm` dependency floor to `>= 0.4.1` for pure provider discovery and alias metadata.
+
+### Fixed
+- Preserve streaming thinking chunks that providers emit as plain strings.
+- Normalize discovered provider instance ids from provider offerings before memory-gate and availability checks.
+- Use canonical offering aliases for metering cost estimates and fall back to caller agent metadata for fleet context.
+- Preserve extension, string, and top-level caller identity metadata in audit events and transport headers.
+
+## [0.9.0] - 2026-05-06
+
+### Changed
+- Added shared provider-owned fleet responder execution support for lex-llm provider gems.
+- Moved fleet dispatch defaults to top-level `fleet.dispatch`, removed legacy gateway defaults, and rejected `routing.use_fleet` / `openai_compat.gateways` settings during validation.
+- OpenAI-compatible routing now resolves through registered `lex-llm-openai` provider instances instead of gateway interceptor configuration.
+- Native dispatch now routes chat, stream, embed, image, and health calls through the canonical lex-llm provider-instance adapter contract.
+- Native inference now records direct/fleet route attempts with dispatch path, idempotency key, selected lane, failure reason, and escalation context.
+- Native inference and `/api/llm/inference` now strip provider thinking from caller-visible content and expose thinking only through explicit diagnostic fields/events.
+- Inventory and provider/model API reads now use cached discovery and non-live provider offerings, so explicit discovery refresh remains the only path that probes provider endpoints.
+- Fleet dispatch now publishes shared lex-llm protocol-v2 envelopes with canonical `operation`, `request_id`, `correlation_id`, `idempotency_key`, signed tokens, and strict reply matching.
+- Fleet worker handling now validates protocol-v2 envelopes, enforces token/idempotency policy, dispatches local providers through canonical lex-llm methods, and publishes shared lex-llm response/error envelopes.
+- Bumped dependency floors to `lex-llm >= 0.4.0` and `legion-transport >= 1.4.14` for shared provider contracts and fleet envelopes.
+
+### Removed
+- Removed the gateway interceptor runtime path and gateway metering fallback.
+- Retired `Legion::LLM::Transport::Messages::FleetRequest`, `FleetResponse`, and `FleetError` as fleet message authorities in favor of `Legion::Extensions::Llm::Transport::Messages::*`.
+
+## [0.8.51] - 2026-05-03
+
+### Changed
+- Native `/api/llm/inference` streaming hides provider thinking deltas by default, with `include_thinking: true` as the explicit diagnostic opt-in.
+- Pipeline metering events now carry wall-clock latency, estimated cost, conversation/correlation ids, billing, task, agent, identity, and routing context.
+- LLM transport messages now preserve caller identity, credential, and caller type headers from nested caller metadata, top-level identity metadata, and extension callers.
+
+### Fixed
+- Prompt audit events now include provider response thinking separately from assistant response content.
+- Native lex-llm dispatch now carries provider thinking separately from response content before API responses are emitted.
+- Native discovery now normalizes provider offering objects before generating routing candidates, preserving provider instance, tier, capabilities, context length, and parameter metadata.
+
+## [0.8.50] - 2026-05-03
+
+### Fixed
+- Native discovery now normalizes lex-llm `ModelOffering` objects before generating routing candidates, allowing auto-rules to populate from provider adapters again.
+
+## [0.8.49] - 2026-04-29
+
+### Changed
+- `Settings.register_defaults!` now calls `Legion::Settings.register_library` instead of `merge_settings`, using the idempotent legion-settings 1.4.0 API that prevents double-registration.
+- Bumped `legion-settings` dependency floor to `>= 1.4.0`.
+- Test stub `Legion::Settings` now exposes `register_library` matching the real 1.4.0 API.
+
+## [0.8.48] - 2026-04-29
+
+### Added
+- `ToolDefinition.from_registry_entry` builds tool definitions from `Legion::Settings::Extensions` registry entries.
+- `Dispatcher` checks `Settings::Extensions` for tool override resolution; when no matching entry is found it falls back to settings-based MCP overrides (no `Tools::Registry` or `Catalog::Registry` fallback).
+- `Executor#add_registry_tool_definitions` reads from `Settings::Extensions` when available, falling back to `Legion::Tools::Registry` for backward compatibility.
+- `Steps::ToolDiscovery` discovers tools from `Settings::Extensions` when available, falling back to `Legion::Tools::Registry`.
+
+### Changed
+- Bumped `legion-settings` dependency floor to `>= 1.4.0` (requires `Settings::Extensions` module).
+
 ## [0.8.47] - 2026-04-29
 
 ### Fixed

data/CLAUDE.md

@@ -8,7 +8,7 @@
 Core LegionIO gem providing LLM capabilities to all extensions through Legion-native provider dispatch. Includes a dynamic weighted routing engine that dispatches requests across local, fleet, and cloud tiers based on caller intent, priority rules, time schedules, cost multipliers, and real-time provider health.
 
 **GitHub**: https://github.com/LegionIO/legion-llm
-**Version**: 0.8.0
+**Version**: 0.8.49
 **License**: Apache-2.0
 
 ## Architecture
@@ -61,7 +61,10 @@ Legion::LLM (lib/legion/llm.rb)          # Thin facade — delegates to Inferenc
 │   ├── Compressor   # Deterministic prompt compression (3 levels, code-block-aware)
 │   └── Curator      # Async conversation curation: strip thinking, distill tools, fold resolved exchanges
 ├── Discovery                            # Runtime introspection
-│   ├── Ollama       # Queries Ollama /api/tags for pulled models (TTL-cached)
+│   ├── Ollama       # Multi-instance Ollama /api/tags + /api/show discovery (TTL-cached)
+│   ├── Vllm         # Multi-instance vLLM /v1/models + /health discovery (TTL-cached)
+│   ├── RuleGenerator # Auto-generates routing rules from discovered instances/models
+│   ├── MemoryGate   # Checks available RAM before routing to local models
 │   └── System       # Queries OS memory: macOS (vm_stat/sysctl), Linux (/proc/meminfo)
 ├── Quality                              # Response quality evaluation
 │   ├── Checker      # Quality heuristics (empty, too_short, repetition, json_parse) + pluggable (was QualityChecker)
@@ -364,24 +367,21 @@ Settings read from `Legion::Settings[:llm]`:
 
 ### Provider Settings
 
-Each provider has: `enabled`, `api_key`, `vault_path`, plus provider-specific keys.
+Provider defaults now live in each `lex-llm-*` provider extension's `default_settings`. The `providers:` key in `Settings.default` ships as an empty hash; settings files and extension registrations populate it at runtime. Each provider has: `enabled`, `api_key`, plus provider-specific keys.
 
-Vault credential resolution: When `vault_path` is set and Legion::Crypt::Vault is connected, credentials are fetched from Vault at startup. Keys map to provider-specific fields automatically.
+Local/fleet providers (Ollama, vLLM, MLX) support multi-instance configs via an `instances:` hash. Discovery scans all instances in parallel, enriches models with real capability metadata, and generates per-instance routing rules.
 
-Bedrock supports two auth modes:
-- **SigV4** (default): `api_key` + `secret_key` (+ optional `session_token`)
-- **Bearer token**: `bearer_token` for AWS Identity Center/SSO. Native Bedrock providers consume it through lex-llm configuration.
+### Capability-Aware Routing
 
-### Auto-Detection Priority
+Routing rules carry `model_capabilities`, `context_length`, and `parameter_count` from provider-supplied `Model::Info`. The `RuleGenerator` creates rules from discovered instances without a static capability map -- each provider supplies real metadata.
+
+### Memory Gate
 
-When no defaults are configured, the first enabled provider is used:
+`Discovery::MemoryGate` checks available system memory before routing to local models. Models that exceed available RAM minus `discovery.memory_floor_mb` are silently skipped.
+
+### Auto-Detection Priority
 
-1. Bedrock -> `us.anthropic.claude-sonnet-4-6-v1`
-2. Anthropic -> `claude-sonnet-4-6`
-3. OpenAI -> `gpt-4o`
-4. Gemini -> `gemini-2.0-flash`
-5. Azure -> (endpoint-specific, from `api_base`)
-6. Ollama -> `llama3`
+When no defaults are configured, the first enabled provider is used. Detection order and default models are defined by each `lex-llm-*` provider extension.
 
 ### Routing Settings
 
@@ -501,7 +501,10 @@ In-memory signal consumer with pluggable handlers. Adjusts effective priorities
 | `lib/legion/llm/context/compressor.rb` | Deterministic prompt compression: 3 levels, code-block-aware, stopword removal |
 | `lib/legion/llm/context/curator.rb` | Async heuristic conversation curation (was ContextCurator) |
 | `lib/legion/llm/discovery.rb` | Discovery entry point: run, detect_embedding_capability, can_embed? |
-| `lib/legion/llm/discovery/ollama.rb` | Ollama /api/tags discovery with TTL cache |
+| `lib/legion/llm/discovery/ollama.rb` | Multi-instance Ollama /api/tags + /api/show discovery with TTL cache |
+| `lib/legion/llm/discovery/vllm.rb` | Multi-instance vLLM /v1/models + /health discovery with TTL cache |
+| `lib/legion/llm/discovery/rule_generator.rb` | Auto-generates routing rules from discovered instances/models |
+| `lib/legion/llm/discovery/memory_gate.rb` | Checks available RAM vs model size before routing to local models |
 | `lib/legion/llm/discovery/system.rb` | OS memory introspection (macOS + Linux) with TTL cache |
 | `lib/legion/llm/quality.rb` | Quality entry point |
 | `lib/legion/llm/quality/checker.rb` | Quality heuristics + pluggable callable (was QualityChecker) |
@@ -715,7 +718,7 @@ The legacy `vault_path` per-provider setting was removed in v0.3.1.
 Tests run without the full LegionIO stack. `spec/spec_helper.rb` uses real `Legion::Logging` and `Legion::Settings` (no stubs — hard dependencies are always present). Each test resets settings to defaults via `before(:each)`.
 
 ```bash
-bundle exec rspec    # 1661 examples, 0 failures
+bundle exec rspec    # 2379 examples, 0 failures
 bundle exec rubocop  # 0 offenses
 ```
 

data/Gemfile

@@ -4,6 +4,9 @@ source 'https://rubygems.org'
 
 gemspec
 
+legion_settings_path = File.expand_path('../legion-settings', __dir__)
+gem 'legion-settings', path: legion_settings_path if Dir.exist?(legion_settings_path)
+
 group :test do
   lex_llm_path = File.expand_path('../extensions-ai/lex-llm', __dir__)
   if Dir.exist?(lex_llm_path)

data/README.md

@@ -2,7 +2,7 @@
 
 LLM routing and provider orchestration for the [LegionIO](https://github.com/LegionIO/LegionIO) framework. Routes chat, embeddings, tool use, fleet dispatch, auditing, and provider metadata through Legion-native `lex-llm-*` provider extensions.
 
-**Version**: 0.8.47
+**Version**: 0.9.0
 
 ## Installation
 
@@ -60,7 +60,7 @@ Requests flow through the full Inference pipeline — routing, metering, audit,
 Both formats supported with correct SSE shapes:
 - **OpenAI**: `data: {"choices":[{"delta":{"content":"..."}}]}` chunks, terminated by `data: [DONE]`
 - **Anthropic**: Typed events — `message_start`, `content_block_start`, `content_block_delta`, `content_block_stop`, `message_delta`, `message_stop`
-- **Native**: `/api/llm/inference` streams `text-delta`, `thinking-delta`, tool lifecycle events, and a final `done` event. Structured provider content blocks are flattened to plain text in both streaming and non-streaming native responses so `content` remains a string for daemon clients.
+- **Native**: `/api/llm/inference` streams `text-delta`, optional `thinking-delta` events when `include_thinking: true`, tool lifecycle events, and a final `done` event. Structured provider content blocks are flattened to plain text in both streaming and non-streaming native responses so `content` remains a string for daemon clients.
 
 ### API Authentication
 
@@ -90,6 +90,8 @@ When enabled, validates `Authorization: Bearer <token>` or `x-api-key` headers a
 
 ## Configuration
 
+Provider defaults now live in each `lex-llm-*` provider extension. `legion-llm` ships an empty `providers: {}` hash; settings files and extension registrations populate it at runtime.
+
 Add to your LegionIO settings directory (e.g. `~/.legionio/settings/llm.json`):
 
 ```json
@@ -103,53 +105,60 @@ Add to your LegionIO settings directory (e.g. `~/.legionio/settings/llm.json`):
         "region": "us-east-2",
         "bearer_token": ["vault://secret/data/llm/bedrock#bearer_token", "env://AWS_BEARER_TOKEN"]
       },
-      "anthropic": {
-        "enabled": false,
-        "api_key": "env://ANTHROPIC_API_KEY"
-      },
-      "openai": {
-        "enabled": false,
-        "api_key": "env://OPENAI_API_KEY"
-      },
       "ollama": {
-        "enabled": false,
-        "base_url": "http://localhost:11434"
-      },
-      "vllm": {
-        "enabled": false,
-        "base_url": "http://localhost:8000/v1",
-        "default_model": "qwen3.6-27b",
-        "enable_thinking": true
-      },
-      "mlx": {
-        "enabled": false,
-        "base_url": "http://localhost:8000"
+        "enabled": true,
+        "base_url": "http://localhost:11434",
+        "instances": {
+          "default": { "base_url": "http://localhost:11434" },
+          "gpu_server": { "base_url": "http://gpu-server:11434" }
+        }
       }
     }
   }
 }
 ```
 
-Credentials are resolved automatically by the universal secret resolver in `legion-settings` (v1.3.0+). Use `vault://` URIs for Vault secrets, `env://` for environment variables, or plain strings for static values. Array values act as fallback chains — the first non-nil result wins.
+Credentials are resolved automatically by the universal secret resolver in `legion-settings` (v1.3.0+). Use `vault://` URIs for Vault secrets, `env://` for environment variables, or plain strings for static values. Array values act as fallback chains -- the first non-nil result wins.
+
+### Provider Extensions (lex-llm-*)
+
+Each provider is a standalone `lex-llm-*` gem that ships its own `default_settings`, model catalog, capability declarations, and optional provider-owned fleet worker actor. When a provider gem is loaded, `legion-llm` discovers it through the shared `lex-llm` provider contract and registers provider instances for routing. Provider gems implement:
+
+- **`default_settings`** -- Connection defaults (base_url, region, API key env vars)
+- **`model_allowed?(model_name)`** -- Provider-level model filtering
+- **`Model::Info`** -- Real capabilities, context lengths, and parameter counts for each model
+
+The routing layer only sees models the provider has already filtered and annotated.
+
+### Multi-Instance Providers
+
+Local and fleet providers (Ollama, vLLM, MLX) support multiple named instances:
+
+```json
+{
+  "ollama": {
+    "enabled": true,
+    "instances": {
+      "macbook":    { "base_url": "http://localhost:11434" },
+      "gpu_server": { "base_url": "http://gpu-server:11434" }
+    }
+  }
+}
+```
+
+Discovery scans all instances in parallel, enriches models with `/api/show` metadata, and generates per-instance routing rules. Each instance appears independently in the routing table so the router can target the exact hardware.
 
-### Provider Configuration
+### Capability-Aware Routing
 
-Each provider supports these common fields:
+Routing rules and auto-generated rules carry `model_capabilities`, `context_length`, and `parameter_count` from provider-supplied `Model::Info`. The router uses these to match capability requirements (e.g., `thinking`, `vision`, `tools`) without a static lookup table.
 
-| Field | Type | Description |
-|-------|------|-------------|
-| `enabled` | Boolean | Enable this provider (default: `false`) |
-| `api_key` | String | API key (supports `vault://`, `env://`, or plain string) |
+### Generic Dispatch
 
-Provider-specific fields:
+`Call::Dispatch.call` accepts a `capability:` parameter (`:chat`, `:stream`, `:embed`) and routes to the registered `lex-llm-*` adapter. This replaces the old provider-specific dispatch paths.
 
-| Provider | Additional Fields |
-|----------|------------------|
-| **Bedrock** | `secret_key`, `session_token`, `region` (default: `us-east-2`), `bearer_token` (alternative to SigV4 — for AWS Identity Center/SSO) |
-| **Azure** | `api_base` (Azure OpenAI endpoint URL, required), `auth_token` (bearer token alternative to `api_key`) |
-| **Ollama** | `base_url` (default: `http://localhost:11434`) |
-| **vLLM** | `base_url` (default: `http://localhost:8000/v1`), `api_key`, `enable_thinking` |
-| **MLX** | `base_url` (default: `http://localhost:8000`), `api_key` |
+### Memory Gate
+
+Discovery checks available system memory (macOS `vm_stat`/`sysctl`, Linux `/proc/meminfo`) before routing to local models. Models that exceed available RAM minus `discovery.memory_floor_mb` are silently skipped.
 
 ### Credential Resolution
 
@@ -171,18 +180,7 @@ By the time `Legion::LLM.start` runs, all `vault://` and `env://` references hav
 
 ### Auto-Detection
 
-If no `default_model` or `default_provider` is set, legion-llm auto-detects from the first enabled provider in priority order:
-
-| Priority | Provider | Default Model |
-|----------|----------|---------------|
-| 1 | Bedrock | `us.anthropic.claude-sonnet-4-6-v1` |
-| 2 | Anthropic | `claude-sonnet-4-6` |
-| 3 | OpenAI | `gpt-4o` |
-| 4 | Gemini | `gemini-2.0-flash` |
-| 5 | Azure | (endpoint-specific) |
-| 6 | Ollama | `qwen3.5:latest` |
-| 7 | vLLM | `qwen3.6-27b` |
-| 8 | MLX | (configured model) |
+If no `default_model` or `default_provider` is set, legion-llm auto-detects from the first enabled provider. The detection order and default models are defined by each `lex-llm-*` provider extension's `default_settings`.
 
 ## Core API
 
@@ -372,9 +370,9 @@ Legion::LLM (lib/legion/llm.rb)          # Thin facade — delegates to Inferenc
 │   ├── Arbitrage    # Cost-aware model selection when no rules match
 │   └── Escalation/
 │       └── History  # EscalationHistory mixin
-├── Fleet                                # Fleet RPC dispatch over AMQP (built-in)
+├── Fleet                                # Fleet dispatch over AMQP; provider responders live in lex-llm-* gems
 │   ├── Dispatcher   # Fleet RPC dispatch with routing key building, per-type timeouts
-│   ├── Handler      # Fleet request handler for GPU worker nodes
+│   ├── TokenIssuer  # Request-side JWT minting for provider-owned responders
 │   └── ReplyDispatcher # Correlation-based reply routing
 ├── API                                  # All external HTTP interfaces
 │   ├── Auth         # Config-driven Bearer/x-api-key auth for /v1/ routes
@@ -394,10 +392,10 @@ Legion::LLM (lib/legion/llm.rb)          # Thin facade — delegates to Inferenc
 │       ├── OpenAIRequest / OpenAIResponse
 │       └── AnthropicRequest / AnthropicResponse
 ├── Audit                                # Prompt, tool, and skill audit event emission
-├── Transport                            # Centralized AMQP exchange and message definitions
+├── Transport                            # Centralized AMQP exchange and non-fleet message definitions
 │   ├── Message      # LLM base message: context propagation, LLM headers
 │   ├── Exchanges/   # Fleet, Metering, Audit, Escalation
-│   └── Messages/    # FleetRequest, FleetResponse, FleetError, MeteringEvent, etc.
+│   └── Messages/    # MeteringEvent, prompt/tool audit, escalation, and compatibility wrappers
 ├── Scheduling                           # Deferred execution
 │   ├── Batch        # Non-urgent request batching with priority queue and auto-flush
 │   └── OffPeak      # Peak-hour deferral
@@ -499,8 +497,8 @@ legion-llm includes a dynamic weighted routing engine that dispatches requests a
 │  Tier 1: LOCAL  → Ollama on this machine (direct HTTP)   │
 │          Zero network overhead, no Transport              │
 │                                                          │
-│  Tier 2: FLEET  → vLLM/Ollama on GPU workers             │
-│          Built-in Fleet RPC over AMQP                     │
+│  Tier 2: FLEET  → provider-owned lex-llm-* responders     │
+│          Shared lex-llm fleet envelopes over AMQP         │
 │                                                          │
 │  Tier 3: CLOUD  → Bedrock / Azure / Gemini               │
 │  Tier 4: FRONTIER → Anthropic / OpenAI                   │
@@ -511,12 +509,12 @@ legion-llm includes a dynamic weighted routing engine that dispatches requests a
 | Tier | Target | Use Case |
 |------|--------|----------|
 | `local` | Ollama on localhost | Privacy-sensitive, offline, or low-latency workloads |
-| `fleet` | Shared hardware via built-in Fleet dispatcher (AMQP) | Larger vLLM/Ollama models on dedicated GPU servers |
-| `openai_compat` | OpenAI-compatible gateways | Self-hosted or proxy endpoints with OpenAI-compatible APIs |
+| `fleet` | Shared hardware via provider-owned lex-llm responders over AMQP | Larger vLLM/Ollama models on dedicated GPU servers |
+| `openai_compat` | OpenAI-compatible provider instances | Self-hosted or proxy endpoints with OpenAI-compatible APIs |
 | `cloud` | API providers (Bedrock, Azure, Gemini) | Managed cloud inference |
 | `frontier` | API providers (Anthropic, OpenAI) | Frontier models, full-capability inference |
 
-Fleet dispatch is built into legion-llm. The `Fleet::Dispatcher` publishes shared-lane requests to keys such as `llm.fleet.inference.qwen3-6-27b.ctx32000` or `llm.fleet.embed.nomic-embed-text`; `Fleet::Handler` processes them on GPU worker nodes and replies through correlated live responses. Keep `routing.tiers.fleet.routing_style` set to `shared_lane` for the default pooled model lanes, set it to `offering_lane` for exact provider-instance lanes such as `llm.fleet.offering.vllm-gpu-01.qwen3-6.inference`, or use any other value only for the legacy `llm.request.{provider}.{type}.{model}` keys.
+Fleet dispatch is built into `legion-llm`, but fleet consumption is provider-owned. `Fleet::Dispatcher` publishes shared `lex-llm` protocol-v2 `FleetRequest` envelopes to keys such as `llm.fleet.inference.qwen3-6-27b.ctx32000` or `llm.fleet.embed.nomic-embed-text`; the enabled provider gem actor consumes the request, validates the signed token and idempotency key through `Legion::Extensions::Llm::Fleet::ProviderResponder`, calls its local provider instance through the canonical `lex-llm` provider methods, and replies with shared `FleetResponse` or `FleetError` envelopes. Keep `routing.tiers.fleet.routing_style` set to `shared_lane` for the default pooled model lanes, or set it to `offering_lane` for exact provider-instance lanes such as `llm.fleet.offering.vllm-gpu-01.qwen3-6.inference`.
 
 #### Intent-Based Dispatch
 
@@ -587,7 +585,7 @@ Add routing configuration under the `llm` key:
           "timeout_seconds": 30,
           "timeouts": { "embed": 10, "chat": 30, "generate": 30, "default": 30 }
         },
-        "openai_compat": { "gateways": [] },
+        "openai_compat": { "providers": ["openai"] },
         "cloud": { "providers": ["bedrock", "azure", "gemini"] },
         "frontier": { "providers": ["anthropic", "openai"] }
       },
@@ -855,6 +853,8 @@ Legion::Service#initialize
   load_extensions      # LEX extensions (can use LLM if available)
 ```
 
+LegionIO hosts these routes through `mount_library_routes('llm', Routes::Llm, 'Legion::LLM::Routes')`. The route modules remain owned by `legion-llm`; LegionIO no longer registers provider gateway fallback routes when the library is available.
+
 - **Service**: `setup_llm` called between data and supervision in startup sequence
 - **Extensions**: `llm_required?` method on extension module, checked at load time
 - **Helpers**: `Legion::Extensions::Helpers::LLM` auto-loaded when gem is present
@@ -889,8 +889,9 @@ bundle exec rubocop -A
 | `legion-json` | Legion JSON serialization |
 | `legion-logging` | Logging |
 | `legion-settings` | Configuration defaults and file overrides |
+| `legion-transport` (>= 1.4.14) | AMQP transport for fleet dispatch, metering, and audit |
 | `lex-knowledge` | Optional knowledge chunking integration when loaded |
-| `lex-llm` (>= 0.1.6) | Provider-neutral model offering and adapter base |
+| `lex-llm` (>= 0.4.3) | Provider-neutral contract, model offerings, response normalization, fleet envelopes, and responder-side fleet execution helpers |
 | `pdf-reader` | PDF extraction support |
 | `tzinfo` (>= 2.0) | IANA timezone conversion for schedule windows |
 

data/legion-llm.gemspec

@@ -30,9 +30,10 @@ Gem::Specification.new do |spec|
   spec.add_dependency 'legion-cache', '>= 1.4.2'
   spec.add_dependency 'legion-json', '>= 1.2.0'
   spec.add_dependency 'legion-logging', '>= 1.2.8'
-  spec.add_dependency 'legion-settings', '>= 1.3.12'
+  spec.add_dependency 'legion-settings', '>= 1.4.0'
+  spec.add_dependency 'legion-transport', '>= 1.4.14'
   spec.add_dependency 'lex-knowledge'
-  spec.add_dependency 'lex-llm', '>= 0.1.6'
+  spec.add_dependency 'lex-llm', '>= 0.4.3'
   spec.add_dependency 'pdf-reader'
   spec.add_dependency 'tzinfo', '>= 2.0'
 end

data/lib/legion/llm/api/native/helpers.rb

@@ -10,8 +10,10 @@ require 'legion/llm/types'
 begin
   require 'legion/identity/request'
   require 'legion/identity/process'
-rescue LoadError
-  # legion-llm can still be loaded outside a full LegionIO runtime.
+rescue LoadError => e
+  Object.new.extend(Legion::Logging::Helper).handle_exception(
+    e, level: :debug, handled: true, operation: 'llm.api.native.helpers.optional_identity_require'
+  )
 end
 
 module Legion
@@ -299,7 +301,7 @@ module Legion
                   name:        tname,
                   description: tdesc,
                   parameters:  tschema || {},
-                  source:      { type: :client, executable: true }
+                  source:      { type: :client, executable: false }
                 )
               rescue StandardError => e
                 handle_exception(e, level: :warn, handled: true, operation: "llm.api.build_client_tool_class.#{tname}")
@@ -334,7 +336,7 @@ module Legion
                   text = content.key?(:text) || content.key?('text') ? (content[:text] || content['text']) : (content[:content] || content['content'])
                   extract_text_content(text)
                 else
-                  content.to_s
+                  content.respond_to?(:text) ? content.text.to_s : content.to_s
                 end
               end
 

data/lib/legion/llm/api/native/inference.rb

@@ -26,6 +26,7 @@ module Legion
               caller_context  = body[:caller]
               conversation_id = body[:conversation_id]
               request_id      = body[:request_id] || SecureRandom.uuid
+              include_thinking = body[:include_thinking] == true
 
               unless messages.is_a?(Array)
                 halt 400, { 'Content-Type' => 'application/json' },
@@ -145,7 +146,7 @@ module Legion
 
                   pipeline_response = executor.call_stream do |chunk|
                     thinking = extract_text_content(chunk.thinking) if chunk.respond_to?(:thinking)
-                    emit_sse_event(out, 'thinking-delta', { delta: thinking }) unless thinking.to_s.empty?
+                    emit_sse_event(out, 'thinking-delta', { delta: thinking }) if include_thinking && !thinking.to_s.empty?
 
                     text = extract_text_content(chunk.respond_to?(:content) ? chunk.content : chunk)
                     next if text.empty?
@@ -161,14 +162,18 @@ module Legion
 
                   routing = pipeline_response.routing || {}
                   tokens = pipeline_response.tokens || {}
+                  done_payload = {
+                    request_id:      request_id,
+                    content:         full_text,
+                    model:           (routing[:model] || routing['model']).to_s,
+                    input_tokens:    token_value(tokens, :input),
+                    output_tokens:   token_value(tokens, :output),
+                    tool_calls:      extract_tool_calls(pipeline_response),
+                    conversation_id: pipeline_response.conversation_id
+                  }
+                  done_payload[:thinking] = pipeline_response.thinking if include_thinking && pipeline_response.thinking
                   emit_sse_event(out, 'done', {
-                                   request_id:      request_id,
-                                   content:         full_text,
-                                   model:           (routing[:model] || routing['model']).to_s,
-                                   input_tokens:    token_value(tokens, :input),
-                                   output_tokens:   token_value(tokens, :output),
-                                   tool_calls:      extract_tool_calls(pipeline_response),
-                                   conversation_id: pipeline_response.conversation_id
+                                   **done_payload
                                  })
 
                   log.info(
@@ -208,16 +213,18 @@ module Legion
                   "stop_reason=#{pipeline_response.stop&.dig(:reason) || 'unknown'} stream=false"
                 )
 
-                json_response({
-                                request_id:      request_id,
-                                content:         content,
-                                tool_calls:      tool_calls,
-                                stop_reason:     pipeline_response.stop&.dig(:reason)&.to_s,
-                                model:           (routing[:model] || routing['model']).to_s,
-                                input_tokens:    token_value(tokens, :input),
-                                output_tokens:   token_value(tokens, :output),
-                                conversation_id: pipeline_response.conversation_id
-                              }, status_code: 200)
+                payload = {
+                  request_id:      request_id,
+                  content:         content,
+                  tool_calls:      tool_calls,
+                  stop_reason:     pipeline_response.stop&.dig(:reason)&.to_s,
+                  model:           (routing[:model] || routing['model']).to_s,
+                  input_tokens:    token_value(tokens, :input),
+                  output_tokens:   token_value(tokens, :output),
+                  conversation_id: pipeline_response.conversation_id
+                }
+                payload[:thinking] = pipeline_response.thinking if include_thinking && pipeline_response.thinking
+                json_response(payload, status_code: 200)
               end
             rescue Legion::LLM::AuthError => e
               handle_exception(e, level: :error, handled: true, operation: 'llm.api.inference.auth', request_id: request_id)