legion-llm 0.15.2 → 0.16.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 63f3720eceeff21e7c4d1ca9c13858256b93bd2744d0ae3fc1bd0c7452b40d1a
-  data.tar.gz: 686e8646720ba92310f58ab7e675d380003e89e465a7b02711b916b38ee89953
+  metadata.gz: c593f0ce469cb6391a8aecfea4bbccfb99220e598616e980bc005594fe789972
+  data.tar.gz: 2cb436d417122b94e0aa9eb02775aeee25047bf94ba7cb03ff8bf73f6677787f
 SHA512:
-  metadata.gz: cf42fe3fc6d6128ca111f2d92488496079496357d03ddf5c8fdf45d1372d245fb19f997a941caff59b13a5c8c4ddf52ac1dcc28ad2aace7f5f5accd1fdb6c48d
-  data.tar.gz: be0e5fbd569a12eb65a0952c58ee91a42f06955858a073c52e83ae1e2165fec4f595dd2677b3287ffc1b5cf99fb53ed8f8905849876e93cbb21b100ea4f47d02
+  metadata.gz: 2cb61a834758040085f1f8479119e1e36e3b77b7145fc1ee42b900454993f2356cac77ad4cd54898ded97380ae66bf905487e2ed2c26bf27a43cdf0f598ce6a7
+  data.tar.gz: 0ec4deffe1350de416f2f3f9366ebe97d20ac32154f76d25e603767b089e9155ed7896cb84ed43ceeb8306211318494c78ff8e3e8f58d0189b089b5db9e6d0d5

data/CHANGELOG.md

@@ -1,5 +1,50 @@
 # Legion LLM Changelog
 
+## [0.16.3] - 2026-08-17
+
+### Changed
+- **`llm.context_curation.tool_result_max_chars` 2,000 → 10,000.** The tool-result distillation cap is raised so larger tool outputs (file reads, command logs) survive curation intact before the heuristic summary kicks in.
+- **Strict direct-first tier weights.** `llm.routing.tier_weights` is now `{ direct: 150, local: 120, fleet: 115, cloud: 110, frontier: 105 }` — a strict direct → local → fleet → cloud → frontier order with no ties. The previous default ranked frontier first (`{ direct: 105, local: 110, fleet: 110, cloud: 120, frontier: 150 }`, local/fleet tied at 110).
+
+### Removed
+- **`llm.routing.last_resort_model` / `last_resort_provider` defaults.** No code path reads either key (verified); the stale `claude-sonnet-4-6` / `anthropic` last-resort fallback is gone.
+
+### Notes
+- **Curation threshold audit (blocking ceiling / no-curation floor), mapped by behavior.** (1) The *no-curation floor* — `llm.context_curation.target_context_tokens`, at/below which drop-and-archive curation is a no-op (`context/curator.rb`) — is 120,000 → 120,000 (confirmed unchanged); the conversation auto-compact floor (`llm.conversation.summarize_threshold`, `inference/executor.rb`) sits at the same 120,000. (2) The *size-based blocking ceiling* that compacts the outgoing call — `llm.context_curation.context_window_threshold` (`inference/executor/context_window.rb`) — 0.90 → 0.85: the synchronous curation ceiling is now 85% of the model's context window (850k on the 1M-window vllm lanes).
+
+## [0.16.2] - 2026-08-17
+
+### Fixed
+- **Status API repointed off the dead legacy store.** `/api/llm/providers` and `/instances` now read the per-instance health hash from `Legion::Settings` (4-key legacy shape keyed by config name, written by the discovery actors); `/tiers` and `/offerings` read the live `Inventory::Registry` snapshot. `routing_enabled?` is derived from the Registry instead of the hardcoded `false` stub.
+- **Dispatch-boundary response normalization (latent 500 after a provider 200).** `Call::Dispatch.normalize_response` is public; the SSOT direct-dispatch path (`ssot_v3_direct_dispatch` in `route_attempts.rb`) wraps the raw provider `Message` in a canonical `Canonical::Response` at the executor consumption point, so the tool loop no longer NoMethodErrors on duck-typed `result[:tool_calls]`. Regression: `dispatch_message_normalization_spec.rb`.
+- **SelectionDispatch logs the original dispatch error.** The `normalize` rescue records the original error's class + message (scrubbed), so a raising normalizer (e.g. a non-UTF-8 `ArgumentError` masked as `reason is not valid UTF-8`) can no longer hide the real error behind an infinite-retry 500.
+- **Bounded 529 / typed rejection diagnostics.** A required capability that is `:unknown` on a settled (`:complete`) candidate set is now a terminal typed 400 instead of unbounded 529 `too_early` retries; tripped instances report before unknown (503, not 529); `too_early` is reserved for genuinely-`:initializing` candidates.
+- **Pin-aware `fit_available` (release bar 6a).** A pinned tools+thinking request against an unserviceable provider no longer falls to unbounded 529 when pin-mismatched fit+available siblings exist — it returns the terminal typed 400.
+- **Per-instance tuning keyed by config name.** `SettingsSnapshot` weight/`preferred_context` lookups used the derived `host:port/ak` id while config is keyed by name — silently inert. Tuning now resolves via the shared `SettingsCascade` keyed by the operator's config name.
+- **Upper-exclusive binning seam restored.** `preferred_context_sieve` is back to `budget < max`; adjacent bins no longer double-match at the shared boundary.
+
+### Changed
+- **`enable_*` keys are operator overrides consumed by the router.** The `enable_thinking`/`enable_tools`/`enable_streaming` cascade (provider → instance → model) is applied as a routing override — `true` satisfies the axis, `false` makes the candidate ineligible, unset falls back to provider evidence. Provider capability evidence itself is unchanged.
+- **`legion-settings >= 1.4.2` floor.** The published 1.4.0 gem resolved nested `lex-llm-*` extensions to flat settings keys, leaving `settings[:instances]` nil so discovery actors saw zero instances; 1.4.2 ships the segments-based nested-path fix.
+- **Fail-forward release bar locked in.** New 12-example frozen-config behavioral spec (`ssot_v3_fail_forward_release_bar_spec.rb`) drives the real router against the no-regression bar: fail forward, no config changes required, typed failures, recovery without restart.
+
+## [0.16.1] - 2026-08-13
+
+### Fixed
+- **P1: native_dispatch_options dropped temperature and all generation params before provider dispatch in the native tool loop.** Root cause of the parallel tool-call dead stop: `native_dispatch_options` (tool_injection.rb) built the dispatch Hash from request fields but never included generation sampling params from `@request.generation`. The provider receives `opts[:temperature]` — when nil, vLLM/Ollama/any provider runs at its default temperature instead of the caller's explicit `temperature: 0`, producing nondeterministic output. For tool-call requests, this manifests as empty-argument tool calls (the model commits to tool call openers but generates EOS before argument tokens). Now propagates the full canonical generation params (temperature, top_p, top_k, frequency_penalty, presence_penalty, seed) via `apply_generation_params!`, using `.key?` to preserve explicit 0 values. Provider-agnostic fix at the executor boundary — all providers benefit.
+
+## [0.16.0] - 2026-08-13
+
+### Fixed
+- **Never-recovering open circuit eliminated (routing incident).** An exact provider+instance whose circuit tripped could stay open permanently, yielding continual `no_lanes_available`. The legacy `HealthTracker` circuit engine (opened on error; its success-close path was dead behind a hardcoded gate) is deleted. The only health mechanism now is a normalized `instance_unavailable` mark on the exact instance via the Phase-1 Registry, cleared automatically when provider readiness republishes the instance (probe-cleared recovery). Transient provider errors (overload/timeout/429/5xx) stay request-local and never poison an instance. Locked in by `spec/legion/llm/router/ssot_v3_instance_recovery_regression_spec.rb`.
+
+### Changed
+- **SSOT v3: one stateless selector.** `Router.next_lane(requirements:, exclusions:, snapshot:)` is the single selection path — a pure function of one Registry snapshot plus one settings snapshot. Removed every competing selector and legacy fallback: `Router.request_lane` and its hard-filter helpers, `infer_provider_for_model`, `inventory_default_model`, the executor's old resolution chain, the `chat_single`/`chat_with_escalation` escalation cluster, and the `pipeline_enabled?` routing gate — `dispatch_chat` now always routes through the pipeline. Exhaustion/empty returns a typed, retriable Rejection (`service_unavailable`/`too_early`, 503/425/529 + Retry-After), never `nil` or a fabricated default.
+- **Dead modules removed.** Deleted `Router::Availability` and `Inventory::Sweeper` (unreachable).
+
+### Notes
+- Tracked follow-ups (out of scope for this change): `lex-llm-*` provider gems auto-publishing to the Phase-1 Registry at boot — unblocks removing the residual `Call::Registry`/`Call::Providers` boot layer; and wiring GAIA `preferred_provider`/`preferred_model` into `RequestRequirements` pins on the SSOT path (`CandidateEvaluator` already enforces pins).
+
 ## [0.15.2] - 2026-08-04
 
 ### Fixed

data/legion-llm.gemspec

@@ -35,10 +35,10 @@ Gem::Specification.new do |spec|
   spec.add_dependency 'legion-cache', '>= 1.4.2'
   spec.add_dependency 'legion-json', '>= 1.2.0'
   spec.add_dependency 'legion-logging', '>= 1.2.8'
-  spec.add_dependency 'legion-settings', '>= 1.4.0'
+  spec.add_dependency 'legion-settings', '>= 1.4.2'
   spec.add_dependency 'legion-transport', '>= 1.4.14'
   spec.add_dependency 'lex-knowledge'
-  spec.add_dependency 'lex-llm', '>= 0.6.3'
+  spec.add_dependency 'lex-llm', '>= 0.7.0'
   spec.add_dependency 'pdf-reader'
   spec.add_dependency 'sinatra-contrib', '>= 2.0'
   spec.add_dependency 'tzinfo', '>= 2.0'

data/lib/legion/llm/api/client_translators/anthropic_messages.rb

@@ -101,6 +101,10 @@ module Legion
               messages:        messages,
               system:          canonical_request.system,
               routing:         canonical_request.routing,
+              # Body-model routing hint (SSOT v3 D19): the sole input to
+              # BodyModelHintPolicy. The trusted X-Legion-Model pin in
+              # `routing` supersedes it — RequestRequirements keeps that order.
+              client_model:    canonical_request.metadata[:client_model],
               tools:           tool_defs,
               tool_choice:     canonical_request.tool_choice,
               caller:          server_caller,

data/lib/legion/llm/api/client_translators/openai_chat.rb

@@ -95,6 +95,10 @@ module Legion
               messages:        messages,
               system:          canonical_request.system,
               routing:         canonical_request.routing,
+              # Body-model routing hint (SSOT v3 D19): the sole input to
+              # BodyModelHintPolicy. The trusted X-Legion-Model pin in
+              # `routing` supersedes it — RequestRequirements keeps that order.
+              client_model:    canonical_request.metadata[:client_model],
               tools:           tool_defs,
               tool_choice:     canonical_request.tool_choice,
               caller:          server_caller,
@@ -305,7 +309,7 @@ module Legion
               # 'tool_calls' on the final done covers it.
             end
 
-            def on_tool_call_abort(block_index:, reason:) # rubocop:disable Lint/UnusedMethodArgument
+            def on_tool_call_abort(**)
               nil
             end
 

data/lib/legion/llm/api/client_translators/openai_responses.rb

@@ -97,6 +97,10 @@ module Legion
               messages:        messages,
               system:          canonical_request.system,
               routing:         canonical_request.routing,
+              # Body-model routing hint (SSOT v3 D19): the sole input to
+              # BodyModelHintPolicy. The trusted X-Legion-Model pin in
+              # `routing` supersedes it — RequestRequirements keeps that order.
+              client_model:    canonical_request.metadata[:client_model],
               tools:           tool_defs,
               tool_choice:     canonical_request.tool_choice,
               caller:          server_caller,
@@ -395,7 +399,7 @@ module Legion
               @output_items[state[:output_index]] = completed
             end
 
-            def on_tool_call_abort(block_index:, reason:) # rubocop:disable Lint/UnusedMethodArgument
+            def on_tool_call_abort(**)
               nil
             end
 

data/lib/legion/llm/api/error_translator.rb

@@ -1,6 +1,7 @@
 # frozen_string_literal: true
 
 require 'legion/logging/helper'
+require 'legion/llm/api/routing_error_mapper'
 
 module Legion
   module LLM
@@ -10,6 +11,8 @@ module Legion
       # NoLaneAvailable  → 400  (filters excluded everything; caller can fix the request)
       # EscalationExhausted → 503 + Retry-After (tried lanes, all failed; transient upstream degradation)
       # InvalidHeader    → 400  (x-legion-* header carries unrecognized value; caller can fix)
+      # RoutingRejected  → SSOT v3 §18 dialect status table (native 425 / openai 503 /
+      #                    anthropic 529 for too_early, etc.) via RoutingErrorMapper.
       #
       # Included into API Helpers so every inference route gets the mapping without
       # duplicating rescue clauses. Must be included BEFORE the route-level rescue
@@ -65,6 +68,22 @@ module Legion
           status 400
           Legion::JSON.dump(body)
         end
+
+        # SSOT v3 §18 / D16: render a typed Routing::Rejection carried by
+        # Errors::RoutingRejected through the dialect status/header/body table.
+        # dialect is :native | :openai | :anthropic. RoutingErrorMapper owns the
+        # status divergence (native 425 vs openai 503 vs anthropic 529 for
+        # too_early) and the Retry-After header for retryable kinds.
+        def translate_routing_rejected(error, dialect:, operation:)
+          handle_exception(error, level: :warn, handled: true, operation: operation)
+          response = Legion::LLM::API::RoutingErrorMapper.call(
+            rejection: error.rejection, dialect: dialect
+          )
+          response.headers.each { |k, v| headers k => v }
+          content_type :json
+          status response.status
+          Legion::JSON.dump(response.body)
+        end
       end
     end
   end

data/lib/legion/llm/api/model_catalog.rb

@@ -0,0 +1,254 @@
+# frozen_string_literal: true
+
+require 'time'
+require 'legion/logging/helper'
+
+module Legion
+  module LLM
+    module API
+      # Snapshot-only model catalog — §17.3 / D19 Copilot compatibility.
+      #
+      # .list  → frozen Array<Hash>  all models in the requested dialect
+      # .fetch → frozen Hash or nil  single-model lookup; nil if not catalog-visible
+      #
+      # The catalog iterates the supplied snapshot once and NEVER calls
+      # Router.next_lane, Ranker, or any callable.  Selection, lane weight,
+      # and availability do not determine which models appear in the compat
+      # view — only publication completion, supported-operation evidence, and
+      # the §9.5 model policy (from settings_snapshot.model_policy_for) do.
+      #
+      # X-Legion-Model is intentionally exempt from the body-hint policy
+      # evaluated here. This module handles only compat-view construction;
+      # it does NOT re-apply D19 body-hint logic during GET /v1/models listing.
+      module ModelCatalog
+        include Legion::Logging::Helper
+        extend  Legion::Logging::Helper
+
+        VALID_DIALECTS     = %i[native openai anthropic].freeze
+        OPENAI_OBJECT_TYPE = 'model'
+        DEFAULT_OWNED_BY   = 'legion'
+        private_constant :VALID_DIALECTS, :OPENAI_OBJECT_TYPE, :DEFAULT_OWNED_BY
+
+        # ------------------------------------------------------------------ #
+        # Public API                                                           #
+        # ------------------------------------------------------------------ #
+
+        # Returns a frozen Array<Hash> of model objects for the requested dialect.
+        # dialect must be :native, :openai, or :anthropic; any other value raises
+        # ArgumentError immediately.
+        def self.list(snapshot:, settings_snapshot:, dialect:)
+          validate_dialect!(dialect)
+          log.debug("[llm][model_catalog] action=list dialect=#{dialect} " \
+                    "snapshot_generation=#{snapshot.generation} " \
+                    "settings_generation=#{settings_snapshot.generation}")
+          case dialect
+          when :native    then native_list(snapshot: snapshot)
+          when :openai    then compat_list(snapshot: snapshot, settings_snapshot: settings_snapshot, dialect: :openai)
+          when :anthropic then compat_list(snapshot: snapshot, settings_snapshot: settings_snapshot, dialect: :anthropic)
+          end
+        end
+
+        # Returns the frozen dialect model object for +id+, or nil when:
+        # - compat dialect: the model is not catalog-visible (body hint would be
+        #   ignored or not whitelisted); auto-routing alias returns nil when the
+        #   compat set is empty.
+        # - native dialect: the id does not match any offering in the snapshot.
+        def self.fetch(id:, snapshot:, settings_snapshot:, dialect:)
+          validate_dialect!(dialect)
+          list(snapshot: snapshot, settings_snapshot: settings_snapshot, dialect: dialect)
+            .find { |m| m[:id] == id.to_s }
+        end
+
+        # ------------------------------------------------------------------ #
+        # Private class methods                                                #
+        # ------------------------------------------------------------------ #
+
+        # Raise ArgumentError unless dialect is one of the three accepted symbols.
+        def self.validate_dialect!(dialect)
+          return if VALID_DIALECTS.include?(dialect)
+
+          raise ArgumentError,
+                "[llm][model_catalog] dialect must be one of #{VALID_DIALECTS.inspect}, " \
+                "got #{dialect.inspect}"
+        end
+        private_class_method :validate_dialect!
+
+        # Native: complete diagnostic view of every offering.
+        # Every offering is included regardless of availability or policy so
+        # operators can see exactly what the registry holds for diagnostics.
+        # Enriched with the exact instance availability state and the publication
+        # state from the three Phase 1 snapshot enumerators.
+        def self.native_list(snapshot:)
+          pub_by_key  = {}
+          inst_by_key = {}
+          snapshot.each_publication_status { |ps|   pub_by_key[ps.instance_key] = ps }
+          snapshot.each_instance           { |inst| inst_by_key[inst.instance_key] = inst }
+
+          entries = []
+          snapshot.each_offering do |offering|
+            ik = offering.instance_key
+            entries << {
+              id:                     offering.model.to_s,
+              offering_id:            offering.offering_id.to_s,
+              provider_family:        ik.provider_family.to_s,
+              instance_id:            ik.instance_id.to_s,
+              tier:                   offering.tier.to_s,
+              supported_operations:   offering.supported_operations.map(&:to_s).freeze,
+              unsupported_operations: offering.unsupported_operations.map(&:to_s).freeze,
+              unknown_operations:     offering.unknown_operations.map(&:to_s).freeze,
+              publication_state:      pub_by_key[ik]&.state&.to_s,
+              availability_state:     inst_by_key[ik]&.availability&.state&.to_s, # rubocop:disable Style/SafeNavigationChainLength
+              publication_source:     offering.publication_source.to_s,
+              metadata:               offering.metadata
+            }.freeze
+          end
+          entries.freeze
+        end
+        private_class_method :native_list
+
+        # Compat: one entry per unique model that has at least one COMPLETE,
+        # policy-permitted offering with a SUPPORTED operation.
+        # Availability does NOT remove a model; an initializing claim with no
+        # offering does NOT manufacture one (§17.3 behavioural rule).
+        # Auto-routing aliases are appended only when the compat set is non-empty.
+        def self.compat_list(snapshot:, settings_snapshot:, dialect:)
+          # One-pass publication-status index (keyed by InstanceKey).
+          pub_by_key = {}
+          snapshot.each_publication_status { |ps| pub_by_key[ps.instance_key] = ps }
+
+          # Collect eligible unique model identifiers; first-seen provider wins
+          # for the owned_by field when the same model appears on multiple providers.
+          seen = {} # model_id (String) => provider_family (String)
+          snapshot.each_offering do |offering|
+            model_id = offering.model.to_s
+            next if seen.key?(model_id)
+
+            ik = offering.instance_key
+            ps = pub_by_key[ik]
+
+            # Publication must be complete (not initializing).
+            next unless ps&.state == :complete
+
+            # Must advertise at least one supported operation.
+            next if offering.supported_operations.empty?
+
+            # §9.5 fail-closed whitelist-AND-blacklist model policy.
+            next unless policy_permits?(offering: offering, settings_snapshot: settings_snapshot)
+
+            seen[model_id] = ik.provider_family.to_s
+          end
+
+          # Build sorted, frozen model entries for deterministic list order.
+          entries = seen.keys.sort.map do |model_id|
+            format_compat_entry(id: model_id, owned_by: seen[model_id], dialect: dialect)
+          end
+
+          # Auto-routing aliases are appended ONLY when the compat set is non-empty.
+          # Alias copilot-utility-small MUST return owned_by: 'legionio' via its
+          # configured alias metadata.
+          unless entries.empty?
+            settings_snapshot.auto_routing_model_aliases.each do |alias_id|
+              entries << format_alias_entry(
+                alias_id:          alias_id,
+                settings_snapshot: settings_snapshot,
+                dialect:           dialect
+              )
+            end
+          end
+
+          entries.freeze
+        end
+        private_class_method :compat_list
+
+        # §9.5 fail-closed primitive: case-insensitive literal substring matching.
+        # A nonempty effective whitelist must match the offering model.
+        # A matching blacklist always denies, including when the whitelist matched.
+        def self.policy_permits?(offering:, settings_snapshot:)
+          policy    = settings_snapshot.model_policy_for(offering: offering)
+          whitelist = policy[:whitelist]
+          blacklist = policy[:blacklist]
+          model_lc  = offering.model.to_s.downcase
+
+          # nonempty whitelist — at least one entry must match
+          return false if whitelist.any? && whitelist.none? { |e| model_lc.include?(e.downcase) }
+
+          # any blacklist match denies, even when whitelist also matched
+          return false if blacklist.any? { |e| model_lc.include?(e.downcase) }
+
+          true
+        end
+        private_class_method :policy_permits?
+
+        # Build a real-model compat entry. No limits for real models because they
+        # are deduplicated across providers and no single canonical context window
+        # is available without selecting a specific lane.
+        def self.format_compat_entry(id:, owned_by:, dialect:)
+          case dialect
+          when :openai    then format_openai_entry(id: id, owned_by: owned_by)
+          when :anthropic then format_anthropic_entry(id: id)
+          end
+        end
+        private_class_method :format_compat_entry
+
+        # Build an auto-routing alias entry.
+        # Limits: alias metadata[:context_window/:max_output_tokens] first, then
+        # the registered llm.context_window / llm.max_output_tokens envelope.
+        # No field comes from a selected, max-weight, or first lane.
+        def self.format_alias_entry(alias_id:, settings_snapshot:, dialect:)
+          meta     = settings_snapshot.auto_routing_model_alias_metadata[alias_id] || {}
+          owned_by = meta[:owned_by] || DEFAULT_OWNED_BY
+          created  = meta[:created]
+          ctx      = meta[:context_window]    || Legion::Settings[:llm][:context_window]
+          max_out  = meta[:max_output_tokens] || Legion::Settings[:llm][:max_output_tokens]
+          limits   = { context_window: ctx, max_output_tokens: max_out }.freeze
+
+          case dialect
+          when :openai
+            format_openai_entry(id: alias_id, owned_by: owned_by, created: created, limits: limits)
+          when :anthropic
+            format_anthropic_entry(id: alias_id, created: created, limits: limits)
+          end
+        end
+        private_class_method :format_alias_entry
+
+        # OpenAI model object shape: { id:, object: 'model', created:, owned_by: }
+        # Limits are optional and included only for alias entries.
+        def self.format_openai_entry(id:, owned_by:, created: nil, limits: nil)
+          obj = {
+            id:       id.to_s,
+            object:   OPENAI_OBJECT_TYPE,
+            created:  created || Time.now.to_i,
+            owned_by: owned_by.to_s
+          }
+          if limits.is_a?(Hash)
+            if limits[:context_window]
+              obj[:context_window] = limits[:context_window]
+              obj[:context_size]   = limits[:context_window]
+            end
+            obj[:max_output_tokens] = limits[:max_output_tokens] if limits[:max_output_tokens]
+          end
+          obj.freeze
+        end
+        private_class_method :format_openai_entry
+
+        # Anthropic model object shape: { type: 'model', id:, display_name:, created_at: }
+        # Limits are optional and translated to max_input_tokens / max_tokens.
+        def self.format_anthropic_entry(id:, created: nil, limits: nil)
+          ts  = created || Time.now.to_i
+          obj = {
+            type:         'model',
+            id:           id.to_s,
+            display_name: id.to_s,
+            created_at:   Time.at(ts).utc.strftime('%Y-%m-%dT%H:%M:%SZ')
+          }
+          if limits.is_a?(Hash)
+            obj[:max_input_tokens] = limits[:context_window]    if limits[:context_window]
+            obj[:max_tokens]       = limits[:max_output_tokens] if limits[:max_output_tokens]
+          end
+          obj.freeze
+        end
+        private_class_method :format_anthropic_entry
+      end
+    end
+  end
+end

data/lib/legion/llm/api/namespaces/anthropic/messages/batches.rb

@@ -156,13 +156,11 @@ module Legion
                   batch_store.delete(id)
                 end
 
-                # rubocop:disable Lint/UnusedMethodArgument
-                def list_batches(limit:, before_id: nil, after_id: nil)
+                def list_batches(limit:, **)
                   all = batch_store.values.map { |e| e[:batch] }.sort_by { |b| b[:created_at] }.reverse
                   all = all.first(limit.clamp(1, 100))
                   { data: all, has_more: false, first_id: all.first&.dig(:id), last_id: all.last&.dig(:id) }
                 end
-                # rubocop:enable Lint/UnusedMethodArgument
 
                 def batch_results(id)
                   batch_store.dig(id, :results) || []

data/lib/legion/llm/api/namespaces/anthropic/messages/count_tokens.rb

@@ -13,6 +13,7 @@ module Legion
           module Messages
             module CountTokens
               extend Sinatra::Extension
+              extend Legion::Logging::Helper
 
               post '/count_tokens' do
                 require_llm!
@@ -26,7 +27,7 @@ module Legion
                        Legion::JSON.dump({ type: 'error', error: { type: 'invalid_request_error', message: "missing required fields: #{missing.join(', ')}" } })
                 end
 
-                result = Legion::LLM::TokenEstimation.estimate(
+                result = CountTokens.count_tokens_result(
                   messages: body[:messages],
                   model:    body[:model],
                   system:   body[:system],
@@ -40,6 +41,65 @@ module Legion
                 handle_exception(e, level: :error, handled: true, operation: 'llm.ns.anthropic.count_tokens')
                 anthropic_error('api_error', e.message, status_code: 500)
               end
+
+              # SSOT v3 §20 (count_tokens row): choose the EXACT lane first, then use
+              # that lane's callable tokenizer. `model` is a required, caller-supplied
+              # value — never a default. When the SSOT registry publishes a
+              # count_tokens lane for the requested model, the exact selected
+              # callable's tokenizer is authoritative; otherwise (cold registry, no
+              # count_tokens lane, or a callable without a tokenizer) fall back to the
+              # provider-neutral TokenEstimation.
+              def self.count_tokens_result(messages:, model:, system:, tools:)
+                lane_count = count_tokens_via_selected_lane(messages: messages, model: model, system: system, tools: tools)
+                return lane_count if lane_count
+
+                Legion::LLM::TokenEstimation.estimate(messages: messages, model: model, system: system, tools: tools)
+              end
+
+              def self.count_tokens_via_selected_lane(messages:, model:, system:, tools:)
+                snapshot = Legion::Extensions::Llm::Inventory::Registry.snapshot
+                return nil unless snapshot.generation.positive?
+
+                request = Legion::LLM::Inference::Request.build(
+                  messages: messages,
+                  system:   system,
+                  routing:  { model: model },
+                  tools:    tools || []
+                )
+                requirements = Legion::LLM::Router::RequestRequirements.build(
+                  request:                request,
+                  operation:              :count_tokens,
+                  required_capabilities:  Legion::LLM::Router::RequiredCapabilities.call(request: request, operation: :count_tokens),
+                  estimated_input_bound:  0,
+                  required_output_tokens: 0
+                )
+                session = Legion::LLM::Inference::RoutingSession.new(request: request, requirements: requirements)
+                attempt = session.next_attempt(snapshot: snapshot)
+                return nil if attempt.is_a?(Legion::Extensions::Llm::Routing::Rejection)
+
+                dispatch = Legion::LLM::Call::SelectionDispatch.call(
+                  attempt_context: attempt,
+                  arguments:       { messages: messages }
+                )
+                return nil unless dispatch.success?
+
+                normalize_token_count(dispatch.value)
+              rescue ::NoMethodError, ::ArgumentError, ::NotImplementedError
+                # Programming errors are never swallowed — re-raise.
+                raise
+              rescue StandardError => e
+                # Non-programming errors during lane selection/tokenizer dispatch fall
+                # back to provider-neutral estimation.
+                log.warn "[llm][ns][anthropic][count_tokens] action=lane_tokenizer_fallback error=#{e.class}: #{e.message}"
+                nil
+              end
+
+              def self.normalize_token_count(value)
+                return value if value.is_a?(Hash) && (value.key?(:input_tokens) || value.key?('input_tokens'))
+                return { input_tokens: value } if value.is_a?(Integer)
+
+                nil
+              end
             end
           end
         end

data/lib/legion/llm/api/namespaces/anthropic/messages.rb

@@ -6,6 +6,7 @@ require 'sinatra/namespace'
 require 'legion/logging/helper'
 require 'legion/llm/api/client_translators/anthropic_messages'
 require 'legion/llm/api/stream_assembler'
+require 'legion/llm/api/routing_error_mapper'
 require 'legion/llm/api/debug_formats'
 
 module Legion
@@ -48,6 +49,11 @@ module Legion
               echo_request = Legion::LLM::API::DebugFormats.echo_request?(env)
 
               if streaming
+                # SSOT v3 §19: select + acquire the exact lane BEFORE opening SSE.
+                # A routing rejection raises Errors::RoutingRejected here (rescued
+                # below → RoutingErrorMapper) instead of an SSE error after headers.
+                preflight_lane = executor.stream_preflight!
+
                 content_type 'text/event-stream'
                 headers 'Cache-Control' => 'no-cache', 'Connection' => 'keep-alive',
                         'X-Accel-Buffering' => 'no', 'X-Legion-Conversation-Id' => conv_id
@@ -65,9 +71,9 @@ module Legion
                     request_id:   request_id,
                     model:        model,
                     input_tokens: estimate_input_tokens(inference_request.messages),
-                    initial_lane: { id: 'unknown:pending' }
+                    initial_lane: preflight_lane || { id: 'unknown:pending' }
                   )
-                  pipeline_response = executor.call_stream do |chunk|
+                  pipeline_response = executor.call_stream(stream_observer: assembler) do |chunk|
                     assembler.push(chunk)
                   end
                   assembler.finalize(pipeline_response)
@@ -121,6 +127,8 @@ module Legion
               translate_escalation_exhausted(e, operation: 'llm.ns.anthropic.messages.exhausted', client: :anthropic)
             rescue Legion::LLM::Errors::InvalidHeader => e
               translate_invalid_header(e, operation: 'llm.ns.anthropic.messages.invalid_header', client: :anthropic)
+            rescue Legion::LLM::Errors::RoutingRejected => e
+              translate_routing_rejected(e, dialect: :anthropic, operation: 'llm.ns.anthropic.messages.routing_rejected')
             rescue Legion::LLM::AuthError => e
               handle_exception(e, level: :error, handled: true, operation: 'llm.ns.anthropic.messages.auth')
               anthropic_error('authentication_error', e.message, status_code: 401)

data/lib/legion/llm/api/namespaces/helpers.rb

@@ -13,11 +13,12 @@ module Legion
           include Legion::LLM::API::SharedHelpers
           include Legion::LLM::API::ErrorTranslator
 
-          def openai_error(message, type: 'server_error', code: nil, status_code: 500)
+          # OpenAI error envelope — ALWAYS carries all four keys (message, type,
+          # param, code); param/code are null when not applicable.
+          def openai_error(message, type: 'server_error', code: nil, param: nil, status_code: 500)
             content_type :json
             status status_code
-            body = { error: { message: message, type: type } }
-            body[:error][:code] = code if code
+            body = { error: { message: message, type: type, param: param, code: code } }
             Legion::JSON.dump(body)
           end
 
@@ -48,7 +49,6 @@ module Legion
           # Called by inference routes before building the executor request.
           def validate_legion_routing_headers!(rack_env)
             http_headers = rack_env.select { |k, _| k.start_with?('HTTP_') }
-            return unless defined?(Legion::LLM::Inference::Executor::PayloadBuilder)
 
             # Build a flat header map from Rack env (HTTP_X_LEGION_TIERS → x-legion-tiers)
             mapped = http_headers.each_with_object({}) do |(k, v), h|
@@ -65,7 +65,8 @@ module Legion
             end
           rescue Legion::LLM::Errors::InvalidHeader
             raise
-          rescue StandardError
+          rescue StandardError => e
+            handle_exception(e, level: :warn, handled: true, operation: 'llm.api.validate_legion_routing_headers')
             nil
           end
 

data/lib/legion/llm/api/namespaces/native/inference.rb

@@ -241,6 +241,8 @@ module Legion
                   payload.compact!
                   json_response(payload, status_code: 200)
                 end
+              rescue Legion::LLM::Errors::RoutingRejected => e
+                translate_routing_rejected(e, dialect: :native, operation: 'llm.api.inference.routing_rejected')
               rescue Legion::LLM::AuthError => e
                 handle_exception(e, level: :error, handled: true, operation: 'llm.api.inference.auth', request_id: request_id)
                 json_error('auth_error', e.message, status_code: 401)

data/lib/legion/llm/api/namespaces/native/offerings.rb

@@ -21,7 +21,7 @@ module Legion
                 require_llm!
 
                 filters = Legion::LLM::API::Native::Offerings.request_filters(params)
-                raw_offerings = Legion::LLM::Inventory.offerings(filters)
+                raw_offerings = Legion::LLM::API::Native::Offerings.snapshot_offerings(filters)
                 grouped = Legion::LLM::API::Native::Offerings.group_offerings(raw_offerings)
 
                 json_response({
@@ -38,7 +38,7 @@ module Legion
                 log.debug("[llm][api][namespaces][offerings] action=get_offering id=#{offering_id}")
                 require_llm!
 
-                offering = Legion::LLM::Inventory.offerings(offering_id: offering_id).first
+                offering = Legion::LLM::API::Native::Offerings.snapshot_offering(offering_id)
                 halt json_error('offering_not_found', "Offering '#{offering_id}' not found", status_code: 404) unless offering
 
                 json_response({ offering: offering })

data/lib/legion/llm/api/namespaces/native/routing.rb

@@ -19,9 +19,11 @@ module Legion
                 log.debug('[llm][api][namespaces][routing] action=list_rules')
                 require_llm!
 
+                # SSOT has no routing toggle or rule list: routing is enabled
+                # whenever the Registry holds at least one complete publication.
                 json_response({
-                                routing_enabled:      false,
-                                auto_rules_populated: false,
+                                routing_enabled:      Legion::LLM::Router.routing_enabled?,
+                                auto_rules_populated: Legion::LLM::Router.auto_rules_populated?,
                                 rules:                [],
                                 summary:              { total: 0, auto: 0, manual: 0 }
                               })