legion-llm 0.13.0 → 0.14.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: eaa596812e2320baa0d8ae31b44225c80e9ad9b54e62531b3cd1c7640ff09fe6
-  data.tar.gz: a98b90ae07c7040fcd8a13adccf1d07037f253cc845cdc4dc55b65f434805ce0
+  metadata.gz: dac2aadf427ccf4c6e902242ed392c3f9f0f5a3e0ddaeeb6d6c262e3926c87b1
+  data.tar.gz: 48ccd139e47330107147d7745ec7e34c7a1de879cc8939d2d1fd33322d04b336
 SHA512:
-  metadata.gz: a7611f997b2163792aa4f29d8ca2e3b8c10ec11af08ff8d89fae578ab0b0a138fbd9ad4957a5c42b793c30e959ab4c17290f01972cd0d8de8128e7e79873e28c
-  data.tar.gz: 3b0acc643ffe9c06c6d07df9c5dce9a54d79351878354e30c1cc22fb5c5001debf9ac529ddeb873868ff713f455665c6e5fe992558d70ef376151f9b368b7086
+  metadata.gz: 27c3269042cf0b060f0e402fad967f048319a2a98a425d4b68f39569d07e41842505bf947f32ae62a52125a0c81143e4d1fcf076bee53e247fd0cfe76a9e7f6a
+  data.tar.gz: 1fcd99c1a3007b858bc033f57a1cd3ada1fd96e2729d597fd65f8ac8eeb406295f1b73a2d4bcea31793ca6669b0bba545c32ac86474e2c9f1a5adcc60f9550f2

data/.rubocop.yml

@@ -12,6 +12,31 @@ Legion/RescueLogging/NoCapture:
 Legion/ConstantSafety/InheritParam:
   Enabled: false
 
+# rubocop-legion 0.1.9 cops — enabled as of P0 but deferred enforcement:
+# these four cops flag broad pre-existing patterns in the codebase that
+# predate the SSOT refactor. They gate NEW code from P1 onward; existing
+# violations are cleaned up within each phase as the code they govern is
+# rewritten. Do not add new violations; do not suppress them with inline
+# rubocop:disable without a tracking comment.
+#
+# TODO(P1): enable Legion/Llm/TaxonomyEnum repo-wide after lane taxonomy
+#           is established and all :type/:tier/:circuit_state literals updated.
+# TODO(P1): enable Legion/Llm/RescueLogLevel repo-wide after back-compat
+#           :debug rescue handlers are audited and leveled up.
+# TODO(P1): enable Legion/Llm/NoLoopDo repo-wide after cache/drain loops
+#           are converted to bounded iteration.
+Legion/Llm/TaxonomyEnum:
+  Enabled: false
+Legion/Llm/RescueLogLevel:
+  Enabled: false
+Legion/Llm/NoLoopDo:
+  Enabled: false
+# SettingsAccessPath is enabled for lib/ only — specs legitimately write
+# settings via the loader path to set up test fixtures.
+Legion/Llm/SettingsAccessPath:
+  Exclude:
+    - 'spec/**/*'
+
 AllCops:
   TargetRubyVersion: 3.4
   NewCops: enable
@@ -106,6 +131,7 @@ Naming/PredicateMethod:
   Enabled: false
 Metrics/ParameterLists:
   Max: 9
+  CountKeywordArgs: false
 Style/RedundantConstantBase:
   Exclude:
     - 'spec/**/*'

data/CHANGELOG.md

@@ -1,5 +1,185 @@
 # Legion LLM Changelog
 
+## [0.14.3] - 2026-06-22
+
+### Fixed
+
+- Context window enforcement now accounts for tool definition tokens when deciding whether to compact messages. Previously, 235 injected tools (~50k tokens) were invisible to the threshold check, causing context overflow on dispatch.
+- Context overflow errors are no longer unconditionally terminal during escalation. When a lane with a larger context window exists, the executor retries on that lane instead of raising immediately.
+- `X-Legion-Format: canonical` debug surface enabled by default (was gated to dev/lite mode only). The canonical format is a first-class output format, not a debug feature.
+- Completion log now includes `context_tokens_saved=N` showing total tokens saved by curation, archival, thinking-strip, and context window compaction per request.
+- Embedding dispatch now sends chunks individually instead of as an array, fixing intermittent `ContextOverflow` on RAG embedding queries with ollama.
+- Truncated tool results now instruct the model to make smaller targeted requests instead of silently chopping content.
+- `context_window_threshold` moved from hardcoded 0.90 to `context_curation_defaults` setting.
+
+### Changed
+
+- `tool_result_max_dispatch_chars` raised from 2k to 5k (dispatch-time truncation of oversized tool results before provider call).
+- `context_curation.tool_result_max_chars` lowered from 10k to 2k (curator distills tool results earlier, reducing context growth).
+- Default `tier_weights` rebalanced: direct=105, local=110, fleet=110, cloud=120, frontier=150 (prefer higher-capability tiers when lane_weight tie-breaks).
+
+## [0.14.2] - 2026-06-20
+
+### Fixed
+
+- Move auto-routing model aliases into `llm.routing.auto_routing_model_aliases`, so `legionio` and `auto` stay configurable rather than hard-coded.
+- Ignore request-body `model` values as routing hints unless `llm.routing.allow_body_routing_hints` is explicitly enabled; auto-routing aliases still mean "you pick".
+- Stop treating injected special tools as an implicit native-tools routing requirement when the client did not actually request tools.
+
+## [0.14.1] - 2026-06-20
+
+### Fixed
+
+- Treat Bedrock region-prefixed model ids such as `us.anthropic.claude-sonnet-4-6` as equivalent to
+  the inventory's canonical `anthropic.claude-sonnet-4-6` lane during hard model filtering, so
+  routing no longer raises `NoLaneAvailable` for valid Bedrock requests.
+
+## [0.14.0] - 2026-06-19
+
+### Changed (BREAKING — internal API)
+
+- **Inventory is now a single live `Concurrent::Map`** keyed by 5-part lane id
+  `tier:provider:instance:type:model`. The catalog is composed on write (by `lex-llm-*` discovery
+  actors via the `Inventory::ScopedRefresher` mixin), not recomposed on read. Per-request
+  `offerings_calls` collapses from ~4N to ≤1.
+- **`Router.request_lane(**routing_payload)` is the single selection method.** `Router.resolve`,
+  `Router.resolve_chain`, `Router::Candidates`, `Router::EscalationChain`, `Arbitrage`, and the
+  full chain-building machinery are deleted.
+- **`HealthTracker` writes lane health one-directionally.** The old request-time read API
+  (`circuit_state(provider:, instance:)`, `adjustment(...)`, `model_denied?(...)`) is deleted.
+  Health is now read from `lane[:health]`.
+- **Dual error classes replace the old `EscalationExhausted`.** `Errors::NoLaneAvailable` (HTTP 400;
+  filters excluded all candidates from the start) and `Errors::EscalationExhausted` (HTTP 503 +
+  `Retry-After`; max attempts reached mid-flight) are the new error contract. Both inherit from
+  `LLMError`.
+- **Embedding selection uses `Router.request_lane(type: :embedding, models: [pinned])`.** Strict
+  model pin — no cross-model failover. The bespoke embedding-selection machine is deleted.
+- **`while remaining.positive?` loop replaces `loop do`.** The executor's request lifecycle is
+  bounded by construction; `loop do`, `retry`, `redo` are forbidden by the `NoLoopDo` rubocop cop.
+
+### Added
+
+- `Inventory.write_lane(lane:, ttl:, **)` / `.delete_lane(id:, **)` / `.lane(id:, **)` /
+  `.lanes_for(provider:, instance:, type:, model:, **)` / `.lanes(**)` — kwargs-only public API.
+- `Inventory::Sweeper` `::Every` actor — TTL safety net for dead-actor lane orphans.
+- **RANKING v2:** `lane_weight = tier_w × provider_w × instance_w × model_w × health_mult`,
+  precomputed at write time, surfaced in `/api/llm/providers/<p>/models`. Operator-tunable via
+  settings; all weights default to 100.
+- `Legion::Cache::Local` cooldown circuit for auth failures
+  (`llm_auth_failed:<credential_hash>` key). Short-circuits dispatch during the cooldown window
+  without tripping the instance circuit.
+- `PayloadBuilder` single ingress site at `inference/executor/payload_builder.rb`. Validates
+  `x-legion-tiers`, `x-legion-providers`, `x-legion-instances`, `x-legion-models` headers against
+  frozen taxonomies. Unknown values → 400 with `error.type: invalid_header`.
+- `StreamAssembler` mid-stream failover contract: `provider_failover_pending!(from:)` clears the
+  canonical buffer; `finalize` emits debug trailers (`x-legion-failover-from`, `-to`, `-count`)
+  only when failover occurred. No custom SSE event (N×N invariant 5).
+- Admin endpoint `POST /api/llm/inventory/refresh` — operator-triggered catalog refresh.
+- `:fleet` is a first-class tier in the `Taxonomies::TIERS` enum.
+
+### Deprecated
+
+- `Router.populate_auto_rules(_)` — no-op stub. Removed in v0.15.0 after call sites in `lex-llm-*`
+  gems are cleaned up. Tracking issue: [#154](https://github.com/legion-io/legion-llm/issues/154).
+  Remove-stub issue: [#155](https://github.com/legion-io/legion-llm/issues/155).
+
+### Fixed
+
+- `/v1/moderations` 500 error (missing `Call::Registry.providers` method).
+- Compliance leak via discovery path: denied models could enter `/api/llm/offerings` because the
+  discovery feeder bypassed `lex-llm-*` whitelist/blacklist filtering. `Inventory.write_lane` is
+  now the single fail-closed choke point.
+- Mid-stream provider failover now correctly clears the canonical buffer — no thinking tokens from
+  provider A leak into provider B's response context.
+
+### Removed
+
+- `Legion::LLM::EscalationTracker` (dead code, zero callers).
+- `Inventory#native_provider_offerings`, `discovery_offerings`, `dedupe_offerings`, `build_offering`,
+  `add_fleet_lane`, `compose_offerings` — replaced by `lex-llm-*` gem writers via the
+  `Inventory::ScopedRefresher` mixin.
+- `Call::Registry.all_provider_families` (duplicate of `.available`).
+- Hardcoded last-resort tier model literals.
+- `Providers.inject_anthropic_cache_control!` — moved to `lex-llm-anthropic` translator (CLAUDE.md
+  invariant #3).
+- `lib/legion/llm/discovery.rb`, `lib/legion/llm/capabilities.rb`, `lib/legion/llm/discovery/`
+  compat shim forwarders (module paths moved to `inventory/` tree in v0.13.x; shims deleted in
+  v0.14.0).
+- `Router::Candidates`, `Router::Arbitrage`, `Router::EscalationChain` (all deleted; use
+  `Router.request_lane`).
+
+### Breaking change notes
+
+- **Embedding single-instance HA:** single-instance Ollama (or any single embedding provider) will
+  produce 400 `NoLaneAvailable` during the ~5–10s restart window rather than silently retrying.
+  Use two instances for HA.
+- **Rollback requires yanking the entire train.** `lex-llm 0.6.0`'s `ScopedRefresher` calls
+  `Inventory.write_lane` which does not exist on `legion-llm 0.13.x`. Yanking `legion-llm 0.14.0`
+  alone is insufficient — `lex-llm 0.6.0` and all 9 `lex-llm-*` paired versions must be yanked
+  together. See `docs/migration/0.14.0.md` for the 3am rollback procedure.
+
+---
+
+## [0.13.3] - 2026-06-18
+
+### Fixed
+
+- **OpenAI Responses (`/v1/responses`) tool turns now terminate with `response.completed`.** A turn
+  carrying client-callable `function_call` items was emitting a non-standard `response.done` with
+  `status: requires_action` — Assistants-API vocabulary the Responses protocol has no concept of. Real
+  Responses clients wait for `response.completed`, so each tool turn surfaced to the client as
+  "stream disconnected before completion" and forced a reconnect/retry. The terminal event is now
+  always `response.completed` / `status: completed` with the `function_call` items in `output[]`
+  (streaming **and** non-streaming); `requires_action`/`action_required` removed. Server-executed
+  (LegionIO) tools were already `completed` and are unchanged. Specs updated to assert the protocol.
+- **Router no longer manufactures escalation fallbacks the live catalog doesn't offer.**
+  `build_fallback_resolutions` enumerated registered instances and paired each with a default model
+  without checking the catalog offered it, producing dead candidates (a provider + a model it does
+  not serve) that availability rejected on every request — wasted work plus `resolution_unavailable`
+  log noise. Fallbacks are now gated against `Inventory` (the catalog SSOT) via
+  `fallback_model_offered?`, so an unoffered triple is never proposed.
+
+### Changed
+
+- Removed the per-response `extract_thinking` INFO log spam (it fired 4–6× per request, once per
+  extraction site). The extraction is unchanged; only the diagnostic logging was dropped.
+
+## [0.13.2] - 2026-06-17
+
+### Fixed
+
+- **Discovery no longer blocks the request path on a live network refresh.** `Discovery#discovered_models`
+  used to refresh synchronously once its 60s TTL lapsed — a serial, per-instance live fetch
+  (`adapter.offerings(live: true)`) on the request thread, so one unreachable/slow instance stalled
+  routing for its socket timeout (~20s, recurring ~once a minute). It surfaced as a fast `[pipeline][timing]`
+  with the time hidden in the `routing` step (which reads candidates via `model_available?`/`model_size`).
+  The request path now only **reads** the cache; refresh is owned by the provider `DiscoveryRefresh`
+  `::Every` actors (background) + the startup `Discovery.run` warm.
+
+### Changed
+
+- **Discovered-models cache is a `Concurrent::Map` keyed by provider.** Each provider's refresh actor
+  writes its own key atomically (no read-modify-write across providers, no lock); reads flatten all
+  values lock-free. `@discovery_status` is likewise a `Concurrent::Map` (the `@discovery_mutex` is
+  removed). Dead read-path TTL machinery (`discovered_models_stale?`, `discovery_refresh_seconds`)
+  deleted; the `llm.discovery.refresh_seconds` setting is now inert (actors use their own interval).
+
+## [0.13.1] - 2026-06-17
+
+### Fixed
+
+- **Streamed responses no longer leak Ruby object inspect strings to the client.** The
+  `StreamAssembler::ChunkAdapter` — the single chunk→wire normalizer — rendered provider value
+  objects with `.to_s` when they weren't plain strings, so the client SSE could carry
+  `#<Legion::Extensions::Llm::Thinking:0x…>` (Claude Code `/v1/messages`, via the legacy-chunk
+  `legacy_thinking` path that only checked `#content` while the legacy `Thinking` exposes `#text`)
+  or `[#<data …Canonical::ContentBlock…>]` (Codex `/v1/responses`, via a `text_delta` whose delta
+  arrived as a `ContentBlock` array). Both paths now unwrap to text and never `.to_s` a value
+  object onto the wire. The metering/audit ledger was already clean — only the streaming wire was
+  affected; the in-process matrix did not catch it because the `FakeProvider` emits canonical
+  chunks only (the documented provider-shape blind spot), so the regression is locked by direct
+  `StreamAssembler` specs.
+
 ## [0.13.0] - 2026-06-17
 
 Consolidated release. This single version bundles every change from `0.12.14` through `0.12.35`

data/CLAUDE.md

@@ -1,4 +1,4 @@
-# legion-llm (v0.13.0)
+# legion-llm (v0.14.0)
 
 Core LegionIO gem: LLM routing, provider dispatch, the inference pipeline, and the
 OpenAI/Anthropic-compatible API surface. This file is loaded into **every** session — it is
@@ -31,8 +31,8 @@ push. If a regression breaks live e2e but not the matrix, the matrix is missing
 |------|------|
 | Facade (`start`, `chat`, `ask`, `embed`) | `lib/legion/llm.rb` |
 | **Single source of truth for the catalog** | `lib/legion/llm/inventory.rb` |
-| Router (`resolve`, `resolve_chain`, candidates) | `lib/legion/llm/router.rb`, `router/{candidates,availability,resolution,rule,health_tracker}.rb` |
-| Escalation / failover | `lib/legion/llm/router/escalation/`, `inference/executor/escalation.rb` |
+| Router (`request_lane` — single selection) | `lib/legion/llm/router.rb`, `router/{availability,resolution,health_tracker}.rb` |
+| Escalation history / failover | `lib/legion/llm/router/escalation/history.rb`, `inference/executor/escalation.rb` |
 | Pipeline executor (18 steps, streaming) | `lib/legion/llm/inference/executor.rb` (+ `executor/*.rb`) |
 | Pipeline steps | `lib/legion/llm/inference/steps/*.rb` |
 | Client API routes | `lib/legion/llm/api/openai/`, `api/anthropic/`, `api/native/` |
@@ -75,21 +75,27 @@ These have caused production incidents. They are also enforced by `rubocop-legio
    prompt; server-executed tools run server-side; client-passthrough tools surface as pending
    calls for the client. Simplest end-to-end check that the proxy contract holds in both formats.
 
-## Routing rules (current behaviour)
-
-- **`Inventory.offerings` is THE catalog** (registration + liveness + health/circuit/denied).
-  `Call::Registry`, `Discovery`, `HealthTracker` are *feeders*, never read directly for model facts
-  by routing/availability/executor.
-- **Never dispatch a triple that isn't in the live catalog / isn't healthy.** There is no
-  anthropic→qwen; the availability gate rejects models a provider doesn't offer. Fail over, don't
-  hard-fail, unless the chain is genuinely empty.
-- **Multi-instance failover:** exhaust a provider's own instances before crossing providers.
-  Account-scoped errors (credit/quota/payment) **deprioritize** the failing instance via its
-  per-instance circuit (no model-deny) so the healthy sibling wins and auto-recovers on cooldown.
-  Model-intrinsic errors skip all instances. Instance selection prefers closed → half_open → open.
+## Routing rules (RANKING v2 — current behaviour)
+
+- **`Inventory` live `Concurrent::Map` is THE catalog.** Keyed by 5-part lane id
+  `tier:provider:instance:type:model`. Written by `lex-llm-*` discovery actors via the
+  `Inventory::ScopedRefresher` mixin. `HealthTracker` is the only other writer (owns `health`
+  block per lane). Everyone reads the same map, lock-free.
+- **`Router.request_lane(**routing_payload)` is the single selection method.** Returns one lane
+  hash or `nil`. Hard filters → soft filter (lane_weight ≤ 0 excluded) → max-weight bucket →
+  uniform sample. No pre-built chains.
+- **Escalation = "ask again with the failed lane excluded."** Executor calls `request_lane` in a
+  `while remaining.positive?` loop, appending tried lane ids to `tried_lanes`. No `loop do`.
+- **`lane_weight = tier_w × provider_w × instance_w × model_w × health_mult`.** Precomputed on
+  write. Negative = open circuit or policy-denied (excluded by soft filter). Surfaced in
+  `/api/llm/providers/<p>/models`. Tunable via `settings[:llm][:routing][:weights]`.
+- **`:fleet` is a first-class tier** in `Taxonomies::TIERS`. Fleet lanes written by `lex-llm-*`
+  fleet workers appear alongside direct lanes.
+- **`NoLaneAvailable` (400):** hard filters excluded everything before the first attempt.
+  **`EscalationExhausted` (503 + `Retry-After`):** max attempts reached mid-flight.
 - **Model policy is compliance.** `model_whitelist`/`model_blacklist` is honored at dispatch,
-  fail-closed. A policy-denied model is **terminal** — never escalated, never trips circuits/denies.
-  Enforced at the daemon layer here (`call/dispatch.rb` `enforce_model_policy!` →
+  fail-closed. A policy-denied model is **terminal** — never escalated, never trips circuits.
+  Enforced at the daemon layer (`call/dispatch.rb` `enforce_model_policy!` →
   `Errors::ModelNotAllowed`) and in each `lex-llm-*` provider.
 
 ## Coding constraints (enforced in review + cops)

data/README.md

@@ -12,7 +12,7 @@
 <p align="center">
   <img alt="License" src="https://img.shields.io/badge/license-Apache--2.0-blue.svg">
   <img alt="Ruby" src="https://img.shields.io/badge/ruby-3.4%2B-CC342D.svg">
-  <img alt="Version" src="https://img.shields.io/badge/version-0.13.0-informational.svg">
+  <img alt="Version" src="https://img.shields.io/badge/version-0.14.0-informational.svg">
   <img alt="Tests" src="https://img.shields.io/badge/tests-3200%2B%20examples%20·%200%20failures-success.svg">
   <img alt="RuboCop" src="https://img.shields.io/badge/rubocop-0%20offenses-success.svg">
 </p>
@@ -642,24 +642,43 @@ session = llm_session(tier: :local)
 | `capability` | `:basic`, `:moderate`, `:reasoning` | `:moderate` | Higher prefers larger/cloud models |
 | `cost` | `:minimize`, `:normal` | `:normal` | `:minimize` prefers local/fleet |
 
-#### Routing Resolution
+#### Routing Resolution — RANKING v2
 
+`Router.request_lane(**routing_payload)` returns one lane hash from the live `Inventory` catalog
+or `nil`. The catalog is a `Concurrent::Map` of 5-part lane ids
+(`tier:provider:instance:type:model`) populated by `lex-llm-*` discovery actors. No recomputation
+on read.
+
+**Selection algorithm:**
+```
+1. Hard filters applied (provider/instance/model/tier constraints from routing_payload).
+2. Soft filter: lanes with lane_weight ≤ 0 excluded (open circuit or policy-denied).
+3. Max-weight bucket selected (all lanes with the highest lane_weight value).
+4. One lane sampled uniformly within the bucket (seeded RNG for reproducibility).
+5. Returns the lane, or nil if no lanes survive filters.
+```
+
+**RANKING v2 lane_weight formula:**
 ```
-1. Caller passes intent: { privacy: :strict, capability: :basic }
-2. Router merges with default_intent (fills missing dimensions)
-3. Load rules from settings, filter by:
-   a. Intent match (all `when` conditions must match)
-   b. Schedule window (valid_from/valid_until, hours, days)
-   c. Constraints (e.g., never_cloud strips cloud-tier rules)
-   d. Discovery (Ollama model pulled? Model fits in available RAM?)
-   e. Tier availability (is Ollama running? is Transport loaded?)
-4. Score remaining candidates:
-   effective_priority = rule.priority
-                      + health_tracker.adjustment(provider)
-                      + (1.0 - cost_multiplier) * 10
-5. Return Resolution for highest-scoring candidate
+lane_weight = tier_weight × provider_weight × instance_weight × model_weight × health_multiplier
 ```
 
+All weights default to 100. The health multiplier is:
+- `1.0` — closed circuit (full weight)
+- `0.5` — half-open (reduced weight; cautious retry)
+- `-100_000_000` — open circuit (effectively disabled; excluded by soft filter)
+
+Weights are operator-tunable via settings and take effect immediately (no restart required).
+Surfaced in `/api/llm/providers/<provider>/models` as `lane_weight`.
+
+**Escalation:** "try again with the failed lane excluded." The executor calls `request_lane` in a
+`while remaining.positive?` loop, appending each tried lane to `tried_lanes`. This replaces the
+old pre-built escalation chain.
+
+**Errors:**
+- `Errors::NoLaneAvailable` (HTTP 400) — all filters excluded everything before the first attempt.
+- `Errors::EscalationExhausted` (HTTP 503 + `Retry-After`) — attempts exhausted mid-flight.
+
 #### Settings
 
 Add routing configuration under the `llm` key: