lex-llm-mlx 0.5.2 → 0.5.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 7e5f9a7dab605748337a587418be077e253a7e6b497c6b014ecba9875043c6ad
4
- data.tar.gz: 3114258fbcc3dd3b6f184f28d83bc14c866187947a08c42bf6128f86e002c9c3
3
+ metadata.gz: b2be714a855933a9b2eaa90d975550eacf40d0e34d7a072ae9c7cade745cf0c8
4
+ data.tar.gz: 30222c46840faddde023875adf8282efcdf773b6b56b3840497613f03d65ce6b
5
5
  SHA512:
6
- metadata.gz: 12ae5ec7295decb7048148673f35daac26c2737f0739b4d236a2097ac07d6b7c5eddcad7af5bbde8fc7e6c2ca3d4bea570a2021ff6d0b30f2eff1e45711dc836
7
- data.tar.gz: d227264342de019670a47624dcf121ae21c9ed9653d3e65c47d15ff754c05da407b44b29e20de06ee357f21b5bf34fffe7fac9869c1cacecefcd2ed21fd42c62
6
+ metadata.gz: f34a9e7e80330ca83cbc703ac8c2c1e9ee2124c89a246655c525943a2176f7163ddd409ec9df12cd3b9c28ee68a6bd98f0f7d0211ec54608494f3e656c45d8a6
7
+ data.tar.gz: 37027094de2819478977b96140b19c14564f33afa5c0d42d3d7d89338e7009b5dfbb2eedc73679b5742c09ff70534dc6ee64e750e4a465a163e7016f066aa6da
data/.rubocop.yml CHANGED
@@ -14,6 +14,8 @@ Metrics/BlockLength:
14
14
  - spec/**/*
15
15
  Metrics/MethodLength:
16
16
  Enabled: false
17
+ Metrics/ModuleLength:
18
+ Max: 350
17
19
  RSpec/ExampleLength:
18
20
  Enabled: false
19
21
  RSpec/MultipleExpectations:
data/CHANGELOG.md CHANGED
@@ -1,5 +1,43 @@
1
1
  # Changelog
2
2
 
3
+ ## [0.5.4] - 2026-08-20
4
+
5
+ ### Changed
6
+ - **lex-llm 0.8.0 conformance.** Legacy type usage migrates to Canonical; the legacy coordinator wiring is removed; the gemspec floor is `lex-llm >= 0.8.0`.
7
+ - Enforce the canonical dispatch boundary end to end: the `MlxCallable` dispatch operations (chat, stream_chat, count_tokens) call the shared `Provider#enforce_canonical_messages!` helper at the exact-execution boundary (12/O05) before delegating. Plain-Hash input raises a typed `ArgumentError` on every dispatch operation; the lenient hash re-canonicalization that masked the 2026-08-19 hash-bypass defect for 25 failed openai dispatches is gone.
8
+ - Migrate the `MlxCallable` dispatch contract to the 0.8.0 callable shape: `chat`/`stream_chat` take the rehydrated `Array<Canonical::Message>` positionally (the `Fleet::WorkerExecution` dispatch shape and the kit B1/B2 callable contract), the Selection-derived model passes through as a bare String (the 0.8.0 renderer puts `model` on the wire as-is), and the folded wire params become a `Canonical::Params` at the dispatch boundary — `temperature` is a params member (05 O4) and no longer a named kwarg.
9
+ - Migrate the offering call site to the writer path: the legacy read-path production (`offering_from_model` → `Routing::ModelOffering`, deleted from lex-llm 0.8.0) is removed; the base `discover_offerings` now serves the activated inventory offerings published by the discovery actor from the Registry snapshot (07 C5 / 08 D3).
10
+ - Remove the discovery actor's `Inventory::ScopedRefresher::LegacyCoordinatorAdapter` compatibility wiring (the adapter is deleted in the lex-llm 0.8.0 cut; the `Publisher` is constructed registry-only).
11
+ - Raise the `lex-llm` dependency floor to 0.8.0 for the canonical-only funnel, the Registry read path, fleet v3 exact execution, and the conformance kit.
12
+
13
+ ### Removed
14
+ - `Provider#offering_from_model` with its offering-kwarg and capability-policy resolution helpers, and the `CapabilityConfig`/`CapabilityResolution` modules that fed only that read path. Config-cascade capability overrides (provider/instance/model `*_flag` keys) no longer have an MLX consumer: the SSOT evidence model treats config sources as unknown-only (`Taxonomies::UNKNOWN_ONLY_EVIDENCE_SOURCES`).
15
+ - The provider-local `render_payload` message-boundary check — the 0.8.0 base funnel enforces `Canonical::Message` input centrally before rendering (08 F2); providers no longer re-implement the check, and the reference to the deleted legacy `Llm::Message` class is gone.
16
+ - `MlxCallable#model_info` — the `Model::Info` wrapping was a 0.7.7 `model.id` renderer artifact; the 0.8.0 renderer renders the model string as-is.
17
+ - `spec/legion/extensions/llm/mlx/provider_capability_policy_spec.rb` — it tested the removed offering-production path.
18
+
19
+ ### Added
20
+ - The ssot_v3 conformance spec runs the 0.8.0 kit boundary groups B1 (central canonical enforcement) and B2 (canonical outputs asserted by type) against the real `MlxCallable`, alongside the existing SSOT v3 provider-adapter group.
21
+ - `provider_contract_spec` rewritten against the 0.8.0 funnel shape (positional completion messages, named `text`/`prompt`) and the Registry-snapshot read path.
22
+ - RULES.md (the 0.8.0 architecture law, byte-for-byte mirror of the lex-llm 0.8.0 release).
23
+ - A local-tree `lex-llm` path dependency in the test group so the adjacent checkout resolves against the 0.8.0 cut during development.
24
+
25
+ ## [0.5.3] - 2026-08-19
26
+
27
+ ### Changed
28
+ - Publish the immutable four-component lane-weight pair from MLX discovery and reconcile weight-only changes on the existing scheduled writer pass.
29
+ - Serialize initial, recovery, replacement, removal, and shutdown state transitions behind one actor-local mutex without adding a Settings callback or operator workflow.
30
+ - Track configured-but-unpublished weight keys on the existing discovery cadence and log each dormant transition once.
31
+ - Raise the `lex-llm` dependency floor to 0.7.6; the existing `legion-settings` dependency remains unchanged.
32
+
33
+ ### Fixed
34
+ - Compare every offering contract field while excluding only non-authoritative evidence observation timestamps, preventing unchanged discovery passes from republishing snapshots while retaining real evidence and weight changes.
35
+ - Treat discovery catalogs as order-independent multisets, so provider reorderings do not republish while adding or removing a duplicate offering remains a significant change.
36
+ - Validate the complete weighted offering set before constructing or claiming a callable scope, so malformed weights leave no orphaned Registry publication and a later corrected ordinary pass activates without restart or operator cleanup.
37
+
38
+ ### Added
39
+ - Cover the complete writer lifecycle, publication races, failure atomicity, dormant-state cycle, and the actual callable's folded-system OpenAI-compatible wire payload.
40
+
3
41
  ## [0.5.2] - 2026-08-18
4
42
 
5
43
  ### Fixed
data/Gemfile CHANGED
@@ -2,10 +2,15 @@
2
2
 
3
3
  source 'https://rubygems.org'
4
4
 
5
- gem 'lex-llm'
6
-
7
5
  gemspec
8
6
 
7
+ # Local-tree resolution: use the adjacent lex-llm checkout when present so the
8
+ # canonical-boundary work resolves against the unreleased 0.7.7.
9
+ group :test do
10
+ lex_llm_path = File.expand_path('../lex-llm', __dir__)
11
+ gem 'lex-llm', path: lex_llm_path if Dir.exist?(lex_llm_path)
12
+ end
13
+
9
14
  group :development do
10
15
  gem 'bundler', '>= 2.0'
11
16
  gem 'rake', '>= 13.0'
data/RULES.md ADDED
@@ -0,0 +1,97 @@
1
+ RULES.md — Legion LLM Architecture Law
2
+ These rules apply to every task, file, repository, agent, model, session, test, refactor, migration, incident, and release.
3
+ The requested task defines what may change. These rules define how the system ALWAYS works.
4
+ Every rule remains active 100% of the time. If requested work conflicts with these rules, stop and surface the conflict before changing code.
5
+ These are architecture laws. Scope, compatibility, urgency, convenience, tests, existing behavior, and model judgment do not change them.
6
+ 1. Canonical is the only internal language.
7
+ Every client translates client wire -> Canonical before shared execution.
8
+ Shared execution carries Canonical through context, tools, routing, direct dispatch, fleet dispatch, and response handling.
9
+ Every provider translates Canonical -> provider wire at the provider boundary, then provider wire -> Canonical before returning to shared execution.
10
+ Every internal boundary validates the Canonical type it is defined to receive and raises immediately when that contract is violated.
11
+ Client Wire -> Client Translator -> Canonical -> Shared Execution -> Canonical -> Provider Translator -> Provider Wire.
12
+ 2. Serialization preserves Canonical.
13
+ Transport may serialize Canonical state. The receiving transport boundary ALWAYS rehydrates the exact Canonical type before execution continues.
14
+ Fleet follows Canonical -> serialize -> wire -> deserialize -> rehydrate Canonical -> Canonical.
15
+ Serialization changes encoding only. Ownership, identity, model, operation, capability, selection, and meaning remain exactly the same.
16
+ After rehydration, shared execution continues only with Canonical objects.
17
+ 3. Every authoritative fact has exactly one owner.
18
+ The owner creates the fact once. Every downstream layer carries, projects, serializes, rehydrates, verifies, or executes that exact fact.
19
+ A downstream layer receiving missing or contradictory authoritative state raises and returns the defect to the owning layer.
20
+ Authority ALWAYS moves forward by preservation.
21
+ Authority is created once and is never recreated downstream.
22
+ 4. Requirements describe the request. Inventory describes reality. Router chooses. Dispatch executes.
23
+ Canonical request construction owns request semantics. RequestRequirements expresses operation, capabilities, modality, context, output, tools, and explicit pins.
24
+ Providers publish exact executable facts into Inventory. Inventory owns canonical instance, offering, lane, capability, context, quota, health, and published weight state.
25
+ Router.next_lane consumes Requirements plus one immutable Inventory snapshot and produces one authoritative Selection.
26
+ Dispatch executes that Selection exactly. Once Selection exists, routing is finished.
27
+ 5. Inventory facts are immutable executable facts.
28
+ Providers publish exact instances and complete offering snapshots through the Inventory publication contract.
29
+ Identity, capability evidence, context evidence, quota domains, availability, and write-time weights are consumed from published Inventory state.
30
+ A changed fact becomes authoritative only through the owning publication or reconciliation path and a new Inventory snapshot.
31
+ Routing reads Inventory. Dispatch verifies and executes Inventory-backed Selection.
32
+ 6. Identity, capabilities, weights, and context policy retain exact ownership.
33
+ Inventory::Identity owns instance, offering, and lane identity; canonical instance identity is provider family plus the operator/configured instance name; physical endpoint data remains secondary.
34
+ Providers publish capability evidence. Requirements state required capabilities. Candidate evaluation compares the two and determines capability eligibility.
35
+ The weight owner computes lane weight at publication time; Inventory stores it; ranking consumes that stored weight; a stored zero disables the lane.
36
+ Preferred-context binning orders eligible candidates into preference bands and preserves eligibility. Capability, health, binning, and weight ALWAYS retain distinct meanings.
37
+ 7. Routing chooses exactly once.
38
+ Router.next_lane is the sole routing authority.
39
+ Candidate evaluation determines eligibility from Requirements and Inventory. Ranking orders eligible candidates from published routing facts.
40
+ Selection freezes the exact provider, instance, offering, lane, model, operation, and routing identity required for execution.
41
+ Every downstream component consumes the Selection it receives.
42
+ Selection is preserved, not reconstructed.
43
+ 8. Exact execution stays exact through every boundary.
44
+ Direct dispatch executes the exact Selection-derived binding it receives.
45
+ Fleet dispatch serializes and signs that exact binding; fleet validation verifies it; fleet rehydration restores it; worker resolution verifies it against authoritative Inventory.
46
+ The selected provider, instance, offering, lane, model, and operation remain identical through projection, signing, transport, validation, rehydration, resolution, and callable invocation.
47
+ A mismatch raises before provider execution.
48
+ An exact execution request ALWAYS remains exact execution.
49
+ 9. Health and errors preserve one authoritative meaning.
50
+ Inventory owns exact-instance availability. An authoritative instance-unavailable result removes that exact instance; readiness probing owns recovery; successful readiness republish re-admits it.
51
+ Overload, timeout, rate limit, model-not-ready, and transient provider failures remain request-local according to ProviderOutcome semantics.
52
+ The first layer that can authoritatively classify an error performs that classification once. Every downstream layer preserves it.
53
+ Programming errors remain programming errors. Contract violations remain contract violations. Routing exhaustion remains the defined typed Rejection.
54
+ 10. Compatibility exists only at explicit edges.
55
+ Supported legacy clients and protocols are translated into the current Canonical and SSOT architecture at explicit compatibility boundaries.
56
+ Shared execution remains Canonical. Routing remains SSOT-driven. Exact execution remains exact.
57
+ Compatibility code adapts an external contract to the current internal architecture.
58
+ The current internal architecture ALWAYS has one representation, one routing authority, one identity system, and one execution truth.
59
+ 11. Fix every defect at its owner.
60
+ Trace the incorrect value to the layer that owns it, then fix that owner.
61
+ Fix client wire in the client translator; Canonical shape in Canonical construction; Requirements in Requirements construction; provider facts in publication; identity in Inventory identity; weights in publication/reconciliation.
62
+ Fix eligibility in candidate evaluation; ordering in ranking; choice in Router.next_lane; execution preservation in dispatch; provider wire in the provider translator.
63
+ The layer where a defect becomes visible is evidence. The owning layer is where the correction belongs.
64
+ 12. A discovered issue remains in its owning domain.
65
+ Complete the requested task inside its stated scope.
66
+ When investigation exposes a separate defect owned by another architectural domain, record and surface it as separate work unless the requested task is explicitly expanded.
67
+ Routing work consumes existing Canonical Requirements and Inventory facts. Canonical work changes Canonical contracts. Provider work changes publication or translation. Transport work changes transport.
68
+ Nearby code never changes ownership. “While we are here” never changes architecture.
69
+ 13. N x N ALWAYS converges through Canonical.
70
+ Equivalent client semantics produce equivalent Canonical state before shared execution. Every provider consumes the same Canonical semantics for the same request.
71
+ When two paths disagree, capture the state at every involved boundary and locate the FIRST point where Canonical meaning diverges.
72
+ Fix that first divergent boundary, then run the exact failing path again.
73
+ Client behavior is proven at client-wire <-> Canonical. Provider behavior is proven at Canonical <-> provider-wire. Shared execution is proven with Canonical throughout.
74
+ 14. Debug from captured authoritative state.
75
+ Capture the actual input at the failing boundary before reasoning from symptoms.
76
+ For translation or transport defects, capture Canonical immediately before and after every involved boundary.
77
+ For routing or dispatch defects, capture Requirements, relevant Inventory facts, Selection, execution binding, and ProviderOutcome.
78
+ Compare each captured value to the contract owned by that layer. Find the first divergence. Fix its owner. Re-run the exact path.
79
+ Then inspect sibling implementations for the same defect class.
80
+ 15. Tests prove the real boundary and the invariant.
81
+ A boundary test exercises the real boundary it claims to protect.
82
+ Fleet tests exercise real serialization, deserialization, Canonical rehydration, signing, validation, exact resolution, and callable dispatch.
83
+ Provider tests exercise the real callable boundary and provider translator. Routing tests exercise real Requirements, Inventory records, candidate evaluation, ranking, and Selection.
84
+ Regression tests prove the violated invariant, not only the observed symptom.
85
+ A green suite is release evidence only when the tested path traverses the real architecture.
86
+ 16. Shared contracts are consumed directly.
87
+ Shared Canonical types own execution representation. Shared Inventory types own inventory state. Shared Routing types own routing state.
88
+ Shared taxonomy owns canonical mappings. Shared ProviderOutcome owns provider-neutral outcomes. Shared fleet protocol owns exact execution claims.
89
+ Every repository consumes these shared owners directly.
90
+ A defect in one shared boundary triggers an audit of every sibling implementation of that boundary. Fix the shared owner centrally whenever the defect belongs to a shared contract.
91
+ 17. Architecture is the release gate.
92
+ Every change preserves every rule in this file.
93
+ Tests, compatibility, historical behavior, migration phase, patch urgency, nearby code, task wording, and model judgment are evaluated UNDER these rules.
94
+ A contradiction between existing behavior and these rules is surfaced as an architecture conflict and resolved at the owning boundary before release.
95
+ Limited scope means do less. Limited scope NEVER means fewer rules apply.
96
+ These rules apply 100% of the time.
97
+ These are the law.
data/lex-llm-mlx.gemspec CHANGED
@@ -28,8 +28,9 @@ Gem::Specification.new do |spec|
28
28
  spec.add_dependency 'legion-logging', '>= 1.3.2'
29
29
  spec.add_dependency 'legion-settings', '>= 1.4.2'
30
30
  spec.add_dependency 'legion-transport', '>= 1.4.14'
31
- # 0.7.1 carries the SSOT v3 InstanceKey (instance_id = config name +
32
- # secondary physical_id) and the Publisher physical_id: kwargs the
33
- # discovery actor calls. 0.7.0 publishers reject the physical_id: kwarg.
34
- spec.add_dependency 'lex-llm', '>= 0.7.1'
31
+ # 0.8.0 is the complete contract cut: the canonical-only funnel
32
+ # (positional completion messages, Canonical::Params at the render boundary),
33
+ # the Registry-snapshot read path, fleet v3 exact execution, and the
34
+ # conformance kit the ssot_v3 conformance spec runs.
35
+ spec.add_dependency 'lex-llm', '>= 0.8.0'
35
36
  end
@@ -0,0 +1,24 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'legion/extensions/llm/discovery/actor'
4
+
5
+ # The base discovery actor only exists inside the daemon (it inherits the
6
+ # LegionIO time-based Every actor). In a standalone load, define nothing.
7
+ return unless defined?(Legion::Extensions::Llm::Discovery::Actor)
8
+
9
+ module Legion
10
+ module Extensions
11
+ module Llm
12
+ module Mlx
13
+ module Actor
14
+ # Mlx discovery actor: an EMPTY subclass of the shared base. The
15
+ # timer, dispatch, and runner-resolution convention are inherited — this
16
+ # class redefines nothing. The Mlx-specific work lives in
17
+ # Mlx::Runners::Discovery, resolved by the base from this
18
+ # namespace.
19
+ class Discovery < Legion::Extensions::Llm::Discovery::Actor; end
20
+ end
21
+ end
22
+ end
23
+ end
24
+ end
@@ -0,0 +1,162 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'faraday'
4
+
5
+ require 'legion/extensions/llm/routing/provider_outcome'
6
+ require 'legion/extensions/llm/canonical'
7
+ require 'legion/extensions/llm/mlx/provider'
8
+
9
+ module Legion
10
+ module Extensions
11
+ module Llm
12
+ module Mlx
13
+ module Helpers
14
+ # Callable wrapper for an MLX provider instance. It is the
15
+ # exact-execution dispatch target: it implements the fleet dispatch
16
+ # operations (chat, stream_chat, embed, count_tokens) by delegating
17
+ # to a per-instance Mlx::Provider built from the instance config,
18
+ # plus the `disconnect` and `normalize_dispatch_error(error:)`
19
+ # contracts required by Inventory::CallableHandle and
20
+ # Routing::ProviderOutcome. Provider and Faraday errors are NOT
21
+ # rescued here so the dispatch normalizer can classify them.
22
+ # 0.8.0 callable contract: chat/stream_chat take the rehydrated
23
+ # message array positionally (WorkerExecution dispatch shape) and
24
+ # the Selection-derived model as a bare String.
25
+ class Callable
26
+ # Keys the base Provider exposes as named kwargs for the
27
+ # completion operations. Anything else the fleet passes (sampling
28
+ # scalars, `temperature` — a Canonical::Params member, 05 O4) is
29
+ # folded into Canonical::Params at the dispatch boundary.
30
+ COMPLETION_NAMED_KEYS = %i[tools schema thinking tool_prefs headers].freeze
31
+ EMBED_NAMED_KEYS = %i[dimensions headers].freeze
32
+
33
+ def initialize(instance_cfg:, logger:)
34
+ @instance_cfg = instance_cfg
35
+ @logger = logger
36
+ @disconnected = false
37
+ @inference_calls = 0
38
+ end
39
+
40
+ def call_count
41
+ @inference_calls
42
+ end
43
+
44
+ def disconnected?
45
+ @disconnected
46
+ end
47
+
48
+ def disconnect
49
+ @disconnected = true
50
+ @provider&.disconnect
51
+ @logger.debug { '[mlx][callable] disconnected' }
52
+ end
53
+
54
+ # ── Fleet dispatch operations ───────────────────────────────────
55
+
56
+ def chat(messages, model:, **rest)
57
+ record_inference
58
+ # Canonical boundary (N x N law): pipeline dispatch delivers
59
+ # Canonical::Message objects only. Hash/legacy shapes are the
60
+ # bypass class — reject loudly, never coerce.
61
+ provider.enforce_canonical_messages!(messages)
62
+ named, params = split_fleet_kwargs(rest, COMPLETION_NAMED_KEYS)
63
+ provider.chat(messages, model: model, params: canonical_params(params), **named)
64
+ end
65
+
66
+ def stream_chat(messages, model:, **rest, &)
67
+ record_inference
68
+ provider.enforce_canonical_messages!(messages)
69
+ named, params = split_fleet_kwargs(rest, COMPLETION_NAMED_KEYS)
70
+ provider.stream_chat(messages, model: model, params: canonical_params(params), **named, &)
71
+ end
72
+
73
+ def embed(text:, model:, **rest)
74
+ record_inference
75
+ named, params = split_fleet_kwargs(rest, EMBED_NAMED_KEYS)
76
+ provider.embed(text: text, model: model, params: params, **named)
77
+ end
78
+
79
+ def count_tokens(messages:, model:, **rest)
80
+ record_inference
81
+ provider.enforce_canonical_messages!(messages)
82
+ _named, params = split_fleet_kwargs(rest, [])
83
+ provider.count_tokens(messages: messages, model: model, params: params)
84
+ end
85
+
86
+ def normalize_dispatch_error(error:)
87
+ reason = error.message.to_s[0, 512]
88
+ kind = classify_error_kind(error: error)
89
+
90
+ Legion::Extensions::Llm::Routing::ProviderOutcome.new(
91
+ kind: kind,
92
+ reason: reason.empty? ? 'unknown dispatch error' : reason
93
+ )
94
+ end
95
+
96
+ private
97
+
98
+ def record_inference
99
+ @inference_calls += 1
100
+ end
101
+
102
+ def provider
103
+ @provider ||= Legion::Extensions::Llm::Mlx::Provider.new(@instance_cfg)
104
+ end
105
+
106
+ # The 0.8.0 completion funnel receives canonical values only
107
+ # (08 F3): the folded wire params become a Canonical::Params at
108
+ # the dispatch boundary — temperature is a params member (05 O4),
109
+ # never a kwarg.
110
+ def canonical_params(params)
111
+ Legion::Extensions::Llm::Canonical::Params.from_hash(params)
112
+ end
113
+
114
+ # Split the fleet's **rest into the base Provider's named kwargs
115
+ # and a payload params hash (any passed :params merged with
116
+ # unknown keys).
117
+ def split_fleet_kwargs(rest, named_keys)
118
+ named = rest.slice(*named_keys)
119
+ extra = rest.reject { |key, _| named.key?(key) }
120
+ params = (extra.delete(:params) || {}).to_h.merge(extra)
121
+ [named, params]
122
+ end
123
+
124
+ def classify_error_kind(error:)
125
+ case error
126
+ when Faraday::ConnectionFailed then :connection_failure
127
+ when Faraday::TimeoutError then :timeout
128
+ when Faraday::ClientError then classify_client_error(error: error)
129
+ when Faraday::ServerError then classify_server_error(error: error)
130
+ when Legion::Extensions::Llm::OverloadedError then :overloaded
131
+ else :provider_error
132
+ end
133
+ end
134
+
135
+ def classify_client_error(error:)
136
+ status = error.respond_to?(:response_status) ? error.response_status : nil
137
+ case status
138
+ when 401 then :authentication
139
+ when 403 then :authorization
140
+ when 404 then :model_missing
141
+ when 429 then :rate_limited
142
+ else :invalid_request
143
+ end
144
+ end
145
+
146
+ def classify_server_error(error:)
147
+ # NEVER classify raw 503/529/5xx as instance_unavailable by status alone.
148
+ # Only an explicit flat MLX service/instance-unavailable signal (which MLX
149
+ # does not produce) would justify instance_unavailable. For MLX, connection
150
+ # failure (port unreachable) is the signal the instance is down.
151
+ status = error.respond_to?(:response_status) ? error.response_status : nil
152
+ case status
153
+ when 503, 529 then :overloaded
154
+ else :provider_error
155
+ end
156
+ end
157
+ end
158
+ end
159
+ end
160
+ end
161
+ end
162
+ end
@@ -30,56 +30,6 @@ module Legion
30
30
  end
31
31
  end
32
32
 
33
- # Capability config helpers — mix into Provider.
34
- module CapabilityConfig
35
- private
36
-
37
- def provider_capability_config
38
- conf = Legion::Extensions::Llm::CredentialSources.setting(:extensions, :llm, :mlx)
39
- conf.is_a?(Hash) ? conf.to_h.except(:instances, 'instances') : {}
40
- rescue StandardError => e
41
- handle_exception(e, level: :warn, handled: true, operation: 'mlx.provider_capability_config')
42
- {}
43
- end
44
-
45
- def instance_capability_config
46
- cfg = config
47
- result = {}
48
- %i[capabilities enable_thinking enable_tools enable_streaming enable_vision enable_embeddings
49
- thinking_flag tools_flag streaming_flag vision_flag embedding_flag embeddings_flag
50
- tool_flag images_flag image_flag].each do |key|
51
- next unless cfg.respond_to?(key)
52
-
53
- val = cfg.send(key)
54
- result[key] = val unless val.nil?
55
- rescue StandardError => e
56
- handle_exception(e, level: :warn, handled: true, operation: 'mlx.provider.instance_capability_config')
57
- next
58
- end
59
- result
60
- end
61
-
62
- def model_capability_config(model_id)
63
- models_conf = fetch_models_config
64
- return {} unless models_conf
65
-
66
- hash = models_conf.to_h
67
- hash[model_id.to_s] || hash[model_id.to_sym] || {}
68
- rescue StandardError => e
69
- handle_exception(e, level: :warn, handled: true, operation: 'mlx.model_capability_config')
70
- {}
71
- end
72
-
73
- def fetch_models_config
74
- conf = config.models if config.respond_to?(:models)
75
- conf ||= config[:models] if config.respond_to?(:[])
76
- conf if conf.respond_to?(:to_h)
77
- rescue StandardError => e
78
- handle_exception(e, level: :warn, handled: true, operation: 'mlx.fetch_models_config')
79
- nil
80
- end
81
- end
82
-
83
33
  # Health payload helpers — mix into Provider.
84
34
  module HealthHelpers
85
35
  private
@@ -110,62 +60,10 @@ module Legion
110
60
  end
111
61
  end
112
62
 
113
- # Capability resolution helpers — mix into Provider.
114
- module CapabilityResolution
115
- private
116
-
117
- def resolve_capability_policy(model_info)
118
- Legion::Extensions::Llm::CapabilityPolicy.resolve(
119
- real: extract_real_capabilities(model_info),
120
- provider_catalog: extract_catalog_capabilities(model_info),
121
- probe: {},
122
- provider_envelope: provider_envelope_capabilities,
123
- provider_config: provider_capability_config,
124
- instance_config: instance_capability_config,
125
- model_config: model_capability_config(model_info.id)
126
- )
127
- end
128
-
129
- def extract_real_capabilities(model_info)
130
- return {} unless model_info.respond_to?(:metadata)
131
-
132
- meta = model_info.metadata
133
- return {} unless meta.is_a?(Hash)
134
-
135
- caps = meta[:capabilities]
136
- caps.is_a?(Hash) ? caps : {}
137
- end
138
-
139
- def extract_catalog_capabilities(_model_info)
140
- # Regex-based name matching is not authoritative evidence.
141
- # Unverified capability support is unknown, not promoted to supported.
142
- {}
143
- end
144
-
145
- def embedding_model?(model_id)
146
- model_id.to_s.match?(/embed|bge|e5|nomic/i)
147
- end
148
-
149
- def provider_envelope_capabilities
150
- # No capabilities are advertised at the envelope level without endpoint evidence.
151
- {}
152
- end
153
-
154
- def offering_metadata_for(model_info)
155
- {
156
- raw_model: model_info.id,
157
- parameter_count: model_info.respond_to?(:parameter_count) ? model_info.parameter_count : nil,
158
- quantization: model_info.respond_to?(:quantization) ? model_info.quantization : nil
159
- }.compact
160
- end
161
- end
162
-
163
63
  # MLX provider implementation for local OpenAI-compatible servers.
164
64
  class Provider < Legion::Extensions::Llm::Provider
165
65
  include Legion::Extensions::Llm::Provider::OpenAICompatible
166
- include CapabilityConfig
167
66
  include HealthHelpers
168
- include CapabilityResolution
169
67
 
170
68
  class << self
171
69
  def slug = 'mlx'
@@ -215,45 +113,6 @@ module Legion
215
113
  log.info("Checking MLX readiness (live=#{live})")
216
114
  super
217
115
  end
218
-
219
- def list_models(**)
220
- log.info('Listing available MLX models')
221
- models = super
222
- log.info("Discovered #{Array(models).size} MLX models")
223
- models
224
- end
225
-
226
- def offering_from_model(model_info, health: {})
227
- policy = resolve_capability_policy(model_info)
228
- Legion::Extensions::Llm::Routing::ModelOffering.new(**build_offering_kwargs(
229
- model_info: model_info, policy: policy, health: health
230
- ))
231
- end
232
-
233
- private
234
-
235
- def build_offering_kwargs(model_info:, policy:, health:)
236
- {
237
- provider_family: :mlx,
238
- instance_id: provider_instance_id,
239
- transport: offering_transport,
240
- tier: offering_tier,
241
- model: model_info.id,
242
- canonical_model_alias: model_info.respond_to?(:name) ? model_info.name : nil,
243
- model_family: model_info.respond_to?(:family) ? model_info.family : nil,
244
- usage_type: embedding_model?(model_info.id) ? :embedding : :inference,
245
- capabilities: policy[:capabilities],
246
- capability_sources: policy[:sources],
247
- limits: extract_model_limits(model_info),
248
- health: health,
249
- metadata: offering_metadata_for(model_info).merge(capability_sources: policy[:sources])
250
- }
251
- end
252
-
253
- def extract_model_limits(model_info)
254
- ctx = model_info.respond_to?(:context_length) ? model_info.context_length : nil
255
- { context_window: ctx }.compact
256
- end
257
116
  end
258
117
  end
259
118
  end