legion-llm 0.16.3 → 0.16.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +30 -0
- data/legion-llm.gemspec +1 -1
- data/lib/legion/llm/api/stream_assembler.rb +35 -5
- data/lib/legion/llm/call/selection_dispatch.rb +16 -0
- data/lib/legion/llm/fleet/dispatcher.rb +16 -3
- data/lib/legion/llm/fleet/token_issuer.rb +13 -0
- data/lib/legion/llm/inference/executor/escalation.rb +94 -15
- data/lib/legion/llm/inference/route_attempts.rb +100 -11
- data/lib/legion/llm/inference/routing_session.rb +5 -0
- data/lib/legion/llm/router/candidate_evaluator.rb +15 -9
- data/lib/legion/llm/router/ranker.rb +50 -43
- data/lib/legion/llm/router/settings_snapshot.rb +1 -25
- data/lib/legion/llm/version.rb +1 -1
- metadata +3 -3
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: ff0c6b5e1aff591b55afb3776a7d14fc8c6fc607bcc5836b0c2a3ffbd31d4e83
|
|
4
|
+
data.tar.gz: 4f2e3d428bded9896ce9a4eb1044fc65e7f2025ec5116aba2bd3bdf1a5a94077
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 83e30aa5a9b7f2608b52159b252a7481c44909537e3643c03f1981a6b4eb5926a9642684444ea8e3b61bb0b628e03bb5e89ffcf0d51d2da077f326d7041f8029
|
|
7
|
+
data.tar.gz: 350b3cda23fcd149f9b14b6a11cefb567ab9a3b4b2ca1f2818d9991d8bcf5968156349519a10254e6ea7f7a92e89361fc7de7c6a5f14be28ed06382031b05aad
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,35 @@
|
|
|
1
1
|
# Legion LLM Changelog
|
|
2
2
|
|
|
3
|
+
## [0.16.8] - 2026-08-19
|
|
4
|
+
|
|
5
|
+
### Fixed
|
|
6
|
+
- **Routing consumes the SSOT lane's write-time weight.** The stateless ranker reads the immutable 100-based `weight_inputs`/`base_weight` pair published on `LaneRecord`; request-time settings recomputation and the dormant tier-component substitution are removed.
|
|
7
|
+
- **Zero-weight disable remains explicit.** Any zero stored component disables the lane before ranking, while health remains the binary availability axis.
|
|
8
|
+
- **Exact fleet streaming preserves the selected operation.** SSOT fleet dispatch now sources `operation` from the immutable selection, verifies it against the bound lane before envelope construction, and signs `stream_chat` through to the worker's exact stream callable instead of silently executing the chat lane.
|
|
9
|
+
|
|
10
|
+
## [0.16.7] - 2026-08-19
|
|
11
|
+
|
|
12
|
+
### Fixed
|
|
13
|
+
- **Preferred-context bands now order without excluding.** Routing ranks in-band lanes first, then ranks every remaining ready lane when no band matches. Out-of-band lanes are never dropped, while release-bar in-band steering remains intact.
|
|
14
|
+
|
|
15
|
+
## [0.16.6] - 2026-08-19
|
|
16
|
+
|
|
17
|
+
### Fixed
|
|
18
|
+
- **Daemon faults stay terminal.** `NoMethodError`, `ArgumentError`, `NotImplementedError`, and `TypeError` are never laundered into retryable provider outcomes; caught daemon faults re-raise raw with the original backtrace.
|
|
19
|
+
- **Lease lifecycle parity across sync and stream.** Retired or disposed callable handles classify as `instance_unavailable`, trigger the existing exact-instance transition/probe behavior, and retry a different eligible lane. Streaming preflight uses an explicit bounded loop and rejects before headers when the attempt budget is exhausted.
|
|
20
|
+
|
|
21
|
+
## [0.16.5] - 2026-08-19
|
|
22
|
+
|
|
23
|
+
### Fixed
|
|
24
|
+
- **Shared SSOT dispatch-boundary contract.** Direct and fleet requests now project only provider-callable contract keys, fold the authoritative system prompt into canonical messages exactly once, and move non-contract generation parameters under `params`. Legacy/non-SSOT calls retain their original messages and options.
|
|
25
|
+
- **Signed fleet exact execution.** SSOT fleet envelopes carry and sign `execution_contract: exact_offering_v1` plus a validated nonempty `offering_id`; both dispatcher and independently callable token issuer reject unknown markers instead of silently downgrading to legacy execution. Daemon-only offering metadata never crosses the provider boundary.
|
|
26
|
+
|
|
27
|
+
## [0.16.4] - 2026-08-19
|
|
28
|
+
|
|
29
|
+
### Changed
|
|
30
|
+
- **Human-readable SSOT routing identities.** Ranked, selected, failed-attempt, streaming preflight, and streaming failover decision logs now carry the Inventory five-tuple (`tier:provider:instance:type:model`) instead of opaque `lane:v1:` hashes. StreamAssembler preserves historically accepted public lane inputs while redacting incomplete or malformed identities as `lane_identity_missing=true`; failover-chain state is unchanged.
|
|
31
|
+
- **`lex-llm >= 0.7.4` floor.** Routing logs consume the shared authoritative operation-to-lane-type mapping rather than maintaining a daemon-local copy.
|
|
32
|
+
|
|
3
33
|
## [0.16.3] - 2026-08-17
|
|
4
34
|
|
|
5
35
|
### Changed
|
data/legion-llm.gemspec
CHANGED
|
@@ -38,7 +38,7 @@ Gem::Specification.new do |spec|
|
|
|
38
38
|
spec.add_dependency 'legion-settings', '>= 1.4.2'
|
|
39
39
|
spec.add_dependency 'legion-transport', '>= 1.4.14'
|
|
40
40
|
spec.add_dependency 'lex-knowledge'
|
|
41
|
-
spec.add_dependency 'lex-llm', '>= 0.7.
|
|
41
|
+
spec.add_dependency 'lex-llm', '>= 0.7.5'
|
|
42
42
|
spec.add_dependency 'pdf-reader'
|
|
43
43
|
spec.add_dependency 'sinatra-contrib', '>= 2.0'
|
|
44
44
|
spec.add_dependency 'tzinfo', '>= 2.0'
|
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
require 'legion/logging/helper'
|
|
4
|
+
require 'legion/extensions/llm/taxonomies'
|
|
4
5
|
require 'legion/llm/api/client_translators/shared_extractors'
|
|
5
6
|
|
|
6
7
|
module Legion
|
|
@@ -246,9 +247,14 @@ module Legion
|
|
|
246
247
|
def provider_failover_pending!(from:, **)
|
|
247
248
|
return if @closed
|
|
248
249
|
|
|
249
|
-
|
|
250
|
-
|
|
251
|
-
|
|
250
|
+
identity = lane_identity(from)
|
|
251
|
+
if identity
|
|
252
|
+
log.warn('[llm][stream_assembler] action=provider_failover_pending ' \
|
|
253
|
+
"from=#{identity} request_id=#{@request_id}")
|
|
254
|
+
else
|
|
255
|
+
log.warn('[llm][stream_assembler] action=provider_failover_pending ' \
|
|
256
|
+
"lane_identity_missing=true request_id=#{@request_id}")
|
|
257
|
+
end
|
|
252
258
|
# Clear the partial canonical buffer — partial responses can't cross providers
|
|
253
259
|
# (thinking strip, context invalidation). The next provider starts fresh.
|
|
254
260
|
close_thinking_block if @thinking_block_open
|
|
@@ -272,8 +278,14 @@ module Legion
|
|
|
272
278
|
|
|
273
279
|
@current_lane = lane
|
|
274
280
|
@failover_chain << (lane.is_a?(Hash) ? lane[:id] : lane.to_s)
|
|
275
|
-
|
|
276
|
-
|
|
281
|
+
identity = lane_identity(lane)
|
|
282
|
+
if identity
|
|
283
|
+
log.info("[llm][stream_assembler] action=begin_dispatch lane=#{identity} " \
|
|
284
|
+
"request_id=#{@request_id}")
|
|
285
|
+
else
|
|
286
|
+
log.warn('[llm][stream_assembler] action=begin_dispatch ' \
|
|
287
|
+
"lane_identity_missing=true request_id=#{@request_id}")
|
|
288
|
+
end
|
|
277
289
|
end
|
|
278
290
|
|
|
279
291
|
def safe_replay_snapshot
|
|
@@ -303,6 +315,24 @@ module Legion
|
|
|
303
315
|
|
|
304
316
|
private
|
|
305
317
|
|
|
318
|
+
# Returns the exact five-tuple for complete internal lane hashes.
|
|
319
|
+
# Opaque or partial public values retain control behavior but are never
|
|
320
|
+
# echoed into logs.
|
|
321
|
+
def lane_identity(lane)
|
|
322
|
+
keys = %i[tier provider_family instance_id type model]
|
|
323
|
+
return unless lane.is_a?(Hash)
|
|
324
|
+
return unless keys.all? do |key|
|
|
325
|
+
value = lane[key]
|
|
326
|
+
next false unless lane.key?(key) && (value.is_a?(String) || value.is_a?(Symbol))
|
|
327
|
+
|
|
328
|
+
text = value.to_s
|
|
329
|
+
text.valid_encoding? && !text.match?(/[[:cntrl:]]/) && !text.strip.empty?
|
|
330
|
+
end
|
|
331
|
+
return unless Legion::Extensions::Llm::Taxonomies::TYPES.include?(lane[:type].to_sym)
|
|
332
|
+
|
|
333
|
+
"#{lane[:tier]}:#{lane[:provider_family]}:#{lane[:instance_id]}:#{lane[:type]}:#{lane[:model]}"
|
|
334
|
+
end
|
|
335
|
+
|
|
306
336
|
# G30 / D-F: emit x-legion-failover-* debug trailers after on_done.
|
|
307
337
|
# Only fires when failover actually happened (@failover_chain has markers).
|
|
308
338
|
# Trailers are response metadata, not SSE events — no wire-level event emitted.
|
|
@@ -58,17 +58,29 @@ module Legion
|
|
|
58
58
|
lease = dispatch_lease || Legion::Extensions::Llm::Inventory::Registry.acquire(
|
|
59
59
|
callable_handle: selection.callable_handle
|
|
60
60
|
)
|
|
61
|
+
log.debug("[llm][selection_dispatch] action=lease_acquired handle=#{selection.callable_handle.handle_id} " \
|
|
62
|
+
"lease=#{lease.lease_id}")
|
|
61
63
|
callable = lease.callable
|
|
62
64
|
|
|
63
65
|
begin
|
|
64
66
|
Result.success(value: invocation.call(callable))
|
|
65
67
|
rescue StandardError => e
|
|
68
|
+
log.warn("[llm][selection_dispatch] action=dispatch_failed class=#{e.class.name} " \
|
|
69
|
+
"message=#{e.message.to_s[0, 200]}")
|
|
70
|
+
raise if daemon_fault?(e)
|
|
71
|
+
|
|
66
72
|
Result.failure(outcome: normalize(callable: callable, error: e))
|
|
67
73
|
ensure
|
|
68
74
|
lease.release if owned_lease && !lease.released?
|
|
69
75
|
end
|
|
70
76
|
end
|
|
71
77
|
|
|
78
|
+
def self.daemon_fault?(error)
|
|
79
|
+
error.is_a?(::NoMethodError) || error.is_a?(::ArgumentError) ||
|
|
80
|
+
error.is_a?(::NotImplementedError) || error.is_a?(::TypeError)
|
|
81
|
+
end
|
|
82
|
+
private_class_method :daemon_fault?
|
|
83
|
+
|
|
72
84
|
# Normalize a provider error through the exact callable's Phase 1
|
|
73
85
|
# normalizer. A normalizer that raises or returns the wrong type is a
|
|
74
86
|
# programming failure — report and re-raise, never a silent success.
|
|
@@ -78,6 +90,8 @@ module Legion
|
|
|
78
90
|
raise TypeError,
|
|
79
91
|
"normalize_dispatch_error returned #{outcome.class}, expected ProviderOutcome"
|
|
80
92
|
end
|
|
93
|
+
log.warn("[llm][selection_dispatch] action=dispatch_normalized kind=#{outcome.kind} " \
|
|
94
|
+
"reason=#{outcome.reason.to_s[0, 200]}")
|
|
81
95
|
outcome
|
|
82
96
|
rescue StandardError => e
|
|
83
97
|
# Log the ORIGINAL dispatch error too — a normalizer that raises must never
|
|
@@ -87,6 +101,8 @@ module Legion
|
|
|
87
101
|
handled: false, lane_id: nil,
|
|
88
102
|
original_error_class: error.class.name,
|
|
89
103
|
original_error_message: error.message.to_s.dup.force_encoding(::Encoding::UTF_8).scrub('?')[0, 512])
|
|
104
|
+
log.warn("[llm][selection_dispatch] action=normalizer_failed class=#{e.class.name} " \
|
|
105
|
+
"message=#{e.message.to_s[0, 200]}")
|
|
90
106
|
raise
|
|
91
107
|
end
|
|
92
108
|
private_class_method :normalize
|
|
@@ -16,9 +16,10 @@ module Legion
|
|
|
16
16
|
extend Legion::Logging::Helper
|
|
17
17
|
|
|
18
18
|
ENVELOPE_KEYS = %i[
|
|
19
|
-
app_id caller correlation_id expires_at idempotency_key identity
|
|
20
|
-
model priority protocol_version provider
|
|
21
|
-
signed_token timeout timeout_seconds
|
|
19
|
+
app_id caller correlation_id expires_at execution_contract idempotency_key identity
|
|
20
|
+
message_context offering_id operation model priority protocol_version provider
|
|
21
|
+
provider_instance reply_to request_id routing_key signed_token timeout timeout_seconds
|
|
22
|
+
trace_context ttl
|
|
22
23
|
].freeze
|
|
23
24
|
LEGACY_FIELDS = %i[schema_version request_type fleet_correlation_id].freeze
|
|
24
25
|
|
|
@@ -96,6 +97,18 @@ module Legion
|
|
|
96
97
|
expires_at: (Time.now.utc + timeout).iso8601,
|
|
97
98
|
ttl: effective_ttl(request_opts, timeout)
|
|
98
99
|
}
|
|
100
|
+
execution_contract = fetch_option(request_opts, :execution_contract)
|
|
101
|
+
unless execution_contract.nil?
|
|
102
|
+
unless execution_contract == ::Legion::Extensions::Llm::Fleet::Protocol::EXACT_EXECUTION_CONTRACT
|
|
103
|
+
raise ArgumentError, "unknown fleet execution_contract marker: #{execution_contract.inspect}"
|
|
104
|
+
end
|
|
105
|
+
|
|
106
|
+
offering_id = fetch_option(request_opts, :offering_id)
|
|
107
|
+
raise ArgumentError, 'exact execution contract requires a nonempty String offering_id' unless offering_id.is_a?(String) && !offering_id.strip.empty?
|
|
108
|
+
|
|
109
|
+
envelope[:execution_contract] = execution_contract
|
|
110
|
+
envelope[:offering_id] = offering_id
|
|
111
|
+
end
|
|
99
112
|
envelope[:signed_token] = dispatch_auth_required? ? TokenIssuer.issue(envelope) : 'unsigned'
|
|
100
113
|
envelope
|
|
101
114
|
end
|
|
@@ -3,6 +3,7 @@
|
|
|
3
3
|
require 'securerandom'
|
|
4
4
|
require 'time'
|
|
5
5
|
|
|
6
|
+
require 'legion/extensions/llm/fleet/protocol'
|
|
6
7
|
require 'legion/extensions/llm/fleet/token_error'
|
|
7
8
|
|
|
8
9
|
module Legion
|
|
@@ -60,6 +61,18 @@ module Legion
|
|
|
60
61
|
|
|
61
62
|
claims[key] = value
|
|
62
63
|
end
|
|
64
|
+
execution_contract = payload[:execution_contract]
|
|
65
|
+
unless execution_contract.nil?
|
|
66
|
+
unless execution_contract == ::Legion::Extensions::Llm::Fleet::Protocol::EXACT_EXECUTION_CONTRACT
|
|
67
|
+
raise TokenError, "unknown fleet execution_contract marker: #{execution_contract.inspect}"
|
|
68
|
+
end
|
|
69
|
+
|
|
70
|
+
offering_id = payload[:offering_id]
|
|
71
|
+
raise TokenError, 'missing token claim source: nonempty String offering_id (exact execution marker present)' unless offering_id.is_a?(String) && !offering_id.strip.empty?
|
|
72
|
+
|
|
73
|
+
claims[:execution_contract] = execution_contract
|
|
74
|
+
claims[:offering_id] = offering_id
|
|
75
|
+
end
|
|
63
76
|
claims
|
|
64
77
|
end
|
|
65
78
|
|
|
@@ -1,6 +1,8 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
require 'faraday'
|
|
4
|
+
require 'legion/extensions/llm/inventory/errors'
|
|
5
|
+
require 'legion/extensions/llm/taxonomies'
|
|
4
6
|
|
|
5
7
|
module Legion
|
|
6
8
|
module LLM
|
|
@@ -58,9 +60,21 @@ module Legion
|
|
|
58
60
|
populate_ssot_v3_resolved_state(attempt)
|
|
59
61
|
result = ssot_v3_execute_attempt
|
|
60
62
|
action = @routing_session.classify(dispatch_result: result, attempt_context: attempt)
|
|
63
|
+
unless action.disposition == :success
|
|
64
|
+
lane = attempt.lane
|
|
65
|
+
log.warn("[llm][executor] action=ssot_v3_attempt_failed attempt=#{attempt.attempt_number} " \
|
|
66
|
+
"lane=#{lane.tier}:#{lane.provider_family}:#{lane.instance_id}:#{lane_type_for(lane.operation)}:#{lane.model} " \
|
|
67
|
+
"provider=#{@resolved_provider} " \
|
|
68
|
+
"instance=#{@resolved_instance} model=#{@resolved_model} " \
|
|
69
|
+
"outcome_kind=#{result.outcome.kind} outcome_reason=#{result.outcome.reason.to_s[0, 200]} " \
|
|
70
|
+
"disposition=#{action.disposition}")
|
|
71
|
+
end
|
|
61
72
|
return if action.disposition == :success
|
|
62
73
|
raise Legion::LLM::Errors::RoutingRejected.new(rejection: action.rejection) if action.disposition == :terminal
|
|
63
74
|
end
|
|
75
|
+
log.warn('[llm][executor] action=ssot_v3_attempts_exhausted kind=attempts_exhausted ' \
|
|
76
|
+
"reason=maximum attempts (#{@routing_requirements.maximum_attempts}) reached " \
|
|
77
|
+
"http_status=503 generation=#{Legion::Extensions::Llm::Inventory::Registry.snapshot.generation}")
|
|
64
78
|
raise Legion::LLM::Errors::RoutingRejected.new(rejection: attempts_exhausted_rejection)
|
|
65
79
|
ensure
|
|
66
80
|
@current_attempt_context = nil
|
|
@@ -73,11 +87,24 @@ module Legion
|
|
|
73
87
|
def ssot_v3_execute_attempt
|
|
74
88
|
execute_provider_request
|
|
75
89
|
Legion::LLM::Call::SelectionDispatch::Result.success(value: @raw_response)
|
|
90
|
+
rescue Legion::Extensions::Llm::Inventory::Errors::StaleCallableError,
|
|
91
|
+
Legion::Extensions::Llm::Inventory::Errors::CallableDisposedError => e
|
|
92
|
+
log.warn("[llm][executor] action=ssot_v3_stale_callable class=#{e.class.name} " \
|
|
93
|
+
"message=#{e.message.to_s[0, 200]}")
|
|
94
|
+
Legion::LLM::Call::SelectionDispatch::Result.failure(
|
|
95
|
+
outcome: Legion::Extensions::Llm::Routing::ProviderOutcome.new(
|
|
96
|
+
kind: :instance_unavailable, reason: e.class.name
|
|
97
|
+
)
|
|
98
|
+
)
|
|
76
99
|
rescue StandardError => e
|
|
77
|
-
raise
|
|
100
|
+
raise if non_provider_failure?(e)
|
|
78
101
|
|
|
79
102
|
outcome = @last_ssot_dispatch_outcome
|
|
80
103
|
@last_ssot_dispatch_outcome = nil
|
|
104
|
+
if outcome.nil?
|
|
105
|
+
log.warn("[llm][executor] action=ssot_v3_unnormalized_error class=#{e.class.name} " \
|
|
106
|
+
"message=#{e.message.to_s[0, 200]}")
|
|
107
|
+
end
|
|
81
108
|
outcome ||= Legion::Extensions::Llm::Routing::ProviderOutcome.new(
|
|
82
109
|
kind: :provider_error, reason: e.class.name.to_s
|
|
83
110
|
)
|
|
@@ -104,19 +131,42 @@ module Legion
|
|
|
104
131
|
# never an SSE server_error. Always selects (single engine) — an empty
|
|
105
132
|
# Registry yields a typed Rejection, never a legacy fallback.
|
|
106
133
|
def ssot_v3_stream_preflight
|
|
107
|
-
snap = Legion::Extensions::Llm::Inventory::Registry.snapshot
|
|
108
134
|
@stream_session = Legion::LLM::Inference::RoutingSession.new(
|
|
109
135
|
request: @request, requirements: @routing_requirements
|
|
110
136
|
)
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
137
|
+
remaining = @routing_requirements.maximum_attempts
|
|
138
|
+
while remaining.positive?
|
|
139
|
+
remaining -= 1
|
|
140
|
+
snap = Legion::Extensions::Llm::Inventory::Registry.snapshot
|
|
141
|
+
attempt_context = @stream_session.next_attempt!(snapshot: snap)
|
|
142
|
+
begin
|
|
143
|
+
@preflight_lease = Legion::Extensions::Llm::Inventory::Registry.acquire(
|
|
144
|
+
callable_handle: attempt_context.selection.callable_handle
|
|
145
|
+
)
|
|
146
|
+
rescue Legion::Extensions::Llm::Inventory::Errors::StaleCallableError,
|
|
147
|
+
Legion::Extensions::Llm::Inventory::Errors::CallableDisposedError => e
|
|
148
|
+
log.warn('[llm][executor] action=ssot_v3_stream_preflight_stale_callable ' \
|
|
149
|
+
"class=#{e.class.name} message=#{e.message.to_s[0, 200]}")
|
|
150
|
+
@stream_session.classify(
|
|
151
|
+
dispatch_result: Legion::LLM::Call::SelectionDispatch::Result.failure(
|
|
152
|
+
outcome: Legion::Extensions::Llm::Routing::ProviderOutcome.new(
|
|
153
|
+
kind: :instance_unavailable, reason: e.class.name
|
|
154
|
+
)
|
|
155
|
+
),
|
|
156
|
+
attempt_context: attempt_context
|
|
157
|
+
)
|
|
158
|
+
next
|
|
159
|
+
end
|
|
160
|
+
|
|
161
|
+
populate_ssot_v3_resolved_state(attempt_context)
|
|
162
|
+
lane = ssot_v3_stream_lane_hash(attempt_context)
|
|
163
|
+
log.info '[llm][executor] action=ssot_v3_stream_preflight_selected ' \
|
|
164
|
+
"lane=#{lane[:tier]}:#{lane[:provider_family]}:#{lane[:instance_id]}:#{lane[:type]}:#{lane[:model]} " \
|
|
165
|
+
"provider=#{@resolved_provider} model=#{@resolved_model}"
|
|
166
|
+
return lane
|
|
167
|
+
end
|
|
168
|
+
|
|
169
|
+
raise Legion::LLM::Errors::RoutingRejected.new(rejection: attempts_exhausted_rejection)
|
|
120
170
|
end
|
|
121
171
|
|
|
122
172
|
# Lane Hash consumed by StreamAssembler (#initial_lane, #begin_dispatch_on,
|
|
@@ -124,12 +174,14 @@ module Legion
|
|
|
124
174
|
# the failover debug trailers report real lane IDs — NOT 'unknown:pending'.
|
|
125
175
|
def ssot_v3_stream_lane_hash(attempt_context)
|
|
126
176
|
sel = attempt_context.selection
|
|
177
|
+
lane = attempt_context.lane
|
|
127
178
|
{
|
|
128
179
|
id: sel.lane_id,
|
|
129
180
|
provider_family: sel.provider_family,
|
|
130
181
|
instance_id: sel.instance_id,
|
|
131
182
|
model: sel.model,
|
|
132
|
-
tier:
|
|
183
|
+
tier: lane.tier,
|
|
184
|
+
type: lane_type_for(lane.operation)
|
|
133
185
|
}
|
|
134
186
|
end
|
|
135
187
|
|
|
@@ -192,11 +244,24 @@ module Legion
|
|
|
192
244
|
def ssot_v3_stream_handle_failure(error:, session:, attempt_context:)
|
|
193
245
|
raise error if non_provider_failure?(error)
|
|
194
246
|
|
|
195
|
-
outcome =
|
|
247
|
+
outcome = if error.is_a?(Legion::Extensions::Llm::Inventory::Errors::StaleCallableError) ||
|
|
248
|
+
error.is_a?(Legion::Extensions::Llm::Inventory::Errors::CallableDisposedError)
|
|
249
|
+
log.warn("[llm][executor] action=ssot_v3_stream_stale_callable class=#{error.class.name} " \
|
|
250
|
+
"message=#{error.message.to_s[0, 200]}")
|
|
251
|
+
Legion::Extensions::Llm::Routing::ProviderOutcome.new(
|
|
252
|
+
kind: :instance_unavailable, reason: error.class.name
|
|
253
|
+
)
|
|
254
|
+
else
|
|
255
|
+
ssot_v3_stream_failover_outcome(error)
|
|
256
|
+
end
|
|
196
257
|
action = session.classify(
|
|
197
258
|
dispatch_result: Legion::LLM::Call::SelectionDispatch::Result.failure(outcome: outcome),
|
|
198
259
|
attempt_context: attempt_context
|
|
199
260
|
)
|
|
261
|
+
if action.disposition == :terminal
|
|
262
|
+
log.warn("[llm][executor] action=ssot_v3_stream_terminal class=#{error.class.name} " \
|
|
263
|
+
"message=#{error.message.to_s[0, 200]} outcome_kind=#{outcome.kind}")
|
|
264
|
+
end
|
|
200
265
|
raise error if action.disposition == :terminal
|
|
201
266
|
|
|
202
267
|
# Preserve the existing StreamAssembler failover sequence (§19). The
|
|
@@ -210,14 +275,27 @@ module Legion
|
|
|
210
275
|
|
|
211
276
|
snap = Legion::Extensions::Llm::Inventory::Registry.snapshot
|
|
212
277
|
nxt = session.next_attempt(snapshot: snap)
|
|
278
|
+
if nxt.is_a?(Legion::Extensions::Llm::Routing::Rejection)
|
|
279
|
+
log.warn("[llm][executor] action=ssot_v3_stream_exhausted kind=#{nxt.kind} reason=#{nxt.reason.to_s[0, 200]} " \
|
|
280
|
+
"class=#{error.class.name} message=#{error.message.to_s[0, 200]}")
|
|
281
|
+
end
|
|
213
282
|
raise error if nxt.is_a?(Legion::Extensions::Llm::Routing::Rejection)
|
|
214
283
|
|
|
215
284
|
@stream_observer&.begin_dispatch_on(lane: ssot_v3_stream_lane_hash(nxt))
|
|
285
|
+
next_lane = nxt.lane
|
|
286
|
+
next_identity = [
|
|
287
|
+
next_lane.tier, next_lane.provider_family, next_lane.instance_id,
|
|
288
|
+
lane_type_for(next_lane.operation), next_lane.model
|
|
289
|
+
].join(':')
|
|
216
290
|
log.warn "[llm][executor] action=ssot_v3_stream_failover from_kind=#{outcome.kind} " \
|
|
217
|
-
"
|
|
291
|
+
"outcome_reason=#{outcome.reason.to_s[0, 200]} to_lane=#{next_identity}"
|
|
218
292
|
nxt
|
|
219
293
|
end
|
|
220
294
|
|
|
295
|
+
def lane_type_for(operation)
|
|
296
|
+
Legion::Extensions::Llm::Taxonomies.lane_type_for(operation: operation)
|
|
297
|
+
end
|
|
298
|
+
|
|
221
299
|
# Map a raised streaming provider error to a Phase 1 ProviderOutcome for
|
|
222
300
|
# classification. The exact outcome from SelectionDispatch is preserved
|
|
223
301
|
# losslessly by ssot_v3_direct_dispatch (@last_ssot_dispatch_outcome);
|
|
@@ -291,7 +369,8 @@ module Legion
|
|
|
291
369
|
# shared daemon code — retrying on a different lane guarantees the same crash. Classified as
|
|
292
370
|
# terminal: raise immediately, never retry, never trip circuits, never push to tried_lanes.
|
|
293
371
|
def internal_error?(err)
|
|
294
|
-
err.is_a?(::NoMethodError) || err.is_a?(::ArgumentError) ||
|
|
372
|
+
err.is_a?(::NoMethodError) || err.is_a?(::ArgumentError) ||
|
|
373
|
+
err.is_a?(::NotImplementedError) || err.is_a?(::TypeError)
|
|
295
374
|
end
|
|
296
375
|
|
|
297
376
|
# SSE assembly / canonical parse / translation errors originate inside LegionIO's
|
|
@@ -1,10 +1,16 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require 'legion/extensions/llm/fleet/protocol'
|
|
4
|
+
|
|
3
5
|
module Legion
|
|
4
6
|
module LLM
|
|
5
7
|
module Inference
|
|
6
8
|
# Shared executor helpers for direct/fleet provider dispatch attempt metadata.
|
|
7
9
|
module RouteAttempts
|
|
10
|
+
CONTRACT_OPTION_KEYS = %i[tools temperature params headers schema thinking tool_prefs].freeze
|
|
11
|
+
NONCONTRACT_GENERATION_KEYS = %i[top_p top_k frequency_penalty presence_penalty seed].freeze
|
|
12
|
+
private_constant :CONTRACT_OPTION_KEYS, :NONCONTRACT_GENERATION_KEYS
|
|
13
|
+
|
|
8
14
|
private
|
|
9
15
|
|
|
10
16
|
def fleet_dispatch?
|
|
@@ -12,13 +18,28 @@ module Legion
|
|
|
12
18
|
end
|
|
13
19
|
|
|
14
20
|
def dispatch_provider_request(capability:, operation:, messages:, stream_block: nil)
|
|
21
|
+
raw_options = native_dispatch_options
|
|
22
|
+
dispatch_messages, dispatch_options = if @current_attempt_context
|
|
23
|
+
project_dispatch_arguments(
|
|
24
|
+
messages: messages, options: raw_options
|
|
25
|
+
)
|
|
26
|
+
else
|
|
27
|
+
[messages, raw_options]
|
|
28
|
+
end
|
|
15
29
|
if fleet_dispatch?
|
|
16
|
-
|
|
17
|
-
|
|
30
|
+
fleet_operation = ssot_v3_fleet_operation(operation)
|
|
31
|
+
log.debug "[llm][route_attempts] action=dispatch path=fleet provider=#{@resolved_provider} model=#{@resolved_model} operation=#{fleet_operation}"
|
|
32
|
+
dispatch_fleet_request(
|
|
33
|
+
operation: fleet_operation, messages: dispatch_messages,
|
|
34
|
+
dispatch_options: dispatch_options, stream_block: stream_block
|
|
35
|
+
)
|
|
18
36
|
else
|
|
19
37
|
log.debug "[llm][route_attempts] action=dispatch path=direct provider=#{@resolved_provider} model=#{@resolved_model} operation=#{operation}"
|
|
20
|
-
dispatch_direct_request(
|
|
21
|
-
|
|
38
|
+
dispatch_direct_request(
|
|
39
|
+
capability: capability, operation: operation, messages: dispatch_messages,
|
|
40
|
+
raw_messages: messages, dispatch_options: dispatch_options,
|
|
41
|
+
raw_dispatch_options: raw_options, stream_block: stream_block
|
|
42
|
+
)
|
|
22
43
|
end
|
|
23
44
|
end
|
|
24
45
|
|
|
@@ -27,10 +48,10 @@ module Legion
|
|
|
27
48
|
# The capability :responses capability is a provider wire-format detail — the canonical
|
|
28
49
|
# core only knows :chat and :stream. See lex-llm-* adapters for wire-format decisions.
|
|
29
50
|
|
|
30
|
-
def dispatch_direct_request(capability:, operation:, messages:,
|
|
51
|
+
def dispatch_direct_request(capability:, operation:, messages:, raw_messages:,
|
|
52
|
+
dispatch_options:, raw_dispatch_options:, stream_block: nil)
|
|
31
53
|
idempotency_key = next_route_idempotency_key
|
|
32
|
-
|
|
33
|
-
enforce_final_context_budget!(messages, dispatch_options)
|
|
54
|
+
enforce_final_context_budget!(raw_messages, raw_dispatch_options)
|
|
34
55
|
|
|
35
56
|
if @current_attempt_context
|
|
36
57
|
return ssot_v3_direct_dispatch(
|
|
@@ -128,7 +149,8 @@ module Legion
|
|
|
128
149
|
end
|
|
129
150
|
end
|
|
130
151
|
|
|
131
|
-
def dispatch_fleet_request(operation:, messages:, stream_block: nil)
|
|
152
|
+
def dispatch_fleet_request(operation:, messages:, dispatch_options:, stream_block: nil)
|
|
153
|
+
validate_ssot_v3_fleet_operation!(operation)
|
|
132
154
|
idempotency_key = next_route_idempotency_key
|
|
133
155
|
selected_lane = fleet_selected_lane(operation)
|
|
134
156
|
log.info "[llm][route_attempts] action=fleet_dispatch provider=#{@resolved_provider} model=#{@resolved_model} lane=#{selected_lane} operation=#{operation}"
|
|
@@ -136,7 +158,9 @@ module Legion
|
|
|
136
158
|
operation: operation,
|
|
137
159
|
routing_key: selected_lane,
|
|
138
160
|
message_context: fleet_message_context,
|
|
139
|
-
request: fleet_dispatch_request(
|
|
161
|
+
request: fleet_dispatch_request(
|
|
162
|
+
messages, idempotency_key, dispatch_options: dispatch_options
|
|
163
|
+
)
|
|
140
164
|
)
|
|
141
165
|
if result[:success] == false || result['success'] == false
|
|
142
166
|
failure_reason = result[:error] || result['error'] || 'fleet_error'
|
|
@@ -164,6 +188,28 @@ module Legion
|
|
|
164
188
|
normalized
|
|
165
189
|
end
|
|
166
190
|
|
|
191
|
+
def ssot_v3_fleet_operation(requested_operation)
|
|
192
|
+
return requested_operation unless @current_attempt_context
|
|
193
|
+
|
|
194
|
+
selection_operation = @current_attempt_context.selection.operation
|
|
195
|
+
lane_operation = @current_attempt_context.lane.operation
|
|
196
|
+
unless selection_operation == lane_operation
|
|
197
|
+
raise ArgumentError,
|
|
198
|
+
"SSOT fleet selection/lane operation mismatch: #{selection_operation.inspect} != #{lane_operation.inspect}"
|
|
199
|
+
end
|
|
200
|
+
|
|
201
|
+
selection_operation
|
|
202
|
+
end
|
|
203
|
+
|
|
204
|
+
def validate_ssot_v3_fleet_operation!(operation)
|
|
205
|
+
return unless @current_attempt_context
|
|
206
|
+
return if operation == @current_attempt_context.selection.operation
|
|
207
|
+
|
|
208
|
+
raise ArgumentError,
|
|
209
|
+
"SSOT fleet envelope operation mismatch: #{operation.inspect} != " \
|
|
210
|
+
"#{@current_attempt_context.selection.operation.inspect}"
|
|
211
|
+
end
|
|
212
|
+
|
|
167
213
|
def enforce_final_context_budget!(messages, dispatch_options)
|
|
168
214
|
context_window = final_context_window
|
|
169
215
|
return unless context_window.positive?
|
|
@@ -197,7 +243,7 @@ module Legion
|
|
|
197
243
|
(metadata[:context_window] || metadata['context_window'] || limits[:context_window]).to_i
|
|
198
244
|
end
|
|
199
245
|
|
|
200
|
-
def fleet_dispatch_request(messages, idempotency_key)
|
|
246
|
+
def fleet_dispatch_request(messages, idempotency_key, dispatch_options:)
|
|
201
247
|
{
|
|
202
248
|
provider: @resolved_provider,
|
|
203
249
|
provider_instance: @resolved_instance || :default,
|
|
@@ -207,7 +253,46 @@ module Legion
|
|
|
207
253
|
caller: @request.caller,
|
|
208
254
|
trace_context: @tracing || {},
|
|
209
255
|
timeout: @request.ttl
|
|
210
|
-
}.merge(
|
|
256
|
+
}.merge(dispatch_options).merge(exact_execution_envelope_fields).compact
|
|
257
|
+
end
|
|
258
|
+
|
|
259
|
+
def exact_execution_envelope_fields
|
|
260
|
+
return {} unless @current_attempt_context
|
|
261
|
+
|
|
262
|
+
raise ArgumentError, 'SSOT fleet dispatch requires a nonempty String resolved offering_id' unless @resolved_offering_id.is_a?(String) && !@resolved_offering_id.strip.empty?
|
|
263
|
+
|
|
264
|
+
{
|
|
265
|
+
execution_contract: ::Legion::Extensions::Llm::Fleet::Protocol::EXACT_EXECUTION_CONTRACT,
|
|
266
|
+
offering_id: @resolved_offering_id
|
|
267
|
+
}
|
|
268
|
+
end
|
|
269
|
+
|
|
270
|
+
def project_dispatch_arguments(messages:, options:)
|
|
271
|
+
system_text = options[:system].to_s
|
|
272
|
+
folded = if system_text.strip.empty?
|
|
273
|
+
messages
|
|
274
|
+
else
|
|
275
|
+
fold_system_into_messages(messages: messages, system: system_text)
|
|
276
|
+
end
|
|
277
|
+
projected = options.slice(*CONTRACT_OPTION_KEYS)
|
|
278
|
+
params = projected[:params]
|
|
279
|
+
raise ArgumentError, "dispatch params must be a Hash, got #{params.class}" unless params.nil? || params.is_a?(Hash)
|
|
280
|
+
|
|
281
|
+
extras = NONCONTRACT_GENERATION_KEYS.each_with_object({}) do |key, acc|
|
|
282
|
+
acc[key] = options[key] if options.key?(key) && !options[key].nil?
|
|
283
|
+
end
|
|
284
|
+
projected[:params] = (params || {}).merge(extras) unless extras.empty?
|
|
285
|
+
[folded, projected]
|
|
286
|
+
end
|
|
287
|
+
|
|
288
|
+
def fold_system_into_messages(messages:, system:)
|
|
289
|
+
message_class = Legion::Extensions::Llm::Canonical::Message
|
|
290
|
+
first = messages.first
|
|
291
|
+
if first.is_a?(message_class) && first.role.to_s == 'system'
|
|
292
|
+
[first.with(content: system), *messages[1..]]
|
|
293
|
+
else
|
|
294
|
+
[message_class.build(role: :system, content: system), *messages]
|
|
295
|
+
end
|
|
211
296
|
end
|
|
212
297
|
|
|
213
298
|
def normalize_fleet_result(result)
|
|
@@ -260,6 +345,10 @@ module Legion
|
|
|
260
345
|
end
|
|
261
346
|
|
|
262
347
|
def record_route_attempt(dispatch_path:, operation:, status:, idempotency_key:, selected_lane:, failure_reason: nil)
|
|
348
|
+
log.debug "[llm][route_attempts] action=route_attempt provider=#{@resolved_provider} " \
|
|
349
|
+
"instance=#{@resolved_instance} model=#{@resolved_model} operation=#{operation} " \
|
|
350
|
+
"dispatch_path=#{dispatch_path} status=#{status} " \
|
|
351
|
+
"failure_reason=#{failure_reason.to_s[0, 200] if failure_reason}"
|
|
263
352
|
attempt = {
|
|
264
353
|
provider: @resolved_provider,
|
|
265
354
|
provider_instance: @resolved_instance || :default,
|
|
@@ -72,6 +72,8 @@ module Legion
|
|
|
72
72
|
# remove or overwrite a consumed attempt-target exclusion.
|
|
73
73
|
def add_exclusion(exclusion:)
|
|
74
74
|
@exclusions << exclusion
|
|
75
|
+
log.debug("[llm][routing_session] action=exclusion_added kind=#{exclusion.target_kind} " \
|
|
76
|
+
"reason=#{exclusion.reason} target_kind_seen=#{exclusion.target_kind}")
|
|
75
77
|
exclusion
|
|
76
78
|
end
|
|
77
79
|
|
|
@@ -85,6 +87,8 @@ module Legion
|
|
|
85
87
|
)
|
|
86
88
|
apply_global_transition(action.global_transition) if action.global_transition
|
|
87
89
|
action.exclusions.each { |ex| add_exclusion(exclusion: ex) }
|
|
90
|
+
log.debug("[llm][routing_session] action=outcome_classified kind=#{dispatch_result.outcome.kind} " \
|
|
91
|
+
"disposition=#{action.disposition} exclusions_added=#{action.exclusions.size}")
|
|
88
92
|
action
|
|
89
93
|
end
|
|
90
94
|
|
|
@@ -109,6 +113,7 @@ module Legion
|
|
|
109
113
|
target_kind: :attempt_target, target: key, reason: 'attempt_consumed',
|
|
110
114
|
evidence: { attempt_number: @attempt_count }, lifetime: :request
|
|
111
115
|
)
|
|
116
|
+
log.debug("[llm][routing_session] action=attempt_consumed attempt_number=#{@attempt_count} target=#{key}")
|
|
112
117
|
end
|
|
113
118
|
|
|
114
119
|
def apply_global_transition(transition)
|
|
@@ -48,6 +48,15 @@ module Legion
|
|
|
48
48
|
"candidates=#{candidates.size} " \
|
|
49
49
|
"pub_statuses=#{publication_statuses.size} " \
|
|
50
50
|
"generation=#{snapshot.generation}")
|
|
51
|
+
log.debug('[llm][candidate_evaluator] action=evaluated_summary ' \
|
|
52
|
+
"ready=#{candidates.count(&:ready?)} " \
|
|
53
|
+
"not_ready=#{candidates.count { |c| !c.ready? }} " \
|
|
54
|
+
"policy_denied=#{candidates.count { |c| !c.ready? && c.policy_state == :denied }} " \
|
|
55
|
+
"capability_failed=#{candidates.count { |c| !c.ready? && c.capability_state != :supported }} " \
|
|
56
|
+
"context_rejected=#{candidates.count { |c| !c.ready? && !%i[fits not_applicable].include?(c.context_state) }} " \
|
|
57
|
+
"unavailable=#{candidates.count { |c| !c.ready? && c.availability_state == :unavailable }} " \
|
|
58
|
+
"excluded=#{candidates.count { |c| !c.ready? && c.exclusion_state == :excluded }} " \
|
|
59
|
+
"weight_disabled=#{candidates.count { |c| !c.ready? && c.weight_state == :disabled }}")
|
|
51
60
|
|
|
52
61
|
EvaluationSet.new(
|
|
53
62
|
candidates: candidates,
|
|
@@ -109,11 +118,8 @@ module Legion
|
|
|
109
118
|
# Step 9 — fleet contract marker when tier is :fleet
|
|
110
119
|
fleet_contract_state = evaluate_fleet_contract(offering: offering)
|
|
111
120
|
|
|
112
|
-
# Step 10 —
|
|
113
|
-
weight_state, weight_inputs = evaluate_weight(
|
|
114
|
-
lane: lane,
|
|
115
|
-
settings_snapshot: settings_snapshot
|
|
116
|
-
)
|
|
121
|
+
# Step 10 — stored write-time weight components
|
|
122
|
+
weight_state, weight_inputs = evaluate_weight(lane: lane)
|
|
117
123
|
|
|
118
124
|
# Soft sieve input for Ranker §10.1
|
|
119
125
|
preferred_context_match = evaluate_preferred_context(
|
|
@@ -355,15 +361,15 @@ module Legion
|
|
|
355
361
|
end
|
|
356
362
|
end
|
|
357
363
|
|
|
358
|
-
# §9.7 step 10 —
|
|
364
|
+
# §9.7 step 10 — stored write-time weight components.
|
|
359
365
|
# Any zero component → :disabled with nil weight_inputs.
|
|
360
366
|
# All positive → :enabled with frozen weight_inputs Hash.
|
|
361
367
|
# Nil lane (unsupported/unknown operation) → :disabled.
|
|
362
|
-
def evaluate_weight(lane
|
|
368
|
+
def evaluate_weight(lane:)
|
|
363
369
|
return [:disabled, nil] if lane.nil?
|
|
364
370
|
|
|
365
|
-
inputs =
|
|
366
|
-
if inputs.values.any?(&:zero?)
|
|
371
|
+
inputs = lane.weight_inputs
|
|
372
|
+
if inputs.nil? || inputs.values.any?(&:zero?)
|
|
367
373
|
[:disabled, nil]
|
|
368
374
|
else
|
|
369
375
|
[:enabled, inputs]
|
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
require 'digest'
|
|
4
|
+
require 'legion/extensions/llm/taxonomies'
|
|
4
5
|
|
|
5
6
|
module Legion
|
|
6
7
|
module LLM
|
|
@@ -66,12 +67,22 @@ module Legion
|
|
|
66
67
|
log.debug("[llm][ranker] action=rank ready_count=#{ready.size} " \
|
|
67
68
|
"seed=#{@requirements.routing_seed[0, 8]}...")
|
|
68
69
|
|
|
69
|
-
#
|
|
70
|
-
|
|
70
|
+
# Two-pass band ordering (preferred != required): pass 1 ranks the in-band
|
|
71
|
+
# lanes; pass 2 (only when pass 1 is empty) ranks everything else,
|
|
72
|
+
# band-ignoring. No lane is excluded by the band -- the band is a priority,
|
|
73
|
+
# not a filter. Only the selected pass is computed, so the ranked log lines
|
|
74
|
+
# are exactly the lanes eligible for this attempt.
|
|
75
|
+
in_band, out_of_band = band_partition(ready)
|
|
76
|
+
|
|
77
|
+
log.debug("[llm][ranker] action=preferred_band_partition ready=#{ready.size} " \
|
|
78
|
+
"in_band=#{in_band.size} out_of_band=#{out_of_band.size}")
|
|
71
79
|
|
|
72
80
|
# Steps 3–4 (§10.2 + D17): base weight, affinity, effective weight.
|
|
73
81
|
# Step 5 (§10.3): rendezvous score.
|
|
74
|
-
|
|
82
|
+
# Pass 1 wins if non-empty; otherwise pass 2. Within a pass: greatest
|
|
83
|
+
# effective_weight bucket -> greatest rendezvous score -> ascending lane_id.
|
|
84
|
+
ranked = compute_ranked(in_band) if in_band.any?
|
|
85
|
+
ranked ||= compute_ranked(out_of_band)
|
|
75
86
|
|
|
76
87
|
# Select the winner (greatest effective_weight bucket → greatest rendezvous
|
|
77
88
|
# score → lexicographically ascending lane_id for cryptographically
|
|
@@ -85,24 +96,22 @@ module Legion
|
|
|
85
96
|
# §10.1 Preferred-context soft sieve #
|
|
86
97
|
# ------------------------------------------------------------------ #
|
|
87
98
|
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
ready
|
|
99
|
+
# Partition ready candidates by preferred-band containment. A lane with no
|
|
100
|
+
# configured range is out_of_band (generalist). A lane whose range does not
|
|
101
|
+
# contain the budget is also out_of_band -- never excluded.
|
|
102
|
+
def band_partition(ready)
|
|
103
|
+
budget = @requirements.required_context_budget
|
|
104
|
+
in_band = []
|
|
105
|
+
out_of_band = []
|
|
106
|
+
ready.each do |candidate|
|
|
107
|
+
range = @settings_snapshot.preferred_context_range_for(lane: candidate.lane)
|
|
108
|
+
if range && range_contains?(range, budget)
|
|
109
|
+
in_band << candidate
|
|
110
|
+
else
|
|
111
|
+
out_of_band << candidate
|
|
112
|
+
end
|
|
113
|
+
end
|
|
114
|
+
[in_band, out_of_band]
|
|
106
115
|
end
|
|
107
116
|
|
|
108
117
|
# True when +budget+ falls within the (nil-open) range [min, max) —
|
|
@@ -118,22 +127,25 @@ module Legion
|
|
|
118
127
|
# ------------------------------------------------------------------ #
|
|
119
128
|
|
|
120
129
|
def compute_ranked(candidates)
|
|
121
|
-
# Precompute the tier boost value once; it is the same for every candidate.
|
|
122
|
-
tier_max_plus_one = @settings_snapshot.tier_weights.values.max + 1
|
|
123
|
-
|
|
124
130
|
candidates.map do |candidate|
|
|
125
|
-
|
|
126
|
-
|
|
131
|
+
# Stored write-time scalar: the operator's tier/provider/instance/model
|
|
132
|
+
# weights are baked in by the inventory writer. The former request-time
|
|
133
|
+
# tier_preference substitution is deliberately removed: applying a tier
|
|
134
|
+
# component to the stored scalar would double-count that axis. A future
|
|
135
|
+
# request-time tier preference requires a first-class ppm-domain term.
|
|
136
|
+
base = candidate.lane.base_weight
|
|
127
137
|
ppm = compute_preference_ppm(candidate)
|
|
128
138
|
eff = base * ppm
|
|
129
139
|
score = rendezvous_score(candidate)
|
|
130
140
|
|
|
131
|
-
|
|
141
|
+
lane = candidate.lane
|
|
142
|
+
log.debug('[llm][ranker] action=ranked ' \
|
|
143
|
+
"lane=#{lane.tier}:#{lane.provider_family}:#{lane.instance_id}:#{lane_type_for(lane.operation)}:#{lane.model} " \
|
|
132
144
|
"base=#{base} ppm=#{ppm} eff=#{eff}")
|
|
133
145
|
|
|
134
146
|
RankedCandidate.new(
|
|
135
147
|
evaluation: candidate,
|
|
136
|
-
weight_inputs:
|
|
148
|
+
weight_inputs: candidate.lane.weight_inputs,
|
|
137
149
|
base_weight: base,
|
|
138
150
|
preference_ppm: ppm,
|
|
139
151
|
effective_weight: eff,
|
|
@@ -142,20 +154,6 @@ module Legion
|
|
|
142
154
|
end
|
|
143
155
|
end
|
|
144
156
|
|
|
145
|
-
# §10.2: optionally substitute the tier weight for the preferred tier.
|
|
146
|
-
# All other components are taken verbatim from the evaluation-time snapshot.
|
|
147
|
-
def build_weight_inputs(candidate, tier_max_plus_one)
|
|
148
|
-
base_wi = candidate.weight_inputs
|
|
149
|
-
tier_w = if @requirements.tier_preference &&
|
|
150
|
-
candidate.lane.tier == @requirements.tier_preference
|
|
151
|
-
tier_max_plus_one
|
|
152
|
-
else
|
|
153
|
-
base_wi[:tier]
|
|
154
|
-
end
|
|
155
|
-
{ tier: tier_w, provider: base_wi[:provider],
|
|
156
|
-
instance: base_wi[:instance], model_or_offering: base_wi[:model_or_offering] }.freeze
|
|
157
|
-
end
|
|
158
|
-
|
|
159
157
|
# §10.2/D17: compute preference_ppm using integer floor division throughout.
|
|
160
158
|
# Bounds: 500_000..1_500_000 (clamped as a belt-and-suspenders guard; the
|
|
161
159
|
# input constraints guarantee the range without it).
|
|
@@ -216,7 +214,16 @@ module Legion
|
|
|
216
214
|
def select_winner(ranked)
|
|
217
215
|
max_ew = ranked.map(&:effective_weight).max
|
|
218
216
|
bucket = ranked.select { |rc| rc.effective_weight == max_ew }
|
|
219
|
-
bucket.min_by { |rc| [-rc.rendezvous_score, rc.evaluation.lane.lane_id] }
|
|
217
|
+
winner = bucket.min_by { |rc| [-rc.rendezvous_score, rc.evaluation.lane.lane_id] }
|
|
218
|
+
lane = winner.evaluation.lane
|
|
219
|
+
log.debug('[llm][ranker] action=selected ' \
|
|
220
|
+
"lane=#{lane.tier}:#{lane.provider_family}:#{lane.instance_id}:#{lane_type_for(lane.operation)}:#{lane.model} " \
|
|
221
|
+
"eff=#{winner.effective_weight}")
|
|
222
|
+
winner
|
|
223
|
+
end
|
|
224
|
+
|
|
225
|
+
def lane_type_for(operation)
|
|
226
|
+
Legion::Extensions::Llm::Taxonomies.lane_type_for(operation: operation)
|
|
220
227
|
end
|
|
221
228
|
end
|
|
222
229
|
end
|
|
@@ -15,12 +15,11 @@ module Legion
|
|
|
15
15
|
|
|
16
16
|
TIER_KEYS = %i[direct local fleet cloud frontier].freeze
|
|
17
17
|
ALLOWED_ALIAS_META = %i[owned_by created context_window max_output_tokens].freeze
|
|
18
|
-
IDENTITY_WEIGHT = 1
|
|
19
18
|
# Shared 3-level settings cascade (lex-llm): provider -> instance ->
|
|
20
19
|
# model, most-specific-first. The instance leg is the operator's
|
|
21
20
|
# CONFIG NAME (InstanceKey#instance_id), never a derived id.
|
|
22
21
|
CASCADE = Legion::Extensions::Llm::SettingsCascade
|
|
23
|
-
private_constant :TIER_KEYS, :ALLOWED_ALIAS_META, :
|
|
22
|
+
private_constant :TIER_KEYS, :ALLOWED_ALIAS_META, :CASCADE
|
|
24
23
|
|
|
25
24
|
# ------------------------------------------------------------------ #
|
|
26
25
|
# Public factory #
|
|
@@ -51,29 +50,6 @@ module Legion
|
|
|
51
50
|
:auto_routing_model_aliases,
|
|
52
51
|
:auto_routing_model_alias_metadata
|
|
53
52
|
|
|
54
|
-
# ------------------------------------------------------------------ #
|
|
55
|
-
# Per-lane weight inputs (called per candidate during ranking) #
|
|
56
|
-
# ------------------------------------------------------------------ #
|
|
57
|
-
|
|
58
|
-
# Returns a frozen Hash { tier:, provider:, instance:, model_or_offering: }
|
|
59
|
-
# where each value is a positive Integer >= 1 (missing component → identity 1).
|
|
60
|
-
# The provider/instance/model scopes are read through the lex-llm
|
|
61
|
-
# cascade, keyed by the config name (lane.instance_id): the instance
|
|
62
|
-
# leg is instances.<name>, the model leg is the instance's models.<model>
|
|
63
|
-
# entry overriding the provider's models.<model> entry. A zero tier
|
|
64
|
-
# weight is stored as-is (disabled lane); callers check for zero.
|
|
65
|
-
def weight_inputs_for(lane:)
|
|
66
|
-
tier_w = tier_weights[lane.tier] || IDENTITY_WEIGHT
|
|
67
|
-
prov = ext_llm_provider(lane.provider_family)
|
|
68
|
-
inst = ext_llm_instance(prov, lane.instance_id)
|
|
69
|
-
prov_w = CASCADE.lookup(prov, :weight) || IDENTITY_WEIGHT
|
|
70
|
-
inst_w = CASCADE.lookup(inst, :weight) || IDENTITY_WEIGHT
|
|
71
|
-
off_entry = CASCADE.lookup(CASCADE.lookup(prov, :offerings), lane.offering_id)
|
|
72
|
-
model_cfg = CASCADE.merge_model_scopes(provider_conf: prov, instance_cfg: inst, model: lane.model)
|
|
73
|
-
model_w = CASCADE.lookup(off_entry, :weight) || CASCADE.lookup(model_cfg, :weight) || IDENTITY_WEIGHT
|
|
74
|
-
{ tier: tier_w, provider: prov_w, instance: inst_w, model_or_offering: model_w }.freeze
|
|
75
|
-
end
|
|
76
|
-
|
|
77
53
|
# Returns { min: Integer_or_nil, max: Integer_or_nil } or nil when no
|
|
78
54
|
# preferred range is configured. Resolved through the lex-llm 3-level
|
|
79
55
|
# cascade (provider -> instance -> model, most-specific-first) keyed
|
data/lib/legion/llm/version.rb
CHANGED
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: legion-llm
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.16.
|
|
4
|
+
version: 0.16.8
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Esity
|
|
@@ -141,14 +141,14 @@ dependencies:
|
|
|
141
141
|
requirements:
|
|
142
142
|
- - ">="
|
|
143
143
|
- !ruby/object:Gem::Version
|
|
144
|
-
version: 0.7.
|
|
144
|
+
version: 0.7.5
|
|
145
145
|
type: :runtime
|
|
146
146
|
prerelease: false
|
|
147
147
|
version_requirements: !ruby/object:Gem::Requirement
|
|
148
148
|
requirements:
|
|
149
149
|
- - ">="
|
|
150
150
|
- !ruby/object:Gem::Version
|
|
151
|
-
version: 0.7.
|
|
151
|
+
version: 0.7.5
|
|
152
152
|
- !ruby/object:Gem::Dependency
|
|
153
153
|
name: pdf-reader
|
|
154
154
|
requirement: !ruby/object:Gem::Requirement
|