lex-llm 0.8.5 → 0.8.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +21 -0
- data/README.md +25 -2
- data/lex-llm.gemspec +1 -1
- data/lib/legion/extensions/llm/canonical/content_block.rb +9 -0
- data/lib/legion/extensions/llm/canonical/request.rb +5 -0
- data/lib/legion/extensions/llm/connection.rb +3 -1
- data/lib/legion/extensions/llm/fleet/default_exchange_reply.rb +4 -3
- data/lib/legion/extensions/llm/provider/open_ai_compatible.rb +11 -2
- data/lib/legion/extensions/llm/routing/request_requirements.rb +70 -0
- data/lib/legion/extensions/llm/transport/messages/fleet_request.rb +4 -3
- data/lib/legion/extensions/llm/version.rb +1 -1
- data/lib/legion/extensions/llm.rb +1 -0
- data/spec/legion/extensions/llm/canonical/content_block_spec.rb +31 -6
- data/spec/legion/extensions/llm/canonical/request_spec.rb +20 -0
- data/spec/legion/extensions/llm/fleet/exact_offering_spec.rb +10 -1
- data/spec/legion/extensions/llm/fleet_messages_spec.rb +33 -8
- data/spec/legion/extensions/llm/provider/open_ai_compatible_spec.rb +30 -18
- data/spec/legion/extensions/llm/routing/request_requirements_spec.rb +74 -0
- data/spec/support/tls_endpoint.rb +2 -2
- metadata +5 -3
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: f21576528494d0876f279ea9a60a61b0ae66b00d6f2cf42edf023fea4e937b55
|
|
4
|
+
data.tar.gz: b5b4d3a0b3ac21cf23f94e511c059fab5d5f0bf925a137389ee1cf5254b252f5
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 647f69802e60ee7be7c8d428999f337a24fc2d66f65b315631046cbebf44ffcaf36416076857f0efd3127a7728eb84852a143a603b5fb2e28aa7c55257110c20
|
|
7
|
+
data.tar.gz: 575f2293e9a1d8a97135c950d0937b24b42c2e6c14d8c4cb49d69d8001f656e5e8d9d3aecc2b50914cfe6b089b140913bb3382288233446473f98d96e621cf41
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,26 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.8.8 - 2026-11-18
|
|
4
|
+
|
|
5
|
+
### Fixed
|
|
6
|
+
- Handle a peer connection reset in the local TLS test endpoint when a client rejects an untrusted certificate. Linux OpenSSL/curl may reset the socket after the server accepts it; the certificate-rejection and no-authenticated-request assertions remain unchanged.
|
|
7
|
+
|
|
8
|
+
## 0.8.7 - 2026-11-17
|
|
9
|
+
|
|
10
|
+
### Fixed
|
|
11
|
+
- Enforce the canonical system member as a String or absent across direct construction, builders, and JSON rehydration. Client block arrays must be translated at ingress instead of leaking into provider dispatch.
|
|
12
|
+
|
|
13
|
+
## 0.8.6 - 2026-10-31
|
|
14
|
+
|
|
15
|
+
### Fixed
|
|
16
|
+
- Normalize canonical image source types at construction and JSON rehydration so fleet execution preserves base64 and URL image meaning.
|
|
17
|
+
- Render mixed text and images as OpenAI-compatible text/image_url parts through the real provider callable, preserving media bytes, order, and image detail.
|
|
18
|
+
- Decode HTTP JSON responses through `Legion::JSON` at the shared connection boundary; Faraday's positional parser options no longer break JSON 3 responses.
|
|
19
|
+
- Validate the exact encrypted fleet request/reply bytes once before publication using `legion-transport >= 1.4.31`, preserving their matching encryption headers.
|
|
20
|
+
|
|
21
|
+
### Added
|
|
22
|
+
- Immutable shared `Routing::RequestRequirements` for request-owned routing facts; it validates contracts without consulting Inventory or choosing a provider.
|
|
23
|
+
|
|
3
24
|
## 0.8.5 - 2026-10-29
|
|
4
25
|
|
|
5
26
|
### Changed
|
data/README.md
CHANGED
|
@@ -41,11 +41,34 @@ gem 'lex-llm'
|
|
|
41
41
|
Provider extensions should declare `lex-llm` as a gemspec dependency:
|
|
42
42
|
|
|
43
43
|
```ruby
|
|
44
|
-
spec.add_dependency 'lex-llm', '>= 0.
|
|
44
|
+
spec.add_dependency 'lex-llm', '>= 0.8.6'
|
|
45
45
|
```
|
|
46
46
|
|
|
47
47
|
For local development across LegionIO repos, prefer a local path override in the app or test `Gemfile`, not a permanent git dependency in the gemspec.
|
|
48
48
|
|
|
49
|
+
## Canonical execution contracts
|
|
50
|
+
|
|
51
|
+
`Canonical::ContentBlock` owns media source normalization: supported image sources
|
|
52
|
+
are `:base64` and `:url`. String enum values from JSON are normalized by the strict
|
|
53
|
+
constructor, including `from_hash`; an image without an explicit source type is a
|
|
54
|
+
contract error. `ContentBlock.image` explicitly defaults to `:base64`.
|
|
55
|
+
Fleet workers rehydrate serialized messages through these same constructors before
|
|
56
|
+
calling a provider, preserving media bytes, MIME type, ordering and image detail.
|
|
57
|
+
|
|
58
|
+
Fleet publishers require `legion-transport >= 1.4.31`: they encode once, validate
|
|
59
|
+
the final encrypted byte count, then publish those exact bytes with the matching
|
|
60
|
+
encryption headers. HTTP response decoding uses `Legion::JSON` at the shared
|
|
61
|
+
connection boundary and supports Faraday's positional and keyword parser APIs.
|
|
62
|
+
|
|
63
|
+
`Provider::OpenAICompatible` translates canonical text/image blocks into OpenAI
|
|
64
|
+
`text`/`image_url` parts, never raw canonical media fields. Provider-specific cache
|
|
65
|
+
controls remain canonical metadata unless the selected provider supports them.
|
|
66
|
+
|
|
67
|
+
`Routing::RequestRequirements` is the shared immutable record for operation,
|
|
68
|
+
capabilities, context/input/output bounds, modality, pins, hint disposition, routing
|
|
69
|
+
seed and attempt limit. It validates supplied facts without reading settings or
|
|
70
|
+
Inventory. `legion-llm` constructs it before its sole Router chooses a Selection.
|
|
71
|
+
|
|
49
72
|
## Namespace
|
|
50
73
|
|
|
51
74
|
Load the extension through the Legion namespace:
|
|
@@ -178,7 +201,7 @@ end
|
|
|
178
201
|
| `Error` | `lib/.../error.rb` | Base error class for lex-llm. |
|
|
179
202
|
| `Errors::UnsupportedCapability` | `lib/.../errors/unsupported_capability.rb` | Raised when a provider lacks a requested capability. |
|
|
180
203
|
| `Utils` | `lib/.../utils.rb` | Shared utility methods. |
|
|
181
|
-
| `VERSION` | `lib/.../version.rb` | Current gem version (`0.
|
|
204
|
+
| `VERSION` | `lib/.../version.rb` | Current gem version (`0.8.8`). |
|
|
182
205
|
|
|
183
206
|
---
|
|
184
207
|
|
data/lex-llm.gemspec
CHANGED
|
@@ -41,7 +41,7 @@ Gem::Specification.new do |spec|
|
|
|
41
41
|
spec.add_dependency 'legion-json', '>= 1.2.1'
|
|
42
42
|
spec.add_dependency 'legion-logging', '>= 1.3.2'
|
|
43
43
|
spec.add_dependency 'legion-settings', '>= 1.4.2'
|
|
44
|
-
spec.add_dependency 'legion-transport', '>= 1.4.
|
|
44
|
+
spec.add_dependency 'legion-transport', '>= 1.4.31'
|
|
45
45
|
spec.add_dependency 'marcel', '~> 1'
|
|
46
46
|
spec.add_dependency 'ruby_llm-schema', '~> 0'
|
|
47
47
|
end
|
|
@@ -59,6 +59,13 @@ module Legion
|
|
|
59
59
|
Strict.enum!(type_sym, self::CONTENT_BLOCK_TYPES, site, :type)
|
|
60
60
|
end
|
|
61
61
|
|
|
62
|
+
def self.normalize_source_type!(source_type, type, site)
|
|
63
|
+
raise ArgumentError, "#{site}: image source_type is required" if type == :image && source_type.nil?
|
|
64
|
+
|
|
65
|
+
source_type = source_type.to_sym if source_type.is_a?(::String)
|
|
66
|
+
Strict.enum!(source_type, self::SOURCE_TYPES, site, :source_type)
|
|
67
|
+
end
|
|
68
|
+
|
|
62
69
|
# Build a text content block.
|
|
63
70
|
def self.text(content, cache_control: nil)
|
|
64
71
|
build(type: :text, text: content, cache_control:)
|
|
@@ -136,12 +143,14 @@ module Legion
|
|
|
136
143
|
# delegate here.
|
|
137
144
|
Strict.install_strict_new!(self) do |values, site|
|
|
138
145
|
values[:type] = normalize_type!(values[:type], site)
|
|
146
|
+
values[:source_type] = normalize_source_type!(values[:source_type], values[:type], site)
|
|
139
147
|
values[:metadata] = Strict.metadata!(values[:metadata], site)
|
|
140
148
|
values
|
|
141
149
|
end
|
|
142
150
|
end
|
|
143
151
|
|
|
144
152
|
ContentBlock::CONTENT_BLOCK_TYPES = %i[text thinking tool_use tool_result image audio video].freeze
|
|
153
|
+
ContentBlock::SOURCE_TYPES = %i[base64 url].freeze
|
|
145
154
|
ContentBlock::BUILD_SITE = 'Canonical::ContentBlock.build'
|
|
146
155
|
ContentBlock::FROM_HASH_SITE = 'Canonical::ContentBlock.from_hash'
|
|
147
156
|
ContentBlock::NEW_SITE = 'Canonical::ContentBlock.new'
|
|
@@ -120,6 +120,11 @@ module Legion
|
|
|
120
120
|
# delegate here.
|
|
121
121
|
Strict.install_strict_new!(self) do |values, site|
|
|
122
122
|
values[:messages] = normalize_messages!(values[:messages], site)
|
|
123
|
+
# The system string contract: a String (or absent) is the only
|
|
124
|
+
# canonical shape. Client dialects converge multi-part system
|
|
125
|
+
# wire at their own edge; a block array here is a client-edge
|
|
126
|
+
# defect leaking past the boundary — fail loud, never stringify.
|
|
127
|
+
values[:system] = Strict.expect_type!(values[:system], [::String], site, :system)
|
|
123
128
|
values[:tools] = normalize_tools(values[:tools], site)
|
|
124
129
|
values[:tool_choice] = values[:tool_choice].is_a?(::String) ? values[:tool_choice].to_sym : values[:tool_choice]
|
|
125
130
|
values[:params] = normalize_params!(values[:params], site)
|
|
@@ -125,7 +125,9 @@ module Legion
|
|
|
125
125
|
def setup_middleware(faraday)
|
|
126
126
|
faraday.request :multipart
|
|
127
127
|
faraday.request :json
|
|
128
|
-
faraday.response :json
|
|
128
|
+
faraday.response :json, parser_options: {
|
|
129
|
+
decoder: [->(body, *) { Legion::JSON.parse(body, symbolize_names: false) }, :call]
|
|
130
|
+
}
|
|
129
131
|
# Prefer typhoeus (libcurl) over net_http to avoid Ruby 4.0 + net-http-0.9.1 SSL issues
|
|
130
132
|
begin
|
|
131
133
|
require 'faraday/typhoeus'
|
|
@@ -22,13 +22,14 @@ module Legion
|
|
|
22
22
|
|
|
23
23
|
requested_options = DEFAULT_REPLY_PUBLISH_OPTIONS.merge(@options).merge(options || {})
|
|
24
24
|
return_result = return_publish_result?(requested_options)
|
|
25
|
+
payload = encode_message
|
|
26
|
+
validate_payload_size(payload: payload)
|
|
25
27
|
publish_options = reply_publish_options(requested_options)
|
|
26
|
-
validate_payload_size
|
|
27
28
|
default_exchange = channel.default_exchange
|
|
28
29
|
return_state = {}
|
|
29
30
|
install_return_listener(default_exchange, requested_options, return_state)
|
|
30
31
|
prepare_publisher_confirms(default_exchange, requested_options)
|
|
31
|
-
default_exchange.publish(
|
|
32
|
+
default_exchange.publish(payload, **publish_options)
|
|
32
33
|
return nil unless return_result
|
|
33
34
|
|
|
34
35
|
publish_result(default_exchange, requested_options.merge(publish_options), return_state)
|
|
@@ -56,7 +57,7 @@ module Legion
|
|
|
56
57
|
{
|
|
57
58
|
routing_key: routing_key,
|
|
58
59
|
content_type: options[:content_type] || content_type,
|
|
59
|
-
content_encoding: options[:content_encoding] || content_encoding,
|
|
60
|
+
content_encoding: @options[:content_encoding] || content_encoding,
|
|
60
61
|
type: options[:type] || type,
|
|
61
62
|
priority: options[:priority] || priority,
|
|
62
63
|
expiration: options[:expiration] || expiration,
|
|
@@ -85,7 +85,15 @@ module Legion
|
|
|
85
85
|
return content.map { |block| openai_content(block) } if content.is_a?(::Array)
|
|
86
86
|
return content.to_s if content.nil? || content.is_a?(::String)
|
|
87
87
|
|
|
88
|
-
return content.text.to_s if content.text?
|
|
88
|
+
return { type: 'text', text: content.text.to_s } if content.text?
|
|
89
|
+
|
|
90
|
+
if content.type == :image
|
|
91
|
+
url = content.source_type == :base64 ? "data:#{content.media_type};base64,#{content.data}" : content.data
|
|
92
|
+
return {
|
|
93
|
+
type: 'image_url',
|
|
94
|
+
image_url: { url: url, detail: content.detail }.compact
|
|
95
|
+
}.compact
|
|
96
|
+
end
|
|
89
97
|
|
|
90
98
|
{
|
|
91
99
|
type: content.type.to_s,
|
|
@@ -97,7 +105,8 @@ module Legion
|
|
|
97
105
|
id: content.id,
|
|
98
106
|
input: content.input,
|
|
99
107
|
tool_use_id: content.tool_use_id,
|
|
100
|
-
is_error: content.is_error
|
|
108
|
+
is_error: content.is_error,
|
|
109
|
+
cache_control: content.cache_control
|
|
101
110
|
}.compact
|
|
102
111
|
end
|
|
103
112
|
|
|
@@ -0,0 +1,70 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'legion/extensions/llm/capabilities'
|
|
4
|
+
require 'legion/extensions/llm/routing/records'
|
|
5
|
+
|
|
6
|
+
module Legion
|
|
7
|
+
module Extensions
|
|
8
|
+
module Llm
|
|
9
|
+
module Routing
|
|
10
|
+
# Request-owned routing facts. Construction does not consult settings,
|
|
11
|
+
# inventory, or provider state; consumers preserve this exact record.
|
|
12
|
+
RequestRequirements = ::Data.define(
|
|
13
|
+
:operation, :required_capabilities, :input_bound, :required_output_tokens, :context_budget,
|
|
14
|
+
:requested_embedding_dimensions, :modality, :provider_pin, :instance_pin, :model_pin, :tier_pin,
|
|
15
|
+
:body_model_hint_decision, :routing_seed, :maximum_attempts
|
|
16
|
+
) do
|
|
17
|
+
# rubocop:disable-next Metrics/ParameterLists -- every immutable contract field is required
|
|
18
|
+
def initialize(operation:, required_capabilities:, input_bound:, required_output_tokens:, context_budget:,
|
|
19
|
+
requested_embedding_dimensions:, modality:, provider_pin:, instance_pin:, model_pin:, tier_pin:,
|
|
20
|
+
body_model_hint_decision:, routing_seed:, maximum_attempts:)
|
|
21
|
+
support = Legion::Extensions::Llm::Inventory::RecordSupport
|
|
22
|
+
immutable = Legion::Extensions::Llm::Inventory::ImmutableValue
|
|
23
|
+
errors = Legion::Extensions::Llm::Inventory::Errors
|
|
24
|
+
raise errors::ValidationError, 'operation must be canonical' unless Legion::Extensions::Llm::Taxonomies::OPERATIONS.include?(operation)
|
|
25
|
+
unless required_capabilities.is_a?(::Array) && required_capabilities.all? { |capability| Legion::Extensions::Llm::Capabilities::CANONICAL.include?(capability) }
|
|
26
|
+
raise errors::ValidationError, 'required_capabilities must be an Array of canonical capabilities'
|
|
27
|
+
end
|
|
28
|
+
raise errors::ValidationError, 'tier_pin must be nil or a canonical tier' unless tier_pin.nil? || Legion::Extensions::Llm::Taxonomies::TIERS.include?(tier_pin)
|
|
29
|
+
raise errors::ValidationError, 'provider_pin must be nil or a Symbol' unless provider_pin.nil? || provider_pin.is_a?(::Symbol)
|
|
30
|
+
unless body_model_hint_decision.is_a?(Legion::Extensions::Llm::Routing::BodyModelHintDecision)
|
|
31
|
+
raise errors::ValidationError, 'body_model_hint_decision must be a BodyModelHintDecision'
|
|
32
|
+
end
|
|
33
|
+
unless routing_seed.is_a?(::String) && support.valid_utf8?(routing_seed) && routing_seed.match?(/\A[0-9a-f]{32}\z/)
|
|
34
|
+
raise errors::ValidationError, 'routing_seed must be 32 lowercase hexadecimal characters'
|
|
35
|
+
end
|
|
36
|
+
|
|
37
|
+
super(
|
|
38
|
+
operation: operation, required_capabilities: required_capabilities.dup.freeze,
|
|
39
|
+
input_bound: support.nonnegative_integer(value: input_bound, field: :input_bound),
|
|
40
|
+
required_output_tokens: support.nonnegative_integer(value: required_output_tokens, field: :required_output_tokens),
|
|
41
|
+
context_budget: support.nonnegative_integer(value: context_budget, field: :context_budget),
|
|
42
|
+
requested_embedding_dimensions: requested_embedding_dimensions.nil? ? nil : positive_integer(value: requested_embedding_dimensions, field: :requested_embedding_dimensions),
|
|
43
|
+
modality: immutable.copy_and_freeze(value: modality, field: :modality),
|
|
44
|
+
provider_pin: provider_pin, instance_pin: optional_pin_text(value: instance_pin, field: :instance_pin),
|
|
45
|
+
model_pin: optional_pin_text(value: model_pin, field: :model_pin), tier_pin: tier_pin,
|
|
46
|
+
body_model_hint_decision: body_model_hint_decision, routing_seed: routing_seed.dup.freeze,
|
|
47
|
+
maximum_attempts: positive_integer(value: maximum_attempts, field: :maximum_attempts)
|
|
48
|
+
)
|
|
49
|
+
end
|
|
50
|
+
|
|
51
|
+
private
|
|
52
|
+
|
|
53
|
+
def positive_integer(value:, field:)
|
|
54
|
+
Legion::Extensions::Llm::Inventory::RecordSupport.nonnegative_integer(value: value, field: field)
|
|
55
|
+
raise Legion::Extensions::Llm::Inventory::Errors::ValidationError, "#{field} must be positive" if value.zero?
|
|
56
|
+
|
|
57
|
+
value
|
|
58
|
+
end
|
|
59
|
+
|
|
60
|
+
def optional_pin_text(value:, field:)
|
|
61
|
+
return nil if value.nil?
|
|
62
|
+
raise Legion::Extensions::Llm::Inventory::Errors::ValidationError, "#{field} must be nil or a nonempty String" unless value.is_a?(::String) && !value.strip.empty?
|
|
63
|
+
|
|
64
|
+
Legion::Extensions::Llm::Inventory::ImmutableValue.copy_and_freeze(value: value, field: field)
|
|
65
|
+
end
|
|
66
|
+
end
|
|
67
|
+
end
|
|
68
|
+
end
|
|
69
|
+
end
|
|
70
|
+
end
|
|
@@ -55,13 +55,14 @@ module Legion
|
|
|
55
55
|
|
|
56
56
|
requested_options = DEFAULT_PUBLISH_OPTIONS.merge(@options).merge(options || {})
|
|
57
57
|
return_result = return_publish_result?(requested_options)
|
|
58
|
+
payload = encode_message
|
|
59
|
+
validate_payload_size(payload: payload)
|
|
58
60
|
publish_options = request_publish_options(requested_options)
|
|
59
|
-
validate_payload_size
|
|
60
61
|
exchange_dest = fleet_exchange
|
|
61
62
|
return_state = {}
|
|
62
63
|
install_return_listener(exchange_dest, requested_options, return_state)
|
|
63
64
|
prepare_publisher_confirms(exchange_dest, requested_options)
|
|
64
|
-
exchange_dest.publish(
|
|
65
|
+
exchange_dest.publish(payload, **publish_options)
|
|
65
66
|
return nil unless return_result
|
|
66
67
|
|
|
67
68
|
publish_result(exchange_dest, requested_options.merge(publish_options), return_state)
|
|
@@ -125,7 +126,7 @@ module Legion
|
|
|
125
126
|
{
|
|
126
127
|
routing_key: routing_key || '',
|
|
127
128
|
content_type: options[:content_type] || content_type,
|
|
128
|
-
content_encoding: options[:content_encoding] || content_encoding,
|
|
129
|
+
content_encoding: @options[:content_encoding] || content_encoding,
|
|
129
130
|
type: options[:type] || type,
|
|
130
131
|
priority: options[:priority] || priority,
|
|
131
132
|
expiration: options[:expiration] || expiration,
|
|
@@ -61,6 +61,7 @@ module Legion
|
|
|
61
61
|
require_relative 'llm/inventory/publisher'
|
|
62
62
|
require_relative 'llm/inventory/probe_coordinator'
|
|
63
63
|
require_relative 'llm/routing/records'
|
|
64
|
+
require_relative 'llm/routing/request_requirements'
|
|
64
65
|
require_relative 'llm/routing/provider_outcome'
|
|
65
66
|
|
|
66
67
|
# --- Base value objects (no internal deps) ---
|
|
@@ -64,12 +64,37 @@ RSpec.describe Legion::Extensions::Llm::Canonical::ContentBlock do
|
|
|
64
64
|
end
|
|
65
65
|
end
|
|
66
66
|
|
|
67
|
-
describe '
|
|
68
|
-
it '
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
67
|
+
describe 'source semantics at construction and rehydration' do
|
|
68
|
+
it 'normalizes source enums through every constructor' do
|
|
69
|
+
%i[new build from_hash].each do |constructor|
|
|
70
|
+
%w[base64 url].each do |source_type|
|
|
71
|
+
block = described_class.public_send(constructor, type: :image, source_type: source_type)
|
|
72
|
+
expect(block.source_type).to eq(source_type.to_sym)
|
|
73
|
+
end
|
|
74
|
+
end
|
|
75
|
+
end
|
|
76
|
+
|
|
77
|
+
it 'rejects missing and unknown image source types through every constructor' do
|
|
78
|
+
%i[new build from_hash].each do |constructor|
|
|
79
|
+
expect { described_class.public_send(constructor, type: :image) }.to raise_error(ArgumentError, /source_type/)
|
|
80
|
+
expect { described_class.public_send(constructor, type: :image, source_type: :file) }
|
|
81
|
+
.to raise_error(ArgumentError, /Invalid source_type/)
|
|
82
|
+
end
|
|
83
|
+
end
|
|
84
|
+
|
|
85
|
+
it 'preserves typed ordered content and image meaning across message JSON rehydration' do
|
|
86
|
+
content = [
|
|
87
|
+
described_class.text('compare', cache_control: { type: 'ephemeral' }),
|
|
88
|
+
described_class.image(data: 'AAA=', media_type: 'image/png', detail: 'high'),
|
|
89
|
+
described_class.image(data: 'https://example.org/image.png', media_type: nil, source_type: :url, detail: 'low')
|
|
90
|
+
]
|
|
91
|
+
message = Legion::Extensions::Llm::Canonical::Message.build(role: :user, content: content)
|
|
92
|
+
expect(message.content.first).to be(content.first)
|
|
93
|
+
|
|
94
|
+
restored = Legion::Extensions::Llm::Canonical::Message.from_hash(Legion::JSON.load(Legion::JSON.dump(message)))
|
|
95
|
+
|
|
96
|
+
expect(restored.content).to all(be_a(described_class))
|
|
97
|
+
expect(restored.content).to eq(content)
|
|
73
98
|
end
|
|
74
99
|
end
|
|
75
100
|
|
|
@@ -102,6 +102,26 @@ RSpec.describe Legion::Extensions::Llm::Canonical::Request do
|
|
|
102
102
|
end
|
|
103
103
|
end
|
|
104
104
|
|
|
105
|
+
describe 'system member contract — String or absent' do
|
|
106
|
+
it 'accepts a String system' do
|
|
107
|
+
expect(described_class.build(system: 'be brief').system).to eq('be brief')
|
|
108
|
+
end
|
|
109
|
+
|
|
110
|
+
it 'accepts an absent system' do
|
|
111
|
+
expect(described_class.build.system).to be_nil
|
|
112
|
+
end
|
|
113
|
+
|
|
114
|
+
it 'rejects a block-array system from every construction path' do
|
|
115
|
+
blocks = [{ type: 'text', text: 'Rule A' }]
|
|
116
|
+
expect { described_class.build(system: blocks) }
|
|
117
|
+
.to raise_error(ArgumentError, /system expected String, got Array/)
|
|
118
|
+
expect { described_class.new(system: blocks) }
|
|
119
|
+
.to raise_error(ArgumentError, /system expected String, got Array/)
|
|
120
|
+
expect { described_class.from_hash(system: blocks) }
|
|
121
|
+
.to raise_error(ArgumentError, /system expected String, got Array/)
|
|
122
|
+
end
|
|
123
|
+
end
|
|
124
|
+
|
|
105
125
|
it 'defaults stream false, routing {} and auto-generates req ids' do
|
|
106
126
|
request = described_class.build(messages: [])
|
|
107
127
|
expect(request.stream).to be(false)
|
|
@@ -143,6 +143,13 @@ RSpec.describe Legion::Extensions::Llm::Fleet::ExactOffering do
|
|
|
143
143
|
end
|
|
144
144
|
|
|
145
145
|
it 'rehydrates JSON-round-tripped messages before every message operation' do
|
|
146
|
+
content = [
|
|
147
|
+
Legion::Extensions::Llm::Canonical::ContentBlock.text('compare', cache_control: { type: 'ephemeral' }),
|
|
148
|
+
Legion::Extensions::Llm::Canonical::ContentBlock.image(data: 'AAA=', media_type: 'image/png', detail: 'high'),
|
|
149
|
+
Legion::Extensions::Llm::Canonical::ContentBlock.image(
|
|
150
|
+
data: 'https://example.org/image.png', media_type: nil, source_type: :url, detail: 'low'
|
|
151
|
+
)
|
|
152
|
+
]
|
|
146
153
|
raw_messages = [
|
|
147
154
|
{ role: 'system', content: 'follow the rules' },
|
|
148
155
|
{
|
|
@@ -150,7 +157,7 @@ RSpec.describe Legion::Extensions::Llm::Fleet::ExactOffering do
|
|
|
150
157
|
tool_calls: [{ id: 'call-1', name: 'lookup', arguments: { query: 'status' } }]
|
|
151
158
|
},
|
|
152
159
|
{ role: 'tool', tool_call_id: 'call-1', content: 'clean' },
|
|
153
|
-
|
|
160
|
+
Legion::Extensions::Llm::Canonical::Message.build(role: :user, content: content)
|
|
154
161
|
]
|
|
155
162
|
|
|
156
163
|
%w[chat stream_chat count_tokens].each do |operation|
|
|
@@ -166,6 +173,8 @@ RSpec.describe Legion::Extensions::Llm::Fleet::ExactOffering do
|
|
|
166
173
|
expect(call[:messages].map(&:role)).to eq(%i[system assistant tool user])
|
|
167
174
|
expect(call[:messages][1].tool_calls.first.name).to eq('lookup')
|
|
168
175
|
expect(call[:messages][2].tool_call_id).to eq('call-1')
|
|
176
|
+
expect(call[:messages][3].content).to all(be_a(Legion::Extensions::Llm::Canonical::ContentBlock))
|
|
177
|
+
expect(call[:messages][3].content).to eq(content)
|
|
169
178
|
end
|
|
170
179
|
end
|
|
171
180
|
end
|
|
@@ -15,6 +15,15 @@ RSpec.describe 'LLM fleet message envelopes' do
|
|
|
15
15
|
end
|
|
16
16
|
end
|
|
17
17
|
|
|
18
|
+
around do |example|
|
|
19
|
+
previous_crypt = Legion::Settings[:crypt].dup
|
|
20
|
+
Legion::Settings[:crypt][:cluster_secret] = 'a' * 64
|
|
21
|
+
Legion::Settings[:crypt][:cs_encrypt_ready] = true
|
|
22
|
+
example.run
|
|
23
|
+
ensure
|
|
24
|
+
Legion::Settings.loader.settings[:crypt] = previous_crypt
|
|
25
|
+
end
|
|
26
|
+
|
|
18
27
|
def valid_request_options
|
|
19
28
|
{
|
|
20
29
|
routing_key: 'llm.fleet.inference.qwen.ctx8192',
|
|
@@ -176,6 +185,11 @@ RSpec.describe 'LLM fleet message envelopes' do
|
|
|
176
185
|
type: protocol::REQUEST_TYPE
|
|
177
186
|
)
|
|
178
187
|
)
|
|
188
|
+
expect(exchange).to have_received(:publish) do |payload, **wire_options|
|
|
189
|
+
expect(wire_options[:content_encoding]).to eq('encrypted/cs')
|
|
190
|
+
decoded = Legion::JSON.load(Legion::Crypt.decrypt(payload, wire_options[:headers].fetch(:iv)))
|
|
191
|
+
expect(decoded).to include(model: 'qwen', params: { messages: [{ role: 'user', content: 'hello' }] })
|
|
192
|
+
end
|
|
179
193
|
expect(channel).to have_received(:on_return)
|
|
180
194
|
expect(channel).to have_received(:confirm_select)
|
|
181
195
|
expect(channel).to have_received(:wait_for_confirms)
|
|
@@ -200,6 +214,13 @@ RSpec.describe 'LLM fleet message envelopes' do
|
|
|
200
214
|
expect(channel).to have_received(:wait_for_confirms).with(no_args)
|
|
201
215
|
expect(result).to include(status: :accepted, accepted: true)
|
|
202
216
|
end
|
|
217
|
+
|
|
218
|
+
it 'rejects an encrypted request that exceeds the limit even when plaintext fits' do
|
|
219
|
+
message = described_class.new(**valid_request_options)
|
|
220
|
+
allow(described_class).to receive(:max_payload_bytes).and_return(Legion::JSON.dump(message.message).bytesize)
|
|
221
|
+
|
|
222
|
+
expect { message.publish }.to raise_error(Legion::Transport::PayloadTooLarge)
|
|
223
|
+
end
|
|
203
224
|
end
|
|
204
225
|
|
|
205
226
|
describe Legion::Extensions::Llm::Transport::Messages::FleetResponse do
|
|
@@ -241,14 +262,6 @@ RSpec.describe 'LLM fleet message envelopes' do
|
|
|
241
262
|
end.to raise_error(ArgumentError, /protocol_version must be 3/)
|
|
242
263
|
end
|
|
243
264
|
|
|
244
|
-
it 'is a pass-through for the response payload — the G5 thinking exclusion happens exactly once at the responder builder' do
|
|
245
|
-
message = described_class.new(**valid_response_options, response: { text: 'visible', stop_reason: :end_turn })
|
|
246
|
-
|
|
247
|
-
expect(message.message[:response]).to eq(text: 'visible', stop_reason: :end_turn)
|
|
248
|
-
expect(message.message).not_to have_key(:thinking)
|
|
249
|
-
expect(message.encode_message).not_to include('thinking')
|
|
250
|
-
end
|
|
251
|
-
|
|
252
265
|
it 'rejects legacy fleet envelope fields' do
|
|
253
266
|
%i[schema_version request_type fleet_correlation_id].each do |field|
|
|
254
267
|
expect do
|
|
@@ -274,6 +287,18 @@ RSpec.describe 'LLM fleet message envelopes' do
|
|
|
274
287
|
type: protocol::RESPONSE_TYPE
|
|
275
288
|
)
|
|
276
289
|
)
|
|
290
|
+
expect(default_exchange).to have_received(:publish) do |payload, **wire_options|
|
|
291
|
+
expect(wire_options[:content_encoding]).to eq('encrypted/cs')
|
|
292
|
+
decoded = Legion::JSON.load(Legion::Crypt.decrypt(payload, wire_options[:headers].fetch(:iv)))
|
|
293
|
+
expect(decoded[:response]).to include(text: 'hello', stop_reason: 'end_turn')
|
|
294
|
+
end
|
|
295
|
+
end
|
|
296
|
+
|
|
297
|
+
it 'rejects an encrypted reply that exceeds the limit even when plaintext fits' do
|
|
298
|
+
message = described_class.new(**valid_response_options)
|
|
299
|
+
allow(described_class).to receive(:max_payload_bytes).and_return(Legion::JSON.dump(message.message).bytesize)
|
|
300
|
+
|
|
301
|
+
expect { message.publish }.to raise_error(Legion::Transport::PayloadTooLarge)
|
|
277
302
|
end
|
|
278
303
|
|
|
279
304
|
it 'accepts standard transport publish options and returns accepted reply results' do
|
|
@@ -72,29 +72,41 @@ RSpec.describe Legion::Extensions::Llm::Provider::OpenAICompatible do
|
|
|
72
72
|
end
|
|
73
73
|
end
|
|
74
74
|
|
|
75
|
-
it '
|
|
75
|
+
it 'sends ordered typed text and image_url parts through the callable HTTP boundary' do
|
|
76
76
|
message = canonical::Message.build(
|
|
77
77
|
role: :user,
|
|
78
|
-
content: [
|
|
79
|
-
|
|
78
|
+
content: [
|
|
79
|
+
canonical::ContentBlock.text('compare these', cache_control: { type: 'ephemeral' }),
|
|
80
|
+
canonical::ContentBlock.image(data: 'AAA=', media_type: 'image/png', detail: 'high'),
|
|
81
|
+
canonical::ContentBlock.text('with this'),
|
|
82
|
+
canonical::ContentBlock.build(type: :image, source_type: :url, data: 'https://example.org/image.png',
|
|
83
|
+
detail: 'low', cache_control: { type: 'ephemeral' })
|
|
84
|
+
]
|
|
80
85
|
)
|
|
86
|
+
stubs = Faraday::Adapter::Test::Stubs.new do |stub|
|
|
87
|
+
stub.post('/v1/chat/completions') do |env|
|
|
88
|
+
payload = Legion::JSON.load(env.body)
|
|
89
|
+
expect(payload[:messages]).to eq(
|
|
90
|
+
[{
|
|
91
|
+
role: 'user',
|
|
92
|
+
content: [
|
|
93
|
+
{ type: 'text', text: 'compare these' },
|
|
94
|
+
{ type: 'image_url', image_url: { url: 'data:image/png;base64,AAA=', detail: 'high' } },
|
|
95
|
+
{ type: 'text', text: 'with this' },
|
|
96
|
+
{ type: 'image_url', image_url: { url: 'https://example.org/image.png', detail: 'low' } }
|
|
97
|
+
]
|
|
98
|
+
}]
|
|
99
|
+
)
|
|
100
|
+
[200, { 'content-type' => 'application/json' }, Legion::JSON.dump(completion_body)]
|
|
101
|
+
end
|
|
102
|
+
end
|
|
103
|
+
provider.connection.connection.builder.adapter :test, stubs
|
|
81
104
|
|
|
82
|
-
|
|
83
|
-
:render_payload,
|
|
84
|
-
[message],
|
|
85
|
-
tools: {},
|
|
86
|
-
model: model_id,
|
|
87
|
-
stream: false,
|
|
88
|
-
schema: nil,
|
|
89
|
-
thinking: nil,
|
|
90
|
-
tool_prefs: nil,
|
|
91
|
-
params: nil
|
|
92
|
-
)
|
|
105
|
+
response = provider.chat([message], model: model_id)
|
|
93
106
|
|
|
94
|
-
|
|
95
|
-
expect(
|
|
96
|
-
|
|
97
|
-
)
|
|
107
|
+
expect(response).to be_a(canonical::Response)
|
|
108
|
+
expect(response.text).to eq('hi')
|
|
109
|
+
stubs.verify_stubbed_calls
|
|
98
110
|
end
|
|
99
111
|
|
|
100
112
|
it 'parses chat completion responses to a Canonical::Response with usage and tool calls (08 R2)' do
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'spec_helper'
|
|
4
|
+
require 'legion/extensions/llm/routing/request_requirements'
|
|
5
|
+
|
|
6
|
+
RSpec.describe Legion::Extensions::Llm::Routing::RequestRequirements do
|
|
7
|
+
let(:validation_error) { Legion::Extensions::Llm::Inventory::Errors::ValidationError }
|
|
8
|
+
let(:attributes) do
|
|
9
|
+
{
|
|
10
|
+
operation: :chat, required_capabilities: %i[completion vision], input_bound: 0,
|
|
11
|
+
required_output_tokens: 0, context_budget: 0, requested_embedding_dimensions: nil,
|
|
12
|
+
modality: nil, provider_pin: nil, instance_pin: nil, model_pin: nil, tier_pin: nil,
|
|
13
|
+
body_model_hint_decision: Legion::Extensions::Llm::Routing::BodyModelHintDecision.new(
|
|
14
|
+
requested_model: nil, disposition: :absent, settings_generation: 0
|
|
15
|
+
),
|
|
16
|
+
routing_seed: '0123456789abcdef0123456789abcdef', maximum_attempts: 1
|
|
17
|
+
}
|
|
18
|
+
end
|
|
19
|
+
|
|
20
|
+
it 'isolates routing facts from later mutations of request-owned values' do
|
|
21
|
+
capabilities = %i[completion vision]
|
|
22
|
+
modality = { output: [+ 'image', { format: + 'png' }] }
|
|
23
|
+
model = + 'model-a'
|
|
24
|
+
instance = + 'primary'
|
|
25
|
+
seed = attributes[:routing_seed].dup
|
|
26
|
+
requirements = described_class.new(**attributes, required_capabilities: capabilities, modality: modality,
|
|
27
|
+
model_pin: model, instance_pin: instance, routing_seed: seed)
|
|
28
|
+
|
|
29
|
+
capabilities.clear
|
|
30
|
+
modality[:output].first.replace('audio')
|
|
31
|
+
modality[:output].last[:format].replace('wav')
|
|
32
|
+
model.replace('other-model')
|
|
33
|
+
instance.replace('other-instance')
|
|
34
|
+
seed.replace('f' * 32)
|
|
35
|
+
|
|
36
|
+
expect(requirements.required_capabilities).to eq(%i[completion vision])
|
|
37
|
+
expect(requirements.modality).to eq(output: ['image', { format: 'png' }])
|
|
38
|
+
expect(requirements.model_pin).to eq('model-a')
|
|
39
|
+
expect(requirements.instance_pin).to eq('primary')
|
|
40
|
+
expect(requirements.routing_seed).to eq(attributes[:routing_seed])
|
|
41
|
+
expect { requirements.modality[:output].last[:format].replace('jpeg') }.to raise_error(FrozenError)
|
|
42
|
+
expect { requirements.required_capabilities.clear }.to raise_error(FrozenError)
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
it 'rejects untranslated vocabulary and incorrectly typed explicit pins' do
|
|
46
|
+
{
|
|
47
|
+
operation: :completion, required_capabilities: [:function_calling], tier_pin: 'local',
|
|
48
|
+
provider_pin: 'openai', instance_pin: :primary, model_pin: :model
|
|
49
|
+
}.each do |field, value|
|
|
50
|
+
expect { described_class.new(**attributes, field => value) }.to raise_error(validation_error, /#{field}/)
|
|
51
|
+
end
|
|
52
|
+
end
|
|
53
|
+
|
|
54
|
+
it 'rejects invalid budgets, dimensions, and attempt limits rather than coercing them' do
|
|
55
|
+
{
|
|
56
|
+
input_bound: -1, required_output_tokens: 1.5, context_budget: nil,
|
|
57
|
+
requested_embedding_dimensions: 0, maximum_attempts: 0
|
|
58
|
+
}.each do |field, value|
|
|
59
|
+
expect { described_class.new(**attributes, field => value) }.to raise_error(validation_error, /#{field}/)
|
|
60
|
+
end
|
|
61
|
+
end
|
|
62
|
+
|
|
63
|
+
it 'requires request ownership of the routing seed and model hint decision' do
|
|
64
|
+
expect { described_class.new(**attributes.except(:routing_seed)) }.to raise_error(ArgumentError, /routing_seed/)
|
|
65
|
+
expect { described_class.new(**attributes, routing_seed: 'A' * 32) }.to raise_error(validation_error, /routing_seed/)
|
|
66
|
+
expect { described_class.new(**attributes, body_model_hint_decision: attributes[:body_model_hint_decision].to_h) }
|
|
67
|
+
.to raise_error(validation_error, /body_model_hint_decision/)
|
|
68
|
+
end
|
|
69
|
+
|
|
70
|
+
it 'rejects unsupported mutable values embedded in modality' do
|
|
71
|
+
expect { described_class.new(**attributes, modality: { image: Object.new }) }
|
|
72
|
+
.to raise_error(validation_error, /modality.image/)
|
|
73
|
+
end
|
|
74
|
+
end
|
|
@@ -83,8 +83,8 @@ module SpecSupport
|
|
|
83
83
|
@requests << request
|
|
84
84
|
body = '{"status":"ok"}'
|
|
85
85
|
socket.write("HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nContent-Length: #{body.bytesize}\r\nConnection: close\r\n\r\n#{body}")
|
|
86
|
-
rescue OpenSSL::SSL::SSLError => e
|
|
87
|
-
Legion::Logging.debug("TLS fixture
|
|
86
|
+
rescue OpenSSL::SSL::SSLError, Errno::ECONNRESET => e
|
|
87
|
+
Legion::Logging.debug("TLS fixture peer rejected the connection: #{e.class}")
|
|
88
88
|
ensure
|
|
89
89
|
socket&.close
|
|
90
90
|
end
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: lex-llm
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.8.
|
|
4
|
+
version: 0.8.8
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- LegionIO
|
|
@@ -198,14 +198,14 @@ dependencies:
|
|
|
198
198
|
requirements:
|
|
199
199
|
- - ">="
|
|
200
200
|
- !ruby/object:Gem::Version
|
|
201
|
-
version: 1.4.
|
|
201
|
+
version: 1.4.31
|
|
202
202
|
type: :runtime
|
|
203
203
|
prerelease: false
|
|
204
204
|
version_requirements: !ruby/object:Gem::Requirement
|
|
205
205
|
requirements:
|
|
206
206
|
- - ">="
|
|
207
207
|
- !ruby/object:Gem::Version
|
|
208
|
-
version: 1.4.
|
|
208
|
+
version: 1.4.31
|
|
209
209
|
- !ruby/object:Gem::Dependency
|
|
210
210
|
name: marcel
|
|
211
211
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -314,6 +314,7 @@ files:
|
|
|
314
314
|
- lib/legion/extensions/llm/routing.rb
|
|
315
315
|
- lib/legion/extensions/llm/routing/provider_outcome.rb
|
|
316
316
|
- lib/legion/extensions/llm/routing/records.rb
|
|
317
|
+
- lib/legion/extensions/llm/routing/request_requirements.rb
|
|
317
318
|
- lib/legion/extensions/llm/settings_cascade.rb
|
|
318
319
|
- lib/legion/extensions/llm/stop_reason_mapping.rb
|
|
319
320
|
- lib/legion/extensions/llm/stream_accumulator.rb
|
|
@@ -422,6 +423,7 @@ files:
|
|
|
422
423
|
- spec/legion/extensions/llm/responses/thinking_extractor_spec.rb
|
|
423
424
|
- spec/legion/extensions/llm/routing/provider_outcome_spec.rb
|
|
424
425
|
- spec/legion/extensions/llm/routing/records_spec.rb
|
|
426
|
+
- spec/legion/extensions/llm/routing/request_requirements_spec.rb
|
|
425
427
|
- spec/legion/extensions/llm/settings_cascade_spec.rb
|
|
426
428
|
- spec/legion/extensions/llm/stop_reason_mapping_spec.rb
|
|
427
429
|
- spec/legion/extensions/llm/stream_accumulator_spec.rb
|