lex-llm 0.8.4 → 0.8.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +33 -0
- data/README.md +59 -2
- data/lex-llm.gemspec +1 -1
- data/lib/legion/extensions/llm/canonical/content_block.rb +9 -0
- data/lib/legion/extensions/llm/canonical/request.rb +5 -0
- data/lib/legion/extensions/llm/connection.rb +4 -2
- data/lib/legion/extensions/llm/credential_sources.rb +1 -5
- data/lib/legion/extensions/llm/fleet/default_exchange_reply.rb +4 -3
- data/lib/legion/extensions/llm/provider/open_ai_compatible.rb +11 -2
- data/lib/legion/extensions/llm/routing/request_requirements.rb +70 -0
- data/lib/legion/extensions/llm/transport/messages/fleet_request.rb +4 -3
- data/lib/legion/extensions/llm/version.rb +1 -1
- data/lib/legion/extensions/llm.rb +4 -0
- data/spec/legion/extensions/llm/canonical/content_block_spec.rb +31 -6
- data/spec/legion/extensions/llm/canonical/request_spec.rb +20 -0
- data/spec/legion/extensions/llm/connection_tls_spec.rb +103 -0
- data/spec/legion/extensions/llm/credential_sources_spec.rb +16 -7
- data/spec/legion/extensions/llm/fleet/exact_offering_spec.rb +10 -1
- data/spec/legion/extensions/llm/fleet_messages_spec.rb +33 -8
- data/spec/legion/extensions/llm/provider/open_ai_compatible_spec.rb +30 -18
- data/spec/legion/extensions/llm/routing/request_requirements_spec.rb +74 -0
- data/spec/support/tls_endpoint.rb +92 -0
- metadata +7 -3
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: f21576528494d0876f279ea9a60a61b0ae66b00d6f2cf42edf023fea4e937b55
|
|
4
|
+
data.tar.gz: b5b4d3a0b3ac21cf23f94e511c059fab5d5f0bf925a137389ee1cf5254b252f5
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 647f69802e60ee7be7c8d428999f337a24fc2d66f65b315631046cbebf44ffcaf36416076857f0efd3127a7728eb84852a143a603b5fb2e28aa7c55257110c20
|
|
7
|
+
data.tar.gz: 575f2293e9a1d8a97135c950d0937b24b42c2e6c14d8c4cb49d69d8001f656e5e8d9d3aecc2b50914cfe6b089b140913bb3382288233446473f98d96e621cf41
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,38 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.8.8 - 2026-11-18
|
|
4
|
+
|
|
5
|
+
### Fixed
|
|
6
|
+
- Handle a peer connection reset in the local TLS test endpoint when a client rejects an untrusted certificate. Linux OpenSSL/curl may reset the socket after the server accepts it; the certificate-rejection and no-authenticated-request assertions remain unchanged.
|
|
7
|
+
|
|
8
|
+
## 0.8.7 - 2026-11-17
|
|
9
|
+
|
|
10
|
+
### Fixed
|
|
11
|
+
- Enforce the canonical system member as a String or absent across direct construction, builders, and JSON rehydration. Client block arrays must be translated at ingress instead of leaking into provider dispatch.
|
|
12
|
+
|
|
13
|
+
## 0.8.6 - 2026-10-31
|
|
14
|
+
|
|
15
|
+
### Fixed
|
|
16
|
+
- Normalize canonical image source types at construction and JSON rehydration so fleet execution preserves base64 and URL image meaning.
|
|
17
|
+
- Render mixed text and images as OpenAI-compatible text/image_url parts through the real provider callable, preserving media bytes, order, and image detail.
|
|
18
|
+
- Decode HTTP JSON responses through `Legion::JSON` at the shared connection boundary; Faraday's positional parser options no longer break JSON 3 responses.
|
|
19
|
+
- Validate the exact encrypted fleet request/reply bytes once before publication using `legion-transport >= 1.4.31`, preserving their matching encryption headers.
|
|
20
|
+
|
|
21
|
+
### Added
|
|
22
|
+
- Immutable shared `Routing::RequestRequirements` for request-owned routing facts; it validates contracts without consulting Inventory or choosing a provider.
|
|
23
|
+
|
|
24
|
+
## 0.8.5 - 2026-10-29
|
|
25
|
+
|
|
26
|
+
### Changed
|
|
27
|
+
- Make shared provider TLS certificate and hostname verification configurable through
|
|
28
|
+
`Legion::Settings` at `extensions.llm.ssl.verify`, defaulting to `false`. Set it to
|
|
29
|
+
`true` to enable verification with Typhoeus or Net::HTTP; explicitly trusted custom
|
|
30
|
+
certificate authorities remain supported. (#50)
|
|
31
|
+
|
|
32
|
+
### Fixed
|
|
33
|
+
- Correct asynchronous TCP probes that could report refused connections as open
|
|
34
|
+
on Darwin; inspect the completed socket's error status instead of reconnecting.
|
|
35
|
+
|
|
3
36
|
## 0.8.4 - 2026-08-25
|
|
4
37
|
|
|
5
38
|
### Changed
|
data/README.md
CHANGED
|
@@ -41,11 +41,34 @@ gem 'lex-llm'
|
|
|
41
41
|
Provider extensions should declare `lex-llm` as a gemspec dependency:
|
|
42
42
|
|
|
43
43
|
```ruby
|
|
44
|
-
spec.add_dependency 'lex-llm', '>= 0.
|
|
44
|
+
spec.add_dependency 'lex-llm', '>= 0.8.6'
|
|
45
45
|
```
|
|
46
46
|
|
|
47
47
|
For local development across LegionIO repos, prefer a local path override in the app or test `Gemfile`, not a permanent git dependency in the gemspec.
|
|
48
48
|
|
|
49
|
+
## Canonical execution contracts
|
|
50
|
+
|
|
51
|
+
`Canonical::ContentBlock` owns media source normalization: supported image sources
|
|
52
|
+
are `:base64` and `:url`. String enum values from JSON are normalized by the strict
|
|
53
|
+
constructor, including `from_hash`; an image without an explicit source type is a
|
|
54
|
+
contract error. `ContentBlock.image` explicitly defaults to `:base64`.
|
|
55
|
+
Fleet workers rehydrate serialized messages through these same constructors before
|
|
56
|
+
calling a provider, preserving media bytes, MIME type, ordering and image detail.
|
|
57
|
+
|
|
58
|
+
Fleet publishers require `legion-transport >= 1.4.31`: they encode once, validate
|
|
59
|
+
the final encrypted byte count, then publish those exact bytes with the matching
|
|
60
|
+
encryption headers. HTTP response decoding uses `Legion::JSON` at the shared
|
|
61
|
+
connection boundary and supports Faraday's positional and keyword parser APIs.
|
|
62
|
+
|
|
63
|
+
`Provider::OpenAICompatible` translates canonical text/image blocks into OpenAI
|
|
64
|
+
`text`/`image_url` parts, never raw canonical media fields. Provider-specific cache
|
|
65
|
+
controls remain canonical metadata unless the selected provider supports them.
|
|
66
|
+
|
|
67
|
+
`Routing::RequestRequirements` is the shared immutable record for operation,
|
|
68
|
+
capabilities, context/input/output bounds, modality, pins, hint disposition, routing
|
|
69
|
+
seed and attempt limit. It validates supplied facts without reading settings or
|
|
70
|
+
Inventory. `legion-llm` constructs it before its sole Router chooses a Selection.
|
|
71
|
+
|
|
49
72
|
## Namespace
|
|
50
73
|
|
|
51
74
|
Load the extension through the Legion namespace:
|
|
@@ -178,7 +201,7 @@ end
|
|
|
178
201
|
| `Error` | `lib/.../error.rb` | Base error class for lex-llm. |
|
|
179
202
|
| `Errors::UnsupportedCapability` | `lib/.../errors/unsupported_capability.rb` | Raised when a provider lacks a requested capability. |
|
|
180
203
|
| `Utils` | `lib/.../utils.rb` | Shared utility methods. |
|
|
181
|
-
| `VERSION` | `lib/.../version.rb` | Current gem version (`0.
|
|
204
|
+
| `VERSION` | `lib/.../version.rb` | Current gem version (`0.8.8`). |
|
|
182
205
|
|
|
183
206
|
---
|
|
184
207
|
|
|
@@ -489,10 +512,44 @@ Key settings paths:
|
|
|
489
512
|
- `extensions.llm.fleet.endpoint` -- endpoint-style worker configuration
|
|
490
513
|
- `extensions.llm.fleet.compliance.encrypt_fleet` -- encrypt fleet envelopes (default true)
|
|
491
514
|
- `extensions.llm.fleet.auth.verify_issuer` -- validate JWT issuer (default true)
|
|
515
|
+
- `extensions.llm.ssl.verify` -- verify provider HTTPS certificates and hostnames (default false)
|
|
492
516
|
- `extensions.llm.security.credential_source_probing` -- gate credential probing (default true)
|
|
493
517
|
- `extensions.llm.model_whitelist` / `model_blacklist` -- provider-level model filters
|
|
494
518
|
- `extensions.llm.<family>.instance.<name>.model_whitelist` -- per-instance override
|
|
495
519
|
|
|
520
|
+
### TLS verification and custom certificate authorities
|
|
521
|
+
|
|
522
|
+
Shared provider connections read `extensions.llm.ssl.verify` from `Legion::Settings`
|
|
523
|
+
when constructed. The default is `false` for both Typhoeus and Net::HTTP.
|
|
524
|
+
With verification disabled, untrusted and wrong-host certificates are accepted,
|
|
525
|
+
so credentials can be sent to an unauthenticated peer.
|
|
526
|
+
|
|
527
|
+
Enable verification before creating provider connections:
|
|
528
|
+
|
|
529
|
+
```json
|
|
530
|
+
{
|
|
531
|
+
"extensions": {
|
|
532
|
+
"llm": {
|
|
533
|
+
"ssl": { "verify": true }
|
|
534
|
+
}
|
|
535
|
+
}
|
|
536
|
+
}
|
|
537
|
+
```
|
|
538
|
+
|
|
539
|
+
With verification enabled, an untrusted certificate or hostname mismatch fails
|
|
540
|
+
before an authenticated HTTP request is sent.
|
|
541
|
+
|
|
542
|
+
When constructing a shared connection for a private or corporate CA, configure
|
|
543
|
+
the underlying Faraday SSL options before making the first request:
|
|
544
|
+
|
|
545
|
+
```ruby
|
|
546
|
+
connection = Legion::Extensions::Llm::Connection.new(provider, config)
|
|
547
|
+
connection.connection.ssl.ca_file = '/path/to/trusted-ca-bundle.pem'
|
|
548
|
+
```
|
|
549
|
+
|
|
550
|
+
When verification is enabled, private or self-signed HTTPS servers need explicit
|
|
551
|
+
CA trust. Plain HTTP behavior is unchanged.
|
|
552
|
+
|
|
496
553
|
---
|
|
497
554
|
|
|
498
555
|
## Provider Dependencies
|
data/lex-llm.gemspec
CHANGED
|
@@ -41,7 +41,7 @@ Gem::Specification.new do |spec|
|
|
|
41
41
|
spec.add_dependency 'legion-json', '>= 1.2.1'
|
|
42
42
|
spec.add_dependency 'legion-logging', '>= 1.3.2'
|
|
43
43
|
spec.add_dependency 'legion-settings', '>= 1.4.2'
|
|
44
|
-
spec.add_dependency 'legion-transport', '>= 1.4.
|
|
44
|
+
spec.add_dependency 'legion-transport', '>= 1.4.31'
|
|
45
45
|
spec.add_dependency 'marcel', '~> 1'
|
|
46
46
|
spec.add_dependency 'ruby_llm-schema', '~> 0'
|
|
47
47
|
end
|
|
@@ -59,6 +59,13 @@ module Legion
|
|
|
59
59
|
Strict.enum!(type_sym, self::CONTENT_BLOCK_TYPES, site, :type)
|
|
60
60
|
end
|
|
61
61
|
|
|
62
|
+
def self.normalize_source_type!(source_type, type, site)
|
|
63
|
+
raise ArgumentError, "#{site}: image source_type is required" if type == :image && source_type.nil?
|
|
64
|
+
|
|
65
|
+
source_type = source_type.to_sym if source_type.is_a?(::String)
|
|
66
|
+
Strict.enum!(source_type, self::SOURCE_TYPES, site, :source_type)
|
|
67
|
+
end
|
|
68
|
+
|
|
62
69
|
# Build a text content block.
|
|
63
70
|
def self.text(content, cache_control: nil)
|
|
64
71
|
build(type: :text, text: content, cache_control:)
|
|
@@ -136,12 +143,14 @@ module Legion
|
|
|
136
143
|
# delegate here.
|
|
137
144
|
Strict.install_strict_new!(self) do |values, site|
|
|
138
145
|
values[:type] = normalize_type!(values[:type], site)
|
|
146
|
+
values[:source_type] = normalize_source_type!(values[:source_type], values[:type], site)
|
|
139
147
|
values[:metadata] = Strict.metadata!(values[:metadata], site)
|
|
140
148
|
values
|
|
141
149
|
end
|
|
142
150
|
end
|
|
143
151
|
|
|
144
152
|
ContentBlock::CONTENT_BLOCK_TYPES = %i[text thinking tool_use tool_result image audio video].freeze
|
|
153
|
+
ContentBlock::SOURCE_TYPES = %i[base64 url].freeze
|
|
145
154
|
ContentBlock::BUILD_SITE = 'Canonical::ContentBlock.build'
|
|
146
155
|
ContentBlock::FROM_HASH_SITE = 'Canonical::ContentBlock.from_hash'
|
|
147
156
|
ContentBlock::NEW_SITE = 'Canonical::ContentBlock.new'
|
|
@@ -120,6 +120,11 @@ module Legion
|
|
|
120
120
|
# delegate here.
|
|
121
121
|
Strict.install_strict_new!(self) do |values, site|
|
|
122
122
|
values[:messages] = normalize_messages!(values[:messages], site)
|
|
123
|
+
# The system string contract: a String (or absent) is the only
|
|
124
|
+
# canonical shape. Client dialects converge multi-part system
|
|
125
|
+
# wire at their own edge; a block array here is a client-edge
|
|
126
|
+
# defect leaking past the boundary — fail loud, never stringify.
|
|
127
|
+
values[:system] = Strict.expect_type!(values[:system], [::String], site, :system)
|
|
123
128
|
values[:tools] = normalize_tools(values[:tools], site)
|
|
124
129
|
values[:tool_choice] = values[:tool_choice].is_a?(::String) ? values[:tool_choice].to_sym : values[:tool_choice]
|
|
125
130
|
values[:params] = normalize_params!(values[:params], site)
|
|
@@ -40,7 +40,7 @@ module Legion
|
|
|
40
40
|
@config = config
|
|
41
41
|
|
|
42
42
|
@connection ||= Faraday.new(provider.api_base) do |faraday|
|
|
43
|
-
faraday.ssl.verify =
|
|
43
|
+
faraday.ssl.verify = Legion::Settings.dig(:extensions, :llm, :ssl, :verify) == true
|
|
44
44
|
setup_timeout(faraday)
|
|
45
45
|
setup_logging(faraday)
|
|
46
46
|
setup_retry(faraday)
|
|
@@ -125,7 +125,9 @@ module Legion
|
|
|
125
125
|
def setup_middleware(faraday)
|
|
126
126
|
faraday.request :multipart
|
|
127
127
|
faraday.request :json
|
|
128
|
-
faraday.response :json
|
|
128
|
+
faraday.response :json, parser_options: {
|
|
129
|
+
decoder: [->(body, *) { Legion::JSON.parse(body, symbolize_names: false) }, :call]
|
|
130
|
+
}
|
|
129
131
|
# Prefer typhoeus (libcurl) over net_http to avoid Ruby 4.0 + net-http-0.9.1 SSL issues
|
|
130
132
|
begin
|
|
131
133
|
require 'faraday/typhoeus'
|
|
@@ -114,11 +114,7 @@ module Legion
|
|
|
114
114
|
rescue IO::WaitWritable
|
|
115
115
|
return false unless sock.wait_writable(timeout)
|
|
116
116
|
|
|
117
|
-
|
|
118
|
-
sock.connect_nonblock(addr)
|
|
119
|
-
rescue Errno::EISCONN
|
|
120
|
-
# already connected — success
|
|
121
|
-
end
|
|
117
|
+
return sock.getsockopt(Socket::SOL_SOCKET, Socket::SO_ERROR).int.zero?
|
|
122
118
|
end
|
|
123
119
|
true
|
|
124
120
|
rescue StandardError => e
|
|
@@ -22,13 +22,14 @@ module Legion
|
|
|
22
22
|
|
|
23
23
|
requested_options = DEFAULT_REPLY_PUBLISH_OPTIONS.merge(@options).merge(options || {})
|
|
24
24
|
return_result = return_publish_result?(requested_options)
|
|
25
|
+
payload = encode_message
|
|
26
|
+
validate_payload_size(payload: payload)
|
|
25
27
|
publish_options = reply_publish_options(requested_options)
|
|
26
|
-
validate_payload_size
|
|
27
28
|
default_exchange = channel.default_exchange
|
|
28
29
|
return_state = {}
|
|
29
30
|
install_return_listener(default_exchange, requested_options, return_state)
|
|
30
31
|
prepare_publisher_confirms(default_exchange, requested_options)
|
|
31
|
-
default_exchange.publish(
|
|
32
|
+
default_exchange.publish(payload, **publish_options)
|
|
32
33
|
return nil unless return_result
|
|
33
34
|
|
|
34
35
|
publish_result(default_exchange, requested_options.merge(publish_options), return_state)
|
|
@@ -56,7 +57,7 @@ module Legion
|
|
|
56
57
|
{
|
|
57
58
|
routing_key: routing_key,
|
|
58
59
|
content_type: options[:content_type] || content_type,
|
|
59
|
-
content_encoding: options[:content_encoding] || content_encoding,
|
|
60
|
+
content_encoding: @options[:content_encoding] || content_encoding,
|
|
60
61
|
type: options[:type] || type,
|
|
61
62
|
priority: options[:priority] || priority,
|
|
62
63
|
expiration: options[:expiration] || expiration,
|
|
@@ -85,7 +85,15 @@ module Legion
|
|
|
85
85
|
return content.map { |block| openai_content(block) } if content.is_a?(::Array)
|
|
86
86
|
return content.to_s if content.nil? || content.is_a?(::String)
|
|
87
87
|
|
|
88
|
-
return content.text.to_s if content.text?
|
|
88
|
+
return { type: 'text', text: content.text.to_s } if content.text?
|
|
89
|
+
|
|
90
|
+
if content.type == :image
|
|
91
|
+
url = content.source_type == :base64 ? "data:#{content.media_type};base64,#{content.data}" : content.data
|
|
92
|
+
return {
|
|
93
|
+
type: 'image_url',
|
|
94
|
+
image_url: { url: url, detail: content.detail }.compact
|
|
95
|
+
}.compact
|
|
96
|
+
end
|
|
89
97
|
|
|
90
98
|
{
|
|
91
99
|
type: content.type.to_s,
|
|
@@ -97,7 +105,8 @@ module Legion
|
|
|
97
105
|
id: content.id,
|
|
98
106
|
input: content.input,
|
|
99
107
|
tool_use_id: content.tool_use_id,
|
|
100
|
-
is_error: content.is_error
|
|
108
|
+
is_error: content.is_error,
|
|
109
|
+
cache_control: content.cache_control
|
|
101
110
|
}.compact
|
|
102
111
|
end
|
|
103
112
|
|
|
@@ -0,0 +1,70 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'legion/extensions/llm/capabilities'
|
|
4
|
+
require 'legion/extensions/llm/routing/records'
|
|
5
|
+
|
|
6
|
+
module Legion
|
|
7
|
+
module Extensions
|
|
8
|
+
module Llm
|
|
9
|
+
module Routing
|
|
10
|
+
# Request-owned routing facts. Construction does not consult settings,
|
|
11
|
+
# inventory, or provider state; consumers preserve this exact record.
|
|
12
|
+
RequestRequirements = ::Data.define(
|
|
13
|
+
:operation, :required_capabilities, :input_bound, :required_output_tokens, :context_budget,
|
|
14
|
+
:requested_embedding_dimensions, :modality, :provider_pin, :instance_pin, :model_pin, :tier_pin,
|
|
15
|
+
:body_model_hint_decision, :routing_seed, :maximum_attempts
|
|
16
|
+
) do
|
|
17
|
+
# rubocop:disable-next Metrics/ParameterLists -- every immutable contract field is required
|
|
18
|
+
def initialize(operation:, required_capabilities:, input_bound:, required_output_tokens:, context_budget:,
|
|
19
|
+
requested_embedding_dimensions:, modality:, provider_pin:, instance_pin:, model_pin:, tier_pin:,
|
|
20
|
+
body_model_hint_decision:, routing_seed:, maximum_attempts:)
|
|
21
|
+
support = Legion::Extensions::Llm::Inventory::RecordSupport
|
|
22
|
+
immutable = Legion::Extensions::Llm::Inventory::ImmutableValue
|
|
23
|
+
errors = Legion::Extensions::Llm::Inventory::Errors
|
|
24
|
+
raise errors::ValidationError, 'operation must be canonical' unless Legion::Extensions::Llm::Taxonomies::OPERATIONS.include?(operation)
|
|
25
|
+
unless required_capabilities.is_a?(::Array) && required_capabilities.all? { |capability| Legion::Extensions::Llm::Capabilities::CANONICAL.include?(capability) }
|
|
26
|
+
raise errors::ValidationError, 'required_capabilities must be an Array of canonical capabilities'
|
|
27
|
+
end
|
|
28
|
+
raise errors::ValidationError, 'tier_pin must be nil or a canonical tier' unless tier_pin.nil? || Legion::Extensions::Llm::Taxonomies::TIERS.include?(tier_pin)
|
|
29
|
+
raise errors::ValidationError, 'provider_pin must be nil or a Symbol' unless provider_pin.nil? || provider_pin.is_a?(::Symbol)
|
|
30
|
+
unless body_model_hint_decision.is_a?(Legion::Extensions::Llm::Routing::BodyModelHintDecision)
|
|
31
|
+
raise errors::ValidationError, 'body_model_hint_decision must be a BodyModelHintDecision'
|
|
32
|
+
end
|
|
33
|
+
unless routing_seed.is_a?(::String) && support.valid_utf8?(routing_seed) && routing_seed.match?(/\A[0-9a-f]{32}\z/)
|
|
34
|
+
raise errors::ValidationError, 'routing_seed must be 32 lowercase hexadecimal characters'
|
|
35
|
+
end
|
|
36
|
+
|
|
37
|
+
super(
|
|
38
|
+
operation: operation, required_capabilities: required_capabilities.dup.freeze,
|
|
39
|
+
input_bound: support.nonnegative_integer(value: input_bound, field: :input_bound),
|
|
40
|
+
required_output_tokens: support.nonnegative_integer(value: required_output_tokens, field: :required_output_tokens),
|
|
41
|
+
context_budget: support.nonnegative_integer(value: context_budget, field: :context_budget),
|
|
42
|
+
requested_embedding_dimensions: requested_embedding_dimensions.nil? ? nil : positive_integer(value: requested_embedding_dimensions, field: :requested_embedding_dimensions),
|
|
43
|
+
modality: immutable.copy_and_freeze(value: modality, field: :modality),
|
|
44
|
+
provider_pin: provider_pin, instance_pin: optional_pin_text(value: instance_pin, field: :instance_pin),
|
|
45
|
+
model_pin: optional_pin_text(value: model_pin, field: :model_pin), tier_pin: tier_pin,
|
|
46
|
+
body_model_hint_decision: body_model_hint_decision, routing_seed: routing_seed.dup.freeze,
|
|
47
|
+
maximum_attempts: positive_integer(value: maximum_attempts, field: :maximum_attempts)
|
|
48
|
+
)
|
|
49
|
+
end
|
|
50
|
+
|
|
51
|
+
private
|
|
52
|
+
|
|
53
|
+
def positive_integer(value:, field:)
|
|
54
|
+
Legion::Extensions::Llm::Inventory::RecordSupport.nonnegative_integer(value: value, field: field)
|
|
55
|
+
raise Legion::Extensions::Llm::Inventory::Errors::ValidationError, "#{field} must be positive" if value.zero?
|
|
56
|
+
|
|
57
|
+
value
|
|
58
|
+
end
|
|
59
|
+
|
|
60
|
+
def optional_pin_text(value:, field:)
|
|
61
|
+
return nil if value.nil?
|
|
62
|
+
raise Legion::Extensions::Llm::Inventory::Errors::ValidationError, "#{field} must be nil or a nonempty String" unless value.is_a?(::String) && !value.strip.empty?
|
|
63
|
+
|
|
64
|
+
Legion::Extensions::Llm::Inventory::ImmutableValue.copy_and_freeze(value: value, field: field)
|
|
65
|
+
end
|
|
66
|
+
end
|
|
67
|
+
end
|
|
68
|
+
end
|
|
69
|
+
end
|
|
70
|
+
end
|
|
@@ -55,13 +55,14 @@ module Legion
|
|
|
55
55
|
|
|
56
56
|
requested_options = DEFAULT_PUBLISH_OPTIONS.merge(@options).merge(options || {})
|
|
57
57
|
return_result = return_publish_result?(requested_options)
|
|
58
|
+
payload = encode_message
|
|
59
|
+
validate_payload_size(payload: payload)
|
|
58
60
|
publish_options = request_publish_options(requested_options)
|
|
59
|
-
validate_payload_size
|
|
60
61
|
exchange_dest = fleet_exchange
|
|
61
62
|
return_state = {}
|
|
62
63
|
install_return_listener(exchange_dest, requested_options, return_state)
|
|
63
64
|
prepare_publisher_confirms(exchange_dest, requested_options)
|
|
64
|
-
exchange_dest.publish(
|
|
65
|
+
exchange_dest.publish(payload, **publish_options)
|
|
65
66
|
return nil unless return_result
|
|
66
67
|
|
|
67
68
|
publish_result(exchange_dest, requested_options.merge(publish_options), return_state)
|
|
@@ -125,7 +126,7 @@ module Legion
|
|
|
125
126
|
{
|
|
126
127
|
routing_key: routing_key || '',
|
|
127
128
|
content_type: options[:content_type] || content_type,
|
|
128
|
-
content_encoding: options[:content_encoding] || content_encoding,
|
|
129
|
+
content_encoding: @options[:content_encoding] || content_encoding,
|
|
129
130
|
type: options[:type] || type,
|
|
130
131
|
priority: options[:priority] || priority,
|
|
131
132
|
expiration: options[:expiration] || expiration,
|
|
@@ -61,6 +61,7 @@ module Legion
|
|
|
61
61
|
require_relative 'llm/inventory/publisher'
|
|
62
62
|
require_relative 'llm/inventory/probe_coordinator'
|
|
63
63
|
require_relative 'llm/routing/records'
|
|
64
|
+
require_relative 'llm/routing/request_requirements'
|
|
64
65
|
require_relative 'llm/routing/provider_outcome'
|
|
65
66
|
|
|
66
67
|
# --- Base value objects (no internal deps) ---
|
|
@@ -156,6 +157,9 @@ module Legion
|
|
|
156
157
|
|
|
157
158
|
def self.default_settings
|
|
158
159
|
{
|
|
160
|
+
ssl: {
|
|
161
|
+
verify: false
|
|
162
|
+
},
|
|
159
163
|
fleet: {
|
|
160
164
|
consumer: {
|
|
161
165
|
enabled: false,
|
|
@@ -64,12 +64,37 @@ RSpec.describe Legion::Extensions::Llm::Canonical::ContentBlock do
|
|
|
64
64
|
end
|
|
65
65
|
end
|
|
66
66
|
|
|
67
|
-
describe '
|
|
68
|
-
it '
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
67
|
+
describe 'source semantics at construction and rehydration' do
|
|
68
|
+
it 'normalizes source enums through every constructor' do
|
|
69
|
+
%i[new build from_hash].each do |constructor|
|
|
70
|
+
%w[base64 url].each do |source_type|
|
|
71
|
+
block = described_class.public_send(constructor, type: :image, source_type: source_type)
|
|
72
|
+
expect(block.source_type).to eq(source_type.to_sym)
|
|
73
|
+
end
|
|
74
|
+
end
|
|
75
|
+
end
|
|
76
|
+
|
|
77
|
+
it 'rejects missing and unknown image source types through every constructor' do
|
|
78
|
+
%i[new build from_hash].each do |constructor|
|
|
79
|
+
expect { described_class.public_send(constructor, type: :image) }.to raise_error(ArgumentError, /source_type/)
|
|
80
|
+
expect { described_class.public_send(constructor, type: :image, source_type: :file) }
|
|
81
|
+
.to raise_error(ArgumentError, /Invalid source_type/)
|
|
82
|
+
end
|
|
83
|
+
end
|
|
84
|
+
|
|
85
|
+
it 'preserves typed ordered content and image meaning across message JSON rehydration' do
|
|
86
|
+
content = [
|
|
87
|
+
described_class.text('compare', cache_control: { type: 'ephemeral' }),
|
|
88
|
+
described_class.image(data: 'AAA=', media_type: 'image/png', detail: 'high'),
|
|
89
|
+
described_class.image(data: 'https://example.org/image.png', media_type: nil, source_type: :url, detail: 'low')
|
|
90
|
+
]
|
|
91
|
+
message = Legion::Extensions::Llm::Canonical::Message.build(role: :user, content: content)
|
|
92
|
+
expect(message.content.first).to be(content.first)
|
|
93
|
+
|
|
94
|
+
restored = Legion::Extensions::Llm::Canonical::Message.from_hash(Legion::JSON.load(Legion::JSON.dump(message)))
|
|
95
|
+
|
|
96
|
+
expect(restored.content).to all(be_a(described_class))
|
|
97
|
+
expect(restored.content).to eq(content)
|
|
73
98
|
end
|
|
74
99
|
end
|
|
75
100
|
|
|
@@ -102,6 +102,26 @@ RSpec.describe Legion::Extensions::Llm::Canonical::Request do
|
|
|
102
102
|
end
|
|
103
103
|
end
|
|
104
104
|
|
|
105
|
+
describe 'system member contract — String or absent' do
|
|
106
|
+
it 'accepts a String system' do
|
|
107
|
+
expect(described_class.build(system: 'be brief').system).to eq('be brief')
|
|
108
|
+
end
|
|
109
|
+
|
|
110
|
+
it 'accepts an absent system' do
|
|
111
|
+
expect(described_class.build.system).to be_nil
|
|
112
|
+
end
|
|
113
|
+
|
|
114
|
+
it 'rejects a block-array system from every construction path' do
|
|
115
|
+
blocks = [{ type: 'text', text: 'Rule A' }]
|
|
116
|
+
expect { described_class.build(system: blocks) }
|
|
117
|
+
.to raise_error(ArgumentError, /system expected String, got Array/)
|
|
118
|
+
expect { described_class.new(system: blocks) }
|
|
119
|
+
.to raise_error(ArgumentError, /system expected String, got Array/)
|
|
120
|
+
expect { described_class.from_hash(system: blocks) }
|
|
121
|
+
.to raise_error(ArgumentError, /system expected String, got Array/)
|
|
122
|
+
end
|
|
123
|
+
end
|
|
124
|
+
|
|
105
125
|
it 'defaults stream false, routing {} and auto-generates req ids' do
|
|
106
126
|
request = described_class.build(messages: [])
|
|
107
127
|
expect(request.stream).to be(false)
|
|
@@ -0,0 +1,103 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'spec_helper'
|
|
4
|
+
require 'faraday/net_http'
|
|
5
|
+
require_relative '../../../support/tls_endpoint'
|
|
6
|
+
|
|
7
|
+
RSpec.describe Legion::Extensions::Llm::Connection do
|
|
8
|
+
around do |example|
|
|
9
|
+
saved = Legion::Settings.loader.settings[:extensions]
|
|
10
|
+
Legion::Settings.loader.settings[:extensions] = { llm: Legion::Extensions::Llm.default_settings }
|
|
11
|
+
example.run
|
|
12
|
+
ensure
|
|
13
|
+
if saved.nil?
|
|
14
|
+
Legion::Settings.loader.settings.delete(:extensions)
|
|
15
|
+
else
|
|
16
|
+
Legion::Settings.loader.settings[:extensions] = saved
|
|
17
|
+
end
|
|
18
|
+
end
|
|
19
|
+
|
|
20
|
+
{ typhoeus: Faraday::ConnectionFailed, net_http: Faraday::SSLError }.each do |adapter, error_class|
|
|
21
|
+
context "with the #{adapter} adapter" do
|
|
22
|
+
let(:endpoint) { SpecSupport::TlsEndpoint.new }
|
|
23
|
+
let(:config) do
|
|
24
|
+
Legion::Extensions::Llm::Configuration.new.tap do |configuration|
|
|
25
|
+
configuration.logger = Logger.new(File::NULL)
|
|
26
|
+
configuration.max_retries = 0
|
|
27
|
+
configuration.request_timeout = 3
|
|
28
|
+
end
|
|
29
|
+
end
|
|
30
|
+
let(:provider) do
|
|
31
|
+
instance_double(Legion::Extensions::Llm::Provider,
|
|
32
|
+
api_base: endpoint.url,
|
|
33
|
+
headers: { 'Authorization' => 'Bearer tls-fixture' },
|
|
34
|
+
parse_error: nil)
|
|
35
|
+
end
|
|
36
|
+
let(:connection) do
|
|
37
|
+
described_class.new(provider, config).tap do |client|
|
|
38
|
+
client.connection.builder.adapter(adapter)
|
|
39
|
+
client.connection.proxy = nil
|
|
40
|
+
end
|
|
41
|
+
end
|
|
42
|
+
|
|
43
|
+
after do
|
|
44
|
+
connection.close
|
|
45
|
+
endpoint.close
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
context 'with the default disabled verification' do
|
|
49
|
+
let(:endpoint) { SpecSupport::TlsEndpoint.new(subject_alt_name: 'DNS:wrong.example.test') }
|
|
50
|
+
|
|
51
|
+
it 'allows an authenticated request despite an untrusted and mismatched certificate' do
|
|
52
|
+
response = connection.get('/tls-check')
|
|
53
|
+
endpoint.finish
|
|
54
|
+
|
|
55
|
+
expect(response.status).to eq(200)
|
|
56
|
+
expect(response.body).to eq('status' => 'ok')
|
|
57
|
+
expect(endpoint.requests.one? { |request| request.match?(/^authorization: Bearer tls-fixture\r?$/i) }).to be(true)
|
|
58
|
+
end
|
|
59
|
+
end
|
|
60
|
+
|
|
61
|
+
context 'when TLS verification is enabled in Legion settings' do
|
|
62
|
+
before do
|
|
63
|
+
Legion::Settings.loader.settings[:extensions][:llm][:ssl][:verify] = true
|
|
64
|
+
end
|
|
65
|
+
|
|
66
|
+
it 'rejects an untrusted certificate before sending an authenticated HTTP request' do
|
|
67
|
+
expect { connection.get('/tls-check') }.to raise_error(error_class)
|
|
68
|
+
endpoint.finish
|
|
69
|
+
|
|
70
|
+
expect(endpoint.requests).to be_empty
|
|
71
|
+
end
|
|
72
|
+
|
|
73
|
+
it 'accepts a matching certificate through an explicitly trusted custom CA' do
|
|
74
|
+
connection.connection.ssl.ca_file = endpoint.ca_file.path
|
|
75
|
+
|
|
76
|
+
response = connection.get('/tls-check')
|
|
77
|
+
endpoint.finish
|
|
78
|
+
|
|
79
|
+
expect(response.status).to eq(200)
|
|
80
|
+
expect(response.body).to eq('status' => 'ok')
|
|
81
|
+
expect(endpoint.requests.one? { |request| request.match?(/^authorization: Bearer tls-fixture\r?$/i) }).to be(true)
|
|
82
|
+
end
|
|
83
|
+
end
|
|
84
|
+
|
|
85
|
+
context 'when TLS verification is enabled for a trusted wrong-host certificate' do
|
|
86
|
+
let(:endpoint) { SpecSupport::TlsEndpoint.new(subject_alt_name: 'DNS:wrong.example.test') }
|
|
87
|
+
|
|
88
|
+
before do
|
|
89
|
+
Legion::Settings.loader.settings[:extensions][:llm][:ssl][:verify] = true
|
|
90
|
+
end
|
|
91
|
+
|
|
92
|
+
it 'rejects the peer before sending an authenticated HTTP request' do
|
|
93
|
+
connection.connection.ssl.ca_file = endpoint.ca_file.path
|
|
94
|
+
|
|
95
|
+
expect { connection.get('/tls-check') }.to raise_error(error_class)
|
|
96
|
+
endpoint.finish
|
|
97
|
+
|
|
98
|
+
expect(endpoint.requests).to be_empty
|
|
99
|
+
end
|
|
100
|
+
end
|
|
101
|
+
end
|
|
102
|
+
end
|
|
103
|
+
end
|
|
@@ -247,7 +247,12 @@ RSpec.describe Legion::Extensions::Llm::CredentialSources do
|
|
|
247
247
|
end
|
|
248
248
|
|
|
249
249
|
it 'returns false when port is closed' do
|
|
250
|
-
|
|
250
|
+
socket = Socket.new(:INET, :STREAM)
|
|
251
|
+
socket.bind(Socket.sockaddr_in(0, '127.0.0.1'))
|
|
252
|
+
|
|
253
|
+
expect(mod.socket_open?('127.0.0.1', socket.local_address.ip_port, timeout: 0.05)).to be false
|
|
254
|
+
ensure
|
|
255
|
+
socket&.close
|
|
251
256
|
end
|
|
252
257
|
|
|
253
258
|
it 'returns false when connection is refused' do
|
|
@@ -260,12 +265,16 @@ RSpec.describe Legion::Extensions::Llm::CredentialSources do
|
|
|
260
265
|
expect(mod.socket_open?('192.0.2.1', 80, timeout: 0.05)).to be false
|
|
261
266
|
end
|
|
262
267
|
|
|
263
|
-
it '
|
|
264
|
-
|
|
265
|
-
|
|
266
|
-
|
|
267
|
-
|
|
268
|
-
|
|
268
|
+
it 'returns false when an asynchronous connection completes with an error' do
|
|
269
|
+
socket_error = Socket::Option.int(Socket::AF_INET, Socket::SOL_SOCKET, Socket::SO_ERROR, Errno::ECONNREFUSED::Errno)
|
|
270
|
+
fake_sock = instance_double(Socket, setsockopt: nil, wait_writable: true, getsockopt: socket_error, close: nil)
|
|
271
|
+
allow(Socket).to receive(:new).and_return(fake_sock)
|
|
272
|
+
allow(fake_sock).to receive(:connect_nonblock).and_invoke(
|
|
273
|
+
->(*) { raise IO::EINPROGRESSWaitWritable },
|
|
274
|
+
->(*) { 0 }
|
|
275
|
+
)
|
|
276
|
+
|
|
277
|
+
expect(mod.socket_open?('192.0.2.1', 80, timeout: 0.05)).to be false
|
|
269
278
|
end
|
|
270
279
|
end
|
|
271
280
|
|
|
@@ -143,6 +143,13 @@ RSpec.describe Legion::Extensions::Llm::Fleet::ExactOffering do
|
|
|
143
143
|
end
|
|
144
144
|
|
|
145
145
|
it 'rehydrates JSON-round-tripped messages before every message operation' do
|
|
146
|
+
content = [
|
|
147
|
+
Legion::Extensions::Llm::Canonical::ContentBlock.text('compare', cache_control: { type: 'ephemeral' }),
|
|
148
|
+
Legion::Extensions::Llm::Canonical::ContentBlock.image(data: 'AAA=', media_type: 'image/png', detail: 'high'),
|
|
149
|
+
Legion::Extensions::Llm::Canonical::ContentBlock.image(
|
|
150
|
+
data: 'https://example.org/image.png', media_type: nil, source_type: :url, detail: 'low'
|
|
151
|
+
)
|
|
152
|
+
]
|
|
146
153
|
raw_messages = [
|
|
147
154
|
{ role: 'system', content: 'follow the rules' },
|
|
148
155
|
{
|
|
@@ -150,7 +157,7 @@ RSpec.describe Legion::Extensions::Llm::Fleet::ExactOffering do
|
|
|
150
157
|
tool_calls: [{ id: 'call-1', name: 'lookup', arguments: { query: 'status' } }]
|
|
151
158
|
},
|
|
152
159
|
{ role: 'tool', tool_call_id: 'call-1', content: 'clean' },
|
|
153
|
-
|
|
160
|
+
Legion::Extensions::Llm::Canonical::Message.build(role: :user, content: content)
|
|
154
161
|
]
|
|
155
162
|
|
|
156
163
|
%w[chat stream_chat count_tokens].each do |operation|
|
|
@@ -166,6 +173,8 @@ RSpec.describe Legion::Extensions::Llm::Fleet::ExactOffering do
|
|
|
166
173
|
expect(call[:messages].map(&:role)).to eq(%i[system assistant tool user])
|
|
167
174
|
expect(call[:messages][1].tool_calls.first.name).to eq('lookup')
|
|
168
175
|
expect(call[:messages][2].tool_call_id).to eq('call-1')
|
|
176
|
+
expect(call[:messages][3].content).to all(be_a(Legion::Extensions::Llm::Canonical::ContentBlock))
|
|
177
|
+
expect(call[:messages][3].content).to eq(content)
|
|
169
178
|
end
|
|
170
179
|
end
|
|
171
180
|
end
|
|
@@ -15,6 +15,15 @@ RSpec.describe 'LLM fleet message envelopes' do
|
|
|
15
15
|
end
|
|
16
16
|
end
|
|
17
17
|
|
|
18
|
+
around do |example|
|
|
19
|
+
previous_crypt = Legion::Settings[:crypt].dup
|
|
20
|
+
Legion::Settings[:crypt][:cluster_secret] = 'a' * 64
|
|
21
|
+
Legion::Settings[:crypt][:cs_encrypt_ready] = true
|
|
22
|
+
example.run
|
|
23
|
+
ensure
|
|
24
|
+
Legion::Settings.loader.settings[:crypt] = previous_crypt
|
|
25
|
+
end
|
|
26
|
+
|
|
18
27
|
def valid_request_options
|
|
19
28
|
{
|
|
20
29
|
routing_key: 'llm.fleet.inference.qwen.ctx8192',
|
|
@@ -176,6 +185,11 @@ RSpec.describe 'LLM fleet message envelopes' do
|
|
|
176
185
|
type: protocol::REQUEST_TYPE
|
|
177
186
|
)
|
|
178
187
|
)
|
|
188
|
+
expect(exchange).to have_received(:publish) do |payload, **wire_options|
|
|
189
|
+
expect(wire_options[:content_encoding]).to eq('encrypted/cs')
|
|
190
|
+
decoded = Legion::JSON.load(Legion::Crypt.decrypt(payload, wire_options[:headers].fetch(:iv)))
|
|
191
|
+
expect(decoded).to include(model: 'qwen', params: { messages: [{ role: 'user', content: 'hello' }] })
|
|
192
|
+
end
|
|
179
193
|
expect(channel).to have_received(:on_return)
|
|
180
194
|
expect(channel).to have_received(:confirm_select)
|
|
181
195
|
expect(channel).to have_received(:wait_for_confirms)
|
|
@@ -200,6 +214,13 @@ RSpec.describe 'LLM fleet message envelopes' do
|
|
|
200
214
|
expect(channel).to have_received(:wait_for_confirms).with(no_args)
|
|
201
215
|
expect(result).to include(status: :accepted, accepted: true)
|
|
202
216
|
end
|
|
217
|
+
|
|
218
|
+
it 'rejects an encrypted request that exceeds the limit even when plaintext fits' do
|
|
219
|
+
message = described_class.new(**valid_request_options)
|
|
220
|
+
allow(described_class).to receive(:max_payload_bytes).and_return(Legion::JSON.dump(message.message).bytesize)
|
|
221
|
+
|
|
222
|
+
expect { message.publish }.to raise_error(Legion::Transport::PayloadTooLarge)
|
|
223
|
+
end
|
|
203
224
|
end
|
|
204
225
|
|
|
205
226
|
describe Legion::Extensions::Llm::Transport::Messages::FleetResponse do
|
|
@@ -241,14 +262,6 @@ RSpec.describe 'LLM fleet message envelopes' do
|
|
|
241
262
|
end.to raise_error(ArgumentError, /protocol_version must be 3/)
|
|
242
263
|
end
|
|
243
264
|
|
|
244
|
-
it 'is a pass-through for the response payload — the G5 thinking exclusion happens exactly once at the responder builder' do
|
|
245
|
-
message = described_class.new(**valid_response_options, response: { text: 'visible', stop_reason: :end_turn })
|
|
246
|
-
|
|
247
|
-
expect(message.message[:response]).to eq(text: 'visible', stop_reason: :end_turn)
|
|
248
|
-
expect(message.message).not_to have_key(:thinking)
|
|
249
|
-
expect(message.encode_message).not_to include('thinking')
|
|
250
|
-
end
|
|
251
|
-
|
|
252
265
|
it 'rejects legacy fleet envelope fields' do
|
|
253
266
|
%i[schema_version request_type fleet_correlation_id].each do |field|
|
|
254
267
|
expect do
|
|
@@ -274,6 +287,18 @@ RSpec.describe 'LLM fleet message envelopes' do
|
|
|
274
287
|
type: protocol::RESPONSE_TYPE
|
|
275
288
|
)
|
|
276
289
|
)
|
|
290
|
+
expect(default_exchange).to have_received(:publish) do |payload, **wire_options|
|
|
291
|
+
expect(wire_options[:content_encoding]).to eq('encrypted/cs')
|
|
292
|
+
decoded = Legion::JSON.load(Legion::Crypt.decrypt(payload, wire_options[:headers].fetch(:iv)))
|
|
293
|
+
expect(decoded[:response]).to include(text: 'hello', stop_reason: 'end_turn')
|
|
294
|
+
end
|
|
295
|
+
end
|
|
296
|
+
|
|
297
|
+
it 'rejects an encrypted reply that exceeds the limit even when plaintext fits' do
|
|
298
|
+
message = described_class.new(**valid_response_options)
|
|
299
|
+
allow(described_class).to receive(:max_payload_bytes).and_return(Legion::JSON.dump(message.message).bytesize)
|
|
300
|
+
|
|
301
|
+
expect { message.publish }.to raise_error(Legion::Transport::PayloadTooLarge)
|
|
277
302
|
end
|
|
278
303
|
|
|
279
304
|
it 'accepts standard transport publish options and returns accepted reply results' do
|
|
@@ -72,29 +72,41 @@ RSpec.describe Legion::Extensions::Llm::Provider::OpenAICompatible do
|
|
|
72
72
|
end
|
|
73
73
|
end
|
|
74
74
|
|
|
75
|
-
it '
|
|
75
|
+
it 'sends ordered typed text and image_url parts through the callable HTTP boundary' do
|
|
76
76
|
message = canonical::Message.build(
|
|
77
77
|
role: :user,
|
|
78
|
-
content: [
|
|
79
|
-
|
|
78
|
+
content: [
|
|
79
|
+
canonical::ContentBlock.text('compare these', cache_control: { type: 'ephemeral' }),
|
|
80
|
+
canonical::ContentBlock.image(data: 'AAA=', media_type: 'image/png', detail: 'high'),
|
|
81
|
+
canonical::ContentBlock.text('with this'),
|
|
82
|
+
canonical::ContentBlock.build(type: :image, source_type: :url, data: 'https://example.org/image.png',
|
|
83
|
+
detail: 'low', cache_control: { type: 'ephemeral' })
|
|
84
|
+
]
|
|
80
85
|
)
|
|
86
|
+
stubs = Faraday::Adapter::Test::Stubs.new do |stub|
|
|
87
|
+
stub.post('/v1/chat/completions') do |env|
|
|
88
|
+
payload = Legion::JSON.load(env.body)
|
|
89
|
+
expect(payload[:messages]).to eq(
|
|
90
|
+
[{
|
|
91
|
+
role: 'user',
|
|
92
|
+
content: [
|
|
93
|
+
{ type: 'text', text: 'compare these' },
|
|
94
|
+
{ type: 'image_url', image_url: { url: 'data:image/png;base64,AAA=', detail: 'high' } },
|
|
95
|
+
{ type: 'text', text: 'with this' },
|
|
96
|
+
{ type: 'image_url', image_url: { url: 'https://example.org/image.png', detail: 'low' } }
|
|
97
|
+
]
|
|
98
|
+
}]
|
|
99
|
+
)
|
|
100
|
+
[200, { 'content-type' => 'application/json' }, Legion::JSON.dump(completion_body)]
|
|
101
|
+
end
|
|
102
|
+
end
|
|
103
|
+
provider.connection.connection.builder.adapter :test, stubs
|
|
81
104
|
|
|
82
|
-
|
|
83
|
-
:render_payload,
|
|
84
|
-
[message],
|
|
85
|
-
tools: {},
|
|
86
|
-
model: model_id,
|
|
87
|
-
stream: false,
|
|
88
|
-
schema: nil,
|
|
89
|
-
thinking: nil,
|
|
90
|
-
tool_prefs: nil,
|
|
91
|
-
params: nil
|
|
92
|
-
)
|
|
105
|
+
response = provider.chat([message], model: model_id)
|
|
93
106
|
|
|
94
|
-
|
|
95
|
-
expect(
|
|
96
|
-
|
|
97
|
-
)
|
|
107
|
+
expect(response).to be_a(canonical::Response)
|
|
108
|
+
expect(response.text).to eq('hi')
|
|
109
|
+
stubs.verify_stubbed_calls
|
|
98
110
|
end
|
|
99
111
|
|
|
100
112
|
it 'parses chat completion responses to a Canonical::Response with usage and tool calls (08 R2)' do
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'spec_helper'
|
|
4
|
+
require 'legion/extensions/llm/routing/request_requirements'
|
|
5
|
+
|
|
6
|
+
RSpec.describe Legion::Extensions::Llm::Routing::RequestRequirements do
|
|
7
|
+
let(:validation_error) { Legion::Extensions::Llm::Inventory::Errors::ValidationError }
|
|
8
|
+
let(:attributes) do
|
|
9
|
+
{
|
|
10
|
+
operation: :chat, required_capabilities: %i[completion vision], input_bound: 0,
|
|
11
|
+
required_output_tokens: 0, context_budget: 0, requested_embedding_dimensions: nil,
|
|
12
|
+
modality: nil, provider_pin: nil, instance_pin: nil, model_pin: nil, tier_pin: nil,
|
|
13
|
+
body_model_hint_decision: Legion::Extensions::Llm::Routing::BodyModelHintDecision.new(
|
|
14
|
+
requested_model: nil, disposition: :absent, settings_generation: 0
|
|
15
|
+
),
|
|
16
|
+
routing_seed: '0123456789abcdef0123456789abcdef', maximum_attempts: 1
|
|
17
|
+
}
|
|
18
|
+
end
|
|
19
|
+
|
|
20
|
+
it 'isolates routing facts from later mutations of request-owned values' do
|
|
21
|
+
capabilities = %i[completion vision]
|
|
22
|
+
modality = { output: [+ 'image', { format: + 'png' }] }
|
|
23
|
+
model = + 'model-a'
|
|
24
|
+
instance = + 'primary'
|
|
25
|
+
seed = attributes[:routing_seed].dup
|
|
26
|
+
requirements = described_class.new(**attributes, required_capabilities: capabilities, modality: modality,
|
|
27
|
+
model_pin: model, instance_pin: instance, routing_seed: seed)
|
|
28
|
+
|
|
29
|
+
capabilities.clear
|
|
30
|
+
modality[:output].first.replace('audio')
|
|
31
|
+
modality[:output].last[:format].replace('wav')
|
|
32
|
+
model.replace('other-model')
|
|
33
|
+
instance.replace('other-instance')
|
|
34
|
+
seed.replace('f' * 32)
|
|
35
|
+
|
|
36
|
+
expect(requirements.required_capabilities).to eq(%i[completion vision])
|
|
37
|
+
expect(requirements.modality).to eq(output: ['image', { format: 'png' }])
|
|
38
|
+
expect(requirements.model_pin).to eq('model-a')
|
|
39
|
+
expect(requirements.instance_pin).to eq('primary')
|
|
40
|
+
expect(requirements.routing_seed).to eq(attributes[:routing_seed])
|
|
41
|
+
expect { requirements.modality[:output].last[:format].replace('jpeg') }.to raise_error(FrozenError)
|
|
42
|
+
expect { requirements.required_capabilities.clear }.to raise_error(FrozenError)
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
it 'rejects untranslated vocabulary and incorrectly typed explicit pins' do
|
|
46
|
+
{
|
|
47
|
+
operation: :completion, required_capabilities: [:function_calling], tier_pin: 'local',
|
|
48
|
+
provider_pin: 'openai', instance_pin: :primary, model_pin: :model
|
|
49
|
+
}.each do |field, value|
|
|
50
|
+
expect { described_class.new(**attributes, field => value) }.to raise_error(validation_error, /#{field}/)
|
|
51
|
+
end
|
|
52
|
+
end
|
|
53
|
+
|
|
54
|
+
it 'rejects invalid budgets, dimensions, and attempt limits rather than coercing them' do
|
|
55
|
+
{
|
|
56
|
+
input_bound: -1, required_output_tokens: 1.5, context_budget: nil,
|
|
57
|
+
requested_embedding_dimensions: 0, maximum_attempts: 0
|
|
58
|
+
}.each do |field, value|
|
|
59
|
+
expect { described_class.new(**attributes, field => value) }.to raise_error(validation_error, /#{field}/)
|
|
60
|
+
end
|
|
61
|
+
end
|
|
62
|
+
|
|
63
|
+
it 'requires request ownership of the routing seed and model hint decision' do
|
|
64
|
+
expect { described_class.new(**attributes.except(:routing_seed)) }.to raise_error(ArgumentError, /routing_seed/)
|
|
65
|
+
expect { described_class.new(**attributes, routing_seed: 'A' * 32) }.to raise_error(validation_error, /routing_seed/)
|
|
66
|
+
expect { described_class.new(**attributes, body_model_hint_decision: attributes[:body_model_hint_decision].to_h) }
|
|
67
|
+
.to raise_error(validation_error, /body_model_hint_decision/)
|
|
68
|
+
end
|
|
69
|
+
|
|
70
|
+
it 'rejects unsupported mutable values embedded in modality' do
|
|
71
|
+
expect { described_class.new(**attributes, modality: { image: Object.new }) }
|
|
72
|
+
.to raise_error(validation_error, /modality.image/)
|
|
73
|
+
end
|
|
74
|
+
end
|
|
@@ -0,0 +1,92 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'openssl'
|
|
4
|
+
require 'socket'
|
|
5
|
+
require 'tempfile'
|
|
6
|
+
require 'timeout'
|
|
7
|
+
|
|
8
|
+
module SpecSupport
|
|
9
|
+
# One real HTTPS exchange, using a fresh private CA rather than external trust.
|
|
10
|
+
class TlsEndpoint
|
|
11
|
+
attr_reader :url, :ca_file, :requests
|
|
12
|
+
|
|
13
|
+
def initialize(subject_alt_name: 'IP:127.0.0.1')
|
|
14
|
+
ca_key = OpenSSL::PKey::EC.generate('prime256v1')
|
|
15
|
+
ca = certificate(key: ca_key, name: 'lex-llm test CA')
|
|
16
|
+
ca_extensions = OpenSSL::X509::ExtensionFactory.new
|
|
17
|
+
ca_extensions.subject_certificate = ca
|
|
18
|
+
ca_extensions.issuer_certificate = ca
|
|
19
|
+
ca.add_extension(ca_extensions.create_extension('basicConstraints', 'CA:TRUE', true))
|
|
20
|
+
ca.add_extension(ca_extensions.create_extension('keyUsage', 'keyCertSign,cRLSign', true))
|
|
21
|
+
ca.sign(ca_key, OpenSSL::Digest.new('SHA256'))
|
|
22
|
+
|
|
23
|
+
key = OpenSSL::PKey::EC.generate('prime256v1')
|
|
24
|
+
cert = certificate(key: key, name: 'lex-llm test endpoint')
|
|
25
|
+
cert.issuer = ca.subject
|
|
26
|
+
extensions = OpenSSL::X509::ExtensionFactory.new
|
|
27
|
+
extensions.subject_certificate = cert
|
|
28
|
+
extensions.issuer_certificate = ca
|
|
29
|
+
cert.add_extension(extensions.create_extension('basicConstraints', 'CA:FALSE', true))
|
|
30
|
+
cert.add_extension(extensions.create_extension('keyUsage', 'digitalSignature', true))
|
|
31
|
+
cert.add_extension(extensions.create_extension('extendedKeyUsage', 'serverAuth'))
|
|
32
|
+
cert.add_extension(extensions.create_extension('subjectAltName', subject_alt_name))
|
|
33
|
+
cert.sign(ca_key, OpenSSL::Digest.new('SHA256'))
|
|
34
|
+
|
|
35
|
+
@ca_file = Tempfile.new(['lex-llm-ca', '.pem'])
|
|
36
|
+
@ca_file.write(ca.to_pem)
|
|
37
|
+
@ca_file.flush
|
|
38
|
+
@requests = []
|
|
39
|
+
@listener = TCPServer.new('127.0.0.1', 0)
|
|
40
|
+
@url = "https://127.0.0.1:#{@listener.addr[1]}"
|
|
41
|
+
context = OpenSSL::SSL::SSLContext.new
|
|
42
|
+
context.cert = cert
|
|
43
|
+
context.key = key
|
|
44
|
+
server = OpenSSL::SSL::SSLServer.new(@listener, context)
|
|
45
|
+
@worker = Thread.new { serve(server: server) }
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
def finish
|
|
49
|
+
raise Timeout::Error, 'TLS fixture did not finish its exchange' unless @worker.join(5)
|
|
50
|
+
|
|
51
|
+
@worker.value
|
|
52
|
+
end
|
|
53
|
+
|
|
54
|
+
def close
|
|
55
|
+
@worker&.kill
|
|
56
|
+
@worker&.join
|
|
57
|
+
@listener&.close
|
|
58
|
+
@ca_file&.close!
|
|
59
|
+
end
|
|
60
|
+
|
|
61
|
+
private
|
|
62
|
+
|
|
63
|
+
def certificate(key:, name:)
|
|
64
|
+
cert = OpenSSL::X509::Certificate.new
|
|
65
|
+
cert.version = 2
|
|
66
|
+
cert.serial = OpenSSL::BN.rand(128).to_i
|
|
67
|
+
cert.subject = OpenSSL::X509::Name.parse("/CN=#{name}")
|
|
68
|
+
cert.issuer = cert.subject
|
|
69
|
+
cert.public_key = key
|
|
70
|
+
cert.not_before = Time.now - 60
|
|
71
|
+
cert.not_after = Time.now + 3600
|
|
72
|
+
cert
|
|
73
|
+
end
|
|
74
|
+
|
|
75
|
+
def serve(server:)
|
|
76
|
+
socket = server.accept
|
|
77
|
+
request = +''
|
|
78
|
+
while (line = socket.gets) && line != "\r\n"
|
|
79
|
+
request << line
|
|
80
|
+
end
|
|
81
|
+
return if request.empty?
|
|
82
|
+
|
|
83
|
+
@requests << request
|
|
84
|
+
body = '{"status":"ok"}'
|
|
85
|
+
socket.write("HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nContent-Length: #{body.bytesize}\r\nConnection: close\r\n\r\n#{body}")
|
|
86
|
+
rescue OpenSSL::SSL::SSLError, Errno::ECONNRESET => e
|
|
87
|
+
Legion::Logging.debug("TLS fixture peer rejected the connection: #{e.class}")
|
|
88
|
+
ensure
|
|
89
|
+
socket&.close
|
|
90
|
+
end
|
|
91
|
+
end
|
|
92
|
+
end
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: lex-llm
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.8.
|
|
4
|
+
version: 0.8.8
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- LegionIO
|
|
@@ -198,14 +198,14 @@ dependencies:
|
|
|
198
198
|
requirements:
|
|
199
199
|
- - ">="
|
|
200
200
|
- !ruby/object:Gem::Version
|
|
201
|
-
version: 1.4.
|
|
201
|
+
version: 1.4.31
|
|
202
202
|
type: :runtime
|
|
203
203
|
prerelease: false
|
|
204
204
|
version_requirements: !ruby/object:Gem::Requirement
|
|
205
205
|
requirements:
|
|
206
206
|
- - ">="
|
|
207
207
|
- !ruby/object:Gem::Version
|
|
208
|
-
version: 1.4.
|
|
208
|
+
version: 1.4.31
|
|
209
209
|
- !ruby/object:Gem::Dependency
|
|
210
210
|
name: marcel
|
|
211
211
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -314,6 +314,7 @@ files:
|
|
|
314
314
|
- lib/legion/extensions/llm/routing.rb
|
|
315
315
|
- lib/legion/extensions/llm/routing/provider_outcome.rb
|
|
316
316
|
- lib/legion/extensions/llm/routing/records.rb
|
|
317
|
+
- lib/legion/extensions/llm/routing/request_requirements.rb
|
|
317
318
|
- lib/legion/extensions/llm/settings_cascade.rb
|
|
318
319
|
- lib/legion/extensions/llm/stop_reason_mapping.rb
|
|
319
320
|
- lib/legion/extensions/llm/stream_accumulator.rb
|
|
@@ -388,6 +389,7 @@ files:
|
|
|
388
389
|
- spec/legion/extensions/llm/connection_close_spec.rb
|
|
389
390
|
- spec/legion/extensions/llm/connection_logging_spec.rb
|
|
390
391
|
- spec/legion/extensions/llm/connection_retry_spec.rb
|
|
392
|
+
- spec/legion/extensions/llm/connection_tls_spec.rb
|
|
391
393
|
- spec/legion/extensions/llm/credential_sources_spec.rb
|
|
392
394
|
- spec/legion/extensions/llm/error_middleware_spec.rb
|
|
393
395
|
- spec/legion/extensions/llm/error_spec.rb
|
|
@@ -421,6 +423,7 @@ files:
|
|
|
421
423
|
- spec/legion/extensions/llm/responses/thinking_extractor_spec.rb
|
|
422
424
|
- spec/legion/extensions/llm/routing/provider_outcome_spec.rb
|
|
423
425
|
- spec/legion/extensions/llm/routing/records_spec.rb
|
|
426
|
+
- spec/legion/extensions/llm/routing/request_requirements_spec.rb
|
|
424
427
|
- spec/legion/extensions/llm/settings_cascade_spec.rb
|
|
425
428
|
- spec/legion/extensions/llm/stop_reason_mapping_spec.rb
|
|
426
429
|
- spec/legion/extensions/llm/stream_accumulator_spec.rb
|
|
@@ -439,6 +442,7 @@ files:
|
|
|
439
442
|
- spec/support/rspec_configuration.rb
|
|
440
443
|
- spec/support/simplecov_configuration.rb
|
|
441
444
|
- spec/support/ssot_registry_helpers.rb
|
|
445
|
+
- spec/support/tls_endpoint.rb
|
|
442
446
|
homepage: https://github.com/LegionIO/lex-llm
|
|
443
447
|
licenses:
|
|
444
448
|
- MIT
|