lex-llm-bedrock 0.1.4 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +20 -0
- data/README.md +68 -1
- data/lex-llm-bedrock.gemspec +1 -1
- data/lib/legion/extensions/llm/bedrock/provider.rb +61 -21
- data/lib/legion/extensions/llm/bedrock/version.rb +1 -1
- data/lib/legion/extensions/llm/bedrock.rb +21 -23
- metadata +3 -7
- data/lib/legion/extensions/llm/bedrock/registry_event_builder.rb +0 -93
- data/lib/legion/extensions/llm/bedrock/registry_publisher.rb +0 -100
- data/lib/legion/extensions/llm/bedrock/transport/exchanges/llm_registry.rb +0 -24
- data/lib/legion/extensions/llm/bedrock/transport/messages/registry_event.rb +0 -42
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: eafb04fa29d1bf032323c4040c7cdbe0ea2d3cf1c72328864ec54b5e2f7ce117
|
|
4
|
+
data.tar.gz: 6da27c531876ff32af29b783548b86bed7a4146f45f0b94da679320dd594310b
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 7b7a1bc938f269380f0310cfd0e4a2f3561e986ed09db904dbe6a31663451658bac03e56b54c57f3c32efd85c7d83eb64e8d50fdbeac20efe2314f6a9e5d0a38
|
|
7
|
+
data.tar.gz: db293ebc885b48aa2902ecccfd5cf8e47bf58fd775ec211d31cb9a491dcecb60c5146989175affd2e1c59bd3f2429492eea94f36d202f41955708d72a3ec5780
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,25 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.2.0 - 2026-04-30
|
|
4
|
+
|
|
5
|
+
- Adopt lex-llm 0.1.9 base contract: flat `default_settings`, base `RegistryPublisher`, base `RegistryEventBuilder`.
|
|
6
|
+
- Replace `provider_settings` call with flat default_settings hash (default_model, region, credentials, whitelist/blacklist, TLS, instances).
|
|
7
|
+
- Remove `Provider.register` call; register configuration options directly via `Configuration.register_provider_options`.
|
|
8
|
+
- Delete local `RegistryPublisher`, `RegistryEventBuilder`, and `transport/` directory; use parameterized base classes from lex-llm.
|
|
9
|
+
- Move `registry_publisher` from `Provider` class method to `Bedrock` module method using `Legion::Extensions::Llm::RegistryPublisher.new(provider_family: :bedrock)`.
|
|
10
|
+
- Rewrite `list_models` to return `Model::Info` with `capabilities`, `modalities_input`, and `modalities_output` derived from Bedrock `inputModalities`/`outputModalities`.
|
|
11
|
+
- Publish discovered models via `publish_models_async` (base contract) instead of `publish_offerings_async`.
|
|
12
|
+
- Bump gemspec dependency to `lex-llm >= 0.1.9`.
|
|
13
|
+
|
|
14
|
+
## 0.1.5 - 2026-04-30
|
|
15
|
+
|
|
16
|
+
- Audit logging, rescue blocks, and README for full observability.
|
|
17
|
+
- Add `include Legion::Logging::Helper` to Provider, RegistryPublisher, and RegistryEventBuilder.
|
|
18
|
+
- Replace all bare rescue blocks with `handle_exception(e, level:, handled:, operation:)` calls.
|
|
19
|
+
- Add `log.info` for key actions: chat, stream, embed, health, discovery, list_models.
|
|
20
|
+
- Remove custom `log_publish_failure` method in favor of standard `handle_exception`.
|
|
21
|
+
- Update README with architecture, file map, dependency table, and development guide.
|
|
22
|
+
|
|
3
23
|
## 0.1.4 - 2026-04-30
|
|
4
24
|
|
|
5
25
|
- Add headers: parameter to complete method signature matching base provider contract
|
data/README.md
CHANGED
|
@@ -4,6 +4,48 @@ Amazon Bedrock provider extension for `Legion::Extensions::Llm`.
|
|
|
4
4
|
|
|
5
5
|
This gem adds a hosted Bedrock provider surface for Legion LLM routing without depending on the old `legion-llm` gem. It uses the official AWS SDK for Ruby and keeps discovery offline by default, so loading the extension or running tests does not require live AWS credentials. It requires `lex-llm >= 0.1.5` for the shared model offering, alias, readiness, and fleet lane contract.
|
|
6
6
|
|
|
7
|
+
## Architecture
|
|
8
|
+
|
|
9
|
+
```
|
|
10
|
+
Legion::Extensions::Llm::Bedrock
|
|
11
|
+
├── Provider # Bedrock implementation of the lex-llm Provider contract
|
|
12
|
+
│ ├── Capabilities # Capability predicates inferred from model IDs
|
|
13
|
+
│ ├── chat / stream # Converse / ConverseStream API calls
|
|
14
|
+
│ ├── embed # Titan InvokeModel embedding
|
|
15
|
+
│ ├── count_tokens # CountTokens API call
|
|
16
|
+
│ ├── discover_offerings # Static catalog + live ListFoundationModels
|
|
17
|
+
│ ├── health / readiness # Provider health checks with live AWS verification
|
|
18
|
+
│ └── list_models # Live model enumeration
|
|
19
|
+
├── RegistryEventBuilder # Builds sanitized lex-llm registry envelopes
|
|
20
|
+
├── RegistryPublisher # Best-effort async publisher for registry events
|
|
21
|
+
└── Transport
|
|
22
|
+
├── Exchanges::LlmRegistry # Topic exchange for llm.registry events
|
|
23
|
+
└── Messages::RegistryEvent # AMQP message for registry event publishing
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
## Dependencies
|
|
27
|
+
|
|
28
|
+
| Gem | Required | Purpose |
|
|
29
|
+
|-----|----------|---------|
|
|
30
|
+
| `aws-sdk-bedrock` | Yes | Bedrock management client (ListFoundationModels) |
|
|
31
|
+
| `aws-sdk-bedrockruntime` | Yes | Bedrock runtime client (Converse, InvokeModel) |
|
|
32
|
+
| `legion-json` (>= 1.2.1) | Yes | JSON serialization |
|
|
33
|
+
| `legion-logging` (>= 1.3.2) | Yes | Structured logging via Helper |
|
|
34
|
+
| `legion-settings` (>= 1.3.14) | Yes | Configuration |
|
|
35
|
+
| `lex-llm` (>= 0.1.5) | Yes | Shared provider contract, model offerings, routing |
|
|
36
|
+
|
|
37
|
+
## File Map
|
|
38
|
+
|
|
39
|
+
| Path | Purpose |
|
|
40
|
+
|------|---------|
|
|
41
|
+
| `lib/legion/extensions/llm/bedrock.rb` | Entry point: namespace, default settings, provider registration |
|
|
42
|
+
| `lib/legion/extensions/llm/bedrock/provider.rb` | Full Bedrock provider implementation |
|
|
43
|
+
| `lib/legion/extensions/llm/bedrock/registry_event_builder.rb` | Builds lex-llm registry event envelopes |
|
|
44
|
+
| `lib/legion/extensions/llm/bedrock/registry_publisher.rb` | Best-effort async registry event publishing |
|
|
45
|
+
| `lib/legion/extensions/llm/bedrock/transport/exchanges/llm_registry.rb` | AMQP topic exchange definition |
|
|
46
|
+
| `lib/legion/extensions/llm/bedrock/transport/messages/registry_event.rb` | AMQP message class for registry events |
|
|
47
|
+
| `lib/legion/extensions/llm/bedrock/version.rb` | `VERSION` constant |
|
|
48
|
+
|
|
7
49
|
## Install
|
|
8
50
|
|
|
9
51
|
```ruby
|
|
@@ -31,6 +73,8 @@ If explicit keys are not configured, the AWS SDK default credential provider cha
|
|
|
31
73
|
Legion::Extensions::Llm::Bedrock.default_settings
|
|
32
74
|
```
|
|
33
75
|
|
|
76
|
+
Configuration options: `bedrock_region`, `bedrock_endpoint`, `bedrock_access_key_id`, `bedrock_secret_access_key`, `bedrock_session_token`, `bedrock_profile`, `bedrock_stub_responses`.
|
|
77
|
+
|
|
34
78
|
## Provider Surface
|
|
35
79
|
|
|
36
80
|
```ruby
|
|
@@ -58,6 +102,8 @@ Every offering uses:
|
|
|
58
102
|
|
|
59
103
|
Known aliases are intentionally small and conservative. For example, `claude-3-haiku` resolves to `anthropic.claude-3-haiku-20240307-v1:0`, while the preserved Bedrock model ID remains the routing model.
|
|
60
104
|
|
|
105
|
+
Static models: `claude-3-haiku`, `titan-text-express`, `titan-embed-text-v2`, `llama-3.2-11b-instruct`, `mistral-large-3`.
|
|
106
|
+
|
|
61
107
|
## API Contract
|
|
62
108
|
|
|
63
109
|
The implementation is intentionally limited to Bedrock operations documented by AWS:
|
|
@@ -70,10 +116,31 @@ The implementation is intentionally limited to Bedrock operations documented by
|
|
|
70
116
|
|
|
71
117
|
Provider-specific request bodies are not guessed. Non-Titan embedding models raise until their documented body shape is added explicitly.
|
|
72
118
|
|
|
73
|
-
|
|
119
|
+
## Observability
|
|
120
|
+
|
|
121
|
+
All classes include `Legion::Logging::Helper` for structured logging:
|
|
122
|
+
|
|
123
|
+
- **Info-level**: provider connections, API calls (chat, stream, embed), model listing, health checks
|
|
124
|
+
- **Debug-level**: offline health checks, readiness probes, token counting, registry event scheduling
|
|
125
|
+
- **Rescue blocks**: every rescue calls `handle_exception(e, level:, handled:, operation:)` with dot-separated operation names (e.g., `bedrock.provider.health`, `bedrock.registry_publisher.publish_event`)
|
|
126
|
+
|
|
127
|
+
## Development
|
|
128
|
+
|
|
129
|
+
```bash
|
|
130
|
+
bundle install
|
|
131
|
+
bundle exec rspec --format progress # all pass
|
|
132
|
+
bundle exec rubocop -A # auto-fix
|
|
133
|
+
bundle exec rubocop # lint check (0 offenses expected)
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
## AWS References
|
|
74
137
|
|
|
75
138
|
- [Converse](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html)
|
|
76
139
|
- [ConverseStream](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ConverseStream.html)
|
|
77
140
|
- [CountTokens](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_CountTokens.html)
|
|
78
141
|
- [ListFoundationModels](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_ListFoundationModels.html)
|
|
79
142
|
- [Foundation model information](https://docs.aws.amazon.com/bedrock/latest/userguide/foundation-models-reference.html)
|
|
143
|
+
|
|
144
|
+
## License
|
|
145
|
+
|
|
146
|
+
MIT
|
data/lex-llm-bedrock.gemspec
CHANGED
|
@@ -28,5 +28,5 @@ Gem::Specification.new do |spec|
|
|
|
28
28
|
spec.add_dependency 'legion-json', '>= 1.2.1'
|
|
29
29
|
spec.add_dependency 'legion-logging', '>= 1.3.2'
|
|
30
30
|
spec.add_dependency 'legion-settings', '>= 1.3.14'
|
|
31
|
-
spec.add_dependency 'lex-llm', '>= 0.
|
|
31
|
+
spec.add_dependency 'lex-llm', '>= 0.2.0'
|
|
32
32
|
end
|
|
@@ -12,6 +12,8 @@ module Legion
|
|
|
12
12
|
module Bedrock
|
|
13
13
|
# Amazon Bedrock provider implementation for the Legion::Extensions::Llm contract.
|
|
14
14
|
class Provider < Legion::Extensions::Llm::Provider # rubocop:disable Metrics/ClassLength
|
|
15
|
+
include Legion::Logging::Helper
|
|
16
|
+
|
|
15
17
|
DEFAULT_REGION = 'us-east-1'
|
|
16
18
|
|
|
17
19
|
STATIC_MODELS = [
|
|
@@ -25,8 +27,6 @@ module Legion
|
|
|
25
27
|
ALIASES = STATIC_MODELS.to_h { |entry| [entry.fetch(:alias), entry.fetch(:model)] }.freeze
|
|
26
28
|
|
|
27
29
|
class << self
|
|
28
|
-
attr_writer :registry_publisher
|
|
29
|
-
|
|
30
30
|
def slug = 'bedrock'
|
|
31
31
|
|
|
32
32
|
def configuration_options
|
|
@@ -45,7 +45,7 @@ module Legion
|
|
|
45
45
|
def capabilities = Capabilities
|
|
46
46
|
|
|
47
47
|
def registry_publisher
|
|
48
|
-
|
|
48
|
+
Bedrock.registry_publisher
|
|
49
49
|
end
|
|
50
50
|
|
|
51
51
|
def resolve_model_id(model_id, config: nil) # rubocop:disable Lint/UnusedMethodArgument
|
|
@@ -85,11 +85,15 @@ module Legion
|
|
|
85
85
|
end
|
|
86
86
|
|
|
87
87
|
def discover_offerings(live: false, **filters)
|
|
88
|
-
|
|
88
|
+
unless live
|
|
89
|
+
log.debug { 'bedrock.provider.discover_offerings: returning static catalog' }
|
|
90
|
+
return static_offerings(**filters)
|
|
91
|
+
end
|
|
89
92
|
|
|
93
|
+
log.info { "bedrock.provider.discover_offerings: listing foundation models (region=#{region})" }
|
|
90
94
|
response = bedrock_client.list_foundation_models(**filters)
|
|
91
95
|
Array(value(response, :model_summaries)).map { |summary| offering_from_summary(summary) }.tap do |offerings|
|
|
92
|
-
|
|
96
|
+
log.info { "bedrock.provider.discover_offerings: found #{offerings.size} models" }
|
|
93
97
|
end
|
|
94
98
|
end
|
|
95
99
|
|
|
@@ -114,15 +118,22 @@ module Legion
|
|
|
114
118
|
live: live,
|
|
115
119
|
credentials: credential_source
|
|
116
120
|
}
|
|
117
|
-
|
|
121
|
+
unless live
|
|
122
|
+
log.debug { "bedrock.provider.health: offline check (region=#{region})" }
|
|
123
|
+
return baseline.merge(checked: false)
|
|
124
|
+
end
|
|
118
125
|
|
|
126
|
+
log.info { "bedrock.provider.health: live check (region=#{region})" }
|
|
119
127
|
bedrock_client.list_foundation_models
|
|
128
|
+
log.info { 'bedrock.provider.health: live check passed' }
|
|
120
129
|
baseline.merge(checked: true)
|
|
121
130
|
rescue StandardError => e
|
|
131
|
+
handle_exception(e, level: :warn, handled: true, operation: 'bedrock.provider.health')
|
|
122
132
|
baseline.merge(checked: true, ready: false, error: e.class.name, message: e.message)
|
|
123
133
|
end
|
|
124
134
|
|
|
125
135
|
def readiness(live: false)
|
|
136
|
+
log.debug { "bedrock.provider.readiness: checking (live=#{live})" }
|
|
126
137
|
health(live: live).merge(local: false, remote: true, api_base: api_base,
|
|
127
138
|
endpoints: endpoint_manifest).tap do |metadata|
|
|
128
139
|
self.class.registry_publisher.publish_readiness_async(metadata) if live
|
|
@@ -130,19 +141,16 @@ module Legion
|
|
|
130
141
|
end
|
|
131
142
|
|
|
132
143
|
def list_models
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
capabilities: offering.capabilities.map(&:to_s),
|
|
140
|
-
metadata: offering.to_h
|
|
141
|
-
)
|
|
142
|
-
end
|
|
144
|
+
log.info { 'bedrock.provider.list_models: fetching live model list' }
|
|
145
|
+
response = bedrock_client.list_foundation_models
|
|
146
|
+
models = Array(value(response, :model_summaries)).filter_map { |summary| model_info_from_summary(summary) }
|
|
147
|
+
log.info { "bedrock.provider.list_models: found #{models.size} models" }
|
|
148
|
+
self.class.registry_publisher.publish_models_async(models, readiness: readiness(live: false))
|
|
149
|
+
models
|
|
143
150
|
end
|
|
144
151
|
|
|
145
152
|
def chat(messages, model:, temperature: nil, max_tokens: nil, tools: {}, tool_prefs: nil, params: {})
|
|
153
|
+
log.info { "bedrock.provider.chat: model=#{model_id(model)} messages=#{messages.size}" }
|
|
146
154
|
request = Utils.deep_merge(
|
|
147
155
|
converse_request(messages, model:, temperature:, max_tokens:, tools:, tool_prefs:),
|
|
148
156
|
params
|
|
@@ -152,6 +160,7 @@ module Legion
|
|
|
152
160
|
|
|
153
161
|
def stream(messages, model:, temperature: nil, max_tokens: nil, tools: {}, tool_prefs: nil, params: {},
|
|
154
162
|
&)
|
|
163
|
+
log.info { "bedrock.provider.stream: model=#{model_id(model)} messages=#{messages.size}" }
|
|
155
164
|
request = Utils.deep_merge(
|
|
156
165
|
converse_request(messages, model:, temperature:, max_tokens:, tools:, tool_prefs:),
|
|
157
166
|
params
|
|
@@ -160,6 +169,7 @@ module Legion
|
|
|
160
169
|
end
|
|
161
170
|
|
|
162
171
|
def count_tokens(messages, model:, system: nil, params: {})
|
|
172
|
+
log.debug { "bedrock.provider.count_tokens: model=#{model_id(model)}" }
|
|
163
173
|
request = Utils.deep_merge(
|
|
164
174
|
{
|
|
165
175
|
model_id: model_id(model),
|
|
@@ -172,20 +182,21 @@ module Legion
|
|
|
172
182
|
end
|
|
173
183
|
|
|
174
184
|
def embed(text, model:, dimensions: nil)
|
|
175
|
-
|
|
176
|
-
unless titan_embed?(
|
|
185
|
+
mid = model_id(model)
|
|
186
|
+
unless titan_embed?(mid)
|
|
177
187
|
raise NotImplementedError,
|
|
178
|
-
"Bedrock embedding payload for #{
|
|
188
|
+
"Bedrock embedding payload for #{mid} is not standardized"
|
|
179
189
|
end
|
|
180
190
|
|
|
191
|
+
log.info { "bedrock.provider.embed: model=#{mid}" }
|
|
181
192
|
body = { inputText: text, dimensions: dimensions }.compact
|
|
182
193
|
response = runtime_client.invoke_model(
|
|
183
|
-
model_id:
|
|
194
|
+
model_id: mid,
|
|
184
195
|
content_type: 'application/json',
|
|
185
196
|
accept: 'application/json',
|
|
186
197
|
body: Legion::JSON.generate(body)
|
|
187
198
|
)
|
|
188
|
-
parse_embedding_response(response, model:
|
|
199
|
+
parse_embedding_response(response, model: mid)
|
|
189
200
|
end
|
|
190
201
|
|
|
191
202
|
def complete(messages, tools:, temperature:, model:, params: {}, headers: {}, schema: nil, thinking: nil, # rubocop:disable Lint/UnusedMethodArgument
|
|
@@ -225,6 +236,23 @@ module Legion
|
|
|
225
236
|
)
|
|
226
237
|
end
|
|
227
238
|
|
|
239
|
+
def model_info_from_summary(summary)
|
|
240
|
+
model = value(summary, :model_id)
|
|
241
|
+
input_mods = Array(value(summary, :input_modalities)).map { |m| m.to_s.downcase }
|
|
242
|
+
output_mods = Array(value(summary, :output_modalities)).map { |m| m.to_s.downcase }
|
|
243
|
+
|
|
244
|
+
Legion::Extensions::Llm::Model::Info.new(
|
|
245
|
+
id: model,
|
|
246
|
+
name: alias_for(model) || model,
|
|
247
|
+
provider: :bedrock,
|
|
248
|
+
family: (normalize_provider(value(summary, :provider_name)) || model_family_for(model)).to_s,
|
|
249
|
+
capabilities: capabilities_from_modalities(input_mods, output_mods, summary),
|
|
250
|
+
modalities_input: input_mods,
|
|
251
|
+
modalities_output: output_mods,
|
|
252
|
+
metadata: normalize_response(summary)
|
|
253
|
+
)
|
|
254
|
+
end
|
|
255
|
+
|
|
228
256
|
def build_offering(model:, model_family:, usage_type:, instance_id: :default, alias_name: nil,
|
|
229
257
|
capabilities: nil, metadata: {})
|
|
230
258
|
Legion::Extensions::Llm::Routing::ModelOffering.new(
|
|
@@ -461,6 +489,18 @@ module Legion
|
|
|
461
489
|
capabilities
|
|
462
490
|
end
|
|
463
491
|
|
|
492
|
+
def capabilities_from_modalities(input_mods, output_mods, summary)
|
|
493
|
+
caps = []
|
|
494
|
+
caps << :embedding if output_mods.include?('embedding')
|
|
495
|
+
unless caps.include?(:embedding)
|
|
496
|
+
caps << :completion
|
|
497
|
+
caps << :streaming if value(summary, :response_streaming_supported)
|
|
498
|
+
end
|
|
499
|
+
caps << :vision if input_mods.include?('image')
|
|
500
|
+
caps << :tools if caps.include?(:completion)
|
|
501
|
+
caps
|
|
502
|
+
end
|
|
503
|
+
|
|
464
504
|
def model_family_for(model)
|
|
465
505
|
normalize_provider(model.to_s.split('.').first)
|
|
466
506
|
end
|
|
@@ -2,8 +2,6 @@
|
|
|
2
2
|
|
|
3
3
|
require 'legion/extensions/llm'
|
|
4
4
|
require 'legion/extensions/llm/bedrock/provider'
|
|
5
|
-
require 'legion/extensions/llm/bedrock/registry_event_builder'
|
|
6
|
-
require 'legion/extensions/llm/bedrock/registry_publisher'
|
|
7
5
|
require 'legion/extensions/llm/bedrock/version'
|
|
8
6
|
|
|
9
7
|
module Legion
|
|
@@ -16,34 +14,34 @@ module Legion
|
|
|
16
14
|
PROVIDER_FAMILY = :bedrock
|
|
17
15
|
|
|
18
16
|
def self.default_settings
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
},
|
|
34
|
-
usage: { inference: true, embedding: true, token_counting: true },
|
|
35
|
-
limits: { concurrency: 4 }
|
|
36
|
-
}
|
|
37
|
-
)
|
|
17
|
+
{
|
|
18
|
+
enabled: false,
|
|
19
|
+
default_model: 'us.anthropic.claude-sonnet-4-6',
|
|
20
|
+
region: 'us-east-2',
|
|
21
|
+
bearer_token: nil,
|
|
22
|
+
api_key: nil,
|
|
23
|
+
secret_key: nil,
|
|
24
|
+
session_token: nil,
|
|
25
|
+
model_whitelist: [],
|
|
26
|
+
model_blacklist: [],
|
|
27
|
+
model_cache_ttl: 3600,
|
|
28
|
+
tls: { enabled: false, verify: :peer },
|
|
29
|
+
instances: {}
|
|
30
|
+
}
|
|
38
31
|
end
|
|
39
32
|
|
|
40
33
|
def self.provider_class
|
|
41
34
|
Provider
|
|
42
35
|
end
|
|
36
|
+
|
|
37
|
+
def self.registry_publisher
|
|
38
|
+
@registry_publisher ||= Legion::Extensions::Llm::RegistryPublisher.new(provider_family: PROVIDER_FAMILY)
|
|
39
|
+
end
|
|
43
40
|
end
|
|
44
41
|
end
|
|
45
42
|
end
|
|
46
43
|
end
|
|
47
44
|
|
|
48
|
-
Legion::Extensions::Llm::
|
|
49
|
-
|
|
45
|
+
Legion::Extensions::Llm::Configuration.register_provider_options(
|
|
46
|
+
Legion::Extensions::Llm::Bedrock::Provider.configuration_options
|
|
47
|
+
)
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: lex-llm-bedrock
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.2.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- LegionIO
|
|
@@ -85,14 +85,14 @@ dependencies:
|
|
|
85
85
|
requirements:
|
|
86
86
|
- - ">="
|
|
87
87
|
- !ruby/object:Gem::Version
|
|
88
|
-
version: 0.
|
|
88
|
+
version: 0.2.0
|
|
89
89
|
type: :runtime
|
|
90
90
|
prerelease: false
|
|
91
91
|
version_requirements: !ruby/object:Gem::Requirement
|
|
92
92
|
requirements:
|
|
93
93
|
- - ">="
|
|
94
94
|
- !ruby/object:Gem::Version
|
|
95
|
-
version: 0.
|
|
95
|
+
version: 0.2.0
|
|
96
96
|
description: Amazon Bedrock provider integration for the LegionIO LLM routing framework.
|
|
97
97
|
email:
|
|
98
98
|
- matthewdiverson@gmail.com
|
|
@@ -112,10 +112,6 @@ files:
|
|
|
112
112
|
- lex-llm-bedrock.gemspec
|
|
113
113
|
- lib/legion/extensions/llm/bedrock.rb
|
|
114
114
|
- lib/legion/extensions/llm/bedrock/provider.rb
|
|
115
|
-
- lib/legion/extensions/llm/bedrock/registry_event_builder.rb
|
|
116
|
-
- lib/legion/extensions/llm/bedrock/registry_publisher.rb
|
|
117
|
-
- lib/legion/extensions/llm/bedrock/transport/exchanges/llm_registry.rb
|
|
118
|
-
- lib/legion/extensions/llm/bedrock/transport/messages/registry_event.rb
|
|
119
115
|
- lib/legion/extensions/llm/bedrock/version.rb
|
|
120
116
|
homepage: https://github.com/LegionIO/lex-llm-bedrock
|
|
121
117
|
licenses:
|
|
@@ -1,93 +0,0 @@
|
|
|
1
|
-
# frozen_string_literal: true
|
|
2
|
-
|
|
3
|
-
module Legion
|
|
4
|
-
module Extensions
|
|
5
|
-
module Llm
|
|
6
|
-
module Bedrock
|
|
7
|
-
# Builds sanitized lex-llm registry envelopes for Bedrock provider state.
|
|
8
|
-
class RegistryEventBuilder
|
|
9
|
-
def readiness(readiness)
|
|
10
|
-
registry_event_class.public_send(
|
|
11
|
-
readiness[:ready] ? :available : :unavailable,
|
|
12
|
-
provider_offering(readiness),
|
|
13
|
-
runtime: runtime_metadata,
|
|
14
|
-
health: readiness_health(readiness),
|
|
15
|
-
metadata: readiness_metadata(readiness)
|
|
16
|
-
)
|
|
17
|
-
end
|
|
18
|
-
|
|
19
|
-
def offering_available(offering, readiness:)
|
|
20
|
-
registry_event_class.available(
|
|
21
|
-
offering,
|
|
22
|
-
runtime: runtime_metadata,
|
|
23
|
-
health: offering_health(readiness),
|
|
24
|
-
metadata: offering_metadata
|
|
25
|
-
)
|
|
26
|
-
end
|
|
27
|
-
|
|
28
|
-
private
|
|
29
|
-
|
|
30
|
-
def provider_offering(readiness)
|
|
31
|
-
{
|
|
32
|
-
provider_family: :bedrock,
|
|
33
|
-
provider_instance: provider_instance,
|
|
34
|
-
transport: :aws_sdk,
|
|
35
|
-
model: 'provider-readiness',
|
|
36
|
-
usage_type: :inference,
|
|
37
|
-
capabilities: [],
|
|
38
|
-
health: readiness_health(readiness),
|
|
39
|
-
metadata: { lex: :llm_bedrock, provider_readiness: true }
|
|
40
|
-
}
|
|
41
|
-
end
|
|
42
|
-
|
|
43
|
-
def readiness_health(readiness)
|
|
44
|
-
health = {
|
|
45
|
-
ready: readiness[:ready] == true,
|
|
46
|
-
status: readiness[:ready] ? :available : :unavailable,
|
|
47
|
-
checked: readiness[:checked] != false
|
|
48
|
-
}
|
|
49
|
-
add_readiness_error(health, readiness)
|
|
50
|
-
end
|
|
51
|
-
|
|
52
|
-
def add_readiness_error(health, source)
|
|
53
|
-
error_class = source[:error] || source['error']
|
|
54
|
-
error_message = source[:message] || source['message']
|
|
55
|
-
health[:error_class] = error_class if error_class
|
|
56
|
-
health[:error] = error_message if error_message
|
|
57
|
-
health
|
|
58
|
-
end
|
|
59
|
-
|
|
60
|
-
def offering_health(readiness)
|
|
61
|
-
ready = readiness.fetch(:ready, true) == true
|
|
62
|
-
{ ready:, status: ready ? :available : :degraded, checked: readiness[:checked] != false }
|
|
63
|
-
end
|
|
64
|
-
|
|
65
|
-
def readiness_metadata(readiness)
|
|
66
|
-
{
|
|
67
|
-
extension: :lex_llm_bedrock,
|
|
68
|
-
provider: :bedrock,
|
|
69
|
-
configured: readiness[:configured] == true,
|
|
70
|
-
live: readiness[:live] == true
|
|
71
|
-
}
|
|
72
|
-
end
|
|
73
|
-
|
|
74
|
-
def offering_metadata
|
|
75
|
-
{ extension: :lex_llm_bedrock, provider: :bedrock }
|
|
76
|
-
end
|
|
77
|
-
|
|
78
|
-
def runtime_metadata
|
|
79
|
-
{ node: provider_instance }
|
|
80
|
-
end
|
|
81
|
-
|
|
82
|
-
def provider_instance
|
|
83
|
-
:bedrock
|
|
84
|
-
end
|
|
85
|
-
|
|
86
|
-
def registry_event_class
|
|
87
|
-
::Legion::Extensions::Llm::Routing::RegistryEvent
|
|
88
|
-
end
|
|
89
|
-
end
|
|
90
|
-
end
|
|
91
|
-
end
|
|
92
|
-
end
|
|
93
|
-
end
|
|
@@ -1,100 +0,0 @@
|
|
|
1
|
-
# frozen_string_literal: true
|
|
2
|
-
|
|
3
|
-
module Legion
|
|
4
|
-
module Extensions
|
|
5
|
-
module Llm
|
|
6
|
-
module Bedrock
|
|
7
|
-
# Best-effort publisher for Bedrock provider availability events.
|
|
8
|
-
class RegistryPublisher
|
|
9
|
-
APP_ID = 'lex-llm-bedrock'
|
|
10
|
-
|
|
11
|
-
def initialize(builder: RegistryEventBuilder.new)
|
|
12
|
-
@builder = builder
|
|
13
|
-
end
|
|
14
|
-
|
|
15
|
-
def publish_readiness_async(readiness)
|
|
16
|
-
schedule { publish_event(@builder.readiness(readiness)) }
|
|
17
|
-
end
|
|
18
|
-
|
|
19
|
-
def publish_offerings_async(offerings, readiness:)
|
|
20
|
-
schedule do
|
|
21
|
-
Array(offerings).each do |offering|
|
|
22
|
-
publish_event(@builder.offering_available(offering, readiness:))
|
|
23
|
-
end
|
|
24
|
-
end
|
|
25
|
-
end
|
|
26
|
-
|
|
27
|
-
private
|
|
28
|
-
|
|
29
|
-
def schedule(&)
|
|
30
|
-
return false unless publishing_available?
|
|
31
|
-
|
|
32
|
-
Thread.new do
|
|
33
|
-
Thread.current.abort_on_exception = false
|
|
34
|
-
yield
|
|
35
|
-
rescue StandardError => e
|
|
36
|
-
log_publish_failure(e, level: :debug)
|
|
37
|
-
end
|
|
38
|
-
rescue StandardError => e
|
|
39
|
-
log_publish_failure(e, level: :debug)
|
|
40
|
-
false
|
|
41
|
-
end
|
|
42
|
-
|
|
43
|
-
def publish_event(event)
|
|
44
|
-
return false unless publishing_available?
|
|
45
|
-
|
|
46
|
-
message_class.new(event:, app_id: APP_ID).publish(spool: false)
|
|
47
|
-
rescue StandardError => e
|
|
48
|
-
log_publish_failure(e)
|
|
49
|
-
false
|
|
50
|
-
end
|
|
51
|
-
|
|
52
|
-
def publishing_available?
|
|
53
|
-
return false unless registry_event_available?
|
|
54
|
-
return false unless transport_message_available?
|
|
55
|
-
return true unless defined?(::Legion::Transport::Connection)
|
|
56
|
-
return true unless ::Legion::Transport::Connection.respond_to?(:session_open?)
|
|
57
|
-
|
|
58
|
-
::Legion::Transport::Connection.session_open?
|
|
59
|
-
rescue StandardError
|
|
60
|
-
false
|
|
61
|
-
end
|
|
62
|
-
|
|
63
|
-
def registry_event_available?
|
|
64
|
-
defined?(::Legion::Extensions::Llm::Routing::RegistryEvent)
|
|
65
|
-
end
|
|
66
|
-
|
|
67
|
-
def transport_message_available?
|
|
68
|
-
return true if message_class_defined?
|
|
69
|
-
return false unless defined?(::Legion::Transport::Message) && defined?(::Legion::Transport::Exchange)
|
|
70
|
-
|
|
71
|
-
require 'legion/extensions/llm/bedrock/transport/messages/registry_event'
|
|
72
|
-
message_class_defined?
|
|
73
|
-
rescue LoadError
|
|
74
|
-
false
|
|
75
|
-
end
|
|
76
|
-
|
|
77
|
-
def message_class_defined?
|
|
78
|
-
defined?(::Legion::Extensions::Llm::Bedrock::Transport::Messages::RegistryEvent)
|
|
79
|
-
end
|
|
80
|
-
|
|
81
|
-
def message_class
|
|
82
|
-
::Legion::Extensions::Llm::Bedrock::Transport::Messages::RegistryEvent
|
|
83
|
-
end
|
|
84
|
-
|
|
85
|
-
def log_publish_failure(error, level: :warn)
|
|
86
|
-
message = "[lex-llm-bedrock] llm.registry publish failed: #{error.class}: #{error.message}"
|
|
87
|
-
logger = ::Legion::Extensions::Llm.logger if defined?(::Legion::Extensions::Llm)
|
|
88
|
-
if logger.respond_to?(level)
|
|
89
|
-
logger.public_send(level, message)
|
|
90
|
-
elsif logger.respond_to?(:debug)
|
|
91
|
-
logger.debug(message)
|
|
92
|
-
end
|
|
93
|
-
rescue StandardError
|
|
94
|
-
nil
|
|
95
|
-
end
|
|
96
|
-
end
|
|
97
|
-
end
|
|
98
|
-
end
|
|
99
|
-
end
|
|
100
|
-
end
|
|
@@ -1,24 +0,0 @@
|
|
|
1
|
-
# frozen_string_literal: true
|
|
2
|
-
|
|
3
|
-
module Legion
|
|
4
|
-
module Extensions
|
|
5
|
-
module Llm
|
|
6
|
-
module Bedrock
|
|
7
|
-
module Transport
|
|
8
|
-
module Exchanges
|
|
9
|
-
# Topic exchange for Bedrock provider availability events.
|
|
10
|
-
class LlmRegistry < ::Legion::Transport::Exchange
|
|
11
|
-
def exchange_name
|
|
12
|
-
'llm.registry'
|
|
13
|
-
end
|
|
14
|
-
|
|
15
|
-
def default_type
|
|
16
|
-
'topic'
|
|
17
|
-
end
|
|
18
|
-
end
|
|
19
|
-
end
|
|
20
|
-
end
|
|
21
|
-
end
|
|
22
|
-
end
|
|
23
|
-
end
|
|
24
|
-
end
|
|
@@ -1,42 +0,0 @@
|
|
|
1
|
-
# frozen_string_literal: true
|
|
2
|
-
|
|
3
|
-
require 'legion/extensions/llm/bedrock/transport/exchanges/llm_registry'
|
|
4
|
-
|
|
5
|
-
module Legion
|
|
6
|
-
module Extensions
|
|
7
|
-
module Llm
|
|
8
|
-
module Bedrock
|
|
9
|
-
module Transport
|
|
10
|
-
module Messages
|
|
11
|
-
# Publishes lex-llm RegistryEvent envelopes to the llm.registry exchange.
|
|
12
|
-
class RegistryEvent < ::Legion::Transport::Message
|
|
13
|
-
def initialize(event:, **options)
|
|
14
|
-
super(**event.to_h.merge(options))
|
|
15
|
-
end
|
|
16
|
-
|
|
17
|
-
def exchange
|
|
18
|
-
Transport::Exchanges::LlmRegistry
|
|
19
|
-
end
|
|
20
|
-
|
|
21
|
-
def routing_key
|
|
22
|
-
@options[:routing_key] || "llm.registry.#{@options.fetch(:event_type)}"
|
|
23
|
-
end
|
|
24
|
-
|
|
25
|
-
def type
|
|
26
|
-
'llm.registry.event'
|
|
27
|
-
end
|
|
28
|
-
|
|
29
|
-
def app_id
|
|
30
|
-
@options[:app_id] || RegistryPublisher::APP_ID
|
|
31
|
-
end
|
|
32
|
-
|
|
33
|
-
def persistent # rubocop:disable Naming/PredicateMethod
|
|
34
|
-
false
|
|
35
|
-
end
|
|
36
|
-
end
|
|
37
|
-
end
|
|
38
|
-
end
|
|
39
|
-
end
|
|
40
|
-
end
|
|
41
|
-
end
|
|
42
|
-
end
|