lex-llm-bedrock 0.3.0 → 0.3.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.github/workflows/ci.yml +13 -1
- data/CHANGELOG.md +37 -0
- data/Gemfile +2 -0
- data/README.md +40 -21
- data/lex-llm-bedrock.gemspec +2 -1
- data/lib/legion/extensions/llm/bedrock/actors/fleet_worker.rb +43 -0
- data/lib/legion/extensions/llm/bedrock/provider.rb +39 -11
- data/lib/legion/extensions/llm/bedrock/runners/fleet_worker.rb +30 -0
- data/lib/legion/extensions/llm/bedrock/version.rb +1 -1
- data/lib/legion/extensions/llm/bedrock.rb +73 -25
- metadata +19 -3
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: ba5cbf8b2adeb7ce05f969bfef32582c03d238febed07790345841d1cf038eba
|
|
4
|
+
data.tar.gz: 96fd62bc8575afa30a836c084e3d4f8745dfd59566f7210ec6226f4a466444f5
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 4cf60483077635545837818151d76674d3a8f69550dce991f3c8def75f1c9fabe882b30efedbeffe57c051040b53e7b473a92d34199c9edf7da2f6441fecddcc
|
|
7
|
+
data.tar.gz: 9098f7257c42b29f48ea2cb6d212ef737b8473f7d53e3f9bddedbde7e4be7c2887ce1d82e998905fbf86621975537fe49bd3181792d8a23925dbada75fa5fc33
|
data/.github/workflows/ci.yml
CHANGED
|
@@ -8,8 +8,20 @@ jobs:
|
|
|
8
8
|
ci:
|
|
9
9
|
uses: LegionIO/.github/.github/workflows/ci.yml@main
|
|
10
10
|
|
|
11
|
+
excluded-files:
|
|
12
|
+
uses: LegionIO/.github/.github/workflows/excluded-files.yml@main
|
|
13
|
+
|
|
14
|
+
security:
|
|
15
|
+
uses: LegionIO/.github/.github/workflows/security-scan.yml@main
|
|
16
|
+
|
|
17
|
+
version-changelog:
|
|
18
|
+
uses: LegionIO/.github/.github/workflows/version-changelog.yml@main
|
|
19
|
+
|
|
20
|
+
dependency-review:
|
|
21
|
+
uses: LegionIO/.github/.github/workflows/dependency-review.yml@main
|
|
22
|
+
|
|
11
23
|
release:
|
|
12
|
-
needs: ci
|
|
24
|
+
needs: [ci, excluded-files, security]
|
|
13
25
|
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
|
|
14
26
|
uses: LegionIO/.github/.github/workflows/release.yml@main
|
|
15
27
|
secrets:
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,42 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.3.6 - 2026-05-08
|
|
4
|
+
|
|
5
|
+
- Accept keyword arguments in `list_models` to match the base provider contract called by `discover_offerings`.
|
|
6
|
+
|
|
7
|
+
## 0.3.5 - 2026-05-06
|
|
8
|
+
|
|
9
|
+
- Load provider-owned fleet actors through the LegionIO subscription base and the canonical Bedrock provider root.
|
|
10
|
+
- Keep fleet runners anchored on the provider root namespace so provider constants and instance discovery are always loaded.
|
|
11
|
+
- Preserve configured transport and tier metadata when Bedrock builds routing offerings.
|
|
12
|
+
- Strip temporary generic API key fields from discovered Bedrock instance configs after credential deduplication.
|
|
13
|
+
- Clean up provider method signatures and README examples from Copilot review feedback.
|
|
14
|
+
- Gate release publishing on the shared security workflow.
|
|
15
|
+
|
|
16
|
+
## 0.3.4 - 2026-05-06
|
|
17
|
+
|
|
18
|
+
- Use the shared `lex-llm` fleet provider responder helper for provider-owned fleet workers.
|
|
19
|
+
- Remove the runtime `legion-llm` dependency and require `lex-llm >= 0.4.3` for responder-side fleet execution.
|
|
20
|
+
- Refresh README architecture, file map, fleet responder, and development verification guidance for the current provider-owned fleet implementation.
|
|
21
|
+
- Silence test logging so the required full-suite RSpec gate writes only to the configured output files.
|
|
22
|
+
|
|
23
|
+
## 0.3.3 - 2026-05-06
|
|
24
|
+
|
|
25
|
+
- Remove require-time provider self-registration; `legion-llm` now owns adapter creation and registry writes from loaded provider discovery metadata.
|
|
26
|
+
- Bump dependency floors to `lex-llm >= 0.4.1` and `legion-llm >= 0.9.1`.
|
|
27
|
+
|
|
28
|
+
## 0.3.2 - 2026-05-06
|
|
29
|
+
|
|
30
|
+
- Enforce the shared keyword-only `lex-llm` provider contract for chat, streaming, embeddings, and token counting.
|
|
31
|
+
- Move defaults back to `Legion::Extensions::Llm.provider_settings` with AWS credentials/provider metadata under the default instance and instance-level fleet responder settings.
|
|
32
|
+
- Add provider-owned fleet responder actor and runner backed by `legion-llm` fleet policy execution.
|
|
33
|
+
- Bump the transport dependency floor to `legion-transport >= 1.4.14`.
|
|
34
|
+
|
|
35
|
+
## 0.3.1 - 2026-05-03
|
|
36
|
+
|
|
37
|
+
- Normalize generic settings keys to Bedrock provider config keys during instance discovery.
|
|
38
|
+
- Support named Bedrock instances from extension settings.
|
|
39
|
+
|
|
3
40
|
## 0.3.0 - 2026-05-01
|
|
4
41
|
|
|
5
42
|
- Add auto-discovery via CredentialSources and AutoRegistration from lex-llm 0.3.0
|
data/Gemfile
CHANGED
|
@@ -4,6 +4,8 @@ source 'https://rubygems.org'
|
|
|
4
4
|
|
|
5
5
|
group :test do
|
|
6
6
|
llm_base_path = ENV.fetch('LEX_LLM_PATH', File.expand_path('../lex-llm', __dir__))
|
|
7
|
+
transport_path = ENV.fetch('LEGION_TRANSPORT_PATH', File.expand_path('../../legion-transport', __dir__))
|
|
8
|
+
gem 'legion-transport', path: transport_path if File.directory?(transport_path)
|
|
7
9
|
gem 'lex-llm', path: llm_base_path if File.directory?(llm_base_path)
|
|
8
10
|
end
|
|
9
11
|
|
data/README.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
Amazon Bedrock provider extension for `Legion::Extensions::Llm`.
|
|
4
4
|
|
|
5
|
-
This gem adds a hosted Bedrock provider surface for Legion LLM routing
|
|
5
|
+
This gem adds a hosted Bedrock provider surface for Legion LLM routing. It uses the official AWS SDK for Ruby and keeps discovery offline by default, so loading the extension or running tests does not require live AWS credentials. It requires `lex-llm >= 0.4.3` for the shared provider contract, response normalization, model offering, readiness, fleet envelope contract, and provider-owned fleet responder execution.
|
|
6
6
|
|
|
7
7
|
## Architecture
|
|
8
8
|
|
|
@@ -16,11 +16,8 @@ Legion::Extensions::Llm::Bedrock
|
|
|
16
16
|
│ ├── discover_offerings # Static catalog + live ListFoundationModels
|
|
17
17
|
│ ├── health / readiness # Provider health checks with live AWS verification
|
|
18
18
|
│ └── list_models # Live model enumeration
|
|
19
|
-
├──
|
|
20
|
-
|
|
21
|
-
└── Transport
|
|
22
|
-
├── Exchanges::LlmRegistry # Topic exchange for llm.registry events
|
|
23
|
-
└── Messages::RegistryEvent # AMQP message for registry event publishing
|
|
19
|
+
├── Actor::FleetWorker # Provider-owned fleet subscription gate
|
|
20
|
+
└── Runners::FleetWorker # Delegates fleet requests to lex-llm ProviderResponder
|
|
24
21
|
```
|
|
25
22
|
|
|
26
23
|
## Dependencies
|
|
@@ -32,18 +29,17 @@ Legion::Extensions::Llm::Bedrock
|
|
|
32
29
|
| `legion-json` (>= 1.2.1) | Yes | JSON serialization |
|
|
33
30
|
| `legion-logging` (>= 1.3.2) | Yes | Structured logging via Helper |
|
|
34
31
|
| `legion-settings` (>= 1.3.14) | Yes | Configuration |
|
|
35
|
-
| `lex-llm` (>= 0.
|
|
32
|
+
| `lex-llm` (>= 0.4.3) | Yes | Shared provider contract, response normalization, model offerings, fleet envelopes, and fleet responder execution |
|
|
33
|
+
| `legion-transport` (>= 1.4.14) | Yes | AMQP subscriptions and replies |
|
|
36
34
|
|
|
37
35
|
## File Map
|
|
38
36
|
|
|
39
37
|
| Path | Purpose |
|
|
40
38
|
|------|---------|
|
|
41
|
-
| `lib/legion/extensions/llm/bedrock.rb` | Entry point: namespace, default settings, provider registration |
|
|
39
|
+
| `lib/legion/extensions/llm/bedrock.rb` | Entry point: namespace, default settings, discovery, and shared provider registration metadata |
|
|
42
40
|
| `lib/legion/extensions/llm/bedrock/provider.rb` | Full Bedrock provider implementation |
|
|
43
|
-
| `lib/legion/extensions/llm/bedrock/
|
|
44
|
-
| `lib/legion/extensions/llm/bedrock/
|
|
45
|
-
| `lib/legion/extensions/llm/bedrock/transport/exchanges/llm_registry.rb` | AMQP topic exchange definition |
|
|
46
|
-
| `lib/legion/extensions/llm/bedrock/transport/messages/registry_event.rb` | AMQP message class for registry events |
|
|
41
|
+
| `lib/legion/extensions/llm/bedrock/actors/fleet_worker.rb` | Starts the provider-owned fleet subscriber when an instance opts in |
|
|
42
|
+
| `lib/legion/extensions/llm/bedrock/runners/fleet_worker.rb` | Hands provider fleet requests to `Legion::Extensions::Llm::Fleet::ProviderResponder` |
|
|
47
43
|
| `lib/legion/extensions/llm/bedrock/version.rb` | `VERSION` constant |
|
|
48
44
|
|
|
49
45
|
## Install
|
|
@@ -67,7 +63,7 @@ Legion::Extensions::Llm.configure do |config|
|
|
|
67
63
|
end
|
|
68
64
|
```
|
|
69
65
|
|
|
70
|
-
If explicit keys are not configured, the AWS SDK default credential provider chain is used. Default settings
|
|
66
|
+
If explicit keys are not configured, the AWS SDK default credential provider chain is used. Default settings define the Bedrock provider family, default instance metadata, AWS credential slots, and opt-in fleet responder controls:
|
|
71
67
|
|
|
72
68
|
```ruby
|
|
73
69
|
Legion::Extensions::Llm::Bedrock.default_settings
|
|
@@ -75,6 +71,27 @@ Legion::Extensions::Llm::Bedrock.default_settings
|
|
|
75
71
|
|
|
76
72
|
Configuration options: `bedrock_region`, `bedrock_endpoint`, `bedrock_access_key_id`, `bedrock_secret_access_key`, `bedrock_session_token`, `bedrock_profile`, `bedrock_stub_responses`.
|
|
77
73
|
|
|
74
|
+
## Fleet Responder
|
|
75
|
+
|
|
76
|
+
Provider instances can opt in to consuming Legion LLM fleet requests. The provider-owned fleet actor only starts when at least one configured instance enables `respond_to_requests`.
|
|
77
|
+
|
|
78
|
+
```yaml
|
|
79
|
+
extensions:
|
|
80
|
+
llm:
|
|
81
|
+
bedrock:
|
|
82
|
+
instances:
|
|
83
|
+
local:
|
|
84
|
+
fleet:
|
|
85
|
+
enabled: true
|
|
86
|
+
respond_to_requests: true
|
|
87
|
+
capabilities:
|
|
88
|
+
- chat
|
|
89
|
+
- stream_chat
|
|
90
|
+
- embed
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
Fleet execution stays inside this provider extension until the final handoff to `lex-llm`'s shared `ProviderResponder` helper. This gem does not depend on `legion-llm` at runtime.
|
|
94
|
+
|
|
78
95
|
## Provider Surface
|
|
79
96
|
|
|
80
97
|
```ruby
|
|
@@ -83,10 +100,12 @@ provider = Legion::Extensions::Llm::Bedrock::Provider.new(Legion::Extensions::Ll
|
|
|
83
100
|
provider.discover_offerings(live: false)
|
|
84
101
|
provider.offering_for(model: 'anthropic.claude-3-haiku-20240307-v1:0')
|
|
85
102
|
provider.health(live: false)
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
provider.
|
|
89
|
-
provider.
|
|
103
|
+
messages = [Legion::Extensions::Llm::Message.new(role: :user, content: 'hello')]
|
|
104
|
+
model = 'anthropic.claude-3-haiku-20240307-v1:0'
|
|
105
|
+
provider.chat(messages: messages, model: model)
|
|
106
|
+
provider.stream(messages: messages, model: model) { |chunk| chunk.content }
|
|
107
|
+
provider.embed(text: 'hello', model: 'amazon.titan-embed-text-v2:0')
|
|
108
|
+
provider.count_tokens(messages: messages, model: model)
|
|
90
109
|
```
|
|
91
110
|
|
|
92
111
|
`discover_offerings(live: false)` returns a small static catalog that is useful for routing defaults and unit tests. `discover_offerings(live: true)` calls Bedrock `ListFoundationModels` and maps the returned model summaries into `Legion::Extensions::Llm::Routing::ModelOffering` records.
|
|
@@ -118,17 +137,17 @@ Provider-specific request bodies are not guessed. Non-Titan embedding models rai
|
|
|
118
137
|
|
|
119
138
|
## Observability
|
|
120
139
|
|
|
121
|
-
|
|
140
|
+
The Bedrock namespace and provider implementation include `Legion::Logging::Helper` for structured logging:
|
|
122
141
|
|
|
123
142
|
- **Info-level**: provider connections, API calls (chat, stream, embed), model listing, health checks
|
|
124
|
-
- **Debug-level**: offline health checks, readiness probes, token counting
|
|
125
|
-
- **Rescue blocks**:
|
|
143
|
+
- **Debug-level**: offline health checks, readiness probes, and token counting
|
|
144
|
+
- **Rescue blocks**: handled provider failures call `handle_exception(e, level:, handled:, operation:)` with dot-separated operation names such as `bedrock.provider.health`
|
|
126
145
|
|
|
127
146
|
## Development
|
|
128
147
|
|
|
129
148
|
```bash
|
|
130
149
|
bundle install
|
|
131
|
-
bundle exec rspec --format progress
|
|
150
|
+
bundle exec rspec --format json --out tmp/rspec_results.json --format progress --out tmp/rspec_progress.txt
|
|
132
151
|
bundle exec rubocop -A # auto-fix
|
|
133
152
|
bundle exec rubocop # lint check (0 offenses expected)
|
|
134
153
|
```
|
data/lex-llm-bedrock.gemspec
CHANGED
|
@@ -28,5 +28,6 @@ Gem::Specification.new do |spec|
|
|
|
28
28
|
spec.add_dependency 'legion-json', '>= 1.2.1'
|
|
29
29
|
spec.add_dependency 'legion-logging', '>= 1.3.2'
|
|
30
30
|
spec.add_dependency 'legion-settings', '>= 1.3.14'
|
|
31
|
-
spec.add_dependency '
|
|
31
|
+
spec.add_dependency 'legion-transport', '>= 1.4.14'
|
|
32
|
+
spec.add_dependency 'lex-llm', '>= 0.4.3'
|
|
32
33
|
end
|
|
@@ -0,0 +1,43 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
begin
|
|
4
|
+
require 'legion/extensions/actors/subscription'
|
|
5
|
+
rescue LoadError => e
|
|
6
|
+
warn(e.message) if $VERBOSE
|
|
7
|
+
end
|
|
8
|
+
|
|
9
|
+
unless defined?(Legion::Extensions::Actors::Subscription)
|
|
10
|
+
raise LoadError, 'LegionIO actor runtime is required for Bedrock fleet worker'
|
|
11
|
+
end
|
|
12
|
+
|
|
13
|
+
require 'legion/extensions/llm/bedrock'
|
|
14
|
+
require 'legion/extensions/llm/fleet/provider_responder'
|
|
15
|
+
|
|
16
|
+
module Legion
|
|
17
|
+
module Extensions
|
|
18
|
+
module Llm
|
|
19
|
+
module Bedrock
|
|
20
|
+
module Actor
|
|
21
|
+
# Subscription actor for Bedrock fleet request consumption.
|
|
22
|
+
class FleetWorker < Legion::Extensions::Actors::Subscription
|
|
23
|
+
def runner_class
|
|
24
|
+
'Legion::Extensions::Llm::Bedrock::Runners::FleetWorker'
|
|
25
|
+
end
|
|
26
|
+
|
|
27
|
+
def runner_function
|
|
28
|
+
'handle_fleet_request'
|
|
29
|
+
end
|
|
30
|
+
|
|
31
|
+
def use_runner?
|
|
32
|
+
false
|
|
33
|
+
end
|
|
34
|
+
|
|
35
|
+
def enabled?
|
|
36
|
+
Legion::Extensions::Llm::Fleet::ProviderResponder.enabled_for?(Bedrock.discover_instances)
|
|
37
|
+
end
|
|
38
|
+
end
|
|
39
|
+
end
|
|
40
|
+
end
|
|
41
|
+
end
|
|
42
|
+
end
|
|
43
|
+
end
|
|
@@ -140,7 +140,7 @@ module Legion
|
|
|
140
140
|
end
|
|
141
141
|
end
|
|
142
142
|
|
|
143
|
-
def list_models
|
|
143
|
+
def list_models(**)
|
|
144
144
|
log.info { 'bedrock.provider.list_models: fetching live model list' }
|
|
145
145
|
response = bedrock_client.list_foundation_models
|
|
146
146
|
models = Array(value(response, :model_summaries)).filter_map { |summary| model_info_from_summary(summary) }
|
|
@@ -149,7 +149,16 @@ module Legion
|
|
|
149
149
|
models
|
|
150
150
|
end
|
|
151
151
|
|
|
152
|
-
def chat(
|
|
152
|
+
def chat(
|
|
153
|
+
messages:,
|
|
154
|
+
model:,
|
|
155
|
+
temperature: nil,
|
|
156
|
+
max_tokens: nil,
|
|
157
|
+
tools: {},
|
|
158
|
+
tool_prefs: nil,
|
|
159
|
+
params: {},
|
|
160
|
+
**_provider_options
|
|
161
|
+
)
|
|
153
162
|
log.info { "bedrock.provider.chat: model=#{model_id(model)} messages=#{messages.size}" }
|
|
154
163
|
request = Utils.deep_merge(
|
|
155
164
|
converse_request(messages, model:, temperature:, max_tokens:, tools:, tool_prefs:),
|
|
@@ -158,8 +167,8 @@ module Legion
|
|
|
158
167
|
parse_converse_response(runtime_client.converse(**request), model_id(model))
|
|
159
168
|
end
|
|
160
169
|
|
|
161
|
-
def stream(messages
|
|
162
|
-
&)
|
|
170
|
+
def stream(messages:, model:, temperature: nil, max_tokens: nil, tools: {}, tool_prefs: nil, params: {},
|
|
171
|
+
**_provider_options, &)
|
|
163
172
|
log.info { "bedrock.provider.stream: model=#{model_id(model)} messages=#{messages.size}" }
|
|
164
173
|
request = Utils.deep_merge(
|
|
165
174
|
converse_request(messages, model:, temperature:, max_tokens:, tools:, tool_prefs:),
|
|
@@ -168,7 +177,12 @@ module Legion
|
|
|
168
177
|
stream_converse(request, model_id(model), &)
|
|
169
178
|
end
|
|
170
179
|
|
|
171
|
-
def count_tokens(
|
|
180
|
+
def count_tokens(
|
|
181
|
+
messages:,
|
|
182
|
+
model:,
|
|
183
|
+
system: nil,
|
|
184
|
+
params: {}
|
|
185
|
+
)
|
|
172
186
|
log.debug { "bedrock.provider.count_tokens: model=#{model_id(model)}" }
|
|
173
187
|
request = Utils.deep_merge(
|
|
174
188
|
{
|
|
@@ -181,7 +195,13 @@ module Legion
|
|
|
181
195
|
{ input_tokens: value(response, :input_tokens), raw: normalize_response(response) }
|
|
182
196
|
end
|
|
183
197
|
|
|
184
|
-
def embed(
|
|
198
|
+
def embed(
|
|
199
|
+
text:,
|
|
200
|
+
model:,
|
|
201
|
+
dimensions: nil,
|
|
202
|
+
params: {},
|
|
203
|
+
**_provider_options
|
|
204
|
+
)
|
|
185
205
|
mid = model_id(model)
|
|
186
206
|
unless titan_embed?(mid)
|
|
187
207
|
raise NotImplementedError,
|
|
@@ -189,7 +209,7 @@ module Legion
|
|
|
189
209
|
end
|
|
190
210
|
|
|
191
211
|
log.info { "bedrock.provider.embed: model=#{mid}" }
|
|
192
|
-
body = { inputText: text, dimensions: dimensions }.compact
|
|
212
|
+
body = Utils.deep_merge({ inputText: text, dimensions: dimensions }.compact, params)
|
|
193
213
|
response = runtime_client.invoke_model(
|
|
194
214
|
model_id: mid,
|
|
195
215
|
content_type: 'application/json',
|
|
@@ -207,9 +227,9 @@ module Legion
|
|
|
207
227
|
payload[:additional_model_request_fields][:response_format] = schema if schema
|
|
208
228
|
|
|
209
229
|
if block_given?
|
|
210
|
-
stream(messages
|
|
230
|
+
stream(messages:, model:, temperature:, tools:, tool_prefs:, params: payload, &)
|
|
211
231
|
else
|
|
212
|
-
chat(messages
|
|
232
|
+
chat(messages:, model:, temperature:, tools:, tool_prefs:, params: payload)
|
|
213
233
|
end
|
|
214
234
|
end
|
|
215
235
|
|
|
@@ -258,8 +278,8 @@ module Legion
|
|
|
258
278
|
Legion::Extensions::Llm::Routing::ModelOffering.new(
|
|
259
279
|
provider_family: :bedrock,
|
|
260
280
|
instance_id: instance_id,
|
|
261
|
-
transport: :aws_sdk,
|
|
262
|
-
tier: :frontier,
|
|
281
|
+
transport: configured_transport(:aws_sdk),
|
|
282
|
+
tier: configured_tier(:frontier),
|
|
263
283
|
model: model,
|
|
264
284
|
usage_type: usage_type,
|
|
265
285
|
capabilities: capabilities || default_capabilities(model),
|
|
@@ -267,6 +287,14 @@ module Legion
|
|
|
267
287
|
)
|
|
268
288
|
end
|
|
269
289
|
|
|
290
|
+
def configured_transport(default)
|
|
291
|
+
config.respond_to?(:transport) ? config.transport : default
|
|
292
|
+
end
|
|
293
|
+
|
|
294
|
+
def configured_tier(default)
|
|
295
|
+
config.respond_to?(:tier) ? config.tier : default
|
|
296
|
+
end
|
|
297
|
+
|
|
270
298
|
def converse_request(messages, model:, temperature:, max_tokens:, tools:, tool_prefs:)
|
|
271
299
|
{
|
|
272
300
|
model_id: model_id(model),
|
|
@@ -0,0 +1,30 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'legion/extensions/llm/fleet/provider_responder'
|
|
4
|
+
require 'legion/extensions/llm/bedrock'
|
|
5
|
+
|
|
6
|
+
module Legion
|
|
7
|
+
module Extensions
|
|
8
|
+
module Llm
|
|
9
|
+
module Bedrock
|
|
10
|
+
module Runners
|
|
11
|
+
# Runner entrypoint for Bedrock fleet request execution.
|
|
12
|
+
module FleetWorker
|
|
13
|
+
module_function
|
|
14
|
+
|
|
15
|
+
def handle_fleet_request(payload, delivery: nil, properties: nil)
|
|
16
|
+
Legion::Extensions::Llm::Fleet::ProviderResponder.call(
|
|
17
|
+
payload: payload,
|
|
18
|
+
provider_family: Bedrock::PROVIDER_FAMILY,
|
|
19
|
+
provider_class: Bedrock::Provider,
|
|
20
|
+
provider_instances: -> { Bedrock.discover_instances },
|
|
21
|
+
delivery: delivery,
|
|
22
|
+
properties: properties
|
|
23
|
+
)
|
|
24
|
+
end
|
|
25
|
+
end
|
|
26
|
+
end
|
|
27
|
+
end
|
|
28
|
+
end
|
|
29
|
+
end
|
|
30
|
+
end
|
|
@@ -8,7 +8,7 @@ module Legion
|
|
|
8
8
|
module Extensions
|
|
9
9
|
module Llm
|
|
10
10
|
# Amazon Bedrock provider extension namespace.
|
|
11
|
-
module Bedrock
|
|
11
|
+
module Bedrock # rubocop:disable Metrics/ModuleLength
|
|
12
12
|
extend ::Legion::Extensions::Core if ::Legion::Extensions.const_defined?(:Core, false)
|
|
13
13
|
extend Legion::Logging::Helper
|
|
14
14
|
extend Legion::Extensions::Llm::AutoRegistration
|
|
@@ -18,20 +18,36 @@ module Legion
|
|
|
18
18
|
DEFAULT_REGION = 'us-east-2'
|
|
19
19
|
|
|
20
20
|
def self.default_settings
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
21
|
+
::Legion::Extensions::Llm.provider_settings(
|
|
22
|
+
family: PROVIDER_FAMILY,
|
|
23
|
+
instance: {
|
|
24
|
+
default_model: 'us.anthropic.claude-sonnet-4-6',
|
|
25
|
+
tier: :frontier,
|
|
26
|
+
transport: :aws_sdk,
|
|
27
|
+
credentials: {
|
|
28
|
+
bearer_token: nil,
|
|
29
|
+
access_key_id: nil,
|
|
30
|
+
secret_access_key: nil,
|
|
31
|
+
session_token: nil,
|
|
32
|
+
profile: nil
|
|
33
|
+
},
|
|
34
|
+
provider: {
|
|
35
|
+
region: DEFAULT_REGION,
|
|
36
|
+
endpoint: nil,
|
|
37
|
+
stub_responses: false
|
|
38
|
+
},
|
|
39
|
+
usage: { inference: true, embedding: true, image: false },
|
|
40
|
+
limits: { concurrency: 4 },
|
|
41
|
+
fleet: {
|
|
42
|
+
enabled: false,
|
|
43
|
+
respond_to_requests: false,
|
|
44
|
+
capabilities: %i[chat stream_chat embed],
|
|
45
|
+
lanes: [],
|
|
46
|
+
concurrency: 4,
|
|
47
|
+
queue_suffix: nil
|
|
48
|
+
}
|
|
49
|
+
}
|
|
50
|
+
)
|
|
35
51
|
end
|
|
36
52
|
|
|
37
53
|
def self.provider_class
|
|
@@ -49,7 +65,7 @@ module Legion
|
|
|
49
65
|
discover_env_sigv4(candidates)
|
|
50
66
|
discover_settings(candidates)
|
|
51
67
|
discover_broker(candidates)
|
|
52
|
-
CredentialSources.dedup_credentials(candidates)
|
|
68
|
+
CredentialSources.dedup_credentials(candidates).transform_values { |config| sanitize_instance_config(config) }
|
|
53
69
|
end
|
|
54
70
|
|
|
55
71
|
def self.discover_env_bearer(candidates)
|
|
@@ -89,7 +105,16 @@ module Legion
|
|
|
89
105
|
|
|
90
106
|
def self.discover_settings(candidates)
|
|
91
107
|
settings = CredentialSources.setting(:extensions, :llm, :bedrock)
|
|
92
|
-
|
|
108
|
+
return unless settings.is_a?(Hash) && !settings.empty?
|
|
109
|
+
|
|
110
|
+
default_config = dedup_config(normalize_instance_config(settings))
|
|
111
|
+
candidates[:settings] = default_config.merge(tier: :cloud) unless default_config.empty?
|
|
112
|
+
|
|
113
|
+
settings_instances(settings).each do |name, config|
|
|
114
|
+
next unless config.is_a?(Hash)
|
|
115
|
+
|
|
116
|
+
candidates[name.to_sym] = dedup_config(normalize_instance_config(config)).merge(tier: :cloud)
|
|
117
|
+
end
|
|
93
118
|
end
|
|
94
119
|
|
|
95
120
|
def self.discover_broker(candidates)
|
|
@@ -128,15 +153,38 @@ module Legion
|
|
|
128
153
|
bedrock_session_token: creds[:session_token] || creds['session_token'],
|
|
129
154
|
bedrock_region: creds[:region] || creds['region'] || DEFAULT_REGION }.compact
|
|
130
155
|
end
|
|
156
|
+
|
|
157
|
+
def self.settings_instances(config)
|
|
158
|
+
instances = config[:instances] || config['instances']
|
|
159
|
+
instances.is_a?(Hash) ? instances : {}
|
|
160
|
+
end
|
|
161
|
+
|
|
162
|
+
def self.normalize_instance_config(config)
|
|
163
|
+
normalized = config.to_h.transform_keys { |key| key.respond_to?(:to_sym) ? key.to_sym : key }
|
|
164
|
+
normalized[:bedrock_region] ||= normalized.delete(:region)
|
|
165
|
+
normalized[:bedrock_endpoint] ||= normalized.delete(:endpoint)
|
|
166
|
+
normalized[:bedrock_endpoint] ||= normalized.delete(:base_url)
|
|
167
|
+
normalized[:bedrock_endpoint] ||= normalized.delete(:api_base)
|
|
168
|
+
normalized[:bedrock_access_key_id] ||= normalized.delete(:api_key) || normalized.delete(:access_key_id)
|
|
169
|
+
normalized[:bedrock_secret_access_key] ||= normalized.delete(:secret_key)
|
|
170
|
+
normalized[:bedrock_secret_access_key] ||= normalized.delete(:secret_access_key)
|
|
171
|
+
normalized[:bedrock_session_token] ||= normalized.delete(:session_token)
|
|
172
|
+
normalized[:bedrock_profile] ||= normalized.delete(:profile)
|
|
173
|
+
normalized.compact.except(:instances)
|
|
174
|
+
end
|
|
175
|
+
|
|
176
|
+
def self.dedup_config(config)
|
|
177
|
+
key = config[:bedrock_access_key_id]
|
|
178
|
+
key ? config.merge(api_key: key) : config
|
|
179
|
+
end
|
|
180
|
+
|
|
181
|
+
def self.sanitize_instance_config(config)
|
|
182
|
+
config.except(:api_key)
|
|
183
|
+
end
|
|
184
|
+
|
|
185
|
+
Legion::Extensions::Llm::Configuration.register_provider_options(Provider.configuration_options) if
|
|
186
|
+
Legion::Extensions::Llm::Configuration.respond_to?(:register_provider_options)
|
|
131
187
|
end
|
|
132
188
|
end
|
|
133
189
|
end
|
|
134
190
|
end
|
|
135
|
-
|
|
136
|
-
if Legion::Extensions::Llm::Configuration.respond_to?(:register_provider_options)
|
|
137
|
-
Legion::Extensions::Llm::Configuration.register_provider_options(
|
|
138
|
-
Legion::Extensions::Llm::Bedrock::Provider.configuration_options
|
|
139
|
-
)
|
|
140
|
-
end
|
|
141
|
-
|
|
142
|
-
Legion::Extensions::Llm::Bedrock.register_discovered_instances
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: lex-llm-bedrock
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.3.
|
|
4
|
+
version: 0.3.6
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- LegionIO
|
|
@@ -79,20 +79,34 @@ dependencies:
|
|
|
79
79
|
- - ">="
|
|
80
80
|
- !ruby/object:Gem::Version
|
|
81
81
|
version: 1.3.14
|
|
82
|
+
- !ruby/object:Gem::Dependency
|
|
83
|
+
name: legion-transport
|
|
84
|
+
requirement: !ruby/object:Gem::Requirement
|
|
85
|
+
requirements:
|
|
86
|
+
- - ">="
|
|
87
|
+
- !ruby/object:Gem::Version
|
|
88
|
+
version: 1.4.14
|
|
89
|
+
type: :runtime
|
|
90
|
+
prerelease: false
|
|
91
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
92
|
+
requirements:
|
|
93
|
+
- - ">="
|
|
94
|
+
- !ruby/object:Gem::Version
|
|
95
|
+
version: 1.4.14
|
|
82
96
|
- !ruby/object:Gem::Dependency
|
|
83
97
|
name: lex-llm
|
|
84
98
|
requirement: !ruby/object:Gem::Requirement
|
|
85
99
|
requirements:
|
|
86
100
|
- - ">="
|
|
87
101
|
- !ruby/object:Gem::Version
|
|
88
|
-
version: 0.3
|
|
102
|
+
version: 0.4.3
|
|
89
103
|
type: :runtime
|
|
90
104
|
prerelease: false
|
|
91
105
|
version_requirements: !ruby/object:Gem::Requirement
|
|
92
106
|
requirements:
|
|
93
107
|
- - ">="
|
|
94
108
|
- !ruby/object:Gem::Version
|
|
95
|
-
version: 0.3
|
|
109
|
+
version: 0.4.3
|
|
96
110
|
description: Amazon Bedrock provider integration for the LegionIO LLM routing framework.
|
|
97
111
|
email:
|
|
98
112
|
- matthewdiverson@gmail.com
|
|
@@ -111,7 +125,9 @@ files:
|
|
|
111
125
|
- README.md
|
|
112
126
|
- lex-llm-bedrock.gemspec
|
|
113
127
|
- lib/legion/extensions/llm/bedrock.rb
|
|
128
|
+
- lib/legion/extensions/llm/bedrock/actors/fleet_worker.rb
|
|
114
129
|
- lib/legion/extensions/llm/bedrock/provider.rb
|
|
130
|
+
- lib/legion/extensions/llm/bedrock/runners/fleet_worker.rb
|
|
115
131
|
- lib/legion/extensions/llm/bedrock/version.rb
|
|
116
132
|
homepage: https://github.com/LegionIO/lex-llm-bedrock
|
|
117
133
|
licenses:
|