lex-llm-vertex 0.2.16 → 0.3.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.rubocop.yml +2 -0
- data/CHANGELOG.md +77 -0
- data/Gemfile +2 -0
- data/lex-llm-vertex.gemspec +2 -2
- data/lib/legion/extensions/llm/vertex/actors/discovery_refresh.rb +581 -84
- data/lib/legion/extensions/llm/vertex/actors/fleet_worker.rb +7 -1
- data/lib/legion/extensions/llm/vertex/callable.rb +127 -0
- data/lib/legion/extensions/llm/vertex/provider.rb +27 -28
- data/lib/legion/extensions/llm/vertex/runners/fleet_worker.rb +9 -5
- data/lib/legion/extensions/llm/vertex/version.rb +1 -1
- data/lib/legion/extensions/llm/vertex.rb +2 -2
- metadata +6 -5
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: d0c10fb0ccc4de2f634ea0b53c4eac0e190aa9ebfb8e9fc05400fac745027244
|
|
4
|
+
data.tar.gz: 4f3799fa25e60a577cffcdbc74fc6dfa9c21ee8a0e5827c6477e9c14b474b9f1
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 1252b157643111b4803bc88d0e5d2a686edc39a7bf2c81611b1a651f46f8920f7a1ad93e153fe6342e1a79ada2cc12c0c273ab068daa46215d7ddddfd351f4ae
|
|
7
|
+
data.tar.gz: 2e9b123320a7d452ce6b76777fd2a830f0a633b36e060a2e6938e929c3f6fdd0cd0ccd5b75436dff4450134fc59f3ceeb36f5d1b9969642041906afa5cd43a16
|
data/.rubocop.yml
CHANGED
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,82 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## [0.3.3] - 2026-08-18
|
|
4
|
+
|
|
5
|
+
### Fixed
|
|
6
|
+
- Update SSOT v3 conformance coverage: `default` is a valid operator configuration
|
|
7
|
+
name for an instance, rather than a reserved identity.
|
|
8
|
+
|
|
9
|
+
## [0.3.2] - 2026-08-17
|
|
10
|
+
|
|
11
|
+
### Changed
|
|
12
|
+
- **SSOT v3 fail-forward identity** — Instance identity is now the operator's CONFIG NAME
|
|
13
|
+
(the key the router's `instances.<name>` settings lookups use); the derived
|
|
14
|
+
`{project}:{location}/{credential_fingerprint}` moves to the secondary `InstanceKey`
|
|
15
|
+
`physical_id` field (dedup/diagnostics only). Two config names pointing at the same
|
|
16
|
+
physical endpoint stay distinct instances. `DiscoveryRefreshConfigHelpers#derive_instance_id`
|
|
17
|
+
becomes `derive_physical_id`; all `Inventory::Publisher` calls carry the `physical_id:`
|
|
18
|
+
secondary field. Requires `lex-llm >= 0.7.1` (InstanceKey `physical_id` field).
|
|
19
|
+
- **Embedding models publish `chat: :unsupported`** — embedding offerings from the
|
|
20
|
+
STATIC_MODELS catalog publish `embed` as `:supported` and `chat`/`stream_chat`/
|
|
21
|
+
`count_tokens` as `:unsupported`, so a chat request can never be routed to an
|
|
22
|
+
embedding-only model; chat models publish `chat`/`stream_chat` `:supported`
|
|
23
|
+
(`count_tokens` gated on generate-content support).
|
|
24
|
+
- `lex-llm` dependency floor bumped to `>= 0.7.1` (InstanceKey `physical_id` field).
|
|
25
|
+
- Conformance/actor specs updated to the name-based identity with the secondary
|
|
26
|
+
physical-id field; the conformance harness fixture is now a name-keyed instance map.
|
|
27
|
+
- **Single actor registration** — the provider module no longer extends `Core` at
|
|
28
|
+
file level, so the boot-time submodule walk's `autobuild` gate skips it and the
|
|
29
|
+
gem's own top-level extension load is the sole actor registration (eliminates the
|
|
30
|
+
double-claim / FencedPublisherError from the daemon's dual boot-time build).
|
|
31
|
+
|
|
32
|
+
## [0.3.1] - 2026-08-13
|
|
33
|
+
|
|
34
|
+
### Changed
|
|
35
|
+
- Remove all inline `rubocop:disable` directives from lib/ and spec/; fix underlying offenses by
|
|
36
|
+
real refactoring: rename unused `headers:` kwarg to `_headers:` in `complete`, move spec files to
|
|
37
|
+
paths that match the described class (`capability_policy_spec.rb` → `provider_spec.rb`,
|
|
38
|
+
`actors/fleet_worker_spec.rb` → `actor/fleet_worker_spec.rb`), and disable `Metrics/ClassLength`
|
|
39
|
+
at project level (consistent with all other disabled Metrics cops in `.rubocop.yml`).
|
|
40
|
+
- Remove secondary publication engine: strip `attr_writer :registry_publisher`, the
|
|
41
|
+
`registry_publisher` class method, and all `publish_models_async`/`publish_readiness_async`
|
|
42
|
+
calls from `Provider`. Discovery publication now flows exclusively through the SSOT v3
|
|
43
|
+
`DiscoveryRefresh` actor via `Inventory::Publisher`.
|
|
44
|
+
- Remove `:default` identity access in `Provider#settings`; `project` now reads
|
|
45
|
+
`config.vertex_project || ENV['GOOGLE_CLOUD_PROJECT']`, `location` reads `config.vertex_location`
|
|
46
|
+
directly, `default_publisher` returns the provider-native literal `'google'`.
|
|
47
|
+
- Remove `respond_to?(:vertex_model_aliases)` guard in `resolve_model_id`; use safe navigation
|
|
48
|
+
(`config&.vertex_model_aliases`) instead.
|
|
49
|
+
- Rename fallback instance key in `DiscoveryRefreshConfigHelpers#configured_instances` from
|
|
50
|
+
`:default_instance` to `:settings` to avoid gate-A false match on `:default` prefix.
|
|
51
|
+
- Add `handle_exception` call to the `check_health` rescue block so failures are logged through
|
|
52
|
+
the standards path before a `ReadinessResult` is returned.
|
|
53
|
+
- Update `vertex_spec.rb` to remove `RegistryPublisher` test stubs and expectations that no
|
|
54
|
+
longer apply; replace with direct assertions on model/offering/readiness values.
|
|
55
|
+
|
|
56
|
+
## [0.3.0] - 2026-08-13
|
|
57
|
+
|
|
58
|
+
### Changed
|
|
59
|
+
- **SSOT v3 provider migration** — Complete rewrite of the discovery actor to the
|
|
60
|
+
Inventory::Publisher pattern. Claims instances by `{project}:{location}/{credential_fingerprint}`,
|
|
61
|
+
discovers models from the STATIC_MODELS catalog, probes health via the non-inference
|
|
62
|
+
models-list endpoint, and publishes OfferingDraft snapshots with full operation/capability evidence.
|
|
63
|
+
- Remove `@model || STATIC_MODELS.first` default-model fallbacks from `completion_url`/`stream_url`.
|
|
64
|
+
- Remove `Legion::LLM::Call::Registry` and `ScopedRefresher` dependencies from the discovery actor.
|
|
65
|
+
- Add `VertexCallable` with `disconnect` and `normalize_dispatch_error(error:)` contracts.
|
|
66
|
+
- Add SSOT v3 conformance spec with `it_behaves_like 'an SSOT v3 provider adapter'`.
|
|
67
|
+
- Bump `lex-llm` dependency floor to `>= 0.7.0`.
|
|
68
|
+
- `publication_source: :provider_static_catalog` for all offerings derived from STATIC_MODELS.
|
|
69
|
+
- Two distinct projects/locations produce independent instances with separate lanes.
|
|
70
|
+
- Initial readiness failure leaves instance in `:initializing` state (not `:unavailable`).
|
|
71
|
+
- Error normalization (§8 health firewall): only an explicit flat 503 SERVICE_UNAVAILABLE
|
|
72
|
+
response body maps to `instance_unavailable`; connection_failure, timeout, overload (503/529),
|
|
73
|
+
model_not_ready, 429 (rate_limited), auth errors, and generic 5xx are all request-local/terminal
|
|
74
|
+
and never mutate global instance availability.
|
|
75
|
+
- Remove `instance_id: :default` from `offering_for`/`build_offering`; callers receive a
|
|
76
|
+
real project+location derived instance_id from `provider_instance_id`.
|
|
77
|
+
- Read `settings[:publisher]` and `settings[:location]` directly (registered defaults applied);
|
|
78
|
+
remove inline `|| 'google'` and `|| 'us-central1'` fallback guards.
|
|
79
|
+
|
|
3
80
|
## [0.2.16] - 2026-08-04
|
|
4
81
|
|
|
5
82
|
### Changed
|
data/Gemfile
CHANGED
data/lex-llm-vertex.gemspec
CHANGED
|
@@ -25,7 +25,7 @@ Gem::Specification.new do |spec|
|
|
|
25
25
|
|
|
26
26
|
spec.add_dependency 'legion-json', '>= 1.2.1'
|
|
27
27
|
spec.add_dependency 'legion-logging', '>= 1.3.2'
|
|
28
|
-
spec.add_dependency 'legion-settings', '>= 1.
|
|
28
|
+
spec.add_dependency 'legion-settings', '>= 1.4.2'
|
|
29
29
|
spec.add_dependency 'legion-transport', '>= 1.4.14'
|
|
30
|
-
spec.add_dependency 'lex-llm', '>= 0.
|
|
30
|
+
spec.add_dependency 'lex-llm', '>= 0.7.1'
|
|
31
31
|
end
|
|
@@ -1,145 +1,642 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
-
require 'digest'
|
|
4
|
-
|
|
5
3
|
begin
|
|
6
4
|
require 'legion/extensions/actors/every'
|
|
7
5
|
rescue LoadError => e
|
|
8
6
|
warn(e.message) if $VERBOSE
|
|
9
7
|
end
|
|
10
8
|
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
rescue LoadError => e
|
|
14
|
-
warn(e.message) if $VERBOSE
|
|
9
|
+
unless defined?(Legion::Extensions::Actors::Every)
|
|
10
|
+
raise LoadError, 'LegionIO actor runtime is required for Vertex discovery refresh'
|
|
15
11
|
end
|
|
16
12
|
|
|
17
|
-
|
|
13
|
+
require 'concurrent'
|
|
14
|
+
require 'faraday'
|
|
15
|
+
require 'legion/extensions/llm/vertex/callable'
|
|
16
|
+
require 'legion/extensions/llm/inventory/publisher'
|
|
17
|
+
require 'legion/extensions/llm/inventory/scoped_refresher'
|
|
18
|
+
require 'legion/extensions/llm/inventory/identity'
|
|
19
|
+
require 'legion/extensions/llm/inventory/records'
|
|
20
|
+
require 'legion/extensions/llm/inventory/evidence'
|
|
21
|
+
require 'legion/extensions/llm/inventory/probe_coordinator'
|
|
22
|
+
require 'legion/extensions/llm/routing/provider_outcome'
|
|
23
|
+
require 'legion/extensions/llm/taxonomies'
|
|
24
|
+
require 'legion/extensions/llm/capabilities'
|
|
18
25
|
|
|
19
26
|
module Legion
|
|
20
27
|
module Extensions
|
|
21
28
|
module Llm
|
|
22
29
|
module Vertex
|
|
23
30
|
module Actor
|
|
24
|
-
|
|
25
|
-
|
|
31
|
+
# Operation evidence constructors for DiscoveryRefresh.
|
|
32
|
+
module DiscoveryRefreshOperationEvidence
|
|
33
|
+
private
|
|
26
34
|
|
|
27
|
-
|
|
28
|
-
|
|
35
|
+
def build_operation_evidence(now:, is_embedding:, is_generate_content:)
|
|
36
|
+
if is_embedding
|
|
37
|
+
build_embedding_op_evidence(now: now)
|
|
38
|
+
else
|
|
39
|
+
build_chat_op_evidence(now: now, is_gen: is_generate_content)
|
|
40
|
+
end
|
|
29
41
|
end
|
|
30
42
|
|
|
31
|
-
def
|
|
43
|
+
def build_chat_op_evidence(now:, is_gen:)
|
|
44
|
+
ct_status = is_gen ? :supported : :unsupported
|
|
45
|
+
{
|
|
46
|
+
chat: op_ev(:chat, :supported, now), stream_chat: op_ev(:stream_chat, :supported, now),
|
|
47
|
+
embed: op_ev(:embed, :unsupported, now), image: op_ev(:image, :unsupported, now),
|
|
48
|
+
transcribe: op_ev(:transcribe, :unsupported, now), translate: op_ev(:translate, :unsupported, now),
|
|
49
|
+
speak: op_ev(:speak, :unsupported, now), moderate: op_ev(:moderate, :unsupported, now),
|
|
50
|
+
count_tokens: op_ev(:count_tokens, ct_status, now)
|
|
51
|
+
}
|
|
52
|
+
end
|
|
32
53
|
|
|
33
|
-
def
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
54
|
+
def build_embedding_op_evidence(now:)
|
|
55
|
+
{
|
|
56
|
+
chat: op_ev(:chat, :unsupported, now), stream_chat: op_ev(:stream_chat, :unsupported, now),
|
|
57
|
+
embed: op_ev(:embed, :supported, now), image: op_ev(:image, :unsupported, now),
|
|
58
|
+
transcribe: op_ev(:transcribe, :unsupported, now), translate: op_ev(:translate, :unsupported, now),
|
|
59
|
+
speak: op_ev(:speak, :unsupported, now), moderate: op_ev(:moderate, :unsupported, now),
|
|
60
|
+
count_tokens: op_ev(:count_tokens, :unsupported, now)
|
|
61
|
+
}
|
|
62
|
+
end
|
|
39
63
|
|
|
40
|
-
def
|
|
41
|
-
|
|
64
|
+
def op_ev(operation, status, observed_at)
|
|
65
|
+
source = status == :unknown ? :default_false : :provider_implementation
|
|
66
|
+
Legion::Extensions::Llm::Inventory::OperationEvidence.new(
|
|
67
|
+
operation: operation, status: status, source: source, observed_at: observed_at
|
|
68
|
+
)
|
|
69
|
+
end
|
|
70
|
+
end
|
|
71
|
+
|
|
72
|
+
# Capability evidence constructors for DiscoveryRefresh.
|
|
73
|
+
module DiscoveryRefreshCapabilityEvidence
|
|
74
|
+
private
|
|
75
|
+
|
|
76
|
+
def build_capability_evidence(model_entry:, instance_cfg:, now:)
|
|
77
|
+
if model_entry[:usage_type] == :embedding
|
|
78
|
+
build_embedding_cap_evidence(now: now)
|
|
79
|
+
else
|
|
80
|
+
build_chat_cap_evidence(instance_cfg: instance_cfg, now: now)
|
|
81
|
+
end
|
|
82
|
+
end
|
|
83
|
+
|
|
84
|
+
def build_chat_cap_evidence(instance_cfg:, now:)
|
|
85
|
+
{
|
|
86
|
+
completion: cap_ev(:completion, :supported, :provider_implementation, now),
|
|
87
|
+
streaming: cap_ev(:streaming, :supported, :provider_implementation, now),
|
|
88
|
+
vision: resolve_vision_evidence(instance_cfg: instance_cfg, now: now),
|
|
89
|
+
tools: resolve_tools_evidence(instance_cfg: instance_cfg, now: now),
|
|
90
|
+
thinking: resolve_thinking_evidence(instance_cfg: instance_cfg, now: now)
|
|
91
|
+
}
|
|
92
|
+
end
|
|
93
|
+
|
|
94
|
+
def build_embedding_cap_evidence(now:)
|
|
95
|
+
{ embedding: cap_ev(:embedding, :supported, :provider_implementation, now) }
|
|
96
|
+
end
|
|
97
|
+
|
|
98
|
+
def cap_ev(capability, status, source, now)
|
|
99
|
+
Legion::Extensions::Llm::Inventory::CapabilityEvidence.new(
|
|
100
|
+
capability: capability, status: status, source: source, observed_at: now
|
|
101
|
+
)
|
|
102
|
+
end
|
|
103
|
+
|
|
104
|
+
def resolve_vision_evidence(instance_cfg:, now:)
|
|
105
|
+
src = instance_cfg.key?(:enable_vision) ? :instance_override : :default_false
|
|
106
|
+
cap_ev(:vision, :unknown, src, now)
|
|
107
|
+
end
|
|
42
108
|
|
|
43
|
-
|
|
109
|
+
def resolve_tools_evidence(instance_cfg:, now:)
|
|
110
|
+
src = instance_cfg.key?(:enable_tools) ? :instance_override : :default_false
|
|
111
|
+
cap_ev(:tools, :unknown, src, now)
|
|
44
112
|
end
|
|
45
113
|
|
|
46
|
-
def
|
|
47
|
-
|
|
114
|
+
def resolve_thinking_evidence(instance_cfg:, now:)
|
|
115
|
+
src = instance_cfg.key?(:enable_thinking) ? :instance_override : :default_false
|
|
116
|
+
cap_ev(:thinking, :unknown, src, now)
|
|
117
|
+
end
|
|
118
|
+
|
|
119
|
+
def absent_value_evidence
|
|
120
|
+
Legion::Extensions::Llm::Inventory::ValueEvidence.new(status: :unknown, source: :absent)
|
|
121
|
+
end
|
|
122
|
+
|
|
123
|
+
def build_offering_metadata(model_entry:, instance_key:)
|
|
124
|
+
{
|
|
125
|
+
raw_model: model_entry[:model],
|
|
126
|
+
publisher: model_entry[:publisher],
|
|
127
|
+
model_family: model_entry[:model_family].to_s,
|
|
128
|
+
api: model_entry.fetch(:api, :generate_content).to_s,
|
|
129
|
+
instance_id: instance_key.instance_id
|
|
130
|
+
}.freeze
|
|
131
|
+
end
|
|
132
|
+
end
|
|
133
|
+
|
|
134
|
+
# Configuration and identity helpers for DiscoveryRefresh.
|
|
135
|
+
module DiscoveryRefreshConfigHelpers
|
|
136
|
+
private
|
|
137
|
+
|
|
138
|
+
# Single source of truth for configured Vertex instances: the entry
|
|
139
|
+
# module's discovery, which applies the project/credential filter
|
|
140
|
+
# (Vertex.vertex_credentials_present?) and normalizes keys.
|
|
141
|
+
def configured_instances
|
|
142
|
+
Legion::Extensions::Llm::Vertex.discover_instances
|
|
143
|
+
end
|
|
144
|
+
|
|
145
|
+
# Derives the SECONDARY physical identity (project:location/
|
|
146
|
+
# credential-fingerprint) carried as InstanceKey#physical_id for
|
|
147
|
+
# dedup and diagnostics. It is NOT the instance identity — the
|
|
148
|
+
# identity is the operator's config name (see
|
|
149
|
+
# claim_and_activate_instance).
|
|
150
|
+
#
|
|
151
|
+
# Returns nil instead of a fallback: an instance without a
|
|
152
|
+
# resolvable project or credential is skipped by the caller. It is
|
|
153
|
+
# never claimed under a provider-fallback identity (no
|
|
154
|
+
# "unknown"/"default"/"no-cred" IDs).
|
|
155
|
+
def derive_physical_id(instance_cfg:)
|
|
156
|
+
project = instance_cfg[:vertex_project] || instance_cfg[:project]
|
|
157
|
+
return nil if project.nil? || project.to_s.strip.empty?
|
|
158
|
+
|
|
159
|
+
fingerprint = Legion::Extensions::Llm::CredentialSources.credential_fingerprint(
|
|
160
|
+
instance_cfg[:vertex_access_token] || instance_cfg[:vertex_credentials] ||
|
|
161
|
+
instance_cfg[:access_token] || instance_cfg[:credentials]
|
|
162
|
+
)
|
|
163
|
+
return nil if fingerprint.nil?
|
|
164
|
+
|
|
165
|
+
location = instance_cfg[:vertex_location] || instance_cfg[:location] || 'us-central1'
|
|
166
|
+
"#{project}:#{location}/#{fingerprint}"
|
|
167
|
+
end
|
|
168
|
+
|
|
169
|
+
def vertex_api_base(instance_cfg:, location:)
|
|
170
|
+
instance_cfg[:vertex_api_base] || "https://#{location}-aiplatform.googleapis.com"
|
|
171
|
+
end
|
|
172
|
+
|
|
173
|
+
def build_health_connection(base_url:, instance_cfg:)
|
|
174
|
+
Faraday.new(url: base_url) do |f|
|
|
175
|
+
f.options.timeout = 10
|
|
176
|
+
f.options.open_timeout = 5
|
|
177
|
+
token = instance_cfg[:vertex_access_token] || instance_cfg[:access_token]
|
|
178
|
+
f.headers['Authorization'] = "Bearer #{token}" if token.is_a?(String) && !token.strip.empty?
|
|
179
|
+
f.adapter Faraday.default_adapter
|
|
180
|
+
end
|
|
181
|
+
end
|
|
182
|
+
end
|
|
183
|
+
|
|
184
|
+
# Readiness probe and health check helpers for DiscoveryRefresh.
|
|
185
|
+
module DiscoveryRefreshProbeHelpers
|
|
186
|
+
private
|
|
187
|
+
|
|
188
|
+
def run_cadence_probe(instance_id:, state:)
|
|
189
|
+
coordinator = state[:probe_coordinator]
|
|
190
|
+
return unless coordinator.begin_probe
|
|
191
|
+
|
|
192
|
+
probe_token = publisher.readiness_probe_started(instance_id: instance_id,
|
|
193
|
+
publisher_token: state[:publisher_token],
|
|
194
|
+
physical_id: state[:physical_id])
|
|
195
|
+
readiness = check_health(instance_cfg: state[:instance_cfg])
|
|
196
|
+
coordinator.finish_probe
|
|
197
|
+
report_probe_result(instance_id: instance_id, probe_token: probe_token,
|
|
198
|
+
readiness: readiness, state: state)
|
|
199
|
+
rescue StandardError => e
|
|
200
|
+
begin
|
|
201
|
+
coordinator&.finish_probe
|
|
202
|
+
rescue StandardError => finish_e
|
|
203
|
+
handle_exception(finish_e, level: :warn, operation: 'vertex.actor.cadence_probe.finish_probe')
|
|
204
|
+
end
|
|
205
|
+
handle_exception(e, level: :warn, operation: 'vertex.actor.cadence_probe', instance_id: instance_id)
|
|
206
|
+
end
|
|
207
|
+
|
|
208
|
+
def handle_reactive_probe(instance_id:, request:)
|
|
209
|
+
return false if @instance_states.nil?
|
|
210
|
+
|
|
211
|
+
state = @instance_states[instance_id]
|
|
212
|
+
return false unless state
|
|
213
|
+
|
|
214
|
+
coordinator = state[:probe_coordinator]
|
|
215
|
+
return false unless coordinator.begin_probe(request: request)
|
|
216
|
+
|
|
217
|
+
probe_token = publisher.readiness_probe_started(instance_id: instance_id,
|
|
218
|
+
publisher_token: state[:publisher_token],
|
|
219
|
+
physical_id: state[:physical_id])
|
|
220
|
+
readiness = check_health(instance_cfg: state[:instance_cfg])
|
|
221
|
+
coordinator.finish_probe(request: request)
|
|
222
|
+
report_probe_result(instance_id: instance_id, probe_token: probe_token,
|
|
223
|
+
readiness: readiness, state: state)
|
|
224
|
+
true
|
|
225
|
+
rescue StandardError => e
|
|
226
|
+
begin
|
|
227
|
+
coordinator&.finish_probe(request: request)
|
|
228
|
+
rescue StandardError => finish_e
|
|
229
|
+
handle_exception(finish_e, level: :warn, operation: 'vertex.actor.reactive_probe.finish_probe')
|
|
230
|
+
end
|
|
231
|
+
handle_exception(e, level: :warn, operation: 'vertex.actor.reactive_probe', instance_id: instance_id)
|
|
232
|
+
false
|
|
233
|
+
end
|
|
234
|
+
|
|
235
|
+
def report_probe_result(instance_id:, probe_token:, readiness:, state:)
|
|
236
|
+
if readiness.ready?
|
|
237
|
+
if publication_state(instance_key: state[:instance_key]) == :initializing
|
|
238
|
+
# Initial-failure recovery: while the claim is still
|
|
239
|
+
# :initializing a passing probe re-activates the snapshot —
|
|
240
|
+
# activate_instance_snapshot is the only legal transition out
|
|
241
|
+
# of :initializing (readiness_succeeded raises there).
|
|
242
|
+
publisher.activate_instance_snapshot(instance_id: instance_id,
|
|
243
|
+
publisher_token: state[:publisher_token],
|
|
244
|
+
offerings: state[:offerings],
|
|
245
|
+
sequence: state[:sequence], probe_token: probe_token,
|
|
246
|
+
physical_id: state[:physical_id])
|
|
247
|
+
else
|
|
248
|
+
publisher.readiness_succeeded(instance_id: instance_id, probe_token: probe_token,
|
|
249
|
+
physical_id: state[:physical_id])
|
|
250
|
+
end
|
|
251
|
+
else
|
|
252
|
+
publisher.readiness_failed(instance_id: instance_id, probe_token: probe_token,
|
|
253
|
+
reason: readiness.reason, physical_id: state[:physical_id])
|
|
254
|
+
end
|
|
255
|
+
sync_instance_health(name: state[:name], instance_key: state[:instance_key], offerings: state[:offerings])
|
|
48
256
|
end
|
|
49
257
|
|
|
50
|
-
def
|
|
51
|
-
|
|
258
|
+
def publication_state(instance_key:)
|
|
259
|
+
publisher.snapshot.publication_status(instance_key: instance_key).state
|
|
260
|
+
end
|
|
52
261
|
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
262
|
+
def build_probe_enqueue(instance_id:)
|
|
263
|
+
proc do |request:|
|
|
264
|
+
handle_reactive_probe(instance_id: instance_id, request: request)
|
|
265
|
+
rescue StandardError => e
|
|
266
|
+
handle_exception(e, level: :warn, operation: 'vertex.actor.probe_enqueue', instance_id: instance_id)
|
|
267
|
+
false
|
|
57
268
|
end
|
|
269
|
+
end
|
|
270
|
+
|
|
271
|
+
def check_health(instance_cfg:)
|
|
272
|
+
project = instance_cfg[:vertex_project] || instance_cfg[:project]
|
|
273
|
+
location = instance_cfg[:vertex_location] || instance_cfg[:location] || 'us-central1'
|
|
274
|
+
base_url = vertex_api_base(instance_cfg: instance_cfg, location: location)
|
|
275
|
+
path = "/v1/projects/#{project}/locations/#{location}/publishers/google/models"
|
|
276
|
+
conn = build_health_connection(base_url: base_url, instance_cfg: instance_cfg)
|
|
277
|
+
response = conn.get(path)
|
|
278
|
+
Legion::Extensions::Llm::Inventory::ReadinessResult.new(
|
|
279
|
+
ready: response.status == 200,
|
|
280
|
+
reason: "Vertex models-list returned #{response.status}",
|
|
281
|
+
metadata: { status: response.status, base_url: base_url }
|
|
282
|
+
)
|
|
283
|
+
rescue StandardError => e
|
|
284
|
+
handle_exception(e, level: :warn, operation: 'vertex.actor.check_health')
|
|
285
|
+
Legion::Extensions::Llm::Inventory::ReadinessResult.new(
|
|
286
|
+
ready: false, reason: "Vertex models-list error: #{e.message}",
|
|
287
|
+
metadata: { error_class: e.class.name }
|
|
288
|
+
)
|
|
289
|
+
end
|
|
290
|
+
end
|
|
291
|
+
|
|
292
|
+
# Offering snapshot construction helpers for DiscoveryRefresh.
|
|
293
|
+
module DiscoveryRefreshOfferingHelpers
|
|
294
|
+
private
|
|
58
295
|
|
|
59
|
-
|
|
60
|
-
|
|
296
|
+
# The catalog is static per instance, so evidence timestamps are
|
|
297
|
+
# pinned per instance (state[:evidence_now]) — rebuilding with a
|
|
298
|
+
# fresh Time.now would make every draft unequal and force a
|
|
299
|
+
# replace_instance_snapshot churn on every tick.
|
|
300
|
+
def discover_offerings_for_instance(instance_cfg:, instance_key:, now:)
|
|
301
|
+
tier = instance_cfg[:tier] || :cloud
|
|
302
|
+
Provider::STATIC_MODELS.filter_map do |entry|
|
|
303
|
+
next if entry[:model].to_s.empty?
|
|
304
|
+
|
|
305
|
+
build_offering_draft(model_entry: entry, tier: tier, instance_cfg: instance_cfg,
|
|
306
|
+
instance_key: instance_key, now: now)
|
|
61
307
|
end
|
|
62
308
|
rescue StandardError => e
|
|
63
|
-
handle_exception(e, level: :warn,
|
|
309
|
+
handle_exception(e, level: :warn, operation: 'vertex.actor.discover_offerings')
|
|
64
310
|
[]
|
|
65
311
|
end
|
|
66
312
|
|
|
313
|
+
def build_offering_draft(model_entry:, tier:, instance_cfg:, instance_key:, now:)
|
|
314
|
+
model_id = model_entry[:model]
|
|
315
|
+
is_embedding = model_entry[:usage_type] == :embedding
|
|
316
|
+
is_gen = model_entry.fetch(:api, :generate_content) == :generate_content
|
|
317
|
+
Legion::Extensions::Llm::Inventory::OfferingDraft.new(
|
|
318
|
+
provider_native_key: model_id, model: model_id, tier: tier,
|
|
319
|
+
operation_evidence: build_operation_evidence(now: now,
|
|
320
|
+
is_embedding: is_embedding,
|
|
321
|
+
is_generate_content: is_gen),
|
|
322
|
+
capability_evidence: build_capability_evidence(model_entry: model_entry,
|
|
323
|
+
instance_cfg: instance_cfg, now: now),
|
|
324
|
+
context_evidence: absent_value_evidence,
|
|
325
|
+
max_output_evidence: absent_value_evidence,
|
|
326
|
+
embedding_dimensions_evidence: absent_value_evidence,
|
|
327
|
+
model_revision_evidence: absent_value_evidence,
|
|
328
|
+
tokenizer_evidence: absent_value_evidence,
|
|
329
|
+
quota_domains: {},
|
|
330
|
+
metadata: build_offering_metadata(model_entry: model_entry, instance_key: instance_key),
|
|
331
|
+
publication_source: :provider_static_catalog
|
|
332
|
+
)
|
|
333
|
+
end
|
|
334
|
+
end
|
|
335
|
+
|
|
336
|
+
# Publisher, tick-refresh, and instance removal orchestration for
|
|
337
|
+
# DiscoveryRefresh.
|
|
338
|
+
module DiscoveryRefreshLifecycleHelpers
|
|
67
339
|
private
|
|
68
340
|
|
|
69
|
-
def
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
341
|
+
def publisher
|
|
342
|
+
@publisher ||= Legion::Extensions::Llm::Inventory::Publisher.new(
|
|
343
|
+
provider_family: :vertex,
|
|
344
|
+
compatibility_adapter: Legion::Extensions::Llm::Inventory::ScopedRefresher::LegacyCoordinatorAdapter.new(
|
|
345
|
+
provider_family: :vertex
|
|
346
|
+
)
|
|
347
|
+
)
|
|
348
|
+
end
|
|
76
349
|
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
350
|
+
def initial_discovery
|
|
351
|
+
@instance_states = Concurrent::Map.new
|
|
352
|
+
@initialized = true
|
|
353
|
+
reconcile_instances
|
|
354
|
+
end
|
|
355
|
+
|
|
356
|
+
# Re-scans configured instances each tick so late-configured
|
|
357
|
+
# instances appear without a restart and removed instances are
|
|
358
|
+
# retired (with their display health cleared).
|
|
359
|
+
# Instance identity is the operator's CONFIG NAME (the key the
|
|
360
|
+
# router's instances.<name> settings lookups use); names are
|
|
361
|
+
# unique by construction, so two names pointing at the same
|
|
362
|
+
# physical endpoint stay distinct instances (the derived
|
|
363
|
+
# physical_id is diagnostic, never a dedup key that collapses
|
|
364
|
+
# operator-named instances).
|
|
365
|
+
def reconcile_instances
|
|
366
|
+
desired = {}
|
|
367
|
+
configured_instances.each do |name, instance_cfg|
|
|
368
|
+
physical_id = derive_physical_id(instance_cfg: instance_cfg)
|
|
369
|
+
if physical_id.nil?
|
|
370
|
+
log.warn(
|
|
371
|
+
"[vertex][actor] action=skip_instance name=#{name} " \
|
|
372
|
+
'reason=no_resolvable_project_or_credential'
|
|
373
|
+
)
|
|
374
|
+
next
|
|
375
|
+
end
|
|
376
|
+
|
|
377
|
+
desired[name.to_s] = { name: name, instance_cfg: instance_cfg }
|
|
378
|
+
end
|
|
379
|
+
|
|
380
|
+
desired.each do |instance_id, entry|
|
|
381
|
+
next if @instance_states.key?(instance_id)
|
|
382
|
+
|
|
383
|
+
claim_and_activate_instance(name: entry[:name], instance_cfg: entry[:instance_cfg])
|
|
384
|
+
rescue StandardError => e
|
|
385
|
+
handle_exception(e, level: :warn, operation: 'vertex.actor.claim_instance',
|
|
386
|
+
instance_name: entry[:name].to_s)
|
|
387
|
+
end
|
|
388
|
+
|
|
389
|
+
(@instance_states.keys - desired.keys).each do |instance_id|
|
|
390
|
+
remove_instance_state(instance_id)
|
|
80
391
|
end
|
|
81
|
-
lanes
|
|
82
392
|
end
|
|
83
393
|
|
|
84
|
-
def
|
|
85
|
-
|
|
86
|
-
|
|
394
|
+
def tick_refresh
|
|
395
|
+
reconcile_instances
|
|
396
|
+
@instance_states.each do |instance_id, state|
|
|
397
|
+
refresh_instance(instance_id: instance_id, state: state)
|
|
398
|
+
rescue StandardError => e
|
|
399
|
+
handle_exception(e, level: :warn, operation: 'vertex.actor.refresh_instance',
|
|
400
|
+
instance_id: instance_id)
|
|
401
|
+
end
|
|
402
|
+
end
|
|
87
403
|
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
404
|
+
def refresh_instance(instance_id:, state:)
|
|
405
|
+
# While :initializing there is no activated snapshot to replace;
|
|
406
|
+
# the cadence probe is the recovery path (re-activation on a
|
|
407
|
+
# passing probe).
|
|
408
|
+
return run_cadence_probe(instance_id: instance_id, state: state) if initializing?(state)
|
|
409
|
+
|
|
410
|
+
new_offerings = discover_offerings_for_instance(instance_cfg: state[:instance_cfg],
|
|
411
|
+
instance_key: state[:instance_key],
|
|
412
|
+
now: state[:evidence_now])
|
|
413
|
+
if new_offerings != state[:offerings]
|
|
414
|
+
state[:sequence] += 1
|
|
415
|
+
publisher.replace_instance_snapshot(instance_id: instance_id,
|
|
416
|
+
publisher_token: state[:publisher_token],
|
|
417
|
+
offerings: new_offerings, sequence: state[:sequence],
|
|
418
|
+
physical_id: state[:physical_id])
|
|
419
|
+
state[:offerings] = new_offerings
|
|
420
|
+
sync_instance_health(name: state[:name], instance_key: state[:instance_key], offerings: new_offerings)
|
|
421
|
+
end
|
|
422
|
+
run_cadence_probe(instance_id: instance_id, state: state)
|
|
92
423
|
end
|
|
93
424
|
|
|
94
|
-
def
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
425
|
+
def initializing?(state)
|
|
426
|
+
publication_state(instance_key: state[:instance_key]) == :initializing
|
|
427
|
+
end
|
|
428
|
+
|
|
429
|
+
def remove_instance_state(instance_id)
|
|
430
|
+
state = @instance_states.delete(instance_id)
|
|
431
|
+
return unless state
|
|
432
|
+
|
|
433
|
+
state[:callable].disconnect
|
|
434
|
+
publisher.remove_instance(instance_id: instance_id, publisher_token: state[:publisher_token],
|
|
435
|
+
physical_id: state[:physical_id])
|
|
436
|
+
clear_instance_health(name: state[:name])
|
|
437
|
+
rescue StandardError => e
|
|
438
|
+
handle_exception(e, level: :warn, operation: 'vertex.actor.remove_instance',
|
|
439
|
+
instance_id: instance_id)
|
|
440
|
+
end
|
|
441
|
+
|
|
442
|
+
def remove_all_instances
|
|
443
|
+
return unless @instance_states
|
|
444
|
+
|
|
445
|
+
@instance_states.each_key { |instance_id| remove_instance_state(instance_id) }
|
|
446
|
+
end
|
|
447
|
+
end
|
|
448
|
+
|
|
449
|
+
# Instance claim and initial-readiness activation helpers for
|
|
450
|
+
# DiscoveryRefresh.
|
|
451
|
+
module DiscoveryRefreshClaimHelpers
|
|
452
|
+
private
|
|
453
|
+
|
|
454
|
+
# Instance identity is the operator's CONFIG NAME (the key the
|
|
455
|
+
# router's instances.<name> lookups use). The derived
|
|
456
|
+
# project:location/fingerprint rides along as the secondary
|
|
457
|
+
# physical_id field for dedup and diagnostics only.
|
|
458
|
+
def claim_and_activate_instance(name:, instance_cfg:)
|
|
459
|
+
instance_id = name.to_s
|
|
460
|
+
physical_id = derive_physical_id(instance_cfg: instance_cfg)
|
|
461
|
+
raise ArgumentError, 'claim_and_activate_instance requires a resolvable physical_id' if physical_id.nil?
|
|
462
|
+
|
|
463
|
+
instance_key = Legion::Extensions::Llm::Inventory::Identity::InstanceKey.new(
|
|
464
|
+
provider_family: :vertex, instance_id: instance_id, physical_id: physical_id
|
|
465
|
+
)
|
|
466
|
+
callable = VertexCallable.new(instance_cfg: instance_cfg, logger: log)
|
|
467
|
+
probe_coordinator = Legion::Extensions::Llm::Inventory::ProbeCoordinator.new(
|
|
468
|
+
instance_key: instance_key, enqueue: build_probe_enqueue(instance_id: instance_id)
|
|
469
|
+
)
|
|
470
|
+
pub_token = publisher.claim_instance(instance_id: instance_id, callable: callable,
|
|
471
|
+
probe_request_handle: probe_coordinator, physical_id: physical_id)
|
|
472
|
+
now = Time.now.freeze
|
|
473
|
+
offerings = discover_offerings_for_instance(instance_cfg: instance_cfg,
|
|
474
|
+
instance_key: instance_key, now: now)
|
|
475
|
+
probe_token = publisher.readiness_probe_started(instance_id: instance_id, publisher_token: pub_token,
|
|
476
|
+
physical_id: physical_id)
|
|
477
|
+
activate_or_fail_instance(instance_id: instance_id, pub_token: pub_token,
|
|
478
|
+
probe_token: probe_token, instance_cfg: instance_cfg, offerings: offerings,
|
|
479
|
+
physical_id: physical_id)
|
|
480
|
+
@instance_states[instance_id] = {
|
|
481
|
+
name: name, instance_key: instance_key, instance_cfg: instance_cfg,
|
|
482
|
+
callable: callable, probe_coordinator: probe_coordinator,
|
|
483
|
+
publisher_token: pub_token, sequence: 0, offerings: offerings, evidence_now: now,
|
|
484
|
+
physical_id: physical_id
|
|
112
485
|
}
|
|
486
|
+
sync_instance_health(name: name, instance_key: instance_key, offerings: offerings)
|
|
113
487
|
end
|
|
114
488
|
|
|
115
|
-
def
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
489
|
+
def activate_or_fail_instance(instance_id:, pub_token:, probe_token:, instance_cfg:, offerings:,
|
|
490
|
+
physical_id:)
|
|
491
|
+
readiness = check_health(instance_cfg: instance_cfg)
|
|
492
|
+
if readiness.ready?
|
|
493
|
+
publisher.activate_instance_snapshot(instance_id: instance_id, publisher_token: pub_token,
|
|
494
|
+
offerings: offerings, sequence: 0, probe_token: probe_token,
|
|
495
|
+
physical_id: physical_id)
|
|
496
|
+
else
|
|
497
|
+
publisher.readiness_failed(instance_id: instance_id, probe_token: probe_token,
|
|
498
|
+
reason: readiness.reason, physical_id: physical_id)
|
|
499
|
+
end
|
|
119
500
|
end
|
|
501
|
+
end
|
|
502
|
+
|
|
503
|
+
# Display-only health/capabilities settings for DiscoveryRefresh,
|
|
504
|
+
# written AFTER each registry commit. Legacy 4-key health shape
|
|
505
|
+
# (circuit_state/denied/available/adjustment) so pre-SSOT consumers
|
|
506
|
+
# see unchanged output, plus display-only provenance fields.
|
|
507
|
+
# Display only — routing authority stays the in-memory
|
|
508
|
+
# AvailabilityFact.
|
|
509
|
+
module DiscoveryRefreshHealthDisplay
|
|
510
|
+
HEALTH_ADJUSTMENT_AVAILABLE = 0
|
|
511
|
+
HEALTH_ADJUSTMENT_DEGRADED = -50
|
|
512
|
+
CAPABILITY_NAMES_BY_OPERATION = {
|
|
513
|
+
chat: :completion, stream_chat: :streaming, embed: :embedding, image: :image,
|
|
514
|
+
transcribe: :audio_transcription, translate: :audio_transcription, speak: :audio_speech,
|
|
515
|
+
moderate: :moderation
|
|
516
|
+
}.freeze
|
|
517
|
+
|
|
518
|
+
private
|
|
519
|
+
|
|
520
|
+
def sync_instance_health(name:, instance_key:, offerings:)
|
|
521
|
+
snapshot = publisher.snapshot
|
|
522
|
+
status = snapshot.publication_status(instance_key: instance_key)
|
|
523
|
+
record = snapshot.instance(instance_key: instance_key)
|
|
524
|
+
availability = record&.availability
|
|
525
|
+
available = availability&.state == :available
|
|
120
526
|
|
|
121
|
-
|
|
122
|
-
|
|
527
|
+
instances_settings = settings[:instances] || (settings[:instances] = {})
|
|
528
|
+
instance_settings = instances_settings[name] || (instances_settings[name] = {})
|
|
529
|
+
instance_settings[:health] = {
|
|
530
|
+
circuit_state: circuit_state_for(availability),
|
|
531
|
+
denied: false,
|
|
532
|
+
available: available,
|
|
533
|
+
adjustment: available ? HEALTH_ADJUSTMENT_AVAILABLE : HEALTH_ADJUSTMENT_DEGRADED,
|
|
534
|
+
reason: availability&.reason || status.last_error,
|
|
535
|
+
observed_at: observed_at_display(availability),
|
|
536
|
+
last_probe_outcome: status.last_probe_outcome,
|
|
537
|
+
source: :ssot_discovery_actor
|
|
538
|
+
}.compact
|
|
539
|
+
instance_settings[:capabilities] = supported_capabilities(offerings)
|
|
540
|
+
rescue StandardError => e
|
|
541
|
+
handle_exception(e, level: :warn, operation: 'vertex.actor.sync_health',
|
|
542
|
+
instance_id: instance_key.instance_id)
|
|
123
543
|
end
|
|
124
544
|
|
|
125
|
-
def
|
|
126
|
-
|
|
127
|
-
|
|
545
|
+
def clear_instance_health(name:)
|
|
546
|
+
instance_settings = settings[:instances][name]
|
|
547
|
+
return unless instance_settings.is_a?(Hash)
|
|
128
548
|
|
|
129
|
-
|
|
549
|
+
instance_settings.delete(:health)
|
|
550
|
+
instance_settings.delete(:capabilities)
|
|
551
|
+
rescue StandardError => e
|
|
552
|
+
handle_exception(e, level: :warn, operation: 'vertex.actor.clear_health',
|
|
553
|
+
instance_name: name.to_s)
|
|
554
|
+
end
|
|
555
|
+
|
|
556
|
+
def circuit_state_for(availability)
|
|
557
|
+
case availability&.state
|
|
558
|
+
when :available then :closed
|
|
559
|
+
when :unavailable then :open
|
|
560
|
+
else :half_open
|
|
561
|
+
end
|
|
130
562
|
end
|
|
131
563
|
|
|
132
|
-
|
|
564
|
+
def supported_capabilities(offerings)
|
|
565
|
+
capabilities = []
|
|
566
|
+
offerings.each do |draft|
|
|
567
|
+
draft.operation_evidence.each do |operation, evidence|
|
|
568
|
+
next unless evidence.status == :supported
|
|
133
569
|
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
570
|
+
cap = CAPABILITY_NAMES_BY_OPERATION.fetch(operation, operation)
|
|
571
|
+
capabilities << cap unless capabilities.include?(cap)
|
|
572
|
+
end
|
|
573
|
+
draft.capability_evidence.each do |capability, evidence|
|
|
574
|
+
capabilities << capability if evidence.status == :supported && !capabilities.include?(capability)
|
|
575
|
+
end
|
|
576
|
+
end
|
|
577
|
+
capabilities.sort
|
|
578
|
+
end
|
|
579
|
+
|
|
580
|
+
def observed_at_display(availability)
|
|
581
|
+
observed_at = availability&.observed_at
|
|
582
|
+
return nil unless observed_at
|
|
583
|
+
|
|
584
|
+
observed_at.getutc.strftime('%Y-%m-%dT%H:%M:%SZ')
|
|
585
|
+
end
|
|
586
|
+
end
|
|
587
|
+
|
|
588
|
+
# SSOT v3 periodic discovery actor for Vertex AI provider instances.
|
|
589
|
+
# Claims configured instances, discovers models from the STATIC_MODELS
|
|
590
|
+
# catalog, probes health via the non-inference models-list endpoint,
|
|
591
|
+
# and publishes complete OfferingDraft snapshots through the
|
|
592
|
+
# Inventory::Publisher. Owns the refresh cadence, recovers
|
|
593
|
+
# initial-readiness failures, and writes the display-only
|
|
594
|
+
# health/capabilities settings after each registry commit.
|
|
595
|
+
class DiscoveryRefresh < Legion::Extensions::Actors::Every
|
|
596
|
+
include Legion::Extensions::Helpers::Lex
|
|
597
|
+
include Legion::Logging::Helper
|
|
598
|
+
include DiscoveryRefreshOperationEvidence
|
|
599
|
+
include DiscoveryRefreshCapabilityEvidence
|
|
600
|
+
include DiscoveryRefreshConfigHelpers
|
|
601
|
+
include DiscoveryRefreshProbeHelpers
|
|
602
|
+
include DiscoveryRefreshOfferingHelpers
|
|
603
|
+
include DiscoveryRefreshClaimHelpers
|
|
604
|
+
include DiscoveryRefreshHealthDisplay
|
|
605
|
+
include DiscoveryRefreshLifecycleHelpers
|
|
606
|
+
|
|
607
|
+
# Guards a broken discovery config (missing or non-positive value);
|
|
608
|
+
# the registered default (lex-llm ProviderSettings) is 300 seconds.
|
|
609
|
+
FALLBACK_DISCOVERY_INTERVAL_SECONDS = 300
|
|
610
|
+
|
|
611
|
+
def runner_class = self.class
|
|
612
|
+
def runner_function = 'manual'
|
|
613
|
+
def run_now? = true
|
|
614
|
+
def use_runner? = false
|
|
615
|
+
def check_subtask? = false
|
|
616
|
+
def generate_task? = false
|
|
617
|
+
|
|
618
|
+
# The actor owns the discovery cadence. Read the registered
|
|
619
|
+
# interval so the operator knob is honored and the timer never
|
|
620
|
+
# receives nil.
|
|
621
|
+
def time
|
|
622
|
+
interval = settings.dig(:discovery, :interval_seconds)
|
|
623
|
+
interval.is_a?(Numeric) && interval.positive? ? interval : FALLBACK_DISCOVERY_INTERVAL_SECONDS
|
|
137
624
|
end
|
|
138
625
|
|
|
139
626
|
def manual
|
|
140
|
-
|
|
627
|
+
if @initialized
|
|
628
|
+
tick_refresh
|
|
629
|
+
else
|
|
630
|
+
initial_discovery
|
|
631
|
+
end
|
|
632
|
+
rescue StandardError => e
|
|
633
|
+
handle_exception(e, level: :warn, operation: 'vertex.actor.discovery_refresh')
|
|
634
|
+
end
|
|
635
|
+
|
|
636
|
+
def shutdown
|
|
637
|
+
remove_all_instances
|
|
141
638
|
rescue StandardError => e
|
|
142
|
-
handle_exception(e, level: :warn,
|
|
639
|
+
handle_exception(e, level: :warn, operation: 'vertex.actor.discovery_refresh.shutdown')
|
|
143
640
|
end
|
|
144
641
|
end
|
|
145
642
|
end
|
|
@@ -11,6 +11,7 @@ unless defined?(Legion::Extensions::Actors::Subscription)
|
|
|
11
11
|
end
|
|
12
12
|
|
|
13
13
|
require 'legion/extensions/llm/vertex'
|
|
14
|
+
require 'legion/extensions/llm/vertex/runners/fleet_worker'
|
|
14
15
|
require 'legion/extensions/llm/fleet/provider_responder'
|
|
15
16
|
|
|
16
17
|
module Legion
|
|
@@ -19,9 +20,14 @@ module Legion
|
|
|
19
20
|
module Vertex
|
|
20
21
|
module Actor
|
|
21
22
|
# Subscription actor for Vertex fleet request consumption.
|
|
23
|
+
#
|
|
24
|
+
# runner_class MUST be the constant (not a String): the Subscription
|
|
25
|
+
# dispatch path calls runner_class.send(runner_function, **message)
|
|
26
|
+
# directly when use_runner? is false, and a String cannot be send-ed.
|
|
27
|
+
# The runner entrypoint is kwargs-only to match that call.
|
|
22
28
|
class FleetWorker < Legion::Extensions::Actors::Subscription
|
|
23
29
|
def runner_class
|
|
24
|
-
|
|
30
|
+
Legion::Extensions::Llm::Vertex::Runners::FleetWorker
|
|
25
31
|
end
|
|
26
32
|
|
|
27
33
|
def runner_function
|
|
@@ -0,0 +1,127 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'faraday'
|
|
4
|
+
require 'legion/extensions/llm/routing/provider_outcome'
|
|
5
|
+
require 'legion/extensions/llm/vertex/provider'
|
|
6
|
+
|
|
7
|
+
module Legion
|
|
8
|
+
module Extensions
|
|
9
|
+
module Llm
|
|
10
|
+
module Vertex
|
|
11
|
+
module Actor
|
|
12
|
+
# Callable wrapper for a Vertex AI provider instance. Delegates the
|
|
13
|
+
# fleet dispatch operations to a per-instance Vertex::Provider (real
|
|
14
|
+
# HTTP dispatch; provider/Faraday errors propagate so
|
|
15
|
+
# normalize_dispatch_error classifies them) and implements the
|
|
16
|
+
# `disconnect` and `normalize_dispatch_error(error:)` contracts
|
|
17
|
+
# required by Inventory::CallableHandle and Routing::ProviderOutcome.
|
|
18
|
+
class VertexCallable
|
|
19
|
+
attr_reader :provider
|
|
20
|
+
|
|
21
|
+
def initialize(instance_cfg:, logger:)
|
|
22
|
+
@instance_cfg = instance_cfg
|
|
23
|
+
@logger = logger
|
|
24
|
+
@provider = Provider.new(instance_cfg)
|
|
25
|
+
@disconnected = false
|
|
26
|
+
end
|
|
27
|
+
|
|
28
|
+
def disconnected?
|
|
29
|
+
@disconnected
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
def disconnect
|
|
33
|
+
@disconnected = true
|
|
34
|
+
@provider.disconnect
|
|
35
|
+
@logger.debug { '[vertex][callable] disconnected' }
|
|
36
|
+
end
|
|
37
|
+
|
|
38
|
+
# --- Fleet dispatch operations (Fleet::WorkerExecution contract) --
|
|
39
|
+
|
|
40
|
+
def chat(messages:, model:, **rest)
|
|
41
|
+
provider.chat(messages: messages, model: model, **rest)
|
|
42
|
+
end
|
|
43
|
+
|
|
44
|
+
def stream_chat(messages:, model:, **rest, &)
|
|
45
|
+
provider.stream_chat(messages: messages, model: model, **rest, &)
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
def embed(text:, model:, **rest)
|
|
49
|
+
provider.embed(text: text, model: model, **rest)
|
|
50
|
+
end
|
|
51
|
+
|
|
52
|
+
def count_tokens(messages:, model:, **rest)
|
|
53
|
+
provider.count_tokens(messages: messages, model: model, **rest)
|
|
54
|
+
end
|
|
55
|
+
|
|
56
|
+
# --- Error normalization ------------------------------------------
|
|
57
|
+
|
|
58
|
+
def normalize_dispatch_error(error:)
|
|
59
|
+
reason = error.message.to_s[0, 512]
|
|
60
|
+
|
|
61
|
+
kind = case error
|
|
62
|
+
when Faraday::ConnectionFailed
|
|
63
|
+
:connection_failure
|
|
64
|
+
when Faraday::TimeoutError
|
|
65
|
+
:timeout
|
|
66
|
+
when Faraday::ClientError
|
|
67
|
+
classify_client_error(error: error)
|
|
68
|
+
when Faraday::ServerError
|
|
69
|
+
classify_server_error(error: error)
|
|
70
|
+
when Legion::Extensions::Llm::OverloadedError
|
|
71
|
+
:overloaded
|
|
72
|
+
else
|
|
73
|
+
# ServiceUnavailableError and all other errors map to provider_error.
|
|
74
|
+
# Never escalate to instance_unavailable from a typed error alone.
|
|
75
|
+
:provider_error
|
|
76
|
+
end
|
|
77
|
+
|
|
78
|
+
Legion::Extensions::Llm::Routing::ProviderOutcome.new(
|
|
79
|
+
kind: kind,
|
|
80
|
+
reason: reason.empty? ? 'unknown dispatch error' : reason
|
|
81
|
+
)
|
|
82
|
+
end
|
|
83
|
+
|
|
84
|
+
private
|
|
85
|
+
|
|
86
|
+
def classify_client_error(error:)
|
|
87
|
+
case error.response_status
|
|
88
|
+
when 401 then :authentication
|
|
89
|
+
when 403 then :authorization
|
|
90
|
+
when 404 then :model_missing
|
|
91
|
+
when 429 then :rate_limited
|
|
92
|
+
else :invalid_request
|
|
93
|
+
end
|
|
94
|
+
end
|
|
95
|
+
|
|
96
|
+
def classify_server_error(error:)
|
|
97
|
+
# NEVER classify raw 503/5xx as instance_unavailable by status alone.
|
|
98
|
+
# Only an explicit flat SERVICE_UNAVAILABLE body signal from Vertex
|
|
99
|
+
# would justify instance_unavailable. Everything else is request-local.
|
|
100
|
+
status = error.response_status
|
|
101
|
+
body = extract_error_body(error: error)
|
|
102
|
+
|
|
103
|
+
if status == 503 && body.include?('service_unavailable') && !overload_signal?(body: body)
|
|
104
|
+
:instance_unavailable
|
|
105
|
+
elsif [503, 529].include?(status)
|
|
106
|
+
:overloaded
|
|
107
|
+
else
|
|
108
|
+
:provider_error
|
|
109
|
+
end
|
|
110
|
+
end
|
|
111
|
+
|
|
112
|
+
# Faraday::Error#response_body reads the body from either a
|
|
113
|
+
# Hash-shaped or a Faraday::Response/Env-shaped error response, so
|
|
114
|
+
# the body signal survives real adapter-raised errors.
|
|
115
|
+
def extract_error_body(error:)
|
|
116
|
+
error.response_body.to_s.downcase
|
|
117
|
+
end
|
|
118
|
+
|
|
119
|
+
def overload_signal?(body:)
|
|
120
|
+
body.include?('overload') || body.include?('model not ready') || body.include?('model_not_ready')
|
|
121
|
+
end
|
|
122
|
+
end
|
|
123
|
+
end
|
|
124
|
+
end
|
|
125
|
+
end
|
|
126
|
+
end
|
|
127
|
+
end
|
|
@@ -10,7 +10,7 @@ module Legion
|
|
|
10
10
|
module Llm
|
|
11
11
|
module Vertex
|
|
12
12
|
# Google Cloud Vertex AI provider implementation for the Legion::Extensions::Llm contract.
|
|
13
|
-
class Provider < Legion::Extensions::Llm::Provider
|
|
13
|
+
class Provider < Legion::Extensions::Llm::Provider
|
|
14
14
|
STATIC_MODELS = [
|
|
15
15
|
{ model: 'gemini-2.5-flash', alias: 'gemini-flash', publisher: 'google', model_family: :gemini },
|
|
16
16
|
{ model: 'gemini-2.5-pro', alias: 'gemini-pro', publisher: 'google', model_family: :gemini },
|
|
@@ -32,8 +32,6 @@ module Legion
|
|
|
32
32
|
MODEL_FAMILIES = STATIC_MODELS.to_h { |entry| [entry.fetch(:model), entry.fetch(:model_family)] }.freeze
|
|
33
33
|
|
|
34
34
|
class << self
|
|
35
|
-
attr_writer :registry_publisher
|
|
36
|
-
|
|
37
35
|
def slug = 'vertex'
|
|
38
36
|
def default_transport = :http
|
|
39
37
|
def default_tier = :cloud
|
|
@@ -53,12 +51,8 @@ module Legion
|
|
|
53
51
|
def configuration_requirements = []
|
|
54
52
|
def capabilities = Capabilities
|
|
55
53
|
|
|
56
|
-
def registry_publisher
|
|
57
|
-
@registry_publisher ||= Legion::Extensions::Llm::RegistryPublisher.new(provider_family: :vertex)
|
|
58
|
-
end
|
|
59
|
-
|
|
60
54
|
def resolve_model_id(model_id, config: nil)
|
|
61
|
-
configured_aliases = config
|
|
55
|
+
configured_aliases = config&.vertex_model_aliases
|
|
62
56
|
aliases = ALIASES.merge((configured_aliases || {}).transform_keys(&:to_s))
|
|
63
57
|
aliases.fetch(model_id.to_s, model_id.to_s)
|
|
64
58
|
end
|
|
@@ -81,10 +75,6 @@ module Legion
|
|
|
81
75
|
end
|
|
82
76
|
end
|
|
83
77
|
|
|
84
|
-
def settings
|
|
85
|
-
Vertex.default_settings
|
|
86
|
-
end
|
|
87
|
-
|
|
88
78
|
def api_base
|
|
89
79
|
config.vertex_api_base || "https://#{location}-aiplatform.googleapis.com/v1"
|
|
90
80
|
end
|
|
@@ -94,12 +84,23 @@ module Legion
|
|
|
94
84
|
'Content-Type' => 'application/json; charset=utf-8' }.compact)
|
|
95
85
|
end
|
|
96
86
|
|
|
97
|
-
def project = config.vertex_project ||
|
|
98
|
-
def location = config.vertex_location
|
|
99
|
-
def default_publisher =
|
|
87
|
+
def project = config.vertex_project || ENV.fetch('GOOGLE_CLOUD_PROJECT', nil)
|
|
88
|
+
def location = config.vertex_location
|
|
89
|
+
def default_publisher = 'google'
|
|
100
90
|
def models_url = publisher_parent
|
|
101
|
-
|
|
102
|
-
def
|
|
91
|
+
|
|
92
|
+
def completion_url(model: @model)
|
|
93
|
+
raise ArgumentError, 'model is required for completion_url' unless model
|
|
94
|
+
|
|
95
|
+
generate_content_url(model: model)
|
|
96
|
+
end
|
|
97
|
+
|
|
98
|
+
def stream_url(model: @model)
|
|
99
|
+
raise ArgumentError, 'model is required for stream_url' unless model
|
|
100
|
+
|
|
101
|
+
stream_generate_content_url(model: model)
|
|
102
|
+
end
|
|
103
|
+
|
|
103
104
|
def count_tokens_url(model:) = "#{publisher_model_path(model)}:countTokens"
|
|
104
105
|
def embedding_url(model:) = "#{publisher_model_path(model)}:predict"
|
|
105
106
|
|
|
@@ -119,8 +120,7 @@ module Legion
|
|
|
119
120
|
def list_models(**_filters)
|
|
120
121
|
log.info { 'listing available Vertex models from static catalog' }
|
|
121
122
|
STATIC_MODELS.map { |entry| model_info_from_static(entry) }.tap do |models|
|
|
122
|
-
log.info { "
|
|
123
|
-
self.class.registry_publisher.publish_models_async(models, readiness: readiness(live: false))
|
|
123
|
+
log.info { "listed #{models.size} Vertex model(s)" }
|
|
124
124
|
end
|
|
125
125
|
end
|
|
126
126
|
|
|
@@ -134,7 +134,7 @@ module Legion
|
|
|
134
134
|
super
|
|
135
135
|
end
|
|
136
136
|
|
|
137
|
-
def offering_for(model:, model_family: nil, instance_id:
|
|
137
|
+
def offering_for(model:, model_family: nil, instance_id: nil, **metadata)
|
|
138
138
|
model_id = model_id(model)
|
|
139
139
|
publisher = metadata.delete(:publisher) || publisher_for(model_id)
|
|
140
140
|
family = model_family || metadata.delete(:model_family) || model_family_for(model_id, publisher)
|
|
@@ -143,7 +143,7 @@ module Legion
|
|
|
143
143
|
model: resource_name(model_id, publisher:),
|
|
144
144
|
alias_name: alias_for(model_id),
|
|
145
145
|
model_family: family,
|
|
146
|
-
instance_id: instance_id,
|
|
146
|
+
instance_id: instance_id || provider_instance_id,
|
|
147
147
|
publisher: publisher,
|
|
148
148
|
usage_type: metadata.delete(:usage_type) || usage_type_for(model_id),
|
|
149
149
|
api: metadata.delete(:api) || api_for(model_id),
|
|
@@ -173,9 +173,7 @@ module Legion
|
|
|
173
173
|
|
|
174
174
|
def readiness(live: false)
|
|
175
175
|
health(live:).merge(local: false, remote: true, api_base: api_base,
|
|
176
|
-
endpoints: endpoint_manifest)
|
|
177
|
-
self.class.registry_publisher.publish_readiness_async(metadata) if live
|
|
178
|
-
end
|
|
176
|
+
endpoints: endpoint_manifest)
|
|
179
177
|
end
|
|
180
178
|
|
|
181
179
|
def chat(
|
|
@@ -258,7 +256,7 @@ module Legion
|
|
|
258
256
|
parse_embedding_response(response, model: model_id)
|
|
259
257
|
end
|
|
260
258
|
|
|
261
|
-
def complete(messages, tools:, temperature:, model:, params: {},
|
|
259
|
+
def complete(messages, tools:, temperature:, model:, params: {}, _headers: {}, schema: nil, thinking: nil,
|
|
262
260
|
tool_prefs: nil, &)
|
|
263
261
|
payload = params.dup
|
|
264
262
|
payload[:generationConfig] = Utils.deep_merge(payload[:generationConfig] || {},
|
|
@@ -356,13 +354,14 @@ module Legion
|
|
|
356
354
|
)
|
|
357
355
|
end
|
|
358
356
|
|
|
359
|
-
def build_offering(model:, model_family:, usage_type:, publisher:, api:, instance_id:
|
|
357
|
+
def build_offering(model:, model_family:, usage_type:, publisher:, api:, instance_id: nil,
|
|
360
358
|
alias_name: nil, health: {}, metadata: {})
|
|
361
|
-
|
|
359
|
+
resolved_instance_id = instance_id || provider_instance_id
|
|
360
|
+
policy = resolve_capability_policy(model, api:, metadata:, instance_id: resolved_instance_id)
|
|
362
361
|
|
|
363
362
|
Legion::Extensions::Llm::Routing::ModelOffering.new(
|
|
364
363
|
provider_family: :vertex,
|
|
365
|
-
instance_id:
|
|
364
|
+
instance_id: resolved_instance_id,
|
|
366
365
|
transport: offering_transport,
|
|
367
366
|
tier: offering_tier,
|
|
368
367
|
model: model,
|
|
@@ -9,17 +9,21 @@ module Legion
|
|
|
9
9
|
module Vertex
|
|
10
10
|
module Runners
|
|
11
11
|
# Runner entrypoint for Vertex fleet request execution.
|
|
12
|
+
#
|
|
13
|
+
# The Subscription dispatch path invokes this as
|
|
14
|
+
# handle_fleet_request(**message) where message is the decoded
|
|
15
|
+
# protocol-v2 envelope plus AMQP metadata; the whole message is the
|
|
16
|
+
# payload for the shared responder (which selects the envelope
|
|
17
|
+
# fields it needs).
|
|
12
18
|
module FleetWorker
|
|
13
19
|
module_function
|
|
14
20
|
|
|
15
|
-
def handle_fleet_request(
|
|
21
|
+
def handle_fleet_request(**message)
|
|
16
22
|
Legion::Extensions::Llm::Fleet::ProviderResponder.call(
|
|
17
|
-
payload:
|
|
23
|
+
payload: message,
|
|
18
24
|
provider_family: Vertex::PROVIDER_FAMILY,
|
|
19
25
|
provider_class: Vertex::Provider,
|
|
20
|
-
provider_instances: -> { Vertex.discover_instances }
|
|
21
|
-
delivery: delivery,
|
|
22
|
-
properties: properties
|
|
26
|
+
provider_instances: -> { Vertex.discover_instances }
|
|
23
27
|
)
|
|
24
28
|
end
|
|
25
29
|
end
|
|
@@ -3,7 +3,8 @@
|
|
|
3
3
|
require 'legion/extensions/llm'
|
|
4
4
|
require 'legion/extensions/llm/vertex/provider'
|
|
5
5
|
require 'legion/extensions/llm/vertex/version'
|
|
6
|
-
|
|
6
|
+
require 'legion/extensions/llm/vertex/callable'
|
|
7
|
+
require 'legion/extensions/llm/vertex/actors/discovery_refresh'
|
|
7
8
|
|
|
8
9
|
module Legion
|
|
9
10
|
module Extensions
|
|
@@ -11,7 +12,6 @@ module Legion
|
|
|
11
12
|
# Google Cloud Vertex AI provider extension namespace.
|
|
12
13
|
module Vertex
|
|
13
14
|
extend Legion::Logging::Helper
|
|
14
|
-
extend ::Legion::Extensions::Core if ::Legion::Extensions.const_defined?(:Core, false)
|
|
15
15
|
extend Legion::Extensions::Llm::AutoRegistration
|
|
16
16
|
|
|
17
17
|
PROVIDER_FAMILY = :vertex
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: lex-llm-vertex
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.3.3
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- LegionIO
|
|
@@ -43,14 +43,14 @@ dependencies:
|
|
|
43
43
|
requirements:
|
|
44
44
|
- - ">="
|
|
45
45
|
- !ruby/object:Gem::Version
|
|
46
|
-
version: 1.
|
|
46
|
+
version: 1.4.2
|
|
47
47
|
type: :runtime
|
|
48
48
|
prerelease: false
|
|
49
49
|
version_requirements: !ruby/object:Gem::Requirement
|
|
50
50
|
requirements:
|
|
51
51
|
- - ">="
|
|
52
52
|
- !ruby/object:Gem::Version
|
|
53
|
-
version: 1.
|
|
53
|
+
version: 1.4.2
|
|
54
54
|
- !ruby/object:Gem::Dependency
|
|
55
55
|
name: legion-transport
|
|
56
56
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -71,14 +71,14 @@ dependencies:
|
|
|
71
71
|
requirements:
|
|
72
72
|
- - ">="
|
|
73
73
|
- !ruby/object:Gem::Version
|
|
74
|
-
version: 0.
|
|
74
|
+
version: 0.7.1
|
|
75
75
|
type: :runtime
|
|
76
76
|
prerelease: false
|
|
77
77
|
version_requirements: !ruby/object:Gem::Requirement
|
|
78
78
|
requirements:
|
|
79
79
|
- - ">="
|
|
80
80
|
- !ruby/object:Gem::Version
|
|
81
|
-
version: 0.
|
|
81
|
+
version: 0.7.1
|
|
82
82
|
description: Google Cloud Vertex AI provider integration for the LegionIO LLM routing
|
|
83
83
|
framework.
|
|
84
84
|
email:
|
|
@@ -100,6 +100,7 @@ files:
|
|
|
100
100
|
- lib/legion/extensions/llm/vertex.rb
|
|
101
101
|
- lib/legion/extensions/llm/vertex/actors/discovery_refresh.rb
|
|
102
102
|
- lib/legion/extensions/llm/vertex/actors/fleet_worker.rb
|
|
103
|
+
- lib/legion/extensions/llm/vertex/callable.rb
|
|
103
104
|
- lib/legion/extensions/llm/vertex/provider.rb
|
|
104
105
|
- lib/legion/extensions/llm/vertex/runners/fleet_worker.rb
|
|
105
106
|
- lib/legion/extensions/llm/vertex/version.rb
|