lex-llm-openai 0.4.10 → 0.6.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +69 -0
- data/Gemfile +2 -0
- data/lex-llm-openai.gemspec +2 -2
- data/lib/legion/extensions/llm/openai/actors/discovery_refresh.rb +283 -107
- data/lib/legion/extensions/llm/openai/actors/fleet_worker.rb +11 -0
- data/lib/legion/extensions/llm/openai/discovery_drafts.rb +113 -0
- data/lib/legion/extensions/llm/openai/discovery_evidence_builders.rb +140 -0
- data/lib/legion/extensions/llm/openai/discovery_health_display.rb +92 -0
- data/lib/legion/extensions/llm/openai/discovery_identity.rb +69 -0
- data/lib/legion/extensions/llm/openai/discovery_probing.rb +135 -0
- data/lib/legion/extensions/llm/openai/discovery_transport.rb +37 -0
- data/lib/legion/extensions/llm/openai/instance_discovery.rb +108 -0
- data/lib/legion/extensions/llm/openai/openai_callable.rb +135 -0
- data/lib/legion/extensions/llm/openai/provider.rb +37 -20
- data/lib/legion/extensions/llm/openai/runners/fleet_worker.rb +9 -13
- data/lib/legion/extensions/llm/openai/translator.rb +85 -94
- data/lib/legion/extensions/llm/openai/version.rb +1 -1
- data/lib/legion/extensions/llm/openai.rb +5 -96
- metadata +13 -5
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 84567e208562add7567e8de290c9d69ce3a7ae7e10cfab7e7e6ddf92173d2b7d
|
|
4
|
+
data.tar.gz: 912a0379793e6e7dc10353aaf198ddc2c08b50310eb1642b8e2f7e5f1f244010
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: a92b8cc928c93cf271ab575f15a9c582ddb4a53ceab09f8d0817a9b4268abc4dddcb997a11e926490c739798a9f25b763ea7cdbe3e348b328570f6ee3714b1fb
|
|
7
|
+
data.tar.gz: f8686fa71f7b9a23b7ee2f0a26bd74b8ea44afbefc33f6c47f0ed3836c34fe3e75acce1cd49bc98407a9ecf41e4937642f84b3fe0ce50f5d24c66b1f0da29884
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,74 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## [Unreleased]
|
|
4
|
+
|
|
5
|
+
### Fixed
|
|
6
|
+
- **Credential-less configured instances are discovered normally** — removed the synthetic-default suppression and its one-time warning cache. A configured instance, including one named `default`, now follows the ordinary discovery and readiness path without a skip warning.
|
|
7
|
+
|
|
8
|
+
## [0.6.2] - 2026-08-17
|
|
9
|
+
|
|
10
|
+
### Fixed
|
|
11
|
+
- **Instance identity is the operator's config name** — `InstanceKey.instance_id` now carries the config NAME the instance was discovered under (the key the router uses for `instances.<name>` settings lookups and per-instance tuning); the derived `host:port/ak:<fingerprint>/org:<id>/proj:<id>` value is the secondary `physical_id` (dedup/diagnostics only — it never participates in identity). Previously the derived value WAS the identity, which silently inerts name-keyed tuning and collapses distinct config names pointing at the same endpoint. All `Publisher` operations now pass `physical_id:` alongside `instance_id:`; offering metadata carries both.
|
|
12
|
+
- **Authoritative operation evidence verified** — embedding models (`text-embedding-*`) publish `chat: :unsupported` (not `:unknown`, not `:supported`) so a plain chat request cannot misroute to an embedding model; pinned by spec.
|
|
13
|
+
- **D1 Callable dispatch** — `OpenaiCallable` now implements the fleet dispatch ops (`chat`, `stream_chat`, `embed`, `count_tokens`, `image`, `moderate`) by delegating to a per-instance `Openai::Provider` (previously `NotImplementedError` stubs); errors propagate for `normalize_dispatch_error`; `disconnect` closes the Provider. Optional `provider:` injection seam for specs (production builds the real Provider lazily).
|
|
14
|
+
- **D15 Raw-string model at the dispatch boundary** — the fleet passes `model:` as the offering's raw id (String). `chat`/`stream_chat` render paths call `model.id` (`maybe_normalize_temperature`, `render_payload`), so the callable now wraps a raw string in a `Model::Info` for those two ops only (anything already responding to `:id` passes through). `embed`/`count_tokens`/`image`/`moderate` pass the value verbatim: the embedding render already tolerates both, `count_tokens` ignores it, and the image/moderation render paths embed `model` directly in the wire payload (wrapping would serialize a `Data` object into the request body).
|
|
15
|
+
- **D4 Initial-failure recovery** — an instance whose initial readiness failed stays claimable: each tick probes while `:initializing` and re-activates via `activate_instance_snapshot` (fresh probe token, current offerings, next sequence) on the first passing probe. Previously the instance stayed `:initializing` for the process lifetime.
|
|
16
|
+
- **D4 Tick reconcile** — `discover_instances` is re-scanned every tick: instances configured after boot are claimed without a restart; removed instances are released from the Registry and their settings health cleared. Credential-less candidates (e.g. the synthetic `instances.default` placeholder) are skipped with a warn, not claimed.
|
|
17
|
+
- **D3 Snapshot churn** — offerings are compared on identity/status fields (not `Data#==`, which `Time.now` `observed_at` stamps poison) so an unchanged catalog no longer triggers `replace_instance_snapshot` every tick.
|
|
18
|
+
- **Latent NameError in draft building** — `STANDARD_CAPABILITY_CHECKS` moved from the actor class into `DiscoveryEvidenceBuilders`: constant lookup from the included module only walks that module's lexical scope, so `build_offering_draft` raised `NameError` in production (swallowed to `[]` by the discovery rescue) — every activated instance published an empty offering set.
|
|
19
|
+
- **D2 Bridge** — `Publisher` constructed with the `LegacyCoordinatorAdapter` so SSOT commits project into the old `Legion::LLM::Inventory` coordinator during the mixed-version window.
|
|
20
|
+
- **D9 Cadence interval** — actor `time` reads `settings[:discovery][:interval_seconds]` (never nil; falls back to the registered default); dead `self.every_seconds` removed.
|
|
21
|
+
- **D13 Fleet dispatch** — fleet `Subscription` actor sets `use_runner? = true` (Legion::Runner.run resolves the String runner class; the direct `runner_class.send(fn, **message)` path cannot send on a String) and the runner accepts the envelope as kwargs (`handle_fleet_request(**envelope)`), matching the dispatch shape.
|
|
22
|
+
- **D14 Health display** — after each registry commit the actor writes `settings[:instances][<config_name>][:health]` (legacy 4-key shape + display keys) and `[:capabilities]`; cleared on removal.
|
|
23
|
+
- **D5 Fail loud** — the actor-runtime `rescue LoadError → warn if $VERBOSE` + `return unless defined?` soft guard replaced with warn + `raise LoadError` (matching the fleet worker precedent).
|
|
24
|
+
- **D6 Nits** — `require 'faraday'` hoisted to file tops; `require` instead of `require_relative`; dead `|| instance_cfg[:endpoint]` branch removed; `log.debug` block form; dead `stub_registry_publisher` spec helper removed.
|
|
25
|
+
- **Standard sweep** — discovery/identity/probing/transport/health logic extracted from the actor class into `InstanceDiscovery`, `DiscoveryDrafts`, `DiscoveryIdentity`, `DiscoveryProbing`, `DiscoveryTransport`, `DiscoveryHealthDisplay` modules so all files sit under Metrics limits; conformance harness now drives the production callable and the actor's real identity/draft helpers (no harness re-implementation); new `actor/discovery_refresh_spec.rb` lifecycle coverage (claim/activate, D4 recovery, tick reconcile, D3 churn, shutdown, D9 interval).
|
|
26
|
+
- **legion-settings floor bumped to >= 1.4.2** — nested-extension settings-path resolution: the actor's `settings[:...]` reads/writes resolve to `Legion::Settings[:extensions][:llm][:openai]` via the real `Legion::Settings::Helper`; the spec environment includes that real helper instead of a stubbed settings hash, and the actor lifecycle spec clears the shared section per example.
|
|
27
|
+
- **lex-llm floor bumped to >= 0.7.1** — the fail-forward identity contract first shipped in lex-llm 0.7.1: `InstanceKey` gains the optional secondary `physical_id` member (0.7.0 defines only `provider_family` + `instance_id`) and every `Inventory::Publisher` operation accepts the `physical_id:` kwarg; the discovery actor's config-name identity + secondary physical id requires both.
|
|
28
|
+
- **Single actor registration** — the provider module no longer extends Core at file level, so the boot-time submodule walk skips it and the gem's own top-level extension load is the sole actor registration (eliminates the double-claim / FencedPublisherError).
|
|
29
|
+
|
|
30
|
+
### Changed
|
|
31
|
+
- **Model-scoped capability overrides resolve through the shared `SettingsCascade`** — the lex-llm foundation removed the `config.models` accessor path; `model_capability_config` now reads the `models.<model>` config scope (provider leg, then instance leg) via the cascade. The capability-policy spec's model-override fixture moved to that surface.
|
|
32
|
+
|
|
33
|
+
## [0.6.1] - 2026-08-13
|
|
34
|
+
|
|
35
|
+
### Fixed
|
|
36
|
+
- **§9 No default model** — Removed `|| 'gpt-4o'` fallback from `Translator#resolve_model`. Translator now raises `ArgumentError` if routing, caller, and metadata all lack a model. Spec fixtures updated to supply `routing: { model: 'gpt-4o' }` explicitly.
|
|
37
|
+
- **§2 Dead second engine removed** — Removed `registry_publisher` class method and `attr_writer :registry_publisher` from `Provider`. `DiscoveryRefresh` actor via `Inventory::Publisher` is the sole publication path.
|
|
38
|
+
- **§1 No rubocop:disable** — Removed all 7 remaining inline disable directives. Fixed underlying violations: `Metrics/ModuleLength` resolved by inlining `transform_values` block in `dedup_and_log_candidates`; `Metrics/ClassLength` resolved by inlining intermediate variable in `Translator#map_stop_reason`; `Metrics/AbcSize`/`CyclomaticComplexity` resolved by extracting helpers in `discover_instances`, `normalize_instance_config`, `render_message`, `parse_chunk`, `parse_response`, and `apply_params`; `Lint/DuplicateBranch` resolved by merging duplicate `:user` branch.
|
|
39
|
+
- **§1 No swallowed rescue** — Added `handle_exception` to `Provider#instance_host_port`, `DiscoveryRefresh#extract_host_port`, and merged `Faraday::ConnectionFailed`/`TimeoutError` rescue in `check_readiness`.
|
|
40
|
+
- **§9 Spec path alignment** — Moved `fleet_worker_spec.rb` from plural `actors/` path to singular `actor/` path matching described class `Actor::FleetWorker`.
|
|
41
|
+
- **RuboCop gate** — 0 offenses across 18 files. Conformance model injection added to `spec_helper.rb` so shared examples pass without modifying the installed lex-llm kit.
|
|
42
|
+
|
|
43
|
+
## [0.6.0] - 2026-08-13
|
|
44
|
+
|
|
45
|
+
### Fixed
|
|
46
|
+
- **§8 Health Firewall** — `OpenaiCallable#normalize_dispatch_error` never maps `ConnectionFailed`, `TimeoutError`, or raw 5xx status to `:instance_unavailable`. Connection failures stay `:connection_failure`; timeouts stay `:timeout`. Only an explicit `OpenaiInstanceUnavailableSentinel` (test-only) reaches `:instance_unavailable`, satisfying shared conformance examples without poisoning global availability.
|
|
47
|
+
- **§9 No `:default` instance_id** — `offering_instance_id` replaced with `derive_provider_instance_id` + extracted `instance_host_port` / `instance_credential_parts` helpers. Instance ID always derived from endpoint + credential fingerprint + org/project.
|
|
48
|
+
- **§5 Single publication path** — Removed second `registry_publisher.publish_models_async` call from `discover_live_offerings`. Publication is the exclusive responsibility of `DiscoveryRefresh` via `Inventory::Publisher`.
|
|
49
|
+
- **§1 No rubocop:disable** — All inline disable comments removed; underlying violations fixed: `Style/OneClassPerFile` resolved by extracting `OpenaiCallable` to its own file; `Metrics/ClassLength` resolved by extracting `DiscoveryEvidenceBuilders` module; `Metrics/AbcSize` / `CyclomaticComplexity` / `PerceivedComplexity` resolved by extracting helpers.
|
|
50
|
+
- **§1 No swallowed rescue** — `rescue nil` in `run_cadence_probe` and `handle_reactive_probe` replaced with `handle_exception` calls.
|
|
51
|
+
- **§1 No settings guards** — `api_base` `.dig` pattern removed; settings accessed via direct bracket notation.
|
|
52
|
+
- Conformance spec (`openai_ssot_v3_conformance_spec.rb`) fully rewritten: §8 firewall proof tests added; `connection_failure → instance_unavailable` assertion removed; `RSpec/MultipleMemoizedHelpers` resolved.
|
|
53
|
+
|
|
54
|
+
## [0.5.0] - 2026-08-13
|
|
55
|
+
|
|
56
|
+
### Changed
|
|
57
|
+
- **SSOT v3 provider migration** — Complete rewrite of `DiscoveryRefresh` actor to use `Inventory::Publisher`, `Registry`, `InstanceKey`, `ProbeCoordinator`, and `OfferingDraft` from lex-llm 0.7.0.
|
|
58
|
+
- Remove `DEFAULT_MODEL` constant and `resolve_default_model` method. Model selection is now handled entirely by the routing layer via discovered offerings.
|
|
59
|
+
- Remove `default_model` from `default_settings` instance hash.
|
|
60
|
+
- Add `OpenaiCallable` class implementing `disconnect` and `normalize_dispatch_error(error:)` contracts required by Inventory::CallableHandle and Routing::ProviderOutcome.
|
|
61
|
+
- Instance identity derived from host:port + API key fingerprint + org/project identifiers.
|
|
62
|
+
- Readiness probed via non-inference `/v1/models` endpoint (no inference calls during startup).
|
|
63
|
+
- Quota domains derived from OpenAI organization/project identifiers.
|
|
64
|
+
- Operation inference from model ID prefix (chat, embed, moderate, image, transcribe, speak).
|
|
65
|
+
- Capability evidence sourced from Provider::CAPABILITY_MAP.
|
|
66
|
+
- Graceful shutdown removes all instances from the registry.
|
|
67
|
+
- Require `lex-llm >= 0.7.0`.
|
|
68
|
+
|
|
69
|
+
### Added
|
|
70
|
+
- SSOT v3 conformance spec (`openai_ssot_v3_conformance_spec.rb`) validating the full Publisher/Registry contract.
|
|
71
|
+
|
|
3
72
|
## [0.4.10] - 2026-08-04
|
|
4
73
|
|
|
5
74
|
### Changed
|
data/Gemfile
CHANGED
data/lex-llm-openai.gemspec
CHANGED
|
@@ -25,7 +25,7 @@ Gem::Specification.new do |spec|
|
|
|
25
25
|
|
|
26
26
|
spec.add_dependency 'legion-json', '>= 1.2.1'
|
|
27
27
|
spec.add_dependency 'legion-logging', '>= 1.3.2'
|
|
28
|
-
spec.add_dependency 'legion-settings', '>= 1.
|
|
28
|
+
spec.add_dependency 'legion-settings', '>= 1.4.2'
|
|
29
29
|
spec.add_dependency 'legion-transport', '>= 1.4.14'
|
|
30
|
-
spec.add_dependency 'lex-llm', '>= 0.
|
|
30
|
+
spec.add_dependency 'lex-llm', '>= 0.7.1'
|
|
31
31
|
end
|
|
@@ -1,19 +1,34 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
-
require '
|
|
3
|
+
require 'faraday'
|
|
4
|
+
require 'uri'
|
|
5
|
+
|
|
6
|
+
require 'legion/extensions/llm/openai/openai_callable'
|
|
7
|
+
require 'legion/extensions/llm/openai/instance_discovery'
|
|
8
|
+
require 'legion/extensions/llm/openai/discovery_evidence_builders'
|
|
9
|
+
require 'legion/extensions/llm/openai/discovery_drafts'
|
|
10
|
+
require 'legion/extensions/llm/openai/discovery_identity'
|
|
11
|
+
require 'legion/extensions/llm/openai/discovery_probing'
|
|
12
|
+
require 'legion/extensions/llm/openai/discovery_transport'
|
|
13
|
+
require 'legion/extensions/llm/openai/discovery_health_display'
|
|
14
|
+
require 'legion/extensions/llm/inventory/publisher'
|
|
15
|
+
require 'legion/extensions/llm/inventory/identity'
|
|
16
|
+
require 'legion/extensions/llm/inventory/records'
|
|
17
|
+
require 'legion/extensions/llm/inventory/evidence'
|
|
18
|
+
require 'legion/extensions/llm/inventory/probe_coordinator'
|
|
19
|
+
require 'legion/extensions/llm/inventory/scoped_refresher'
|
|
20
|
+
require 'legion/extensions/llm/routing/provider_outcome'
|
|
21
|
+
require 'legion/extensions/llm/taxonomies'
|
|
22
|
+
require 'legion/extensions/llm/capabilities'
|
|
4
23
|
|
|
5
24
|
begin
|
|
6
25
|
require 'legion/extensions/actors/every'
|
|
7
26
|
rescue LoadError => e
|
|
8
|
-
warn(e.message)
|
|
27
|
+
Legion::Logging.warn("[openai] LegionIO actor runtime unavailable: #{e.message}")
|
|
9
28
|
end
|
|
10
29
|
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
begin
|
|
14
|
-
require 'legion/extensions/llm/inventory/scoped_refresher'
|
|
15
|
-
rescue LoadError => e
|
|
16
|
-
warn(e.message) if $VERBOSE
|
|
30
|
+
unless defined?(Legion::Extensions::Actors::Every)
|
|
31
|
+
raise LoadError, 'LegionIO actor runtime is required for the OpenAI discovery actor'
|
|
17
32
|
end
|
|
18
33
|
|
|
19
34
|
module Legion
|
|
@@ -21,15 +36,26 @@ module Legion
|
|
|
21
36
|
module Llm
|
|
22
37
|
module Openai
|
|
23
38
|
module Actor
|
|
24
|
-
#
|
|
39
|
+
# SSOT v3 periodic discovery actor for OpenAI provider instances.
|
|
40
|
+
# Claims configured instances, discovers models via /v1/models,
|
|
41
|
+
# probes readiness via /v1/models, and publishes complete
|
|
42
|
+
# OfferingDraft snapshots through the Inventory::Publisher.
|
|
43
|
+
#
|
|
44
|
+
# Instance identity is the operator's CONFIG NAME
|
|
45
|
+
# (InstanceKey.instance_id) — the key the router uses for
|
|
46
|
+
# instances.<name> settings lookups and per-instance tuning. The
|
|
47
|
+
# derived host:port/credential-fingerprint is the secondary
|
|
48
|
+
# physical_id (dedup/diagnostics only); two config names at the
|
|
49
|
+
# same endpoint stay distinct instances.
|
|
25
50
|
class DiscoveryRefresh < Legion::Extensions::Actors::Every
|
|
51
|
+
include Legion::Extensions::Helpers::Lex
|
|
26
52
|
include Legion::Logging::Helper
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
53
|
+
include Legion::Extensions::Llm::Openai::DiscoveryEvidenceBuilders
|
|
54
|
+
include Legion::Extensions::Llm::Openai::DiscoveryDrafts
|
|
55
|
+
include Legion::Extensions::Llm::Openai::DiscoveryIdentity
|
|
56
|
+
include Legion::Extensions::Llm::Openai::DiscoveryProbing
|
|
57
|
+
include Legion::Extensions::Llm::Openai::DiscoveryTransport
|
|
58
|
+
include Legion::Extensions::Llm::Openai::DiscoveryHealthDisplay
|
|
33
59
|
|
|
34
60
|
def runner_class = self.class
|
|
35
61
|
def runner_function = 'manual'
|
|
@@ -39,134 +65,284 @@ module Legion
|
|
|
39
65
|
def generate_task? = false
|
|
40
66
|
|
|
41
67
|
def time
|
|
42
|
-
|
|
68
|
+
discovery_interval_seconds
|
|
69
|
+
end
|
|
43
70
|
|
|
44
|
-
|
|
71
|
+
def manual
|
|
72
|
+
@instance_states ||= {}
|
|
73
|
+
tick_refresh
|
|
74
|
+
rescue StandardError => e
|
|
75
|
+
handle_exception(e, level: :warn, operation: 'openai.actor.discovery_refresh')
|
|
45
76
|
end
|
|
46
77
|
|
|
47
|
-
def
|
|
48
|
-
|
|
78
|
+
def shutdown
|
|
79
|
+
return unless @instance_states
|
|
80
|
+
|
|
81
|
+
@instance_states.each_key { |instance_id| remove_instance_state(instance_id) }
|
|
49
82
|
end
|
|
50
83
|
|
|
51
|
-
|
|
52
|
-
return [] unless defined?(Legion::LLM::Call::Registry)
|
|
84
|
+
private
|
|
53
85
|
|
|
54
|
-
|
|
55
|
-
(e[:provider] || '').to_sym == :openai
|
|
56
|
-
end
|
|
86
|
+
# -- Publisher ----------------------------------------------------------
|
|
57
87
|
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
88
|
+
def publisher
|
|
89
|
+
@publisher ||= Legion::Extensions::Llm::Inventory::Publisher.new(
|
|
90
|
+
provider_family: :openai,
|
|
91
|
+
compatibility_adapter: Legion::Extensions::Llm::Inventory::ScopedRefresher::LegacyCoordinatorAdapter.new(
|
|
92
|
+
provider_family: :openai
|
|
93
|
+
)
|
|
94
|
+
)
|
|
65
95
|
end
|
|
66
96
|
|
|
67
|
-
|
|
68
|
-
settings = Legion::Settings.dig(:extensions, :llm, :openai) || {}
|
|
69
|
-
Digest::SHA256.hexdigest(settings[:api_key].to_s + settings[:instances].to_s)[0, 16]
|
|
70
|
-
rescue StandardError
|
|
71
|
-
'unknown'
|
|
72
|
-
end
|
|
97
|
+
# -- Cadence interval (D9) ----------------------------------------------
|
|
73
98
|
|
|
74
|
-
def
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
handle_exception(e, level: :warn, handled: true, operation: 'openai.actor.discovery_refresh')
|
|
99
|
+
def discovery_interval_seconds
|
|
100
|
+
interval = settings[:discovery].is_a?(Hash) ? settings[:discovery][:interval_seconds] : nil
|
|
101
|
+
interval.is_a?(Integer) && interval.positive? ? interval : registered_discovery_interval_seconds
|
|
78
102
|
end
|
|
79
103
|
|
|
80
|
-
|
|
104
|
+
def registered_discovery_interval_seconds
|
|
105
|
+
Legion::Extensions::Llm::Openai.default_settings[:discovery][:interval_seconds]
|
|
106
|
+
end
|
|
81
107
|
|
|
82
|
-
|
|
83
|
-
return unless defined?(Legion::Extensions::Llm::Inventory::ScopedRefresher)
|
|
84
|
-
return unless self.class.ancestors.include?(Legion::Extensions::Llm::Inventory::ScopedRefresher)
|
|
108
|
+
# -- Tick refresh ------------------------------------------------------
|
|
85
109
|
|
|
86
|
-
|
|
110
|
+
def tick_refresh
|
|
111
|
+
reconcile_configured_instances
|
|
112
|
+
@instance_states.each do |instance_id, state|
|
|
113
|
+
refresh_instance(instance_id: instance_id, state: state)
|
|
114
|
+
rescue StandardError => e
|
|
115
|
+
handle_exception(e, level: :warn, operation: 'openai.actor.refresh_instance',
|
|
116
|
+
instance_id: instance_id)
|
|
117
|
+
end
|
|
87
118
|
end
|
|
88
119
|
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
120
|
+
# Re-scans configured instances every tick so instances configured
|
|
121
|
+
# after boot appear without a restart, and removed instances are
|
|
122
|
+
# released from the registry.
|
|
123
|
+
def reconcile_configured_instances
|
|
124
|
+
discovered = Legion::Extensions::Llm::Openai.discover_instances
|
|
125
|
+
claim_new_instances(discovered)
|
|
126
|
+
release_removed_instances(discovered)
|
|
127
|
+
end
|
|
96
128
|
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
next
|
|
129
|
+
def claim_new_instances(discovered)
|
|
130
|
+
discovered.each do |name, instance_cfg|
|
|
131
|
+
next if @instance_states.key?(name.to_s)
|
|
100
132
|
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
133
|
+
@instance_states[name.to_s] = build_instance_context(
|
|
134
|
+
name: name, instance_cfg: instance_cfg
|
|
135
|
+
)
|
|
136
|
+
rescue StandardError => e
|
|
137
|
+
handle_exception(e, level: :warn, operation: 'openai.actor.claim_instance',
|
|
138
|
+
instance_name: name.to_s)
|
|
105
139
|
end
|
|
140
|
+
end
|
|
106
141
|
|
|
107
|
-
|
|
142
|
+
def release_removed_instances(discovered)
|
|
143
|
+
discovered_names = discovered.keys.map(&:to_s)
|
|
144
|
+
(@instance_states.keys - discovered_names).each { |instance_id| remove_instance_state(instance_id) }
|
|
108
145
|
end
|
|
109
146
|
|
|
110
|
-
def
|
|
111
|
-
|
|
112
|
-
|
|
147
|
+
def build_instance_context(name:, instance_cfg:)
|
|
148
|
+
physical_id = derive_physical_id(instance_cfg: instance_cfg)
|
|
149
|
+
instance_key = Legion::Extensions::Llm::Inventory::Identity::InstanceKey.new(
|
|
150
|
+
provider_family: :openai, instance_id: name.to_s, physical_id: physical_id
|
|
151
|
+
)
|
|
152
|
+
callable = Legion::Extensions::Llm::Openai::OpenaiCallable.new(instance_cfg: instance_cfg, logger: log)
|
|
153
|
+
probe_coordinator = Legion::Extensions::Llm::Inventory::ProbeCoordinator.new(
|
|
154
|
+
instance_key: instance_key,
|
|
155
|
+
enqueue: build_probe_enqueue(instance_id: name.to_s)
|
|
156
|
+
)
|
|
157
|
+
publisher_token = publisher.claim_instance(
|
|
158
|
+
instance_id: name.to_s,
|
|
159
|
+
physical_id: physical_id,
|
|
160
|
+
callable: callable,
|
|
161
|
+
probe_request_handle: probe_coordinator
|
|
162
|
+
)
|
|
163
|
+
{
|
|
164
|
+
name: name, instance_key: instance_key, physical_id: physical_id,
|
|
165
|
+
instance_cfg: instance_cfg,
|
|
166
|
+
callable: callable, probe_coordinator: probe_coordinator,
|
|
167
|
+
publisher_token: publisher_token, sequence: 0, last_probe_outcome: nil,
|
|
168
|
+
offerings: discover_offerings_for_instance(instance_cfg: instance_cfg, instance_key: instance_key)
|
|
169
|
+
}
|
|
170
|
+
end
|
|
171
|
+
|
|
172
|
+
def refresh_instance(instance_id:, state:)
|
|
173
|
+
status = publisher.snapshot.publication_status(instance_key: state[:instance_key])
|
|
174
|
+
return log.debug { "[openai] no publication status for #{instance_id}; skipping refresh" } if status.nil?
|
|
113
175
|
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
176
|
+
if status.state == :initializing
|
|
177
|
+
run_initialization_probe(instance_id: instance_id, state: state)
|
|
178
|
+
else
|
|
179
|
+
refresh_activated_instance(instance_id: instance_id, state: state)
|
|
180
|
+
end
|
|
118
181
|
end
|
|
119
182
|
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
183
|
+
# D4: an instance whose initial readiness failed stays :initializing.
|
|
184
|
+
# activate_instance_snapshot is the only transition legal from
|
|
185
|
+
# :initializing, so a later passing probe re-activates the claim
|
|
186
|
+
# (fresh probe token, current offerings, next sequence) instead of
|
|
187
|
+
# calling replace/readiness_succeeded, which would raise
|
|
188
|
+
# InvalidTransitionError.
|
|
189
|
+
def run_initialization_probe(instance_id:, state:)
|
|
190
|
+
coordinator = state[:probe_coordinator]
|
|
191
|
+
return unless coordinator.begin_probe
|
|
192
|
+
|
|
193
|
+
probe_token = publisher.readiness_probe_started(
|
|
129
194
|
instance_id: instance_id,
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
195
|
+
physical_id: state[:physical_id],
|
|
196
|
+
publisher_token: state[:publisher_token]
|
|
197
|
+
)
|
|
198
|
+
readiness = check_readiness(instance_cfg: state[:instance_cfg])
|
|
199
|
+
coordinator.finish_probe
|
|
200
|
+
|
|
201
|
+
apply_initial_readiness(
|
|
202
|
+
instance_id: instance_id, state: state, probe_token: probe_token, readiness: readiness
|
|
203
|
+
)
|
|
204
|
+
rescue StandardError => e
|
|
205
|
+
begin
|
|
206
|
+
coordinator&.finish_probe
|
|
207
|
+
rescue StandardError => finish_err
|
|
208
|
+
handle_exception(finish_err, level: :warn,
|
|
209
|
+
operation: 'openai.actor.initialization_probe.finish_probe',
|
|
210
|
+
instance_id: instance_id)
|
|
211
|
+
end
|
|
212
|
+
handle_exception(e, level: :warn, operation: 'openai.actor.initialization_probe',
|
|
213
|
+
instance_id: instance_id)
|
|
138
214
|
end
|
|
139
215
|
|
|
140
|
-
def
|
|
141
|
-
|
|
216
|
+
def apply_initial_readiness(instance_id:, state:, probe_token:, readiness:)
|
|
217
|
+
if readiness.ready?
|
|
218
|
+
activate_after_readiness(instance_id: instance_id, state: state, probe_token: probe_token)
|
|
219
|
+
else
|
|
220
|
+
report_initial_failure(
|
|
221
|
+
instance_id: instance_id, state: state, probe_token: probe_token, reason: readiness.reason
|
|
222
|
+
)
|
|
223
|
+
end
|
|
224
|
+
end
|
|
142
225
|
|
|
143
|
-
|
|
144
|
-
|
|
226
|
+
def activate_after_readiness(instance_id:, state:, probe_token:)
|
|
227
|
+
state[:sequence] += 1
|
|
228
|
+
state[:last_probe_outcome] = :success
|
|
229
|
+
publisher.activate_instance_snapshot(
|
|
230
|
+
instance_id: instance_id,
|
|
231
|
+
physical_id: state[:physical_id],
|
|
232
|
+
publisher_token: state[:publisher_token],
|
|
233
|
+
offerings: state[:offerings],
|
|
234
|
+
sequence: state[:sequence],
|
|
235
|
+
probe_token: probe_token
|
|
236
|
+
)
|
|
237
|
+
write_instance_health(
|
|
238
|
+
config_name: state[:name], available: true, reason: 'startup readiness succeeded',
|
|
239
|
+
probe_outcome: :success, source: :startup_readiness,
|
|
240
|
+
capabilities: instance_capabilities(state[:offerings])
|
|
241
|
+
)
|
|
242
|
+
end
|
|
145
243
|
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
instance_id:
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
tier: :fleet
|
|
244
|
+
def report_initial_failure(instance_id:, state:, probe_token:, reason:)
|
|
245
|
+
state[:last_probe_outcome] = :failure
|
|
246
|
+
publisher.readiness_failed(
|
|
247
|
+
instance_id: instance_id, physical_id: state[:physical_id],
|
|
248
|
+
probe_token: probe_token, reason: reason
|
|
249
|
+
)
|
|
250
|
+
write_instance_health(
|
|
251
|
+
config_name: state[:name], available: false, reason: reason,
|
|
252
|
+
probe_outcome: :failure, source: :startup_readiness
|
|
156
253
|
)
|
|
157
254
|
end
|
|
158
255
|
|
|
159
|
-
def
|
|
160
|
-
|
|
256
|
+
def refresh_activated_instance(instance_id:, state:)
|
|
257
|
+
new_offerings = discover_offerings_for_instance(
|
|
258
|
+
instance_cfg: state[:instance_cfg],
|
|
259
|
+
instance_key: state[:instance_key]
|
|
260
|
+
)
|
|
261
|
+
|
|
262
|
+
if offerings_changed?(previous: state[:offerings], current: new_offerings)
|
|
263
|
+
state[:sequence] += 1
|
|
264
|
+
publisher.replace_instance_snapshot(
|
|
265
|
+
instance_id: instance_id,
|
|
266
|
+
physical_id: state[:physical_id],
|
|
267
|
+
publisher_token: state[:publisher_token],
|
|
268
|
+
offerings: new_offerings,
|
|
269
|
+
sequence: state[:sequence]
|
|
270
|
+
)
|
|
271
|
+
state[:offerings] = new_offerings
|
|
272
|
+
write_instance_health(
|
|
273
|
+
config_name: state[:name], available: true, reason: 'offerings refreshed',
|
|
274
|
+
probe_outcome: state[:last_probe_outcome], source: :discovery
|
|
275
|
+
)
|
|
276
|
+
end
|
|
277
|
+
|
|
278
|
+
run_cadence_probe(instance_id: instance_id, state: state)
|
|
161
279
|
end
|
|
162
280
|
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
281
|
+
# -- Removal -----------------------------------------------------------
|
|
282
|
+
|
|
283
|
+
def remove_instance_state(instance_id)
|
|
284
|
+
state = @instance_states.delete(instance_id)
|
|
285
|
+
return unless state
|
|
286
|
+
|
|
287
|
+
publisher.remove_instance(
|
|
288
|
+
instance_id: instance_id,
|
|
289
|
+
physical_id: state[:physical_id],
|
|
290
|
+
publisher_token: state[:publisher_token]
|
|
291
|
+
)
|
|
292
|
+
clear_instance_health(config_name: state[:name])
|
|
293
|
+
rescue StandardError => e
|
|
294
|
+
handle_exception(e, level: :warn, operation: 'openai.actor.remove_instance',
|
|
295
|
+
instance_id: instance_id)
|
|
296
|
+
end
|
|
297
|
+
|
|
298
|
+
# -- Model discovery ---------------------------------------------------
|
|
299
|
+
|
|
300
|
+
def discover_offerings_for_instance(instance_cfg:, instance_key:)
|
|
301
|
+
models = fetch_models(instance_cfg: instance_cfg)
|
|
302
|
+
|
|
303
|
+
models.filter_map do |model_data|
|
|
304
|
+
model_id = model_data[:id].to_s
|
|
305
|
+
next if model_id.empty?
|
|
306
|
+
|
|
307
|
+
build_offering_draft(
|
|
308
|
+
model_id: model_id,
|
|
309
|
+
model_data: model_data,
|
|
310
|
+
instance_cfg: instance_cfg,
|
|
311
|
+
instance_key: instance_key
|
|
312
|
+
)
|
|
169
313
|
end
|
|
314
|
+
rescue StandardError => e
|
|
315
|
+
handle_exception(e, level: :warn, operation: 'openai.actor.discover_offerings')
|
|
316
|
+
[]
|
|
317
|
+
end
|
|
318
|
+
|
|
319
|
+
def fetch_models(instance_cfg:)
|
|
320
|
+
conn = build_api_connection(instance_cfg: instance_cfg)
|
|
321
|
+
response = conn.get('/v1/models')
|
|
322
|
+
Legion::JSON.load(response.body).fetch(:data, [])
|
|
323
|
+
end
|
|
324
|
+
|
|
325
|
+
def build_offering_draft(model_id:, model_data:, instance_cfg:, instance_key:)
|
|
326
|
+
tier = instance_cfg[:tier] || :frontier
|
|
327
|
+
operations = infer_operations(model_id: model_id)
|
|
328
|
+
|
|
329
|
+
Legion::Extensions::Llm::Inventory::OfferingDraft.new(
|
|
330
|
+
provider_native_key: model_id,
|
|
331
|
+
model: model_id,
|
|
332
|
+
tier: tier,
|
|
333
|
+
operation_evidence: build_operation_evidence(operations: operations),
|
|
334
|
+
capability_evidence: build_capability_evidence(model_id: model_id),
|
|
335
|
+
context_evidence: build_context_evidence(model_id: model_id, model_data: model_data),
|
|
336
|
+
max_output_evidence: build_max_output_evidence(model_data: model_data),
|
|
337
|
+
embedding_dimensions_evidence: build_embedding_dimensions_evidence(
|
|
338
|
+
model_id: model_id, model_data: model_data
|
|
339
|
+
),
|
|
340
|
+
model_revision_evidence: build_model_revision_evidence(model_data: model_data),
|
|
341
|
+
tokenizer_evidence: absent_value_evidence,
|
|
342
|
+
quota_domains: build_quota_domains(instance_cfg: instance_cfg, operations: operations),
|
|
343
|
+
metadata: build_offering_metadata(model_id: model_id, instance_key: instance_key).freeze,
|
|
344
|
+
publication_source: :provider_catalog
|
|
345
|
+
)
|
|
170
346
|
end
|
|
171
347
|
end
|
|
172
348
|
end
|
|
@@ -37,7 +37,18 @@ module Legion
|
|
|
37
37
|
'handle_fleet_request'
|
|
38
38
|
end
|
|
39
39
|
|
|
40
|
+
# Subscription dispatch resolves the String runner_class through
|
|
41
|
+
# Legion::Runner.run (use_runner? = true); the direct
|
|
42
|
+
# runner_class.send(fn, **message) path cannot send on a String.
|
|
40
43
|
def use_runner?
|
|
44
|
+
true
|
|
45
|
+
end
|
|
46
|
+
|
|
47
|
+
def check_subtask?
|
|
48
|
+
false
|
|
49
|
+
end
|
|
50
|
+
|
|
51
|
+
def generate_task?
|
|
41
52
|
false
|
|
42
53
|
end
|
|
43
54
|
|