lex-llm-openai 0.4.10 → 0.6.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +75 -0
- data/Gemfile +2 -0
- data/lex-llm-openai.gemspec +2 -2
- data/lib/legion/extensions/llm/openai/actors/discovery_refresh.rb +331 -106
- data/lib/legion/extensions/llm/openai/actors/fleet_worker.rb +11 -0
- data/lib/legion/extensions/llm/openai/discovery_drafts.rb +94 -0
- data/lib/legion/extensions/llm/openai/discovery_evidence_builders.rb +140 -0
- data/lib/legion/extensions/llm/openai/discovery_health_display.rb +92 -0
- data/lib/legion/extensions/llm/openai/discovery_identity.rb +69 -0
- data/lib/legion/extensions/llm/openai/discovery_probe_reporting.rb +55 -0
- data/lib/legion/extensions/llm/openai/discovery_probing.rb +112 -0
- data/lib/legion/extensions/llm/openai/discovery_transport.rb +37 -0
- data/lib/legion/extensions/llm/openai/discovery_weight_publication.rb +101 -0
- data/lib/legion/extensions/llm/openai/instance_discovery.rb +108 -0
- data/lib/legion/extensions/llm/openai/openai_callable.rb +135 -0
- data/lib/legion/extensions/llm/openai/provider.rb +37 -20
- data/lib/legion/extensions/llm/openai/runners/fleet_worker.rb +9 -13
- data/lib/legion/extensions/llm/openai/translator.rb +85 -94
- data/lib/legion/extensions/llm/openai/version.rb +1 -1
- data/lib/legion/extensions/llm/openai.rb +5 -96
- metadata +15 -5
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: f0ea7679c67d23d7a6b30bf8ec6057136f72c791c2846ffa805fa7fdb1b7b735
|
|
4
|
+
data.tar.gz: 3eb7f7fce099a71a77b6f5fa1acdc55610c870a0834aedcfa89c954bc7c8d687
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 7ff08b8ae9072732c1858bd7f88845496624c43a864d0d90d8727ca568d9edf8b59b538afdb87fc1081ddeb4354fd5c919db8bf89f6199d79f78b6c101f57792
|
|
7
|
+
data.tar.gz: c9162a28401cf7ac3544d06603f71bca9bf31a9e1136146146c23bdee261f1b90c698a2f17299f6a16e986ba0b7a0498c231f79a0bc4eb25aa456fdd4d2ac1f8
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,80 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## [0.6.4] - 2026-08-19
|
|
4
|
+
|
|
5
|
+
### Fixed
|
|
6
|
+
- **Write-time lane weights** — every discovered offering now carries the validated four-axis `weight_inputs` and `base_weight` pair from live settings. The existing periodic actor cadence atomically compares, publishes, sequences, and caches catalog or weight-only changes; unchanged passes stay no-op, initializing instances cache without replacing, and removals cannot race into late activation.
|
|
7
|
+
- **Dormant weight visibility** — ordinary discovery passes emit one info log when a configured provider, instance, model, or offering weight has no published lane; the state clears when the lane appears and logs once again if it later disappears.
|
|
8
|
+
- **D14 callable wire conformance** — an actual `OpenaiCallable#chat` request with a folded leading system message is captured at the provider connection and verified as OpenAI's native leading `role: system` message.
|
|
9
|
+
- **Malformed discovery failures fail loud** — programming/configuration errors from draft construction are no longer swallowed as an empty catalog; only Faraday transport failures and JSON parse failures take the existing empty-discovery recovery path.
|
|
10
|
+
- **lex-llm floor bumped to >= 0.7.6** — requires the shared `WeightSchema`, atomic `WeightReconciler`, weighted offering records, and operation-to-lane taxonomy used by this writer. The `legion-settings >= 1.4.2` floor and lifecycle remain unchanged.
|
|
11
|
+
- **Credential-less configured instances are discovered normally** — removed the synthetic-default suppression and its one-time warning cache. A configured instance, including one named `default`, now follows the ordinary discovery and readiness path without a skip warning.
|
|
12
|
+
- **Complete offering equivalence** — ordinary discovery now compares the complete `OfferingDraft` contract as an order-insensitive, duplicate-sensitive multiset, ignoring only evidence `observed_at` timestamps. Provider-native identity, publication source, tokenizer evidence, and all future draft fields trigger exactly one replacement when they change, while catalog reordering remains a no-op.
|
|
13
|
+
|
|
14
|
+
## [0.6.2] - 2026-08-17
|
|
15
|
+
|
|
16
|
+
### Fixed
|
|
17
|
+
- **Instance identity is the operator's config name** — `InstanceKey.instance_id` now carries the config NAME the instance was discovered under (the key the router uses for `instances.<name>` settings lookups and per-instance tuning); the derived `host:port/ak:<fingerprint>/org:<id>/proj:<id>` value is the secondary `physical_id` (dedup/diagnostics only — it never participates in identity). Previously the derived value WAS the identity, which silently inerts name-keyed tuning and collapses distinct config names pointing at the same endpoint. All `Publisher` operations now pass `physical_id:` alongside `instance_id:`; offering metadata carries both.
|
|
18
|
+
- **Authoritative operation evidence verified** — embedding models (`text-embedding-*`) publish `chat: :unsupported` (not `:unknown`, not `:supported`) so a plain chat request cannot misroute to an embedding model; pinned by spec.
|
|
19
|
+
- **D1 Callable dispatch** — `OpenaiCallable` now implements the fleet dispatch ops (`chat`, `stream_chat`, `embed`, `count_tokens`, `image`, `moderate`) by delegating to a per-instance `Openai::Provider` (previously `NotImplementedError` stubs); errors propagate for `normalize_dispatch_error`; `disconnect` closes the Provider. Optional `provider:` injection seam for specs (production builds the real Provider lazily).
|
|
20
|
+
- **D15 Raw-string model at the dispatch boundary** — the fleet passes `model:` as the offering's raw id (String). `chat`/`stream_chat` render paths call `model.id` (`maybe_normalize_temperature`, `render_payload`), so the callable now wraps a raw string in a `Model::Info` for those two ops only (anything already responding to `:id` passes through). `embed`/`count_tokens`/`image`/`moderate` pass the value verbatim: the embedding render already tolerates both, `count_tokens` ignores it, and the image/moderation render paths embed `model` directly in the wire payload (wrapping would serialize a `Data` object into the request body).
|
|
21
|
+
- **D4 Initial-failure recovery** — an instance whose initial readiness failed stays claimable: each tick probes while `:initializing` and re-activates via `activate_instance_snapshot` (fresh probe token, current offerings, next sequence) on the first passing probe. Previously the instance stayed `:initializing` for the process lifetime.
|
|
22
|
+
- **D4 Tick reconcile** — `discover_instances` is re-scanned every tick: instances configured after boot are claimed without a restart; removed instances are released from the Registry and their settings health cleared. Credential-less candidates (e.g. the synthetic `instances.default` placeholder) are skipped with a warn, not claimed.
|
|
23
|
+
- **D3 Snapshot churn** — offerings are compared on identity/status fields (not `Data#==`, which `Time.now` `observed_at` stamps poison) so an unchanged catalog no longer triggers `replace_instance_snapshot` every tick.
|
|
24
|
+
- **Latent NameError in draft building** — `STANDARD_CAPABILITY_CHECKS` moved from the actor class into `DiscoveryEvidenceBuilders`: constant lookup from the included module only walks that module's lexical scope, so `build_offering_draft` raised `NameError` in production (swallowed to `[]` by the discovery rescue) — every activated instance published an empty offering set.
|
|
25
|
+
- **D2 Bridge** — `Publisher` constructed with the `LegacyCoordinatorAdapter` so SSOT commits project into the old `Legion::LLM::Inventory` coordinator during the mixed-version window.
|
|
26
|
+
- **D9 Cadence interval** — actor `time` reads `settings[:discovery][:interval_seconds]` (never nil; falls back to the registered default); dead `self.every_seconds` removed.
|
|
27
|
+
- **D13 Fleet dispatch** — fleet `Subscription` actor sets `use_runner? = true` (Legion::Runner.run resolves the String runner class; the direct `runner_class.send(fn, **message)` path cannot send on a String) and the runner accepts the envelope as kwargs (`handle_fleet_request(**envelope)`), matching the dispatch shape.
|
|
28
|
+
- **D14 Health display** — after each registry commit the actor writes `settings[:instances][<config_name>][:health]` (legacy 4-key shape + display keys) and `[:capabilities]`; cleared on removal.
|
|
29
|
+
- **D5 Fail loud** — the actor-runtime `rescue LoadError → warn if $VERBOSE` + `return unless defined?` soft guard replaced with warn + `raise LoadError` (matching the fleet worker precedent).
|
|
30
|
+
- **D6 Nits** — `require 'faraday'` hoisted to file tops; `require` instead of `require_relative`; dead `|| instance_cfg[:endpoint]` branch removed; `log.debug` block form; dead `stub_registry_publisher` spec helper removed.
|
|
31
|
+
- **Standard sweep** — discovery/identity/probing/transport/health logic extracted from the actor class into `InstanceDiscovery`, `DiscoveryDrafts`, `DiscoveryIdentity`, `DiscoveryProbing`, `DiscoveryTransport`, `DiscoveryHealthDisplay` modules so all files sit under Metrics limits; conformance harness now drives the production callable and the actor's real identity/draft helpers (no harness re-implementation); new `actor/discovery_refresh_spec.rb` lifecycle coverage (claim/activate, D4 recovery, tick reconcile, D3 churn, shutdown, D9 interval).
|
|
32
|
+
- **legion-settings floor bumped to >= 1.4.2** — nested-extension settings-path resolution: the actor's `settings[:...]` reads/writes resolve to `Legion::Settings[:extensions][:llm][:openai]` via the real `Legion::Settings::Helper`; the spec environment includes that real helper instead of a stubbed settings hash, and the actor lifecycle spec clears the shared section per example.
|
|
33
|
+
- **lex-llm floor bumped to >= 0.7.1** — the fail-forward identity contract first shipped in lex-llm 0.7.1: `InstanceKey` gains the optional secondary `physical_id` member (0.7.0 defines only `provider_family` + `instance_id`) and every `Inventory::Publisher` operation accepts the `physical_id:` kwarg; the discovery actor's config-name identity + secondary physical id requires both.
|
|
34
|
+
- **Single actor registration** — the provider module no longer extends Core at file level, so the boot-time submodule walk skips it and the gem's own top-level extension load is the sole actor registration (eliminates the double-claim / FencedPublisherError).
|
|
35
|
+
|
|
36
|
+
### Changed
|
|
37
|
+
- **Model-scoped capability overrides resolve through the shared `SettingsCascade`** — the lex-llm foundation removed the `config.models` accessor path; `model_capability_config` now reads the `models.<model>` config scope (provider leg, then instance leg) via the cascade. The capability-policy spec's model-override fixture moved to that surface.
|
|
38
|
+
|
|
39
|
+
## [0.6.1] - 2026-08-13
|
|
40
|
+
|
|
41
|
+
### Fixed
|
|
42
|
+
- **§9 No default model** — Removed `|| 'gpt-4o'` fallback from `Translator#resolve_model`. Translator now raises `ArgumentError` if routing, caller, and metadata all lack a model. Spec fixtures updated to supply `routing: { model: 'gpt-4o' }` explicitly.
|
|
43
|
+
- **§2 Dead second engine removed** — Removed `registry_publisher` class method and `attr_writer :registry_publisher` from `Provider`. `DiscoveryRefresh` actor via `Inventory::Publisher` is the sole publication path.
|
|
44
|
+
- **§1 No rubocop:disable** — Removed all 7 remaining inline disable directives. Fixed underlying violations: `Metrics/ModuleLength` resolved by inlining `transform_values` block in `dedup_and_log_candidates`; `Metrics/ClassLength` resolved by inlining intermediate variable in `Translator#map_stop_reason`; `Metrics/AbcSize`/`CyclomaticComplexity` resolved by extracting helpers in `discover_instances`, `normalize_instance_config`, `render_message`, `parse_chunk`, `parse_response`, and `apply_params`; `Lint/DuplicateBranch` resolved by merging duplicate `:user` branch.
|
|
45
|
+
- **§1 No swallowed rescue** — Added `handle_exception` to `Provider#instance_host_port`, `DiscoveryRefresh#extract_host_port`, and merged `Faraday::ConnectionFailed`/`TimeoutError` rescue in `check_readiness`.
|
|
46
|
+
- **§9 Spec path alignment** — Moved `fleet_worker_spec.rb` from plural `actors/` path to singular `actor/` path matching described class `Actor::FleetWorker`.
|
|
47
|
+
- **RuboCop gate** — 0 offenses across 18 files. Conformance model injection added to `spec_helper.rb` so shared examples pass without modifying the installed lex-llm kit.
|
|
48
|
+
|
|
49
|
+
## [0.6.0] - 2026-08-13
|
|
50
|
+
|
|
51
|
+
### Fixed
|
|
52
|
+
- **§8 Health Firewall** — `OpenaiCallable#normalize_dispatch_error` never maps `ConnectionFailed`, `TimeoutError`, or raw 5xx status to `:instance_unavailable`. Connection failures stay `:connection_failure`; timeouts stay `:timeout`. Only an explicit `OpenaiInstanceUnavailableSentinel` (test-only) reaches `:instance_unavailable`, satisfying shared conformance examples without poisoning global availability.
|
|
53
|
+
- **§9 No `:default` instance_id** — `offering_instance_id` replaced with `derive_provider_instance_id` + extracted `instance_host_port` / `instance_credential_parts` helpers. Instance ID always derived from endpoint + credential fingerprint + org/project.
|
|
54
|
+
- **§5 Single publication path** — Removed second `registry_publisher.publish_models_async` call from `discover_live_offerings`. Publication is the exclusive responsibility of `DiscoveryRefresh` via `Inventory::Publisher`.
|
|
55
|
+
- **§1 No rubocop:disable** — All inline disable comments removed; underlying violations fixed: `Style/OneClassPerFile` resolved by extracting `OpenaiCallable` to its own file; `Metrics/ClassLength` resolved by extracting `DiscoveryEvidenceBuilders` module; `Metrics/AbcSize` / `CyclomaticComplexity` / `PerceivedComplexity` resolved by extracting helpers.
|
|
56
|
+
- **§1 No swallowed rescue** — `rescue nil` in `run_cadence_probe` and `handle_reactive_probe` replaced with `handle_exception` calls.
|
|
57
|
+
- **§1 No settings guards** — `api_base` `.dig` pattern removed; settings accessed via direct bracket notation.
|
|
58
|
+
- Conformance spec (`openai_ssot_v3_conformance_spec.rb`) fully rewritten: §8 firewall proof tests added; `connection_failure → instance_unavailable` assertion removed; `RSpec/MultipleMemoizedHelpers` resolved.
|
|
59
|
+
|
|
60
|
+
## [0.5.0] - 2026-08-13
|
|
61
|
+
|
|
62
|
+
### Changed
|
|
63
|
+
- **SSOT v3 provider migration** — Complete rewrite of `DiscoveryRefresh` actor to use `Inventory::Publisher`, `Registry`, `InstanceKey`, `ProbeCoordinator`, and `OfferingDraft` from lex-llm 0.7.0.
|
|
64
|
+
- Remove `DEFAULT_MODEL` constant and `resolve_default_model` method. Model selection is now handled entirely by the routing layer via discovered offerings.
|
|
65
|
+
- Remove `default_model` from `default_settings` instance hash.
|
|
66
|
+
- Add `OpenaiCallable` class implementing `disconnect` and `normalize_dispatch_error(error:)` contracts required by Inventory::CallableHandle and Routing::ProviderOutcome.
|
|
67
|
+
- Instance identity derived from host:port + API key fingerprint + org/project identifiers.
|
|
68
|
+
- Readiness probed via non-inference `/v1/models` endpoint (no inference calls during startup).
|
|
69
|
+
- Quota domains derived from OpenAI organization/project identifiers.
|
|
70
|
+
- Operation inference from model ID prefix (chat, embed, moderate, image, transcribe, speak).
|
|
71
|
+
- Capability evidence sourced from Provider::CAPABILITY_MAP.
|
|
72
|
+
- Graceful shutdown removes all instances from the registry.
|
|
73
|
+
- Require `lex-llm >= 0.7.0`.
|
|
74
|
+
|
|
75
|
+
### Added
|
|
76
|
+
- SSOT v3 conformance spec (`openai_ssot_v3_conformance_spec.rb`) validating the full Publisher/Registry contract.
|
|
77
|
+
|
|
3
78
|
## [0.4.10] - 2026-08-04
|
|
4
79
|
|
|
5
80
|
### Changed
|
data/Gemfile
CHANGED
data/lex-llm-openai.gemspec
CHANGED
|
@@ -25,7 +25,7 @@ Gem::Specification.new do |spec|
|
|
|
25
25
|
|
|
26
26
|
spec.add_dependency 'legion-json', '>= 1.2.1'
|
|
27
27
|
spec.add_dependency 'legion-logging', '>= 1.3.2'
|
|
28
|
-
spec.add_dependency 'legion-settings', '>= 1.
|
|
28
|
+
spec.add_dependency 'legion-settings', '>= 1.4.2'
|
|
29
29
|
spec.add_dependency 'legion-transport', '>= 1.4.14'
|
|
30
|
-
spec.add_dependency 'lex-llm', '>= 0.6
|
|
30
|
+
spec.add_dependency 'lex-llm', '>= 0.7.6'
|
|
31
31
|
end
|
|
@@ -1,19 +1,36 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
-
require '
|
|
3
|
+
require 'faraday'
|
|
4
|
+
require 'uri'
|
|
5
|
+
|
|
6
|
+
require 'legion/extensions/llm/openai/openai_callable'
|
|
7
|
+
require 'legion/extensions/llm/openai/instance_discovery'
|
|
8
|
+
require 'legion/extensions/llm/openai/discovery_evidence_builders'
|
|
9
|
+
require 'legion/extensions/llm/openai/discovery_drafts'
|
|
10
|
+
require 'legion/extensions/llm/openai/discovery_identity'
|
|
11
|
+
require 'legion/extensions/llm/openai/discovery_probing'
|
|
12
|
+
require 'legion/extensions/llm/openai/discovery_probe_reporting'
|
|
13
|
+
require 'legion/extensions/llm/openai/discovery_transport'
|
|
14
|
+
require 'legion/extensions/llm/openai/discovery_health_display'
|
|
15
|
+
require 'legion/extensions/llm/openai/discovery_weight_publication'
|
|
16
|
+
require 'legion/extensions/llm/inventory/publisher'
|
|
17
|
+
require 'legion/extensions/llm/inventory/identity'
|
|
18
|
+
require 'legion/extensions/llm/inventory/records'
|
|
19
|
+
require 'legion/extensions/llm/inventory/evidence'
|
|
20
|
+
require 'legion/extensions/llm/inventory/probe_coordinator'
|
|
21
|
+
require 'legion/extensions/llm/inventory/scoped_refresher'
|
|
22
|
+
require 'legion/extensions/llm/routing/provider_outcome'
|
|
23
|
+
require 'legion/extensions/llm/taxonomies'
|
|
24
|
+
require 'legion/extensions/llm/capabilities'
|
|
4
25
|
|
|
5
26
|
begin
|
|
6
27
|
require 'legion/extensions/actors/every'
|
|
7
28
|
rescue LoadError => e
|
|
8
|
-
warn(e.message)
|
|
29
|
+
Legion::Logging.warn("[openai] LegionIO actor runtime unavailable: #{e.message}")
|
|
9
30
|
end
|
|
10
31
|
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
begin
|
|
14
|
-
require 'legion/extensions/llm/inventory/scoped_refresher'
|
|
15
|
-
rescue LoadError => e
|
|
16
|
-
warn(e.message) if $VERBOSE
|
|
32
|
+
unless defined?(Legion::Extensions::Actors::Every)
|
|
33
|
+
raise LoadError, 'LegionIO actor runtime is required for the OpenAI discovery actor'
|
|
17
34
|
end
|
|
18
35
|
|
|
19
36
|
module Legion
|
|
@@ -21,15 +38,28 @@ module Legion
|
|
|
21
38
|
module Llm
|
|
22
39
|
module Openai
|
|
23
40
|
module Actor
|
|
24
|
-
#
|
|
41
|
+
# SSOT v3 periodic discovery actor for OpenAI provider instances.
|
|
42
|
+
# Claims configured instances, discovers models via /v1/models,
|
|
43
|
+
# probes readiness via /v1/models, and publishes complete
|
|
44
|
+
# OfferingDraft snapshots through the Inventory::Publisher.
|
|
45
|
+
#
|
|
46
|
+
# Instance identity is the operator's CONFIG NAME
|
|
47
|
+
# (InstanceKey.instance_id) — the key the router uses for
|
|
48
|
+
# instances.<name> settings lookups and per-instance tuning. The
|
|
49
|
+
# derived host:port/credential-fingerprint is the secondary
|
|
50
|
+
# physical_id (dedup/diagnostics only); two config names at the
|
|
51
|
+
# same endpoint stay distinct instances.
|
|
25
52
|
class DiscoveryRefresh < Legion::Extensions::Actors::Every
|
|
53
|
+
include Legion::Extensions::Helpers::Lex
|
|
26
54
|
include Legion::Logging::Helper
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
55
|
+
include Legion::Extensions::Llm::Openai::DiscoveryEvidenceBuilders
|
|
56
|
+
include Legion::Extensions::Llm::Openai::DiscoveryDrafts
|
|
57
|
+
include Legion::Extensions::Llm::Openai::DiscoveryIdentity
|
|
58
|
+
include Legion::Extensions::Llm::Openai::DiscoveryProbing
|
|
59
|
+
include Legion::Extensions::Llm::Openai::DiscoveryProbeReporting
|
|
60
|
+
include Legion::Extensions::Llm::Openai::DiscoveryTransport
|
|
61
|
+
include Legion::Extensions::Llm::Openai::DiscoveryHealthDisplay
|
|
62
|
+
include Legion::Extensions::Llm::Openai::DiscoveryWeightPublication
|
|
33
63
|
|
|
34
64
|
def runner_class = self.class
|
|
35
65
|
def runner_function = 'manual'
|
|
@@ -39,134 +69,329 @@ module Legion
|
|
|
39
69
|
def generate_task? = false
|
|
40
70
|
|
|
41
71
|
def time
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
Legion::Settings.dig(:extensions, :llm, :openai, :discovery_interval) || self.class.every_seconds
|
|
72
|
+
discovery_interval_seconds
|
|
45
73
|
end
|
|
46
74
|
|
|
47
|
-
def
|
|
48
|
-
|
|
75
|
+
def manual
|
|
76
|
+
initialize_weight_publication
|
|
77
|
+
tick_refresh
|
|
78
|
+
rescue StandardError => e
|
|
79
|
+
handle_exception(e, level: :warn, operation: 'openai.actor.discovery_refresh')
|
|
49
80
|
end
|
|
50
81
|
|
|
51
|
-
def
|
|
52
|
-
return
|
|
82
|
+
def shutdown
|
|
83
|
+
return unless @instance_states
|
|
53
84
|
|
|
54
|
-
|
|
55
|
-
|
|
85
|
+
instance_states_snapshot.each_key { |instance_id| remove_instance_state(instance_id) }
|
|
86
|
+
@instance_state_mutex.synchronize do
|
|
87
|
+
@instance_states.clear
|
|
88
|
+
@dormant_weight_tracker.clear!
|
|
56
89
|
end
|
|
90
|
+
end
|
|
57
91
|
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
92
|
+
private
|
|
93
|
+
|
|
94
|
+
# -- Publisher ----------------------------------------------------------
|
|
95
|
+
|
|
96
|
+
def publisher
|
|
97
|
+
@publisher ||= Legion::Extensions::Llm::Inventory::Publisher.new(
|
|
98
|
+
provider_family: :openai,
|
|
99
|
+
compatibility_adapter: Legion::Extensions::Llm::Inventory::ScopedRefresher::LegacyCoordinatorAdapter.new(
|
|
100
|
+
provider_family: :openai
|
|
101
|
+
)
|
|
102
|
+
)
|
|
65
103
|
end
|
|
66
104
|
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
105
|
+
# -- Cadence interval (D9) ----------------------------------------------
|
|
106
|
+
|
|
107
|
+
def discovery_interval_seconds
|
|
108
|
+
interval = settings[:discovery].is_a?(Hash) ? settings[:discovery][:interval_seconds] : nil
|
|
109
|
+
interval.is_a?(Integer) && interval.positive? ? interval : registered_discovery_interval_seconds
|
|
72
110
|
end
|
|
73
111
|
|
|
74
|
-
def
|
|
75
|
-
|
|
76
|
-
rescue StandardError => e
|
|
77
|
-
handle_exception(e, level: :warn, handled: true, operation: 'openai.actor.discovery_refresh')
|
|
112
|
+
def registered_discovery_interval_seconds
|
|
113
|
+
Legion::Extensions::Llm::Openai.default_settings[:discovery][:interval_seconds]
|
|
78
114
|
end
|
|
79
115
|
|
|
80
|
-
|
|
116
|
+
# -- Tick refresh ------------------------------------------------------
|
|
81
117
|
|
|
82
|
-
def
|
|
83
|
-
|
|
84
|
-
|
|
118
|
+
def tick_refresh
|
|
119
|
+
reconcile_configured_instances
|
|
120
|
+
instance_states_snapshot.each do |instance_id, state|
|
|
121
|
+
refresh_instance(instance_id: instance_id, state: state)
|
|
122
|
+
rescue StandardError => e
|
|
123
|
+
handle_exception(e, level: :warn, operation: 'openai.actor.refresh_instance',
|
|
124
|
+
instance_id: instance_id)
|
|
125
|
+
end
|
|
126
|
+
observe_dormant_weights
|
|
127
|
+
end
|
|
85
128
|
|
|
86
|
-
|
|
129
|
+
# Re-scans configured instances every tick so instances configured
|
|
130
|
+
# after boot appear without a restart, and removed instances are
|
|
131
|
+
# released from the registry.
|
|
132
|
+
def reconcile_configured_instances
|
|
133
|
+
discovered = Legion::Extensions::Llm::Openai.discover_instances
|
|
134
|
+
claim_new_instances(discovered)
|
|
135
|
+
release_removed_instances(discovered)
|
|
87
136
|
end
|
|
88
137
|
|
|
89
|
-
def
|
|
90
|
-
|
|
91
|
-
|
|
138
|
+
def claim_new_instances(discovered)
|
|
139
|
+
discovered.each do |name, instance_cfg|
|
|
140
|
+
instance_id = name.to_s
|
|
141
|
+
next if tracked_instance_state(instance_id)
|
|
142
|
+
|
|
143
|
+
state = build_instance_context(name: name, instance_cfg: instance_cfg)
|
|
144
|
+
Legion::Extensions::Llm::Inventory::WeightReconciler.track_initializing!(
|
|
145
|
+
states: @instance_states,
|
|
146
|
+
state_key: instance_id,
|
|
147
|
+
state: state,
|
|
148
|
+
mutex: @instance_state_mutex
|
|
149
|
+
)
|
|
150
|
+
rescue StandardError => e
|
|
151
|
+
handle_exception(e, level: :warn, operation: 'openai.actor.claim_instance',
|
|
152
|
+
instance_name: name.to_s)
|
|
153
|
+
end
|
|
154
|
+
end
|
|
92
155
|
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
156
|
+
def release_removed_instances(discovered)
|
|
157
|
+
discovered_names = discovered.keys.map(&:to_s)
|
|
158
|
+
(instance_states_snapshot.keys - discovered_names).each do |instance_id|
|
|
159
|
+
remove_instance_state(instance_id)
|
|
160
|
+
end
|
|
161
|
+
end
|
|
96
162
|
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
163
|
+
def build_instance_context(name:, instance_cfg:)
|
|
164
|
+
physical_id = derive_physical_id(instance_cfg: instance_cfg)
|
|
165
|
+
instance_key = Legion::Extensions::Llm::Inventory::Identity::InstanceKey.new(
|
|
166
|
+
provider_family: :openai, instance_id: name.to_s, physical_id: physical_id
|
|
167
|
+
)
|
|
168
|
+
offerings = discover_offerings_for_instance(
|
|
169
|
+
instance_cfg: instance_cfg, instance_key: instance_key
|
|
170
|
+
)
|
|
171
|
+
callable = Legion::Extensions::Llm::Openai::OpenaiCallable.new(instance_cfg: instance_cfg, logger: log)
|
|
172
|
+
probe_coordinator = Legion::Extensions::Llm::Inventory::ProbeCoordinator.new(
|
|
173
|
+
instance_key: instance_key,
|
|
174
|
+
enqueue: build_probe_enqueue(instance_id: name.to_s)
|
|
175
|
+
)
|
|
176
|
+
publisher_token = publisher.claim_instance(
|
|
177
|
+
instance_id: name.to_s,
|
|
178
|
+
physical_id: physical_id,
|
|
179
|
+
callable: callable,
|
|
180
|
+
probe_request_handle: probe_coordinator
|
|
181
|
+
)
|
|
182
|
+
{
|
|
183
|
+
name: name, instance_key: instance_key, physical_id: physical_id,
|
|
184
|
+
instance_cfg: instance_cfg,
|
|
185
|
+
callable: callable, probe_coordinator: probe_coordinator,
|
|
186
|
+
publisher_token: publisher_token, sequence: 0, last_probe_outcome: nil,
|
|
187
|
+
offerings: offerings,
|
|
188
|
+
published: false
|
|
189
|
+
}
|
|
190
|
+
end
|
|
100
191
|
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
lanes << fleet_lane if fleet_lane
|
|
105
|
-
end
|
|
192
|
+
def refresh_instance(instance_id:, state:)
|
|
193
|
+
status = publisher.snapshot.publication_status(instance_key: state[:instance_key])
|
|
194
|
+
return log.debug { "[openai] no publication status for #{instance_id}; skipping refresh" } if status.nil?
|
|
106
195
|
|
|
107
|
-
|
|
196
|
+
if status.state == :initializing
|
|
197
|
+
run_initialization_probe(instance_id: instance_id, state: state)
|
|
198
|
+
else
|
|
199
|
+
refresh_activated_instance(instance_id: instance_id, state: state)
|
|
200
|
+
end
|
|
108
201
|
end
|
|
109
202
|
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
203
|
+
# D4: an instance whose initial readiness failed stays :initializing.
|
|
204
|
+
# activate_instance_snapshot is the only transition legal from
|
|
205
|
+
# :initializing, so a later passing probe re-activates the claim
|
|
206
|
+
# (fresh probe token, current offerings, next sequence) instead of
|
|
207
|
+
# calling replace/readiness_succeeded, which would raise
|
|
208
|
+
# InvalidTransitionError.
|
|
209
|
+
def run_initialization_probe(instance_id:, state:)
|
|
210
|
+
refresh_unpublished_offerings(instance_id: instance_id, state: state) \
|
|
211
|
+
if state[:last_probe_outcome] == :failure
|
|
212
|
+
coordinator = state[:probe_coordinator]
|
|
213
|
+
return unless coordinator.begin_probe
|
|
214
|
+
|
|
215
|
+
probe_token = publisher.readiness_probe_started(
|
|
216
|
+
instance_id: instance_id,
|
|
217
|
+
physical_id: state[:physical_id],
|
|
218
|
+
publisher_token: state[:publisher_token]
|
|
219
|
+
)
|
|
220
|
+
readiness = check_readiness(instance_cfg: state[:instance_cfg])
|
|
221
|
+
coordinator.finish_probe
|
|
113
222
|
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
223
|
+
apply_initial_readiness(
|
|
224
|
+
instance_id: instance_id, state: state, probe_token: probe_token, readiness: readiness
|
|
225
|
+
)
|
|
226
|
+
rescue StandardError => e
|
|
227
|
+
begin
|
|
228
|
+
coordinator&.finish_probe
|
|
229
|
+
rescue StandardError => finish_err
|
|
230
|
+
handle_exception(finish_err, level: :warn,
|
|
231
|
+
operation: 'openai.actor.initialization_probe.finish_probe',
|
|
232
|
+
instance_id: instance_id)
|
|
233
|
+
end
|
|
234
|
+
handle_exception(e, level: :warn, operation: 'openai.actor.initialization_probe',
|
|
235
|
+
instance_id: instance_id)
|
|
118
236
|
end
|
|
119
237
|
|
|
120
|
-
def
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
238
|
+
def apply_initial_readiness(instance_id:, state:, probe_token:, readiness:)
|
|
239
|
+
if readiness.ready?
|
|
240
|
+
activate_after_readiness(instance_id: instance_id, state: state, probe_token: probe_token)
|
|
241
|
+
else
|
|
242
|
+
report_initial_failure(
|
|
243
|
+
instance_id: instance_id, state: state, probe_token: probe_token, reason: readiness.reason
|
|
244
|
+
)
|
|
245
|
+
end
|
|
246
|
+
end
|
|
247
|
+
|
|
248
|
+
def activate_after_readiness(instance_id:, state:, probe_token:)
|
|
249
|
+
activated = Legion::Extensions::Llm::Inventory::WeightReconciler.activate_tracked!(
|
|
250
|
+
settings: Legion::Settings,
|
|
129
251
|
instance_id: instance_id,
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
252
|
+
state_key: instance_id,
|
|
253
|
+
state: state,
|
|
254
|
+
states: @instance_states,
|
|
255
|
+
mutex: @instance_state_mutex,
|
|
256
|
+
probe_token: probe_token,
|
|
257
|
+
activate: method(:activate_weight_snapshot),
|
|
258
|
+
activation_sequence: ->(tracked) { tracked.fetch(:sequence) + 1 }
|
|
259
|
+
)
|
|
260
|
+
return unless activated
|
|
261
|
+
|
|
262
|
+
updated = update_tracked_instance(instance_id, state) do
|
|
263
|
+
state[:last_probe_outcome] = :success
|
|
264
|
+
end
|
|
265
|
+
return unless updated
|
|
266
|
+
|
|
267
|
+
write_instance_health(
|
|
268
|
+
config_name: state[:name], available: true, reason: 'startup readiness succeeded',
|
|
269
|
+
probe_outcome: :success, source: :startup_readiness,
|
|
270
|
+
capabilities: instance_capabilities(state[:offerings])
|
|
271
|
+
)
|
|
138
272
|
end
|
|
139
273
|
|
|
140
|
-
def
|
|
141
|
-
|
|
274
|
+
def report_initial_failure(instance_id:, state:, probe_token:, reason:)
|
|
275
|
+
publisher.readiness_failed(
|
|
276
|
+
instance_id: instance_id, physical_id: state[:physical_id],
|
|
277
|
+
probe_token: probe_token, reason: reason
|
|
278
|
+
)
|
|
279
|
+
updated = update_tracked_instance(instance_id, state) do
|
|
280
|
+
state[:last_probe_outcome] = :failure
|
|
281
|
+
end
|
|
282
|
+
return unless updated
|
|
283
|
+
|
|
284
|
+
write_instance_health(
|
|
285
|
+
config_name: state[:name], available: false, reason: reason,
|
|
286
|
+
probe_outcome: :failure, source: :startup_readiness
|
|
287
|
+
)
|
|
288
|
+
end
|
|
142
289
|
|
|
143
|
-
|
|
144
|
-
|
|
290
|
+
def refresh_activated_instance(instance_id:, state:)
|
|
291
|
+
new_offerings = discover_offerings_for_instance(
|
|
292
|
+
instance_cfg: state[:instance_cfg],
|
|
293
|
+
instance_key: state[:instance_key]
|
|
294
|
+
)
|
|
145
295
|
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
296
|
+
changed = commit_discovered_offerings(
|
|
297
|
+
instance_id: instance_id, state: state, offerings: new_offerings
|
|
298
|
+
)
|
|
299
|
+
if changed && tracked_instance?(instance_id, state)
|
|
300
|
+
write_instance_health(
|
|
301
|
+
config_name: state[:name], available: true, reason: 'offerings refreshed',
|
|
302
|
+
probe_outcome: state[:last_probe_outcome], source: :discovery
|
|
303
|
+
)
|
|
304
|
+
end
|
|
305
|
+
|
|
306
|
+
run_cadence_probe(instance_id: instance_id, state: state)
|
|
307
|
+
end
|
|
308
|
+
|
|
309
|
+
def refresh_unpublished_offerings(instance_id:, state:)
|
|
310
|
+
offerings = discover_offerings_for_instance(
|
|
311
|
+
instance_cfg: state[:instance_cfg], instance_key: state[:instance_key]
|
|
156
312
|
)
|
|
313
|
+
commit_discovered_offerings(instance_id: instance_id, state: state, offerings: offerings)
|
|
157
314
|
end
|
|
158
315
|
|
|
159
|
-
|
|
160
|
-
|
|
316
|
+
# -- Removal -----------------------------------------------------------
|
|
317
|
+
|
|
318
|
+
def remove_instance_state(instance_id)
|
|
319
|
+
state = @instance_state_mutex.synchronize do
|
|
320
|
+
tracked = @instance_states[instance_id]
|
|
321
|
+
next unless tracked
|
|
322
|
+
|
|
323
|
+
publisher.remove_instance(
|
|
324
|
+
instance_id: instance_id,
|
|
325
|
+
physical_id: tracked[:physical_id],
|
|
326
|
+
publisher_token: tracked[:publisher_token]
|
|
327
|
+
)
|
|
328
|
+
@instance_states.delete(instance_id)
|
|
329
|
+
end
|
|
330
|
+
return unless state
|
|
331
|
+
|
|
332
|
+
clear_instance_health(config_name: state[:name])
|
|
333
|
+
rescue StandardError => e
|
|
334
|
+
handle_exception(e, level: :warn, operation: 'openai.actor.remove_instance',
|
|
335
|
+
instance_id: instance_id)
|
|
161
336
|
end
|
|
162
337
|
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
338
|
+
# -- Model discovery ---------------------------------------------------
|
|
339
|
+
|
|
340
|
+
def discover_offerings_for_instance(instance_cfg:, instance_key:)
|
|
341
|
+
models = fetch_models(instance_cfg: instance_cfg)
|
|
342
|
+
|
|
343
|
+
models.filter_map do |model_data|
|
|
344
|
+
model_id = model_data[:id].to_s
|
|
345
|
+
next if model_id.empty?
|
|
346
|
+
|
|
347
|
+
build_offering_draft(
|
|
348
|
+
model_id: model_id,
|
|
349
|
+
model_data: model_data,
|
|
350
|
+
instance_cfg: instance_cfg,
|
|
351
|
+
instance_key: instance_key
|
|
352
|
+
)
|
|
169
353
|
end
|
|
354
|
+
rescue Faraday::Error, Legion::JSON::ParseError => e
|
|
355
|
+
handle_exception(e, level: :warn, operation: 'openai.actor.discover_offerings')
|
|
356
|
+
[]
|
|
357
|
+
end
|
|
358
|
+
|
|
359
|
+
def fetch_models(instance_cfg:)
|
|
360
|
+
conn = build_api_connection(instance_cfg: instance_cfg)
|
|
361
|
+
response = conn.get('/v1/models')
|
|
362
|
+
Legion::JSON.load(response.body).fetch(:data, [])
|
|
363
|
+
end
|
|
364
|
+
|
|
365
|
+
def build_offering_draft(model_id:, model_data:, instance_cfg:, instance_key:)
|
|
366
|
+
tier = instance_cfg[:tier] || :frontier
|
|
367
|
+
operations = infer_operations(model_id: model_id)
|
|
368
|
+
weight_inputs = Legion::Extensions::Llm::Inventory::WeightSchema.weight_inputs(
|
|
369
|
+
settings: Legion::Settings,
|
|
370
|
+
instance_key: instance_key,
|
|
371
|
+
provider_native_key: model_id,
|
|
372
|
+
model: model_id,
|
|
373
|
+
tier: tier
|
|
374
|
+
)
|
|
375
|
+
|
|
376
|
+
Legion::Extensions::Llm::Inventory::OfferingDraft.new(
|
|
377
|
+
provider_native_key: model_id,
|
|
378
|
+
model: model_id,
|
|
379
|
+
tier: tier,
|
|
380
|
+
weight_inputs: weight_inputs,
|
|
381
|
+
base_weight: Legion::Extensions::Llm::Inventory::WeightSchema.base_weight(weight_inputs),
|
|
382
|
+
operation_evidence: build_operation_evidence(operations: operations),
|
|
383
|
+
capability_evidence: build_capability_evidence(model_id: model_id),
|
|
384
|
+
context_evidence: build_context_evidence(model_id: model_id, model_data: model_data),
|
|
385
|
+
max_output_evidence: build_max_output_evidence(model_data: model_data),
|
|
386
|
+
embedding_dimensions_evidence: build_embedding_dimensions_evidence(
|
|
387
|
+
model_id: model_id, model_data: model_data
|
|
388
|
+
),
|
|
389
|
+
model_revision_evidence: build_model_revision_evidence(model_data: model_data),
|
|
390
|
+
tokenizer_evidence: absent_value_evidence,
|
|
391
|
+
quota_domains: build_quota_domains(instance_cfg: instance_cfg, operations: operations),
|
|
392
|
+
metadata: build_offering_metadata(model_id: model_id, instance_key: instance_key).freeze,
|
|
393
|
+
publication_source: :provider_catalog
|
|
394
|
+
)
|
|
170
395
|
end
|
|
171
396
|
end
|
|
172
397
|
end
|
|
@@ -37,7 +37,18 @@ module Legion
|
|
|
37
37
|
'handle_fleet_request'
|
|
38
38
|
end
|
|
39
39
|
|
|
40
|
+
# Subscription dispatch resolves the String runner_class through
|
|
41
|
+
# Legion::Runner.run (use_runner? = true); the direct
|
|
42
|
+
# runner_class.send(fn, **message) path cannot send on a String.
|
|
40
43
|
def use_runner?
|
|
44
|
+
true
|
|
45
|
+
end
|
|
46
|
+
|
|
47
|
+
def check_subtask?
|
|
48
|
+
false
|
|
49
|
+
end
|
|
50
|
+
|
|
51
|
+
def generate_task?
|
|
41
52
|
false
|
|
42
53
|
end
|
|
43
54
|
|