lex-llm-azure-foundry 0.3.4 → 0.4.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +148 -0
- data/Gemfile +9 -2
- data/README.md +24 -39
- data/RULES.md +97 -0
- data/lex-llm-azure-foundry.gemspec +4 -1
- data/lib/legion/extensions/llm/azure_foundry/actors/discovery.rb +24 -0
- data/lib/legion/extensions/llm/azure_foundry/helpers/callable.rb +143 -0
- data/lib/legion/extensions/llm/azure_foundry/helpers/offering_evidence.rb +150 -0
- data/lib/legion/extensions/llm/azure_foundry/model_catalog_parser.rb +119 -0
- data/lib/legion/extensions/llm/azure_foundry/provider.rb +50 -358
- data/lib/legion/extensions/llm/azure_foundry/runners/discovery.rb +191 -0
- data/lib/legion/extensions/llm/azure_foundry/runners/fleet_worker.rb +3 -4
- data/lib/legion/extensions/llm/azure_foundry/version.rb +1 -1
- data/lib/legion/extensions/llm/azure_foundry.rb +22 -12
- metadata +9 -4
- data/lib/legion/extensions/llm/azure_foundry/actors/discovery_refresh.rb +0 -1026
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: c1198fb2ce8ff987bdcde592fd99985fea88baf60104fa1424e240201ce26989
|
|
4
|
+
data.tar.gz: ddc59cb92b40d93ffd687ae7fb0e833af86f2d8ea64f3080f709f2044e6f451b
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: cd60624cc57ee1c7031f1a7392f01822be1abace9a94b2c0aaa9628c14a5801738f7bb2d4f00e717d7e760c5ead49918b72f07bc2a2185df5a329d65c1ef29b3
|
|
7
|
+
data.tar.gz: 5c25b3e222403710aafe6d54dc6d91e83a6b1d8bd442b849a8f40da76af231bfcdf2e41ca2b7686245dcf1178494ea6aedc7bb44f96c2c892f216e4fcc0df787
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,153 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## [0.4.2] - 2026-08-25
|
|
4
|
+
|
|
5
|
+
### Fixed
|
|
6
|
+
- **Duplicate `provider_native_key` on discovery (collapsing distinct
|
|
7
|
+
deployments).** Model-id resolution used a multi-key fallback
|
|
8
|
+
(`id name model_name deployment_name model`) that could resolve two
|
|
9
|
+
DISTINCT deployments of one base model to the SAME id — the shared base
|
|
10
|
+
keys `model_name` / `model` ranked at or above the unique
|
|
11
|
+
`deployment_name`. Two drafts then carried the same `provider_native_key`
|
|
12
|
+
and the registry `Store#build_records` raised
|
|
13
|
+
`Legion::Extensions::Llm::Inventory::Errors::ValidationError:
|
|
14
|
+
duplicate provider_native_key`, so the instance failed to publish.
|
|
15
|
+
`MODEL_ID_KEYS` now resolves only the unique per-deployment identity
|
|
16
|
+
(`id deployment_name name`) and never the shared base model — one
|
|
17
|
+
unambiguous key per offering, matching the bedrock provider's single
|
|
18
|
+
`:model_id`. Envelope recognition (`looks_like_model?`) keeps its
|
|
19
|
+
original breadth via a separate `MODEL_SHAPE_KEYS` set, so a model-shaped
|
|
20
|
+
entry without a unique id is still recognized and then dropped by the
|
|
21
|
+
empty-id filter rather than raising an unrecognized-envelope error.
|
|
22
|
+
|
|
23
|
+
## [0.4.1] - 2026-08-20
|
|
24
|
+
|
|
25
|
+
### Changed
|
|
26
|
+
- **0.8.0 conformance (lex-llm SSOT v4 contract cut).** Legacy type usage
|
|
27
|
+
is gone: the dispatch entries accept
|
|
28
|
+
`Legion::Extensions::Llm::Canonical::Message` only, and the shared
|
|
29
|
+
lex-llm `enforce_canonical_messages!` helper runs at every
|
|
30
|
+
message-operation entry on both entry forms (the provider entries and
|
|
31
|
+
the `AzureFoundryCallable` — 08 F2, 12/O05). The per-provider
|
|
32
|
+
`enforce_message_boundary!` re-implementation (which still accepted the
|
|
33
|
+
deleted legacy `Llm::Message` shape) is removed.
|
|
34
|
+
- **Render from canonical, model untouched (08 R1/F3, B4).** `chat`/`stream`
|
|
35
|
+
build `Canonical::Params` from the canonical-spelling dispatch kwargs
|
|
36
|
+
(temperature lives only in `Canonical::Params`, 05 O4) and pass the
|
|
37
|
+
Selection-derived model String to the wire unchanged — the
|
|
38
|
+
`Model::Info` dispatch-time wrapping (and the callable's `to_model_info`
|
|
39
|
+
fabrication) is deleted.
|
|
40
|
+
- **Offering read path is the Registry snapshot (07 C5).** The legacy
|
|
41
|
+
`offering_from_model` → `Routing::ModelOffering` production (with its
|
|
42
|
+
`CapabilityPolicy` cascade) is deleted; the discovery actor's writer
|
|
43
|
+
(`OfferingDraft` + `Registry` publication) is the sole offering path and
|
|
44
|
+
`discover_offerings` serves the activated inventory offerings.
|
|
45
|
+
- **Legacy coordinator wiring removed.** The
|
|
46
|
+
`Inventory::ScopedRefresher::LegacyCoordinatorAdapter` compatibility
|
|
47
|
+
adapter is deleted from the discovery actor's `Inventory::Publisher`
|
|
48
|
+
construction (the file no longer exists in lex-llm 0.8.0).
|
|
49
|
+
- **count_tokens returns the 05 §2 Integer heuristic.** The legacy Hash
|
|
50
|
+
artifact (`supported: false`, `estimated_input_characters`) is deleted
|
|
51
|
+
and the base heuristic estimate is inherited — operation support is
|
|
52
|
+
carried by the SSOT data plane (writer operation evidence +
|
|
53
|
+
`WorkerExecution.require_supported!`), not by a per-call artifact.
|
|
54
|
+
- **§2 single publication engine (0.8.0 core contract).** The gem-level
|
|
55
|
+
`registry_publisher` singleton and the `Provider.registry_publisher`
|
|
56
|
+
class method are removed: nothing in the gem, the lex-llm core, or
|
|
57
|
+
legion-llm calls them (the discovery actor publishes through
|
|
58
|
+
`Inventory::Publisher`, the sole engine), and the 0.8.0
|
|
59
|
+
`RegistryPublisher` now carries the operator's `provider_instance`
|
|
60
|
+
identity — a gem-level construction site has no instance scope to
|
|
61
|
+
carry one.
|
|
62
|
+
- **L6 fleet responder kwargs.** `FleetWorker.handle_fleet_request` no
|
|
63
|
+
longer passes the `provider_class:`/`provider_instances:` params the
|
|
64
|
+
0.8.0 core deleted from `ProviderResponder.call` (v3 dispatch is
|
|
65
|
+
exact-only and never constructs a provider at the runner).
|
|
66
|
+
- **Dependency floor** — `lex-llm` raised to `>= 0.8.0` for the SSOT v4
|
|
67
|
+
contract cut (Canonical types, shared boundary helper, Registry read
|
|
68
|
+
path).
|
|
69
|
+
|
|
70
|
+
### Added
|
|
71
|
+
- **Dispatch-boundary regression coverage.** Loud-reject examples at each
|
|
72
|
+
public entry for plain-Hash input, and a canonical passthrough example
|
|
73
|
+
verifying canonical messages reach the rendered wire payload unchanged
|
|
74
|
+
and the sync response is a `Canonical::Response`.
|
|
75
|
+
- **Raw-string-model passthrough spec.** The fleet model id travels to the
|
|
76
|
+
provider boundary verbatim for every operation (no Model::Info
|
|
77
|
+
fabrication, 08 F3/B4).
|
|
78
|
+
|
|
79
|
+
## [0.4.0] - 2026-08-19
|
|
80
|
+
|
|
81
|
+
### Changed
|
|
82
|
+
- **Live model catalog discovery — standard lex-llm interface.**
|
|
83
|
+
`list_models` and `discover_offerings` now fetch the model catalog from
|
|
84
|
+
the instance's discovery endpoint (`GET models/info?api-version=...` on the
|
|
85
|
+
model-inference surface, `GET /models` on the OpenAI-compatible surface)
|
|
86
|
+
and derive offerings through the shared base-class flow — the same
|
|
87
|
+
endpoint-driven discovery every other provider uses. Offerings are
|
|
88
|
+
published with `publication_source: :provider_catalog`.
|
|
89
|
+
- **Shared `ModelCatalogParser`.** Both the Provider's `list_models` and the
|
|
90
|
+
SSOT v3 `DiscoveryRefresh` actor parse the wire catalog through one module
|
|
91
|
+
(envelope: `data` / `models` / `value` / `deployments` list keys, bare
|
|
92
|
+
arrays, or a single model object). An unrecognized envelope raises instead
|
|
93
|
+
of producing a silent empty catalog.
|
|
94
|
+
- **Discovery actor fetches the live catalog.** `discover_offerings_for_instance`
|
|
95
|
+
hits the same endpoint (and auth) as the readiness probe and builds one
|
|
96
|
+
OfferingDraft per catalog entry. A failed fetch yields nil so the refresh
|
|
97
|
+
loop keeps the last complete snapshot rather than deleting it.
|
|
98
|
+
|
|
99
|
+
### Removed
|
|
100
|
+
- **Configured deployments — the static discovery path is gone.**
|
|
101
|
+
`azure_foundry_deployments` (and the `deployments` / `provider.deployments`
|
|
102
|
+
settings aliases) no longer exist. `ProviderClassMethods#resolve_model_id`,
|
|
103
|
+
`#deployment_config`, `#normalize_deployments`, and the config-driven
|
|
104
|
+
offering path (`allowed_offerings` / `configured_deployments` /
|
|
105
|
+
`offering_from_config`) are deleted. The model set is whatever the endpoint
|
|
106
|
+
reports — nothing about models lives in settings.
|
|
107
|
+
- `provider_native_key` and `model` are both the catalog model id (the
|
|
108
|
+
routable id the endpoint accepts); the base model name rides along as
|
|
109
|
+
`canonical_model_alias` when the catalog reports one.
|
|
110
|
+
|
|
111
|
+
### Fixed
|
|
112
|
+
- **Bearer-token auth on the actor path.** `apply_auth_headers` now sends
|
|
113
|
+
`Authorization: Bearer` when the instance carries a bearer token
|
|
114
|
+
(previously api-key only — bearer-only instances failed the readiness
|
|
115
|
+
probe and stayed `:initializing` forever).
|
|
116
|
+
|
|
117
|
+
### Notes
|
|
118
|
+
- No captured fixture of the model-inference `models/info` list envelope
|
|
119
|
+
exists in the monorepo. The parser accepts the known shapes; live
|
|
120
|
+
confirmation of the wire response against a real project is the one
|
|
121
|
+
remaining UAT item for this change.
|
|
122
|
+
|
|
123
|
+
## [0.3.5] - 2026-08-19
|
|
124
|
+
|
|
125
|
+
### Changed
|
|
126
|
+
- **Write-time SSOT lane weights** — Build Azure deployment drafts with the shared
|
|
127
|
+
four-component weight pair using the deployment name as the provider-native
|
|
128
|
+
offering identity, and reconcile weight-only changes atomically on the existing
|
|
129
|
+
discovery cadence. Initial/recovery activation now rebuilds from current settings,
|
|
130
|
+
removal wins readiness races, and dormant configured weights log once per absence
|
|
131
|
+
period without any Settings lifecycle callback.
|
|
132
|
+
- **Callable-path system conformance** — Capture the rendered Azure/OpenAI-compatible
|
|
133
|
+
HTTP payload and verify a folded leading system message remains in the dialect-native
|
|
134
|
+
`role: system` message position.
|
|
135
|
+
- **Dependency floor** — Raise `lex-llm` to `>= 0.7.6` for `WeightSchema`,
|
|
136
|
+
`WeightReconciler`, and the immutable record weight pair. The `legion-settings`
|
|
137
|
+
dependency and provider/operator workflow are unchanged.
|
|
138
|
+
|
|
139
|
+
### Fixed
|
|
140
|
+
- **Malformed-weight startup cleanup** — Validate and build weighted offering
|
|
141
|
+
drafts before constructing or claiming any callable scope, so invalid weight
|
|
142
|
+
values cannot leave an orphaned initializing Registry publication. A later
|
|
143
|
+
corrected discovery pass claims and activates the instance normally without
|
|
144
|
+
requiring a restart or operator cleanup.
|
|
145
|
+
- **Complete offering replacement comparison** — Ordinary discovery now compares
|
|
146
|
+
every authoritative `OfferingDraft` field while ignoring only evidence
|
|
147
|
+
observation timestamps. Deployment order does not churn, duplicate counts stay
|
|
148
|
+
significant, and native-key, evidence, quota, metadata, publication-source,
|
|
149
|
+
tier, or weight drift advances the Registry snapshot exactly once.
|
|
150
|
+
|
|
3
151
|
## [0.3.4] - 2026-08-18
|
|
4
152
|
|
|
5
153
|
### Fixed
|
data/Gemfile
CHANGED
|
@@ -2,8 +2,6 @@
|
|
|
2
2
|
|
|
3
3
|
source 'https://rubygems.org'
|
|
4
4
|
|
|
5
|
-
gem 'lex-llm'
|
|
6
|
-
|
|
7
5
|
gemspec
|
|
8
6
|
|
|
9
7
|
group :development do
|
|
@@ -15,3 +13,12 @@ group :development do
|
|
|
15
13
|
gem 'rubocop-rake', '>= 0.6'
|
|
16
14
|
gem 'rubocop-rspec'
|
|
17
15
|
end
|
|
16
|
+
|
|
17
|
+
group :test do
|
|
18
|
+
# The published lex-llm (>= 0.8.0, declared in the gemspec) provides the
|
|
19
|
+
# WeightSchema/WeightReconciler and Canonical types these specs require.
|
|
20
|
+
# Use the local checkout when present (development); CI resolves the
|
|
21
|
+
# published gem via the gemspec dependency.
|
|
22
|
+
lex_llm_path = File.expand_path('../lex-llm', __dir__)
|
|
23
|
+
gem 'lex-llm', path: lex_llm_path if Dir.exist?(lex_llm_path)
|
|
24
|
+
end
|
data/README.md
CHANGED
|
@@ -15,11 +15,15 @@ require 'legion/extensions/llm/azure_foundry'
|
|
|
15
15
|
- Provider family `:azure_foundry`
|
|
16
16
|
- Azure AI Foundry model inference chat completions through `POST /models/chat/completions?api-version=...`
|
|
17
17
|
- Azure AI Foundry model inference embeddings through `POST /models/embeddings?api-version=...`
|
|
18
|
-
-
|
|
18
|
+
- Live model catalog discovery through the surface's discovery endpoint
|
|
19
|
+
(`GET /models/info?api-version=...` on the model-inference surface, `GET /models` on the
|
|
20
|
+
OpenAI-compatible surface) — the same endpoint doubles as the readiness probe
|
|
19
21
|
- Azure OpenAI v1-compatible endpoint support through `/openai/v1/chat/completions` and `/openai/v1/embeddings`
|
|
20
|
-
-
|
|
21
|
-
|
|
22
|
-
|
|
22
|
+
- Standard lex-llm offering discovery: `list_models` fetches the live catalog and
|
|
23
|
+
`discover_offerings` derives offerings through the shared base-class flow,
|
|
24
|
+
exactly like every other provider
|
|
25
|
+
- Shared `ModelCatalogParser` — the single module both the Provider's `list_models`
|
|
26
|
+
and the SSOT v3 `DiscoveryRefresh` actor parse the wire catalog through
|
|
23
27
|
- Shared OpenAI-compatible request and response mapping through `Legion::Extensions::Llm::Provider::OpenAICompatible`
|
|
24
28
|
- Shared registry availability publishing through `Legion::Extensions::Llm::RegistryPublisher` when transport is available
|
|
25
29
|
- Provider-owned fleet request handling through `Legion::Extensions::Llm::Fleet::ProviderResponder`
|
|
@@ -29,7 +33,10 @@ require 'legion/extensions/llm/azure_foundry'
|
|
|
29
33
|
```text
|
|
30
34
|
Legion::Extensions::Llm::AzureFoundry
|
|
31
35
|
|-- Provider # Azure AI Foundry and Azure OpenAI hosted provider surface
|
|
32
|
-
|
|
|
36
|
+
| |-- Capabilities # Capability predicates inferred from catalog metadata and model naming
|
|
37
|
+
| `-- catalog discovery # live list_models through ModelCatalogParser
|
|
38
|
+
|-- ModelCatalogParser # shared live-catalog envelope/entry parsing
|
|
39
|
+
|-- Actor::DiscoveryRefresh # SSOT v3 periodic discovery (catalog fetch, OfferingDrafts, probes)
|
|
33
40
|
|-- Actor::FleetWorker # Subscription actor for provider-owned fleet requests
|
|
34
41
|
|-- Runners::FleetWorker # Runner entrypoint that delegates to lex-llm ProviderResponder
|
|
35
42
|
`-- VERSION
|
|
@@ -42,7 +49,9 @@ Legion::Extensions::Llm::AzureFoundry
|
|
|
42
49
|
| Path | Purpose |
|
|
43
50
|
|------|---------|
|
|
44
51
|
| `lib/legion/extensions/llm/azure_foundry.rb` | Entry point, provider defaults, instance discovery, shared registry publisher |
|
|
45
|
-
| `lib/legion/extensions/llm/azure_foundry/provider.rb` | Provider implementation with chat, stream, embed, health, readiness, model listing
|
|
52
|
+
| `lib/legion/extensions/llm/azure_foundry/provider.rb` | Provider implementation with chat, stream, embed, health, readiness, and live model listing |
|
|
53
|
+
| `lib/legion/extensions/llm/azure_foundry/model_catalog_parser.rb` | Shared live-catalog envelope and entry parsing |
|
|
54
|
+
| `lib/legion/extensions/llm/azure_foundry/actors/discovery_refresh.rb` | SSOT v3 periodic discovery actor (catalog fetch, OfferingDrafts, readiness probes) |
|
|
46
55
|
| `lib/legion/extensions/llm/azure_foundry/actors/fleet_worker.rb` | Subscription actor gated by ProviderResponder fleet settings |
|
|
47
56
|
| `lib/legion/extensions/llm/azure_foundry/runners/fleet_worker.rb` | Fleet request runner that delegates execution to `ProviderResponder.call` |
|
|
48
57
|
| `lib/legion/extensions/llm/azure_foundry/version.rb` | `VERSION` constant |
|
|
@@ -51,6 +60,8 @@ Legion::Extensions::Llm::AzureFoundry
|
|
|
51
60
|
|
|
52
61
|
Configured instances can be supplied through Legion settings under `extensions.llm.azure_foundry`. A top-level endpoint creates a `:settings` instance; entries under `instances` create named instances.
|
|
53
62
|
|
|
63
|
+
Models are **discovered from the endpoint**, not configured: nothing about the model set lives in settings. The discovery actor (and `list_models`) fetches the model catalog from the instance's discovery endpoint and publishes whatever it reports.
|
|
64
|
+
|
|
54
65
|
```yaml
|
|
55
66
|
extensions:
|
|
56
67
|
llm:
|
|
@@ -60,26 +71,12 @@ extensions:
|
|
|
60
71
|
bearer_token: env://AZURE_FOUNDRY_BEARER_TOKEN
|
|
61
72
|
api_version: 2024-05-01-preview
|
|
62
73
|
surface: model_inference
|
|
63
|
-
deployments:
|
|
64
|
-
- deployment: gpt-4o-prod
|
|
65
|
-
model_family: openai
|
|
66
|
-
canonical_model_alias: gpt-4o
|
|
67
|
-
usage_type: inference
|
|
68
|
-
- deployment: embedding-prod
|
|
69
|
-
model_family: openai
|
|
70
|
-
canonical_model_alias: text-embedding-3-small
|
|
71
|
-
usage_type: embedding
|
|
72
74
|
instances:
|
|
73
75
|
prod:
|
|
74
76
|
endpoint: https://prod.services.ai.azure.com
|
|
75
77
|
api_key: env://AZURE_INFERENCE_CREDENTIAL
|
|
76
78
|
api_version: 2024-05-01-preview
|
|
77
79
|
surface: model_inference
|
|
78
|
-
deployments:
|
|
79
|
-
- deployment: gpt-4o-prod
|
|
80
|
-
model_family: openai
|
|
81
|
-
canonical_model_alias: gpt-4o
|
|
82
|
-
usage_type: inference
|
|
83
80
|
fleet:
|
|
84
81
|
enabled: true
|
|
85
82
|
respond_to_requests: true
|
|
@@ -98,14 +95,6 @@ Legion::Extensions::Llm.configure do |config|
|
|
|
98
95
|
config.azure_foundry_bearer_token = ENV['AZURE_FOUNDRY_BEARER_TOKEN']
|
|
99
96
|
config.azure_foundry_api_version = '2024-05-01-preview'
|
|
100
97
|
config.azure_foundry_surface = :model_inference
|
|
101
|
-
config.azure_foundry_deployments = [
|
|
102
|
-
{
|
|
103
|
-
deployment: 'gpt-4o-prod',
|
|
104
|
-
model_family: :openai,
|
|
105
|
-
canonical_model_alias: 'gpt-4o',
|
|
106
|
-
usage_type: :inference
|
|
107
|
-
}
|
|
108
|
-
]
|
|
109
98
|
end
|
|
110
99
|
```
|
|
111
100
|
|
|
@@ -129,8 +118,7 @@ Legion::Extensions::Llm::AzureFoundry.default_settings
|
|
|
129
118
|
# },
|
|
130
119
|
# provider: {
|
|
131
120
|
# api_version: "2024-05-01-preview",
|
|
132
|
-
# surface: nil
|
|
133
|
-
# deployments: []
|
|
121
|
+
# surface: nil
|
|
134
122
|
# },
|
|
135
123
|
# usage: { inference: true, embedding: true, image: false },
|
|
136
124
|
# limits: { concurrency: 4 },
|
|
@@ -152,20 +140,19 @@ Legion::Extensions::Llm::AzureFoundry.default_settings
|
|
|
152
140
|
```ruby
|
|
153
141
|
provider = Legion::Extensions::Llm::AzureFoundry.provider_class.new(Legion::Extensions::Llm.config)
|
|
154
142
|
|
|
155
|
-
provider.
|
|
156
|
-
provider.
|
|
143
|
+
provider.list_models
|
|
144
|
+
provider.discover_offerings(live: true)
|
|
157
145
|
provider.health(live: false)
|
|
158
146
|
provider.readiness(live: false)
|
|
159
|
-
provider.list_models
|
|
160
147
|
provider.chat(messages: messages, model: 'gpt-4o-prod')
|
|
161
148
|
provider.stream(messages: messages, model: 'gpt-4o-prod') { |chunk| puts chunk.content }
|
|
162
149
|
provider.embed(text: ['hello'], model: 'embedding-prod')
|
|
163
150
|
provider.count_tokens(messages: messages, model: 'gpt-4o-prod')
|
|
164
151
|
```
|
|
165
152
|
|
|
166
|
-
`discover_offerings(live:
|
|
153
|
+
`list_models` and `discover_offerings(live: true)` fetch the model catalog from the instance's discovery endpoint. `discover_offerings(live: false)` returns the last cached discovery result without calling Azure — the periodic `DiscoveryRefresh` actor keeps that cache warm, exactly like the other providers.
|
|
167
154
|
|
|
168
|
-
`health(live: true)` calls the
|
|
155
|
+
`health(live: true)` calls the surface's discovery endpoint (the non-billable readiness probe). Keep `live: false` for startup paths and tests that must not require Azure.
|
|
169
156
|
|
|
170
157
|
`count_tokens` returns a structured unsupported result by default because the Microsoft REST contract used here does not define a portable token-counting endpoint across Azure AI Foundry deployments.
|
|
171
158
|
|
|
@@ -177,7 +164,7 @@ Fleet execution is delegated to `Legion::Extensions::Llm::Fleet::ProviderRespond
|
|
|
177
164
|
|
|
178
165
|
## Routing Metadata
|
|
179
166
|
|
|
180
|
-
|
|
167
|
+
The catalog entry's model identity (`id` / `name` / `model_name` / `deployment_name` / `model`) becomes the offering `model` and `provider_native_key` — the same routable id the endpoint accepts. When the catalog reports a base model name distinct from the routable id, it rides along as `canonical_model_alias`; `model_family` is inferred from the model name as a routing hint (never as capability evidence).
|
|
181
168
|
|
|
182
169
|
Supported `model_family` values are intentionally open-ended symbols, including:
|
|
183
170
|
|
|
@@ -188,8 +175,6 @@ Supported `model_family` values are intentionally open-ended symbols, including:
|
|
|
188
175
|
- `:anthropic`
|
|
189
176
|
- `:microsoft`
|
|
190
177
|
|
|
191
|
-
When `model_family` or `canonical_model_alias` is missing, offerings include `requires_explicit_model_metadata: true`.
|
|
192
|
-
|
|
193
178
|
## Failure Behavior
|
|
194
179
|
|
|
195
|
-
Live discovery and health-check failures are reported with `handle_exception(e, level: :warn, handled: true, operation: ...)` before returning degraded metadata.
|
|
180
|
+
Live discovery and health-check failures are reported with `handle_exception(e, level: :warn, handled: true, operation: ...)` before returning degraded metadata. A failed catalog fetch in the discovery actor keeps the last complete snapshot (it never replaces it with an empty set); provider configuration and fleet actor enablement do not require live Azure connectivity.
|
data/RULES.md
ADDED
|
@@ -0,0 +1,97 @@
|
|
|
1
|
+
RULES.md — Legion LLM Architecture Law
|
|
2
|
+
These rules apply to every task, file, repository, agent, model, session, test, refactor, migration, incident, and release.
|
|
3
|
+
The requested task defines what may change. These rules define how the system ALWAYS works.
|
|
4
|
+
Every rule remains active 100% of the time. If requested work conflicts with these rules, stop and surface the conflict before changing code.
|
|
5
|
+
These are architecture laws. Scope, compatibility, urgency, convenience, tests, existing behavior, and model judgment do not change them.
|
|
6
|
+
1. Canonical is the only internal language.
|
|
7
|
+
Every client translates client wire -> Canonical before shared execution.
|
|
8
|
+
Shared execution carries Canonical through context, tools, routing, direct dispatch, fleet dispatch, and response handling.
|
|
9
|
+
Every provider translates Canonical -> provider wire at the provider boundary, then provider wire -> Canonical before returning to shared execution.
|
|
10
|
+
Every internal boundary validates the Canonical type it is defined to receive and raises immediately when that contract is violated.
|
|
11
|
+
Client Wire -> Client Translator -> Canonical -> Shared Execution -> Canonical -> Provider Translator -> Provider Wire.
|
|
12
|
+
2. Serialization preserves Canonical.
|
|
13
|
+
Transport may serialize Canonical state. The receiving transport boundary ALWAYS rehydrates the exact Canonical type before execution continues.
|
|
14
|
+
Fleet follows Canonical -> serialize -> wire -> deserialize -> rehydrate Canonical -> Canonical.
|
|
15
|
+
Serialization changes encoding only. Ownership, identity, model, operation, capability, selection, and meaning remain exactly the same.
|
|
16
|
+
After rehydration, shared execution continues only with Canonical objects.
|
|
17
|
+
3. Every authoritative fact has exactly one owner.
|
|
18
|
+
The owner creates the fact once. Every downstream layer carries, projects, serializes, rehydrates, verifies, or executes that exact fact.
|
|
19
|
+
A downstream layer receiving missing or contradictory authoritative state raises and returns the defect to the owning layer.
|
|
20
|
+
Authority ALWAYS moves forward by preservation.
|
|
21
|
+
Authority is created once and is never recreated downstream.
|
|
22
|
+
4. Requirements describe the request. Inventory describes reality. Router chooses. Dispatch executes.
|
|
23
|
+
Canonical request construction owns request semantics. RequestRequirements expresses operation, capabilities, modality, context, output, tools, and explicit pins.
|
|
24
|
+
Providers publish exact executable facts into Inventory. Inventory owns canonical instance, offering, lane, capability, context, quota, health, and published weight state.
|
|
25
|
+
Router.next_lane consumes Requirements plus one immutable Inventory snapshot and produces one authoritative Selection.
|
|
26
|
+
Dispatch executes that Selection exactly. Once Selection exists, routing is finished.
|
|
27
|
+
5. Inventory facts are immutable executable facts.
|
|
28
|
+
Providers publish exact instances and complete offering snapshots through the Inventory publication contract.
|
|
29
|
+
Identity, capability evidence, context evidence, quota domains, availability, and write-time weights are consumed from published Inventory state.
|
|
30
|
+
A changed fact becomes authoritative only through the owning publication or reconciliation path and a new Inventory snapshot.
|
|
31
|
+
Routing reads Inventory. Dispatch verifies and executes Inventory-backed Selection.
|
|
32
|
+
6. Identity, capabilities, weights, and context policy retain exact ownership.
|
|
33
|
+
Inventory::Identity owns instance, offering, and lane identity; canonical instance identity is provider family plus the operator/configured instance name; physical endpoint data remains secondary.
|
|
34
|
+
Providers publish capability evidence. Requirements state required capabilities. Candidate evaluation compares the two and determines capability eligibility.
|
|
35
|
+
The weight owner computes lane weight at publication time; Inventory stores it; ranking consumes that stored weight; a stored zero disables the lane.
|
|
36
|
+
Preferred-context binning orders eligible candidates into preference bands and preserves eligibility. Capability, health, binning, and weight ALWAYS retain distinct meanings.
|
|
37
|
+
7. Routing chooses exactly once.
|
|
38
|
+
Router.next_lane is the sole routing authority.
|
|
39
|
+
Candidate evaluation determines eligibility from Requirements and Inventory. Ranking orders eligible candidates from published routing facts.
|
|
40
|
+
Selection freezes the exact provider, instance, offering, lane, model, operation, and routing identity required for execution.
|
|
41
|
+
Every downstream component consumes the Selection it receives.
|
|
42
|
+
Selection is preserved, not reconstructed.
|
|
43
|
+
8. Exact execution stays exact through every boundary.
|
|
44
|
+
Direct dispatch executes the exact Selection-derived binding it receives.
|
|
45
|
+
Fleet dispatch serializes and signs that exact binding; fleet validation verifies it; fleet rehydration restores it; worker resolution verifies it against authoritative Inventory.
|
|
46
|
+
The selected provider, instance, offering, lane, model, and operation remain identical through projection, signing, transport, validation, rehydration, resolution, and callable invocation.
|
|
47
|
+
A mismatch raises before provider execution.
|
|
48
|
+
An exact execution request ALWAYS remains exact execution.
|
|
49
|
+
9. Health and errors preserve one authoritative meaning.
|
|
50
|
+
Inventory owns exact-instance availability. An authoritative instance-unavailable result removes that exact instance; readiness probing owns recovery; successful readiness republish re-admits it.
|
|
51
|
+
Overload, timeout, rate limit, model-not-ready, and transient provider failures remain request-local according to ProviderOutcome semantics.
|
|
52
|
+
The first layer that can authoritatively classify an error performs that classification once. Every downstream layer preserves it.
|
|
53
|
+
Programming errors remain programming errors. Contract violations remain contract violations. Routing exhaustion remains the defined typed Rejection.
|
|
54
|
+
10. Compatibility exists only at explicit edges.
|
|
55
|
+
Supported legacy clients and protocols are translated into the current Canonical and SSOT architecture at explicit compatibility boundaries.
|
|
56
|
+
Shared execution remains Canonical. Routing remains SSOT-driven. Exact execution remains exact.
|
|
57
|
+
Compatibility code adapts an external contract to the current internal architecture.
|
|
58
|
+
The current internal architecture ALWAYS has one representation, one routing authority, one identity system, and one execution truth.
|
|
59
|
+
11. Fix every defect at its owner.
|
|
60
|
+
Trace the incorrect value to the layer that owns it, then fix that owner.
|
|
61
|
+
Fix client wire in the client translator; Canonical shape in Canonical construction; Requirements in Requirements construction; provider facts in publication; identity in Inventory identity; weights in publication/reconciliation.
|
|
62
|
+
Fix eligibility in candidate evaluation; ordering in ranking; choice in Router.next_lane; execution preservation in dispatch; provider wire in the provider translator.
|
|
63
|
+
The layer where a defect becomes visible is evidence. The owning layer is where the correction belongs.
|
|
64
|
+
12. A discovered issue remains in its owning domain.
|
|
65
|
+
Complete the requested task inside its stated scope.
|
|
66
|
+
When investigation exposes a separate defect owned by another architectural domain, record and surface it as separate work unless the requested task is explicitly expanded.
|
|
67
|
+
Routing work consumes existing Canonical Requirements and Inventory facts. Canonical work changes Canonical contracts. Provider work changes publication or translation. Transport work changes transport.
|
|
68
|
+
Nearby code never changes ownership. “While we are here” never changes architecture.
|
|
69
|
+
13. N x N ALWAYS converges through Canonical.
|
|
70
|
+
Equivalent client semantics produce equivalent Canonical state before shared execution. Every provider consumes the same Canonical semantics for the same request.
|
|
71
|
+
When two paths disagree, capture the state at every involved boundary and locate the FIRST point where Canonical meaning diverges.
|
|
72
|
+
Fix that first divergent boundary, then run the exact failing path again.
|
|
73
|
+
Client behavior is proven at client-wire <-> Canonical. Provider behavior is proven at Canonical <-> provider-wire. Shared execution is proven with Canonical throughout.
|
|
74
|
+
14. Debug from captured authoritative state.
|
|
75
|
+
Capture the actual input at the failing boundary before reasoning from symptoms.
|
|
76
|
+
For translation or transport defects, capture Canonical immediately before and after every involved boundary.
|
|
77
|
+
For routing or dispatch defects, capture Requirements, relevant Inventory facts, Selection, execution binding, and ProviderOutcome.
|
|
78
|
+
Compare each captured value to the contract owned by that layer. Find the first divergence. Fix its owner. Re-run the exact path.
|
|
79
|
+
Then inspect sibling implementations for the same defect class.
|
|
80
|
+
15. Tests prove the real boundary and the invariant.
|
|
81
|
+
A boundary test exercises the real boundary it claims to protect.
|
|
82
|
+
Fleet tests exercise real serialization, deserialization, Canonical rehydration, signing, validation, exact resolution, and callable dispatch.
|
|
83
|
+
Provider tests exercise the real callable boundary and provider translator. Routing tests exercise real Requirements, Inventory records, candidate evaluation, ranking, and Selection.
|
|
84
|
+
Regression tests prove the violated invariant, not only the observed symptom.
|
|
85
|
+
A green suite is release evidence only when the tested path traverses the real architecture.
|
|
86
|
+
16. Shared contracts are consumed directly.
|
|
87
|
+
Shared Canonical types own execution representation. Shared Inventory types own inventory state. Shared Routing types own routing state.
|
|
88
|
+
Shared taxonomy owns canonical mappings. Shared ProviderOutcome owns provider-neutral outcomes. Shared fleet protocol owns exact execution claims.
|
|
89
|
+
Every repository consumes these shared owners directly.
|
|
90
|
+
A defect in one shared boundary triggers an audit of every sibling implementation of that boundary. Fix the shared owner centrally whenever the defect belongs to a shared contract.
|
|
91
|
+
17. Architecture is the release gate.
|
|
92
|
+
Every change preserves every rule in this file.
|
|
93
|
+
Tests, compatibility, historical behavior, migration phase, patch urgency, nearby code, task wording, and model judgment are evaluated UNDER these rules.
|
|
94
|
+
A contradiction between existing behavior and these rules is surfaced as an architecture conflict and resolved at the owning boundary before release.
|
|
95
|
+
Limited scope means do less. Limited scope NEVER means fewer rules apply.
|
|
96
|
+
These rules apply 100% of the time.
|
|
97
|
+
These are the law.
|
|
@@ -27,5 +27,8 @@ Gem::Specification.new do |spec|
|
|
|
27
27
|
spec.add_dependency 'legion-logging', '>= 1.3.2'
|
|
28
28
|
spec.add_dependency 'legion-settings', '>= 1.4.2'
|
|
29
29
|
spec.add_dependency 'legion-transport', '>= 1.4.14'
|
|
30
|
-
|
|
30
|
+
# 0.8.0 is the SSOT v4 contract cut: Canonical is the only internal
|
|
31
|
+
# language, the shared enforce_canonical_messages! boundary helper, and
|
|
32
|
+
# the Registry-snapshot offering read path this gem conforms to.
|
|
33
|
+
spec.add_dependency 'lex-llm', '>= 0.8.0'
|
|
31
34
|
end
|
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'legion/extensions/llm/discovery/actor'
|
|
4
|
+
|
|
5
|
+
# The base discovery actor only exists inside the daemon (it inherits the
|
|
6
|
+
# LegionIO time-based Every actor). In a standalone load, define nothing.
|
|
7
|
+
return unless defined?(Legion::Extensions::Llm::Discovery::Actor)
|
|
8
|
+
|
|
9
|
+
module Legion
|
|
10
|
+
module Extensions
|
|
11
|
+
module Llm
|
|
12
|
+
module AzureFoundry
|
|
13
|
+
module Actor
|
|
14
|
+
# Azure Foundry discovery actor: an EMPTY subclass of the shared
|
|
15
|
+
# base. The timer, dispatch, and runner-resolution convention are
|
|
16
|
+
# inherited — this class redefines nothing. The Azure Foundry-
|
|
17
|
+
# specific work lives in AzureFoundry::Runners::Discovery, resolved
|
|
18
|
+
# by the base from this namespace.
|
|
19
|
+
class Discovery < Legion::Extensions::Llm::Discovery::Actor; end
|
|
20
|
+
end
|
|
21
|
+
end
|
|
22
|
+
end
|
|
23
|
+
end
|
|
24
|
+
end
|
|
@@ -0,0 +1,143 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'legion/extensions/llm/routing/provider_outcome'
|
|
4
|
+
require 'legion/extensions/llm/azure_foundry/provider'
|
|
5
|
+
|
|
6
|
+
module Legion
|
|
7
|
+
module Extensions
|
|
8
|
+
module Llm
|
|
9
|
+
module AzureFoundry
|
|
10
|
+
module Helpers
|
|
11
|
+
# Callable wrapper for an Azure Foundry provider instance. Implements
|
|
12
|
+
# the fleet dispatch operations by delegating to the per-instance
|
|
13
|
+
# AzureFoundry::Provider (errors propagate so
|
|
14
|
+
# normalize_dispatch_error can classify them), plus the `disconnect`
|
|
15
|
+
# and `normalize_dispatch_error(error:)` contracts required by
|
|
16
|
+
# Inventory::CallableHandle and Routing::ProviderOutcome.
|
|
17
|
+
#
|
|
18
|
+
# The callable is the second canonical entry form (08 F2, 12/O05):
|
|
19
|
+
# the shared lex-llm enforce_canonical_messages! helper runs at each
|
|
20
|
+
# message-operation entry. The fleet passes `model:` as the
|
|
21
|
+
# offering's model id (String) — it travels to the wire untouched
|
|
22
|
+
# (08 F3, B4: no model re-derivation, no Model::Info fabrication).
|
|
23
|
+
class Callable
|
|
24
|
+
def initialize(instance_cfg:, logger:, provider: nil)
|
|
25
|
+
@instance_cfg = instance_cfg
|
|
26
|
+
@logger = logger
|
|
27
|
+
@provider = provider
|
|
28
|
+
@disconnected = false
|
|
29
|
+
end
|
|
30
|
+
|
|
31
|
+
def provider
|
|
32
|
+
@provider ||= Legion::Extensions::Llm::AzureFoundry::Provider.new(@instance_cfg)
|
|
33
|
+
end
|
|
34
|
+
|
|
35
|
+
def disconnected?
|
|
36
|
+
@disconnected
|
|
37
|
+
end
|
|
38
|
+
|
|
39
|
+
def disconnect
|
|
40
|
+
@disconnected = true
|
|
41
|
+
@provider&.disconnect
|
|
42
|
+
@logger.debug { '[azure_foundry][callable] disconnected' }
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
def chat(messages, model:, **rest)
|
|
46
|
+
provider.enforce_canonical_messages!(messages)
|
|
47
|
+
provider.chat(messages, model: model, **rest)
|
|
48
|
+
end
|
|
49
|
+
|
|
50
|
+
def stream_chat(messages, model:, **rest, &)
|
|
51
|
+
provider.enforce_canonical_messages!(messages)
|
|
52
|
+
provider.stream(messages, model: model, **rest, &)
|
|
53
|
+
end
|
|
54
|
+
|
|
55
|
+
def embed(text:, model:, **rest)
|
|
56
|
+
provider.embed(text: text, model: model, **rest)
|
|
57
|
+
end
|
|
58
|
+
|
|
59
|
+
def count_tokens(messages:, model:, **rest)
|
|
60
|
+
provider.enforce_canonical_messages!(messages)
|
|
61
|
+
# The base heuristic ignores params (05 §2) — the fleet rest
|
|
62
|
+
# folds into the params: slot rather than splatting into the
|
|
63
|
+
# fixed base signature.
|
|
64
|
+
provider.count_tokens(messages: messages, model: model, params: rest)
|
|
65
|
+
end
|
|
66
|
+
|
|
67
|
+
# D17: production dispatch raises Legion::Extensions::Llm::*Error
|
|
68
|
+
# (ErrorMiddleware), not raw Faraday — the base
|
|
69
|
+
# Provider#normalize_dispatch_error classifies those. On top, Azure
|
|
70
|
+
# wire semantics supply two stronger signals: an explicit
|
|
71
|
+
# EndpointDeactivated body (the ONLY instance_unavailable signal)
|
|
72
|
+
# and a model-not-ready 503 body (request-local, not instance-down).
|
|
73
|
+
def normalize_dispatch_error(error:)
|
|
74
|
+
outcome = base_provider_outcome(error: error)
|
|
75
|
+
return outcome_with_kind(outcome, :instance_unavailable) if explicit_endpoint_deactivated?(error: error)
|
|
76
|
+
return outcome_with_kind(outcome, :model_not_ready) if model_not_ready_outcome?(outcome, error: error)
|
|
77
|
+
return outcome_with_kind(outcome, :model_missing) if model_missing_outcome?(outcome, error: error)
|
|
78
|
+
|
|
79
|
+
outcome
|
|
80
|
+
end
|
|
81
|
+
|
|
82
|
+
private
|
|
83
|
+
|
|
84
|
+
# The base Provider#normalize_dispatch_error is a pure classifier
|
|
85
|
+
# (it reads no instance state), so it is bound to an allocated
|
|
86
|
+
# base instance — the callable classifies dispatch errors without
|
|
87
|
+
# constructing a provider connection.
|
|
88
|
+
def base_provider_outcome(error:)
|
|
89
|
+
@base_classifier ||= Legion::Extensions::Llm::Provider.allocate
|
|
90
|
+
@base_classifier.normalize_dispatch_error(error: error)
|
|
91
|
+
end
|
|
92
|
+
|
|
93
|
+
def outcome_with_kind(outcome, kind)
|
|
94
|
+
Legion::Extensions::Llm::Routing::ProviderOutcome.new(kind: kind, reason: outcome.reason)
|
|
95
|
+
end
|
|
96
|
+
|
|
97
|
+
def request_local_kind?(kind)
|
|
98
|
+
%i[overloaded provider_error].include?(kind)
|
|
99
|
+
end
|
|
100
|
+
|
|
101
|
+
def model_not_ready_outcome?(outcome, error:)
|
|
102
|
+
model_not_ready?(error: error) && request_local_kind?(outcome.kind)
|
|
103
|
+
end
|
|
104
|
+
|
|
105
|
+
def model_missing_outcome?(outcome, error:)
|
|
106
|
+
outcome.kind == :provider_error && error_status(error: error) == 404
|
|
107
|
+
end
|
|
108
|
+
|
|
109
|
+
# An explicit Azure endpoint-deactivation code is the ONLY signal
|
|
110
|
+
# that justifies instance_unavailable. The body is read from the
|
|
111
|
+
# Llm error's wrapped response (Faraday::Response in production,
|
|
112
|
+
# Faraday::Env on raw error paths, or a Hash).
|
|
113
|
+
def explicit_endpoint_deactivated?(error:)
|
|
114
|
+
body = extract_error_body(error: error)
|
|
115
|
+
body.include?('EndpointDeactivated') || body.include?('endpoint_deactivated')
|
|
116
|
+
end
|
|
117
|
+
|
|
118
|
+
def model_not_ready?(error:)
|
|
119
|
+
body = extract_error_body(error: error).downcase
|
|
120
|
+
body.include?('model not ready') || body.include?('deployment is warming up')
|
|
121
|
+
end
|
|
122
|
+
|
|
123
|
+
def extract_error_body(error:)
|
|
124
|
+
response = error.respond_to?(:response) ? error.response : nil
|
|
125
|
+
return response[:body].to_s if response.is_a?(Hash)
|
|
126
|
+
return '' if response.nil?
|
|
127
|
+
|
|
128
|
+
response.body.to_s
|
|
129
|
+
end
|
|
130
|
+
|
|
131
|
+
def error_status(error:)
|
|
132
|
+
response = error.respond_to?(:response) ? error.response : nil
|
|
133
|
+
return response[:status] if response.is_a?(Hash)
|
|
134
|
+
return nil if response.nil?
|
|
135
|
+
|
|
136
|
+
response.status
|
|
137
|
+
end
|
|
138
|
+
end
|
|
139
|
+
end
|
|
140
|
+
end
|
|
141
|
+
end
|
|
142
|
+
end
|
|
143
|
+
end
|