legion-llm 0.16.8 → 0.16.20
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.github/workflows/ci.yml +1 -1
- data/.gitignore +1 -0
- data/.rubocop.yml +4 -58
- data/CHANGELOG.md +118 -0
- data/CLAUDE.md +11 -11
- data/Gemfile +31 -7
- data/README.md +135 -4
- data/RULES.md +97 -0
- data/legion-llm.gemspec +3 -2
- data/lib/legion/llm/api/anthropic/messages.rb +7 -0
- data/lib/legion/llm/api/client_translators/anthropic_messages.rb +130 -125
- data/lib/legion/llm/api/client_translators/mcp_items.rb +30 -0
- data/lib/legion/llm/api/client_translators/openai_chat.rb +92 -93
- data/lib/legion/llm/api/client_translators/openai_responses.rb +238 -185
- data/lib/legion/llm/api/client_translators/server_tool_transcript.rb +35 -0
- data/lib/legion/llm/api/client_translators/shared_extractors.rb +191 -21
- data/lib/legion/llm/api/debug_formats.rb +11 -37
- data/lib/legion/llm/api/inventory_admin.rb +16 -12
- data/lib/legion/llm/api/model_catalog.rb +25 -28
- data/lib/legion/llm/api/namespaces/anthropic/messages/count_tokens.rb +6 -10
- data/lib/legion/llm/api/namespaces/anthropic/messages.rb +9 -1
- data/lib/legion/llm/api/namespaces/helpers.rb +6 -0
- data/lib/legion/llm/api/namespaces/native/inference.rb +4 -1
- data/lib/legion/llm/api/namespaces/native/models.rb +2 -2
- data/lib/legion/llm/api/namespaces/native/providers.rb +6 -24
- data/lib/legion/llm/api/namespaces/native/routing.rb +3 -4
- data/lib/legion/llm/api/namespaces/openai/chat/completions.rb +3 -0
- data/lib/legion/llm/api/namespaces/openai/models.rb +9 -40
- data/lib/legion/llm/api/namespaces/openai/responses.rb +4 -4
- data/lib/legion/llm/api/namespaces/registration.rb +2 -2
- data/lib/legion/llm/api/native/models.rb +139 -5
- data/lib/legion/llm/api/native/offerings.rb +55 -61
- data/lib/legion/llm/api/native/providers.rb +38 -50
- data/lib/legion/llm/api/native/routing.rb +3 -4
- data/lib/legion/llm/api/native/tiers.rb +31 -24
- data/lib/legion/llm/api/openai/chat_completions.rb +8 -0
- data/lib/legion/llm/api/openai/models.rb +2 -1
- data/lib/legion/llm/api/openai/responses.rb +6 -0
- data/lib/legion/llm/api/routing_error_mapper.rb +4 -3
- data/lib/legion/llm/api/stream_assembler.rb +181 -223
- data/lib/legion/llm/api/translators/anthropic_response.rb +13 -4
- data/lib/legion/llm/api/translators/openai_response.rb +5 -0
- data/lib/legion/llm/api.rb +7 -0
- data/lib/legion/llm/audit.rb +11 -3
- data/lib/legion/llm/audit_projection.rb +90 -0
- data/lib/legion/llm/cache/response.rb +3 -3
- data/lib/legion/llm/cache.rb +0 -14
- data/lib/legion/llm/call/daemon_client.rb +1 -1
- data/lib/legion/llm/call/dispatch.rb +34 -112
- data/lib/legion/llm/call/embeddings.rb +198 -311
- data/lib/legion/llm/call/lex_llm_adapter.rb +230 -309
- data/lib/legion/llm/call/selection_dispatch.rb +35 -8
- data/lib/legion/llm/compat.rb +6 -21
- data/lib/legion/llm/context/compressor.rb +12 -1
- data/lib/legion/llm/context/curator.rb +2 -2
- data/lib/legion/llm/errors.rb +22 -6
- data/lib/legion/llm/fleet/dispatcher.rb +56 -37
- data/lib/legion/llm/fleet/token_issuer.rb +14 -9
- data/lib/legion/llm/fleet.rb +0 -1
- data/lib/legion/llm/inference/attempt_context.rb +9 -2
- data/lib/legion/llm/inference/audit_publisher.rb +69 -27
- data/lib/legion/llm/inference/embed_pipeline.rb +4 -2
- data/lib/legion/llm/inference/executor/context_window.rb +100 -29
- data/lib/legion/llm/inference/executor/escalation.rb +48 -67
- data/lib/legion/llm/inference/executor/payload_builder.rb +16 -5
- data/lib/legion/llm/inference/executor/routing.rb +31 -41
- data/lib/legion/llm/inference/executor/tool_injection.rb +29 -10
- data/lib/legion/llm/inference/executor.rb +228 -190
- data/lib/legion/llm/inference/native_tool_loop.rb +96 -51
- data/lib/legion/llm/inference/prompt.rb +7 -4
- data/lib/legion/llm/inference/request.rb +45 -26
- data/lib/legion/llm/inference/response.rb +4 -22
- data/lib/legion/llm/inference/route_attempts.rb +78 -29
- data/lib/legion/llm/inference/steps/classification.rb +8 -15
- data/lib/legion/llm/inference/steps/debate.rb +22 -17
- data/lib/legion/llm/inference/steps/metering.rb +2 -1
- data/lib/legion/llm/inference/steps/post_response.rb +16 -18
- data/lib/legion/llm/inference/steps/prompt_cache.rb +20 -2
- data/lib/legion/llm/inference/steps/rag_context.rb +27 -40
- data/lib/legion/llm/inference/steps/rag_guard.rb +4 -1
- data/lib/legion/llm/inference/steps/tier_assigner.rb +8 -12
- data/lib/legion/llm/inference/steps/tool_calls.rb +66 -50
- data/lib/legion/llm/inference/steps/tool_history.rb +5 -1
- data/lib/legion/llm/inference/steps/trigger_match.rb +6 -16
- data/lib/legion/llm/inference.rb +67 -80
- data/lib/legion/llm/inventory/discovery.rb +20 -478
- data/lib/legion/llm/inventory.rb +37 -335
- data/lib/legion/llm/metering.rb +20 -7
- data/lib/legion/llm/quality/checker.rb +4 -2
- data/lib/legion/llm/router.rb +513 -222
- data/lib/legion/llm/routing/TODO.md +46 -0
- data/lib/legion/llm/{router/candidate_evaluation.rb → routing/evaluation.rb} +14 -17
- data/lib/legion/llm/routing/filter.rb +309 -0
- data/lib/legion/llm/routing/fleet.rb +32 -0
- data/lib/legion/llm/{router → routing}/header_constraints.rb +1 -1
- data/lib/legion/llm/{router/outcome_classifier.rb → routing/outcome.rb} +39 -28
- data/lib/legion/llm/routing/rank.rb +210 -0
- data/lib/legion/llm/routing/requirements.rb +198 -0
- data/lib/legion/llm/{router → routing}/settings_snapshot.rb +20 -14
- data/lib/legion/llm/{router → routing}/settings_state.rb +4 -4
- data/lib/legion/llm/settings/router.rb +98 -0
- data/lib/legion/llm/settings/tools.rb +5 -0
- data/lib/legion/llm/settings.rb +54 -46
- data/lib/legion/llm/tools/dispatcher.rb +6 -1
- data/lib/legion/llm/tools/mcp_execution.rb +76 -0
- data/lib/legion/llm/tools/special.rb +9 -2
- data/lib/legion/llm/transport/messages/prompt_event.rb +119 -0
- data/lib/legion/llm/types.rb +0 -4
- data/lib/legion/llm/version.rb +1 -1
- data/lib/legion/llm.rb +16 -53
- metadata +35 -29
- data/lib/legion/llm/fleet/handler.rb +0 -267
- data/lib/legion/llm/inference/routing_session.rb +0 -144
- data/lib/legion/llm/inventory/capabilities.rb +0 -48
- data/lib/legion/llm/inventory/discovery/memory_gate.rb +0 -55
- data/lib/legion/llm/inventory/discovery/system.rb +0 -138
- data/lib/legion/llm/inventory/settings_observer.rb +0 -61
- data/lib/legion/llm/router/body_model_hint_policy.rb +0 -97
- data/lib/legion/llm/router/candidate_evaluator.rb +0 -399
- data/lib/legion/llm/router/escalation/history.rb +0 -28
- data/lib/legion/llm/router/input_bound.rb +0 -151
- data/lib/legion/llm/router/ranker.rb +0 -231
- data/lib/legion/llm/router/rejection_diagnostics.rb +0 -273
- data/lib/legion/llm/router/request_requirements.rb +0 -162
- data/lib/legion/llm/router/required_capabilities.rb +0 -144
- data/lib/legion/llm/router/resolution.rb +0 -74
- data/lib/legion/llm/types/chunk.rb +0 -45
- data/lib/legion/llm/types/content_block.rb +0 -55
- data/lib/legion/llm/types/message.rb +0 -103
- data/lib/legion/llm/types/tool_call.rb +0 -87
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: cf4c21744c43d3e9ab870e0775b991947e5251e1cfe9c0cbdebc4ebb60c1f89d
|
|
4
|
+
data.tar.gz: 6c6a86bf0bb8f56612ea47fdacdeae4ae772d60a2d6b7a33369fb9e65f967a9c
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 6100698937fe2989bf58d6aaf00734f1b7dd97849f53c954ad90297c103337ec0b1dcdfebd18ffc7e2252fcf3efa251b48ac7f13c878dfabfade604a8f9cca83
|
|
7
|
+
data.tar.gz: 5eca66202c9c79d15f6f37f207c2abd75d091d9c7ccc2b7fa952252087ef1b71fa73fc47df2b4733d04735b5670f5a53a0d8945c3de5606b8509ee2482d8852c
|
data/.github/workflows/ci.yml
CHANGED
data/.gitignore
CHANGED
data/.rubocop.yml
CHANGED
|
@@ -1,38 +1,12 @@
|
|
|
1
1
|
plugins:
|
|
2
2
|
- rubocop-legion
|
|
3
3
|
|
|
4
|
-
# These rubocop-legion cops surface in the local-path 0.1.7 build but were not
|
|
5
|
-
# in the published 0.1.7 gem the repo previously tracked. They flag broad
|
|
6
|
-
# pre-existing patterns unrelated to the N×N enforcement pass; deferred to
|
|
7
|
-
# their own cleanup task. NoUnderscorePrefixedKwargs / NoInlineSettingDefaults
|
|
8
|
-
# / NoDirectDispatch / NoShapeDuckTyping (the B4 set Phase 6 adopts) remain
|
|
9
|
-
# enabled.
|
|
10
4
|
Legion/RescueLogging/NoCapture:
|
|
11
5
|
Enabled: false
|
|
12
|
-
Legion/ConstantSafety/InheritParam:
|
|
13
|
-
Enabled: false
|
|
14
|
-
|
|
15
|
-
# rubocop-legion 0.1.9 cops — enabled as of P0 but deferred enforcement:
|
|
16
|
-
# these four cops flag broad pre-existing patterns in the codebase that
|
|
17
|
-
# predate the SSOT refactor. They gate NEW code from P1 onward; existing
|
|
18
|
-
# violations are cleaned up within each phase as the code they govern is
|
|
19
|
-
# rewritten. Do not add new violations; do not suppress them with inline
|
|
20
|
-
# rubocop:disable without a tracking comment.
|
|
21
|
-
#
|
|
22
|
-
# TODO(P1): enable Legion/Llm/TaxonomyEnum repo-wide after lane taxonomy
|
|
23
|
-
# is established and all :type/:tier/:circuit_state literals updated.
|
|
24
|
-
# TODO(P1): enable Legion/Llm/RescueLogLevel repo-wide after back-compat
|
|
25
|
-
# :debug rescue handlers are audited and leveled up.
|
|
26
|
-
# TODO(P1): enable Legion/Llm/NoLoopDo repo-wide after cache/drain loops
|
|
27
|
-
# are converted to bounded iteration.
|
|
28
6
|
Legion/Llm/TaxonomyEnum:
|
|
29
7
|
Enabled: false
|
|
30
8
|
Legion/Llm/RescueLogLevel:
|
|
31
9
|
Enabled: false
|
|
32
|
-
Legion/Llm/NoLoopDo:
|
|
33
|
-
Enabled: false
|
|
34
|
-
# SettingsAccessPath is enabled for lib/ only — specs legitimately write
|
|
35
|
-
# settings via the loader path to set up test fixtures.
|
|
36
10
|
Legion/Llm/SettingsAccessPath:
|
|
37
11
|
Exclude:
|
|
38
12
|
- 'spec/**/*'
|
|
@@ -41,38 +15,14 @@ AllCops:
|
|
|
41
15
|
TargetRubyVersion: 3.4
|
|
42
16
|
NewCops: enable
|
|
43
17
|
SuggestExtensions: false
|
|
44
|
-
|
|
45
|
-
# N×N routing guard cops (Phase 6 enforcement; defaults from rubocop-legion config/default.yml).
|
|
46
|
-
#
|
|
47
|
-
# - NoUnderscorePrefixedKwargs / NoInlineSettingDefaults / NoDirectDispatch are
|
|
48
|
-
# enabled repo-wide.
|
|
49
|
-
# - NoShapeDuckTyping is enabled on the canonical-only surface where the shape
|
|
50
|
-
# contract is fully established by translators. Code at the HTTP/wire ingress
|
|
51
|
-
# (client_translator parse_request, StreamAssembler chunk adapter, DebugFormats
|
|
52
|
-
# request envelope reader, Response.from_provider_message bridge) legitimately
|
|
53
|
-
# inspects shape because that's the layer responsible for normalising into
|
|
54
|
-
# canonical. The cop's scope expands here as the executor's canonical
|
|
55
|
-
# migration (Phase 4 follow-up) lands.
|
|
56
18
|
Legion/Framework/NoShapeDuckTyping:
|
|
57
|
-
Enabled:
|
|
19
|
+
Enabled: false
|
|
58
20
|
Include:
|
|
21
|
+
#- 'lib/legion/llm/api/**/*.rb'
|
|
22
|
+
# - 'lib/legion/llm/inference/**/*.rb'
|
|
23
|
+
Exclude:
|
|
59
24
|
- 'lib/legion/llm/api/**/*.rb'
|
|
60
25
|
- 'lib/legion/llm/inference/**/*.rb'
|
|
61
|
-
Exclude:
|
|
62
|
-
# Legacy tree deprecated this release (R11); deleted next minor.
|
|
63
|
-
- 'lib/legion/llm/api/translators/**/*.rb'
|
|
64
|
-
- 'lib/legion/llm/api/anthropic/**/*.rb'
|
|
65
|
-
- 'lib/legion/llm/api/openai/**/*.rb'
|
|
66
|
-
- 'lib/legion/llm/api/native/**/*.rb'
|
|
67
|
-
- 'lib/legion/llm/api/shared_helpers.rb'
|
|
68
|
-
# Boundary code that legitimately bridges wire ↔ canonical: parse_request
|
|
69
|
-
# at the HTTP ingress, the StreamAssembler chunk adapter (P5 explicitly
|
|
70
|
-
# accepts both Canonical::Chunk and the legacy StreamChunk shape during
|
|
71
|
-
# migration), DebugFormats (reads raw env / reflects request), and
|
|
72
|
-
# shared_extractors (normalises arbitrary thinking content shapes).
|
|
73
|
-
# Pre-canonical inference steps still navigate raw wire hashes; that
|
|
74
|
-
# scope tightens once the executor finishes the canonical migration
|
|
75
|
-
# (Phase 4 follow-up).
|
|
76
26
|
- 'lib/legion/llm/api/client_translators/anthropic_messages.rb'
|
|
77
27
|
- 'lib/legion/llm/api/client_translators/openai_chat.rb'
|
|
78
28
|
- 'lib/legion/llm/api/client_translators/openai_responses.rb'
|
|
@@ -108,14 +58,12 @@ Metrics/BlockLength:
|
|
|
108
58
|
Max: 150
|
|
109
59
|
Exclude:
|
|
110
60
|
- 'spec/**/*'
|
|
111
|
-
|
|
112
61
|
Metrics/AbcSize:
|
|
113
62
|
Max: 110
|
|
114
63
|
Metrics/BlockNesting:
|
|
115
64
|
Max: 4
|
|
116
65
|
Metrics/CyclomaticComplexity:
|
|
117
66
|
Max: 50
|
|
118
|
-
|
|
119
67
|
Metrics/PerceivedComplexity:
|
|
120
68
|
Max: 50
|
|
121
69
|
Style/Documentation:
|
|
@@ -125,8 +73,6 @@ Style/SymbolArray:
|
|
|
125
73
|
Style/FrozenStringLiteralComment:
|
|
126
74
|
Enabled: true
|
|
127
75
|
EnforcedStyle: always
|
|
128
|
-
Naming/FileName:
|
|
129
|
-
Enabled: false
|
|
130
76
|
Naming/PredicateMethod:
|
|
131
77
|
Enabled: false
|
|
132
78
|
Metrics/ParameterLists:
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,123 @@
|
|
|
1
1
|
# Legion LLM Changelog
|
|
2
2
|
|
|
3
|
+
## [0.16.20] - 2026-11-18
|
|
4
|
+
|
|
5
|
+
### Fixed
|
|
6
|
+
- Replace unconditional RAG entry dumps with lazy DEBUG diagnostics limited to Apollo entry ID, content type, numeric confidence/distance, content length, and request/strategy context. Retrieved text, raw source, tags, and arbitrary metadata stay out of these logs while enrichment retains the complete entries. Move the adjacent strategy diagnostic to DEBUG ([#193](https://github.com/LegionIO/legion-llm/issues/193)).
|
|
7
|
+
|
|
8
|
+
## [0.16.19] - 2026-11-18
|
|
9
|
+
|
|
10
|
+
### Fixed
|
|
11
|
+
- Normalize image MIME case and the `image/jpg` alias, strip ASCII whitespace from base64, and restore omitted padding without changing the caller's input. Preserve other image MIME types for provider validation instead of imposing one provider's image-format allowlist at ingress.
|
|
12
|
+
- Reject unsupported content block types with HTTP 400 before inference or streaming begins across Messages, Chat Completions, and Responses.
|
|
13
|
+
- Fingerprint oversized audit message content, tool arguments/results, and metadata before omitting older messages. Compact bulky event fields first, preserve tool IDs, names, message associations and content digests, and retain bounded plaintext/encrypted payloads. Shared audit projection now lives below inference and transport.
|
|
14
|
+
- Report durable spooled prompt audits as `spooled`, distinct from published, disabled, or dropped events. Preserve this status in the execution timeline and local audit observation.
|
|
15
|
+
- Initialize native tool-choice spec cleanup state before setup can raise. Remove temporary companion-gem CI pins after publication so pull requests, main, scheduled checks, and releases resolve published dependencies.
|
|
16
|
+
|
|
17
|
+
## [0.16.18] - 2026-11-18
|
|
18
|
+
|
|
19
|
+
### Fixed
|
|
20
|
+
- Preserve routed embedding provenance on fresh, batch, and cached results: provider/model, selected instance, authoritative revision when available, actual dimensions, document/query prefix pair, normalization, task, and stable space ID. Share a space across instances only with matching revision evidence; isolate unknown revisions by instance ([#199](https://github.com/LegionIO/legion-llm/pull/199), incorporated into #200).
|
|
21
|
+
- Validate complete embedding attempts before returning or caching vectors. Reject missing, nested, nonnumeric, nonfinite, zero, and inconsistent-dimension vectors through the existing routing retry policy; never discard malformed chunks or label vectors with dimensions they do not have. Consume both canonical embedding artifacts and the defined `LexLLMAdapter` result shape.
|
|
22
|
+
- Regenerate legacy or incompatible cache entries instead of assigning them new provenance. Compatible hits preserve cached token accounting and identify the currently selected instance; changing either task prefix invalidates the shared embedding space.
|
|
23
|
+
|
|
24
|
+
## [0.16.17] - 2026-11-18
|
|
25
|
+
|
|
26
|
+
### Fixed
|
|
27
|
+
- Preserve embedding request `input_tokens` and `total_tokens` on cache hits so ledger can account for served workload and counterfactual savings. Cached requests retain `cache_hit: true`, zero output tokens, and `cost_usd: 0` without another provider execution. Missing provider usage remains zero. Regression coverage verifies serialized miss/hit events through real selection, caching, and metering, alongside routing and provider-error paths ([#201](https://github.com/LegionIO/legion-llm/issues/201)).
|
|
28
|
+
|
|
29
|
+
## [0.16.16] - 2026-11-18
|
|
30
|
+
|
|
31
|
+
### Fixed
|
|
32
|
+
- Execute local Legion tools through the existing MCP SDK adapter in-process, without an HTTP hop or daemon re-entry. Render documented `mcp_call` items and lifecycle events in OpenAI Responses, preserve full results and replay their call/result history. Chat Completions retains visible assistant transcript content; executable function calls remain reserved for client-owned tools. Preserve the Anthropic server-tool representation and withhold unclassified streaming fragments until ownership is known.
|
|
33
|
+
- Preserve complete server-tool results in canonical execution records delivered to stream observers and final responses. Diagnostic event previews remain bounded without truncating the client-visible execution result.
|
|
34
|
+
- Retain earlier server executions when a later provider round hands off a client tool, and prevent completed tool calls from executing again in the post-response step.
|
|
35
|
+
- Render the same terminal state in synchronous and streaming Responses: token exhaustion and content filtering include the documented incomplete reason, and streams emit `response.incomplete` instead of claiming completion. Preserve already-completed MCP results when a later answer is truncated.
|
|
36
|
+
|
|
37
|
+
## [0.16.15] - 2026-11-18
|
|
38
|
+
|
|
39
|
+
### Fixed
|
|
40
|
+
- Require thinking-capable inventory lanes when canonical thinking is enabled. Native request input uses the shared canonical normalizer at ingress; routing reads the canonical flag rather than looking for a Hash, preserving explicit budget and effort independently. Disabled or absent thinking leaves text-only lanes eligible.
|
|
41
|
+
|
|
42
|
+
## [0.16.14] - 2026-11-18
|
|
43
|
+
|
|
44
|
+
### Fixed
|
|
45
|
+
- Include tool counts in `legion_list_special_tools` results so callers can answer inventory-size questions without enumerating every definition. Counts come from the same returned snapshot, total distinct callable names, and preserve both complete tool lists and runtime details.
|
|
46
|
+
|
|
47
|
+
## [0.16.13] - 2026-11-18
|
|
48
|
+
|
|
49
|
+
### Fixed
|
|
50
|
+
- Preserve canonical thinking-token usage through the shared executor and stream assembler. Responses clients receive provider-reported reasoning counts in both synchronous usage and stream trailers, even when no thinking text is returned, without inventing a reasoning block or double-counting output tokens.
|
|
51
|
+
- Retain canonical usage during post-response extraction and carry its thinking count into metering. Ledger totals count input plus output once; thinking remains an output-token breakdown.
|
|
52
|
+
- Pin CI and security checks to the shared canonical system-string validation used by the local matrix suite.
|
|
53
|
+
- Install the Bedrock test dependency in standalone checkouts and pin its verified companion in CI so overload-isolation coverage runs without an adjacent workspace.
|
|
54
|
+
|
|
55
|
+
## [0.16.12] - 2026-11-17
|
|
56
|
+
|
|
57
|
+
### Fixed
|
|
58
|
+
- Preserve output-token limits, stop sequences and sampling parameters as canonical dispatch params across all client dialects and both direct/fleet execution paths.
|
|
59
|
+
- Record native server-tool executions in the shared execution timeline so completion summaries and audit evidence agree with actual tool calls.
|
|
60
|
+
- Read provider listings and family details from one authoritative Inventory snapshot. Pending configuration claims no longer masquerade as published instances; health and capabilities come from the published records in both API route trees.
|
|
61
|
+
|
|
62
|
+
## [0.16.11] - 2026-11-17
|
|
63
|
+
|
|
64
|
+
### Fixed
|
|
65
|
+
- **Request-scoped routing requirements.** Include `Routing::Requirements` in Router so each instance initializes its own request facts once and exposes direct readers such as `maximum_attempts`. Preserve shared immutable-value validation, canonical tool replay, capabilities, pins, context budgets and ingress policy generation.
|
|
66
|
+
- **Router constructor kwargs.** Restore trailing `**` so `Router.new(request:, operation:, body_model:)` accepts additional keyword arguments.
|
|
67
|
+
- **Tool-result citation preservation.** Retain text-block citations and other metadata before canonical construction instead of dropping annotations and collapsing the result to a string. Normalization does not mutate the client's metadata.
|
|
68
|
+
- **Cached system instructions.** Normalize OpenAI Chat system text to the canonical string contract, preserving exact newlines instead of injecting Ruby hash representations into the prompt.
|
|
69
|
+
- **Cached history safeguards.** Strip leading thinking across adjacent annotated text blocks and enforce one historical tool-result text cap across blocks, retaining surviving annotations, media, tool associations, and current-turn content.
|
|
70
|
+
- **Streaming tool identity.** Preserve Hash and typed Canonical tool-call fragments, correlate parallel argument deltas by provider index, and reset that index mapping between provider rounds without fabricating client-side identities.
|
|
71
|
+
- **Live server-tool delivery.** Publish execution-owned tool calls and results through the shared stream observer before continuation completes, upgrade unresolved source classification, and avoid duplicate call/result events during finalization.
|
|
72
|
+
- **Named tool-choice lifecycle.** Release a named tool choice only after that server-side call completes, allowing continuation without forcing the same tool again while leaving the original request and unrelated choice modes unchanged.
|
|
73
|
+
- **Typed streaming tool snapshots.** Hold replaceable Hash argument snapshots until tool-call completion instead of emitting contradictory append-only JSON. Raw JSON fragments still stream incrementally, and Anthropic Messages, Chat Completions, and Responses all receive valid final arguments and terminal events.
|
|
74
|
+
- **Anthropic streaming tool input.** Start tool blocks with an empty input object; argument snapshots and raw JSON fragments are delivered only through `input_json_delta`, preventing intermediate values or partial JSON strings from leaking through `content_block_start`.
|
|
75
|
+
- Preserve shared registry trigger words during request matching. Repeated requests no longer append matching aliases to registered tools or inflate server-tool inventory responses.
|
|
76
|
+
- Propagate stream-consumer exceptions unchanged through exact callable dispatch, releasing the lease without misclassifying client disconnects or translation errors as provider failures.
|
|
77
|
+
|
|
78
|
+
### Changed
|
|
79
|
+
- Require `lex-llm >= 0.8.6` for shared request-fact validation; executor, embedding, token-counting and cache consumers construct the per-request Router directly.
|
|
80
|
+
|
|
81
|
+
## [0.16.10] - 2026-10-30
|
|
82
|
+
|
|
83
|
+
### Fixed
|
|
84
|
+
- **Canonical prompt auditing.** Histories beyond `llm.compliance.audit_max_messages` no longer call Hash accessors on canonical messages. Audit projections retain the newest messages without changing inference history, and record original, captured, and omitted counts.
|
|
85
|
+
- **Whole-event audit payload limits.** Prompt events account for attribution and encrypted wire size against `transport.max_payload_bytes`. Oversized binary content becomes explicit byte-count/SHA256 fingerprints, including nested image/audio/video content; aggregate overflow removes remaining binary content before older messages and bulky nonessential fields. An essential envelope that cannot fit is explicitly dropped.
|
|
86
|
+
- **Multimodal audit identity.** Request audit hashes cover the full original message history, media fingerprints, roles, and tool state, excluding generated message IDs and timestamps. The text-only embedding/RAG hash contract is unchanged.
|
|
87
|
+
- **Honest publication outcomes.** Post-response timelines distinguish published, disabled, and dropped audit delivery. GAIA can still observe the constructed local event when transport does not accept publication.
|
|
88
|
+
- **Anthropic pasted images and tool results.** Normalize base64 and URL source envelopes at both user and tool-result boundaries, preserving mixed content order, tool association, and cache-control metadata.
|
|
89
|
+
- **OpenAI image inputs.** Normalize Responses `input_image` and Chat `image_url` parts without dropping base64 MIME or image detail. Malformed image sources, provider-specific file references, and unsupported generated-image history return HTTP 400 before streaming begins.
|
|
90
|
+
- **N×N text convergence.** All client dialects preserve text-fragment whitespace and converge to the same canonical text representation without collapsing annotated blocks.
|
|
91
|
+
- **Request-owned routing facts.** Construct shared immutable `RequestRequirements` before routing, including canonical tool replay, capabilities, pins and context budgets. Router consumes those facts without reconstructing them.
|
|
92
|
+
- **Real execution boundary coverage.** Exercise client ingress, Inventory selection, direct callable execution, signed fleet serialization/validation/rehydration, exact worker resolution, provider rendering and rejection of tampered bindings/media.
|
|
93
|
+
- **Router constructor kwargs.** Restore trailing `**` so `Router.new(requirements:)` accepts additional keyword arguments.
|
|
94
|
+
- **Prompt audit require path.** Load `AuditPublisher` through its gem-rooted require path instead of `require_relative`.
|
|
95
|
+
|
|
96
|
+
### Changed
|
|
97
|
+
- Declare the vLLM renderer as a test dependency in standalone checkouts so image regressions run in CI as well as the multi-repo workspace. The RAG integration regression now observes the active provider boundary instead of the retired dispatch path.
|
|
98
|
+
- Require `legion-transport >= 1.4.31` and use its encoded-payload validation boundary directly, removing the private pre-encoding validator override.
|
|
99
|
+
- Require `lex-llm >= 0.8.6` and use `Router.new(requirements:)`; all executor, embedding, token-counting and cache consumers use the same requirements constructor.
|
|
100
|
+
|
|
101
|
+
## [0.16.9] - 2026-08-20
|
|
102
|
+
|
|
103
|
+
### Changed
|
|
104
|
+
- **lex-llm 0.8.0 contract conformance.** Gemspec floor raised to `lex-llm >= 0.8.0`; the repo conforms to the 0.8.0 contract cut (Canonical-only internal language, fleet protocol v3, documented embed/image artifacts).
|
|
105
|
+
- **Canonical messages end-to-end.** Client translators no longer de-canonicalize (`inference_messages` bridges deleted); `Inference::Request` canonicalizes inbound messages at the pipeline entry (plain strings, wire hashes, and Canonical objects are accepted; anything else raises). Executor `context_window`, `native_tool_loop`, `step_context_store`, and `prompt_cache` consume strict `Canonical::Message` accessors.
|
|
106
|
+
- **Fleet requestor is protocol v3.** Dispatcher emits `protocol_version: 3` (explicit, no default fill), requires `execution_contract: exact_offering_v1` + a nonempty `offering_id` (no default provider/instance fill), signs both exact claims unconditionally (S3), and rejects legacy field names at the edge. The reply path rehydrates the serialized `Canonical::Response` under `response:` (E3) — the v2 `content`/`finish_reason` projection is gone.
|
|
107
|
+
- **Client-edge dialect aliasing (O03a).** The three client translators translate client-wire spellings at the edge: params (`max_output_tokens`/`num_predict`/`max_completion_tokens` → `max_tokens`, `budget_tokens`/`thinking_budget` → `max_thinking_tokens`, `stop` → `stop_sequences`), content-type aliases (`input_text`/`output_text` → `text`, `image_url` → canonical `image` block), and JSON-string tool arguments → Hash. Canonical types accept canonical keys/types only.
|
|
108
|
+
- **`/api/llm/providers` surfaces SSOT-activated instances.** The listing unions the Inventory Registry snapshot (deduped by provider+instance) with the compatibility call registry.
|
|
109
|
+
- **`RULES.md` installed** (architecture law, byte-for-byte from the SSOT v4 reference).
|
|
110
|
+
- **SSOT v4: routing behavior extracted into stateless mixins under `Legion::LLM::Routing::`.** `Filter` (lane-eligibility axes including the body-model-hint decision ladder, policy enforcement, capability/context/dimension checks), `Rank` (weight/affinity/rendezvous ranking with band partition), `Fleet` (fleet-contract evaluation), `Outcome` (provider-outcome classification — renamed from `OutcomeClassifier`), and immutable records `CandidateEvaluation`/`EvaluationSet` (`routing/evaluation.rb`, 10-axis per-lane readiness snapshot). Each mixin is independently unit-testable with no cross-mixin or Router class dependency.
|
|
111
|
+
- **SSOT v4: Router is a per-request CLASS** (written in `router_new.rb`, wiring pending). One `Legion::LLM::Router.new(request:, operation:, body_model:)` per logical request; it holds the request's attempt state (exclusions, consumed targets, attempt budget, last rejection) — absorbing the former `Inference::RoutingSession` role. Request-derived facts (filters, pins, required capabilities, input bound, context budget, the body-model-hint decision) are computed once in `initialize` and are immutable for the request's lifetime. Inventory is fetched live at each lane decision from `Inventory::Registry` — no captured snapshot.
|
|
112
|
+
- **Routing settings read directly from `Legion::Settings[:llm][:router]`.** Defaults centralized in `lib/legion/llm/settings/router.rb`. Per-provider/instance/model cascades (model policy allow/deny, `enable_<capability>` overrides, preferred-context ranges) are resolved live from `Legion::Settings[:extensions][:llm]` via the shared `SettingsCascade`. The per-request `SettingsState`/`SettingsSnapshot` objects are eliminated from the routing path.
|
|
113
|
+
|
|
114
|
+
### Removed
|
|
115
|
+
- **Legacy rip (0.8.0 cut).** The `Responses::StreamChunk` detection in `StreamAssembler` (type deleted upstream) and its tool-call accumulator; dead `Types::Message#to_provider_hash`; the legacy `Fleet::Handler` responder path (replaced by the lex-llm `ProviderResponder` registry topology); provider-native `Embedding` value-object handling in the embed consumer (replaced by the documented 0.8.0 embed artifact `{ text:, model:, embedding:, usage: }`).
|
|
116
|
+
- **Legacy `lib/legion/llm/router/` support folder retired.** 11 files deleted (`candidate_evaluator`, `ranker`, `rejection_diagnostics`, `required_capabilities`, `input_bound`, `request_requirements`, `body_model_hint_policy`, `resolution`, `settings_state`, `settings_snapshot`, `header_constraints`) and 2 renamed into the new namespace (`candidate_evaluation` → `routing/evaluation.rb`, `outcome_classifier` → `routing/outcome.rb`) — their logic reproduced across the `routing/` mixins and the Router class. (`escalation/history.rb` remains until the `llm.rb` require graph is rewired.) The 18 old specs under `spec/legion/llm/router/` were removed: 16 reproduced or subsumed by the new `spec/legion/llm/routing/` suites + `router_new_spec.rb`, and the two behavioral regressions (fail-forward release bar, instance-recovery) re-encoded against the Router class as `spec/legion/llm/router/{fail_forward_release_bar,instance_recovery_regression}_spec.rb`.
|
|
117
|
+
|
|
118
|
+
### Notes
|
|
119
|
+
- **Migration is mid-flight.** The routing mixins (`routing/*.rb`) are landed and spec-covered; the old support folder is deleted. The Router class replacement (`router_new.rb`) is written but not yet wired into the require graph (module→class swap and consumer rewiring are pending). The existing `router.rb` stateless module remains the active entry point until the swap lands.
|
|
120
|
+
|
|
3
121
|
## [0.16.8] - 2026-08-19
|
|
4
122
|
|
|
5
123
|
### Fixed
|
data/CLAUDE.md
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
|
-
# legion-llm
|
|
1
|
+
# legion-llm
|
|
2
2
|
|
|
3
3
|
Core LegionIO gem: LLM routing, provider dispatch, the inference pipeline, and the
|
|
4
|
-
OpenAI/Anthropic-compatible API surface.
|
|
5
|
-
|
|
4
|
+
OpenAI/Anthropic-compatible API surface. Detailed reference lives in `README.md`; deep history
|
|
5
|
+
lives in git.
|
|
6
6
|
|
|
7
7
|
## What it is (the one-paragraph model)
|
|
8
8
|
|
|
@@ -14,11 +14,6 @@ gems. The daemon is an **execution proxy**, not a passthrough (see Invariants).
|
|
|
14
14
|
|
|
15
15
|
## Build & Test
|
|
16
16
|
|
|
17
|
-
```bash
|
|
18
|
-
bundle exec rspec # ~3200 examples, must be 0 failures before any commit
|
|
19
|
-
bundle exec rubocop # 0 offenses required (rubocop-legion plugin)
|
|
20
|
-
```
|
|
21
|
-
|
|
22
17
|
The **in-process matrix harness** (`spec/legion/llm/api/matrix/`) is the **commit gate**. It boots
|
|
23
18
|
the real Sinatra app, mounts `/v1/messages` `/v1/responses` `/v1/chat/completions`, and replays the
|
|
24
19
|
full client × scenario matrix against a deterministic `FakeProvider` in ~250ms. Any change to
|
|
@@ -51,9 +46,14 @@ These have caused production incidents. They are also enforced by `rubocop-legio
|
|
|
51
46
|
`Legion/Framework` cops, lex-llm conformance fixtures, and the matrix harness.
|
|
52
47
|
|
|
53
48
|
1. **Execution-proxy contract.** To the *client*, the daemon is a server-side execution surface:
|
|
54
|
-
LegionIO-resolved tools run server-side and
|
|
55
|
-
`server_tool_use`/`server_tool_result`; OpenAI Responses
|
|
56
|
-
|
|
49
|
+
LegionIO-resolved tools run server-side and render from canonical execution records (Anthropic
|
|
50
|
+
`server_tool_use`/`server_tool_result`; OpenAI Responses `mcp_call` items for MCP executions;
|
|
51
|
+
Chat visible assistant content carrying call ID, name, arguments, and full result). Local
|
|
52
|
+
Legion handlers execute through the existing MCP SDK adapter in-process, without HTTP or
|
|
53
|
+
daemon API re-entry. Canonical metadata preserves the MCP server label; direct legacy
|
|
54
|
+
executions without MCP provenance retain a visible text fallback. OpenAI `function_call`/`tool_calls` items
|
|
55
|
+
request client execution even with `status: completed`, so they are reserved for client-owned
|
|
56
|
+
calls. Unclassified streaming fragments wait for execution ownership to resolve. To the
|
|
57
57
|
*provider*, the daemon looks like the client: the same tool-use/tool-result exchange appears in
|
|
58
58
|
the next turn in that provider's wire format. The model must know files changed.
|
|
59
59
|
2. **Always translate; never passthrough.** Every request → `Canonical::Request`; every response
|
data/Gemfile
CHANGED
|
@@ -7,21 +7,45 @@ gemspec
|
|
|
7
7
|
legion_settings_path = File.expand_path('../legion-settings', __dir__)
|
|
8
8
|
gem 'legion-settings', path: legion_settings_path if Dir.exist?(legion_settings_path)
|
|
9
9
|
|
|
10
|
+
legion_mcp_path = File.expand_path('../legion-mcp', __dir__)
|
|
11
|
+
gem 'legion-mcp', path: legion_mcp_path if Dir.exist?(legion_mcp_path)
|
|
12
|
+
|
|
10
13
|
group :test do
|
|
11
|
-
#
|
|
12
|
-
#
|
|
13
|
-
|
|
14
|
-
lex_llm_path = File.expand_path('../extensions-ai/lex-llm', __dir__)
|
|
14
|
+
# Immutable CI companions and adjacent development checkouts both consume the
|
|
15
|
+
# shared contract directly. Standalone installs use the gemspec dependency.
|
|
16
|
+
lex_llm_path = File.expand_path('.ci-dependencies/lex-llm', __dir__)
|
|
17
|
+
lex_llm_path = File.expand_path('../extensions-ai/lex-llm', __dir__) unless Dir.exist?(lex_llm_path)
|
|
15
18
|
gem 'lex-llm', path: lex_llm_path if Dir.exist?(lex_llm_path)
|
|
16
19
|
|
|
20
|
+
# Execution regressions exercise real provider callables in every checkout.
|
|
21
|
+
vllm_path = File.expand_path('.ci-dependencies/lex-llm-vllm', __dir__)
|
|
22
|
+
vllm_path = File.expand_path('../extensions-ai/lex-llm-vllm', __dir__) unless Dir.exist?(vllm_path)
|
|
23
|
+
if Dir.exist?(vllm_path)
|
|
24
|
+
gem 'lex-llm-vllm', path: vllm_path
|
|
25
|
+
else
|
|
26
|
+
gem 'lex-llm-vllm', '>= 0.4.9'
|
|
27
|
+
end
|
|
28
|
+
|
|
29
|
+
openai_path = File.expand_path('../extensions-ai/lex-llm-openai', __dir__)
|
|
30
|
+
if Dir.exist?(openai_path)
|
|
31
|
+
gem 'lex-llm-openai', path: openai_path
|
|
32
|
+
else
|
|
33
|
+
gem 'lex-llm-openai', '>= 0.6.6'
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
bedrock_path = File.expand_path('.ci-dependencies/lex-llm-bedrock', __dir__)
|
|
37
|
+
bedrock_path = File.expand_path('../extensions-ai/lex-llm-bedrock', __dir__) unless Dir.exist?(bedrock_path)
|
|
38
|
+
if Dir.exist?(bedrock_path)
|
|
39
|
+
gem 'lex-llm-bedrock', path: bedrock_path
|
|
40
|
+
else
|
|
41
|
+
gem 'lex-llm-bedrock', '>= 0.5.9'
|
|
42
|
+
end
|
|
43
|
+
|
|
17
44
|
%w[
|
|
18
45
|
lex-llm-ollama
|
|
19
|
-
lex-llm-vllm
|
|
20
46
|
lex-llm-anthropic
|
|
21
|
-
lex-llm-openai
|
|
22
47
|
lex-llm-gemini
|
|
23
48
|
lex-llm-mlx
|
|
24
|
-
lex-llm-bedrock
|
|
25
49
|
lex-llm-azure-foundry
|
|
26
50
|
lex-llm-vertex
|
|
27
51
|
].each do |provider_gem|
|
data/README.md
CHANGED
|
@@ -12,7 +12,7 @@
|
|
|
12
12
|
<p align="center">
|
|
13
13
|
<img alt="License" src="https://img.shields.io/badge/license-Apache--2.0-blue.svg">
|
|
14
14
|
<img alt="Ruby" src="https://img.shields.io/badge/ruby-3.4%2B-CC342D.svg">
|
|
15
|
-
<img alt="Version" src="https://img.shields.io/badge/version-0.
|
|
15
|
+
<img alt="Version" src="https://img.shields.io/badge/version-0.16.20-informational.svg">
|
|
16
16
|
<img alt="Tests" src="https://img.shields.io/badge/tests-3200%2B%20examples%20·%200%20failures-success.svg">
|
|
17
17
|
<img alt="RuboCop" src="https://img.shields.io/badge/rubocop-0%20offenses-success.svg">
|
|
18
18
|
</p>
|
|
@@ -61,6 +61,8 @@ The result: **any client → any provider, in every direction.** A Claude Code s
|
|
|
61
61
|
|
|
62
62
|
To the **client**, Legion looks like a server-side execution surface: LegionIO-resolved tools run *server-side* and appear in canonical server-tool shapes — the client never sees a pending action for a tool Legion already executed. To the **provider**, Legion looks like the client: the same tool-use/tool-result exchange shows up in the next turn in that provider's wire format, so the model always knows what happened. This "always translate, never passthrough" contract is verified on every test run by an in-process matrix harness that replays the full client × scenario matrix in milliseconds.
|
|
63
63
|
|
|
64
|
+
Local tools use the MCP SDK adapter in-process. `/v1/responses` exposes these executions as documented `mcp_call` items with arguments, results, and streaming lifecycle events. `/v1/messages` retains its server-tool blocks; `/v1/chat/completions` includes a visible call/result transcript in assistant content. Tool execution makes no HTTP request back into the daemon.
|
|
65
|
+
|
|
64
66
|
### Context curation that actually works
|
|
65
67
|
|
|
66
68
|
Long agentic sessions die by a thousand tokens: every turn re-sends the entire history, thinking blocks, and stale tool output, and context grows without bound. Legion's **Curator** (`Legion::LLM::Context::Curator`) runs asynchronously after each turn and keeps the payload bounded — without losing the information the model needs to stay coherent.
|
|
@@ -164,6 +166,61 @@ response = client.messages.create(
|
|
|
164
166
|
|
|
165
167
|
Requests flow through the full Inference pipeline — routing, metering, audit, quality checks — then the response is translated back into the caller's expected format.
|
|
166
168
|
|
|
169
|
+
### Image inputs
|
|
170
|
+
|
|
171
|
+
Anthropic `image` source envelopes, OpenAI Chat `image_url` parts, and Responses
|
|
172
|
+
`input_image` parts normalize to canonical image blocks. Supported sources are
|
|
173
|
+
absolute HTTP(S) URLs and nonempty base64 data with an explicit image MIME type.
|
|
174
|
+
MIME types are lowercased and `image/jpg` becomes `image/jpeg`. ASCII whitespace
|
|
175
|
+
and omitted base64 padding are normalized; invalid encodings still return 400.
|
|
176
|
+
Image formats such as HEIC remain available to providers that support them;
|
|
177
|
+
the selected provider determines format support.
|
|
178
|
+
|
|
179
|
+
Mixed text/image order is preserved. Anthropic `tool_result` images retain their
|
|
180
|
+
tool-call association. Image `detail` and `cache_control` survive the canonical
|
|
181
|
+
inference boundary; their provider-wire rendering belongs to the selected
|
|
182
|
+
`lex-llm-*` adapter. Image requests require a vision-capable routing candidate.
|
|
183
|
+
|
|
184
|
+
Malformed sources, provider-specific `file_id` references, Responses
|
|
185
|
+
`image_generation_call` history, and unsupported content block types are rejected with HTTP 400
|
|
186
|
+
(`invalid_request_error`) before an event stream opens. They are not silently
|
|
187
|
+
converted to empty images or discarded.
|
|
188
|
+
|
|
189
|
+
### Canonical execution and routing ownership
|
|
190
|
+
|
|
191
|
+
Equivalent text fragments preserve their exact whitespace across all three client
|
|
192
|
+
dialects. Plain text converges to one string; annotated text and mixed media retain
|
|
193
|
+
their ordered canonical blocks.
|
|
194
|
+
|
|
195
|
+
Text-block citations and metadata survive ingress without changing the original
|
|
196
|
+
client history. System instructions use the canonical string contract, including
|
|
197
|
+
cached OpenAI Chat system text. History reduction applies leading-thinking removal
|
|
198
|
+
and `tools.result_max_dispatch_chars` to both strings and annotated text blocks;
|
|
199
|
+
surviving blocks retain their metadata and media remain in order. The tool-result
|
|
200
|
+
cap is shared across text blocks in a message, not applied separately to each block.
|
|
201
|
+
|
|
202
|
+
`Router.new(request:, operation:, body_model:, **)` creates one Router for the
|
|
203
|
+
logical request. Its included `Routing::Requirements` module initializes required
|
|
204
|
+
capabilities, input/output bounds, modality, explicit pins, the body-model hint
|
|
205
|
+
decision and the attempt limit once during construction. The shared
|
|
206
|
+
`Legion::Extensions::Llm::Routing::RequestRequirements` contract validates and
|
|
207
|
+
freezes those values before they become Router instance fields. Callers use direct
|
|
208
|
+
readers such as `router.maximum_attempts` and `router.required_capabilities`; there
|
|
209
|
+
is no standalone requirements builder or nested requirements API.
|
|
210
|
+
|
|
211
|
+
Canonical tool history requires tool capability even when no new tool definitions
|
|
212
|
+
are supplied. Request policy uses the settings generation captured at ingress.
|
|
213
|
+
Each lane decision reads one immutable live Inventory snapshot; filters compare
|
|
214
|
+
the Router's request facts with published evidence, ranking consumes stored weights,
|
|
215
|
+
and dispatch preserves the resulting exact Selection. Retries retain the same
|
|
216
|
+
Router, immutable facts and request-local attempt state.
|
|
217
|
+
|
|
218
|
+
Fleet serialization preserves canonical media source enums through the shared
|
|
219
|
+
rehydration boundary. Signed binding/media tampering and Inventory mismatches
|
|
220
|
+
raise before provider execution. Boundary regressions exercise real provider
|
|
221
|
+
callables and signed worker execution with only external HTTP/key provisioning
|
|
222
|
+
substituted; they do not certify live model-service availability.
|
|
223
|
+
|
|
167
224
|
### Streaming
|
|
168
225
|
|
|
169
226
|
Both formats supported with correct SSE shapes:
|
|
@@ -171,6 +228,22 @@ Both formats supported with correct SSE shapes:
|
|
|
171
228
|
- **Anthropic**: Typed events — `message_start`, `content_block_start`, `content_block_delta`, `content_block_stop`, `message_delta`, `message_stop`
|
|
172
229
|
- **Native**: `/api/llm/inference` streams `text-delta`, optional `thinking-delta` events when `include_thinking: true`, tool lifecycle events, and a final `done` event. Structured provider content blocks are flattened to plain text in both streaming and non-streaming native responses so `content` remains a string for daemon clients.
|
|
173
230
|
|
|
231
|
+
Server-executed tool calls and results reach the shared stream observer as typed
|
|
232
|
+
Canonical tool calls when execution starts and completes. Responses and Anthropic
|
|
233
|
+
emit their corresponding lifecycle events before any subsequent model round
|
|
234
|
+
finishes; finalization does not replay already delivered events. Client-owned
|
|
235
|
+
calls never receive fabricated server results.
|
|
236
|
+
|
|
237
|
+
Tool-call fragments retain their authoritative IDs, names and incremental JSON
|
|
238
|
+
arguments. Provider-index correlation resets for each completion. An explicit
|
|
239
|
+
named tool choice remains enforced until the matching server call completes,
|
|
240
|
+
then continuation uses automatic choice without mutating the original request.
|
|
241
|
+
Typed Hash arguments are replaceable snapshots: their final value is emitted at
|
|
242
|
+
tool-call completion. Raw JSON argument fragments continue streaming incrementally;
|
|
243
|
+
an intermediate snapshot cannot prematurely close the client's argument JSON.
|
|
244
|
+
Anthropic tool blocks start with an empty `input` object; their arguments arrive
|
|
245
|
+
through `input_json_delta`, never through a premature opening snapshot.
|
|
246
|
+
|
|
174
247
|
### API Authentication
|
|
175
248
|
|
|
176
249
|
Config-driven via `settings[:llm][:api][:auth]`. Disabled by default for local dev and lite mode.
|
|
@@ -357,12 +430,33 @@ result = Legion::LLM.chat(
|
|
|
357
430
|
|
|
358
431
|
```ruby
|
|
359
432
|
embedding = Legion::LLM.embed("some text to embed")
|
|
360
|
-
embedding
|
|
433
|
+
embedding[:vector] # -> Array of numbers
|
|
434
|
+
embedding[:embedding_provenance][:space_id] # -> Stable identity for comparison
|
|
361
435
|
|
|
362
436
|
# Specific model
|
|
363
437
|
embedding = Legion::LLM.embed("text", model: "text-embedding-3-small")
|
|
364
438
|
```
|
|
365
439
|
|
|
440
|
+
Fresh, batch, and cached Ruby results include `embedding_provenance` with the
|
|
441
|
+
routed provider/model, selected instance, authoritative model revision when
|
|
442
|
+
available, actual dimensions, document/query prefix pair, normalization, task,
|
|
443
|
+
and `space_id`. Document and query tasks share a space when their prefix policy
|
|
444
|
+
matches. Instances share a space only when they publish matching revision
|
|
445
|
+
evidence; unknown revisions remain isolated by instance. Persist this provenance
|
|
446
|
+
with vectors and compare only compatible spaces.
|
|
447
|
+
|
|
448
|
+
The embedding path validates every vector and chunk before returning a result.
|
|
449
|
+
Malformed output is retried through the existing Router policy rather than
|
|
450
|
+
silently dropping chunks. Legacy cache entries without provenance and entries
|
|
451
|
+
with incompatible provenance or invalid vectors are regenerated.
|
|
452
|
+
|
|
453
|
+
Embedding metering counts served request workload on both cache misses and cache
|
|
454
|
+
hits. A cache hit preserves the cached input-token count in `input_tokens` and
|
|
455
|
+
`total_tokens`, while emitting `cache_hit: true`, `output_tokens: 0`, and
|
|
456
|
+
`cost_usd: 0`. Ledger can use those tokens to calculate counterfactual spend and
|
|
457
|
+
framework savings without charging for another provider execution. Missing
|
|
458
|
+
provider usage remains zero; embedding dimensions are not counted as output tokens.
|
|
459
|
+
|
|
366
460
|
### Tool Use
|
|
367
461
|
|
|
368
462
|
Define tools as native tool definitions or registered Legion tool classes. The inference executor forwards tool definitions to native providers and dispatches tool calls through `Inference::ToolDispatcher`:
|
|
@@ -593,6 +687,36 @@ Legion::LLM.chat(
|
|
|
593
687
|
|
|
594
688
|
System callers (type: `:system`) derive the `:system` profile, which skips governance steps to prevent recursion.
|
|
595
689
|
|
|
690
|
+
### Prompt audit capture
|
|
691
|
+
|
|
692
|
+
Prompt audit capture is a projection, not a replacement for inference history:
|
|
693
|
+
|
|
694
|
+
- `llm.compliance.audit_max_messages` retains the newest 20 messages by default.
|
|
695
|
+
- `llm.compliance.audit_max_binary_data_bytes` replaces individual binary values
|
|
696
|
+
larger than 131,072 bytes with `binary_fingerprint` metadata: byte count,
|
|
697
|
+
SHA256, MIME type, and original source type. These fingerprints are not
|
|
698
|
+
reusable base64 image data.
|
|
699
|
+
- `transport.max_payload_bytes` also bounds the complete attributed prompt
|
|
700
|
+
event and its encoded/encrypted body. Aggregate overflow removes remaining
|
|
701
|
+
binary content first, fingerprints oversized message fields and bulky event
|
|
702
|
+
fields, then omits oldest messages if necessary. Message roles and tool-result
|
|
703
|
+
associations survive field compaction. Essential identifiers, routing, and
|
|
704
|
+
usage remain; an envelope that still cannot fit is dropped and logged.
|
|
705
|
+
|
|
706
|
+
The emitted `capture` object reports `full` versus `projected` mode, message
|
|
707
|
+
counts, binary redaction counts/bytes, omitted-field count, and reasons.
|
|
708
|
+
`request_content_hash` identifies the full original semantic message history,
|
|
709
|
+
including media and tool state, independently of projection or generated
|
|
710
|
+
message IDs. The separate embedding/RAG content hash is unchanged.
|
|
711
|
+
|
|
712
|
+
The post-response timeline records `published`, `spooled`, `disabled`, or `dropped`.
|
|
713
|
+
`spooled` means transport retained the event for later delivery; it does not
|
|
714
|
+
claim broker acceptance or successful delivery.
|
|
715
|
+
GAIA observes the constructed local audit event even if delivery is disabled
|
|
716
|
+
or dropped; additional transport-only compaction can make that local projection
|
|
717
|
+
differ from the transmitted one. Consumers should use the transmitted `capture`
|
|
718
|
+
metadata rather than assuming every event contains the full request.
|
|
719
|
+
|
|
596
720
|
### Routing
|
|
597
721
|
|
|
598
722
|
legion-llm includes a dynamic weighted routing engine that dispatches requests across local, fleet, OpenAI-compatible, cloud, and frontier tiers based on caller intent, priority rules, time schedules, cost multipliers, and real-time provider health. Routing is enabled by default; set `routing.enabled: false` to bypass routing and call the configured provider directly.
|
|
@@ -1017,12 +1141,19 @@ bundle exec rubocop -A
|
|
|
1017
1141
|
| `legion-json` | Legion JSON serialization |
|
|
1018
1142
|
| `legion-logging` | Logging |
|
|
1019
1143
|
| `legion-settings` | Configuration defaults and file overrides |
|
|
1020
|
-
| `legion-transport` (>= 1.4.
|
|
1144
|
+
| `legion-transport` (>= 1.4.31) | Encoded-payload validation and AMQP transport for fleet dispatch, metering, and audit |
|
|
1021
1145
|
| `lex-knowledge` | Optional knowledge chunking integration when loaded |
|
|
1022
|
-
| `lex-llm` (>= 0.
|
|
1146
|
+
| `lex-llm` (>= 0.8.6) | Shared Canonical types, immutable `Routing::RequestRequirements`, Inventory/Selection contracts, and exact fleet execution/rehydration |
|
|
1023
1147
|
| `pdf-reader` | PDF extraction support |
|
|
1024
1148
|
| `tzinfo` (>= 2.0) | IANA timezone conversion for schedule windows |
|
|
1025
1149
|
|
|
1150
|
+
For the coordinated LLM release, publish shared `lex-llm` first, the provider
|
|
1151
|
+
gems next, and `legion-llm` last. The current targets are `lex-llm 0.8.8`,
|
|
1152
|
+
`lex-llm-vllm 0.4.9`, `lex-llm-anthropic 0.3.8`, and `lex-llm-bedrock 0.5.10`.
|
|
1153
|
+
These companion versions are published. All CI runs resolve published gems;
|
|
1154
|
+
temporary companion commit pins have been removed. Release depends on CI passing
|
|
1155
|
+
against those published dependencies.
|
|
1156
|
+
|
|
1026
1157
|
## License
|
|
1027
1158
|
|
|
1028
1159
|
Apache-2.0
|