lex-llm-ledger 0.4.2 → 0.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (25) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +46 -0
  3. data/lib/legion/extensions/llm/ledger/actors/escalations.rb +34 -0
  4. data/lib/legion/extensions/llm/ledger/actors/reconciliation.rb +48 -0
  5. data/lib/legion/extensions/llm/ledger/actors/retention_purge.rb +47 -0
  6. data/lib/legion/extensions/llm/ledger/actors/skills.rb +34 -0
  7. data/lib/legion/extensions/llm/ledger/backfill/legacy_llm_records.rb +8 -2
  8. data/lib/legion/extensions/llm/ledger/helpers/caller_identity.rb +16 -2
  9. data/lib/legion/extensions/llm/ledger/helpers/retention.rb +1 -1
  10. data/lib/legion/extensions/llm/ledger/runners/escalations.rb +115 -0
  11. data/lib/legion/extensions/llm/ledger/runners/metering.rb +14 -12
  12. data/lib/legion/extensions/llm/ledger/runners/prompts.rb +19 -4
  13. data/lib/legion/extensions/llm/ledger/runners/provider_stats.rb +1 -1
  14. data/lib/legion/extensions/llm/ledger/runners/reconciliation.rb +96 -0
  15. data/lib/legion/extensions/llm/ledger/runners/retention_purge.rb +62 -0
  16. data/lib/legion/extensions/llm/ledger/runners/skills.rb +99 -0
  17. data/lib/legion/extensions/llm/ledger/runners/tools.rb +73 -68
  18. data/lib/legion/extensions/llm/ledger/transport/exchanges/escalation.rb +23 -0
  19. data/lib/legion/extensions/llm/ledger/transport/queues/audit_escalations.rb +23 -0
  20. data/lib/legion/extensions/llm/ledger/transport/queues/audit_skills.rb +23 -0
  21. data/lib/legion/extensions/llm/ledger/transport/transport.rb +11 -0
  22. data/lib/legion/extensions/llm/ledger/version.rb +1 -1
  23. data/lib/legion/extensions/llm/ledger/writers/official_record_writer.rb +71 -7
  24. data/lib/legion/extensions/llm/ledger.rb +15 -1
  25. metadata +12 -1
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 9d96639d6f6bf123c338c0df58c4e25d1375d0102873318197f6297eeda70a80
4
- data.tar.gz: 1ebcbc88558e3783595e3c07c13eb0f9d0f3fbb82a2c096cb52752592c469d2a
3
+ metadata.gz: 70deb6cfb3814da49ebbc893a2025e06ddcf96ee7789a65f794fa66910502f41
4
+ data.tar.gz: e0837642064bf0c1ee6bf480704d297f961144f5f6cfb0814b88d8de438bf0f6
5
5
  SHA512:
6
- metadata.gz: 670f7c45555f557aae91ba858831f1ce40e74eaf92a5390a965e2d5e6487a6b6deb9700527a6098bd8454b2085fbe1de567195d4a7f54b8f910921d6dde9535f
7
- data.tar.gz: a72b7dac0f47bf49ca637f060698899d8eb2fc1d7436e2271afbddb8ef5684a00e5b75ae264ac6d635bc9ac58b9e79fab84070b4b51878324615d939942a9ef7
6
+ metadata.gz: '024169e420f778950b01bd074b03b1e63ccc642cb6c0148a14046dc91a74426c8a4d202a70fb33b6a669fbe3248fba62940e72a3a63ba1d776cef0044e0b9e36'
7
+ data.tar.gz: 207e66a936b63bf8c52437d2dc1d92931afdcb12caea03976b4e293bf8e67d43f963e62e9ba336c1e80789b3d60ad960b56eb6b85081f3146509f84c760aaf6f
data/CHANGELOG.md CHANGED
@@ -1,5 +1,51 @@
1
1
  # Changelog
2
2
 
3
+ ## [0.6.0] - 2026-05-31
4
+
5
+ ### Added
6
+ - **Skills audit actor** — new queue `llm.audit.skills`, actor, and runner consuming `audit.skill.#` events from the `llm.audit` exchange. Creates `llm_skill_events` records.
7
+ - **Escalation audit actor** — new exchange `llm.escalation` binding, queue `llm.audit.escalations`, actor, and runner. Creates `llm_escalation_events` records.
8
+ - **Retention purge actor** — actively deletes expired records (hourly, batched). Enforces `session_only` and PHI TTL retention policies that were previously passive.
9
+ - **Reconciliation actor** — links orphaned tool calls (null `response_id`) and metering requests (null `latest_message_id`) within a 5-minute lookback window every 2 minutes.
10
+ - **Migration 013** — `parent_request_id` FK on requests, `schema_version` on all official tables, `pii_types_json` and `jurisdictions_json` on conversations.
11
+ - **Migration 014** — creates `llm_skill_events` and `llm_escalation_events` tables.
12
+ - **PHI field-level encryption** — `request_json`, `response_json`, `response_thinking_json` encrypted at rest via `Legion::Crypt` when `contains_phi: true`.
13
+
14
+ ### Fixed
15
+ - **LEDGER-01**: `session_only` retention maps to 0 days (immediate expiry) instead of `nil`, distinguishing it from `permanent`.
16
+ - **IDENTITY-05**: `CallerIdentity.normalize` recognizes transport identity headers (`x-legion-identity-canonical-name`, `x-legion-identity-kind`, `x-legion-identity-db-*`).
17
+ - **IDENTITY-06**: `OfficialRecordWriter.caller_identity_refs` falls back to pre-resolved AMQP header IDs.
18
+ - **GAP-06**: `ProviderStats#health_report` uses `recorded_at` instead of `inserted_at` for 24h window.
19
+ - **GAP-10**: Backfill `metering_payload` maps `provider_instance` from correct column.
20
+ - **GAP-11**: Backfill `registry_reason` extracts from metadata JSON instead of storing raw blob.
21
+ - Dead `Retention.resolve` call removed from tools runner.
22
+
23
+ ### Changed
24
+ - **Jurisdictions** stored as JSON array instead of comma-joined string.
25
+ - **Classification level** validated against controlled vocabulary (`public`/`internal`/`confidential`/`restricted`), defaults to `internal`.
26
+ - **Schema version** written to all official table inserts (constant `SCHEMA_VERSION = 13`).
27
+ - All CallerIdentity-resolved payloads now pass `__header_principal_id`/`__header_identity_id` for direct FK resolution.
28
+
29
+ ### Requires
30
+ - legion-data >= 1.8.9 (migrations 013-014)
31
+
32
+ ## [0.5.0] - 2026-05-26
33
+
34
+ ### Changed
35
+ - Tool audit writes no longer dead-letter when the parent response row is missing. The runner retries up to 3 times (1s delay each, configurable via `tool_write` settings), then inserts with a NULL `message_inference_response_id` instead of raising `UnrecoverableMessageError`.
36
+ - Removed `ResponseNotReady` exception class — tool calls are always persisted now.
37
+ - Populate `conversation_id` FK on `llm_tool_calls` from the message payload/headers, providing conversation-level traceability even when the response FK is NULL.
38
+ - Retry configuration moved to `default_settings[:tool_write]` (`response_retry_attempts`, `response_retry_delay`) — tunable at runtime without code changes.
39
+
40
+ ### Requires
41
+ - legion-data >= 1.8.9 (migrations 116-117)
42
+
43
+ ## [0.4.3] - 2026-05-22
44
+
45
+ ### Fixed
46
+ - Persist `llm.registry.availability` publisher identity from current transport headers into `llm_registry_availability_records`, including best-effort `identity_principal_id` and `identity_id` from DB id headers.
47
+ - Preserve legacy identity header and body fallbacks for registry availability records when current transport identity headers are absent.
48
+
3
49
  ## [0.4.2] - 2026-05-22
4
50
 
5
51
  ### Fixed
@@ -0,0 +1,34 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'legion/extensions/actors/subscription'
4
+ require_relative '../helpers/subscription_actor'
5
+
6
+ module Legion
7
+ module Extensions
8
+ module Llm
9
+ module Ledger
10
+ module Actor
11
+ class Escalations < Legion::Extensions::Actors::Subscription
12
+ include Helpers::SubscriptionActor
13
+
14
+ prefetch 1
15
+
16
+ def runner_class = Legion::Extensions::Llm::Ledger::Runners::Escalations
17
+
18
+ def runner_function
19
+ 'write_escalation_record'
20
+ end
21
+
22
+ def use_runner?
23
+ false
24
+ end
25
+
26
+ def queue
27
+ Legion::Extensions::Llm::Ledger::Transport::Queues::AuditEscalations
28
+ end
29
+ end
30
+ end
31
+ end
32
+ end
33
+ end
34
+ end
@@ -0,0 +1,48 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Legion
4
+ module Extensions
5
+ module Llm
6
+ module Ledger
7
+ module Actor
8
+ class Reconciliation < Legion::Extensions::Actors::Every # rubocop:disable Legion/Extension/EveryActorRequiresTime
9
+ def runner_class
10
+ 'Legion::Extensions::Llm::Ledger::Runners::Reconciliation'
11
+ end
12
+
13
+ def runner_function
14
+ 'link_orphaned_tool_calls'
15
+ end
16
+
17
+ def time
18
+ 120
19
+ end
20
+
21
+ def run
22
+ Runners::Reconciliation.link_orphaned_tool_calls
23
+ Runners::Reconciliation.link_metering_messages
24
+ rescue StandardError => e
25
+ handle_exception(e, level: :warn, handled: true, operation: 'reconciliation')
26
+ end
27
+
28
+ def run_now?
29
+ false
30
+ end
31
+
32
+ def use_runner?
33
+ false
34
+ end
35
+
36
+ def check_subtask?
37
+ false
38
+ end
39
+
40
+ def generate_task?
41
+ false
42
+ end
43
+ end
44
+ end
45
+ end
46
+ end
47
+ end
48
+ end
@@ -0,0 +1,47 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Legion
4
+ module Extensions
5
+ module Llm
6
+ module Ledger
7
+ module Actor
8
+ class RetentionPurge < Legion::Extensions::Actors::Every # rubocop:disable Legion/Extension/EveryActorRequiresTime
9
+ def runner_class
10
+ 'Legion::Extensions::Llm::Ledger::Runners::RetentionPurge'
11
+ end
12
+
13
+ def runner_function
14
+ 'purge_expired'
15
+ end
16
+
17
+ def time
18
+ 3600
19
+ end
20
+
21
+ def run
22
+ Runners::RetentionPurge.purge_expired
23
+ rescue StandardError => e
24
+ handle_exception(e, level: :warn, handled: true, operation: 'retention_purge')
25
+ end
26
+
27
+ def run_now?
28
+ false
29
+ end
30
+
31
+ def use_runner?
32
+ false
33
+ end
34
+
35
+ def check_subtask?
36
+ false
37
+ end
38
+
39
+ def generate_task?
40
+ false
41
+ end
42
+ end
43
+ end
44
+ end
45
+ end
46
+ end
47
+ end
@@ -0,0 +1,34 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'legion/extensions/actors/subscription'
4
+ require_relative '../helpers/subscription_actor'
5
+
6
+ module Legion
7
+ module Extensions
8
+ module Llm
9
+ module Ledger
10
+ module Actor
11
+ class Skills < Legion::Extensions::Actors::Subscription
12
+ include Helpers::SubscriptionActor
13
+
14
+ prefetch 1
15
+
16
+ def runner_class = Legion::Extensions::Llm::Ledger::Runners::Skills
17
+
18
+ def runner_function
19
+ 'write_skill_record'
20
+ end
21
+
22
+ def use_runner?
23
+ false
24
+ end
25
+
26
+ def queue
27
+ Legion::Extensions::Llm::Ledger::Transport::Queues::AuditSkills
28
+ end
29
+ end
30
+ end
31
+ end
32
+ end
33
+ end
34
+ end
@@ -110,7 +110,8 @@ module Legion
110
110
  exchange_id: row[:exchange_id],
111
111
  operation: row[:request_type],
112
112
  provider: row[:provider],
113
- provider_instance: row[:worker_id],
113
+ provider_instance: row[:provider_instance],
114
+ worker_id: row[:worker_id],
114
115
  model_id: row[:model_id],
115
116
  tier: row[:tier],
116
117
  input_tokens: row[:input_tokens],
@@ -161,7 +162,7 @@ module Legion
161
162
  model_key: row[:model_id],
162
163
  event_type: row[:event_type],
163
164
  status: registry_status(row),
164
- reason: row[:metadata_json],
165
+ reason: registry_reason(row),
165
166
  recorded_at: row[:occurred_at],
166
167
  inserted_at: Time.now.utc
167
168
  }, operation: 'legacy_llm_backfill.registry_event')
@@ -197,6 +198,11 @@ module Legion
197
198
  health[:status] || health['status'] || row[:event_type] || 'unknown'
198
199
  end
199
200
 
201
+ def registry_reason(row)
202
+ metadata = json_load(row[:metadata_json])
203
+ metadata[:reason] || metadata['reason'] || metadata[:message] || metadata['message'] || row[:event_type]
204
+ end
205
+
200
206
  def table_present?(table)
201
207
  db.table_exists?(table)
202
208
  end
@@ -18,12 +18,16 @@ module Legion
18
18
  extension = hash_value(caller_hash, :extension)
19
19
  type = first_present(
20
20
  hash_value(identity_hash, :type),
21
+ hash_value(identity_hash, :kind),
22
+ header_value(headers, 'x-legion-identity-kind'),
21
23
  header_value(headers, 'x-legion-caller-type'),
22
24
  hash_value(caller, :type),
25
+ hash_value(caller, :kind),
23
26
  extension && 'extension'
24
27
  )
25
28
 
26
29
  raw_identity = first_present(
30
+ header_value(headers, 'x-legion-identity-canonical-name'),
27
31
  hash_value(identity_hash, :id),
28
32
  hash_value(identity_hash, :canonical_name),
29
33
  hash_value(identity_hash, :identity),
@@ -38,8 +42,10 @@ module Legion
38
42
  )
39
43
 
40
44
  {
41
- identity: normalize_identity_value(raw_identity, type),
42
- type: type
45
+ identity: normalize_identity_value(raw_identity, type),
46
+ type: type,
47
+ principal_id: integer_header(headers, 'x-legion-identity-db-principal-id'),
48
+ identity_id: integer_header(headers, 'x-legion-identity-db-identity-id')
43
49
  }.compact
44
50
  end
45
51
 
@@ -71,6 +77,14 @@ module Legion
71
77
  headers[key] || headers[key.to_sym]
72
78
  end
73
79
 
80
+ def integer_header(headers, key)
81
+ raw = header_value(headers, key)
82
+ return nil if raw.nil?
83
+
84
+ int = raw.to_i
85
+ int.positive? ? int : nil
86
+ end
87
+
74
88
  def present?(value)
75
89
  !value.nil? && !(value.respond_to?(:empty?) && value.empty?)
76
90
  end
@@ -13,7 +13,7 @@ module Legion
13
13
  PHI_TTL_DEFAULT_DAYS = 30
14
14
 
15
15
  RETENTION_MAP = {
16
- 'session_only' => nil,
16
+ 'session_only' => 0,
17
17
  'days_30' => 30,
18
18
  'days_90' => 90,
19
19
  'permanent' => nil
@@ -0,0 +1,115 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'digest'
4
+ require 'securerandom'
5
+ require_relative '../helpers/caller_identity'
6
+ require_relative '../helpers/json'
7
+ require_relative '../helpers/persistence_logging'
8
+
9
+ module Legion
10
+ module Extensions
11
+ module Llm
12
+ module Ledger
13
+ module Runners
14
+ module Escalations
15
+ extend self
16
+
17
+ def write_escalation_record(payload = nil, metadata = {}, **message)
18
+ payload, metadata = normalize_runner_args(payload, metadata, message)
19
+ headers = Helpers::SubscriptionMessage.extract_headers(payload, metadata)
20
+ props = metadata[:properties] || {}
21
+
22
+ body = payload.is_a?(Hash) ? payload : Helpers::Decryption.decrypt_if_needed(payload, metadata)
23
+
24
+ db = ::Legion::Data.connection
25
+ record = build_escalation_record(db, body, props, headers)
26
+
27
+ Helpers::PersistenceLogging.insert_row(
28
+ db, :llm_escalation_events, record,
29
+ operation: 'write_escalation_record'
30
+ )
31
+ { result: :ok }
32
+ rescue Sequel::UniqueConstraintViolation => e
33
+ log.warn("write_escalation_record duplicate insert ignored: #{e.message}")
34
+ { result: :duplicate }
35
+ rescue Helpers::DecryptionUnavailable => e
36
+ handle_exception(e, level: :warn, handled: true, operation: 'write_escalation_record.decrypt')
37
+ raise
38
+ rescue Helpers::DecryptionFailed => e
39
+ handle_exception(e, level: :error, handled: true, operation: 'write_escalation_record.decrypt')
40
+ raise
41
+ rescue StandardError => e
42
+ handle_exception(e, level: :error, handled: true, operation: 'write_escalation_record')
43
+ { result: :error, error: e.message }
44
+ end
45
+
46
+ private
47
+
48
+ def normalize_runner_args(payload, metadata, message)
49
+ Helpers::SubscriptionMessage.runner_args(payload, metadata, message)
50
+ end
51
+
52
+ def build_escalation_record(db, body, props, headers) # rubocop:disable Metrics/AbcSize, Metrics/CyclomaticComplexity, Metrics/PerceivedComplexity
53
+ history = Array(body[:history])
54
+ identity = Helpers::CallerIdentity.normalize(
55
+ caller_raw: body[:caller], identity: body[:identity], headers: headers
56
+ )
57
+
58
+ first_attempt = history.first || {}
59
+ last_attempt = history.last || {}
60
+
61
+ {
62
+ uuid: stable_uuid(props[:message_id] || body[:event_id] || SecureRandom.uuid),
63
+ conversation_id: resolve_conversation_id(db, body, headers),
64
+ request_ref: body[:request_id] || props[:correlation_id],
65
+ from_provider: first_attempt[:provider] || body[:from_provider],
66
+ from_instance: first_attempt[:instance] || body[:from_instance],
67
+ from_model: first_attempt[:model] || body[:from_model],
68
+ to_provider: last_attempt[:provider] || body[:to_provider],
69
+ to_instance: last_attempt[:instance] || body[:to_instance],
70
+ to_model: last_attempt[:model] || body[:to_model],
71
+ reason: body[:reason] || (first_attempt[:outcome] == 'failure' ? 'provider_failover' : nil),
72
+ error_category: body[:error_category] || extract_error_category(first_attempt),
73
+ attempt_no: history.size || (body[:attempt_no] || 1),
74
+ latency_ms: history.sum { |a| (a[:duration_ms] || 0).to_i } || body[:latency_ms].to_i,
75
+ identity_canonical_name: identity[:identity],
76
+ identity_principal_id: identity[:principal_id],
77
+ identity_id: identity[:identity_id],
78
+ history_json: history.any? ? Helpers::Json.dump(history) : nil,
79
+ outcome: body[:outcome]&.to_s,
80
+ total_attempts: body[:attempts] ? body[:attempts].to_i : history.size,
81
+ recorded_at: body[:recorded_at] || body[:timestamp] || Time.now.utc,
82
+ inserted_at: Time.now.utc
83
+ }.compact
84
+ end
85
+
86
+ def resolve_conversation_id(db, body, headers)
87
+ conv_ref = body[:conversation_id] || headers['x-legion-llm-conversation-id']
88
+ return nil unless conv_ref # rubocop:disable Legion/Extension/RunnerReturnHash
89
+
90
+ conv = db[:llm_conversations].where(uuid: stable_uuid(conv_ref)).first ||
91
+ db[:llm_conversations].where(uuid: conv_ref).first
92
+ conv&.[](:id)
93
+ end
94
+
95
+ def extract_error_category(attempt)
96
+ failures = Array(attempt[:failures])
97
+ failures.first.is_a?(Hash) ? failures.first[:category].to_s : nil
98
+ end
99
+
100
+ def stable_uuid(value)
101
+ raw = value.to_s
102
+ return raw if raw.length <= 36 # rubocop:disable Legion/Extension/RunnerReturnHash
103
+
104
+ hex = Digest::SHA256.hexdigest(raw)[0, 32]
105
+ "#{hex[0, 8]}-#{hex[8, 4]}-#{hex[12, 4]}-#{hex[16, 4]}-#{hex[20, 12]}"
106
+ end
107
+
108
+ include Legion::Extensions::Helpers::Lex if Legion::Extensions.const_defined?(:Helpers, false) &&
109
+ Legion::Extensions::Helpers.const_defined?(:Lex, false)
110
+ end
111
+ end
112
+ end
113
+ end
114
+ end
115
+ end
@@ -92,18 +92,20 @@ module Legion
92
92
  caller_raw: payload[:caller], identity: payload[:identity], headers: headers
93
93
  )
94
94
  payload.merge(
95
- message_id: props[:message_id] || payload[:message_id] || ctx[:message_id],
96
- correlation_id: props[:correlation_id] || payload[:correlation_id],
97
- conversation_id: ctx[:conversation_id] || payload[:conversation_id],
98
- request_id: ctx[:request_id] || payload[:request_id],
99
- exchange_id: ctx[:exchange_id] || payload[:exchange_id],
100
- operation: payload[:operation] || payload[:request_type] || headers['x-legion-llm-request-type'],
101
- provider: payload[:provider] || headers['x-legion-llm-provider'],
102
- provider_instance: payload[:provider_instance] || payload[:instance],
103
- model_id: payload[:model_id] || headers['x-legion-llm-model'],
104
- tier: payload[:tier] || headers['x-legion-llm-tier'],
105
- caller_identity: identity[:identity],
106
- caller_type: identity[:type]
95
+ message_id: props[:message_id] || payload[:message_id] || ctx[:message_id],
96
+ correlation_id: props[:correlation_id] || payload[:correlation_id],
97
+ conversation_id: ctx[:conversation_id] || payload[:conversation_id],
98
+ request_id: ctx[:request_id] || payload[:request_id],
99
+ exchange_id: ctx[:exchange_id] || payload[:exchange_id],
100
+ operation: payload[:operation] || payload[:request_type] || headers['x-legion-llm-request-type'],
101
+ provider: payload[:provider] || headers['x-legion-llm-provider'],
102
+ provider_instance: payload[:provider_instance] || payload[:instance],
103
+ model_id: payload[:model_id] || headers['x-legion-llm-model'],
104
+ tier: payload[:tier] || headers['x-legion-llm-tier'],
105
+ caller_identity: identity[:identity],
106
+ caller_type: identity[:type],
107
+ __header_principal_id: identity[:principal_id],
108
+ __header_identity_id: identity[:identity_id]
107
109
  )
108
110
  end
109
111
 
@@ -10,6 +10,8 @@ module Legion
10
10
  module Ledger
11
11
  module Runners
12
12
  module Prompts
13
+ ALLOWED_CLASSIFICATION_LEVELS = %w[public internal confidential restricted].freeze
14
+
13
15
  extend self
14
16
 
15
17
  def write_prompt_record(payload = nil, metadata = {}, **message)
@@ -137,8 +139,10 @@ module Legion
137
139
  caller_raw: body[:caller], identity: body[:identity], headers: headers
138
140
  )
139
141
  {
140
- caller_identity: identity[:identity],
141
- caller_type: identity[:type]
142
+ caller_identity: identity[:identity],
143
+ caller_type: identity[:type],
144
+ __header_principal_id: identity[:principal_id],
145
+ __header_identity_id: identity[:identity_id]
142
146
  }.compact
143
147
  end
144
148
 
@@ -165,14 +169,25 @@ module Legion
165
169
  end
166
170
 
167
171
  def official_compliance_payload(body, headers, expires_at)
172
+ cls = body[:classification] || {}
168
173
  {
169
174
  retention_policy: headers['x-legion-retention'] || body[:retention_policy],
170
175
  expires_at: expires_at,
171
- contains_phi: headers['x-legion-contains-phi'] == 'true' || body.dig(:classification, :contains_phi),
172
- classification_level: body.dig(:classification, :level) || headers['x-legion-classification']
176
+ contains_phi: headers['x-legion-contains-phi'] == 'true' || cls[:contains_phi],
177
+ contains_pii: cls[:contains_pii] ? true : false,
178
+ pii_types_json: Helpers::Json.dump(Array(cls[:pii_types])),
179
+ classification_level: normalize_classification(cls[:level] || headers['x-legion-classification']),
180
+ jurisdictions: Helpers::Json.dump(Array(cls[:jurisdictions]))
173
181
  }
174
182
  end
175
183
 
184
+ def normalize_classification(level)
185
+ return 'internal' if level.nil? || level.to_s.empty? # rubocop:disable Legion/Extension/RunnerReturnHash
186
+
187
+ normalized = level.to_s.downcase
188
+ ALLOWED_CLASSIFICATION_LEVELS.include?(normalized) ? normalized : 'internal'
189
+ end
190
+
176
191
  include Legion::Extensions::Helpers::Lex if Legion::Extensions.const_defined?(:Helpers, false) &&
177
192
  Legion::Extensions::Helpers.const_defined?(:Lex, false)
178
193
  end
@@ -10,7 +10,7 @@ module Legion
10
10
 
11
11
  def health_report
12
12
  ds = official_metrics
13
- .where { Sequel[:llm_message_inference_metrics][:inserted_at] >= Time.now.utc - 86_400 }
13
+ .where { Sequel[:llm_message_inference_metrics][:recorded_at] >= Time.now.utc - 86_400 }
14
14
  .select_group(
15
15
  Sequel[:llm_message_inference_metrics][:provider],
16
16
  Sequel[:llm_message_inference_responses][:provider_instance],
@@ -0,0 +1,96 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'legion/logging'
4
+
5
+ module Legion
6
+ module Extensions
7
+ module Llm
8
+ module Ledger
9
+ module Runners
10
+ module Reconciliation
11
+ extend self
12
+ extend Legion::Logging::Helper
13
+
14
+ BATCH_SIZE = 200
15
+ LOOKBACK_SECONDS = 300
16
+
17
+ def link_orphaned_tool_calls
18
+ db = ::Legion::Data.connection
19
+ linked = 0
20
+
21
+ orphans = db[:llm_tool_calls]
22
+ .where(message_inference_response_id: nil)
23
+ .where { inserted_at >= Time.now.utc - LOOKBACK_SECONDS }
24
+ .limit(BATCH_SIZE)
25
+ .all
26
+
27
+ orphans.each do |tool_call|
28
+ response = find_response_for_tool_call(db, tool_call)
29
+ next unless response
30
+
31
+ db[:llm_tool_calls].where(id: tool_call[:id]).update(
32
+ message_inference_response_id: response[:id]
33
+ )
34
+ linked += 1
35
+ end
36
+
37
+ log.info("[ledger] reconciliation: linked #{linked} orphaned tool calls") if linked.positive?
38
+ { result: :ok, linked: linked }
39
+ rescue StandardError => e
40
+ handle_exception(e, level: :error, handled: true, operation: 'reconciliation.tool_calls')
41
+ { result: :error, error: e.message }
42
+ end
43
+
44
+ def link_metering_messages
45
+ db = ::Legion::Data.connection
46
+ linked = 0
47
+
48
+ requests_without_messages = db[:llm_message_inference_requests]
49
+ .where(latest_message_id: nil)
50
+ .where { inserted_at >= Time.now.utc - LOOKBACK_SECONDS }
51
+ .limit(BATCH_SIZE)
52
+ .all
53
+
54
+ requests_without_messages.each do |request|
55
+ next unless request[:conversation_id]
56
+
57
+ message = db[:llm_messages]
58
+ .where(conversation_id: request[:conversation_id])
59
+ .order(Sequel.desc(:seq))
60
+ .first
61
+ next unless message
62
+
63
+ db[:llm_message_inference_requests]
64
+ .where(id: request[:id])
65
+ .update(latest_message_id: message[:id])
66
+ linked += 1
67
+ end
68
+
69
+ log.info("[ledger] reconciliation: linked #{linked} metering requests to messages") if linked.positive?
70
+ { result: :ok, linked: linked }
71
+ rescue StandardError => e
72
+ handle_exception(e, level: :error, handled: true, operation: 'reconciliation.metering_messages')
73
+ { result: :error, error: e.message }
74
+ end
75
+
76
+ private
77
+
78
+ def find_response_for_tool_call(db, tool_call)
79
+ return nil unless tool_call[:conversation_id] # rubocop:disable Legion/Extension/RunnerReturnHash
80
+
81
+ request = db[:llm_message_inference_requests]
82
+ .where(conversation_id: tool_call[:conversation_id])
83
+ .order(Sequel.desc(:inserted_at))
84
+ .first
85
+ return nil unless request # rubocop:disable Legion/Extension/RunnerReturnHash
86
+
87
+ db[:llm_message_inference_responses]
88
+ .where(message_inference_request_id: request[:id])
89
+ .first
90
+ end
91
+ end
92
+ end
93
+ end
94
+ end
95
+ end
96
+ end