lex-metering 0.1.15 → 0.1.16
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +11 -0
- data/CLAUDE.md +21 -106
- data/README.md +182 -35
- data/lib/legion/extensions/metering/data/migrations/001_add_metering_records.rb +0 -5
- data/lib/legion/extensions/metering/data/migrations/003_add_metering_indexes.rb +21 -0
- data/lib/legion/extensions/metering/runners/cost_optimizer.rb +6 -2
- data/lib/legion/extensions/metering/runners/metering.rb +1 -1
- data/lib/legion/extensions/metering/version.rb +1 -1
- data/lib/legion/extensions/metering.rb +3 -3
- metadata +2 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 4b90e69ccdc11eb888b7876e51028b09f31eb1af90c1bd13cd157094206e08cf
|
|
4
|
+
data.tar.gz: cbf0e8c74af206b2473efc67ec982a9aa3ae75afe794a461035d2caa9306fc97
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 671d819df21bbd2d220fd18aba534847d3c048e2d131122c68d212938a6d63270a4161c22f49208599c19682c147cf4b331ecc05a04c7c3c0c01d105e0079d89
|
|
7
|
+
data.tar.gz: 241c993213715d59c198dcae98e1145a1d0122a95b1ef8639af85466683114f609df577f6c01dec682d490bf08cd344881008195b99afe792169d8bdb2b8c1f0
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,16 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## [0.1.16] - 2026-05-17
|
|
4
|
+
### Fixed
|
|
5
|
+
- Migration 001: remove indexes from `create_table?` block (Sequel creates indexes even when table exists, causing DuplicateTable error)
|
|
6
|
+
- New migration 003: add indexes idempotently via `add_index :col, if_not_exists: true`
|
|
7
|
+
- `routing_stats` GROUP BY query: replace `select_append` with explicit `select` projection to avoid PG::GroupingError on non-aggregate columns
|
|
8
|
+
- Set `data_required? true` so extension loader reliably discovers and runs migrations on fresh installs
|
|
9
|
+
|
|
10
|
+
## [0.1.14] - 2026-05-03
|
|
11
|
+
### Fixed
|
|
12
|
+
- Cost optimization recommendations now pass prompts directly into native `Legion::LLM.chat` dispatch instead of routing through the legacy nil-input `llm_chat` helper path.
|
|
13
|
+
|
|
3
14
|
## [0.1.13] - 2026-03-30
|
|
4
15
|
|
|
5
16
|
### Changed
|
data/CLAUDE.md
CHANGED
|
@@ -1,113 +1,28 @@
|
|
|
1
|
-
# lex-metering
|
|
1
|
+
# lex-metering
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
- **Parent**: `/Users/miverso2/rubymine/legion/extensions-core/CLAUDE.md`
|
|
5
|
-
- **Grandparent**: `/Users/miverso2/rubymine/legion/CLAUDE.md`
|
|
3
|
+
LLM cost metering for LegionIO. Records token usage per task for cost attribution and intelligent routing: input/output/thinking tokens, latency, wall-clock time, CPU time, external API calls, cost in USD, trace context, and routing reason. Supports per-worker, per-team, and aggregate queries. Hourly rollup to summary table.
|
|
6
4
|
|
|
7
|
-
##
|
|
8
|
-
|
|
9
|
-
Captures LLM token usage metrics per task for cost attribution and intelligent routing. Records input/output/thinking tokens, latency, wall-clock time, CPU time, external API call counts, cost in USD, trace context (extension, runner_function, event_type, status), and routing reason per metered call. Supports per-worker, per-team, and aggregate routing statistics queries. Hourly rollup to a summary table for reporting efficiency.
|
|
10
|
-
|
|
11
|
-
## Gem Info
|
|
12
|
-
|
|
13
|
-
- **Gem name**: `lex-metering`
|
|
14
|
-
- **Version**: `0.1.11`
|
|
15
|
-
- **Module**: `Legion::Extensions::Metering`
|
|
16
|
-
- **Ruby**: `>= 3.4`
|
|
17
|
-
- **License**: MIT
|
|
18
|
-
- **GitHub**: https://github.com/LegionIO/lex-metering
|
|
19
|
-
|
|
20
|
-
## File Structure
|
|
5
|
+
## Architecture
|
|
21
6
|
|
|
22
7
|
```
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
runners/
|
|
34
|
-
metering.rb # record, worker_costs, team_costs, routing_stats, cleanup_old_records
|
|
35
|
-
cost_optimizer.rb # analyze_costs (LLM-powered model rightsizing recommendations)
|
|
36
|
-
rollup.rb # rollup_hour, purge_raw_records
|
|
8
|
+
Legion::Extensions::Metering
|
|
9
|
+
├── Runners/
|
|
10
|
+
│ ├── Metering # record, worker_costs, team_costs, routing_stats, cleanup_old_records
|
|
11
|
+
│ ├── CostOptimizer # analyze_costs (LLM-powered model rightsizing recommendations)
|
|
12
|
+
│ └── Rollup # rollup_hour, purge_raw_records
|
|
13
|
+
├── Actors/
|
|
14
|
+
│ ├── Cleanup(86400s) # Daily: cleanup_old_records; use_runner? false
|
|
15
|
+
│ ├── CostOptimizer(604800s) # Weekly: analyze_costs; Singleton mixin
|
|
16
|
+
│ └── Rollup(3600s) # Hourly: rollup_hour; run_now? false
|
|
17
|
+
└── Data/Migrations/ # 001_add_metering_records, 002_add_trace_columns, 003_add_metering_indexes
|
|
37
18
|
```
|
|
38
19
|
|
|
39
|
-
##
|
|
40
|
-
|
|
41
|
-
| Column | Type | Description |
|
|
42
|
-
|--------|------|-------------|
|
|
43
|
-
| `worker_id` | String(36) | Digital worker ID (nullable for non-worker tasks) |
|
|
44
|
-
| `task_id` | Integer | Legion task ID |
|
|
45
|
-
| `provider` | String(100) | LLM provider (e.g., 'anthropic', 'openai', 'bedrock') |
|
|
46
|
-
| `model_id` | String(255) | Model identifier |
|
|
47
|
-
| `input_tokens` | Integer | Prompt tokens |
|
|
48
|
-
| `output_tokens` | Integer | Completion tokens |
|
|
49
|
-
| `thinking_tokens` | Integer | Thinking/reasoning tokens (Anthropic extended thinking) |
|
|
50
|
-
| `total_tokens` | Integer | Sum of all token types |
|
|
51
|
-
| `input_context_bytes` | Integer | Raw context size in bytes |
|
|
52
|
-
| `latency_ms` | Integer | LLM API round-trip time |
|
|
53
|
-
| `wall_clock_ms` | Integer | Total wall-clock time for the task |
|
|
54
|
-
| `cpu_time_ms` | Integer | CPU time consumed |
|
|
55
|
-
| `external_api_calls` | Integer | Non-LLM external API calls made |
|
|
56
|
-
| `routing_reason` | String(255) | Why this model/provider was chosen |
|
|
57
|
-
| `cost_usd` | Float | Estimated cost in USD (default: 0.0) |
|
|
58
|
-
| `status` | String(50) | Pipeline status at time of record |
|
|
59
|
-
| `event_type` | String(100) | Event type for SIEM/audit correlation |
|
|
60
|
-
| `extension` | String(255) | Calling extension name |
|
|
61
|
-
| `runner_function` | String(255) | Calling runner function name |
|
|
62
|
-
| `recorded_at` | DateTime | Timestamp (indexed) |
|
|
63
|
-
|
|
64
|
-
## Runner Methods
|
|
65
|
-
|
|
66
|
-
| Method | Parameters | Returns |
|
|
67
|
-
|--------|-----------|---------|
|
|
68
|
-
| `record` | All schema fields as kwargs | Record hash (also inserted to DB when connected) |
|
|
69
|
-
| `worker_costs` | `worker_id:`, `period: 'daily'` | Aggregated token/call/latency metrics |
|
|
70
|
-
| `team_costs` | `team:`, `period: 'daily'` | Team-wide aggregation across all team workers |
|
|
71
|
-
| `routing_stats` | `worker_id: nil` | Breakdowns by routing_reason, provider, model, avg latency |
|
|
72
|
-
| `cleanup_old_records` | `retention_days: 90` | Deletes records older than cutoff |
|
|
73
|
-
|
|
74
|
-
`period` values: `'daily'`, `'weekly'`, `'monthly'`
|
|
75
|
-
|
|
76
|
-
### Rollup (`Runners::Rollup`)
|
|
77
|
-
|
|
78
|
-
`rollup_hour` — groups `metering_records` by worker/provider/model for the current hour into `metering_hourly_rollup` table with upsert semantics (one row per worker+provider+model+hour).
|
|
79
|
-
|
|
80
|
-
`purge_raw_records(retention_days: 7)` — deletes raw `metering_records` older than the retention window after they have been rolled up.
|
|
81
|
-
|
|
82
|
-
### CostOptimizer (`Runners::CostOptimizer`)
|
|
83
|
-
|
|
84
|
-
`analyze_costs(window_days: 7, top_n: 10)` — aggregates cost drivers from `metering_records` for the past `window_days` days. Calls `Legion::LLM.chat` (with `caller: { extension: 'lex-metering', operation: 'cost_optimization' }`) to generate model rightsizing recommendations. Returns `{ status:, cost_drivers:, recommendations: }`.
|
|
85
|
-
|
|
86
|
-
Built-in rate table (per 1M tokens): Anthropic claude-opus-4-6 $15, claude-sonnet-4-6 $3, claude-haiku-4-5 $0.25; OpenAI gpt-4o $5, gpt-4o-mini $0.15; Bedrock/Azure default $3.
|
|
87
|
-
|
|
88
|
-
## Actors
|
|
89
|
-
|
|
90
|
-
| Actor | Interval | Behaviour |
|
|
91
|
-
|-------|----------|-----------|
|
|
92
|
-
| `Cleanup` | 86400s daily | `cleanup_old_records`; `use_runner? false` |
|
|
93
|
-
| `CostOptimizer` | 604800s weekly | `analyze_costs`; `use_runner? false`; Singleton mixin |
|
|
94
|
-
| `Rollup` | 3600s hourly | `rollup_hour`; `run_now? false` |
|
|
95
|
-
|
|
96
|
-
## Integration Points
|
|
97
|
-
|
|
98
|
-
- **legion-data**: `data_required? false` — loads without DB. `record` returns hash for RMQ publishing; query methods require `Legion::Data`.
|
|
99
|
-
- **LegionIO MCP**: `legion.routing_stats` MCP tool calls `routing_stats` runner
|
|
100
|
-
- **REST API**: `GET /api/tasks/:id` includes a `:metering` block when lex-metering data exists for the task
|
|
101
|
-
- **Digital Workers**: `legion worker costs` CLI command delegates to `worker_costs` runner
|
|
102
|
-
- **lex-llm-gateway**: MeteringWriter actor writes to the same `metering_records` table
|
|
103
|
-
|
|
104
|
-
## Development Notes
|
|
105
|
-
|
|
106
|
-
- Actor module is `module Actor` (singular) per framework convention
|
|
107
|
-
- `routing_stats` uses `select_append { avg(latency_ms).as(avg_latency) }` — Sequel virtual row syntax
|
|
108
|
-
- Time interval filtering uses `Sequel.lit('recorded_at >= ?', cutoff)` with Ruby `Time` arithmetic for cross-database compatibility
|
|
109
|
-
- `CostOptimizer` actor includes `Legion::Extensions::Actors::Singleton` when available to prevent duplicate weekly runs in a cluster
|
|
110
|
-
|
|
111
|
-
---
|
|
20
|
+
## Key Design Decisions
|
|
112
21
|
|
|
113
|
-
|
|
22
|
+
- **`data_required? true`**: migrations auto-run on boot. `record` returns hash for RMQ publishing; query methods require `Legion::Data`.
|
|
23
|
+
- **Rollup**: groups by worker/provider/model per hour into `metering_hourly_rollup` (upsert semantics). `purge_raw_records(retention_days: 7)` cleans rolled-up data.
|
|
24
|
+
- **CostOptimizer**: weekly LLM-powered analysis with built-in rate table (Opus $15, Sonnet $3, Haiku $0.25, GPT-4o $5, etc. per 1M tokens). Uses `caller: { extension: 'lex-metering', operation: 'cost_optimization' }`.
|
|
25
|
+
- **Period values**: `'daily'`, `'weekly'`, `'monthly'` for worker_costs/team_costs.
|
|
26
|
+
- **routing_stats**: uses `select { [provider, avg(latency_ms).as(avg_latency)] }` Sequel virtual row syntax with explicit projection.
|
|
27
|
+
- Actor module is singular (`module Actor`) per framework convention.
|
|
28
|
+
- Integration: MCP tool `legion.routing_stats`, REST API includes `:metering` block, CLI `legion worker costs`, shared `metering_records` table with lex-llm-gateway.
|
data/README.md
CHANGED
|
@@ -1,67 +1,214 @@
|
|
|
1
1
|
# lex-metering
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
LLM cost metering for LegionIO. Records token usage, latency, and routing metrics per task for cost attribution, budget forecasting, and intelligent model routing.
|
|
4
4
|
|
|
5
5
|
**Ruby >= 3.4** | **License**: MIT | **Author**: [@Esity](https://github.com/Esity)
|
|
6
6
|
|
|
7
|
-
## Purpose
|
|
8
|
-
|
|
9
|
-
`lex-metering` records every LLM call made through a Legion digital worker: tokens consumed (input, output, thinking), latency, wall-clock time, CPU time, external API calls, and routing reason. Data is persisted to the `metering_records` table and queried for cost attribution and routing statistics.
|
|
10
|
-
|
|
11
7
|
## Installation
|
|
12
8
|
|
|
13
9
|
Included with the LegionIO framework. No separate installation needed.
|
|
14
10
|
|
|
11
|
+
```ruby
|
|
12
|
+
# Gemfile
|
|
13
|
+
gem 'lex-metering'
|
|
14
|
+
```
|
|
15
|
+
|
|
16
|
+
## Architecture
|
|
17
|
+
|
|
18
|
+
```
|
|
19
|
+
Legion::Extensions::Metering
|
|
20
|
+
├── Runners/
|
|
21
|
+
│ ├── Metering # record, worker_costs, team_costs, routing_stats, cleanup_old_records
|
|
22
|
+
│ ├── CostOptimizer # analyze_costs (LLM-powered model rightsizing)
|
|
23
|
+
│ └── Rollup # rollup_hour, purge_raw_records
|
|
24
|
+
├── Actors/
|
|
25
|
+
│ ├── Cleanup # Daily (86400s) — prunes records older than 90 days
|
|
26
|
+
│ ├── CostOptimizer # Weekly (604800s) — generates model rightsizing recommendations
|
|
27
|
+
│ └── Rollup # Hourly (3600s) — aggregates raw records into hourly summaries
|
|
28
|
+
├── Helpers/
|
|
29
|
+
│ └── Economics # payroll_summary, worker_report, budget_forecast
|
|
30
|
+
└── Data/Migrations/
|
|
31
|
+
├── 001 # metering_records table
|
|
32
|
+
├── 002 # trace columns (cost_usd, status, event_type, extension, runner_function)
|
|
33
|
+
└── 003 # indexes (worker_id, task_id, provider, recorded_at)
|
|
34
|
+
```
|
|
35
|
+
|
|
15
36
|
## Usage
|
|
16
37
|
|
|
38
|
+
### Recording Metrics
|
|
39
|
+
|
|
17
40
|
```ruby
|
|
18
|
-
# Record an LLM call
|
|
19
41
|
Legion::Extensions::Metering::Runners::Metering.record(
|
|
20
|
-
worker_id:
|
|
21
|
-
task_id:
|
|
22
|
-
provider:
|
|
23
|
-
model_id:
|
|
24
|
-
input_tokens:
|
|
25
|
-
output_tokens:
|
|
26
|
-
|
|
27
|
-
|
|
42
|
+
worker_id: 'worker-abc',
|
|
43
|
+
task_id: 42,
|
|
44
|
+
provider: 'anthropic',
|
|
45
|
+
model_id: 'claude-sonnet-4-6',
|
|
46
|
+
input_tokens: 1000,
|
|
47
|
+
output_tokens: 500,
|
|
48
|
+
thinking_tokens: 200,
|
|
49
|
+
latency_ms: 1200,
|
|
50
|
+
wall_clock_ms: 1500,
|
|
51
|
+
cpu_time_ms: 80,
|
|
52
|
+
cost_usd: 0.0051,
|
|
53
|
+
routing_reason: 'cost_optimization',
|
|
54
|
+
extension: 'lex-developer',
|
|
55
|
+
runner_function: 'generate_code',
|
|
56
|
+
status: 'success',
|
|
57
|
+
event_type: 'llm_completion'
|
|
28
58
|
)
|
|
59
|
+
```
|
|
29
60
|
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
61
|
+
### Querying Costs
|
|
62
|
+
|
|
63
|
+
```ruby
|
|
64
|
+
# Per-worker costs (daily/weekly/monthly)
|
|
65
|
+
Legion::Extensions::Metering::Runners::Metering.worker_costs(
|
|
66
|
+
worker_id: 'worker-abc',
|
|
67
|
+
period: 'weekly'
|
|
34
68
|
)
|
|
69
|
+
# => { worker_id:, period:, total_tokens:, input_tokens:, output_tokens:,
|
|
70
|
+
# thinking_tokens:, total_calls:, avg_latency_ms:, by_provider:, by_model: }
|
|
71
|
+
|
|
72
|
+
# Per-team costs
|
|
73
|
+
Legion::Extensions::Metering::Runners::Metering.team_costs(
|
|
74
|
+
team: 'engineering',
|
|
75
|
+
period: 'monthly'
|
|
76
|
+
)
|
|
77
|
+
# => { team:, period:, worker_count:, total_tokens:, total_calls:, by_worker: }
|
|
78
|
+
```
|
|
35
79
|
|
|
36
|
-
|
|
37
|
-
|
|
80
|
+
### Routing Statistics
|
|
81
|
+
|
|
82
|
+
```ruby
|
|
83
|
+
Legion::Extensions::Metering::Runners::Metering.routing_stats(worker_id: 'worker-abc')
|
|
84
|
+
# => { by_routing_reason: [...], by_provider: [...], by_model: [...],
|
|
85
|
+
# avg_latency_by_provider: [{ provider: 'anthropic', avg_latency: 820.0 }] }
|
|
38
86
|
```
|
|
39
87
|
|
|
40
|
-
|
|
88
|
+
### Hourly Rollup
|
|
41
89
|
|
|
42
|
-
|
|
90
|
+
Raw records are aggregated hourly into `metering_hourly_rollup` for efficient reporting:
|
|
43
91
|
|
|
44
|
-
|
|
92
|
+
```ruby
|
|
93
|
+
# Manually trigger (normally runs via actor)
|
|
94
|
+
Legion::Extensions::Metering::Runners::Rollup.rollup_hour
|
|
95
|
+
# => { rolled_up: 15, hour: "2026-05-17T10:00:00Z", raw_records: 340 }
|
|
96
|
+
|
|
97
|
+
# Purge rolled-up raw records (default: 7 days retention)
|
|
98
|
+
Legion::Extensions::Metering::Runners::Rollup.purge_raw_records(retention_days: 7)
|
|
99
|
+
# => { purged: 2400, retention_days: 7, cutoff: "2026-05-10T11:00:00Z" }
|
|
100
|
+
```
|
|
45
101
|
|
|
46
|
-
|
|
102
|
+
### Cost Optimization
|
|
47
103
|
|
|
48
|
-
|
|
104
|
+
Weekly LLM-powered analysis identifies model rightsizing opportunities:
|
|
49
105
|
|
|
50
106
|
```ruby
|
|
107
|
+
Legion::Extensions::Metering::Runners::CostOptimizer.analyze_costs(window_days: 7, top_n: 10)
|
|
108
|
+
# => { status: 'analyzed', window_days: 7, cost_drivers: [...],
|
|
109
|
+
# recommendations: [{ extension: 'lex-developer', current_model: 'claude-opus-4-6',
|
|
110
|
+
# suggested_model: 'claude-sonnet-4-6', rationale: '...',
|
|
111
|
+
# estimated_savings_pct: 80 }] }
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
Built-in rate table (per 1M tokens):
|
|
115
|
+
|
|
116
|
+
| Provider | Model | Rate |
|
|
117
|
+
|----------|-------|------|
|
|
118
|
+
| Anthropic | claude-opus-4-6 | $15.00 |
|
|
119
|
+
| Anthropic | claude-sonnet-4-6 | $3.00 |
|
|
120
|
+
| Anthropic | claude-haiku-4-5 | $0.25 |
|
|
121
|
+
| OpenAI | gpt-4o | $5.00 |
|
|
122
|
+
| OpenAI | gpt-4o-mini | $0.15 |
|
|
123
|
+
| OpenAI | gpt-4.1 | $2.00 |
|
|
124
|
+
| Bedrock | default | $3.00 |
|
|
125
|
+
| Azure AI | default | $3.00 |
|
|
126
|
+
|
|
127
|
+
### Economics Helper
|
|
128
|
+
|
|
129
|
+
Labor economics reporting for digital worker cost attribution:
|
|
130
|
+
|
|
131
|
+
```ruby
|
|
132
|
+
include Legion::Extensions::Metering::Helpers::Economics
|
|
133
|
+
|
|
134
|
+
payroll_summary(period: :weekly)
|
|
135
|
+
# => { workers: [{ worker_id:, task_count:, cost:, autonomy: }], total_cost:, avg_productivity: }
|
|
136
|
+
|
|
137
|
+
worker_report(worker_id: 'worker-abc', period: :daily)
|
|
138
|
+
# => { worker_id:, salary:, overtime:, productivity:, avg_latency:, autonomy_level: }
|
|
139
|
+
|
|
140
|
+
budget_forecast(days: 30)
|
|
141
|
+
# => { projected_cost: 4.50, daily_average: 0.15, days: 30, trend: :active }
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
## Database Schema
|
|
145
|
+
|
|
146
|
+
### `metering_records`
|
|
147
|
+
|
|
148
|
+
| Column | Type | Description |
|
|
149
|
+
|--------|------|-------------|
|
|
150
|
+
| `id` | Integer (PK) | Auto-increment primary key |
|
|
151
|
+
| `worker_id` | String(36) | Digital worker ID |
|
|
152
|
+
| `task_id` | Integer | Legion task ID |
|
|
153
|
+
| `provider` | String(100) | LLM provider name |
|
|
154
|
+
| `model_id` | String(255) | Model identifier |
|
|
155
|
+
| `input_tokens` | Integer | Prompt tokens |
|
|
156
|
+
| `output_tokens` | Integer | Completion tokens |
|
|
157
|
+
| `thinking_tokens` | Integer | Reasoning tokens |
|
|
158
|
+
| `total_tokens` | Integer | Sum of all token types |
|
|
159
|
+
| `input_context_bytes` | Integer | Raw context size in bytes |
|
|
160
|
+
| `latency_ms` | Integer | LLM API round-trip time |
|
|
161
|
+
| `wall_clock_ms` | Integer | Total wall-clock time |
|
|
162
|
+
| `cpu_time_ms` | Integer | CPU time consumed |
|
|
163
|
+
| `external_api_calls` | Integer | Non-LLM external API calls |
|
|
164
|
+
| `routing_reason` | String(255) | Model/provider selection rationale |
|
|
165
|
+
| `cost_usd` | Float | Estimated cost in USD |
|
|
166
|
+
| `status` | String(50) | Pipeline status at time of record |
|
|
167
|
+
| `event_type` | String(100) | Event type for audit correlation |
|
|
168
|
+
| `extension` | String(255) | Calling extension name |
|
|
169
|
+
| `runner_function` | String(255) | Calling runner function |
|
|
170
|
+
| `recorded_at` | DateTime | Timestamp (indexed) |
|
|
171
|
+
|
|
172
|
+
Indexes: `worker_id`, `task_id`, `provider`, `recorded_at`, `status`, `event_type`, `extension`
|
|
173
|
+
|
|
174
|
+
### `metering_hourly_rollup`
|
|
175
|
+
|
|
176
|
+
Aggregated hourly summaries grouped by worker/provider/model. One row per unique combination per hour, upserted on each rollup cycle.
|
|
177
|
+
|
|
178
|
+
## Record Retention
|
|
179
|
+
|
|
180
|
+
| Actor | Interval | Retention | Description |
|
|
181
|
+
|-------|----------|-----------|-------------|
|
|
182
|
+
| Cleanup | Daily | 90 days | Prunes raw `metering_records` older than cutoff |
|
|
183
|
+
| Rollup | Hourly | — | Aggregates into `metering_hourly_rollup` |
|
|
184
|
+
| Purge | On-demand | 7 days | Removes rolled-up raw records via `purge_raw_records` |
|
|
185
|
+
|
|
186
|
+
```ruby
|
|
187
|
+
# Manual cleanup
|
|
51
188
|
Legion::Extensions::Metering::Runners::Metering.cleanup_old_records(retention_days: 90)
|
|
52
|
-
# => { purged: 1234, retention_days: 90, cutoff:
|
|
189
|
+
# => { purged: 1234, retention_days: 90, cutoff: 2026-02-16 00:00:00 UTC }
|
|
53
190
|
```
|
|
54
191
|
|
|
55
|
-
##
|
|
192
|
+
## Integration Points
|
|
193
|
+
|
|
194
|
+
| System | Interface | Description |
|
|
195
|
+
|--------|-----------|-------------|
|
|
196
|
+
| MCP | `legion.routing_stats` | Tool for querying routing statistics |
|
|
197
|
+
| REST API | `GET /api/metering` | Returns routing stats and recent records |
|
|
198
|
+
| CLI | `legion worker costs` | Worker cost attribution from terminal |
|
|
199
|
+
| lex-llm-gateway | Shared table | Gateway publishes metering events over AMQP |
|
|
200
|
+
| legion-data | Migrations 021, 046 | Archive table and hourly rollup DDL |
|
|
56
201
|
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
202
|
+
## Development
|
|
203
|
+
|
|
204
|
+
```bash
|
|
205
|
+
bundle install
|
|
206
|
+
bundle exec rspec
|
|
207
|
+
bundle exec rubocop
|
|
208
|
+
```
|
|
62
209
|
|
|
63
210
|
## Related
|
|
64
211
|
|
|
65
|
-
- [LegionIO](https://github.com/LegionIO/LegionIO)
|
|
66
|
-
- [legion-data](https://github.com/LegionIO/legion-data)
|
|
67
|
-
- [lex-llm-gateway](https://github.com/LegionIO/lex-llm-gateway)
|
|
212
|
+
- [LegionIO](https://github.com/LegionIO/LegionIO) — Framework
|
|
213
|
+
- [legion-data](https://github.com/LegionIO/legion-data) — Persistence layer
|
|
214
|
+
- [lex-llm-gateway](https://github.com/LegionIO/lex-llm-gateway) — Gateway that publishes metering events over AMQP
|
|
@@ -19,11 +19,6 @@ Sequel.migration do
|
|
|
19
19
|
Integer :external_api_calls, null: false, default: 0
|
|
20
20
|
String :routing_reason, null: true, size: 255
|
|
21
21
|
DateTime :recorded_at, null: false, default: Sequel::CURRENT_TIMESTAMP
|
|
22
|
-
|
|
23
|
-
index :worker_id
|
|
24
|
-
index :task_id
|
|
25
|
-
index :provider
|
|
26
|
-
index :recorded_at
|
|
27
22
|
end
|
|
28
23
|
end
|
|
29
24
|
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
Sequel.migration do
|
|
4
|
+
up do
|
|
5
|
+
alter_table(:metering_records) do
|
|
6
|
+
add_index :worker_id, if_not_exists: true
|
|
7
|
+
add_index :task_id, if_not_exists: true
|
|
8
|
+
add_index :provider, if_not_exists: true
|
|
9
|
+
add_index :recorded_at, if_not_exists: true
|
|
10
|
+
end
|
|
11
|
+
end
|
|
12
|
+
|
|
13
|
+
down do
|
|
14
|
+
alter_table(:metering_records) do
|
|
15
|
+
drop_index :worker_id, if_exists: true
|
|
16
|
+
drop_index :task_id, if_exists: true
|
|
17
|
+
drop_index :provider, if_exists: true
|
|
18
|
+
drop_index :recorded_at, if_exists: true
|
|
19
|
+
end
|
|
20
|
+
end
|
|
21
|
+
end
|
|
@@ -72,8 +72,12 @@ module Legion
|
|
|
72
72
|
return { recommendations: [] } unless defined?(Legion::LLM)
|
|
73
73
|
|
|
74
74
|
prompt = build_recommendation_prompt(drivers)
|
|
75
|
-
result =
|
|
76
|
-
|
|
75
|
+
result = Legion::LLM.chat( # rubocop:disable Legion/HelperMigration/DirectLlm
|
|
76
|
+
message: prompt,
|
|
77
|
+
caller: { extension: 'lex-metering', operation: 'cost_optimization' }
|
|
78
|
+
)
|
|
79
|
+
content = result.is_a?(Hash) ? result[:content] : result.content
|
|
80
|
+
::JSON.parse(content || '{}', symbolize_names: true)
|
|
77
81
|
rescue StandardError => _e
|
|
78
82
|
{ recommendations: [] }
|
|
79
83
|
end
|
|
@@ -83,7 +83,7 @@ module Legion
|
|
|
83
83
|
by_routing_reason: ds.group_and_count(:routing_reason).all,
|
|
84
84
|
by_provider: ds.group_and_count(:provider).all,
|
|
85
85
|
by_model: ds.group_and_count(:model_id).all,
|
|
86
|
-
avg_latency_by_provider: ds.group(:provider).
|
|
86
|
+
avg_latency_by_provider: ds.group(:provider).select { [provider, avg(latency_ms).as(avg_latency)] }.all
|
|
87
87
|
}
|
|
88
88
|
end
|
|
89
89
|
|
|
@@ -9,12 +9,12 @@ module Legion
|
|
|
9
9
|
module Metering
|
|
10
10
|
extend Legion::Extensions::Core if Legion::Extensions.const_defined? :Core, false
|
|
11
11
|
|
|
12
|
-
def self.data_required?
|
|
13
|
-
|
|
12
|
+
def self.data_required? # rubocop:disable Legion/Extension/DataRequiredWithoutMigrations
|
|
13
|
+
true
|
|
14
14
|
end
|
|
15
15
|
|
|
16
16
|
def data_required?
|
|
17
|
-
|
|
17
|
+
true
|
|
18
18
|
end
|
|
19
19
|
end
|
|
20
20
|
end
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: lex-metering
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.1.
|
|
4
|
+
version: 0.1.16
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Esity
|
|
@@ -189,6 +189,7 @@ files:
|
|
|
189
189
|
- lib/legion/extensions/metering/actors/rollup.rb
|
|
190
190
|
- lib/legion/extensions/metering/data/migrations/001_add_metering_records.rb
|
|
191
191
|
- lib/legion/extensions/metering/data/migrations/002_add_trace_columns.rb
|
|
192
|
+
- lib/legion/extensions/metering/data/migrations/003_add_metering_indexes.rb
|
|
192
193
|
- lib/legion/extensions/metering/helpers/economics.rb
|
|
193
194
|
- lib/legion/extensions/metering/runners/cost_optimizer.rb
|
|
194
195
|
- lib/legion/extensions/metering/runners/metering.rb
|