lex-metering 0.1.4 → 0.1.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +5 -0
- data/CLAUDE.md +9 -2
- data/Gemfile +5 -0
- data/README.md +13 -5
- data/lib/legion/extensions/metering/helpers/economics.rb +86 -0
- data/lib/legion/extensions/metering/version.rb +1 -1
- metadata +2 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 13443e923c7850c17785f0e750d1bdb995575da237592758c7578de30c321081
|
|
4
|
+
data.tar.gz: e054cb9fd8222b5d0616f82afb46669541fccfa54c9de4dbd7f83360cb4e7967
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 561748036537315786f810e594f005f58de6f4b2793ec497967dd00ae7488ac7c75a7fd4f7ddfeeea0f5cdd2da9442b3ffe434f4a419d3f083c63c3cea958a47
|
|
7
|
+
data.tar.gz: 0dfdb4731304dadc64f56c02e3c71edd939137026b51f057cf5c181412edc4b4b23c02d7bae8ae65451ce8e41c222bd5bbc5a9944991bec3c3dc282ee08c4b21
|
data/CHANGELOG.md
CHANGED
data/CLAUDE.md
CHANGED
|
@@ -22,11 +22,13 @@ Captures LLM token usage metrics per task for cost attribution and intelligent r
|
|
|
22
22
|
```
|
|
23
23
|
lib/legion/extensions/metering/
|
|
24
24
|
version.rb
|
|
25
|
+
actors/
|
|
26
|
+
cleanup.rb # Every actor (86400s/daily): calls cleanup_old_records
|
|
25
27
|
data/
|
|
26
28
|
migrations/
|
|
27
29
|
001_add_metering_records.rb # Creates metering_records table
|
|
28
30
|
runners/
|
|
29
|
-
metering.rb # record, worker_costs, team_costs, routing_stats
|
|
31
|
+
metering.rb # record, worker_costs, team_costs, routing_stats, cleanup_old_records
|
|
30
32
|
```
|
|
31
33
|
|
|
32
34
|
## Database Schema (`metering_records`)
|
|
@@ -57,9 +59,14 @@ lib/legion/extensions/metering/
|
|
|
57
59
|
| `worker_costs` | `worker_id:`, `period: 'daily'` | Aggregated token/call/latency metrics |
|
|
58
60
|
| `team_costs` | `team:`, `period: 'daily'` | Team-wide aggregation across all team workers |
|
|
59
61
|
| `routing_stats` | `worker_id: nil` | Breakdowns by routing_reason, provider, model, avg latency |
|
|
62
|
+
| `cleanup_old_records` | `retention_days: 90` | Deletes records older than cutoff; returns `{ purged:, retention_days:, cutoff: }` |
|
|
60
63
|
|
|
61
64
|
`period` values: `'daily'`, `'weekly'`, `'monthly'`
|
|
62
65
|
|
|
66
|
+
## Cleanup Actor
|
|
67
|
+
|
|
68
|
+
`Actor::Cleanup` is an Every actor that calls `cleanup_old_records` once per day (86,400s). It runs with `run_now? false`, `use_runner? false`, `check_subtask? false`, `generate_task? false` — a minimal background trigger that delegates directly to the runner method.
|
|
69
|
+
|
|
63
70
|
## Integration Points
|
|
64
71
|
|
|
65
72
|
- **legion-data**: `data_required? false` — loads without DB. `record` returns hash only (for RabbitMQ publishing). Query methods (`worker_costs`, `team_costs`, `routing_stats`, `cleanup_old_records`) still access `metering_records` as a raw Sequel dataset when `Legion::Data` is available.
|
|
@@ -70,6 +77,6 @@ lib/legion/extensions/metering/
|
|
|
70
77
|
## Development Notes
|
|
71
78
|
|
|
72
79
|
- Extension has `data_required? false` — loads without `legion-data`; `record` builds hash only (no DB insert), query methods still require `Legion::Data`
|
|
73
|
-
-
|
|
80
|
+
- Has one explicit actor (`Cleanup`); auto-generated subscription actors are created for runner methods
|
|
74
81
|
- `routing_stats` uses `select_append { avg(latency_ms).as(avg_latency) }` — Sequel virtual row syntax
|
|
75
82
|
- Time interval filtering uses `Sequel.lit('recorded_at >= ?', cutoff)` with Ruby `Time` arithmetic for cross-database compatibility (PostgreSQL, SQLite, MySQL)
|
data/Gemfile
CHANGED
data/README.md
CHANGED
|
@@ -6,7 +6,7 @@ Captures LLM token usage, latency, and routing metrics per task for cost attribu
|
|
|
6
6
|
|
|
7
7
|
## Purpose
|
|
8
8
|
|
|
9
|
-
`lex-metering` records every LLM call made through a Legion digital worker
|
|
9
|
+
`lex-metering` records every LLM call made through a Legion digital worker: tokens consumed (input, output, thinking), latency, wall-clock time, CPU time, external API calls, and routing reason. Data is persisted to the `metering_records` table and queried for cost attribution and routing statistics.
|
|
10
10
|
|
|
11
11
|
## Installation
|
|
12
12
|
|
|
@@ -39,7 +39,7 @@ stats = Legion::Extensions::Metering::Runners::Metering.routing_stats
|
|
|
39
39
|
|
|
40
40
|
## Database
|
|
41
41
|
|
|
42
|
-
Requires `legion-data`. Creates the `metering_records` table via Sequel migration.
|
|
42
|
+
Requires `legion-data`. Creates the `metering_records` table via Sequel migration. Extension loads without `legion-data` (`data_required? false`) — `record` returns a hash for RMQ publishing; query methods require `Legion::Data` to be available.
|
|
43
43
|
|
|
44
44
|
## Record Retention
|
|
45
45
|
|
|
@@ -52,8 +52,16 @@ Legion::Extensions::Metering::Runners::Metering.cleanup_old_records(retention_da
|
|
|
52
52
|
# => { purged: 1234, retention_days: 90, cutoff: 2025-12-15 00:00:00 UTC }
|
|
53
53
|
```
|
|
54
54
|
|
|
55
|
+
## Query Methods
|
|
56
|
+
|
|
57
|
+
| Method | Parameters | Period Values |
|
|
58
|
+
|--------|-----------|---------------|
|
|
59
|
+
| `worker_costs` | `worker_id:`, `period: 'daily'` | `'daily'`, `'weekly'`, `'monthly'` |
|
|
60
|
+
| `team_costs` | `team:`, `period: 'daily'` | `'daily'`, `'weekly'`, `'monthly'` |
|
|
61
|
+
| `routing_stats` | `worker_id: nil` | — |
|
|
62
|
+
|
|
55
63
|
## Related
|
|
56
64
|
|
|
57
|
-
- [LegionIO](https://github.com/LegionIO/LegionIO)
|
|
58
|
-
- [legion-data](https://github.com/LegionIO/legion-data)
|
|
59
|
-
- [
|
|
65
|
+
- [LegionIO](https://github.com/LegionIO/LegionIO) - Framework
|
|
66
|
+
- [legion-data](https://github.com/LegionIO/legion-data) - Persistence layer
|
|
67
|
+
- [lex-llm-gateway](https://github.com/LegionIO/lex-llm-gateway) - Gateway that publishes metering events over RMQ
|
|
@@ -0,0 +1,86 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Legion
|
|
4
|
+
module Extensions
|
|
5
|
+
module Metering
|
|
6
|
+
module Helpers
|
|
7
|
+
module Economics
|
|
8
|
+
PERIOD_DAYS = { daily: 1, weekly: 7, monthly: 30 }.freeze
|
|
9
|
+
|
|
10
|
+
def payroll_summary(period: :daily, **)
|
|
11
|
+
return { workers: [], total_cost: 0, avg_productivity: 0 } unless defined?(Legion::Data)
|
|
12
|
+
|
|
13
|
+
days = PERIOD_DAYS.fetch(period.to_sym, 1)
|
|
14
|
+
cutoff = Time.now - (days * 86_400)
|
|
15
|
+
ds = Legion::Data.connection[:metering_records].where(Sequel.lit('recorded_at >= ?', cutoff))
|
|
16
|
+
|
|
17
|
+
workers = ds.group_and_count(:worker_id).all.map do |row|
|
|
18
|
+
{
|
|
19
|
+
worker_id: row[:worker_id],
|
|
20
|
+
task_count: row[:count],
|
|
21
|
+
cost: ds.where(worker_id: row[:worker_id]).sum(:input_tokens).to_f * cost_per_token,
|
|
22
|
+
autonomy: synapse_autonomy(row[:worker_id])
|
|
23
|
+
}
|
|
24
|
+
end
|
|
25
|
+
|
|
26
|
+
total = workers.sum { |w| w[:cost] }
|
|
27
|
+
avg_prod = workers.empty? ? 0 : workers.sum { |w| w[:task_count] } / workers.size.to_f
|
|
28
|
+
|
|
29
|
+
{ workers: workers, total_cost: total, avg_productivity: avg_prod, period: period }
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
def worker_report(worker_id:, period: :daily, **)
|
|
33
|
+
return { salary: 0, overtime: 0, productivity: 0 } unless defined?(Legion::Data)
|
|
34
|
+
|
|
35
|
+
days = PERIOD_DAYS.fetch(period.to_sym, 1)
|
|
36
|
+
cutoff = Time.now - (days * 86_400)
|
|
37
|
+
ds = Legion::Data.connection[:metering_records]
|
|
38
|
+
.where(worker_id: worker_id)
|
|
39
|
+
.where(Sequel.lit('recorded_at >= ?', cutoff))
|
|
40
|
+
|
|
41
|
+
task_count = ds.count
|
|
42
|
+
total_tokens = ds.sum(:input_tokens).to_i + ds.sum(:output_tokens).to_i
|
|
43
|
+
salary = total_tokens.to_f * cost_per_token
|
|
44
|
+
avg_latency = ds.avg(:latency_ms).to_f
|
|
45
|
+
|
|
46
|
+
{
|
|
47
|
+
worker_id: worker_id,
|
|
48
|
+
salary: salary,
|
|
49
|
+
overtime: 0,
|
|
50
|
+
productivity: task_count,
|
|
51
|
+
avg_latency: avg_latency,
|
|
52
|
+
autonomy_level: synapse_autonomy(worker_id),
|
|
53
|
+
period: period
|
|
54
|
+
}
|
|
55
|
+
end
|
|
56
|
+
|
|
57
|
+
def budget_forecast(days: 30, **)
|
|
58
|
+
return { projected_cost: 0, trend: :flat } unless defined?(Legion::Data)
|
|
59
|
+
|
|
60
|
+
recent_ds = Legion::Data.connection[:metering_records]
|
|
61
|
+
.where(Sequel.lit('recorded_at >= ?', Time.now - 86_400))
|
|
62
|
+
daily_cost = (recent_ds.sum(:input_tokens).to_i + recent_ds.sum(:output_tokens).to_i) *
|
|
63
|
+
cost_per_token
|
|
64
|
+
|
|
65
|
+
{ projected_cost: daily_cost * days, daily_average: daily_cost, days: days,
|
|
66
|
+
trend: daily_cost.positive? ? :active : :flat }
|
|
67
|
+
end
|
|
68
|
+
|
|
69
|
+
private
|
|
70
|
+
|
|
71
|
+
def cost_per_token
|
|
72
|
+
0.000003
|
|
73
|
+
end
|
|
74
|
+
|
|
75
|
+
def synapse_autonomy(worker_id)
|
|
76
|
+
return :unknown unless defined?(Legion::Extensions::Synapse)
|
|
77
|
+
|
|
78
|
+
Legion::Extensions::Synapse::Client.new.autonomy_level(worker_id: worker_id)
|
|
79
|
+
rescue StandardError
|
|
80
|
+
:unknown
|
|
81
|
+
end
|
|
82
|
+
end
|
|
83
|
+
end
|
|
84
|
+
end
|
|
85
|
+
end
|
|
86
|
+
end
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: lex-metering
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.1.
|
|
4
|
+
version: 0.1.5
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Esity
|
|
@@ -114,6 +114,7 @@ files:
|
|
|
114
114
|
- lib/legion/extensions/metering.rb
|
|
115
115
|
- lib/legion/extensions/metering/actors/cleanup.rb
|
|
116
116
|
- lib/legion/extensions/metering/data/migrations/001_add_metering_records.rb
|
|
117
|
+
- lib/legion/extensions/metering/helpers/economics.rb
|
|
117
118
|
- lib/legion/extensions/metering/runners/metering.rb
|
|
118
119
|
- lib/legion/extensions/metering/version.rb
|
|
119
120
|
homepage: https://github.com/LegionIO/lex-metering
|