llm_cost_tracker 0.1.3 → 0.2.0.alpha1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (71) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +64 -81
  3. data/PLAN_0.2.md +488 -0
  4. data/README.md +141 -316
  5. data/app/controllers/llm_cost_tracker/application_controller.rb +42 -0
  6. data/app/controllers/llm_cost_tracker/calls_controller.rb +77 -0
  7. data/app/controllers/llm_cost_tracker/dashboard_controller.rb +54 -0
  8. data/app/controllers/llm_cost_tracker/data_quality_controller.rb +10 -0
  9. data/app/controllers/llm_cost_tracker/models_controller.rb +12 -0
  10. data/app/controllers/llm_cost_tracker/tags_controller.rb +21 -0
  11. data/app/helpers/llm_cost_tracker/application_helper.rb +113 -0
  12. data/app/services/llm_cost_tracker/dashboard/data_quality.rb +38 -0
  13. data/app/services/llm_cost_tracker/dashboard/filter.rb +109 -0
  14. data/app/services/llm_cost_tracker/dashboard/overview_stats.rb +87 -0
  15. data/app/services/llm_cost_tracker/dashboard/provider_breakdown.rb +44 -0
  16. data/app/services/llm_cost_tracker/dashboard/tag_breakdown.rb +58 -0
  17. data/app/services/llm_cost_tracker/dashboard/tag_key_explorer.rb +125 -0
  18. data/app/services/llm_cost_tracker/dashboard/time_series.rb +44 -0
  19. data/app/services/llm_cost_tracker/dashboard/top_models.rb +89 -0
  20. data/app/services/llm_cost_tracker/pagination.rb +59 -0
  21. data/app/views/layouts/llm_cost_tracker/application.html.erb +342 -0
  22. data/app/views/llm_cost_tracker/calls/index.html.erb +127 -0
  23. data/app/views/llm_cost_tracker/calls/show.html.erb +67 -0
  24. data/app/views/llm_cost_tracker/dashboard/index.html.erb +145 -0
  25. data/app/views/llm_cost_tracker/data_quality/index.html.erb +110 -0
  26. data/app/views/llm_cost_tracker/errors/database.html.erb +8 -0
  27. data/app/views/llm_cost_tracker/errors/invalid_filter.html.erb +4 -0
  28. data/app/views/llm_cost_tracker/errors/not_found.html.erb +5 -0
  29. data/app/views/llm_cost_tracker/models/index.html.erb +95 -0
  30. data/app/views/llm_cost_tracker/shared/_bar.html.erb +5 -0
  31. data/app/views/llm_cost_tracker/shared/setup_required.html.erb +6 -0
  32. data/app/views/llm_cost_tracker/tags/index.html.erb +34 -0
  33. data/app/views/llm_cost_tracker/tags/show.html.erb +69 -0
  34. data/config/routes.rb +10 -0
  35. data/lib/llm_cost_tracker/budget.rb +16 -38
  36. data/lib/llm_cost_tracker/configuration.rb +3 -1
  37. data/lib/llm_cost_tracker/cost.rb +1 -3
  38. data/lib/llm_cost_tracker/engine.rb +13 -0
  39. data/lib/llm_cost_tracker/engine_compatibility.rb +15 -0
  40. data/lib/llm_cost_tracker/errors.rb +2 -0
  41. data/lib/llm_cost_tracker/event.rb +1 -3
  42. data/lib/llm_cost_tracker/event_metadata.rb +9 -18
  43. data/lib/llm_cost_tracker/llm_api_call.rb +43 -9
  44. data/lib/llm_cost_tracker/middleware/faraday.rb +4 -4
  45. data/lib/llm_cost_tracker/parsed_usage.rb +5 -9
  46. data/lib/llm_cost_tracker/parsers/anthropic.rb +4 -5
  47. data/lib/llm_cost_tracker/parsers/base.rb +3 -8
  48. data/lib/llm_cost_tracker/parsers/gemini.rb +3 -3
  49. data/lib/llm_cost_tracker/parsers/openai_usage.rb +3 -3
  50. data/lib/llm_cost_tracker/parsers/registry.rb +5 -12
  51. data/lib/llm_cost_tracker/period_grouping.rb +68 -0
  52. data/lib/llm_cost_tracker/price_registry.rb +22 -30
  53. data/lib/llm_cost_tracker/pricing.rb +10 -19
  54. data/lib/llm_cost_tracker/report.rb +4 -4
  55. data/lib/llm_cost_tracker/report_data.rb +23 -29
  56. data/lib/llm_cost_tracker/report_formatter.rb +11 -3
  57. data/lib/llm_cost_tracker/storage/active_record_store.rb +1 -3
  58. data/lib/llm_cost_tracker/tag_accessors.rb +0 -8
  59. data/lib/llm_cost_tracker/tag_key.rb +16 -0
  60. data/lib/llm_cost_tracker/tracker.rb +35 -1
  61. data/lib/llm_cost_tracker/unknown_pricing.rb +1 -1
  62. data/lib/llm_cost_tracker/version.rb +1 -1
  63. data/lib/llm_cost_tracker.rb +3 -6
  64. data/llm_cost_tracker.gemspec +13 -9
  65. metadata +92 -21
  66. data/.rubocop.yml +0 -44
  67. data/lib/llm_cost_tracker/storage/active_record_backend.rb +0 -19
  68. data/lib/llm_cost_tracker/storage/backends.rb +0 -26
  69. data/lib/llm_cost_tracker/storage/custom_backend.rb +0 -16
  70. data/lib/llm_cost_tracker/storage/log_backend.rb +0 -28
  71. data/lib/llm_cost_tracker/value_object.rb +0 -45
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 0d1192ed209333057bd2522173d05530b4f45c6bb63242189c75354a83b5a746
4
- data.tar.gz: 486555221b66a0da6cb867d207fb76349f0d56a23da623418da35aab7672875f
3
+ metadata.gz: fc82847cd3008ce0d7015928be7a594a595cf1f4e6eea88380532e5ac3f2e6e4
4
+ data.tar.gz: aa47d647351e14b84b170f148e53f2aa2ad406543b889ed5e4bc6bbf5037b4ca
5
5
  SHA512:
6
- metadata.gz: 8e74531effe3fc425de0384c13c7717c54b8be8d683493c92665b356ed8142d2629c9ce555f5fff703c2cd4a676d69e82efb39de767fc75fdbdcabdca9289f2c
7
- data.tar.gz: 38e9744e157248e67bebcdd818b356c06b923d75a216b406f96f0b1b10368d4a118cbb9593461e2a19d19bdb5b10fc115c2f871d15fcf8669e656ad8ea8e034a
6
+ metadata.gz: 945fe190de9c123a65a7b2a2692fb74090adc0ed461c3b8265d40d8b1da61bb48934b792bb6069029fa96294b87c43dcc924250b4181c3c7bb665a8617ed302e
7
+ data.tar.gz: 4c6843af269557a262144de52a003c9f1c79e71427f5000b95c49066a5ecb54b54d00b2fa8096cb82361d8eb24da3925d81852aba981565d601f5a4f827ae86b
data/CHANGELOG.md CHANGED
@@ -1,117 +1,100 @@
1
1
  # Changelog
2
2
 
3
- All notable changes to this project will be documented in this file.
3
+ Format: [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). Versioning: [SemVer](https://semver.org/spec/v2.0.0.html).
4
4
 
5
- The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
6
- and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
5
+ ## [0.2.0.alpha1] - 2026-04-20
7
6
 
8
- ## [0.1.3] - 2026-04-18
7
+ ### Breaking
8
+
9
+ - Require Ruby 3.3+ (was 3.1), Rails/ActiveRecord 7.1+ (was 7.0), Faraday 2.0+ (was 1.0).
10
+ - `Event`, `Cost`, and `ParsedUsage` are plain `Data.define` value objects; use method access (`event.cost.total_cost`) instead of Hash lookups. `ActiveSupport::Notifications` payloads are unchanged.
11
+ - Rename `LlmCostTracker::InvalidFilter` → `InvalidFilterError`.
12
+ - Drop `LlmApiCall.by_provider` / `by_model` scopes — use `where(provider:)` / `where(model:)`.
13
+ - `ReportData` no longer hardcodes a `"feature"` tag breakdown. Configure `config.report_tag_breakdowns = %w[feature env]` (or pass `tag_breakdowns:` to `ReportData.build` / `Report.generate`). Default is empty.
14
+
15
+ ### Added
9
16
 
10
- ### Thread-safety, pricing UX, and internal hardening
17
+ - `LlmApiCall.group_by_period(:day/:month)` SQL-side period grouping.
18
+ - Opt-in `LlmCostTracker::Engine` dashboard (Rails 7.1+): overview with delta-vs-previous-period, provider rollup, models, filterable call list with CSV export and outlier sort modes, call details, tag key explorer, per-key tag breakdown, data quality. PostgreSQL/SQLite use adapter-specific SQL; MySQL falls back to an in-Ruby scan capped at 50k rows. Core middleware still works without Rails.
11
19
 
12
- **Thread-safety**
20
+ ## [0.1.4] - 2026-04-18
13
21
 
14
- - Guard `PriceRegistry.file_prices` and `Pricing.sorted_price_keys` memoization with mutexes.
22
+ ### Breaking
23
+
24
+ - Drop `LlmApiCall.by_user` / `by_feature` scopes and `LlmApiCall#user_id` / `#feature` accessors. Use `by_tag("user_id", id)` / `by_tag("feature", name)` or `by_tags(...)`; read stored tags via `parsed_tags[...]`.
25
+ - Drop `ReportData#cost_by_feature` — use `cost_by_tags.fetch("feature")` or `LlmApiCall.cost_by_tag("feature")`.
26
+
27
+ ### Added
15
28
 
16
- **Pricing UX**
29
+ - `group_by_tag(key)` / `cost_by_tag(key)` SQL aggregations across any tag key.
30
+ - Generic tag breakdowns in reports.
17
31
 
32
+ ## [0.1.3] - 2026-04-18
33
+
34
+ ### Fixed / Changed
35
+
36
+ - Mutex-guard `PriceRegistry.file_prices` and `Pricing.sorted_price_keys` memoization.
18
37
  - Warn on unknown keys in local prices files.
19
- - Add `llm_cost_tracker:prices` generator for creating a local price override template.
20
- - Document that budget enforcement skips events with unknown pricing.
38
+ - Document that budget guardrails skip events with unknown pricing.
21
39
 
22
- **Onboarding UX**
40
+ ### Added
23
41
 
24
- - Add callable Faraday `tags:` support for per-request Rails attribution with `Current`.
25
- - Add `llm_cost_tracker:report` rake task for a quick terminal cost report.
26
- - Rework README with a no-database quick try, report output, and safety guarantees.
42
+ - `llm_cost_tracker:prices` generator for a local price override template.
43
+ - Callable Faraday `tags:` for per-request Rails attribution via `Current`.
44
+ - `llm_cost_tracker:report` rake task.
27
45
 
28
- **Internal refactor (no behavior change)**
46
+ ### Internal
29
47
 
30
- - Extract `Logging` module and remove duplicated warning helpers.
31
- - Extract `TagQuery`, `TagsColumn`, and `TagAccessors` helpers from `LlmApiCall`.
32
- - Introduce typed `Cost`, `Event`, and `ParsedUsage` value objects while preserving hash-like access.
33
- - Move storage dispatch into dedicated backend objects with a uniform save contract.
34
- - Split `Report` into `ReportData` and `ReportFormatter`.
35
- - Use `OpenaiUsage` composition for OpenAI-compatible providers instead of parser inheritance.
36
- - Move config enum validation into `Configuration` setters.
37
- - Memoize the merged built-in/file/override prices table.
38
- - Restrict the Gemini parser to `generateContent` and `streamGenerateContent` paths.
48
+ - Extract `Logging`, `TagQuery`, `TagsColumn`, `TagAccessors` helpers; `Cost`, `Event`, `ParsedUsage` value objects; storage backend objects; split `Report` into data + formatter; `OpenaiUsage` composition for OpenAI-compatible providers; move enum validation into `Configuration`; memoize merged prices table; restrict Gemini parser to `generateContent` / `streamGenerateContent`.
39
49
 
40
50
  ## [0.1.2] - 2026-04-18
41
51
 
42
52
  ### Added
43
53
 
44
- - Auto-detect OpenRouter and DeepSeek as OpenAI-compatible providers.
45
- - Add `openai_compatible_providers` configuration for private OpenAI-compatible gateways.
46
- - Add `BudgetExceededError` and `budget_exceeded_behavior` for best-effort budget guardrails.
47
- - Add `:raise` and `:block_requests` budget behaviors; `:block_requests` is not a hard cap under concurrency.
48
- - Add `StorageError` and `storage_error_behavior` so storage failures do not have to break host LLM calls.
49
- - Add `UnknownPricingError` and `unknown_pricing_behavior` for unknown model pricing.
50
- - Add built-in `prices.json` registry with metadata and source URLs.
51
- - Add `prices_file` configuration for local JSON/YAML pricing overrides.
52
- - Add `with_cost`, `without_cost`, and `unknown_pricing` ActiveRecord scopes.
53
- - Add `latency_ms` tracking for Faraday calls, manual tracking, notifications, and ActiveRecord storage.
54
- - Add `with_latency`, `average_latency_ms`, `latency_by_model`, and `latency_by_provider`.
55
- - Use PostgreSQL `jsonb` storage for tags in newly generated migrations.
56
- - Add a GIN index on `llm_api_calls.tags` for PostgreSQL installs.
57
- - Add adapter-aware `by_tag` querying with JSONB containment on PostgreSQL and text fallback elsewhere.
58
- - Add `by_tags`, `by_user`, and `by_feature` scopes for common attribution queries.
59
- - Add `llm_cost_tracker:upgrade_tags_to_jsonb` generator for existing PostgreSQL installs.
60
- - Add `llm_cost_tracker:upgrade_cost_precision` generator for widening stored cost columns.
61
- - Add `llm_cost_tracker:add_latency_ms` generator for existing installs.
54
+ - Auto-detect OpenRouter and DeepSeek as OpenAI-compatible.
55
+ - `openai_compatible_providers` config for private gateways.
56
+ - `BudgetExceededError` + `budget_exceeded_behavior` (`:notify`, `:raise`, `:block_requests`). `:block_requests` is best-effort under concurrency.
57
+ - `StorageError` + `storage_error_behavior`; `UnknownPricingError` + `unknown_pricing_behavior`.
58
+ - Built-in `prices.json` registry with metadata and source URLs; `prices_file` for local JSON/YAML overrides.
59
+ - `with_cost`, `without_cost`, `unknown_pricing` scopes.
60
+ - `latency_ms` tracking end-to-end; `with_latency`, `average_latency_ms`, `latency_by_model`, `latency_by_provider`.
61
+ - `jsonb` tags + GIN index on PostgreSQL in new migrations; adapter-aware `by_tag` (JSONB containment on PG, text fallback elsewhere); `by_tags` / `by_user` / `by_feature`.
62
+ - Generators: `upgrade_tags_to_jsonb`, `upgrade_cost_precision`, `add_latency_ms`.
62
63
 
63
64
  ### Changed
64
65
 
65
- - Store tags as a Hash for JSON-backed columns and as JSON text for fallback columns.
66
- - Keep internal usage metadata such as cache token counts out of stored attribution tags.
67
- - Normalize provider-prefixed model IDs like `openai/gpt-4o-mini` for built-in price lookup.
68
- - Normalize configured OpenAI-compatible host keys to lowercase after configuration.
69
- - Avoid double fuzzy-match passes during price lookup.
70
- - Widen generated cost decimal columns to `precision: 20, scale: 8`.
71
- - Count Gemini `thoughtsTokenCount` as output tokens for better thinking-mode cost estimates.
72
- - Warn when Faraday exposes an unreadable streaming/SSE response body.
73
- - Document tag storage behavior, budget guardrail limits, known limitations, common tag scopes, and upgrade flows.
74
- - Clarify that budget errors raised after a response occur after the event has been recorded.
75
- - Route custom storage exceptions that inherit from `LlmCostTracker::Error` through `storage_error_behavior`.
66
+ - Tags stored as Hash for JSON-backed columns, JSON text for fallback.
67
+ - Normalize provider-prefixed model IDs (e.g. `openai/gpt-4o-mini`) for price lookup.
68
+ - Widen generated cost columns to `precision: 20, scale: 8`.
69
+ - Count Gemini `thoughtsTokenCount` as output tokens.
70
+ - Warn on unreadable streaming/SSE response bodies.
71
+ - Route storage exceptions inheriting from `LlmCostTracker::Error` through `storage_error_behavior`.
76
72
 
77
73
  ## [0.1.1] - 2026-04-17
78
74
 
79
75
  ### Fixed
80
76
 
81
- - Lazy-load ActiveRecord storage so `storage_backend = :active_record` persists events reliably.
82
- - Avoid double-counting the latest ActiveRecord event in monthly budget callbacks.
83
- - Track OpenAI Responses API usage via `/v1/responses`.
84
- - Parse OpenAI cached input token details for cache-aware cost estimates.
85
- - Parse Anthropic cache read and cache creation token usage under canonical metadata keys.
86
- - Parse Gemini cached content token usage when present.
87
- - Store ActiveRecord tag values as strings so `by_tag("user_id", "42")` works for numeric IDs.
77
+ - Lazy-load ActiveRecord storage so `:active_record` persists events reliably.
78
+ - Stop double-counting the latest event in monthly budget callbacks.
79
+ - Track OpenAI Responses API (`/v1/responses`).
80
+ - Parse cached/cache-read/cache-creation token details across OpenAI, Anthropic, Gemini.
81
+ - Store tag values as strings so `by_tag("user_id", "42")` matches numeric IDs.
88
82
 
89
83
  ### Changed
90
84
 
91
- - Refresh built-in pricing for current OpenAI, Anthropic, and Gemini models.
92
- - Add cache-aware cost calculation fields for cached input, cache reads, and cache creation.
93
- - Tighten OpenAI URL matching to supported endpoint families only.
94
- - Reposition README around self-hosted Rails/Ruby cost tracking for Faraday-based clients.
85
+ - Refresh built-in pricing for current OpenAI, Anthropic, Gemini models.
86
+ - Cache-aware cost fields (cached input, cache reads, cache creation).
87
+ - Tighten OpenAI URL matching to supported endpoint families.
95
88
 
96
89
  ### Added
97
90
 
98
- - Add ActiveRecord integration specs for persistence, tag querying, and budget callbacks.
99
- - Add RuboCop configuration, rake task, and CI lint step.
100
- - Require MFA metadata for RubyGems publishing.
91
+ - ActiveRecord integration specs; RuboCop config + CI lint step; RubyGems MFA metadata.
101
92
 
102
93
  ## [0.1.0] - 2026-04-16
103
94
 
104
- ### Added
105
-
106
- - Faraday middleware for automatic LLM API call interception
107
- - Provider parsers: OpenAI, Anthropic, Google Gemini
108
- - Built-in pricing table for 20+ models
109
- - Fuzzy model name matching (e.g. `gpt-4o-2024-08-06` `gpt-4o`)
110
- - ActiveSupport::Notifications integration
111
- - ActiveRecord storage backend with scopes and aggregations
112
- - Manual `LlmCostTracker.track()` for non-Faraday clients
113
- - Per-user / per-feature tagging
114
- - Monthly budget alerts with configurable callbacks
115
- - Rails generator: `rails generate llm_cost_tracker:install`
116
- - Custom storage backend support
117
- - Pricing overrides via configuration
95
+ - Faraday middleware for LLM call interception.
96
+ - Parsers: OpenAI, Anthropic, Gemini. Built-in pricing for 20+ models with fuzzy matching.
97
+ - `ActiveSupport::Notifications` integration; ActiveRecord backend with scopes and aggregations.
98
+ - Manual `LlmCostTracker.track(...)` for non-Faraday clients.
99
+ - Per-user / per-feature tagging; monthly budget alerts with configurable callbacks.
100
+ - `rails generate llm_cost_tracker:install`; custom storage backend; pricing overrides.
data/PLAN_0.2.md ADDED
@@ -0,0 +1,488 @@
1
+ # llm_cost_tracker v0.2.0 Plan
2
+
3
+ ## Goal
4
+
5
+ Ship an opt-in Rails dashboard engine:
6
+
7
+ ```ruby
8
+ require "llm_cost_tracker/engine"
9
+
10
+ mount LlmCostTracker::Engine => "/llm-costs"
11
+ ```
12
+
13
+ The dashboard is read-only and shows where a Rails app spends money on LLM APIs:
14
+
15
+ - spend over time
16
+ - top models
17
+ - recent calls
18
+ - call details
19
+ - tag breakdowns such as `feature`
20
+ - optional monthly budget status
21
+
22
+ The first milestone is a useful Overview page that can be used as the README screenshot.
23
+
24
+ ## Non-Goals
25
+
26
+ These are intentionally out of scope for v0.2.0:
27
+
28
+ - built-in authorization
29
+ - CSV or JSON export
30
+ - alerts, webhooks, or Slack notifications
31
+ - JavaScript charts or live updates
32
+ - editing settings from the dashboard
33
+ - persistent budget models
34
+ - `/budgets`
35
+ - `/tags` index or automatic tag-key discovery
36
+ - timezone-aware SQL bucketing
37
+ - hourly period grouping
38
+ - SaaS or remote sync
39
+
40
+ ## Compatibility Decisions
41
+
42
+ ### Core Gem
43
+
44
+ The core gem remains lightweight and usable outside Rails:
45
+
46
+ - keep `activesupport >= 7.0, < 9.0`
47
+ - do not add `railties` as a runtime dependency
48
+ - keep middleware-only usage working for plain Faraday, Sinatra, Hanami, and service objects
49
+
50
+ ### Dashboard Engine
51
+
52
+ The Engine is Rails-only and opt-in:
53
+
54
+ - users must `require "llm_cost_tracker/engine"`
55
+ - Engine requires Rails 7.1+
56
+ - the Rails version guard must run at top level in `engine.rb`, before defining `class Engine < ::Rails::Engine`
57
+ - `railties` is a development/test dependency only
58
+
59
+ Example guard:
60
+
61
+ ```ruby
62
+ unless Gem::Version.new(Rails.version) >= Gem::Version.new("7.1.0")
63
+ raise LlmCostTracker::Error, "LlmCostTracker::Engine requires Rails 7.1+"
64
+ end
65
+ ```
66
+
67
+ ## Phase 1: Public Query Primitive
68
+
69
+ Add SQL-side period grouping to `LlmCostTracker::LlmApiCall`.
70
+
71
+ ### API
72
+
73
+ ```ruby
74
+ LlmCostTracker::LlmApiCall.group_by_period(:day).sum(:total_cost)
75
+ LlmCostTracker::LlmApiCall.group_by_period(:month).count
76
+ LlmCostTracker::LlmApiCall.group_by_period(:day, column: :created_at)
77
+ ```
78
+
79
+ Default column: `:tracked_at`.
80
+
81
+ ### Supported Periods
82
+
83
+ Only:
84
+
85
+ - `:day`
86
+ - `:month`
87
+
88
+ Do not add `:hour` or `:week` in v0.2.0.
89
+
90
+ ### Return Keys
91
+
92
+ Return string keys across all adapters:
93
+
94
+ ```ruby
95
+ :day # "2026-04-18"
96
+ :month # "2026-04"
97
+ ```
98
+
99
+ ### SQL Strategy
100
+
101
+ - PostgreSQL: `TO_CHAR(DATE_TRUNC(...), ...)`
102
+ - MySQL: `DATE_FORMAT(...)`
103
+ - SQLite: `strftime(...)`
104
+
105
+ ### Validation
106
+
107
+ - period must be whitelisted
108
+ - column must exist in `column_names`
109
+ - invalid period or column raises `ArgumentError`
110
+ - no `time_zone:` parameter in v0.2.0
111
+
112
+ ### Tests
113
+
114
+ - real SQLite grouping specs
115
+ - generated SQL expression specs for PostgreSQL/MySQL if those adapters are not in CI
116
+ - injection tests for period and column
117
+ - composition test:
118
+
119
+ ```ruby
120
+ LlmCostTracker::LlmApiCall
121
+ .this_month
122
+ .where(provider: "openai")
123
+ .group_by_period(:day)
124
+ .sum(:total_cost)
125
+ ```
126
+
127
+ ## Phase 2: Engine Skeleton
128
+
129
+ Add the minimal Rails Engine structure:
130
+
131
+ ```text
132
+ lib/llm_cost_tracker/engine.rb
133
+ app/controllers/llm_cost_tracker/application_controller.rb
134
+ app/controllers/llm_cost_tracker/dashboard_controller.rb
135
+ app/views/layouts/llm_cost_tracker/application.html.erb
136
+ config/routes.rb
137
+ spec/dummy/
138
+ ```
139
+
140
+ ### Requirements
141
+
142
+ - isolated namespace: `isolate_namespace LlmCostTracker`
143
+ - no automatic loading for non-Rails users
144
+ - no asset pipeline dependency
145
+ - no JavaScript
146
+ - inline CSS in the Engine layout
147
+ - all CSS classes prefixed with `.lct-`
148
+ - minimal `spec/dummy` Rails app for Engine request specs
149
+
150
+ ### Acceptance
151
+
152
+ - dummy app mounts the Engine at `/llm-costs`
153
+ - `GET /llm-costs` returns 200
154
+ - empty database shows an intentional empty state
155
+ - missing `llm_api_calls` table shows a friendly setup error
156
+
157
+ ## Phase 3: Dashboard Data Layer
158
+
159
+ Use plain Ruby objects under:
160
+
161
+ ```text
162
+ app/services/llm_cost_tracker/dashboard/
163
+ ```
164
+
165
+ ### Filter
166
+
167
+ Parses dashboard params and returns an ActiveRecord relation.
168
+
169
+ Supported params:
170
+
171
+ - `from`
172
+ - `to`
173
+ - `provider`
174
+ - `model`
175
+ - `tag[key]=value`
176
+
177
+ Rules:
178
+
179
+ - parse `from` and `to` with `Date.iso8601`
180
+ - ignore invalid dates
181
+ - use AR placeholders for provider/model
182
+ - parse all `params[:tag]` keys as a hash
183
+ - pass multi-key tag filters to `by_tags`
184
+ - validate tag keys with the same whitelist as `group_by_tag`
185
+
186
+ Example:
187
+
188
+ ```text
189
+ ?tag[feature]=chat&tag[user_id]=42
190
+ ```
191
+
192
+ becomes:
193
+
194
+ ```ruby
195
+ scope.by_tags("feature" => "chat", "user_id" => "42")
196
+ ```
197
+
198
+ ### Page
199
+
200
+ Use a small immutable object, not a large service:
201
+
202
+ ```ruby
203
+ Dashboard::Page = Data.define(:page, :per)
204
+ ```
205
+
206
+ Rules:
207
+
208
+ - `page` minimum: 1
209
+ - `per` default: 50
210
+ - `per` maximum: 200
211
+ - expose `limit`, `offset`, `prev_page?`, and `next_page?`
212
+
213
+ ### OverviewStats
214
+
215
+ Compute:
216
+
217
+ - total spend
218
+ - total calls
219
+ - average cost per call
220
+ - average latency only if `latency_ms` exists
221
+ - monthly budget status only if `LlmCostTracker.configuration.monthly_budget` is set
222
+
223
+ Do not compute `known_pricing_rate` in v0.2.0.
224
+
225
+ ### TimeSeries
226
+
227
+ Uses `group_by_period(:day).sum(:total_cost)`.
228
+
229
+ Rules:
230
+
231
+ - default range: last 30 days
232
+ - fill missing days with zero
233
+ - output array:
234
+
235
+ ```ruby
236
+ [{ label: "2026-04-01", cost: 0.0 }]
237
+ ```
238
+
239
+ ### TopModels
240
+
241
+ Compute:
242
+
243
+ - provider
244
+ - model
245
+ - calls
246
+ - total cost
247
+ - average cost per call
248
+ - input tokens
249
+ - output tokens
250
+ - average latency if available
251
+
252
+ ### TopTags
253
+
254
+ Only for configured keys.
255
+
256
+ Default keys:
257
+
258
+ ```ruby
259
+ ["feature"]
260
+ ```
261
+
262
+ No automatic tag-key discovery in v0.2.0.
263
+
264
+ ## Phase 4: Dashboard Pages
265
+
266
+ ### Overview: `GET /`
267
+
268
+ The main screenshot-worthy page.
269
+
270
+ Show:
271
+
272
+ - total spend
273
+ - total calls
274
+ - average cost per call
275
+ - average latency if available
276
+ - monthly budget status if configured
277
+ - daily spend table with CSS bars
278
+ - top 5 models
279
+ - cost by `feature` tag if data exists
280
+
281
+ Budget wording:
282
+
283
+ > Soft monthly limit. Blocking is not atomic under concurrency.
284
+
285
+ ### Calls Index: `GET /calls`
286
+
287
+ Filters:
288
+
289
+ - `from`
290
+ - `to`
291
+ - `provider`
292
+ - `model`
293
+ - `tag[key]=value`
294
+ - `page`
295
+ - `per`
296
+
297
+ Columns:
298
+
299
+ - created_at
300
+ - provider
301
+ - model
302
+ - input tokens
303
+ - output tokens
304
+ - cache tokens if present
305
+ - total cost
306
+ - latency if available
307
+ - tags summary
308
+ - details link
309
+
310
+ Rules:
311
+
312
+ - newest first
313
+ - max `per=200`
314
+ - one count query and one paginated query
315
+ - no N+1 behavior
316
+
317
+ ### Call Details: `GET /calls/:id`
318
+
319
+ Show:
320
+
321
+ - all stored columns
322
+ - token breakdown
323
+ - cost breakdown
324
+ - latency if available
325
+ - tags as pretty JSON
326
+ - metadata if present
327
+
328
+ Missing records render a friendly 404 inside the Engine layout.
329
+
330
+ ### Models: `GET /models`
331
+
332
+ Show rows grouped by provider/model:
333
+
334
+ - provider
335
+ - model
336
+ - calls
337
+ - total cost
338
+ - average cost per call
339
+ - input tokens
340
+ - output tokens
341
+ - average latency if available
342
+
343
+ Default sort: total cost descending.
344
+
345
+ ### Tag Breakdown: `GET /tags/:key`
346
+
347
+ Validate `key` using the same tag-key whitelist as `group_by_tag`.
348
+
349
+ Show:
350
+
351
+ - tag value
352
+ - calls
353
+ - total cost
354
+ - average cost per call
355
+
356
+ Invalid keys render a friendly 400.
357
+
358
+ Do not add `/tags` index in v0.2.0.
359
+
360
+ ## Graceful Degradation
361
+
362
+ The dashboard must not explode when optional pieces are missing.
363
+
364
+ Handle:
365
+
366
+ - empty database
367
+ - missing `llm_api_calls` table
368
+ - missing `latency_ms`
369
+ - text `tags` fallback instead of JSONB
370
+ - malformed legacy tag JSON
371
+ - unknown-pricing rows with `total_cost = nil`
372
+ - no configured monthly budget
373
+
374
+ ## Security
375
+
376
+ - dashboard is read-only
377
+ - routes are GET-only
378
+ - keep `protect_from_forgery with: :exception`
379
+ - no built-in auth
380
+ - README must show Basic Auth and Devise examples
381
+ - validate every param through `Dashboard::Filter`
382
+ - do not log full tags or request details from the Engine
383
+
384
+ README warning:
385
+
386
+ > Do not expose this dashboard publicly. Tags may contain internal user, tenant, or feature identifiers.
387
+
388
+ ## Styling
389
+
390
+ - ERB templates
391
+ - no JS
392
+ - no external fonts
393
+ - no external CSS
394
+ - no Chart.js
395
+ - no Tailwind dependency
396
+ - inline `<style>` in layout
397
+ - `.lct-` class prefix
398
+ - CSS bar charts through inline custom properties
399
+
400
+ Example:
401
+
402
+ ```html
403
+ <div class="lct-bar" style="--lct-width: 73%"></div>
404
+ ```
405
+
406
+ ## Testing Plan
407
+
408
+ ### Core Specs
409
+
410
+ - `group_by_period(:day)`
411
+ - `group_by_period(:month)`
412
+ - invalid period
413
+ - invalid column
414
+ - composition with existing scopes
415
+
416
+ ### Engine Request Specs
417
+
418
+ - `/` empty DB
419
+ - `/` seeded DB
420
+ - `/calls`
421
+ - `/calls` filters narrow results
422
+ - `/calls/:id`
423
+ - `/calls/:missing`
424
+ - `/models`
425
+ - `/tags/feature`
426
+ - invalid tag key returns 400
427
+ - missing table renders setup error
428
+
429
+ ### Service Specs
430
+
431
+ - `Dashboard::Filter`
432
+ - `Dashboard::Page`
433
+ - `Dashboard::TimeSeries`
434
+ - `Dashboard::OverviewStats`
435
+ - `Dashboard::TopModels`
436
+ - `Dashboard::TopTags`
437
+
438
+ ### CI
439
+
440
+ - Ruby 3.3, 3.4
441
+ - Rails Engine specs on Rails 7.1, 7.2, 8.0
442
+ - SQLite default
443
+ - PostgreSQL job for period grouping and tag grouping
444
+ - MySQL best-effort only; not required for v0.2.0 CI
445
+
446
+ ## Documentation
447
+
448
+ README additions:
449
+
450
+ - Dashboard quick start
451
+ - mount snippet
452
+ - Basic Auth snippet
453
+ - Devise snippet
454
+ - warning not to expose dashboard publicly
455
+ - note that Engine requires Rails 7.1+
456
+ - note that core middleware remains usable without Rails
457
+
458
+ ## Release Strategy
459
+
460
+ 1. Work on `codex/engine-dashboard`.
461
+ 2. Implement `group_by_period`.
462
+ 3. Add Engine skeleton.
463
+ 4. Build Overview first.
464
+ 5. Build Calls index.
465
+ 6. Build Call details.
466
+ 7. Build Models.
467
+ 8. Build `/tags/:key`.
468
+ 9. Dogfood in a real Rails app.
469
+ 10. Release `0.2.0.alpha1`.
470
+ 11. Fix issues from real usage.
471
+ 12. Release `0.2.0.rc1`.
472
+ 13. Release final `0.2.0`.
473
+
474
+ ## v0.2.0 Acceptance Criteria
475
+
476
+ The release is ready when:
477
+
478
+ - `group_by_period(:day/:month)` is tested and documented
479
+ - Engine is opt-in and requires Rails 7.1+
480
+ - non-Rails core usage does not gain Rails runtime dependencies
481
+ - `/llm-costs` renders without JS or asset pipeline dependencies
482
+ - empty DB and missing table states are friendly
483
+ - seeded dashboard shows useful spend data
484
+ - filters cannot SQL-inject
485
+ - Overview is good enough for README screenshot
486
+ - README includes auth snippets
487
+ - full test suite and RuboCop pass
488
+ - alpha has been tested in one real Rails app