llm_cost_tracker 0.13.0 → 0.14.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: ec6d42366ce33ab5b2d4847d4ad425dd4d4262a302212427be2a3fc55db53fcc
-  data.tar.gz: 57f6eaa22546be19cc3b17afeed107fb35782ad5de4b6ed9646a4d7955cab589
+  metadata.gz: ffc5fef0c3c18c6926b6c414d56fb0b7c9c4afef6db315139f8bc4626864611a
+  data.tar.gz: 3b012c1014c7246dd0eebb5413e13c02023cce056312e719a96328e86de749a8
 SHA512:
-  metadata.gz: 716c326903ac571988211ee34f2ff2a43d6cec3d6b8d00a1682df5d245877e7884f8b3e3b3e9d86e37b0089a73f97f3dc1b4c7cbbf1c40a1365702acef7e5f6b
-  data.tar.gz: ef41e457bafea063396e7ab08d2c7f6c051c1831ec7476071e36b08af7812853c7e1501fc4d7986658de726316c136e562b747eec26316ffd97a7ee6ee7eda2a
+  metadata.gz: b28d7d695a1d6c45fe6c9d77d4f5caebbbde84682b7a261e25123d280181d00361baae80e9ddd65def0b9602c385d48813467ae741a5cde4bc378e4652a45829
+  data.tar.gz: 617cafb3be05250fd7c9c0b049a37474e1e0a2737d72cfd2744110cd531b68c362e7d15928f54d9a9c2d810da3f451516ff9ae747103bca2d618b77159359a13

data/CHANGELOG.md

@@ -2,7 +2,54 @@
 
 Format: [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). Versioning: [SemVer](https://semver.org/spec/v2.0.0.html).
 
-## [Unreleased]
+## [0.14.0] - 2026-08-26
+
+### Added
+
+- `config.budgets.per_tag` applies one budget to every distinct value of each declared tag — `{ tenant_id: { monthly: 1000 }, user_id: { daily: 25 } }` gives every tenant its own 1000 a month and every user its own 25 a day, for as many tags as you declare. Windows are `daily`, `weekly`, and `monthly`; a rule can set its own `behavior` and `on_exceeded` or fall back to the global ones, and the payload names the tag and value that crossed. A fresh install is ready for it; an install created before v0.14 runs `bin/rails generate llm_cost_tracker:upgrade_per_tag_budgets`, then `bin/rails llm_cost_tracker:backfill_tag_costs` to count spend recorded before the upgrade.
+- The Data Quality page names any `config.budgets.per_tag` tag that no recorded call carries, so a mistyped tag name shows up as a budget that can never fire instead of silently enforcing nothing. The list clears itself as soon as a call arrives with that tag, and stays hidden until something has been tagged at all.
+- Gemini calls that ground against Google Search are costed. The rate comes from each model's own published `grounding_request` price — $35 per 1,000 on Gemini 2.x, $14 on 3.x — so a grounded call lands `complete` instead of `unknown`. Google's free monthly allowance is account-level and is not modelled, so a project still inside it is over-reported.
+- OpenAI's duration-billed audio models — `gpt-transcribe`, `gpt-live-transcribe`, `gpt-realtime-whisper`, `gpt-realtime-translate` — are priced per minute of audio. They publish no token price, so these calls previously recorded no cost at all.
+- OpenAI gpt-5.6-sol, gpt-5.6-terra, and gpt-5.6-luna are priced, including their cache-write rates across standard, batch, flex, priority, long-context, and data-residency tiers.
+- Cache writes reported in OpenAI usage (`cache_write_tokens`, GPT-5.6 and later) are captured and costed at the model's cache-write rate instead of being counted as regular input.
+- Anthropic thinking tokens are counted as hidden output on the Data Quality page, so reasoning Claude already billed inside `output_tokens` is visible instead of reading as zero. Cost is unchanged — `output_tokens` stays the billable total.
+
+### Changed
+
+- BREAKING: a call the gem never found a rate for records `cost_status: unknown` instead of `free`. `free` covered both "priced at zero" and "never priced", so token-billed endpoints the parser captured no quantities for — `gpt-4o-mini-tts`, Whisper transcription, any unrecognised model — reported as costing nothing. Genuinely unbilled endpoints such as moderations move to `unknown` as well and appear on the Data Quality page.
+- BREAKING: `enforce_budget: true` on `LlmCostTracker.track` records the call before it raises, and the error carries `stage: :post_spend` instead of `:pre_send`. `track` reports a request the provider already served, so the old order threw away real spend — and with the ledger total never advancing, every later call raised and was dropped too. `LlmCostTracker.track_stream` still raises `:pre_send`, before your block runs.
+- BREAKING: `pricing_snapshot["rates"]` is keyed by the rate actually applied, so a batch call reads `batch_input` rather than `input`. The old key named a row whose value in the price table was a different number.
+- `bin/rails llm_cost_tracker:backfill_unknown_pricing` no longer scans the whole ledger on every batch — unpriced calls are found through a partial index. Existing installs pick this up with `bin/rails generate llm_cost_tracker:upgrade_indexes`, which also drops the ingestion inbox lock index the drain never uses.
+- `bin/rails llm_cost_tracker:prune` warns with the count and cost when it deletes inbox rows that never reached the ledger, instead of dropping that spend silently.
+- The unpriced-model warning names the tier as well, so a call at a pricing mode you have no rate for reads `model "gpt-5.5" at pricing_mode "scale"` instead of pointing at the model.
+- The models page caps at 200 rows and an out-of-range `page` no longer renders a database error.
+
+### Deprecated
+
+- Configuration options are grouped into `budgets`, `capture`, `ingestion`, `pricing`, and `tags` — `config.budgets.monthly` replaces `config.monthly_budget`, and so on. Flat names still work and warn with their replacement; they are removed in 1.0. See [docs/upgrading.md](docs/upgrading.md#v013--v014) for the full mapping.
+- `config.log_level` is dropped with no replacement — it never affected any log output.
+
+### Removed
+
+- Bundled prices for OpenAI models past their published shutdown date are dropped, so repricing a call to a retired model reports unknown pricing instead of a stale rate.
+
+### Fixed
+
+- OpenAI responses that break the completion into `text_tokens` are costed on the full billed output again. Reasoning tokens were dropped from the total, so a 1,000-token completion with 800 reasoning tokens recorded $0.011 instead of $0.035 while still reporting `cost_status: complete`.
+- Anthropic calls are priced from the `speed` the response reports rather than the one the request asked for, so a `fast` response no longer records at standard rates — half its real cost on `claude-opus-5`.
+- OpenAI regional calls on gpt-5.6 models are billed at their data-residency rates. Eligibility now comes from the price table instead of a hard-coded model list, which had not caught up with the codename models.
+- Apps that set `config.logger` to a plain `Logger` no longer take a `NoMethodError` from inside the tracker. Every warning went through `Rails.logger.tagged`, which those apps do not have, so a rollup or ingestion failure raised into the request instead of being logged, and the async worker thread died on its first warning.
+- Filtering the dashboard by a tag survives the date, provider, model and stream filters. Submitting any of them flattened `tag[env]=prod` into a single `tag` value, silently dropping the filter and showing a larger total under an unchanged header.
+- A call whose tag value is a large hash or array is recorded instead of failing the whole insert. The encoded value is capped to fit the tag index; the scalar cap alone did not bound a composite.
+- `Call.unknown_pricing` composes with `cost_by_tag` and `group_by_tag` again instead of raising on an ambiguous `total_cost`.
+- A rollup increment that may already have been applied is no longer retried, so a dropped connection cannot leave the cache — and every budget read — above the real spend.
+- The dashboard recovers on its own when the schema catches up. A process that started before `db:migrate` cached "Setup required" until it was restarted.
+- Async inbox writes work after a fork, instead of raising into the request for the life of the child process.
+- A completed OpenAI batch is retried when its result download fails, instead of being marked captured and never recorded.
+- `bin/rails llm_cost_tracker:doctor` reports drift in the async inbox and lease tables, not just their absence.
+- OpenAI gpt-5.4, gpt-5.4-pro, and gpt-5.5 prompts above 272K input tokens are costed at OpenAI's published long-context premium (2x input, 1.5x output on standard, batch, and flex) instead of the flat short-context rate.
+- `bin/rails llm_cost_tracker:backfill_unknown_pricing` no longer aborts on the default configuration; repricing calls with unknown pricing no longer requires opting into `config.budgets.totals_source = :cache`.
+- Setting `config.budgets.totals_source = :cache` without creating `llm_cost_tracker_call_rollups` no longer breaks dashboard and budget reads; totals fall back to aggregating the calls ledger and a log warning names the missing table.
 
 ## [0.13.0] - 2026-06-26
 
@@ -30,7 +77,7 @@ Format: [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). Versioning: [S
 
 ### Removed
 
-- BREAKING: the experimental `Reconciliation` subsystem (provider invoice import + diff, the `/reconciliation` dashboard page, `bin/rails llm_cost_tracker:reconcile:*` rake tasks, `config.reconciliation_enabled`, `config.reconciliation_importers`, the `llm_cost_tracker:reconciliation` generator, and the `llm_cost_tracker_provider_invoices` / `_provider_invoice_imports` tables) is gone. It was never finished and never billing-accurate. `calls.provider_response_id` (captured on every call) already covers invoice cross-reference; if invoice-vs-ledger reconciliation ships again it lives in a separate gem. Existing installs can drop the two tables — see [docs/upgrading.md](docs/upgrading.md#v011--v012-unreleased).
+- BREAKING: the experimental `Reconciliation` subsystem (provider invoice import + diff, the `/reconciliation` dashboard page, `bin/rails llm_cost_tracker:reconcile:*` rake tasks, `config.reconciliation_enabled`, `config.reconciliation_importers`, the `llm_cost_tracker:reconciliation` generator, and the `llm_cost_tracker_provider_invoices` / `_provider_invoice_imports` tables) is gone. It was never finished and never billing-accurate. `calls.provider_response_id` (captured on every call) already covers invoice cross-reference; if invoice-vs-ledger reconciliation ships again it lives in a separate gem. Existing installs can drop the two tables — see [docs/upgrading.md](docs/upgrading.md#v011--v012).
 - `config.instrument :gemnii` (or any other typo / unknown integration name) no longer raises at config time — it now logs `Logging.warn("Unknown integration: :gemnii. Known: ...")` once when integrations install, and `bin/rails llm_cost_tracker:doctor` shows the unknown name as a `:warn` row so the typo is visible without crashing boot.
 - Pre-call budget enforcement for Azure-hosted OpenAI calls now keys on `"azure_openai"` (matching the recorded `Call.provider`), so `pricing_overrides` for Azure rates actually gate the call. Previously it always keyed on `"openai"` regardless of the SDK client's `base_url`.
 - BREAKING: removed the `batch:` keyword argument from `LlmCostTracker.track`, `LlmCostTracker.track_stream`, and `stream.usage` (inside `track_stream` blocks). Signal a batch-tier call via `pricing_mode: :batch` (or any pricing_mode containing the `batch` token like `:batch_flex`) — that's the single source of truth now. Previously `batch:` and `pricing_mode:` could disagree, especially after request-side pricing_mode merge inside `Tracker.record` overwrote the parser's mode but left the stored `batch` flag stale, so `calls.batch` could read `true` while `calls.pricing_mode` read `flex` (or vice versa) for the same row.
@@ -40,7 +87,7 @@ Format: [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). Versioning: [S
 
 - The RubyLLM SDK integration now requires `ruby_llm >= 1.15.0` (was `>= 1.14.1`).
 - Engine no longer adds `tag` / `tag_value` to Rails `filter_parameters` — the Symbol filter was substring-matching unrelated host-app params (`tags`, `meta_tag`, etc.) into `[FILTERED]`. `Tags::Sanitizer` continues redacting secret-shaped tag values at storage.
-- BREAKING: the serialized event `cost` (the `llm_request.llm_cost_tracker` notification payload and the async-ingestion inbox payload) is now `{ components: {...}, total:, currency: }` (was flat with a top-level `total_cost:`). Notification subscribers should read `cost[:total]`; `ingestion: :async` rolling deploys should drain the inbox first — see [docs/upgrading.md](docs/upgrading.md#v011--v012-unreleased).
+- BREAKING: the serialized event `cost` (the `llm_request.llm_cost_tracker` notification payload and the async-ingestion inbox payload) is now `{ components: {...}, total:, currency: }` (was flat with a top-level `total_cost:`). Notification subscribers should read `cost[:total]`; `ingestion: :async` rolling deploys should drain the inbox first — see [docs/upgrading.md](docs/upgrading.md#v011--v012).
 - BREAKING: `pricing_mode` in the `llm_request.llm_cost_tracker` notification payload is now a String (e.g. `"batch"`, `"fast_data_residency"`), not a Symbol — subscribers matching it against a Symbol must compare to the String.
 - BREAKING: `LlmCostTracker.track(tokens:)` now takes the same `_tokens`-suffixed keys as `stream.usage` and the stored columns — `input_tokens`, `output_tokens`, `cache_read_input_tokens`, `audio_input_tokens`, etc. (was the short `input`, `output`, `cache_read_input`, …). Update manual `track` calls. Pricing-file / `pricing_overrides` field names are unchanged — they stay `input`, `output`, … (per-component rates, a separate vocabulary).
 
@@ -134,17 +181,11 @@ Format: [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). Versioning: [S
 
 ## [0.9.0] - 2026-05-12
 
-0.9 leans the default install: only `calls`, `call_line_items`, and `call_tags`
-are mandatory. Durable ingestion, rollup-cached budget reads, and provider
-invoice reconciliation are opt-in behind config flags and dedicated generators.
-Plus expanded SDK capture (OpenAI embeddings/audio/images/moderation, RubyLLM
-paint/moderate), correct handling of Anthropic data residency and Priority
-Tier, and a security-hardened dashboard. Existing installs need a migration —
-see [Upgrading](docs/upgrading.md).
+0.9 leans the default install: only `calls`, `call_line_items`, and `call_tags` are mandatory. Durable ingestion, rollup-cached budget reads, and provider invoice reconciliation are opt-in behind config flags and dedicated generators. Plus expanded SDK capture (OpenAI embeddings/audio/images/moderation, RubyLLM paint/moderate), correct handling of Anthropic data residency and Priority Tier, and a security-hardened dashboard. Existing installs need a migration — see [Upgrading](docs/upgrading.md).
 
 ### Added
 
-- **Experimental:** opt-in provider invoice reconciliation. Set `config.reconciliation_enabled = true` and run `bin/rails generate llm_cost_tracker:reconciliation`. Public surface: `LlmCostTracker::Reconciliation.import / .diff`, `config.register_reconciliation_importer(:source) { … }`, rake tasks `llm_cost_tracker:reconcile:import` and `:reconcile:diff`. Doctor warns when drift exceeds 5% or imports go stale past 14 days. See [Configuration](docs/configuration.md#reconciliation-experimental-opt-in).
+- **Experimental:** opt-in provider invoice reconciliation. Set `config.reconciliation_enabled = true` and run `bin/rails generate llm_cost_tracker:reconciliation`. Public surface: `LlmCostTracker::Reconciliation.import / .diff`, `config.register_reconciliation_importer(:source) { … }`, rake tasks `llm_cost_tracker:reconcile:import` and `:reconcile:diff`. Doctor warns when drift exceeds 5% or imports go stale past 14 days.
 - Dashboard Data Quality page now shows a "Streaming health by provider" breakdown (streams, with-usage, unknown, unknown share) so a misconfigured OpenAI-compatible host shipping streams without `stream_options.include_usage` is visible at a glance.
 - Dashboard tag detail page drills into a single value via `?tag_value=…` with total cost, call count, average per call, and a daily spend timeseries.
 - Bundled rates for OpenAI embeddings (`text-embedding-3-small` / `-3-large` / `-ada-002`, including 50% batch discount) and token-priced transcription (`gpt-4o-transcribe`, `gpt-4o-mini-transcribe`). Token-priced transcription splits audio and text inputs at their separate rates. DALL-E and Whisper still record as zero-token visibility events until their per-image / per-minute pricing components land.
@@ -240,11 +281,7 @@ see [Upgrading](docs/upgrading.md).
 
 ## [0.8.0] - 2026-05-07
 
-0.8 is a storage rebuild. Tokens and tool/runtime charges share one shape
-(`Billing::LineItem`) and live in a dedicated line items table. Per-component
-cost columns and the standalone service charges table are gone. Several tables
-were also renamed during the cycle. See [Upgrading](docs/upgrading.md) for the
-migration path — there is no rolling-deploy upgrade.
+0.8 is a storage rebuild. Tokens and tool/runtime charges share one shape (`Billing::LineItem`) and live in a dedicated line items table. Per-component cost columns and the standalone service charges table are gone. Several tables were also renamed during the cycle. See [Upgrading](docs/upgrading.md) for the migration path — there is no rolling-deploy upgrade.
 
 ### Added
 

data/README.md

@@ -2,53 +2,46 @@
 
 Self-hosted LLM cost tracking for Rails.
 
-[![Gem Version](https://img.shields.io/gem/v/llm_cost_tracker.svg)](https://rubygems.org/gems/llm_cost_tracker)
-[![CI](https://github.com/sergey-homenko/llm_cost_tracker/actions/workflows/ruby.yml/badge.svg)](https://github.com/sergey-homenko/llm_cost_tracker/actions)
-[![codecov](https://codecov.io/gh/sergey-homenko/llm_cost_tracker/branch/main/graph/badge.svg)](https://codecov.io/gh/sergey-homenko/llm_cost_tracker)
+[![Gem Version](https://img.shields.io/gem/v/llm_cost_tracker.svg)](https://rubygems.org/gems/llm_cost_tracker) [![CI](https://github.com/sergey-homenko/llm_cost_tracker/actions/workflows/ruby.yml/badge.svg)](https://github.com/sergey-homenko/llm_cost_tracker/actions) [![codecov](https://codecov.io/gh/sergey-homenko/llm_cost_tracker/branch/main/graph/badge.svg)](https://codecov.io/gh/sergey-homenko/llm_cost_tracker)
 
-Every call your app makes to OpenAI, Anthropic, Gemini, RubyLLM, or any
-OpenAI-compatible API gets logged: tokens, cost, latency, tags. Calls go
-app → provider direct. No proxy.
+Every call your app makes through RubyLLM, the official OpenAI and Anthropic SDKs, Gemini, or any OpenAI-compatible API gets logged: tokens, cost, latency, tags. Calls go app → provider direct. No proxy.
 
-Not Langfuse, Helicone, or LiteLLM. No prompts, no traces, no replay. Spend
-attribution only.
+Not Langfuse, Helicone, or LiteLLM. No prompts, no traces, no replay. Spend attribution only.
 
 Requires Ruby 3.4+, Rails 7.1+, PostgreSQL or MySQL.
 
-<picture>
-  <source media="(prefers-color-scheme: dark)" srcset="docs/dashboard-overview-dark.png">
-  <img alt="LLM Cost Tracker dashboard" src="docs/dashboard-overview-light.png">
-</picture>
+<picture> <source media="(prefers-color-scheme: dark)" srcset="docs/dashboard-overview-dark.png"> <img alt="LLM Cost Tracker dashboard" src="docs/dashboard-overview-light.png"> </picture>
 
 ## Quickstart
 
+Shown with RubyLLM; the flow is identical for the official OpenAI and Anthropic SDKs — swap the gem and the `instrument` name (see the [cookbook](docs/cookbook.md)).
+
 ```ruby
 # Gemfile
 gem "llm_cost_tracker"
-gem "openai"
+gem "ruby_llm"
 ```
 
 ```bash
 bin/rails llm_cost_tracker:setup
 ```
 
-Runs the install generator, drops a price snapshot, migrates the database, and verifies via `llm_cost_tracker:doctor`. The generated `config/initializers/llm_cost_tracker.rb` looks like:
+Runs the install generator, drops a price snapshot, migrates the database, and verifies via `llm_cost_tracker:doctor`. Then enable the integration in the generated `config/initializers/llm_cost_tracker.rb`:
 
 ```ruby
 LlmCostTracker.configure do |config|
-  config.default_tags = -> { { environment: Rails.env } }
-  config.instrument :openai
+  config.tags.default = -> { { environment: Rails.env } }
+  config.instrument :ruby_llm
 end
 ```
 
 Edit it in place to add tags, switch on async ingestion, etc.
 
-Tag your calls to attribute spend:
+Your RubyLLM calls stay unchanged — every chat, embedding, transcription, image, and moderation call now lands in the ledger. Tag them to attribute spend:
 
 ```ruby
 LlmCostTracker.with_tags(user_id: Current.user&.id, feature: "chat") do
-  client = OpenAI::Client.new(api_key: ENV["OPENAI_API_KEY"])
-  client.responses.create(model: "gpt-4o", input: "Hello")
+  RubyLLM.chat.ask("Hello")
 end
 ```
 
@@ -65,30 +58,25 @@ The engine ships without authentication on purpose.
 ## What lands in the ledger
 
 - **Calls.** Provider, model, total tokens, total cost, latency, status.
-- **Line items.** Per-component breakdown — text/audio/cached tokens, tool
-  charges (web search, code execution, grounding, container sessions).
+- **Line items.** Per-component breakdown — text/audio/cached tokens, tool charges (web search, code execution, grounding, container sessions).
 - **Tags.** Whatever attribution you pass — user, feature, tenant, env.
-- **Provider IDs.** Response, project, API key, workspace — for downstream
-  audits.
+- **Provider IDs.** Response, project, API key, workspace — for downstream audits.
 - **Pricing snapshot.** So historical numbers don't drift when prices change.
 
 ## Capture surfaces
 
 | Surface | Path |
 | --- | --- |
+| RubyLLM | Provider layer |
 | OpenAI | Official SDK or Faraday |
 | Anthropic | Official SDK or Faraday |
 | Azure OpenAI | Faraday or official SDK (auto-detected on `*.openai.azure.com` and Foundry `*.services.ai.azure.com`, both deployments and `/openai/v1/...`) |
 | Google Gemini | Faraday |
-| RubyLLM | Provider layer |
 | `ruby-openai` | Faraday |
 | OpenRouter, DeepSeek, Groq, LiteLLM-style gateways | OpenAI-compatible Faraday |
 | Anything else | `LlmCostTracker.track` |
 
-Streams capture when the provider emits final usage. OpenAI Faraday streams
-get `stream_options: { include_usage: true }` auto-injected so the final
-usage chunk lands in the ledger (opt out via
-`config.auto_enable_stream_usage = false`).
+Streams capture when the provider emits final usage. OpenAI Faraday streams to `/chat/completions` get `stream_options: { include_usage: true }` auto-injected so the final usage chunk lands in the ledger (opt out via `config.capture.request_stream_usage = false`).
 
 ## What it isn't
 

data/app/controllers/llm_cost_tracker/application_controller.rb

@@ -28,6 +28,8 @@ module LlmCostTracker
 
       @setup_message = drift.message
       @setup_details = drift.details
+      return head :service_unavailable unless request.format.html?
+
       render template: "llm_cost_tracker/shared/setup_required"
     end
 
@@ -37,8 +39,7 @@ module LlmCostTracker
       @to_date = range.to
     end
 
-    def render_database_error(error)
-      @error = error
+    def render_database_error(_error)
       render "llm_cost_tracker/errors/database", status: :internal_server_error
     end
 

data/app/controllers/llm_cost_tracker/data_quality_controller.rb

@@ -21,6 +21,7 @@ module LlmCostTracker
         total_streaming: @summary.streaming_count
       )
       @quarantined_inbox = Dashboard::DataQuality.quarantined_inbox
+      @unseen_budget_tags = Dashboard::DataQuality.unseen_budget_tags
     end
   end
 end

data/app/controllers/llm_cost_tracker/models_controller.rb

@@ -2,12 +2,14 @@
 
 module LlmCostTracker
   class ModelsController < ApplicationController
+    MAX_ROWS = 200
+
     def index
       @sort = params[:sort].to_s
       @dir = params[:dir].to_s
       @rows = Dashboard::TopModels.call(
         scope: Dashboard::Filter.call(params: params),
-        limit: nil,
+        limit: MAX_ROWS,
         sort: @sort,
         direction: @dir
       )

data/app/helpers/llm_cost_tracker/dashboard_query_helper.rb

@@ -16,6 +16,17 @@ module LlmCostTracker
       query
     end
 
+    def hidden_query_fields(query, prefix: nil)
+      safe_join(query.flat_map do |key, value|
+        name = prefix ? "#{prefix}[#{key}]" : key.to_s
+        case value
+        when Hash then hidden_query_fields(value, prefix: name)
+        when Array then value.map { |item| hidden_field_tag("#{name}[]", item, id: nil) }
+        else hidden_field_tag(name, value, id: nil)
+        end
+      end)
+    end
+
     private
 
     def clean_dashboard_query(value)

data/app/models/llm_cost_tracker/call.rb

@@ -10,7 +10,8 @@ module LlmCostTracker
     scope :without_cost, -> { where(total_cost: nil) }
     scope :unknown_pricing,
           lambda {
-            where(Charges::CostStatus.unknown_pricing_sql)
+            where(Charges::CostStatus.unknown_pricing_sql(total_cost: qualified(:total_cost),
+                                                          cost_status: qualified(:cost_status)))
           }
     scope :with_latency, -> { where.not(latency_ms: nil) }
     scope :streaming,     -> { where(stream: true) }
@@ -64,13 +65,14 @@ module LlmCostTracker
       end
 
       def cost_by_tag(key, limit: nil)
+        cost = qualified(:total_cost)
         label = Ledger::Tags::Breakdown.label_sql(connection)
         raw_value = Ledger::Tags::Breakdown.raw_value_sql(connection)
         relation = Ledger::Tags::Breakdown.join_relation(self, key)
-                                          .select("#{label} AS name", "COALESCE(SUM(total_cost), 0) AS total_cost")
+                                          .select("#{label} AS name", "COALESCE(SUM(#{cost}), 0) AS total_cost")
                                           .group(Arel.sql(label))
                                           .order(
-                                            Arel.sql("COALESCE(SUM(total_cost), 0) DESC"),
+                                            Arel.sql("COALESCE(SUM(#{cost}), 0) DESC"),
                                             Arel.sql("MAX(CASE WHEN #{raw_value} IS NULL THEN 1 ELSE 0 END) ASC"),
                                             Arel.sql("#{label} DESC")
                                           )
@@ -94,6 +96,10 @@ module LlmCostTracker
           .sum(:total_cost)
       end
 
+      def qualified(column)
+        "#{quoted_table_name}.#{connection.quote_column_name(column)}"
+      end
+
       private
 
       def cost_by_column(column, limit:)

data/app/models/llm_cost_tracker/call_rollup.rb

@@ -7,16 +7,32 @@ module LlmCostTracker
         upsert_all(rows, on_duplicate: increment_on_duplicate, record_timestamps: true, unique_by: increment_unique_by)
       end
 
+      DECREMENT_SLICE = 100
+
       def decrement(buckets)
         now = Time.now.utc
-        buckets.each do |(period, period_start, currency, provider), amount|
-          where(period: period, period_start: period_start, currency: currency, provider: provider)
-            .update_all(["total_cost = GREATEST(0, total_cost - ?), updated_at = ?", amount, now])
+        buckets.each_slice(DECREMENT_SLICE) do |slice|
+          where(decrement_scope(slice)).update_all(decrement_assignment(slice, now))
         end
       end
 
       private
 
+      def decrement_scope(slice)
+        rows = Array.new(slice.size, "(?, ?, ?, ?)").join(", ")
+        binds = slice.flat_map { |bucket, _| bucket }
+        sanitize_sql_array(["(period, period_start, currency, provider) IN (#{rows})", *binds])
+      end
+
+      def decrement_assignment(slice, now)
+        branches = slice.map { "WHEN period = ? AND period_start = ? AND currency = ? AND provider = ? THEN ?" }
+        binds = slice.flat_map { |bucket, amount| [*bucket, amount] }
+        sanitize_sql_array(
+          ["total_cost = GREATEST(0, total_cost - CASE #{branches.join(' ')} ELSE 0 END), updated_at = ?",
+           *binds, now]
+        )
+      end
+
       def increment_on_duplicate
         return Arel.sql(mysql_increment_sql) if Ledger::Schema::Adapter.mysql?(connection)
         return Arel.sql(postgres_increment_sql) if Ledger::Schema::Adapter.postgresql?(connection)

data/app/services/llm_cost_tracker/dashboard/data_quality.rb

@@ -5,6 +5,7 @@ module LlmCostTracker
     module DataQuality
       UnknownPricingRow = ::Data.define(:provider, :model, :calls, :share_percent)
       QuarantinedInbox = ::Data.define(:count, :total_cost)
+      UnseenBudgetTags = ::Data.define(:keys)
       StreamingHealthRow = ::Data.define(:provider, :streams, :with_usage, :unknown, :unknown_share)
       Summary = ::Data.define(:total,
                               :unknown_pricing_count,
@@ -45,6 +46,18 @@ module LlmCostTracker
           QuarantinedInbox.new(count: row.quarantined_count.to_i, total_cost: row.quarantined_cost.to_d)
         end
 
+        def unseen_budget_tags
+          budgeted = Budget::PerTag.configured
+          return nil if budgeted.empty?
+          return nil unless Budget::PerTag.columns?
+          return nil unless LlmCostTracker::CallTag.exists?
+
+          unseen = budgeted.keys.reject { |key| LlmCostTracker::CallTag.exists?(key: key) }
+          return nil if unseen.empty?
+
+          UnseenBudgetTags.new(keys: unseen)
+        end
+
         def summary(stats)
           total = stats.total_calls.to_i
           unknown_pricing_count = stats.unknown_pricing_count.to_i

data/app/services/llm_cost_tracker/dashboard/monthly_budget.rb

@@ -6,7 +6,7 @@ module LlmCostTracker
       module_function
 
       def status
-        budget = LlmCostTracker.configuration.monthly_budget
+        budget = LlmCostTracker.configuration.budgets.monthly
         return nil unless budget
 
         budget = budget.to_f

data/app/services/llm_cost_tracker/dashboard/pagination.rb

@@ -6,6 +6,7 @@ module LlmCostTracker
       DEFAULT_PER = 50
       MAX_PER = 200
       MIN_PAGE = 1
+      MAX_PAGE = 100_000
       MIN_PER = 1
 
       attr_reader :page, :per
@@ -13,7 +14,7 @@ module LlmCostTracker
       def self.call(params)
         params = Params.to_hash(params).symbolize_keys
         new(
-          page: integer_param(params, :page, default: MIN_PAGE, min: MIN_PAGE),
+          page: integer_param(params, :page, default: MIN_PAGE, min: MIN_PAGE, max: MAX_PAGE),
           per: integer_param(params, :per, default: DEFAULT_PER, min: MIN_PER, max: MAX_PER)
         )
       end

data/app/services/llm_cost_tracker/dashboard/pricing_overview.rb

@@ -46,8 +46,8 @@ module LlmCostTracker
 
       def subtitle_for(key)
         case key
-        when :overrides then "config.pricing_overrides"
-        when :file then LlmCostTracker.configuration.prices_file.to_s
+        when :overrides then "config.pricing.overrides"
+        when :file then LlmCostTracker.configuration.pricing.file.to_s
         when :bundled then "ships with the gem"
         end
       end
@@ -55,7 +55,7 @@ module LlmCostTracker
       def updated_at_for(key)
         case key
         when :file
-          path = LlmCostTracker.configuration.prices_file
+          path = LlmCostTracker.configuration.pricing.file
           Pricing::Registry.file_metadata(path)["updated_at"] || Pricing::Registry.prices_file_mtime_iso
         when :bundled
           Pricing::Registry.metadata["updated_at"]

data/app/services/llm_cost_tracker/dashboard/setup_state.rb

@@ -7,13 +7,15 @@ module LlmCostTracker
 
       class << self
         def current
-          return @current if defined?(@current)
+          return if @healthy
 
-          @current = compute
+          state = compute
+          @healthy = state.nil?
+          state
         end
 
         def reset!
-          remove_instance_variable(:@current) if defined?(@current)
+          @healthy = false
         end
 
         private

data/app/views/llm_cost_tracker/calls/show.html.erb

@@ -2,7 +2,7 @@
 <% line_item_costs_by_component = call_line_item_costs_by_component(@call) %>
 <% token_segments = priced_components.map do |component|
   token_key = component.fetch(:token_key)
-  value = @call.has_attribute?(token_key) ? @call[token_key] : 0
+  value = @call[token_key]
   { label: component.fetch(:label), value: value, formatted_value: number_with_delimiter(value), css_class: component.fetch(:css_class) }
 end %>
 <% cost_segments = [] %>
@@ -68,13 +68,11 @@ end %>
       <dt>Batch</dt>
       <dd><%= @call.batch? ? "yes" : "no" %></dd>
     </div>
-    <% if @call.has_attribute?(:stream) %>
-      <div class="lct-meta-strip-item">
-        <dt>Stream</dt>
-        <dd><%= @call.stream? ? "yes" : "no" %></dd>
-      </div>
-    <% end %>
-    <% if @call.has_attribute?(:usage_source) && @call.usage_source.present? %>
+    <div class="lct-meta-strip-item">
+      <dt>Stream</dt>
+      <dd><%= @call.stream? ? "yes" : "no" %></dd>
+    </div>
+    <% if @call.usage_source.present? %>
       <div class="lct-meta-strip-item">
         <dt>Usage source</dt>
         <dd><code class="lct-code-id"><%= @call.usage_source %></code></dd>

data/app/views/llm_cost_tracker/data_quality/index.html.erb

@@ -21,6 +21,17 @@
   </div>
 <% end %>
 
+<% if @unseen_budget_tags %>
+  <h3 class="lct-stat-section-label">Budgets</h3>
+  <div class="lct-stat-grid">
+    <div class="lct-stat lct-stat-warn">
+      <div class="lct-stat-head"><p class="lct-stat-label">Budgeted tags never recorded</p></div>
+      <p class="lct-stat-value"><%= @unseen_budget_tags.keys.join(", ") %></p>
+      <p class="lct-stat-foot">No call carries <%= @unseen_budget_tags.keys.one? ? "this tag" : "these tags" %> yet, so <%= @unseen_budget_tags.keys.one? ? "its budget never fires" : "their budgets never fire" %>. Expected on a young ledger; otherwise check the name against the tags you pass.</p>
+    </div>
+  </div>
+<% end %>
+
 <% if @summary.total.zero? %>
   <section class="lct-panel lct-empty">
     <h2 class="lct-state-title">No data yet</h2>

data/app/views/llm_cost_tracker/shared/_filter_pill_date.html.erb

@@ -9,9 +9,7 @@
   </summary>
   <%= form_with url: path, method: :get, local: true, html: { class: "lct-filter-pop-body" } do %>
     <% extra_hidden.each do |k, v| %><%= hidden_field_tag k, v %><% end %>
-    <% current_query.except(:from, :to, :page, :per, *extra_except).each do |key, value| %>
-      <% Array(value).each do |v| %><%= hidden_field_tag(value.is_a?(Array) ? "#{key}[]" : key.to_s, v) %><% end %>
-    <% end %>
+    <%= hidden_query_fields(current_query.except(:from, :to, :page, :per, *extra_except)) %>
     <div class="lct-filter-pop-field"><label for="lct-filter-from">From</label><input type="date" name="from" id="lct-filter-from" value="<%= @from_date.iso8601 %>"></div>
     <div class="lct-filter-pop-field"><label for="lct-filter-to">To</label><input type="date" name="to" id="lct-filter-to" value="<%= @to_date.iso8601 %>"></div>
     <button type="submit" class="lct-button lct-button-primary">Apply</button>

data/app/views/llm_cost_tracker/shared/_filter_pill_model.html.erb

@@ -10,9 +10,7 @@
   </summary>
   <%= form_with url: path, method: :get, local: true, html: { class: "lct-filter-pop-body" } do %>
     <% extra_hidden.each do |k, v| %><%= hidden_field_tag k, v %><% end %>
-    <% current_query.except(:model, :page, :per, *extra_except).each do |key, value| %>
-      <% Array(value).each do |v| %><%= hidden_field_tag(value.is_a?(Array) ? "#{key}[]" : key.to_s, v) %><% end %>
-    <% end %>
+    <%= hidden_query_fields(current_query.except(:model, :page, :per, *extra_except)) %>
     <div class="lct-filter-pop-field">
       <label for="lct-filter-model">Model</label>
       <%= select_tag :model, options_for_select(filter_options_for(:model), active), include_blank: "All models", id: "lct-filter-model" %>

data/app/views/llm_cost_tracker/shared/_filter_pill_provider.html.erb

@@ -10,9 +10,7 @@
   </summary>
   <%= form_with url: path, method: :get, local: true, html: { class: "lct-filter-pop-body" } do %>
     <% extra_hidden.each do |k, v| %><%= hidden_field_tag k, v %><% end %>
-    <% current_query.except(:provider, :page, :per, *extra_except).each do |key, value| %>
-      <% Array(value).each do |v| %><%= hidden_field_tag(value.is_a?(Array) ? "#{key}[]" : key.to_s, v) %><% end %>
-    <% end %>
+    <%= hidden_query_fields(current_query.except(:provider, :page, :per, *extra_except)) %>
     <div class="lct-filter-pop-field">
       <label for="lct-filter-provider">Provider</label>
       <%= select_tag :provider, options_for_select(filter_options_for(:provider), active), include_blank: "All providers", id: "lct-filter-provider" %>

data/app/views/llm_cost_tracker/shared/_filter_pill_stream.html.erb

@@ -11,9 +11,7 @@
   </summary>
   <%= form_with url: path, method: :get, local: true, html: { class: "lct-filter-pop-body" } do %>
     <% extra_hidden.each do |k, v| %><%= hidden_field_tag k, v %><% end %>
-    <% current_query.except(:stream, :page, :per, *extra_except).each do |key, value| %>
-      <% Array(value).each do |v| %><%= hidden_field_tag(value.is_a?(Array) ? "#{key}[]" : key.to_s, v) %><% end %>
-    <% end %>
+    <%= hidden_query_fields(current_query.except(:stream, :page, :per, *extra_except)) %>
     <div class="lct-filter-pop-field">
       <label for="lct-filter-stream">Stream</label>
       <%= select_tag :stream, options_for_select(LlmCostTracker::Dashboard::Filter::STREAM_FILTER_OPTIONS, active), include_blank: "All calls", id: "lct-filter-stream" %>