llm_cost_tracker 0.14.1 → 0.14.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: af338b0d20bfcbdfd9b95a88f903fa2a6b22dab16c72857a48bb2ff6ac0e9ac6
-  data.tar.gz: e2688d3cc3d29795787602ccf79b70dbe541347884644b3d5d234d1635f3f4b9
+  metadata.gz: 2c1f13f9f919518298bf4aa5bb9bf33f6fb48851a963d132d48000c2c9ab2477
+  data.tar.gz: 1de730cdb5061df2588c74dd0c56ec29aa59ec0fe812188e2ae30df4e206c682
 SHA512:
-  metadata.gz: d4e06cd0bab7dd941fb958ca5fff5b6844ba17879f04bd3370258864e7ebd390088e091d2e0bd7f6fd839fcf05ecf3bc79f954b918c5b0b1cea471e1d14a413b
-  data.tar.gz: 107d59de744aabfbf946cbc38ce9ad705b4562a516446163dfb4834102ea263065d67e72c302c75f90ac887c12dfe11c404e85256e46300cd5a2ebe73b61b3af
+  metadata.gz: ee3ac1c8844c4bc478f85899d9e74734d67040d4b4f124c1751a970924a6d4ccf6fd2d680a2ed2a0d0e51ea79d7e985ad623ff49ed88148129df0276ba6c0925
+  data.tar.gz: 2aaa9cad0cd45a6e7df0920f054c258d12f03b2edc99c1c3bcf846d4ba33f4ce62972419cc92c5014711e417cb5f12eee63df2ac2f3c3e691acf47c8c620fdb8

data/CHANGELOG.md

@@ -2,6 +2,88 @@
 
 Format: [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). Versioning: [SemVer](https://semver.org/spec/v2.0.0.html).
 
+## [0.14.2] - 2026-09-28
+
+### Added
+
+- `bin/rails llm_cost_tracker:reprice FROM=... [TO=...]` reprices recorded calls, their rollups and per-tag costs at current prices, except provider-billed costs and costs passed to `track`.
+- Bundled xAI and Mistral prices, with batch, priority and regional rates; Faraday captures both once their hosts are in `capture.openai_compatible_providers`.
+- `bin/rails llm_cost_tracker:doctor` warns when `pricing.file` is older than the bundled prices or this month's `:cache` rollups do not match the calls ledger.
+- Gemini Interactions API calls through the Faraday middleware, streamed and `background: true` ones included, are recorded and priced with their grounding.
+- Creating an explicit Gemini context cache through Faraday or `RubyLLM.cache` on RubyLLM 2.x records its estimated storage cost until the cache expires.
+- Price fields with a `_from_YYYY-MM-DD` suffix apply to calls from that date, including in `backfill_unknown_pricing` and `reprice`; bundled Gemini prices use it for announced price changes.
+
+### Changed
+
+- BREAKING: Rails 8.0+ required; Rails 7.1 and 7.2 no longer receive security fixes upstream.
+- Calls returning a billed `usage.cost`, such as OpenRouter's, are recorded at the billed amount instead of a list-price estimate or `pricing.overrides` rate.
+- The official openai gem pointed at a host in `capture.openai_compatible_providers` records that provider instead of `openai`.
+- A provider-scoped price (`<provider>/<model>`) prices another provider's calls only when that provider has no price for the model.
+- With `budgets.totals_source = :cache`, monthly budgets read past days from the rollups: run `bin/rails llm_cost_tracker:rebuild_rollups` once after deploying and after switching to `:cache`, or they are under-counted.
+- OpenAI Realtime cached audio and image and Gemini cached audio are priced at their own cached rates and stored as `audio_token` / `image_token` line items with `cache_state: read`.
+- `bin/rails llm_cost_tracker:prices:refresh` no longer takes `PREVIEW=1` and stops when it is set; use `bin/rails llm_cost_tracker:prices:check` instead.
+
+### Removed
+
+- The boot warning for `:ruby_llm` enabled together with `:openai` or `:anthropic`: RubyLLM doesn't call those SDKs, so nothing was recorded twice, and disabling one as it advised lost those calls.
+
+### Fixed
+
+- Bundled prices cover more OpenAI and Gemini models that recorded unknown cost, and `omni-moderation` calls are `free` instead of `unknown`.
+- `gpt-5.5-pro` prompts over 272K tokens use long-context rates (unknown on Batch and Flex), and `gpt-5.4` Batch and Flex long-context cached input is corrected.
+- OpenAI cached input on Pro models and cache writes on models before GPT-5.6 are priced at the input rate; these calls were `partial`.
+- OpenAI duration-billed transcriptions are priced per second instead of per started minute, and are no longer free in streams or missing through RubyLLM.
+- Transcriptions returned without usage are recorded with unknown cost; they were recorded at $0 or not at all.
+- OpenAI image, transcription and batch calls on regional hosts, and Anthropic batches with `inference_geo: us`, include the data-residency uplift for models with regional rates.
+- The Faraday middleware reads the model from multipart uploads, which were recorded as `unknown`, and records `/v1/audio/speech` by input characters.
+- OpenAI Hosted Shell calls in a hosted container are captured as `container_session` line items, like Code Interpreter.
+- OpenAI Responses created with `background: true` are recorded once, when a poll through the OpenAI SDK or Faraday returns them finished.
+- Responses calls using the `image_generation` tool, RubyLLM 2.x OpenAI chats included, are recorded `partial` instead of `complete` without the image charge.
+- Chat Completions web search fees apply to every call to OpenAI's search models, streams included, and no longer to other models' `url_citation` annotations.
+- A stream on a priced model that ends without usage and has no priced tool charges stores a `nil` total instead of `0.0`, so it shows as unpriced and is in `Call.without_cost`.
+- OpenAI, Azure OpenAI and Anthropic SDK streams price the service tier and speed the provider served, not the requested `priority` or `fast`.
+- OpenAI streams with logprobs record their usage instead of 0 tokens and $0, and an overflowing SDK or `track_stream` capture logs a warning.
+- Groq streams whose usage arrives only in `x_groq.usage` record their tokens instead of 0 tokens and `unknown`.
+- Anthropic streams price cache writes made after `message_start`.
+- Anthropic compaction tokens, which the top-level usage leaves out, are counted; on-demand compactions were recorded as free (not yet through RubyLLM).
+- Anthropic server-side fallback calls are recorded under the serving model, and billed fallback attempts and advisor iterations use their own model's rates; an unpriced one triggers `pricing.unknown_model_behavior` (not yet through RubyLLM).
+- Refusals Anthropic does not bill are recorded at $0 (not yet through RubyLLM), and refusals `Anthropic::BetaRefusalFallbackMiddleware` retried are recorded instead of dropped.
+- Gemini image and audio prompt tokens on single-rate models are priced at the input rate; they were unpriced.
+- Gemini image models are priced per 1M image tokens and price text and thinking output at the text rate; `gemini-2.5-flash-image` images were priced about 770x too low and 3.x image models' images 10-20x too low. Calls already recorded keep their cost. With a local pricing file, run `bin/rails llm_cost_tracker:prices:check`, check that only Gemini image models are flagged, then run `bin/rails llm_cost_tracker:prices:refresh FORCE=1`.
+- Gemini 3 grounding is priced per unique non-empty query, `track_stream` included; image search queries, Maps grounding and the 3.x image models' grounding are now priced.
+- Gemini calls are recorded under the response's `modelVersion`, so `-latest` aliases are priced and `track_stream(provider: :gemini)` works without `model:`.
+- `track_stream(provider: :gemini)` parses native Gemini chunks when `generativelanguage.googleapis.com` is also an OpenAI-compatible provider named `gemini`.
+- Amazon Bedrock Claude ids and inference profiles price as the Anthropic model, with the regional-profile premium from Claude 4.5; GovCloud is not priced at its rate.
+- Groq batch calls are priced at Groq's batch rate, cached tokens included.
+- OpenAI batches that end `expired` or `cancelled` record their completed requests, and image batch results use the batch's model and batch image rates.
+- OpenAI and Anthropic batch results are stored once however often or concurrently the batch is fetched; they could be recorded twice.
+- With `config.enabled = false`, batch retrieval no longer downloads OpenAI output files or queries the database.
+- RubyLLM prices Anthropic US inference, fast mode and OpenAI regional hosts, and its streams outside Bedrock read the service tier, 1-hour cache writes and response id.
+- RubyLLM chats, streamed ones included, record Anthropic and OpenAI web search and Gemini grounding fees; they were stored `complete` without the fee.
+- RubyLLM Anthropic streams count the final cumulative input and every `pause_turn` segment's input, and chats keep earlier segments' cache writes.
+- RubyLLM Gemini chats, Interactions protocol included, are priced from the raw usage, keeping audio and URL-context tool tokens, tiers, grounding and response ids.
+- RubyLLM `paint` prices image output at the image output rate for Gemini image models, `gpt-image-1` and `gpt-image-1-mini`; the image was unpriced.
+- RubyLLM transcriptions price prompt text and audio at their own rates when the response splits them (Gemini without it as text), and streams record `stream: true`.
+- RubyLLM Bedrock Converse chats with prompt caching no longer subtract cache tokens from input twice, and split cache writes into 5-minute and 1-hour writes.
+- RubyLLM Gemini embeddings record their tokens instead of 0, and `gemini-embedding-2` prices image, PDF, audio and video parts by modality, adding a `video_input` rate.
+- The pre-send budget estimate covers RubyLLM calls, whose estimate was $0, and ignores base64 images, PDFs and audio, which inflated it and blocked ordinary vision requests.
+- `llm_cost_tracker:backfill_unknown_pricing` also reprices `partial` calls whose already-priced rates are unchanged, and adds only the difference to rollups.
+- Mode rates derived from a standard cache rate are named by their mode key, such as `batch_cache_read_input`, in `pricing_snapshot` and on line items.
+- With inline ingestion, a per-tag `on_exceeded` fires when several calls for one tag value cross the limit together; each call read the others' spend, so none saw itself as the crossing call and the alert never fired.
+- Under `:raise` and `:block_requests`, every budget a call crosses fires its `on_exceeded` before the first error is raised.
+- `budgets.per_tag` without a `:block_requests` rule no longer checks the database schema before an LLM call is sent, so a process that starts during a database outage no longer fails its LLM calls.
+- A tag key given as both a Symbol and a String is stored once, with the later value, not twice; requeue any async inbox rows quarantined because of it.
+- Tags with a nil or empty value count as untagged on the dashboard, in `cost_by_tag` and for `budgets.per_tag`.
+- An OpenAI SDK response without usage logs a warning instead of being skipped silently.
+- The async worker checks its tables through the Rails schema cache, so an idle poll runs one query instead of six.
+- With `ingestion.mode = :async` and `budgets.totals_source = :cache`, a missing rollups table no longer stops the worker from draining the inbox; it warns once and budget reads use the calls ledger, as inline ingestion does.
+- `track_stream` records events passed as symbol-keyed hashes, such as `to_h` of an OpenAI Realtime `response.done` event; they were ignored and the call was stored as `unknown` with 0 tokens.
+- An OpenAI SDK `chat.completions.stream` without `stream_options: { include_usage: true }` logs the same warning as the Faraday path instead of being stored as `unknown` silently.
+- The overview's monthly budget bar and projection marker and the Data Quality coverage bars are visible again, and the dark theme gets its missing chart colors.
+- In apps whose `Time.zone` is not UTC, daily charts, the previous-period line and the spend-anomaly banner use local days when the database knows the zone name.
+- A tag value page's spend chart covers the selected date range instead of the last 30 days.
+- Sortable dashboard tables no longer return a 500 when the host sets `config.action_controller.include_all_helpers = false`.
+
 ## [0.14.1] - 2026-09-25
 
 ### Added

data/README.md

@@ -1,14 +1,14 @@
 # LLM Cost Tracker
 
-Self-hosted LLM cost tracking for Rails.
+Per-tenant LLM spend attribution and budgets for Rails — in your database, no proxy.
 
 [![Gem Version](https://img.shields.io/gem/v/llm_cost_tracker.svg)](https://rubygems.org/gems/llm_cost_tracker) [![CI](https://github.com/sergey-homenko/llm_cost_tracker/actions/workflows/ruby.yml/badge.svg)](https://github.com/sergey-homenko/llm_cost_tracker/actions) [![codecov](https://codecov.io/gh/sergey-homenko/llm_cost_tracker/branch/main/graph/badge.svg)](https://codecov.io/gh/sergey-homenko/llm_cost_tracker)
 
-Every call your app makes through RubyLLM, the official OpenAI and Anthropic SDKs, Gemini, or any OpenAI-compatible API gets logged: tokens, cost, latency, tags. Calls go app → provider direct. No proxy.
+Every call through RubyLLM, the official OpenAI and Anthropic SDKs, Gemini, or any OpenAI-compatible API is logged with tokens, cost, and your tags. Budgets can block a tenant's next call before it is sent.
 
 Not Langfuse, Helicone, or LiteLLM. No prompts, no traces, no replay. Spend attribution only.
 
-Requires Ruby 3.3+, Rails 7.1+, PostgreSQL or MySQL.
+Requires Ruby 3.3+, Rails 8.0+, PostgreSQL or MySQL.
 
 <picture> <source media="(prefers-color-scheme: dark)" srcset="docs/dashboard-overview-dark.png"> <img alt="LLM Cost Tracker dashboard" src="docs/dashboard-overview-light.png"> </picture>
 
@@ -58,7 +58,7 @@ The engine ships without authentication on purpose.
 ## What lands in the ledger
 
 - **Calls.** Provider, model, total tokens, total cost, latency, status.
-- **Line items.** Per-component breakdown — text/audio/cached tokens, tool charges (web search, code execution, grounding, container sessions).
+- **Line items.** Per-component breakdown — text/audio/cached tokens, tool charges (web search, grounding, container sessions).
 - **Tags.** Whatever attribution you pass — user, feature, tenant, env.
 - **Provider IDs.** Response, project, API key, workspace — for downstream audits.
 - **Pricing snapshot.** So historical numbers don't drift when prices change.
@@ -73,11 +73,22 @@ The engine ships without authentication on purpose.
 | Azure OpenAI | Faraday or official SDK (auto-detected on `*.openai.azure.com` and Foundry `*.services.ai.azure.com`, both deployments and `/openai/v1/...`) |
 | Google Gemini | Faraday |
 | `ruby-openai` | Faraday |
-| OpenRouter, DeepSeek, Groq, LiteLLM-style gateways | OpenAI-compatible Faraday |
+| OpenRouter, DeepSeek, Groq; xAI, Mistral, and other gateways (LiteLLM etc.) once their host is added to `config.capture.openai_compatible_providers` | OpenAI-compatible Faraday, or the official OpenAI SDK with `base_url` on that host |
 | Anything else | `LlmCostTracker.track` |
 
 Streams capture when the provider emits final usage. OpenAI Faraday streams to `/chat/completions` get `stream_options: { include_usage: true }` auto-injected so the final usage chunk lands in the ledger (opt out via `config.capture.request_stream_usage = false`).
 
+Captured does not always mean priced:
+
+| Cost comes from | Calls |
+| --- | --- |
+| The amount billed, from `usage.cost` in the response or final stream chunk | OpenRouter through Faraday, the official OpenAI SDK, or `track_stream`, and any OpenAI-compatible gateway that returns `usage.cost`; bundled prices apply when it is missing or the call comes through RubyLLM, other than an OpenRouter chat |
+| Bundled [`prices.json`](lib/llm_cost_tracker/prices.json) | The OpenAI, Anthropic, Gemini, Groq, OpenRouter, xAI, and Mistral models it lists, and the same Claude models on Bedrock through RubyLLM |
+| The OpenAI, Anthropic, or Gemini price for the same model name | Azure OpenAI (by the model in the response, not the deployment name), Vertex AI through RubyLLM, gateways that pass a listed model name through |
+| Nothing: recorded with `cost_status: unknown` | DeepSeek, and through RubyLLM also Perplexity, Ollama, other Bedrock models, and Claude on GovCloud (`us-gov.` profiles) |
+
+Add missing prices to `config.pricing.file` or `config.pricing.overrides` ([Pricing](docs/pricing.md)), then run `bin/rails llm_cost_tracker:backfill_unknown_pricing` to price the calls already recorded.
+
 ## What it isn't
 
 - No proxy. Direct calls only.

data/app/assets/llm_cost_tracker/application.css

@@ -154,6 +154,9 @@
     --lct-shadow-md: 0 2px 6px rgba(0, 0, 0, 0.35), 0 1px 2px rgba(0, 0, 0, 0.25);
     --lct-shadow-lg: 0 12px 32px rgba(0, 0, 0, 0.5), 0 2px 6px rgba(0, 0, 0, 0.3);
     --lct-focus-ring: 0 0 0 3px rgba(124, 131, 255, 0.30);
+      --lct-chart-marker: rgba(230, 236, 245, 0.45);
+      --lct-chart-secondary: rgba(230, 236, 245, 0.32);
+      --lct-chip-remove: rgba(230, 236, 245, 0.58);
   }
 
   * { box-sizing: border-box; }
@@ -613,6 +616,19 @@
   .lct-stack-swatch { width: 10px; height: 10px; border-radius: 2px; display: inline-block; }
   .lct-stack-meta { color: var(--lct-muted); }
   .lct-stack-empty { color: var(--lct-muted); font-size: var(--fs-sm); margin: 0; }
+  .lct-bar-track,
+  .lct-budget-track { background: var(--lct-surface-2); border-radius: 999px; overflow: hidden; }
+  .lct-bar-track { height: 8px; min-width: 96px; }
+  .lct-budget-track { height: 10px; position: relative; }
+  .lct-bar-fill,
+  .lct-budget-fill { height: 100%; display: block; background: var(--lct-accent); }
+  .lct-budget-fill--warn { background: var(--lct-warning); }
+  .lct-budget-fill--over { background: var(--lct-danger); }
+  .lct-budget-marker { position: absolute; top: 0; bottom: 0; border-left: 2px dashed var(--lct-chart-marker); }
+  .lct-budget-projection { display: flex; flex-wrap: wrap; gap: 8px 12px; align-items: baseline; margin: 10px 0 0; color: var(--lct-muted); font-size: var(--fs-sm); }
+  .lct-budget-projection strong { color: var(--lct-text); }
+  .lct-budget-projection-status { font-weight: 600; }
+  .lct-budget-projection-status--over { color: var(--lct-warning-copy); }
 
   .lct-legend { display: inline-flex; align-items: center; gap: 6px; font-size: var(--fs-xs); color: var(--lct-muted); }
   .lct-legend-dot { width: 9px; height: 9px; border-radius: 2px; }

data/app/controllers/llm_cost_tracker/calls_controller.rb

@@ -16,11 +16,9 @@ module LlmCostTracker
     }.freeze
 
     def index
-      @sort = params[:sort].to_s
-      @dir = params[:dir].to_s
       scope = Dashboard::Filter.call(params: params)
       scope = scope.unknown_pricing if params[:cost_status].to_s == "incomplete"
-      ordered_scope = scope.order(*calls_order(@sort, @dir))
+      ordered_scope = scope.order(*calls_order(params[:sort].to_s, params[:dir].to_s))
 
       respond_to do |format|
         format.html do

data/app/controllers/llm_cost_tracker/models_controller.rb

@@ -5,13 +5,11 @@ module LlmCostTracker
     MAX_ROWS = 200
 
     def index
-      @sort = params[:sort].to_s
-      @dir = params[:dir].to_s
       @rows = Dashboard::TopModels.call(
         scope: Dashboard::Filter.call(params: params),
         limit: MAX_ROWS,
-        sort: @sort,
-        direction: @dir
+        sort: params[:sort].to_s,
+        direction: params[:dir].to_s
       )
     end
   end

data/app/controllers/llm_cost_tracker/tags_controller.rb

@@ -10,10 +10,11 @@ module LlmCostTracker
       @value = Dashboard::Params.scalar(params[:tag_value], :tag_value)
 
       if @value.empty?
-        @sort = params[:sort].to_s
-        @dir = params[:dir].to_s
         @breakdown = Dashboard::TagBreakdown.call(
-          scope: Dashboard::Filter.call(params: params), key: params[:key], sort: @sort, direction: @dir
+          scope: Dashboard::Filter.call(params: params),
+          key: params[:key],
+          sort: params[:sort].to_s,
+          direction: params[:dir].to_s
         )
       else
         @key = LlmCostTracker::Tags::Key.validate!(
@@ -23,7 +24,7 @@ module LlmCostTracker
         value_scope = Dashboard::Filter.call(params: params, tags: { @key => @value })
         @value_total_cost = value_scope.sum(:total_cost).to_f
         @value_calls = value_scope.count
-        @value_points = Dashboard::TimeSeries.call(scope: value_scope)
+        @value_points = Dashboard::TimeSeries.call(scope: value_scope, from: @from_date, to: @to_date)
       end
     end
   end

data/app/helpers/llm_cost_tracker/application_helper.rb

@@ -13,6 +13,7 @@ module LlmCostTracker
     include PaginationHelper
     include TokenUsageHelper
     include InlineStyleHelper
+    include SortableTableHelper
 
     def dashboard_section
       path = request.path.to_s
@@ -120,13 +121,10 @@ module LlmCostTracker
     end
 
     def tag_chip_entries(tags, limit: 3)
-      normalized = normalized_tags(tags)
-      return [] if normalized.empty?
-
-      visible = normalized.first(limit).map do |key, value|
-        { key: key.to_s, value: tag_value_summary(value) }
+      visible = tags.first(limit).map do |key, value|
+        { key: key.to_s, value: truncate_text(value.to_s, TAG_VALUE_SUMMARY_BYTES) }
       end
-      visible << { more: normalized.size - limit } if normalized.size > limit
+      visible << { more: tags.size - limit } if tags.size > limit
       visible
     end
 
@@ -144,25 +142,6 @@ module LlmCostTracker
 
     private
 
-    def normalized_tags(tags)
-      return tags.transform_keys(&:to_s) if tags.is_a?(Hash)
-
-      JSON.parse(tags || "{}")
-    rescue JSON::ParserError, TypeError
-      {}
-    end
-
-    def tag_value_summary(value)
-      string = case value
-               when Hash, Array
-                 JSON.generate(value)
-               else
-                 value.to_s
-               end
-
-      truncate_text(string, TAG_VALUE_SUMMARY_BYTES)
-    end
-
     def truncate_text(string, limit)
       return string if string.bytesize <= limit
 

data/app/helpers/llm_cost_tracker/chart_helper.rb

@@ -7,7 +7,7 @@ module LlmCostTracker
 
       cfg = chart_config(points, comparison_points, height, y_ticks)
       parts = [chart_svg_open(cfg), "<title>Daily spend trend</title>", chart_area_gradient_def]
-      parts.concat(chart_grid_and_axis(cfg))
+      parts.concat((0..cfg[:y_ticks]).map { |i| chart_tick_line(cfg, i) })
       parts << chart_paths(cfg)
       parts.concat(chart_dots(cfg))
       parts.concat(chart_x_labels(cfg))
@@ -34,7 +34,7 @@ module LlmCostTracker
       peak_index = points.each_with_index.max_by { |point, _| point[:cost].to_f }&.last
       { width: width, height: height, pad: pad, plot_w: plot_w, plot_h: plot_h,
         max_cost: max_cost, n: points.size, y_ticks: y_ticks, points: points, coords: coords,
-        comparison_points: comparison_points, comparison_coords: comparison_coords,
+        comparison_coords: comparison_coords,
         peak_index: peak_index }
     end
 
@@ -60,21 +60,17 @@ module LlmCostTracker
       "<svg #{attrs}>"
     end
 
-    def chart_grid_and_axis(cfg)
-      (0..cfg[:y_ticks]).map { |i| chart_tick_line(cfg, i) }
-    end
-
     def chart_tick_line(cfg, idx)
       pad = cfg[:pad]
       right_x = chart_fmt(pad[:left] + cfg[:plot_w])
       left_x = chart_fmt(pad[:left])
       text_x = chart_fmt(pad[:left] - 8)
       value = cfg[:max_cost] * (cfg[:y_ticks] - idx).to_f / cfg[:y_ticks]
-      y = chart_fmt(pad[:top] + (cfg[:plot_h] * idx.to_f / cfg[:y_ticks]))
-      label_y = chart_fmt(pad[:top] + (cfg[:plot_h] * idx.to_f / cfg[:y_ticks]) + 3)
+      tick_y = pad[:top] + (cfg[:plot_h] * idx.to_f / cfg[:y_ticks])
+      y = chart_fmt(tick_y)
+      label_y = chart_fmt(tick_y + 3)
       grid = %(<line class="lct-chart-grid" x1="#{left_x}" x2="#{right_x}" y1="#{y}" y2="#{y}"/>)
-      label = format("%.2f", value)
-      text = %(<text class="lct-chart-axis" x="#{text_x}" y="#{label_y}" text-anchor="end">$#{label}</text>)
+      text = %(<text class="lct-chart-axis" x="#{text_x}" y="#{label_y}" text-anchor="end">$#{chart_fmt(value)}</text>)
       "#{grid}#{text}"
     end
 

data/app/helpers/llm_cost_tracker/token_usage_helper.rb

@@ -33,41 +33,21 @@ module LlmCostTracker
     }.freeze
 
     def token_usage_stack_components
-      token_usage_display_components(labels: COMPONENT_LABELS).select do |component|
-        component.fetch(:cost_key)
-      end
-    end
-
-    def call_line_item_costs_by_component(call)
-      call.line_items.each_with_object({}) do |line_item, accumulator|
-        component = LlmCostTracker::Usage::Catalog.token_priced_for(
-          kind: line_item.kind, direction: line_item.direction, cache_state: line_item.cache_state
-        )
-        accumulator[component.key] = line_item.cost if component && line_item.cost
-      end
-    end
-
-    private
-
-    def token_usage_display_components(labels:)
       LlmCostTracker::Usage::Catalog.token_priced.map do |component|
         token_key = component.token_key
         {
           token_key: token_key,
-          cost_key: component.cost_key,
           price_key: component.key,
-          label: labels.fetch(token_key),
+          label: COMPONENT_LABELS.fetch(token_key),
           css_class: STACK_CLASSES[token_key]
         }
-      end + [
-        {
-          token_key: :hidden_output_tokens,
-          cost_key: nil,
-          price_key: nil,
-          label: labels.fetch(:hidden_output_tokens),
-          css_class: nil
-        }
-      ]
+      end
+    end
+
+    def call_line_item_costs_by_component(call)
+      LlmCostTracker::Usage::Catalog.costs_by_component(
+        call.line_items.map { |item| [item.kind, item.direction, item.cache_state, item.cost] }
+      )
     end
   end
 end

data/app/models/llm_cost_tracker/call.rb

@@ -45,7 +45,10 @@ module LlmCostTracker
       def already_recorded?(provider:, provider_response_id:)
         return false if provider_response_id.to_s.empty?
 
-        Ledger::Isolation.guard(self) { where(provider: provider, provider_response_id: provider_response_id).exists? }
+        Ledger::Isolation.guard(self) do
+          where(provider: provider, provider_response_id: provider_response_id)
+            .where.not(usage_source: Usage::Source::UNKNOWN).exists?
+        end
       end
 
       def by_tag(key, value) = by_tags(key => value)
@@ -86,8 +89,12 @@ module LlmCostTracker
 
       def latency_by_provider = group(:provider).average(:latency_ms).transform_values(&:to_f)
 
-      def group_by_period(period, column: :tracked_at)
-        group(Arel.sql(period_group_expression(period, column: column)))
+      def group_by_period(period, column: :tracked_at, time_zone: nil)
+        column = column.to_s
+        raise ArgumentError, "invalid period column: #{column.inspect}" unless column_names.include?(column)
+
+        bucket = Ledger::Schema::Adapter.period_bucket_sql(connection, period, qualified(column), time_zone: time_zone)
+        group(Arel.sql(bucket))
       end
 
       def daily_costs(days: 30)
@@ -109,17 +116,6 @@ module LlmCostTracker
         relation = relation.limit(limit) if limit
         relation
       end
-
-      def period_group_expression(period, column:)
-        Ledger::Schema::Adapter.period_bucket_sql(connection, period, period_column_expression(column))
-      end
-
-      def period_column_expression(column)
-        column = column.to_s
-        return "#{quoted_table_name}.#{connection.quote_column_name(column)}" if column_names.include?(column)
-
-        raise ArgumentError, "invalid period column: #{column.inspect}"
-      end
     end
 
     def tag_pairs

data/app/services/llm_cost_tracker/dashboard/data_quality.rb

@@ -13,7 +13,6 @@ module LlmCostTracker
                               :missing_latency_count,
                               :streaming_count,
                               :streaming_missing_usage,
-                              :missing_provider_response_id_count,
                               :calls_with_pricing,
                               :tagged_calls,
                               :calls_with_latency,
@@ -52,7 +51,7 @@ module LlmCostTracker
           return nil unless Budget::PerTag.columns?
           return nil unless LlmCostTracker::CallTag.exists?
 
-          unseen = budgeted.keys.reject { |key| LlmCostTracker::CallTag.exists?(key: key) }
+          unseen = budgeted.keys.reject { |key| LlmCostTracker::CallTag.where(key: key).where.not(value: "").exists? }
           return nil if unseen.empty?
 
           UnseenBudgetTags.new(keys: unseen)
@@ -79,7 +78,6 @@ module LlmCostTracker
             missing_latency_count,
             streaming_count,
             streaming_missing_usage,
-            missing_provider_response_id_count,
             calls_with_pricing,
             tagged_calls,
             calls_with_latency,
@@ -142,7 +140,6 @@ module LlmCostTracker
             token_value = stats[component.token_key].to_i
 
             {
-              price_key: component.key,
               token_key: component.token_key,
               cost_key: component.cost_key,
               token_value: token_value,
@@ -154,7 +151,6 @@ module LlmCostTracker
 
           rows + [
             {
-              price_key: nil,
               token_key: :hidden_output_tokens,
               cost_key: nil,
               token_value: stats.hidden_output_tokens.to_i,
@@ -178,7 +174,7 @@ module LlmCostTracker
                         Arel.sql("#{line_item_table}.direction"),
                         Arel.sql("#{line_item_table}.cache_state"),
                         Arel.sql("COALESCE(SUM(#{line_item_table}.cost), 0)"))
-          index_costs_by_component(rows)
+          Usage::Catalog.costs_by_component(rows)
         end
 
         def streaming_health_rows(scope, total_streaming:)
@@ -221,13 +217,6 @@ module LlmCostTracker
 
         private
 
-        def index_costs_by_component(rows)
-          rows.each_with_object({}) do |(kind, direction, cache_state, cost), accumulator|
-            component = Usage::Catalog.token_priced_for(kind: kind, direction: direction, cache_state: cache_state)
-            accumulator[component.key] = cost if component
-          end
-        end
-
         def percentage(numerator, denominator)
           return 0.0 unless denominator.positive?
 
@@ -239,7 +228,6 @@ module LlmCostTracker
           selects = [
             "COUNT(*) AS total_calls",
             "#{conditional_count_sql(unknown_pricing)} AS unknown_pricing_count",
-            "#{tagged_calls_sql(scope)} AS tagged_calls_count",
             "COUNT(*) - #{tagged_calls_sql(scope)} AS untagged_calls_count",
             "#{conditional_count_sql('latency_ms IS NULL')} AS missing_latency_count",
             "#{conditional_count_sql('stream')} AS streaming_count",
@@ -301,7 +289,8 @@ module LlmCostTracker
           tags_table = LlmCostTracker::CallTag.quoted_table_name
 
           "COALESCE(SUM(CASE WHEN EXISTS (SELECT 1 FROM #{tags_table} " \
-            "WHERE #{tags_table}.llm_cost_tracker_call_id = #{calls_table}.id) THEN 1 ELSE 0 END), 0)"
+            "WHERE #{tags_table}.llm_cost_tracker_call_id = #{calls_table}.id " \
+            "AND #{tags_table}.#{scope.connection.quote_column_name('value')} != '') THEN 1 ELSE 0 END), 0)"
         end
       end
     end

data/app/services/llm_cost_tracker/dashboard/filter.rb

@@ -39,17 +39,13 @@ module LlmCostTracker
       attr_reader :scope, :params, :extra_tags
 
       def apply_date_filters(relation)
-        from_date = Dashboard::DateRange.parse(params, :from)
-        to_date = Dashboard::DateRange.parse(params, :to)
-        Dashboard::DateRange.validate!(from: from_date, to: to_date)
-
-        default_range = Dashboard::DateRange.call(params: params)
-        from_date ||= default_range.from
-        to_date ||= default_range.to
+        Dashboard::DateRange.validate!(from: Dashboard::DateRange.parse(params, :from),
+                                       to: Dashboard::DateRange.parse(params, :to))
+        range = Dashboard::DateRange.call(params: params)
 
         relation
-          .where(tracked_at: from_date.beginning_of_day..)
-          .where(tracked_at: ..to_date.end_of_day)
+          .where(tracked_at: range.from.beginning_of_day..)
+          .where(tracked_at: ..range.to.end_of_day)
       end
 
       def apply_exact_filter(relation, key)

data/app/services/llm_cost_tracker/dashboard/pagination.rb

@@ -19,11 +19,8 @@ module LlmCostTracker
         )
       end
 
-      def self.integer_param(params, key, default:, min:, max: nil)
-        value = Integer(params[key], 10)
-        value = [value, min].max
-        value = [value, max].min if max
-        value
+      def self.integer_param(params, key, default:, min:, max:)
+        Integer(params[key], 10).clamp(min, max)
       rescue ArgumentError, TypeError
         default
       end

data/app/services/llm_cost_tracker/dashboard/pricing_overview.rb

@@ -63,9 +63,10 @@ module LlmCostTracker
       end
 
       def build_rows(prices)
+        today = Time.now.utc.to_date.iso8601
         rows = prices.map do |key, rates|
           provider, model = split_key(key.to_s)
-          Row.new(provider: provider, model: model, rates: rates)
+          Row.new(provider: provider, model: model, rates: Pricing::Matcher.prices_on(rates, today))
         end
         rows.sort_by { |row| [row.provider || "~", row.model] }
       end

data/app/services/llm_cost_tracker/dashboard/spend_anomaly.rb

@@ -59,7 +59,7 @@ module LlmCostTracker
           .where(tracked_at: window)
           .where.not(total_cost: nil)
           .group(:provider, :model)
-          .group_by_period(:day)
+          .group_by_period(:day, time_zone: Time.zone)
           .sum(:total_cost)
           .each do |(provider, model, day), total_cost|
             grouped[[provider, model]][Date.iso8601(day.to_s)] += total_cost.to_f

data/app/services/llm_cost_tracker/dashboard/tag_breakdown.rb

@@ -4,7 +4,6 @@ module LlmCostTracker
   module Dashboard
     class TagBreakdown
       DEFAULT_LIMIT = 100
-      SORT_OPTIONS = %w[value calls cost avg_cost].freeze
       DEFAULT_DIRECTIONS = { "value" => "asc", "calls" => "desc", "cost" => "desc", "avg_cost" => "desc" }.freeze
       Row = Data.define(:value, :calls, :total_cost, :average_cost_per_call, :share_percent)
 
@@ -21,7 +20,7 @@ module LlmCostTracker
         @key = LlmCostTracker::Tags::Key.validate!(key, error_class: LlmCostTracker::InvalidFilterError)
         limit = limit.to_i
         @limit = limit.positive? ? [limit, DEFAULT_LIMIT].min : DEFAULT_LIMIT
-        @sort = SORT_OPTIONS.include?(sort.to_s) ? sort.to_s : "cost"
+        @sort = DEFAULT_DIRECTIONS.key?(sort.to_s) ? sort.to_s : "cost"
         @direction = Sort::DIRECTIONS.include?(direction.to_s) ? direction.to_s : DEFAULT_DIRECTIONS[@sort]
       end
 
@@ -89,7 +88,7 @@ module LlmCostTracker
       def summary_sql
         <<~SQL.squish
           SELECT COUNT(*) AS total_calls,
-                 COUNT(t.#{quote_column('value')}) AS tagged_calls,
+                 COUNT(CASE WHEN #{tag_present_predicate} THEN 1 END) AS tagged_calls,
                  COUNT(DISTINCT CASE WHEN #{tag_present_predicate} THEN #{tag_value_column} END) AS distinct_values
           FROM (#{scope.to_sql}) AS sub
           LEFT OUTER JOIN #{call_tag_table} t ON t.llm_cost_tracker_call_id = sub.id AND t.#{quote_column('key')} = #{quoted_key}

data/app/services/llm_cost_tracker/dashboard/tag_key_explorer.rb

@@ -38,6 +38,7 @@ module LlmCostTracker
                  COUNT(DISTINCT t.#{value_column}) AS distinct_values
           FROM (#{scope.to_sql}) AS sub
           INNER JOIN #{tags_table} t ON t.llm_cost_tracker_call_id = sub.id
+          WHERE t.#{value_column} != ''
           GROUP BY t.#{key_column}
           ORDER BY calls_count DESC
           LIMIT #{limit}

data/app/services/llm_cost_tracker/dashboard/time_series.rb

@@ -5,18 +5,16 @@ require "date"
 module LlmCostTracker
   module Dashboard
     class TimeSeries
-      DEFAULT_DAYS = 30
-
       class << self
-        def call(scope: LlmCostTracker::Call.all, from: nil, to: Date.current)
+        def call(from:, to:, scope: LlmCostTracker::Call.all)
           new(scope: scope, from: from, to: to).points
         end
       end
 
       def initialize(scope:, from:, to:)
         @scope = scope
+        @from = from.to_date
         @to = to.to_date
-        @from = from ? from.to_date : (@to - (DEFAULT_DAYS - 1))
       end
 
       def points
@@ -35,7 +33,7 @@ module LlmCostTracker
       def scoped_costs
         scope
           .where(tracked_at: from.beginning_of_day..to.end_of_day)
-          .group_by_period(:day)
+          .group_by_period(:day, time_zone: Time.zone)
           .sum(:total_cost)
           .transform_values(&:to_f)
       end