llm_cost_tracker 0.7.1 → 0.7.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: fbf918d9a4886e24ba99f93dc1125e016e50b45208437dd3254adda76f58033a
-  data.tar.gz: 8a22ccfa517549f55d2a302287660a8a5d7faf03568e80c253e6024ad4988743
+  metadata.gz: 6950dae400eac9294a57a0ba2fd2bce7977658837962eafffc3836fa4ab9bd2b
+  data.tar.gz: c398f5271d3d0fa53cb27e1206e418e0242fb9dff73ed2c405903b92dfaf8a48
 SHA512:
-  metadata.gz: 4ef6bc278f6f98ce37a91e515e8b4a004aaff5f573fc8580e312e53b903d1e7700b2cd58b9bea72fc0ec904010cba43fb0209e7993c31bb69205b42564554884
-  data.tar.gz: 4c9a193d16bb5e8bfa58aaefb6fb9b7d98a632a8e19203927c3fd29b957da6088af71caf5c14f5f2b7452660f78001d7898745f8d7611a4fd6e94311b723bc71
+  metadata.gz: c52638e31e7eb0f46308312339bd40cfce87227a8c7ec77c94b3af08ffc931c3cffb9566f2ce15ec70a87700084e5e9bb6d05fe670028b57a12066af4a9ebaf6
+  data.tar.gz: 12da45f4cd8c485bd6fde5f9376bdfa2c8e618abd41e7be11c022878ec005348ca5d98578216170f1b7105dcc9e2c0c4b037cb5394e2085e2526e04ee8d5a885

data/CHANGELOG.md

@@ -4,6 +4,27 @@ Format: [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). Versioning: [S
 
 ## [Unreleased]
 
+## [0.7.3] - 2026-05-01
+
+### Fixed
+
+- Gemini API thinking tokens no longer get added to output tokens twice.
+
+## [0.7.2] - 2026-05-01
+
+### Added
+
+- Groq auto-detection, price scraping, and bundled production text model prices.
+
+### Changed
+
+- Bundled prices refreshed from official provider pricing as of 2026-05-01.
+- Bundled prices now include OpenAI Flex/Priority/regional processing, Gemini Flex/Priority, and Anthropic fast/data residency rates.
+
+### Fixed
+
+- Streaming capture now snapshots tags when the stream starts.
+
 ## [0.7.1] - 2026-04-30
 
 ### Changed

data/README.md

@@ -1,12 +1,12 @@
 # LLM Cost Tracker
 
-A Rails-native ledger for what your LLM calls actually cost.
+A Rails-native ledger for estimating LLM API spend.
 
 [![Gem Version](https://img.shields.io/gem/v/llm_cost_tracker.svg)](https://rubygems.org/gems/llm_cost_tracker)
 [![CI](https://github.com/sergey-homenko/llm_cost_tracker/actions/workflows/ruby.yml/badge.svg)](https://github.com/sergey-homenko/llm_cost_tracker/actions)
 [![codecov](https://codecov.io/gh/sergey-homenko/llm_cost_tracker/branch/main/graph/badge.svg)](https://codecov.io/gh/sergey-homenko/llm_cost_tracker)
 
-If you have OpenAI, Anthropic, or Gemini in production and someone keeps asking "where did that bill come from?", this gem records every call into your own database, prices it locally, and gives you a dashboard you can mount in five minutes. No proxy, no SaaS account, no extra service to deploy.
+If someone keeps asking "where did that LLM bill come from?", this gem records provider-reported usage into your own database, prices it locally, and gives you a dashboard you can mount in five minutes. No proxy, no SaaS account, no extra service to deploy.
 
 It is not Langfuse, Helicone, or LiteLLM. It does not capture prompts, score completions, or replay traces. It does one thing: tells you which provider, which model, which feature, and which user burned how much money. That's the entire pitch.
 
@@ -14,6 +14,10 @@ Requires Ruby 3.3+, Rails 7.1+, PostgreSQL or MySQL, and Faraday 2.0+.
 
 ![Dashboard overview](docs/dashboard-overview.png)
 
+## Accuracy model
+
+LLM Cost Tracker estimates spend from provider-reported usage and configured prices. It is useful for explaining spend by provider, model, and tags, but it is not invoice-grade billing. For reconciliation, each call keeps `provider_response_id`, `usage_source`, token breakdowns, and `pricing_mode`.
+
 ## Quickstart
 
 Add to your Gemfile alongside whatever LLM client you already use:
@@ -35,7 +39,7 @@ Drop this into `config/initializers/llm_cost_tracker.rb`:
 
 ```ruby
 LlmCostTracker.configure do |config|
-  config.default_tags    = -> { { environment: Rails.env } }
+  config.default_tags = -> { { environment: Rails.env } }
   config.instrument :openai
 end
 ```
@@ -78,7 +82,7 @@ Drop-in for RubyLLM and the official `openai` and `anthropic` gems. `config.inst
 
 ```ruby
 LlmCostTracker.configure do |config|
-  config.instrument :openai      # or :anthropic / :ruby_llm
+  config.instrument :openai # or :anthropic / :ruby_llm
 end
 
 LlmCostTracker.with_tags(feature: "support_chat") do
@@ -98,7 +102,7 @@ This patches **only** RubyLLM and the official Ruby SDKs. `ruby-openai` (alexrud
 
 ### 2. Faraday middleware
 
-For `ruby-openai`, the Gemini REST API, custom Faraday clients, or anything OpenAI-compatible (OpenRouter, DeepSeek, LiteLLM proxies):
+For `ruby-openai`, the Gemini REST API, custom Faraday clients, or anything OpenAI-compatible (OpenRouter, DeepSeek, Groq, LiteLLM proxies):
 
 ```ruby
 conn = Faraday.new(url: "https://api.openai.com") do |f|
@@ -137,13 +141,15 @@ For streaming the same way, `track_stream` accepts a block, parses provider even
 Tags answer the only question that matters in attribution: which feature, which user, which job, which tenant. They're free-form strings, stored as JSONB on PostgreSQL or JSON on MySQL, and queryable from both Ruby and the dashboard.
 
 ```ruby
-LlmCostTracker.with_tags(user_id: current_user.id, feature: "support_chat", trace_id: request.uuid) do
+LlmCostTracker.with_tags(user_id: current_user.id, feature: "support_chat") do
   client.chat(parameters: { model: "gpt-4o", messages: [...] })
 end
 ```
 
 `with_tags` is thread- and fiber-isolated, so concurrent requests in Puma or jobs in Sidekiq don't bleed into each other. A `default_tags` callable on configuration runs on every event for things you always want — `environment`, `region`, deployment SHA. Explicit tags passed to `track` win over scoped tags, scoped tags win over defaults.
 
+Streaming capture snapshots tags when the stream starts, so attribution survives delayed or cross-thread stream consumption.
+
 What you put in tags is **your** input — they're queryable strings. Don't put prompts, completions, emails, or secrets there. Use IDs.
 
 ## Pricing
@@ -184,7 +190,7 @@ Budgets are guardrails, not transactional caps:
 config.monthly_budget           = 500.00
 config.daily_budget             = 50.00
 config.per_call_budget          = 2.00
-config.budget_exceeded_behavior = :block_requests   # or :notify, :raise
+config.budget_exceeded_behavior = :block_requests # or :notify, :raise
 config.on_budget_exceeded       = ->(data) { SlackNotifier.notify("#alerts", "...") }
 ```
 
@@ -233,6 +239,7 @@ Auth is your job. Examples for basic auth and Devise: [`docs/dashboard.md`](docs
 | Google Gemini | Yes | Gemini 2.5 Pro/Flash/Flash-Lite, 2.0 Flash/Flash-Lite |
 | OpenRouter | Yes | OpenAI-compatible usage; provider-prefixed model IDs are normalized |
 | DeepSeek | Yes | OpenAI-compatible usage; add `pricing_overrides` for DeepSeek-specific rates |
+| Groq | Yes | OpenAI-compatible usage with bundled prices for production text models |
 | Other OpenAI-compatible hosts | Configurable | Register the host via `config.openai_compatible_providers` |
 | Anything else | Manual | Use `LlmCostTracker.track` / `track_stream` |
 
@@ -275,10 +282,10 @@ is still brief.
 
 ```bash
 bundle install
-bin/check       # rubocop + rspec + coverage gate
+bin/check # rubocop + rspec + coverage gate
 ```
 
-Architecture rules and conventions for contributions live in [`AGENTS.md`](AGENTS.md) and [`docs/architecture.md`](docs/architecture.md).
+Architecture rules and conventions for contributions live in [`docs/architecture.md`](docs/architecture.md).
 
 ## License
 

data/app/models/llm_cost_tracker/ledger/call.rb

@@ -38,7 +38,7 @@ module LlmCostTracker
       end
 
       def self.by_tags(tags)
-        Ledger::Tags::Query.apply(self, tags)
+        Ledger::Tags::Query.apply(tags)
       end
     end
   end

data/app/models/llm_cost_tracker/ledger/call_metrics.rb

@@ -48,7 +48,7 @@ module LlmCostTracker
       end
 
       def tag_value_expression(key, table_name: quoted_table_name)
-        Ledger::Tags::Sql.value_expression(self, key, table_name: table_name)
+        Ledger::Tags::Sql.value_expression(key, table_name: table_name)
       end
 
       private

data/app/services/llm_cost_tracker/dashboard/data_quality.rb

@@ -8,8 +8,7 @@ module LlmCostTracker
     class DataQuality
       class << self
         def call(scope: LlmCostTracker::Ledger::Call.all)
-          model = scope.klass
-          scope.unscope(:order).select(aggregate_selects(scope, model:)).take
+          scope.unscope(:order).select(aggregate_selects(scope)).take
         end
 
         def unknown_pricing_by_model(scope)
@@ -70,12 +69,12 @@ module LlmCostTracker
 
         private
 
-        def aggregate_selects(scope, model:)
+        def aggregate_selects(scope)
           selects = [
             "COUNT(*) AS total_calls",
             "#{conditional_count_sql('total_cost IS NULL')} AS unknown_pricing_count",
-            "#{tagged_calls_sql(model)} AS tagged_calls_count",
-            "COUNT(*) - #{tagged_calls_sql(model)} AS untagged_calls_count",
+            "#{tagged_calls_sql(scope)} AS tagged_calls_count",
+            "COUNT(*) - #{tagged_calls_sql(scope)} AS untagged_calls_count",
             "#{conditional_count_sql('latency_ms IS NULL')} AS missing_latency_count",
             "#{conditional_count_sql('stream')} AS streaming_count",
             "#{streaming_missing_usage_select} AS streaming_missing_usage_count",
@@ -127,11 +126,12 @@ module LlmCostTracker
           conditional_count_sql(predicate)
         end
 
-        def tagged_calls_sql(model)
-          table = model.quoted_table_name
-          column = "#{table}.#{model.connection.quote_column_name('tags')}"
+        def tagged_calls_sql(scope)
+          table = scope.klass.quoted_table_name
+          connection = scope.connection
+          column = "#{table}.#{connection.quote_column_name('tags')}"
 
-          if Ledger::Schema::Adapter.postgresql?(model.connection)
+          if Ledger::Schema::Adapter.postgresql?(connection)
             "COALESCE(SUM(CASE WHEN #{column} <> '{}'::jsonb THEN 1 ELSE 0 END), 0)"
           else
             "COALESCE(SUM(CASE WHEN JSON_LENGTH(#{column}) > 0 THEN 1 ELSE 0 END), 0)"

data/lib/llm_cost_tracker/capture/stream_collector.rb

@@ -10,13 +10,15 @@ module LlmCostTracker
     class StreamCollector
       attr_reader :provider
 
-      def initialize(provider:, model:, latency_ms: nil, provider_response_id: nil, pricing_mode: nil, metadata: {})
+      def initialize(provider:, model:, latency_ms: nil, provider_response_id: nil, pricing_mode: nil, metadata: {},
+                     context_tags: nil)
         @provider = provider.to_s
         @model = model
         @latency_ms = latency_ms
         @provider_response_id = provider_response_id
         @pricing_mode = pricing_mode
         @metadata = (metadata || {}).deep_dup
+        @context_tags = (context_tags || LlmCostTracker::Tags::Context.tags).deep_dup
         @events = []
         @captured_bytes = 0
         @overflowed = false
@@ -85,7 +87,8 @@ module LlmCostTracker
             latency_ms: @latency_ms,
             provider_response_id: @provider_response_id,
             pricing_mode: @pricing_mode,
-            metadata: @metadata.deep_dup
+            metadata: @metadata.deep_dup,
+            context_tags: @context_tags.deep_dup
           }
         end
 
@@ -98,7 +101,8 @@ module LlmCostTracker
           latency_ms: snapshot[:latency_ms] ||
             ((Process.clock_gettime(Process::CLOCK_MONOTONIC) - @started_at) * 1000).round,
           pricing_mode: snapshot[:pricing_mode],
-          metadata: (errored ? { stream_errored: true } : {}).merge(snapshot[:metadata])
+          metadata: (errored ? { stream_errored: true } : {}).merge(snapshot[:metadata]),
+          context_tags: snapshot[:context_tags]
         )
       end
 
@@ -114,7 +118,10 @@ module LlmCostTracker
         return build_from_explicit_usage(snapshot) if snapshot[:explicit_usage]
         return build_unknown_usage(snapshot) if snapshot[:overflowed]
 
-        capture = Parsers.find_for_provider(@provider)&.parse_stream(nil, nil, 200, snapshot[:events])
+        capture = Parsers.find_for_provider(@provider)&.parse_stream(
+          response_status: 200,
+          events: snapshot[:events]
+        )
         if capture
           model = present_model(capture.model) || present_model(snapshot[:model]) || UsageCapture::UNKNOWN_MODEL
           return capture.with(provider: @provider, model: model)

data/lib/llm_cost_tracker/capture/stream_tracker.rb

@@ -8,7 +8,7 @@ require_relative "../logging"
 module LlmCostTracker
   module Capture
     class StreamTracker
-      def initialize(stream, collector, active, finish)
+      def initialize(stream:, collector:, active:, finish: nil)
         @stream = stream
         @collector = collector
         @active = active

data/lib/llm_cost_tracker/configuration.rb

@@ -8,7 +8,11 @@ module LlmCostTracker
   class Configuration
     include ConfigurationInstrumentation
 
-    OPENAI_COMPATIBLE_PROVIDERS = { "openrouter.ai" => "openrouter", "api.deepseek.com" => "deepseek" }.freeze
+    OPENAI_COMPATIBLE_PROVIDERS = {
+      "openrouter.ai" => "openrouter",
+      "api.deepseek.com" => "deepseek",
+      "api.groq.com" => "groq"
+    }.freeze
 
     BUDGET_EXCEEDED_BEHAVIORS = %i[notify raise block_requests].freeze
     UNKNOWN_PRICING_BEHAVIORS = %i[ignore warn raise].freeze

data/lib/llm_cost_tracker/integrations/anthropic.rb

@@ -49,9 +49,8 @@ module LlmCostTracker
               capture: UsageCapture.build(
                 provider: "anthropic",
                 model: object_value(message, :model) || request[:model],
-                pricing_mode: object_value(usage, :service_tier) || object_value(message, :service_tier) ||
-                  request[:service_tier],
-                token_usage: token_usage(usage, input_tokens, output_tokens),
+                pricing_mode: pricing_mode(message: message, request: request, usage: usage),
+                token_usage: token_usage(usage: usage, input_tokens: input_tokens, output_tokens: output_tokens),
                 usage_source: :sdk_response,
                 provider_response_id: object_value(message, :id)
               ),
@@ -60,7 +59,7 @@ module LlmCostTracker
           end
         end
 
-        def token_usage(usage, input_tokens, output_tokens)
+        def token_usage(usage:, input_tokens:, output_tokens:)
           cache_write_1h = object_dig(usage, :cache_creation, :ephemeral_1h_input_tokens).to_i
           cache_write_5m = object_dig(usage, :cache_creation, :ephemeral_5m_input_tokens)
           cache_write = if cache_write_5m.nil?
@@ -84,14 +83,32 @@ module LlmCostTracker
           )
         end
 
+        def pricing_mode(message:, request:, usage:)
+          modes = [
+            Pricing.normalize_mode(object_value(usage, :speed) || object_value(message, :speed) || request[:speed]),
+            Pricing.normalize_mode(
+              object_value(usage, :service_tier) || object_value(message, :service_tier) || request[:service_tier]
+            )
+          ]
+          modes << "data_residency" if inference_geo(message: message, request: request, usage: usage).to_s == "us"
+          modes = modes.compact.uniq
+          modes.empty? ? nil : modes.join("_")
+        end
+
+        def inference_geo(message:, request:, usage:)
+          object_value(usage, :inference_geo) ||
+            object_value(message, :inference_geo) ||
+            request[:inference_geo]
+        end
+
         def track_stream(stream, collector:)
           return stream unless active?
 
           LlmCostTracker::Capture::StreamTracker.new(
-            stream,
-            collector,
-            -> { active? },
-            ->(errored:) { finish_stream(collector, errored: errored) }
+            stream: stream,
+            collector: collector,
+            active: -> { active? },
+            finish: ->(errored:) { finish_stream(collector, errored: errored) }
           ).wrap
         end
 

data/lib/llm_cost_tracker/integrations/openai.rb

@@ -90,10 +90,10 @@ module LlmCostTracker
           return stream unless active?
 
           LlmCostTracker::Capture::StreamTracker.new(
-            stream,
-            collector,
-            -> { active? },
-            ->(errored:) { finish_stream(collector, errored: errored) }
+            stream: stream,
+            collector: collector,
+            active: -> { active? },
+            finish: ->(errored:) { finish_stream(collector, errored: errored) }
           ).wrap
         end
 

data/lib/llm_cost_tracker/ledger/rollups/upsert_sql.rb

@@ -6,12 +6,8 @@ module LlmCostTracker
   module Ledger
     class Rollups
       class UpsertSql
-        def self.call(model)
-          new(model).call
-        end
-
-        def initialize(model)
-          @model = model
+        def self.call
+          new.call
         end
 
         def call
@@ -23,13 +19,11 @@ module LlmCostTracker
 
         private
 
-        attr_reader :model
-
         def postgres_sql
           total_cost = connection.quote_column_name("total_cost")
           updated_at = connection.quote_column_name("updated_at")
 
-          "#{total_cost} = #{model.quoted_table_name}.#{total_cost} + excluded.#{total_cost}, " \
+          "#{total_cost} = #{Period::Total.quoted_table_name}.#{total_cost} + excluded.#{total_cost}, " \
             "#{updated_at} = excluded.#{updated_at}"
         end
 
@@ -38,7 +32,7 @@ module LlmCostTracker
         end
 
         def connection
-          model.connection
+          Period::Total.connection
         end
       end
     end

data/lib/llm_cost_tracker/ledger/rollups.rb

@@ -15,9 +15,9 @@ module LlmCostTracker
 
           Period::Total.upsert_all(
             period_rows(event),
-            on_duplicate: Ledger::Rollups::UpsertSql.call(Period::Total),
+            on_duplicate: Ledger::Rollups::UpsertSql.call,
             record_timestamps: true,
-            unique_by: unique_by(Period::Total, %i[period period_start])
+            unique_by: period_totals_unique_by
           )
         end
 
@@ -27,9 +27,9 @@ module LlmCostTracker
 
           Period::Total.upsert_all(
             Ledger::Rollups::Batch.rows(events),
-            on_duplicate: Ledger::Rollups::UpsertSql.call(Period::Total),
+            on_duplicate: Ledger::Rollups::UpsertSql.call,
             record_timestamps: true,
-            unique_by: unique_by(Period::Total, %i[period period_start])
+            unique_by: period_totals_unique_by
           )
         end
 
@@ -76,10 +76,10 @@ module LlmCostTracker
           end
         end
 
-        def unique_by(model, column)
-          return unless model.connection.supports_insert_conflict_target?
+        def period_totals_unique_by
+          return unless Period::Total.connection.supports_insert_conflict_target?
 
-          column
+          %i[period period_start]
         end
       end
     end

data/lib/llm_cost_tracker/ledger/store.rb

@@ -11,12 +11,11 @@ module LlmCostTracker
           events = Array(events)
           return [] if events.empty?
 
-          model = LlmCostTracker::Ledger::Call
-          insertable = new_events(model, events)
+          insertable = insertable_events(events)
 
           if insertable.any?
             rows = insertable.map { |event| attributes_for(event) }
-            model.insert_all!(rows, record_timestamps: true, returning: false)
+            Ledger::Call.insert_all!(rows, record_timestamps: true, returning: false)
             Ledger::Rollups.increment_many!(insertable)
           end
           events
@@ -25,14 +24,11 @@ module LlmCostTracker
         private
 
         def attributes_for(event)
-          tags = (event.tags || {}).transform_keys(&:to_s).transform_values { |value| stringify_tag_value(value) }
-          usage = event.token_usage.stored_attributes
-
           attributes = {
             event_id: event.event_id,
             provider: event.provider,
             model: event.model,
-            tags: tags,
+            tags: stored_tags(event.tags),
             tracked_at: event.tracked_at,
             pricing_mode: event.pricing_mode,
             latency_ms: event.latency_ms,
@@ -41,16 +37,29 @@ module LlmCostTracker
             provider_response_id: event.provider_response_id
           }
 
-          attributes.merge(usage).merge(Pricing.stored_cost_attributes(event.cost || {}))
+          attributes
+            .merge(event.token_usage.stored_attributes)
+            .merge(Pricing.stored_cost_attributes(event.cost || {}))
+        end
+
+        def insertable_events(events)
+          existing_ids = Ledger::Call.where(event_id: events.map(&:event_id)).pluck(:event_id).to_set
+          seen_ids = Set.new
+
+          events.select do |event|
+            event_id = event.event_id
+            !existing_ids.include?(event_id) && seen_ids.add?(event_id)
+          end
         end
 
-        def new_events(model, events)
-          existing_ids = model.where(event_id: events.map(&:event_id)).pluck(:event_id).to_set
-          events.reject { |event| existing_ids.include?(event.event_id) }
+        def stored_tags(tags)
+          (tags || {}).transform_keys(&:to_s).transform_values { |value| stored_tag_value(value) }
         end
 
-        def stringify_tag_value(value)
-          return value.transform_values { |nested| stringify_tag_value(nested) } if value.is_a?(Hash)
+        def stored_tag_value(value)
+          if value.is_a?(Hash)
+            return value.transform_keys(&:to_s).transform_values { |nested| stored_tag_value(nested) }
+          end
 
           value.to_s
         end

data/lib/llm_cost_tracker/ledger/tags/query.rb

@@ -9,17 +9,17 @@ module LlmCostTracker
     module Tags
       module Query
         class << self
-          def apply(model, tags)
+          def apply(tags)
             normalized_tags = (tags || {}).to_h.transform_keys(&:to_s).transform_values(&:to_s)
-            return model.all if normalized_tags.empty?
+            return Ledger::Call.all if normalized_tags.empty?
 
-            connection = model.connection
+            connection = Ledger::Call.connection
             json = normalized_tags.to_json
 
             if Schema::Adapter.postgresql?(connection)
-              model.where("tags @> ?::jsonb", json)
+              Ledger::Call.where("tags @> ?::jsonb", json)
             else
-              model.where("JSON_CONTAINS(tags, ?)", json)
+              Ledger::Call.where("JSON_CONTAINS(tags, ?)", json)
             end
           end
         end

data/lib/llm_cost_tracker/ledger/tags/sql.rb

@@ -8,16 +8,17 @@ module LlmCostTracker
     module Tags
       module Sql
         class << self
-          def value_expression(model, key, table_name:)
+          def value_expression(key, table_name:)
             key = LlmCostTracker::Tags::Key.validate!(key)
-            column = "#{table_name}.#{model.connection.quote_column_name('tags')}"
+            connection = Ledger::Call.connection
+            column = "#{table_name}.#{connection.quote_column_name('tags')}"
 
-            if Ledger::Schema::Adapter.postgresql?(model.connection)
-              "#{column}->>#{model.connection.quote(key)}"
-            elsif Ledger::Schema::Adapter.mysql?(model.connection)
-              "JSON_UNQUOTE(JSON_EXTRACT(#{column}, #{model.connection.quote(json_path(key))}))"
+            if Ledger::Schema::Adapter.postgresql?(connection)
+              "#{column}->>#{connection.quote(key)}"
+            elsif Ledger::Schema::Adapter.mysql?(connection)
+              "JSON_UNQUOTE(JSON_EXTRACT(#{column}, #{connection.quote(json_path(key))}))"
             else
-              Ledger::Schema::Adapter.ensure_supported!(model.connection)
+              Ledger::Schema::Adapter.ensure_supported!(connection)
             end
           end