llm_cost_tracker 0.1.3 → 0.1.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 0d1192ed209333057bd2522173d05530b4f45c6bb63242189c75354a83b5a746
-  data.tar.gz: 486555221b66a0da6cb867d207fb76349f0d56a23da623418da35aab7672875f
+  metadata.gz: cb8f35c3d464bc7adea3b8376055e23c2ff704d815247de8ec40c82cbab41d29
+  data.tar.gz: ec3fecd6c98f9c3c9664f12db2ca411440d698f5743d2ac77827e86f04fca380
 SHA512:
-  metadata.gz: 8e74531effe3fc425de0384c13c7717c54b8be8d683493c92665b356ed8142d2629c9ce555f5fff703c2cd4a676d69e82efb39de767fc75fdbdcabdca9289f2c
-  data.tar.gz: 38e9744e157248e67bebcdd818b356c06b923d75a216b406f96f0b1b10368d4a118cbb9593461e2a19d19bdb5b10fc115c2f871d15fcf8669e656ad8ea8e034a
+  metadata.gz: 76ccf3b2d160eb3d9e7dd22545e7a53515f48334f6fd7a6aae89684ad660721eb1fcedc27f572b669f95da91e4afe01b9bc5e6762801a36b0ac78fbbb5229b80
+  data.tar.gz: 7456155944b169f3a5c293c0d256b1a4200de22f24f1433f387475fdefedb9a5ad1e66776b0263eba9ea7be4de2e9fbd4315346c05cc1d0dfa76cadd94699199

data/CHANGELOG.md

@@ -5,6 +5,22 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.1.4] - 2026-04-18
+
+### Breaking Changes
+
+- Removed `LlmApiCall.by_user(id)` and `LlmApiCall.by_feature(name)` convenience scopes. Use
+  `by_tag("user_id", id)`, `by_tag("feature", name)`, or `by_tags(...)` for filters.
+- Removed `LlmApiCall#user_id` and `LlmApiCall#feature` tag accessors. Use
+  `parsed_tags["user_id"]` or `parsed_tags["feature"]` when reading stored tags.
+- Removed `ReportData#cost_by_feature`. Use `ReportData#cost_by_tags.fetch("feature")` or
+  `LlmApiCall.cost_by_tag("feature")`.
+
+### Added
+
+- Add SQL-side `group_by_tag(key)` and `cost_by_tag(key)` aggregations across any attribution tag.
+- Use generic tag breakdowns in reports instead of feature-specific report data.
+
 ## [0.1.3] - 2026-04-18
 
 ### Thread-safety, pricing UX, and internal hardening
@@ -17,7 +33,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 - Warn on unknown keys in local prices files.
 - Add `llm_cost_tracker:prices` generator for creating a local price override template.
-- Document that budget enforcement skips events with unknown pricing.
+- Document that budget guardrails skip events with unknown pricing.
 
 **Onboarding UX**
 

data/README.md

@@ -20,7 +20,7 @@ By model:
   claude-sonnet-4-6           $31.200000
   gemini-2.5-flash            $14.120000
 
-By feature:
+By tag (feature):
   chat                        $73.500000
   summarizer                  $29.220000
   translate                   $24.700000
@@ -104,7 +104,7 @@ OpenAI.configure do |config|
     f.use :llm_cost_tracker, tags: -> {
       {
         user_id: Current.user&.id,
-        feature: Current.llm_feature || "openai"
+        feature: Current.llm_feature || "chat"
       }
     }
   end
@@ -220,7 +220,7 @@ config.unknown_pricing_behavior = :raise # fail fast with UnknownPricingError
 config.unknown_pricing_behavior = :ignore # keep tracking tokens silently
 ```
 
-When pricing is unknown, the event can still be recorded with token counts, but `cost` is `nil` and budget enforcement is skipped for that event. Use `prices_file` or `pricing_overrides` to ensure all production models are priced. Check this ActiveRecord query for a list of unpriced models in your data:
+When pricing is unknown, the event can still be recorded with token counts, but `cost` is `nil` and budget guardrails are skipped for that event. Use `prices_file` or `pricing_overrides` to ensure all production models are priced. Check this ActiveRecord query for a list of unpriced models in your data:
 
 ```ruby
 LlmCostTracker::LlmApiCall.unknown_pricing.group(:model).count
@@ -289,7 +289,7 @@ end
 
 Pre-request blocking needs `storage_backend = :active_record` because the middleware must query your stored monthly total before sending the request. With `:log` or `:custom` storage, `:raise` and the post-response part of `:block_requests` still work for the event being tracked.
 
-`:block_requests` is a best-effort guardrail, not a transactional hard quota. In highly concurrent deployments, multiple workers can pass the preflight check at the same time before any of them records its final cost. The request that first pushes the month over budget is stored before the post-response `BudgetExceededError` is raised; later Faraday requests are blocked during preflight once the stored monthly total is exhausted. Use provider-side limits or a gateway-level quota if you need strict cross-process enforcement.
+`:block_requests` is a best-effort guardrail, not a transactional hard quota. In highly concurrent deployments, multiple workers can pass the preflight check at the same time before any of them records its final cost. The request that first pushes the month over budget is stored before the post-response `BudgetExceededError` is raised; later Faraday requests are blocked during preflight once the stored monthly total is exhausted. Use provider-side limits or a gateway-level quota if you need strict cross-process caps.
 
 ## Querying Costs (ActiveRecord)
 
@@ -332,6 +332,15 @@ LlmCostTracker::LlmApiCall.this_month.cost_by_model
 LlmCostTracker::LlmApiCall.this_month.cost_by_provider
 # => { "openai" => 8.20, "anthropic" => 4.25 }
 
+# SQL-side cost breakdown by any tag key
+calls = LlmCostTracker::LlmApiCall.this_month
+calls.group_by_tag("feature").sum(:total_cost)
+# => { "chat" => 7.10, "summarizer" => 1.10 }
+
+# Convenience wrapper with "(untagged)" labels and float values
+calls.cost_by_tag("feature")
+# => { "chat" => 7.10, "summarizer" => 1.10 }
+
 # Daily cost trend
 LlmCostTracker::LlmApiCall.daily_costs(days: 7)
 # => { "2026-04-10" => 1.5, "2026-04-11" => 2.3, ... }
@@ -340,19 +349,15 @@ LlmCostTracker::LlmApiCall.daily_costs(days: 7)
 LlmCostTracker::LlmApiCall.with_latency.average_latency_ms
 LlmCostTracker::LlmApiCall.this_month.latency_by_model
 
-# Filter by feature
+# Filter by one tag
 LlmCostTracker::LlmApiCall.by_tag("feature", "chat").this_month.total_cost
 
-# Filter by user
+# Filter by another tag
 LlmCostTracker::LlmApiCall.by_tag("user_id", "42").today.total_cost
-LlmCostTracker::LlmApiCall.by_user(42).today.total_cost
 
 # Filter by multiple tags
 LlmCostTracker::LlmApiCall.by_tags(user_id: 42, feature: "chat").this_month.total_cost
 
-# Feature shortcut
-LlmCostTracker::LlmApiCall.by_feature("summarizer").this_month.total_cost
-
 # Find models without pricing
 LlmCostTracker::LlmApiCall.unknown_pricing.group(:model).count
 LlmCostTracker::LlmApiCall.with_cost.this_month.total_cost
@@ -481,7 +486,7 @@ This covers OpenRouter, DeepSeek, and private gateways that expose OpenAI-style
 - Treat `:block_requests` as best-effort in concurrent systems, not a strict quota.
 - Keep `unknown_pricing_behavior = :warn` or `:raise` until pricing overrides are complete.
 - Add `pricing_overrides` for custom, fine-tuned, gateway-specific, or newly released models.
-- Tag calls with `tenant_id`, `user_id`, and `feature` where possible.
+- Tag calls with useful business context such as `tenant_id`, `user_id`, and `feature`.
 - Check `LlmCostTracker::LlmApiCall.unknown_pricing.group(:model).count` after deploys.
 - Track `latency_ms` and watch `latency_by_model` for slow or degraded providers.
 

data/lib/llm_cost_tracker/llm_api_call.rb

@@ -8,6 +8,10 @@ require_relative "tags_column"
 
 module LlmCostTracker
   class LlmApiCall < ActiveRecord::Base
+    TAG_KEY_PATTERN = /\A[\w.-]+\z/
+
+    private_constant :TAG_KEY_PATTERN
+
     extend TagsColumn
     include TagAccessors
 
@@ -16,8 +20,6 @@ module LlmCostTracker
     # Scopes for querying
     scope :by_provider, ->(provider) { where(provider: provider) }
     scope :by_model,    ->(model)    { where(model: model) }
-    scope :by_user,    ->(user_id) { by_tag("user_id", user_id) }
-    scope :by_feature, ->(feature) { by_tag("feature", feature) }
     scope :with_cost, -> { where.not(total_cost: nil) }
     scope :without_cost, -> { where(total_cost: nil) }
     scope :unknown_pricing, -> { without_cost }
@@ -61,6 +63,17 @@ module LlmCostTracker
       group(:provider).sum(:total_cost)
     end
 
+    def self.group_by_tag(key)
+      group(Arel.sql(tag_group_expression(key)))
+    end
+
+    def self.cost_by_tag(key)
+      costs = group_by_tag(key).sum(:total_cost).each_with_object(Hash.new(0.0)) do |(tag_value, cost), grouped|
+        grouped[tag_label(tag_value)] += cost.to_f
+      end
+      costs.sort_by { |_label, cost| -cost }.to_h
+    end
+
     def self.average_latency_ms
       return nil unless latency_column?
 
@@ -85,5 +98,39 @@ module LlmCostTracker
         .sum(:total_cost)
         .transform_keys(&:to_s)
     end
+
+    def self.tag_label(value)
+      value.nil? || value == "" ? "(untagged)" : value.to_s
+    end
+    private_class_method :tag_label
+
+    def self.tag_group_expression(key)
+      key = validated_tag_key(key)
+      column = "#{quoted_table_name}.#{connection.quote_column_name('tags')}"
+
+      case connection.adapter_name
+      when /postgres/i
+        json_column = tags_json_column? ? column : "(#{column})::jsonb"
+        "#{json_column}->>#{connection.quote(key)}"
+      when /mysql/i
+        "JSON_UNQUOTE(JSON_EXTRACT(#{column}, #{connection.quote(json_path(key))}))"
+      else
+        "json_extract(#{column}, #{connection.quote(json_path(key))})"
+      end
+    end
+    private_class_method :tag_group_expression
+
+    def self.validated_tag_key(key)
+      key = key.to_s
+      return key if key.match?(TAG_KEY_PATTERN)
+
+      raise ArgumentError, "invalid tag key: #{key.inspect}"
+    end
+    private_class_method :validated_tag_key
+
+    def self.json_path(key)
+      "$.\"#{key}\""
+    end
+    private_class_method :json_path
   end
 end

data/lib/llm_cost_tracker/report_data.rb

@@ -15,12 +15,13 @@ module LlmCostTracker
     :unknown_pricing_count,
     :cost_by_provider,
     :cost_by_model,
-    :cost_by_feature,
+    :cost_by_tags,
     :top_calls
   )
 
   ReportData.const_set(:DEFAULT_DAYS, 30)
   ReportData.const_set(:TOP_LIMIT, 5)
+  ReportData.const_set(:DEFAULT_TAG_BREAKDOWNS, %w[feature].freeze)
 
   class << ReportData
     def build(days: ReportData::DEFAULT_DAYS, now: Time.now.utc)
@@ -39,7 +40,7 @@ module LlmCostTracker
         unknown_pricing_count: scope.where(total_cost: nil).count,
         cost_by_provider: cost_by(scope, :provider),
         cost_by_model: cost_by(scope, :model),
-        cost_by_feature: cost_by_feature(scope),
+        cost_by_tags: cost_by_tags(scope, ReportData::DEFAULT_TAG_BREAKDOWNS),
         top_calls: top_calls(scope)
       )
     end
@@ -65,12 +66,8 @@ module LlmCostTracker
       scope.group(column).sum(:total_cost).transform_values(&:to_f).sort_by { |_name, cost| -cost }
     end
 
-    def cost_by_feature(scope)
-      costs = Hash.new(0.0)
-      scope.select(:id, :tags, :total_cost).find_each do |call|
-        costs[call.feature || "(untagged)"] += call.total_cost.to_f
-      end
-      costs.sort_by { |_feature, cost| -cost }
+    def cost_by_tags(scope, keys)
+      keys.to_h { |key| [key, scope.cost_by_tag(key).to_a] }
     end
 
     def top_calls(scope)

data/lib/llm_cost_tracker/report_formatter.rb

@@ -13,7 +13,7 @@ module LlmCostTracker
       append_summary(lines)
       append_cost_section(lines, "By provider", @data.cost_by_provider)
       append_cost_section(lines, "By model", @data.cost_by_model)
-      append_cost_section(lines, "By feature", @data.cost_by_feature)
+      append_tag_sections(lines)
       append_top_calls(lines)
       lines.join("\n")
     end
@@ -37,6 +37,12 @@ module LlmCostTracker
       end
     end
 
+    def append_tag_sections(lines)
+      @data.cost_by_tags.each do |tag_key, rows|
+        append_cost_section(lines, "By tag (#{tag_key})", rows)
+      end
+    end
+
     def append_top_calls(lines)
       lines << ""
       lines << "Top expensive calls:"

data/lib/llm_cost_tracker/tag_accessors.rb

@@ -11,13 +11,5 @@ module LlmCostTracker
     rescue JSON::ParserError
       {}
     end
-
-    def feature
-      parsed_tags["feature"]
-    end
-
-    def user_id
-      parsed_tags["user_id"]
-    end
   end
 end

data/lib/llm_cost_tracker/unknown_pricing.rb

@@ -27,7 +27,7 @@ module LlmCostTracker
       def warn_missing(model)
         Logging.warn(
           "No pricing configured for model #{model.inspect}. " \
-          "Cost and budget enforcement will be skipped for this event. " \
+          "Cost and budget guardrails will be skipped for this event. " \
           "Add a pricing_overrides entry or set unknown_pricing_behavior."
         )
       end

data/lib/llm_cost_tracker/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module LlmCostTracker
-  VERSION = "0.1.3"
+  VERSION = "0.1.4"
 end

data/llm_cost_tracker.gemspec

@@ -12,7 +12,7 @@ Gem::Specification.new do |spec|
   spec.description   = "Tracks token usage and estimated costs for OpenAI, Anthropic, Google Gemini, " \
                        "OpenRouter, DeepSeek, and OpenAI-compatible calls. " \
                        "Works as Faraday middleware for Ruby clients, with ActiveRecord storage, " \
-                       "per-user/per-feature attribution, budget alerts, and budget enforcement."
+                       "per-user/per-feature attribution, and budget guardrails."
   spec.homepage      = "https://github.com/sergey-homenko/llm_cost_tracker"
   spec.license       = "MIT"
 

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: llm_cost_tracker
 version: !ruby/object:Gem::Version
-  version: 0.1.3
+  version: 0.1.4
 platform: ruby
 authors:
 - Sergii Khomenko
@@ -142,8 +142,8 @@ dependencies:
         version: '3.0'
 description: Tracks token usage and estimated costs for OpenAI, Anthropic, Google
   Gemini, OpenRouter, DeepSeek, and OpenAI-compatible calls. Works as Faraday middleware
-  for Ruby clients, with ActiveRecord storage, per-user/per-feature attribution, budget
-  alerts, and budget enforcement.
+  for Ruby clients, with ActiveRecord storage, per-user/per-feature attribution, and
+  budget guardrails.
 email:
 - sergey@mm.st
 executables: []