llm_cost_tracker 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 16c6c4c230300b2caebc69e1f0aeb9a7af232278a6e77401aba0d490bb16b4a2
+  data.tar.gz: 4b32fb25d22c645e4d66767264130bc44ce730e553636cefca8e4884dede7b94
+SHA512:
+  metadata.gz: f403ebeeb6a98164bc2318b9f3b8f49e03750fd5c5d328ac0b0f3c0557f16c39d8f247aa663bdd15b605df779be7d4b4db7445fb79d10eb4c89ef17a687a77d7
+  data.tar.gz: fd04a24708901e998127d6582290da90b35f62f7a36d794805a4677656be9191eed14235b634b72924ac4178837aa9dbace3a57fde18829d47b6a8208e295af2

data/.rspec

@@ -0,0 +1,3 @@
+--require spec_helper
+--format documentation
+--color

data/CHANGELOG.md

@@ -0,0 +1,23 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [0.1.0] - 2026-04-16
+
+### Added
+
+- Faraday middleware for automatic LLM API call interception
+- Provider parsers: OpenAI, Anthropic, Google Gemini
+- Built-in pricing table for 20+ models
+- Fuzzy model name matching (e.g. `gpt-4o-2024-08-06` → `gpt-4o`)
+- ActiveSupport::Notifications integration
+- ActiveRecord storage backend with scopes and aggregations
+- Manual `LlmCostTracker.track()` for non-Faraday clients
+- Per-user / per-feature tagging
+- Monthly budget alerts with configurable callbacks
+- Rails generator: `rails generate llm_cost_tracker:install`
+- Custom storage backend support
+- Pricing overrides via configuration

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2026 Sergii Khomenko
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,248 @@
+# LlmCostTracker
+
+**Provider-agnostic LLM API cost tracking for Ruby.**
+
+Track token usage and costs for every LLM API call your app makes — OpenAI, Anthropic, Google Gemini, and any OpenAI-compatible provider. Works as Faraday middleware, so it plugs into **any** Ruby LLM client without code changes.
+
+[![Gem Version](https://badge.fury.io/rb/llm_cost_tracker.svg)](https://rubygems.org/gems/llm_cost_tracker)
+
+## Why?
+
+Every Rails app integrating LLMs faces the same problem: **you don't know how much AI is costing you** until the invoice arrives. Existing solutions either lock you into a specific LLM gem (like `ruby_llm-monitoring`) or require external SaaS (Langfuse, Helicone).
+
+`llm_cost_tracker` takes a different approach:
+
+- 🔌 **Provider-agnostic** — intercepts HTTP responses at the Faraday level
+- 🏠 **Self-hosted** — your data stays in your database
+- 🧩 **Zero coupling** — works with `ruby-openai`, `anthropic-rb`, `ruby_llm`, or raw Faraday
+- ⚡ **Zero config** — add the middleware, done
+
+## Installation
+
+Add to your Gemfile:
+
+```ruby
+gem "llm_cost_tracker"
+```
+
+For ActiveRecord storage (recommended for production):
+
+```bash
+bin/rails generate llm_cost_tracker:install
+bin/rails db:migrate
+```
+
+## Quick Start
+
+### Option 1: Faraday Middleware (automatic)
+
+If your LLM client uses Faraday (most do), just add the middleware:
+
+```ruby
+conn = Faraday.new(url: "https://api.openai.com") do |f|
+  f.use :llm_cost_tracker, tags: { feature: "chat", user_id: current_user.id }
+  f.request :json
+  f.response :json
+  f.adapter Faraday.default_adapter
+end
+
+# Every request through this connection is now tracked automatically
+response = conn.post("/v1/chat/completions", {
+  model: "gpt-4o",
+  messages: [{ role: "user", content: "Hello!" }]
+})
+```
+
+### Option 2: Patch an existing client
+
+Most LLM gems expose their Faraday connection. For example, with `ruby-openai`:
+
+```ruby
+# config/initializers/openai.rb
+OpenAI.configure do |config|
+  config.access_token = ENV["OPENAI_API_KEY"]
+
+  config.faraday do |f|
+    f.use :llm_cost_tracker, tags: { feature: "openai_default" }
+  end
+end
+```
+
+### Option 3: Manual tracking
+
+For non-Faraday clients, track manually:
+
+```ruby
+LlmCostTracker.track(
+  provider: :anthropic,
+  model: "claude-sonnet-4-6",
+  input_tokens: 1500,
+  output_tokens: 320,
+  feature: "summarizer",
+  user_id: current_user.id
+)
+```
+
+## Configuration
+
+```ruby
+# config/initializers/llm_cost_tracker.rb
+LlmCostTracker.configure do |config|
+  # Storage: :log (default), :active_record, or :custom
+  config.storage_backend = :active_record
+
+  # Default tags on every event
+  config.default_tags = { app: "my_app", environment: Rails.env }
+
+  # Monthly budget in USD
+  config.monthly_budget = 500.00
+
+  # Alert callback
+  config.on_budget_exceeded = ->(data) {
+    SlackNotifier.notify(
+      "#alerts",
+      "🚨 LLM budget exceeded! $#{data[:monthly_total].round(2)} / $#{data[:budget]}"
+    )
+  }
+
+  # Override pricing for custom/fine-tuned models (per 1M tokens)
+  config.pricing_overrides = {
+    "ft:gpt-4o-mini:my-org" => { input: 0.30, output: 1.20 }
+  }
+end
+```
+
+## Querying Costs (ActiveRecord)
+
+```ruby
+# Today's total spend
+LlmCostTracker::LlmApiCall.today.total_cost
+# => 12.45
+
+# Cost breakdown by model this month
+LlmCostTracker::LlmApiCall.this_month.cost_by_model
+# => { "gpt-4o" => 8.20, "claude-sonnet-4-6" => 4.25 }
+
+# Cost by provider
+LlmCostTracker::LlmApiCall.this_month.cost_by_provider
+# => { "openai" => 8.20, "anthropic" => 4.25 }
+
+# Daily cost trend
+LlmCostTracker::LlmApiCall.daily_costs(days: 7)
+# => { "2026-04-10" => 1.5, "2026-04-11" => 2.3, ... }
+
+# Filter by feature
+LlmCostTracker::LlmApiCall.by_tag("feature", "chat").this_month.total_cost
+
+# Filter by user
+LlmCostTracker::LlmApiCall.by_tag("user_id", "42").today.total_cost
+
+# Custom date range
+LlmCostTracker::LlmApiCall.between(1.week.ago, Time.current).cost_by_model
+```
+
+## ActiveSupport::Notifications
+
+Every tracked call emits an `llm_request.llm_cost_tracker` event:
+
+```ruby
+ActiveSupport::Notifications.subscribe("llm_request.llm_cost_tracker") do |*, payload|
+  # payload =>
+  # {
+  #   provider: "openai",
+  #   model: "gpt-4o",
+  #   input_tokens: 150,
+  #   output_tokens: 42,
+  #   total_tokens: 192,
+  #   cost: { input_cost: 0.000375, output_cost: 0.00042, total_cost: 0.000795, currency: "USD" },
+  #   tags: { feature: "chat", user_id: 42 },
+  #   tracked_at: 2026-04-16 14:30:00 UTC
+  # }
+
+  StatsD.increment("llm.requests", tags: ["provider:#{payload[:provider]}"])
+  StatsD.histogram("llm.cost", payload[:cost][:total_cost])
+end
+```
+
+## Custom Storage Backend
+
+```ruby
+LlmCostTracker.configure do |config|
+  config.storage_backend = :custom
+  config.custom_storage = ->(event) {
+    InfluxDB.write("llm_costs", {
+      values: { cost: event[:cost][:total_cost], tokens: event[:total_tokens] },
+      tags: { provider: event[:provider], model: event[:model] }
+    })
+  }
+end
+```
+
+## Adding a Custom Provider Parser
+
+```ruby
+class DeepSeekParser < LlmCostTracker::Parsers::Base
+  def match?(url)
+    url.to_s.include?("api.deepseek.com")
+  end
+
+  def parse(request_url, request_body, response_status, response_body)
+    return nil unless response_status == 200
+
+    response = safe_json_parse(response_body)
+    usage = response["usage"]
+    return nil unless usage
+
+    {
+      provider: "deepseek",
+      model: response["model"],
+      input_tokens: usage["prompt_tokens"] || 0,
+      output_tokens: usage["completion_tokens"] || 0
+    }
+  end
+end
+
+# Register it
+LlmCostTracker::Parsers::Registry.register(DeepSeekParser.new)
+```
+
+## Supported Providers
+
+| Provider | Auto-detected | Models with pricing |
+|----------|:---:|---|
+| OpenAI | ✅ | GPT-4o, GPT-4o-mini, GPT-4-turbo, GPT-4, GPT-3.5-turbo, o1, o1-mini, o3-mini |
+| Anthropic | ✅ | Claude Opus 4.6, Sonnet 4.6, Haiku 4.5, Claude 3.5 Sonnet, Claude 3 Opus |
+| Google Gemini | ✅ | Gemini 2.5 Pro/Flash, 2.0 Flash, 1.5 Pro/Flash |
+| Any other | 🔧 | Via custom parser (see above) |
+
+## How It Works
+
+```
+Your App → Faraday → [LlmCostTracker Middleware] → LLM API
+                              ↓
+                     Parses response body
+                     Extracts token usage
+                     Calculates cost
+                              ↓
+               ActiveSupport::Notifications
+               ActiveRecord / Log / Custom
+```
+
+The middleware intercepts **outgoing** HTTP responses (not incoming requests), parses the `usage` object from the LLM provider's response body, looks up pricing, and records the event. It never modifies requests or responses — it's read-only.
+
+## Development
+
+```bash
+git clone https://github.com/sergey-homenko/llm_cost_tracker.git
+cd llm_cost_tracker
+bundle install
+bundle exec rspec
+```
+
+## Contributing
+
+Bug reports and pull requests are welcome on [GitHub](https://github.com/sergey-homenko/llm_cost_tracker).
+
+## License
+
+The gem is available as open source under the terms of the [MIT License](LICENSE.txt).

data/Rakefile

@@ -0,0 +1,8 @@
+# frozen_string_literal: true
+
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+
+RSpec::Core::RakeTask.new(:spec)
+
+task default: :spec

data/lib/llm_cost_tracker/configuration.rb

@@ -0,0 +1,33 @@
+# frozen_string_literal: true
+
+module LlmCostTracker
+  class Configuration
+    attr_accessor :enabled,
+                  :storage_backend,    # :log, :active_record, :custom
+                  :custom_storage,     # callable object for :custom backend
+                  :default_tags,       # Hash of default tags added to every event
+                  :on_budget_exceeded, # callable, receives event hash
+                  :monthly_budget,     # Float, in USD — nil means no limit
+                  :log_level,          # :debug, :info, :warn
+                  :pricing_overrides   # Hash to override built-in pricing
+
+    def initialize
+      @enabled            = true
+      @storage_backend    = :log
+      @custom_storage     = nil
+      @default_tags       = {}
+      @on_budget_exceeded = nil
+      @monthly_budget     = nil
+      @log_level          = :info
+      @pricing_overrides  = {}
+    end
+
+    def active_record?
+      storage_backend == :active_record
+    end
+
+    def log?
+      storage_backend == :log
+    end
+  end
+end

data/lib/llm_cost_tracker/generators/llm_cost_tracker/install_generator.rb

@@ -0,0 +1,36 @@
+# frozen_string_literal: true
+
+require "rails/generators"
+require "rails/generators/active_record"
+
+module LlmCostTracker
+  module Generators
+    class InstallGenerator < Rails::Generators::Base
+      include ActiveRecord::Generators::Migration
+
+      source_root File.expand_path("templates", __dir__)
+
+      desc "Creates the LlmCostTracker migration and initializer"
+
+      def create_migration_file
+        migration_template(
+          "create_llm_api_calls.rb.erb",
+          "db/migrate/create_llm_api_calls.rb"
+        )
+      end
+
+      def create_initializer
+        template(
+          "initializer.rb.erb",
+          "config/initializers/llm_cost_tracker.rb"
+        )
+      end
+
+      private
+
+      def migration_version
+        "[#{ActiveRecord::VERSION::MAJOR}.#{ActiveRecord::VERSION::MINOR}]"
+      end
+    end
+  end
+end

data/lib/llm_cost_tracker/generators/llm_cost_tracker/templates/create_llm_api_calls.rb.erb

@@ -0,0 +1,23 @@
+class CreateLlmApiCalls < ActiveRecord::Migration<%= migration_version %>
+  def change
+    create_table :llm_api_calls do |t|
+      t.string  :provider,      null: false
+      t.string  :model,         null: false
+      t.integer :input_tokens,  null: false, default: 0
+      t.integer :output_tokens, null: false, default: 0
+      t.integer :total_tokens,  null: false, default: 0
+      t.decimal :input_cost,    precision: 12, scale: 8
+      t.decimal :output_cost,   precision: 12, scale: 8
+      t.decimal :total_cost,    precision: 12, scale: 8
+      t.text    :tags
+      t.datetime :tracked_at,   null: false
+
+      t.timestamps
+    end
+
+    add_index :llm_api_calls, :provider
+    add_index :llm_api_calls, :model
+    add_index :llm_api_calls, :tracked_at
+    add_index :llm_api_calls, [:provider, :tracked_at]
+  end
+end

data/lib/llm_cost_tracker/generators/llm_cost_tracker/templates/initializer.rb.erb

@@ -0,0 +1,27 @@
+# frozen_string_literal: true
+
+LlmCostTracker.configure do |config|
+  # Enable/disable tracking
+  config.enabled = true
+
+  # Storage backend: :log, :active_record, or :custom
+  config.storage_backend = :active_record
+
+  # Default tags added to every tracked event
+  # config.default_tags = { environment: Rails.env, app: "my_app" }
+
+  # Monthly budget in USD. Set to nil to disable budget alerts.
+  # config.monthly_budget = 100.00
+
+  # Callback when monthly budget is exceeded.
+  # config.on_budget_exceeded = ->(data) {
+  #   Rails.logger.warn "[LlmCostTracker] Budget exceeded! " \
+  #     "Monthly total: $#{data[:monthly_total]}, Budget: $#{data[:budget]}"
+  #   # Or send a Slack notification, email, etc.
+  # }
+
+  # Override built-in pricing for specific models (per 1M tokens, USD)
+  # config.pricing_overrides = {
+  #   "my-custom-model" => { input: 1.00, output: 2.00 }
+  # }
+end

data/lib/llm_cost_tracker/llm_api_call.rb

@@ -0,0 +1,56 @@
+# frozen_string_literal: true
+
+require "active_record"
+
+module LlmCostTracker
+  class LlmApiCall < ActiveRecord::Base
+    self.table_name = "llm_api_calls"
+
+    # Scopes for querying
+    scope :by_provider, ->(provider) { where(provider: provider) }
+    scope :by_model,    ->(model)    { where(model: model) }
+    scope :by_tag,      ->(key, value) { where("tags LIKE ?", "%\"#{key}\":\"#{value}\"%") }
+
+    scope :today,       -> { where(tracked_at: Time.now.utc.beginning_of_day..) }
+    scope :this_week,   -> { where(tracked_at: Time.now.utc.beginning_of_week..) }
+    scope :this_month,  -> { where(tracked_at: Time.now.utc.beginning_of_month..) }
+    scope :between,     ->(from, to) { where(tracked_at: from..to) }
+
+    # Aggregations
+    def self.total_cost
+      sum(:total_cost).to_f
+    end
+
+    def self.total_tokens
+      sum(:total_tokens).to_i
+    end
+
+    def self.cost_by_model
+      group(:model).sum(:total_cost)
+    end
+
+    def self.cost_by_provider
+      group(:provider).sum(:total_cost)
+    end
+
+    def self.daily_costs(days: 30)
+      where(tracked_at: days.days.ago..)
+        .group("DATE(tracked_at)")
+        .sum(:total_cost)
+    end
+
+    def parsed_tags
+      JSON.parse(tags || "{}")
+    rescue JSON::ParserError
+      {}
+    end
+
+    def feature
+      parsed_tags["feature"]
+    end
+
+    def user_id
+      parsed_tags["user_id"]
+    end
+  end
+end

data/lib/llm_cost_tracker/middleware/faraday.rb

@@ -0,0 +1,61 @@
+# frozen_string_literal: true
+
+require "faraday"
+
+module LlmCostTracker
+  module Middleware
+    class Faraday < ::Faraday::Middleware
+      def initialize(app, **options)
+        super(app)
+        @tags = options.fetch(:tags, {})
+      end
+
+      def call(request_env)
+        return @app.call(request_env) unless LlmCostTracker.configuration.enabled
+
+        request_url  = request_env.url.to_s
+        request_body = read_body(request_env.body)
+
+        @app.call(request_env).on_complete do |response_env|
+          process(request_url, request_body, response_env)
+        end
+      rescue StandardError => e
+        # Never break the actual request — log and re-raise
+        raise e
+      end
+
+      private
+
+      def process(request_url, request_body, response_env)
+        parser = Parsers::Registry.find_for(request_url)
+        return unless parser
+
+        parsed = parser.parse(
+          request_url,
+          request_body,
+          response_env.status,
+          read_body(response_env.body)
+        )
+        return unless parsed
+
+        Tracker.record(
+          provider: parsed[:provider],
+          model: parsed[:model],
+          input_tokens: parsed[:input_tokens],
+          output_tokens: parsed[:output_tokens],
+          metadata: @tags.merge(parsed.except(:provider, :model, :input_tokens, :output_tokens, :total_tokens))
+        )
+      rescue StandardError => e
+        warn "[LlmCostTracker] Error processing response: #{e.message}" if LlmCostTracker.configuration.log_level == :debug
+      end
+
+      def read_body(body)
+        case body
+        when String then body
+        when nil then ""
+        else body.to_s
+        end
+      end
+    end
+  end
+end

data/lib/llm_cost_tracker/parsers/anthropic.rb

@@ -0,0 +1,38 @@
+# frozen_string_literal: true
+
+require "json"
+
+module LlmCostTracker
+  module Parsers
+    class Anthropic < Base
+      HOSTS = %w[api.anthropic.com].freeze
+
+      def match?(url)
+        uri = URI.parse(url.to_s)
+        HOSTS.include?(uri.host) && uri.path.include?("/v1/messages")
+      rescue URI::InvalidURIError
+        false
+      end
+
+      def parse(request_url, request_body, response_status, response_body)
+        return nil unless response_status == 200
+
+        response = safe_json_parse(response_body)
+        usage    = response["usage"]
+        return nil unless usage
+
+        request = safe_json_parse(request_body)
+
+        {
+          provider: "anthropic",
+          model: response["model"] || request["model"],
+          input_tokens: usage["input_tokens"] || 0,
+          output_tokens: usage["output_tokens"] || 0,
+          total_tokens: (usage["input_tokens"] || 0) + (usage["output_tokens"] || 0),
+          cache_read_tokens: usage["cache_read_input_tokens"],
+          cache_creation_tokens: usage["cache_creation_input_tokens"]
+        }.compact
+      end
+    end
+  end
+end

data/lib/llm_cost_tracker/parsers/base.rb

@@ -0,0 +1,35 @@
+# frozen_string_literal: true
+
+module LlmCostTracker
+  module Parsers
+    class Base
+      # Returns a hash with parsed usage data, or nil if not applicable.
+      #
+      # Expected return format:
+      # {
+      #   provider: "openai",
+      #   model: "gpt-4o",
+      #   input_tokens: 150,
+      #   output_tokens: 42
+      # }
+      def parse(request_url, request_body, response_status, response_body)
+        raise NotImplementedError
+      end
+
+      # Returns true if this parser can handle the given URL.
+      def match?(url)
+        raise NotImplementedError
+      end
+
+      private
+
+      def safe_json_parse(body)
+        return {} if body.nil? || body.empty?
+
+        JSON.parse(body)
+      rescue JSON::ParserError
+        {}
+      end
+    end
+  end
+end

data/lib/llm_cost_tracker/parsers/gemini.rb

@@ -0,0 +1,47 @@
+# frozen_string_literal: true
+
+require "json"
+
+module LlmCostTracker
+  module Parsers
+    class Gemini < Base
+      HOSTS = %w[generativelanguage.googleapis.com].freeze
+
+      def match?(url)
+        uri = URI.parse(url.to_s)
+        HOSTS.include?(uri.host)
+      rescue URI::InvalidURIError
+        false
+      end
+
+      def parse(request_url, request_body, response_status, response_body)
+        return nil unless response_status == 200
+
+        response = safe_json_parse(response_body)
+        usage    = response["usageMetadata"]
+        return nil unless usage
+
+        # Extract model from URL: /v1beta/models/gemini-2.5-flash:generateContent
+        model = extract_model_from_url(request_url)
+
+        {
+          provider: "gemini",
+          model: model,
+          input_tokens: usage["promptTokenCount"] || 0,
+          output_tokens: usage["candidatesTokenCount"] || 0,
+          total_tokens: usage["totalTokenCount"] || 0
+        }
+      end
+
+      private
+
+      def extract_model_from_url(url)
+        uri = URI.parse(url.to_s)
+        match = uri.path.match(%r{/models/([^/:]+)})
+        match ? match[1] : "unknown"
+      rescue URI::InvalidURIError
+        "unknown"
+      end
+    end
+  end
+end

data/lib/llm_cost_tracker/parsers/openai.rb

@@ -0,0 +1,37 @@
+# frozen_string_literal: true
+
+require "json"
+
+module LlmCostTracker
+  module Parsers
+    class Openai < Base
+      HOSTS = %w[api.openai.com].freeze
+      TRACKED_PATHS = %w[/v1/chat/completions /v1/completions /v1/embeddings].freeze
+
+      def match?(url)
+        uri = URI.parse(url.to_s)
+        HOSTS.include?(uri.host) && TRACKED_PATHS.any? { |p| uri.path.start_with?(p) }
+      rescue URI::InvalidURIError
+        false
+      end
+
+      def parse(request_url, request_body, response_status, response_body)
+        return nil unless response_status == 200
+
+        response = safe_json_parse(response_body)
+        usage    = response["usage"]
+        return nil unless usage
+
+        request = safe_json_parse(request_body)
+
+        {
+          provider: "openai",
+          model: response["model"] || request["model"],
+          input_tokens: usage["prompt_tokens"] || 0,
+          output_tokens: usage["completion_tokens"] || 0,
+          total_tokens: usage["total_tokens"] || 0
+        }
+      end
+    end
+  end
+end

data/lib/llm_cost_tracker/parsers/registry.rb

@@ -0,0 +1,29 @@
+# frozen_string_literal: true
+
+module LlmCostTracker
+  module Parsers
+    class Registry
+      class << self
+        def parsers
+          @parsers ||= [
+            Openai.new,
+            Anthropic.new,
+            Gemini.new
+          ]
+        end
+
+        def register(parser)
+          parsers.unshift(parser)
+        end
+
+        def find_for(url)
+          parsers.find { |p| p.match?(url) }
+        end
+
+        def reset!
+          @parsers = nil
+        end
+      end
+    end
+  end
+end

data/lib/llm_cost_tracker/pricing.rb

@@ -0,0 +1,73 @@
+# frozen_string_literal: true
+
+module LlmCostTracker
+  # Prices per 1M tokens in USD.
+  # Updated: April 2026. Override via configuration.
+  module Pricing
+    PRICES = {
+      # OpenAI
+      "gpt-4o"             => { input: 2.50,  output: 10.00 },
+      "gpt-4o-mini"        => { input: 0.15,  output: 0.60 },
+      "gpt-4-turbo"        => { input: 10.00, output: 30.00 },
+      "gpt-4"              => { input: 30.00, output: 60.00 },
+      "gpt-3.5-turbo"      => { input: 0.50,  output: 1.50 },
+      "o1"                 => { input: 15.00, output: 60.00 },
+      "o1-mini"            => { input: 3.00,  output: 12.00 },
+      "o3-mini"            => { input: 1.10,  output: 4.40 },
+
+      # Anthropic
+      "claude-sonnet-4-6"  => { input: 3.00,  output: 15.00 },
+      "claude-opus-4-6"    => { input: 15.00, output: 75.00 },
+      "claude-haiku-4-5"   => { input: 0.80,  output: 4.00 },
+      "claude-3-5-sonnet-20241022" => { input: 3.00,  output: 15.00 },
+      "claude-3-5-haiku-20241022"  => { input: 0.80,  output: 4.00 },
+      "claude-3-opus-20240229"     => { input: 15.00, output: 75.00 },
+
+      # Google Gemini
+      "gemini-2.5-pro"     => { input: 1.25,  output: 10.00 },
+      "gemini-2.5-flash"   => { input: 0.15,  output: 0.60 },
+      "gemini-2.0-flash"   => { input: 0.10,  output: 0.40 },
+      "gemini-1.5-pro"     => { input: 1.25,  output: 5.00 },
+      "gemini-1.5-flash"   => { input: 0.075, output: 0.30 },
+    }.freeze
+
+    class << self
+      def cost_for(model:, input_tokens:, output_tokens:)
+        prices = lookup(model)
+        return nil unless prices
+
+        input_cost  = (input_tokens.to_f / 1_000_000) * prices[:input]
+        output_cost = (output_tokens.to_f / 1_000_000) * prices[:output]
+
+        {
+          input_cost: input_cost.round(8),
+          output_cost: output_cost.round(8),
+          total_cost: (input_cost + output_cost).round(8),
+          currency: "USD"
+        }
+      end
+
+      def lookup(model)
+        overrides = LlmCostTracker.configuration.pricing_overrides
+        overrides[model] || PRICES[model] || fuzzy_match(model)
+      end
+
+      def models
+        PRICES.keys | LlmCostTracker.configuration.pricing_overrides.keys
+      end
+
+      private
+
+      # Try to match model names like "gpt-4o-2024-08-06" to "gpt-4o"
+      def fuzzy_match(model)
+        return nil unless model
+
+        PRICES.each do |key, value|
+          return value if model.start_with?(key)
+        end
+
+        nil
+      end
+    end
+  end
+end

data/lib/llm_cost_tracker/railtie.rb

@@ -0,0 +1,19 @@
+# frozen_string_literal: true
+
+module LlmCostTracker
+  class Railtie < Rails::Railtie
+    generators do
+      require_relative "generators/llm_cost_tracker/install_generator"
+    end
+
+    initializer "llm_cost_tracker.configure" do
+      # Auto-require ActiveRecord storage if configured
+      ActiveSupport.on_load(:active_record) do
+        if LlmCostTracker.configuration.active_record?
+          require_relative "llm_api_call"
+          require_relative "storage/active_record_store"
+        end
+      end
+    end
+  end
+end

data/lib/llm_cost_tracker/storage/active_record_store.rb

@@ -0,0 +1,37 @@
+# frozen_string_literal: true
+
+module LlmCostTracker
+  module Storage
+    class ActiveRecordStore
+      class << self
+        def save(event)
+          model_class.create!(
+            provider:      event[:provider],
+            model:         event[:model],
+            input_tokens:  event[:input_tokens],
+            output_tokens: event[:output_tokens],
+            total_tokens:  event[:total_tokens],
+            input_cost:    event.dig(:cost, :input_cost),
+            output_cost:   event.dig(:cost, :output_cost),
+            total_cost:    event.dig(:cost, :total_cost),
+            tags:          event[:tags].to_json,
+            tracked_at:    event[:tracked_at]
+          )
+        end
+
+        def monthly_total(time: Time.now.utc)
+          beginning_of_month = Time.new(time.year, time.month, 1, 0, 0, 0, "+00:00")
+
+          model_class
+            .where(tracked_at: beginning_of_month..time)
+            .sum(:total_cost)
+            .to_f
+        end
+
+        def model_class
+          LlmCostTracker::LlmApiCall
+        end
+      end
+    end
+  end
+end

data/lib/llm_cost_tracker/tracker.rb

@@ -0,0 +1,107 @@
+# frozen_string_literal: true
+
+module LlmCostTracker
+  class Tracker
+    EVENT_NAME = "llm_request.llm_cost_tracker"
+
+    class << self
+      def record(provider:, model:, input_tokens:, output_tokens:, metadata: {})
+        cost_data = Pricing.cost_for(
+          model: model,
+          input_tokens: input_tokens,
+          output_tokens: output_tokens
+        )
+
+        event = {
+          provider: provider,
+          model: model,
+          input_tokens: input_tokens,
+          output_tokens: output_tokens,
+          total_tokens: input_tokens + output_tokens,
+          cost: cost_data,
+          tags: LlmCostTracker.configuration.default_tags.merge(metadata),
+          tracked_at: Time.now.utc
+        }
+
+        # Emit ActiveSupport::Notifications event
+        ActiveSupport::Notifications.instrument(EVENT_NAME, event)
+
+        # Store based on backend
+        store(event)
+
+        # Budget check
+        check_budget(event)
+
+        event
+      end
+
+      private
+
+      def store(event)
+        config = LlmCostTracker.configuration
+
+        case config.storage_backend
+        when :log
+          log_event(event)
+        when :active_record
+          store_active_record(event)
+        when :custom
+          config.custom_storage&.call(event)
+        end
+      end
+
+      def log_event(event)
+        cost_str = event[:cost] ? "$#{'%.6f' % event[:cost][:total_cost]}" : "unknown"
+
+        message = "[LlmCostTracker] #{event[:provider]}/#{event[:model]} " \
+                  "tokens=#{event[:input_tokens]}+#{event[:output_tokens]} " \
+                  "cost=#{cost_str}"
+        message += " tags=#{event[:tags]}" unless event[:tags].empty?
+
+        case LlmCostTracker.configuration.log_level
+        when :debug
+          Rails.logger.debug(message) if defined?(Rails)
+        when :warn
+          Rails.logger.warn(message) if defined?(Rails)
+        else
+          Rails.logger.info(message) if defined?(Rails)
+        end
+
+        # Fallback if Rails is not available
+        warn(message) unless defined?(Rails)
+      end
+
+      def store_active_record(event)
+        return unless defined?(LlmCostTracker::Storage::ActiveRecordStore)
+
+        LlmCostTracker::Storage::ActiveRecordStore.save(event)
+      end
+
+      def check_budget(event)
+        config = LlmCostTracker.configuration
+        return unless config.monthly_budget && config.on_budget_exceeded
+        return unless event[:cost]
+
+        monthly_total = calculate_monthly_total(event[:cost][:total_cost])
+        return unless monthly_total > config.monthly_budget
+
+        config.on_budget_exceeded.call(
+          monthly_total: monthly_total,
+          budget: config.monthly_budget,
+          last_event: event
+        )
+      end
+
+      def calculate_monthly_total(latest_cost)
+        # For :active_record backend, query the DB
+        if LlmCostTracker.configuration.active_record? &&
+           defined?(LlmCostTracker::Storage::ActiveRecordStore)
+          LlmCostTracker::Storage::ActiveRecordStore.monthly_total + latest_cost
+        else
+          # For other backends, we can only report the latest cost
+          latest_cost
+        end
+      end
+    end
+  end
+end

data/lib/llm_cost_tracker/version.rb

@@ -0,0 +1,5 @@
+# frozen_string_literal: true
+
+module LlmCostTracker
+  VERSION = "0.1.0"
+end

data/lib/llm_cost_tracker.rb

@@ -0,0 +1,65 @@
+# frozen_string_literal: true
+
+require "active_support"
+require "active_support/notifications"
+
+require_relative "llm_cost_tracker/version"
+require_relative "llm_cost_tracker/configuration"
+require_relative "llm_cost_tracker/pricing"
+require_relative "llm_cost_tracker/parsers/base"
+require_relative "llm_cost_tracker/parsers/openai"
+require_relative "llm_cost_tracker/parsers/anthropic"
+require_relative "llm_cost_tracker/parsers/gemini"
+require_relative "llm_cost_tracker/parsers/registry"
+require_relative "llm_cost_tracker/middleware/faraday"
+require_relative "llm_cost_tracker/tracker"
+
+module LlmCostTracker
+  class Error < StandardError; end
+
+  class << self
+    attr_writer :configuration
+
+    def configuration
+      @configuration ||= Configuration.new
+    end
+
+    def configure
+      yield(configuration)
+    end
+
+    def reset_configuration!
+      @configuration = Configuration.new
+    end
+
+    # Manual tracking for non-Faraday clients
+    #
+    #   LlmCostTracker.track(
+    #     provider: :openai,
+    #     model: "gpt-4o",
+    #     input_tokens: 150,
+    #     output_tokens: 50,
+    #     feature: "chat",
+    #     user_id: current_user.id
+    #   )
+    def track(provider:, model:, input_tokens:, output_tokens:, **metadata)
+      Tracker.record(
+        provider: provider.to_s,
+        model: model,
+        input_tokens: input_tokens,
+        output_tokens: output_tokens,
+        metadata: metadata
+      )
+    end
+  end
+end
+
+# Load Railtie if Rails is present
+require_relative "llm_cost_tracker/railtie" if defined?(Rails::Railtie)
+
+# Auto-register Faraday middleware
+if defined?(Faraday)
+  Faraday::Middleware.register_middleware(
+    llm_cost_tracker: LlmCostTracker::Middleware::Faraday
+  )
+end

data/llm_cost_tracker.gemspec

@@ -0,0 +1,41 @@
+# frozen_string_literal: true
+
+require_relative "lib/llm_cost_tracker/version"
+
+Gem::Specification.new do |spec|
+  spec.name          = "llm_cost_tracker"
+  spec.version       = LlmCostTracker::VERSION
+  spec.authors       = ["Sergii Khomenko"]
+  spec.email         = ["sergey@mm.st"]
+
+  spec.summary       = "Provider-agnostic LLM API cost tracking for Ruby"
+  spec.description   = "Automatically tracks token usage and costs for LLM API calls (OpenAI, Anthropic, Google Gemini, and more). " \
+                        "Works as Faraday middleware — plugs into any Ruby HTTP client. " \
+                        "Provides ActiveRecord storage, per-user/per-feature attribution, and budget alerts."
+  spec.homepage      = "https://github.com/sergey-homenko/llm_cost_tracker"
+  spec.license       = "MIT"
+
+  spec.required_ruby_version = ">= 3.1.0"
+
+  spec.metadata["homepage_uri"]  = spec.homepage
+  spec.metadata["changelog_uri"] = "#{spec.homepage}/blob/main/CHANGELOG.md"
+
+  spec.files = Dir.chdir(__dir__) do
+    `git ls-files -z`.split("\x0").reject do |f|
+      (File.expand_path(f) == __FILE__) ||
+        f.start_with?("bin/", "test/", "spec/", ".git", ".github", "Gemfile")
+    end
+  end
+
+  spec.require_paths = ["lib"]
+
+  spec.add_dependency "faraday", ">= 1.0", "< 3.0"
+  spec.add_dependency "activesupport", ">= 7.0", "< 9.0"
+
+  spec.add_development_dependency "activerecord", ">= 7.0", "< 9.0"
+  spec.add_development_dependency "rake", "~> 13.0"
+  spec.add_development_dependency "rspec", "~> 3.0"
+  spec.add_development_dependency "webmock", "~> 3.0"
+  spec.add_development_dependency "sqlite3", "~> 2.0"
+  spec.add_development_dependency "rubocop", "~> 1.0"
+end

metadata

@@ -0,0 +1,201 @@
+--- !ruby/object:Gem::Specification
+name: llm_cost_tracker
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- Sergii Khomenko
+autorequire:
+bindir: bin
+cert_chain: []
+date: 2026-04-16 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: faraday
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '1.0'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '1.0'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+- !ruby/object:Gem::Dependency
+  name: activesupport
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '7.0'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '9.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '7.0'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '9.0'
+- !ruby/object:Gem::Dependency
+  name: activerecord
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '7.0'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '9.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '7.0'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '9.0'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '13.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '13.0'
+- !ruby/object:Gem::Dependency
+  name: rspec
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+- !ruby/object:Gem::Dependency
+  name: webmock
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+- !ruby/object:Gem::Dependency
+  name: sqlite3
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.0'
+- !ruby/object:Gem::Dependency
+  name: rubocop
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.0'
+description: Automatically tracks token usage and costs for LLM API calls (OpenAI,
+  Anthropic, Google Gemini, and more). Works as Faraday middleware — plugs into any
+  Ruby HTTP client. Provides ActiveRecord storage, per-user/per-feature attribution,
+  and budget alerts.
+email:
+- sergey@mm.st
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- ".rspec"
+- CHANGELOG.md
+- LICENSE.txt
+- README.md
+- Rakefile
+- lib/llm_cost_tracker.rb
+- lib/llm_cost_tracker/configuration.rb
+- lib/llm_cost_tracker/generators/llm_cost_tracker/install_generator.rb
+- lib/llm_cost_tracker/generators/llm_cost_tracker/templates/create_llm_api_calls.rb.erb
+- lib/llm_cost_tracker/generators/llm_cost_tracker/templates/initializer.rb.erb
+- lib/llm_cost_tracker/llm_api_call.rb
+- lib/llm_cost_tracker/middleware/faraday.rb
+- lib/llm_cost_tracker/parsers/anthropic.rb
+- lib/llm_cost_tracker/parsers/base.rb
+- lib/llm_cost_tracker/parsers/gemini.rb
+- lib/llm_cost_tracker/parsers/openai.rb
+- lib/llm_cost_tracker/parsers/registry.rb
+- lib/llm_cost_tracker/pricing.rb
+- lib/llm_cost_tracker/railtie.rb
+- lib/llm_cost_tracker/storage/active_record_store.rb
+- lib/llm_cost_tracker/tracker.rb
+- lib/llm_cost_tracker/version.rb
+- llm_cost_tracker.gemspec
+homepage: https://github.com/sergey-homenko/llm_cost_tracker
+licenses:
+- MIT
+metadata:
+  homepage_uri: https://github.com/sergey-homenko/llm_cost_tracker
+  changelog_uri: https://github.com/sergey-homenko/llm_cost_tracker/blob/main/CHANGELOG.md
+post_install_message:
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: 3.1.0
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.5.9
+signing_key:
+specification_version: 4
+summary: Provider-agnostic LLM API cost tracking for Ruby
+test_files: []