llm_cost_tracker 0.5.1 → 0.5.3

data/docs/configuration.md

@@ -0,0 +1,65 @@
+# Configuration
+
+Configuration is the contract between the host app and the ledger: where events
+go, which integrations are enabled, how attribution is attached, and how the app
+reacts when storage, pricing, or budgets need attention.
+
+The full option reference is moving here from the README. Until that migration is
+complete, the README anchors below remain canonical.
+
+## Canonical Sources
+
+Until this page is expanded, use:
+
+- [Quickstart](../README.md#quickstart)
+- [Capturing calls](../README.md#capturing-calls)
+- [Tags](../README.md#tags-who-burned-this-money)
+- [Pricing](../README.md#pricing)
+- [Budgets](../README.md#budgets)
+
+## Scope
+
+This page is scoped to:
+
+- `storage_backend`: `:log`, `:active_record`, and `:custom`
+- `default_tags`: static tags and per-request callable tags
+- `instrument`: RubyLLM and official SDK integrations
+- `prices_file` and `pricing_overrides`
+- `monthly_budget`, `daily_budget`, and `per_call_budget`
+- `budget_exceeded_behavior`
+- `storage_error_behavior`
+- `unknown_pricing_behavior`
+- `openai_compatible_providers`
+- `report_tag_breakdowns`
+
+## Minimal Production Config
+
+```ruby
+LlmCostTracker.configure do |config|
+  config.storage_backend = :active_record
+  config.default_tags = -> { { environment: Rails.env } }
+  config.prices_file = Rails.root.join("config/llm_cost_tracker_prices.yml")
+  config.instrument :openai
+end
+```
+
+Keep configuration at boot. Mutable shared settings are frozen after
+`configure` returns so request-time code cannot silently change global tracking
+behavior.
+
+Enabled SDK integrations are fail-fast. The client gem must be loaded, meet the
+minimum supported version, and expose the expected classes and methods before
+LLM Cost Tracker installs its wrapper.
+
+## Capture Verification
+
+After boot, run:
+
+```bash
+bin/rails llm_cost_tracker:verify_capture
+```
+
+For ActiveRecord storage, the task records a synthetic manual event inside a
+rollback and checks that notifications and persistence both work. For log and
+custom storage, it reports the configured capture path without invoking external
+sinks.

data/docs/cookbook.md

@@ -0,0 +1,185 @@
+# Cookbook
+
+Short integration recipes for common Ruby clients. Prefer SDK integrations or middleware. Use `track` and `track_stream` only as fallback helpers for unsupported clients.
+
+| Client | Best path | Why |
+|---|---|---|
+| RubyLLM | `config.instrument :ruby_llm` | The integration wraps RubyLLM's provider layer without adding a third-party instrumentation gem. |
+| Official `openai` gem | `config.instrument :openai` | The integration wraps SDK resource methods without changing call sites. |
+| Official `anthropic` gem | `config.instrument :anthropic` | The integration records returned message usage without changing call sites. |
+| `ruby-openai` | Faraday middleware | The client is built on Faraday and accepts middleware via the constructor block. |
+| OpenAI-compatible proxy | Faraday middleware | Use `ruby-openai` or a direct Faraday client against the proxy host. |
+| Custom Faraday client | Faraday middleware | The middleware can parse known provider responses automatically. |
+| Other clients | Adapter first, fallback helpers second | Add a stable integration instead of scattering per-call ledger code. |
+
+## RubyLLM
+
+Enable the integration once, then keep normal RubyLLM calls unchanged.
+
+```ruby
+LlmCostTracker.configure do |config|
+  config.instrument :ruby_llm
+end
+
+LlmCostTracker.with_tags(feature: "support_chat") do
+  RubyLLM.chat.ask("Hello")
+  RubyLLM.embed("text to embed")
+end
+```
+
+The RubyLLM integration supports `ruby_llm >= 1.14.1` and checks RubyLLM's provider contract at boot. Chat, embedding, and transcription calls are captured. Image generation, moderation, and tool execution are not recorded as separate ledger rows.
+
+## Official OpenAI SDK
+
+Enable the integration once, then keep normal `openai` gem calls unchanged.
+
+```ruby
+LlmCostTracker.configure do |config|
+  config.instrument :openai
+end
+
+client = OpenAI::Client.new(api_key: ENV["OPENAI_API_KEY"])
+
+client.responses.create(model: "gpt-4o", input: "Hello")
+client.chat.completions.create(
+  model: "gpt-4o",
+  messages: [{ role: "user", content: "Hello" }]
+)
+
+client.responses.stream(model: "gpt-4o", input: "Hello").each do |event|
+  puts event.type
+end
+
+client.responses.stream_raw(model: "gpt-4o", input: "Hello").each do |event|
+  puts event.type
+end
+
+client.chat.completions.stream_raw(
+  model: "gpt-4o",
+  messages: [{ role: "user", content: "Hello" }],
+  stream_options: { include_usage: true }
+).each do |event|
+  puts event
+end
+```
+
+The OpenAI SDK integration supports `openai >= 0.59.0`. Streaming calls are recorded after the returned stream is consumed. Chat Completions streams need `stream_options: { include_usage: true }` for final usage.
+
+## Official Anthropic SDK
+
+Enable the integration once, then keep normal `anthropic` gem calls unchanged.
+
+```ruby
+LlmCostTracker.configure do |config|
+  config.instrument :anthropic
+end
+
+client = Anthropic::Client.new(api_key: ENV["ANTHROPIC_API_KEY"])
+
+client.messages.create(
+  max_tokens: 1024,
+  model: "claude-sonnet-4-5-20250929",
+  messages: [{ role: "user", content: "Hello" }]
+)
+
+client.messages.stream(
+  max_tokens: 1024,
+  model: "claude-sonnet-4-5-20250929",
+  messages: [{ role: "user", content: "Hello" }]
+).each do |event|
+  puts event.type
+end
+
+client.messages.stream_raw(
+  max_tokens: 1024,
+  model: "claude-sonnet-4-5-20250929",
+  messages: [{ role: "user", content: "Hello" }]
+).each do |event|
+  puts event.type
+end
+```
+
+The Anthropic SDK integration supports `anthropic >= 1.36.0`. Streaming calls are recorded after the returned stream is consumed.
+
+## ruby-openai
+
+`ruby-openai` is a community client that occupies the same `OpenAI::Client` constant as the official gem; only one of the two can be loaded. `config.instrument :openai` is for the official gem. For `ruby-openai`, attach the Faraday middleware via the constructor block:
+
+```ruby
+client = OpenAI::Client.new(access_token: ENV["OPENAI_API_KEY"]) do |f|
+  f.use :llm_cost_tracker, tags: { feature: "chat" }
+end
+
+client.chat(
+  parameters: {
+    model: "gpt-4o",
+    messages: [{ role: "user", content: "Hello" }],
+    stream: proc { |chunk, _bytesize| puts chunk.dig("choices", 0, "delta", "content") },
+    stream_options: { include_usage: true }
+  }
+)
+```
+
+Use the constructor block on every client you build, or wrap client creation in your own factory.
+
+## Azure OpenAI
+
+Azure's v1 API works with OpenAI-compatible HTTP shapes, but pricing and deployment names are yours. Register the Azure host, use the Faraday middleware path, and keep Azure-specific prices in `prices_file` or `pricing_overrides`.
+
+```ruby
+LlmCostTracker.configure do |config|
+  config.openai_compatible_providers["my-resource.openai.azure.com"] = "azure_openai"
+end
+
+conn = Faraday.new(url: "https://my-resource.openai.azure.com") do |f|
+  f.use :llm_cost_tracker, tags: { feature: "chat" }
+  f.request :json
+  f.response :json
+  f.adapter Faraday.default_adapter
+end
+
+conn.post(
+  "/openai/v1/responses",
+  { model: "gpt-4o-prod", input: "Hello" },
+  { "api-key" => ENV.fetch("AZURE_OPENAI_API_KEY") }
+)
+```
+
+## Gemini API
+
+Google's official Gemini SDKs do not include Ruby. Use a Faraday client against the REST API so the Gemini parser can capture usage automatically.
+
+```ruby
+conn = Faraday.new(url: "https://generativelanguage.googleapis.com") do |f|
+  f.use :llm_cost_tracker, tags: { feature: "chat" }
+  f.request :json
+  f.response :json
+  f.adapter Faraday.default_adapter
+end
+
+conn.post(
+  "/v1beta/models/gemini-2.5-flash:generateContent?key=#{ENV.fetch("GOOGLE_API_KEY")}",
+  { contents: [{ role: "user", parts: [{ text: "Hello" }] }] }
+)
+```
+
+## LiteLLM proxy
+
+LiteLLM Proxy speaks an OpenAI-compatible HTTP shape, so register the proxy host once and keep using the normal middleware path.
+
+```ruby
+LlmCostTracker.configure do |config|
+  config.openai_compatible_providers["proxy.internal.example"] = "litellm"
+end
+
+client = OpenAI::Client.new(
+  access_token: ENV["LITELLM_API_KEY"],
+  uri_base: "https://proxy.internal.example"
+) do |f|
+  f.use :llm_cost_tracker, tags: { gateway: "litellm" }
+end
+
+client.chat(parameters: { model: "openai/gpt-5-mini", messages: [{ role: "user", content: "Hello" }] })
+```
+
+If your proxy exposes custom model IDs or discounts, add them in `prices_file` or `pricing_overrides`.

data/docs/dashboard.md

@@ -0,0 +1,38 @@
+# Dashboard
+
+The dashboard is a Rails Engine for humans reviewing spend, attribution, and data
+quality. It is server-rendered ERB, has no JavaScript bundle, and reads from the
+host app's `llm_api_calls` table.
+
+The detailed dashboard guide is moving here from the README: mounting, route
+constraints, authentication examples, page map, and operational notes.
+
+## Canonical Sources
+
+Until this page is expanded, use:
+
+- [Dashboard](../README.md#dashboard)
+- [Privacy](../README.md#privacy)
+- [Operations](operations.md)
+
+## Mounting
+
+```ruby
+mount LlmCostTracker::Engine => "/llm-costs"
+```
+
+Use `storage_backend = :active_record` for apps that mount the dashboard.
+
+## Pages
+
+- Overview: spend trend, budget status, anomaly banner, provider rollup, top models
+- Models: spend and usage by provider and model
+- Calls: filterable call ledger with CSV export
+- Tags: tag keys and tag value breakdowns
+- Data Quality: unknown pricing, untagged calls, missing latency, incomplete streams
+
+## Authentication
+
+The gem does not ship dashboard auth. Mount the engine behind the host app's
+existing authentication layer: Devise, basic auth, Cloudflare Access, or your own
+constraints.

data/docs/extending.md

@@ -0,0 +1,32 @@
+# Extending LLM Cost Tracker
+
+Extensions belong at clear boundaries: parsers for response shapes, integrations
+for SDK hooks, pricing files for rates, and custom storage for apps that own
+persistence themselves.
+
+The practical extension guide is moving here from the README. The lower-level
+contracts already live in the technical extension reference.
+
+## Canonical Sources
+
+Until this page is expanded, use:
+
+- [Capturing calls](../README.md#capturing-calls)
+- [Pricing](pricing.md)
+- [Technical extension points](technical/extension-points.md)
+
+## Extension Points
+
+- Custom parser: translate a provider response into `ParsedUsage`.
+- OpenAI-compatible host: register the host-to-provider mapping.
+- Custom storage: receive the canonical `Event` and write it elsewhere.
+- Notifications subscriber: observe `llm_request.llm_cost_tracker`.
+- Local price file: model gateway IDs, contract rates, or unsupported models.
+
+## Parser Boundary
+
+A parser matches request URLs, translates provider response shapes into
+`ParsedUsage`, and returns `nil` when the response is outside its contract.
+
+Do provider-specific translation at this boundary. Keep `Tracker`, storage,
+dashboard, and pricing in canonical ledger terms.

data/docs/operations.md

@@ -0,0 +1,44 @@
+# Operations
+
+Production use is mostly about choosing the right storage backend, keeping the
+database healthy, and understanding where the gem is intentionally best effort.
+
+The operational guide is moving here from the README: retention, tag storage,
+thread safety, connection pools, and deployment notes.
+
+## Canonical Sources
+
+Until this page is expanded, use:
+
+- [Privacy](../README.md#privacy)
+- [Known limitations](../README.md#known-limitations)
+- [Technical operational notes](technical/operational-notes.md)
+
+## Production Defaults
+
+- Use `storage_backend = :active_record` for the shared ledger, dashboard, and
+  cross-process budget guardrails.
+- Size the ActiveRecord connection pool for your app plus ledger writes.
+- Keep `storage_error_behavior = :warn` unless losing the LLM response is better
+  than losing the ledger event.
+- Treat `:block_requests` as a guardrail, not a hard quota.
+- Keep `default_tags` callables fast and thread-safe.
+
+## Retention
+
+Retention is explicit. Use the prune task when the ledger should not grow
+forever:
+
+```bash
+DAYS=90 bin/rails llm_cost_tracker:prune
+```
+
+When ActiveRecord period rollups are installed, pruning decrements the
+affected daily and monthly buckets in the same batch transaction as the ledger
+delete.
+
+## Data Shape
+
+Tags are JSONB with a GIN index on PostgreSQL and JSON text elsewhere. The
+dashboard and query helpers work across supported adapters, but PostgreSQL is the
+strongest path for large tag-heavy ledgers.

data/docs/pricing.md

@@ -0,0 +1,94 @@
+# Pricing and Price Refresh
+
+LLM Cost Tracker prices calls locally from recorded usage and a versioned price
+registry. Providers usually return token counts, not a stable per-request price,
+so the gem stores the calculated cost with each ledger row.
+
+The full pricing reference is moving here from the README: registry shape,
+refresh tasks, precedence, provider-qualified keys, and mode-specific rates.
+
+## Canonical Sources
+
+Until this page is expanded, use:
+
+- [Pricing](../README.md#pricing)
+- [Supported providers](../README.md#supported-providers)
+- [Known limitations](../README.md#known-limitations)
+
+## Registry Rules
+
+- Built-in prices live in `lib/llm_cost_tracker/prices.json`.
+- Local snapshots live wherever `config.prices_file` points.
+- Precedence is `pricing_overrides`, then `prices_file`, then bundled prices.
+- Provider-qualified keys like `openai/gpt-4o-mini` win over model-only keys.
+- Historical rows keep the cost calculated when the call was recorded.
+
+## Refresh Commands
+
+```bash
+bin/rails generate llm_cost_tracker:prices
+bin/rails llm_cost_tracker:prices:refresh
+bin/rails llm_cost_tracker:prices:check
+PROVIDER=openai MODEL=gpt-4o bin/rails llm_cost_tracker:prices:explain
+```
+
+The refresh task reads the maintained LLM Cost Tracker snapshot and writes to
+`ENV["OUTPUT"]`, then `config.prices_file`, then
+`config/llm_cost_tracker_prices.yml`.
+
+## Price Fields
+
+Base fields:
+
+- `input`
+- `output`
+- `cache_read_input`
+- `cache_write_input`
+
+Mode-prefixed fields use the same base terms:
+
+- `batch_input`
+- `batch_output`
+- `priority_input`
+- `batch_cache_read_input`
+
+## Pricing Modes
+
+Pass `pricing_mode: :batch` when usage came from a provider batch job or another
+discounted mode:
+
+```ruby
+LlmCostTracker.track(
+  provider: "openai",
+  model: "gpt-4o",
+  input_tokens: 1_000_000,
+  output_tokens: 250_000,
+  pricing_mode: :batch,
+  feature: "offline_eval"
+)
+```
+
+The calculator uses `batch_input`, `batch_output`, and other matching
+mode-prefixed fields when present, then falls back to the base field for missing
+mode-specific rates.
+
+## Price Explain
+
+Use `prices:explain` when Data Quality shows unknown pricing or a local override
+does not behave as expected:
+
+```bash
+PROVIDER=openai MODEL=gpt-4o PRICING_MODE=batch bin/rails llm_cost_tracker:prices:explain
+```
+
+Optional token env vars let the command check the exact buckets that a call used:
+
+```bash
+PROVIDER=custom MODEL=gateway-model INPUT_TOKENS=1000 OUTPUT_TOKENS=200 CACHE_READ_INPUT_TOKENS=50 bin/rails llm_cost_tracker:prices:explain
+```
+
+The command reports the matched source, matched key, match strategy, effective
+rates, and any missing rate needed to price the event.
+
+Provider-specific pricing pages belong in scrapers and snapshots. Runtime
+pricing should stay in canonical billing terms.

data/docs/querying.md

@@ -0,0 +1,36 @@
+# Querying and Reports
+
+Once calls are in `llm_api_calls`, the host app owns the data. Query it from a
+console, a scheduled job, your admin UI, or the mounted dashboard.
+
+The full querying reference is moving here from the README: ActiveRecord scopes,
+reporting helpers, tag breakdowns, and SQL-side grouping patterns.
+
+## Canonical Sources
+
+Until this page is expanded, use:
+
+- [Querying](../README.md#querying)
+- [Dashboard](dashboard.md)
+- [Operations](operations.md)
+
+## Common Queries
+
+```ruby
+LlmCostTracker::LlmApiCall.today.total_cost
+LlmCostTracker::LlmApiCall.this_month.cost_by_model
+LlmCostTracker::LlmApiCall.this_month.cost_by_provider
+LlmCostTracker::LlmApiCall.this_month.cost_by_tag("feature")
+LlmCostTracker::LlmApiCall.by_tags(user_id: 42, feature: "chat").this_month.total_cost
+LlmCostTracker::LlmApiCall.daily_costs(days: 7)
+```
+
+## Report Task
+
+```bash
+bin/rails llm_cost_tracker:report
+DAYS=7 bin/rails llm_cost_tracker:report
+```
+
+This page is scoped to cost scopes, tag grouping, period grouping, latency
+helpers, unknown-pricing queries, and report tag breakdowns.

data/docs/streaming.md

@@ -0,0 +1,70 @@
+# Streaming Capture
+
+Streaming calls should appear in the ledger instead of disappearing into a live
+callback. LLM Cost Tracker records them when the provider emits final usage or
+when the app supplies explicit totals.
+
+The full streaming reference is moving here from the README: Faraday streaming,
+`track_stream`, provider response IDs, final usage events, and data-quality
+states.
+
+## Canonical Sources
+
+Until this page is expanded, use:
+
+- [Capturing calls](../README.md#capturing-calls)
+- [Known limitations](../README.md#known-limitations)
+- [Cookbook](cookbook.md)
+
+## Faraday Path
+
+The middleware tees Faraday's `on_data` callback, keeps chunks flowing to the
+caller, and records usage when the response completes.
+
+OpenAI streams need final usage:
+
+```ruby
+stream_options: { include_usage: true }
+```
+
+Anthropic and Gemini are parsed from their provider stream event shapes when
+usage is present.
+
+## SDK Path
+
+Official OpenAI and Anthropic SDK streams are captured when `config.instrument`
+is enabled for the provider. The returned stream object is preserved, and usage
+is recorded after the stream is consumed.
+
+```ruby
+config.instrument :openai
+config.instrument :anthropic
+```
+
+Captured SDK helpers:
+
+- OpenAI `responses.stream`, `responses.stream_raw`, `responses.retrieve_streaming`, and `chat.completions.stream_raw`.
+- Anthropic `messages.stream` and `messages.stream_raw`.
+
+OpenAI Chat Completions streams need final usage:
+
+```ruby
+stream_options: { include_usage: true }
+```
+
+## Manual Path
+
+```ruby
+LlmCostTracker.track_stream(provider: "openai", model: "gpt-4o") do |stream|
+  my_client.stream(...) { |event| stream.event(event.to_h) }
+end
+```
+
+If the client already knows totals, skip provider event parsing:
+
+```ruby
+stream.usage(input_tokens: 120, output_tokens: 45)
+```
+
+Missing final usage is stored with `usage_source: "unknown"` so the Data Quality
+page can surface it.

data/docs/technical/README.md

@@ -0,0 +1,10 @@
+# Technical Documentation
+
+These files describe the internal module boundaries for LLM Cost Tracker.
+
+- [Module map](module-map.md)
+- [Data flow](data-flow.md)
+- [Extension points](extension-points.md)
+- [Operational notes](operational-notes.md)
+
+The main rule is simple: provider-specific API shapes stop at ingestion boundaries. The ledger, storage, budgets, dashboard, and reports work with canonical billing concepts.

data/docs/technical/data-flow.md

@@ -0,0 +1,67 @@
+# Data Flow
+
+This is the normal path from an application LLM call to stored ledger data.
+
+## Faraday Requests
+
+1. The host app sends an HTTP request through Faraday.
+2. `LlmCostTracker::Middleware::Faraday` checks whether a parser matches the request URL.
+3. For non-streaming responses, the middleware passes request and response data to the parser.
+4. For streaming responses, the middleware tees `on_data`, collects stream events, and parses final usage when the stream completes.
+5. The parser returns `ParsedUsage` with canonical fields.
+6. `Tracker.record` prices and persists the event.
+
+## SDK Integrations
+
+1. The host app enables an integration with `config.instrument`.
+2. `LlmCostTracker::Integrations` checks the SDK version, target classes, and target methods once at install time.
+3. `LlmCostTracker::Integrations` prepends a narrow wrapper to supported SDK resource methods.
+4. The host app keeps calling the provider SDK normally.
+5. The wrapper measures latency, extracts usage from the SDK response object, and sends canonical fields to `Tracker.record`.
+6. If an explicitly enabled SDK is not loaded or does not satisfy the install contract, boot raises before the app silently misses usage.
+
+## Explicit Tracking
+
+1. The host app calls `LlmCostTracker.track` with known usage totals, or `LlmCostTracker.track_stream` with stream events.
+2. `track` sends manual totals directly to `Tracker.record`.
+3. `track_stream` uses `StreamCollector`, then parser lookup by provider when events need parsing.
+4. `Tracker.record` prices and persists the event.
+
+## Canonical Event Build
+
+`Tracker.record` performs the central normalization step:
+
+1. Blank model identifiers become `unknown`.
+2. Input, output, cache-read, cache-write, hidden-output, and pricing-mode values are extracted from metadata.
+3. `Pricing.cost_for` calculates a `Cost` object or returns `nil` for unknown pricing.
+4. Tags are merged from `with_tags`, `default_tags`, middleware tags, and explicit metadata.
+5. An `Event` is created and emitted through `ActiveSupport::Notifications`.
+6. The configured storage backend receives the event.
+7. Budget checks run unless storage explicitly returns `false`.
+
+## ActiveRecord Storage
+
+1. `Storage::ActiveRecordStore.save` converts tags for JSON or text storage.
+2. Optional fields are written only when their columns exist.
+3. The call row and period rollup updates happen in one transaction.
+4. `ActiveRecordRollups.increment!` updates daily and monthly totals atomically.
+5. Budget reads use period totals when available.
+
+## Dashboard Reads
+
+1. Controllers build a filtered `LlmApiCall` scope.
+2. Dashboard services run targeted aggregate queries.
+3. Helpers render filters, charts, pagination, CSV links, and numeric formatting.
+4. Views render plain ERB with the engine CSS asset.
+
+Dashboard reads do not mutate ledger state. They can be heavier than request-time code, but they still need explicit grouping and indexes.
+
+## Pricing Refresh
+
+1. `llm_cost_tracker:prices:refresh` chooses `ENV["OUTPUT"]`, then `config.prices_file`, then `config/llm_cost_tracker_prices.yml`.
+2. `PriceSync::Fetcher` fetches the maintained LLM Cost Tracker price snapshot.
+3. `PriceSync` validates schema compatibility, gem-version compatibility, and model price shape.
+4. `RegistryWriter` writes a local JSON or YAML registry.
+5. Runtime pricing reloads the local file when its mtime changes.
+
+The gem never fetches pricing from the network during normal request tracking.