llm_cost_tracker 0.3.1 → 0.3.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +7 -0
- data/CODE_OF_CONDUCT.md +23 -0
- data/README.md +72 -7
- data/SECURITY.md +36 -0
- data/app/controllers/llm_cost_tracker/assets_controller.rb +0 -2
- data/lib/llm_cost_tracker/railtie.rb +2 -0
- data/lib/llm_cost_tracker/version.rb +1 -1
- data/llm_cost_tracker.gemspec +3 -1
- metadata +31 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 6952282e6f93b4e5658ef9d2b9527d2a332cb2d6f483da25540c3a0d6672ed9b
|
|
4
|
+
data.tar.gz: e66eaaeb99698abf9c0ff9e3f1305e6bb27a8b6c25355e94bce4baec5f5d3a50
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: '078d695498ed6f254a700ccb3381ace3feaa1f4880691a1a69be4bb435097202ecf28f2cbf065202a543186bdd3894aaedcc44bfacc2d2ffa25a54ddf6d1cc76'
|
|
7
|
+
data.tar.gz: 2638ae3c579bd2c0f71a73b06719eb0a35d7b82e8614a74c5100f41b69693babfd3b613ecf18e7f6e7ea33210222432efa5a7ca718325c4c8fd12be9b8ab806e
|
data/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,13 @@ Format: [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). Versioning: [S
|
|
|
4
4
|
|
|
5
5
|
## [Unreleased]
|
|
6
6
|
|
|
7
|
+
## [0.3.2] - 2026-04-22
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
|
|
11
|
+
- Test coverage reporting via SimpleCov with LCOV upload to Codecov from CI.
|
|
12
|
+
- Repository governance files: `CODE_OF_CONDUCT.md`, `SECURITY.md`, `CONTRIBUTING.md`, and GitHub issue templates.
|
|
13
|
+
|
|
7
14
|
## [0.3.1] - 2026-04-22
|
|
8
15
|
|
|
9
16
|
### Added
|
data/CODE_OF_CONDUCT.md
ADDED
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
# Code of Conduct
|
|
2
|
+
|
|
3
|
+
This project adopts the [Contributor Covenant](https://www.contributor-covenant.org/), version 2.1, as its code of conduct. The full text is available at:
|
|
4
|
+
|
|
5
|
+
<https://www.contributor-covenant.org/version/2/1/code_of_conduct/>
|
|
6
|
+
|
|
7
|
+
## Our pledge
|
|
8
|
+
|
|
9
|
+
We as contributors and maintainers pledge to make participation in this project a welcoming, respectful, and inclusive experience for everyone.
|
|
10
|
+
|
|
11
|
+
## Scope
|
|
12
|
+
|
|
13
|
+
This Code of Conduct applies within all project spaces (issues, pull requests, discussions, commit messages, review comments) and in public spaces when an individual is representing the project.
|
|
14
|
+
|
|
15
|
+
## Reporting
|
|
16
|
+
|
|
17
|
+
Instances of unacceptable behavior may be reported to the project maintainer at **sergey@mm.st**. All reports will be reviewed and investigated promptly and fairly. The maintainer is obligated to respect the privacy and safety of the reporter of any incident.
|
|
18
|
+
|
|
19
|
+
## Enforcement
|
|
20
|
+
|
|
21
|
+
Maintainers are responsible for clarifying and enforcing the standards of acceptable behavior and will take appropriate and fair corrective action in response to any behavior that they deem inappropriate, threatening, or harmful. This may include warnings, temporary bans, or permanent removal from project participation.
|
|
22
|
+
|
|
23
|
+
For the full set of community standards, enforcement guidelines, and attribution, see the canonical document linked above.
|
data/README.md
CHANGED
|
@@ -4,14 +4,70 @@
|
|
|
4
4
|
|
|
5
5
|
[](https://rubygems.org/gems/llm_cost_tracker)
|
|
6
6
|
[](https://github.com/sergey-homenko/llm_cost_tracker/actions)
|
|
7
|
+
[](https://codecov.io/gh/sergey-homenko/llm_cost_tracker)
|
|
8
|
+
|
|
9
|
+
Requires Ruby 3.3+, Rails/ActiveRecord 7.1+, and Faraday 2.0+.
|
|
10
|
+
Core tracking works without Rails; the mounted dashboard requires Rails 7.1+.
|
|
7
11
|
|
|
8
12
|
## Why
|
|
9
13
|
|
|
10
14
|
Every Rails app with LLM integrations eventually runs into the same question: where did that invoice come from? Full observability platforms like Langfuse and Helicone solve a broader set of problems; sometimes you just need a small Rails-native ledger in your own database.
|
|
11
15
|
|
|
12
|
-
|
|
16
|
+
## What You Get
|
|
17
|
+
|
|
18
|
+
- A local ActiveRecord ledger of provider, model, tokens, cost, latency, tags, streaming usage, and provider response IDs
|
|
19
|
+
- Faraday middleware plus explicit `track` / `track_stream` helpers for non-Faraday clients
|
|
20
|
+
- Server-rendered Rails dashboard with overview, calls, tags, CSV export, and data-quality pages
|
|
21
|
+
- Local pricing snapshots, price sync tasks, and budget guardrails
|
|
22
|
+
- Prompt and response bodies are never persisted
|
|
23
|
+
|
|
24
|
+
## Dashboard
|
|
25
|
+
|
|
26
|
+
LLM Cost Tracker ships with an optional server-rendered Rails Engine dashboard for spend review, attribution, and data quality checks.
|
|
27
|
+
|
|
28
|
+

|
|
29
|
+
|
|
30
|
+
The overview page includes spend trend, budget status, provider breakdown, top models, and filterable slices. The engine also includes Calls, Tags, and Data Quality pages. Plain ERB, no JavaScript bundle.
|
|
31
|
+
|
|
32
|
+
## Quickstart
|
|
33
|
+
|
|
34
|
+
```ruby
|
|
35
|
+
gem "llm_cost_tracker"
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
```bash
|
|
39
|
+
bin/rails generate llm_cost_tracker:install
|
|
40
|
+
bin/rails db:migrate
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
```ruby
|
|
44
|
+
LlmCostTracker.configure do |config|
|
|
45
|
+
config.storage_backend = :active_record
|
|
46
|
+
config.default_tags = { app: "my_app", environment: Rails.env }
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
OpenAI.configure do |config|
|
|
50
|
+
config.access_token = ENV["OPENAI_API_KEY"]
|
|
51
|
+
config.faraday do |f|
|
|
52
|
+
f.use :llm_cost_tracker, tags: -> { { user_id: Current.user&.id, feature: "chat" } }
|
|
53
|
+
end
|
|
54
|
+
end
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
```ruby
|
|
58
|
+
mount LlmCostTracker::Engine => "/llm-costs"
|
|
59
|
+
```
|
|
13
60
|
|
|
14
|
-
|
|
61
|
+
After that, LLM Cost Tracker starts recording calls into `llm_api_calls` and the dashboard becomes available at `/llm-costs`.
|
|
62
|
+
Protect the mounted engine with your application's authentication before exposing it outside development.
|
|
63
|
+
|
|
64
|
+
## Tradeoffs
|
|
65
|
+
|
|
66
|
+
- Self-hosted ledger first: no proxy, no SaaS, no separate service to operate
|
|
67
|
+
- Best-effort pricing for spend review and attribution, not invoice-grade billing
|
|
68
|
+
- No prompt or response body storage
|
|
69
|
+
- No built-in auth on the mounted dashboard
|
|
70
|
+
- Use `:active_record` when you want shared dashboards and budget checks across Puma workers and Sidekiq processes
|
|
15
71
|
|
|
16
72
|
## Installation
|
|
17
73
|
|
|
@@ -293,7 +349,7 @@ bin/rails generate llm_cost_tracker:add_latency_ms
|
|
|
293
349
|
bin/rails db:migrate
|
|
294
350
|
```
|
|
295
351
|
|
|
296
|
-
##
|
|
352
|
+
## Mounting the dashboard
|
|
297
353
|
|
|
298
354
|
Optional Rails Engine. Plain ERB, no JavaScript framework, no asset pipeline required. Requires Rails 7.1+; the core middleware works without Rails.
|
|
299
355
|
|
|
@@ -315,7 +371,7 @@ Routes (GET-only; CSV export included):
|
|
|
315
371
|
- `/llm-costs/tags/:key` — breakdown by values of a given tag key
|
|
316
372
|
- `/llm-costs/data_quality` — unknown pricing share, untagged calls, missing latency
|
|
317
373
|
|
|
318
|
-
|
|
374
|
+
No built-in auth is included. Tags carry whatever your app puts in them, so protect the mount point with your application's authentication.
|
|
319
375
|
|
|
320
376
|
### Basic auth
|
|
321
377
|
|
|
@@ -381,20 +437,26 @@ Configured hosts are parsed using the OpenAI-compatible usage shape (`prompt_tok
|
|
|
381
437
|
For providers with a non-OpenAI usage shape:
|
|
382
438
|
|
|
383
439
|
```ruby
|
|
440
|
+
require "uri"
|
|
441
|
+
|
|
384
442
|
class AcmeParser < LlmCostTracker::Parsers::Base
|
|
385
443
|
def match?(url)
|
|
386
|
-
url.to_s
|
|
444
|
+
uri = URI.parse(url.to_s)
|
|
445
|
+
uri.host == "api.acme-llm.example" && uri.path == "/v1/generate"
|
|
446
|
+
rescue URI::InvalidURIError
|
|
447
|
+
false
|
|
387
448
|
end
|
|
388
449
|
|
|
389
450
|
def parse(request_url, request_body, response_status, response_body)
|
|
390
451
|
return nil unless response_status == 200
|
|
391
452
|
|
|
392
|
-
|
|
453
|
+
payload = safe_json_parse(response_body)
|
|
454
|
+
usage = payload&.dig("usage")
|
|
393
455
|
return nil unless usage
|
|
394
456
|
|
|
395
457
|
LlmCostTracker::ParsedUsage.build(
|
|
396
458
|
provider: "acme",
|
|
397
|
-
model:
|
|
459
|
+
model: payload["model"],
|
|
398
460
|
input_tokens: usage["input"] || 0,
|
|
399
461
|
output_tokens: usage["output"] || 0
|
|
400
462
|
)
|
|
@@ -420,9 +482,12 @@ Endpoints: OpenAI Chat Completions / Responses / Completions / Embeddings; OpenA
|
|
|
420
482
|
|
|
421
483
|
## Safety
|
|
422
484
|
|
|
485
|
+
**By design, `llm_cost_tracker` never persists prompt or response content.** The only data stored per call is the metadata needed for a cost ledger (provider, model, token counts, cost, latency, tags, provider response ID, HTTP status, and a timestamp). Tags carry whatever your application passes in — treat them as user-controlled input and avoid putting request bodies, completions, or secrets into them.
|
|
486
|
+
|
|
423
487
|
- No external HTTP calls at request-tracking time.
|
|
424
488
|
- No prompt or response bodies stored.
|
|
425
489
|
- Faraday responses not modified.
|
|
490
|
+
- Authorization headers and API keys are never stored or logged.
|
|
426
491
|
- Storage failures non-fatal by default (`storage_error_behavior = :warn`).
|
|
427
492
|
- Budget and unknown-pricing errors are raised only when you opt in.
|
|
428
493
|
|
data/SECURITY.md
ADDED
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# Security Policy
|
|
2
|
+
|
|
3
|
+
## Supported versions
|
|
4
|
+
|
|
5
|
+
The gem is pre-1.0. Only the latest released version receives security fixes — please upgrade rather than expecting backports to older releases.
|
|
6
|
+
|
|
7
|
+
## Reporting a vulnerability
|
|
8
|
+
|
|
9
|
+
Please **do not open a public GitHub issue** for security reports.
|
|
10
|
+
|
|
11
|
+
Email **sergey@mm.st** with:
|
|
12
|
+
|
|
13
|
+
- A description of the issue and its potential impact
|
|
14
|
+
- Steps to reproduce (a minimal proof-of-concept is ideal)
|
|
15
|
+
- The affected version(s)
|
|
16
|
+
- Any suggested mitigation or fix
|
|
17
|
+
|
|
18
|
+
You will receive an acknowledgment within **72 hours**. I will work with you on a disclosure timeline — typically a fix plus a coordinated release within 14 days for confirmed vulnerabilities, longer if the issue is complex.
|
|
19
|
+
|
|
20
|
+
## Scope
|
|
21
|
+
|
|
22
|
+
In scope:
|
|
23
|
+
|
|
24
|
+
- Vulnerabilities in the gem's middleware, parsers, storage adapters, dashboard controllers, or generators
|
|
25
|
+
- Data-exposure issues (unintended persistence of prompt content, API keys, or response bodies)
|
|
26
|
+
- Injection, auth-bypass, or privilege-escalation in the mounted dashboard
|
|
27
|
+
|
|
28
|
+
Out of scope:
|
|
29
|
+
|
|
30
|
+
- Issues in third-party dependencies (report those upstream; mention them here only if this gem's usage pattern creates the vulnerability)
|
|
31
|
+
- Missing security hardening recommendations that are not vulnerabilities (open a regular issue instead)
|
|
32
|
+
- Social engineering or physical attacks
|
|
33
|
+
|
|
34
|
+
## Credit
|
|
35
|
+
|
|
36
|
+
Reporters who follow this policy will be credited in the release notes for the fix unless they request anonymity.
|
|
@@ -2,8 +2,6 @@
|
|
|
2
2
|
|
|
3
3
|
module LlmCostTracker
|
|
4
4
|
class AssetsController < ActionController::Base
|
|
5
|
-
skip_forgery_protection if respond_to?(:skip_forgery_protection)
|
|
6
|
-
|
|
7
5
|
def stylesheet
|
|
8
6
|
response.set_header("Cache-Control", "public, max-age=31536000, immutable")
|
|
9
7
|
send_file LlmCostTracker::Assets::STYLESHEET_PATH, type: "text/css", disposition: "inline"
|
|
@@ -4,6 +4,8 @@ module LlmCostTracker
|
|
|
4
4
|
class Railtie < Rails::Railtie
|
|
5
5
|
generators do
|
|
6
6
|
require_relative "generators/llm_cost_tracker/add_latency_ms_generator"
|
|
7
|
+
require_relative "generators/llm_cost_tracker/add_provider_response_id_generator"
|
|
8
|
+
require_relative "generators/llm_cost_tracker/add_streaming_generator"
|
|
7
9
|
require_relative "generators/llm_cost_tracker/install_generator"
|
|
8
10
|
require_relative "generators/llm_cost_tracker/prices_generator"
|
|
9
11
|
require_relative "generators/llm_cost_tracker/upgrade_cost_precision_generator"
|
data/llm_cost_tracker.gemspec
CHANGED
|
@@ -28,7 +28,7 @@ Gem::Specification.new do |spec|
|
|
|
28
28
|
spec.files = Dir.chdir(__dir__) do
|
|
29
29
|
`git ls-files -z`.split("\x0").reject do |f|
|
|
30
30
|
(File.expand_path(f) == __FILE__) ||
|
|
31
|
-
f.start_with?("bin/", "test/", "spec/", ".git", ".github", "gemfiles/", ".rubocop", "Gemfile")
|
|
31
|
+
f.start_with?("bin/", "docs/", "test/", "spec/", ".git", ".github", "gemfiles/", ".rubocop", "Gemfile")
|
|
32
32
|
end
|
|
33
33
|
end
|
|
34
34
|
|
|
@@ -43,6 +43,8 @@ Gem::Specification.new do |spec|
|
|
|
43
43
|
spec.add_development_dependency "rake", "~> 13.0"
|
|
44
44
|
spec.add_development_dependency "rspec", "~> 3.0"
|
|
45
45
|
spec.add_development_dependency "rubocop", "~> 1.0"
|
|
46
|
+
spec.add_development_dependency "simplecov", "~> 0.22"
|
|
47
|
+
spec.add_development_dependency "simplecov-lcov", "~> 0.8"
|
|
46
48
|
spec.add_development_dependency "sqlite3", ">= 1.4", "< 3.0"
|
|
47
49
|
spec.add_development_dependency "webmock", "~> 3.0"
|
|
48
50
|
end
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: llm_cost_tracker
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.3.
|
|
4
|
+
version: 0.3.2
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Sergii Khomenko
|
|
@@ -146,6 +146,34 @@ dependencies:
|
|
|
146
146
|
- - "~>"
|
|
147
147
|
- !ruby/object:Gem::Version
|
|
148
148
|
version: '1.0'
|
|
149
|
+
- !ruby/object:Gem::Dependency
|
|
150
|
+
name: simplecov
|
|
151
|
+
requirement: !ruby/object:Gem::Requirement
|
|
152
|
+
requirements:
|
|
153
|
+
- - "~>"
|
|
154
|
+
- !ruby/object:Gem::Version
|
|
155
|
+
version: '0.22'
|
|
156
|
+
type: :development
|
|
157
|
+
prerelease: false
|
|
158
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
159
|
+
requirements:
|
|
160
|
+
- - "~>"
|
|
161
|
+
- !ruby/object:Gem::Version
|
|
162
|
+
version: '0.22'
|
|
163
|
+
- !ruby/object:Gem::Dependency
|
|
164
|
+
name: simplecov-lcov
|
|
165
|
+
requirement: !ruby/object:Gem::Requirement
|
|
166
|
+
requirements:
|
|
167
|
+
- - "~>"
|
|
168
|
+
- !ruby/object:Gem::Version
|
|
169
|
+
version: '0.8'
|
|
170
|
+
type: :development
|
|
171
|
+
prerelease: false
|
|
172
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
173
|
+
requirements:
|
|
174
|
+
- - "~>"
|
|
175
|
+
- !ruby/object:Gem::Version
|
|
176
|
+
version: '0.8'
|
|
149
177
|
- !ruby/object:Gem::Dependency
|
|
150
178
|
name: sqlite3
|
|
151
179
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -192,9 +220,11 @@ extra_rdoc_files: []
|
|
|
192
220
|
files:
|
|
193
221
|
- ".rspec"
|
|
194
222
|
- CHANGELOG.md
|
|
223
|
+
- CODE_OF_CONDUCT.md
|
|
195
224
|
- LICENSE.txt
|
|
196
225
|
- README.md
|
|
197
226
|
- Rakefile
|
|
227
|
+
- SECURITY.md
|
|
198
228
|
- app/assets/llm_cost_tracker/application.css
|
|
199
229
|
- app/controllers/llm_cost_tracker/application_controller.rb
|
|
200
230
|
- app/controllers/llm_cost_tracker/assets_controller.rb
|