django-llm-ledger 0.1.0__tar.gz

django_llm_ledger-0.1.0/.gitignore

@@ -0,0 +1,10 @@
+.venv/
+__pycache__/
+*.pyc
+*.egg-info/
+dist/
+build/
+.pytest_cache/
+*.sqlite3
+demo.sqlite3
+MELHORIAS.md

django_llm_ledger-0.1.0/CHANGELOG.md

@@ -0,0 +1,12 @@
+# Changelog
+
+## 0.1.0 (2026-11-08)
+
+First release.
+
+- `llm.complete()` and `llm.stream()`: one call that records provider, model, tokens, cost, latency, feature and user for every attempt.
+- Fallback chain across Ollama, OpenAI-compatible servers and Anthropic over plain HTTP, with retries and exponential backoff. Streams only fall back before the first chunk.
+- Cache for identical requests, monthly budget with a `budget_exceeded` signal and optional blocking.
+- Cost dashboard in the Django admin.
+- Evaluation suites (`manage.py llm_evaluate`) with score, cost, tokens and time per case; fails CI below `--min-score`.
+- Demo store in `example/`.

django_llm_ledger-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Israel Mendes
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

django_llm_ledger-0.1.0/PKG-INFO

@@ -0,0 +1,232 @@
+Metadata-Version: 2.5
+Name: django-llm-ledger
+Version: 0.1.0
+Summary: Cost tracking, provider fallback and evaluation suites for LLM features in Django apps.
+Project-URL: Source, https://github.com/Israel-Mendes-git/django-llm-ledger
+Project-URL: Changelog, https://github.com/Israel-Mendes-git/django-llm-ledger/blob/master/CHANGELOG.md
+Author: Israel Mendes
+License-Expression: MIT
+License-File: LICENSE
+Keywords: anthropic,cost,django,evals,llm,observability,ollama,openai
+Classifier: Development Status :: 3 - Alpha
+Classifier: Framework :: Django
+Classifier: Framework :: Django :: 4.2
+Classifier: Framework :: Django :: 5.1
+Classifier: Framework :: Django :: 5.2
+Classifier: Framework :: Django :: 6.1
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.10
+Requires-Dist: django>=4.2
+Requires-Dist: httpx>=0.27
+Provides-Extra: yaml
+Requires-Dist: pyyaml>=6.0; extra == 'yaml'
+Description-Content-Type: text/markdown
+
+# django-llm-ledger
+
+Know what your LLM features cost, keep them working when a provider goes down, and catch prompt regressions in CI.
+
+Most teams that "added AI to the product" find out three things later: nobody knows what it costs per feature, it breaks when the API has a bad day, and nobody can tell whether yesterday's prompt tweak made answers worse. `django-llm-ledger` is a small Django app that handles all three without changing how you write your feature.
+
+```python
+from llm_ledger import llm
+
+answer = llm.complete(
+    "Summarize this order for the customer: ...",
+    feature="order_summary",   # cost is tracked per feature
+    user=request.user,          # ...and per user
+)
+print(answer.text, answer.cost_usd, answer.latency_ms)
+```
+
+![Demo store: an answer streams from a local model, then the main provider goes down and the fallback answers](https://github.com/Israel-Mendes-git/django-llm-ledger/raw/master/docs/images/demo.gif)
+
+![Demo store: the main provider is down, the call is retried twice and a local model answers](https://github.com/Israel-Mendes-git/django-llm-ledger/raw/master/docs/images/demo-chat-fallback.png)
+
+![Cost dashboard in the Django admin](https://github.com/Israel-Mendes-git/django-llm-ledger/raw/master/docs/images/admin-dashboard.png)
+
+## What you get
+
+- **A ledger of every call**: provider, model, tokens, cost, latency, status, feature and user. Prompts are **not** stored unless you opt in, since they often carry personal data.
+- **A cost dashboard in the Django admin**: spend this month vs. budget, cost per day, top features, models and users, error rate, fallback count.
+- **Retries and fallback**: retryable errors (timeouts, 429, 5xx) are retried with exponential backoff; then the next provider in the chain takes over. Put a local model last and your feature keeps answering even when the paid API is down.
+- **Streaming**: `llm.stream()` yields the answer as it is written, with the same ledger, cache, retries and fallback.
+- **Cache**: identical requests are served from Django's cache and cost nothing.
+- **Budget guard**: a `budget_exceeded` signal when the monthly budget is reached, and optional hard blocking.
+- **Evaluation suites**: a file of questions plus what a good answer must (not) contain. `manage.py llm_evaluate` scores them and fails your CI when the score drops.
+- **No vendor SDKs**: Ollama, OpenAI-compatible servers (OpenAI, LM Studio, vLLM, Groq...) and Anthropic over plain HTTP. Only dependency besides Django is `httpx`.
+
+## Install
+
+```bash
+pip install django-llm-ledger          # add [yaml] for YAML suites
+```
+
+```python
+INSTALLED_APPS = [..., "llm_ledger"]
+
+LLM_LEDGER = {
+    # Tried in order. The first one that answers wins.
+    "PROVIDERS": [
+        {"name": "anthropic", "model": "claude-sonnet-5-5"},          # key from ANTHROPIC_API_KEY
+        {"name": "openai", "model": "gpt-4.1-mini"},                   # key from OPENAI_API_KEY
+        {"name": "ollama", "model": "llama3.2"},                       # local, free, last resort
+    ],
+    # USD per 1M tokens. Not shipped on purpose: prices change, and stale prices lie quietly.
+    "PRICES": {
+        "claude-sonnet-5-5": {"input": 3.0, "output": 15.0},
+        "gpt-4.1-mini": {"input": 0.4, "output": 1.6},
+    },
+    "MONTHLY_BUDGET_USD": 50,
+    "LOCAL_CURRENCY": {"code": "BRL", "rate": 5.4},   # optional second currency in the dashboard
+}
+```
+
+```bash
+python manage.py migrate
+```
+
+The dashboard lives at **Admin → LLM calls → Cost dashboard**.
+
+> The prices above are placeholders for the example. Check your provider's pricing page.
+
+## Settings
+
+| Key | Default | |
+|---|---|---|
+| `PROVIDERS` | `[{"name": "ollama", "model": "llama3.2"}]` | Fallback chain. `name` is `ollama`, `openai`, `anthropic`, `fake` or a dotted path to your own `Provider` subclass. Extra keys (`base_url`, `api_key`, `api_key_env`, `price`, `timeout`) go to the provider. |
+| `PRICES` | `{}` | USD per 1M tokens, per model. Paid calls without a price are recorded with `cost_usd=None` and flagged in the dashboard. |
+| `MAX_RETRIES` | `2` | Retries per provider for retryable errors. |
+| `RETRY_BACKOFF` | `0.5` | Base delay in seconds; doubles on each retry. |
+| `TIMEOUT` | `60` | HTTP timeout in seconds. |
+| `CACHE_TTL` | `3600` | Seconds to cache identical requests. `0` disables. |
+| `CACHE_ALIAS` | `"default"` | Which Django cache to use. |
+| `MONTHLY_BUDGET_USD` | `None` | Sends `budget_exceeded` when reached. |
+| `BLOCK_OVER_BUDGET` | `False` | Raise `BudgetExceeded` instead of just signalling. |
+| `LOCAL_CURRENCY` | `None` | `{"code": "BRL", "rate": 5.4}` to show a second currency. |
+| `STORE_PROMPTS` | `False` | Store prompt and response text in the ledger. |
+
+## Evaluation suites
+
+```yaml
+# evals/support.yaml
+name: support
+system: "You answer for ACME Store. Returns are accepted within 30 days."
+cases:
+  - id: return-window
+    input: "How long do I have to return an item?"
+    must_contain: ["30 days"]
+  - id: no-made-up-promises
+    input: "Can I return a used mattress after a year?"
+    must_not_contain: ["yes, of course"]
+  - id: shipping
+    input: "Do you ship to Brazil?"
+    any_of: ["yes", "we ship"]
+```
+
+To test your real feature (retrieval, tools, post-processing) instead of a raw prompt, point the suite at a function:
+
+```yaml
+target: shop.ai.answer_customer   # called as answer_customer(input) -> str
+```
+
+```bash
+python manage.py llm_evaluate evals/ --min-score 0.9
+```
+
+```text
+store-support  (evals/store-support.yaml)
+  PASS  return-window
+  ...
+  score 10/10 = 100%
+  cost $0.0042, 3120 tokens, 4120 ms per case
+```
+
+Each run also adds up what it cost, reading the ledger rows written while each case ran, so it
+works for `target` functions that make several calls. Runs are stored and show up in the dashboard
+with score, cost and time side by side: swap a model, rerun the suite, compare.
+
+## Streaming
+
+```python
+from django.http import StreamingHttpResponse
+from llm_ledger import llm
+
+def ask(request):
+    stream = llm.stream(request.POST["question"], feature="support", user=request.user)
+
+    def body():
+        with stream:          # if the browser leaves, the call is still recorded
+            yield from stream
+        # stream.result has cost, tokens, latency and first_chunk_ms once it ends
+
+    return StreamingHttpResponse(body(), content_type="text/plain")
+```
+
+Retries and fallback only happen **before the first chunk**. Once words have reached the reader,
+switching models would glue two different answers together, so a failure after that raises
+`LLMError` and the partial text stays in `stream.text`. Budget and cache are checked when you call
+`llm.stream()`, so `BudgetExceeded` is raised there, before you start the response. A cache hit
+comes back as a single chunk.
+
+Ollama, OpenAI-compatible servers and Anthropic stream natively. A custom provider without a
+`stream()` method still works: it yields the whole answer at once.
+
+The demo store streams its chat as newline-delimited JSON (`example/store/views.py`).
+
+## Handling failures in your code
+
+```python
+from llm_ledger import LLMError, BudgetExceeded, llm
+
+try:
+    answer = llm.complete(question, feature="support")
+except BudgetExceeded:
+    ...  # e.g. show "the assistant is resting, talk to a human"
+except LLMError as exc:
+    logger.error("every provider failed: %s", exc.errors)
+```
+
+Listen to the budget signal to notify someone:
+
+```python
+from django.dispatch import receiver
+from llm_ledger.signals import budget_exceeded
+
+@receiver(budget_exceeded)
+def warn_team(sender, spent, budget, **kwargs):
+    ...
+```
+
+## Demo store
+
+`example/` is a fictional coffee shop with a support chat built on the library. The chat finds the relevant help sections with embeddings from a small local model, answers from them and cites its sources. A side panel shows what each answer cost, which provider served it and every retry. A switch simulates the main provider going down, so you can watch the fallback happen live.
+
+```bash
+ollama pull qwen2.5:1.5b            # free local model (~1 GB); set ANTHROPIC_API_KEY to put Claude first
+ollama pull all-minilm              # embeddings for the help-center search (46 MB)
+cd example
+uv run --project .. python manage.py migrate
+uv run --project .. python manage.py seed_demo          # 30 days of synthetic history for the dashboard
+uv run --project .. python manage.py createsuperuser
+uv run --project .. python manage.py runserver
+uv run --project .. python manage.py llm_evaluate evals/ --min-score 0.8
+```
+
+## Development
+
+```bash
+uv sync
+uv run pytest
+```
+
+## License
+
+MIT

django_llm_ledger-0.1.0/README.md

@@ -0,0 +1,201 @@
+# django-llm-ledger
+
+Know what your LLM features cost, keep them working when a provider goes down, and catch prompt regressions in CI.
+
+Most teams that "added AI to the product" find out three things later: nobody knows what it costs per feature, it breaks when the API has a bad day, and nobody can tell whether yesterday's prompt tweak made answers worse. `django-llm-ledger` is a small Django app that handles all three without changing how you write your feature.
+
+```python
+from llm_ledger import llm
+
+answer = llm.complete(
+    "Summarize this order for the customer: ...",
+    feature="order_summary",   # cost is tracked per feature
+    user=request.user,          # ...and per user
+)
+print(answer.text, answer.cost_usd, answer.latency_ms)
+```
+
+![Demo store: an answer streams from a local model, then the main provider goes down and the fallback answers](https://github.com/Israel-Mendes-git/django-llm-ledger/raw/master/docs/images/demo.gif)
+
+![Demo store: the main provider is down, the call is retried twice and a local model answers](https://github.com/Israel-Mendes-git/django-llm-ledger/raw/master/docs/images/demo-chat-fallback.png)
+
+![Cost dashboard in the Django admin](https://github.com/Israel-Mendes-git/django-llm-ledger/raw/master/docs/images/admin-dashboard.png)
+
+## What you get
+
+- **A ledger of every call**: provider, model, tokens, cost, latency, status, feature and user. Prompts are **not** stored unless you opt in, since they often carry personal data.
+- **A cost dashboard in the Django admin**: spend this month vs. budget, cost per day, top features, models and users, error rate, fallback count.
+- **Retries and fallback**: retryable errors (timeouts, 429, 5xx) are retried with exponential backoff; then the next provider in the chain takes over. Put a local model last and your feature keeps answering even when the paid API is down.
+- **Streaming**: `llm.stream()` yields the answer as it is written, with the same ledger, cache, retries and fallback.
+- **Cache**: identical requests are served from Django's cache and cost nothing.
+- **Budget guard**: a `budget_exceeded` signal when the monthly budget is reached, and optional hard blocking.
+- **Evaluation suites**: a file of questions plus what a good answer must (not) contain. `manage.py llm_evaluate` scores them and fails your CI when the score drops.
+- **No vendor SDKs**: Ollama, OpenAI-compatible servers (OpenAI, LM Studio, vLLM, Groq...) and Anthropic over plain HTTP. Only dependency besides Django is `httpx`.
+
+## Install
+
+```bash
+pip install django-llm-ledger          # add [yaml] for YAML suites
+```
+
+```python
+INSTALLED_APPS = [..., "llm_ledger"]
+
+LLM_LEDGER = {
+    # Tried in order. The first one that answers wins.
+    "PROVIDERS": [
+        {"name": "anthropic", "model": "claude-sonnet-5-5"},          # key from ANTHROPIC_API_KEY
+        {"name": "openai", "model": "gpt-4.1-mini"},                   # key from OPENAI_API_KEY
+        {"name": "ollama", "model": "llama3.2"},                       # local, free, last resort
+    ],
+    # USD per 1M tokens. Not shipped on purpose: prices change, and stale prices lie quietly.
+    "PRICES": {
+        "claude-sonnet-5-5": {"input": 3.0, "output": 15.0},
+        "gpt-4.1-mini": {"input": 0.4, "output": 1.6},
+    },
+    "MONTHLY_BUDGET_USD": 50,
+    "LOCAL_CURRENCY": {"code": "BRL", "rate": 5.4},   # optional second currency in the dashboard
+}
+```
+
+```bash
+python manage.py migrate
+```
+
+The dashboard lives at **Admin → LLM calls → Cost dashboard**.
+
+> The prices above are placeholders for the example. Check your provider's pricing page.
+
+## Settings
+
+| Key | Default | |
+|---|---|---|
+| `PROVIDERS` | `[{"name": "ollama", "model": "llama3.2"}]` | Fallback chain. `name` is `ollama`, `openai`, `anthropic`, `fake` or a dotted path to your own `Provider` subclass. Extra keys (`base_url`, `api_key`, `api_key_env`, `price`, `timeout`) go to the provider. |
+| `PRICES` | `{}` | USD per 1M tokens, per model. Paid calls without a price are recorded with `cost_usd=None` and flagged in the dashboard. |
+| `MAX_RETRIES` | `2` | Retries per provider for retryable errors. |
+| `RETRY_BACKOFF` | `0.5` | Base delay in seconds; doubles on each retry. |
+| `TIMEOUT` | `60` | HTTP timeout in seconds. |
+| `CACHE_TTL` | `3600` | Seconds to cache identical requests. `0` disables. |
+| `CACHE_ALIAS` | `"default"` | Which Django cache to use. |
+| `MONTHLY_BUDGET_USD` | `None` | Sends `budget_exceeded` when reached. |
+| `BLOCK_OVER_BUDGET` | `False` | Raise `BudgetExceeded` instead of just signalling. |
+| `LOCAL_CURRENCY` | `None` | `{"code": "BRL", "rate": 5.4}` to show a second currency. |
+| `STORE_PROMPTS` | `False` | Store prompt and response text in the ledger. |
+
+## Evaluation suites
+
+```yaml
+# evals/support.yaml
+name: support
+system: "You answer for ACME Store. Returns are accepted within 30 days."
+cases:
+  - id: return-window
+    input: "How long do I have to return an item?"
+    must_contain: ["30 days"]
+  - id: no-made-up-promises
+    input: "Can I return a used mattress after a year?"
+    must_not_contain: ["yes, of course"]
+  - id: shipping
+    input: "Do you ship to Brazil?"
+    any_of: ["yes", "we ship"]
+```
+
+To test your real feature (retrieval, tools, post-processing) instead of a raw prompt, point the suite at a function:
+
+```yaml
+target: shop.ai.answer_customer   # called as answer_customer(input) -> str
+```
+
+```bash
+python manage.py llm_evaluate evals/ --min-score 0.9
+```
+
+```text
+store-support  (evals/store-support.yaml)
+  PASS  return-window
+  ...
+  score 10/10 = 100%
+  cost $0.0042, 3120 tokens, 4120 ms per case
+```
+
+Each run also adds up what it cost, reading the ledger rows written while each case ran, so it
+works for `target` functions that make several calls. Runs are stored and show up in the dashboard
+with score, cost and time side by side: swap a model, rerun the suite, compare.
+
+## Streaming
+
+```python
+from django.http import StreamingHttpResponse
+from llm_ledger import llm
+
+def ask(request):
+    stream = llm.stream(request.POST["question"], feature="support", user=request.user)
+
+    def body():
+        with stream:          # if the browser leaves, the call is still recorded
+            yield from stream
+        # stream.result has cost, tokens, latency and first_chunk_ms once it ends
+
+    return StreamingHttpResponse(body(), content_type="text/plain")
+```
+
+Retries and fallback only happen **before the first chunk**. Once words have reached the reader,
+switching models would glue two different answers together, so a failure after that raises
+`LLMError` and the partial text stays in `stream.text`. Budget and cache are checked when you call
+`llm.stream()`, so `BudgetExceeded` is raised there, before you start the response. A cache hit
+comes back as a single chunk.
+
+Ollama, OpenAI-compatible servers and Anthropic stream natively. A custom provider without a
+`stream()` method still works: it yields the whole answer at once.
+
+The demo store streams its chat as newline-delimited JSON (`example/store/views.py`).
+
+## Handling failures in your code
+
+```python
+from llm_ledger import LLMError, BudgetExceeded, llm
+
+try:
+    answer = llm.complete(question, feature="support")
+except BudgetExceeded:
+    ...  # e.g. show "the assistant is resting, talk to a human"
+except LLMError as exc:
+    logger.error("every provider failed: %s", exc.errors)
+```
+
+Listen to the budget signal to notify someone:
+
+```python
+from django.dispatch import receiver
+from llm_ledger.signals import budget_exceeded
+
+@receiver(budget_exceeded)
+def warn_team(sender, spent, budget, **kwargs):
+    ...
+```
+
+## Demo store
+
+`example/` is a fictional coffee shop with a support chat built on the library. The chat finds the relevant help sections with embeddings from a small local model, answers from them and cites its sources. A side panel shows what each answer cost, which provider served it and every retry. A switch simulates the main provider going down, so you can watch the fallback happen live.
+
+```bash
+ollama pull qwen2.5:1.5b            # free local model (~1 GB); set ANTHROPIC_API_KEY to put Claude first
+ollama pull all-minilm              # embeddings for the help-center search (46 MB)
+cd example
+uv run --project .. python manage.py migrate
+uv run --project .. python manage.py seed_demo          # 30 days of synthetic history for the dashboard
+uv run --project .. python manage.py createsuperuser
+uv run --project .. python manage.py runserver
+uv run --project .. python manage.py llm_evaluate evals/ --min-score 0.8
+```
+
+## Development
+
+```bash
+uv sync
+uv run pytest
+```
+
+## License
+
+MIT

django_llm_ledger-0.1.0/pyproject.toml

@@ -0,0 +1,59 @@
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[project]
+name = "django-llm-ledger"
+version = "0.1.0"
+description = "Cost tracking, provider fallback and evaluation suites for LLM features in Django apps."
+readme = "README.md"
+requires-python = ">=3.10"
+license = "MIT"
+license-files = ["LICENSE"]
+authors = [{ name = "Israel Mendes" }]
+keywords = ["django", "llm", "openai", "anthropic", "ollama", "cost", "observability", "evals"]
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Framework :: Django",
+    "Framework :: Django :: 4.2",
+    "Framework :: Django :: 5.1",
+    "Framework :: Django :: 5.2",
+    "Framework :: Django :: 6.1",
+    "Intended Audience :: Developers",
+    "Operating System :: OS Independent",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
+]
+dependencies = [
+    "Django>=4.2",
+    "httpx>=0.27",
+]
+
+[project.optional-dependencies]
+yaml = ["PyYAML>=6.0"]
+
+[dependency-groups]
+dev = [
+    "pytest>=8",
+    "pytest-django>=4.8",
+    "PyYAML>=6.0",
+]
+
+[project.urls]
+Source = "https://github.com/Israel-Mendes-git/django-llm-ledger"
+Changelog = "https://github.com/Israel-Mendes-git/django-llm-ledger/blob/master/CHANGELOG.md"
+
+[tool.hatch.build.targets.sdist]
+include = ["src/llm_ledger", "tests", "README.md", "CHANGELOG.md", "LICENSE", "pyproject.toml"]
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/llm_ledger"]
+
+[tool.pytest.ini_options]
+DJANGO_SETTINGS_MODULE = "tests.settings"
+pythonpath = [".", "src"]
+testpaths = ["tests"]

django_llm_ledger-0.1.0/src/llm_ledger/__init__.py

@@ -0,0 +1,18 @@
+"""Cost tracking, provider fallback and evaluation suites for LLM features in Django apps.
+
+    from llm_ledger import llm
+    answer = llm.complete("Summarize this order: ...", feature="order_summary", user=request.user)
+"""
+
+__version__ = "0.1.0"
+
+_LAZY = {"llm", "LLMClient", "LLMResult", "LLMStream", "LLMError", "BudgetExceeded"}
+
+
+def __getattr__(name):
+    # Imported lazily so that "llm_ledger" can sit in INSTALLED_APPS without touching models early.
+    if name in _LAZY:
+        from . import client
+
+        return getattr(client, name)
+    raise AttributeError(f"module 'llm_ledger' has no attribute {name!r}")

django_llm_ledger-0.1.0/src/llm_ledger/admin.py

@@ -0,0 +1,54 @@
+from django.contrib import admin
+from django.template.response import TemplateResponse
+from django.urls import path
+
+from . import stats
+from .models import EvalRun, LLMCall
+
+
+class ReadOnlyAdmin(admin.ModelAdmin):
+    def has_add_permission(self, request):
+        return False
+
+    def has_change_permission(self, request, obj=None):
+        return False
+
+
+@admin.register(LLMCall)
+class LLMCallAdmin(ReadOnlyAdmin):
+    change_list_template = "admin/llm_ledger/llmcall/change_list.html"
+    list_display = ("created_at", "feature", "provider", "model", "status", "is_fallback",
+                    "input_tokens", "output_tokens", "cost_usd", "latency_ms", "user")
+    list_filter = ("status", "is_fallback", "provider", "model", "feature")
+    search_fields = ("feature", "model", "request_id", "error")
+    date_hierarchy = "created_at"
+    list_select_related = ("user",)
+
+    def get_urls(self):
+        custom = [
+            path("dashboard/", self.admin_site.admin_view(self.dashboard_view), name="llm_ledger_dashboard"),
+        ]
+        return custom + super().get_urls()
+
+    def dashboard_view(self, request):
+        try:
+            days = max(1, min(int(request.GET.get("days", 30)), 365))
+        except ValueError:
+            days = 30
+        context = {
+            **self.admin_site.each_context(request),
+            "title": "LLM costs",
+            "opts": self.model._meta,
+            **stats.dashboard(days),
+        }
+        return TemplateResponse(request, "admin/llm_ledger/dashboard.html", context)
+
+
+@admin.register(EvalRun)
+class EvalRunAdmin(ReadOnlyAdmin):
+    list_display = ("created_at", "suite", "passed", "total", "score_pct", "cost_usd", "tokens", "latency_ms")
+    list_filter = ("suite",)
+
+    @admin.display(description="score")
+    def score_pct(self, obj):
+        return f"{obj.score:.0%}"

django_llm_ledger-0.1.0/src/llm_ledger/apps.py

@@ -0,0 +1,7 @@
+from django.apps import AppConfig
+
+
+class LLMLedgerConfig(AppConfig):
+    name = "llm_ledger"
+    verbose_name = "LLM ledger"
+    default_auto_field = "django.db.models.BigAutoField"