callm-toolkit 0.1.0__tar.gz

callm_toolkit-0.1.0/.gitignore

@@ -0,0 +1,21 @@
+__pycache__/
+*.py[cod]
+*.egg-info/
+.venv/
+venv/
+build/
+dist/
+site/
+.coverage
+.coverage.*
+coverage.xml
+htmlcov/
+.pytest_cache/
+.mypy_cache/
+.ruff_cache/
+.DS_Store
+.idea/
+.vscode/
+*.db
+*.db-wal
+*.db-shm

callm_toolkit-0.1.0/CHANGELOG.md

@@ -0,0 +1,42 @@
+# Changelog
+
+All notable changes to this project are documented here. The format follows
+[Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and the project uses
+[Semantic Versioning](https://semver.org/).
+
+## [Unreleased]
+
+## [0.1.0] - 2026-09-15
+
+Initial release.
+
+### Added
+
+- `@callm` decorator for sync and async functions and methods, intercepting OpenAI
+  (`chat.completions.create`), Anthropic (`messages.create`) and Google Gen AI
+  (`models.generate_content`) SDK calls and returning the SDK's native response types.
+- `shield` context manager and provider-neutral `complete` / `acomplete`.
+- Retry engine with exponential backoff and jitter that honours `retry-after`,
+  `retry-after-ms`, OpenAI and Anthropic rate-limit reset headers and Gemini `RetryInfo`.
+- Response cache: exact-match by default, optional semantic matching (sentence-transformers,
+  OpenAI embeddings, or a custom embedder); SQLite, in-memory and Redis backends; TTLs.
+- Provider fallback chains with request translation between OpenAI, Anthropic, Gemini, Ollama
+  and OpenAI-compatible providers; provider-specific requests only fall back within their
+  provider.
+- Cost tracking from a bundled price table (`callm pricing update` refreshes it), including
+  prompt-cache read/write pricing; `max_cost` per call; thread-safe `Budget`s per function,
+  session (`callm.budget`) or key (`callm.budget_for`).
+- PII redaction for emails, phone numbers, SSNs, payment cards (Luhn), IPv4 addresses and IBANs
+  (checksum), plus optional spaCy person names; mask or block.
+- Heuristic prompt-injection detection (instruction overrides, prompt extraction, role
+  hijacking, chat-template tokens, hidden unicode, multilingual variants) with an optional
+  Hugging Face classifier; flag or block.
+- Structured output validation for any Pydantic type with automatic re-asking on invalid output.
+- Telemetry records (no prompt text), `on_call` hooks, OpenTelemetry spans, `callm.stats()` and
+  `callm.last_call()`.
+- Nested scopes: protections (PII masking, injection detection, the strictest `max_cost`, all
+  budgets) from enclosing decorators and shields apply to inner calls.
+- `callm` CLI: `stats`, `calls`, `cache`, `pricing`, `telemetry`, `info`.
+
+[Unreleased]: https://github.com/TanbirRamim/callm/compare/v0.1.0...HEAD
+[0.1.0]: https://github.com/TanbirRamim/callm/releases/tag/v0.1.0

callm_toolkit-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Tanbir Hossain Ramim
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

callm_toolkit-0.1.0/PKG-INFO

@@ -0,0 +1,363 @@
+Metadata-Version: 2.5
+Name: callm-toolkit
+Version: 0.1.0
+Summary: The production toolkit for LLM calls: caching, retries, fallback, cost tracking, security and structured output in one decorator.
+Project-URL: Homepage, https://github.com/TanbirRamim/callm
+Project-URL: Documentation, https://tanbirramim.github.io/callm
+Project-URL: Repository, https://github.com/TanbirRamim/callm
+Project-URL: Issues, https://github.com/TanbirRamim/callm/issues
+Project-URL: Changelog, https://github.com/TanbirRamim/callm/blob/main/CHANGELOG.md
+Author: Tanbir Hossain Ramim
+License-Expression: MIT
+License-File: LICENSE
+Keywords: anthropic,caching,fallback,gemini,llm,openai,pii,production,prompt-injection,pydantic,retry
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Typing :: Typed
+Requires-Python: >=3.10
+Provides-Extra: all
+Requires-Dist: anthropic>=0.40; extra == 'all'
+Requires-Dist: google-genai>=1.0; extra == 'all'
+Requires-Dist: openai>=1.40; extra == 'all'
+Requires-Dist: opentelemetry-api>=1.20; extra == 'all'
+Requires-Dist: pydantic>=2.6; extra == 'all'
+Requires-Dist: redis>=5.0; extra == 'all'
+Requires-Dist: rich>=13.0; extra == 'all'
+Requires-Dist: sentence-transformers>=2.7; extra == 'all'
+Requires-Dist: spacy>=3.7; extra == 'all'
+Requires-Dist: tiktoken>=0.7; extra == 'all'
+Provides-Extra: anthropic
+Requires-Dist: anthropic>=0.40; extra == 'anthropic'
+Provides-Extra: cache
+Requires-Dist: sentence-transformers>=2.7; extra == 'cache'
+Provides-Extra: cli
+Requires-Dist: rich>=13.0; extra == 'cli'
+Provides-Extra: google
+Requires-Dist: google-genai>=1.0; extra == 'google'
+Provides-Extra: openai
+Requires-Dist: openai>=1.40; extra == 'openai'
+Provides-Extra: otel
+Requires-Dist: opentelemetry-api>=1.20; extra == 'otel'
+Provides-Extra: redis
+Requires-Dist: redis>=5.0; extra == 'redis'
+Provides-Extra: security
+Requires-Dist: spacy>=3.7; extra == 'security'
+Provides-Extra: tokens
+Requires-Dist: tiktoken>=0.7; extra == 'tokens'
+Provides-Extra: validation
+Requires-Dist: pydantic>=2.6; extra == 'validation'
+Description-Content-Type: text/markdown
+
+<div align="center">
+
+<img src="https://raw.githubusercontent.com/TanbirRamim/callm/main/docs/assets/logo.svg" alt="callm" width="96" height="96">
+
+# callm
+
+**The production toolkit for LLM calls.**
+
+Caching, retries, provider fallback, cost tracking, budgets, PII redaction, prompt-injection
+detection, structured output validation and telemetry — in one decorator, on top of the SDKs
+you already use. No proxy. No database server. Zero required dependencies.
+
+[![CI](https://github.com/TanbirRamim/callm/actions/workflows/ci.yml/badge.svg)](https://github.com/TanbirRamim/callm/actions/workflows/ci.yml)
+![Python 3.10+](https://img.shields.io/badge/python-3.10%20%7C%203.11%20%7C%203.12%20%7C%203.13-blue.svg)
+[![PyPI](https://img.shields.io/pypi/v/callm-toolkit.svg)](https://pypi.org/project/callm-toolkit/)
+[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/TanbirRamim/callm/blob/main/LICENSE)
+
+[Documentation](https://tanbirramim.github.io/callm) ·
+[Quickstart](#quickstart) ·
+[Features](#features) ·
+[How it works](#how-it-works) ·
+[CLI](#the-callm-cli)
+
+</div>
+
+---
+
+```python
+from callm import callm
+
+@callm(cache=True, retry=3, fallback=["anthropic/claude-sonnet-5"], max_cost=0.25,
+       block_pii=True, detect_injection=True, output_schema=Summary)
+def summarize(text: str) -> Summary:
+    return openai.chat.completions.create(
+        model="gpt-4o", messages=[{"role": "user", "content": text}]
+    )
+```
+
+`summarize()` still calls OpenAI with your code, your client and your API key — but now every
+call is cached, retried on rate limits, failed over to Claude if OpenAI is down, refused if it
+would cost more than 25¢, stripped of emails and phone numbers before it leaves your process,
+scanned for prompt injection, validated into a `Summary` object, and recorded in a local cost
+dashboard.
+
+## Why callm
+
+Every team shipping LLM features writes the same production checklist: retry on 429s, cache
+repeated prompts, fall back when a provider has an outage, track spend, keep PII out of prompts,
+catch injection attempts, and make the model return valid JSON. That usually means stitching
+together `tenacity`, a cache, a PII library, an output-parsing library and a pile of glue code —
+or deploying a proxy service.
+
+callm is a library: install it, add a decorator, ship.
+
+- **No rewrite.** Keep calling `openai`, `anthropic` or `google-genai` directly. callm intercepts
+  the SDK call inside decorated functions and hands you back the SDK's own response type.
+- **No infrastructure.** State lives in a local SQLite file (or memory, or Redis if you want a
+  shared cache).
+- **No required dependencies.** The core is standard library only; features that need extra
+  packages are optional extras.
+- **Sync and async.** Same behaviour for `def` and `async def`, including concurrent tasks.
+
+## Quickstart
+
+```bash
+pip install "callm-toolkit[openai,validation]"   # or callm-toolkit[all]
+```
+
+The package is published as `callm-toolkit`; you import it as `callm` and the CLI is `callm`.
+
+```python
+import openai
+from callm import callm
+
+client = openai.OpenAI()
+
+@callm()  # zero-config: retries + cost tracking
+def ask(question: str) -> str:
+    response = client.chat.completions.create(
+        model="gpt-4o-mini", messages=[{"role": "user", "content": question}]
+    )
+    return response.choices[0].message.content
+
+print(ask("What is the capital of France?"))
+```
+
+```console
+$ callm stats
+Provider   Calls   Tokens    Cost   Cache Hits   Saved   Errors   Latency
+──────────────────────────────────────────────────────────────────────────
+openai         1       31   $0.0000     0 (0%)   $0.00        0     412ms
+```
+
+### Try it without an API key
+
+`examples/offline_demo.py` runs the real OpenAI and Anthropic SDKs against a scripted fake
+server and walks through a rate-limit retry, PII masking, schema validation, a cache hit, a
+fallback to Claude during an outage, a blocked expensive call and a flagged injection:
+
+```bash
+pip install "callm-toolkit[openai,anthropic,validation]"
+export CALLM_HOME=/tmp/callm-demo      # keep demo data out of ~/.callm
+python examples/offline_demo.py
+callm stats
+```
+
+## Features
+
+| | Feature | What you get |
+|---|---|---|
+| 🔄 | **Smart retries** | Exponential backoff with full jitter on 408/409/429/5xx/529, timeouts and connection errors. Honours `retry-after`, `retry-after-ms`, OpenAI `x-ratelimit-reset-*`, Anthropic `anthropic-ratelimit-*-reset` and Gemini `RetryInfo`. |
+| 🗄️ | **Response cache** | Exact-match by default; opt-in semantic matching with sentence-transformers, OpenAI embeddings or your own embedder. SQLite, memory or Redis. TTLs. Refusals and invalid output are never cached. |
+| 🔀 | **Provider fallback** | Ordered chains across OpenAI, Anthropic, Gemini, Ollama and any OpenAI-compatible endpoint. Requests are translated between providers, and your code still receives the response type of the SDK it called. |
+| 💰 | **Cost tracking & budgets** | Per-call cost from a bundled price table (refreshable with `callm pricing update`), including prompt-cache read/write pricing. `max_cost` per call, shared `Budget`s per function, session or user — enforced *before* the request is sent. |
+| 🛡️ | **Input security** | PII redaction (emails, phones, SSNs, Luhn-checked cards, IPs, checksum-validated IBANs, optional spaCy names) with stable placeholders. Prompt-injection scoring with a fast heuristic detector and an optional local ML classifier; flag or block. |
+| ✅ | **Structured output** | Pass any Pydantic type as `output_schema`. Invalid output is re-requested with the validation errors appended, and the function returns the validated object. |
+| 📊 | **Telemetry** | Every call records provider, model, tokens, cost, savings, latency, retries, fallbacks and security flags — never prompt text. `callm stats`, `callm calls`, JSON export, `on_call` hooks and OpenTelemetry spans. |
+
+## Examples
+
+### Structured output with automatic repair
+
+```python
+from pydantic import BaseModel
+from callm import callm
+
+class Invoice(BaseModel):
+    vendor: str
+    total: float
+    currency: str
+
+@callm(output_schema=Invoice, validation_retries=2)
+def extract(text: str):
+    return anthropic_client.messages.create(
+        model="claude-sonnet-5", max_tokens=1024,
+        messages=[{"role": "user", "content": f"Extract the invoice as JSON:\n{text}"}],
+    )
+
+invoice = extract(raw_text)   # -> Invoice(vendor=..., total=..., currency=...)
+```
+
+### Fallback across providers
+
+```python
+@callm(retry=2, fallback=["anthropic/claude-sonnet-5", "google/gemini-2.5-flash", "ollama/llama3.1"])
+def answer(question: str):
+    return openai_client.chat.completions.create(
+        model="gpt-4o", messages=[{"role": "user", "content": question}]
+    )
+```
+
+If OpenAI keeps returning 429/5xx, callm retries, then sends the same conversation to Claude,
+then Gemini, then a local model — and `answer()` still returns an OpenAI `ChatCompletion`.
+Requests that use provider-specific features (tools, images, response formats) only fall back
+to models of the same provider, so a fallback never silently changes what you asked for.
+
+### Budgets per user
+
+```python
+import callm
+
+@callm.callm(max_cost=0.05)
+def chat(messages): ...
+
+def handle(user_id: str, messages):
+    with callm.budget_for(f"user:{user_id}", limit=2.00):
+        return chat(messages)   # raises callm.BudgetExceeded once the user spent $2
+```
+
+### Security without a decorator
+
+```python
+from callm import shield
+
+with shield(block_pii=True, detect_injection=True) as s:
+    # Any supported SDK call inside the block is protected...
+    openai_client.chat.completions.create(model="gpt-4o-mini", messages=user_messages)
+    # ...and you can make provider-neutral calls directly.
+    response = s.complete(provider="anthropic", model="claude-sonnet-5", messages=user_messages)
+    print(response.text, response.cost)
+```
+
+### Direct, provider-neutral calls
+
+```python
+response = callm.complete("gemini/gemini-2.5-flash", "Summarize: ...", max_tokens=200, cache=True)
+response.text, response.usage.total_tokens, response.cost, response.raw
+```
+
+More in the [cookbook](https://tanbirramim.github.io/callm/cookbook/): a support chatbot, RAG answers with citations and data
+extraction.
+
+## How it works
+
+```text
+your function                         callm middleware stack
+─────────────                         ──────────────────────
+@callm(...)                    ┌────────────────────────────────────┐
+def summarize():   ─────────►  │ 1. Telemetry     cost, latency, tokens
+    client.chat.completions    │ 2. Security      injection scan, PII masking
+        .create(...)           │ 3. Cache         exact / semantic lookup
+                               │ 4. Validator     parse + re-ask on invalid output
+                               │ 5. Fallback      next provider when one keeps failing
+                               │ 6. Cost guard    max_cost and budgets, before sending
+                               │ 7. Retry         backoff honouring retry-after
+                               │ 8. Transport ──► the real SDK call
+                               └────────────────────────────────────┘
+```
+
+1. `@callm` sets a scope (a `ContextVar`) while your function runs.
+2. The official SDK methods (`chat.completions.create`, `messages.create`,
+   `models.generate_content`, sync and async) are instrumented on first use. Outside a callm
+   scope they call straight through, so importing callm never changes other code.
+3. Inside a scope, the SDK call is converted into a provider-neutral request and sent through
+   the middleware chain. Each layer is independent and only enabled when configured.
+4. The response is converted back into the SDK's native type before it is returned to you.
+
+The middleware is written once as generators and executed by a sync or an async driver, so
+`def` and `async def` functions behave identically.
+
+## Configuration
+
+```python
+import callm
+
+callm.configure(
+    home="~/.callm",          # SQLite database and price overrides
+    storage="sqlite",         # "sqlite" | "memory" | a storage object
+    telemetry=True,
+    otel=False,               # emit OpenTelemetry spans
+    default_retries=2,
+    on_call=[print],          # called with every CallRecord
+)
+```
+
+| Environment variable | Effect |
+|---|---|
+| `CALLM_HOME` | Data directory (default `~/.callm`) |
+| `CALLM_STORAGE` | `sqlite` or `memory` |
+| `CALLM_TELEMETRY=0` | Do not persist call records |
+| `CALLM_OTEL=1` | Export OpenTelemetry spans |
+| `CALLM_DISABLED=1` | Kill switch: decorated functions run untouched |
+
+Every option of `@callm` is documented in the [API reference](https://tanbirramim.github.io/callm/api-reference/).
+
+## The `callm` CLI
+
+```console
+$ callm stats --since 7d                 # cost, tokens, cache hits and savings by provider
+$ callm stats --by function --json       # machine-readable, per function
+$ callm calls --limit 20                 # recent calls with retries, fallbacks, flags
+$ callm cache stats | clear
+$ callm pricing show claude-sonnet-5     # USD per 1M tokens
+$ callm pricing update                   # refresh prices from the LiteLLM price list
+$ callm info                             # environment and installed extras
+```
+
+## Installation extras
+
+| Extra | Installs | Needed for |
+|---|---|---|
+| `openai` / `anthropic` / `google` | provider SDKs | calling those providers |
+| `validation` | `pydantic>=2` | `output_schema` |
+| `cache` | `sentence-transformers` | semantic caching with local embeddings |
+| `security` | `spacy` | person-name redaction (`PIIConfig(ner=True)`) |
+| `redis` | `redis` | shared cache across hosts |
+| `otel` | `opentelemetry-api` | OpenTelemetry spans |
+| `tokens` | `tiktoken` | exact OpenAI token estimates for the cost guard |
+| `cli` | `rich` | prettier `callm stats` tables |
+| `all` | everything above | |
+
+## Design decisions and limits
+
+- **The cache is exact-match unless you opt into semantic matching.** A semantic cache with a
+  similarity threshold would happily return the answer for *"Summarize https://a.example"* when
+  asked about *"https://b.example"*. Use `cache="semantic"` (or `CacheConfig(semantic=True)`) for
+  FAQ-style traffic; semantic matches never cross different system prompts, histories,
+  parameters or schemas.
+- **Injection detection flags by default.** Heuristics have false positives, so the default
+  logs a warning and records the score; use `InjectionConfig(action="block")` to refuse. No
+  detector catches every attack — keep treating model output as untrusted.
+- **Budgets use estimates before a call and actual cost after it.** Set `max_tokens` for a
+  tight worst-case estimate; without it the cost guard assumes 1,024 output tokens.
+- **Streaming calls** get security, retries on connection setup, budgets and telemetry, but are
+  not cached or validated.
+- **Threads:** the callm scope follows `asyncio` tasks automatically. Work handed to a thread
+  pool needs `contextvars.copy_context().run(...)`, or decorate the function running in the
+  thread.
+- **Prices** are a bundled snapshot. Run `callm pricing update` or `callm.set_price(...)` for
+  current or negotiated rates.
+
+## Contributing
+
+Contributions are welcome — see [CONTRIBUTING.md](https://github.com/TanbirRamim/callm/blob/main/CONTRIBUTING.md). The test suite runs
+entirely offline against the real provider SDKs with mocked HTTP transports:
+
+```bash
+uv sync
+uv run pytest
+```
+
+## License
+
+[MIT](https://github.com/TanbirRamim/callm/blob/main/LICENSE)