callm-toolkit 0.1.2__tar.gz → 0.3.0__tar.gz

{callm_toolkit-0.1.2 → callm_toolkit-0.3.0}/CHANGELOG.md

@@ -6,6 +6,34 @@ All notable changes to this project are documented here. The format follows
 
 ## [Unreleased]
 
+## [0.3.0] - 2026-10-02
+
+### Added
+
+- OpenAI structured outputs: `client.chat.completions.parse(...)`, sync and async, is now
+  intercepted. PII masking, budgets, caching, retries, cost and telemetry apply, and the
+  caller still gets a `ParsedChatCompletion` with `.parsed`, also on cache hits. Different
+  `response_format` models never share a cache entry.
+
+### Fixed
+
+- Storing a `ParsedChatCompletion` for the cache no longer prints pydantic serializer
+  warnings.
+
+## [0.2.0] - 2026-09-29
+
+### Added
+
+- OpenAI Responses API support: `client.responses.create(...)`, sync and async, is intercepted
+  like `chat.completions.create`. `instructions`, string and item `input` and tool outputs get
+  PII masking and injection checks; budgets, caching, output validation, retries, cost and
+  telemetry apply; fallbacks to other providers return a real `Response` object. The adapter is
+  also available as the `openai-responses` provider for `callm.complete` and fallback chains.
+- Inside a callm function, SDK helpers callm does not intercept yet (`responses.parse/stream`,
+  `chat.completions.parse/stream`, `beta.chat.completions.parse/stream`, Anthropic
+  `messages.stream`) now log a one-time warning naming the protections that do not apply,
+  instead of silently skipping PII masking, budgets and telemetry.
+
 ## [0.1.2] - 2026-09-27
 
 ### Fixed
@@ -86,7 +114,9 @@ Initial release.
   budgets) from enclosing decorators and shields apply to inner calls.
 - `callm` CLI: `stats`, `calls`, `cache`, `pricing`, `telemetry`, `info`.
 
-[Unreleased]: https://github.com/TanbirRamim/callm/compare/v0.1.1...HEAD
+[Unreleased]: https://github.com/TanbirRamim/callm/compare/v0.3.0...HEAD
+[0.3.0]: https://github.com/TanbirRamim/callm/compare/v0.2.0...v0.3.0
+[0.2.0]: https://github.com/TanbirRamim/callm/compare/v0.1.2...v0.2.0
 [0.1.2]: https://github.com/TanbirRamim/callm/compare/v0.1.1...v0.1.2
 [0.1.1]: https://github.com/TanbirRamim/callm/compare/v0.1.0...v0.1.1
 [0.1.0]: https://github.com/TanbirRamim/callm/releases/tag/v0.1.0

{callm_toolkit-0.1.2 → callm_toolkit-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: callm-toolkit
-Version: 0.1.2
+Version: 0.3.0
 Summary: The production toolkit for LLM calls: caching, retries, fallback, cost tracking, security and structured output in one decorator.
 Project-URL: Homepage, https://github.com/TanbirRamim/callm
 Project-URL: Documentation, https://tanbirramim.github.io/callm
@@ -10,7 +10,7 @@ Project-URL: Changelog, https://github.com/TanbirRamim/callm/blob/main/CHANGELOG
 Author: Tanbir Hossain Ramim
 License-Expression: MIT
 License-File: LICENSE
-Keywords: anthropic,caching,fallback,gemini,llm,openai,pii,production,prompt-injection,pydantic,retry
+Keywords: anthropic,budget,caching,cost-tracking,fallback,gemini,llm,llmops,openai,pii,production,prompt-injection,pydantic,retry,telemetry
 Classifier: Development Status :: 4 - Beta
 Classifier: Intended Audience :: Developers
 Classifier: Operating System :: OS Independent
@@ -364,8 +364,8 @@ def summarize():   ─────────►  │ 1. Telemetry     cost, la
 ```
 
 1. `@callm` sets a scope (a `ContextVar`) while your function runs.
-2. The official SDK methods (`chat.completions.create`, `messages.create`,
-   `models.generate_content`, sync and async) are instrumented on first use. Outside a callm
+2. The official SDK methods (`chat.completions.create` and `.parse`, OpenAI's `responses.create`,
+   `messages.create`, `models.generate_content`, sync and async) are instrumented on first use. Outside a callm
    scope they call straight through, so importing callm never changes other code.
 3. Inside a scope, the SDK call is converted into a provider-neutral request and sent through
    the middleware chain. Each layer is independent and only enabled when configured.
@@ -473,7 +473,7 @@ Measured offline with the real OpenAI SDK against a fake server
 
 ## Contributing
 
-Contributions are welcome — see [CONTRIBUTING.md](https://github.com/TanbirRamim/callm/blob/main/CONTRIBUTING.md). The test suite runs
+Contributions are welcome. The [good first issues](https://github.com/TanbirRamim/callm/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) are small and scoped, and every pull request gets a review. See [CONTRIBUTING.md](https://github.com/TanbirRamim/callm/blob/main/CONTRIBUTING.md). The test suite runs
 entirely offline against the real provider SDKs with mocked HTTP transports:
 
 ```bash
@@ -481,6 +481,8 @@ uv sync
 uv run pytest
 ```
 
+If callm is useful to you, a star helps other people find it.
+
 ## License
 
 [MIT](https://github.com/TanbirRamim/callm/blob/main/LICENSE)

{callm_toolkit-0.1.2 → callm_toolkit-0.3.0}/README.md

@@ -305,8 +305,8 @@ def summarize():   ─────────►  │ 1. Telemetry     cost, la
 ```
 
 1. `@callm` sets a scope (a `ContextVar`) while your function runs.
-2. The official SDK methods (`chat.completions.create`, `messages.create`,
-   `models.generate_content`, sync and async) are instrumented on first use. Outside a callm
+2. The official SDK methods (`chat.completions.create` and `.parse`, OpenAI's `responses.create`,
+   `messages.create`, `models.generate_content`, sync and async) are instrumented on first use. Outside a callm
    scope they call straight through, so importing callm never changes other code.
 3. Inside a scope, the SDK call is converted into a provider-neutral request and sent through
    the middleware chain. Each layer is independent and only enabled when configured.
@@ -414,7 +414,7 @@ Measured offline with the real OpenAI SDK against a fake server
 
 ## Contributing
 
-Contributions are welcome — see [CONTRIBUTING.md](https://github.com/TanbirRamim/callm/blob/main/CONTRIBUTING.md). The test suite runs
+Contributions are welcome. The [good first issues](https://github.com/TanbirRamim/callm/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) are small and scoped, and every pull request gets a review. See [CONTRIBUTING.md](https://github.com/TanbirRamim/callm/blob/main/CONTRIBUTING.md). The test suite runs
 entirely offline against the real provider SDKs with mocked HTTP transports:
 
 ```bash
@@ -422,6 +422,8 @@ uv sync
 uv run pytest
 ```
 
+If callm is useful to you, a star helps other people find it.
+
 ## License
 
 [MIT](https://github.com/TanbirRamim/callm/blob/main/LICENSE)

{callm_toolkit-0.1.2 → callm_toolkit-0.3.0}/pyproject.toml

@@ -11,7 +11,7 @@ license = "MIT"
 license-files = ["LICENSE"]
 requires-python = ">=3.10"
 authors = [{ name = "Tanbir Hossain Ramim" }]
-keywords = ["llm", "openai", "anthropic", "gemini", "caching", "retry", "fallback", "pii", "prompt-injection", "pydantic", "production"]
+keywords = ["llm", "openai", "anthropic", "gemini", "caching", "retry", "fallback", "pii", "prompt-injection", "pydantic", "production", "llmops", "cost-tracking", "budget", "telemetry"]
 classifiers = [
   "Development Status :: 4 - Beta",
   "Intended Audience :: Developers",

callm_toolkit-0.3.0/src/callm/__about__.py

@@ -0,0 +1 @@
+__version__ = "0.3.0"

{callm_toolkit-0.1.2 → callm_toolkit-0.3.0}/src/callm/cli.py

@@ -272,7 +272,7 @@ def _cmd_info(args: argparse.Namespace) -> int:
         "enabled": settings.enabled,
         "providers": provider_names(),
         "instrumented_methods": sorted(
-            {f"{t.provider}: {t.cls}.{t.method}" for t in PATCH_TARGETS}
+            {f"{t.provider}: {t.cls}.{t.method}" for t in PATCH_TARGETS if not t.warn_only}
         ),
         "optional_packages": optional,
     }

{callm_toolkit-0.1.2 → callm_toolkit-0.3.0}/src/callm/decorator.py

@@ -42,7 +42,9 @@ def detect_provider(value: Any) -> str | None:
     """Which provider SDK produced ``value`` (a native response object), if any."""
     module = type(value).__module__ or ""
     if module.startswith("openai"):
-        return "openai" if hasattr(value, "choices") else None
+        if hasattr(value, "choices"):
+            return "openai"
+        return "openai-responses" if hasattr(value, "output") and hasattr(value, "usage") else None
     if module.startswith("anthropic"):
         return "anthropic" if hasattr(value, "content") and hasattr(value, "usage") else None
     if module.startswith("google.genai"):
@@ -50,6 +52,8 @@ def detect_provider(value: Any) -> str | None:
     if isinstance(value, AttrDict):
         if "choices" in value:
             return "openai"
+        if value.get("object") == "response":
+            return "openai-responses"
         if value.get("type") == "message":
             return "anthropic"
         if "candidates" in value:

{callm_toolkit-0.1.2 → callm_toolkit-0.3.0}/src/callm/interception.py

@@ -95,17 +95,60 @@ class PatchTarget:
     cls: str
     method: str
     is_async: bool
+    #: True for SDK methods callm does not intercept yet: inside @callm they only log a
+    #: one-time warning, so nobody assumes PII masking or budgets apply when they do not.
+    warn_only: bool = False
 
 
 PATCH_TARGETS: tuple[PatchTarget, ...] = (
     PatchTarget("openai", "openai.resources.chat.completions", "Completions", "create", False),
     PatchTarget("openai", "openai.resources.chat.completions", "AsyncCompletions", "create", True),
+    PatchTarget("openai", "openai.resources.chat.completions", "Completions", "parse", False),
+    PatchTarget("openai", "openai.resources.chat.completions", "AsyncCompletions", "parse", True),
+    PatchTarget("openai-responses", "openai.resources.responses", "Responses", "create", False),
+    PatchTarget("openai-responses", "openai.resources.responses", "AsyncResponses", "create", True),
     PatchTarget("anthropic", "anthropic.resources.messages", "Messages", "create", False),
     PatchTarget("anthropic", "anthropic.resources.messages", "AsyncMessages", "create", True),
     PatchTarget("google", "google.genai.models", "Models", "generate_content", False),
     PatchTarget("google", "google.genai.models", "AsyncModels", "generate_content", True),
+    *(
+        PatchTarget(provider, module, cls, method, cls.startswith("Async"), warn_only=True)
+        for provider, module, classes, methods in (
+            (
+                "openai-responses",
+                "openai.resources.responses",
+                ("Responses", "AsyncResponses"),
+                ("parse", "stream"),
+            ),
+            (
+                "openai",
+                "openai.resources.chat.completions",
+                ("Completions", "AsyncCompletions"),
+                ("stream",),
+            ),
+            (
+                "openai",
+                "openai.resources.beta.chat.completions",
+                ("Completions", "AsyncCompletions"),
+                ("parse", "stream"),
+            ),
+            (
+                "anthropic",
+                "anthropic.resources.messages",
+                ("Messages", "AsyncMessages"),
+                ("stream",),
+            ),
+        )
+        for cls in classes
+        for method in methods
+    ),
 )
-_ROOT_PACKAGES = {"openai": "openai", "anthropic": "anthropic", "google": "google.genai"}
+_ROOT_PACKAGES = {
+    "openai": "openai",
+    "openai-responses": "openai",
+    "anthropic": "anthropic",
+    "google": "google.genai",
+}
 _RAW_RESPONSE_HEADERS = frozenset({"x-stainless-raw-response", "x-stainless-streamed-raw-response"})
 
 _lock = threading.RLock()
@@ -160,6 +203,7 @@ def client_endpoint(resource: Any) -> str | None:
 # Where each patched resource lives on its SDK client, to rebuild it on a client copy.
 _RESOURCE_PATHS: dict[str, tuple[str, ...]] = {
     "openai": ("chat", "completions"),
+    "openai-responses": ("responses",),
     "anthropic": ("messages",),
 }
 _no_retry_resources: weakref.WeakKeyDictionary[Any, Any] = weakref.WeakKeyDictionary()
@@ -238,7 +282,99 @@ def clean_kwargs(kwargs: dict[str, Any]) -> dict[str, Any]:
 def _finish(target: PatchTarget, state: CallState, response: LLMResponse) -> Any:
     if state.request.stream:
         return response.raw
-    return get_provider(target.provider).native_for(response)
+    native = get_provider(target.provider).native_for(response)
+    if target.provider == "openai" and target.method == "parse":
+        native = _as_parsed_completion(native, state.request.native_extra)
+    return native
+
+
+def _as_parsed_completion(native: Any, extra: dict[str, Any]) -> Any:
+    """The ``ParsedChatCompletion`` that ``chat.completions.parse`` promises its caller.
+
+    A live call already returns one. A cache hit, or a response synthesized after a
+    fallback, is a plain ``ChatCompletion``, so run it through the SDK's own parser
+    with the caller's ``response_format`` and ``tools``.
+    """
+    if type(native).__name__.startswith("Parsed"):
+        return native
+    try:
+        import openai
+        from openai.lib._parsing import parse_chat_completion
+    except ImportError:
+        return native
+    not_given: Any = getattr(openai, "omit", None) or getattr(openai, "NOT_GIVEN", None)
+    response_format: Any = extra.get("response_format", not_given)
+    input_tools: Any = extra.get("tools", not_given)
+    try:
+        return parse_chat_completion(
+            response_format=response_format,
+            input_tools=input_tools,
+            chat_completion=native,
+        )
+    except (TypeError, AttributeError):  # an AttrDict without the SDK types
+        logger.debug("callm: could not rebuild a ParsedChatCompletion", exc_info=True)
+        return native
+
+
+_warned_methods: set[str] = set()
+_SUPPORTED_ALTERNATIVE = {
+    "anthropic": "messages.create",
+    "openai-responses": "responses.create",
+}
+
+
+def _unprotected_features(config: CallConfig) -> str:
+    features = []
+    if config.pii is not None:
+        features.append("PII masking")
+    if config.injection is not None:
+        features.append("injection checks")
+    if config.max_cost is not None or config.budgets:
+        features.append("budgets")
+    if config.cache is not None:
+        features.append("caching")
+    if config.output_schema is not None:
+        features.append("output validation")
+    if config.fallback:
+        features.append("fallback")
+    if config.retry.max_retries:
+        features.append("callm retries")
+    features.append("cost tracking and telemetry")
+    return ", ".join(features)
+
+
+def _make_warning_wrapper(target: PatchTarget, original: Callable[..., Any]) -> Callable[..., Any]:
+    """Pass-through wrapper for SDK methods callm does not intercept yet.
+
+    Plain (non-async) on purpose: async originals return their coroutine or stream manager
+    unchanged, so behaviour and return types stay exactly those of the SDK.
+    """
+    name = f"{target.provider} {target.cls}.{target.method}"
+    label = (
+        "responses." + target.method
+        if "responses" in target.module
+        else ("messages." if target.provider == "anthropic" else "chat.completions.")
+        + target.method
+    )
+
+    @functools.wraps(original)
+    def wrapper(self: Any, *args: Any, **kwargs: Any) -> Any:
+        ctx = ACTIVE.get()
+        if ctx is not None and not BYPASS.get() and name not in _warned_methods:
+            _warned_methods.add(name)
+            logger.warning(
+                "callm: %s (%s) is not intercepted by callm yet, so inside %s it runs without %s. "
+                "Use %s for now: https://tanbirramim.github.io/callm/faq/",
+                label,
+                target.provider,
+                ctx.name or "@callm",
+                _unprotected_features(ctx.config),
+                _SUPPORTED_ALTERNATIVE.get(target.provider, "chat.completions.create"),
+            )
+        return original(self, *args, **kwargs)
+
+    wrapper.__callm_original__ = original  # type: ignore[attr-defined]
+    return wrapper
 
 
 def _make_sync_wrapper(target: PatchTarget, original: Callable[..., Any]) -> Callable[..., Any]:
@@ -304,7 +440,10 @@ def _patch_module(module: ModuleType, provider: str) -> None:
             if existing is not None:  # already instrumented (e.g. module reloaded)
                 _patched[target] = existing
                 continue
-            make = _make_async_wrapper if target.is_async else _make_sync_wrapper
+            if target.warn_only:
+                make = _make_warning_wrapper
+            else:
+                make = _make_async_wrapper if target.is_async else _make_sync_wrapper
             setattr(cls, target.method, make(target, original))
             _patched[target] = original
             logger.debug("callm: instrumented %s.%s.%s", target.module, target.cls, target.method)

{callm_toolkit-0.1.2 → callm_toolkit-0.3.0}/src/callm/middleware/telemetry.py

@@ -103,7 +103,7 @@ def _export_span(record: CallRecord, start_ns: int | None) -> None:
         )
         attributes: dict[str, Any] = {
             "gen_ai.operation.name": "chat",
-            "gen_ai.system": record.provider,
+            "gen_ai.system": "openai" if record.provider == "openai-responses" else record.provider,
             "gen_ai.request.model": record.model,
             "gen_ai.usage.input_tokens": record.input_tokens
             + record.cache_read_tokens

{callm_toolkit-0.1.2 → callm_toolkit-0.3.0}/src/callm/pricing.py

@@ -161,8 +161,11 @@ def _lookup(table: dict[str, ModelPrice], name: str) -> ModelPrice | None:
     return table[best] if best is not None else None
 
 
+_PRICE_ALIASES = {"gemini": "google", "openai-responses": "openai"}
+
+
 def _provider_key(provider: str) -> str:
-    return "google" if provider == "gemini" else provider
+    return _PRICE_ALIASES.get(provider, provider)
 
 
 def get_price(provider: str, model: str) -> ModelPrice | None:
@@ -235,7 +238,7 @@ def estimate_tokens(text: str, model: str | None = None, provider: str | None =
     """Estimate a token count. Uses ``tiktoken`` for OpenAI models when it is installed."""
     if not text:
         return 0
-    if provider in (None, "openai") and model is not None:
+    if provider in (None, "openai", "openai-responses") and model is not None:
         encoder = _tiktoken_encoder(model)
         if encoder is not None:
             return len(encoder.encode(text, disallowed_special=()))

{callm_toolkit-0.1.2 → callm_toolkit-0.3.0}/src/callm/providers/__init__.py

@@ -4,12 +4,14 @@ from callm.providers.anthropic import AnthropicProvider
 from callm.providers.base import Provider
 from callm.providers.google import GoogleProvider
 from callm.providers.openai import OpenAIProvider
+from callm.providers.openai_responses import OpenAIResponsesProvider
 from callm.providers.registry import get_provider, parse_target, provider_names, register_provider
 
 __all__ = [
     "AnthropicProvider",
     "GoogleProvider",
     "OpenAIProvider",
+    "OpenAIResponsesProvider",
     "Provider",
     "get_provider",
     "parse_target",

{callm_toolkit-0.1.2 → callm_toolkit-0.3.0}/src/callm/providers/base.py

@@ -61,7 +61,14 @@ def dump_json(obj: Any) -> dict[str, Any] | None:
     dump = getattr(obj, "model_dump", None)
     if callable(dump):
         try:
-            result = dump(mode="json", exclude_none=True)
+            # warnings=False: ParsedChatCompletion types `parsed` loosely, so pydantic
+            # would warn on every call even though the dump is correct.
+            result = dump(mode="json", exclude_none=True, warnings=False)
+        except TypeError:  # an object whose model_dump has no `warnings` option
+            try:
+                result = dump(mode="json", exclude_none=True)
+            except Exception:
+                return None
         except Exception:
             return None
         return result if isinstance(result, dict) else None