frugal-sdk-python 0.2.0__tar.gz → 0.3.0__tar.gz

{frugal_sdk_python-0.2.0 → frugal_sdk_python-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: frugal-sdk-python
-Version: 0.2.0
+Version: 0.3.0
 Summary: Instrumentation for Cost-to-Code attribution.
 Project-URL: Homepage, https://frugal.co
 Author: Frugal AI Inc.
@@ -78,6 +78,7 @@ Set both or the wrapper silently returns the client unmodified:
 | `FRUGAL_OTEL_EXPORTER_OTLP_ENDPOINT`     | OTLP collector URL for Frugal metrics. Defaults to `https://metrics.frugal.co/opentelemetry/v1/metrics`. Falls back to `OTEL_EXPORTER_OTLP_ENDPOINT`. |
 | `FRUGAL_OTEL_EXPORTER_OTLP_HEADERS`      | Export auth headers (e.g. `Authorization=Bearer ...`). Overrides the header built from `FRUGAL_API_KEY`. Falls back to `OTEL_EXPORTER_OTLP_HEADERS`. |
 | `FRUGAL_COMPONENTS`                      | Comma-separated logical components (e.g. `ai-handler,reco`). When set, each metric is ALSO emitted once per component (tagged `frugal.component`) on top of the base project series, so usage can be viewed for the whole project or split by component. |
+| `FRUGAL_CUSTOM_PROVIDERS`                | Comma-separated allowlist of provider names beyond the built-in registry (e.g. `together_ai,fireworks`) that should pass through verbatim on `gen_ai.system` / `gen_ai.provider.name` instead of being coerced to `custom`. Capped at 10 (extras are coerced) to keep label cardinality bounded. |
 | `FRUGAL_PATH_BLOCK_LIST`                 | Comma-separated repo-relative paths to skip for caller attribution (see below).          |
 | `FRUGAL_METRIC_EXPORT_INTERVAL_MS`       | Export cadence in milliseconds. Default `60000`.                                         |
 | `FRUGAL_MAX_CALL_SITES`                  | Cardinality cap — max distinct `(caller_file, caller_function)` pairs to attribute. Default `500`. Set to `0` to disable. |
@@ -113,9 +114,9 @@ Every metric carries this base attribute set:
 
 - `frugal.project_id` — from your env vars (`frugal.customer_id` too when set); `frugal.component` is added on the per-component series when `FRUGAL_COMPONENTS` is set
 - `frugal.caller_file`, `frugal.caller_function` — auto-detected from the call stack
-- `gen_ai.system` — `openai`, `anthropic`, `aws.bedrock`, or `vertex_ai`
+- `gen_ai.system` + `gen_ai.provider.name` — the provider, dual-emitted under both the legacy and current OTel attribute (`openai`, `anthropic`, `aws.bedrock`, `vertex_ai`/`gcp.vertex_ai`, …, or `custom`). Set via the `system` kwarg; an unrecognized value is coerced to `custom` with a one-time warning. See [`docs/metrics-reference.md`](docs/metrics-reference.md#attributes).
 - `gen_ai.request.model`, `gen_ai.response.model` — the model used
-- `gen_ai.operation.name` — `chat`, `text_completion`, or `embeddings`
+- `gen_ai.operation.name` — `chat`, `text_completion`, `embeddings`, `generate_content`, `execute_tool`, `create_agent`, `invoke_agent`, or `other` (the sentinel an unrecognized operation coerces to)
 - `frugal.batched` — `true` for batch-API submissions, `false` otherwise
 
 The "Additional attributes" column above lists what each metric carries on top of the base set. See [`docs/metrics-reference.md`](docs/metrics-reference.md) for the values each attribute takes, sentinel values for caller fields, and the full internal-error-stage list.

{frugal_sdk_python-0.2.0 → frugal_sdk_python-0.3.0}/README.md

@@ -49,6 +49,7 @@ Set both or the wrapper silently returns the client unmodified:
 | `FRUGAL_OTEL_EXPORTER_OTLP_ENDPOINT`     | OTLP collector URL for Frugal metrics. Defaults to `https://metrics.frugal.co/opentelemetry/v1/metrics`. Falls back to `OTEL_EXPORTER_OTLP_ENDPOINT`. |
 | `FRUGAL_OTEL_EXPORTER_OTLP_HEADERS`      | Export auth headers (e.g. `Authorization=Bearer ...`). Overrides the header built from `FRUGAL_API_KEY`. Falls back to `OTEL_EXPORTER_OTLP_HEADERS`. |
 | `FRUGAL_COMPONENTS`                      | Comma-separated logical components (e.g. `ai-handler,reco`). When set, each metric is ALSO emitted once per component (tagged `frugal.component`) on top of the base project series, so usage can be viewed for the whole project or split by component. |
+| `FRUGAL_CUSTOM_PROVIDERS`                | Comma-separated allowlist of provider names beyond the built-in registry (e.g. `together_ai,fireworks`) that should pass through verbatim on `gen_ai.system` / `gen_ai.provider.name` instead of being coerced to `custom`. Capped at 10 (extras are coerced) to keep label cardinality bounded. |
 | `FRUGAL_PATH_BLOCK_LIST`                 | Comma-separated repo-relative paths to skip for caller attribution (see below).          |
 | `FRUGAL_METRIC_EXPORT_INTERVAL_MS`       | Export cadence in milliseconds. Default `60000`.                                         |
 | `FRUGAL_MAX_CALL_SITES`                  | Cardinality cap — max distinct `(caller_file, caller_function)` pairs to attribute. Default `500`. Set to `0` to disable. |
@@ -84,9 +85,9 @@ Every metric carries this base attribute set:
 
 - `frugal.project_id` — from your env vars (`frugal.customer_id` too when set); `frugal.component` is added on the per-component series when `FRUGAL_COMPONENTS` is set
 - `frugal.caller_file`, `frugal.caller_function` — auto-detected from the call stack
-- `gen_ai.system` — `openai`, `anthropic`, `aws.bedrock`, or `vertex_ai`
+- `gen_ai.system` + `gen_ai.provider.name` — the provider, dual-emitted under both the legacy and current OTel attribute (`openai`, `anthropic`, `aws.bedrock`, `vertex_ai`/`gcp.vertex_ai`, …, or `custom`). Set via the `system` kwarg; an unrecognized value is coerced to `custom` with a one-time warning. See [`docs/metrics-reference.md`](docs/metrics-reference.md#attributes).
 - `gen_ai.request.model`, `gen_ai.response.model` — the model used
-- `gen_ai.operation.name` — `chat`, `text_completion`, or `embeddings`
+- `gen_ai.operation.name` — `chat`, `text_completion`, `embeddings`, `generate_content`, `execute_tool`, `create_agent`, `invoke_agent`, or `other` (the sentinel an unrecognized operation coerces to)
 - `frugal.batched` — `true` for batch-API submissions, `false` otherwise
 
 The "Additional attributes" column above lists what each metric carries on top of the base set. See [`docs/metrics-reference.md`](docs/metrics-reference.md) for the values each attribute takes, sentinel values for caller fields, and the full internal-error-stage list.

{frugal_sdk_python-0.2.0 → frugal_sdk_python-0.3.0}/docs/metrics-reference.md

@@ -45,12 +45,31 @@ calls, token usage) are 100% — never sampled.
 
 | Attribute                  | Example                                    |
 | -------------------------- | ------------------------------------------ |
-| `gen_ai.system`            | `openai` \| `anthropic` \| `aws.bedrock` \| `vertex_ai` \| `custom` |
-| `gen_ai.operation.name`    | `chat` \| `text_completion` \| `embeddings` |
+| `gen_ai.system`            | `openai` \| `anthropic` \| `aws.bedrock` \| `vertex_ai` \| `az.ai.openai` \| `cohere` \| `mistral_ai` \| `deepseek` \| `groq` \| `perplexity` \| `x_ai` \| `ibm.watsonx.ai` \| `gcp.gemini` \| `custom` (legacy spelling — see below) |
+| `gen_ai.provider.name`     | `openai` \| `anthropic` \| `aws.bedrock` \| `gcp.vertex_ai` \| `azure.ai.openai` \| `cohere` \| `mistral_ai` \| `deepseek` \| `groq` \| `perplexity` \| `x_ai` \| `ibm.watsonx.ai` \| `gcp.gemini` \| `custom` (current OTel attribute) |
+| `gen_ai.operation.name`    | `chat` \| `text_completion` \| `embeddings` \| `generate_content` \| `execute_tool` \| `create_agent` \| `invoke_agent` \| `other` (sentinel for an unrecognized operation) |
 | `gen_ai.request.model`     | `gpt-4o` / `claude-3-5-sonnet-latest` / …   |
 | `gen_ai.response.model`    | Provider-returned model id (often identical)|
 | `gen_ai.token.type`        | `input` \| `output` (token usage metric only) |
 
+> **Provider attribute migration.** OTel renamed `gen_ai.system` to
+> `gen_ai.provider.name` (some spellings changed, e.g. `vertex_ai` →
+> `gcp.vertex_ai`). The SDK **dual-emits both** during the transition: every
+> metric carries the legacy `gen_ai.system` value (so existing queries/history
+> keep matching) and the new `gen_ai.provider.name` value. The `system` kwarg
+> accepts either spelling. A value outside the well-known set (e.g. a typo like
+> `amazon-bedrock`) is **coerced to `custom`** on both labels — so a wrong label
+> never reaches metrics — and surfaced once through the debug callback
+> (`set_debug_callback`), per distinct value (bounded so a high-cardinality
+> caller can't grow the warning set without limit). To keep a legitimate provider
+> that isn't in the registry, register it via `FRUGAL_CUSTOM_PROVIDERS` (up to 10)
+> and it passes through verbatim on both labels. An `operation` outside the accepted set is coerced to
+> the `other` sentinel (NOT `chat`, so a typo can't inflate the real chat series) with the same
+> one-time warning — the seven listed values plus an explicit `other` pass through unchanged. The
+> exported `GenAiSystem` / `GenAiOperation` `Literal` aliases give strict typing for callers who want
+> it — `GenAiSystem` lists both provider spellings the runtime accepts (the kwargs stay `str` so a
+> free-form custom value remains legal).
+
 ### Frugal-added
 
 | Attribute                 | Value                                                      |

{frugal_sdk_python-0.2.0 → frugal_sdk_python-0.3.0}/docs/quickstart.md

@@ -92,6 +92,13 @@ token counts are not (the OpenAI SDK doesn't send them).
 Use `wrap_anthropic` with the Anthropic SDK's `AnthropicBedrock` client. The
 wrapper detects it and tags metrics with `gen_ai.system = aws.bedrock`.
 
+> Every metric is **dual-emitted** under both `gen_ai.system` (legacy) and
+> `gen_ai.provider.name` (current OTel spelling — e.g. `aws.bedrock`,
+> `gcp.vertex_ai`). A provider or operation outside the known set is coerced to
+> the `custom` / `other` sentinel with a one-time warning (via
+> `set_debug_callback`); register extra providers with `FRUGAL_CUSTOM_PROVIDERS`.
+> See [metrics-reference.md](metrics-reference.md#attributes).
+
 ```python
 from anthropic import AnthropicBedrock
 from frugal_metrics import wrap_anthropic

{frugal_sdk_python-0.2.0 → frugal_sdk_python-0.3.0}/src/frugal_metrics/__init__.py

@@ -3,6 +3,14 @@
 from importlib.metadata import PackageNotFoundError, version
 
 from frugal_metrics.debug import set_debug_callback
+from frugal_metrics.providers import (
+    PROVIDERS,
+    GenAiOperation,
+    GenAiSystem,
+    ProviderEntry,
+    normalize_operation,
+    normalize_provider,
+)
 from frugal_metrics.shutdown import enable_auto_shutdown, flush, shutdown
 from frugal_metrics.track import AICallSpan, start_ai_call, track_ai_call
 from frugal_metrics.wrappers.openai import wrap_openai
@@ -20,6 +28,12 @@ __all__ = [
     "flush",
     "shutdown",
     "enable_auto_shutdown",
+    "PROVIDERS",
+    "ProviderEntry",
+    "GenAiSystem",
+    "GenAiOperation",
+    "normalize_provider",
+    "normalize_operation",
 ]
 try:
     __version__ = version("frugal-sdk-python")

{frugal_sdk_python-0.2.0 → frugal_sdk_python-0.3.0}/src/frugal_metrics/config.py

@@ -35,7 +35,7 @@ import os
 import threading
 from dataclasses import dataclass, field
 
-from frugal_metrics.debug import emit_log
+from frugal_metrics.debug import emit_log, emit_log_once
 
 logger = logging.getLogger("frugal_metrics")
 
@@ -43,6 +43,7 @@ _ENV_API_KEY = "FRUGAL_API_KEY"
 _ENV_CUSTOMER_ID = "FRUGAL_CUSTOMER_ID"
 _ENV_PROJECT_ID = "FRUGAL_PROJECT_ID"
 _ENV_COMPONENTS = "FRUGAL_COMPONENTS"
+_ENV_CUSTOM_PROVIDERS = "FRUGAL_CUSTOM_PROVIDERS"
 _ENV_REPO_ROOT = "FRUGAL_REPO_ROOT"
 _ENV_PATH_BLOCK_LIST = "FRUGAL_PATH_BLOCK_LIST"
 _ENV_DISABLED = "FRUGAL_DISABLED"
@@ -58,6 +59,8 @@ _ENV_PROMPT_ANALYSIS_SAMPLE_RATE = "FRUGAL_PROMPT_ANALYSIS_SAMPLE_RATE"
 _DEFAULT_OTLP_ENDPOINT = "https://metrics.frugal.co/opentelemetry/v1/metrics"
 _DEFAULT_MAX_CALL_SITES = 500
 _DEFAULT_PROMPT_ANALYSIS_SAMPLE_RATE = 0.1
+# Upper bound on customer-registered provider names — keeps label cardinality bounded.
+_MAX_CUSTOM_PROVIDERS = 10
 
 _warned_once = False
 _warned_lock = threading.Lock()
@@ -72,6 +75,10 @@ class Config:
     repo_root: str
     # Optional logical components; metrics fan out one extra series per entry.
     components: tuple[str, ...] = field(default_factory=tuple)
+    # Customer-registered provider names (beyond the built-in OTel registry) that
+    # pass through verbatim instead of being coerced to "custom". Capped to keep
+    # gen_ai.system / gen_ai.provider.name cardinality bounded.
+    custom_providers: tuple[str, ...] = field(default_factory=tuple)
     path_block_list: tuple[str, ...] = field(default_factory=tuple)
     otlp_endpoint: str = _DEFAULT_OTLP_ENDPOINT
     otlp_headers: str | None = None
@@ -157,6 +164,28 @@ def load_config() -> Config | None:
         entry.strip() for entry in components_raw.split(",") if entry.strip()
     )
 
+    # Optional comma-separated allowlist of custom provider names that pass
+    # through verbatim (e.g. "together_ai,fireworks"). Deduped (order-preserving)
+    # and capped at _MAX_CUSTOM_PROVIDERS so a misconfigured value can't blow up
+    # label cardinality.
+    custom_providers_raw = os.environ.get(_ENV_CUSTOM_PROVIDERS, "")
+    custom_providers_all = list(
+        dict.fromkeys(
+            entry.strip()
+            for entry in custom_providers_raw.split(",")
+            if entry.strip()
+        )
+    )
+    custom_providers = tuple(custom_providers_all[:_MAX_CUSTOM_PROVIDERS])
+    if len(custom_providers_all) > _MAX_CUSTOM_PROVIDERS:
+        emit_log_once(
+            "custom_providers_truncated",
+            "warn",
+            f"{_ENV_CUSTOM_PROVIDERS} lists {len(custom_providers_all)} entries; "
+            f"only the first {_MAX_CUSTOM_PROVIDERS} are honored, the rest are "
+            f'coerced to "custom".',
+        )
+
     otlp_endpoint = (
         os.environ.get(_ENV_FRUGAL_OTLP_ENDPOINT)
         or os.environ.get(_ENV_OTLP_ENDPOINT)
@@ -185,6 +214,7 @@ def load_config() -> Config | None:
         project_id=project_id,  # type: ignore[arg-type]
         repo_root=repo_root_abs,
         components=components,
+        custom_providers=custom_providers,
         path_block_list=block_list,
         otlp_endpoint=otlp_endpoint,
         otlp_headers=otlp_headers,

{frugal_sdk_python-0.2.0 → frugal_sdk_python-0.3.0}/src/frugal_metrics/instrument.py

@@ -23,6 +23,7 @@ from frugal_metrics.caller import (
 from frugal_metrics.config import Config
 from frugal_metrics.debug import emit_log_once
 from frugal_metrics.otel import Instruments, get_instruments
+from frugal_metrics.providers import normalize_operation, normalize_provider
 from frugal_metrics.prompt_analyzer import (
     RequestPayload,
     analyze_request,
@@ -56,9 +57,14 @@ class UsageReport:
 
 
 def _base_attrs(config: Config, ctx: CallContext, caller: CallerInfo) -> dict[str, Any]:
+    # Normalize the constrained-vocabulary labels at the single chokepoint every
+    # metric flows through: coerce typos to the sentinel and dual-emit both the
+    # legacy gen_ai.system value and the new gen_ai.provider.name value.
+    provider = normalize_provider(ctx.system, config.custom_providers)
     attrs: dict[str, Any] = {
-        semconv.GEN_AI_SYSTEM: ctx.system,
-        semconv.GEN_AI_OPERATION_NAME: ctx.operation,
+        semconv.GEN_AI_SYSTEM: provider.system,
+        semconv.GEN_AI_PROVIDER_NAME: provider.provider_name,
+        semconv.GEN_AI_OPERATION_NAME: normalize_operation(ctx.operation),
         semconv.GEN_AI_REQUEST_MODEL: ctx.request_model,
         semconv.FRUGAL_PROJECT_ID: config.project_id,
         semconv.FRUGAL_CALLER_FILE: caller.file,

frugal_sdk_python-0.3.0/src/frugal_metrics/providers.py

@@ -0,0 +1,219 @@
+"""Canonical provider registry + sentinel normalization.
+
+``gen_ai.system`` is the OpenTelemetry GenAI semantic-convention attribute that
+identifies the provider/platform a model runs on. OTel has since renamed it to
+``gen_ai.provider.name``, with a few new value spellings (e.g. ``vertex_ai`` ->
+``gcp.vertex_ai``). We dual-emit both attributes: the legacy ``gen_ai.system``
+value (unchanged for the providers we already supported, so existing queries and
+historical series keep matching) plus the new ``gen_ai.provider.name`` value.
+Consumers migrate to the new attribute at their own pace; the old one is dropped
+afterward.
+
+Any value outside the well-known set is a bug (a typo like ``amazon-bedrock``
+silently produces a series no query ever matches). Such values are coerced to the
+``custom`` sentinel -- a wrong label never reaches metrics -- and surfaced once
+via the debug channel. Everything here is pure and never raises: the SDK is safe
+by construction.
+"""
+
+from __future__ import annotations
+
+import threading
+from typing import Literal, NamedTuple
+
+from frugal_metrics import semconv
+from frugal_metrics.debug import emit_log
+
+
+class ProviderEntry(NamedTuple):
+    """A provider's legacy and current attribute spellings."""
+
+    system: str  # legacy gen_ai.system value (kept for back-compat)
+    provider_name: str  # new gen_ai.provider.name value (current OTel spelling)
+
+
+# Single source of truth. ``system`` preserves the spelling Frugal already
+# emitted for the providers it historically supported (so existing dashboards /
+# history keep matching); ``provider_name`` is the current OTel spelling. For
+# providers we never emitted before, both columns are the same OTel string.
+PROVIDERS: tuple[ProviderEntry, ...] = (
+    ProviderEntry("openai", "openai"),
+    ProviderEntry("anthropic", "anthropic"),
+    ProviderEntry("aws.bedrock", "aws.bedrock"),
+    ProviderEntry("vertex_ai", "gcp.vertex_ai"),
+    ProviderEntry("az.ai.openai", "azure.ai.openai"),
+    ProviderEntry("cohere", "cohere"),
+    ProviderEntry("mistral_ai", "mistral_ai"),
+    ProviderEntry("deepseek", "deepseek"),
+    ProviderEntry("groq", "groq"),
+    ProviderEntry("perplexity", "perplexity"),
+    ProviderEntry("x_ai", "x_ai"),
+    ProviderEntry("ibm.watsonx.ai", "ibm.watsonx.ai"),
+    ProviderEntry("gcp.gemini", "gcp.gemini"),
+)
+
+# Accepted provider values. Exported for callers who want strict typing; the
+# public params stay ``str`` so a free-form custom value remains legal
+# (``Literal[...] | str`` collapses to ``str`` for type checkers, so it cannot
+# give both autocomplete *and* free-form). Runtime coercion is the real
+# enforcement. Lists BOTH spellings (legacy ``system`` and new
+# ``provider_name``) since ``normalize_provider`` accepts either — a strict-typed
+# caller must be able to annotate with the new ``gcp.vertex_ai`` form. The
+# test ``test_genai_system_literal_matches_registry`` guards this against drift.
+GenAiSystem = Literal[
+    "openai",
+    "anthropic",
+    "aws.bedrock",
+    "vertex_ai",
+    "gcp.vertex_ai",
+    "az.ai.openai",
+    "azure.ai.openai",
+    "cohere",
+    "mistral_ai",
+    "deepseek",
+    "groq",
+    "perplexity",
+    "x_ai",
+    "ibm.watsonx.ai",
+    "gcp.gemini",
+    "custom",
+]
+
+# Must enumerate the same values as ``_OPERATIONS`` below (a ``Literal`` can't be
+# derived from a runtime ``frozenset``). Keep the two in sync.
+# Lists the accepted operations plus the ``other`` coercion sentinel. Guarded
+# against drift from ``_OPERATIONS_ORDERED`` by
+# ``test_genai_operation_literal_matches_runtime``.
+GenAiOperation = Literal[
+    "chat",
+    "text_completion",
+    "embeddings",
+    "generate_content",
+    "execute_tool",
+    "create_agent",
+    "invoke_agent",
+    "other",
+]
+
+# Operations accepted without warning: the three defaults plus other OTel-defined
+# values. Ordered to match the Node SDK's OPERATIONS tuple so both SDKs emit an
+# identical "use one of" diagnostic; the frozenset is derived for O(1) membership.
+# Single source of truth for ``GenAiOperation`` above too.
+_OPERATIONS_ORDERED: tuple[str, ...] = (
+    "chat",
+    "text_completion",
+    "embeddings",
+    "generate_content",
+    "execute_tool",
+    "create_agent",
+    "invoke_agent",
+)
+_OPERATIONS: frozenset[str] = frozenset(_OPERATIONS_ORDERED)
+
+_CUSTOM = ProviderEntry(semconv.SYSTEM_CUSTOM, semconv.SYSTEM_CUSTOM)
+
+_BY_VALUE: dict[str, ProviderEntry] = {}
+for _entry in PROVIDERS:
+    _BY_VALUE[_entry.system] = _entry
+    _BY_VALUE[_entry.provider_name] = _entry
+
+
+# Bounded warn-once: surface each distinct invalid provider/operation value once,
+# but cap how many distinct values we track so a high-cardinality caller (e.g. a
+# gateway labeling calls from request data) can't grow this set without bound.
+# Past the cap, further unknown values are coerced silently after a single
+# generic notice. We do our own de-dup here (emit_log, not emit_log_once) so the
+# cap is enforced in one place -- and so every distinct typo is surfaced, not just
+# the first one.
+_MAX_TRACKED_INVALID = 32
+_warned_invalid: set[str] = set()
+_cap_notice_emitted = False
+# A metrics SDK is called from concurrent request threads, so the check-then-act
+# on the shared warn state must be synchronized (mirrors debug._lock /
+# config._warned_lock) or the same typo could warn twice and the set could grow
+# past the cap.
+_warned_lock = threading.Lock()
+
+_CAP_NOTICE = (
+    f"frugal-metrics: {_MAX_TRACKED_INVALID}+ distinct invalid provider/operation "
+    "values seen; further unknown values are coerced to the sentinel without an "
+    "individual warning."
+)
+
+
+def _warn_invalid_once(key: str, message: str) -> None:
+    global _cap_notice_emitted
+    # Decide what (if anything) to emit while holding the lock; emit outside it so
+    # a slow debug callback can't serialize other threads' normalization.
+    emit: str | None = None
+    with _warned_lock:
+        if key in _warned_invalid:
+            return
+        if len(_warned_invalid) >= _MAX_TRACKED_INVALID:
+            if _cap_notice_emitted:
+                return
+            _cap_notice_emitted = True
+            emit = _CAP_NOTICE
+        else:
+            _warned_invalid.add(key)
+            emit = message
+    emit_log("warn", emit)
+
+
+def reset_provider_warnings_for_tests() -> None:
+    """Test-only -- clear the bounded invalid-value warning state."""
+    global _cap_notice_emitted
+    with _warned_lock:
+        _warned_invalid.clear()
+        _cap_notice_emitted = False
+
+
+def normalize_provider(
+    value: str | None, custom_providers: tuple[str, ...] = ()
+) -> ProviderEntry:
+    """Resolve a caller-supplied provider string to its canonical pair.
+
+    Accepts either spelling (legacy ``gen_ai.system`` or new
+    ``gen_ai.provider.name``). A value the customer registered via
+    ``custom_providers`` passes through verbatim (system == provider_name). Any
+    other unknown value is coerced to the ``custom`` sentinel and reported
+    (bounded warn-once). ``None`` resolves to ``custom`` silently (the intended
+    default). Never raises.
+    """
+    if value is None or value == semconv.SYSTEM_CUSTOM:
+        return _CUSTOM
+    entry = _BY_VALUE.get(value)
+    if entry is not None:
+        return entry
+    # Customer-registered passthrough: no OTel mapping, so both labels carry the
+    # raw name. The set is capped at config-load time, so cardinality stays bounded.
+    if value in custom_providers:
+        return ProviderEntry(value, value)
+    _warn_invalid_once(
+        f"provider:{value}",
+        f'unknown gen_ai provider "{value}" -- coerced to "custom". Use a '
+        'well-known OpenTelemetry GenAI provider value (e.g. "aws.bedrock", '
+        '"vertex_ai", "anthropic", "openai"), or register it via '
+        "FRUGAL_CUSTOM_PROVIDERS.",
+    )
+    return _CUSTOM
+
+
+def normalize_operation(value: str | None) -> str:
+    """Resolve a caller-supplied operation string.
+
+    Unknown values are coerced to the ``other`` sentinel -- NOT ``chat`` -- so a
+    typo lands in a distinguishable bucket instead of inflating the real chat
+    series, and reported (bounded warn-once). ``None`` resolves to ``chat`` (the
+    default); an explicit ``other`` passes through silently. Never raises.
+    """
+    if value is None:
+        return semconv.OP_CHAT
+    if value == semconv.OP_OTHER or value in _OPERATIONS:
+        return value
+    _warn_invalid_once(
+        f"operation:{value}",
+        f'unknown gen_ai.operation.name "{value}" -- coerced to "other". Use one '
+        f'of: {", ".join(_OPERATIONS_ORDERED)}.',
+    )
+    return semconv.OP_OTHER

{frugal_sdk_python-0.2.0 → frugal_sdk_python-0.3.0}/src/frugal_metrics/semconv.py

@@ -2,12 +2,18 @@
 
 # GenAI semconv attribute names (stable as of semconv v1.28+).
 GEN_AI_SYSTEM = "gen_ai.system"
+# OTel renamed gen_ai.system -> gen_ai.provider.name; we dual-emit both during
+# the transition. See providers.py for the value mapping.
+GEN_AI_PROVIDER_NAME = "gen_ai.provider.name"
 GEN_AI_REQUEST_MODEL = "gen_ai.request.model"
 GEN_AI_RESPONSE_MODEL = "gen_ai.response.model"
 GEN_AI_OPERATION_NAME = "gen_ai.operation.name"
 GEN_AI_TOKEN_TYPE = "gen_ai.token.type"
 
-# GenAI semconv system identifiers.
+# Legacy gen_ai.system spellings exposed as named constants for the wrapper
+# modules. The full cross-provider registry (cohere, mistral_ai, deepseek,
+# groq, …) lives solely in the PROVIDERS table in providers.py — the single
+# source of truth — and those values are not re-declared here.
 SYSTEM_OPENAI = "openai"
 SYSTEM_ANTHROPIC = "anthropic"
 SYSTEM_AZURE_OPENAI = "az.ai.openai"
@@ -21,6 +27,9 @@ SYSTEM_CUSTOM = "custom"
 OP_CHAT = "chat"
 OP_TEXT_COMPLETION = "text_completion"
 OP_EMBEDDINGS = "embeddings"
+# Sentinel for an operation outside the accepted set — parallels SYSTEM_CUSTOM so
+# a typo lands in a distinguishable bucket instead of corrupting the real chat series.
+OP_OTHER = "other"
 
 # GenAI semconv token types.
 TOKEN_INPUT = "input"

{frugal_sdk_python-0.2.0 → frugal_sdk_python-0.3.0}/tests/conftest.py

@@ -21,6 +21,7 @@ from frugal_metrics.block_hash_lru import (
 from frugal_metrics.caller import reset_caches_for_tests as reset_caller_caches
 from frugal_metrics.debug import reset_debug_for_tests
 from frugal_metrics.otel import Instruments
+from frugal_metrics.providers import reset_provider_warnings_for_tests
 
 
 @dataclass
@@ -80,6 +81,7 @@ def required_env(monkeypatch, tmp_path) -> str:
     reset_caller_caches()
     reset_shared_block_hash_lru_for_tests()
     reset_debug_for_tests()
+    reset_provider_warnings_for_tests()
     return repo_root
 
 

{frugal_sdk_python-0.2.0 → frugal_sdk_python-0.3.0}/tests/test_config.py

@@ -73,6 +73,24 @@ def test_parses_components(monkeypatch):
     assert cfg.components == ("ai-handler", "reco", "trailing")
 
 
+def test_parses_dedupes_and_caps_custom_providers(monkeypatch):
+    monkeypatch.setenv("FRUGAL_PROJECT_ID", "p")
+    monkeypatch.setenv("FRUGAL_OTEL_EXPORTER_OTLP_HEADERS", "Authorization=Bearer x")
+    monkeypatch.setenv("FRUGAL_API_KEY", "sk-test")
+    monkeypatch.setenv("FRUGAL_REPO_ROOT", "/app")
+    monkeypatch.setenv("FRUGAL_CUSTOM_PROVIDERS", "a, b ,, a")  # dedupe + drop empties
+    cfg = config_module.load_config()
+    assert cfg is not None
+    assert cfg.custom_providers == ("a", "b")
+    # More than 10 distinct -> capped to the first 10.
+    monkeypatch.setenv(
+        "FRUGAL_CUSTOM_PROVIDERS", ",".join(f"p{i}" for i in range(13))
+    )
+    cfg = config_module.load_config()
+    assert cfg is not None
+    assert len(cfg.custom_providers) == 10
+
+
 def test_parses_block_list(monkeypatch):
     monkeypatch.setenv("FRUGAL_CUSTOMER_ID", "c")
     monkeypatch.setenv("FRUGAL_PROJECT_ID", "p")

frugal_sdk_python-0.3.0/tests/test_providers.py

@@ -0,0 +1,180 @@
+"""Provider/operation sentinel normalization.
+
+Dual-emit of gen_ai.system + gen_ai.provider.name, coerce-to-custom on typos,
+and de-duped warnings. Mirrors the Node providers.test.ts coverage.
+"""
+
+from __future__ import annotations
+
+import pytest
+
+from typing import get_args
+
+from frugal_metrics import semconv
+from frugal_metrics.debug import set_debug_callback
+from frugal_metrics.providers import (
+    _OPERATIONS_ORDERED,
+    PROVIDERS,
+    GenAiOperation,
+    GenAiSystem,
+    ProviderEntry,
+    normalize_operation,
+    normalize_provider,
+)
+from frugal_metrics.track import track_ai_call
+
+
+def _by_name(points, name):
+    return [p for p in points if p["name"] == name]
+
+
+# --- unit ---------------------------------------------------------------------
+
+
+def test_known_legacy_value_maps_to_both_spellings():
+    assert normalize_provider("vertex_ai") == ProviderEntry("vertex_ai", "gcp.vertex_ai")
+    assert normalize_provider("aws.bedrock") == ProviderEntry("aws.bedrock", "aws.bedrock")
+
+
+def test_accepts_new_provider_name_spelling():
+    assert normalize_provider("gcp.vertex_ai") == ProviderEntry(
+        "vertex_ai", "gcp.vertex_ai"
+    )
+
+
+def test_azure_dual_spelling_maps_both_ways():
+    # az.ai.openai is the only provider besides vertex_ai where the legacy and
+    # new spellings differ; both must resolve to the same canonical pair.
+    expected = ProviderEntry("az.ai.openai", "azure.ai.openai")
+    assert normalize_provider("az.ai.openai") == expected
+    assert normalize_provider("azure.ai.openai") == expected
+
+
+def test_unknown_provider_coerces_to_custom():
+    assert normalize_provider("amazon-bedrock") == ProviderEntry("custom", "custom")
+
+
+def test_none_and_custom_resolve_silently():
+    assert normalize_provider(None) == ProviderEntry("custom", "custom")
+    assert normalize_provider("custom") == ProviderEntry("custom", "custom")
+
+
+def test_operation_coercion():
+    assert normalize_operation("embeddings") == "embeddings"
+    # Unknown -> 'other' sentinel (a distinguishable bucket), not chat.
+    assert normalize_operation("bogus") == "other"
+    # Explicit 'other' passes through; absent defaults to chat.
+    assert normalize_operation("other") == "other"
+    assert normalize_operation(None) == "chat"
+
+
+def test_genai_system_literal_matches_registry():
+    # Guard against drift: the exported Literal must list exactly what
+    # normalize_provider accepts (both spellings of every registry entry + custom).
+    expected = (
+        {p.system for p in PROVIDERS}
+        | {p.provider_name for p in PROVIDERS}
+        | {"custom"}
+    )
+    assert set(get_args(GenAiSystem)) == expected
+
+
+def test_genai_operation_literal_matches_runtime():
+    # Guard against drift: the accepted operations plus the 'other' sentinel.
+    assert set(get_args(GenAiOperation)) == set(_OPERATIONS_ORDERED) | {"other"}
+
+
+def test_non_default_operations_pass_through():
+    # OTel-defined operations beyond the default trio are accepted unchanged
+    # (not coerced to chat).
+    for op in ("generate_content", "execute_tool", "create_agent", "invoke_agent"):
+        assert normalize_operation(op) == op
+
+
+# --- integration through track_ai_call ----------------------------------------
+
+
+@pytest.fixture
+def warnings(required_env):
+    captured_warnings: list[tuple[str, str]] = []
+    set_debug_callback(lambda level, message, context=None: captured_warnings.append((level, message)))
+    yield captured_warnings
+    set_debug_callback(None)
+
+
+def test_dual_emits_system_and_provider_name(captured, warnings):
+    with track_ai_call(system="vertex_ai") as span:
+        span.set_usage(input_tokens=1)
+    attrs = _by_name(captured.collect(), semconv.METRIC_CALLS)[0]["attributes"]
+    assert attrs[semconv.GEN_AI_SYSTEM] == "vertex_ai"
+    assert attrs[semconv.GEN_AI_PROVIDER_NAME] == "gcp.vertex_ai"
+    assert warnings == []
+
+
+def test_typo_coerces_to_custom_and_warns_once(captured, warnings):
+    for _ in range(2):
+        with track_ai_call(system="amazon-bedrock") as span:
+            span.set_usage(input_tokens=1)
+    points = _by_name(captured.collect(), semconv.METRIC_CALLS)
+    for p in points:
+        assert p["attributes"][semconv.GEN_AI_SYSTEM] == "custom"
+        assert p["attributes"][semconv.GEN_AI_PROVIDER_NAME] == "custom"
+    provider_warnings = [w for w in warnings if "amazon-bedrock" in w[1]]
+    assert len(provider_warnings) == 1
+    assert provider_warnings[0][0] == "warn"
+
+
+def test_unknown_operation_coerces_to_other_and_warns_once(captured, warnings):
+    for _ in range(2):
+        with track_ai_call(system="openai", operation="summarize") as span:
+            span.set_usage(input_tokens=1)
+    attrs = _by_name(captured.collect(), semconv.METRIC_CALLS)[0]["attributes"]
+    assert attrs[semconv.GEN_AI_OPERATION_NAME] == "other"
+    # Coercion runs on both calls, but the operation warning de-dupes.
+    assert len([w for w in warnings if "summarize" in w[1]]) == 1
+
+
+# --- custom providers (FRUGAL_CUSTOM_PROVIDERS) -------------------------------
+
+
+def test_registered_custom_provider_passes_through_unit():
+    assert normalize_provider("together_ai", ("together_ai",)) == ProviderEntry(
+        "together_ai", "together_ai"
+    )
+    # Same value, but not registered -> coerced to custom.
+    assert normalize_provider("together_ai", ()) == ProviderEntry("custom", "custom")
+
+
+def test_custom_provider_emits_verbatim_through_track(captured, warnings, monkeypatch):
+    monkeypatch.setenv("FRUGAL_CUSTOM_PROVIDERS", "together_ai,fireworks")
+    with track_ai_call(system="together_ai") as span:
+        span.set_usage(input_tokens=1)
+    attrs = _by_name(captured.collect(), semconv.METRIC_CALLS)[0]["attributes"]
+    assert attrs[semconv.GEN_AI_SYSTEM] == "together_ai"
+    assert attrs[semconv.GEN_AI_PROVIDER_NAME] == "together_ai"
+    assert warnings == []
+
+
+def test_unregistered_provider_still_coerces(captured, warnings, monkeypatch):
+    monkeypatch.setenv("FRUGAL_CUSTOM_PROVIDERS", "together_ai")
+    with track_ai_call(system="ai21") as span:
+        span.set_usage(input_tokens=1)
+    attrs = _by_name(captured.collect(), semconv.METRIC_CALLS)[0]["attributes"]
+    assert attrs[semconv.GEN_AI_SYSTEM] == "custom"
+    assert len([w for w in warnings if "ai21" in w[1]]) == 1
+
+
+# --- bounded invalid-value warnings ------------------------------------------
+
+
+def test_warns_once_per_distinct_invalid_provider(required_env):
+    # The old fixed-key dedup surfaced only the first invalid value process-wide;
+    # each DISTINCT value must now warn (bounded by a cap).
+    msgs: list[str] = []
+    set_debug_callback(lambda level, message, context=None: msgs.append(message))
+    normalize_provider("amazon-bedrock")
+    normalize_provider("amazon-bedrock")  # duplicate -> no second warning
+    normalize_provider("gpt-typo")  # distinct -> must still warn
+    set_debug_callback(None)
+    assert len([m for m in msgs if "amazon-bedrock" in m]) == 1
+    assert len([m for m in msgs if "gpt-typo" in m]) == 1