frugal-sdk-python 0.3.0__tar.gz → 0.4.0__tar.gz

{frugal_sdk_python-0.3.0 → frugal_sdk_python-0.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.4
+Metadata-Version: 2.5
 Name: frugal-sdk-python
-Version: 0.3.0
+Version: 0.4.0
 Summary: Instrumentation for Cost-to-Code attribution.
 Project-URL: Homepage, https://frugal.co
 Author: Frugal AI Inc.
@@ -79,10 +79,16 @@ Set both or the wrapper silently returns the client unmodified:
 | `FRUGAL_OTEL_EXPORTER_OTLP_HEADERS`      | Export auth headers (e.g. `Authorization=Bearer ...`). Overrides the header built from `FRUGAL_API_KEY`. Falls back to `OTEL_EXPORTER_OTLP_HEADERS`. |
 | `FRUGAL_COMPONENTS`                      | Comma-separated logical components (e.g. `ai-handler,reco`). When set, each metric is ALSO emitted once per component (tagged `frugal.component`) on top of the base project series, so usage can be viewed for the whole project or split by component. |
 | `FRUGAL_CUSTOM_PROVIDERS`                | Comma-separated allowlist of provider names beyond the built-in registry (e.g. `together_ai,fireworks`) that should pass through verbatim on `gen_ai.system` / `gen_ai.provider.name` instead of being coerced to `custom`. Capped at 10 (extras are coerced) to keep label cardinality bounded. |
+| `FRUGAL_CUSTOM_LABEL`                   | Comma-separated allowlist of values for the `frugal.custom_label` label (e.g. `billing,search`). Applies only to `track_ai_call` / `start_ai_call` calls that pass `custom_label=...`; a value outside the first 20 entries (hard cap, not configurable) is coerced to `other`. |
 | `FRUGAL_PATH_BLOCK_LIST`                 | Comma-separated repo-relative paths to skip for caller attribution (see below).          |
 | `FRUGAL_METRIC_EXPORT_INTERVAL_MS`       | Export cadence in milliseconds. Default `60000`.                                         |
 | `FRUGAL_MAX_CALL_SITES`                  | Cardinality cap — max distinct `(caller_file, caller_function)` pairs to attribute. Default `500`. Set to `0` to disable. |
 | `FRUGAL_PROMPT_ANALYSIS_SAMPLE_RATE`     | Fraction of calls (0..1) that run the lightweight prompt analyzer. Default `0.1`. Set to `0` to disable, `1` for every call. |
+| `FRUGAL_DUPLICATE_REQUEST_DETECTION`     | Duplicate-request detection (the `frugal.gen_ai.duplicate_requests` metric) is **on by default** and hashes each chat request in-process (the hash never leaves the SDK). Set to `0`/`false`/`no`/`off` to disable — no hashing, no filter memory, no metric. |
+| `FRUGAL_DUPLICATE_WINDOW_SECONDS`        | How long a request signature is remembered for duplicate detection. Default `3600`, max `604800` (7 days). |
+| `FRUGAL_DUPLICATE_CAPACITY`              | Expected distinct request signatures per window (sizes the in-memory filter). Default `10000`, max `1000000`. Raise it if the SDK warns the filter is saturated. |
+| `FRUGAL_DUPLICATE_FP_RATE`               | Target false-positive rate of the duplicate filter. Default `0.01`, clamped to `[1e-6, 0.5]`. |
+| `FRUGAL_DUPLICATE_GENERATIONS`           | Rotation granularity of the duplicate filter's time window. Default `2`, max `16`.        |
 | `FRUGAL_DISABLED`                        | Set to `1` to disable all instrumentation without uninstalling.                          |
 
 ## What gets captured
@@ -92,32 +98,38 @@ Every wrapped call emits OTel metrics tagged with the caller's source file and f
 | Metric                                  | Kind      | Description                                                  | Additional attributes                  |
 | --------------------------------------- | --------- | ------------------------------------------------------------ | -------------------------------------- |
 | `gen_ai.client.operation.duration`      | histogram | Call latency in seconds                                      | —                                      |
-| `gen_ai.client.token.usage`             | histogram | Input and output token counts                                | `gen_ai.token.type`                    |
-| `frugal.gen_ai.calls`                   | counter   | Number of calls per site                                     | —                                      |
+| `gen_ai.client.token.usage`             | histogram | Input and output token counts                                | `gen_ai.token.type`, `frugal.history_bucket` |
+| `frugal.gen_ai.calls`                   | counter   | Number of calls per site                                     | `frugal.history_bucket`, `frugal.structured_output_mode` |
 | `frugal.gen_ai.cache_read_tokens`       | histogram | Prompt cache hits                                            | —                                      |
 | `frugal.gen_ai.cache_write_tokens`      | histogram | Prompt cache writes                                          | —                                      |
 | `frugal.gen_ai.reasoning_tokens`        | histogram | OpenAI reasoning tokens (o-series), Anthropic thinking tokens| —                                      |
 | `frugal.gen_ai.errors`                  | counter   | Errors by exception type                                     | `frugal.error.type`                    |
 | `frugal.gen_ai.internal_errors`         | counter   | frugal-metrics internal failures (cardinality cap, bugs)     | `frugal.internal_error.stage`          |
 | `frugal.gen_ai.output_cap_utilization`  | histogram | `output_tokens / max_tokens` ratio when `max_tokens` is set  | `frugal.structured_output_mode`        |
-| `frugal.gen_ai.history_depth`           | histogram | `messages.length` (analyzer-sampled)                         | —                                      |
+| `frugal.gen_ai.history_depth`           | histogram | `messages.length` — every chat call (100%, not sampled)      | —                                      |
+| `frugal.gen_ai.duplicate_requests`      | counter   | Repeat-request detection — every chat call (100%, not sampled; disable via `FRUGAL_DUPLICATE_REQUEST_DETECTION`) | `frugal.dup_state` |
 | `frugal.gen_ai.few_shot_count`          | histogram | Few-shot exemplar pairs before final user (analyzer-sampled) | —                                      |
+| `frugal.gen_ai.few_shot_tokens`         | histogram | Approx token size of the few-shot example block (analyzer-sampled) | —                                |
 | `frugal.gen_ai.noise_ratio`             | histogram | HTML/base64/whitespace fraction of input (analyzer-sampled)  | —                                      |
 | `frugal.gen_ai.cache_prefix_stable`     | counter   | Per-block prompt-cache stability (analyzer-sampled)          | `frugal.block`, `frugal.stable`        |
 | `frugal.gen_ai.structured_output_hint`  | counter   | "Return JSON" prompts missing `response_format`/`tools`      | —                                      |
 
-The bottom block of analyzer-sampled metrics fires on a fraction of calls
-(`FRUGAL_PROMPT_ANALYSIS_SAMPLE_RATE`, default 10%); the token-side metrics are
-100% — never sampled.
+The analyzer-sampled metrics fire on a fraction of calls
+(`FRUGAL_PROMPT_ANALYSIS_SAMPLE_RATE`, default 10%); the token-side metrics,
+`history_depth`, and `duplicate_requests` are 100% — never sampled.
 
-Every metric carries this base attribute set:
+Every metric carries this base attribute set (except
+`frugal.gen_ai.internal_errors`, which is emitted from failure paths where
+these attributes may not exist yet and carries only `frugal.project_id`, the
+stage/type, and `frugal.customer_id` / `frugal.custom_label` when set):
 
 - `frugal.project_id` — from your env vars (`frugal.customer_id` too when set); `frugal.component` is added on the per-component series when `FRUGAL_COMPONENTS` is set
 - `frugal.caller_file`, `frugal.caller_function` — auto-detected from the call stack
 - `gen_ai.system` + `gen_ai.provider.name` — the provider, dual-emitted under both the legacy and current OTel attribute (`openai`, `anthropic`, `aws.bedrock`, `vertex_ai`/`gcp.vertex_ai`, …, or `custom`). Set via the `system` kwarg; an unrecognized value is coerced to `custom` with a one-time warning. See [`docs/metrics-reference.md`](docs/metrics-reference.md#attributes).
 - `gen_ai.request.model`, `gen_ai.response.model` — the model used
 - `gen_ai.operation.name` — `chat`, `text_completion`, `embeddings`, `generate_content`, `execute_tool`, `create_agent`, `invoke_agent`, or `other` (the sentinel an unrecognized operation coerces to)
-- `frugal.batched` — `true` for batch-API submissions, `false` otherwise
+- `frugal.batched` — `true` for batch-API paths (OpenAI Batch API submissions; Anthropic Message Batches submissions and results), `false` otherwise
+- `frugal.custom_label` — only on `track_ai_call` / `start_ai_call` calls that pass `custom_label=...`; the value must be in the `FRUGAL_CUSTOM_LABEL` allowlist or it is coerced to `other`
 
 The "Additional attributes" column above lists what each metric carries on top of the base set. See [`docs/metrics-reference.md`](docs/metrics-reference.md) for the values each attribute takes, sentinel values for caller fields, and the full internal-error-stage list.
 
@@ -179,7 +191,7 @@ except BaseException as exc:
 
 `success()` and `error()` finalize the call exactly once and are idempotent — extra calls are ignored.
 
-**Metadata** — `track_ai_call` / `start_ai_call` kwargs (all optional): `system` (→ `gen_ai.system`, default `"custom"`), `operation` (default `"chat"`), `request_model` (default `"<unknown>"`). To also run the sampled prompt analyzer, pass any of `messages` / `system_prompt` / `tools` / `response_format` / `max_tokens`.
+**Metadata** — `track_ai_call` / `start_ai_call` kwargs (all optional): `system` (→ `gen_ai.system`, default `"custom"`), `operation` (default `"chat"`), `request_model` (default `"<unknown>"`), `custom_label` (→ the `frugal.custom_label` label on every metric of the call; trimmed, and coerced to `other` unless in the `FRUGAL_CUSTOM_LABEL` allowlist). To also run the sampled prompt analyzer, pass any of `messages` / `system_prompt` / `tools` / `response_format` / `max_tokens`.
 
 **Usage** — `set_usage(...)` / `success(...)` kwargs (all optional): `input_tokens`, `output_tokens`, `cache_read_tokens`, `cache_write_tokens`, `reasoning_tokens`, `response_model`.
 

{frugal_sdk_python-0.3.0 → frugal_sdk_python-0.4.0}/README.md

@@ -50,10 +50,16 @@ Set both or the wrapper silently returns the client unmodified:
 | `FRUGAL_OTEL_EXPORTER_OTLP_HEADERS`      | Export auth headers (e.g. `Authorization=Bearer ...`). Overrides the header built from `FRUGAL_API_KEY`. Falls back to `OTEL_EXPORTER_OTLP_HEADERS`. |
 | `FRUGAL_COMPONENTS`                      | Comma-separated logical components (e.g. `ai-handler,reco`). When set, each metric is ALSO emitted once per component (tagged `frugal.component`) on top of the base project series, so usage can be viewed for the whole project or split by component. |
 | `FRUGAL_CUSTOM_PROVIDERS`                | Comma-separated allowlist of provider names beyond the built-in registry (e.g. `together_ai,fireworks`) that should pass through verbatim on `gen_ai.system` / `gen_ai.provider.name` instead of being coerced to `custom`. Capped at 10 (extras are coerced) to keep label cardinality bounded. |
+| `FRUGAL_CUSTOM_LABEL`                   | Comma-separated allowlist of values for the `frugal.custom_label` label (e.g. `billing,search`). Applies only to `track_ai_call` / `start_ai_call` calls that pass `custom_label=...`; a value outside the first 20 entries (hard cap, not configurable) is coerced to `other`. |
 | `FRUGAL_PATH_BLOCK_LIST`                 | Comma-separated repo-relative paths to skip for caller attribution (see below).          |
 | `FRUGAL_METRIC_EXPORT_INTERVAL_MS`       | Export cadence in milliseconds. Default `60000`.                                         |
 | `FRUGAL_MAX_CALL_SITES`                  | Cardinality cap — max distinct `(caller_file, caller_function)` pairs to attribute. Default `500`. Set to `0` to disable. |
 | `FRUGAL_PROMPT_ANALYSIS_SAMPLE_RATE`     | Fraction of calls (0..1) that run the lightweight prompt analyzer. Default `0.1`. Set to `0` to disable, `1` for every call. |
+| `FRUGAL_DUPLICATE_REQUEST_DETECTION`     | Duplicate-request detection (the `frugal.gen_ai.duplicate_requests` metric) is **on by default** and hashes each chat request in-process (the hash never leaves the SDK). Set to `0`/`false`/`no`/`off` to disable — no hashing, no filter memory, no metric. |
+| `FRUGAL_DUPLICATE_WINDOW_SECONDS`        | How long a request signature is remembered for duplicate detection. Default `3600`, max `604800` (7 days). |
+| `FRUGAL_DUPLICATE_CAPACITY`              | Expected distinct request signatures per window (sizes the in-memory filter). Default `10000`, max `1000000`. Raise it if the SDK warns the filter is saturated. |
+| `FRUGAL_DUPLICATE_FP_RATE`               | Target false-positive rate of the duplicate filter. Default `0.01`, clamped to `[1e-6, 0.5]`. |
+| `FRUGAL_DUPLICATE_GENERATIONS`           | Rotation granularity of the duplicate filter's time window. Default `2`, max `16`.        |
 | `FRUGAL_DISABLED`                        | Set to `1` to disable all instrumentation without uninstalling.                          |
 
 ## What gets captured
@@ -63,32 +69,38 @@ Every wrapped call emits OTel metrics tagged with the caller's source file and f
 | Metric                                  | Kind      | Description                                                  | Additional attributes                  |
 | --------------------------------------- | --------- | ------------------------------------------------------------ | -------------------------------------- |
 | `gen_ai.client.operation.duration`      | histogram | Call latency in seconds                                      | —                                      |
-| `gen_ai.client.token.usage`             | histogram | Input and output token counts                                | `gen_ai.token.type`                    |
-| `frugal.gen_ai.calls`                   | counter   | Number of calls per site                                     | —                                      |
+| `gen_ai.client.token.usage`             | histogram | Input and output token counts                                | `gen_ai.token.type`, `frugal.history_bucket` |
+| `frugal.gen_ai.calls`                   | counter   | Number of calls per site                                     | `frugal.history_bucket`, `frugal.structured_output_mode` |
 | `frugal.gen_ai.cache_read_tokens`       | histogram | Prompt cache hits                                            | —                                      |
 | `frugal.gen_ai.cache_write_tokens`      | histogram | Prompt cache writes                                          | —                                      |
 | `frugal.gen_ai.reasoning_tokens`        | histogram | OpenAI reasoning tokens (o-series), Anthropic thinking tokens| —                                      |
 | `frugal.gen_ai.errors`                  | counter   | Errors by exception type                                     | `frugal.error.type`                    |
 | `frugal.gen_ai.internal_errors`         | counter   | frugal-metrics internal failures (cardinality cap, bugs)     | `frugal.internal_error.stage`          |
 | `frugal.gen_ai.output_cap_utilization`  | histogram | `output_tokens / max_tokens` ratio when `max_tokens` is set  | `frugal.structured_output_mode`        |
-| `frugal.gen_ai.history_depth`           | histogram | `messages.length` (analyzer-sampled)                         | —                                      |
+| `frugal.gen_ai.history_depth`           | histogram | `messages.length` — every chat call (100%, not sampled)      | —                                      |
+| `frugal.gen_ai.duplicate_requests`      | counter   | Repeat-request detection — every chat call (100%, not sampled; disable via `FRUGAL_DUPLICATE_REQUEST_DETECTION`) | `frugal.dup_state` |
 | `frugal.gen_ai.few_shot_count`          | histogram | Few-shot exemplar pairs before final user (analyzer-sampled) | —                                      |
+| `frugal.gen_ai.few_shot_tokens`         | histogram | Approx token size of the few-shot example block (analyzer-sampled) | —                                |
 | `frugal.gen_ai.noise_ratio`             | histogram | HTML/base64/whitespace fraction of input (analyzer-sampled)  | —                                      |
 | `frugal.gen_ai.cache_prefix_stable`     | counter   | Per-block prompt-cache stability (analyzer-sampled)          | `frugal.block`, `frugal.stable`        |
 | `frugal.gen_ai.structured_output_hint`  | counter   | "Return JSON" prompts missing `response_format`/`tools`      | —                                      |
 
-The bottom block of analyzer-sampled metrics fires on a fraction of calls
-(`FRUGAL_PROMPT_ANALYSIS_SAMPLE_RATE`, default 10%); the token-side metrics are
-100% — never sampled.
+The analyzer-sampled metrics fire on a fraction of calls
+(`FRUGAL_PROMPT_ANALYSIS_SAMPLE_RATE`, default 10%); the token-side metrics,
+`history_depth`, and `duplicate_requests` are 100% — never sampled.
 
-Every metric carries this base attribute set:
+Every metric carries this base attribute set (except
+`frugal.gen_ai.internal_errors`, which is emitted from failure paths where
+these attributes may not exist yet and carries only `frugal.project_id`, the
+stage/type, and `frugal.customer_id` / `frugal.custom_label` when set):
 
 - `frugal.project_id` — from your env vars (`frugal.customer_id` too when set); `frugal.component` is added on the per-component series when `FRUGAL_COMPONENTS` is set
 - `frugal.caller_file`, `frugal.caller_function` — auto-detected from the call stack
 - `gen_ai.system` + `gen_ai.provider.name` — the provider, dual-emitted under both the legacy and current OTel attribute (`openai`, `anthropic`, `aws.bedrock`, `vertex_ai`/`gcp.vertex_ai`, …, or `custom`). Set via the `system` kwarg; an unrecognized value is coerced to `custom` with a one-time warning. See [`docs/metrics-reference.md`](docs/metrics-reference.md#attributes).
 - `gen_ai.request.model`, `gen_ai.response.model` — the model used
 - `gen_ai.operation.name` — `chat`, `text_completion`, `embeddings`, `generate_content`, `execute_tool`, `create_agent`, `invoke_agent`, or `other` (the sentinel an unrecognized operation coerces to)
-- `frugal.batched` — `true` for batch-API submissions, `false` otherwise
+- `frugal.batched` — `true` for batch-API paths (OpenAI Batch API submissions; Anthropic Message Batches submissions and results), `false` otherwise
+- `frugal.custom_label` — only on `track_ai_call` / `start_ai_call` calls that pass `custom_label=...`; the value must be in the `FRUGAL_CUSTOM_LABEL` allowlist or it is coerced to `other`
 
 The "Additional attributes" column above lists what each metric carries on top of the base set. See [`docs/metrics-reference.md`](docs/metrics-reference.md) for the values each attribute takes, sentinel values for caller fields, and the full internal-error-stage list.
 
@@ -150,7 +162,7 @@ except BaseException as exc:
 
 `success()` and `error()` finalize the call exactly once and are idempotent — extra calls are ignored.
 
-**Metadata** — `track_ai_call` / `start_ai_call` kwargs (all optional): `system` (→ `gen_ai.system`, default `"custom"`), `operation` (default `"chat"`), `request_model` (default `"<unknown>"`). To also run the sampled prompt analyzer, pass any of `messages` / `system_prompt` / `tools` / `response_format` / `max_tokens`.
+**Metadata** — `track_ai_call` / `start_ai_call` kwargs (all optional): `system` (→ `gen_ai.system`, default `"custom"`), `operation` (default `"chat"`), `request_model` (default `"<unknown>"`), `custom_label` (→ the `frugal.custom_label` label on every metric of the call; trimmed, and coerced to `other` unless in the `FRUGAL_CUSTOM_LABEL` allowlist). To also run the sampled prompt analyzer, pass any of `messages` / `system_prompt` / `tools` / `response_format` / `max_tokens`.
 
 **Usage** — `set_usage(...)` / `success(...)` kwargs (all optional): `input_tokens`, `output_tokens`, `cache_read_tokens`, `cache_write_tokens`, `reasoning_tokens`, `response_model`.
 

{frugal_sdk_python-0.3.0 → frugal_sdk_python-0.4.0}/docs/metrics-reference.md

@@ -16,18 +16,20 @@ metric may carry the extra attributes shown in the rightmost column.
 | ------------------------------------- | --------- | ----------- | ----------------------------------------------------- | ---------------------------------------------- |
 | `gen_ai.client.operation.duration`    | histogram | `s`         | every call, success or error                          | —                                              |
 | `gen_ai.client.token.usage`           | histogram | `{token}`   | every call with a usage payload                       | `gen_ai.token.type`                            |
-| `frugal.gen_ai.calls`                 | counter   | `{call}`    | every call, success or error                          | —                                              |
+| `frugal.gen_ai.calls`                 | counter   | `{call}`    | every call, success or error                          | `frugal.history_bucket` (chat calls), `frugal.structured_output_mode` (with payload) |
 | `frugal.gen_ai.cache_read_tokens`     | histogram | `{token}`   | when provider reports cache reads                     | —                                              |
 | `frugal.gen_ai.cache_write_tokens`    | histogram | `{token}`   | when provider reports cache writes                    | —                                              |
 | `frugal.gen_ai.reasoning_tokens`      | histogram | `{token}`   | OpenAI o-series reasoning models                      | —                                              |
 | `frugal.gen_ai.errors`                | counter   | `{error}`   | when the wrapped call raises                          | `frugal.error.type`                            |
 | `frugal.gen_ai.internal_errors`       | counter   | `{error}`   | frugal-metrics internal failures (cardinality cap, instrumentation bugs) | `frugal.internal_error.stage`     |
 | `frugal.gen_ai.output_cap_utilization`| histogram | `1`         | when request had `max_tokens` (ratio 0..1)            | `frugal.structured_output_mode`                |
-| `frugal.gen_ai.history_depth`         | histogram | `{message}` | analyzer-sampled chat call (`messages.length`)        | —                                              |
+| `frugal.gen_ai.history_depth`         | histogram | `{message}` | every chat call (`messages.length`) — 100%, not sampled | —                                            |
 | `frugal.gen_ai.few_shot_count`        | histogram | `{example}` | analyzer-sampled single-turn chat (count of `(user, assistant)` pairs before final user) | —           |
+| `frugal.gen_ai.few_shot_tokens`       | histogram | `{token}`   | analyzer-sampled single-turn chat — approx token size of the `(user, assistant)` example block, beside `few_shot_count` | —      |
 | `frugal.gen_ai.noise_ratio`           | histogram | `1`         | analyzer-sampled chat call (HTML / base64 / whitespace fraction of input) | —                          |
 | `frugal.gen_ai.cache_prefix_stable`   | counter   | `{call}`    | analyzer-sampled chat call, when previous call to the same call site existed (one point per block, see Attributes) | `frugal.block`, `frugal.stable` |
 | `frugal.gen_ai.structured_output_hint`| counter   | `{call}`    | analyzer-sampled chat call where the prompt asks for JSON without `response_format` or `tools` | —             |
+| `frugal.gen_ai.duplicate_requests`    | counter   | `{call}`    | every chat call with a canonicalizable request, success or error — 100%, not sampled; off when `FRUGAL_DUPLICATE_REQUEST_DETECTION=false` | `frugal.dup_state` |
 
 `gen_ai.client.token.usage` is split across input and output via the
 `gen_ai.token.type` attribute (`input` | `output`).
@@ -37,7 +39,90 @@ The analyzer-sampled metrics are gated by `FRUGAL_PROMPT_ANALYSIS_SAMPLE_RATE`
 minute)` — within a minute a given call site is either fully sampled or fully
 skipped, so the LRU comparisons that drive `cache_prefix_stable` see contiguous
 data when active. The token-side metrics (cap utilization, batched label,
-calls, token usage) are 100% — never sampled.
+calls, token usage) are 100% — never sampled. `frugal.gen_ai.history_depth` and
+the `frugal.history_bucket` label are likewise 100% — `messages.length` is a
+trivially-cheap integer needing no prompt analysis, so it is emitted on every chat
+call (`few_shot_count`, `few_shot_tokens`, `noise_ratio`, `cache_prefix_stable`, and
+`structured_output_hint` remain analyzer-sampled). `few_shot_tokens` is emitted from
+the same analyzer pass as `few_shot_count` (identifying which turns are examples
+needs prompt inspection, so there is no cheap 100% path); its token count is an
+approximate chars/4 estimate over the example block.
+
+Label placement and cardinality: `frugal.history_bucket` rides BOTH the
+`frugal.gen_ai.calls` counter and the `gen_ai.client.token.usage` histogram —
+on token.usage deliberately, so spend can be cut by conversation depth.
+Worst-case that multiplies the token.usage series by 7 buckets × 2 token types
+per call site (in practice a site occupies 1–3 buckets, so expect 2–6× — but
+provision for 14×). `frugal.structured_output_mode` rides ONLY `calls` (and
+`output_cap_utilization`), NOT token.usage — the full 7×4 bucket×mode
+cross-tab lives on `calls` alone, where worst-case it is 28 series per call
+site.
+
+`frugal.gen_ai.duplicate_requests` is also 100% (independent of
+`FRUGAL_PROMPT_ANALYSIS_SAMPLE_RATE`). Each chat call's request is canonicalized
+(`model + system + messages + tools + response_format + max_tokens` — plus
+`previous_response_id` for stateful OpenAI Responses-API calls — with the call
+site folded in) and hashed; the hash is checked against a shared, time-and-size-
+limited rotating Bloom filter and classified `first_seen` / `repeat_seen`. Errored
+calls are classified too — a failed-then-retried identical request reads
+`repeat_seen`, keeping retry storms visible. The raw signature never leaves the
+SDK — only the 2-value state is exported. Hashing is incremental and hard-capped
+at 256 KiB of canonicalized request per call: the small discriminators (call
+site, model, system, tools, response_format, max_tokens, previous_response_id)
+are hashed FIRST, then `messages` under the remaining budget — so an over-budget
+history truncates the tail of the conversation (plus a truncation marker), never
+the fields that keep distinct call sites or models from colliding. Binary/image
+content contributes only its type and length — never its bytes — covering raw
+`bytes` blocks AND the base64-string forms providers actually send (OpenAI
+`data:` URLs, Anthropic base64 source blocks). If the filter saturates (actual
+traffic far above `FRUGAL_DUPLICATE_CAPACITY`, generation union over 50% full),
+a one-time `warn` diagnostic is emitted through the debug callback. If the
+filter's lazy bit-array allocation fails (`MemoryError`), duplicate detection is
+**disabled for the rest of the process**: a one-time `error` diagnostic is
+emitted, the first failing call records a
+`frugal.gen_ai.internal_errors{stage=duplicate_detection}` point, and every
+subsequent call skips detection silently (no `duplicate_requests` metric).
+Controls:
+
+- `FRUGAL_DUPLICATE_REQUEST_DETECTION` (default `true`) — kill switch. When
+  `false`/`0`/`no`/`off`, no filter is allocated, no signature is hashed, and the
+  metric is not emitted. Any other value keeps detection enabled and emits a
+  one-time `warn` diagnostic naming the unrecognized value.
+- `FRUGAL_DUPLICATE_WINDOW_SECONDS` (default `3600`, clamped to `604800` — 7
+  days) — retention window. Because the filter rotates in `generations` slots,
+  a signature is remembered for **at least**
+  `window × (generations−1)/generations` and **at most** `window` seconds
+  (30–60 min at defaults) — treat the counterfactual cache TTL as that range,
+  not a flat window.
+- `FRUGAL_DUPLICATE_CAPACITY` (default `10000`, clamped to `1000000`) — expected
+  distinct signatures per window (Bloom sizing target).
+- `FRUGAL_DUPLICATE_FP_RATE` (default `0.01`, clamped to `[1e-6, 0.5]`) —
+  target false-positive rate.
+- `FRUGAL_DUPLICATE_GENERATIONS` (default `2`, clamped to `16`) — rotating-Bloom
+  generation count.
+
+Out-of-range or non-finite values for any of these fall back to the default (or
+clamp to the bound) with a one-time named `warn` through the debug callback.
+
+Reading `duplicate_requests` correctly:
+
+- **Denominator.** Coverage is a per-endpoint subset of `calls` (embeddings,
+  legacy text completions, bare `track_ai_call` without messages, and other
+  non-chat surfaces emit no duplicate signal), so compute repeat rates as
+  `repeat_seen / (first_seen + repeat_seen)` — never against `calls`.
+- **Retries count as repeats.** Errored calls insert into the filter, so a
+  retried-after-failure request reads `repeat_seen`. That keeps retry storms
+  visible, but it also means `repeat_seen` includes required retry traffic that
+  a response cache would not eliminate — net it against
+  `frugal.gen_ai.errors` when estimating cacheable waste.
+- **Sampling params are ignored.** The signature deliberately excludes
+  `temperature`, `top_p`, `seed`, `n`, and `tool_choice`, so best-of-N /
+  self-consistency workloads read as 100% duplicates. Fine for spotting
+  caching opportunities; wrong for assuming every repeat is waste.
+- **Why a separate counter, not a label on `calls`:** the metric has its own
+  kill switch, covers only a subset of calls, and its 2-value state would
+  otherwise double every `calls` series; keeping it separate keeps `calls`
+  stable when detection is toggled.
 
 ## Attributes
 
@@ -80,10 +165,13 @@ calls, token usage) are 100% — never sampled.
 | `frugal.caller_file`      | Source file repo-relative to `FRUGAL_REPO_ROOT`             |
 | `frugal.caller_function`  | Function/method name at the call site                       |
 | `frugal.error.type`       | Exception class name (error-path metrics only)              |
-| `frugal.batched`          | `true` when emitted from a batch-API method, `false` otherwise. Present on every metric. The batch-API wrap paths themselves are a follow-up milestone — until then this attribute is always `false`. |
-| `frugal.structured_output_mode` | `none` \| `json_object` \| `json_schema` \| `tools`. Only on `frugal.gen_ai.output_cap_utilization`. |
+| `frugal.batched`          | `true` when the metric came from a provider batch-API path, `false` otherwise. Present on every metric. Batch submissions (OpenAI `batches.create`, Anthropic `messages.batches.create`) emit `frugal.gen_ai.calls` with `batched="true"` — one point per request for Anthropic (the requests array is in hand, grouped by model), one per submission for OpenAI (per-request counts are not reported at submission time, so OpenAI batched call volume is a floor, never an overcount). Iterating Anthropic `messages.batches.results()` additionally emits per-entry `gen_ai.client.token.usage` (+ cache metrics) with `batched="true"`, attributed to the call site that submitted the batch when the submission happened in the same process (bounded in-process memory; falls back to the results call site). OpenAI batch token usage lives in the output file read through the generic files API and is not intercepted. The manual track API accepts a `batched` flag for hand-instrumented batch paths. An app on an SDK predating batch support reads `false` on genuinely batched traffic — the label reflects what the deployed SDK can see. |
+| `frugal.structured_output_mode` | `none` \| `json_object` \| `json_schema` \| `tools`. Derived from request params by precedence: non-empty `tools` → `tools`; else `response_format.type` `json_object` / `json_schema` → the matching value; else `none`. On `frugal.gen_ai.output_cap_utilization` (when `max_tokens` set), and on `frugal.gen_ai.calls` for every call with a request payload (omitted when there is no payload). |
 | `frugal.block`            | `system` \| `user` \| `tools` \| `schema`. Only on `frugal.gen_ai.cache_prefix_stable`. |
 | `frugal.stable`           | `true` when the block hash matches the previous call at this site. Only on `frugal.gen_ai.cache_prefix_stable`. |
+| `frugal.history_bucket`   | Bounded bucket of `messages.length`: `0` \| `1-2` \| `3-5` \| `6-10` \| `11-20` \| `21-50` \| `51+`. On `frugal.gen_ai.calls` and `gen_ai.client.token.usage`, and only for chat calls (payload has a `messages` array); omitted otherwise. 100% (not analyzer-sampled). |
+| `frugal.custom_label`    | Custom label (e.g. a business unit) passed as `custom_label=` to `track_ai_call` / `start_ai_call` — never set by the provider wrappers. When passed, it rides EVERY metric of that call (success and error paths alike). The value is trimmed of surrounding whitespace (matching the Node SDK), then must match one of the first 20 entries of `FRUGAL_CUSTOM_LABEL` (hard cap, not configurable) or it is coerced to `other`. Omitted when not passed. |
+| `frugal.dup_state`        | `first_seen` \| `repeat_seen` — whether this request's canonical signature was already seen within the duplicate-detection window. Only on `frugal.gen_ai.duplicate_requests`. |
 
 ### Sentinel values for `frugal.caller_file` / `frugal.caller_function`
 
@@ -118,8 +206,16 @@ attribute describing where our own code failed. Non-exhaustive list:
 | `setup_failed`          | Our wrapper setup (OTel init, caller resolution) threw; original call continued normally |
 | `stream_proxy_failed`   | Constructing the stream wrapper threw; original response returned unwrapped |
 | `emission`              | Emitting a success/error metric threw                                  |
+| `duplicate_detection`   | The duplicate-request path (signature, filter, or emission) threw — including the one-time filter allocation failure; carries `frugal.internal_error.type` with the exception class |
 | `cardinality_cap_hit`   | `FRUGAL_MAX_CALL_SITES` was just reached (once per process)            |
 
+Unlike every other metric, `frugal.gen_ai.internal_errors` does **not** carry
+the full base attribute set — it is emitted from failure paths where the
+caller/provider/model attributes may not exist yet. Its points carry only
+`frugal.project_id`, `frugal.internal_error.stage` (plus
+`frugal.internal_error.type` where noted), and — when set —
+`frugal.customer_id` and `frugal.custom_label`.
+
 ## Provider coverage
 
 | Provider                                                | Wrapper                              | `gen_ai.system`  |

{frugal_sdk_python-0.3.0 → frugal_sdk_python-0.4.0}/docs/quickstart.md

@@ -26,10 +26,16 @@ Optional:
 | `FRUGAL_OTEL_EXPORTER_OTLP_ENDPOINT`  | Frugal-specific OTLP collector URL. Defaults to `https://metrics.frugal.co/opentelemetry/v1/metrics`. Falls back to `OTEL_EXPORTER_OTLP_ENDPOINT`. |
 | `FRUGAL_OTEL_EXPORTER_OTLP_HEADERS`   | Export auth headers (e.g. `Authorization=Bearer ...`). Overrides the header built from `FRUGAL_API_KEY`. Falls back to `OTEL_EXPORTER_OTLP_HEADERS`. |
 | `FRUGAL_COMPONENTS`                   | Comma-separated logical components (e.g. `ai-handler,reco`). When set, each metric is ALSO emitted once per component (tagged `frugal.component`) on top of the base project series, so usage can be viewed for the whole project or split by component. |
+| `FRUGAL_CUSTOM_LABEL`                | Comma-separated allowlist of values for the `frugal.custom_label` label (e.g. `billing,search`). Applies only to `track_ai_call` / `start_ai_call` calls that pass `custom_label=...`; a value outside the first 20 entries (hard cap, not configurable) is coerced to `other`. |
 | `FRUGAL_PATH_BLOCK_LIST`              | Comma-separated repo-relative paths to skip when attributing the caller (see below).        |
 | `FRUGAL_METRIC_EXPORT_INTERVAL_MS`    | Export cadence. Default `60000`.                                                            |
 | `FRUGAL_MAX_CALL_SITES`               | Cardinality cap — max distinct `(caller_file, caller_function)` pairs to attribute before overflow. Default `500`, set to `0` to disable. See [Cardinality cap](#cardinality-cap). |
-| `FRUGAL_PROMPT_ANALYSIS_SAMPLE_RATE`  | Fraction of calls (0..1) where the lightweight prompt analyzer runs and emits derived metrics (`history_depth`, `noise_ratio`, `cache_prefix_stable`, …). Default `0.1`. Set to `0` to disable, `1` for every call. See [Prompt analysis](#prompt-analysis). |
+| `FRUGAL_PROMPT_ANALYSIS_SAMPLE_RATE`  | Fraction of calls (0..1) where the lightweight prompt analyzer runs and emits derived metrics (`noise_ratio`, `cache_prefix_stable`, `few_shot_*`, …). Default `0.1`. Set to `0` to disable, `1` for every call. See [Prompt analysis](#prompt-analysis). |
+| `FRUGAL_DUPLICATE_REQUEST_DETECTION`  | Duplicate-request detection (`frugal.gen_ai.duplicate_requests`) is **on by default**: every chat request is canonicalized and SHA-256 hashed in-process to classify it `first_seen`/`repeat_seen`. Set to `0`/`false`/`no`/`off` to turn it off entirely (no hashing, no filter memory, no metric). |
+| `FRUGAL_DUPLICATE_WINDOW_SECONDS`     | How long a request signature is remembered for duplicate detection. Default `3600`, max `604800` (7 days). |
+| `FRUGAL_DUPLICATE_CAPACITY`           | Expected distinct request signatures per window — sizes the in-memory duplicate filter. Default `10000`, max `1000000`. Raise it if the debug callback warns the filter is saturated. If the filter's memory can't be allocated at all, detection disables itself for the rest of the process (one-time `error` diagnostic + one `internal_errors` point). |
+| `FRUGAL_DUPLICATE_FP_RATE`            | Target false-positive rate of the duplicate filter. Default `0.01`, clamped to `[1e-6, 0.5]`. |
+| `FRUGAL_DUPLICATE_GENERATIONS`        | Rotation granularity of the duplicate filter's time window. Default `2`, max `16`.          |
 | `FRUGAL_DISABLED`                     | Kill switch. Set to `1` to disable without uninstalling.                                    |
 
 Standard `OTEL_EXPORTER_OTLP_*` env vars are honored as fallbacks if the
@@ -254,7 +260,9 @@ extra calls are ignored.
 
 **Metadata** — `track_ai_call` / `start_ai_call` kwargs (all optional):
 `system` (→ `gen_ai.system`, default `"custom"`), `operation` (default
-`"chat"`), `request_model` (default `"<unknown>"`). To also run the sampled
+`"chat"`), `request_model` (default `"<unknown>"`), `custom_label` (→ the
+`frugal.custom_label` label on every metric of the call; values outside the
+`FRUGAL_CUSTOM_LABEL` allowlist become `other`). To also run the sampled
 prompt analyzer, pass any of `messages` / `system_prompt` / `tools` /
 `response_format` / `max_tokens`.
 
@@ -316,17 +324,28 @@ detect common AI cost traps:
 
 | Signal | Metric | Cost trap |
 | --- | --- | --- |
-| Conversation depth | `frugal.gen_ai.history_depth` | unbounded history re-sent on every turn |
 | Few-shot example count | `frugal.gen_ai.few_shot_count` | over-provisioned exemplars where accuracy plateaus |
+| Few-shot example size | `frugal.gen_ai.few_shot_tokens` | oversized exemplar blocks re-sent on every call |
 | Input noise (HTML tags, base64 runs, large whitespace) | `frugal.gen_ai.noise_ratio` | filterable boilerplate paying token cost |
 | Per-block cache stability | `frugal.gen_ai.cache_prefix_stable` | system / tools / schema churn breaking provider prompt cache |
 | Structured-output keyword in prose | `frugal.gen_ai.structured_output_hint` | "return JSON" prompts that should use `response_format` / `tools` |
 
+Two related signals run on **every** chat call (100%, independent of the
+analyzer sample rate): `frugal.gen_ai.history_depth` (the `messages.length`
+count — unbounded history re-sent on every turn) and
+`frugal.gen_ai.duplicate_requests` (repeat-request detection; disable via
+`FRUGAL_DUPLICATE_REQUEST_DETECTION`).
+
 > **What we read vs what we send.** The analyzer inspects the request's
 > `messages`, `tools`, `response_format`, `system`, and `max_tokens` kwargs to
-> compute the signals above. We never send raw request content over the wire —
-> only stats (counts, ratios) and SHA-256 hashes used in-process for the
-> stability comparisons. The hashes never leave the process.
+> compute the signals above. Duplicate detection additionally canonicalizes and
+> SHA-256 hashes those same fields on every chat call (capped at 256 KiB per
+> request; binary/image content — raw bytes and the base64 strings providers
+> actually send — contributes only its type and length). We never send raw
+> request content over the wire — only stats (counts, ratios); the hashes are
+> used in-process for the stability comparisons and the duplicate filter, and
+> never leave the process. For `duplicate_requests`, only the two-value
+> `first_seen`/`repeat_seen` state is exported.
 
 Sampling defaults to **10% per call site** and is deterministic by call site +
 minute, so a given call site is either fully sampled or fully skipped within a
@@ -335,7 +354,8 @@ paying analyzer CPU cost on every call. Tune via
 `FRUGAL_PROMPT_ANALYSIS_SAMPLE_RATE`.
 
 The token-side metrics (`gen_ai.client.token.usage`, `frugal.gen_ai.calls`,
-`frugal.gen_ai.cache_*`, `frugal.gen_ai.output_cap_utilization`) are 100% —
+`frugal.gen_ai.cache_*`, `frugal.gen_ai.output_cap_utilization`) plus
+`frugal.gen_ai.history_depth` and `frugal.gen_ai.duplicate_requests` are 100% —
 never sampled.
 
 ## See why nothing is exporting (debug callback)

{frugal_sdk_python-0.3.0 → frugal_sdk_python-0.4.0}/src/frugal_metrics/__init__.py

@@ -3,6 +3,7 @@
 from importlib.metadata import PackageNotFoundError, version
 
 from frugal_metrics.debug import set_debug_callback
+from frugal_metrics.semconv import DupState, HistoryBucket, StructuredOutputMode
 from frugal_metrics.providers import (
     PROVIDERS,
     GenAiOperation,
@@ -32,6 +33,9 @@ __all__ = [
     "ProviderEntry",
     "GenAiSystem",
     "GenAiOperation",
+    "HistoryBucket",
+    "StructuredOutputMode",
+    "DupState",
     "normalize_provider",
     "normalize_operation",
 ]