mca-sdk 0.9.4.dev1__tar.gz → 0.9.7__tar.gz

{mca_sdk-0.9.4.dev1 → mca_sdk-0.9.7}/CHANGELOG.md

@@ -2,7 +2,389 @@
 
 All notable changes to the MCA SDK will be documented in this file.
 
-## [Unreleased]
+## [0.9.7] - 2026-07-25
+### Fixed
+- `mca-verify` console script is now installed by `pip install mca-sdk`. The v0.9.6 wheel advertised the CLI in its release notes but never registered it in `pyproject.toml`'s `[project.scripts]` table, so the entry point was silently dropped by the setuptools build backend (`pyproject.toml` wins over `setup.py`'s `entry_points`). `MCAClient.verify_baseline_emissions()` (the Python API surface) was unaffected in v0.9.6 and remains unchanged in v0.9.7.
+- `pip install mca-sdk[test]` and `[dev]` now pull `mistune>=3.0.0` so `tests/unit/test_for_ai_assistants_doc_lint.py` runs instead of `pytest.importorskip`-skipping when consumers run the SDK's test suite against the sdist.
+
+## [0.9.6] - 2026-07-24
+### Added
+- `MCAClient.verify_baseline_emissions(*, sample_size=10, timeout_seconds=5.0, strict=False) -> BaselineVerificationReport` — inspects the last-emitted metrics/logs/spans against the §2A baseline emission contract keyed off `(model_category, model_type)` and returns a structured report. Backed by a presence-only in-memory `ObservationBuffer` (256 items/signal) that records attribute-key names and datapoint counts but never attribute values, so the serialized report is PHI-safe by construction. Gated by a new `baseline_verify_enabled` config field (env `MCA_BASELINE_VERIFY_ENABLED`); off by default. Bypasses `flush()` coalescing via a `force_flush` under `_shutdown_lock` — if `shutdown()` has already run, raises the new `SDKShutdownError` rather than NPE. A new `mca-verify` console script (`mca_sdk.cli.verify:main`) wraps the same check for CI, exit codes 0/2/3 (`--strict`, `--format {text,json}`). New public exceptions `BaselineVerificationError` (inherits `MCASDKError` directly, NOT `ValidationError`, so existing `except ValidationError` handlers do not silently swallow verify failures) and `SDKShutdownError`. The verify path is additive — the OTLP export path is untouched when the flag is off.
+- `MCAClient.record_error(err, ..., input_data=None)` — new keyword. When set, the failing input flows through the same `_safe_sanitize_for_telemetry` size-cap / secret-scrubber pipeline `record_prediction` uses and lands in `log_extra["input_data"]` with matching `input_shape` / `input_truncated` metadata. Gated by the existing `capture_input_data` flag. Restores Evidently drift coverage on the failure path once autolog's error branch swaps to `record_error` (see the `### Changed` entry below).
+
+### Fixed
+- `MCALangChainCallback._extract_model_name` no longer returns the LangChain constructor class (`ChatOpenAI`, `AzureChatOpenAI`, ...) as a model identifier. Lookup now walks `serialized["kwargs"]["model_name"]` → `kwargs["model"]` → `kwargs["deployment_name"]` (Azure), falling through to `"unknown"`; `serialized["name"]` is ignored because LangChain populates it with the class name. Dashboards and cost slices keyed on the real deployed model (`gpt-4o-mini`, `claude-3-sonnet`, ...) now receive the correct value for LangChain traffic (tk_faffe8808cf5444c).
+- `MCALangChainCallback` now emits the OpenTelemetry GenAI semantic-convention attributes on LLM spans by delegating to the shared `build_request_attributes` / `build_usage_attributes` in `_genai_attrs.py` — the same builders `MCALiteLLMCallback` and `MCAClient.record_llm_generation` use. Start spans gain `gen_ai.system`, `gen_ai.request.model`, `gen_ai.operation.name` (`chat` for chat models, `text_completion` for LLMs), `gen_ai.request.temperature|max_tokens|top_p`, and `langfuse.observation.type="generation"`. End spans gain `gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens`, `gen_ai.usage.total_tokens`, `gen_ai.response.model` (from `llm_output.model_name` or `llm_output.model`), and `gen_ai.latency_seconds` (mirrors LiteLLM callback). Legacy `langchain.*` and `genai.*` attributes — including `genai.tokens_prompt|completion|total` and `genai.latency_seconds` — remain for backward compatibility with dashboards and trace filters keyed on the pre-PR names. The `genai.cost_dollars` span attribute is dropped as of tk_e0237bafa19a4365 in the same `[Unreleased]` window because Langfuse now prices generation spans server-side; dashboards must migrate to `gen_ai.usage.input_tokens` × Langfuse pricing (tk_faffe8808cf5444c).
+- `MCALlamaIndexCallback` now emits OpenTelemetry GenAI semantic-convention attributes by delegating to the shared `build_request_attributes` / `build_usage_attributes` builders in `_genai_attrs.py` — the same source of truth `MCALiteLLMCallback`, `MCALangChainCallback`, and `MCAClient.record_llm_generation` use. Start spans gain `gen_ai.system`, `gen_ai.provider.name`, `gen_ai.request.model`, `gen_ai.operation.name="chat"`, and `langfuse.observation.type="generation"`; end spans gain `gen_ai.usage.input_tokens|output_tokens|total_tokens` (with the transitional `prompt_tokens|completion_tokens|total_tokens` legacy aliases dual-emitted by the builder). `gen_ai.latency_seconds` and the legacy `genai.latency_seconds` are both written for the one-release compat window; `genai.cost_dollars` is retained on LlamaIndex spans for the same window pending the Langfuse pricing handoff. `_extract_provider` now delegates to the shared `extract_provider` helper, so LiteLLM-style prefixes (`openai/gpt-4`, `vertex_ai/gemini-1.5-pro`) resolve correctly instead of returning `"unknown"`. The legacy `genai.tokens_prompt|completion|total` span attributes are NOT re-emitted alongside the builder output (would triple-stack under the `tk_e0237bafa19a4365` deprecation window).
+- `MCALangChainCallback._extract_provider` now delegates to the shared `extract_provider` helper in `mca_sdk/integrations/_genai_attrs.py` (a one-line body replacing the 22-line local implementation). Fixes LiteLLM-style provider prefixes on LangChain spans — `openai/gpt-4`, `vertex_ai/gemini-2.0-flash-001`, `bedrock/anthropic.claude-3-sonnet-20240229-v1:0` now resolve to their prefix instead of returning `"unknown"`, so Langfuse pricing and provider slices work for LangChain traffic that flows through LiteLLM. Narrow intentional regression: substring fallback matches (`"my-openai-fork"` → `"openai"`, `"foo-anthropic-bar"` → `"anthropic"`, `"google-x"` → `"google"`, `"meta-y"` → `"meta"`) now return `"unknown"`; callers relying on this ambiguous behavior should switch to explicit `provider/model` naming.
+- Consolidated the `genai.latency_seconds` histogram registration onto a single canonical instrument via a new `MCAClient._get_or_create_histogram` accessor. `_initialize_standard_metrics`, `MCALiteLLMCallback.__init__`, the shared GenAI instrument cache in `integrations/genai.py`, and `MCAClient.record_metric`'s lazy histogram path all now resolve to the same instrument object, eliminating the duplicate-TimeSeries rejection GCP produced when `model_type="generative"` was combined with the LiteLLM callback. Description text on later registrations is discarded (first-caller-wins on metadata).
+- `MCAClient.flush()` now coalesces force-flushes that arrive within one metric-export-interval window (plus 0-10% jitter). Explicit flushes overlapping the periodic reader's export tick no longer produce two CUMULATIVE datapoints for the same `(metric, labels, resource)` tuple. Suppressed calls are logged at DEBUG and return `True` because the datapoint is already in flight via the periodic reader. `shutdown()` bypasses this method and drains providers directly, so the exit path is unaffected.
+- example_deployed/internal-agentic-vertex: /predict handler now fans instances out via asyncio.gather + asyncio.to_thread instead of a serial sync loop inside async def, unblocking the Vertex 60s per-request ceiling for batch_size>1 (root cause of mca-predict-agentic 504 outage on 2026-07-23).
+- example_deployed/internal-agentic-vertex: removed per-instance record_prediction calls from agent_instrumented.research_question in favor of one batch-level record_prediction, matching the canonical shape in FOR_AI_ASSISTANTS.md 2.1.
+- example_deployed/internal-agentic-vertex: dropped synthetic time.sleep from tools.py mock implementations that added 1.5-3.5 s per instance to the request path.
+- example_deployed/internal-genai-vertex, example_deployed/autolog-demo-vertex: /predict handlers converted to asyncio.gather fan-out; flush moved out of per-instance loops.
+- example_deployed/data-generation-automation/cloudrun_server.py: endpoint.predict now uses timeout=55 so a client-side deadline fires before Vertex's 60s ceiling with a clean DeadlineExceeded stack trace; batch_size default lowered from 10 to 1.
+
+### Changed
+- `mca_sdk.autolog._base._instrumented_call` now routes the exception branch through `MCAClient.record_error(...)` instead of `MCAClient.record_prediction(output=None, ...)`. Failed `sklearn`, `xgboost`, and `lightgbm` predictions therefore emit `{prefix}.predictions_total{status="error"}` and `{prefix}.errors_total` rather than incrementing the success counter. This is a **breaking dashboard behavior** change for any Grafana / Cloud Monitoring panel, alert policy, or SLO burn calculation keyed on `status="success"` for the prediction counter — success rate will drop for models whose failed predictions were previously mis-bucketed as success. A dual-emit compatibility flag `MCA_AUTOLOG_LEGACY_ERROR_COUNTER` (defaults to `1` in v0.10.x, flips to `0` in v0.11.x, removed in v0.12.x) preserves the legacy emission while dashboards migrate. See [`docs/migration-guides/autolog-error-counter.md`](../../docs/migration-guides/autolog-error-counter.md) for the alert-policy rewrite recipes and the release schedule. `mca_sdk.autolog._openai` and `mca_sdk.autolog._anthropic` are unaffected — they use `_record_genai_metrics` against a different metric family.
+- example_deployed/internal-agentic-vertex, example_deployed/internal-genai-vertex, example_deployed/autolog-demo-vertex: capture_input_data and capture_output_data now default to False for generative/agentic model_types (they still work when explicitly enabled after an application-side scrubber is in place).
+- example_deployed/data-generation-automation: new README.md runbook for cadence + batch_size changes via gcloud (Dockerfile already had PYTHONUNBUFFERED=1).
+- `MCALangChainCallback` no longer computes cost client-side. The `genai.cost_dollars` histogram, the matching span attribute, and the `cost_dollars` field in the success log record are all removed; `_estimate_cost` and the callback's `_cost_histogram` instrument have been deleted. Cost is now resolved by Langfuse server-side from `gen_ai.provider.name` + `gen_ai.request.model` + `gen_ai.usage.input_tokens|output_tokens` via its per-model price registry. The unnamespaced `model` and `provider` span attributes are also dropped in the same pass — consumers should key on `gen_ai.request.model` and `gen_ai.system` (and, transitionally, `gen_ai.provider.name`) instead. `MCALiteLLMCallback`'s local `genai.cost_usd` histogram is retained because its value is provider-computed via `litellm.completion_cost()`, not derived from the deprecated static pricing tables (tk_e0237bafa19a4365).
+- `build_request_attributes` in `mca_sdk/integrations/_genai_attrs.py` now dual-emits `gen_ai.provider.name` alongside `gen_ai.system` during the OTel semantic-conventions 1.34+ transition window. Consumers keyed on either name see the same value. `gen_ai.system` is scheduled for removal in v0.11.0 once Langfuse's OTLP mapper picks up the new attribute name (tk_e0237bafa19a4365).
+
+### Deprecated
+- `mca_sdk.integrations.genai.estimate_openai_cost`, `estimate_anthropic_cost`, `OPENAI_PRICING`, and `ANTHROPIC_PRICING` are deprecated. Access via `from mca_sdk.integrations.genai import ...` (all four names) or `from mca_sdk.integrations import estimate_openai_cost` (only the historically re-exported name) now emits a `DeprecationWarning` routed through a PEP 562 module `__getattr__` shim; the underlying values continue to work correctly for one release. Langfuse's per-model price registry supersedes them — callers should emit tokens only and let Langfuse compute cost from `gen_ai.provider.name` + `gen_ai.request.model` + `gen_ai.usage.*_tokens`. Callers with irregular provider/model combos should upstream pricing entries to Langfuse rather than pin the SDK's static tables. Removal target: v0.11.0 (tk_e0237bafa19a4365).
+- `MCAClient.record_llm_generation` yields an `_LLMGenerationRecorder` whose `.set_cost(cost_usd)` handle is deprecated for the same reason; the call still works and emits a `DeprecationWarning` (`stacklevel=2`, so the warning is attributed to the caller). Emit tokens via `gen.set_usage` / `gen.set_usage_from_response` and let Langfuse price the span server-side. Removal target: v0.11.0 (tk_e0237bafa19a4365).
+- Follow-up: `mca_sdk.autolog._openai`, `mca_sdk.autolog._anthropic`, and `MCALlamaIndexCallback` still call the legacy pricing helpers via the private aliases (`_estimate_openai_cost_impl`, `_estimate_anthropic_cost_impl`) so this release does not spam DeprecationWarnings on downstream unit tests. Their migration to token-only emission (Langfuse handoff) is tracked as a separate ticket and lands ahead of the v0.11.0 removal (tk_e0237bafa19a4365).
+
+### Docs
+- `mca_sdk/FOR_AI_ASSISTANTS.md`: added §2A "Required baseline emission contract" — a 7-row table (regression×internal, classification×internal, time-series×internal, generative×internal, agentic×internal, vendor in-process wrapper, vendor out-of-process bridge) declaring the metrics, log-extras, per-emission attributes, and downstream failure modes each `(model_type, model_category)` shape MUST produce regardless of which AI coding assistant runs the instrumentation prompt. Row 4 mandates `gen_ai.response.model` for OpenAI ChatCompletion auto-fallback paths. Row 5 phrases `association_id` as a stable run-scoped identifier (LangGraph `thread_id` given as one example, not a hard bind). Row 7 explicitly disclaims `record_prediction` for out-of-process vendor bridges. Added §2A.1 declaring three known SDK deviations that assistants must NOT hand-work-around (LlamaIndex legacy-only span attrs — now fixed above; LangChain deprecated key dual-emit; autolog OpenAI/Anthropic emitting `openai.*` / `anthropic.*` metric families). Stamped `> Baseline: §2A row N` back-references at the top of every §2.x pattern subsection. Replaced §6 with a two-layer verification checklist (universal items 1–4 preserved; per-type OTLP grep sub-checklists layer 2 including a `gen_ai.response.model` auto-fallback check). Appended one cross-reference sentence to §10 Variant A and Variant B. A new mistune-based doc-lint test (`tests/unit/test_for_ai_assistants_doc_lint.py`, 19 tests) asserts the table row set matches `METRIC_PREFIX_ROUTING` in `utils/routing.py`, each row's `metric_prefix` cell matches `get_metric_prefix()`, row 4's per-emission attrs match `build_request_attributes` / `build_usage_attributes` output, row 5 names every `agent.*` counter in `core/client.py`, row 7 disclaims `record_prediction` (verified against `integrations/bridge.py`), and every §2.x heading is followed by a §2A back-reference. Follow-up: `mistune>=3.0.0` needs adding to `setup.py`'s `extras_require["dev"]`.
+- `docs/integration-guides/ai-assistant-guide.md`: appended a §2A cross-reference sentence to both Variant A and Variant B prompts so downstream assistants cross-check emitted OTLP against the baseline contract before signing off.
+- `docs/integration-guides/{predictive,agentic,genai,vendor}-models.md` and `vendor-models-playbook.md`: one-line pointer added to the top of each: "Required baseline emissions per model type: see `mca_sdk/FOR_AI_ASSISTANTS.md` §2A."
+- mca_sdk/FOR_AI_ASSISTANTS.md: added section 2.8 (Vertex online prediction batching), a new flush-placement rule in section 3, a new capture-flag rule in section 7, and section 8 (escape hatches: rawPredict, BatchPredictionJob).
+
+## [0.9.5] - 2026-07-20
+### Added
+- Zero-code LiteLLM auto-instrumentation via LiteLLMInstrumentor BaseInstrumentor (registered as opentelemetry_instrumentor entry point).
+- mca_sdk.integrations._autoregister helper: mca-instrument now auto-registers MCALiteLLMCallback / MCALangChainCallback / MCALlamaIndexCallback when the respective libraries are installed. Opt-out via MCA_DISABLE_LLM_CALLBACKS.
+- Experimental Google ADK PoC instrumentation (opt-in via MCA_ENABLE_ADK_POC=1) in mca_sdk.instrumentation.adk_poc.
+- Jupyter IPython magics (%load_ext mca_sdk.magics) - %mca_init line magic and %%mca_instrument cell magic.
+- Apache Airflow integration (mca_sdk.integrations.airflow): MCAAirflowHook, @mca_task decorator, MCAOnSuccessCallback, MCAOnFailureCallback.
+- KubeFlow Pipelines compile-time hook (mca_sdk.integrations.kubeflow.instrument_pipeline_yaml).
+- New integration guides: jupyter-notebooks, airflow, kubeflow, time-series-models, time-series-models-playbook, adk-poc.
+- New extras: otel-litellm, airflow, kubeflow.
+### Changed
+- autolog() now logs discovered opentelemetry_instrumentor entry points at INFO for observability.
+- GenAI integration guides note auto-registration; manual callback wiring no longer required under mca-instrument.
+- mca-instrument bootstrap invokes autoregister_llm_callbacks() after autolog().
+- Time-series forecasting integration docs restructured. Standalone
+  `time-series-models.md` and `time-series-models-playbook.md` are folded
+  into a dedicated §Time-Series Forecasting section of
+  `docs/integration-guides/predictive-models.md`. Two SDK-supported
+  emission patterns are now documented: Pattern A (per-step
+  `record_prediction` with per-step future ISO 8601 `association_id` and
+  1-based `forecast_horizon_step`) is the recommended shape and matches
+  `example_deployed/uc-brickell-wait-time/main.py:143-163`; Pattern B
+  (per-run metric + per-step logs) remains as an alternative. The
+  zero-code exemption rationale and the Path B envelope-middleware ADR
+  are preserved. Cross-references in the integration-guides README,
+  `predictive-models-playbook.md`, `zero-code.md`, and
+  `zero-code-playbook.md` are repointed to the new section anchor.
+- `mca_sdk/FOR_AI_ASSISTANTS.md` (guide shipped inside the wheel) gains a
+  §2.1.1 that routes Prophet / ARIMA / SARIMAX / multi-step-return code
+  signals to Pattern A, with the five per-step rules (1-based
+  `forecast_horizon_step`; ISO 8601 UTC `association_id` stored
+  verbatim, no SDK-side timezone interpretation; no caller-supplied
+  `timestamp` kwarg; per-step-varying `association_id`; flat kwargs on
+  `client.trace`). §2.1's batch rationale and §3 rule 3 are qualified to
+  acknowledge time-series as the intentional exception to the
+  one-metric-per-batch rule.
+### Fixed
+- Zero-code integration modules now pass the SDK's strict mypy and Black CI gates.
+- Airflow 3 uses its supported `airflow.sdk` hook import while retaining the Airflow 2 fallback.
+- The docs extra now installs the OpenTelemetry instrumentation dependency required to import
+  and document the LiteLLM instrumentor.
+- ADK positional tool calls now record the actual tool name instead of `unknown`.
+- Direct-provider and LiteLLM generation spans now distinguish missing usage
+  and cost from genuine numeric zero using controlled `mca.*.status`
+  attributes and provide `set_usage_from_response()` for common Vertex,
+  OpenAI/LiteLLM, Anthropic, and Bedrock response shapes without estimating
+  missing tokens.
+  Usage emits current OTel input/output aliases and the `gen_ai.usage.cost`
+  attribute consumed by deployed Langfuse v3, while retaining legacy names.
+### Notes
+- Time-series forecasting models remain exempt from the zero-code path - see the §Time-Series Forecasting section of `docs/integration-guides/predictive-models.md`. FastAPI/Flask response-parsing middleware (Path B in to-do.md) was intentionally deferred.
+- Standalone apache-airflow-providers-mca PyPI package is planned as follow-up work.
+
+## [0.9.4] - 2026-07-16
+
+### Added
+- **`MCA_STRICT_REGISTRY_AUTH` env var** (config field
+  `strict_registry_auth: bool = False` in `MCAConfig`, auto-wired through
+  `_load_from_env` and `from_env` in `mca_sdk/config/settings.py`). Set to
+  `true` (or any truthy `1` / `yes` / `on`) to restore the pre-
+  tk_dcaefe49fd364823 fail-closed contract: `MCAClient(...)` will raise
+  `ConfigurationError` when `registry_url` + `model_id` are provided without
+  an explicit `registry_token` / `use_gcp_auth` / `tls_insecure_registry`,
+  even if `registry_url` resolves to an RFC1918 / loopback / IPv6 ULA host.
+  Default is `False` so the soft-default from tk_dcaefe49fd364823 continues
+  to apply — see the `### Fixed` entry below.
+
+### Breaking
+- **Metric shape** — `genai.tokens_total` (and its `vendor.*` / `emms_*`
+  renderings) is now emitted with two `token_type` values only: `prompt` and
+  `completion` (or `input` / `output` on the Anthropic autolog path). The
+  redundant `token_type="total"` bucket that previously accompanied every
+  request was removed. Consumer migration: dashboards that filtered
+  `token_type != "total"` can drop the filter; dashboards that summed
+  `token_type == "total"` alone must switch to a filterless
+  `sum by(...) (rate(genai_tokens_total[5m]))` or equivalent. Prior behavior
+  aggregated to `prompt + completion + total = 2 × actual`, so this is a
+  correctness fix but any dashboard that depended on the old shape needs to
+  be updated. Full detail and rationale in the `### Changed` entry below.
+  Fixes tk_e9e917c20ce34be9.
+
+### Known issues
+- **`registry_token` as a kwarg is a hard `ConfigurationError`** ("Security violation: registry_token must be provided via environment variable (MCA_REGISTRY_TOKEN) only, not kwargs"). The intent — prevent tokens ending up in source control — is sound, but the implementation is a footgun for anyone reading `MCAClient.__init__` and trying to be explicit about auth wiring. **Proposed fix:** downgrade to a `DeprecationWarning` on kwarg use, still honor the value, and refuse only when the passed literal is a string literal at the call site (detectable by inspecting the caller frame, or by requiring an opt-in `MCA_ALLOW_REGISTRY_TOKEN_KWARG=true` env for kwarg use in prod-detected environments — mirrors the `MCA_ALLOW_ENV_KEYS_IN_PROD=true` pattern already in [0.8.31]). Document the env-var preference in the `MCAClient` docstring so IDE tooltips surface it before the caller writes the kwarg.
+- **`MCA_USE_GCP_AUTH` env var has no native support.** `MCAConfig.from_env` (`mca_sdk/config/settings.py::_load_from_env`) reads `MCA_REGISTRY_TOKEN`, `MCA_REGISTRY_URL`, `MCA_TLS_INSECURE_REGISTRY`, etc., but there is no env-driven path to enable `use_gcp_auth`. This forces every consumer app that wants ADC-based registry auth (the pattern in `sdk-examples/internal-agentic-vertex/agent_instrumented.py:354`) to either hard-code `use_gcp_auth=True` in code or reinvent the env-var read. Parity with the existing `MCA_REGISTRY_TOKEN` env would let ops flip auth modes across environments via helm values without a code change. **Proposed fix:** add `MCA_USE_GCP_AUTH` (truthy: `1`/`true`/`yes`/`on`, case-insensitive) to `_load_from_env`, mapping to `MCAConfig.use_gcp_auth`. Precedence: explicit `use_gcp_auth=True` kwarg wins over env; env wins over the default `False`.
+- **Init failure surfacing is the caller's problem.** Every consumer app that wraps `MCAClient(...)` in a `try/except Exception` re-implements the same pattern: log at ERROR with a distinctive marker, optionally re-raise for fail-fast in prod. The 2026-07-14 outage is a case study — the app-level `try/except` swallowed a `ConfigurationError` with only a `logger.warning`, and the eventual diagnosis required an SDK-source-code trace to figure out which of four possible failure modes had actually fired. **Proposed fix:** expose an `MCA_STRICT_INIT` env var (or equivalent `MCAClient(strict_init=True)` kwarg) that turns SDK-detected init-time errors into a re-raise instead of the current behavior. Complementary: add a positive-signal API (`MCAClient.is_telemetry_active() -> bool`) so consumer health checks can wire "telemetry disabled" into the same probe surface without duplicating the init try/except state. Reduces the swallow-and-log pattern to a two-line block in every consumer.
+
+### Fixed
+- `_validate_registry_requirements` in `mca_sdk/core/client.py` no
+  longer fail-closes with `ConfigurationError` when `registry_url`
+  resolves to a private endpoint (RFC1918 10/8, 172.16/12, 192.168/16,
+  IPv4 loopback 127/8, IPv6 loopback `::1`, IPv6 ULA `fc00::/7`, or the
+  literal `localhost`) and none of `registry_token` / `use_gcp_auth` /
+  `tls_insecure_registry` is set. IPv4 link-local `169.254/16` and IPv6
+  link-local `fe80::/10` are deliberately **excluded** from the soft-
+  default because those ranges host cloud instance-metadata endpoints
+  (IMDS at `169.254.169.254` on GCE / GKE / EKS / AKS) — auto-enabling
+  `tls_insecure_registry` for such a URL would create an SSRF-adjacent
+  footgun when `MCA_REGISTRY_URL` is derived from user-controlled
+  config. The SDK now auto-selects `tls_insecure_registry=True` for the
+  supported private classes and emits a WARNING that names the parsed
+  host (URL redacted) plus the three explicit-auth kwargs the caller
+  could set for production. The WARNING fires on **every** construction
+  even when `tls_insecure_registry` is already True (defense against
+  the shared-`MCAConfig` factory pattern that used to silently short-
+  circuit the branch on the second init). Public URLs still raise the pre-fix
+  `ConfigurationError` unchanged. New env
+  `MCA_STRICT_REGISTRY_AUTH=true` (config field
+  `strict_registry_auth: bool = False`) restores the pre-fix
+  fail-closed behavior for operators who want the old contract.
+  Endpoint classification lives in a new module
+  `mca_sdk/utils/network.py` (`_is_private_endpoint`,
+  `_extract_host`) so IPv6 ULA and link-local support are covered by
+  Python's stdlib `ipaddress` module — no bespoke CIDR parsing.
+  Motivating incident: deployment-agent 2026-07-14 17:38 UTC outage —
+  `MCA_REGISTRY_URL=https://10.164.76.59` was added without auth
+  kwargs; the caller's `try/except Exception` swallowed the
+  `ConfigurationError` and left `_client=None` for the whole pod
+  lifetime. Regression tests at
+  `tests/unit/test_registry_rfc1918_softdefault.py` cover 10 private
+  and 3 public URL classes, the strict-mode escape hatch, and the
+  explicit-token bypass. Fixes tk_dcaefe49fd364823.
+- `record_policy_violation` in `mca_sdk/core/client.py:4886` now
+  increments a new `agent.policy_violations_total` counter (created in
+  `_initialize_agentic_metrics` at `client.py:2256-2324`) on every call.
+  Previously the method emitted only a `genai.agent.policy_violation`
+  span event, forcing the V1 agentic dashboard's "Unauthorized actions"
+  tile to scan trace/span-event storage — slow and coupled to trace
+  retention. The counter fires unconditionally, including when there is
+  no active recording span (auth checks, request validators outside a
+  goal context), so consumers can build the rate metric purely from the
+  metric surface. The span event is preserved unchanged inside the
+  existing `is_recording()` branch. Metric label is `policy_type`,
+  sanitized to 100 chars (matching the existing span-event
+  sanitization). Documented in `docs/data-models.md` and
+  `docs/integration-guides/agentic-models.md`. Fixes tk_5faf104898aa4db3.
+- `_resolve_temporality_preference` in `mca_sdk/core/providers.py:389-476`
+  now auto-selects CUMULATIVE for sum-type instruments on Cloud Run
+  (detected via the `K_SERVICE` env var the runtime injects on every
+  container) when neither `MCA_OTEL_TEMPORALITY` nor
+  `OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE` is set. Root
+  cause: sum-type instruments (Counter, UpDownCounter, ObservableCounter,
+  ObservableUpDownCounter) previously exported as DELTA unconditionally
+  — `SumAggregator.collect()` resets to 0 on every export cycle. On
+  Cloud Run Gen2 with `cpu-throttling=true` and `minScale=0`, the
+  `PeriodicExportingMetricReader`'s daemon worker is starved during
+  the post-request throttle window, `BufferedOTLPMetricExporter`
+  exhausts retries at three drop points (`exporters.py:178-188`,
+  `exporters.py:207`, `exporters.py:518-526`), and every dropped delta
+  is permanently lost — counters stuck at 0 in Cloud Monitoring despite
+  verified `.add(1)` calls. Cumulative histograms survive because they
+  don't reset on collect. Under CUMULATIVE, a dropped export just
+  means the next export re-sends the same absolute value. Diagnostic
+  visibility added: `setup_meter_provider` in `providers.py:943` now
+  emits a startup INFO log with `service.instance.id`, resolved
+  counter temporality, and Cloud Run detection state — closes the
+  "can't tell which hypothesis is real from container logs" gap.
+  Overrides preserved: `MCA_OTEL_TEMPORALITY=delta` or
+  `OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE=delta` restore
+  DELTA behavior; `MCA_DISABLE_CLOUD_RUN_TEMPORALITY_AUTOTUNE=true`
+  disables the auto-tune branch entirely for operators whose
+  collector expects DELTA-only ingestion. No regression: GKE
+  long-running pods and local development (neither injects
+  `K_SERVICE`) continue to receive the DELTA default and are
+  unaffected by this change. Fixes tk_e65bfdf5eae7431c and
+  tk_f033a2d1f0cc4d34 (predictive counter
+  `emms_model_predictions_total` shared the same root cause and
+  is closed by the same autotune; regression test at
+  `tests/unit/test_predictive_counter_cumulative.py` pins the
+  `model.predictions_total` cumulative behavior specifically).
+- `_detect_staging_collector_endpoint` in `mca_sdk.config.settings` now
+  falls back to `http://localhost:4318` for any socket-layer failure,
+  not just `socket.gaierror`. The prior narrow except clause let
+  `pytest_socket`'s `SocketBlockedError` — which inherits from
+  `RuntimeError` and whose `__init__` calls `warnings.warn(...)` before
+  raising — escape the handler and abort `MCAConfig` construction under
+  the test conftest that sets `MCA_ENV=staging`. Under `pytest.ini`'s
+  `filterwarnings = error`, the `warnings.warn` is promoted to a raised
+  `UserWarning` before the outer `RuntimeError` surfaces; under a
+  relaxed warning filter, the raw `RuntimeError` surfaces instead.
+  Result: 34 pre-existing failures in `tests/unit/test_attribute_validator.py`
+  (and additional cascaded failures across the suite) that all shared
+  this single root cause. The docstring already promised graceful
+  fallback; the except clause now catches `(OSError, RuntimeError, Warning)`
+  to cover all three surfaces (`socket.gaierror` real DNS miss,
+  `SocketBlockedError` raw, and warning-promoted-to-error). Real
+  DNS-miss behavior is unchanged because `socket.gaierror` is a
+  subclass of `OSError`. Fixes tk_a2fac7d943ed4b22.
+- `secret_filter` no longer over-redacts metric labels whose key name
+  merely contains a credential word as a substring. The previous
+  `_key_matches_secret` used plain substring containment against a
+  fragment list that included bare `"token"`, so `token_type=prompt`
+  was wholesale-replaced with `[REDACTED:SECRET]`, collapsing every
+  `emms_*_tokens_total` series into a single indistinguishable label
+  and destroying the prompt-vs-completion breakdown at Cloud Monitoring
+  and ClickStack. The predicate now matches on a normalized key against
+  a two-tier vocabulary: `_SECRET_KEY_EXACT` (frozenset, whole-key
+  equality — covers bare credential words `password`, `passwd`,
+  `password_hash`, `secret`, `token`, `bearer`, `authorization`,
+  `credential`, `credentials` plus underscore-form and no-separator
+  compounds `api_key`, `apikey`, `private_key`, `access_key`,
+  `secret_key`, `signing_key`, `session_key`, `encryption_key`,
+  `apitoken`, `authtoken`, `accesstoken`, `refreshtoken`, `idtoken`,
+  `bearertoken`, `secretkey`, `accesskey`, `privatekey`, `signingkey`,
+  `sessionkey`, `encryptionkey`, `credentialsfile`, `credentialscache`,
+  `credentialsjson`) and `_SECRET_KEY_SUFFIXES` (underscore-anchored
+  `endswith` — covers compound keys `_password`, `_password_hash`,
+  `_passwd`, `_secret`, `_token`, `_bearer`, `_authorization`,
+  `_apikey`, `_api_key`, `_credential`, `_credentials`, `_private_key`,
+  `_access_key`, `_secret_key`, `_signing_key`, `_encryption_key`,
+  `_session_key`). A bare `_key` suffix is deliberately absent to
+  avoid over-matching legitimate labels like `partition_key`,
+  `sort_key`, `foreign_key`, `cache_key`, and `safe_key`. Key
+  normalization folds camelCase (`authToken` → `auth_token`),
+  hyphenated headers (`x-authorization`), dotted paths
+  (`oauth.access.token`), whitespace-separated
+  (`API KEY`), and leading/trailing whitespace into the same shape
+  before matching. Fixes tk_e02c93d75a9d4e78.
+- `vendor.tokens_total` (and the peer `*.embedding_requests_total`,
+  `*.embedding_tokens_total`) were being emitted as a Histogram in
+  addition to their pre-cached Counter, because `track_llm_completion`
+  and `track_llm_embedding` in `mca_sdk.integrations.genai` called
+  `record_metric` without `instrument_type="counter"` and inherited
+  the default `"histogram"`. Two instruments registered on the same
+  Meter under the same name caused a descriptor collision: Cloud
+  Monitoring locked the descriptor to whichever type it observed
+  first and rejected the other, and in ClickStack the same MetricName
+  split across `otel_metrics_sum` and `otel_metrics_histogram`. The
+  fix also passes `instrument_type="counter"` to the
+  `embedding_errors_total` call site; that name did not have a
+  pre-cached Counter to collide with, but it was being registered as
+  a histogram under a `*_total` name, which is misleading. `record_metric`
+  now also defensively rejects NaN/Inf and coerces integer-valued
+  floats (e.g. `1234.0` from a vendor-bridge `transform_metrics()`)
+  to `int` on the **counter** path only, so gauge/up_down_counter
+  precision on values like `accuracy=0.92` is preserved. Fixes
+  tk_4b30e228b16f4cae; supersedes the "out of scope" note in the
+  [0.8.27] entry.
+
+### Changed
+- **Metric shape** — `genai.tokens_total` (and its `vendor.*` /
+  `emms_*` renderings) is now emitted with two `token_type` values
+  only: `prompt` and `completion` (or `input` / `output` on the
+  Anthropic autolog path). The redundant `token_type="total"` bucket
+  that previously accompanied every request was removed so
+  `sum(genai.tokens_total)` without a `token_type` filter equals
+  actual tokens consumed. Prior behavior was
+  `sum = prompt + completion + total = 2 × actual`. Affected call
+  sites: `mca_sdk/integrations/genai.py:track_llm_completion`,
+  `mca_sdk/integrations/litellm_callback.py:log_success_event`,
+  `mca_sdk/integrations/langchain_callback.py`,
+  `mca_sdk/integrations/llamaindex_callback.py`,
+  `mca_sdk/autolog/_openai.py`, `mca_sdk/autolog/_anthropic.py`, and
+  the `sdk-examples/rest-api-wrapper-example.py` example. Instrument
+  descriptions and `MCALiteLLMCallback` docstrings updated to remove
+  the "always filter by token_type" warning. Consumer migration:
+  dashboards that filtered `token_type != "total"` can drop the
+  filter; dashboards that summed `token_type == "total"` alone must
+  switch to a filterless `sum by(...) (rate(genai_tokens_total[5m]))`
+  or equivalent. Regression coverage in
+  `tests/unit/test_genai_helpers.py`,
+  `tests/unit/test_litellm_callback.py`,
+  `tests/unit/test_genai_metric_naming.py`, and
+  `tests/unit/test_autolog_anthropic.py` — each asserts no emission
+  carries `token_type="total"`. Documented in
+  `docs/api-contracts.md` and
+  `docs/integration-guides/genai-models.md`. Fixes
+  tk_e9e917c20ce34be9.
+- `secret_filter` internals: `_SECRET_KEY_SUBSTRINGS` is replaced by
+  `_SECRET_KEY_EXACT` (frozenset) plus `_SECRET_KEY_SUFFIXES` (tuple);
+  `_SECRET_KEY_ALLOWLIST` is removed because every prior entry
+  (`prediction_id`, `association_id`, `user_id`, `session_id`,
+  `request_id`, `trace_id`, `span_id`) ends in `_id` and structurally
+  misses both the exact set and the suffix tuple — no bespoke carve-out
+  needed. Documented behavior change: `aws_access_key_id` (and any
+  bare `_id`-suffixed key) no longer redacts on the key name alone.
+  There is no downstream safety net for this class: AWS access-key
+  IDs (`AKIA…`, `ASIA…`, `AGPA…`, `AIDA…`) are 20-char alphanumeric
+  strings and match none of the `sk-*` / JWT / PEM value-shape
+  patterns. Callers that pass AWS-style identifiers as attribute
+  keys MUST either rename the key to a covered suffix
+  (e.g. `aws_access_key`) or scrub upstream. Value-shape redaction
+  is unchanged and still catches `sk-*`, JWT, and PEM material
+  stored under any key, so
+  callers that emit credential-shaped AWS keys as values are unaffected.
+  Callers depending on key-only redaction for AWS-style identifiers
+  should rename the key to end in a covered suffix or scrub upstream.
+
+### SDK version pinning (previously staged as 0.9.4)
+
+**Added**
+
+- `PinnedVersionError` exception in `mca_sdk.utils.exceptions`, exported
+  from the top-level `mca_sdk` package. Raised at MCAClient bootstrap
+  when the running SDK version does not satisfy a registry- or
+  config-declared pin.
+- `ModelConfig` (registry response) now carries optional
+  `pinned_sdk_version`, `sdk_version_spec`, and `sdk_compat_notes`
+  fields for per-model SDK version control. Backward compatible: older
+  registries that omit these keys parse cleanly with the fields
+  defaulting to None.
+- `MCAConfig` gains three fields: `sdk_version_enforcement`
+  (`"off"`|`"warn"`|`"fail"`, defaults to `"warn"` in all
+  environments; explicit opt-in via
+  `MCA_SDK_VERSION_ENFORCEMENT=fail`), `local_pinned_sdk_version`,
+  and `local_sdk_version_spec`. Corresponding env vars:
+  `MCA_SDK_VERSION_ENFORCEMENT`, `MCA_LOCAL_PINNED_SDK_VERSION`,
+  `MCA_LOCAL_SDK_VERSION_SPEC`. A future 0.10.0 release will
+  consider making `fail` the prod default after warn-mode
+  dashboards prove out.
+- `mca.registry.sdk_version_mismatch_total` counter on
+  RegistryTelemetry with attributes `expected`, `actual`, `source`,
+  `mode`. Emitted in both warn and fail modes when the running SDK
+  version does not satisfy the resolved pin.
+- `X-Mca-Sdk-Version` and `User-Agent: mca-sdk/{version} python/{X.Y}`
+  headers are attached to every registry HTTP request so the backend
+  can record which SDK version is calling.
+- New module `mca_sdk.compatibility` exposing
+  `MODEL_VERSION_SDK_MATRIX` and `lookup_spec(model_version)` as a
+  last-resort fallback when neither the registry nor local config
+  declares a pin. Ships empty; ops populates via PR review.
+
+**Changed**
+
+- `_validate_sdk_version` runs at MCAClient bootstrap between registry
+  hydration and configuration validation. Precedence: registry pin >
+  registry spec > config pin > config spec > shipped matrix.
+  Enforcement is opt-in (`sdk_version_enforcement` defaults to
+  `"warn"` outside prod so existing users see no change beyond a log
+  line on mismatch).
+- `packaging>=21.0` added to install_requires. Already a transitive
+  dep of pip; making it explicit is a safe promotion.
+
+**Notes**
+
+- Registry SDK-compat fields (`pinned_sdk_version`,
+  `sdk_version_spec`) that arrive as non-string types are logged
+  and dropped; the client does not fail.
+- PEP 440 prerelease semantics apply: a spec like `>=0.9,<0.11`
+  rejects `0.10.0.dev0`; declare prereleases explicitly (e.g.
+  `>=0.9.4rc1`) if you need them.
+- Exact pins compare by release tuple, so `0.9.4+editable` and
+  `0.9.4.post1` satisfy a pin of `0.9.4`.
 
 ## [0.9.3] - 2026-07-08
 
@@ -1455,4 +1837,4 @@ client = MCAClient(service_name="my-model")  # Works!
 
 [unreleased]: https://gitlab.com/bhsf/ai_ml/mca-sdk/compare/v0.8.0...HEAD
 [0.8.0]: https://gitlab.com/bhsf/ai_ml/mca-sdk/releases/tag/v0.8.0
-[0.1.0]: https://gitlab.com/bhsf/ai_ml/mca-sdk/releases/tag/v0.1.0
+[0.1.0]: https://gitlab.com/bhsf/ai_ml/mca-sdk/releases/tag/v0.1.0

{mca_sdk-0.9.4.dev1/mca_sdk.egg-info → mca_sdk-0.9.7}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mca-sdk
-Version: 0.9.4.dev1
+Version: 0.9.7
 Summary: Model Collector Agent SDK for OpenTelemetry monitoring
 Author-email: Baptist Health South Florida <ai-ml@baptisthealth.net>
 License: Apache-2.0
@@ -22,6 +22,7 @@ Requires-Dist: pyyaml>=6.0
 Requires-Dist: pybreaker>=1.4.1
 Requires-Dist: cryptography>=46.0.5
 Requires-Dist: requests>=2.31.0
+Requires-Dist: packaging>=21.0
 Provides-Extra: genai
 Requires-Dist: litellm>=1.17.0; extra == "genai"
 Provides-Extra: vendor
@@ -59,6 +60,7 @@ Requires-Dist: pytest-mock>=3.12.0; extra == "test"
 Requires-Dist: pytest-asyncio>=0.23.0; extra == "test"
 Requires-Dist: opentelemetry-exporter-prometheus>=0.48b0; extra == "test"
 Requires-Dist: ipython>=7.0; extra == "test"
+Requires-Dist: mistune>=3.0.0; extra == "test"
 Provides-Extra: dev
 Requires-Dist: black>=24.0.0; extra == "dev"
 Requires-Dist: mypy>=1.8.0; extra == "dev"
@@ -70,8 +72,10 @@ Requires-Dist: pytest-asyncio>=0.23.0; extra == "dev"
 Requires-Dist: opentelemetry-exporter-prometheus>=0.48b0; extra == "dev"
 Requires-Dist: pdoc>=14.0.0; extra == "dev"
 Requires-Dist: ipython>=7.0; extra == "dev"
+Requires-Dist: mistune>=3.0.0; extra == "dev"
 Provides-Extra: docs
 Requires-Dist: pdoc>=14.0.0; extra == "docs"
+Requires-Dist: opentelemetry-instrumentation>=0.48b0; extra == "docs"
 Provides-Extra: all
 Requires-Dist: litellm>=1.17.0; extra == "all"
 Requires-Dist: flask>=3.0.0; extra == "all"
@@ -289,6 +293,11 @@ make dev-setup
 python -m venv venv
 source venv/bin/activate
 pip install -e ".[dev]"
+
+# Install pre-commit hooks so black, ruff, and mypy run on every commit.
+# CI enforces the same checks (see .gitlab-ci.yml python:lint / python:typecheck),
+# so skipping this step will surface failures at MR time instead of commit time.
+pre-commit install
 ```
 
 ## Instrumentation Approaches
@@ -1150,11 +1159,8 @@ volumes:
 ### Running Tests
 
 ```bash
-# Run all tests with coverage (requires 85% coverage)
+# Run all tests with coverage
 make test
-
-# Run tests manually
-pytest tests/ -v --cov=mca-prototype/mca_sdk --cov-fail-under=85
 ```
 
 ### Code Quality & Linting
@@ -1428,6 +1434,7 @@ See `.env.example` for all available configuration options.
   - `agent.tool_latency_seconds` (histogram per tool)
   - `agent.human_interventions_total` (counter)
   - `agent.reasoning_steps_total` (counter)
+  - `agent.policy_violations_total` (counter with policy_type label)
 - Traces:
   - `agent.goal` (parent span for entire goal)
     - `agent.planning` (search strategy)

{mca_sdk-0.9.4.dev1 → mca_sdk-0.9.7}/README.md

@@ -198,6 +198,11 @@ make dev-setup
 python -m venv venv
 source venv/bin/activate
 pip install -e ".[dev]"
+
+# Install pre-commit hooks so black, ruff, and mypy run on every commit.
+# CI enforces the same checks (see .gitlab-ci.yml python:lint / python:typecheck),
+# so skipping this step will surface failures at MR time instead of commit time.
+pre-commit install
 ```
 
 ## Instrumentation Approaches
@@ -1059,11 +1064,8 @@ volumes:
 ### Running Tests
 
 ```bash
-# Run all tests with coverage (requires 85% coverage)
+# Run all tests with coverage
 make test
-
-# Run tests manually
-pytest tests/ -v --cov=mca-prototype/mca_sdk --cov-fail-under=85
 ```
 
 ### Code Quality & Linting
@@ -1337,6 +1339,7 @@ See `.env.example` for all available configuration options.
   - `agent.tool_latency_seconds` (histogram per tool)
   - `agent.human_interventions_total` (counter)
   - `agent.reasoning_steps_total` (counter)
+  - `agent.policy_violations_total` (counter with policy_type label)
 - Traces:
   - `agent.goal` (parent span for entire goal)
     - `agent.planning` (search strategy)