insightfactory-databricks-langgraph-tracer 1.0.0.dev5__py3-none-any.whl

databricks_langgraph_tracer/CHANGELOG.md

@@ -0,0 +1,176 @@
+# Changelog
+
+All notable changes to the Python package
+(`insightfactory-databricks-langgraph-tracer`) are
+documented here. The format is based on
+[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the project follows
+[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+See [`../typescript/CHANGELOG.md`](../typescript/CHANGELOG.md) for changes to the
+separately versioned TypeScript package.
+
+<!--
+Releasing: add changes under [Unreleased] as you go. To cut a release, move those
+items into a new `## [x.y.z] - YYYY-MM-DD` section, bump `version` in
+pyproject.toml, and add the version's link reference at the bottom. CI requires a
+matching `## [x.y.z]` section whenever pyproject.toml is version-bumped on a PR to
+main; pushing to main then tags `python-vx.y.z` and publishes a GitHub Release
+from this section.
+-->
+
+## [Unreleased]
+
+## [1.0.0] - 2026-09-01
+### Changed
+- Renamed the PyPI distribution from `databricks-langgraph-tracer` to
+  `insightfactory-databricks-langgraph-tracer`. The Python import package remains
+  `databricks_langgraph_tracer`, so application imports do not change.
+- Made the stable Python package available through public PyPI.
+- Raised the minimum supported MLflow version to 3.15.0 and LangChain version to
+  1.3.9 so the public release does not resolve versions with known vulnerabilities.
+- Added `ty` type checking to the Python development dependencies and CI gates,
+  matching the Insight Factory CLI and SDK toolchain.
+
+## [0.6.1] - 2026-07-29
+### Changed
+- Refreshed the vendored LiteLLM pricing catalog (2,719 → 2,984 entries), adding cost
+  coverage for the latest Foundry / Azure OpenAI models — notably the **GPT-5.6 series**
+  (`gpt-5.6`, `gpt-5.6-luna`, `gpt-5.6-sol`, `gpt-5.6-terra`, plus their `azure/us/` and
+  `azure/eu/` regional variants). Deployments named after these models now resolve to a
+  pricing key and are costed, rather than being tagged `cost.unknown_model`. The manual
+  `azure_ai/deepseek-v4-flash` / `azure_ai/deepseek-v4-pro` overrides survive the refresh
+  unchanged.
+- The refresh is not purely additive: upstream also revised rates for 21 non-Azure
+  entries (e.g. `sambanova/gpt-oss-120b`, `mistral/mistral-medium-latest`) and dropped
+  two `fireworks_ai/...whisper-v3*` keys. **No `azure/` or `azure_ai/` rate changed**, and
+  `resolve_pricing_key` only probes those two prefixes, so default-path costs are
+  unaffected; consumers passing an explicit pricing key may see reported costs move.
+
+### Known limitations
+- Long-context tiering is still not applied: `calculate_llm_cost` reads only the flat
+  per-token rates, never `input_cost_per_token_above_272k_tokens` or the `_priority`
+  rates. Pre-existing and affecting all of gpt-5.x, but easier to reach now that the
+  GPT-5.6 entries carry 1,050,000-token windows — a 400k-input call under-reports by
+  roughly half.
+
+## [0.6.0] - 2026-06-25
+### Added
+- `max_string_chars` option on `configure_databricks_tracing(...)` and
+  `DatabricksLangGraphTracer(...)` (env fallback `DATABRICKS_TRACING_MAX_STRING_CHARS`).
+  Opt-in (default off): when set, any plain-text string leaf longer than the
+  threshold in a span's inputs **or** outputs is replaced with a compact
+  `{"_truncated": true, "chars": N, "bytes": M, "preview": "…"}` placeholder, so
+  large text (not just multimodal bytes) cannot push a Unity Catalog trace past the
+  SQL inline read limit (issue #23). It caps string values only — never keys or
+  structural fields — and is a per-leaf mitigation, not a hard per-trace budget, so
+  pair it with consumer-side payload reduction for the heaviest spans. This is a
+  **distinct** knob from the removed multimodal `max_content_part_bytes`: that gated
+  inline binary payloads; this caps arbitrary text.
+
+### Changed
+- The text cap now also applies to span **outputs**, not only inputs — the spans that
+  breach the inline read limit in the reported case (`final_output` /
+  `aggregate_results`) are output-side. Multimodal payload externalization stays
+  **input-only**: outputs are arbitrary chain / tool JSON, so running the
+  `data`/`base64` probe on them would mis-strip legitimate non-binary output. Runs on
+  a copy; the live values are untouched.
+
+### Fixed
+- The `max_string_chars` cap now also reaches text nested inside **structured objects**
+  in span inputs/outputs — Pydantic models, dataclasses, tuples/namedtuples, and sets.
+  Previously the walk only descended through `str`/`dict`/`list`, so a long string on a
+  Pydantic model returned by a LangGraph node (e.g. `final_output` / `aggregate_results`)
+  escaped the cap while MLflow's serializer still expanded the model and recorded the
+  field in full. Such objects are now normalized to the same shape MLflow records (e.g.
+  `model_dump()`), so their leaves are capped too; normalization runs only when the cap
+  is set and is best-effort (a failure records the value as-is). (The TypeScript package
+  already JSON-normalizes inputs/outputs before the walk, so it was unaffected.)
+- Autolog mode: the post-`end()` attribute force-write (`mlflow.llm.*` enrichment)
+  now works on `opentelemetry-sdk >= 1.43`, which made `BoundedAttributes` immutable
+  (`__setitem__` raises). The helper writes into the mapping's backing `_dict`
+  instead, restoring the enriched reduced schema on the mlflow floor.
+
+## [0.5.0] - 2026-06-24
+### Added
+- `content_ref_resolver` option on `configure_databricks_tracing(...)` and
+  `DatabricksLangGraphTracer(...)`. It is called once per multimodal content part
+  (image / PDF / file) with the part and a `ContentPartContext` (part index/page,
+  inline byte length, and a shallow copy of the run metadata), and returns the
+  reference dict to store in the trace — e.g. a Unity Catalog volume path — or
+  `None` to fall back to the placeholder (issue #21).
+
+### Changed
+- Inline base64 payloads of multimodal parts are now **never** stored in the trace:
+  they are always stripped before recording and replaced with a lightweight
+  reference (a `content_ref_resolver` result, otherwise a
+  `{"type": ..., "_omitted": true, "bytes": N}` placeholder), keeping Unity Catalog
+  trace reads under the SQL inline limit. Remote `http(s)://` image URLs are kept
+  verbatim. The transform runs on a copy, so the cache-stable message is untouched.
+
+### Removed
+- The per-content-part byte cap, including the
+  `DATABRICKS_TRACING_MAX_CONTENT_PART_BYTES` env var and the
+  `max_content_part_bytes` option. Inline payloads are now always externalized, so
+  the cap is obsolete.
+
+### Security
+- A resolver result that would re-introduce an inline payload — a `data:` URI, a
+  recognized base64 content part, or the part's raw base64 echoed back — is rejected
+  and the placeholder is used instead, guaranteeing no inline image bytes are
+  persisted. `ctx.metadata` is a shallow copy, so a resolver cannot corrupt run
+  state.
+
+## [0.4.0] - 2026-06-22
+### Added
+- Capture multimodal chat-model inputs (image / PDF / file content blocks) on LLM
+  spans.
+
+### Fixed
+- Keep multimodal content inline when exporting to Unity Catalog table-backed
+  experiments, instead of dropping it to an attachment reference.
+
+## [0.3.0] - 2026-06-17
+### Added
+- Trace-level per-model cost and token rollup, written to the `cost.by_model` tag
+  on the live trace before the root span ends. Unpriceable-but-resolved models
+  still contribute their token split (zero cost) so the totals stay complete.
+
+## [0.2.2] - 2026-06-09
+### Changed
+- Emit OpenInference-style `llm.*` span keys so the Databricks dashboard cost
+  charts read token usage and cost correctly.
+
+## [0.2.1] - 2026-06-04
+### Fixed
+- Chunk the Unity Catalog OTLP span export so payloads stay under the 10 MiB
+  ingest cap.
+
+## [0.2.0] - 2026-06-02
+### Changed
+- Harden Unity Catalog configuration guards and move to experiment-id-only
+  configuration.
+
+## [0.1.0] - 2026-06-01
+### Added
+- Initial release: shared LangGraph → Databricks MLflow tracer built on a custom
+  `BaseTracer` (not autolog), writing all trace fields and the cost rollup on the
+  **live** trace before the root span ends, with fail-fast configuration and an
+  autolog reduced-schema fallback. Per-model pricing comes from a vendored LiteLLM
+  catalog with overrides and a refresh script.
+
+<!-- Historical links (0.1.0-0.4.0) use commit SHAs: those release tags were
+reserved by the repo's immutable-releases policy and cannot be created. Restore
+python-vX.Y.Z tag links if the names are ever reclaimed. 0.5.0's tag is created
+out-of-band (its bump predates the release workflow); 0.5.1+ tags are minted by
+release.yml on push to main. -->
+[Unreleased]: https://github.com/insightfactory-ai/if_s_langraph_mlflow_tracer/compare/python-v1.0.0...HEAD
+[1.0.0]: https://github.com/insightfactory-ai/if_s_langraph_mlflow_tracer/compare/python-v0.6.1...python-v1.0.0
+[0.6.1]: https://github.com/insightfactory-ai/if_s_langraph_mlflow_tracer/compare/python-v0.6.0...python-v0.6.1
+[0.6.0]: https://github.com/insightfactory-ai/if_s_langraph_mlflow_tracer/compare/python-v0.5.0...python-v0.6.0
+[0.5.0]: https://github.com/insightfactory-ai/if_s_langraph_mlflow_tracer/compare/99462dbc95c43f20085cae0bfd7a2dc0aab319b1...python-v0.5.0
+[0.4.0]: https://github.com/insightfactory-ai/if_s_langraph_mlflow_tracer/compare/951a9299c4beffcdd23fc7442bd79b49e2d9b52f...99462dbc95c43f20085cae0bfd7a2dc0aab319b1
+[0.3.0]: https://github.com/insightfactory-ai/if_s_langraph_mlflow_tracer/compare/1e4130acb98eb947f65302e060d6a017da0f2feb...951a9299c4beffcdd23fc7442bd79b49e2d9b52f
+[0.2.2]: https://github.com/insightfactory-ai/if_s_langraph_mlflow_tracer/compare/1a868d88a6227a0528e5725c49b63848f0e03aa9...1e4130acb98eb947f65302e060d6a017da0f2feb
+[0.2.1]: https://github.com/insightfactory-ai/if_s_langraph_mlflow_tracer/compare/03e34a320a8e90e046d574dab0e58cdfe58ab738...1a868d88a6227a0528e5725c49b63848f0e03aa9
+[0.2.0]: https://github.com/insightfactory-ai/if_s_langraph_mlflow_tracer/compare/83c62e28dbd04ef8e987689983e1e5a53e9d204f...03e34a320a8e90e046d574dab0e58cdfe58ab738
+[0.1.0]: https://github.com/insightfactory-ai/if_s_langraph_mlflow_tracer/commit/83c62e28dbd04ef8e987689983e1e5a53e9d204f

databricks_langgraph_tracer/__init__.py

@@ -0,0 +1,28 @@
+"""Shared LangGraph -> Databricks MLflow tracer (Python).
+
+Public API. No auto-bootstrap at import (design "Patterns to NOT follow") —
+consumers call ``configure_databricks_tracing`` explicitly and attach the
+callbacks returned by ``get_tracing_callbacks``.
+"""
+
+from .errors import DatabricksTracingConfigurationError
+from .mlflow_utils import (
+    TracingConfiguration,
+    configure_databricks_tracing,
+    get_tracing_callbacks,
+)
+from .tracer import (
+    ContentPartContext,
+    ContentRefResolver,
+    DatabricksLangGraphTracer,
+)
+
+__all__ = [
+    "ContentPartContext",
+    "ContentRefResolver",
+    "DatabricksLangGraphTracer",
+    "DatabricksTracingConfigurationError",
+    "TracingConfiguration",
+    "configure_databricks_tracing",
+    "get_tracing_callbacks",
+]

databricks_langgraph_tracer/_generated_keys.py

@@ -0,0 +1,43 @@
+"""Constant key names for the shared trace schema.
+
+AUTO-GENERATED from ``schema/trace-schema.json`` — DO NOT EDIT BY HAND.
+Regenerate with ``uv run python scripts/generate_keys.py``;
+validate with ``uv run python scripts/generate_keys.py --check``.
+"""
+
+from typing import Final
+
+
+class TraceTagKey:
+    SOURCE: Final[str] = "source"
+    COST_UNKNOWN_MODEL: Final[str] = "cost.unknown_model"
+    COST_BY_MODEL: Final[str] = "cost.by_model"
+    LANGGRAPH_RUN_ID: Final[str] = "langgraph.run_id"
+    LANGGRAPH_GRAPH_ID: Final[str] = "langgraph.graph_id"
+    LANGGRAPH_ENV: Final[str] = "langgraph.env"
+    LANGGRAPH_API_REVISION: Final[str] = "langgraph.api_revision"
+    LANGGRAPH_THREAD_ID: Final[str] = "langgraph.thread_id"
+
+
+class TraceMetadataKey:
+    TRACE_COST: Final[str] = "mlflow.trace.cost"
+    TRACE_SESSION: Final[str] = "mlflow.trace.session"
+    TRACE_USER: Final[str] = "mlflow.trace.user"
+
+
+class SpanAttrKey:
+    LLM_COST: Final[str] = "mlflow.llm.cost"
+    LLM_MODEL: Final[str] = "mlflow.llm.model"
+    LLM_PROVIDER: Final[str] = "mlflow.llm.provider"
+    TOKEN_USAGE: Final[str] = "mlflow.chat.tokenUsage"
+    INPUT_TOKENS_CACHED: Final[str] = "gen_ai.usage.input_tokens.cached"
+    INPUT_TOKENS_CACHE_CREATION: Final[str] = "gen_ai.usage.input_tokens.cache_creation"
+    OUTPUT_TOKENS_REASONING: Final[str] = "gen_ai.usage.output_tokens.reasoning"
+    INPUT_TOKENS_CACHED_COST: Final[str] = "gen_ai.usage.input_tokens.cached_cost"
+    INPUT_TOKENS_CACHE_CREATION_COST: Final[str] = "gen_ai.usage.input_tokens.cache_creation_cost"
+    LLM_MODEL_NAME: Final[str] = "llm.model_name"
+    LLM_MODEL_PROVIDER: Final[str] = "llm.model_provider"
+    LLM_PROMPT_TOKENS_COST: Final[str] = "llm.usage.prompt_tokens_cost"
+    LLM_COMPLETION_TOKENS_COST: Final[str] = "llm.usage.completion_tokens_cost"
+    LANGGRAPH_NODE: Final[str] = "langgraph.node"
+    LANGGRAPH_STEP: Final[str] = "langgraph.step"

databricks_langgraph_tracer/aggregate.py

@@ -0,0 +1,60 @@
+"""Per-root cost accumulator.
+
+MLflow aggregates ``mlflow.chat.tokenUsage`` span attributes itself, but it has
+no cost aggregation — so the tracer tracks running cost totals here per root run
+and flushes the reserved 3-key ``mlflow.trace.cost`` metadata on root end. Also
+collects the set of unknown-model names for the ``cost.unknown_model`` trace tag.
+
+Only the reserved 3-key totals roll up to the trace: the cache-read /
+cache-creation cost split persists **per-span** as non-reserved attrs (Option B),
+so it is not accumulated here.
+
+A per-model breakdown (``by_model``) is *also* accumulated here for the
+``cost.by_model`` trace tag: model identity is a span-level attribute and a
+single trace mixes several models, so per-model cost/token totals don't exist at
+the trace level otherwise. Unlike the reserved 3-key rollup, the per-model bucket
+also carries **tokens** — MLflow's native token aggregation is whole-trace, not
+per-model, so the tracer must accumulate them itself (issue #13).
+"""
+
+from dataclasses import dataclass, field
+
+
+@dataclass
+class ModelRollup:
+    """Per-model cost + token totals for one root trace (the ``cost.by_model`` map value).
+
+    Cost fields reconcile with the reserved ``mlflow.trace.cost`` rollup; token
+    fields reconcile with the whole-trace ``mlflow.chat.tokenUsage``. A
+    resolved-but-unpriceable model still contributes tokens (with zero cost) so
+    the token split stays complete.
+    """
+
+    input_cost: float = 0.0
+    output_cost: float = 0.0
+    total_cost: float = 0.0
+    input_tokens: int = 0
+    output_tokens: int = 0
+    cache_read_input_tokens: int = 0
+    cache_creation_input_tokens: int = 0
+
+
+@dataclass
+class TraceAggregate:
+    root_trace_id: str
+    input_cost: float = 0.0
+    output_cost: float = 0.0
+    total_cost: float = 0.0
+    unknown_models: set[str] = field(default_factory=set)
+    has_any_cost: bool = False
+    # Per-model cost/token breakdown, keyed by the resolved model identity
+    # (``mlflow.llm.model``); flushed as the ``cost.by_model`` trace tag.
+    by_model: dict[str, ModelRollup] = field(default_factory=dict)
+
+    def model_bucket(self, model: str) -> ModelRollup:
+        """Return the per-model rollup for ``model``, creating it on first use."""
+        bucket = self.by_model.get(model)
+        if bucket is None:
+            bucket = ModelRollup()
+            self.by_model[model] = bucket
+        return bucket