insightfactory-databricks-langgraph-tracer 1.0.0.dev5__tar.gz

insightfactory_databricks_langgraph_tracer-1.0.0.dev5/.gitignore

@@ -0,0 +1,32 @@
+# Python
+__pycache__/
+*.py[cod]
+.venv/
+*.egg-info/
+.pytest_cache/
+.ruff_cache/
+build/
+dist/
+
+# MLflow local stores
+mlruns/
+
+# Scratch space (databricks-traces skill scripts, ad-hoc verification)
+.tmp/
+mlartifacts/
+mlflow.db
+
+# Node / TypeScript (Phase 7)
+node_modules/
+*.tsbuildinfo
+# Yarn 4 (node-modules linker; yarn provided via corepack, not committed)
+.yarn/
+.pnp.*
+
+# Secrets / local env
+.env
+.env.*
+!.env.example
+
+# OS / editor
+.DS_Store

insightfactory_databricks_langgraph_tracer-1.0.0.dev5/CHANGELOG.md

@@ -0,0 +1,176 @@
+# Changelog
+
+All notable changes to the Python package
+(`insightfactory-databricks-langgraph-tracer`) are
+documented here. The format is based on
+[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the project follows
+[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+See [`../typescript/CHANGELOG.md`](../typescript/CHANGELOG.md) for changes to the
+separately versioned TypeScript package.
+
+<!--
+Releasing: add changes under [Unreleased] as you go. To cut a release, move those
+items into a new `## [x.y.z] - YYYY-MM-DD` section, bump `version` in
+pyproject.toml, and add the version's link reference at the bottom. CI requires a
+matching `## [x.y.z]` section whenever pyproject.toml is version-bumped on a PR to
+main; pushing to main then tags `python-vx.y.z` and publishes a GitHub Release
+from this section.
+-->
+
+## [Unreleased]
+
+## [1.0.0] - 2026-09-01
+### Changed
+- Renamed the PyPI distribution from `databricks-langgraph-tracer` to
+  `insightfactory-databricks-langgraph-tracer`. The Python import package remains
+  `databricks_langgraph_tracer`, so application imports do not change.
+- Made the stable Python package available through public PyPI.
+- Raised the minimum supported MLflow version to 3.15.0 and LangChain version to
+  1.3.9 so the public release does not resolve versions with known vulnerabilities.
+- Added `ty` type checking to the Python development dependencies and CI gates,
+  matching the Insight Factory CLI and SDK toolchain.
+
+## [0.6.1] - 2026-07-29
+### Changed
+- Refreshed the vendored LiteLLM pricing catalog (2,719 → 2,984 entries), adding cost
+  coverage for the latest Foundry / Azure OpenAI models — notably the **GPT-5.6 series**
+  (`gpt-5.6`, `gpt-5.6-luna`, `gpt-5.6-sol`, `gpt-5.6-terra`, plus their `azure/us/` and
+  `azure/eu/` regional variants). Deployments named after these models now resolve to a
+  pricing key and are costed, rather than being tagged `cost.unknown_model`. The manual
+  `azure_ai/deepseek-v4-flash` / `azure_ai/deepseek-v4-pro` overrides survive the refresh
+  unchanged.
+- The refresh is not purely additive: upstream also revised rates for 21 non-Azure
+  entries (e.g. `sambanova/gpt-oss-120b`, `mistral/mistral-medium-latest`) and dropped
+  two `fireworks_ai/...whisper-v3*` keys. **No `azure/` or `azure_ai/` rate changed**, and
+  `resolve_pricing_key` only probes those two prefixes, so default-path costs are
+  unaffected; consumers passing an explicit pricing key may see reported costs move.
+
+### Known limitations
+- Long-context tiering is still not applied: `calculate_llm_cost` reads only the flat
+  per-token rates, never `input_cost_per_token_above_272k_tokens` or the `_priority`
+  rates. Pre-existing and affecting all of gpt-5.x, but easier to reach now that the
+  GPT-5.6 entries carry 1,050,000-token windows — a 400k-input call under-reports by
+  roughly half.
+
+## [0.6.0] - 2026-06-25
+### Added
+- `max_string_chars` option on `configure_databricks_tracing(...)` and
+  `DatabricksLangGraphTracer(...)` (env fallback `DATABRICKS_TRACING_MAX_STRING_CHARS`).
+  Opt-in (default off): when set, any plain-text string leaf longer than the
+  threshold in a span's inputs **or** outputs is replaced with a compact
+  `{"_truncated": true, "chars": N, "bytes": M, "preview": "…"}` placeholder, so
+  large text (not just multimodal bytes) cannot push a Unity Catalog trace past the
+  SQL inline read limit (issue #23). It caps string values only — never keys or
+  structural fields — and is a per-leaf mitigation, not a hard per-trace budget, so
+  pair it with consumer-side payload reduction for the heaviest spans. This is a
+  **distinct** knob from the removed multimodal `max_content_part_bytes`: that gated
+  inline binary payloads; this caps arbitrary text.
+
+### Changed
+- The text cap now also applies to span **outputs**, not only inputs — the spans that
+  breach the inline read limit in the reported case (`final_output` /
+  `aggregate_results`) are output-side. Multimodal payload externalization stays
+  **input-only**: outputs are arbitrary chain / tool JSON, so running the
+  `data`/`base64` probe on them would mis-strip legitimate non-binary output. Runs on
+  a copy; the live values are untouched.
+
+### Fixed
+- The `max_string_chars` cap now also reaches text nested inside **structured objects**
+  in span inputs/outputs — Pydantic models, dataclasses, tuples/namedtuples, and sets.
+  Previously the walk only descended through `str`/`dict`/`list`, so a long string on a
+  Pydantic model returned by a LangGraph node (e.g. `final_output` / `aggregate_results`)
+  escaped the cap while MLflow's serializer still expanded the model and recorded the
+  field in full. Such objects are now normalized to the same shape MLflow records (e.g.
+  `model_dump()`), so their leaves are capped too; normalization runs only when the cap
+  is set and is best-effort (a failure records the value as-is). (The TypeScript package
+  already JSON-normalizes inputs/outputs before the walk, so it was unaffected.)
+- Autolog mode: the post-`end()` attribute force-write (`mlflow.llm.*` enrichment)
+  now works on `opentelemetry-sdk >= 1.43`, which made `BoundedAttributes` immutable
+  (`__setitem__` raises). The helper writes into the mapping's backing `_dict`
+  instead, restoring the enriched reduced schema on the mlflow floor.
+
+## [0.5.0] - 2026-06-24
+### Added
+- `content_ref_resolver` option on `configure_databricks_tracing(...)` and
+  `DatabricksLangGraphTracer(...)`. It is called once per multimodal content part
+  (image / PDF / file) with the part and a `ContentPartContext` (part index/page,
+  inline byte length, and a shallow copy of the run metadata), and returns the
+  reference dict to store in the trace — e.g. a Unity Catalog volume path — or
+  `None` to fall back to the placeholder (issue #21).
+
+### Changed
+- Inline base64 payloads of multimodal parts are now **never** stored in the trace:
+  they are always stripped before recording and replaced with a lightweight
+  reference (a `content_ref_resolver` result, otherwise a
+  `{"type": ..., "_omitted": true, "bytes": N}` placeholder), keeping Unity Catalog
+  trace reads under the SQL inline limit. Remote `http(s)://` image URLs are kept
+  verbatim. The transform runs on a copy, so the cache-stable message is untouched.
+
+### Removed
+- The per-content-part byte cap, including the
+  `DATABRICKS_TRACING_MAX_CONTENT_PART_BYTES` env var and the
+  `max_content_part_bytes` option. Inline payloads are now always externalized, so
+  the cap is obsolete.
+
+### Security
+- A resolver result that would re-introduce an inline payload — a `data:` URI, a
+  recognized base64 content part, or the part's raw base64 echoed back — is rejected
+  and the placeholder is used instead, guaranteeing no inline image bytes are
+  persisted. `ctx.metadata` is a shallow copy, so a resolver cannot corrupt run
+  state.
+
+## [0.4.0] - 2026-06-22
+### Added
+- Capture multimodal chat-model inputs (image / PDF / file content blocks) on LLM
+  spans.
+
+### Fixed
+- Keep multimodal content inline when exporting to Unity Catalog table-backed
+  experiments, instead of dropping it to an attachment reference.
+
+## [0.3.0] - 2026-06-17
+### Added
+- Trace-level per-model cost and token rollup, written to the `cost.by_model` tag
+  on the live trace before the root span ends. Unpriceable-but-resolved models
+  still contribute their token split (zero cost) so the totals stay complete.
+
+## [0.2.2] - 2026-06-09
+### Changed
+- Emit OpenInference-style `llm.*` span keys so the Databricks dashboard cost
+  charts read token usage and cost correctly.
+
+## [0.2.1] - 2026-06-04
+### Fixed
+- Chunk the Unity Catalog OTLP span export so payloads stay under the 10 MiB
+  ingest cap.
+
+## [0.2.0] - 2026-06-02
+### Changed
+- Harden Unity Catalog configuration guards and move to experiment-id-only
+  configuration.
+
+## [0.1.0] - 2026-06-01
+### Added
+- Initial release: shared LangGraph → Databricks MLflow tracer built on a custom
+  `BaseTracer` (not autolog), writing all trace fields and the cost rollup on the
+  **live** trace before the root span ends, with fail-fast configuration and an
+  autolog reduced-schema fallback. Per-model pricing comes from a vendored LiteLLM
+  catalog with overrides and a refresh script.
+
+<!-- Historical links (0.1.0-0.4.0) use commit SHAs: those release tags were
+reserved by the repo's immutable-releases policy and cannot be created. Restore
+python-vX.Y.Z tag links if the names are ever reclaimed. 0.5.0's tag is created
+out-of-band (its bump predates the release workflow); 0.5.1+ tags are minted by
+release.yml on push to main. -->
+[Unreleased]: https://github.com/insightfactory-ai/if_s_langraph_mlflow_tracer/compare/python-v1.0.0...HEAD
+[1.0.0]: https://github.com/insightfactory-ai/if_s_langraph_mlflow_tracer/compare/python-v0.6.1...python-v1.0.0
+[0.6.1]: https://github.com/insightfactory-ai/if_s_langraph_mlflow_tracer/compare/python-v0.6.0...python-v0.6.1
+[0.6.0]: https://github.com/insightfactory-ai/if_s_langraph_mlflow_tracer/compare/python-v0.5.0...python-v0.6.0
+[0.5.0]: https://github.com/insightfactory-ai/if_s_langraph_mlflow_tracer/compare/99462dbc95c43f20085cae0bfd7a2dc0aab319b1...python-v0.5.0
+[0.4.0]: https://github.com/insightfactory-ai/if_s_langraph_mlflow_tracer/compare/951a9299c4beffcdd23fc7442bd79b49e2d9b52f...99462dbc95c43f20085cae0bfd7a2dc0aab319b1
+[0.3.0]: https://github.com/insightfactory-ai/if_s_langraph_mlflow_tracer/compare/1e4130acb98eb947f65302e060d6a017da0f2feb...951a9299c4beffcdd23fc7442bd79b49e2d9b52f
+[0.2.2]: https://github.com/insightfactory-ai/if_s_langraph_mlflow_tracer/compare/1a868d88a6227a0528e5725c49b63848f0e03aa9...1e4130acb98eb947f65302e060d6a017da0f2feb
+[0.2.1]: https://github.com/insightfactory-ai/if_s_langraph_mlflow_tracer/compare/03e34a320a8e90e046d574dab0e58cdfe58ab738...1a868d88a6227a0528e5725c49b63848f0e03aa9
+[0.2.0]: https://github.com/insightfactory-ai/if_s_langraph_mlflow_tracer/compare/83c62e28dbd04ef8e987689983e1e5a53e9d204f...03e34a320a8e90e046d574dab0e58cdfe58ab738
+[0.1.0]: https://github.com/insightfactory-ai/if_s_langraph_mlflow_tracer/commit/83c62e28dbd04ef8e987689983e1e5a53e9d204f

insightfactory_databricks_langgraph_tracer-1.0.0.dev5/LICENSE

@@ -0,0 +1,5 @@
+Copyright 2026 insightfactory.ai. All rights reserved.
+
+This software is proprietary. No permission is granted to use, copy, modify,
+distribute, sublicense, or sell it except under a separate written agreement
+with insightfactory.ai.

insightfactory_databricks_langgraph_tracer-1.0.0.dev5/PKG-INFO

@@ -0,0 +1,224 @@
+Metadata-Version: 2.5
+Name: insightfactory-databricks-langgraph-tracer
+Version: 1.0.0.dev5
+Summary: LangGraph tracing for Databricks MLflow
+Project-URL: Homepage, https://insightfactory.ai
+Author-email: "insightfactory.ai" <support@insightfactory.ai>
+License-Expression: LicenseRef-Proprietary
+License-File: LICENSE
+License-File: THIRD_PARTY_NOTICES
+Keywords: databricks,langchain,langgraph,mlflow,tracing
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Intended Audience :: Developers
+Classifier: License :: Other/Proprietary License
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Typing :: Typed
+Requires-Python: <3.13,>=3.12
+Requires-Dist: databricks-sdk>=0.82.0
+Requires-Dist: langchain-core>=1.0.0
+Requires-Dist: langchain>=1.3.9
+Requires-Dist: langgraph>=1.0.0
+Requires-Dist: mlflow<4.0.0,>=3.15.0
+Description-Content-Type: text/markdown
+
+# insightfactory-databricks-langgraph-tracer
+
+LangGraph tracer for Databricks MLflow. It uses a custom LangChain `BaseTracer` to write
+trace tags, metadata, token usage, and cost rollups before the root span ends.
+
+## Install
+
+Published on PyPI:
+
+```bash
+uv add insightfactory-databricks-langgraph-tracer
+# or: pip install insightfactory-databricks-langgraph-tracer
+```
+
+Requires Python 3.12 and resolves `mlflow>=3.15.0,<4`.
+
+## Quickstart
+
+```python
+from databricks_langgraph_tracer import (
+    configure_databricks_tracing,
+    get_tracing_callbacks,
+)
+
+# 1. Bootstrap once at startup (reads env-first; kwargs override).
+configure_databricks_tracing(experiment_id="<mlflow-experiment-id>", source="my-agent")
+
+# 2. Attach the callbacks to your LangGraph / LangChain run.
+graph = build_graph().with_config({"callbacks": get_tracing_callbacks()})
+graph.invoke(state)
+```
+
+Traces appear in the configured Databricks MLflow experiment carrying the full shared schema:
+the `source` tag, session/user/thread metadata, per-span model / provider / token-usage + cost,
+the trace-level `mlflow.trace.cost` rollup, and the per-model `cost.by_model` cost/token rollup tag.
+
+## Configuration
+
+Env-first; any kwarg to `configure_databricks_tracing(...)` overrides the matching env var.
+
+| Setting | Env var | Notes |
+|---------|---------|-------|
+| Tracking URI | `MLFLOW_TRACKING_URI` | `databricks` or `databricks://<profile>` (required) |
+| Experiment | `MLFLOW_EXPERIMENT_ID` | by **id** only (required) |
+| Source tag | — | `source=` kwarg, default `langgraph` |
+| Multimodal refs | — | inline image/PDF/file bytes are externalized to a reference; `content_ref_resolver=` chooses it — see below |
+| Text cap | `DATABRICKS_TRACING_MAX_STRING_CHARS` | opt-in; `max_string_chars=` truncates over-long plain-text span content — see below |
+| Disable | `TESTING` / `BUILDING` = `true`, or `enabled=False` | no-op — the only non-raising path |
+| UC-backed tracing | `MLFLOW_TRACING_UC_BACKED=true` or `uc_tracing=True` | see below |
+| SQL warehouse | `MLFLOW_TRACING_SQL_WAREHOUSE_ID` → `DATABRICKS_WAREHOUSE_ID` | required when UC-backed |
+
+**Auth** (resolved by `databricks_utils`): a service principal
+(`DATABRICKS_HOST` / `DATABRICKS_CLIENT_ID` / `DATABRICKS_CLIENT_SECRET`) **or** a CLI profile
+(`DATABRICKS_CONFIG_PROFILE` / `profile=`).
+
+Missing required config **raises** `DatabricksTracingConfigurationError` (fail-fast) — disable
+explicitly for local/dev runs.
+
+### Unity-Catalog-backed tracing
+
+For experiments whose traces are stored in **Unity Catalog** `_otel_*` tables, set
+`uc_tracing=True` (or `MLFLOW_TRACING_UC_BACKED=true`) and provide a SQL warehouse. The library
+validates the warehouse, resolves the experiment's UC trace location from its binding tag, and
+passes it to `set_experiment` so spans actually persist to the `_otel_spans` table — without it,
+MLflow silently skips span export to UC. Classic (workspace) experiments need none of this and
+are the default. UC-backed tracing requires an MLflow release that provides the
+`UnityCatalog` trace-location API.
+
+### Multimodal inputs (image / PDF / file)
+
+Chat-model spans record their inputs as **structured messages**, preserving the
+multimodal content parts a graph sends to the model — OpenAI / LangChain
+`image_url`, OpenAI `file`, and Anthropic `image` / `document`. (Internally the
+tracer runs the LangChain `BaseTracer` in `original+chat` mode; the default would
+otherwise flatten chat messages to a text-only `prompts` string and drop every
+attachment.)
+
+The inline base64 of each such part is **never stored** in the trace: it is removed
+before recording and replaced with a lightweight reference, so the Unity Catalog
+trace tables stay readable (multi-MB data URIs previously pushed large invoice
+traces past the SQL inline read limit — issue #21). A remote `http(s)://` image URL
+is already a small reference, so this step keeps it verbatim (but see the text cap
+below — if enabled, it still truncates *any* string over its threshold, URLs
+included). The transform runs on a copy of the inputs, so the live message sent to
+the model is untouched and prompt caching is unaffected.
+
+By default a part becomes a `{"type": ..., "_omitted": true, "bytes": N}`
+placeholder. To store a meaningful reference instead — e.g. the Unity Catalog volume
+path the image was loaded from, so it can be re-fetched at runtime — pass a
+`content_ref_resolver`:
+
+```python
+from databricks_langgraph_tracer import (
+    ContentPartContext,
+    configure_databricks_tracing,
+)
+
+def image_ref(part: dict, ctx: ContentPartContext) -> dict | None:
+    # ctx.metadata is the run metadata — pass per-run data (e.g. a source volume
+    # path) via the invoke config's `metadata`, which propagates to the LLM run
+    # alongside langgraph_node etc.
+    path = ctx.metadata.get("encoded_images_path")
+    if path:
+        return {"type": part.get("type"), "ref": path, "page": ctx.index}
+    return None  # fall back to the default placeholder
+
+configure_databricks_tracing(experiment_id="...", content_ref_resolver=image_ref)
+
+# ... then carry the per-run ref data on the invoke config metadata:
+graph.invoke(state, config={
+    "callbacks": get_tracing_callbacks(),
+    "metadata": {"encoded_images_path": "/Volumes/cat/sch/vol/inv/pages.txt"},
+})
+```
+
+The resolver is called once per multimodal part with the part and a
+`ContentPartContext` — `index` (position within the message content array, i.e. the
+page number for a one-image-per-page invoice), `bytes` (the inline payload length),
+and the run `metadata`. Return a dict to store as the reference, or `None` for the
+default placeholder. It is also accepted by
+`DatabricksLangGraphTracer(content_ref_resolver=...)` for per-graph wiring.
+
+The library guarantees no inline image bytes are stored: if a resolver result
+re-introduces an inline payload (a `data:` URI, a recognized base64 content part, or
+the part's own payload echoed back under any key — anywhere in the returned object),
+it is rejected and the placeholder is used. Beyond that, keep the reference
+**compact** — a *fabricated* large string under a custom key is the consumer's
+responsibility (the library strips inline payloads but does not otherwise bound what
+a resolver returns).
+`ctx.metadata` is a shallow copy of the run metadata, so a resolver cannot corrupt run
+state by setting top-level keys (don't mutate its nested values, which are shared).
+
+### Capping large text (issue #23)
+
+Multimodal externalization handles inline *bytes*, but large **plain text** can also
+push a trace past the SQL inline read limit — e.g. a classification vocabulary or
+aggregated result set threaded through every fan-out span's inputs and outputs. Set
+`max_string_chars` (or `DATABRICKS_TRACING_MAX_STRING_CHARS`) to truncate it:
+
+```python
+configure_databricks_tracing(experiment_id="...", max_string_chars=50_000)
+```
+
+When set, any string **value** longer than the threshold — in a span's inputs *or*
+outputs — is replaced with a compact placeholder:
+
+```json
+{"_truncated": true, "chars": 812345, "bytes": 812345, "preview": "first 256 chars…"}
+```
+
+It is **opt-in / off by default** (generic truncation costs debuggability, so you
+choose the threshold). It caps string values only — never keys or structural fields —
+and runs on a copy, so the live messages are untouched. The cap also reaches text
+nested inside **Pydantic models / dataclasses / tuples** (e.g. a model returned by a
+node such as `final_output`), normalizing them to the same shape MLflow records. Also
+accepted by `DatabricksLangGraphTracer(max_string_chars=...)` for per-graph wiring.
+
+This is a **distinct** knob from the multimodal handling above: `content_ref_resolver`
+chooses references for inline image/PDF/file bytes; `max_string_chars` caps arbitrary
+text (and never truncates a resolver's reference). Because it is generic, it also
+truncates any other over-threshold string — including a remote `http(s)://` image URL
+the multimodal step keeps verbatim — so set the threshold comfortably above your
+reference / URL lengths. Note it is a **per-leaf** mitigation, not a hard per-trace
+byte budget — enough sub-threshold leaves can still sum past the limit — so for the
+heaviest spans also reduce what you record (pass ids/references through node state
+rather than full payloads).
+
+The threshold counts **characters** (code points here; the TypeScript package counts
+UTF-16 units, so the two can differ on non-BMP text), while the inline limit is in
+bytes — multibyte text can be up to ~4× larger in bytes than characters, so for
+CJK/emoji-heavy content size the cap below `limit / 4`. The placeholder's `bytes`
+field always reports the exact UTF-8 size of the original.
+
+### Autolog fallback
+
+`mode="autolog"` wires MLflow's built-in LangChain autologging plus compatibility patches and
+emits a **reduced** schema — everything except a tracer-computed
+`mlflow.trace.cost` rollup (the backend may still aggregate it server-side). The default
+`mode="tracer"` (custom `BaseTracer`) is the full-parity path.
+
+## Development
+
+```bash
+cd python
+uv sync                                            # latest allowed mlflow (ceiling)
+uv run pytest                                      # unit + lifecycle tests
+uv run ruff check src tests
+uv run ty check
+uv run python scripts/generate_keys.py --check     # schema/keys parity
+
+# mlflow floor matrix (CI runs both cells via UV_RESOLUTION):
+UV_RESOLUTION=lowest-direct uv sync && UV_RESOLUTION=lowest-direct uv run pytest
+```
+
+Tests use a local sqlite MLflow tracking backend; checks that need a live Databricks backend are
+marked `integration`.
+
+## Changelog
+
+See [`CHANGELOG.md`](CHANGELOG.md).

insightfactory_databricks_langgraph_tracer-1.0.0.dev5/README.md

@@ -0,0 +1,200 @@
+# insightfactory-databricks-langgraph-tracer
+
+LangGraph tracer for Databricks MLflow. It uses a custom LangChain `BaseTracer` to write
+trace tags, metadata, token usage, and cost rollups before the root span ends.
+
+## Install
+
+Published on PyPI:
+
+```bash
+uv add insightfactory-databricks-langgraph-tracer
+# or: pip install insightfactory-databricks-langgraph-tracer
+```
+
+Requires Python 3.12 and resolves `mlflow>=3.15.0,<4`.
+
+## Quickstart
+
+```python
+from databricks_langgraph_tracer import (
+    configure_databricks_tracing,
+    get_tracing_callbacks,
+)
+
+# 1. Bootstrap once at startup (reads env-first; kwargs override).
+configure_databricks_tracing(experiment_id="<mlflow-experiment-id>", source="my-agent")
+
+# 2. Attach the callbacks to your LangGraph / LangChain run.
+graph = build_graph().with_config({"callbacks": get_tracing_callbacks()})
+graph.invoke(state)
+```
+
+Traces appear in the configured Databricks MLflow experiment carrying the full shared schema:
+the `source` tag, session/user/thread metadata, per-span model / provider / token-usage + cost,
+the trace-level `mlflow.trace.cost` rollup, and the per-model `cost.by_model` cost/token rollup tag.
+
+## Configuration
+
+Env-first; any kwarg to `configure_databricks_tracing(...)` overrides the matching env var.
+
+| Setting | Env var | Notes |
+|---------|---------|-------|
+| Tracking URI | `MLFLOW_TRACKING_URI` | `databricks` or `databricks://<profile>` (required) |
+| Experiment | `MLFLOW_EXPERIMENT_ID` | by **id** only (required) |
+| Source tag | — | `source=` kwarg, default `langgraph` |
+| Multimodal refs | — | inline image/PDF/file bytes are externalized to a reference; `content_ref_resolver=` chooses it — see below |
+| Text cap | `DATABRICKS_TRACING_MAX_STRING_CHARS` | opt-in; `max_string_chars=` truncates over-long plain-text span content — see below |
+| Disable | `TESTING` / `BUILDING` = `true`, or `enabled=False` | no-op — the only non-raising path |
+| UC-backed tracing | `MLFLOW_TRACING_UC_BACKED=true` or `uc_tracing=True` | see below |
+| SQL warehouse | `MLFLOW_TRACING_SQL_WAREHOUSE_ID` → `DATABRICKS_WAREHOUSE_ID` | required when UC-backed |
+
+**Auth** (resolved by `databricks_utils`): a service principal
+(`DATABRICKS_HOST` / `DATABRICKS_CLIENT_ID` / `DATABRICKS_CLIENT_SECRET`) **or** a CLI profile
+(`DATABRICKS_CONFIG_PROFILE` / `profile=`).
+
+Missing required config **raises** `DatabricksTracingConfigurationError` (fail-fast) — disable
+explicitly for local/dev runs.
+
+### Unity-Catalog-backed tracing
+
+For experiments whose traces are stored in **Unity Catalog** `_otel_*` tables, set
+`uc_tracing=True` (or `MLFLOW_TRACING_UC_BACKED=true`) and provide a SQL warehouse. The library
+validates the warehouse, resolves the experiment's UC trace location from its binding tag, and
+passes it to `set_experiment` so spans actually persist to the `_otel_spans` table — without it,
+MLflow silently skips span export to UC. Classic (workspace) experiments need none of this and
+are the default. UC-backed tracing requires an MLflow release that provides the
+`UnityCatalog` trace-location API.
+
+### Multimodal inputs (image / PDF / file)
+
+Chat-model spans record their inputs as **structured messages**, preserving the
+multimodal content parts a graph sends to the model — OpenAI / LangChain
+`image_url`, OpenAI `file`, and Anthropic `image` / `document`. (Internally the
+tracer runs the LangChain `BaseTracer` in `original+chat` mode; the default would
+otherwise flatten chat messages to a text-only `prompts` string and drop every
+attachment.)
+
+The inline base64 of each such part is **never stored** in the trace: it is removed
+before recording and replaced with a lightweight reference, so the Unity Catalog
+trace tables stay readable (multi-MB data URIs previously pushed large invoice
+traces past the SQL inline read limit — issue #21). A remote `http(s)://` image URL
+is already a small reference, so this step keeps it verbatim (but see the text cap
+below — if enabled, it still truncates *any* string over its threshold, URLs
+included). The transform runs on a copy of the inputs, so the live message sent to
+the model is untouched and prompt caching is unaffected.
+
+By default a part becomes a `{"type": ..., "_omitted": true, "bytes": N}`
+placeholder. To store a meaningful reference instead — e.g. the Unity Catalog volume
+path the image was loaded from, so it can be re-fetched at runtime — pass a
+`content_ref_resolver`:
+
+```python
+from databricks_langgraph_tracer import (
+    ContentPartContext,
+    configure_databricks_tracing,
+)
+
+def image_ref(part: dict, ctx: ContentPartContext) -> dict | None:
+    # ctx.metadata is the run metadata — pass per-run data (e.g. a source volume
+    # path) via the invoke config's `metadata`, which propagates to the LLM run
+    # alongside langgraph_node etc.
+    path = ctx.metadata.get("encoded_images_path")
+    if path:
+        return {"type": part.get("type"), "ref": path, "page": ctx.index}
+    return None  # fall back to the default placeholder
+
+configure_databricks_tracing(experiment_id="...", content_ref_resolver=image_ref)
+
+# ... then carry the per-run ref data on the invoke config metadata:
+graph.invoke(state, config={
+    "callbacks": get_tracing_callbacks(),
+    "metadata": {"encoded_images_path": "/Volumes/cat/sch/vol/inv/pages.txt"},
+})
+```
+
+The resolver is called once per multimodal part with the part and a
+`ContentPartContext` — `index` (position within the message content array, i.e. the
+page number for a one-image-per-page invoice), `bytes` (the inline payload length),
+and the run `metadata`. Return a dict to store as the reference, or `None` for the
+default placeholder. It is also accepted by
+`DatabricksLangGraphTracer(content_ref_resolver=...)` for per-graph wiring.
+
+The library guarantees no inline image bytes are stored: if a resolver result
+re-introduces an inline payload (a `data:` URI, a recognized base64 content part, or
+the part's own payload echoed back under any key — anywhere in the returned object),
+it is rejected and the placeholder is used. Beyond that, keep the reference
+**compact** — a *fabricated* large string under a custom key is the consumer's
+responsibility (the library strips inline payloads but does not otherwise bound what
+a resolver returns).
+`ctx.metadata` is a shallow copy of the run metadata, so a resolver cannot corrupt run
+state by setting top-level keys (don't mutate its nested values, which are shared).
+
+### Capping large text (issue #23)
+
+Multimodal externalization handles inline *bytes*, but large **plain text** can also
+push a trace past the SQL inline read limit — e.g. a classification vocabulary or
+aggregated result set threaded through every fan-out span's inputs and outputs. Set
+`max_string_chars` (or `DATABRICKS_TRACING_MAX_STRING_CHARS`) to truncate it:
+
+```python
+configure_databricks_tracing(experiment_id="...", max_string_chars=50_000)
+```
+
+When set, any string **value** longer than the threshold — in a span's inputs *or*
+outputs — is replaced with a compact placeholder:
+
+```json
+{"_truncated": true, "chars": 812345, "bytes": 812345, "preview": "first 256 chars…"}
+```
+
+It is **opt-in / off by default** (generic truncation costs debuggability, so you
+choose the threshold). It caps string values only — never keys or structural fields —
+and runs on a copy, so the live messages are untouched. The cap also reaches text
+nested inside **Pydantic models / dataclasses / tuples** (e.g. a model returned by a
+node such as `final_output`), normalizing them to the same shape MLflow records. Also
+accepted by `DatabricksLangGraphTracer(max_string_chars=...)` for per-graph wiring.
+
+This is a **distinct** knob from the multimodal handling above: `content_ref_resolver`
+chooses references for inline image/PDF/file bytes; `max_string_chars` caps arbitrary
+text (and never truncates a resolver's reference). Because it is generic, it also
+truncates any other over-threshold string — including a remote `http(s)://` image URL
+the multimodal step keeps verbatim — so set the threshold comfortably above your
+reference / URL lengths. Note it is a **per-leaf** mitigation, not a hard per-trace
+byte budget — enough sub-threshold leaves can still sum past the limit — so for the
+heaviest spans also reduce what you record (pass ids/references through node state
+rather than full payloads).
+
+The threshold counts **characters** (code points here; the TypeScript package counts
+UTF-16 units, so the two can differ on non-BMP text), while the inline limit is in
+bytes — multibyte text can be up to ~4× larger in bytes than characters, so for
+CJK/emoji-heavy content size the cap below `limit / 4`. The placeholder's `bytes`
+field always reports the exact UTF-8 size of the original.
+
+### Autolog fallback
+
+`mode="autolog"` wires MLflow's built-in LangChain autologging plus compatibility patches and
+emits a **reduced** schema — everything except a tracer-computed
+`mlflow.trace.cost` rollup (the backend may still aggregate it server-side). The default
+`mode="tracer"` (custom `BaseTracer`) is the full-parity path.
+
+## Development
+
+```bash
+cd python
+uv sync                                            # latest allowed mlflow (ceiling)
+uv run pytest                                      # unit + lifecycle tests
+uv run ruff check src tests
+uv run ty check
+uv run python scripts/generate_keys.py --check     # schema/keys parity
+
+# mlflow floor matrix (CI runs both cells via UV_RESOLUTION):
+UV_RESOLUTION=lowest-direct uv sync && UV_RESOLUTION=lowest-direct uv run pytest
+```
+
+Tests use a local sqlite MLflow tracking backend; checks that need a live Databricks backend are
+marked `integration`.
+
+## Changelog
+
+See [`CHANGELOG.md`](CHANGELOG.md).

insightfactory_databricks_langgraph_tracer-1.0.0.dev5/THIRD_PARTY_NOTICES

@@ -0,0 +1,26 @@
+LiteLLM model pricing data
+==========================
+
+This package includes model pricing data derived from LiteLLM.
+
+MIT License
+
+Copyright (c) 2023 Berri AI
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.