agent-tracegraph 0.3.0__tar.gz → 0.4.0__tar.gz

{agent_tracegraph-0.3.0 → agent_tracegraph-0.4.0}/CHANGELOG.md

@@ -6,6 +6,86 @@ The **artifact** schema version and the **report**/**review-candidate** JSON sch
 separate contracts from this version number; changes to any of them are called out
 explicitly.
 
+## 0.4.0 - 2026-10-14
+
+### Added
+
+- **LangGraph tasks paused by `interrupt()` are reported.** A task whose only write on the
+  thread's newest root checkpoint is `__interrupt__` — it paused and the history holds no
+  resume and no result — becomes an `unset` step with the new source value `interrupt`,
+  linked by `CAUSED_BY` to that checkpoint and named by the same task-id reconstruction as
+  failed tasks. The step describes the recorded history, not whether LangGraph would resume
+  the task now. An interrupt recorded beside a resume or an error and no result is ambiguous
+  in the checkpoint (under LangGraph 1.2.2 a resumed task that wrote nothing looks exactly
+  like one paused twice; 1.2.15 marks it finished), so it is disclosed at ingest instead. Trace `status` is unchanged: a pause is not an error.
+- **Artifact schema version 5**, stamped by content: only an artifact carrying an
+  `interrupt` step declares it, so every other artifact serializes byte for byte as before.
+  0.3.1 and earlier refuse a version 5 artifact by name; this build reads versions 1 to 5,
+  and still rejects an envelope that declares less than its content needs.
+
+### Known limitations
+
+- Tasks scheduled but never started — the recursion limit, a killed process, an
+  `interrupt_before`/`interrupt_after` breakpoint — leave no write and are still not
+  reported. Telling them apart from finished work needs the compiled graph.
+- An interrupt inside a subgraph is reported as the subgraph's node at the root, not as the
+  node inside the subgraph that paused.
+
+## 0.3.1 - 2026-10-13
+
+### Fixed
+
+- **An unreadable cost no longer understates the run's total.** The importers dropped a
+  `llm.cost.total` that was not a finite non-negative decimal (`"abc"`, `-1`, a boolean), so
+  `analyze` summed the remaining costs and printed a total that was knowably too low, with no
+  warning. The documented rule that a single unparseable cost withholds the whole total only
+  applied to artifacts built by hand. An unreadable total cost is now stored as the fixed
+  marker `"unreadable"`, never the producer's value, and the report withholds `total_cost`
+  and its delta with the existing note.
+- Token counts, prompt and completion costs, cost currencies and Phoenix evaluation scores
+  that cannot be read were dropped without a trace. They are still reported as unavailable,
+  and `ingest-otlp`, `ingest-phoenix`, `analyze` and `phoenix diagnose` now say so on stderr,
+  naming the kind of value and how many spans (for scores, evaluations) carried it, never
+  the value itself. A boolean score still reads as `1.0` or `0.0`, as before.
+- An OTLP `intValue` sent as a fractional JSON number was truncated before validation:
+  `1.5` became `1` and `-0.5` became `0`, so an invalid token count looked valid. It is now
+  validated as sent.
+- **A checkpoint history LangGraph could not have written is a usage error, not a
+  traceback or a different trace.** About half of 64 one-field mutations of a real history
+  (a non-object `metadata` or `channel_values`, a numeric `checkpoint_ns`, a boolean `v`, an
+  unhashable parent checkpoint id, ...) crashed `ingest` with a traceback, and several were
+  read silently as something else: a malformed parent config or `metadata.parents` dropped
+  the declared parent and made the checkpoint a root, and `metadata.step` of `true` or `1.5`
+  was read as step 1. Each checkpoint is now checked against the types LangGraph declares
+  for the fields tracegraph reads, strictly, and rejected with a message naming the field.
+  A config `checkpoint_id` that disagrees with the checkpoint's own id is rejected too.
+- A checkpoint row the saver cannot decode (an unknown serialization type, metadata nested
+  past the recursion limit, a non-string metadata column) crashed `ingest` and `discover`
+  with a traceback, except that `discover` reported an unknown serialization type as "does
+  not support listing all threads". Both now report the history as undecodable.
+- Pending writes that are not a `(task id, channel, value)` triple were skipped without a
+  word, and any 3-character string was unpacked as one (`"abc"` became task `a`). They are
+  still ignored, but `ingest` now says how many on stderr, since one could have carried a
+  task failure.
+
+### Compatibility notes for imports
+
+- An artifact imported from a span with an unreadable total cost now carries
+  `"total_cost": "unreadable"` instead of no cost, and its reports withhold the run's
+  total cost. No schema change: 0.3.0 reads such an artifact and withholds the same way.
+- Because `intValue` is no longer truncated, `llm.cost.total = {"intValue": 1.5}` is stored as
+  `"1.5"` instead of `"1"`, and a `toolgraph.graph_generation` of `{"intValue": 1.5}` is
+  rejected like any other invalid generation instead of being read as `1`.
+- The committed Phoenix fixture carries an unreadable `llm.cost.prompt`; reading it now prints
+  one stderr disclosure. Every committed fixture's artifact, report, memo, candidate file and
+  CLI stdout is unchanged.
+- LangGraph histories that `ingest` used to accept with a malformed field listed above are
+  now refused. Histories written by LangGraph's own savers are unaffected: the checks cover
+  only fields tracegraph reads (`counters_since_delta_snapshot`, which savers return as
+  lists although it is declared as tuples, is not checked), a missing `updated_channels`
+  and a legacy `pending_sends` are accepted, and an unknown `metadata.source` string still
+  reads as a loop. The committed LangGraph fixture is byte-identical.
+
 ## 0.3.0 - 2026-10-13
 
 ### Fixed

{agent_tracegraph-0.3.0 → agent_tracegraph-0.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: agent-tracegraph
-Version: 0.3.0
+Version: 0.4.0
 Summary: Time-travel debugging and causal RCA for LangGraph agent traces, on a causal graph.
 Project-URL: Homepage, https://github.com/memtomem/tracegraph
 Project-URL: Repository, https://github.com/memtomem/tracegraph
@@ -289,9 +289,11 @@ and any change to them is called out in the [changelog](CHANGELOG.md).
 
 Known limits:
 
-- A LangGraph task that was scheduled but left no result and no recorded error (a run
-  paused by `interrupt()`, stopped by the recursion limit, or killed) is not reported, so
-  such a run reads `ok` unless some other step recorded an error.
+- A LangGraph task that was scheduled but never started (a run stopped by the recursion
+  limit, killed, or held at an `interrupt_before`/`interrupt_after` breakpoint) leaves no
+  write and is not reported, so such a run reads `ok` unless some other step recorded an
+  error. A task paused by `interrupt()` *is* reported (below), but only at the root graph:
+  the subgraph node is named, not the node inside it that paused.
 - Phoenix exports carry no span links, so their artifacts are parent-only; additional fan-in
   causes may be missing.
 - Report privacy (`safe-v1`) applies to reports; OTLP and LangGraph artifacts keep the
@@ -376,9 +378,16 @@ cancels its siblings and records that through the same channel; a cancelled task
 as an `unset` step with no message, because it was torn down rather than at fault. Only the
 latest persisted error survives per task, so repeated attempts are not reconstructed, and
 `Send` packets are nameable only where `langgraph.types` is importable — an ordinary install
-discloses them instead. An artifact whose derived tree relies on EdgeOrigin precedence declares
-artifact `schema_version` 4; otherwise, an artifact carrying derived task steps declares
-`schema_version` 3, and artifacts without them still serialize as version 2, byte for byte.
+discloses them instead. A task that called `interrupt()` and whose only write on the thread's
+newest root checkpoint is that interrupt becomes an `unset` step with source `interrupt` under
+that checkpoint: the recorded history holds no resume and no result for it. That describes the
+history, not whether LangGraph would resume the task today. An interrupt recorded together with
+a resume or an error and no result is ambiguous in the checkpoint itself — under LangGraph 1.2.2
+a task resumed with nothing to write looks exactly like one paused a second time — so it is
+disclosed at ingest, not reported. An artifact carrying interrupt steps declares artifact `schema_version` 5; one
+whose derived tree relies on EdgeOrigin precedence declares 4; otherwise, an artifact carrying
+derived task steps declares `schema_version` 3, and artifacts without them still serialize as
+version 2, byte for byte.
 Missing observed errors still do not
 prove success: an artifact ingested by an older build, or one using an unrecognized checkpoint
 layout, carries no derived step and looks exactly like a clean run, which is why the capture
@@ -387,7 +396,10 @@ means valid in-trace links only. Structural diff compares the derived tree and d
 lossy step counts. Metrics sum available span observations: partial coverage and producer
 aggregation can undercount or double count. Coverage gaps are disclosed rather than absorbed
 — a wall duration derived from a subset of steps says so, one whose end precedes its start is
-withheld, and a single unparseable cost withholds the whole total instead of understating it. File input has a size limit, but stdin/Phoenix
+withheld, and a single unparseable cost withholds the whole total instead of understating it.
+An OTLP or Phoenix import records an unreadable total cost as `unreadable` for that reason, and
+reports token counts, other costs, currencies and evaluation scores it cannot read as
+unavailable, disclosing each on stderr; the producer's value is never kept. File input has a size limit, but stdin/Phoenix
 subprocess output and query materialization are not streaming memory guarantees.
 
 Both stores isolate returned mutable objects. InMemory node upsert replaces by ID;

{agent_tracegraph-0.3.0 → agent_tracegraph-0.4.0}/README.md

@@ -258,9 +258,11 @@ and any change to them is called out in the [changelog](CHANGELOG.md).
 
 Known limits:
 
-- A LangGraph task that was scheduled but left no result and no recorded error (a run
-  paused by `interrupt()`, stopped by the recursion limit, or killed) is not reported, so
-  such a run reads `ok` unless some other step recorded an error.
+- A LangGraph task that was scheduled but never started (a run stopped by the recursion
+  limit, killed, or held at an `interrupt_before`/`interrupt_after` breakpoint) leaves no
+  write and is not reported, so such a run reads `ok` unless some other step recorded an
+  error. A task paused by `interrupt()` *is* reported (below), but only at the root graph:
+  the subgraph node is named, not the node inside it that paused.
 - Phoenix exports carry no span links, so their artifacts are parent-only; additional fan-in
   causes may be missing.
 - Report privacy (`safe-v1`) applies to reports; OTLP and LangGraph artifacts keep the
@@ -345,9 +347,16 @@ cancels its siblings and records that through the same channel; a cancelled task
 as an `unset` step with no message, because it was torn down rather than at fault. Only the
 latest persisted error survives per task, so repeated attempts are not reconstructed, and
 `Send` packets are nameable only where `langgraph.types` is importable — an ordinary install
-discloses them instead. An artifact whose derived tree relies on EdgeOrigin precedence declares
-artifact `schema_version` 4; otherwise, an artifact carrying derived task steps declares
-`schema_version` 3, and artifacts without them still serialize as version 2, byte for byte.
+discloses them instead. A task that called `interrupt()` and whose only write on the thread's
+newest root checkpoint is that interrupt becomes an `unset` step with source `interrupt` under
+that checkpoint: the recorded history holds no resume and no result for it. That describes the
+history, not whether LangGraph would resume the task today. An interrupt recorded together with
+a resume or an error and no result is ambiguous in the checkpoint itself — under LangGraph 1.2.2
+a task resumed with nothing to write looks exactly like one paused a second time — so it is
+disclosed at ingest, not reported. An artifact carrying interrupt steps declares artifact `schema_version` 5; one
+whose derived tree relies on EdgeOrigin precedence declares 4; otherwise, an artifact carrying
+derived task steps declares `schema_version` 3, and artifacts without them still serialize as
+version 2, byte for byte.
 Missing observed errors still do not
 prove success: an artifact ingested by an older build, or one using an unrecognized checkpoint
 layout, carries no derived step and looks exactly like a clean run, which is why the capture
@@ -356,7 +365,10 @@ means valid in-trace links only. Structural diff compares the derived tree and d
 lossy step counts. Metrics sum available span observations: partial coverage and producer
 aggregation can undercount or double count. Coverage gaps are disclosed rather than absorbed
 — a wall duration derived from a subset of steps says so, one whose end precedes its start is
-withheld, and a single unparseable cost withholds the whole total instead of understating it. File input has a size limit, but stdin/Phoenix
+withheld, and a single unparseable cost withholds the whole total instead of understating it.
+An OTLP or Phoenix import records an unreadable total cost as `unreadable` for that reason, and
+reports token counts, other costs, currencies and evaluation scores it cannot read as
+unavailable, disclosing each on stderr; the producer's value is never kept. File input has a size limit, but stdin/Phoenix
 subprocess output and query materialization are not streaming memory guarantees.
 
 Both stores isolate returned mutable objects. InMemory node upsert replaces by ID;

{agent_tracegraph-0.3.0 → agent_tracegraph-0.4.0}/pyproject.toml

@@ -4,7 +4,7 @@
 # the schema `kind` constants and the TRACEGRAPH_* environment variables — those are package
 # identity, not distribution identity, and renaming them would break published contracts.
 name = "agent-tracegraph"
-version = "0.3.0"
+version = "0.4.0"
 description = "Time-travel debugging and causal RCA for LangGraph agent traces, on a causal graph."
 readme = "README.md"
 requires-python = ">=3.12"

{agent_tracegraph-0.3.0 → agent_tracegraph-0.4.0}/src/tracegraph/adapters/langgraph_checkpoint.py

@@ -54,6 +54,17 @@ steps carrying no message, and only the task that actually raised is an ``ERROR`
 Only the *latest* persisted error survives per task id — a fail/fail/succeed sequence leaves
 one record — so attempts are never reconstructed. Errors are recorded wherever they appear;
 this module does not infer anything from their *absence*.
+
+Interrupted tasks
+-----------------
+A node that calls ``interrupt()`` leaves an ``__interrupt__`` pending write on the checkpoint
+that scheduled it, and LangGraph resumes from the thread's newest root checkpoint. A task whose
+*only* write on that checkpoint is ``__interrupt__`` therefore never received a resume or
+produced a result, and becomes an ``UNSET`` step with ``source=INTERRUPT``. That is a statement
+about the recorded history, not a prediction that LangGraph would resume it now. Tasks that were
+scheduled but never started (the recursion limit, a killed process, a static breakpoint) leave no
+write at all and are not reported: telling them apart from finished work needs the compiled
+graph, which this adapter does not have.
 """
 
 from __future__ import annotations
@@ -62,8 +73,12 @@ from binascii import Error as BinasciiError, unhexlify
 from hashlib import sha1
 import heapq
 import re
-from typing import Any
+from typing import Any, Iterable, Iterator, get_type_hints
+
+from langgraph.checkpoint.base import Checkpoint, CheckpointMetadata
+from pydantic import TypeAdapter, ValidationError
 
+from tracegraph.input_validation import object_field
 from tracegraph.model import (
     CausalFidelity,
     Edge,
@@ -82,12 +97,33 @@ _BRANCH_PREFIX = "branch:to:"
 _JOIN_PREFIX = "join:"
 _TOOL_TOKENS = {"tool", "tools"}
 
+# The shape of every field this adapter reads, taken from the types LangGraph itself declares
+# rather than restated here, so a LangGraph type change surfaces (in the upstream drift
+# workflow) instead of hiding behind a stale copy. Only fields the adapter reads are checked:
+# LangGraph declares ``counters_since_delta_snapshot`` as tuples, but both savers hand it back
+# as lists, so checking everything would reject histories LangGraph itself wrote. ``source``
+# is any string, not LangGraph's Literal, because an unknown source still reads as a loop.
+_CHECKPOINT_FIELDS = {
+    name: TypeAdapter(get_type_hints(Checkpoint)[name])
+    for name in ("id", "ts", "v", "channel_values", "channel_versions", "versions_seen")
+}
+_METADATA_FIELDS = {
+    **{name: TypeAdapter(get_type_hints(CheckpointMetadata)[name]) for name in ("step", "parents")},
+    "source": TypeAdapter(str),
+}
+_CONFIG_FIELDS = {name: TypeAdapter(str) for name in ("thread_id", "checkpoint_ns", "checkpoint_id")}
+
 # LangGraph's reserved pending-write channels and task-path markers, spelled out rather than
 # imported: they live under ``langgraph._internal``, which is private and not a dependency of
 # this package. They are stable on-disk names — a checkpoint written by any 4.x saver uses
 # them — so copying them is safer than importing a private module that may move.
 _ERROR = "__error__"
 _ERROR_SOURCE_NODE = "__error_source_node__"
+_INTERRUPT = "__interrupt__"
+_RESUME = "__resume__"
+#: Writes that can sit beside an ``__interrupt__`` without recording whether the task is still
+#: paused. Any other write is a result, and the task is no longer waiting.
+_UNDECIDED_WITH_INTERRUPT = frozenset({_INTERRUPT, _RESUME, _ERROR, _ERROR_SOURCE_NODE})
 _START = "__start__"
 _TASKS = "__pregel_tasks"
 _PULL = "__pregel_pull"
@@ -165,18 +201,18 @@ class LangGraphCheckpointAdapter:
                 "this checkpointer does not support listing all threads"
             )
         try:
-            tuples = list(list_fn(None))
+            listing = list_fn(None)
         except NotImplementedError as exc:
-            # Only the documented capability signal becomes "unsupported"; any other
-            # failure (sqlite corruption, I/O error, a saver bug) must propagate, not
-            # masquerade as a missing capability.
+            # Only the documented capability signal becomes "unsupported", and only when the
+            # call itself raises it. The same exception raised while iterating is the
+            # serializer failing to decode a row, which `_decoded` reports as corrupt input.
             raise NotImplementedError(
                 "this checkpointer does not support listing all threads"
             ) from exc
         seen: set[str] = set()
         ordered: list[str] = []
-        for t in tuples:
-            tid = (t.config.get("configurable") or {}).get("thread_id")
+        for t in _decoded(listing):
+            tid = _checked(t)[0]["thread_id"]
             if tid is not None and tid not in seen:
                 seen.add(tid)
                 ordered.append(tid)
@@ -186,8 +222,8 @@ class LangGraphCheckpointAdapter:
         config = {"configurable": {"thread_id": trace_id}}
         tuples = [
             t
-            for t in self._saver.list(config)
-            if (t.config.get("configurable") or {}).get("thread_id") == trace_id
+            for t in _decoded(self._saver.list(config))
+            if _checked(t)[0]["thread_id"] == trace_id
         ]
         if not tuples:
             raise KeyError(f"no checkpoints for thread_id {trace_id!r}")
@@ -199,12 +235,11 @@ class LangGraphCheckpointAdapter:
         original_seq: dict[str, int] = {}
         steps: dict[str, Step] = {}
         writes_by_step: dict[str, dict[str, list[tuple[str, Any]]]] = {}
+        malformed_writes = 0
         for t in tuples:
-            cfg = t.config.get("configurable") or {}
-            cp = t.checkpoint
-            md = t.metadata or {}
+            cfg, cp, md, parent_cfg = _checked(t)
             ns = cfg.get("checkpoint_ns", _ROOT_NS) or _ROOT_NS
-            cid = cfg.get("checkpoint_id") or cp["id"]
+            cid = cp["id"]
             key = (ns, cid)
             if key in step_id_by_key:
                 raise ValueError(f"duplicate checkpoint {cid!r} in namespace {ns!r}")
@@ -216,15 +251,12 @@ class LangGraphCheckpointAdapter:
             step_id_by_key[key] = step_id
             key_by_step_id[step_id] = key
             checkpoints[step_id] = cp
-            try:
-                original_seq[step_id] = int(md.get("step", 0))
-            except (TypeError, ValueError) as exc:
-                raise ValueError(
-                    f"checkpoint {cid!r} in thread {trace_id!r} has a non-integer "
-                    f"metadata step: {md.get('step')!r}"
-                ) from exc
-            parent_keys[step_id] = self._declared_parent_keys(t, ns)
-            writes_by_step[step_id] = self._writes_by_task(getattr(t, "pending_writes", None))
+            original_seq[step_id] = md.get("step", 0)
+            parent_keys[step_id] = self._declared_parent_keys(parent_cfg, md, ns)
+            writes_by_step[step_id], malformed = self._writes_by_task(
+                getattr(t, "pending_writes", None)
+            )
+            malformed_writes += malformed
             steps[step_id] = Step(
                 step_id=step_id,
                 trace_id=trace_id,
@@ -304,6 +336,16 @@ class LangGraphCheckpointAdapter:
         ingest_warnings = self._derive_task_steps(
             trace_id, steps, checkpoints, key_by_step_id, original_seq, writes_by_step, edges
         )
+        ingest_warnings.extend(
+            self._derive_interrupt_steps(
+                trace_id, steps, checkpoints, key_by_step_id, original_seq, writes_by_step, edges
+            )
+        )
+        if malformed_writes:
+            ingest_warnings.append(
+                f"{malformed_writes} pending write(s) are not a (task id, channel, value) "
+                "triple and were ignored; a task failure recorded in one would not be reported."
+            )
 
         any_error = any(s.status is StepStatus.ERROR for s in steps.values())
         trace = Trace(
@@ -418,24 +460,115 @@ class LangGraphCheckpointAdapter:
             )
         return warnings
 
+    @classmethod
+    def _derive_interrupt_steps(
+        cls,
+        trace_id: str,
+        steps: dict[str, Step],
+        checkpoints: dict[str, dict],
+        key_by_step_id: dict[str, _CheckpointKey],
+        original_seq: dict[str, int],
+        writes_by_step: dict[str, dict[str, list[tuple[str, Any]]]],
+        edges: list[Edge],
+    ) -> list[str]:
+        """Add one UNSET step per task the head checkpoint records as interrupted, never resumed.
+
+        Only the thread's head — the newest root checkpoint, which is what LangGraph itself
+        resumes from — is read. An ``__interrupt__`` write on any earlier checkpoint was resumed,
+        forked past, or abandoned, and is history rather than an open pause.
+
+        A task is reported only when ``__interrupt__`` is its *only* write. Anything else that
+        accompanies it leaves the task's state unrecorded, and it is disclosed rather than
+        guessed: under LangGraph 1.2.2 a resumed task that returned nothing leaves
+        ``{__interrupt__, __resume__}``, exactly like one that reached a second ``interrupt()``
+        (LangGraph overwrites the interrupt in place; 1.2.15 adds ``__no_writes__`` to the
+        first, which then reads as finished); a failure followed by a retry that interrupts leaves the same
+        ``{__error__, __interrupt__}`` as the reverse order, because savers keep no order.
+        """
+        root_ids = [sid for sid, (ns, _) in key_by_step_id.items() if ns == _ROOT_NS]
+        if not root_ids:
+            return []
+        head_id = max(root_ids, key=lambda sid: checkpoints[sid]["id"])
+        head = steps[head_id]
+        cp = checkpoints[head_id]
+        _, cid = key_by_step_id[head_id]
+        warnings: list[str] = []
+        undecided = 0
+        for task_id, task_writes in (writes_by_step.get(head_id) or {}).items():
+            if task_id == _NULL_TASK_ID:
+                continue
+            channels = {channel for channel, _ in task_writes}
+            if _INTERRUPT not in channels:
+                continue
+            if channels != {_INTERRUPT}:
+                if channels <= _UNDECIDED_WITH_INTERRUPT:
+                    undecided += 1
+                continue
+            name = cls._resolve_task_name(cp, _ROOT_NS, original_seq[head_id], task_id, task_writes)
+            if name is None:
+                warnings.append(
+                    f"checkpoint {cid}: interrupted task {task_id} could not be attributed to a "
+                    "node; no candidate reproduced its task id, which is also what a change in "
+                    "LangGraph's task-id algorithm would look like."
+                )
+            derived_id = f"{head_id}#interrupt:{task_id}"
+            if derived_id in steps:
+                raise ValueError(
+                    f"derived interrupt step id {derived_id!r} collides with a checkpoint id "
+                    "or another derived step"
+                )
+            steps[derived_id] = Step(
+                step_id=derived_id,
+                trace_id=trace_id,
+                seq=head.seq + 1,
+                ts=head.ts,
+                kind=StepKind.TOOL if name and _looks_like_tool_node(name) else StepKind.CHAIN,
+                source=StepSource.INTERRUPT,
+                name=name,
+                # The task has no outcome yet. The interrupt's value is whatever the node asked
+                # its caller, which is user content, so it is not carried as a message.
+                status=StepStatus.UNSET,
+            )
+            edges.append(
+                Edge(
+                    type=EdgeType.CAUSED_BY,
+                    src=derived_id,
+                    dst=head_id,
+                    # LangGraph keys the write to the head checkpoint itself.
+                    origin=EdgeOrigin.CHECKPOINT_PARENT,
+                )
+            )
+        if undecided:
+            warnings.append(
+                f"checkpoint {cid}: {undecided} task(s) recorded an interrupt together with a "
+                "resume or an error and no result; the checkpoint does not record whether each "
+                "is still paused, so none is reported as interrupted."
+            )
+        return warnings
+
     @staticmethod
-    def _writes_by_task(pending_writes: Any) -> dict[str, list[tuple[str, Any]]]:
+    def _writes_by_task(pending_writes: Any) -> tuple[dict[str, list[tuple[str, Any]]], int]:
         """Group a checkpoint's pending writes by task id, in first-seen order.
 
         Savers are free to order writes as they like — the SQLite saver sorts by task id and
         index, the in-memory one keeps insertion order — so nothing here may depend on
         position. ``pending_writes`` is also optional on ``CheckpointTuple``; a third-party
-        saver that omits it reads as "no writes", never as an error.
+        saver that omits it reads as "no writes", never as an error. Returns the grouping and
+        how many entries were not a ``(str, str, value)`` triple: those are not ours to
+        interpret, but the caller discloses them because one could have carried a failure.
         """
         grouped: dict[str, list[tuple[str, Any]]] = {}
+        malformed = 0
         for write in pending_writes or ():
-            try:
-                task_id, channel, value = write
-            except (TypeError, ValueError):
-                continue  # not a (task_id, channel, value) triple; not ours to interpret
+            if not isinstance(write, (tuple, list)) or len(write) != 3:
+                malformed += 1
+                continue
+            task_id, channel, value = write
             if isinstance(task_id, str) and isinstance(channel, str):
                 grouped.setdefault(task_id, []).append((channel, value))
-        return grouped
+            else:
+                malformed += 1
+        return grouped, malformed
 
     @staticmethod
     def _hash_task_id(cid: str, legacy: bool, *parts: str) -> str | None:
@@ -593,22 +726,18 @@ class LangGraphCheckpointAdapter:
         return str(value)
 
     @staticmethod
-    def _declared_parent_keys(t: Any, ns: str) -> list[_CheckpointKey]:
-        parent_cfg = (t.parent_config or {}).get("configurable") or {}
+    def _declared_parent_keys(parent_cfg: dict, md: dict, ns: str) -> list[_CheckpointKey]:
         parent_id = parent_cfg.get("checkpoint_id")
         if parent_id is not None:
             parent_ns = parent_cfg.get("checkpoint_ns", ns) or _ROOT_NS
             return [(parent_ns, parent_id)]
 
-        md = t.metadata or {}
+        # `_checked` has already established `parents` as a str -> str mapping.
         parents = md.get("parents") or {}
-        if not isinstance(parents, dict):
-            return []
         closest = LangGraphCheckpointAdapter._closest_parent_ns(ns, parents)
         if closest is None:
             return []
-        checkpoint_id = parents[closest]
-        return [] if checkpoint_id is None else [(closest or _ROOT_NS, checkpoint_id)]
+        return [(closest or _ROOT_NS, parents[closest])]
 
     @staticmethod
     def _closest_parent_ns(ns: str, parents: dict[str, str]) -> str | None:
@@ -809,3 +938,80 @@ class LangGraphCheckpointAdapter:
         if previous:  # error was already present upstream -> not introduced here
             return None
         return str(current)
+
+
+def _decoded(listing: Iterable[Any]) -> Iterator[Any]:
+    """Iterate a saver's listing, turning a failure to decode a row into a usage error.
+
+    Only saver code runs while the next tuple is produced: the serializer reviving a blob,
+    ``json.loads`` on the metadata column, the database driver. Whatever it raises there
+    (an unknown serialization type, JSON nested past the recursion limit, a non-string column)
+    means the history cannot be read, which is a property of the input, not a crash.
+    """
+    iterator = iter(listing)
+    while True:
+        try:
+            item = next(iterator)
+        except StopIteration:
+            return
+        except Exception as exc:
+            raise ValueError(
+                f"checkpoint history cannot be decoded ({type(exc).__name__})"
+            ) from exc
+        yield item
+
+
+def _checked(t: Any) -> tuple[dict, dict, dict, dict]:
+    """The tuple's ``configurable``, checkpoint and metadata, each checked against the shape
+    LangGraph declares for the fields this adapter reads (strictly: ``True`` is not ``1``).
+
+    A field is checked only when present; ``checkpoint.id`` is required, and when the config
+    also names the checkpoint the two must agree, because step identity comes from one and
+    chronology and task hashing from the other.
+    """
+    # A config without a thread id is not "empty": `ingest` filters on the thread id, so it
+    # would silently drop the checkpoint, and with it any failure it recorded.
+    config = getattr(t, "config", None)
+    if not isinstance(config, dict) or not isinstance(config.get("configurable"), dict):
+        raise ValueError("checkpoint config must be an object with a configurable object")
+    cfg = config["configurable"]
+    if "thread_id" not in cfg:
+        raise ValueError("checkpoint config has no thread_id")
+    cp = getattr(t, "checkpoint", None)
+    if not isinstance(cp, dict):
+        raise ValueError("checkpoint must be an object")
+    cid = cp.get("id")
+    label = f"checkpoint {cid!r}" if isinstance(cid, str) else "a checkpoint"
+    if "id" not in cp:
+        raise ValueError(f"{label} has no id")
+    md = object_field(getattr(t, "metadata", None), f"{label} metadata")
+    # No parent config is legitimate (a thread's first checkpoint). One that is present but
+    # names no checkpoint is not: read as "no parent", it would make this checkpoint a root.
+    parent = getattr(t, "parent_config", None)
+    parent_cfg: dict = {}
+    if parent is not None:
+        if not isinstance(parent, dict) or not isinstance(parent.get("configurable"), dict):
+            raise ValueError(f"{label} parent config must be an object with a configurable object")
+        parent_cfg = parent["configurable"]
+        if "checkpoint_id" not in parent_cfg:
+            raise ValueError(f"{label} parent config has no checkpoint_id")
+    for where, fields, values in (
+        ("checkpoint", _CHECKPOINT_FIELDS, cp),
+        ("metadata", _METADATA_FIELDS, md),
+        ("config", _CONFIG_FIELDS, cfg),
+        ("parent config", _CONFIG_FIELDS, parent_cfg),
+    ):
+        for name, adapter in fields.items():
+            if name not in values:
+                continue
+            try:
+                adapter.validate_python(values[name], strict=True)
+            except ValidationError as exc:
+                reason = exc.errors()[0]["msg"]
+                raise ValueError(f"{label} has a malformed {where} {name}: {reason}") from None
+    if "checkpoint_id" in cfg and cfg["checkpoint_id"] != cid:
+        raise ValueError(f"{label} disagrees with its config checkpoint_id")
+    writes = getattr(t, "pending_writes", None)
+    if writes is not None and not isinstance(writes, (list, tuple)):
+        raise ValueError(f"{label} has pending writes that are not a list")
+    return cfg, cp, md, parent_cfg

{agent_tracegraph-0.3.0 → agent_tracegraph-0.4.0}/src/tracegraph/adapters/otlp_spans.py

@@ -118,6 +118,8 @@ def _attr_value(v: Any) -> Any:
             raw = v[key]
             if isinstance(raw, bool):
                 return raw  # `int(True) == 1` would hide that the value was a boolean
+            if isinstance(raw, float) and not raw.is_integer():
+                return raw  # `int(1.5) == 1` would hand validation a plausible count
             try:
                 return int(raw)
             except (TypeError, ValueError, OverflowError):
@@ -229,43 +231,83 @@ def _iso_ts(start_nano: Any) -> str | None:
     return datetime.fromtimestamp(nanos / 1e9, tz=timezone.utc).isoformat()
 
 
-def _int_attr(attrs: dict[str, Any], *names: str) -> int | None:
-    for name in names:
-        value = attrs.get(name)
-        if value is None or isinstance(value, bool):
-            continue
-        try:
-            parsed = int(value)
-        except (TypeError, ValueError, OverflowError):
-            continue
-        if parsed < 0 or (isinstance(value, float) and not value.is_integer()):
-            continue
-        return parsed
-    return None
+def _as_count(value: Any) -> int | None:
+    if value is None or isinstance(value, bool):
+        return None
+    try:
+        parsed = int(value)
+    except (TypeError, ValueError, OverflowError):
+        return None
+    if parsed < 0 or (isinstance(value, float) and not value.is_integer()):
+        return None
+    return parsed
 
 
-def _decimal_attr(attrs: dict[str, Any], *names: str) -> str | None:
-    for name in names:
-        value = attrs.get(name)
-        if value is None or not isinstance(value, (int, float, str)):
-            continue
-        try:
-            parsed = Decimal(str(value))
-        except InvalidOperation:
-            continue
-        if parsed.is_finite() and parsed >= 0:
-            return format(parsed, "f")
-    return None
+def _as_decimal(value: Any) -> str | None:
+    if value is None or isinstance(value, bool) or not isinstance(value, (int, float, str)):
+        return None
+    try:
+        parsed = Decimal(str(value))
+    except InvalidOperation:
+        return None
+    return format(parsed, "f") if parsed.is_finite() and parsed >= 0 else None
 
 
-def _currency_attr(attrs: dict[str, Any]) -> str | None:
-    value = attrs.get("llm.cost.currency")
+def _as_currency(value: Any) -> str | None:
     if not isinstance(value, str):
         return None
     normalized = value.upper()
     return normalized if re.fullmatch(r"[A-Z]{3,8}", normalized) else None
 
 
+def _first_readable(attrs: dict[str, Any], read: Any, *names: str) -> Any:
+    for name in names:
+        parsed = read(attrs.get(name))
+        if parsed is not None:
+            return parsed
+    return None
+
+
+def _unreadable(attrs: dict[str, Any], read: Any, *names: str) -> bool:
+    """The producer sent a value under one of ``names`` and none of them can be read."""
+    return _first_readable(attrs, read, *names) is None and any(
+        attrs.get(name) is not None for name in names
+    )
+
+
+_TOKEN_ATTRS = (
+    ("llm.token_count.prompt", "llm.token_count.input"),
+    ("llm.token_count.completion", "llm.token_count.output"),
+    ("llm.token_count.total",),
+)
+_COST_ATTRS = (
+    ("llm.cost.prompt", "llm.cost.input"),
+    ("llm.cost.completion", "llm.cost.output"),
+    ("llm.cost.total",),
+)
+_CURRENCY_ATTR = "llm.cost.currency"
+#: Stored in ``total_cost`` when the producer sent a total cost that cannot be read. It is not a
+#: decimal, so analysis withholds the run's total instead of summing the rest into a figure that
+#: is knowably too low. Fixed text: the producer's own value never reaches the artifact.
+UNREADABLE_COST = "unreadable"
+
+
+def _unreadable_telemetry(attrs: dict[str, Any]) -> dict[str, set[str]]:
+    """Which kinds of telemetry this span sent but tracegraph cannot read, and under which of
+    our fixed attribute names (never the producer's value)."""
+    groups = (
+        [("token count", _as_count, names) for names in _TOKEN_ATTRS]
+        + [("prompt or completion cost", _as_decimal, names) for names in _COST_ATTRS[:2]]
+        + [("total cost", _as_decimal, _COST_ATTRS[2])]
+        + [("cost currency", _as_currency, (_CURRENCY_ATTR,))]
+    )
+    found: dict[str, set[str]] = {}
+    for kind, read, names in groups:
+        if _unreadable(attrs, read, *names):
+            found.setdefault(kind, set()).update(n for n in names if attrs.get(n) is not None)
+    return found
+
+
 def _digest_attr(attrs: dict[str, Any], name: str) -> str | None:
     value = attrs.get(name)
     if value is None:
@@ -294,15 +336,16 @@ def _evidence(
     evidence = StepEvidence(
         end_ts=_iso_ts(end),
         duration_ms=duration_ms,
-        prompt_tokens=_int_attr(attrs, "llm.token_count.prompt", "llm.token_count.input"),
-        completion_tokens=_int_attr(
-            attrs, "llm.token_count.completion", "llm.token_count.output"
+        prompt_tokens=_first_readable(attrs, _as_count, *_TOKEN_ATTRS[0]),
+        completion_tokens=_first_readable(attrs, _as_count, *_TOKEN_ATTRS[1]),
+        total_tokens=_first_readable(attrs, _as_count, *_TOKEN_ATTRS[2]),
+        prompt_cost=_first_readable(attrs, _as_decimal, *_COST_ATTRS[0]),
+        completion_cost=_first_readable(attrs, _as_decimal, *_COST_ATTRS[1]),
+        total_cost=(
+            _first_readable(attrs, _as_decimal, *_COST_ATTRS[2])
+            or (UNREADABLE_COST if _unreadable(attrs, _as_decimal, *_COST_ATTRS[2]) else None)
         ),
-        total_tokens=_int_attr(attrs, "llm.token_count.total"),
-        prompt_cost=_decimal_attr(attrs, "llm.cost.prompt", "llm.cost.input"),
-        completion_cost=_decimal_attr(attrs, "llm.cost.completion", "llm.cost.output"),
-        total_cost=_decimal_attr(attrs, "llm.cost.total"),
-        cost_currency=_currency_attr(attrs),
+        cost_currency=_as_currency(attrs.get(_CURRENCY_ATTR)),
         artifact_digest=_digest_attr(attrs, "syncmill.artifact_digest"),
         evaluations=evaluations,
     )
@@ -471,6 +514,19 @@ class OTLPSpanAdapter:
             if unreadable
             else []
         )
+        dropped = [_unreadable_telemetry(attrs[sid]) for sid in by_id]
+        for kind, consequence in (
+            ("token count", "that count is reported as unavailable"),
+            ("prompt or completion cost", "that cost is reported as unavailable"),
+            ("total cost", "the run's total cost is withheld rather than understated"),
+            ("cost currency", "that cost is reported without a currency"),
+        ):
+            count = sum(1 for found in dropped if kind in found)
+            if count:
+                names = ", ".join(sorted(set().union(*(found.get(kind, set()) for found in dropped))))
+                warnings.append(
+                    f"{count} span(s) carry a {kind} that cannot be read ({names}); {consequence}."
+                )
         return RawTrace(trace=trace, steps=steps, causal_edges=edges, ingest_warnings=warnings)
 
     # --- helpers ---

{agent_tracegraph-0.3.0 → agent_tracegraph-0.4.0}/src/tracegraph/adapters/phoenix_export.py

@@ -73,7 +73,19 @@ def _attrs(span: dict[str, Any]) -> dict[str, Any]:
     return safe
 
 
+def _as_score(value: Any) -> float | None:
+    if value is None:
+        return None
+    try:
+        score = float(value)
+    except (TypeError, ValueError, OverflowError):
+        return None
+    return score if math.isfinite(score) else None
+
+
 def _evaluations(span: dict[str, Any]) -> list[dict[str, Any]]:
+    """The span's evaluations, each with ``unreadable_score`` set when a score was sent but
+    cannot be represented; the caller strips that key and discloses the count."""
     out: list[dict[str, Any]] = []
     annotations = span.get("annotations")
     if annotations is None:
@@ -93,16 +105,12 @@ def _evaluations(span: dict[str, Any]) -> list[dict[str, Any]]:
         if name is None:
             continue
         label = _first(result, "label") or annotation.get("label")
-        score = _first(result, "score")
-        if score is None:
-            score = annotation.get("score")
-        try:
-            score = float(score) if score is not None else None
-        except (TypeError, ValueError, OverflowError):
-            score = None
-        if score is not None and not math.isfinite(score):
-            score = None
-        out.append({"name": name, "label": _safe_operation_name(label), "score": score})
+        sent = _first(result, "score")
+        if sent is None:
+            sent = annotation.get("score")
+        score = _as_score(sent)
+        out.append({"name": name, "label": _safe_operation_name(label), "score": score,
+                    "unreadable_score": sent is not None and score is None})
     return out
 
 
@@ -153,8 +161,11 @@ class PhoenixExportAdapter:
         except KeyError as exc:
             raise KeyError(f"no Phoenix trace {trace_id!r}") from exc
         spans = [self._span(trace_id, span) for span in trace["spans"]]
+        unreadable_scores = sum(
+            item.pop("unreadable_score") for span in spans for item in span["_tracegraph_evaluations"]
+        )
         document = {"resourceSpans": [{"scopeSpans": [{"spans": spans}]}]}
-        return OTLPSpanAdapter(
+        raw = OTLPSpanAdapter(
             document,
             source_kind="phoenix_cli",
             links_as_causes=False,
@@ -162,6 +173,12 @@ class PhoenixExportAdapter:
             preserve_unset_status=True,
             evaluation_channel="_tracegraph_evaluations",
         ).ingest(trace_id)
+        if unreadable_scores:
+            raw.ingest_warnings.append(
+                f"{unreadable_scores} evaluation score(s) cannot be read as a finite number; "
+                "each is reported as null."
+            )
+        return raw
 
     @staticmethod
     def _trace_id(trace: dict[str, Any]) -> str:

{agent_tracegraph-0.3.0 → agent_tracegraph-0.4.0}/src/tracegraph/artifact.py

@@ -19,13 +19,13 @@ from tracegraph.input_validation import loads as json_loads
 from tracegraph.model import NormalizedTrace, StepSource
 
 #: Bump when the on-disk shape changes incompatibly.
-ARTIFACT_SCHEMA_VERSION = 4
+ARTIFACT_SCHEMA_VERSION = 5
 
 #: Versions this build can read. 1 needs a migration (see ``_migrate_v1``); 2 -> 3 is the
 #: identity, because 3 adds only a new ``Step.source`` value and changes nothing existing.
 #: 4 signals that TREE_PARENT was derived under EdgeOrigin precedence rather than
-#: legacy temporal-only selection.
-_READABLE_VERSIONS = (1, 2, 3, ARTIFACT_SCHEMA_VERSION)
+#: legacy temporal-only selection. 5 adds only the ``interrupt`` ``Step.source`` value.
+_READABLE_VERSIONS = (1, 2, 3, 4, ARTIFACT_SCHEMA_VERSION)
 
 
 def _required_version(nt: NormalizedTrace) -> int:
@@ -38,6 +38,9 @@ def _required_version(nt: NormalizedTrace) -> int:
     bump would invalidate digests already exported in review-candidate reports for artifacts
     whose content did not change at all.
 
+    * An artifact that carries an interrupt step is stamped 5, whatever its projection: a
+      v5 reader also reads the v4 projection, and an older reader would otherwise fail on
+      the unknown enum value instead of refusing the version by name.
     * An artifact whose derived TREE_PARENT relies on EdgeOrigin priority is stamped 4,
       so older readers (which used temporal-only parent selection) refuse it with an
       explicit "unsupported artifact schema_version 4" rather than failing canonical
@@ -47,6 +50,9 @@ def _required_version(nt: NormalizedTrace) -> int:
     """
     from tracegraph.normalize import requires_origin_priority_projection
 
+    interrupt = StepSource.INTERRUPT
+    if any(step.source is interrupt for step in nt.steps):
+        return 5
     if requires_origin_priority_projection(nt):
         return 4
     task = StepSource.TASK

{agent_tracegraph-0.3.0 → agent_tracegraph-0.4.0}/src/tracegraph/model.py

@@ -54,10 +54,15 @@ class StepKind(str, Enum):
 class StepSource(str, Enum):
     """How the step entered the trace (mirrors LangGraph ``metadata.source``).
 
-    ``TASK`` is the one value that does *not* come from ``metadata.source``: it marks a step
+    ``TASK`` and ``INTERRUPT`` do *not* come from ``metadata.source``. ``TASK`` marks a step
     derived from a checkpoint's **pending writes** (one LangGraph task) rather than from a
     checkpoint itself. It is explicit provenance, not a naming convention — ``step_id`` is an
     unrestricted string, so nothing about an id may be read as evidence of how a step was made.
+
+    ``INTERRUPT`` is likewise derived from pending writes: a LangGraph task that called ``interrupt()`` and
+    for which the recorded history holds no resume and no result. It states what the
+    checkpoints record, not that LangGraph would resume the task today — a thread re-run under
+    a revised graph can finish without writing a checkpoint, leaving the record unchanged.
     """
 
     INPUT = "input"
@@ -65,6 +70,7 @@ class StepSource(str, Enum):
     UPDATE = "update"
     FORK = "fork"
     TASK = "task"
+    INTERRUPT = "interrupt"
 
 
 class StepStatus(str, Enum):