agent-tracegraph 0.4.0__tar.gz → 0.5.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {agent_tracegraph-0.4.0 → agent_tracegraph-0.5.0}/CHANGELOG.md +223 -0
- {agent_tracegraph-0.4.0 → agent_tracegraph-0.5.0}/PKG-INFO +115 -16
- {agent_tracegraph-0.4.0 → agent_tracegraph-0.5.0}/README.md +114 -15
- {agent_tracegraph-0.4.0 → agent_tracegraph-0.5.0}/SECURITY.md +56 -10
- agent_tracegraph-0.5.0/contracts/check-report.schema.json +396 -0
- {agent_tracegraph-0.4.0 → agent_tracegraph-0.5.0}/pyproject.toml +5 -1
- agent_tracegraph-0.5.0/src/tracegraph/adapters/base.py +59 -0
- {agent_tracegraph-0.4.0 → agent_tracegraph-0.5.0}/src/tracegraph/adapters/langgraph_checkpoint.py +294 -28
- {agent_tracegraph-0.4.0 → agent_tracegraph-0.5.0}/src/tracegraph/adapters/otlp_spans.py +111 -32
- {agent_tracegraph-0.4.0 → agent_tracegraph-0.5.0}/src/tracegraph/adapters/phoenix_export.py +7 -2
- agent_tracegraph-0.5.0/src/tracegraph/analysis/check.py +601 -0
- {agent_tracegraph-0.4.0 → agent_tracegraph-0.5.0}/src/tracegraph/analysis/patterns.py +117 -72
- {agent_tracegraph-0.4.0 → agent_tracegraph-0.5.0}/src/tracegraph/artifact.py +55 -13
- {agent_tracegraph-0.4.0 → agent_tracegraph-0.5.0}/src/tracegraph/cli.py +773 -126
- agent_tracegraph-0.5.0/src/tracegraph/graph_ref.py +126 -0
- {agent_tracegraph-0.4.0 → agent_tracegraph-0.5.0}/src/tracegraph/input_validation.py +8 -1
- {agent_tracegraph-0.4.0 → agent_tracegraph-0.5.0}/src/tracegraph/model.py +163 -2
- {agent_tracegraph-0.4.0 → agent_tracegraph-0.5.0}/src/tracegraph/sqlite_snapshot.py +7 -2
- {agent_tracegraph-0.4.0 → agent_tracegraph-0.5.0}/src/tracegraph/store/ladybug.py +17 -2
- agent_tracegraph-0.4.0/src/tracegraph/adapters/base.py +0 -22
- {agent_tracegraph-0.4.0 → agent_tracegraph-0.5.0}/.gitignore +0 -0
- {agent_tracegraph-0.4.0 → agent_tracegraph-0.5.0}/LICENSE +0 -0
- {agent_tracegraph-0.4.0 → agent_tracegraph-0.5.0}/contracts/analysis-report.schema.json +0 -0
- {agent_tracegraph-0.4.0 → agent_tracegraph-0.5.0}/contracts/phoenix-runtime.json +0 -0
- {agent_tracegraph-0.4.0 → agent_tracegraph-0.5.0}/contracts/review-candidates.schema.json +0 -0
- {agent_tracegraph-0.4.0 → agent_tracegraph-0.5.0}/src/tracegraph/__init__.py +0 -0
- {agent_tracegraph-0.4.0 → agent_tracegraph-0.5.0}/src/tracegraph/_atomic_write.py +0 -0
- {agent_tracegraph-0.4.0 → agent_tracegraph-0.5.0}/src/tracegraph/adapters/__init__.py +0 -0
- {agent_tracegraph-0.4.0 → agent_tracegraph-0.5.0}/src/tracegraph/analysis/__init__.py +0 -0
- {agent_tracegraph-0.4.0 → agent_tracegraph-0.5.0}/src/tracegraph/analysis/ahu.py +0 -0
- {agent_tracegraph-0.4.0 → agent_tracegraph-0.5.0}/src/tracegraph/analysis/diagnose.py +0 -0
- {agent_tracegraph-0.4.0 → agent_tracegraph-0.5.0}/src/tracegraph/analysis/explain.py +0 -0
- {agent_tracegraph-0.4.0 → agent_tracegraph-0.5.0}/src/tracegraph/analysis/memo.py +0 -0
- {agent_tracegraph-0.4.0 → agent_tracegraph-0.5.0}/src/tracegraph/normalize.py +0 -0
- {agent_tracegraph-0.4.0 → agent_tracegraph-0.5.0}/src/tracegraph/py.typed +0 -0
- {agent_tracegraph-0.4.0 → agent_tracegraph-0.5.0}/src/tracegraph/review_candidates.py +0 -0
- {agent_tracegraph-0.4.0 → agent_tracegraph-0.5.0}/src/tracegraph/store/__init__.py +0 -0
- {agent_tracegraph-0.4.0 → agent_tracegraph-0.5.0}/src/tracegraph/store/base.py +0 -0
- {agent_tracegraph-0.4.0 → agent_tracegraph-0.5.0}/src/tracegraph/store/in_memory.py +0 -0
|
@@ -6,6 +6,229 @@ The **artifact** schema version and the **report**/**review-candidate** JSON sch
|
|
|
6
6
|
separate contracts from this version number; changes to any of them are called out
|
|
7
7
|
explicitly.
|
|
8
8
|
|
|
9
|
+
## 0.5.0 - 2026-10-18
|
|
10
|
+
|
|
11
|
+
### Added
|
|
12
|
+
|
|
13
|
+
- **Dropped OTLP span links are disclosed** (#49). A link the OTLP importer could not record as a
|
|
14
|
+
cause was skipped silently, while the header still read `causal_fidelity=declared_dag` and
|
|
15
|
+
`links_preserved=true`. Each kind now prints one stderr line with its count and becomes a
|
|
16
|
+
capture gap: `otlp.unresolved_span_link` (names this trace but no span in the export,
|
|
17
|
+
possibly a partial export), `otlp.unscoped_span_link` (names no trace, missing or empty `traceId`; it may
|
|
18
|
+
refer to a span of this trace, so it is no longer treated as foreign) and
|
|
19
|
+
`otlp.foreign_trace_link` (names another trace). `check` treats the first two as gaps that may
|
|
20
|
+
have left a step out (one-sided for the count rules, insufficient for topology) and the third
|
|
21
|
+
as having no effect. The schema stays v7. Only artifacts ingested from exports with such links
|
|
22
|
+
change, and with them the digests of reports made from them; every committed fixture resolves
|
|
23
|
+
all its links. An artifact holding none of these codes reads and checks as before.
|
|
24
|
+
- **`--strict` for `query` and `export-review-candidates`** (#50). A `*.json` file in a directory
|
|
25
|
+
that is not a valid artifact is skipped with a warning, so a truncated artifact holding the
|
|
26
|
+
failure let a "pattern must not appear" gate pass with exit 1 ("no matches"), and let
|
|
27
|
+
`export-review-candidates` write a report without it. With `--strict`, any skipped directory
|
|
28
|
+
file fails the command with exit 2 (`--strict: N discovered file(s) could not be loaded as
|
|
29
|
+
tracegraph artifacts`) after the usual skip warning, and no report is written. An explicitly named bad
|
|
30
|
+
file still fails first, as before; `--strict` is checked before "no valid artifacts" and
|
|
31
|
+
duplicate trace ids. Without the flag nothing changes, and with it and no strays the output
|
|
32
|
+
is byte-identical.
|
|
33
|
+
- **`tracegraph check`, an explicit regression verdict** (#59): one current run, one baseline the
|
|
34
|
+
caller names, and rules the caller names (`failures-not-increased`,
|
|
35
|
+
`pattern-not-increased:<preset>`, `topology-unchanged[:shape]`; no default set). Exits 0 PASS,
|
|
36
|
+
1 FAIL, 3 INSUFFICIENT_EVIDENCE, 2 invalid input (no report written). Rules compare counts
|
|
37
|
+
per step `(kind, name)`; unknown evidence the selected rule depends on — an `unset` status, a
|
|
38
|
+
substituted value or missing name on a step it could count, a capture gap that may have
|
|
39
|
+
left a step out of the artifact — blocks PASS on the current side and
|
|
40
|
+
FAIL on the baseline side instead of counting as zero (a step-omitting gap on either side makes
|
|
41
|
+
the topology rule insufficient, since it compares trees, not counts). Pairs from different
|
|
42
|
+
sources, or with either artifact lacking capture provenance (written before artifact v7),
|
|
43
|
+
are insufficient for every rule, and pairs read with different causal settings for the
|
|
44
|
+
rules that read causal structure. New contract `tracegraph.check` v1
|
|
45
|
+
(`contracts/check-report.schema.json`), bound to both canonical artifact digests and each
|
|
46
|
+
rule's id, version and parameters. `diff`, `query` and `analyze` keep their exit codes.
|
|
47
|
+
- **Capture provenance, artifact schema version 7** (groundwork for #59): every ingest now
|
|
48
|
+
records in the artifact what the adapter could not observe, so a later comparison can tell
|
|
49
|
+
"recorded, nothing disclosed" from "not recorded". `trace.capture.gaps` keeps one fixed code and count
|
|
50
|
+
per ingest disclosure (the stderr text is unchanged and stays out of the artifact; the code
|
|
51
|
+
carries no checkpoint id, task id or producer value), and `capture` is present with an empty
|
|
52
|
+
list when the read raised no disclosure; values substituted on one step are recorded
|
|
53
|
+
separately. `steps[].unknown_fields` names values an adapter substituted:
|
|
54
|
+
`kind` and `name` (a span without a recognized OpenInference kind or a usable name, including
|
|
55
|
+
Phoenix names dropped as non-identifiers; an unattributed LangGraph task; and every LangGraph
|
|
56
|
+
checkpoint step except the input checkpoint and the loop step that applies it, named or not,
|
|
57
|
+
because a checkpoint does not record its writer and its name is inferred from the parent's
|
|
58
|
+
routing) and `error_msg` (OTLP's `"error"` fallback, withheld messages, LangGraph's
|
|
59
|
+
unserialized-exception label). Task, interrupt and scheduled steps named from LangGraph's own
|
|
60
|
+
task ids are not flagged. Two OTLP parent resolutions that were silent are now recorded too: a `graph.node.parent_id` resolved by start time among
|
|
61
|
+
several spans, and one not honored in favour of `parentSpanId`; each also prints a new stderr
|
|
62
|
+
warning. Every newly ingested artifact declares 7 and 0.4.x refuses it by name; an artifact
|
|
63
|
+
written by an earlier release re-serializes exactly as the previous release wrote it (v2–v6
|
|
64
|
+
byte for byte, v1 migrated to v2 as before) and never gains provenance. Newly ingested
|
|
65
|
+
artifacts therefore have new digests, as do analysis reports and incident memos built from
|
|
66
|
+
them; report and memo shapes, other CLI output and exit codes are unchanged.
|
|
67
|
+
- **`analyze` reads OTLP span exports directly** (#57): a JSON or Collector JSONL export
|
|
68
|
+
(`resourceSpans`) is diagnosed in memory with the same adapter settings as `ingest-otlp`, so
|
|
69
|
+
the report is byte-identical to analysing the ingested artifact; `--trace` selects one trace of
|
|
70
|
+
a multi-trace export and `--baseline` accepts an OTLP export too. Inputs are routed by shape
|
|
71
|
+
before any adapter runs (`schema_version` → artifact, `resourceSpans` → OTLP, `spans` or a
|
|
72
|
+
root `traces`/`data` list → Phoenix); record validity stays the adapter's call, so the direct
|
|
73
|
+
path accepts and rejects what the explicit ingest commands do. A payload carrying both OTLP
|
|
74
|
+
and Phoenix records exits 2: a single object names the ingest command that reads each format,
|
|
75
|
+
and a list, JSONL or Phoenix envelope says to extract one format's records first. An unknown
|
|
76
|
+
shape exits 2 listing the supported ones instead of failing as a Phoenix export.
|
|
77
|
+
`ingest-otlp --file -` reads stdin, and stdin now goes through the same
|
|
78
|
+
`TRACEGRAPH_MAX_ARTIFACT_BYTES` ceiling as files for `analyze`, `ingest-otlp` and
|
|
79
|
+
`ingest-phoenix` (the stdin half of #53; the `px` output half is the "Phoenix `px` output is bounded" entry below);
|
|
80
|
+
`ingest-phoenix` reads its input with the Phoenix adapter only (its accepted inputs are
|
|
81
|
+
unchanged). Next investigation steps after a direct OTLP
|
|
82
|
+
diagnosis print an `ingest-otlp` save template. No file is written without `--json-out`;
|
|
83
|
+
analysis JSON schemas, exit codes and privacy boundaries are unchanged.
|
|
84
|
+
- **Guided investigation after `analyze` and `phoenix diagnose`** (#58): displayed failure
|
|
85
|
+
candidates include full step IDs, with POSIX-quoted `explain` commands for existing artifacts
|
|
86
|
+
or explicitly labelled save/replay templates for exports, stdin, and unsaved Phoenix traces.
|
|
87
|
+
Evidence IDs, paths and commands preserve literal emoji-like tokens and are not forcibly
|
|
88
|
+
line-wrapped; consumed pipe inputs and descriptor aliases (`/dev/stdin`, `/dev/fd/N`,
|
|
89
|
+
`/proc/<pid>/fd/N`, or a symlink to one, including regular-file stdin redirects; a symlink
|
|
90
|
+
to a directory descriptor is not detected) require replay. Nonprinting arguments are shown as
|
|
91
|
+
escaped data instead of copyable commands. Hidden
|
|
92
|
+
candidates show the required `--limit`; report paths are acknowledged only after successful
|
|
93
|
+
writes, without duplicate report-path notices. Existing edge-origin/fidelity warnings and
|
|
94
|
+
missing-telemetry disclosures continue to distinguish evidence from causal claims; this
|
|
95
|
+
change adds navigation, not new causal inference. Analysis JSON schemas, exit codes, and
|
|
96
|
+
artifact privacy boundaries are unchanged.
|
|
97
|
+
- **`ingest --graph` reports LangGraph tasks that were scheduled but recorded nothing.**
|
|
98
|
+
Given the compiled graph (`path/to/file.py:attr` or `pkg.module:attr`), LangGraph's
|
|
99
|
+
`get_state` lists the tasks it schedules at the thread's newest root checkpoint, and each one
|
|
100
|
+
with no write there becomes an `unset` step with the new source value `scheduled`, linked by
|
|
101
|
+
`CAUSED_BY` to that checkpoint: the recursion limit, a killed process, a static breakpoint,
|
|
102
|
+
one step per `Send` packet. Finished, failed and paused tasks are still read from the
|
|
103
|
+
checkpoint's writes, so these steps never duplicate `task` or `interrupt` steps. The result
|
|
104
|
+
is LangGraph's answer for the supplied graph; the trace header records its structural
|
|
105
|
+
fingerprint as `schedule_graph`, also when nothing is scheduled. An unapplied
|
|
106
|
+
`Command(goto=...)` input and `Send` packets to a node the graph lacks are disclosed at
|
|
107
|
+
ingest. `--graph` imports the module and runs the graph's state callbacks while reading,
|
|
108
|
+
but no node. `inspect` marks such a step `◌ scheduled` and adds `· N scheduled` to the
|
|
109
|
+
summary.
|
|
110
|
+
- **Artifact schema version 6**, stamped by content: an artifact read with `--graph` (it has
|
|
111
|
+
`schedule_graph`, or a `scheduled` step) declares 6, and 0.4.x refuses it by name. The header
|
|
112
|
+
field is omitted, not written as `null`. (A fresh ingest by this release declares 7 for its
|
|
113
|
+
capture record, which outranks 6.)
|
|
114
|
+
- **A hands-on getting-started notebook in Korean**, `examples/getting_started_ko.ipynb`, for
|
|
115
|
+
first-time users working from a repository checkout. It runs the example agent and the
|
|
116
|
+
offline commands other than `check` and `export-review-candidates`, interprets each output, and works through two applied scenarios:
|
|
117
|
+
checking a change against a baseline, and aggregating failures across many runs from
|
|
118
|
+
`analyze --json-out` reports. Its outputs are committed, and `tests/test_notebooks.py`
|
|
119
|
+
executes it so its commands and exit codes cannot drift from the CLI. `nbclient` and
|
|
120
|
+
`ipykernel` join the development dependencies. Nothing ships in the wheel or sdist, and no
|
|
121
|
+
artifact or report contract changes.
|
|
122
|
+
|
|
123
|
+
### Changed
|
|
124
|
+
|
|
125
|
+
- **`artifact.save()` and `artifact.save_atomic()` refuse a trace the readers would reject**
|
|
126
|
+
(#67). Pydantic does not revalidate on attribute assignment, so a `NormalizedTrace` changed in
|
|
127
|
+
place could be written in a form tracegraph's own readers refuse: `save()` wrote a mistyped
|
|
128
|
+
`status`, and both wrote a `CAUSED_BY` edge to a missing step (which `artifact.load` accepted
|
|
129
|
+
and only `validate` / `analyze` rejected), a trace status that disagrees with its steps, a
|
|
130
|
+
wrong `projection_lossy` flag, or a missing `TREE_PARENT`. Both writers now round-trip the
|
|
131
|
+
model and run `validate_normalized`, the check the stores and `validate` apply on load
|
|
132
|
+
(legacy projections included), before touching the destination, and raise `ValueError`
|
|
133
|
+
(pydantic's `ValidationError` is one), leaving an existing file untouched. A mistyped `seq` used to escape
|
|
134
|
+
every writer as a raw `TypeError`; it is now a `ValidationError` too. A valid trace is written
|
|
135
|
+
byte for byte as before, from the validated copy, so a value the model coerces (`"0"` for a
|
|
136
|
+
`seq`) is written as it would be read. Cost on a 6000-step chain: `save()` 0.16 s → 0.29 s,
|
|
137
|
+
`save_atomic()` 0.19 s → 0.30 s; `ingest` already validated before saving, so it now checks
|
|
138
|
+
twice. Because the model now checks first, a capture-provenance
|
|
139
|
+
violation reaches a writer's caller as pydantic's `ValidationError` rather than the
|
|
140
|
+
`check_provenance` message (`artifact.dumps()` still raises the latter). `artifact.dumps()`
|
|
141
|
+
stays a serializer without the model and structure checks (its `TypeError` on a mistyped
|
|
142
|
+
`seq` is #82).
|
|
143
|
+
- `inspect` marks a step paused by `interrupt()` as `⏸ interrupted`, in yellow rather than
|
|
144
|
+
the green used for finished steps, and the summary line gains `· N interrupted` when there
|
|
145
|
+
is one. The output for every trace with neither interrupted nor scheduled steps is unchanged.
|
|
146
|
+
|
|
147
|
+
### Fixed
|
|
148
|
+
|
|
149
|
+
- **A LangGraph node writing the error channel while it is still set is disclosed** (#47). A
|
|
150
|
+
checkpoint is an error only where the error channel is set and every causal parent's is
|
|
151
|
+
empty, so a node failing again before the key was cleared stayed `ok`, silently, and the
|
|
152
|
+
retry presets could not match. The rule is unchanged: a checkpoint does not record which
|
|
153
|
+
node raised, and both a value-change rule and the issue's version rule were measured to
|
|
154
|
+
invent failures (a node returning the whole state, a reducer that grows the value, a
|
|
155
|
+
handler rewording the error). Instead, a node-written checkpoint whose error channel was
|
|
156
|
+
written again while set (its `channel_versions` entry moved, or without versions its value
|
|
157
|
+
changed) counts toward a stderr disclosure and the capture gap
|
|
158
|
+
`langgraph.rewritten_error_channel`. `check` reads it for the count rules (one-sided on that
|
|
159
|
+
side) and for success-dependent patterns such as `gate-failure-after-success` (insufficient
|
|
160
|
+
on either side), not for topology. Input, `__start__`, update and fork-copy checkpoints, and
|
|
161
|
+
parents in another namespace (a subgraph's entry or exit), are not compared, so propagation
|
|
162
|
+
and a caller's input disclose nothing; a propagated error recorded again at a subgraph's
|
|
163
|
+
`__start__` is pre-existing and filed as #85. Measured on LangGraph 1.2.2 and 1.2.15 with
|
|
164
|
+
InMemorySaver and SqliteSaver. Artifacts change only for histories with such writes; the
|
|
165
|
+
committed fixture has none. Schema stays v7.
|
|
166
|
+
- **`analyze --trace` and `--baseline-trace` no longer ignore the selector on an artifact**
|
|
167
|
+
(#64). An artifact holds one trace, and a selector naming a different one was accepted: the
|
|
168
|
+
command exited 0 with a report on the artifact's trace, not the one requested. It now exits 2
|
|
169
|
+
with `Invalid value for --trace: <input> holds trace '<id>', not '<selector>'` (or
|
|
170
|
+
`--baseline-trace`) and writes no report, as `check` already did and as a missing trace in an
|
|
171
|
+
OTLP or Phoenix export does. A selector naming the artifact's own trace is still accepted, and
|
|
172
|
+
the report is byte-identical to the one without it.
|
|
173
|
+
- **`ingest-phoenix --trace` no longer ignores the selector on a Phoenix-sourced artifact** (#69).
|
|
174
|
+
Re-ingesting such an artifact with a selector naming another trace exited 0 and re-saved the
|
|
175
|
+
artifact's own trace. It now exits 2 with the same message as `analyze` and writes nothing, to
|
|
176
|
+
`--out` or to the default path. A matching selector re-saves the artifact byte for byte, as
|
|
177
|
+
before. An artifact from any other source keeps its existing error, `--file must contain a
|
|
178
|
+
Phoenix CLI trace export`.
|
|
179
|
+
- **`phoenix doctor` and `phoenix diagnose` no longer crash when `px` cannot be run** (#56). A
|
|
180
|
+
`px` on PATH without the exec bit, a directory named `px`, or a file the OS cannot execute
|
|
181
|
+
raised `PermissionError` or `OSError` with a traceback. They now report that `px` failed with
|
|
182
|
+
an OS error, naming the operating system's text for the error number and never the exception
|
|
183
|
+
message: `doctor` prints `FAIL` and exits 1, `diagnose` exits 2, as for their other `px`
|
|
184
|
+
failures. When the OS refused to launch `px`, the message adds a hint to check that `px` on
|
|
185
|
+
PATH is an executable file.
|
|
186
|
+
- **Phoenix `px` output is bounded** (#53, the half #63 left open). `phoenix doctor` and
|
|
187
|
+
`phoenix diagnose` held all of `px`'s stdout and stderr in memory with only a time limit. Stdout
|
|
188
|
+
now goes to an unlinked temporary file held to the same `TRACEGRAPH_MAX_ARTIFACT_BYTES` ceiling
|
|
189
|
+
as file and stdin input (default 256 MiB, `0` disables it). `px` is killed and reaped once the
|
|
190
|
+
file passes the limit, and the command reports `... output exceeded the N-byte limit` without
|
|
191
|
+
echoing the output: doctor `FAIL` exit 1, diagnose exit 2. Stderr is discarded; it was never
|
|
192
|
+
shown. Two smaller changes come with it. Output that is not UTF-8 is now reported as such:
|
|
193
|
+
before, `doctor` printed the codec error and `diagnose` crashed with a traceback. And a
|
|
194
|
+
descendant of `px` that keeps stdout open no longer holds the call until the timeout, and
|
|
195
|
+
the capture stops at the size sampled once `px` has exited, read by position on POSIX so a
|
|
196
|
+
descendant's later writes cannot overwrite it. Such descendants are not
|
|
197
|
+
stopped: `px` is trusted like any installed executable. Timeout, nonzero-exit and
|
|
198
|
+
launch-error messages are unchanged.
|
|
199
|
+
- **`SECURITY.md` now states the `ingest --graph` trust boundary** (#55). It said tracegraph
|
|
200
|
+
"does not execute graphs" and called a checkpoint database the one input that can run code,
|
|
201
|
+
but `--graph` imports the module you name, running its top-level code, and then runs the
|
|
202
|
+
graph's state callbacks while reading. A new section says what runs (no node does), that
|
|
203
|
+
checkpoint payloads stay under the empty serializer allowlist while the named code itself is
|
|
204
|
+
unsandboxed, and that the graph is never inferred. A table at the top separates JSON parsing,
|
|
205
|
+
hardened checkpoint deserialization and explicitly supplied graph code. The same file no
|
|
206
|
+
longer says piped stdin is read without a size cap: since direct OTLP analysis (#63), `-` goes
|
|
207
|
+
through the same `TRACEGRAPH_MAX_ARTIFACT_BYTES` ceiling as a named file. Documentation only.
|
|
208
|
+
- **`query` no longer keeps every trace and every match in memory** (part of #54). It loaded
|
|
209
|
+
and validated all artifacts, matched them all, and only then sliced the result to `--limit`,
|
|
210
|
+
so `--limit 1` over 200 artifacts of 2000 steps (248 MB on disk) peaked at 1374 MiB. It now
|
|
211
|
+
loads, validates and matches one artifact after another, keeps only the matches it prints
|
|
212
|
+
(and, with `--explain`, the traces they belong to), and builds labels only for those. With
|
|
213
|
+
`--limit`, peak memory grows with the number of artifacts only by a small per-file record:
|
|
214
|
+
70 MiB for the same query. Without `--limit` it holds the matches it prints: 105 MiB. Output, exit codes and error precedence are
|
|
215
|
+
unchanged: every input is still validated, a later file's load error or a duplicate
|
|
216
|
+
trace_id still wins over a matching failure, and the footer's total stays exact. Matches past
|
|
217
|
+
`--limit` are counted but no longer labelled, so a labelling fault there (an internal bug:
|
|
218
|
+
labels look up step ids the matcher just returned) would no longer surface. One trace
|
|
219
|
+
with very many matches still enumerates them all (3.1M matches: 971 MiB, down from 986 MiB
|
|
220
|
+
but 5.4 s instead of 17 s); bounding that needs a decision about the `--limit` total and is
|
|
221
|
+
tracked as #79. An interrupt or `MemoryError` now surfaces at the artifact being read,
|
|
222
|
+
after the skip warnings collected so far, rather than after every file has loaded.
|
|
223
|
+
- **`repeated-agent-failure` no longer takes quadratic time when no names can match** (#52).
|
|
224
|
+
The unbounded two-step matcher it shares with `tool-repeat-failure-heuristic`, and which
|
|
225
|
+
`analyze` runs on every trace, walked and sorted every failing endpoint's whole causal
|
|
226
|
+
history before checking the name back-reference: a 2000-step chain of failing agents with distinct names
|
|
227
|
+
took 1.3s for zero matches, 4x per doubling. It now skips an endpoint no earlier candidate
|
|
228
|
+
can pair with and walks back only as far as the earliest one (2000 steps: 0.001s; a chain
|
|
229
|
+
whose only repeats are disjoint adjacent pairs is linear too). The longest-path pass also stopped rescanning a
|
|
230
|
+
high-fan-out cause once per endpoint. Matches, their order and every report are unchanged.
|
|
231
|
+
|
|
9
232
|
## 0.4.0 - 2026-10-14
|
|
10
233
|
|
|
11
234
|
### Added
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.5
|
|
2
2
|
Name: agent-tracegraph
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.5.0
|
|
4
4
|
Summary: Time-travel debugging and causal RCA for LangGraph agent traces, on a causal graph.
|
|
5
5
|
Project-URL: Homepage, https://github.com/memtomem/tracegraph
|
|
6
6
|
Project-URL: Repository, https://github.com/memtomem/tracegraph
|
|
@@ -45,7 +45,9 @@ then runs analyses the mainstream observability tools don't:
|
|
|
45
45
|
It is **not** a checkpointer — a graph-backed `BaseCheckpointSaver` already exists.
|
|
46
46
|
tracegraph is the read-path *analysis* layer. See [`docs/FEASIBILITY.md`](docs/FEASIBILITY.md)
|
|
47
47
|
for the full rationale and competitive landscape. Korean readers can start with
|
|
48
|
-
[`docs/USAGE_KO.md`](docs/USAGE_KO.md)
|
|
48
|
+
[`docs/USAGE_KO.md`](docs/USAGE_KO.md), or run the hands-on notebook
|
|
49
|
+
[`examples/getting_started_ko.ipynb`](examples/getting_started_ko.ipynb) from a repository
|
|
50
|
+
checkout.
|
|
49
51
|
Current delivery evidence, known limitations, and next work are maintained in
|
|
50
52
|
[`docs/HANDOFF.md`](docs/HANDOFF.md).
|
|
51
53
|
|
|
@@ -94,6 +96,7 @@ tracegraph explain A.json <step_id> # raw causal chain back to the root cause
|
|
|
94
96
|
tracegraph diff A.json B.json # structural regression diff (AHU isomorphism)
|
|
95
97
|
tracegraph validate A.json traces/ # validate artifacts before CI/query jobs
|
|
96
98
|
tracegraph analyze A.json # automatically select and explain failures
|
|
99
|
+
tracegraph check B.json --baseline A.json --rule failures-not-increased # regression verdict for CI
|
|
97
100
|
|
|
98
101
|
# Phoenix: one command after configuring the official `px` CLI
|
|
99
102
|
tracegraph phoenix doctor
|
|
@@ -119,13 +122,58 @@ Frequently-used options, beyond `--out` / `--json-out`:
|
|
|
119
122
|
|
|
120
123
|
| option | commands | effect |
|
|
121
124
|
|---|---|---|
|
|
122
|
-
| `--trace`, `-t` | `ingest-otlp`, `ingest-phoenix`, `analyze` | Pick one trace out of a multi-trace export. |
|
|
125
|
+
| `--trace`, `-t` | `ingest-otlp`, `ingest-phoenix`, `analyze`, `check` | Pick one trace out of a multi-trace export. For `analyze`, `check` and `ingest-phoenix` on an artifact, it must name the artifact's own trace (exit 2 otherwise). |
|
|
123
126
|
| `--error-channel` | `ingest` | LangGraph state channel that signals a step error (default `error`). |
|
|
124
|
-
| `--baseline
|
|
127
|
+
| `--baseline` | `analyze`, `check`, `phoenix diagnose` | Compare against a known-good run: a file for `analyze` and `check`, a Phoenix trace id for `phoenix diagnose`. |
|
|
128
|
+
| `--baseline-trace` | `analyze`, `check` | Pick the baseline's trace out of a multi-trace file. On an artifact baseline, it must name its own trace (exit 2 otherwise). |
|
|
125
129
|
| `--limit`, `-l` | `analyze`, `phoenix diagnose`, `query` | Cap how many failure candidates or matches are rendered (default 3 for diagnosis). |
|
|
126
130
|
| `--explain` | `query` | Print each match's raw causal chain back to its root cause. |
|
|
127
131
|
| `--structure` | `diff` | Compare topology only, ignoring labels. |
|
|
128
132
|
| `--backend` | `query` | Use the optional LadybugDB Cypher accelerator instead of the pure-Python matcher. |
|
|
133
|
+
| `--strict` | `query`, `export-review-candidates` | Fail with exit 2 when a directory holds a `*.json` file that is not a valid artifact, instead of skipping it with a warning. Use it in CI gates, where a skipped corrupt run would otherwise read as "no match". |
|
|
134
|
+
| `--rule` | `check` | Rule to evaluate, repeatable; there is no default rule set. |
|
|
135
|
+
|
|
136
|
+
### Regression check
|
|
137
|
+
|
|
138
|
+
`check` judges one current run against one baseline you name, with rules you name, and answers
|
|
139
|
+
with an exit code a CI job can use directly:
|
|
140
|
+
|
|
141
|
+
```bash
|
|
142
|
+
tracegraph check current.json --baseline baseline.json \
|
|
143
|
+
--rule failures-not-increased --rule pattern-not-increased:tool-retry-failure \
|
|
144
|
+
--json-out check.json
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
| exit | verdict |
|
|
148
|
+
|---|---|
|
|
149
|
+
| 0 | `PASS` — every rule held on the recorded evidence |
|
|
150
|
+
| 1 | `FAIL` — some rule was definitely violated (more failures or matches, or a changed tree) |
|
|
151
|
+
| 3 | `INSUFFICIENT_EVIDENCE` — some rule could not decide, and none failed |
|
|
152
|
+
| 2 | invalid input or configuration; no report is written |
|
|
153
|
+
|
|
154
|
+
Rules: `failures-not-increased` (recorded errors per step `(kind, name)` and in total),
|
|
155
|
+
`pattern-not-increased:<preset>` (confirmed matches of a preset), and `topology-unchanged`
|
|
156
|
+
(derived tree with labels) or `topology-unchanged:shape`. Counts compare step signatures,
|
|
157
|
+
because no source records which step of one run is which step of another; two same-named steps
|
|
158
|
+
trading which one fails is not detected (`analyze --baseline` shows the positional view).
|
|
159
|
+
Evidence that is unknown is never read as zero. When something the selected rule depends on is
|
|
160
|
+
unknown — an `unset` status, a value the adapter substituted (`unknown_fields`) or a missing
|
|
161
|
+
name on a step the rule could count, or a capture gap that may have left a step out of the artifact — it blocks PASS
|
|
162
|
+
on the current side and FAIL on the baseline side, so the rule reports `INSUFFICIENT_EVIDENCE`.
|
|
163
|
+
The topology rule compares trees rather than counts, so such a gap on either side makes it
|
|
164
|
+
insufficient even when the observed trees match. Unknowns a rule cannot count leave it decided: an unset CHAIN does not block
|
|
165
|
+
`pattern-not-increased:tool-failure`, and a withheld error message blocks only `timeout`.
|
|
166
|
+
Pairs from different sources or ingested before capture provenance (artifact version 7) are
|
|
167
|
+
insufficient for every rule; pairs read with different causal settings, or with causal edges
|
|
168
|
+
chosen heuristically at ingest, are insufficient for the rules that read causal structure
|
|
169
|
+
(multi-step presets and topology). A step without a name is
|
|
170
|
+
treated as unknown wherever a rule compares names, and LangGraph checkpoint names are inferred,
|
|
171
|
+
so the labeled topology rule is insufficient for LangGraph runs; the shape rule is not.
|
|
172
|
+
`check` takes exactly one current and one baseline input (artifact, OTLP export, Phoenix export
|
|
173
|
+
or `-`), never a directory, and never picks a baseline. It also assumes each input is a
|
|
174
|
+
complete record of its run: an export missing a failing span cannot be told from a clean one.
|
|
175
|
+
The `--json-out` report is `tracegraph.check` v1 (`contracts/check-report.schema.json`), bound
|
|
176
|
+
to both canonical artifact digests and each rule's id, version and parameters.
|
|
129
177
|
|
|
130
178
|
## Phoenix-first workflow
|
|
131
179
|
|
|
@@ -198,10 +246,21 @@ For link-preserving analysis, use the optional Collector fan-out example at
|
|
|
198
246
|
It sends the normal stream to Phoenix and an allowlisted JSONL copy to tracegraph. Then run:
|
|
199
247
|
|
|
200
248
|
```bash
|
|
249
|
+
tracegraph analyze /tmp/tracegraph-otlp.jsonl --trace <trace-id> # diagnose in memory
|
|
201
250
|
tracegraph ingest-otlp --file /tmp/tracegraph-otlp.jsonl --trace <trace-id> -o full-dag.json
|
|
202
251
|
tracegraph analyze full-dag.json
|
|
203
252
|
```
|
|
204
253
|
|
|
254
|
+
`analyze` reads an OTLP span export (JSON or Collector JSONL) directly, with the same adapter
|
|
255
|
+
settings as `ingest-otlp`, so the report is byte-identical to analysing the ingested artifact;
|
|
256
|
+
nothing is written unless `--json-out` asks for a report. The input is identified by shape alone
|
|
257
|
+
(`schema_version` → artifact, `resourceSpans` → OTLP, `spans`/`traces`/`data` → Phoenix); a
|
|
258
|
+
payload carrying both OTLP and Phoenix records is refused, naming the explicit ingest command
|
|
259
|
+
for each format (or, for a list, JSONL or envelope, asking you to extract one format's records). Already have an export? The shortest offline first run is
|
|
260
|
+
`tracegraph analyze your-export.otlp.json` (add `--trace <id>` when the file holds several), then
|
|
261
|
+
copy the `explain` command it prints after saving the artifact it names. The artifact caveat
|
|
262
|
+
above still applies: `ingest-otlp` keeps error text, reports stay `safe-v1`.
|
|
263
|
+
|
|
205
264
|
The JSON analysis-report contract is versioned at
|
|
206
265
|
[`contracts/analysis-report.schema.json`](contracts/analysis-report.schema.json).
|
|
207
266
|
Schema v2 can also carry body-free Toolgraph preflight evidence: an exact SHA-256 artifact
|
|
@@ -253,6 +312,8 @@ id/version, tool key, and the SHA-256 digest of the exact normalized artifact th
|
|
|
253
312
|
There are no prompts, outputs, errors, step IDs, trace IDs, or local paths, and the command
|
|
254
313
|
never changes a Toolgraph manifest or policy. A matching artifact without `run_id` fails the
|
|
255
314
|
entire batch and leaves an existing output untouched; export never silently drops evidence.
|
|
315
|
+
A `*.json` file in a directory that is not a valid artifact is skipped with a warning, as for
|
|
316
|
+
`query`; pass `--strict` to fail the batch instead.
|
|
256
317
|
The public schema is a repository-vendored contract (not wheel package data) at
|
|
257
318
|
[`contracts/review-candidates.schema.json`](contracts/review-candidates.schema.json); G3
|
|
258
319
|
consumers should vendor that file explicitly.
|
|
@@ -289,11 +350,12 @@ and any change to them is called out in the [changelog](CHANGELOG.md).
|
|
|
289
350
|
|
|
290
351
|
Known limits:
|
|
291
352
|
|
|
292
|
-
- A LangGraph task that was scheduled but
|
|
353
|
+
- A LangGraph task that was scheduled but recorded nothing (a run stopped by the recursion
|
|
293
354
|
limit, killed, or held at an `interrupt_before`/`interrupt_after` breakpoint) leaves no
|
|
294
|
-
write
|
|
295
|
-
|
|
296
|
-
|
|
355
|
+
write. It is reported only when `ingest --graph` names the compiled graph (below); without
|
|
356
|
+
it such a run reads `ok` unless some other step recorded an error. Trace status stays `ok`
|
|
357
|
+
either way. A task paused by `interrupt()` is reported without a graph. Both are reported
|
|
358
|
+
only at the root graph: the subgraph node is named, not the node inside it.
|
|
297
359
|
- Phoenix exports carry no span links, so their artifacts are parent-only; additional fan-in
|
|
298
360
|
causes may be missing.
|
|
299
361
|
- Report privacy (`safe-v1`) applies to reports; OTLP and LangGraph artifacts keep the
|
|
@@ -369,7 +431,14 @@ source; mixing them can report a step whose status differs only by source as a b
|
|
|
369
431
|
change (`ok` to `unset`, or the reverse).
|
|
370
432
|
|
|
371
433
|
LangGraph ingestion reads both the configured error state channel and native task failures
|
|
372
|
-
from checkpoint pending writes. A
|
|
434
|
+
from checkpoint pending writes. A checkpoint is an error where the channel first holds an error.
|
|
435
|
+
A node that writes the channel again while it still holds one may have failed again, or only
|
|
436
|
+
have returned the whole state (and a reducer can change the value either way); the checkpoint
|
|
437
|
+
cannot tell which, so such a write is not recorded as a failure but disclosed as the capture
|
|
438
|
+
gap `langgraph.rewritten_error_channel`. That makes `check`'s count rules one-sided and
|
|
439
|
+
success-dependent patterns such as `gate-failure-after-success` insufficient either way.
|
|
440
|
+
Clear a plain (last-value) error key before retrying (`{"error": None}`) to have every attempt
|
|
441
|
+
recorded; a reducer on the channel decides whether `None` clears it. A node that raises becomes a derived `task` step under the
|
|
373
442
|
checkpoint that scheduled it, never a mark on that checkpoint, whose own name belongs to the
|
|
374
443
|
node that produced it. The failing node is named by recomputing LangGraph's task id from
|
|
375
444
|
checkpoint data; when nothing proves which node ran, the step stays unnamed rather than
|
|
@@ -384,23 +453,53 @@ that checkpoint: the recorded history holds no resume and no result for it. That
|
|
|
384
453
|
history, not whether LangGraph would resume the task today. An interrupt recorded together with
|
|
385
454
|
a resume or an error and no result is ambiguous in the checkpoint itself — under LangGraph 1.2.2
|
|
386
455
|
a task resumed with nothing to write looks exactly like one paused a second time — so it is
|
|
387
|
-
disclosed at ingest, not reported.
|
|
388
|
-
|
|
389
|
-
|
|
390
|
-
|
|
456
|
+
disclosed at ingest, not reported. Scheduled tasks need the compiled graph: `tracegraph ingest --sqlite trace.db --thread A
|
|
457
|
+
--graph path/to/agent.py:graph` (the `langgraph.json` form) or `--graph pkg.module:graph`.
|
|
458
|
+
LangGraph's own `get_state` then lists the tasks that graph schedules at the thread's newest
|
|
459
|
+
root checkpoint, and each one with no write there becomes an `unset` step with source
|
|
460
|
+
`scheduled`. Whether a task finished, failed or paused is still read from the checkpoint's
|
|
461
|
+
writes, so a `scheduled` step never duplicates a `task` or `interrupt` step. A `scheduled`
|
|
462
|
+
step does not claim that the task never started: a task killed mid-run records nothing
|
|
463
|
+
either. The answer belongs to the graph you pass: a different graph can schedule different
|
|
464
|
+
tasks from the same checkpoint, and nothing in one checkpoint proves which graph wrote it, so
|
|
465
|
+
the artifact records the graph's structural fingerprint (`trace.schedule_graph`) — even when
|
|
466
|
+
nothing is scheduled, which is how "checked, nothing pending" differs from "not checked".
|
|
467
|
+
A `Command(goto=...)` input saved on that checkpoint but not yet applied, and `Send` packets
|
|
468
|
+
addressed to a node the graph lacks, are left out of LangGraph's own answer; ingest discloses
|
|
469
|
+
both instead.
|
|
470
|
+
`--graph` runs code: it imports the module (top-level code runs), and reading the checkpoint
|
|
471
|
+
runs the graph's state callbacks — schema mappers and validators, reducers, custom channels,
|
|
472
|
+
managed values, cache-key functions — though no node runs. It is never inferred.
|
|
473
|
+
Every ingest records what the adapter could not observe: `trace.capture` is an object whose
|
|
474
|
+
`gaps` list holds a fixed code and count per ingest disclosure (empty when there was none),
|
|
475
|
+
and a step's `unknown_fields` names a kind, name or error message the adapter substituted. That
|
|
476
|
+
record makes a fresh artifact declare `schema_version` 7. The envelope otherwise follows
|
|
477
|
+
content, which is what an artifact written by an earlier release keeps when it is loaded and
|
|
478
|
+
saved again: a graph read (`--graph`) needs 6, interrupt steps 5, a derived tree that relies on
|
|
479
|
+
EdgeOrigin precedence 4, derived task steps 3, and anything else 2, byte for byte (a v1 file is
|
|
480
|
+
migrated to 2 as before).
|
|
391
481
|
Missing observed errors still do not
|
|
392
482
|
prove success: an artifact ingested by an older build, or one using an unrecognized checkpoint
|
|
393
483
|
layout, carries no derived step and looks exactly like a clean run, which is why the capture
|
|
394
484
|
disclosure is unconditional. Link preservation
|
|
395
|
-
means valid in-trace links only
|
|
485
|
+
means valid in-trace links only: `declared_dag` and `links_preserved` say how links were read,
|
|
486
|
+
not that every declared link resolved. A link that names this trace but no span in the export,
|
|
487
|
+
names no trace, or names another trace is not recorded as a cause; each kind is disclosed on
|
|
488
|
+
stderr and recorded as a capture gap (`otlp.unresolved_span_link`, `otlp.unscoped_span_link`,
|
|
489
|
+
`otlp.foreign_trace_link`), and `check` treats the first two like a step that may be missing. Structural diff compares the derived tree and discloses
|
|
396
490
|
lossy step counts. Metrics sum available span observations: partial coverage and producer
|
|
397
491
|
aggregation can undercount or double count. Coverage gaps are disclosed rather than absorbed
|
|
398
492
|
— a wall duration derived from a subset of steps says so, one whose end precedes its start is
|
|
399
493
|
withheld, and a single unparseable cost withholds the whole total instead of understating it.
|
|
400
494
|
An OTLP or Phoenix import records an unreadable total cost as `unreadable` for that reason, and
|
|
401
495
|
reports token counts, other costs, currencies and evaluation scores it cannot read as
|
|
402
|
-
unavailable, disclosing each on stderr; the producer's value is never kept. File input
|
|
403
|
-
|
|
496
|
+
unavailable, disclosing each on stderr; the producer's value is never kept. File input, stdin
|
|
497
|
+
and Phoenix `px` output share one size limit (`TRACEGRAPH_MAX_ARTIFACT_BYTES`). `query` reads
|
|
498
|
+
artifacts one after another and keeps only the matches it prints (plus the traces `--explain`
|
|
499
|
+
renders). With `--limit`, its peak memory therefore grows with the number of artifacts only
|
|
500
|
+
by a small per-file record (path, trace id); without it, by the matches it prints. It still
|
|
501
|
+
enumerates every match of the trace it is reading, to report the exact total: one trace with
|
|
502
|
+
very many matches is not bounded by `--limit`.
|
|
404
503
|
|
|
405
504
|
Both stores isolate returned mutable objects. InMemory node upsert replaces by ID;
|
|
406
505
|
Ladybug's existing insertion semantics are unchanged. Use `is_isomorphic` or `diff` for
|