@tangle-network/traces 0.11.5 → 0.11.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +54 -2
- package/dist/{chunk-VTYOCFGF.js → chunk-MEAEXKUV.js} +1496 -271
- package/dist/cli.js +163 -61
- package/dist/index.d.ts +451 -1
- package/dist/index.js +33 -1
- package/docs/trace-analysts.md +39 -1
- package/examples/external-engines.ts +14 -1
- package/package.json +6 -5
- package/skills/build-trace-analyst/SKILL.md +13 -0
- package/skills/inspect-agent-traces/SKILL.md +30 -4
package/README.md
CHANGED
|
@@ -31,6 +31,7 @@ Emitting the contract is the supported way to integrate a new system. The adapte
|
|
|
31
31
|
- [Watch a run tree](#watch-a-run-tree)
|
|
32
32
|
- [Improvement engine](#improvement-engine)
|
|
33
33
|
- [Session index](#session-index)
|
|
34
|
+
- [Session bundle](#session-bundle) · [Two views](#two-views-two-consumers)
|
|
34
35
|
- [Policy-mining evidence](#policy-mining-evidence)
|
|
35
36
|
- [Upload to the Intelligence Platform](#upload-to-the-intelligence-platform)
|
|
36
37
|
- [Trace analysts](#trace-analysts)
|
|
@@ -243,6 +244,8 @@ traces analyze --otlp spans.otlp.jsonl # analyse foreign OTLP, no ad
|
|
|
243
244
|
traces analyze --otlp results/sessions --out report.md
|
|
244
245
|
traces convert --harness claude-code --last 1 --otlp-out spans.jsonl # OTLP only
|
|
245
246
|
traces index --all --since 24h --out session-index.json
|
|
247
|
+
traces bundle --harness claude-code --session <id|path> --out bundle-dir # one session's durable evidence dir
|
|
248
|
+
traces bundle-view bundle-dir --view evidence-only --out writer-dir # same session, without its own words
|
|
246
249
|
traces inspect session-index.json --out inspection-report.md
|
|
247
250
|
traces evidence --harness codex --last 20 --out policy-evidence.jsonl
|
|
248
251
|
traces evidence --harness codex-exec --session /tmp/codex.jsonl --cwd "$PWD" --out policy-evidence.jsonl
|
|
@@ -450,11 +453,58 @@ The index contains:
|
|
|
450
453
|
- one row per session with harness, session id, path, cwd, repo labels, and time bounds
|
|
451
454
|
- behavior metrics: spans, LLM turns, tool calls, tool errors, tokens, models, and tools
|
|
452
455
|
- signal summaries: stuck loops and tool error rate
|
|
453
|
-
- nearby context files for joins: `AGENTS.md`, `CLAUDE.md`, and `.evolve` JSONL /
|
|
456
|
+
- nearby context files for joins: `AGENTS.md`, `CLAUDE.md`, and `.evolve` artifacts — the JSONL ledgers, `scorecard.json` / `current.json`, reflections, `handoffs/` plus the flat `handoff-*.md` convention, and `progress.md` — with markdown heading/ToC and JSONL key summaries
|
|
454
457
|
|
|
455
458
|
`traces inspect` reads that index back and prints ranked improvement findings over the sessions and nearby context.
|
|
456
459
|
It is intentionally read-only: it points to repeated-call loops, high tool-error sessions, missing repo attribution, long docs without Contents, invalid JSONL rows, and skill-run rows that cannot be joined back to a session.
|
|
457
460
|
|
|
461
|
+
## Session bundle
|
|
462
|
+
|
|
463
|
+
`traces bundle` assembles ONE session's durable evidence directory — the input for any downstream consumer that must cite the session after the live stores rotate.
|
|
464
|
+
It composes commands this CLI already owns and never spends a model call.
|
|
465
|
+
|
|
466
|
+
```bash
|
|
467
|
+
traces bundle --harness claude-code --session <id|path> --out bundle-dir
|
|
468
|
+
```
|
|
469
|
+
|
|
470
|
+
The bundle directory:
|
|
471
|
+
|
|
472
|
+
- `session/` — the transcript byte-for-byte, plus the sibling subagents directory when one exists
|
|
473
|
+
- `derived/` — `session-index.json`, the deterministic `report.md`, `evidence.jsonl`, and the OTLP span artifact
|
|
474
|
+
- `ledger/` — the repo's `.evolve` slices for the session window: `experiments.jsonl` and `skill-runs.jsonl` rows inside the padded span window, `current.json` / `scorecard.json` / `progress.md` copied whole, the latest flat `handoff-*.md`, and reflections dated inside the window
|
|
475
|
+
- `repo/git-log.txt` — commits in the session window from the session's cwd
|
|
476
|
+
- `manifest.json` — `view`, SHA-256 + byte count per file, provenance (session id, harness, cwd, transcript hash, session window), every absent artifact with its reason, per-slice row counts, and the bundle's known limits
|
|
477
|
+
|
|
478
|
+
A missing transcript fails the assembly loudly.
|
|
479
|
+
Every optional input that is absent (no subagents, no `.evolve`, no git repo) is recorded in `manifest.absent` with the probed path — a fact, not an error.
|
|
480
|
+
|
|
481
|
+
### Two views, two consumers
|
|
482
|
+
|
|
483
|
+
A bundle has two kinds of reader, and they need opposite things.
|
|
484
|
+
|
|
485
|
+
- An **auditor** cites the session and needs every source byte. That is the FULL view — what `traces bundle` writes, `manifest.view: "full"`.
|
|
486
|
+
- A **writer** must reach its own conclusion about the session, so it must not be able to read the conclusion that session already reached. That is the EVIDENCE-ONLY view, `manifest.view: "evidence-only"`.
|
|
487
|
+
|
|
488
|
+
The full bundle cannot serve the writer. Its transcript holds every word the session wrote, including the shell call that authored an earlier report, so a check that looks for the report FILE passes while the report CONTENT is right there. Projecting is the fix, not grepping:
|
|
489
|
+
|
|
490
|
+
```bash
|
|
491
|
+
traces bundle --harness claude-code --session <id|path> --out bundle-dir
|
|
492
|
+
traces bundle-view bundle-dir --view evidence-only --out writer-dir
|
|
493
|
+
```
|
|
494
|
+
|
|
495
|
+
The evidence-only view carries an explicit allow-list — `derived/session-index.json`, `derived/evidence.jsonl`, and the structured `ledger/` records (`current.json`, `scorecard.json`, `skill-runs.jsonl`). Everything else is excluded **by name** in `manifest.excluded`, with its SHA-256 and one of four rules:
|
|
496
|
+
|
|
497
|
+
| rule | what it covers |
|
|
498
|
+
|---|---|
|
|
499
|
+
| `session-source` | `session/` — the transcript and every subagent transcript |
|
|
500
|
+
| `session-text-derived` | `derived/report.md` (quotes the first prompt line and evidence excerpts) and `derived/trace.otlp.jsonl` (spans carry the full prompt and response text) |
|
|
501
|
+
| `authored-prose` | `ledger/progress.md`, `ledger/handoff-*.md`, `ledger/reflections/`, `ledger/experiments.jsonl` free-text, `repo/git-log.txt` |
|
|
502
|
+
| `not-allow-listed` | any other path, including one a later version of `traces` adds — the view can only narrow by accident, never widen |
|
|
503
|
+
|
|
504
|
+
The allow-list is the structure. A content check is the proof it is right: before anything is written, every candidate is compared against every excluded file for shared 8-word runs of prose, with identifiers (paths, URLs, hashes) and this package's own constant strings scrubbed from both sides first. A shared run in a repo ledger file drops that file with `rule: "content-signature"` and the match count; a shared run in an artifact `traces` derives is a defect in the derivation and no view is written at all. `manifest.projection.leakCheck` records the width, the sources compared, and the result — always `matches: 0`, because a match means no view exists.
|
|
505
|
+
|
|
506
|
+
A projected view also carries the full bundle's `manifest.json` hash, so an auditor can prove which record it came from. Projecting a view of a view is refused.
|
|
507
|
+
|
|
458
508
|
## Policy-mining evidence
|
|
459
509
|
|
|
460
510
|
`traces` does **not** emit benchmark campaign cells. It emits normalized coding-agent session evidence that another system can mine.
|
|
@@ -548,11 +598,13 @@ traces analyze --last 1
|
|
|
548
598
|
traces analyze --last 1 --llm --budget 0.50
|
|
549
599
|
traces analyze --last 1 --analyzer halo --analyzer-prompt "find token waste"
|
|
550
600
|
traces analyze --all --last 20 --analyzer hodoscope
|
|
601
|
+
traces analyze --last 1 --analyzer prime
|
|
551
602
|
traces analyze --last 1 --analyzer my-installed-command
|
|
552
603
|
```
|
|
553
604
|
|
|
554
605
|
HALO returns a diagnosis report.
|
|
555
606
|
Hodoscope samples distinct behaviors and marks every sample `needs_review`.
|
|
607
|
+
Prime posts the full span projection to an OpenAI-compatible bridge (`TRACES_PRIME_BRIDGE_URL`, default `http://localhost:4181`) and returns validated findings with span evidence.
|
|
556
608
|
An arbitrary command returns a raw report unless its SDK adapter explicitly parses a stricter output type.
|
|
557
609
|
|
|
558
610
|
Read [Trace analysts](./docs/trace-analysts.md) for the output contract, a minimal custom analyst, and labeled benchmark setup.
|
|
@@ -564,7 +616,7 @@ Read [Trace analysts](./docs/trace-analysts.md) for the output contract, a minim
|
|
|
564
616
|
traces upload --since 24h --dry-run --redactor "my-pii-scrubber"
|
|
565
617
|
```
|
|
566
618
|
|
|
567
|
-
In the SDK these are the `ExternalAnalyzer` and `Redactor` interfaces (`haloAnalyzer`, `hodoscopeAnalyzer`, `commandAnalyzer`, `commandRedactor`, `applyRedactor`, `runExternalAnalyzers`).
|
|
619
|
+
In the SDK these are the `ExternalAnalyzer` and `Redactor` interfaces (`haloAnalyzer`, `hodoscopeAnalyzer`, `primeAnalyzer`, `commandAnalyzer`, `commandRedactor`, `applyRedactor`, `runExternalAnalyzers`).
|
|
568
620
|
See [`examples/external-engines.ts`](./examples/external-engines.ts).
|
|
569
621
|
|
|
570
622
|
> The built-in agentic analysts (`--llm`) run on the Tangle router by default: set `TANGLE_API_KEY`.
|