auditable 0.2.0__tar.gz → 0.2.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (80) hide show
  1. {auditable-0.2.0 → auditable-0.2.1}/PKG-INFO +18 -15
  2. {auditable-0.2.0 → auditable-0.2.1}/README.md +345 -342
  3. {auditable-0.2.0 → auditable-0.2.1}/examples/README.md +228 -226
  4. {auditable-0.2.0 → auditable-0.2.1}/examples/example_langchain_capture.py +97 -97
  5. {auditable-0.2.0 → auditable-0.2.1}/examples/example_langgraph_capture.py +102 -102
  6. {auditable-0.2.0 → auditable-0.2.1}/examples/example_mcp_capture.py +116 -116
  7. {auditable-0.2.0 → auditable-0.2.1}/examples/example_payment_audit.py +113 -113
  8. {auditable-0.2.0 → auditable-0.2.1}/examples/example_post_rank_run.py +92 -92
  9. {auditable-0.2.0 → auditable-0.2.1}/examples/example_state_anomaly.py +121 -120
  10. {auditable-0.2.0 → auditable-0.2.1}/pyproject.toml +79 -79
  11. {auditable-0.2.0 → auditable-0.2.1}/src/auditable/__init__.py +79 -79
  12. {auditable-0.2.0 → auditable-0.2.1}/src/auditable/analysis.py +615 -615
  13. {auditable-0.2.0 → auditable-0.2.1}/src/auditable/audit_report.py +1056 -1056
  14. {auditable-0.2.0 → auditable-0.2.1}/src/auditable/data/__init__.py +256 -256
  15. {auditable-0.2.0 → auditable-0.2.1}/src/auditable/data/inject.py +225 -225
  16. {auditable-0.2.0 → auditable-0.2.1}/src/auditable/graph/adapters/__init__.py +44 -44
  17. {auditable-0.2.0 → auditable-0.2.1}/src/auditable/graph/adapters/own_record.py +142 -142
  18. {auditable-0.2.0 → auditable-0.2.1}/src/auditable/graph/adapters/protocol.py +64 -64
  19. {auditable-0.2.0 → auditable-0.2.1}/src/auditable/graph/adapters/tau_bench.py +138 -138
  20. {auditable-0.2.0 → auditable-0.2.1}/src/auditable/graph/grounding.py +332 -332
  21. {auditable-0.2.0 → auditable-0.2.1}/src/auditable/graph/loader.py +250 -250
  22. {auditable-0.2.0 → auditable-0.2.1}/src/auditable/graph/risk.py +128 -128
  23. {auditable-0.2.0 → auditable-0.2.1}/src/auditable/graph/session.py +292 -292
  24. {auditable-0.2.0 → auditable-0.2.1}/src/auditable/integrations/__init__.py +10 -10
  25. {auditable-0.2.0 → auditable-0.2.1}/src/auditable/integrations/langchain.py +805 -805
  26. {auditable-0.2.0 → auditable-0.2.1}/src/auditable/integrations/mcp.py +328 -328
  27. {auditable-0.2.0 → auditable-0.2.1}/tests/test_adapters.py +352 -352
  28. {auditable-0.2.0 → auditable-0.2.1}/tests/test_analysis.py +188 -188
  29. {auditable-0.2.0 → auditable-0.2.1}/tests/test_data_inject.py +194 -194
  30. {auditable-0.2.0 → auditable-0.2.1}/tests/test_data_module.py +144 -144
  31. {auditable-0.2.0 → auditable-0.2.1}/tests/test_graph_loader.py +293 -293
  32. {auditable-0.2.0 → auditable-0.2.1}/tests/test_graph_risk.py +134 -134
  33. {auditable-0.2.0 → auditable-0.2.1}/tests/test_grounding.py +249 -249
  34. {auditable-0.2.0 → auditable-0.2.1}/tests/test_langchain_integration.py +292 -292
  35. {auditable-0.2.0 → auditable-0.2.1}/tests/test_live_session.py +372 -372
  36. {auditable-0.2.0 → auditable-0.2.1}/tests/test_mcp_integration.py +264 -264
  37. {auditable-0.2.0 → auditable-0.2.1}/tests/test_public_api.py +99 -99
  38. {auditable-0.2.0 → auditable-0.2.1}/tests/test_session_graph.py +157 -157
  39. {auditable-0.2.0 → auditable-0.2.1}/tests/test_session_graph_projection.py +141 -141
  40. {auditable-0.2.0 → auditable-0.2.1}/.gitignore +0 -0
  41. {auditable-0.2.0 → auditable-0.2.1}/LICENSE +0 -0
  42. {auditable-0.2.0 → auditable-0.2.1}/examples/example_audit_report.py +0 -0
  43. {auditable-0.2.0 → auditable-0.2.1}/examples/example_end_to_end.py +0 -0
  44. {auditable-0.2.0 → auditable-0.2.1}/examples/example_langgraph_live_replay.py +0 -0
  45. {auditable-0.2.0 → auditable-0.2.1}/examples/example_langgraph_llm_agent.py +0 -0
  46. {auditable-0.2.0 → auditable-0.2.1}/examples/example_live_monitor.py +0 -0
  47. {auditable-0.2.0 → auditable-0.2.1}/examples/example_live_replay.py +0 -0
  48. {auditable-0.2.0 → auditable-0.2.1}/examples/example_pre_lint_plan.py +0 -0
  49. {auditable-0.2.0 → auditable-0.2.1}/examples/example_standalone_report.py +0 -0
  50. {auditable-0.2.0 → auditable-0.2.1}/examples/example_touch_capture.py +0 -0
  51. {auditable-0.2.0 → auditable-0.2.1}/src/auditable/_report_figures.py +0 -0
  52. {auditable-0.2.0 → auditable-0.2.1}/src/auditable/chain.py +0 -0
  53. {auditable-0.2.0 → auditable-0.2.1}/src/auditable/compound.py +0 -0
  54. {auditable-0.2.0 → auditable-0.2.1}/src/auditable/graph/__init__.py +0 -0
  55. {auditable-0.2.0 → auditable-0.2.1}/src/auditable/graph/adapters/declared_plan.py +0 -0
  56. {auditable-0.2.0 → auditable-0.2.1}/src/auditable/graph/features.py +0 -0
  57. {auditable-0.2.0 → auditable-0.2.1}/src/auditable/graph/pre.py +0 -0
  58. {auditable-0.2.0 → auditable-0.2.1}/src/auditable/graph/touch.py +0 -0
  59. {auditable-0.2.0 → auditable-0.2.1}/src/auditable/harness/__init__.py +0 -0
  60. {auditable-0.2.0 → auditable-0.2.1}/src/auditable/integrations/langgraph.py +0 -0
  61. {auditable-0.2.0 → auditable-0.2.1}/src/auditable/model/__init__.py +0 -0
  62. {auditable-0.2.0 → auditable-0.2.1}/src/auditable/record.py +0 -0
  63. {auditable-0.2.0 → auditable-0.2.1}/src/auditable/report.py +0 -0
  64. {auditable-0.2.0 → auditable-0.2.1}/tests/test_audit_report.py +0 -0
  65. {auditable-0.2.0 → auditable-0.2.1}/tests/test_auditors.py +0 -0
  66. {auditable-0.2.0 → auditable-0.2.1}/tests/test_basic.py +0 -0
  67. {auditable-0.2.0 → auditable-0.2.1}/tests/test_compound.py +0 -0
  68. {auditable-0.2.0 → auditable-0.2.1}/tests/test_gate.py +0 -0
  69. {auditable-0.2.0 → auditable-0.2.1}/tests/test_graph.py +0 -0
  70. {auditable-0.2.0 → auditable-0.2.1}/tests/test_graph_execution_reach.py +0 -0
  71. {auditable-0.2.0 → auditable-0.2.1}/tests/test_graph_features.py +0 -0
  72. {auditable-0.2.0 → auditable-0.2.1}/tests/test_langgraph_integration.py +0 -0
  73. {auditable-0.2.0 → auditable-0.2.1}/tests/test_langgraph_replay_bridge.py +0 -0
  74. {auditable-0.2.0 → auditable-0.2.1}/tests/test_pre.py +0 -0
  75. {auditable-0.2.0 → auditable-0.2.1}/tests/test_pre_coverage.py +0 -0
  76. {auditable-0.2.0 → auditable-0.2.1}/tests/test_replay.py +0 -0
  77. {auditable-0.2.0 → auditable-0.2.1}/tests/test_report.py +0 -0
  78. {auditable-0.2.0 → auditable-0.2.1}/tests/test_sink.py +0 -0
  79. {auditable-0.2.0 → auditable-0.2.1}/tests/test_touch.py +0 -0
  80. {auditable-0.2.0 → auditable-0.2.1}/tests/test_touch_replay_bridge.py +0 -0
@@ -1,6 +1,6 @@
1
- Metadata-Version: 2.4
1
+ Metadata-Version: 2.5
2
2
  Name: auditable
3
- Version: 0.2.0
3
+ Version: 0.2.1
4
4
  Summary: An open-source system of record for AI-agent decisions: it captures what each decision relied on, replays it against live state, and rolls back the committed action when it no longer holds.
5
5
  Project-URL: Homepage, https://github.com/yzhao062/auditable
6
6
  Project-URL: Repository, https://github.com/yzhao062/auditable
@@ -74,32 +74,33 @@ Description-Content-Type: text/markdown
74
74
  | One graph across plan, run, and review | ✅ | 🟡 | ❌ | ❌ |
75
75
  | Framework-agnostic, dependency-free core | ✅ | 🟡 | 🟡 | 🟡 |
76
76
 
77
- <sub>✅ yes · 🟡 partial · ❌ no, per each tool's public docs (2026). These categories are complementary to `auditable`, not competitors: tracing captures execution, evals score quality, guardrails block at runtime. `auditable` is the audit-and-recovery layer that plugs into them. See [References](#references).</sub>
77
+ <sub>✅ yes · 🟡 partial · ❌ no, per the named tools' public docs (2026). These categories are complementary to `auditable`, not competitors: tracing captures execution, evals score quality, and guardrails block at runtime. `auditable` is the audit-and-recovery layer that plugs into them. See [References](#references).</sub>
78
78
 
79
- **`auditable` is the only column with the full set.** What makes it new:
79
+ **Among the named tools in this documentation-based comparison, `auditable` is the only column with the full set.** What makes it new:
80
80
 
81
81
  - **A unified graph representation for agentic AI.** Every agent run becomes one typed graph that PRE, LIVE, and POST all read. One representation from plan to live operation to review, instead of three disconnected tools.
82
82
  - **Recover, do not just observe.** `auditable` captures the dependency state a decision relied on, replays it under the state that is live now, and reverses the committed action through a compensating rail when it no longer holds. Logging tells you what broke; `auditable` undoes it.
83
83
  - **One decision, three spans, judged together.** Data, model, and harness are bound in a single signed, hash-chained record, so a decision is audited as one unit, not three disconnected logs.
84
84
 
85
- ### Proven on Public Agent Benchmarks
85
+ ### Measured on Public Agent Traces
86
86
 
87
- The graph is not just structure. Across six public agent corpora (GRADE, [arXiv:2606.22741](https://arxiv.org/abs/2606.22741)), the dependency layer predicts which runs fail at **ROC-AUC 0.805** where run length carries no signal, and the execution layer localizes the faulting step at **Top-3 0.614**. See [the benchmark detail](docs/post-analysis.md) for the full numbers and corpora.
87
+ The graph is not just structure. On the CatchBench SWE-Gym board, **`auditable (size+deps)` scores ROC-AUC 0.804 against 0.663 for `size (flat)`**. Registered paired comparisons with ECOD and GUARDIAN remain unresolved. See [the benchmark detail](docs/post-analysis.md) for the localization results, full method names, and limitations.
88
88
 
89
89
  <p align="center">
90
- <img src="https://raw.githubusercontent.com/yzhao062/auditable/main/assets/grade-transfer.png" alt="Leave-one-corpus-out transfer ROC-AUC: the size-normalized dependency signal clears chance on all six held-out agent corpora, while run size drops below chance on tau-bench and SWE-Gym" width="49%">
91
- <img src="https://raw.githubusercontent.com/yzhao062/auditable/main/assets/grade-localization.png" alt="Step-level fault localization on Who and When: ranking steps by execution-graph structure beats an early-fault position prior on top-1, top-3, and MRR" width="49%">
90
+ <img src="https://raw.githubusercontent.com/yzhao062/auditable/main/assets/grade-transfer.png" alt="GRADE leave-one-corpus-out transfer ROC-AUC for size-normalized dependency features and run size across six held-out agent corpora" width="70%">
92
91
  </p>
93
- <p align="center"><sub>Figures from GRADE (<a href="https://arxiv.org/abs/2606.22741">arXiv:2606.22741</a>).</sub></p>
92
+ <p align="center"><sub>Leave-one-corpus-out transfer analysis from GRADE (<a href="https://arxiv.org/abs/2606.22741">arXiv:2606.22741</a>), separate from the CatchBench board result above.</sub></p>
94
93
 
95
- Agents act on dependency state that quietly drifts. A budget read minutes ago can fall below the amount already committed; a price pinned at plan time can move before the action lands. Most tools log what happened, yet they cannot re-decide under the state that is live now, so a stale decision stands until a human notices. `auditable` closes that recovery gap across the lifecycle on one graph: it captures the decision, replays it against live state, and reverses the committed action when it no longer holds.
94
+ Agents act on dependency state that quietly drifts. A budget read minutes ago can fall below the amount already committed; a price pinned at plan time can move before the action lands. The named tracing, eval, and guardrail tools above do not combine dependency capture, live-state replay, and rollback in one layer. `auditable` closes that recovery gap across the lifecycle on one graph: it captures the decision, replays it against live state, and reverses the committed action when it no longer holds.
96
95
 
97
96
  ## The Flagship Moment
98
97
 
99
98
  One payment, walked through the whole lifecycle in 18 `auditable` calls. The agent approves a $2,083.20 vendor payment against a budget snapshot that covered it. Six days later the live budget has dropped below the amount. `replay` re-decides on the live state, and the gate reverses the committed payment. This is recovery, not a log line.
100
99
 
101
100
  ```bash
102
- pip install "auditable[graph]"
101
+ git clone https://github.com/yzhao062/auditable.git
102
+ cd auditable
103
+ python -m pip install ".[graph]"
103
104
  python examples/example_end_to_end.py
104
105
  ```
105
106
 
@@ -125,7 +126,9 @@ print(report.keystone) # the step the rest of the run
125
126
  ```
126
127
 
127
128
  ```bash
128
- pip install "auditable[langgraph]"
129
+ git clone https://github.com/yzhao062/auditable.git
130
+ cd auditable
131
+ python -m pip install ".[langgraph]"
129
132
  python examples/example_langgraph_capture.py # a real LangGraph run -> observed=100%, keystone named
130
133
  ```
131
134
 
@@ -163,7 +166,7 @@ The first six rows capture a real run; see [Plug In Your Agent](#plug-in-your-ag
163
166
  |---|---|---|---|
164
167
  | **PRE** | Before deploy | `analyze_plan` | Read-only structural lints on a declared plan. Names the control-flow chokepoint. Dependency-state risk withheld. |
165
168
  | **LIVE** | While running | `audit` + `replay` + `ActionGate` | Capture a decision, re-decide under live state, route a fix (allow, block, review, rollback) through a rail. The sharpest pillar. |
166
- | **POST** | After a run | `analyze_run` | Rank a finished run by structural blast share. Name the keystone the run rests on, so you review that step first. |
169
+ | **POST** | After a run | `analyze_run` | Rank a finished run by structural blast share and name the keystone for structural triage. |
167
170
 
168
171
  ## Install
169
172
 
@@ -238,7 +241,7 @@ See [`examples/example_pre_lint_plan.py`](examples/example_pre_lint_plan.py) and
238
241
  <details>
239
242
  <summary><b>POST: rank a finished run, find the keystone</b> (`analyze_run` over a recorded trajectory)</summary>
240
243
 
241
- `analyze_run` reads a recorded agent run, builds one decision graph, and ranks every step by how much of the run transitively rests on it, so you review the keystone first. On a tau-bench airline trajectory, the one reservation read that both later writes depend on is the keystone.
244
+ `analyze_run` reads a recorded agent run, builds one decision graph, and ranks every step by how much of the run transitively rests on it. The keystone heads this structural triage order. On a tau-bench airline trajectory, the one reservation read that both later writes depend on is the keystone.
242
245
 
243
246
  ![auditable analyze_run ranks a recorded tau-bench run by structural blast share and names the keystone decision](https://raw.githubusercontent.com/yzhao062/auditable/main/assets/analyze_run.png)
244
247
 
@@ -266,7 +269,7 @@ The graph kernel has two edge layers, and the distinction is load-bearing. Execu
266
269
 
267
270
  One agent decision crosses three spans, and `auditable` binds all three in a single signed, hash-chained record:
268
271
 
269
- | Span | What the Record Binds | Signal in v0.1 |
272
+ | Span | What the Record Binds | Current Signal |
270
273
  |---|---|---|
271
274
  | **Data** | What the agent read and the dependency snapshot it relied on | Snapshot freshness |
272
275
  | **Model** | Which model produced the output, and its stated basis | Decision-basis trust flag |