auditable 0.2.0__tar.gz → 0.2.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {auditable-0.2.0 → auditable-0.2.1}/PKG-INFO +18 -15
- {auditable-0.2.0 → auditable-0.2.1}/README.md +345 -342
- {auditable-0.2.0 → auditable-0.2.1}/examples/README.md +228 -226
- {auditable-0.2.0 → auditable-0.2.1}/examples/example_langchain_capture.py +97 -97
- {auditable-0.2.0 → auditable-0.2.1}/examples/example_langgraph_capture.py +102 -102
- {auditable-0.2.0 → auditable-0.2.1}/examples/example_mcp_capture.py +116 -116
- {auditable-0.2.0 → auditable-0.2.1}/examples/example_payment_audit.py +113 -113
- {auditable-0.2.0 → auditable-0.2.1}/examples/example_post_rank_run.py +92 -92
- {auditable-0.2.0 → auditable-0.2.1}/examples/example_state_anomaly.py +121 -120
- {auditable-0.2.0 → auditable-0.2.1}/pyproject.toml +79 -79
- {auditable-0.2.0 → auditable-0.2.1}/src/auditable/__init__.py +79 -79
- {auditable-0.2.0 → auditable-0.2.1}/src/auditable/analysis.py +615 -615
- {auditable-0.2.0 → auditable-0.2.1}/src/auditable/audit_report.py +1056 -1056
- {auditable-0.2.0 → auditable-0.2.1}/src/auditable/data/__init__.py +256 -256
- {auditable-0.2.0 → auditable-0.2.1}/src/auditable/data/inject.py +225 -225
- {auditable-0.2.0 → auditable-0.2.1}/src/auditable/graph/adapters/__init__.py +44 -44
- {auditable-0.2.0 → auditable-0.2.1}/src/auditable/graph/adapters/own_record.py +142 -142
- {auditable-0.2.0 → auditable-0.2.1}/src/auditable/graph/adapters/protocol.py +64 -64
- {auditable-0.2.0 → auditable-0.2.1}/src/auditable/graph/adapters/tau_bench.py +138 -138
- {auditable-0.2.0 → auditable-0.2.1}/src/auditable/graph/grounding.py +332 -332
- {auditable-0.2.0 → auditable-0.2.1}/src/auditable/graph/loader.py +250 -250
- {auditable-0.2.0 → auditable-0.2.1}/src/auditable/graph/risk.py +128 -128
- {auditable-0.2.0 → auditable-0.2.1}/src/auditable/graph/session.py +292 -292
- {auditable-0.2.0 → auditable-0.2.1}/src/auditable/integrations/__init__.py +10 -10
- {auditable-0.2.0 → auditable-0.2.1}/src/auditable/integrations/langchain.py +805 -805
- {auditable-0.2.0 → auditable-0.2.1}/src/auditable/integrations/mcp.py +328 -328
- {auditable-0.2.0 → auditable-0.2.1}/tests/test_adapters.py +352 -352
- {auditable-0.2.0 → auditable-0.2.1}/tests/test_analysis.py +188 -188
- {auditable-0.2.0 → auditable-0.2.1}/tests/test_data_inject.py +194 -194
- {auditable-0.2.0 → auditable-0.2.1}/tests/test_data_module.py +144 -144
- {auditable-0.2.0 → auditable-0.2.1}/tests/test_graph_loader.py +293 -293
- {auditable-0.2.0 → auditable-0.2.1}/tests/test_graph_risk.py +134 -134
- {auditable-0.2.0 → auditable-0.2.1}/tests/test_grounding.py +249 -249
- {auditable-0.2.0 → auditable-0.2.1}/tests/test_langchain_integration.py +292 -292
- {auditable-0.2.0 → auditable-0.2.1}/tests/test_live_session.py +372 -372
- {auditable-0.2.0 → auditable-0.2.1}/tests/test_mcp_integration.py +264 -264
- {auditable-0.2.0 → auditable-0.2.1}/tests/test_public_api.py +99 -99
- {auditable-0.2.0 → auditable-0.2.1}/tests/test_session_graph.py +157 -157
- {auditable-0.2.0 → auditable-0.2.1}/tests/test_session_graph_projection.py +141 -141
- {auditable-0.2.0 → auditable-0.2.1}/.gitignore +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/LICENSE +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/examples/example_audit_report.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/examples/example_end_to_end.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/examples/example_langgraph_live_replay.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/examples/example_langgraph_llm_agent.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/examples/example_live_monitor.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/examples/example_live_replay.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/examples/example_pre_lint_plan.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/examples/example_standalone_report.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/examples/example_touch_capture.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/src/auditable/_report_figures.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/src/auditable/chain.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/src/auditable/compound.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/src/auditable/graph/__init__.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/src/auditable/graph/adapters/declared_plan.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/src/auditable/graph/features.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/src/auditable/graph/pre.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/src/auditable/graph/touch.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/src/auditable/harness/__init__.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/src/auditable/integrations/langgraph.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/src/auditable/model/__init__.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/src/auditable/record.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/src/auditable/report.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/tests/test_audit_report.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/tests/test_auditors.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/tests/test_basic.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/tests/test_compound.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/tests/test_gate.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/tests/test_graph.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/tests/test_graph_execution_reach.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/tests/test_graph_features.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/tests/test_langgraph_integration.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/tests/test_langgraph_replay_bridge.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/tests/test_pre.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/tests/test_pre_coverage.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/tests/test_replay.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/tests/test_report.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/tests/test_sink.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/tests/test_touch.py +0 -0
- {auditable-0.2.0 → auditable-0.2.1}/tests/test_touch_replay_bridge.py +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
|
-
Metadata-Version: 2.
|
|
1
|
+
Metadata-Version: 2.5
|
|
2
2
|
Name: auditable
|
|
3
|
-
Version: 0.2.
|
|
3
|
+
Version: 0.2.1
|
|
4
4
|
Summary: An open-source system of record for AI-agent decisions: it captures what each decision relied on, replays it against live state, and rolls back the committed action when it no longer holds.
|
|
5
5
|
Project-URL: Homepage, https://github.com/yzhao062/auditable
|
|
6
6
|
Project-URL: Repository, https://github.com/yzhao062/auditable
|
|
@@ -74,32 +74,33 @@ Description-Content-Type: text/markdown
|
|
|
74
74
|
| One graph across plan, run, and review | ✅ | 🟡 | ❌ | ❌ |
|
|
75
75
|
| Framework-agnostic, dependency-free core | ✅ | 🟡 | 🟡 | 🟡 |
|
|
76
76
|
|
|
77
|
-
<sub>✅ yes · 🟡 partial · ❌ no, per
|
|
77
|
+
<sub>✅ yes · 🟡 partial · ❌ no, per the named tools' public docs (2026). These categories are complementary to `auditable`, not competitors: tracing captures execution, evals score quality, and guardrails block at runtime. `auditable` is the audit-and-recovery layer that plugs into them. See [References](#references).</sub>
|
|
78
78
|
|
|
79
|
-
|
|
79
|
+
**Among the named tools in this documentation-based comparison, `auditable` is the only column with the full set.** What makes it new:
|
|
80
80
|
|
|
81
81
|
- **A unified graph representation for agentic AI.** Every agent run becomes one typed graph that PRE, LIVE, and POST all read. One representation from plan to live operation to review, instead of three disconnected tools.
|
|
82
82
|
- **Recover, do not just observe.** `auditable` captures the dependency state a decision relied on, replays it under the state that is live now, and reverses the committed action through a compensating rail when it no longer holds. Logging tells you what broke; `auditable` undoes it.
|
|
83
83
|
- **One decision, three spans, judged together.** Data, model, and harness are bound in a single signed, hash-chained record, so a decision is audited as one unit, not three disconnected logs.
|
|
84
84
|
|
|
85
|
-
###
|
|
85
|
+
### Measured on Public Agent Traces
|
|
86
86
|
|
|
87
|
-
The graph is not just structure.
|
|
87
|
+
The graph is not just structure. On the CatchBench SWE-Gym board, **`auditable (size+deps)` scores ROC-AUC 0.804 against 0.663 for `size (flat)`**. Registered paired comparisons with ECOD and GUARDIAN remain unresolved. See [the benchmark detail](docs/post-analysis.md) for the localization results, full method names, and limitations.
|
|
88
88
|
|
|
89
89
|
<p align="center">
|
|
90
|
-
<img src="https://raw.githubusercontent.com/yzhao062/auditable/main/assets/grade-transfer.png" alt="
|
|
91
|
-
<img src="https://raw.githubusercontent.com/yzhao062/auditable/main/assets/grade-localization.png" alt="Step-level fault localization on Who and When: ranking steps by execution-graph structure beats an early-fault position prior on top-1, top-3, and MRR" width="49%">
|
|
90
|
+
<img src="https://raw.githubusercontent.com/yzhao062/auditable/main/assets/grade-transfer.png" alt="GRADE leave-one-corpus-out transfer ROC-AUC for size-normalized dependency features and run size across six held-out agent corpora" width="70%">
|
|
92
91
|
</p>
|
|
93
|
-
<p align="center"><sub>
|
|
92
|
+
<p align="center"><sub>Leave-one-corpus-out transfer analysis from GRADE (<a href="https://arxiv.org/abs/2606.22741">arXiv:2606.22741</a>), separate from the CatchBench board result above.</sub></p>
|
|
94
93
|
|
|
95
|
-
Agents act on dependency state that quietly drifts. A budget read minutes ago can fall below the amount already committed; a price pinned at plan time can move before the action lands.
|
|
94
|
+
Agents act on dependency state that quietly drifts. A budget read minutes ago can fall below the amount already committed; a price pinned at plan time can move before the action lands. The named tracing, eval, and guardrail tools above do not combine dependency capture, live-state replay, and rollback in one layer. `auditable` closes that recovery gap across the lifecycle on one graph: it captures the decision, replays it against live state, and reverses the committed action when it no longer holds.
|
|
96
95
|
|
|
97
96
|
## The Flagship Moment
|
|
98
97
|
|
|
99
98
|
One payment, walked through the whole lifecycle in 18 `auditable` calls. The agent approves a $2,083.20 vendor payment against a budget snapshot that covered it. Six days later the live budget has dropped below the amount. `replay` re-decides on the live state, and the gate reverses the committed payment. This is recovery, not a log line.
|
|
100
99
|
|
|
101
100
|
```bash
|
|
102
|
-
|
|
101
|
+
git clone https://github.com/yzhao062/auditable.git
|
|
102
|
+
cd auditable
|
|
103
|
+
python -m pip install ".[graph]"
|
|
103
104
|
python examples/example_end_to_end.py
|
|
104
105
|
```
|
|
105
106
|
|
|
@@ -125,7 +126,9 @@ print(report.keystone) # the step the rest of the run
|
|
|
125
126
|
```
|
|
126
127
|
|
|
127
128
|
```bash
|
|
128
|
-
|
|
129
|
+
git clone https://github.com/yzhao062/auditable.git
|
|
130
|
+
cd auditable
|
|
131
|
+
python -m pip install ".[langgraph]"
|
|
129
132
|
python examples/example_langgraph_capture.py # a real LangGraph run -> observed=100%, keystone named
|
|
130
133
|
```
|
|
131
134
|
|
|
@@ -163,7 +166,7 @@ The first six rows capture a real run; see [Plug In Your Agent](#plug-in-your-ag
|
|
|
163
166
|
|---|---|---|---|
|
|
164
167
|
| **PRE** | Before deploy | `analyze_plan` | Read-only structural lints on a declared plan. Names the control-flow chokepoint. Dependency-state risk withheld. |
|
|
165
168
|
| **LIVE** | While running | `audit` + `replay` + `ActionGate` | Capture a decision, re-decide under live state, route a fix (allow, block, review, rollback) through a rail. The sharpest pillar. |
|
|
166
|
-
| **POST** | After a run | `analyze_run` | Rank a finished run by structural blast share
|
|
169
|
+
| **POST** | After a run | `analyze_run` | Rank a finished run by structural blast share and name the keystone for structural triage. |
|
|
167
170
|
|
|
168
171
|
## Install
|
|
169
172
|
|
|
@@ -238,7 +241,7 @@ See [`examples/example_pre_lint_plan.py`](examples/example_pre_lint_plan.py) and
|
|
|
238
241
|
<details>
|
|
239
242
|
<summary><b>POST: rank a finished run, find the keystone</b> (`analyze_run` over a recorded trajectory)</summary>
|
|
240
243
|
|
|
241
|
-
`analyze_run` reads a recorded agent run, builds one decision graph, and ranks every step by how much of the run transitively rests on it
|
|
244
|
+
`analyze_run` reads a recorded agent run, builds one decision graph, and ranks every step by how much of the run transitively rests on it. The keystone heads this structural triage order. On a tau-bench airline trajectory, the one reservation read that both later writes depend on is the keystone.
|
|
242
245
|
|
|
243
246
|

|
|
244
247
|
|
|
@@ -266,7 +269,7 @@ The graph kernel has two edge layers, and the distinction is load-bearing. Execu
|
|
|
266
269
|
|
|
267
270
|
One agent decision crosses three spans, and `auditable` binds all three in a single signed, hash-chained record:
|
|
268
271
|
|
|
269
|
-
| Span | What the Record Binds | Signal
|
|
272
|
+
| Span | What the Record Binds | Current Signal |
|
|
270
273
|
|---|---|---|
|
|
271
274
|
| **Data** | What the agent read and the dependency snapshot it relied on | Snapshot freshness |
|
|
272
275
|
| **Model** | Which model produced the output, and its stated basis | Decision-basis trust flag |
|