agentcrash-core 0.2.0__tar.gz → 0.3.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (114) hide show
  1. agentcrash_core-0.3.0/.github/workflows/release.yml +96 -0
  2. agentcrash_core-0.3.0/AGENTS.md +5 -0
  3. agentcrash_core-0.3.0/CHANGELOG.md +83 -0
  4. agentcrash_core-0.3.0/LICENSE +21 -0
  5. agentcrash_core-0.3.0/PKG-INFO +173 -0
  6. agentcrash_core-0.3.0/README.md +140 -0
  7. agentcrash_core-0.3.0/bench/run_overhead.py +164 -0
  8. agentcrash_core-0.3.0/docs/adlc/01-planning-and-feasibility.md +92 -0
  9. agentcrash_core-0.3.0/docs/adlc/02-prd-and-project-brief.md +103 -0
  10. agentcrash_core-0.3.0/docs/adlc/03-requirements-and-srs.md +238 -0
  11. agentcrash_core-0.3.0/docs/adlc/04-glossary-and-domain-model.md +175 -0
  12. agentcrash_core-0.3.0/docs/adlc/05-system-design.md +235 -0
  13. agentcrash_core-0.3.0/docs/adlc/06-data-design.md +257 -0
  14. agentcrash_core-0.3.0/docs/adlc/07-api-contracts.md +380 -0
  15. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/docs/adlc/08-adr-template.md +20 -64
  16. agentcrash_core-0.3.0/docs/adlc/09-agent-context.md +237 -0
  17. agentcrash_core-0.3.0/docs/adlc/10-testing-strategy.md +174 -0
  18. agentcrash_core-0.3.0/docs/adlc/11-security-and-compliance.md +177 -0
  19. agentcrash_core-0.3.0/docs/adlc/12-non-functional-requirements.md +158 -0
  20. agentcrash_core-0.3.0/docs/adlc/13-contributing-and-workflow.md +215 -0
  21. agentcrash_core-0.3.0/docs/adlc/14-runbook.md +214 -0
  22. agentcrash_core-0.3.0/docs/adlc/15-roadmap-and-changelog.md +127 -0
  23. agentcrash_core-0.3.0/docs/adlc/README.md +137 -0
  24. agentcrash_core-0.3.0/docs/architecture-decoupling.md +76 -0
  25. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/docs/overview.md +1 -1
  26. agentcrash_core-0.3.0/docs/phase7-evidence/bench.json +18 -0
  27. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/docs/phase7-evidence/cost-sheet.md +1 -1
  28. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/docs/plans/2026-10-29-agentcrash-implementation.md +14 -82
  29. agentcrash_core-0.3.0/docs/plans/2026-11-14-sota-roadmap.md +113 -0
  30. agentcrash_core-0.3.0/docs/setup.md +137 -0
  31. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/examples/README.md +9 -5
  32. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/examples/minimal_agent/app.py +1 -1
  33. agentcrash_core-0.3.0/pyproject.toml +69 -0
  34. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/src/agentcrash/__init__.py +8 -6
  35. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/src/agentcrash/_registry.py +1 -1
  36. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/src/agentcrash/cli.py +57 -4
  37. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/src/agentcrash/injector.py +12 -21
  38. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/src/agentcrash/invariants.py +1 -1
  39. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/src/agentcrash/lab/index.html +1 -1
  40. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/src/agentcrash/ledger.py +2 -2
  41. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/src/agentcrash/policy.py +18 -104
  42. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/src/agentcrash/sdk.py +143 -54
  43. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/src/agentcrash/server.py +34 -19
  44. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/src/agentcrash/tracer.py +42 -20
  45. agentcrash_core-0.3.0/src/agentcrash/verifiers.py +289 -0
  46. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/tests/test_cli_help.py +1 -0
  47. agentcrash_core-0.3.0/tests/test_gaps.py +365 -0
  48. agentcrash_core-0.3.0/tests/test_gaps_2.py +546 -0
  49. agentcrash_core-0.3.0/tests/test_generic_core.py +161 -0
  50. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/tests/test_lab.py +5 -5
  51. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/tests/test_ledger.py +15 -15
  52. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/tests/test_sdk.py +13 -3
  53. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/tests/test_second_app.py +7 -11
  54. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/tests/test_server_ingestion.py +33 -5
  55. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/tests/test_tracer.py +7 -7
  56. agentcrash_core-0.3.0/uv.lock +803 -0
  57. agentcrash_core-0.2.0/AGENTS.md +0 -3
  58. agentcrash_core-0.2.0/PKG-INFO +0 -106
  59. agentcrash_core-0.2.0/README.md +0 -81
  60. agentcrash_core-0.2.0/bench/run_overhead.py +0 -125
  61. agentcrash_core-0.2.0/docs/adlc/01-planning-and-feasibility.md +0 -181
  62. agentcrash_core-0.2.0/docs/adlc/02-prd-and-project-brief.md +0 -154
  63. agentcrash_core-0.2.0/docs/adlc/03-requirements-and-srs.md +0 -182
  64. agentcrash_core-0.2.0/docs/adlc/04-glossary-and-domain-model.md +0 -159
  65. agentcrash_core-0.2.0/docs/adlc/05-system-design.md +0 -194
  66. agentcrash_core-0.2.0/docs/adlc/06-data-design.md +0 -216
  67. agentcrash_core-0.2.0/docs/adlc/07-api-contracts.md +0 -229
  68. agentcrash_core-0.2.0/docs/adlc/09-agent-context.md +0 -249
  69. agentcrash_core-0.2.0/docs/adlc/10-testing-strategy.md +0 -200
  70. agentcrash_core-0.2.0/docs/adlc/11-security-and-compliance.md +0 -160
  71. agentcrash_core-0.2.0/docs/adlc/12-non-functional-requirements.md +0 -155
  72. agentcrash_core-0.2.0/docs/adlc/13-contributing-and-workflow.md +0 -167
  73. agentcrash_core-0.2.0/docs/adlc/14-runbook.md +0 -228
  74. agentcrash_core-0.2.0/docs/adlc/15-roadmap-and-changelog.md +0 -159
  75. agentcrash_core-0.2.0/docs/adlc/README.md +0 -159
  76. agentcrash_core-0.2.0/docs/architecture-decoupling.md +0 -109
  77. agentcrash_core-0.2.0/docs/nl-to-odrl-sequence.html +0 -14998
  78. agentcrash_core-0.2.0/docs/phase7-evidence/bench.json +0 -12
  79. agentcrash_core-0.2.0/docs/phase7-evidence/demo-bundle/README.md +0 -34
  80. agentcrash_core-0.2.0/docs/phase7-evidence/demo-bundle/fixed-policy.yaml +0 -15
  81. agentcrash_core-0.2.0/docs/phase7-evidence/demo-bundle/post-fix-state.json +0 -1
  82. agentcrash_core-0.2.0/docs/phase7-evidence/demo-bundle/post-fix-trace.jsonl +0 -11
  83. agentcrash_core-0.2.0/docs/phase7-evidence/demo-bundle/post-fix-verdict.json +0 -23
  84. agentcrash_core-0.2.0/docs/phase7-evidence/demo-bundle/pre-fix-state.json +0 -1
  85. agentcrash_core-0.2.0/docs/phase7-evidence/demo-bundle/pre-fix-trace.jsonl +0 -13
  86. agentcrash_core-0.2.0/docs/phase7-evidence/demo-bundle/pre-fix-verdict.json +0 -23
  87. agentcrash_core-0.2.0/docs/phase7-evidence/rehearsals.md +0 -64
  88. agentcrash_core-0.2.0/docs/plans/2026-11-01-next-steps.md +0 -74
  89. agentcrash_core-0.2.0/docs/setup.md +0 -151
  90. agentcrash_core-0.2.0/pyproject.toml +0 -47
  91. agentcrash_core-0.2.0/src/agentcrash/db.py +0 -411
  92. agentcrash_core-0.2.0/src/agentcrash/verifiers.py +0 -216
  93. agentcrash_core-0.2.0/tests/fixtures/odrl_nl.yaml +0 -85
  94. agentcrash_core-0.2.0/tests/fixtures/traces/odrl_convert_timeout-fixed-seed42.jsonl +0 -15
  95. agentcrash_core-0.2.0/tests/fixtures/traces/odrl_convert_timeout-fixed-seed42.state.json +0 -1
  96. agentcrash_core-0.2.0/tests/fixtures/traces/odrl_convert_timeout-naive-seed42.jsonl +0 -13
  97. agentcrash_core-0.2.0/tests/fixtures/traces/odrl_convert_timeout-naive-seed42.state.json +0 -1
  98. agentcrash_core-0.2.0/uv.lock +0 -2908
  99. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/.github/workflows/ci.yml +0 -0
  100. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/.gitignore +0 -0
  101. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/docs/phase7-evidence/regression-report.json +0 -0
  102. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/src/agentcrash/__main__.py +0 -0
  103. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/src/agentcrash/clock.py +0 -0
  104. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/src/agentcrash/evaluate.py +0 -0
  105. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/src/agentcrash/events.py +0 -0
  106. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/src/agentcrash/lab/app.js +0 -0
  107. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/src/agentcrash/lab/styles.css +0 -0
  108. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/src/agentcrash/scan_pii.py +0 -0
  109. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/src/agentcrash/schemas/policy-patch.json +0 -0
  110. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/src/agentcrash/schemas/policy.json +0 -0
  111. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/tests/test_acr001.py +0 -0
  112. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/tests/test_clock.py +0 -0
  113. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/tests/test_no_static_bundle.py +0 -0
  114. {agentcrash_core-0.2.0 → agentcrash_core-0.3.0}/tests/test_skeleton.py +0 -0
@@ -0,0 +1,96 @@
1
+ name: release
2
+
3
+ # Publishes agentcrash-core to PyPI when a version tag is pushed:
4
+ # git tag v0.3.0 && git push origin v0.3.0
5
+ # Uses PyPI trusted publishing (OIDC): no API token is stored in GitHub.
6
+ # One-time setup (PyPI trusted publisher + GitHub environment "pypi") is in
7
+ # docs/adlc/13-contributing-and-workflow.md, section 8.
8
+
9
+ on:
10
+ push:
11
+ tags: ["v*"]
12
+
13
+ permissions:
14
+ contents: read
15
+
16
+ concurrency:
17
+ group: release-${{ github.ref }}
18
+ cancel-in-progress: false
19
+
20
+ jobs:
21
+ gates:
22
+ name: Gates (py${{ matrix.python-version }})
23
+ runs-on: ubuntu-latest
24
+ strategy:
25
+ fail-fast: true
26
+ matrix:
27
+ python-version: ["3.11", "3.12", "3.13"]
28
+ steps:
29
+ - uses: actions/checkout@v4
30
+ - uses: actions/setup-python@v5
31
+ with:
32
+ python-version: ${{ matrix.python-version }}
33
+ - uses: astral-sh/setup-uv@v5
34
+ - name: Install (with dev extras)
35
+ run: uv sync --extra dev
36
+ - name: Lint
37
+ run: uv run --no-sync ruff check .
38
+ - name: Format
39
+ run: uv run --no-sync black --check .
40
+ - name: Tests
41
+ run: uv run --no-sync pytest -q
42
+
43
+ build:
44
+ name: Build sdist + wheel
45
+ needs: gates
46
+ runs-on: ubuntu-latest
47
+ steps:
48
+ - uses: actions/checkout@v4
49
+ - uses: actions/setup-python@v5
50
+ with:
51
+ python-version: "3.12"
52
+ - uses: astral-sh/setup-uv@v5
53
+ - name: Tag must match the package version
54
+ run: |
55
+ tag="${GITHUB_REF_NAME#v}"
56
+ pyproject=$(python -c "import tomllib; print(tomllib.load(open('pyproject.toml','rb'))['project']['version'])")
57
+ module=$(python -c "import re; print(re.search(r'__version__ = \"([^\"]+)\"', open('src/agentcrash/__init__.py').read()).group(1))")
58
+ echo "tag=$tag pyproject=$pyproject module=$module"
59
+ if [ "$tag" != "$pyproject" ] || [ "$tag" != "$module" ]; then
60
+ echo "::error::tag v$tag does not match pyproject ($pyproject) and __version__ ($module)"
61
+ exit 1
62
+ fi
63
+ - name: Build
64
+ run: |
65
+ uv build --out-dir dist
66
+ rm -f dist/.gitignore # uv writes one; it is not a distribution
67
+ ls -l dist
68
+ - name: Check metadata
69
+ run: uvx twine check --strict dist/*.whl dist/*.tar.gz
70
+ - name: Smoke-test the wheel in a clean venv
71
+ run: |
72
+ uv venv --python 3.12 /tmp/smoke
73
+ uv pip install --python /tmp/smoke/bin/python dist/*.whl
74
+ /tmp/smoke/bin/python -m agentcrash --version
75
+ /tmp/smoke/bin/python -c "import agentcrash, sys; assert agentcrash.__version__ == '${GITHUB_REF_NAME#v}'; print('import ok')"
76
+ - uses: actions/upload-artifact@v4
77
+ with:
78
+ name: dist
79
+ path: dist/
80
+ if-no-files-found: error
81
+
82
+ publish:
83
+ name: Publish to PyPI
84
+ needs: build
85
+ runs-on: ubuntu-latest
86
+ environment:
87
+ name: pypi
88
+ url: https://pypi.org/project/agentcrash-core/
89
+ permissions:
90
+ id-token: write # OIDC token for PyPI trusted publishing; nothing else
91
+ steps:
92
+ - uses: actions/download-artifact@v4
93
+ with:
94
+ name: dist
95
+ path: dist/
96
+ - uses: pypa/gh-action-pypi-publish@release/v1
@@ -0,0 +1,5 @@
1
+ # Agent instructions pointer
2
+
3
+ Canonical docs live in `docs/`. New agent sessions: start with `docs/overview.md`, then `docs/architecture-decoupling.md` (core vs host boundary), then the roadmap in `docs/plans/2026-11-14-sota-roadmap.md` (draft status — read it for direction, but do not implement until the owner replies `go`). `docs/adlc/` and `docs/plans/2026-10-29-agentcrash-implementation.md` are the historical lifecycle record.
4
+
5
+ The core package must stay domain-free: `tests/test_generic_core.py` enforces it.
@@ -0,0 +1,83 @@
1
+ # Changelog
2
+
3
+ All notable changes to `agentcrash-core`. Versions follow [SemVer](https://semver.org/);
4
+ while the version is `0.x`, minor releases may break the API.
5
+
6
+ ## 0.3.0 — unreleased
7
+
8
+ Chaos engineering for any agentic workflow: the core no longer ships domain code.
9
+
10
+ ### Breaking changes and how to migrate
11
+
12
+ | Removed | Why | Do this instead |
13
+ |---|---|---|
14
+ | `agentcrash.db` | Its tables were shaped for one demo app (`nl_text`, `odrl_docs`) and nothing in core used it | Use the control plane (`agentcrash serve`, `/v1/runs`, `/v1/events`) or persist `ac.trace(run_id)` yourself |
15
+ | `agentcrash.policy.reference_patch_for` and `reference_patch_s4_*` | Hard-coded fixes for one demo's scenarios | Build a `policy_patch/v1` dict for your workflow and apply it with `policy.apply_patch`; or opt in to model-proposed patches via `fetch_live_patch` |
16
+ | `agentcrash.verifiers.SAVE_ODRL` | Demo tool name | `FreshWriteOnly(tools=["your_store_tool"])` |
17
+ | `host-odrl` extra (`pyshacl`, `dspy`, `numpy`) | Domain stacks do not belong in core | Depend on them directly in your own package |
18
+ | Fault `default_params` naming a tool | Silently targeted a demo tool | Name the tool yourself: `{"tool": "payments.charge", ...}`; `concurrent_update` now requires `key` and `updates` |
19
+ | Exporter de-duplication by `event_id` | Unbounded memory in long-running hosts | Nothing to do: the server de-duplicates on ingestion |
20
+
21
+ `LLAMA_BASE_URL` / `LLAMA_API_KEY` keep working; the preferred names are
22
+ `AGENTCRASH_LLM_BASE_URL` / `AGENTCRASH_LLM_API_KEY`.
23
+
24
+ ### Added
25
+
26
+ - `.github/workflows/release.yml`: on a `v*` tag, run the gates (py3.11-3.13), check the
27
+ tag matches the package version, build sdist + wheel, `twine check --strict`,
28
+ smoke-install the wheel, and publish to PyPI via trusted publishing (OIDC, no token).
29
+ - `verify_sdk_chain(events)` and `agentcrash check --verify-chain` (SDK runs and
30
+ `TraceWriter` files; a broken chain exits 3).
31
+ - `check --invariant fresh_write_only:<tool>[,<tool>]` names your store tools on the CLI.
32
+ - `agentcrash.verifiers.tool_results(trace)`: one normalized view of tool calls from
33
+ either trace format, for writing your own invariants.
34
+ - `tests/test_gaps_2.py`: workflow error path, `enabled=False`, event validation and
35
+ invalid ingestion, `FaultSchedule` refusal and matching, `fault_injection=False`,
36
+ `run_verifiers` error paths, runtime verifier purity, FastAPI bearer auth, Lab
37
+ traversal, `load_policy`, `diff_policies`, live-patch transport and output gating.
38
+ Every FR in doc 03 now cites a test. Dev extra adds `httpx2` (FastAPI test client).
39
+ - `tests/test_gaps.py`: `--scan-pii`, card redaction, `NoLostUpdate`, `check` exit codes
40
+ 2 and 3, patch rejection, report byte-stability.
41
+ - `agentcrash check` registers the built-in invariants (`duplicate_mutating_effect`,
42
+ `fresh_write_only`, `no_lost_update`, `required_step:<name>`), so it works without plugins.
43
+ - `FreshWriteOnly(tools=...)` for your own versioned-store tools.
44
+ - MIT `LICENSE`; PyPI classifiers, keywords and project URLs.
45
+ - `tests/test_generic_core.py`: fails if domain vocabulary returns to `src/`.
46
+
47
+ ### Fixed
48
+
49
+ - `fresh_write_only` and `no_lost_update` only understood `TraceWriter` envelopes, so on
50
+ SDK traces they passed with nothing to check. Both now read SDK events too
51
+ (`kind: tool` / `kind: fault`); `fresh_write_only` says `nothing to judge (vacuous)`
52
+ when no write matched, and `no_lost_update` raises (exit 3) instead of passing when a
53
+ rival writer exists but `--state` has no `record_versions`.
54
+ - SDK `seq` and the hash chain were per `AgentCrash` instance, so one run's trace could not
55
+ be verified alone. Both are now per run (`seq` from 1, chain from `GENESIS`).
56
+ Behavior change: `seq` restarts at 1 in every run.
57
+ - `maybe_fault` always checked call 1, so `occurrence: 2+` never fired. It now counts
58
+ calls per boundary per run (`occurrence: N` fires on exactly the Nth call); pass
59
+ `call_index` to override. Behavior change: an `occurrence: 1` fault no longer fires
60
+ on every call, only the first.
61
+ - Stdlib server lacked `GET /v1/runs/{id}/evaluation`; FastAPI app lacked
62
+ `GET /v1/reliability/{project}` and returned a different `GET /v1/runs` row shape.
63
+ Both servers now expose the same routes and rows.
64
+
65
+ - Tracer redacted its own `prev_hash`/`event_id` fields; a hash with 13+ digits in a row
66
+ could be rewritten as a "card number" and break the chain.
67
+ - SDK hash chain could fork under concurrent steps (`prev_hash` updated outside the lock).
68
+ - `record_tool` raised `TypeError` into host code for non-JSON outputs (objects, dates).
69
+ - Server reset the connection on 401 before reading the body (Windows; flaky auth test).
70
+ - Bench crashed on Windows (unclosed trace files).
71
+
72
+ ### Performance
73
+
74
+ - Tracer: ~75 → ~31 µs per event (compiled key matcher, cached key checks, cheap
75
+ pre-checks before the PII regexes, one serialization per event).
76
+ - SDK: ~14 µs per event (`deque` exporter, prebuilt encoder, no redundant
77
+ per-event validation).
78
+ - `bench/run_overhead.py` now measures NFR-PERF-01 as written (per-run overhead vs a bare
79
+ 12-step agent loop): 5.7% at a 2 ms stub step, target ≤ 10%.
80
+
81
+ ## 0.2.0
82
+
83
+ First installable split of the SDK, control plane and `check` CLI.
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Sahil Satyam
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,173 @@
1
+ Metadata-Version: 2.5
2
+ Name: agentcrash-core
3
+ Version: 0.3.0
4
+ Summary: Chaos engineering for tool-using AI agents: deterministic fault injection, hash-chained traces, invariant verdicts, and policy replay.
5
+ Project-URL: Homepage, https://github.com/SahilSatyam/AgentCrash
6
+ Project-URL: Issues, https://github.com/SahilSatyam/AgentCrash/issues
7
+ Author: Sahil Satyam
8
+ License-Expression: MIT
9
+ License-File: LICENSE
10
+ Keywords: agentic-workflows,ai-agents,chaos-engineering,evaluation,fault-injection,llm,reliability
11
+ Classifier: Development Status :: 3 - Alpha
12
+ Classifier: Intended Audience :: Developers
13
+ Classifier: Programming Language :: Python :: 3
14
+ Classifier: Programming Language :: Python :: 3.11
15
+ Classifier: Programming Language :: Python :: 3.12
16
+ Classifier: Programming Language :: Python :: 3.13
17
+ Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
18
+ Classifier: Topic :: Software Development :: Testing
19
+ Requires-Python: >=3.11
20
+ Requires-Dist: jsonschema
21
+ Requires-Dist: pyyaml
22
+ Provides-Extra: dev
23
+ Requires-Dist: black; extra == 'dev'
24
+ Requires-Dist: fastapi; extra == 'dev'
25
+ Requires-Dist: httpx2; extra == 'dev'
26
+ Requires-Dist: pytest; extra == 'dev'
27
+ Requires-Dist: ruff; extra == 'dev'
28
+ Requires-Dist: uvicorn; extra == 'dev'
29
+ Provides-Extra: server
30
+ Requires-Dist: fastapi; extra == 'server'
31
+ Requires-Dist: uvicorn; extra == 'server'
32
+ Description-Content-Type: text/markdown
33
+
34
+ # AgentCrash — chaos engineering for AI agents
35
+
36
+ [![CI](https://github.com/SahilSatyam/AgentCrash/actions/workflows/ci.yml/badge.svg)](https://github.com/SahilSatyam/AgentCrash/actions/workflows/ci.yml)
37
+ [![PyPI](https://img.shields.io/pypi/v/agentcrash-core.svg)](https://pypi.org/project/agentcrash-core/)
38
+ [![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-blue.svg)](https://www.python.org/downloads/)
39
+ [![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
40
+
41
+ Most agent evals grade the final answer. A tool-using agent can give a
42
+ correct-looking answer while it charges a customer twice, acts on stale
43
+ data, or follows instructions hidden in a document. AgentCrash tests the
44
+ **path**: it breaks your agent's tools on purpose, records every step in
45
+ a hash-chained trace, and judges the trace with deterministic invariants.
46
+ A model may suggest a fix. Only the trace decides whether it passes.
47
+
48
+ It works with any agentic workflow: plain Python, LangGraph, OpenAI
49
+ Agents SDK, CrewAI, MCP tools. You wrap the tool boundary; AgentCrash
50
+ never runs your agent for you.
51
+
52
+ ```bash
53
+ pip install agentcrash-core
54
+ ```
55
+
56
+ ## 60-second example
57
+
58
+ ```python
59
+ from agentcrash import AgentCrash, DuplicateMutatingEffect, WorkflowConfig, evaluate_trace
60
+
61
+ ac = AgentCrash(WorkflowConfig(project="checkout-agent", fault_injection=True, seed=42))
62
+ # The first call to payments.charge commits, then reports a timeout.
63
+ ac.configure_faults({"payments.charge": {"kind": "timeout_after_commit", "occurrence": 1}})
64
+
65
+ def charge(order_id: str) -> dict:
66
+ return {"charged": order_id} # your real side effect
67
+
68
+ with ac.run(name="checkout") as run, run.step("charge") as step:
69
+ result = charge("o-1")
70
+ fault = run.maybe_fault("payments.charge")
71
+ step.record_tool("payments.charge", inputs={"order": "o-1"}, outputs=result)
72
+ if fault: # a naive agent sees TIMEOUT and blindly retries
73
+ step.record_tool("payments.charge", inputs={"order": "o-1"}, outputs=charge("o-1"))
74
+
75
+ report = evaluate_trace(ac.trace(run.run_id), [DuplicateMutatingEffect()], run_id=run.run_id)
76
+ print(report["verdict"]) # FAIL
77
+ print(report["results"][0]["evidence"]) # duplicate effect(s): payments.charge at steps ...
78
+ ```
79
+
80
+ Fix the retry (check status first, or send an idempotency key), run it
81
+ again, and the same invariant passes.
82
+
83
+ ## What you get
84
+
85
+ | Part | What it does |
86
+ |---|---|
87
+ | SDK (`AgentCrash`) | `run` / `step` / `record_tool` / `maybe_fault`; sync and async; no-op when `AGENTCRASH_DISABLED=1` |
88
+ | Fault injection | Seeded and reproducible; fires on tool + op + Nth call. Kinds: `timeout_after_commit`, `timeout_before_commit`, `concurrent_update`, `poisoned_doc` |
89
+ | Traces | Versioned events, `prev_hash` chain, secrets and PII redacted at write time |
90
+ | Invariants | Pure `(trace, state) -> Verdict`; built-ins `required_step:<step>`, `duplicate_mutating_effect`, `fresh_write_only:<tools>`, `no_lost_update`; register your own |
91
+ | `agentcrash check` | Judge a JSONL trace; JSON output, exit codes, `--plugin` for your invariants, `--verify-chain`, `--scan-pii` |
92
+ | Recovery policy | `policy/v1` YAML + RFC 7396 merge patches; optional model-proposed patches, schema-gated |
93
+ | Control plane | `agentcrash serve`: `/v1` API, idempotent ingestion, Lab UI, bearer auth, retention |
94
+
95
+ ## How it fits together
96
+
97
+ ```mermaid
98
+ flowchart LR
99
+ H[Your agent] -->|tool calls| SDK[AgentCrash SDK]
100
+ F[Fault policy<br/>seeded] -.->|inject| SDK
101
+ SDK --> T[Hash-chained trace<br/>+ state snapshot]
102
+ SDK -->|batch flush| S[Control plane<br/>agentcrash serve]
103
+ T --> C[Invariants<br/>no model in the loop]
104
+ C --> V[Verdict + evidence<br/>PASS / FAIL]
105
+ V -.->|optional| P[Policy patch<br/>model may propose]
106
+ P -.->|replay| H
107
+ ```
108
+
109
+ ## Your own invariants
110
+
111
+ ```python
112
+ from agentcrash.verifiers import Verdict, register_verifier
113
+
114
+ class NoRefundOverLimit:
115
+ id = "no_refund_over_limit"
116
+
117
+ def check(self, trace, state):
118
+ big = [e["step"] for e in trace
119
+ if e.get("tool") == "refund" and e.get("inputs", {}).get("amount", 0) > 500]
120
+ return Verdict(self.id, not big, f"refunds over limit at {big}" if big else "ok")
121
+
122
+ register_verifier(NoRefundOverLimit())
123
+ ```
124
+
125
+ ```bash
126
+ agentcrash check --trace run.jsonl --plugin my_agent.invariants \
127
+ --invariant no_refund_over_limit --format json --report-out verdict.json
128
+ ```
129
+
130
+ ## Install
131
+
132
+ | You want | Command |
133
+ |---|---|
134
+ | SDK + `check` | `pip install agentcrash-core` |
135
+ | Control-plane service | `pip install "agentcrash-core[server]"` |
136
+ | Contributors | `uv venv && uv pip install -e ".[dev]"` |
137
+
138
+ Python 3.11+. No API key needed; the whole test suite runs offline.
139
+
140
+ On Windows with Smart App Control or WDAC, unsigned console-script
141
+ launchers (`agentcrash.exe`, and any other pip/uv-installed `.exe`) may be
142
+ blocked. Use `python -m agentcrash ...`, which is equivalent.
143
+
144
+ Upgrading from 0.2? See [CHANGELOG.md](CHANGELOG.md) for the migration table.
145
+
146
+ ## Configuration
147
+
148
+ | Variable | Effect |
149
+ |---|---|
150
+ | `AGENTCRASH_DISABLED=1` | Every SDK call becomes a no-op passthrough |
151
+ | `AGENTCRASH_API_TOKENS` | Comma-separated bearer tokens required on `/v1/*`; empty = loopback mode |
152
+ | `AGENTCRASH_LLM_BASE_URL` | Opt-in: send failing traces to an OpenAI-compatible `/chat/completions` endpoint for a candidate policy patch |
153
+ | `AGENTCRASH_LLM_API_KEY` | Bearer token for that call; sent as a header only, never traced |
154
+
155
+ `LLAMA_BASE_URL` / `LLAMA_API_KEY` still work as fallbacks.
156
+
157
+ ## Develop
158
+
159
+ ```bash
160
+ git clone https://github.com/SahilSatyam/AgentCrash.git && cd AgentCrash
161
+ uv venv && uv pip install -e ".[dev]"
162
+ pytest -q && ruff check . && black --check .
163
+ ```
164
+
165
+ ## Layout
166
+
167
+ - `src/agentcrash/`: `sdk`, `events`, `injector`, `tracer`, `clock`,
168
+ `invariants`, `verifiers`, `evaluate`, `policy`, `ledger`, `server`,
169
+ `cli`, `scan_pii`, `lab/`, `schemas/`
170
+ - `examples/`: reference integrations (`minimal_agent/`)
171
+ - `docs/`: [overview](docs/overview.md), [setup](docs/setup.md),
172
+ [architecture](docs/architecture-decoupling.md),
173
+ [roadmap](docs/plans/2026-11-14-sota-roadmap.md)
@@ -0,0 +1,140 @@
1
+ # AgentCrash — chaos engineering for AI agents
2
+
3
+ [![CI](https://github.com/SahilSatyam/AgentCrash/actions/workflows/ci.yml/badge.svg)](https://github.com/SahilSatyam/AgentCrash/actions/workflows/ci.yml)
4
+ [![PyPI](https://img.shields.io/pypi/v/agentcrash-core.svg)](https://pypi.org/project/agentcrash-core/)
5
+ [![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-blue.svg)](https://www.python.org/downloads/)
6
+ [![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
7
+
8
+ Most agent evals grade the final answer. A tool-using agent can give a
9
+ correct-looking answer while it charges a customer twice, acts on stale
10
+ data, or follows instructions hidden in a document. AgentCrash tests the
11
+ **path**: it breaks your agent's tools on purpose, records every step in
12
+ a hash-chained trace, and judges the trace with deterministic invariants.
13
+ A model may suggest a fix. Only the trace decides whether it passes.
14
+
15
+ It works with any agentic workflow: plain Python, LangGraph, OpenAI
16
+ Agents SDK, CrewAI, MCP tools. You wrap the tool boundary; AgentCrash
17
+ never runs your agent for you.
18
+
19
+ ```bash
20
+ pip install agentcrash-core
21
+ ```
22
+
23
+ ## 60-second example
24
+
25
+ ```python
26
+ from agentcrash import AgentCrash, DuplicateMutatingEffect, WorkflowConfig, evaluate_trace
27
+
28
+ ac = AgentCrash(WorkflowConfig(project="checkout-agent", fault_injection=True, seed=42))
29
+ # The first call to payments.charge commits, then reports a timeout.
30
+ ac.configure_faults({"payments.charge": {"kind": "timeout_after_commit", "occurrence": 1}})
31
+
32
+ def charge(order_id: str) -> dict:
33
+ return {"charged": order_id} # your real side effect
34
+
35
+ with ac.run(name="checkout") as run, run.step("charge") as step:
36
+ result = charge("o-1")
37
+ fault = run.maybe_fault("payments.charge")
38
+ step.record_tool("payments.charge", inputs={"order": "o-1"}, outputs=result)
39
+ if fault: # a naive agent sees TIMEOUT and blindly retries
40
+ step.record_tool("payments.charge", inputs={"order": "o-1"}, outputs=charge("o-1"))
41
+
42
+ report = evaluate_trace(ac.trace(run.run_id), [DuplicateMutatingEffect()], run_id=run.run_id)
43
+ print(report["verdict"]) # FAIL
44
+ print(report["results"][0]["evidence"]) # duplicate effect(s): payments.charge at steps ...
45
+ ```
46
+
47
+ Fix the retry (check status first, or send an idempotency key), run it
48
+ again, and the same invariant passes.
49
+
50
+ ## What you get
51
+
52
+ | Part | What it does |
53
+ |---|---|
54
+ | SDK (`AgentCrash`) | `run` / `step` / `record_tool` / `maybe_fault`; sync and async; no-op when `AGENTCRASH_DISABLED=1` |
55
+ | Fault injection | Seeded and reproducible; fires on tool + op + Nth call. Kinds: `timeout_after_commit`, `timeout_before_commit`, `concurrent_update`, `poisoned_doc` |
56
+ | Traces | Versioned events, `prev_hash` chain, secrets and PII redacted at write time |
57
+ | Invariants | Pure `(trace, state) -> Verdict`; built-ins `required_step:<step>`, `duplicate_mutating_effect`, `fresh_write_only:<tools>`, `no_lost_update`; register your own |
58
+ | `agentcrash check` | Judge a JSONL trace; JSON output, exit codes, `--plugin` for your invariants, `--verify-chain`, `--scan-pii` |
59
+ | Recovery policy | `policy/v1` YAML + RFC 7396 merge patches; optional model-proposed patches, schema-gated |
60
+ | Control plane | `agentcrash serve`: `/v1` API, idempotent ingestion, Lab UI, bearer auth, retention |
61
+
62
+ ## How it fits together
63
+
64
+ ```mermaid
65
+ flowchart LR
66
+ H[Your agent] -->|tool calls| SDK[AgentCrash SDK]
67
+ F[Fault policy<br/>seeded] -.->|inject| SDK
68
+ SDK --> T[Hash-chained trace<br/>+ state snapshot]
69
+ SDK -->|batch flush| S[Control plane<br/>agentcrash serve]
70
+ T --> C[Invariants<br/>no model in the loop]
71
+ C --> V[Verdict + evidence<br/>PASS / FAIL]
72
+ V -.->|optional| P[Policy patch<br/>model may propose]
73
+ P -.->|replay| H
74
+ ```
75
+
76
+ ## Your own invariants
77
+
78
+ ```python
79
+ from agentcrash.verifiers import Verdict, register_verifier
80
+
81
+ class NoRefundOverLimit:
82
+ id = "no_refund_over_limit"
83
+
84
+ def check(self, trace, state):
85
+ big = [e["step"] for e in trace
86
+ if e.get("tool") == "refund" and e.get("inputs", {}).get("amount", 0) > 500]
87
+ return Verdict(self.id, not big, f"refunds over limit at {big}" if big else "ok")
88
+
89
+ register_verifier(NoRefundOverLimit())
90
+ ```
91
+
92
+ ```bash
93
+ agentcrash check --trace run.jsonl --plugin my_agent.invariants \
94
+ --invariant no_refund_over_limit --format json --report-out verdict.json
95
+ ```
96
+
97
+ ## Install
98
+
99
+ | You want | Command |
100
+ |---|---|
101
+ | SDK + `check` | `pip install agentcrash-core` |
102
+ | Control-plane service | `pip install "agentcrash-core[server]"` |
103
+ | Contributors | `uv venv && uv pip install -e ".[dev]"` |
104
+
105
+ Python 3.11+. No API key needed; the whole test suite runs offline.
106
+
107
+ On Windows with Smart App Control or WDAC, unsigned console-script
108
+ launchers (`agentcrash.exe`, and any other pip/uv-installed `.exe`) may be
109
+ blocked. Use `python -m agentcrash ...`, which is equivalent.
110
+
111
+ Upgrading from 0.2? See [CHANGELOG.md](CHANGELOG.md) for the migration table.
112
+
113
+ ## Configuration
114
+
115
+ | Variable | Effect |
116
+ |---|---|
117
+ | `AGENTCRASH_DISABLED=1` | Every SDK call becomes a no-op passthrough |
118
+ | `AGENTCRASH_API_TOKENS` | Comma-separated bearer tokens required on `/v1/*`; empty = loopback mode |
119
+ | `AGENTCRASH_LLM_BASE_URL` | Opt-in: send failing traces to an OpenAI-compatible `/chat/completions` endpoint for a candidate policy patch |
120
+ | `AGENTCRASH_LLM_API_KEY` | Bearer token for that call; sent as a header only, never traced |
121
+
122
+ `LLAMA_BASE_URL` / `LLAMA_API_KEY` still work as fallbacks.
123
+
124
+ ## Develop
125
+
126
+ ```bash
127
+ git clone https://github.com/SahilSatyam/AgentCrash.git && cd AgentCrash
128
+ uv venv && uv pip install -e ".[dev]"
129
+ pytest -q && ruff check . && black --check .
130
+ ```
131
+
132
+ ## Layout
133
+
134
+ - `src/agentcrash/`: `sdk`, `events`, `injector`, `tracer`, `clock`,
135
+ `invariants`, `verifiers`, `evaluate`, `policy`, `ledger`, `server`,
136
+ `cli`, `scan_pii`, `lab/`, `schemas/`
137
+ - `examples/`: reference integrations (`minimal_agent/`)
138
+ - `docs/`: [overview](docs/overview.md), [setup](docs/setup.md),
139
+ [architecture](docs/architecture-decoupling.md),
140
+ [roadmap](docs/plans/2026-11-14-sota-roadmap.md)