agentdebugx 0.2.12__tar.gz → 0.3.0__tar.gz

agentdebugx-0.3.0/PKG-INFO

@@ -0,0 +1,234 @@
+Metadata-Version: 2.4
+Name: agentdebugx
+Version: 0.3.0
+Summary: Portable error analysis, tracing, and recovery framework for agentic AI systems. Import as `agentdebug`.
+License: MIT
+License-File: LICENSE
+Keywords: llm,agents,debugging,observability,failure-analysis,agent-debugging,agentic-ai,tracing,evaluation
+Author: ULab @ UIUC
+Author-email: ulab@illinois.edu
+Requires-Python: >=3.9,<4.0
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Debuggers
+Classifier: Topic :: Software Development :: Quality Assurance
+Classifier: Topic :: System :: Monitoring
+Classifier: Typing :: Typed
+Provides-Extra: all
+Provides-Extra: crewai
+Provides-Extra: gui
+Provides-Extra: hub-hf
+Provides-Extra: langgraph
+Provides-Extra: openai-agents
+Provides-Extra: otel
+Provides-Extra: ui
+Requires-Dist: anthropic ; extra == "gui"
+Requires-Dist: backoff ; extra == "gui"
+Requires-Dist: chromadb ; extra == "gui"
+Requires-Dist: httpx (>=0.24,<1.0)
+Requires-Dist: langchain-chroma ; extra == "gui"
+Requires-Dist: langchain-core ; extra == "langgraph" or extra == "gui" or extra == "all"
+Requires-Dist: langchain-openai ; extra == "gui"
+Requires-Dist: matplotlib ; extra == "gui"
+Requires-Dist: numpy ; extra == "gui"
+Requires-Dist: openai ; extra == "gui"
+Requires-Dist: pandas ; extra == "gui"
+Requires-Dist: pillow ; extra == "gui"
+Requires-Dist: pydantic (>=1.10,<3.0)
+Requires-Dist: python-dotenv ; extra == "gui"
+Requires-Dist: scikit-learn ; extra == "gui"
+Requires-Dist: streamlit ; extra == "gui"
+Requires-Dist: streamlit-adjustable-columns ; extra == "gui"
+Requires-Dist: together ; extra == "gui"
+Requires-Dist: tqdm ; extra == "gui"
+Project-URL: Documentation, https://github.com/ulab-uiuc/AgentDebugX/tree/main/docs
+Project-URL: Homepage, https://github.com/ulab-uiuc/AgentDebugX
+Project-URL: Repository, https://github.com/ulab-uiuc/AgentDebugX
+Description-Content-Type: text/markdown
+
+<div align="center">
+
+# AgentDebugX
+
+**Failure attribution and recovery for LLM agents — from a trace to a root cause to a fix.**
+
+[![PyPI](https://img.shields.io/badge/pip-agentdebugx-3775A9)](https://pypi.org/project/agentdebugx/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
+[![Python](https://img.shields.io/badge/python-3.9%2B-blue)](pyproject.toml)
+
+[Website](https://www.agentdebugx.com) · [GitHub](https://github.com/AgentDebugX/AgentDebugX) · [Demo video](https://youtu.be/ztni6w0o_l8)
+
+</div>
+
+---
+
+When an LLM-agent run fails, the visible crash is usually a downstream symptom of an
+earlier decision — a dropped constraint, a stale memory read, a lost handoff. Tracing
+tools show *what* happened; they leave you to work out *which* step was responsible,
+*why*, and *what to change*.
+
+**AgentDebugX** is a self-hostable debugging layer that closes that gap. Point it at a
+live run or an exported log and it returns a typed diagnosis: the responsible step, a
+taxonomy-grounded explanation with evidence, and a concrete fix you can rerun — all in a
+local web console, a CLI, or an agentic skill your own agents can invoke.
+
+Everything runs on your machine. No account, no hosted service, no data leaves your host.
+
+## Highlights
+
+- **DeepDebug** — a multi-turn, tool-using diagnosis agent that reads the whole trace,
+  re-investigates along its structure (cascade walk / bisect), cross-examines its own
+  hypotheses, and returns an auditable verdict with one concrete fix.
+- **Cost-staged attribution** — free deterministic detectors → a single LLM-judge pass →
+  a family of localizers (all-at-once, step-by-step, binary-search) that trade cost for
+  resolution.
+- **End-to-end recovery** — turn a diagnosis into a rerun-ready fix (Reflexion, CRITIC,
+  Self-Refine, AutoManual, saga-rollback recoverers), gated behind explicit human/policy
+  approval.
+- **Portable trace schema** — one framework-agnostic format with runtime adapters
+  (LangGraph, CrewAI, OpenAI Agents SDK, OpenTelemetry, raw ReAct) and offline importers.
+- **Local console** — a no-build single-page app served straight from the wheel; aligns
+  the agent's own trace with AgentDebugX's error trace, row by row.
+- **Error Hub** — package scrubbed failure bundles (trace + diagnosis + fix) as CI
+  fixtures or a shareable cross-team corpus that doubles as DeepDebug's long-term memory.
+- **Computer-use agents** — an OSWorld importer and a GUI root-cause mode reason over the
+  screenshot-and-action channel, not just text.
+
+## Install
+
+```bash
+pip install agentdebugx
+```
+
+Optional extras (install only what you need):
+
+```bash
+pip install "agentdebugx[ui]"        # local web console (FastAPI + Uvicorn)
+pip install "agentdebugx[langgraph]" # LangGraph adapter
+pip install "agentdebugx[gui]"       # computer-use / OSWorld GUI RCA
+pip install "agentdebugx[all]"       # every optional integration
+```
+
+The package is imported as `agentdebug`:
+
+```python
+import agentdebug
+```
+
+## Quick start (library)
+
+Instrument a run with one context manager and analyze it in place:
+
+```python
+from agentdebug import AgentDebug, EventType
+
+dbg = AgentDebug()
+with dbg.trace(goal="Book a refundable NYC→SFO flight", framework="my-agent") as t:
+    t.record(EventType.PLAN, agent_name="planner",
+             output="search cheapest fares")          # drops the 'refundable' constraint
+    t.record(EventType.TOOL_RESULT, agent_name="browser", step_index=3,
+             error="Checkout failed: refund_policy required")
+    report = t.analyze()
+
+print(report.summary)
+for finding in report.findings:
+    print(finding.failure_mode_id, finding.step_index, finding.confidence)
+```
+
+The report names the **responsible** step (the planner at step 0), not just the visible
+crash (the browser at step 3).
+
+## Quick start (local console)
+
+```bash
+pip install "agentdebugx[ui]"
+agentdebug serve            # opens the console at http://127.0.0.1:7777
+```
+
+The console opens directly into the workspace — inspect traces, re-analyze any step with
+the LLM judge or DeepDebug, request a debug continuation, and rerun from a chosen step.
+It reads the same local store the library writes; nothing is uploaded.
+
+## CLI
+
+The `agentdebug` CLI exposes the full pipeline:
+
+| Stage | Command | Purpose |
+|-------|---------|---------|
+| Normalize | `agentdebug ingest <export>` | Convert an external trace export into `AgentTrajectory` JSON |
+| Diagnose | `agentdebug diagnose --store-jsonl <path>` | Detectors + attribution + recovery planning |
+| Inspect | `agentdebug serve` / `inspect` | Local web console over a store |
+| Deep dive | `agentdebug act deep <trace-id>` | Run the multi-turn DeepDebug agent |
+| Recover | `agentdebug rerun --report <report.json>` | Rerun an agent from a diagnostic report |
+| Share | `agentdebug hub push / pull` | Package or fetch scrubbed Error Hub bundles |
+| Skills | `agentdebug integrations` | Emit host-runtime integrations (e.g. Claude Code skill) |
+| Health | `agentdebug doctor` | Report adapter and integration availability |
+
+LLM-backed commands read credentials from the environment:
+
+```bash
+export AGENTDEBUG_LLM_BASE_URL=...   # any OpenAI-compatible endpoint
+export AGENTDEBUG_LLM_API_KEY=...
+export AGENTDEBUG_LLM_MODEL=...       # optional
+```
+
+Run `agentdebug <command> --help` for the authoritative, version-specific flags.
+
+## How it works
+
+```
+ capture / import ──▶ AgentTrajectory ──▶ Detect ──▶ Attribute ──▶ Recover ──▶ Rerun
+   adapters, logs        portable schema   rules,      responsible   ranked      from before/
+   (any framework)                         judge       step/agent    fixes       at/after cause
+                                             │
+                                             └── hard cases ──▶ DeepDebug (multi-turn agent)
+```
+
+A diagnosis is **layered on top of** the recorded trace, never written back into it, so
+one run can be re-analyzed by any method, compared across runs, and shared without losing
+its ground truth. See [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) for the DeepDebug
+design and [`docs/TRACE_SCHEMA.md`](docs/TRACE_SCHEMA.md) for the event schema.
+
+## Examples
+
+The [`examples/`](examples/) directory contains runnable end-to-end scripts:
+
+- `basic_usage.py` — record, analyze, inspect.
+- `multi_agent_cascade.py` — upstream root-cause attribution across a handoff cascade.
+- `langgraph/`, `crewai/`, `autogen_roundrobin_deepdebug.py` — framework adapters.
+- `taxonomy_induction_demo.py` — propose new failure modes from a corpus.
+- `claude_skill_integration/` — invoke AgentDebugX as an agentic skill.
+
+## Repository layout
+
+```
+src/agentdebug/     core package: schema, detectors, judge, attribution, DeepDebug,
+                    recovery, Error Hub, CLI, and the local web console (inspect/ui/)
+cua_debugger/       computer-use / OSWorld GUI root-cause tooling
+examples/           runnable usage examples
+docs/               architecture and trace-schema reference
+```
+
+## Safety
+
+AgentDebugX is **local-first**: traces stay on your machine and sharing is opt-in. A
+diagnosed fix can write to the world, so recovery is **suggest-only** — application stays
+behind an explicit human or policy gate. Diagnostic labels and fixes carry confidence and
+evidence; treat them as ranked hypotheses, not ground truth. Configure redaction,
+retention, and access control before collecting production traces.
+
+## License
+
+MIT — see [LICENSE](LICENSE).
+

agentdebugx-0.3.0/README.md

@@ -0,0 +1,174 @@
+<div align="center">
+
+# AgentDebugX
+
+**Failure attribution and recovery for LLM agents — from a trace to a root cause to a fix.**
+
+[![PyPI](https://img.shields.io/badge/pip-agentdebugx-3775A9)](https://pypi.org/project/agentdebugx/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
+[![Python](https://img.shields.io/badge/python-3.9%2B-blue)](pyproject.toml)
+
+[Website](https://www.agentdebugx.com) · [GitHub](https://github.com/AgentDebugX/AgentDebugX) · [Demo video](https://youtu.be/ztni6w0o_l8)
+
+</div>
+
+---
+
+When an LLM-agent run fails, the visible crash is usually a downstream symptom of an
+earlier decision — a dropped constraint, a stale memory read, a lost handoff. Tracing
+tools show *what* happened; they leave you to work out *which* step was responsible,
+*why*, and *what to change*.
+
+**AgentDebugX** is a self-hostable debugging layer that closes that gap. Point it at a
+live run or an exported log and it returns a typed diagnosis: the responsible step, a
+taxonomy-grounded explanation with evidence, and a concrete fix you can rerun — all in a
+local web console, a CLI, or an agentic skill your own agents can invoke.
+
+Everything runs on your machine. No account, no hosted service, no data leaves your host.
+
+## Highlights
+
+- **DeepDebug** — a multi-turn, tool-using diagnosis agent that reads the whole trace,
+  re-investigates along its structure (cascade walk / bisect), cross-examines its own
+  hypotheses, and returns an auditable verdict with one concrete fix.
+- **Cost-staged attribution** — free deterministic detectors → a single LLM-judge pass →
+  a family of localizers (all-at-once, step-by-step, binary-search) that trade cost for
+  resolution.
+- **End-to-end recovery** — turn a diagnosis into a rerun-ready fix (Reflexion, CRITIC,
+  Self-Refine, AutoManual, saga-rollback recoverers), gated behind explicit human/policy
+  approval.
+- **Portable trace schema** — one framework-agnostic format with runtime adapters
+  (LangGraph, CrewAI, OpenAI Agents SDK, OpenTelemetry, raw ReAct) and offline importers.
+- **Local console** — a no-build single-page app served straight from the wheel; aligns
+  the agent's own trace with AgentDebugX's error trace, row by row.
+- **Error Hub** — package scrubbed failure bundles (trace + diagnosis + fix) as CI
+  fixtures or a shareable cross-team corpus that doubles as DeepDebug's long-term memory.
+- **Computer-use agents** — an OSWorld importer and a GUI root-cause mode reason over the
+  screenshot-and-action channel, not just text.
+
+## Install
+
+```bash
+pip install agentdebugx
+```
+
+Optional extras (install only what you need):
+
+```bash
+pip install "agentdebugx[ui]"        # local web console (FastAPI + Uvicorn)
+pip install "agentdebugx[langgraph]" # LangGraph adapter
+pip install "agentdebugx[gui]"       # computer-use / OSWorld GUI RCA
+pip install "agentdebugx[all]"       # every optional integration
+```
+
+The package is imported as `agentdebug`:
+
+```python
+import agentdebug
+```
+
+## Quick start (library)
+
+Instrument a run with one context manager and analyze it in place:
+
+```python
+from agentdebug import AgentDebug, EventType
+
+dbg = AgentDebug()
+with dbg.trace(goal="Book a refundable NYC→SFO flight", framework="my-agent") as t:
+    t.record(EventType.PLAN, agent_name="planner",
+             output="search cheapest fares")          # drops the 'refundable' constraint
+    t.record(EventType.TOOL_RESULT, agent_name="browser", step_index=3,
+             error="Checkout failed: refund_policy required")
+    report = t.analyze()
+
+print(report.summary)
+for finding in report.findings:
+    print(finding.failure_mode_id, finding.step_index, finding.confidence)
+```
+
+The report names the **responsible** step (the planner at step 0), not just the visible
+crash (the browser at step 3).
+
+## Quick start (local console)
+
+```bash
+pip install "agentdebugx[ui]"
+agentdebug serve            # opens the console at http://127.0.0.1:7777
+```
+
+The console opens directly into the workspace — inspect traces, re-analyze any step with
+the LLM judge or DeepDebug, request a debug continuation, and rerun from a chosen step.
+It reads the same local store the library writes; nothing is uploaded.
+
+## CLI
+
+The `agentdebug` CLI exposes the full pipeline:
+
+| Stage | Command | Purpose |
+|-------|---------|---------|
+| Normalize | `agentdebug ingest <export>` | Convert an external trace export into `AgentTrajectory` JSON |
+| Diagnose | `agentdebug diagnose --store-jsonl <path>` | Detectors + attribution + recovery planning |
+| Inspect | `agentdebug serve` / `inspect` | Local web console over a store |
+| Deep dive | `agentdebug act deep <trace-id>` | Run the multi-turn DeepDebug agent |
+| Recover | `agentdebug rerun --report <report.json>` | Rerun an agent from a diagnostic report |
+| Share | `agentdebug hub push / pull` | Package or fetch scrubbed Error Hub bundles |
+| Skills | `agentdebug integrations` | Emit host-runtime integrations (e.g. Claude Code skill) |
+| Health | `agentdebug doctor` | Report adapter and integration availability |
+
+LLM-backed commands read credentials from the environment:
+
+```bash
+export AGENTDEBUG_LLM_BASE_URL=...   # any OpenAI-compatible endpoint
+export AGENTDEBUG_LLM_API_KEY=...
+export AGENTDEBUG_LLM_MODEL=...       # optional
+```
+
+Run `agentdebug <command> --help` for the authoritative, version-specific flags.
+
+## How it works
+
+```
+ capture / import ──▶ AgentTrajectory ──▶ Detect ──▶ Attribute ──▶ Recover ──▶ Rerun
+   adapters, logs        portable schema   rules,      responsible   ranked      from before/
+   (any framework)                         judge       step/agent    fixes       at/after cause
+                                             │
+                                             └── hard cases ──▶ DeepDebug (multi-turn agent)
+```
+
+A diagnosis is **layered on top of** the recorded trace, never written back into it, so
+one run can be re-analyzed by any method, compared across runs, and shared without losing
+its ground truth. See [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) for the DeepDebug
+design and [`docs/TRACE_SCHEMA.md`](docs/TRACE_SCHEMA.md) for the event schema.
+
+## Examples
+
+The [`examples/`](examples/) directory contains runnable end-to-end scripts:
+
+- `basic_usage.py` — record, analyze, inspect.
+- `multi_agent_cascade.py` — upstream root-cause attribution across a handoff cascade.
+- `langgraph/`, `crewai/`, `autogen_roundrobin_deepdebug.py` — framework adapters.
+- `taxonomy_induction_demo.py` — propose new failure modes from a corpus.
+- `claude_skill_integration/` — invoke AgentDebugX as an agentic skill.
+
+## Repository layout
+
+```
+src/agentdebug/     core package: schema, detectors, judge, attribution, DeepDebug,
+                    recovery, Error Hub, CLI, and the local web console (inspect/ui/)
+cua_debugger/       computer-use / OSWorld GUI root-cause tooling
+examples/           runnable usage examples
+docs/               architecture and trace-schema reference
+```
+
+## Safety
+
+AgentDebugX is **local-first**: traces stay on your machine and sharing is opt-in. A
+diagnosed fix can write to the world, so recovery is **suggest-only** — application stays
+behind an explicit human or policy gate. Diagnostic labels and fixes carry confidence and
+evidence; treat them as ranked hypotheses, not ground truth. Configure redaction,
+retention, and access control before collecting production traces.
+
+## License
+
+MIT — see [LICENSE](LICENSE).

{agentdebugx-0.2.12 → agentdebugx-0.3.0}/docs/05_adapters.md

@@ -1,5 +1,59 @@
 # 05 — Framework Adapters
 
+AgentDebugX has two adapter families:
+
+- **Runtime adapters** observe a live framework and emit `AgentEvent` records
+  while the agent runs.
+- **Offline import adapters** read an exported log or benchmark rollout after
+  the run has finished and convert it into `AgentTrajectory`.
+
+The shipped offline importer is `agentdebug.adapters.importers`. It is the
+recommended entry point when users already have trace files from another
+system:
+
+```bash
+agentdebug convert external_trace.json --format auto --out trajectory.json
+agentdebug analyze trajectory.json --traceback
+```
+
+```python
+from agentdebug.adapters.importers import convert_file
+
+trajectory = convert_file("external_trace.json", format="auto")
+```
+
+`format="auto"` recognizes native `AgentTrajectory`, OpenAI-style `messages`,
+rollout `conversations`, generic `event_list` exports, WebShop page logs,
+OpenAI Agents span dumps, CrewAI events, and LangGraph/LangChain callback logs.
+
+## 0.1 Analysis packs for imported traces
+
+Deterministic analysis is also pack-oriented. The default `auto` setting loads
+the generic `core` rules for every trace and adds benchmark-specific packs only
+when the trajectory metadata indicates they apply. A pack can include both
+single-event rules and cross-event trajectory rules.
+
+```bash
+agentdebug analyze trajectory.json --rule-pack auto
+agentdebug analyze trajectory.json --rule-pack core
+agentdebug analyze trajectory.json --rule-pack core --rule-pack agenterrorbench
+```
+
+The first shipped packs are:
+
+- `core` — generic single-event rules for explicit errors, LLM limits, format
+  errors, parameter errors, wrong-tool signals, planning loops, handoff loss,
+  visual/perception signals, plus cross-event repeated-tool-call,
+  repeated-state, and step-budget rules.
+- `agenterrorbench` — ALFWorld/WebShop-specific weak signals such as
+  `Nothing happens`, loaded only by explicit request or benchmark detection.
+
+New packs should live under `agentdebug.rules.<pack_name>` and expose a
+`build_event_rules()` and/or `build_trajectory_rules()` function returning
+rule objects that implement the shared `EventRule` or `TrajectoryRule`
+protocols from `agentdebug.rules.base`. `build_rules()` is still accepted as a
+backward-compatible alias for event-only packs.
+
 ## 1. Adapter contract
 
 Every adapter implements:

{agentdebugx-0.2.12 → agentdebugx-0.3.0}/docs/14_api_reference.md

@@ -221,7 +221,8 @@ The public design above is the long-term `agentdebugx` contract. The current
 package already ships a smaller but working `agentdebug` CLI:
 
 ```bash
-agentdebug analyze <trajectory.json> [--suggest] [--traceback]
+agentdebug analyze <trajectory.json> [--suggest] [--traceback] [--rule-pack auto|core|agenterrorbench|all]
+agentdebug convert <external-trace.json|jsonl> [--format auto] --out trajectory.json
 agentdebug list --store-sqlite .agentdebug/errors.sqlite
 agentdebug show <trace_id> --store-sqlite .agentdebug/errors.sqlite
 agentdebug judge <trajectory.json|trace_id> --attribute [--traceback]
@@ -238,6 +239,42 @@ agentdebug doctor
 `--traceback` renders `AgentTraceback`, a Python-traceback-style cascade view
 implemented by `agentdebug.traceback.format_traceback(report, trajectory)`.
 DeepDebug can provide explicit cascade edges through
+
+`agentdebug convert` is the offline import path for traces that were not
+recorded by AgentDebugX. It normalizes common exported formats into
+`AgentTrajectory` so the rest of the current CLI can operate on them:
+
+```bash
+agentdebug convert webshop_rollout.jsonl --format auto --out trajectory.json
+agentdebug analyze trajectory.json --traceback
+```
+
+The Python API is:
+
+```python
+from agentdebug.adapters.importers import convert_file, convert_payload
+
+trajectory = convert_file("external_trace.json", format="auto")
+```
+
+Supported import formats are native `AgentTrajectory`, `messages`,
+`message_list`, `conversations`, `event_list`, `webshop_pages`,
+`openai_agents_spans`, `crewai_events`, and `langgraph_callbacks`.
+
+Deterministic analysis uses analysis packs. `auto` loads `core` for all traces
+and adds `agenterrorbench` when the trajectory metadata/framework looks like
+AgentErrorBench, ALFWorld, or WebShop. A pack can include single-event rules
+and cross-event trajectory rules; `agentdebug analyze` runs both. Repeat
+`--rule-pack` to compose packs:
+
+```bash
+agentdebug analyze trajectory.json --rule-pack core --rule-pack agenterrorbench
+```
+
+Every rule-based finding includes `metadata.rule_pack`, `metadata.rule_id`, and
+`metadata.confidence_basis` for auditability.
+The legacy `agentdebug.detectors` classes remain importable, but the default
+deterministic entry point is `HeuristicAnalyzer().analyze(trajectory)`.
 `finding.metadata['cascading_from_event_id']`; heuristic and single-pass judge
 reports fall back to step-index ordering.