agentdebugx 0.2.12__tar.gz → 0.3.1__tar.gz

agentdebugx-0.3.1/PKG-INFO

@@ -0,0 +1,590 @@
+Metadata-Version: 2.4
+Name: agentdebugx
+Version: 0.3.1
+Summary: Portable error analysis, tracing, and recovery framework for agentic AI systems. Import as `agentdebug`.
+License: MIT
+License-File: LICENSE
+Keywords: llm,agents,debugging,observability,failure-analysis,agent-debugging,agentic-ai,tracing,evaluation
+Author: ULab @ UIUC
+Author-email: ulab@illinois.edu
+Requires-Python: >=3.9,<4.0
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Debuggers
+Classifier: Topic :: Software Development :: Quality Assurance
+Classifier: Topic :: System :: Monitoring
+Classifier: Typing :: Typed
+Provides-Extra: all
+Provides-Extra: crewai
+Provides-Extra: gui
+Provides-Extra: hub-hf
+Provides-Extra: langgraph
+Provides-Extra: openai-agents
+Provides-Extra: otel
+Provides-Extra: ui
+Requires-Dist: anthropic ; extra == "gui" or extra == "all"
+Requires-Dist: backoff ; extra == "gui" or extra == "all"
+Requires-Dist: chromadb ; extra == "gui" or extra == "all"
+Requires-Dist: crewai ; extra == "crewai" or extra == "all"
+Requires-Dist: fastapi ; extra == "ui" or extra == "all"
+Requires-Dist: httpx (>=0.24,<1.0)
+Requires-Dist: huggingface_hub ; extra == "hub-hf" or extra == "all"
+Requires-Dist: langchain-chroma ; extra == "gui" or extra == "all"
+Requires-Dist: langchain-core ; extra == "langgraph" or extra == "gui" or extra == "all"
+Requires-Dist: langchain-openai ; extra == "gui" or extra == "all"
+Requires-Dist: matplotlib ; extra == "gui" or extra == "all"
+Requires-Dist: numpy ; extra == "gui" or extra == "all"
+Requires-Dist: openai ; extra == "gui" or extra == "all"
+Requires-Dist: openai-agents ; extra == "openai-agents" or extra == "all"
+Requires-Dist: opentelemetry-api ; extra == "otel" or extra == "all"
+Requires-Dist: opentelemetry-sdk ; extra == "otel" or extra == "all"
+Requires-Dist: pandas ; extra == "gui" or extra == "all"
+Requires-Dist: pillow ; extra == "gui" or extra == "all"
+Requires-Dist: pydantic (>=1.10,<3.0)
+Requires-Dist: python-dotenv ; extra == "gui" or extra == "all"
+Requires-Dist: scikit-learn ; extra == "gui" or extra == "all"
+Requires-Dist: streamlit ; extra == "gui" or extra == "all"
+Requires-Dist: streamlit-adjustable-columns ; extra == "gui" or extra == "all"
+Requires-Dist: together ; extra == "gui" or extra == "all"
+Requires-Dist: tqdm ; extra == "gui" or extra == "all"
+Requires-Dist: uvicorn ; extra == "ui" or extra == "all"
+Project-URL: Documentation, https://github.com/AgentDebugX/AgentDebugX
+Project-URL: Homepage, https://www.agentdebugx.com
+Project-URL: Repository, https://github.com/AgentDebugX/AgentDebugX
+Description-Content-Type: text/markdown
+
+<div align="center">
+
+<img src="docs/assets/logo.png" alt="AgentDebugX logo" width="360">
+
+# AgentDebugX
+
+**A local-first debugging framework for agentic AI systems: diagnose failures, attribute root causes, recover with evidence, and validate fixes through reruns.**
+
+<a href="https://www.agentdebugx.com"><img src="https://img.shields.io/badge/WEBSITE-208B57?style=for-the-badge&logo=googlechrome&logoColor=white" alt="AgentDebugX website"></a>
+<a href="https://github.com/AgentDebugX/AgentDebugX"><img src="https://img.shields.io/badge/GITHUB-24292F?style=for-the-badge&logo=github&logoColor=white" alt="AgentDebugX GitHub repository"></a>
+<a href="https://youtu.be/ztni6w0o_l8"><img src="https://img.shields.io/badge/DEMO_VIDEO-EA4335?style=for-the-badge&logo=youtube&logoColor=white" alt="AgentDebugX demo video"></a>
+
+[![PyPI](https://img.shields.io/badge/pip-agentdebugx-3775A9)](https://pypi.org/project/agentdebugx/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
+[![Python](https://img.shields.io/badge/python-3.9%2B-blue)](pyproject.toml)
+[![GitHub Stars](https://img.shields.io/github/stars/AgentDebugX/AgentDebugX?style=flat&logo=github&label=Stars)](https://github.com/AgentDebugX/AgentDebugX/stargazers)
+[![GitHub Forks](https://img.shields.io/github/forks/AgentDebugX/AgentDebugX?style=flat&logo=github&label=Forks)](https://github.com/AgentDebugX/AgentDebugX/forks)
+
+
+</div>
+
+---
+
+AgentDebugX turns failed agent runs into structured, auditable debugging
+artifacts. It ingests a live or exported trajectory, detects visible failure
+signals, attributes them to responsible steps or agents, proposes recovery
+actions, and prepares controlled reruns so fixes can be validated instead of
+guessed.
+
+The project is designed for researchers and engineers building complex LLM
+agents: multi-agent systems, tool-using agents, computer-use agents, benchmark
+runners, and local agent development workflows. AgentDebugX is local-first by
+default: traces stay on your machine, sharing is opt-in, and recovery proposals
+carry explicit policy and approval metadata into the Rerun boundary.
+
+## System Overview
+
+<p align="center">
+  <img src="docs/assets/overview.png" alt="AgentDebugX system overview" width="900">
+</p>
+
+AgentDebugX follows the two-stage loop used by the project paper:
+
+```text
+Diagnose = Detect -> Attribute -> Recover
+Rerun    = checkpoint -> retry directive -> branch execution -> evaluation
+```
+
+`Diagnose` explains what failed and why. `Rerun` tests whether the proposed
+recovery actually improves the agent behavior.
+
+## Why AgentDebugX
+
+Tracing tools show what happened. AgentDebugX focuses on the debugging step that
+usually comes next:
+
+- Which earlier decision caused the visible failure?
+- Which agent, tool call, memory read, handoff, or GUI action was responsible?
+- What evidence supports that diagnosis?
+- What concrete recovery should be tried?
+- Did the rerun branch improve the outcome?
+
+The output is a portable diagnostic report that can be inspected in a local UI,
+used by a CLI workflow, stored in an Error Hub bundle, or invoked from an
+agentic skill.
+
+## Core Capabilities
+
+- **Portable trace schema**: framework-agnostic trajectory, event, finding, and
+  diagnostic report models.
+- **Ingest adapters**: normalize raw JSON, LangGraph, CrewAI, OpenAI Agents SDK,
+  OpenTelemetry, GAIA/Open Deep Research, OSWorld, and other exported traces.
+- **Detect**: deterministic analyzers, manifest-backed rule packs, LLM judge
+  mode, GUI-aware signals, and taxonomy induction support.
+- **Attribute**: heuristic attribution, all-at-once analysis, step-by-step
+  localization, binary search, counterfactual attribution, MOE localization,
+  and DeepDebug.
+- **Recover**: Reflexion, CRITIC, Self-Refine, AutoManual, DeepDebug recovery,
+  and saga rollback style strategies.
+- **Rerun**: three explicit modes for plan/export only, labeled simulation, or
+  observed execution in an application-owned process or persistent HTTP runner.
+- **Local inspection UI**: no-build FastAPI dashboard for traces, reports,
+  saved cases, debug branches, and rerun-from-event workflows.
+- **Error Hub**: scrubbed, shareable failure bundles for regression tests,
+  benchmark corpora, and team debugging memory.
+- **Agent integrations**: generate host-runtime assets such as debugging skills
+  for external agent tools.
+
+## Install
+
+```bash
+pip install agentdebugx
+```
+
+Optional extras:
+
+```bash
+pip install "agentdebugx[ui]"             # local FastAPI dashboard
+pip install "agentdebugx[langgraph]"      # LangGraph adapter
+pip install "agentdebugx[crewai]"         # CrewAI adapter
+pip install "agentdebugx[openai-agents]"  # OpenAI Agents SDK adapter
+pip install "agentdebugx[otel]"           # OpenTelemetry ingest
+pip install "agentdebugx[gui]"            # computer-use / OSWorld GUI tooling
+pip install "agentdebugx[all]"            # all optional integrations
+```
+
+The package is installed as `agentdebugx` and imported as `agentdebug`:
+
+```python
+import agentdebug
+```
+
+## Quick Start: Python API
+
+Record a trajectory and analyze it locally:
+
+```python
+from agentdebug import AgentDebug, EventType
+
+debugger = AgentDebug()
+
+with debugger.trace(
+    goal="Book a refundable NYC to SFO flight",
+    framework="my-agent",
+) as trace:
+    trace.record(
+        EventType.PLAN,
+        agent_name="planner",
+        step_index=1,
+        output="Search for the cheapest fares.",
+    )
+    trace.record(
+        EventType.TOOL_RESULT,
+        agent_name="browser",
+        step_index=3,
+        error="Checkout failed: refund_policy is required.",
+    )
+
+    report = trace.analyze()
+
+print(report.summary)
+for finding in report.findings:
+    print(finding.failure_mode.mode_id, finding.step_index, finding.evidence)
+```
+
+The report localizes the responsible upstream step rather than only reporting
+the final visible error.
+
+## Quick Start: CLI
+
+The CLI supports the complete Diagnose -> Rerun workflow. The web console is
+optional and is not required for trace conversion, diagnosis, attribution,
+recovery planning, or rerun preparation.
+
+### 1. Normalize an external trace
+
+AgentDebugX can auto-detect common JSON and JSONL exports:
+
+```bash
+agentdebug ingest raw_trace.json --format auto --out trace.json
+```
+
+Use `--format` when the source is known, for example `messages`,
+`openai_agents_spans`, `crewai_events`, `langgraph_callbacks`, `claude_code`,
+or `osworld`.
+
+Process a directory of independent JSON files or every non-empty line in a
+JSONL dataset:
+
+```bash
+agentdebug batch ingest AgentProcessBench/gaia_dev/test.jsonl \
+  --out-dir data/agentprocessbench/gaia_dev
+```
+
+Batch diagnosis normalizes each record and writes independently rerunnable
+trajectories and reports:
+
+```bash
+agentdebug batch diagnose AgentProcessBench/gaia_dev/test.jsonl \
+  --mode judge \
+  --attributor all-at-once \
+  --recovery self-refine \
+  --out-dir runs/agentprocessbench/gaia_dev
+```
+
+Every batch writes `batch-summary.json`. Invalid records are isolated and do
+not discard successful outputs; a partially failed CLI batch exits with code
+`3`.
+
+### 2. Run a fully local diagnosis
+
+The deterministic pipeline does not require an API key:
+
+```bash
+agentdebug diagnose trace.json \
+  --mode heuristic \
+  --attributor heuristic \
+  --recovery reflexion \
+  --out report.json
+```
+
+Render the same diagnosis as a cascade-oriented traceback:
+
+```bash
+agentdebug diagnose trace.json \
+  --mode heuristic \
+  --attributor heuristic \
+  --recovery reflexion \
+  --traceback
+```
+
+### 3. Enable LLM-backed diagnosis
+
+Save an OpenAI-compatible endpoint once:
+
+```bash
+agentdebug config set-llm \
+  --base-url "https://<openai-compatible-host>/v1" \
+  --api-key "<secret>" \
+  --model "<model>"
+```
+
+Then select the diagnosis, attribution, and recovery implementations
+explicitly:
+
+```bash
+agentdebug diagnose trace.json \
+  --mode judge \
+  --attributor all-at-once \
+  --recovery self-refine \
+  --out report.json
+```
+
+For difficult multi-step or ambiguous failures, DeepDebug runs the complete
+diagnosis workflow and automatically packages its evidence-backed fix as a
+standard retry directive:
+
+```bash
+agentdebug diagnose trace.json \
+  --mode deepdebug \
+  --out report.json
+```
+
+`--recovery deepdebug` can select this packaging explicitly. Existing scripts
+that use `--attributor none --recovery none` remain compatible; explicit
+`--recovery none` disables the standard recovery payload.
+
+Environment variables can be used instead of saved configuration:
+
+```bash
+export AGENTDEBUG_LLM_BASE_URL="https://<openai-compatible-host>/v1"
+export AGENTDEBUG_LLM_API_KEY="<secret>"
+export AGENTDEBUG_LLM_MODEL="<model>"
+```
+
+Use `agentdebug config show` to inspect masked configuration and
+`agentdebug config doctor` to test the configured endpoint.
+
+### 4. Execute the Rerun stage
+
+For repeated, Docker, or remote reruns, keep the application's complete Agent
+environment running as an HTTP runner service. Implement a project callback,
+then start and configure it once:
+
+```bash
+agentdebug runner serve my_project.runner:run_agent \
+  --name my-agent \
+  --framework langgraph \
+  --host 0.0.0.0 \
+  --port 8765 \
+  --token-env MY_RUNNER_TOKEN
+
+agentdebug config set-runner my-agent \
+  --url http://127.0.0.1:8765 \
+  --token-env MY_RUNNER_TOKEN \
+  --default
+
+agentdebug config doctor-runner my-agent
+```
+
+Then run the original agent framework from the beginning of the task:
+
+```bash
+agentdebug rerun report.json \
+  --trajectory trace.json \
+  --out rerun.live.json
+```
+
+The service owns the framework, real model, tools, credentials, environment,
+job lifecycle, and trajectory recorder. A chat-completions URL alone is not an
+Agent environment. Submissions are idempotent, transient failures use bounded
+retries, and unfinished remote jobs are cancelled best-effort. See
+[the runner specification](src/agentdebug/rerun/RUNNER_SPEC.md).
+
+For local scripts and CI, the process compatibility transport remains available:
+
+```bash
+agentdebug rerun report.json \
+  --trajectory trace.json \
+  --runner-command "python path/to/project_rerun_runner.py" \
+  --out rerun.json
+```
+
+Use `--plan-only` for trajectory-only uploads; the plan reports why real
+execution is unavailable and which runtime capabilities are missing.
+
+Export the same request as a pending actor task dataset when another system
+will perform the rollout:
+
+```bash
+agentdebug rerun report.json \
+  --trajectory trace.json \
+  --plan-only \
+  --actor-task-format jsonl \
+  --out rerun-tasks.jsonl
+```
+
+Parquet is also supported with `--actor-task-format parquet` after installing
+`pyarrow`. These rows contain actor inputs and provenance, not responses or
+training labels. See
+[the actor task specification](src/agentdebug/rerun/ACTOR_TASK_SPEC.md).
+
+For prompt experiments only, `--simulate` enables the previous LLM-generated
+trajectory flow. It returns a workflow JSON with `status=simulated`, a validated
+`hypothetical_trajectory`, and model-generated events explicitly marked as
+simulated. It executes no tools and is not evidence that the recovery fixed the
+task. See
+[the simulation specification](src/agentdebug/rerun/SIMULATION_SPEC.md).
+
+### 5. Work with stored traces
+
+The CLI can query SQLite or JSONL stores created by instrumented runs or the
+local console:
+
+```bash
+agentdebug list --store-sqlite .agentdebug/traces.sqlite
+agentdebug show <trace-id> --store-sqlite .agentdebug/traces.sqlite
+agentdebug diagnose <trace-id> \
+  --store-sqlite .agentdebug/traces.sqlite \
+  --mode heuristic \
+  --attributor heuristic \
+  --recovery reflexion
+```
+
+### 6. Package and integrate debugging workflows
+
+Publish a scrubbed failure bundle to a local Error Hub:
+
+```bash
+agentdebug hub push <trace-id> \
+  --store-sqlite .agentdebug/traces.sqlite \
+  --to local:./agentdebug-hub
+```
+
+Generate a debugging skill for a supported host runtime:
+
+```bash
+agentdebug integrations skill --platform claude --target .claude/skills
+```
+
+### Optional: launch the local console
+
+Install the UI extra only when a visual inspection workflow is useful:
+
+```bash
+pip install "agentdebugx[ui]"
+agentdebug serve \
+  --store-sqlite .agentdebug/traces.sqlite \
+  --host 127.0.0.1 \
+  --port 7777
+```
+
+## CLI Reference
+
+| Command | Purpose |
+| --- | --- |
+| `agentdebug ingest` | Normalize an external trace export into AgentDebugX schema |
+| `agentdebug diagnose` | Run detection, attribution, and recovery planning |
+| `agentdebug batch ingest` | Normalize every JSON file or independent JSONL record |
+| `agentdebug batch diagnose` | Normalize and diagnose a JSON/JSONL collection |
+| `agentdebug rerun` | Execute a real framework runner or build a capability-aware plan |
+| `agentdebug runner serve` | Expose an application callback through the live runner protocol |
+| `agentdebug list` / `agentdebug show` | Inspect traces in a local store |
+| `agentdebug config` | Manage and test LLM endpoints and persistent HTTP runners |
+| `agentdebug hub` | Package, scrub, push, and pull Error Hub bundles |
+| `agentdebug integrations` | Generate external runtime integration assets |
+| `agentdebug act` | Compatibility namespace for Hub and integration actions |
+| `agentdebug serve` / `agentdebug inspect` | Launch the optional local web console |
+| `agentdebug doctor` | Report optional dependency and configuration status |
+| `agentdebug analyze` | Compatibility entry point for heuristic diagnosis |
+| `agentdebug convert` | Compatibility alias for `agentdebug ingest` |
+
+Run `agentdebug <command> --help` for version-specific flags.
+
+## Architecture
+
+The repository mirrors the paper-level workflow:
+
+```text
+src/agentdebug/schema/       portable trajectory, event, report, and taxonomy contracts
+src/agentdebug/runtime/      storage, LLM clients, event bus, and plugin registry
+src/agentdebug/ingest/       live capture APIs and offline trace importers
+src/agentdebug/diagnose/     Detect -> Attribute -> Recover pipeline
+src/agentdebug/rerun/        rerun plans, requests, branch comparison, and executors
+src/agentdebug/inspect/      traceback renderer and local inspection UI
+src/agentdebug/hub/          scrubbed failure bundle packaging and backends
+src/agentdebug/integrations/ host skill and runtime integration generators
+cua_debugger/                computer-use / OSWorld GUI root-cause tooling
+examples/                    runnable examples and demo traces
+docs/                        architecture, schema, and project assets
+```
+
+Detailed references:
+
+- [Architecture](docs/ARCHITECTURE.md)
+- [Trace schema](docs/TRACE_SCHEMA.md)
+- [System overview PDF](docs/assets/overview.pdf)
+
+## Component Model
+
+Diagnose components use manifest-backed discovery:
+
+- Detect components and rule packs declare metadata under
+  `src/agentdebug/diagnose/component_manifests/detect/` and
+  `src/agentdebug/diagnose/detect/rules/packs/`.
+- Attribute components declare metadata under
+  `src/agentdebug/diagnose/component_manifests/attribute/`.
+- Recover components declare metadata under
+  `src/agentdebug/diagnose/component_manifests/recover/`.
+
+The shared registry exposes:
+
+```python
+from agentdebug.diagnose import list_components, load_component
+
+for component in list_components():
+    print(component.id, component.stage, component.capabilities)
+```
+
+This keeps the implementation extensible without turning the CLI or UI into
+business-logic containers.
+
+## Local UI
+
+The inspection UI is a local FastAPI application with a no-build HTML/CSS/JS
+frontend. It is intentionally a surface layer:
+
+- routes live in `inspect/ui/routes.py`
+- rendering lives in `inspect/ui/views.py`
+- UI-facing services live in `inspect/ui/services.py`
+- local case and branch stores live in `inspect/ui/branch_store.py`
+- `inspect/ui/server.py` remains a compatibility import path
+
+The UI can inspect traces, save typical error cases, prepare debug
+continuations, and invoke a server-controlled live runner. Set
+`AGENTDEBUG_RUNNER_URL` for the preferred persistent HTTP transport or
+`AGENTDEBUG_RERUN_COMMAND` for process compatibility. UI reruns default to a
+full `from_start` rollout; `from_event` requires runner capability plus the
+explicit server policy `AGENTDEBUG_UI_RERUN_POLICY=from_event`. The browser does
+not accept or persist runner commands, bearer tokens, or model API keys.
+
+## Privacy and Safety
+
+AgentDebugX is local-first:
+
+- Trace capture and diagnosis run locally unless you explicitly configure an
+  external LLM endpoint.
+- Error Hub publishing is opt-in.
+- Bundle scrubbing is available before sharing traces.
+- Recovery is suggest-only. External execution belongs to Rerun and requires an
+  explicitly configured executor. Recovery approval fields are auditable
+  metadata; deployments must enforce their own authorization policy before
+  dispatch.
+
+Diagnostic findings are hypotheses with evidence and provenance, not ground
+truth. LLM Judge reports retain the model's self-reported confidence;
+Heuristic and DeepDebug reports omit uncalibrated confidence values. Configure
+retention, access control, and redaction before collecting production traces.
+
+## Examples
+
+The `examples/` directory contains runnable scripts and demo artifacts:
+
+- `basic_usage.py`
+- `multi_agent_cascade.py`
+- `langgraph/`
+- `crewai/`
+- `autogen_roundrobin_deepdebug.py`
+- `taxonomy_induction_demo.py`
+- `http_agent_runner.py`
+- `live_rerun_runner.py`
+- `claude_skill_integration/`
+- `debug_skills/`
+
+## Development
+
+Run the test suite:
+
+```bash
+python -m pytest tests -q
+```
+
+Run the enforced branch-coverage baseline:
+
+```bash
+python -m pytest tests -q --cov=agentdebug --cov-branch --cov-fail-under=40
+```
+
+Compile-check the package:
+
+```bash
+python -m compileall -q src/agentdebug tests
+```
+
+Build artifacts under `dist/` should not be committed. Generate them only for
+release workflows.
+
+See [CONTRIBUTING.md](CONTRIBUTING.md) for test organization, optional CUA
+tests, quality checks, and pull request expectations.
+
+## License
+
+MIT. See [LICENSE](LICENSE).
+