agent-forensics 0.1.0__tar.gz

agent_forensics-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 ilya
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

agent_forensics-0.1.0/PKG-INFO

@@ -0,0 +1,186 @@
+Metadata-Version: 2.4
+Name: agent-forensics
+Version: 0.1.0
+Summary: Black box for AI agents — capture decisions, generate forensic reports for EU AI Act compliance
+Author: ilya
+License: MIT
+Project-URL: Homepage, https://github.com/ilya/agent-forensics
+Keywords: ai,agent,forensics,compliance,eu-ai-act,observability
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Topic :: Software Development :: Libraries
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: langchain
+Requires-Dist: langchain-core>=0.3; extra == "langchain"
+Provides-Extra: openai-agents
+Requires-Dist: openai-agents>=0.1; extra == "openai-agents"
+Provides-Extra: crewai
+Requires-Dist: crewai>=0.80; extra == "crewai"
+Provides-Extra: pdf
+Requires-Dist: fpdf2>=2.8; extra == "pdf"
+Provides-Extra: all
+Requires-Dist: langchain-core>=0.3; extra == "all"
+Requires-Dist: openai-agents>=0.1; extra == "all"
+Requires-Dist: crewai>=0.80; extra == "all"
+Requires-Dist: fpdf2>=2.8; extra == "all"
+Dynamic: license-file
+
+# Agent Forensics
+
+**Black box for AI agents.** Capture every decision, generate forensic reports for EU AI Act compliance.
+
+When an AI agent makes a wrong purchase, leaks data, or fails silently — you need to know **why**. Agent Forensics records every decision point, tool call, and LLM interaction, then reconstructs the causal chain when things go wrong.
+
+![Agent Forensics Dashboard — Incident Session](content/screenshots/02-incident-overview.png)
+
+## Why
+
+- **EU AI Act** (Aug 2026): High-risk AI systems must provide decision traceability. Fines up to €35M or 7% of global revenue.
+- **AI agents are already causing incidents**: Meta Sev-1 data leak (Mar 2026), unauthorized purchases, fabricated customer responses.
+- **No existing tool** reconstructs the *why* behind agent failures. Monitoring tools watch in real-time. Forensics analyzes after the fact.
+
+## Install
+
+### From PyPI
+
+```bash
+pip install agent-forensics              # Core only (manual recording)
+pip install agent-forensics[langchain]   # + LangChain integration
+pip install agent-forensics[openai-agents]  # + OpenAI Agents SDK
+pip install agent-forensics[crewai]      # + CrewAI
+pip install agent-forensics[all]         # Everything
+```
+
+### From Source
+
+```bash
+git clone https://github.com/ilflow4592/agent-forensics.git
+cd agent-forensics
+pip install -e ".[all]"
+```
+
+## Quick Start
+
+### Manual Recording (Framework-Agnostic)
+
+```python
+from agent_forensics import Forensics
+
+f = Forensics(session="order-123", agent="shopping-agent")
+
+f.decision("search_products", input={"query": "mouse"}, reasoning="User requested product search")
+f.tool_call("search_api", input={"q": "mouse"}, output={"results": [...]})
+f.error("purchase_failed", output={"reason": "Out of stock"})
+f.finish("Could not complete purchase due to stock unavailability.")
+
+print(f.report())       # Markdown forensic report
+f.save_pdf()            # PDF report
+f.dashboard(port=8080)  # Web dashboard at http://localhost:8080
+```
+
+### LangChain — One Line
+
+```python
+from agent_forensics import Forensics
+
+f = Forensics(session="order-123")
+agent.invoke({"input": "..."}, config={"callbacks": [f.langchain()]})
+```
+
+### OpenAI Agents SDK — One Line
+
+```python
+from agent_forensics import Forensics
+from agents import Agent, Runner
+
+f = Forensics(session="order-123")
+agent = Agent(name="shopper", tools=[...], hooks=f.openai_agents())
+result = await Runner.run(agent, "Buy a wireless mouse")
+```
+
+### CrewAI — Two Lines
+
+```python
+from agent_forensics import Forensics
+
+f = Forensics(session="order-123")
+hooks = f.crewai()
+agent = Agent(role="shopper", step_callback=hooks.step_callback)
+task = Task(description="...", agent=agent, callback=hooks.task_callback)
+```
+
+## What You Get
+
+### Forensic Report
+
+Every report includes:
+
+- **Timeline** — Chronological record of all agent actions
+- **Decision Chain** — Each decision with its reasoning
+- **Incident Analysis** — Automatic error detection + root cause chain
+- **Causal Chain** — `Decision → Tool Call → Result → Error` trace
+- **Compliance Notes** — EU AI Act Article 14 (Human Oversight) support
+
+### Web Dashboard
+
+```bash
+python -c "from agent_forensics import Forensics; Forensics(db_path='forensics.db').dashboard()"
+```
+
+Dark-themed dashboard with session selector, color-coded timeline, and causal chain visualization.
+
+![Causal Chain — Root Cause Analysis](content/screenshots/04-causal-chain.png)
+
+![Data Leak Incident](content/screenshots/05-data-leak.png)
+
+### Output Formats
+
+- **Markdown** — `f.save_markdown()`
+- **PDF** — `f.save_pdf()` (requires `pip install agent-forensics[pdf]`)
+- **Web Dashboard** — `f.dashboard(port=8080)`
+- **Raw Events** — `f.events()` returns `list[Event]`
+
+## Architecture
+
+```
+Your Agent (any framework)
+    │
+    │  Callback / Hook (1 line of code)
+    ▼
+┌──────────────────────┐
+│  Forensics Collector  │  Captures every LLM call, tool use, decision
+├──────────────────────┤
+│  Event Store (SQLite) │  Immutable event log
+├──────────────────────┤
+│  Report Generator     │  Markdown / PDF / Dashboard
+└──────────────────────┘
+```
+
+## Supported Frameworks
+
+| Framework | Integration | Method |
+|-----------|------------|--------|
+| **Any** (manual) | `f.decision()`, `f.tool_call()`, `f.error()` | Direct API |
+| **LangChain / LangGraph** | `f.langchain()` | Callback Handler |
+| **OpenAI Agents SDK** | `f.openai_agents()` | AgentHooks |
+| **CrewAI** | `f.crewai()` | step_callback / task_callback |
+
+## Event Types
+
+| Type | When | Why It Matters |
+|------|------|---------------|
+| `decision` | Agent decides what to do next | Core of forensics — the *why* |
+| `tool_call_start` | Tool execution begins | What tool, what input |
+| `tool_call_end` | Tool returns result | What came back |
+| `llm_call_start` | LLM request sent | What was asked |
+| `llm_call_end` | LLM response received | What was answered |
+| `error` | Something went wrong | Incident detection |
+| `final_decision` | Agent produces final output | End of decision chain |
+
+## License
+
+MIT

agent_forensics-0.1.0/README.md

@@ -0,0 +1,155 @@
+# Agent Forensics
+
+**Black box for AI agents.** Capture every decision, generate forensic reports for EU AI Act compliance.
+
+When an AI agent makes a wrong purchase, leaks data, or fails silently — you need to know **why**. Agent Forensics records every decision point, tool call, and LLM interaction, then reconstructs the causal chain when things go wrong.
+
+![Agent Forensics Dashboard — Incident Session](content/screenshots/02-incident-overview.png)
+
+## Why
+
+- **EU AI Act** (Aug 2026): High-risk AI systems must provide decision traceability. Fines up to €35M or 7% of global revenue.
+- **AI agents are already causing incidents**: Meta Sev-1 data leak (Mar 2026), unauthorized purchases, fabricated customer responses.
+- **No existing tool** reconstructs the *why* behind agent failures. Monitoring tools watch in real-time. Forensics analyzes after the fact.
+
+## Install
+
+### From PyPI
+
+```bash
+pip install agent-forensics              # Core only (manual recording)
+pip install agent-forensics[langchain]   # + LangChain integration
+pip install agent-forensics[openai-agents]  # + OpenAI Agents SDK
+pip install agent-forensics[crewai]      # + CrewAI
+pip install agent-forensics[all]         # Everything
+```
+
+### From Source
+
+```bash
+git clone https://github.com/ilflow4592/agent-forensics.git
+cd agent-forensics
+pip install -e ".[all]"
+```
+
+## Quick Start
+
+### Manual Recording (Framework-Agnostic)
+
+```python
+from agent_forensics import Forensics
+
+f = Forensics(session="order-123", agent="shopping-agent")
+
+f.decision("search_products", input={"query": "mouse"}, reasoning="User requested product search")
+f.tool_call("search_api", input={"q": "mouse"}, output={"results": [...]})
+f.error("purchase_failed", output={"reason": "Out of stock"})
+f.finish("Could not complete purchase due to stock unavailability.")
+
+print(f.report())       # Markdown forensic report
+f.save_pdf()            # PDF report
+f.dashboard(port=8080)  # Web dashboard at http://localhost:8080
+```
+
+### LangChain — One Line
+
+```python
+from agent_forensics import Forensics
+
+f = Forensics(session="order-123")
+agent.invoke({"input": "..."}, config={"callbacks": [f.langchain()]})
+```
+
+### OpenAI Agents SDK — One Line
+
+```python
+from agent_forensics import Forensics
+from agents import Agent, Runner
+
+f = Forensics(session="order-123")
+agent = Agent(name="shopper", tools=[...], hooks=f.openai_agents())
+result = await Runner.run(agent, "Buy a wireless mouse")
+```
+
+### CrewAI — Two Lines
+
+```python
+from agent_forensics import Forensics
+
+f = Forensics(session="order-123")
+hooks = f.crewai()
+agent = Agent(role="shopper", step_callback=hooks.step_callback)
+task = Task(description="...", agent=agent, callback=hooks.task_callback)
+```
+
+## What You Get
+
+### Forensic Report
+
+Every report includes:
+
+- **Timeline** — Chronological record of all agent actions
+- **Decision Chain** — Each decision with its reasoning
+- **Incident Analysis** — Automatic error detection + root cause chain
+- **Causal Chain** — `Decision → Tool Call → Result → Error` trace
+- **Compliance Notes** — EU AI Act Article 14 (Human Oversight) support
+
+### Web Dashboard
+
+```bash
+python -c "from agent_forensics import Forensics; Forensics(db_path='forensics.db').dashboard()"
+```
+
+Dark-themed dashboard with session selector, color-coded timeline, and causal chain visualization.
+
+![Causal Chain — Root Cause Analysis](content/screenshots/04-causal-chain.png)
+
+![Data Leak Incident](content/screenshots/05-data-leak.png)
+
+### Output Formats
+
+- **Markdown** — `f.save_markdown()`
+- **PDF** — `f.save_pdf()` (requires `pip install agent-forensics[pdf]`)
+- **Web Dashboard** — `f.dashboard(port=8080)`
+- **Raw Events** — `f.events()` returns `list[Event]`
+
+## Architecture
+
+```
+Your Agent (any framework)
+    │
+    │  Callback / Hook (1 line of code)
+    ▼
+┌──────────────────────┐
+│  Forensics Collector  │  Captures every LLM call, tool use, decision
+├──────────────────────┤
+│  Event Store (SQLite) │  Immutable event log
+├──────────────────────┤
+│  Report Generator     │  Markdown / PDF / Dashboard
+└──────────────────────┘
+```
+
+## Supported Frameworks
+
+| Framework | Integration | Method |
+|-----------|------------|--------|
+| **Any** (manual) | `f.decision()`, `f.tool_call()`, `f.error()` | Direct API |
+| **LangChain / LangGraph** | `f.langchain()` | Callback Handler |
+| **OpenAI Agents SDK** | `f.openai_agents()` | AgentHooks |
+| **CrewAI** | `f.crewai()` | step_callback / task_callback |
+
+## Event Types
+
+| Type | When | Why It Matters |
+|------|------|---------------|
+| `decision` | Agent decides what to do next | Core of forensics — the *why* |
+| `tool_call_start` | Tool execution begins | What tool, what input |
+| `tool_call_end` | Tool returns result | What came back |
+| `llm_call_start` | LLM request sent | What was asked |
+| `llm_call_end` | LLM response received | What was answered |
+| `error` | Something went wrong | Incident detection |
+| `final_decision` | Agent produces final output | End of decision chain |
+
+## License
+
+MIT

agent_forensics-0.1.0/agent_forensics/__init__.py

@@ -0,0 +1,38 @@
+"""
+Agent Forensics — Black box for AI agents.
+
+Usage:
+    from agent_forensics import Forensics
+
+    # 1. Initialize
+    f = Forensics(session="order-123", agent="shopping-agent")
+
+    # 2a. Manual recording (framework-agnostic)
+    f.decision("search_products", input={"query": "mouse"}, reasoning="User request")
+    f.tool_call("search_api", input={"q": "mouse"}, output={"results": [...]})
+    f.error("purchase_failed", output={"reason": "Out of stock"})
+
+    # 2b. LangChain auto-recording
+    agent.invoke(..., config={"callbacks": [f.langchain()]})
+
+    # 2c. OpenAI Agents SDK auto-recording
+    agent = Agent(name="...", hooks=f.openai_agents())
+
+    # 2d. CrewAI auto-recording
+    hooks = f.crewai()
+    agent = Agent(role="...", step_callback=hooks.step_callback)
+
+    # 3. Report
+    print(f.report())       # Markdown
+    f.save_markdown()       # forensics-report-order-123.md
+    f.save_pdf()            # forensics-report-order-123.pdf
+
+    # 4. Dashboard
+    f.dashboard(port=8080)  # http://localhost:8080
+"""
+
+from .core import Forensics
+from .store import Event, EventStore
+
+__version__ = "0.1.0"
+__all__ = ["Forensics", "Event", "EventStore"]

agent_forensics-0.1.0/agent_forensics/core.py

@@ -0,0 +1,183 @@
+"""
+Forensics — Main interface for AI agent forensics.
+
+Provides all functionality through a single class.
+Framework-agnostic, with integrations for LangChain/CrewAI/OpenAI and more.
+"""
+
+from .store import EventStore, Event, now
+from .report import generate_report, save_report, save_pdf
+
+
+class Forensics:
+    """AI Agent Forensics — Black box + report generator."""
+
+    def __init__(
+        self,
+        session: str = "default",
+        agent: str = "default-agent",
+        db_path: str = "forensics.db",
+    ):
+        self.session = session
+        self.agent = agent
+        self.store = EventStore(db_path)
+
+    # -- Manual Recording API --
+
+    def decision(self, action: str, *, input: dict = None, reasoning: str = "") -> str:
+        """Record when the agent makes a decision."""
+        return self.store.save(Event(
+            timestamp=now(),
+            event_type="decision",
+            agent_id=self.agent,
+            action=action,
+            input_data=input or {},
+            output_data={},
+            reasoning=reasoning,
+            session_id=self.session,
+        ))
+
+    def tool_call(self, action: str, *, input: dict = None, output: dict = None, reasoning: str = "") -> str:
+        """Record a tool call."""
+        # start
+        self.store.save(Event(
+            timestamp=now(),
+            event_type="tool_call_start",
+            agent_id=self.agent,
+            action=f"tool:{action}",
+            input_data=input or {},
+            output_data={},
+            reasoning=reasoning or f"Calling tool: {action}",
+            session_id=self.session,
+        ))
+        # end
+        return self.store.save(Event(
+            timestamp=now(),
+            event_type="tool_call_end",
+            agent_id=self.agent,
+            action="tool_result",
+            input_data={},
+            output_data=output or {},
+            reasoning="Tool execution completed",
+            session_id=self.session,
+        ))
+
+    def llm_call(self, *, input: dict = None, output: str = "", reasoning: str = "") -> str:
+        """Record an LLM call."""
+        self.store.save(Event(
+            timestamp=now(),
+            event_type="llm_call_start",
+            agent_id=self.agent,
+            action="llm_call",
+            input_data=input or {},
+            output_data={},
+            reasoning=reasoning or "LLM call",
+            session_id=self.session,
+        ))
+        return self.store.save(Event(
+            timestamp=now(),
+            event_type="llm_call_end",
+            agent_id=self.agent,
+            action="llm_response",
+            input_data={},
+            output_data={"response": output},
+            reasoning="LLM response",
+            session_id=self.session,
+        ))
+
+    def error(self, action: str, *, output: dict = None, reasoning: str = "") -> str:
+        """Record an error/incident."""
+        return self.store.save(Event(
+            timestamp=now(),
+            event_type="error",
+            agent_id=self.agent,
+            action=action,
+            input_data={},
+            output_data=output or {},
+            reasoning=reasoning or f"Error occurred: {action}",
+            session_id=self.session,
+        ))
+
+    def finish(self, output: str = "", *, reasoning: str = "") -> str:
+        """Record the agent's final result."""
+        return self.store.save(Event(
+            timestamp=now(),
+            event_type="final_decision",
+            agent_id=self.agent,
+            action="agent_finish",
+            input_data={},
+            output_data={"response": output},
+            reasoning=reasoning or "Agent determined final answer",
+            session_id=self.session,
+        ))
+
+    def record(self, event_type: str, action: str, *, input: dict = None, output: dict = None, reasoning: str = "") -> str:
+        """Record a generic event."""
+        return self.store.save(Event(
+            timestamp=now(),
+            event_type=event_type,
+            agent_id=self.agent,
+            action=action,
+            input_data=input or {},
+            output_data=output or {},
+            reasoning=reasoning,
+            session_id=self.session,
+        ))
+
+    # -- Report API --
+
+    def report(self) -> str:
+        """Return the Markdown forensics report as a string."""
+        return generate_report(self.store, self.session)
+
+    def save_markdown(self, path: str = None) -> str:
+        """Save the Markdown report to a file."""
+        return save_report(self.store, self.session, output_dir=path or ".")
+
+    def save_pdf(self, path: str = None) -> str:
+        """Save the PDF report to a file."""
+        return save_pdf(self.store, self.session, output_dir=path or ".")
+
+    def events(self) -> list[Event]:
+        """Return all events for the current session."""
+        return self.store.get_session_events(self.session)
+
+    def sessions(self) -> list[str]:
+        """Return a list of all sessions."""
+        return self.store.get_all_sessions()
+
+    # -- Framework Integrations --
+
+    def langchain(self):
+        """Return a LangChain callback handler. agent.invoke(..., config={"callbacks": [f.langchain()]})"""
+        from .integrations.langchain import ForensicsCollector
+        return ForensicsCollector(
+            store=self.store,
+            session_id=self.session,
+            agent_id=self.agent,
+        )
+
+    def openai_agents(self):
+        """Return OpenAI Agents SDK hooks. Agent(hooks=f.openai_agents())"""
+        from .integrations.openai_agents import ForensicsAgentHooks
+        return ForensicsAgentHooks(
+            store=self.store,
+            session_id=self.session,
+            agent_id=self.agent,
+        )
+
+    def crewai(self):
+        """Return CrewAI callback collection. Agent(step_callback=hooks.step_callback)"""
+        from .integrations.crewai import ForensicsCrewAIHooks
+        return ForensicsCrewAIHooks(
+            store=self.store,
+            session_id=self.session,
+            agent_id=self.agent,
+        )
+
+    # -- Dashboard --
+
+    def dashboard(self, port: int = 8080):
+        """Launch the web dashboard."""
+        from .dashboard import run_dashboard
+        run_dashboard(self.store, port=port)