flightrecorder-ai 1.0.0__tar.gz

flightrecorder_ai-1.0.0/LICENSE

@@ -0,0 +1,18 @@
+Apache License
+Version 2.0, January 2004
+http://www.apache.org/licenses/
+
+Copyright 2026 Mohamed Ansar
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+

flightrecorder_ai-1.0.0/PKG-INFO

@@ -0,0 +1,152 @@
+Metadata-Version: 2.4
+Name: flightrecorder-ai
+Version: 1.0.0
+Summary: A local black-box recorder and failure investigator for AI agents
+Home-page: https://github.com/kmsmohamedansar/agent-flight-recorder
+Author: Mohamed Ansar
+License: Apache-2.0
+Project-URL: Source, https://github.com/kmsmohamedansar/agent-flight-recorder
+Project-URL: Issues, https://github.com/kmsmohamedansar/agent-flight-recorder/issues
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Software Development :: Debuggers
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Dynamic: author
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: license
+Dynamic: license-file
+Dynamic: project-url
+Dynamic: requires-python
+Dynamic: summary
+
+# Agent Flight Recorder
+
+A local black-box recorder for AI agents. It captures tool calls, results,
+exceptions, file changes and Git context in SQLite, then connects failures to the
+recorded evidence that most likely caused them.
+
+The runtime has no third-party dependencies and supports Python 3.9+.
+
+The PyPI distribution is named `flightrecorder-ai`; the Python import and command
+remain `flightrecorder`.
+
+## Quick start
+
+```bash
+python3 -m venv .venv
+. .venv/bin/activate
+python -m pip install flightrecorder-ai
+
+# Synthetic example from the product brief
+flightrecorder demo
+flightrecorder investigate run-1042
+
+# Real files, wrapped tools and a subprocess test
+flightrecorder demo-agent
+
+# Local searchable timeline
+flightrecorder serve
+```
+
+Open <http://127.0.0.1:8765> after starting the server. The project supports the
+older pip bundled with some system Python installations; upgrading pip is optional.
+
+## Instrument an agent
+
+Wrap existing synchronous or asynchronous tools:
+
+```python
+from flightrecorder import FlightRecorder
+
+recorder = FlightRecorder(".flightrecorder.sqlite")
+
+with recorder.run("Fix payment bug", metadata={"workspace": "."}) as run:
+    search = run.wrap_tool(search_code)
+    edit = run.wrap_tool(edit_file, watch_paths=["src"])
+    test = run.wrap_tool(run_tests, kind="test")
+
+    matches = search("payment validation")
+    edit("src/payment.py", replacement)
+    test("tests/test_payment.py")
+```
+
+Arguments, return values and exceptions are captured automatically. Watched text
+files are snapshotted before and after a call. Exceptions are recorded and then
+re-raised, so instrumentation does not change control flow.
+
+For lower-level integrations, `run.step(...)` and `run.record(...)` are also
+available. `ReplayTape` feeds recorded outputs back to deterministic tests without
+repeating side effects.
+
+## Commands
+
+| Command | Purpose |
+| --- | --- |
+| `init` | Initialize the SQLite database |
+| `demo` | Record the synthetic payment failure |
+| `demo-agent` | Execute a real file-edit and subprocess-test workflow |
+| `runs` | List recent recordings |
+| `show [run-id]` | Display a timeline |
+| `investigate [run-id]` | Rank the first suspect change and show its evidence |
+| `replay [run-id]` | Display the side-effect-free replay tape |
+| `compare LEFT RIGHT` | Find the first meaningful divergence between runs |
+| `export [run-id]` | Export native JSON or OTLP/JSON spans |
+| `serve` | Start the local searchable timeline UI |
+| `prune` | Preview or apply the retention policy |
+
+All commands accept `--db PATH` before the command name. Commands accepting an
+optional run ID use the latest run when it is omitted.
+
+```bash
+flightrecorder compare run-yesterday run-today --json
+flightrecorder export run-today --format otlp --output trace.json
+flightrecorder prune --older-than-days 30 --keep-latest 100
+flightrecorder prune --older-than-days 30 --keep-latest 100 --yes
+```
+
+Retention is always a dry-run unless `--yes` is supplied.
+
+## Investigation model
+
+The investigator is deliberately explainable. It finds the first failed step,
+ranks earlier file changes using explicit implicated paths, filenames mentioned in
+the failure, removed-code vocabulary and temporal proximity, then returns its diff
+and evidence. It reports a suspect—not unsupported certainty.
+
+Exact model reproduction is not promised. Replay returns previously recorded tool
+outputs and errors without calling live services; nondeterministic model behavior
+and changed external systems remain outside that guarantee.
+
+## Security and privacy
+
+Common credential keys, inline passwords and bearer tokens are redacted before
+SQLite persistence, including exception text and captured file content. Git and
+file capture are bounded to one megabyte per value; binary files and common build
+directories are skipped.
+
+No heuristic filter is perfect. Test the redaction policy for your environment
+before recording production or personal data. The web server binds to
+`127.0.0.1` by default and has no authentication; do not expose it publicly.
+
+See [SECURITY.md](SECURITY.md) for responsible disclosure and deployment notes.
+
+## Development
+
+```bash
+python -m unittest discover -s tests -v
+python -m flightrecorder demo-agent
+```
+
+CI runs the suite on Python 3.9, 3.12 and 3.13. Architecture and integration
+boundaries are described in [ARCHITECTURE.md](ARCHITECTURE.md).
+

flightrecorder_ai-1.0.0/README.md

@@ -0,0 +1,121 @@
+# Agent Flight Recorder
+
+A local black-box recorder for AI agents. It captures tool calls, results,
+exceptions, file changes and Git context in SQLite, then connects failures to the
+recorded evidence that most likely caused them.
+
+The runtime has no third-party dependencies and supports Python 3.9+.
+
+The PyPI distribution is named `flightrecorder-ai`; the Python import and command
+remain `flightrecorder`.
+
+## Quick start
+
+```bash
+python3 -m venv .venv
+. .venv/bin/activate
+python -m pip install flightrecorder-ai
+
+# Synthetic example from the product brief
+flightrecorder demo
+flightrecorder investigate run-1042
+
+# Real files, wrapped tools and a subprocess test
+flightrecorder demo-agent
+
+# Local searchable timeline
+flightrecorder serve
+```
+
+Open <http://127.0.0.1:8765> after starting the server. The project supports the
+older pip bundled with some system Python installations; upgrading pip is optional.
+
+## Instrument an agent
+
+Wrap existing synchronous or asynchronous tools:
+
+```python
+from flightrecorder import FlightRecorder
+
+recorder = FlightRecorder(".flightrecorder.sqlite")
+
+with recorder.run("Fix payment bug", metadata={"workspace": "."}) as run:
+    search = run.wrap_tool(search_code)
+    edit = run.wrap_tool(edit_file, watch_paths=["src"])
+    test = run.wrap_tool(run_tests, kind="test")
+
+    matches = search("payment validation")
+    edit("src/payment.py", replacement)
+    test("tests/test_payment.py")
+```
+
+Arguments, return values and exceptions are captured automatically. Watched text
+files are snapshotted before and after a call. Exceptions are recorded and then
+re-raised, so instrumentation does not change control flow.
+
+For lower-level integrations, `run.step(...)` and `run.record(...)` are also
+available. `ReplayTape` feeds recorded outputs back to deterministic tests without
+repeating side effects.
+
+## Commands
+
+| Command | Purpose |
+| --- | --- |
+| `init` | Initialize the SQLite database |
+| `demo` | Record the synthetic payment failure |
+| `demo-agent` | Execute a real file-edit and subprocess-test workflow |
+| `runs` | List recent recordings |
+| `show [run-id]` | Display a timeline |
+| `investigate [run-id]` | Rank the first suspect change and show its evidence |
+| `replay [run-id]` | Display the side-effect-free replay tape |
+| `compare LEFT RIGHT` | Find the first meaningful divergence between runs |
+| `export [run-id]` | Export native JSON or OTLP/JSON spans |
+| `serve` | Start the local searchable timeline UI |
+| `prune` | Preview or apply the retention policy |
+
+All commands accept `--db PATH` before the command name. Commands accepting an
+optional run ID use the latest run when it is omitted.
+
+```bash
+flightrecorder compare run-yesterday run-today --json
+flightrecorder export run-today --format otlp --output trace.json
+flightrecorder prune --older-than-days 30 --keep-latest 100
+flightrecorder prune --older-than-days 30 --keep-latest 100 --yes
+```
+
+Retention is always a dry-run unless `--yes` is supplied.
+
+## Investigation model
+
+The investigator is deliberately explainable. It finds the first failed step,
+ranks earlier file changes using explicit implicated paths, filenames mentioned in
+the failure, removed-code vocabulary and temporal proximity, then returns its diff
+and evidence. It reports a suspect—not unsupported certainty.
+
+Exact model reproduction is not promised. Replay returns previously recorded tool
+outputs and errors without calling live services; nondeterministic model behavior
+and changed external systems remain outside that guarantee.
+
+## Security and privacy
+
+Common credential keys, inline passwords and bearer tokens are redacted before
+SQLite persistence, including exception text and captured file content. Git and
+file capture are bounded to one megabyte per value; binary files and common build
+directories are skipped.
+
+No heuristic filter is perfect. Test the redaction policy for your environment
+before recording production or personal data. The web server binds to
+`127.0.0.1` by default and has no authentication; do not expose it publicly.
+
+See [SECURITY.md](SECURITY.md) for responsible disclosure and deployment notes.
+
+## Development
+
+```bash
+python -m unittest discover -s tests -v
+python -m flightrecorder demo-agent
+```
+
+CI runs the suite on Python 3.9, 3.12 and 3.13. Architecture and integration
+boundaries are described in [ARCHITECTURE.md](ARCHITECTURE.md).
+

flightrecorder_ai-1.0.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

flightrecorder_ai-1.0.0/setup.py

@@ -0,0 +1,35 @@
+"""Compatibility installer for older pip versions without PEP 660 support."""
+
+from setuptools import find_packages, setup
+
+
+setup(
+    # The shorter agent-flight-recorder distribution name is owned by an
+    # unrelated project on PyPI. The import and CLI names remain flightrecorder.
+    name="flightrecorder-ai",
+    version="1.0.0",
+    description="A local black-box recorder and failure investigator for AI agents",
+    long_description=open("README.md", encoding="utf-8").read(),
+    long_description_content_type="text/markdown",
+    author="Mohamed Ansar",
+    license="Apache-2.0",
+    url="https://github.com/kmsmohamedansar/agent-flight-recorder",
+    project_urls={
+        "Source": "https://github.com/kmsmohamedansar/agent-flight-recorder",
+        "Issues": "https://github.com/kmsmohamedansar/agent-flight-recorder/issues",
+    },
+    python_requires=">=3.9",
+    classifiers=[
+        "Development Status :: 4 - Beta",
+        "Intended Audience :: Developers",
+        "Operating System :: OS Independent",
+        "Programming Language :: Python :: 3",
+        "Programming Language :: Python :: 3.9",
+        "Programming Language :: Python :: 3.12",
+        "Programming Language :: Python :: 3.13",
+        "Topic :: Software Development :: Debuggers",
+    ],
+    package_dir={"": "src"},
+    packages=find_packages(where="src"),
+    entry_points={"console_scripts": ["flightrecorder=flightrecorder.cli:main"]},
+)

flightrecorder_ai-1.0.0/src/flightrecorder/__init__.py

@@ -0,0 +1,15 @@
+"""Agent Flight Recorder public API."""
+
+from .recorder import FlightRecorder, RunSession, StepCapture
+from .replay import RecordedToolError, ReplayMismatch, ReplayTape
+
+__all__ = [
+    "FlightRecorder",
+    "RecordedToolError",
+    "ReplayMismatch",
+    "ReplayTape",
+    "RunSession",
+    "StepCapture",
+]
+__version__ = "1.0.0"
+

flightrecorder_ai-1.0.0/src/flightrecorder/__main__.py

@@ -0,0 +1,6 @@
+from .cli import main
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())
+

flightrecorder_ai-1.0.0/src/flightrecorder/cli.py

@@ -0,0 +1,244 @@
+"""Command-line interface."""
+
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+import uuid
+from datetime import datetime, timedelta, timezone
+from pathlib import Path
+from typing import Any, Dict, Optional, Sequence
+
+from .compare import compare_runs
+from .demo_agent import run_demo as run_agent_demo
+from .export import to_json, to_otlp
+from .investigate import investigate
+from .recorder import FlightRecorder
+from .store import SQLiteStore
+
+
+DEFAULT_DB = ".flightrecorder.sqlite"
+
+
+def _load(store: SQLiteStore, run_id: Optional[str]) -> Dict[str, Any]:
+    run = store.get_run(run_id)
+    if run is None:
+        target = run_id or "latest run"
+        raise SystemExit("No recording found for %s." % target)
+    return run
+
+
+def _demo(recorder: FlightRecorder) -> str:
+    before = """def charge(order):
+    if order.already_paid:
+        raise DuplicatePayment(order.id)
+    return gateway.charge(order)
+"""
+    after = """def charge(order):
+    return gateway.charge(order)
+"""
+    preferred_id = "run-1042"
+    run_id = (
+        preferred_id
+        if recorder.store.get_run(preferred_id) is None
+        else "run-demo-" + uuid.uuid4().hex[:8]
+    )
+    with recorder.run("Payment bug fix", run_id=run_id) as run:
+        run.record(
+            "Read payment_service.py",
+            kind="file_read",
+            input={"path": "payment_service.py"},
+            output={"bytes": len(before)},
+        )
+        with run.step(
+            "Modified payment_service.py",
+            kind="file_write",
+            input={"path": "payment_service.py"},
+        ) as step:
+            step.add_change("payment_service.py", before, after)
+            step.set_output({"written": True})
+        run.record(
+            "Updated database query",
+            kind="database",
+            input={"operation": "share transaction"},
+            output={"rows": 2},
+        )
+        run.record(
+            "Ran unit tests",
+            kind="test",
+            input={"suite": "unit"},
+            output={"passed": 42, "failed": 0},
+        )
+        run.record(
+            "Ran duplicate-payment regression test",
+            kind="test",
+            input={"test": "test_duplicate_payment_rejected"},
+            output={"passed": 0, "failed": 1},
+            status="failed",
+            error="AssertionError: duplicate payment was accepted",
+            metadata={"implicated_paths": ["payment_service.py"]},
+        )
+        run.finish("failed")
+    return run.id
+
+
+def _print_timeline(run: Dict[str, Any]) -> None:
+    print("%s  %s  [%s]" % (run["id"], run["name"], run["status"]))
+    for step in run["steps"]:
+        marker = "x" if step["status"] == "failed" else "+"
+        print("  %s #%d %-12s %s" % (marker, step["sequence"], step["kind"], step["name"]))
+        if step["error"]:
+            print("      %s" % step["error"])
+
+
+def _print_investigation(run: Dict[str, Any]) -> None:
+    result = investigate(run)
+    print("Run: %s — %s" % (run["id"], run["name"]))
+    if result["failure"] is None:
+        print("Failure: none")
+        return
+    failure = result["failure"]
+    print("Failure: %s" % (failure["error"] or failure["name"]))
+    suspect = result.get("suspect")
+    if suspect is None:
+        print("First suspect: unavailable")
+    else:
+        print("First suspect: #%d %s" % (suspect["sequence"], suspect["name"]))
+        print("Changed: %s" % result["change"]["path"])
+        print("Evidence:")
+        for line in result["evidence"]:
+            print("  - %s" % line)
+        if result["diff"]:
+            print("Relevant diff:")
+            for line in result["diff"]:
+                print("  %s" % line)
+
+
+def build_parser() -> argparse.ArgumentParser:
+    parser = argparse.ArgumentParser(prog="flightrecorder")
+    parser.add_argument("--db", default=DEFAULT_DB, help="SQLite recording path")
+    commands = parser.add_subparsers(dest="command", required=True)
+    commands.add_parser("init", help="initialize the recording database")
+    commands.add_parser("demo", help="record the built-in failing-agent demo")
+    agent_demo = commands.add_parser(
+        "demo-agent", help="run a real file-edit and subprocess-test workflow"
+    )
+    agent_demo.add_argument("--workspace", default=".flightrecorder-demo")
+    runs = commands.add_parser("runs", help="list recent runs")
+    runs.add_argument("--limit", type=int, default=20)
+    show = commands.add_parser("show", help="show a run timeline")
+    show.add_argument("run_id", nargs="?")
+    inv = commands.add_parser("investigate", help="find the first suspect change")
+    inv.add_argument("run_id", nargs="?")
+    replay = commands.add_parser("replay", help="replay recorded outputs without side effects")
+    replay.add_argument("run_id", nargs="?")
+    replay.add_argument("--json", action="store_true")
+    compare = commands.add_parser("compare", help="compare two recorded runs")
+    compare.add_argument("left_run_id")
+    compare.add_argument("right_run_id")
+    compare.add_argument("--json", action="store_true")
+    export = commands.add_parser("export", help="export a run as JSON or OTLP/JSON")
+    export.add_argument("run_id", nargs="?")
+    export.add_argument("--format", choices=("json", "otlp"), default="json")
+    export.add_argument("--output", default="-", help="output file, or - for stdout")
+    serve = commands.add_parser("serve", help="start the local timeline UI")
+    serve.add_argument("--host", default="127.0.0.1")
+    serve.add_argument("--port", type=int, default=8765)
+    prune = commands.add_parser("prune", help="remove recordings outside retention limits")
+    prune.add_argument("--older-than-days", type=int, default=30)
+    prune.add_argument("--keep-latest", type=int, default=100)
+    prune.add_argument(
+        "--yes", action="store_true", help="perform deletion; default is dry-run"
+    )
+    return parser
+
+
+def main(argv: Optional[Sequence[str]] = None) -> int:
+    args = build_parser().parse_args(argv)
+    store = SQLiteStore(args.db)
+    if args.command == "init":
+        print("Initialized %s" % Path(args.db).resolve())
+    elif args.command == "demo":
+        run_id = _demo(FlightRecorder(args.db))
+        print("Recorded demo run %s" % run_id)
+        print("Next: flightrecorder --db %s investigate %s" % (args.db, run_id))
+    elif args.command == "demo-agent":
+        workspace = Path(args.workspace).resolve()
+        run_id = run_agent_demo(FlightRecorder(args.db), workspace)
+        print("Recorded real workflow %s" % run_id)
+        print("Workspace: %s" % workspace)
+        _print_investigation(_load(store, run_id))
+    elif args.command == "runs":
+        for run in store.list_runs(args.limit):
+            print("%-14s %-8s %s" % (run["id"], run["status"], run["name"]))
+    elif args.command == "show":
+        _print_timeline(_load(store, args.run_id))
+    elif args.command == "investigate":
+        _print_investigation(_load(store, args.run_id))
+    elif args.command == "replay":
+        run = _load(store, args.run_id)
+        tape = [
+            {
+                "sequence": step["sequence"],
+                "kind": step["kind"],
+                "name": step["name"],
+                "input": step["input"],
+                "recorded_output": step["output"],
+                "status": step["status"],
+                "error": step["error"],
+            }
+            for step in run["steps"]
+        ]
+        if args.json:
+            print(json.dumps(tape, indent=2))
+        else:
+            print("Controlled replay of %s (side effects disabled)" % run["id"])
+            for item in tape:
+                print("  #%d %s -> %s" % (item["sequence"], item["name"], item["status"]))
+    elif args.command == "compare":
+        result = compare_runs(
+            _load(store, args.left_run_id), _load(store, args.right_run_id)
+        )
+        if args.json:
+            print(json.dumps(result, indent=2))
+        elif result["first_divergence"] is None:
+            print("No meaningful divergence found.")
+        else:
+            first = result["first_divergence"]
+            print("First divergence: position #%d" % first["position"])
+            print("Reasons: %s" % ", ".join(first["reasons"]))
+    elif args.command == "export":
+        run = _load(store, args.run_id)
+        payload = to_otlp(run) if args.format == "otlp" else to_json(run)
+        rendered = json.dumps(payload, indent=2) + "\n"
+        if args.output == "-":
+            print(rendered, end="")
+        else:
+            Path(args.output).write_text(rendered, encoding="utf-8")
+            print("Exported %s" % Path(args.output).resolve())
+    elif args.command == "serve":
+        from .web import serve
+
+        serve(store, args.host, args.port)
+    elif args.command == "prune":
+        cutoff = datetime.now(timezone.utc) - timedelta(
+            days=max(args.older_than_days, 0)
+        )
+        run_ids = store.prune_runs(
+            cutoff.isoformat(),
+            keep_latest=max(args.keep_latest, 0),
+            dry_run=not args.yes,
+        )
+        action = "Deleted" if args.yes else "Would delete"
+        print("%s %d run(s)." % (action, len(run_ids)))
+        for run_id in run_ids:
+            print("  %s" % run_id)
+        if run_ids and not args.yes:
+            print("Re-run with --yes to apply.")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())
+

flightrecorder_ai-1.0.0/src/flightrecorder/compare.py

@@ -0,0 +1,49 @@
+"""Run-to-run comparison."""
+
+from __future__ import annotations
+
+from typing import Any, Dict, List, Optional, Tuple
+
+
+def _signature(step: Dict[str, Any]) -> Tuple[str, str]:
+    return step["kind"], step["name"]
+
+
+def compare_runs(left: Dict[str, Any], right: Dict[str, Any]) -> Dict[str, Any]:
+    """Compare two runs and identify their first meaningful divergence."""
+    differences: List[Dict[str, Any]] = []
+    maximum = max(len(left["steps"]), len(right["steps"]))
+    for index in range(maximum):
+        left_step = left["steps"][index] if index < len(left["steps"]) else None
+        right_step = right["steps"][index] if index < len(right["steps"]) else None
+        reasons: List[str] = []
+        if left_step is None or right_step is None:
+            reasons.append("step_missing")
+        else:
+            if _signature(left_step) != _signature(right_step):
+                reasons.append("operation_changed")
+            if left_step["status"] != right_step["status"]:
+                reasons.append("status_changed")
+            if left_step["input"] != right_step["input"]:
+                reasons.append("input_changed")
+            if left_step["output"] != right_step["output"]:
+                reasons.append("output_changed")
+            if left_step["metadata"].get("changes") != right_step["metadata"].get("changes"):
+                reasons.append("file_changes_changed")
+        if reasons:
+            differences.append(
+                {
+                    "position": index + 1,
+                    "reasons": reasons,
+                    "left": left_step,
+                    "right": right_step,
+                }
+            )
+    return {
+        "left_run_id": left["id"],
+        "right_run_id": right["id"],
+        "status_changed": left["status"] != right["status"],
+        "first_divergence": differences[0] if differences else None,
+        "differences": differences,
+    }
+