runproof-engine 0.1.2__tar.gz → 0.2.0__tar.gz

runproof_engine-0.2.0/PKG-INFO

@@ -0,0 +1,164 @@
+Metadata-Version: 2.4
+Name: runproof-engine
+Version: 0.2.0
+Summary: Evidence, replay, and explainable diffs for real Python runs
+Author: RunProof Contributors
+License-Expression: Apache-2.0
+Keywords: reproducibility,provenance,data-lineage,observability,python,replay,explainable-diff
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Topic :: Software Development :: Testing
+Classifier: Topic :: Scientific/Engineering :: Information Analysis
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: data
+Requires-Dist: pandas>=2.0; extra == "data"
+Requires-Dist: polars>=1.0; extra == "data"
+Provides-Extra: http
+Requires-Dist: requests>=2.31; extra == "http"
+Requires-Dist: httpx>=0.27; extra == "http"
+Provides-Extra: database
+Requires-Dist: SQLAlchemy>=2.0; extra == "database"
+Requires-Dist: psycopg[binary]>=3.1; extra == "database"
+Provides-Extra: jupyter
+Requires-Dist: nbclient>=0.10; extra == "jupyter"
+Requires-Dist: nbformat>=5.9; extra == "jupyter"
+Provides-Extra: cloud
+Requires-Dist: boto3>=1.34; extra == "cloud"
+Provides-Extra: ml
+Requires-Dist: joblib>=1.3; extra == "ml"
+Provides-Extra: dev
+Requires-Dist: pytest>=8; extra == "dev"
+Requires-Dist: ruff>=0.6; extra == "dev"
+Requires-Dist: build>=1.2; extra == "dev"
+Requires-Dist: twine>=5; extra == "dev"
+Requires-Dist: hypothesis>=6.100; extra == "dev"
+Requires-Dist: bandit>=1.7; extra == "dev"
+Requires-Dist: pip-audit>=2.7; extra == "dev"
+Dynamic: license-file
+
+# RunProof
+
+RunProof is a Python library for recording, validating, replaying, and comparing real computational runs. It turns a Python execution into an inspectable artifact containing inputs, outputs, code references, environment metadata, checks, and an execution trace.
+
+The distinctive feature is **Explainable Diff**: when two runs differ, RunProof compares input fingerprints, schemas, output summaries, step status, code fingerprints, and environment metadata, then reports evidence-backed causes instead of only saying that the runs are different.
+
+## What it is
+
+RunProof is a local-first execution record and reproducibility layer for Python workflows. It is useful for data analysis, reports, ML experiments, research software, API workflows, and any process where the result must be explained later.
+
+RunProof is not a reverse-engineering tool, a code generator, a replacement for Git, or a guarantee that a scientific conclusion is correct. It records and validates the execution that was declared to it.
+
+## Quick start
+
+```python
+from runproof_engine import verified
+
+
+def clean_rows(rows):
+    return [row for row in rows if row["amount"] >= 0]
+
+
+def total(rows):
+    return sum(row["amount"] for row in rows)
+
+with verified("sales_total", root="runs") as run:
+    rows = run.input("sales.json", name="sales")
+    cleaned = run.step("clean_rows", clean_rows, rows)
+    result = run.step("total", total, cleaned)
+    run.assert_true(result >= 0, "total must be non-negative")
+    run.output("total.json", {"total": result})
+
+print(run.result.status)
+print(run.result.artifact_dir)
+```
+
+This creates a run directory with a manifest, input metadata, output metadata, trace events, checks, and an environment snapshot. The input is not silently replaced by generated data. File contents are fingerprinted, while copying full inputs is explicit and configurable.
+
+## Automatic capture
+
+For whole-process observation, use `auto_run`. It installs a Python execution observer for the lifetime of the context. On Python 3.12 and newer it uses `sys.monitoring` when available; otherwise it falls back to `sys.settrace`. The observer records bounded call, return, and exception events without requiring every function to be wrapped manually.
+
+```python
+from pathlib import Path
+from runproof_engine import auto_run
+
+with auto_run(
+    "automatic-report",
+    root="runs",
+    include_paths=[Path("src")],
+) as run:
+    report = build_report(real_input)
+    run.observe(report, name="report")
+    run.output("report.json", report)
+```
+
+Automatic capture is an evidence layer, not a sandbox. It does not serialize every local variable, intercept every native extension, or make external services deterministic. Use adapters or explicit `run.input`, `run.step`, and `run.external_call` calls when a boundary needs stronger evidence.
+
+### Optional real-library adapters
+
+When an optional dependency is installed, `auto_run` discovers and activates its adapter by default. The current adapters observe real operations through documented hooks or narrowly scoped wrappers:
+
+| Integration | Captured evidence | Boundary behavior |
+| --- | --- | --- |
+| pandas | DataFrame reads/writes, summaries, file hashes | Local file outputs can be archived; in-memory and native execution remain bounded |
+| Polars | DataFrame reads/writes, summaries, file hashes | Requires the optional `polars` package |
+| requests | Prepared URL, redacted headers, status code | Response bodies are not consumed automatically; live network remains a boundary |
+| HTTPX | Sync and async request/response hooks | Streaming bodies are not consumed automatically; live network remains a boundary |
+| SQLite | SQL statements and database path | Database transaction state is a boundary; mutable files can become `non_reproducible` |
+| SQLAlchemy and psycopg | Cursor statements and database dialect | Requires the optional database package and a live database policy |
+| subprocess | Command summary, return code, stdout/stderr digests | Process environment and side effects remain a boundary |
+| Boto3 | Service operation, redacted parameters, response summary | Cloud object/service state remains a boundary |
+| Jupyter/nbclient | Notebook execution lifecycle | Kernel state and external effects remain a boundary |
+| Torch and joblib | Model save/load file evidence | Device/native runtime state may remain a boundary |
+
+The optional adapters never execute installation commands and never send captured data to RunProof services. They can be disabled or replaced by passing an explicit `adapters=` collection to `auto_run`.
+
+## Provenance, environment, and distributed tracing
+
+Every completed run now contains `provenance.json`, a graph connecting the run to inputs, steps, values, outputs, checks, observations, and evidence boundaries. It is available through `load_run(path).provenance` and is designed to make later diffs explainable without pretending to prove causality.
+
+The artifact also contains `environment/environment.lock.json`. It records the interpreter, platform, and installed package versions and can be compared with `compare_environment_lock(...)`. `reconstruction_plan(...)` returns reviewable pip commands but deliberately marks them as requiring approval; RunProof never installs packages silently. Package locks do not reproduce operating-system state, hardware, external services, or changing data.
+
+For service-to-service workflows, use `run.span(...)` and propagate `run.traceparent()`. Spans are stored in `execution/spans.json` using a dependency-free W3C traceparent-compatible format. OpenTelemetry exporters can be added as adapters without making OpenTelemetry a runtime dependency.
+
+## Replay and comparison
+
+```python
+from runproof_engine import load_run
+
+previous = load_run("runs/sales_total/20260823-101500-abc123")
+replayed = previous.replay(mode="strict")
+print(replayed.status)
+
+comparison = previous.diff(replayed)
+print(comparison.to_dict())
+print(comparison.render())
+```
+
+Without a runner, `replay()` verifies the captured artifact and reports `replay_ready`; it does not silently re-execute arbitrary Python. To perform a real replay, provide a user-controlled runner that invokes the original workflow and returns the new artifact. The resulting run can then be compared with the previous run to identify evidence-backed changes.
+
+## Statuses
+
+- `verified`: execution completed and all declared checks passed with no recorded evidence boundary.
+- `verified_with_boundaries`: execution completed and the artifact is intact, but one or more external or non-deterministic effects remain outside the captured evidence.
+- `verified_with_warnings`: execution completed but comparability or external-source evidence is limited.
+- `failed`: execution or a required check failed.
+- `blocked`: a declared policy prevented a sensitive action.
+- `non_reproducible`: replay was attempted but did not match the captured run.
+
+## Privacy and real data
+
+RunProof records metadata and hashes by default. Full input copying is opt-in. Secrets are redacted from environment snapshots and trace values. External adapters must provide privacy-safe request metadata instead of storing credentials or raw authorization headers.
+
+## Project status
+
+The current repository implements the local core, automatic observation, provenance graph, environment lock, distributed trace context, and optional real-library integrations. It remains useful without a cloud account or a specific AI provider. Automatic observation still has explicit coverage boundaries; it is not a universal sandbox or a guarantee of perfect replay for arbitrary Python.
+
+## License
+
+Apache-2.0. See `LICENSE`.

runproof_engine-0.2.0/README.md

@@ -0,0 +1,121 @@
+# RunProof
+
+RunProof is a Python library for recording, validating, replaying, and comparing real computational runs. It turns a Python execution into an inspectable artifact containing inputs, outputs, code references, environment metadata, checks, and an execution trace.
+
+The distinctive feature is **Explainable Diff**: when two runs differ, RunProof compares input fingerprints, schemas, output summaries, step status, code fingerprints, and environment metadata, then reports evidence-backed causes instead of only saying that the runs are different.
+
+## What it is
+
+RunProof is a local-first execution record and reproducibility layer for Python workflows. It is useful for data analysis, reports, ML experiments, research software, API workflows, and any process where the result must be explained later.
+
+RunProof is not a reverse-engineering tool, a code generator, a replacement for Git, or a guarantee that a scientific conclusion is correct. It records and validates the execution that was declared to it.
+
+## Quick start
+
+```python
+from runproof_engine import verified
+
+
+def clean_rows(rows):
+    return [row for row in rows if row["amount"] >= 0]
+
+
+def total(rows):
+    return sum(row["amount"] for row in rows)
+
+with verified("sales_total", root="runs") as run:
+    rows = run.input("sales.json", name="sales")
+    cleaned = run.step("clean_rows", clean_rows, rows)
+    result = run.step("total", total, cleaned)
+    run.assert_true(result >= 0, "total must be non-negative")
+    run.output("total.json", {"total": result})
+
+print(run.result.status)
+print(run.result.artifact_dir)
+```
+
+This creates a run directory with a manifest, input metadata, output metadata, trace events, checks, and an environment snapshot. The input is not silently replaced by generated data. File contents are fingerprinted, while copying full inputs is explicit and configurable.
+
+## Automatic capture
+
+For whole-process observation, use `auto_run`. It installs a Python execution observer for the lifetime of the context. On Python 3.12 and newer it uses `sys.monitoring` when available; otherwise it falls back to `sys.settrace`. The observer records bounded call, return, and exception events without requiring every function to be wrapped manually.
+
+```python
+from pathlib import Path
+from runproof_engine import auto_run
+
+with auto_run(
+    "automatic-report",
+    root="runs",
+    include_paths=[Path("src")],
+) as run:
+    report = build_report(real_input)
+    run.observe(report, name="report")
+    run.output("report.json", report)
+```
+
+Automatic capture is an evidence layer, not a sandbox. It does not serialize every local variable, intercept every native extension, or make external services deterministic. Use adapters or explicit `run.input`, `run.step`, and `run.external_call` calls when a boundary needs stronger evidence.
+
+### Optional real-library adapters
+
+When an optional dependency is installed, `auto_run` discovers and activates its adapter by default. The current adapters observe real operations through documented hooks or narrowly scoped wrappers:
+
+| Integration | Captured evidence | Boundary behavior |
+| --- | --- | --- |
+| pandas | DataFrame reads/writes, summaries, file hashes | Local file outputs can be archived; in-memory and native execution remain bounded |
+| Polars | DataFrame reads/writes, summaries, file hashes | Requires the optional `polars` package |
+| requests | Prepared URL, redacted headers, status code | Response bodies are not consumed automatically; live network remains a boundary |
+| HTTPX | Sync and async request/response hooks | Streaming bodies are not consumed automatically; live network remains a boundary |
+| SQLite | SQL statements and database path | Database transaction state is a boundary; mutable files can become `non_reproducible` |
+| SQLAlchemy and psycopg | Cursor statements and database dialect | Requires the optional database package and a live database policy |
+| subprocess | Command summary, return code, stdout/stderr digests | Process environment and side effects remain a boundary |
+| Boto3 | Service operation, redacted parameters, response summary | Cloud object/service state remains a boundary |
+| Jupyter/nbclient | Notebook execution lifecycle | Kernel state and external effects remain a boundary |
+| Torch and joblib | Model save/load file evidence | Device/native runtime state may remain a boundary |
+
+The optional adapters never execute installation commands and never send captured data to RunProof services. They can be disabled or replaced by passing an explicit `adapters=` collection to `auto_run`.
+
+## Provenance, environment, and distributed tracing
+
+Every completed run now contains `provenance.json`, a graph connecting the run to inputs, steps, values, outputs, checks, observations, and evidence boundaries. It is available through `load_run(path).provenance` and is designed to make later diffs explainable without pretending to prove causality.
+
+The artifact also contains `environment/environment.lock.json`. It records the interpreter, platform, and installed package versions and can be compared with `compare_environment_lock(...)`. `reconstruction_plan(...)` returns reviewable pip commands but deliberately marks them as requiring approval; RunProof never installs packages silently. Package locks do not reproduce operating-system state, hardware, external services, or changing data.
+
+For service-to-service workflows, use `run.span(...)` and propagate `run.traceparent()`. Spans are stored in `execution/spans.json` using a dependency-free W3C traceparent-compatible format. OpenTelemetry exporters can be added as adapters without making OpenTelemetry a runtime dependency.
+
+## Replay and comparison
+
+```python
+from runproof_engine import load_run
+
+previous = load_run("runs/sales_total/20260823-101500-abc123")
+replayed = previous.replay(mode="strict")
+print(replayed.status)
+
+comparison = previous.diff(replayed)
+print(comparison.to_dict())
+print(comparison.render())
+```
+
+Without a runner, `replay()` verifies the captured artifact and reports `replay_ready`; it does not silently re-execute arbitrary Python. To perform a real replay, provide a user-controlled runner that invokes the original workflow and returns the new artifact. The resulting run can then be compared with the previous run to identify evidence-backed changes.
+
+## Statuses
+
+- `verified`: execution completed and all declared checks passed with no recorded evidence boundary.
+- `verified_with_boundaries`: execution completed and the artifact is intact, but one or more external or non-deterministic effects remain outside the captured evidence.
+- `verified_with_warnings`: execution completed but comparability or external-source evidence is limited.
+- `failed`: execution or a required check failed.
+- `blocked`: a declared policy prevented a sensitive action.
+- `non_reproducible`: replay was attempted but did not match the captured run.
+
+## Privacy and real data
+
+RunProof records metadata and hashes by default. Full input copying is opt-in. Secrets are redacted from environment snapshots and trace values. External adapters must provide privacy-safe request metadata instead of storing credentials or raw authorization headers.
+
+## Project status
+
+The current repository implements the local core, automatic observation, provenance graph, environment lock, distributed trace context, and optional real-library integrations. It remains useful without a cloud account or a specific AI provider. Automatic observation still has explicit coverage boundaries; it is not a universal sandbox or a guarantee of perfect replay for arbitrary Python.
+
+## License
+
+Apache-2.0. See `LICENSE`.

{runproof-engine-0.1.2 → runproof_engine-0.2.0}/pyproject.toml

@@ -1,14 +1,14 @@
 [build-system]
-requires = ["setuptools>=68"]
+requires = ["setuptools>=83.0.0", "wheel>=0.46.2"]
 build-backend = "setuptools.build_meta"
 
 [project]
 name = "runproof-engine"
-version = "0.1.2"
+version = "0.2.0"
 description = "Evidence, replay, and explainable diffs for real Python runs"
 readme = "README.md"
 requires-python = ">=3.10"
-license = { text = "Apache-2.0" }
+license = "Apache-2.0"
 authors = [
   { name = "RunProof Contributors" }
 ]
@@ -17,7 +17,6 @@ classifiers = [
   "Development Status :: 3 - Alpha",
   "Intended Audience :: Developers",
   "Intended Audience :: Science/Research",
-  "License :: OSI Approved :: Apache Software License",
   "Programming Language :: Python :: 3",
   "Programming Language :: Python :: 3 :: Only",
   "Topic :: Software Development :: Testing",
@@ -26,8 +25,13 @@ classifiers = [
 dependencies = []
 
 [project.optional-dependencies]
-data = ["pandas>=2.0"]
-dev = ["pytest>=8", "ruff>=0.6"]
+data = ["pandas>=2.0", "polars>=1.0"]
+http = ["requests>=2.31", "httpx>=0.27"]
+database = ["SQLAlchemy>=2.0", "psycopg[binary]>=3.1"]
+jupyter = ["nbclient>=0.10", "nbformat>=5.9"]
+cloud = ["boto3>=1.34"]
+ml = ["joblib>=1.3"]
+dev = ["pytest>=8", "ruff>=0.6", "build>=1.2", "twine>=5", "hypothesis>=6.100", "bandit>=1.7", "pip-audit>=2.7"]
 
 [project.scripts]
 runproof = "runproof_engine.cli:main"

runproof_engine-0.2.0/src/runproof_engine/__init__.py

@@ -0,0 +1,82 @@
+from .adapters import Adapter, AdapterInfo, UrllibAdapter, default_adapters
+from .auto import AutoCapture, auto_run
+from .core import (
+    CheckRecord,
+    ReplayUnavailable,
+    RunContext,
+    RunProofError,
+    RunResult,
+    StepRecord,
+    verified,
+)
+from .diff import Difference, RunDiff, compare_manifests
+from .environment import compare_environment_lock, environment_lock, reconstruction_plan
+from .integrations import (
+    Boto3Adapter,
+    HTTPXAdapter,
+    JoblibAdapter,
+    JupyterAdapter,
+    PandasAdapter,
+    PolarsAdapter,
+    PsycopgAdapter,
+    RequestsAdapter,
+    SQLAlchemyAdapter,
+    SQLiteAdapter,
+    SubprocessAdapter,
+    TorchAdapter,
+    available_adapters,
+)
+from .policy import Policy, PolicyDenied, safe_default_policy
+from .provenance import ProvenanceEdge, ProvenanceGraph, ProvenanceNode
+from .replay import LoadedRun, ReplayReport, load_run
+from .tracing import Span, Tracer, format_traceparent, parse_traceparent
+
+__all__ = [
+    "Adapter",
+    "AdapterInfo",
+    "AutoCapture",
+    "Boto3Adapter",
+    "CheckRecord",
+    "Difference",
+    "HTTPXAdapter",
+    "JoblibAdapter",
+    "JupyterAdapter",
+    "LoadedRun",
+    "PandasAdapter",
+    "PolarsAdapter",
+    "Policy",
+    "PolicyDenied",
+    "ProvenanceEdge",
+    "ProvenanceGraph",
+    "ProvenanceNode",
+    "PsycopgAdapter",
+    "ReplayReport",
+    "ReplayUnavailable",
+    "RequestsAdapter",
+    "RunContext",
+    "RunDiff",
+    "RunProofError",
+    "RunResult",
+    "SQLAlchemyAdapter",
+    "SQLiteAdapter",
+    "Span",
+    "StepRecord",
+    "SubprocessAdapter",
+    "TorchAdapter",
+    "Tracer",
+    "UrllibAdapter",
+    "auto_run",
+    "available_adapters",
+    "compare_environment_lock",
+    "compare_manifests",
+    "default_adapters",
+    "environment_lock",
+    "format_traceparent",
+    "load_run",
+    "parse_traceparent",
+    "reconstruction_plan",
+    "safe_default_policy",
+    "verified",
+]
+
+__version__ = "0.2.0"

runproof_engine-0.2.0/src/runproof_engine/adapters.py

@@ -0,0 +1,136 @@
+"""Optional boundary adapters for automatic RunProof observation."""
+
+from __future__ import annotations
+
+import hashlib
+from dataclasses import dataclass
+from typing import Any, Protocol
+from urllib.parse import parse_qsl, urlencode, urlsplit, urlunsplit
+from urllib.request import Request
+
+from .core import RunContext
+
+
+class Adapter(Protocol):
+    """Contract implemented by an automatic boundary adapter."""
+
+    name: str
+
+    def install(self, context: RunContext) -> Any:
+        """Install hooks and return a zero-argument uninstall callback."""
+
+
+@dataclass(frozen=True)
+class AdapterInfo:
+    name: str
+    version: str
+    capabilities: tuple[str, ...]
+
+
+_SECRET_QUERY_MARKERS = (
+    "token",
+    "key",
+    "secret",
+    "password",
+    "credential",
+    "authorization",
+)
+
+
+def _safe_url(value: str) -> str:
+    """Keep URL structure while redacting likely secret query values."""
+    try:
+        parts = urlsplit(value)
+        query = []
+        for key, item in parse_qsl(parts.query, keep_blank_values=True):
+            safe_item = "[REDACTED]" if any(marker in key.lower() for marker in _SECRET_QUERY_MARKERS) else item
+            query.append((key, safe_item))
+        return urlunsplit((parts.scheme, parts.netloc, parts.path, urlencode(query), parts.fragment))
+    except ValueError:
+        return value[:500]
+
+
+def _request_details(value: Any, data: Any) -> tuple[str, str, str | None]:
+    if isinstance(value, Request):
+        url = value.full_url
+        method = value.method or ("POST" if value.data is not None else "GET")
+        body = value.data
+    else:
+        url = str(value)
+        method = "POST" if data is not None else "GET"
+        body = data
+    body_bytes = body if isinstance(body, bytes) else str(body).encode("utf-8") if body is not None else None
+    digest = hashlib.sha256(body_bytes).hexdigest() if body_bytes else None
+    return method.upper(), _safe_url(url), digest
+
+
+class UrllibAdapter:
+    """Observe calls through ``urllib.request.urlopen`` without consuming responses."""
+
+    name = "urllib"
+    info = AdapterInfo(
+        name="urllib",
+        version="1",
+        capabilities=("http-request-metadata", "response-status", "secret-query-redaction"),
+    )
+
+    def install(self, context: RunContext) -> Any:
+        import urllib.request as urllib_request
+
+        original = urllib_request.urlopen
+        adapter = self
+
+        def wrapped(*args: Any, **kwargs: Any) -> Any:
+            request_value = args[0] if args else kwargs.get("url")
+            data = args[1] if len(args) > 1 else kwargs.get("data")
+            method, url, body_sha256 = _request_details(request_value, data)
+            try:
+                response = original(*args, **kwargs)
+            except Exception as error:
+                context._event(
+                    "http_request_failed",
+                    {
+                        "adapter": adapter.name,
+                        "method": method,
+                        "url": url,
+                        "error_type": f"{type(error).__module__}.{type(error).__qualname__}",
+                        "message": str(error),
+                    },
+                )
+                raise
+            status_code = getattr(response, "status", None) or getattr(response, "code", None)
+            context._event(
+                "http_request_observed",
+                {
+                    "adapter": adapter.name,
+                    "method": method,
+                    "url": url,
+                    "body_sha256": body_sha256,
+                    "status_code": status_code,
+                },
+            )
+            context.boundary(
+                "network_response",
+                target=url,
+                reason="automatic urllib adapter does not consume or archive the response body",
+                replay="requires_live_service",
+            )
+            return response
+
+        urllib_request.urlopen = wrapped
+        context._event("adapter_installed", {"name": self.name, "version": self.info.version})
+
+        def uninstall() -> None:
+            if urllib_request.urlopen is wrapped:
+                urllib_request.urlopen = original
+            context._event("adapter_uninstalled", {"name": self.name})
+
+        return uninstall
+
+
+def default_adapters() -> tuple[Adapter, ...]:
+    """Return safe standard-library adapters enabled by automatic capture."""
+    return (UrllibAdapter(),)
+
+
+__all__ = ["Adapter", "AdapterInfo", "UrllibAdapter", "default_adapters"]