openbox-temporal-sdk-python 1.0.21__tar.gz → 1.1.0__tar.gz

openbox_temporal_sdk_python-1.1.0/CHANGELOG.md

@@ -0,0 +1,68 @@
+# Changelog
+
+All notable changes to OpenBox SDK for Temporal Workflows.
+
+## [1.1.0] - 2026-03-09
+
+### Added
+
+- **Hook-level governance** — real-time, per-operation governance evaluation during activity execution
+  - Every HTTP request, database query, file operation, and traced function call is evaluated at `started` (before, can block) and `completed` (after, informational) stages
+  - Same `POST /api/v1/governance/evaluate` endpoint with new `hook_trigger` field in payload
+  - Automatically enabled when using `create_openbox_worker()`
+- **Database query governance** — per-query started/completed evaluations for psycopg2, pymysql, mysql-connector, asyncpg, pymongo, redis, sqlalchemy
+- **File I/O governance** — per-operation evaluations for open, read, write, readline, readlines, writelines, close (opt-in via `instrument_file_io=True`)
+- **`@traced` decorator** (`openbox.tracing`) — function-level governance with OTel spans; zero overhead when governance not configured
+- **`GovernanceBlockedError`** — new exception type for hook-level blocking with verdict, reason, and resource identifier
+- **Abort propagation** — once one hook blocks, all subsequent hooks for the same activity short-circuit immediately
+- **HALT workflow termination** from hook-level governance via `client.terminate()`
+- **REQUIRE_APPROVAL** from hook-level governance enters the same HITL approval polling flow as activity-level approvals
+- **`duration_ns`** computed for all hook span types (HTTP, file, function — DB already had it)
+
+### Changed
+
+- **`hook_trigger` simplified to boolean** — was a dict with type/stage/data, now just `true`. All data moved to span root fields
+- **Span data consolidation** — all type-specific fields at span root (`hook_type`, `http_method`, `db_system`, `file_path`, `function`, etc.)
+- **`attributes` is OTel-original only** — no custom `openbox.*`, `http.request.*`, `db.result.*` fields injected
+- Hook governance payloads send only the current span per evaluation (not accumulated history)
+- Event-level payloads (ActivityStarted/Completed, Workflow events) no longer include spans
+- Simplified `WorkflowSpanProcessor` — removed span buffering, governed span tracking, body data merging; `on_end()` now only forwards to fallback exporters
+
+### Fixed
+
+- HALT verdict from hooks now correctly terminates the workflow (previously only stopped the activity like BLOCK)
+- REQUIRE_APPROVAL from hooks now enters the approval polling flow (previously raised unhandled error)
+- Stale buffer/verdict from previous workflow run no longer carries over when workflow_id is reused
+- Subsequent hooks no longer fire after the first hook blocks an activity
+
+## [1.0.21] - 2026-03-04
+
+### Added
+
+- Human-in-the-loop approval with expiration handling
+- Approval polling via `POST /api/v1/governance/approval`
+- Guardrails: input/output validation and redaction
+- `GovernanceVerdictResponse.from_dict()` with guardrails_result parsing
+- Output redaction for activity results
+- `_deep_update_dataclass()` for in-place dataclass field updates from redacted dicts
+
+### Fixed
+
+- Temporal Payload objects no longer slip through as non-serializable in governance payloads
+- Stale buffer detection via run_id comparison
+
+## [1.0.0] - 2026-02-15
+
+### Added
+
+- Initial release
+- 6 event types: WorkflowStarted, WorkflowCompleted, WorkflowFailed, SignalReceived, ActivityStarted, ActivityCompleted
+- 5-tier verdict system: ALLOW, CONSTRAIN, REQUIRE_APPROVAL, BLOCK, HALT
+- HTTP instrumentation via OpenTelemetry (httpx, requests, urllib3, urllib)
+- Database instrumentation (psycopg2, pymysql, asyncpg, pymongo, redis, sqlalchemy)
+- File I/O instrumentation (opt-in)
+- Zero-code setup via `create_openbox_worker()` factory
+- Workflow and activity interceptors for governance
+- Span buffering and activity context tracking
+- `fail_open` / `fail_closed` error policies
+- v1.0 backward compatibility for legacy verdict strings

{openbox_temporal_sdk_python-1.0.21 → openbox_temporal_sdk_python-1.1.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: openbox-temporal-sdk-python
-Version: 1.0.21
+Version: 1.1.0
 Summary: OpenBox SDK - Governance and observability for Temporal workflows
 Project-URL: Homepage, https://github.com/OpenBox-AI/temporal-sdk-python
 Project-URL: Documentation, https://github.com/OpenBox-AI/temporal-sdk-python#readme
@@ -36,6 +36,7 @@ Requires-Dist: opentelemetry-instrumentation-pymysql<0.61b0,>=0.59b0
 Requires-Dist: opentelemetry-instrumentation-redis<0.61b0,>=0.59b0
 Requires-Dist: opentelemetry-instrumentation-requests<0.61b0,>=0.59b0
 Requires-Dist: opentelemetry-instrumentation-sqlalchemy<0.61b0,>=0.59b0
+Requires-Dist: opentelemetry-instrumentation-sqlite3<0.61b0,>=0.59b0
 Requires-Dist: opentelemetry-instrumentation-urllib3<0.61b0,>=0.59b0
 Requires-Dist: opentelemetry-instrumentation-urllib<0.61b0,>=0.59b0
 Requires-Dist: opentelemetry-sdk<1.40.0,>=1.38.0
@@ -54,6 +55,7 @@ OpenBox SDK provides **governance and observability** for Temporal workflows by
 **Key Features:**
 - 6 event types (WorkflowStarted, WorkflowCompleted, WorkflowFailed, SignalReceived, ActivityStarted, ActivityCompleted)
 - 5-tier verdict system (ALLOW, CONSTRAIN, REQUIRE_APPROVAL, BLOCK, HALT)
+- **Hook-level governance** — per-operation evaluation (HTTP requests, file I/O, database queries, function tracing) with started/completed stages
 - HTTP/Database/File I/O instrumentation via OpenTelemetry
 - Guardrails: Input/output validation and redaction
 - Human-in-the-loop approval with expiration handling
@@ -215,7 +217,7 @@ OpenBox Core can validate and redact sensitive data before/after activity execut
 
 ## Error Handling
 
-Configure error policy via `on_api_error`:
+Configure error policy via `on_api_error` (constants available as `hook_governance.FAIL_OPEN` / `FAIL_CLOSED`):
 
 | Policy | Behavior |
 |--------|----------|
@@ -233,11 +235,15 @@ Configure error policy via `on_api_error`:
 - `urllib` - request body only
 
 ### Databases
-- PostgreSQL: `psycopg2`, `asyncpg`
-- MySQL: `mysql-connector-python`, `pymysql`
-- MongoDB: `pymongo`
-- Redis: `redis`
-- ORM: `sqlalchemy`
+
+**Fully supported** — any dbapi-compatible library using OTel's `CursorTracer.traced_execution()`:
+- `psycopg2`, `pymysql`, `mysql-connector-python`, and other dbapi-compliant drivers
+
+**Custom hooks (best-effort)** — these libraries use non-dbapi instrumentation paths; governance hooks may not work correctly in all scenarios:
+- `asyncpg` — wrapt wrapper on Connection methods (governance runs outside OTel span context)
+- `pymongo` — CommandListener monitoring + wrapt Collection wrappers (dedup via thread-local flag; some internal commands like `endSessions` only produce `completed` stage)
+- `redis` — native OTel `request_hook`/`response_hook`
+- `sqlalchemy` — `before/after_cursor_execute` event listeners
 
 **SQLAlchemy Note:** If your SQLAlchemy engine is created before `create_openbox_worker()` runs (e.g., at module import time), you must pass it via the `sqlalchemy_engine` parameter. Without this, `SQLAlchemyInstrumentor` only patches future `create_engine()` calls and won't capture queries on pre-existing engines.
 
@@ -257,6 +263,113 @@ worker = create_openbox_worker(
 
 ---
 
+## Hook-Level Governance
+
+Every HTTP request, file operation, and database query made during an activity is evaluated by OpenBox Core in real-time at two stages:
+
+### HTTP Requests
+
+| Stage | Trigger | Data Available |
+|-------|---------|----------------|
+| `started` | Before request is sent | Method, URL, request headers, request body |
+| `completed` | After response received | All of above + response headers, response body, status code |
+
+### File Operations
+
+Per-operation governance evaluates **every** `read()`/`write()`/`readline()`/`readlines()`/`writelines()` call, not just open/close:
+
+| Operation | Stage | Trigger | Data Available |
+|-----------|-------|---------|----------------|
+| `open` | `started` | Before file is opened | File path, open mode |
+| `read` | `started` | Before read executes | File path, mode |
+| `read` | `completed` | After read returns | data (content read), bytes_read |
+| `readline` | `started` | Before readline executes | File path, mode |
+| `readline` | `completed` | After readline returns | data (line read), bytes_read |
+| `readlines` | `started` | Before readlines executes | File path, mode |
+| `readlines` | `completed` | After readlines returns | data (lines read), bytes_read, lines_count |
+| `write` | `started` | Before write executes | File path, mode |
+| `write` | `completed` | After write returns | data (content written), bytes_written |
+| `writelines` | `started` | Before writelines executes | File path, mode |
+| `writelines` | `completed` | After writelines returns | data (lines written), bytes_written, lines_count |
+| `close` | `completed` | After file is closed | bytes_read, bytes_written, operations list |
+
+**How it works (HTTP):**
+
+1. OTel httpx instrumentation fires a **request hook** → SDK sends `started` governance evaluation with request data
+2. If verdict is BLOCK/HALT → request is aborted before it leaves the process
+3. After response arrives → SDK sends `completed` governance evaluation with full request+response data
+4. If verdict is BLOCK/HALT → `GovernanceBlockedError` is raised, activity fails with `GovernanceStop`
+
+**How it works (File I/O):**
+
+1. Activity calls `open()` → SDK sends `started` governance evaluation with file path and mode
+2. If verdict is BLOCK/HALT → file is never opened, `GovernanceBlockedError` is raised
+3. Each `read()`/`write()`/`readline()`/`readlines()`/`writelines()` call sends `started` (before) and `completed` (after) governance evaluations — enabling content-based policy enforcement
+4. After file is closed → SDK sends `completed` governance with lifecycle summary (total bytes, operations list)
+5. File governance requires `instrument_file_io=True` (disabled by default)
+
+A simple open-read-close produces **4 governance evaluations**: open(started) → read(started) → read(completed) → close(completed).
+
+### Database Queries
+
+Every database operation is evaluated at `started` (pre-query, can block) and `completed` (post-query, reports outcome):
+
+| Field | Started | Completed |
+|-------|---------|-----------|
+| `type` | `"db_query"` | `"db_query"` |
+| `stage` | `"started"` | `"completed"` |
+| `db_system` | postgresql, mysql, mongodb, redis, sqlite | same |
+| `db_name` | Database name | same |
+| `db_operation` | SQL verb or command (SELECT, INSERT, GET, etc.) | same |
+| `db_statement` | Query string or command | same |
+| `server_address` | Host | same |
+| `server_port` | Port | same |
+| `duration_ms` | — | Query duration in ms |
+| `error` | — | Error message or None |
+
+**How it works:**
+
+1. Activity executes a DB query (via any supported library)
+2. SDK governance hook intercepts **before** the query → sends `started` evaluation
+3. If verdict is BLOCK/HALT → query is aborted, `GovernanceBlockedError` raised
+4. Query executes normally → SDK sends `completed` evaluation with duration and error status
+5. DB governance is automatic when `instrument_databases=True` (default)
+
+**Per-library strategy:**
+
+| Library | Hook Method | Can Block? | Reliability |
+|---------|------------|------------|-------------|
+| psycopg2, pymysql, mysql-connector-python | `CursorTracer.traced_execution` patch | Yes | Fully supported |
+| asyncpg | `wrapt` wrapper on Connection methods | Yes | Best-effort |
+| pymongo | CommandListener + `wrapt` Collection wrappers | Yes (wrapt only) | Best-effort |
+| redis | Native OTel `request_hook`/`response_hook` | Yes | Best-effort |
+| sqlalchemy | `before/after_cursor_execute` events | Yes | Best-effort |
+
+> **Note:** Libraries marked "Fully supported" use OTel's `CursorTracer`, which guarantees governance hooks run inside the OTel span context. Best-effort libraries use custom hooks that may produce inconsistencies (e.g., missing stages for internal commands). Some C extension types (e.g., `psycopg2.extensions.cursor`) cannot be patched with `wrapt` — in those cases governance hooks are silently skipped, but OTel span capture still works normally.
+
+### Function Tracing
+
+Functions decorated with `@traced` are automatically governed when `hook_governance` is configured:
+
+| Stage | Trigger | Data Available |
+|-------|---------|----------------|
+| `started` | Before function executes | Function name, module, arguments (if `capture_args=True`) |
+| `completed` | After function returns/raises | All of above + result (if `capture_result=True`) or error info |
+
+**How it works (Function Tracing):**
+
+1. `@traced` decorator wrapper starts OTel span
+2. → SDK sends `started` governance evaluation with function name and module
+3. If verdict is BLOCK/HALT → `GovernanceBlockedError` raised, function never executes
+4. Function executes normally
+5. → SDK sends `completed` governance evaluation with result or error info
+6. If verdict is BLOCK/HALT → `GovernanceBlockedError` raised after execution
+7. When `hook_governance` is not configured → zero overhead, no governance calls
+
+**Governed span tracking:** When hook-level governance is active, the SDK marks HTTP spans as "governed" so the OTel `on_end` processor skips buffering them — preventing duplicate spans.
+
+---
+
 ## Architecture
 
 See [System Architecture](./docs/system-architecture.md) for detailed component design.
@@ -269,8 +382,33 @@ Workflow/Activity → Interceptors → Span Processor → OpenBox Core API
                                             Returns Verdict
                                                     ↓
                                     (ALLOW, BLOCK, HALT, etc.)
+
+Hook-Level (per HTTP request):
+Activity HTTP Call → OTel Hook → hook_governance → API (started) → Allow/Block
+                   → Response  → hook_governance → API (completed) → Allow/Block
+
+Hook-Level (per file operation):
+Activity open()       → hook_governance → API (started)   → Allow/Block
+Activity read/write() → hook_governance → API (started)   → Allow/Block
+                      → hook_governance → API (completed) → Allow/Block
+Activity close()      → hook_governance → API (completed) → Allow/Block
+
+Hook-Level (per DB query):
+Activity DB Call → db_governance_hooks → hook_governance → API (started) → Allow/Block
+                → Query executes     → hook_governance → API (completed)
+
+Hook-Level (per @traced function):
+@traced call    → hook_governance → API (started) → Allow/Block
+                → Function runs  → hook_governance → API (completed)
 ```
 
+**Module responsibilities:**
+- `otel_setup.py` — OTel instrumentation, hooks, TracedFile wrapper
+- `hook_governance.py` — Shared governance evaluator (payload building, API calls, verdict handling)
+- `db_governance_hooks.py` — Per-library DB governance wrappers (wrapt, OTel hooks, SQLAlchemy events)
+- `span_processor.py` — Span buffering, activity context tracking
+- `activity_interceptor.py` — Activity lifecycle governance events
+
 ---
 
 ## Advanced Usage
@@ -287,6 +425,7 @@ from openbox import (
 from openbox.otel_setup import setup_opentelemetry_for_governance
 from openbox.activity_interceptor import ActivityGovernanceInterceptor
 from openbox.activities import send_governance_event
+from openbox.hook_governance import FAIL_OPEN, FAIL_CLOSED  # error policy constants
 
 # 1. Initialize SDK
 initialize(api_url="http://localhost:8086", api_key="obx_test_key_1")
@@ -296,9 +435,11 @@ span_processor = WorkflowSpanProcessor(
     ignored_url_prefixes=["http://localhost:8086"]
 )
 
-# 3. Setup OTel instrumentation
+# 3. Setup OTel instrumentation (governance always enabled)
 setup_opentelemetry_for_governance(
     span_processor,
+    api_url="http://localhost:8086",
+    api_key="obx_test_key_1",
     sqlalchemy_engine=engine,  # optional: instrument pre-existing engine
 )
 
@@ -348,13 +489,13 @@ worker = Worker(
 
 ## Testing
 
-The SDK includes comprehensive test coverage with 10 test files:
+The SDK includes comprehensive test coverage with 13 test files:
 
 ```bash
 pytest tests/
 ```
 
-Test files: `test_activities.py`, `test_activity_interceptor.py`, `test_config.py`, `test_otel_setup.py`, `test_span_processor.py`, `test_tracing.py`, `test_types.py`, `test_worker.py`, `test_workflow_interceptor.py`
+Test files: `test_activities.py`, `test_activity_interceptor.py`, `test_config.py`, `test_db_governance_hooks.py`, `test_file_governance_hooks.py`, `test_otel_hook_pause.py`, `test_otel_hook_pause_db.py`, `test_otel_setup.py`, `test_span_processor.py`, `test_tracing.py`, `test_types.py`, `test_worker.py`, `test_workflow_interceptor.py`
 
 ---
 
@@ -371,4 +512,4 @@ MIT License - See LICENSE file for details
 
 ---
 
-**Version:** 1.0.2 | **Last Updated:** 2026-02-12
+**Version:** 1.1.0 | **Last Updated:** 2026-03-09