nl2data-workflow-postgres 0.1.0__tar.gz

nl2data_workflow_postgres-0.1.0/PKG-INFO

@@ -0,0 +1,110 @@
+Metadata-Version: 2.4
+Name: nl2data-workflow-postgres
+Version: 0.1.0
+Summary: PostgreSQL-backed durable workflow state backend for nl2data-core.
+Author: NL2Data Contributors
+License: Apache-2.0
+Keywords: nl2data,postgresql,workflow,state,lease,idempotency
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Software Development :: Libraries
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+Requires-Dist: nl2data-core>=0.1.0
+Requires-Dist: psycopg[binary,pool]<4,>=3.1
+Provides-Extra: dev
+Requires-Dist: pytest>=8.0; extra == "dev"
+Requires-Dist: pytest-asyncio>=0.23; extra == "dev"
+Requires-Dist: mypy>=1.10; extra == "dev"
+Requires-Dist: ruff>=0.5; extra == "dev"
+
+# nl2data-workflow-postgres
+
+Optional PostgreSQL-backed durable workflow state backend for `nl2data-core`.
+
+## Overview
+
+This package provides a production-ready PostgreSQL implementation of the core workflow state-store contracts (`StateStore`, `IdempotencyStore`, `WorkflowLeaseStore`, and `FencedStateStore`) from `nl2data-core`. It persists safe workflow snapshots, idempotency records, and execution leases in PostgreSQL while keeping `psycopg` lazy and optional.
+
+Base `nl2data` / `nl2data-core` imports remain PostgreSQL-free; the driver is loaded only when a `PostgreSQLStateStore` is constructed.
+
+## Installation
+
+```bash
+pip install nl2data-workflow-postgres
+```
+
+This also installs `psycopg[binary,pool]>=3.1,<4` and `nl2data-core>=0.1.0`.
+
+## Quick Start
+
+```python
+from nl2data_core.workflow import WorkflowState
+from nl2data_workflow_postgres import PostgreSQLStateStore, WorkflowPostgresConfig
+
+store = PostgreSQLStateStore(
+    dsn="postgresql://user:pass@localhost/db",
+    config=WorkflowPostgresConfig(namespace="workflow"),
+)
+
+# store.create(state), store.update(...), store.acquire_lease(...), etc.
+```
+
+## Public surface
+
+```python
+from nl2data_workflow_postgres import (
+    MIGRATIONS,
+    SUPPORTED_SCHEMA_VERSION,
+    SQL_TEMPLATES,
+    PostgreSQLStateStore,
+    WorkflowPostgresConfig,
+    build_pool,
+    driver_available,
+)
+```
+
+- `PostgreSQLStateStore` — the core store-contract implementation
+  (`StateStore`, `IdempotencyStore`, `WorkflowLeaseStore`, `FencedStateStore`)
+  with compare-and-set updates and fenced lease checks.
+- `WorkflowPostgresConfig` — bounded pool/timeout/TTL configuration with a
+  derived tenant-scope namespace.
+- `MIGRATIONS` / `SUPPORTED_SCHEMA_VERSION` — versioned DDL in a package-owned
+  schema namespace.
+- `build_pool` / `driver_available` — lazy psycopg pool construction and
+  driver availability probes that never import the driver.
+
+## Compatibility
+
+The in-core `nl2data_core.workflow.postgres_*` modules were removed. All
+PostgreSQL workflow state ships from this package; import from
+`nl2data_workflow_postgres` directly. The core distribution no longer ships a
+PostgreSQL workflow backend.
+
+## Features
+
+- **Durable workflow state** with versioned safe snapshots and compare-and-set semantics
+- **Idempotency records** with reservation, conflict detection, and terminal outcome references
+- **Lease ownership** with monotonic fencing tokens, renewal, release, and stale-worker takeover
+- **Tenant-scope isolation** via opaque fingerprints and derived schema namespaces
+- **Versioned migrations** in a package-owned schema namespace
+- **Normalized errors** without leaking DSNs, credentials, or raw backend text
+- **Lazy driver loading** so the base framework never imports `psycopg`
+
+## Development
+
+Install in editable mode from the repository root:
+
+```bash
+pip install -e packages/nl2data-workflow-postgres
+```
+
+Run package tests:
+
+```bash
+pytest packages/nl2data-workflow-postgres/tests -q
+```

nl2data_workflow_postgres-0.1.0/README.md

@@ -0,0 +1,86 @@
+# nl2data-workflow-postgres
+
+Optional PostgreSQL-backed durable workflow state backend for `nl2data-core`.
+
+## Overview
+
+This package provides a production-ready PostgreSQL implementation of the core workflow state-store contracts (`StateStore`, `IdempotencyStore`, `WorkflowLeaseStore`, and `FencedStateStore`) from `nl2data-core`. It persists safe workflow snapshots, idempotency records, and execution leases in PostgreSQL while keeping `psycopg` lazy and optional.
+
+Base `nl2data` / `nl2data-core` imports remain PostgreSQL-free; the driver is loaded only when a `PostgreSQLStateStore` is constructed.
+
+## Installation
+
+```bash
+pip install nl2data-workflow-postgres
+```
+
+This also installs `psycopg[binary,pool]>=3.1,<4` and `nl2data-core>=0.1.0`.
+
+## Quick Start
+
+```python
+from nl2data_core.workflow import WorkflowState
+from nl2data_workflow_postgres import PostgreSQLStateStore, WorkflowPostgresConfig
+
+store = PostgreSQLStateStore(
+    dsn="postgresql://user:pass@localhost/db",
+    config=WorkflowPostgresConfig(namespace="workflow"),
+)
+
+# store.create(state), store.update(...), store.acquire_lease(...), etc.
+```
+
+## Public surface
+
+```python
+from nl2data_workflow_postgres import (
+    MIGRATIONS,
+    SUPPORTED_SCHEMA_VERSION,
+    SQL_TEMPLATES,
+    PostgreSQLStateStore,
+    WorkflowPostgresConfig,
+    build_pool,
+    driver_available,
+)
+```
+
+- `PostgreSQLStateStore` — the core store-contract implementation
+  (`StateStore`, `IdempotencyStore`, `WorkflowLeaseStore`, `FencedStateStore`)
+  with compare-and-set updates and fenced lease checks.
+- `WorkflowPostgresConfig` — bounded pool/timeout/TTL configuration with a
+  derived tenant-scope namespace.
+- `MIGRATIONS` / `SUPPORTED_SCHEMA_VERSION` — versioned DDL in a package-owned
+  schema namespace.
+- `build_pool` / `driver_available` — lazy psycopg pool construction and
+  driver availability probes that never import the driver.
+
+## Compatibility
+
+The in-core `nl2data_core.workflow.postgres_*` modules were removed. All
+PostgreSQL workflow state ships from this package; import from
+`nl2data_workflow_postgres` directly. The core distribution no longer ships a
+PostgreSQL workflow backend.
+
+## Features
+
+- **Durable workflow state** with versioned safe snapshots and compare-and-set semantics
+- **Idempotency records** with reservation, conflict detection, and terminal outcome references
+- **Lease ownership** with monotonic fencing tokens, renewal, release, and stale-worker takeover
+- **Tenant-scope isolation** via opaque fingerprints and derived schema namespaces
+- **Versioned migrations** in a package-owned schema namespace
+- **Normalized errors** without leaking DSNs, credentials, or raw backend text
+- **Lazy driver loading** so the base framework never imports `psycopg`
+
+## Development
+
+Install in editable mode from the repository root:
+
+```bash
+pip install -e packages/nl2data-workflow-postgres
+```
+
+Run package tests:
+
+```bash
+pytest packages/nl2data-workflow-postgres/tests -q
+```

nl2data_workflow_postgres-0.1.0/pyproject.toml

@@ -0,0 +1,56 @@
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "nl2data-workflow-postgres"
+version = "0.1.0"
+description = "PostgreSQL-backed durable workflow state backend for nl2data-core."
+readme = "README.md"
+requires-python = ">=3.11"
+license = { text = "Apache-2.0" }
+authors = [{ name = "NL2Data Contributors" }]
+keywords = ["nl2data", "postgresql", "workflow", "state", "lease", "idempotency"]
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Developers",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Topic :: Software Development :: Libraries",
+]
+dependencies = [
+    "nl2data-core>=0.1.0",
+    "psycopg[binary,pool]>=3.1,<4",
+]
+
+[project.optional-dependencies]
+dev = [
+    "pytest>=8.0",
+    "pytest-asyncio>=0.23",
+    "mypy>=1.10",
+    "ruff>=0.5",
+]
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.mypy]
+python_version = "3.11"
+files = ["src"]
+mypy_path = ["../../src"]
+check_untyped_defs = true
+disallow_untyped_defs = true
+no_implicit_optional = true
+warn_redundant_casts = true
+warn_unused_ignores = true
+warn_return_any = true
+
+[tool.ruff]
+line-length = 100
+target-version = "py311"
+src = ["src"]
+
+[tool.ruff.lint]
+select = ["E", "F", "I", "UP", "B", "SIM"]

nl2data_workflow_postgres-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

nl2data_workflow_postgres-0.1.0/src/nl2data_workflow_postgres/__init__.py

@@ -0,0 +1,25 @@
+"""PostgreSQL-backed durable workflow state backend for nl2data-core.
+
+The durable workflow runtime composes this backend through the replaceable
+store contract.  Optional dependencies (``psycopg``/``psycopg_pool``) are
+loaded lazily; importing this module never imports the driver.
+"""
+
+from __future__ import annotations
+
+from .client import build_pool, driver_available
+from .config import WorkflowPostgresConfig
+from .schema import MIGRATIONS, SUPPORTED_SCHEMA_VERSION
+from .store import SQL_TEMPLATES, PostgreSQLStateStore
+
+__version__ = "0.1.0"
+
+__all__ = [
+    "MIGRATIONS",
+    "SUPPORTED_SCHEMA_VERSION",
+    "SQL_TEMPLATES",
+    "PostgreSQLStateStore",
+    "WorkflowPostgresConfig",
+    "build_pool",
+    "driver_available",
+]

nl2data_workflow_postgres-0.1.0/src/nl2data_workflow_postgres/client.py

@@ -0,0 +1,138 @@
+"""Lazy optional psycopg driver boundary for the PostgreSQL workflow backend.
+
+The ``psycopg``/``psycopg_pool`` packages are loaded only inside this module
+through :func:`importlib.import_module`, so importing
+``nl2data_workflow_postgres`` or the store module never imports a database
+driver.  The store accepts an injected pool (fake or host-managed) or a DSN;
+the DSN path is constructed here with bounded connect/command timeouts and a
+bounded pool, and DSNs are never included in errors.  Driver errors are
+classified by class name first (so injected fake clients work without the
+driver installed) and by the real driver only when present.
+"""
+
+from __future__ import annotations
+
+from importlib import import_module
+from importlib.util import find_spec
+from typing import Any, cast
+
+from nl2data_core.workflow.shared_errors import SharedStoreError, SharedStoreErrorCode
+
+
+def driver_available() -> bool:
+    """Whether the optional ``psycopg`` driver (and pool) is installed."""
+    return find_spec("psycopg") is not None and find_spec("psycopg_pool") is not None
+
+
+def build_pool(
+    dsn: str,
+    *,
+    pool_size: int,
+    connect_timeout_seconds: float,
+    command_timeout_seconds: float,
+    acquire_timeout_seconds: float,
+    schema: str,
+) -> Any:
+    """Lazily import the driver and build a bounded PostgreSQL connection pool.
+
+    Every checkout is pinned to the configured schema namespace through the
+    connection ``options`` so all unqualified table names resolve inside the
+    deployment's own schema.  Raises a normalized ``STORE_UNAVAILABLE``
+    error when the driver is missing or the pool cannot be constructed; the
+    DSN and any driver exception text are never included in the error.
+    """
+    if not driver_available():
+        raise SharedStoreError(
+            SharedStoreErrorCode.STORE_UNAVAILABLE,
+            "the psycopg driver is not installed; install the 'postgres' extra",
+            details={"cause_type": "ImportError"},
+        )
+    try:
+        pool_module = cast(Any, import_module("psycopg_pool"))
+        driver = cast(Any, import_module("psycopg"))
+        pool = pool_module.ConnectionPool(
+            dsn,
+            min_size=1,
+            max_size=pool_size,
+            open=True,
+            timeout=acquire_timeout_seconds,
+            kwargs={
+                "connect_timeout": connect_timeout_seconds,
+                "options": f"-c search_path={schema}",
+                "row_factory": driver.rows.dict_row,
+            },
+        )
+    except SharedStoreError:
+        raise
+    except Exception as error:
+        raise SharedStoreError(
+            SharedStoreErrorCode.STORE_UNAVAILABLE,
+            "the PostgreSQL connection pool could not be constructed",
+            details={"cause_type": type(error).__name__},
+        ) from error
+    return pool
+
+
+def is_connect_error(error: BaseException) -> bool:
+    """Whether the exception signals a lost or unavailable connection."""
+    if error.__class__.__name__ in {
+        "OperationalError",
+        "InterfaceError",
+        "ConnectionError",
+        "PoolTimeout",
+        "PoolClosed",
+    }:
+        return True
+    try:
+        pool_module = cast(Any, import_module("psycopg_pool"))
+        if isinstance(error, pool_module.PoolTimeout):
+            return True
+    except ImportError:
+        pass
+    try:
+        exceptions = cast(Any, import_module("psycopg"))
+    except ImportError:
+        return False
+    return isinstance(
+        error, (exceptions.OperationalError, exceptions.InterfaceError)
+    )
+
+
+def is_timeout_error(error: BaseException) -> bool:
+    """Whether the exception signals a query canceled by its timeout."""
+    if error.__class__.__name__ in {
+        "TimeoutError",
+        "QueryCanceledError",
+        "QueryCanceled",
+        "Timeout",
+    }:
+        return True
+    try:
+        exceptions = cast(Any, import_module("psycopg.errors"))
+    except ImportError:
+        return False
+    return isinstance(error, exceptions.QueryCanceled)
+
+
+def is_duplicate_key_error(error: BaseException) -> bool:
+    """Whether the exception signals a unique-key violation."""
+    if error.__class__.__name__ in {"UniqueViolation", "IntegrityError"}:
+        return True
+    try:
+        exceptions = cast(Any, import_module("psycopg.errors"))
+    except ImportError:
+        return False
+    return isinstance(error, exceptions.UniqueViolation)
+
+
+def is_serialization_error(error: BaseException) -> bool:
+    """Whether the exception signals a retryable transaction conflict."""
+    if error.__class__.__name__ in {"SerializationFailure", "DeadlockDetected"}:
+        return True
+    try:
+        exceptions = cast(Any, import_module("psycopg.errors"))
+    except ImportError:
+        return False
+    return isinstance(
+        error, (exceptions.SerializationFailure, exceptions.DeadlockDetected)
+    )

nl2data_workflow_postgres-0.1.0/src/nl2data_workflow_postgres/config.py

@@ -0,0 +1,83 @@
+"""Bounded configuration for the PostgreSQL workflow state backend.
+
+The configuration carries only behavior bounds - never connection strings
+or credentials.  A DSN (when the host manages its own pool) is passed to the
+store constructor and is never part of the model, so a dumped configuration
+cannot leak PostgreSQL endpoints or secrets.  Every bound is validated at
+construction so an empty schema namespace, an unsafe pool/timeout/TTL value,
+or an impossible renewal margin fails before any client is built.
+"""
+
+from __future__ import annotations
+
+from pydantic import BaseModel, ConfigDict, Field, model_validator
+
+from .schema import SUPPORTED_SCHEMA_VERSION
+
+#: Schema namespace: bounded identifier so every derived table name stays safe.
+_SCHEMA_PATTERN = r"^[A-Za-z][A-Za-z0-9_]{0,63}$"
+
+#: Hard limits for the validated bounds below.
+_MAX_POOL_SIZE = 64
+_MAX_CONNECT_TIMEOUT_SECONDS = 30.0
+_MAX_COMMAND_TIMEOUT_SECONDS = 120.0
+_MAX_ACQUIRE_TIMEOUT_SECONDS = 60.0
+_MAX_CLEANUP_BATCH = 10_000
+_MAX_LEASE_TTL_SECONDS = 86_400.0  # one day
+_MAX_RENEWAL_MARGIN_SECONDS = 3_600.0
+_MAX_CLOCK_TOLERANCE_SECONDS = 60.0
+
+
+class WorkflowPostgresConfig(BaseModel):
+    """Immutable bounded configuration for the PostgreSQL workflow backend.
+
+    ``namespace`` is required and must be unique per application and
+    environment: it names the PostgreSQL schema that owns every workflow,
+    idempotency, and lease table, so two deployments sharing one database
+    service never observe each other's records.  All other fields bound the
+    store's work so a pathological pool, timeout, cleanup pass, or lease
+    timing can never produce unbounded behavior.
+    """
+
+    model_config = ConfigDict(frozen=True, extra="forbid")
+
+    #: PostgreSQL schema owning all shared records (never a raw tenant id).
+    namespace: str = Field(pattern=_SCHEMA_PATTERN)
+    #: Bounded connection pool size of the lazy psycopg pool.
+    pool_size: int = Field(default=5, ge=1, le=_MAX_POOL_SIZE)
+    #: Bounded connect timeout for lazy pool connections (seconds).
+    connect_timeout_seconds: float = Field(
+        default=5.0, ge=0.1, le=_MAX_CONNECT_TIMEOUT_SECONDS
+    )
+    #: Bounded per-command timeout for every statement (seconds).
+    command_timeout_seconds: float = Field(
+        default=10.0, ge=0.1, le=_MAX_COMMAND_TIMEOUT_SECONDS
+    )
+    #: Bounded pool checkout timeout (seconds).
+    pool_acquire_timeout_seconds: float = Field(
+        default=5.0, ge=0.1, le=_MAX_ACQUIRE_TIMEOUT_SECONDS
+    )
+    #: Maximum schema version this store operates on; newer schemas reject.
+    schema_version: int = Field(
+        default=SUPPORTED_SCHEMA_VERSION, ge=1, le=SUPPORTED_SCHEMA_VERSION
+    )
+    #: Maximum records removed by one bounded cleanup pass.
+    cleanup_batch_size: int = Field(default=500, ge=1, le=_MAX_CLEANUP_BATCH)
+    #: Default lease TTL applied when an acquire call omits ``ttl_seconds``.
+    lease_ttl_seconds: float = Field(default=120.0, ge=1.0, le=_MAX_LEASE_TTL_SECONDS)
+    #: Renewal is attempted when less than this margin remains on the lease.
+    lease_renewal_margin_seconds: float = Field(
+        default=20.0, ge=0.1, le=_MAX_RENEWAL_MARGIN_SECONDS
+    )
+    #: Conservative clock-skew tolerance applied to every server-time check.
+    clock_tolerance_seconds: float = Field(
+        default=2.0, ge=0.0, le=_MAX_CLOCK_TOLERANCE_SECONDS
+    )
+
+    @model_validator(mode="after")
+    def _consistent_lease_timing(self) -> WorkflowPostgresConfig:
+        if self.lease_renewal_margin_seconds >= self.lease_ttl_seconds:
+            raise ValueError(
+                "lease renewal margin must be smaller than the lease TTL"
+            )
+        return self