durable-worker 0.1.0__tar.gz

durable_worker-0.1.0/.gitignore

@@ -0,0 +1,15 @@
+node_modules/
+dist/
+coverage/
+.turbo/
+*.log
+.DS_Store
+.env
+*.tsbuildinfo
+__pycache__/
+*.pyc
+.pytest_cache/
+
+# prisma adapter generated client + sqlite test db
+packages/store-prisma/generated/
+packages/store-prisma/prisma/*.db

durable_worker-0.1.0/PKG-INFO

@@ -0,0 +1,115 @@
+Metadata-Version: 2.4
+Name: durable-worker
+Version: 0.1.0
+Summary: Python worker SDK for nestjs-durable — run durable workflow steps in Python.
+Project-URL: Homepage, https://github.com/DavideCarvalho/nestjs-durable
+Project-URL: Repository, https://github.com/DavideCarvalho/nestjs-durable
+Project-URL: Issues, https://github.com/DavideCarvalho/nestjs-durable/issues
+Author-email: Davide Carvalho <davi@goflip.ai>
+License: MIT
+Keywords: durable,nestjs,worker,workflow
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Software Development :: Libraries
+Classifier: Typing :: Typed
+Requires-Python: >=3.9
+Provides-Extra: redis
+Requires-Dist: bullmq>=2.0; extra == 'redis'
+Description-Content-Type: text/markdown
+
+# durable-worker (Python)
+
+Run [`nestjs-durable`](../../README.md) workflow steps in Python. A TypeScript workflow calls a
+remote step with `ctx.call(chargeCard, input)`; the orchestrator dispatches it over the
+transport; a Python worker registered for the same step **name** runs it and returns the result.
+One workflow, steps split across languages.
+
+```python
+from durable_worker import Worker, FatalError
+
+worker = Worker(group="payments")
+
+@worker.step("payments.charge-card")
+async def charge(data):
+    res = await stripe.charge(data["orderId"], data["amountCents"])
+    return {"chargeId": res.id}
+
+# worker.run(transport=...)  # see "Transports" below
+```
+
+The handler's argument is the step **input** (already schema-validated by the engine); its
+return value is the step **output**. Raise `FatalError` for a non-retryable failure (e.g. a
+declined card); any other exception is treated as retryable and the engine applies the step's
+retry policy.
+
+## Wire protocol
+
+The contract between the orchestrator and a worker is plain JSON — language-agnostic, so a Go or
+Rust worker can implement the same thing. The orchestrator dispatches a **task**:
+
+```jsonc
+{
+  "runId":   "wrun_8Kb2",            // the workflow run
+  "seq":     1,                       // deterministic step position
+  "name":    "payments.charge-card",  // handler name (the contract)
+  "stepId":  "wrun_8Kb2:1",           // stable id — use it to dedupe re-delivery
+  "group":   "payments",              // worker group expected to handle it
+  "input":   { "orderId": "o1", "amountCents": 4200 },
+  "attempt": 1,
+  "traceparent": "00-..."             // optional W3C trace context to continue the span
+}
+```
+
+The worker replies with a **result**:
+
+```jsonc
+// success
+{ "runId": "wrun_8Kb2", "seq": 1, "stepId": "wrun_8Kb2:1", "status": "completed", "output": { "chargeId": "ch_1" } }
+// failure
+{ "runId": "wrun_8Kb2", "seq": 1, "stepId": "wrun_8Kb2:1", "status": "failed",
+  "error": { "message": "card declined", "code": "declined", "retryable": false } }
+```
+
+`Worker.process_task(task) -> result` is the pure core (no transport, fully tested). Idempotency
+note: if the worker dies after running but before the result is recorded, the engine may
+re-dispatch the same `stepId` — make handlers idempotent or dedupe on `stepId`.
+
+## Transports
+
+`process_task` is transport-agnostic. A transport adapter consumes tasks from the broker and
+ships results back:
+
+- **Redis / BullMQ** (`pip install durable-worker[redis]`) — `durable_worker.redis_runner`
+  consumes the same Redis queues `@dudousxd/nestjs-durable-transport-bullmq` dispatches to:
+
+  ```python
+  import asyncio
+  from durable_worker import Worker
+  from durable_worker.redis_runner import run_redis_worker
+
+  worker = Worker(group="payments")
+
+  @worker.step("payments.charge-card")
+  async def charge(data):
+      return {"chargeId": f"ch_{data['amount']}"}
+
+  async def main():
+      await run_redis_worker(worker, group="payments")
+      await asyncio.Event().wait()
+
+  asyncio.run(main())
+  ```
+
+  This is wired end-to-end in [`scripts/py-e2e.sh`](../../scripts/py-e2e.sh): a TypeScript
+  workflow's `ctx.call` runs this Python handler over Redis and gets the result back.
+- Bring your own: anything that can deliver a task dict and accept a result dict.
+
+## Tests
+
+```bash
+python -m unittest discover -s tests
+```

durable_worker-0.1.0/README.md

@@ -0,0 +1,92 @@
+# durable-worker (Python)
+
+Run [`nestjs-durable`](../../README.md) workflow steps in Python. A TypeScript workflow calls a
+remote step with `ctx.call(chargeCard, input)`; the orchestrator dispatches it over the
+transport; a Python worker registered for the same step **name** runs it and returns the result.
+One workflow, steps split across languages.
+
+```python
+from durable_worker import Worker, FatalError
+
+worker = Worker(group="payments")
+
+@worker.step("payments.charge-card")
+async def charge(data):
+    res = await stripe.charge(data["orderId"], data["amountCents"])
+    return {"chargeId": res.id}
+
+# worker.run(transport=...)  # see "Transports" below
+```
+
+The handler's argument is the step **input** (already schema-validated by the engine); its
+return value is the step **output**. Raise `FatalError` for a non-retryable failure (e.g. a
+declined card); any other exception is treated as retryable and the engine applies the step's
+retry policy.
+
+## Wire protocol
+
+The contract between the orchestrator and a worker is plain JSON — language-agnostic, so a Go or
+Rust worker can implement the same thing. The orchestrator dispatches a **task**:
+
+```jsonc
+{
+  "runId":   "wrun_8Kb2",            // the workflow run
+  "seq":     1,                       // deterministic step position
+  "name":    "payments.charge-card",  // handler name (the contract)
+  "stepId":  "wrun_8Kb2:1",           // stable id — use it to dedupe re-delivery
+  "group":   "payments",              // worker group expected to handle it
+  "input":   { "orderId": "o1", "amountCents": 4200 },
+  "attempt": 1,
+  "traceparent": "00-..."             // optional W3C trace context to continue the span
+}
+```
+
+The worker replies with a **result**:
+
+```jsonc
+// success
+{ "runId": "wrun_8Kb2", "seq": 1, "stepId": "wrun_8Kb2:1", "status": "completed", "output": { "chargeId": "ch_1" } }
+// failure
+{ "runId": "wrun_8Kb2", "seq": 1, "stepId": "wrun_8Kb2:1", "status": "failed",
+  "error": { "message": "card declined", "code": "declined", "retryable": false } }
+```
+
+`Worker.process_task(task) -> result` is the pure core (no transport, fully tested). Idempotency
+note: if the worker dies after running but before the result is recorded, the engine may
+re-dispatch the same `stepId` — make handlers idempotent or dedupe on `stepId`.
+
+## Transports
+
+`process_task` is transport-agnostic. A transport adapter consumes tasks from the broker and
+ships results back:
+
+- **Redis / BullMQ** (`pip install durable-worker[redis]`) — `durable_worker.redis_runner`
+  consumes the same Redis queues `@dudousxd/nestjs-durable-transport-bullmq` dispatches to:
+
+  ```python
+  import asyncio
+  from durable_worker import Worker
+  from durable_worker.redis_runner import run_redis_worker
+
+  worker = Worker(group="payments")
+
+  @worker.step("payments.charge-card")
+  async def charge(data):
+      return {"chargeId": f"ch_{data['amount']}"}
+
+  async def main():
+      await run_redis_worker(worker, group="payments")
+      await asyncio.Event().wait()
+
+  asyncio.run(main())
+  ```
+
+  This is wired end-to-end in [`scripts/py-e2e.sh`](../../scripts/py-e2e.sh): a TypeScript
+  workflow's `ctx.call` runs this Python handler over Redis and gets the result back.
+- Bring your own: anything that can deliver a task dict and accept a result dict.
+
+## Tests
+
+```bash
+python -m unittest discover -s tests
+```

durable_worker-0.1.0/durable_worker/__init__.py

@@ -0,0 +1,11 @@
+"""durable-worker — Python SDK for running nestjs-durable remote steps.
+
+A worker registers step handlers by name and processes tasks dispatched by the orchestrator.
+The wire protocol (task in, result out) is plain JSON and identical across languages, so the
+same step name implemented here is callable from a TypeScript workflow via ``ctx.call``.
+"""
+
+from .worker import FatalError, Worker
+
+__all__ = ["Worker", "FatalError"]
+__version__ = "0.1.0"

durable_worker-0.1.0/durable_worker/redis_runner.py

@@ -0,0 +1,42 @@
+"""Run a :class:`Worker` against the BullMQ/Redis transport.
+
+Consumes the orchestrator's per-group tasks queue and publishes results on the shared results
+queue — the same queues a TypeScript ``BullMQTransport`` uses, so steps interoperate across
+languages. Requires the optional ``bullmq`` extra: ``pip install durable-worker[redis]``.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from .worker import Worker
+
+
+def _names(prefix: str, group: str) -> tuple[str, str]:
+    # Must match the TS BullMQTransport: '<prefix>-tasks-<group>' and '<prefix>-results'.
+    return f"{prefix}-tasks-{group}", f"{prefix}-results"
+
+
+async def run_redis_worker(
+    worker: Worker,
+    *,
+    group: str,
+    connection: str = "redis://localhost:6379",
+    prefix: str = "durable",
+) -> Any:
+    """Start a BullMQ worker that runs ``worker``'s handlers. Returns the bullmq Worker.
+
+    The returned worker runs in the background; ``await worker.close()`` to stop it.
+    """
+
+    from bullmq import Queue as BullQueue  # imported lazily so the SDK works without bullmq
+    from bullmq import Worker as BullWorker
+
+    tasks_name, results_name = _names(prefix, group)
+    results = BullQueue(results_name, {"connection": connection})
+
+    async def process(job: Any, _token: str) -> None:
+        result = await worker.aprocess_task(job.data)
+        await results.add("result", result, {"removeOnComplete": True, "removeOnFail": True})
+
+    return BullWorker(tasks_name, process, {"connection": connection})

durable_worker-0.1.0/durable_worker/worker.py

@@ -0,0 +1,108 @@
+"""Core worker: a name->handler registry and the pure task->result dispatch.
+
+Transport (Redis/BullMQ/NATS) is intentionally separate — `process_task` is a pure function of
+the task, so it is fully testable without any broker. A transport adapter just feeds tasks in
+and ships results out.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import inspect
+from typing import Any, Awaitable, Callable, Dict, Union
+
+Handler = Callable[[Any], Union[Any, Awaitable[Any]]]
+
+
+class FatalError(Exception):
+    """Raise inside a handler to signal a non-retryable failure (mirrors the TS ``FatalError``).
+
+    The engine will not retry the step regardless of its ``retries`` setting.
+    """
+
+    def __init__(self, message: str, code: str | None = None) -> None:
+        super().__init__(message)
+        self.code = code
+
+
+class Worker:
+    """Registers step handlers by name and turns a dispatched task into a result.
+
+    Example::
+
+        worker = Worker(group="payments")
+
+        @worker.step("payments.charge-card")
+        async def charge(data):
+            res = await stripe.charge(data["orderId"], data["amountCents"])
+            return {"chargeId": res.id}
+    """
+
+    def __init__(self, group: str = "default") -> None:
+        self.group = group
+        self._handlers: Dict[str, Handler] = {}
+
+    def step(self, name: str) -> Callable[[Handler], Handler]:
+        """Decorator registering ``fn`` as the handler for step ``name``."""
+
+        def register(fn: Handler) -> Handler:
+            self._handlers[name] = fn
+            return fn
+
+        return register
+
+    def handles(self, name: str) -> bool:
+        return name in self._handlers
+
+    def process_task(self, task: Dict[str, Any]) -> Dict[str, Any]:
+        """Run the handler for ``task`` and return a wire-format result.
+
+        Pure and synchronous from the caller's view (async handlers are awaited internally), so a
+        transport can simply ``result = worker.process_task(task); send(result)``.
+        """
+
+        base = {"runId": task["runId"], "seq": task["seq"], "stepId": task["stepId"]}
+        handler = self._handlers.get(task["name"])
+        if handler is None:
+            return self._no_handler(base, task["name"])
+        try:
+            output = handler(task.get("input"))
+            if inspect.isawaitable(output):
+                output = asyncio.run(output)
+            return {**base, "status": "completed", "output": output}
+        except Exception as err:  # noqa: BLE001
+            return self._failure(base, err)
+
+    async def aprocess_task(self, task: Dict[str, Any]) -> Dict[str, Any]:
+        """Async variant — awaits async handlers in the current loop. Use from a transport that
+        already runs inside an event loop (e.g. the BullMQ runner)."""
+
+        base = {"runId": task["runId"], "seq": task["seq"], "stepId": task["stepId"]}
+        handler = self._handlers.get(task["name"])
+        if handler is None:
+            return self._no_handler(base, task["name"])
+        try:
+            output = handler(task.get("input"))
+            if inspect.isawaitable(output):
+                output = await output
+            return {**base, "status": "completed", "output": output}
+        except Exception as err:  # noqa: BLE001
+            return self._failure(base, err)
+
+    @staticmethod
+    def _no_handler(base: Dict[str, Any], name: str) -> Dict[str, Any]:
+        return {
+            **base,
+            "status": "failed",
+            "error": {"message": f"no handler for {name}", "retryable": False},
+        }
+
+    @staticmethod
+    def _failure(base: Dict[str, Any], err: Exception) -> Dict[str, Any]:
+        if isinstance(err, FatalError):
+            return {
+                **base,
+                "status": "failed",
+                "error": {"message": str(err), "code": err.code, "retryable": False},
+            }
+        return {**base, "status": "failed", "error": {"message": str(err)}}

durable_worker-0.1.0/examples/run_worker.py

@@ -0,0 +1,31 @@
+"""Run a Python worker that handles `payments.charge-card` over BullMQ/Redis.
+
+    PYTHONPATH=clients/python python3 clients/python/examples/run_worker.py <prefix>
+
+It consumes the same Redis queues a TypeScript `BullMQTransport` dispatches to, so a TS workflow
+can `ctx.call` this Python step. Returns a chargeId prefixed `ch_py_` to prove it ran in Python.
+"""
+
+import asyncio
+import sys
+
+from durable_worker import Worker
+from durable_worker.redis_runner import run_redis_worker
+
+worker = Worker(group="payments")
+
+
+@worker.step("payments.charge-card")
+async def charge(data):
+    return {"chargeId": f"ch_py_{data['amount']}"}
+
+
+async def main():
+    prefix = sys.argv[1] if len(sys.argv) > 1 else "durable"
+    await run_redis_worker(worker, group="payments", prefix=prefix)
+    print(f"[python worker] consuming {prefix}-tasks-payments", flush=True)
+    await asyncio.Event().wait()  # run until killed
+
+
+if __name__ == "__main__":
+    asyncio.run(main())

durable_worker-0.1.0/pyproject.toml

@@ -0,0 +1,34 @@
+[project]
+name = "durable-worker"
+version = "0.1.0"
+description = "Python worker SDK for nestjs-durable — run durable workflow steps in Python."
+readme = "README.md"
+requires-python = ">=3.9"
+license = { text = "MIT" }
+authors = [{ name = "Davide Carvalho", email = "davi@goflip.ai" }]
+keywords = ["workflow", "durable", "nestjs", "worker"]
+classifiers = [
+  "Development Status :: 3 - Alpha",
+  "Intended Audience :: Developers",
+  "License :: OSI Approved :: MIT License",
+  "Programming Language :: Python :: 3",
+  "Programming Language :: Python :: 3.9",
+  "Programming Language :: Python :: 3.12",
+  "Topic :: Software Development :: Libraries",
+  "Typing :: Typed",
+]
+
+[project.urls]
+Homepage = "https://github.com/DavideCarvalho/nestjs-durable"
+Repository = "https://github.com/DavideCarvalho/nestjs-durable"
+Issues = "https://github.com/DavideCarvalho/nestjs-durable/issues"
+
+[project.optional-dependencies]
+redis = ["bullmq>=2.0"]
+
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[tool.hatch.build.targets.wheel]
+packages = ["durable_worker"]

durable_worker-0.1.0/tests/test_worker.py

@@ -0,0 +1,73 @@
+import unittest
+
+from durable_worker import FatalError, Worker
+
+
+def task(name="payments.charge-card", **over):
+    base = {
+        "runId": "r1",
+        "seq": 0,
+        "name": name,
+        "stepId": "r1:0",
+        "group": "payments",
+        "input": {"orderId": "o1", "amountCents": 4200},
+        "attempt": 1,
+    }
+    base.update(over)
+    return base
+
+
+class WorkerTest(unittest.TestCase):
+    def test_runs_a_registered_sync_handler(self):
+        worker = Worker(group="payments")
+
+        @worker.step("payments.charge-card")
+        def charge(data):
+            return {"chargeId": f"ch_{data['orderId']}"}
+
+        result = worker.process_task(task())
+        self.assertEqual(result["status"], "completed")
+        self.assertEqual(result["output"], {"chargeId": "ch_o1"})
+        self.assertEqual(result["stepId"], "r1:0")
+
+    def test_awaits_async_handlers(self):
+        worker = Worker()
+
+        @worker.step("payments.charge-card")
+        async def charge(data):
+            return {"chargeId": "ch_async"}
+
+        self.assertEqual(worker.process_task(task())["output"], {"chargeId": "ch_async"})
+
+    def test_unknown_step_is_a_non_retryable_failure(self):
+        result = Worker().process_task(task(name="missing"))
+        self.assertEqual(result["status"], "failed")
+        self.assertFalse(result["error"]["retryable"])
+
+    def test_handler_exception_is_a_retryable_failure(self):
+        worker = Worker()
+
+        @worker.step("payments.charge-card")
+        def charge(_data):
+            raise RuntimeError("network blip")
+
+        result = worker.process_task(task())
+        self.assertEqual(result["status"], "failed")
+        self.assertEqual(result["error"]["message"], "network blip")
+        self.assertNotIn("retryable", result["error"])
+
+    def test_fatal_error_is_not_retryable(self):
+        worker = Worker()
+
+        @worker.step("payments.charge-card")
+        def charge(_data):
+            raise FatalError("card declined", code="declined")
+
+        result = worker.process_task(task())
+        self.assertEqual(result["status"], "failed")
+        self.assertFalse(result["error"]["retryable"])
+        self.assertEqual(result["error"]["code"], "declined")
+
+
+if __name__ == "__main__":
+    unittest.main()