python-durable 0.1.0__tar.gz

python_durable-0.1.0/.github/workflows/publish.yml

@@ -0,0 +1,22 @@
+name: Publish to PyPI
+
+on:
+  release:
+    types: [published]
+
+jobs:
+  publish:
+    runs-on: ubuntu-latest
+    permissions:
+      id-token: write
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: astral-sh/setup-uv@v6
+
+      - run: uv build
+
+      - uses: astral-sh/setup-uv@v6
+        with:
+          enable-cache: false
+      - run: uv publish --trusted-publishing always

python_durable-0.1.0/.gitignore

@@ -0,0 +1,10 @@
+# Python-generated files
+__pycache__/
+*.py[oc]
+build/
+dist/
+wheels/
+*.egg-info
+
+# Virtual environments
+.venv

python_durable-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 python-durable contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

python_durable-0.1.0/PKG-INFO

@@ -0,0 +1,133 @@
+Metadata-Version: 2.4
+Name: python-durable
+Version: 0.1.0
+Summary: Lightweight workflow durability for Python — make any async workflow resumable after crashes with just a decorator.
+Project-URL: Repository, https://github.com/WillemDeGroef/python-durable
+Author: Willem
+License-Expression: MIT
+License-File: LICENSE
+Classifier: Development Status :: 3 - Alpha
+Classifier: Framework :: AsyncIO
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Typing :: Typed
+Requires-Python: >=3.12
+Requires-Dist: aiosqlite>=0.20
+Provides-Extra: dev
+Requires-Dist: pytest-asyncio>=0.24; extra == 'dev'
+Requires-Dist: pytest>=8.0; extra == 'dev'
+Requires-Dist: ruff>=0.9; extra == 'dev'
+Requires-Dist: ty>=0.0.1a7; extra == 'dev'
+Provides-Extra: examples
+Requires-Dist: pydantic-ai>=0.1; extra == 'examples'
+Requires-Dist: pydantic>=2.0; extra == 'examples'
+Description-Content-Type: text/markdown
+
+# durable
+
+Lightweight workflow durability for Python. Make any async workflow resumable after crashes with just a decorator.
+
+Backed by SQLite out of the box; swap in any `Store` subclass for production.
+
+## Install
+
+```bash
+pip install python-durable
+```
+
+## Quick start
+
+```python
+from durable import Workflow
+from durable.backoff import exponential
+
+wf = Workflow("my-app")
+
+@wf.task(retries=3, backoff=exponential(base=2, max=60))
+async def fetch_data(url: str) -> dict:
+    async with httpx.AsyncClient() as client:
+        return (await client.get(url)).json()
+
+@wf.task
+async def save_result(data: dict) -> None:
+    await db.insert(data)
+
+@wf.workflow(id="pipeline-{source}")
+async def run_pipeline(source: str) -> None:
+    data = await fetch_data(f"https://api.example.com/{source}")
+    await save_result(data)
+
+# First call: runs all steps and checkpoints each one.
+# If it crashes and you call it again with the same args,
+# completed steps are replayed from SQLite instantly.
+await run_pipeline(source="users")
+```
+
+## How it works
+
+1. **`@wf.task`** wraps an async function with checkpoint + retry logic. When called inside a workflow, results are persisted to the store. On re-run, completed steps return their cached result without re-executing.
+
+2. **`@wf.workflow`** marks the entry point of a durable run. It manages a `RunContext` (via `ContextVar`) so tasks automatically know which run they belong to. The `id` parameter is a template string resolved from function arguments at call time.
+
+3. **`Store`** is the persistence backend. `SQLiteStore` is the default (zero config, backed by aiosqlite). Subclass `Store` to use Postgres, Redis, or anything else.
+
+## Features
+
+- **Crash recovery** — completed steps are never re-executed after a restart
+- **Automatic retries** — configurable per-task with `exponential`, `linear`, or `constant` backoff
+- **Loop support** — use `step_id` to checkpoint each iteration independently
+- **Zero magic outside workflows** — tasks work as plain async functions when called without a workflow context
+- **Pluggable storage** — SQLite by default, bring your own `Store` for production
+
+## Backoff strategies
+
+```python
+from durable.backoff import exponential, linear, constant
+
+@wf.task(retries=5, backoff=exponential(base=2, max=60))  # 2s, 4s, 8s, 16s, 32s
+async def exp_task(): ...
+
+@wf.task(retries=3, backoff=linear(start=2, step=3))      # 2s, 5s, 8s
+async def linear_task(): ...
+
+@wf.task(retries=3, backoff=constant(5))                   # 5s, 5s, 5s
+async def const_task(): ...
+```
+
+## Loops with step_id
+
+When calling the same task in a loop, pass `step_id` so each iteration is checkpointed independently:
+
+```python
+@wf.workflow(id="batch-{batch_id}")
+async def process_batch(batch_id: str) -> None:
+    for i, item in enumerate(items):
+        await process_item(item, step_id=f"item-{i}")
+```
+
+If the workflow crashes mid-loop, only the remaining items are processed on restart.
+
+## Important: JSON serialization
+
+Task return values must be JSON-serializable (dicts, lists, strings, numbers, booleans, `None`). The store uses `json.dumps` internally.
+
+For Pydantic models, return `.model_dump()` from tasks and reconstruct with `.model_validate()` downstream:
+
+```python
+@wf.task
+async def validate_invoice(draft: InvoiceDraft) -> dict:
+    validated = ValidatedInvoice(...)
+    return validated.model_dump()
+
+@wf.task
+async def book_invoice(data: dict) -> dict:
+    invoice = ValidatedInvoice.model_validate(data)
+    ...
+```
+
+## License
+
+MIT

python_durable-0.1.0/README.md

@@ -0,0 +1,105 @@
+# durable
+
+Lightweight workflow durability for Python. Make any async workflow resumable after crashes with just a decorator.
+
+Backed by SQLite out of the box; swap in any `Store` subclass for production.
+
+## Install
+
+```bash
+pip install python-durable
+```
+
+## Quick start
+
+```python
+from durable import Workflow
+from durable.backoff import exponential
+
+wf = Workflow("my-app")
+
+@wf.task(retries=3, backoff=exponential(base=2, max=60))
+async def fetch_data(url: str) -> dict:
+    async with httpx.AsyncClient() as client:
+        return (await client.get(url)).json()
+
+@wf.task
+async def save_result(data: dict) -> None:
+    await db.insert(data)
+
+@wf.workflow(id="pipeline-{source}")
+async def run_pipeline(source: str) -> None:
+    data = await fetch_data(f"https://api.example.com/{source}")
+    await save_result(data)
+
+# First call: runs all steps and checkpoints each one.
+# If it crashes and you call it again with the same args,
+# completed steps are replayed from SQLite instantly.
+await run_pipeline(source="users")
+```
+
+## How it works
+
+1. **`@wf.task`** wraps an async function with checkpoint + retry logic. When called inside a workflow, results are persisted to the store. On re-run, completed steps return their cached result without re-executing.
+
+2. **`@wf.workflow`** marks the entry point of a durable run. It manages a `RunContext` (via `ContextVar`) so tasks automatically know which run they belong to. The `id` parameter is a template string resolved from function arguments at call time.
+
+3. **`Store`** is the persistence backend. `SQLiteStore` is the default (zero config, backed by aiosqlite). Subclass `Store` to use Postgres, Redis, or anything else.
+
+## Features
+
+- **Crash recovery** — completed steps are never re-executed after a restart
+- **Automatic retries** — configurable per-task with `exponential`, `linear`, or `constant` backoff
+- **Loop support** — use `step_id` to checkpoint each iteration independently
+- **Zero magic outside workflows** — tasks work as plain async functions when called without a workflow context
+- **Pluggable storage** — SQLite by default, bring your own `Store` for production
+
+## Backoff strategies
+
+```python
+from durable.backoff import exponential, linear, constant
+
+@wf.task(retries=5, backoff=exponential(base=2, max=60))  # 2s, 4s, 8s, 16s, 32s
+async def exp_task(): ...
+
+@wf.task(retries=3, backoff=linear(start=2, step=3))      # 2s, 5s, 8s
+async def linear_task(): ...
+
+@wf.task(retries=3, backoff=constant(5))                   # 5s, 5s, 5s
+async def const_task(): ...
+```
+
+## Loops with step_id
+
+When calling the same task in a loop, pass `step_id` so each iteration is checkpointed independently:
+
+```python
+@wf.workflow(id="batch-{batch_id}")
+async def process_batch(batch_id: str) -> None:
+    for i, item in enumerate(items):
+        await process_item(item, step_id=f"item-{i}")
+```
+
+If the workflow crashes mid-loop, only the remaining items are processed on restart.
+
+## Important: JSON serialization
+
+Task return values must be JSON-serializable (dicts, lists, strings, numbers, booleans, `None`). The store uses `json.dumps` internally.
+
+For Pydantic models, return `.model_dump()` from tasks and reconstruct with `.model_validate()` downstream:
+
+```python
+@wf.task
+async def validate_invoice(draft: InvoiceDraft) -> dict:
+    validated = ValidatedInvoice(...)
+    return validated.model_dump()
+
+@wf.task
+async def book_invoice(data: dict) -> dict:
+    invoice = ValidatedInvoice.model_validate(data)
+    ...
+```
+
+## License
+
+MIT

python_durable-0.1.0/examples/examples.py

@@ -0,0 +1,267 @@
+"""
+examples.py — durable library usage examples.
+
+Run with: python examples.py
+"""
+
+import asyncio
+from dataclasses import dataclass
+from enum import StrEnum
+
+from pydantic import BaseModel
+
+from durable import Workflow
+from durable.backoff import constant, exponential, linear
+
+# ---------------------------------------------------------------------------
+# Setup
+# ---------------------------------------------------------------------------
+
+wf = Workflow("examples", db="sqlite:///examples.db")
+
+
+# ---------------------------------------------------------------------------
+# Example 1 — Basic pipeline
+# ---------------------------------------------------------------------------
+
+
+@dataclass
+class User:
+    id: str
+    email: str
+    name: str
+
+
+@dataclass
+class Invoice:
+    user_id: str
+    amount: float
+    items: list[str]
+
+
+@wf.task
+async def fetch_user(user_id: str) -> dict:
+    print(f"  [fetch_user] fetching user {user_id}...")
+    await asyncio.sleep(0.1)  # simulate I/O
+    return {"id": user_id, "email": f"{user_id}@example.com", "name": "Alice"}
+
+
+@wf.task
+async def build_invoice(user: dict) -> dict:
+    print(f"  [build_invoice] building for {user['name']}...")
+    await asyncio.sleep(0.1)
+    return {"user_id": user["id"], "amount": 49.99, "items": ["Pro Plan"]}
+
+
+@wf.task
+async def send_email(user: dict, invoice: dict) -> None:
+    print(f"  [send_email] sending to {user['email']} for ${invoice['amount']}")
+    await asyncio.sleep(0.1)
+
+
+@wf.workflow(id="process-order-{order_id}")
+async def process_order(order_id: str) -> None:
+    user = await fetch_user(order_id)
+    invoice = await build_invoice(user)
+    await send_email(user, invoice)
+
+
+# ---------------------------------------------------------------------------
+# Example 2 — Flaky API with aggressive retries
+# ---------------------------------------------------------------------------
+
+_call_count = 0
+
+
+@wf.task(retries=5, backoff=exponential(base=2, max=30))
+async def call_flaky_api(endpoint: str) -> dict:
+    global _call_count
+    _call_count += 1
+    if _call_count < 3:
+        raise ConnectionError(f"API down (attempt {_call_count})")
+    print(f"  [call_flaky_api] succeeded on attempt {_call_count}")
+    return {"status": "ok", "data": [1, 2, 3]}
+
+
+@wf.workflow(id="flaky-{job_id}")
+async def flaky_pipeline(job_id: str) -> dict:
+    return await call_flaky_api("/data")
+
+
+# ---------------------------------------------------------------------------
+# Example 3 — Loop with explicit step_id
+# ---------------------------------------------------------------------------
+
+
+@wf.task(retries=2, backoff=constant(1))
+async def push_record(record: dict) -> bool:
+    print(f"  [push_record] pushing {record['id']}...")
+    await asyncio.sleep(0.05)
+    return True
+
+
+@wf.task
+async def post_summary(count: int) -> None:
+    print(f"  [post_summary] all done — pushed {count} records")
+
+
+@wf.workflow(id="crm-sync-{batch_id}")
+async def sync_to_crm(batch_id: str) -> None:
+    # Records loaded with plain Python — not every function needs to be a task
+    records = [{"id": f"rec-{i}", "value": i * 10} for i in range(5)]
+
+    for i, record in enumerate(records):
+        # step_id disambiguates repeated calls to the same task in a loop.
+        # If the workflow crashes mid-loop, only the remaining records are pushed.
+        await push_record(record, step_id=f"push-record-{i}")
+
+    await post_summary(len(records))
+
+
+# ---------------------------------------------------------------------------
+# Example 4 — Multiple backoff strategies side by side
+# ---------------------------------------------------------------------------
+
+
+@wf.task(retries=3, backoff=exponential(base=2, max=60))  # 2s, 4s, 8s
+async def sync_to_warehouse(row: dict) -> None:
+    print(f"  [sync_to_warehouse] writing row {row['id']}")
+
+
+@wf.task(retries=3, backoff=linear(start=2, step=3))  # 2s, 5s, 8s
+async def notify_slack(message: str) -> None:
+    print(f"  [notify_slack] {message}")
+
+
+@wf.task(retries=5, backoff=constant(5))  # always 5s
+async def send_sms(number: str, body: str) -> None:
+    print(f"  [send_sms] → {number}: {body}")
+
+
+# ---------------------------------------------------------------------------
+# Example 5 — Explicit run ID via .run()
+# ---------------------------------------------------------------------------
+
+
+@wf.workflow(id="report-{date}")
+async def generate_report(date: str) -> dict:
+    data = await fetch_user("analyst-1")  # reusing tasks across workflows!
+    return {"generated_for": date, "by": data["name"]}
+
+
+# ---------------------------------------------------------------------------
+# Example 6 — Pydantic models as task parameters
+#
+# The checkpoint store serializes results with json.dumps / json.loads, so
+# task return values must be JSON-serializable (dicts, lists, primitives).
+#
+# Pattern: tasks return model.model_dump(), downstream tasks reconstruct
+# with Model.model_validate(). This keeps checkpoints portable and lets
+# workflows resume cleanly after a crash.
+# ---------------------------------------------------------------------------
+
+
+class LineItem(BaseModel):
+    description: str
+    quantity: int
+    unit_price: float
+
+
+class Currency(StrEnum):
+    EUR = "EUR"
+    USD = "USD"
+
+
+class InvoiceDraft(BaseModel):
+    customer_name: str
+    currency: Currency
+    lines: list[LineItem]
+
+
+class ValidatedInvoice(BaseModel):
+    customer_name: str
+    currency: Currency
+    lines: list[LineItem]
+    total: float
+    reference: str
+
+
+@wf.task
+async def validate_invoice(draft: InvoiceDraft) -> dict:
+    """Accept a Pydantic model, return a dict for checkpoint storage."""
+    print(
+        f"  [validate_invoice] validating {len(draft.lines)} line(s) for {draft.customer_name}"
+    )
+    total = sum(line.quantity * line.unit_price for line in draft.lines)
+    validated = ValidatedInvoice(
+        customer_name=draft.customer_name,
+        currency=draft.currency,
+        lines=draft.lines,
+        total=round(total, 2),
+        reference=f"INV-{hash(draft.customer_name) % 10000:04d}",
+    )
+    # .model_dump() → plain dict that json.dumps can serialize
+    return validated.model_dump()
+
+
+@wf.task
+async def book_invoice(invoice_data: dict) -> dict:
+    """Reconstruct the Pydantic model from the checkpointed dict."""
+    invoice = ValidatedInvoice.model_validate(invoice_data)
+    print(
+        f"  [book_invoice] booking {invoice.reference}: "
+        f"{invoice.currency} {invoice.total:.2f} for {invoice.customer_name}"
+    )
+    return {"reference": invoice.reference, "status": "booked"}
+
+
+@wf.workflow(id="invoice-{customer}")
+async def process_invoice(customer: str, draft: InvoiceDraft) -> dict:
+    validated = await validate_invoice(draft)
+    return await book_invoice(validated)
+
+
+# ---------------------------------------------------------------------------
+# Demo runner
+# ---------------------------------------------------------------------------
+
+
+async def main():
+    print("\n── Example 1: Basic order pipeline ──")
+    await process_order(order_id="ord-42")
+    print("  (run again → all steps replayed from cache)")
+    await process_order(order_id="ord-42")
+
+    print("\n── Example 2: Flaky API with retries ──")
+    result = await flaky_pipeline(job_id="job-1")
+    print(f"  result: {result}")
+
+    print("\n── Example 3: Loop with step_id ──")
+    await sync_to_crm(batch_id="batch-2024-01")
+    print("  (run again → all records skipped, already pushed)")
+    await sync_to_crm(batch_id="batch-2024-01")
+
+    print("\n── Example 4: Explicit run ID ──")
+    report = await generate_report.run("weekly-report-custom-id", date="2024-01-15")
+    print(f"  report: {report}")
+
+    print("\n── Example 6: Pydantic models as task params ──")
+    draft = InvoiceDraft(
+        customer_name="Acme Corp",
+        currency=Currency.EUR,
+        lines=[
+            LineItem(description="Consulting", quantity=10, unit_price=150.0),
+            LineItem(description="Travel expenses", quantity=1, unit_price=340.50),
+        ],
+    )
+    booking = await process_invoice(customer="acme", draft=draft)
+    print(f"  result: {booking}")
+    print("  (run again → all steps replayed from cache)")
+    booking = await process_invoice(customer="acme", draft=draft)
+    print(f"  result: {booking}")
+
+    print("\n✓ All examples complete. Check examples.db to see the checkpoint store.")
+
+
+if __name__ == "__main__":
+    asyncio.run(main())

python_durable-0.1.0/examples/in_memory_example.py

@@ -0,0 +1,59 @@
+"""
+in_memory_example.py — Using InMemoryStore for testing and ephemeral workflows.
+
+No files, no SQLite — just a dict. Checkpoints live for the duration of the
+process, so retries and deduplication work within a single run, but nothing
+survives a restart.
+
+Run with:
+    uv run python examples/in_memory_example.py
+"""
+
+import asyncio
+
+from durable import Workflow
+from durable.backoff import constant
+from durable.store import InMemoryStore
+
+wf = Workflow("demo", db=InMemoryStore())
+
+call_count = 0
+
+
+@wf.task(retries=3, backoff=constant(0))
+async def flaky_fetch(url: str) -> dict:
+    """Fails twice, then succeeds — retries are handled automatically."""
+    global call_count
+    call_count += 1
+    if call_count < 3:
+        raise ConnectionError(f"attempt {call_count}: connection refused")
+    print(f"  [flaky_fetch] succeeded on attempt {call_count}")
+    return {"url": url, "status": "ok"}
+
+
+@wf.task
+async def transform(data: dict) -> str:
+    print(f"  [transform] processing {data['url']}")
+    return f"transformed-{data['status']}"
+
+
+@wf.workflow(id="pipeline-{job_id}")
+async def pipeline(job_id: str) -> str:
+    data = await flaky_fetch("https://api.example.com/data")
+    return await transform(data)
+
+
+async def main():
+    print("── First run: flaky_fetch retries, then both steps execute ──")
+    result = await pipeline(job_id="job-1")
+    print(f"  result: {result}")
+
+    print("\n── Second run (same id): both steps replayed from memory ──")
+    result = await pipeline(job_id="job-1")
+    print(f"  result: {result}")
+
+    print("\n✓ Done. Nothing written to disk.")
+
+
+if __name__ == "__main__":
+    asyncio.run(main())