pyworkflow-engine 0.1.7__py3-none-any.whl → 0.1.10__py3-none-any.whl

docs/concepts/sleep.mdx

@@ -1,312 +0,0 @@
----
-title: 'Sleep'
-description: 'Pause workflows for any duration without consuming resources'
----
-
-## What is Sleep?
-
-The `sleep()` primitive pauses a workflow for a specified duration. Unlike traditional sleep that blocks a thread, PyWorkflow's sleep **suspends** the workflow completely - no resources are consumed during the sleep period.
-
-```python
-from pyworkflow import workflow, sleep
-
-@workflow()
-async def reminder_sequence(user_id: str):
-    await send_reminder(user_id, "First reminder")
-
-    # Workflow suspends here - zero resources used
-    await sleep("1d")
-
-    # Resumes automatically after 1 day
-    await send_reminder(user_id, "Second reminder")
-
-    await sleep("7d")
-
-    await send_reminder(user_id, "Final reminder")
-```
-
-## How It Works
-
-```
-Workflow Execution
-        │
-        ▼
-┌───────────────┐
-│ Execute Steps │
-└───────┬───────┘
-        │
-        ▼
-┌───────────────┐
-│ sleep("1d")   │
-└───────┬───────┘
-        │
-        ├─── 1. Record sleep_started event
-        │
-        ├─── 2. Schedule Celery Beat task for wake time
-        │
-        ├─── 3. Raise SuspensionSignal
-        │
-        └─── 4. Worker is freed
-                  │
-                  │  ... 1 day passes ...
-                  │
-                  ▼
-        ┌─────────────────┐
-        │ Celery Beat     │
-        │ triggers resume │
-        └────────┬────────┘
-                 │
-                 ▼
-        ┌───────────────┐
-        │ Replay events │
-        │ Resume work   │
-        └───────────────┘
-```
-
-## Duration Formats
-
-### String Format (Recommended)
-
-```python
-await sleep("30s")   # 30 seconds
-await sleep("5m")    # 5 minutes
-await sleep("2h")    # 2 hours
-await sleep("1d")    # 1 day
-await sleep("1w")    # 1 week
-
-# Combined
-await sleep("1h30m") # 1 hour 30 minutes
-await sleep("2d12h") # 2 days 12 hours
-```
-
-### Timedelta
-
-```python
-from datetime import timedelta
-
-await sleep(timedelta(hours=2, minutes=30))
-await sleep(timedelta(days=7))
-```
-
-### Until Specific Time
-
-```python
-from datetime import datetime
-
-# Sleep until a specific datetime
-await sleep(datetime(2025, 12, 25, 9, 0, 0))
-
-# Sleep until next Monday at 9 AM
-next_monday = get_next_monday()
-await sleep(next_monday.replace(hour=9, minute=0))
-```
-
-### Integer (Seconds)
-
-```python
-await sleep(300)  # 300 seconds (5 minutes)
-```
-
-## Zero-Resource Suspension
-
-Traditional async sleep blocks a worker:
-
-```python
-# BAD: This holds a worker for 24 hours!
-import asyncio
-
-async def traditional_sleep():
-    await do_something()
-    await asyncio.sleep(86400)  # Blocks worker for 24h
-    await do_something_else()
-```
-
-PyWorkflow's sleep releases the worker:
-
-```python
-# GOOD: Worker is freed during sleep
-from pyworkflow import sleep
-
-@workflow()
-async def efficient_sleep():
-    await do_something()
-    await sleep("24h")  # Worker freed, resumes later
-    await do_something_else()
-```
-
-<Note>
-  With 100 workflows each sleeping for 1 day, traditional sleep would need 100 workers blocked for 24 hours. PyWorkflow needs 0 workers during the sleep period.
-</Note>
-
-## Use Cases
-
-### Scheduled Reminders
-
-```python
-@workflow()
-async def onboarding_drip(user_id: str):
-    await send_email(user_id, "Welcome!")
-
-    await sleep("1d")
-    await send_email(user_id, "Getting started tips")
-
-    await sleep("3d")
-    await send_email(user_id, "Advanced features")
-
-    await sleep("7d")
-    await send_email(user_id, "How's it going?")
-```
-
-### Delayed Processing
-
-```python
-@workflow()
-async def process_refund(order_id: str):
-    await validate_refund_request(order_id)
-
-    # Wait for potential fraud review
-    await sleep("24h")
-
-    # If not flagged, process refund
-    await execute_refund(order_id)
-    await notify_customer(order_id)
-```
-
-### Rate Limiting
-
-```python
-@workflow()
-async def batch_api_calls(items: list):
-    for i, item in enumerate(items):
-        await call_api(item)
-
-        # Rate limit: 10 calls per minute
-        if (i + 1) % 10 == 0:
-            await sleep("1m")
-```
-
-### Retry with Backoff
-
-```python
-@workflow()
-async def resilient_operation():
-    for attempt in range(5):
-        try:
-            result = await risky_step()
-            return result
-        except TemporaryError:
-            if attempt < 4:
-                # Exponential backoff: 1m, 2m, 4m, 8m
-                delay = f"{2 ** attempt}m"
-                await sleep(delay)
-
-    raise FatalError("All retries exhausted")
-```
-
-## Sleep vs Step Timeout
-
-Sleep and timeouts serve different purposes:
-
-| Feature | Sleep | Timeout |
-|---------|-------|---------|
-| Purpose | Intentional delay | Maximum execution time |
-| Resources | Zero during sleep | Worker is active |
-| Failure | Never fails | Fails if exceeded |
-
-```python
-@step(timeout="30s")  # Fails if step takes > 30s
-async def quick_step():
-    pass
-
-@workflow()
-async def my_workflow():
-    await quick_step()
-    await sleep("1h")  # Intentional 1-hour pause
-    await quick_step()
-```
-
-## Celery Beat Requirement
-
-Sleep resumption requires Celery Beat to be running:
-
-```bash
-# Start Celery Beat for scheduled task execution
-celery -A pyworkflow.celery.app beat --loglevel=info
-```
-
-<Warning>
-  Without Celery Beat, workflows will suspend but never resume automatically. Make sure Beat is running in production.
-</Warning>
-
-### Docker Compose Setup
-
-```yaml
-services:
-  worker:
-    command: celery -A pyworkflow.celery.app worker --loglevel=info
-
-  beat:
-    command: celery -A pyworkflow.celery.app beat --loglevel=info
-```
-
-## Best Practices
-
-<AccordionGroup>
-  <Accordion title="Use sleep for intentional delays only">
-    Don't use sleep as a retry mechanism. Use step retry configuration instead:
-
-    ```python
-    # Good: Use retry config
-    @step(max_retries=3, retry_delay="exponential")
-    async def my_step():
-        pass
-
-    # Avoid: Manual retry with sleep
-    @workflow()
-    async def my_workflow():
-        for i in range(3):
-            try:
-                await my_step()
-                break
-            except:
-                await sleep(f"{2**i}m")
-    ```
-  </Accordion>
-
-  <Accordion title="Consider timezone implications">
-    When sleeping until a specific time, be aware of timezones:
-
-    ```python
-    from datetime import datetime
-    import pytz
-
-    # Explicit timezone
-    eastern = pytz.timezone("US/Eastern")
-    wake_time = eastern.localize(datetime(2025, 1, 15, 9, 0))
-    await sleep(wake_time)
-    ```
-  </Accordion>
-
-  <Accordion title="Keep sleep durations reasonable">
-    Very long sleeps (months, years) work but consider if a different approach is better:
-
-    ```python
-    # Works, but consider alternatives
-    await sleep("365d")
-
-    # Alternative: Schedule a new workflow
-    schedule_workflow(annual_review, run_at="2026-01-01")
-    ```
-  </Accordion>
-</AccordionGroup>
-
-## Next Steps
-
-<CardGroup cols={2}>
-  <Card title="Workflows" icon="diagram-project" href="/concepts/workflows">
-    Learn about workflow orchestration.
-  </Card>
-  <Card title="Deployment" icon="rocket" href="/guides/deployment">
-    Set up Celery Beat in production.
-  </Card>
-</CardGroup>

docs/concepts/steps.mdx

@@ -1,301 +0,0 @@
----
-title: 'Steps'
-description: 'Isolated, retryable units of work that form the building blocks of workflows'
----
-
-## What is a Step?
-
-A step is an isolated, retryable unit of work within a workflow. Steps are where the actual business logic executes - calling APIs, processing data, sending emails, etc. Each step runs on a Celery worker and can be retried independently if it fails.
-
-```python
-from pyworkflow import step
-
-@step()
-async def send_email(to: str, subject: str, body: str):
-    """Send an email - retries automatically on failure."""
-    async with EmailClient() as client:
-        await client.send(to=to, subject=subject, body=body)
-    return {"sent": True, "to": to}
-```
-
-## Key Characteristics
-
-<CardGroup cols={2}>
-  <Card title="Isolated" icon="box">
-    Each step runs independently with its own retry policy and timeout.
-  </Card>
-  <Card title="Retryable" icon="rotate">
-    Failed steps automatically retry with configurable backoff strategies.
-  </Card>
-  <Card title="Cached" icon="database">
-    Completed step results are cached and replayed during workflow resumption.
-  </Card>
-  <Card title="Distributed" icon="server">
-    Steps execute on Celery workers, distributing load across your cluster.
-  </Card>
-</CardGroup>
-
-## Creating Steps
-
-<Tabs>
-  <Tab title="Decorator">
-    ```python
-    from pyworkflow import step
-
-    @step()
-    async def fetch_user(user_id: str):
-        async with httpx.AsyncClient() as client:
-            response = await client.get(f"/api/users/{user_id}")
-            return response.json()
-    ```
-  </Tab>
-  <Tab title="Class">
-    ```python
-    from pyworkflow import Step
-
-    class FetchUserStep(Step):
-        async def execute(self, user_id: str):
-            async with httpx.AsyncClient() as client:
-                response = await client.get(f"/api/users/{user_id}")
-                return response.json()
-
-    # Usage in workflow
-    user = await FetchUserStep()(user_id)
-    ```
-  </Tab>
-</Tabs>
-
-### Configuration Options
-
-<Tabs>
-  <Tab title="Decorator">
-    ```python
-    @step(
-        name="fetch_user_data",      # Custom step name
-        max_retries=5,               # Retry up to 5 times
-        retry_delay="exponential",   # Exponential backoff
-        timeout="30s"                # Step timeout
-    )
-    async def fetch_user(user_id: str):
-        pass
-    ```
-  </Tab>
-  <Tab title="Class">
-    ```python
-    class FetchUserStep(Step):
-        name = "fetch_user_data"
-        max_retries = 5
-        retry_delay = "exponential"
-        timeout = "30s"
-
-        async def execute(self, user_id: str):
-            pass
-    ```
-  </Tab>
-</Tabs>
-
-| Option | Type | Default | Description |
-|--------|------|---------|-------------|
-| `name` | `str` | Function/class name | Unique identifier for the step |
-| `max_retries` | `int` | `3` | Maximum retry attempts |
-| `retry_delay` | `str` | `"fixed"` | Retry strategy: `"fixed"`, `"exponential"`, or duration |
-| `timeout` | `str` | `"60s"` | Maximum step execution time |
-
-## Retry Strategies
-
-### Fixed Delay
-
-Retry with a constant delay between attempts:
-
-```python
-@step(max_retries=3, retry_delay="30s")
-async def call_api():
-    # Retries at: 30s, 60s, 90s
-    pass
-```
-
-### Exponential Backoff
-
-Retry with increasing delays (recommended for external APIs):
-
-```python
-@step(max_retries=5, retry_delay="exponential")
-async def call_external_api():
-    # Retries at: 1s, 2s, 4s, 8s, 16s
-    pass
-```
-
-### Custom Delay in Error
-
-Specify retry delay when raising an error:
-
-```python
-from pyworkflow import step, RetryableError
-
-@step(max_retries=3)
-async def rate_limited_api():
-    response = await call_api()
-
-    if response.status_code == 429:
-        retry_after = response.headers.get("Retry-After", "60")
-        raise RetryableError(
-            "Rate limited",
-            retry_after=f"{retry_after}s"
-        )
-
-    return response.json()
-```
-
-## Error Handling
-
-### RetryableError
-
-Use for transient failures that should be retried:
-
-```python
-from pyworkflow import step, RetryableError
-
-@step(max_retries=3)
-async def fetch_data():
-    try:
-        return await external_api.get_data()
-    except ConnectionError:
-        raise RetryableError("Connection failed")
-    except TimeoutError:
-        raise RetryableError("Request timed out", retry_after="10s")
-```
-
-### FatalError
-
-Use for permanent failures that should stop the workflow:
-
-```python
-from pyworkflow import step, FatalError
-
-@step()
-async def validate_input(data: dict):
-    if "email" not in data:
-        raise FatalError("Email is required")
-
-    if not is_valid_email(data["email"]):
-        raise FatalError(f"Invalid email: {data['email']}")
-
-    return data
-```
-
-### Error Flow
-
-```
-Step Execution
-      │
-      ├─── Success ───────────────────────> Return Result
-      │
-      └─── Exception
-              │
-              ├─── FatalError ────────────> Workflow Failed
-              │
-              └─── RetryableError / Other
-                      │
-                      ├─── Retries Left ──> Wait & Retry
-                      │
-                      └─── No Retries ────> Workflow Failed
-```
-
-## Step Results and Caching
-
-When a workflow resumes after suspension, completed steps are not re-executed. Instead, their cached results are returned:
-
-```python
-@workflow()
-async def my_workflow():
-    # First run: executes the step
-    # After resume: returns cached result
-    user = await fetch_user("user_123")
-
-    await sleep("1h")
-
-    # After 1 hour, workflow resumes
-    # fetch_user is NOT called again - cached result is used
-    await send_email(user["email"], "Hello!")
-```
-
-<Warning>
-  Step results must be serializable. PyWorkflow supports common Python types (dict, list, str, int, datetime, etc.) and uses cloudpickle for complex objects.
-</Warning>
-
-## Parallel Step Execution
-
-Execute multiple steps concurrently using `asyncio.gather()`:
-
-```python
-import asyncio
-from pyworkflow import workflow, step
-
-@step()
-async def fetch_user(user_id: str):
-    return await api.get_user(user_id)
-
-@step()
-async def fetch_orders(user_id: str):
-    return await api.get_orders(user_id)
-
-@step()
-async def fetch_preferences(user_id: str):
-    return await api.get_preferences(user_id)
-
-@workflow()
-async def load_dashboard(user_id: str):
-    # All three steps run in parallel
-    user, orders, preferences = await asyncio.gather(
-        fetch_user(user_id),
-        fetch_orders(user_id),
-        fetch_preferences(user_id)
-    )
-
-    return {
-        "user": user,
-        "orders": orders,
-        "preferences": preferences
-    }
-```
-
-## Best Practices
-
-<AccordionGroup>
-  <Accordion title="Keep steps small and focused">
-    Each step should do one thing well. This makes retries more efficient - if a step fails, only that specific operation is retried.
-  </Accordion>
-
-  <Accordion title="Make steps idempotent">
-    Steps may be retried, so ensure they can be safely re-executed. Use idempotency keys when calling external APIs.
-
-    ```python
-    @step()
-    async def charge_payment(order_id: str, amount: float):
-        # Use order_id as idempotency key
-        return await stripe.charges.create(
-            amount=amount,
-            idempotency_key=f"charge-{order_id}"
-        )
-    ```
-  </Accordion>
-
-  <Accordion title="Use appropriate timeouts">
-    Set timeouts based on expected execution time. External API calls should have shorter timeouts than data processing steps.
-  </Accordion>
-
-  <Accordion title="Handle errors explicitly">
-    Distinguish between retryable and fatal errors. Don't retry errors that will never succeed.
-  </Accordion>
-</AccordionGroup>
-
-## Next Steps
-
-<CardGroup cols={2}>
-  <Card title="Events" icon="timeline" href="/concepts/events">
-    Learn how event sourcing enables durability and replay.
-  </Card>
-  <Card title="Error Handling" icon="shield-check" href="/guides/error-handling">
-    Deep dive into retry strategies and error handling.
-  </Card>
-</CardGroup>