pyworkflow-engine 0.1.7__py3-none-any.whl → 0.1.10__py3-none-any.whl

docs/quickstart.mdx

@@ -1,279 +0,0 @@
----
-title: 'Quick Start'
-description: 'Get up and running with PyWorkflow in under 5 minutes'
----
-
-## Installation
-
-Install PyWorkflow using pip:
-
-```bash
-pip install pyworkflow
-```
-
-## Create a New Project
-
-The fastest way to get started is with the `quickstart` command:
-
-```bash
-pyworkflow quickstart
-```
-
-This interactive command will:
-1. Create a `workflows/` directory with sample workflows
-2. Generate `pyworkflow.config.yaml` configuration
-3. Optionally start Docker services (Redis + Dashboard)
-
-<Accordion title="Quickstart Output Example">
-```
-============================================================
-  PyWorkflow Quickstart
-============================================================
-
-Select a project template:
-  ❯ Basic - Order processing and notifications (2 workflows)
-
-Storage backend:
-  ❯ SQLite - Single file database (recommended)
-
-Start Docker services (Redis + Dashboard)? [Y/n]
-
-Creating project structure...
-  ✓ Created: workflows/__init__.py
-  ✓ Created: workflows/orders.py
-  ✓ Created: workflows/notifications.py
-  ✓ Created: pyworkflow.config.yaml
-
-============================================================
-Project Created!
-============================================================
-
-Next steps:
-
-  1. Start a worker:
-     $ pyworkflow worker start
-
-  2. Run a workflow:
-     $ pyworkflow workflows run process_order \
-         --input '{"order_id": "123", "amount": 49.99}'
-
-  3. View the dashboard:
-     Open http://localhost:5173 in your browser
-```
-</Accordion>
-
-### Non-Interactive Mode
-
-For CI/CD or scripting, use non-interactive mode:
-
-```bash
-# Create project with defaults (SQLite storage, start Docker)
-pyworkflow quickstart --non-interactive
-
-# Create project without Docker
-pyworkflow quickstart --non-interactive --skip-docker
-
-# Use file storage instead of SQLite
-pyworkflow quickstart --non-interactive --storage file
-```
-
----
-
-## Manual Setup
-
-If you prefer to set up manually or need more control:
-
-<Tabs>
-  <Tab title="Docker Compose (Recommended)">
-    Generate Docker configuration and start services:
-
-    ```bash
-    # Generate docker-compose.yml and config
-    pyworkflow setup
-
-    # Start services
-    docker compose up -d
-    ```
-
-    This starts Redis and the PyWorkflow Dashboard.
-  </Tab>
-  <Tab title="Manual">
-    Start each component manually:
-
-    ```bash
-    # 1. Start Redis
-    docker run -d -p 6379:6379 redis:7-alpine
-
-    # 2. Create config file
-    cat > pyworkflow.config.yaml << EOF
-    module: workflows
-    runtime: celery
-    storage:
-      type: sqlite
-      base_path: pyworkflow_data/pyworkflow.db
-    celery:
-      broker: redis://localhost:6379/0
-      result_backend: redis://localhost:6379/1
-    EOF
-
-    # 3. Start Celery worker
-    pyworkflow worker start
-    ```
-  </Tab>
-</Tabs>
-
-## Your First Workflow
-
-Create a simple onboarding workflow that sends emails with delays:
-
-```python
-from pyworkflow import workflow, step, start, sleep
-
-@step()
-async def send_welcome_email(user_id: str):
-    """Send a welcome email to the new user."""
-    print(f"Sending welcome email to user {user_id}")
-    return f"Email sent to {user_id}"
-
-@step()
-async def send_tips_email(user_id: str):
-    """Send helpful tips after the welcome period."""
-    print(f"Sending tips email to user {user_id}")
-    return f"Tips sent to {user_id}"
-
-@workflow()
-async def onboarding_workflow(user_id: str):
-    # Send welcome email immediately
-    await send_welcome_email(user_id)
-
-    # Sleep for 1 day - workflow suspends, zero resources used
-    await sleep("1d")
-
-    # Automatically resumes after 1 day
-    await send_tips_email(user_id)
-
-    return "Onboarding complete"
-
-# Start the workflow
-run_id = start(onboarding_workflow, user_id="user_123")
-print(f"Workflow started: {run_id}")
-```
-
-## What Happens Under the Hood
-
-<Steps>
-  <Step title="Workflow Starts">
-    Your workflow is dispatched to an available Celery worker.
-  </Step>
-  <Step title="Welcome Email Sent">
-    The `send_welcome_email` step executes and the result is recorded.
-  </Step>
-  <Step title="Workflow Suspends">
-    When `sleep("1d")` is called, the workflow suspends and the worker is freed.
-    Zero resources are consumed during the sleep period.
-  </Step>
-  <Step title="Automatic Resumption">
-    After 1 day, Celery Beat automatically schedules the workflow to resume.
-  </Step>
-  <Step title="Tips Email Sent">
-    The workflow picks up where it left off, sending the tips email.
-  </Step>
-  <Step title="Workflow Completes">
-    The final result is recorded and the workflow is marked as complete.
-  </Step>
-</Steps>
-
-## Key Concepts
-
-<CardGroup cols={2}>
-  <Card title="Workflows" icon="diagram-project" href="/concepts/workflows">
-    Top-level orchestration functions that coordinate steps and handle business logic.
-  </Card>
-  <Card title="Steps" icon="stairs" href="/concepts/steps">
-    Isolated, retryable units of work that run on Celery workers.
-  </Card>
-  <Card title="Sleep" icon="clock" href="/concepts/sleep">
-    Pause workflows for any duration without consuming resources.
-  </Card>
-  <Card title="Events" icon="timeline" href="/concepts/events">
-    Event sourcing provides durability and deterministic replay.
-  </Card>
-</CardGroup>
-
-## Adding Error Handling
-
-Make your workflows fault-tolerant with automatic retries:
-
-```python
-from pyworkflow import step, RetryableError, FatalError
-
-@step(max_retries=3, retry_delay="exponential")
-async def call_payment_api(amount: float):
-    """Process payment with automatic retry on failure."""
-    try:
-        result = await payment_gateway.charge(amount)
-        return result
-    except PaymentGatewayTimeoutError:
-        # Retry with exponential backoff
-        raise RetryableError("Gateway timeout", retry_after="10s")
-    except InsufficientFundsError:
-        # Don't retry - this is a permanent failure
-        raise FatalError("Insufficient funds")
-```
-
-<Tip>
-  Use `RetryableError` for transient failures (network issues, timeouts) and `FatalError` for permanent failures (invalid input, business rule violations).
-</Tip>
-
-## Running in Parallel
-
-Execute multiple steps concurrently using `asyncio.gather()`:
-
-```python
-import asyncio
-from pyworkflow import workflow, step
-
-@step()
-async def fetch_user(user_id: str):
-    return {"id": user_id, "name": "Alice"}
-
-@step()
-async def fetch_orders(user_id: str):
-    return [{"id": "ORD-1"}, {"id": "ORD-2"}]
-
-@step()
-async def fetch_recommendations(user_id: str):
-    return ["Product A", "Product B"]
-
-@workflow()
-async def dashboard_data(user_id: str):
-    # Fetch all data in parallel
-    user, orders, recommendations = await asyncio.gather(
-        fetch_user(user_id),
-        fetch_orders(user_id),
-        fetch_recommendations(user_id)
-    )
-
-    return {
-        "user": user,
-        "orders": orders,
-        "recommendations": recommendations
-    }
-```
-
-## Next Steps
-
-<CardGroup cols={2}>
-  <Card title="Core Concepts" icon="book" href="/concepts/workflows">
-    Learn about workflows, steps, and event sourcing in depth.
-  </Card>
-  <Card title="Error Handling" icon="shield-check" href="/guides/error-handling">
-    Build fault-tolerant workflows with retry strategies.
-  </Card>
-  <Card title="Testing" icon="flask-vial" href="/guides/testing">
-    Write unit and integration tests for your workflows.
-  </Card>
-  <Card title="Deployment" icon="rocket" href="/guides/deployment">
-    Deploy to production with Docker and Kubernetes.
-  </Card>
-</CardGroup>

examples/__init__.py

@@ -1 +0,0 @@
-# PyWorkflow Examples Package

examples/celery/__init__.py

@@ -1 +0,0 @@
-# PyWorkflow Celery Examples Package

examples/celery/durable/docker-compose.yml

@@ -1,55 +0,0 @@
-services:
-  redis:
-    image: redis:7-alpine
-    container_name: pyworkflow-redis
-    ports:
-      - "6379:6379"
-    volumes:
-      - redis_data:/data
-    healthcheck:
-      test: ["CMD", "redis-cli", "ping"]
-      interval: 5s
-      timeout: 3s
-      retries: 5
-    restart: unless-stopped
-
-  dashboard-backend:
-    image: yashabro/pyworkflow-dashboard-backend:latest
-    platform: linux/amd64
-    container_name: pyworkflow-dashboard-backend
-    working_dir: /app/project
-    ports:
-      - "8585:8585"
-    environment:
-      - DASHBOARD_PYWORKFLOW_CONFIG_PATH=/app/project/pyworkflow.config.yaml
-      - DASHBOARD_STORAGE_TYPE=sqlite
-      - DASHBOARD_STORAGE_PATH=/app/project/pyworkflow_data
-      - DASHBOARD_HOST=0.0.0.0
-      - DASHBOARD_PORT=8585
-      - DASHBOARD_CORS_ORIGINS=["http://localhost:5173","http://localhost:3000"]
-      - PYWORKFLOW_CELERY_BROKER=redis://redis:6379/0
-      - PYWORKFLOW_CELERY_RESULT_BACKEND=redis://redis:6379/1
-      - PYTHONPATH=/app/project
-    volumes:
-      - .:/app/project:ro
-      - ./pyworkflow_data:/app/project/pyworkflow_data
-    depends_on:
-      redis:
-        condition: service_healthy
-    restart: unless-stopped
-
-  dashboard-frontend:
-    image: yashabro/pyworkflow-dashboard-frontend:latest
-    platform: linux/amd64
-    container_name: pyworkflow-dashboard-frontend
-    ports:
-      - "5173:80"
-    environment:
-      - VITE_API_URL=http://localhost:8585
-    depends_on:
-      - dashboard-backend
-    restart: unless-stopped
-
-volumes:
-  redis_data:
-    driver: local

examples/celery/durable/pyworkflow.config.yaml

@@ -1,12 +0,0 @@
-# PyWorkflow Configuration
-# Generated: 2025-12-27 11:44:23
-# Documentation: https://docs.pyworkflow.dev
-
-module: workflows
-runtime: celery
-storage:
-  type: sqlite
-  base_path: pyworkflow_data/pyworkflow.db
-celery:
-  broker: redis://localhost:6379/0
-  result_backend: redis://localhost:6379/1

examples/celery/durable/workflows/__init__.py

@@ -1,122 +0,0 @@
-"""
-Example workflows demonstrating PyWorkflow features.
-
-This package contains example workflows organized by feature:
-- basic: Simple order processing workflow
-- long_running: Long-running onboarding workflow with sleeps
-- retries: Retry handling with flaky APIs
-- batch_processing: Processing data in batches
-- idempotency: Idempotent payment processing
-- fault_tolerance: Worker crash recovery
-- hooks: Human-in-the-loop approvals with webhooks
-- cancellation: Cancellable workflows with cleanup
-- child_workflows: Parent-child workflow orchestration
-- child_workflow_patterns: Advanced child workflow patterns
-- continue_as_new: Long-running workflows with state reset
-- schedules: Scheduled/recurring workflows
-
-Usage:
-    # Start workers
-    pyworkflow worker start
-
-    # List registered workflows
-    pyworkflow workflows list
-
-    # Trigger a workflow
-    pyworkflow workflows run order_workflow --input '{"order_id": "123", "amount": 99.99}'
-"""
-
-# Basic workflow
-from .basic import order_workflow
-
-# Batch processing
-from .batch_processing import batch_workflow
-
-# Cancellation
-from .cancellation import cancel_demo_simple_workflow, cancellable_order_workflow
-
-# Child workflow patterns
-from .child_workflow_patterns import (
-    error_handling_parent_workflow,
-    failing_child_workflow,
-    level_1_workflow,
-    level_2_workflow,
-    level_3_workflow,
-    parallel_parent_workflow,
-    parallel_task_workflow,
-    try_exceed_max_depth,
-)
-from .child_workflows import (
-    notification_workflow,
-    order_fulfillment_workflow,
-    shipping_workflow,
-)
-
-# Child workflows
-from .child_workflows import (
-    payment_workflow as child_payment_workflow,
-)
-
-# Continue as new
-from .continue_as_new import batch_processor, message_consumer, recurring_report
-
-# Fault tolerance / recovery
-from .fault_tolerance import critical_pipeline, data_pipeline
-
-# Webhooks / Human-in-the-loop
-from .hooks import approval_workflow, multi_approval_workflow, simple_approval_workflow
-
-# Idempotency
-from .idempotency import payment_workflow as idempotent_payment_workflow
-
-# Long-running workflow with sleeps
-from .long_running import onboarding_workflow
-
-# Retry handling
-from .retries import retry_demo_workflow
-
-# Schedules
-from .schedules import cleanup_workflow
-
-__all__ = [
-    # Basic
-    "order_workflow",
-    # Long-running
-    "onboarding_workflow",
-    # Retries
-    "retry_demo_workflow",
-    # Batch processing
-    "batch_workflow",
-    # Idempotency
-    "idempotent_payment_workflow",
-    # Fault tolerance
-    "data_pipeline",
-    "critical_pipeline",
-    # Hooks
-    "simple_approval_workflow",
-    "approval_workflow",
-    "multi_approval_workflow",
-    # Cancellation
-    "cancellable_order_workflow",
-    "cancel_demo_simple_workflow",
-    # Child workflows
-    "child_payment_workflow",
-    "shipping_workflow",
-    "notification_workflow",
-    "order_fulfillment_workflow",
-    # Child workflow patterns
-    "level_1_workflow",
-    "level_2_workflow",
-    "level_3_workflow",
-    "parallel_task_workflow",
-    "parallel_parent_workflow",
-    "failing_child_workflow",
-    "error_handling_parent_workflow",
-    "try_exceed_max_depth",
-    # Continue as new
-    "batch_processor",
-    "message_consumer",
-    "recurring_report",
-    # Schedules
-    "cleanup_workflow",
-]

examples/celery/durable/workflows/basic.py

@@ -1,87 +0,0 @@
-"""
-Celery Durable Workflow - Basic Example
-
-This example demonstrates a simple event-sourced workflow running on Celery workers.
-- 3-step order processing workflow
-- Distributed execution across workers
-- Events recorded for each step
-
-Prerequisites:
-    1. Start Redis: docker run -d -p 6379:6379 redis:7-alpine
-    2. Start worker: pyworkflow --module examples.celery.durable.01_basic_workflow worker run
-
-Run with CLI:
-    pyworkflow --module examples.celery.durable.01_basic_workflow workflows run order_workflow \
-        --arg order_id=order-123 --arg amount=99.99
-
-Check status:
-    pyworkflow runs list
-    pyworkflow runs status <run_id>
-    pyworkflow runs logs <run_id>
-"""
-
-from pyworkflow import step, workflow
-
-
-@step(name="basic_validate_order")
-async def validate_order(order_id: str) -> dict:
-    """Validate the order exists and is processable."""
-    print(f"[Step] Validating order {order_id}...")
-    return {"order_id": order_id, "valid": True}
-
-
-@step(name="basic_process_payment")
-async def process_payment(order: dict, amount: float) -> dict:
-    """Process payment for the order."""
-    print(f"[Step] Processing payment ${amount:.2f} for {order['order_id']}...")
-    return {**order, "paid": True, "amount": amount}
-
-
-@step()
-async def send_confirmation(order: dict) -> dict:
-    """Send order confirmation email."""
-    print(f"[Step] Sending confirmation for {order['order_id']}...")
-    return {**order, "confirmed": True}
-
-
-@workflow(tags=["celery", "durable"])
-async def order_workflow(order_id: str, amount: float) -> dict:
-    """
-    Complete order processing workflow.
-
-    Steps:
-    1. Validate the order
-    2. Process payment
-    3. Send confirmation
-
-    Each step runs on Celery workers and is recorded as an event.
-    """
-    order = await validate_order(order_id)
-    order = await process_payment(order, amount)
-    order = await send_confirmation(order)
-    return order
-
-
-async def main() -> None:
-    """Run the order workflow example."""
-    import argparse
-
-    import pyworkflow
-
-    parser = argparse.ArgumentParser(description="Order Processing Workflow")
-    parser.add_argument("--order-id", default="order-123", help="Order ID to process")
-    parser.add_argument("--amount", type=float, default=99.99, help="Order amount")
-    args = parser.parse_args()
-
-    # Configuration is automatically loaded from pyworkflow.config.yaml
-    # which sets runtime=celery and creates storage backend
-    print(f"Starting order workflow for {args.order_id} (${args.amount:.2f})...")
-    run_id = await pyworkflow.start(order_workflow, args.order_id, args.amount)
-    print(f"Workflow started with run_id: {run_id}")
-    print(f"\nCheck status: pyworkflow runs status {run_id}")
-
-
-if __name__ == "__main__":
-    import asyncio
-
-    asyncio.run(main())

examples/celery/durable/workflows/batch_processing.py

@@ -1,102 +0,0 @@
-"""
-Celery Durable Workflow - Batch Processing
-
-This example demonstrates batch item processing on Celery workers.
-- Fetch items to process
-- Process each item (each as a recorded step)
-- Aggregate results
-
-Prerequisites:
-    1. Start Redis: docker run -d -p 6379:6379 redis:7-alpine
-    2. Start worker: pyworkflow --module examples.celery.durable.04_batch_processing worker run
-
-Run with CLI:
-    pyworkflow --module examples.celery.durable.04_batch_processing workflows run batch_workflow \
-        --arg batch_id=batch-789 --arg limit=5
-
-Check status:
-    pyworkflow runs status <run_id>
-    pyworkflow runs logs <run_id> --filter step_completed
-"""
-
-from pyworkflow import step, workflow
-
-
-@step()
-async def fetch_batch_items(batch_id: str, limit: int = 100) -> list:
-    """Fetch items to process in this batch."""
-    print(f"[Step] Fetching batch {batch_id} with limit {limit}...")
-    # Simulate fetching items from database
-    items = [{"id": f"item-{i}", "batch_id": batch_id} for i in range(min(limit, 10))]
-    print(f"[Step] Fetched {len(items)} items")
-    return items
-
-
-@step(name="batch_process_item")
-async def process_item(item: dict) -> dict:
-    """Process a single item."""
-    print(f"[Step] Processing item {item['id']}...")
-    # Simulate processing
-    return {**item, "processed": True, "result": f"processed_{item['id']}"}
-
-
-@step()
-async def aggregate_results(results: list) -> dict:
-    """Aggregate processing results."""
-    successful = len([r for r in results if r.get("processed")])
-    print(f"[Step] Aggregating {len(results)} results ({successful} successful)...")
-    return {
-        "total": len(results),
-        "successful": successful,
-        "failed": len(results) - successful,
-    }
-
-
-@workflow(tags=["celery", "durable"])
-async def batch_workflow(batch_id: str, limit: int = 100) -> dict:
-    """
-    Batch processing workflow.
-
-    Steps:
-    1. Fetch items to process
-    2. Process each item individually
-    3. Aggregate and return results
-
-    Each item processing is recorded as a separate step event,
-    enabling fine-grained tracking and potential parallel execution.
-    """
-    items = await fetch_batch_items(batch_id, limit)
-
-    # Process items sequentially (each recorded as step)
-    results = []
-    for item in items:
-        result = await process_item(item)
-        results.append(result)
-
-    summary = await aggregate_results(results)
-    return {"batch_id": batch_id, **summary}
-
-
-async def main() -> None:
-    """Run the batch processing workflow example."""
-    import argparse
-
-    import pyworkflow
-
-    parser = argparse.ArgumentParser(description="Batch Processing Workflow")
-    parser.add_argument("--batch-id", default="batch-789", help="Batch ID to process")
-    parser.add_argument("--limit", type=int, default=5, help="Maximum items to process")
-    args = parser.parse_args()
-
-    # Configuration is automatically loaded from pyworkflow.config.yaml
-    print(f"Starting batch workflow for {args.batch_id} (limit: {args.limit})...")
-    run_id = await pyworkflow.start(batch_workflow, args.batch_id, args.limit)
-    print(f"Workflow started with run_id: {run_id}")
-    print(f"\nCheck status: pyworkflow runs status {run_id}")
-    print(f"View step logs: pyworkflow runs logs {run_id} --filter step_completed")
-
-
-if __name__ == "__main__":
-    import asyncio
-
-    asyncio.run(main())