@sylix/coworker 2.0.11 → 2.0.12

package/dist/skills/defaults/backend-development/python-anti-patterns.md

@@ -0,0 +1,349 @@
+---
+name: python-anti-patterns
+description: Common Python anti-patterns to avoid. Use as a checklist when reviewing code, before finalizing implementations, or when debugging issues that might stem from known bad practices.
+---
+
+# Python Anti-Patterns Checklist
+
+A reference checklist of common mistakes and anti-patterns in Python code. Review this before finalizing implementations to catch issues early.
+
+## When to Use This Skill
+
+- Reviewing code before merge
+- Debugging mysterious issues
+- Teaching or learning Python best practices
+- Establishing team coding standards
+- Refactoring legacy code
+
+**Note:** This skill focuses on what to avoid. For guidance on positive patterns and architecture, see the `python-design-patterns` skill.
+
+## Infrastructure Anti-Patterns
+
+### Scattered Timeout/Retry Logic
+
+```python
+# BAD: Timeout logic duplicated everywhere
+def fetch_user(user_id):
+    try:
+        return requests.get(url, timeout=30)
+    except Timeout:
+        logger.warning("Timeout fetching user")
+        return None
+
+def fetch_orders(user_id):
+    try:
+        return requests.get(url, timeout=30)
+    except Timeout:
+        logger.warning("Timeout fetching orders")
+        return None
+```
+
+**Fix:** Centralize in decorators or client wrappers.
+
+```python
+# GOOD: Centralized retry logic
+@retry(stop=stop_after_attempt(3), wait=wait_exponential())
+def http_get(url: str) -> Response:
+    return requests.get(url, timeout=30)
+```
+
+### Double Retry
+
+```python
+# BAD: Retrying at multiple layers
+@retry(max_attempts=3)  # Application retry
+def call_service():
+    return client.request()  # Client also has retry configured!
+```
+
+**Fix:** Retry at one layer only. Know your infrastructure's retry behavior.
+
+### Hard-Coded Configuration
+
+```python
+# BAD: Secrets and config in code
+DB_HOST = "prod-db.example.com"
+API_KEY = "sk-12345"
+
+def connect():
+    return psycopg.connect(f"host={DB_HOST}...")
+```
+
+**Fix:** Use environment variables with typed settings.
+
+```python
+# GOOD
+from pydantic_settings import BaseSettings
+
+class Settings(BaseSettings):
+    db_host: str = Field(alias="DB_HOST")
+    api_key: str = Field(alias="API_KEY")
+
+settings = Settings()
+```
+
+## Architecture Anti-Patterns
+
+### Exposed Internal Types
+
+```python
+# BAD: Leaking ORM model to API
+@app.get("/users/{id}")
+def get_user(id: str) -> UserModel:  # SQLAlchemy model
+    return db.query(UserModel).get(id)
+```
+
+**Fix:** Use DTOs/response models.
+
+```python
+# GOOD
+@app.get("/users/{id}")
+def get_user(id: str) -> UserResponse:
+    user = db.query(UserModel).get(id)
+    return UserResponse.from_orm(user)
+```
+
+### Mixed I/O and Business Logic
+
+```python
+# BAD: SQL embedded in business logic
+def calculate_discount(user_id: str) -> float:
+    user = db.query("SELECT * FROM users WHERE id = ?", user_id)
+    orders = db.query("SELECT * FROM orders WHERE user_id = ?", user_id)
+    # Business logic mixed with data access
+    if len(orders) > 10:
+        return 0.15
+    return 0.0
+```
+
+**Fix:** Repository pattern. Keep business logic pure.
+
+```python
+# GOOD
+def calculate_discount(user: User, orders: list[Order]) -> float:
+    # Pure business logic, easily testable
+    if len(orders) > 10:
+        return 0.15
+    return 0.0
+```
+
+## Error Handling Anti-Patterns
+
+### Bare Exception Handling
+
+```python
+# BAD: Swallowing all exceptions
+try:
+    process()
+except Exception:
+    pass  # Silent failure - bugs hidden forever
+```
+
+**Fix:** Catch specific exceptions. Log or handle appropriately.
+
+```python
+# GOOD
+try:
+    process()
+except ConnectionError as e:
+    logger.warning("Connection failed, will retry", error=str(e))
+    raise
+except ValueError as e:
+    logger.error("Invalid input", error=str(e))
+    raise BadRequestError(str(e))
+```
+
+### Ignored Partial Failures
+
+```python
+# BAD: Stops on first error
+def process_batch(items):
+    results = []
+    for item in items:
+        result = process(item)  # Raises on error - batch aborted
+        results.append(result)
+    return results
+```
+
+**Fix:** Capture both successes and failures.
+
+```python
+# GOOD
+def process_batch(items) -> BatchResult:
+    succeeded = {}
+    failed = {}
+    for idx, item in enumerate(items):
+        try:
+            succeeded[idx] = process(item)
+        except Exception as e:
+            failed[idx] = e
+    return BatchResult(succeeded, failed)
+```
+
+### Missing Input Validation
+
+```python
+# BAD: No validation
+def create_user(data: dict):
+    return User(**data)  # Crashes deep in code on bad input
+```
+
+**Fix:** Validate early at API boundaries.
+
+```python
+# GOOD
+def create_user(data: dict) -> User:
+    validated = CreateUserInput.model_validate(data)
+    return User.from_input(validated)
+```
+
+## Resource Anti-Patterns
+
+### Unclosed Resources
+
+```python
+# BAD: File never closed
+def read_file(path):
+    f = open(path)
+    return f.read()  # What if this raises?
+```
+
+**Fix:** Use context managers.
+
+```python
+# GOOD
+def read_file(path):
+    with open(path) as f:
+        return f.read()
+```
+
+### Blocking in Async
+
+```python
+# BAD: Blocks the entire event loop
+async def fetch_data():
+    time.sleep(1)  # Blocks everything!
+    response = requests.get(url)  # Also blocks!
+```
+
+**Fix:** Use async-native libraries.
+
+```python
+# GOOD
+async def fetch_data():
+    await asyncio.sleep(1)
+    async with httpx.AsyncClient() as client:
+        response = await client.get(url)
+```
+
+## Type Safety Anti-Patterns
+
+### Missing Type Hints
+
+```python
+# BAD: No types
+def process(data):
+    return data["value"] * 2
+```
+
+**Fix:** Annotate all public functions.
+
+```python
+# GOOD
+def process(data: dict[str, int]) -> int:
+    return data["value"] * 2
+```
+
+### Untyped Collections
+
+```python
+# BAD: Generic list without type parameter
+def get_users() -> list:
+    ...
+```
+
+**Fix:** Use type parameters.
+
+```python
+# GOOD
+def get_users() -> list[User]:
+    ...
+```
+
+## Testing Anti-Patterns
+
+### Only Testing Happy Paths
+
+```python
+# BAD: Only tests success case
+def test_create_user():
+    user = service.create_user(valid_data)
+    assert user.id is not None
+```
+
+**Fix:** Test error conditions and edge cases.
+
+```python
+# GOOD
+def test_create_user_success():
+    user = service.create_user(valid_data)
+    assert user.id is not None
+
+def test_create_user_invalid_email():
+    with pytest.raises(ValueError, match="Invalid email"):
+        service.create_user(invalid_email_data)
+
+def test_create_user_duplicate_email():
+    service.create_user(valid_data)
+    with pytest.raises(ConflictError):
+        service.create_user(valid_data)
+```
+
+### Over-Mocking
+
+```python
+# BAD: Mocking everything
+def test_user_service():
+    mock_repo = Mock()
+    mock_cache = Mock()
+    mock_logger = Mock()
+    mock_metrics = Mock()
+    # Test doesn't verify real behavior
+```
+
+**Fix:** Use integration tests for critical paths. Mock only external services.
+
+## Quick Review Checklist
+
+Before finalizing code, verify:
+
+- [ ] No scattered timeout/retry logic (centralized)
+- [ ] No double retry (app + infrastructure)
+- [ ] No hard-coded configuration or secrets
+- [ ] No exposed internal types (ORM models, protobufs)
+- [ ] No mixed I/O and business logic
+- [ ] No bare `except Exception: pass`
+- [ ] No ignored partial failures in batches
+- [ ] No missing input validation
+- [ ] No unclosed resources (using context managers)
+- [ ] No blocking calls in async code
+- [ ] All public functions have type hints
+- [ ] Collections have type parameters
+- [ ] Error paths are tested
+- [ ] Edge cases are covered
+
+## Common Fixes Summary
+
+| Anti-Pattern | Fix |
+|-------------|-----|
+| Scattered retry logic | Centralized decorators |
+| Hard-coded config | Environment variables + pydantic-settings |
+| Exposed ORM models | DTO/response schemas |
+| Mixed I/O + logic | Repository pattern |
+| Bare except | Catch specific exceptions |
+| Batch stops on error | Return BatchResult with successes/failures |
+| No validation | Validate at boundaries with Pydantic |
+| Unclosed resources | Context managers |
+| Blocking in async | Async-native libraries |
+| Missing types | Type annotations on all public APIs |
+| Only happy path tests | Test errors and edge cases |

package/dist/skills/defaults/backend-development/python-background-jobs.md

@@ -0,0 +1,364 @@
+---
+name: python-background-jobs
+description: Python background job patterns including task queues, workers, and event-driven architecture. Use when implementing async task processing, job queues, long-running operations, or decoupling work from request/response cycles.
+---
+
+# Python Background Jobs & Task Queues
+
+Decouple long-running or unreliable work from request/response cycles. Return immediately to the user while background workers handle the heavy lifting asynchronously.
+
+## When to Use This Skill
+
+- Processing tasks that take longer than a few seconds
+- Sending emails, notifications, or webhooks
+- Generating reports or exporting data
+- Processing uploads or media transformations
+- Integrating with unreliable external services
+- Building event-driven architectures
+
+## Core Concepts
+
+### 1. Task Queue Pattern
+
+API accepts request, enqueues a job, returns immediately with a job ID. Workers process jobs asynchronously.
+
+### 2. Idempotency
+
+Tasks may be retried on failure. Design for safe re-execution.
+
+### 3. Job State Machine
+
+Jobs transition through states: pending → running → succeeded/failed.
+
+### 4. At-Least-Once Delivery
+
+Most queues guarantee at-least-once delivery. Your code must handle duplicates.
+
+## Quick Start
+
+This skill uses Celery for examples, a widely adopted task queue. Alternatives like RQ, Dramatiq, and cloud-native solutions (AWS SQS, GCP Tasks) are equally valid choices.
+
+```python
+from celery import Celery
+
+app = Celery("tasks", broker="redis://localhost:6379")
+
+@app.task
+def send_email(to: str, subject: str, body: str) -> None:
+    # This runs in a background worker
+    email_client.send(to, subject, body)
+
+# In your API handler
+send_email.delay("user@example.com", "Welcome!", "Thanks for signing up")
+```
+
+## Fundamental Patterns
+
+### Pattern 1: Return Job ID Immediately
+
+For operations exceeding a few seconds, return a job ID and process asynchronously.
+
+```python
+from uuid import uuid4
+from dataclasses import dataclass
+from enum import Enum
+from datetime import datetime
+
+class JobStatus(Enum):
+    PENDING = "pending"
+    RUNNING = "running"
+    SUCCEEDED = "succeeded"
+    FAILED = "failed"
+
+@dataclass
+class Job:
+    id: str
+    status: JobStatus
+    created_at: datetime
+    started_at: datetime | None = None
+    completed_at: datetime | None = None
+    result: dict | None = None
+    error: str | None = None
+
+# API endpoint
+async def start_export(request: ExportRequest) -> JobResponse:
+    """Start export job and return job ID."""
+    job_id = str(uuid4())
+
+    # Persist job record
+    await jobs_repo.create(Job(
+        id=job_id,
+        status=JobStatus.PENDING,
+        created_at=datetime.utcnow(),
+    ))
+
+    # Enqueue task for background processing
+    await task_queue.enqueue(
+        "export_data",
+        job_id=job_id,
+        params=request.model_dump(),
+    )
+
+    # Return immediately with job ID
+    return JobResponse(
+        job_id=job_id,
+        status="pending",
+        poll_url=f"/jobs/{job_id}",
+    )
+```
+
+### Pattern 2: Celery Task Configuration
+
+Configure Celery tasks with proper retry and timeout settings.
+
+```python
+from celery import Celery
+
+app = Celery("tasks", broker="redis://localhost:6379")
+
+# Global configuration
+app.conf.update(
+    task_time_limit=3600,          # Hard limit: 1 hour
+    task_soft_time_limit=3000,     # Soft limit: 50 minutes
+    task_acks_late=True,           # Acknowledge after completion
+    task_reject_on_worker_lost=True,
+    worker_prefetch_multiplier=1,  # Don't prefetch too many tasks
+)
+
+@app.task(
+    bind=True,
+    max_retries=3,
+    default_retry_delay=60,
+    autoretry_for=(ConnectionError, TimeoutError),
+)
+def process_payment(self, payment_id: str) -> dict:
+    """Process payment with automatic retry on transient errors."""
+    try:
+        result = payment_gateway.charge(payment_id)
+        return {"status": "success", "transaction_id": result.id}
+    except PaymentDeclinedError as e:
+        # Don't retry permanent failures
+        return {"status": "declined", "reason": str(e)}
+    except TransientError as e:
+        # Retry with exponential backoff
+        raise self.retry(exc=e, countdown=2 ** self.request.retries * 60)
+```
+
+### Pattern 3: Make Tasks Idempotent
+
+Workers may retry on crash or timeout. Design for safe re-execution.
+
+```python
+@app.task(bind=True)
+def process_order(self, order_id: str) -> None:
+    """Process order idempotently."""
+    order = orders_repo.get(order_id)
+
+    # Already processed? Return early
+    if order.status == OrderStatus.COMPLETED:
+        logger.info("Order already processed", order_id=order_id)
+        return
+
+    # Already in progress? Check if we should continue
+    if order.status == OrderStatus.PROCESSING:
+        # Use idempotency key to avoid double-charging
+        pass
+
+    # Process with idempotency key
+    result = payment_provider.charge(
+        amount=order.total,
+        idempotency_key=f"order-{order_id}",  # Critical!
+    )
+
+    orders_repo.update(order_id, status=OrderStatus.COMPLETED)
+```
+
+**Idempotency Strategies:**
+
+1. **Check-before-write**: Verify state before action
+2. **Idempotency keys**: Use unique tokens with external services
+3. **Upsert patterns**: `INSERT ... ON CONFLICT UPDATE`
+4. **Deduplication window**: Track processed IDs for N hours
+
+### Pattern 4: Job State Management
+
+Persist job state transitions for visibility and debugging.
+
+```python
+class JobRepository:
+    """Repository for managing job state."""
+
+    async def create(self, job: Job) -> Job:
+        """Create new job record."""
+        await self._db.execute(
+            """INSERT INTO jobs (id, status, created_at)
+               VALUES ($1, $2, $3)""",
+            job.id, job.status.value, job.created_at,
+        )
+        return job
+
+    async def update_status(
+        self,
+        job_id: str,
+        status: JobStatus,
+        **fields,
+    ) -> None:
+        """Update job status with timestamp."""
+        updates = {"status": status.value, **fields}
+
+        if status == JobStatus.RUNNING:
+            updates["started_at"] = datetime.utcnow()
+        elif status in (JobStatus.SUCCEEDED, JobStatus.FAILED):
+            updates["completed_at"] = datetime.utcnow()
+
+        await self._db.execute(
+            "UPDATE jobs SET status = $1, ... WHERE id = $2",
+            updates, job_id,
+        )
+
+        logger.info(
+            "Job status updated",
+            job_id=job_id,
+            status=status.value,
+        )
+```
+
+## Advanced Patterns
+
+### Pattern 5: Dead Letter Queue
+
+Handle permanently failed tasks for manual inspection.
+
+```python
+@app.task(bind=True, max_retries=3)
+def process_webhook(self, webhook_id: str, payload: dict) -> None:
+    """Process webhook with DLQ for failures."""
+    try:
+        result = send_webhook(payload)
+        if not result.success:
+            raise WebhookFailedError(result.error)
+    except Exception as e:
+        if self.request.retries >= self.max_retries:
+            # Move to dead letter queue for manual inspection
+            dead_letter_queue.send({
+                "task": "process_webhook",
+                "webhook_id": webhook_id,
+                "payload": payload,
+                "error": str(e),
+                "attempts": self.request.retries + 1,
+                "failed_at": datetime.utcnow().isoformat(),
+            })
+            logger.error(
+                "Webhook moved to DLQ after max retries",
+                webhook_id=webhook_id,
+                error=str(e),
+            )
+            return
+
+        # Exponential backoff retry
+        raise self.retry(exc=e, countdown=2 ** self.request.retries * 60)
+```
+
+### Pattern 6: Status Polling Endpoint
+
+Provide an endpoint for clients to check job status.
+
+```python
+from fastapi import FastAPI, HTTPException
+
+app = FastAPI()
+
+@app.get("/jobs/{job_id}")
+async def get_job_status(job_id: str) -> JobStatusResponse:
+    """Get current status of a background job."""
+    job = await jobs_repo.get(job_id)
+
+    if job is None:
+        raise HTTPException(404, f"Job {job_id} not found")
+
+    return JobStatusResponse(
+        job_id=job.id,
+        status=job.status.value,
+        created_at=job.created_at,
+        started_at=job.started_at,
+        completed_at=job.completed_at,
+        result=job.result if job.status == JobStatus.SUCCEEDED else None,
+        error=job.error if job.status == JobStatus.FAILED else None,
+        # Helpful for clients
+        is_terminal=job.status in (JobStatus.SUCCEEDED, JobStatus.FAILED),
+    )
+```
+
+### Pattern 7: Task Chaining and Workflows
+
+Compose complex workflows from simple tasks.
+
+```python
+from celery import chain, group, chord
+
+# Simple chain: A → B → C
+workflow = chain(
+    extract_data.s(source_id),
+    transform_data.s(),
+    load_data.s(destination_id),
+)
+
+# Parallel execution: A, B, C all at once
+parallel = group(
+    send_email.s(user_email),
+    send_sms.s(user_phone),
+    update_analytics.s(event_data),
+)
+
+# Chord: Run tasks in parallel, then a callback
+# Process all items, then send completion notification
+workflow = chord(
+    [process_item.s(item_id) for item_id in item_ids],
+    send_completion_notification.s(batch_id),
+)
+
+workflow.apply_async()
+```
+
+### Pattern 8: Alternative Task Queues
+
+Choose the right tool for your needs.
+
+**RQ (Redis Queue)**: Simple, Redis-based
+```python
+from rq import Queue
+from redis import Redis
+
+queue = Queue(connection=Redis())
+job = queue.enqueue(send_email, "user@example.com", "Subject", "Body")
+```
+
+**Dramatiq**: Modern Celery alternative
+```python
+import dramatiq
+from dramatiq.brokers.redis import RedisBroker
+
+dramatiq.set_broker(RedisBroker())
+
+@dramatiq.actor
+def send_email(to: str, subject: str, body: str) -> None:
+    email_client.send(to, subject, body)
+```
+
+**Cloud-native options:**
+- AWS SQS + Lambda
+- Google Cloud Tasks
+- Azure Functions
+
+## Best Practices Summary
+
+1. **Return immediately** - Don't block requests for long operations
+2. **Persist job state** - Enable status polling and debugging
+3. **Make tasks idempotent** - Safe to retry on any failure
+4. **Use idempotency keys** - For external service calls
+5. **Set timeouts** - Both soft and hard limits
+6. **Implement DLQ** - Capture permanently failed tasks
+7. **Log transitions** - Track job state changes
+8. **Retry appropriately** - Exponential backoff for transient errors
+9. **Don't retry permanent failures** - Validation errors, invalid credentials
+10. **Monitor queue depth** - Alert on backlog growth