@coralai/sps-cli 0.41.2 → 0.43.0

package/skills/python/references/async.md

@@ -0,0 +1,218 @@
+# Python Async / Concurrency
+
+`asyncio` patterns for I/O-bound concurrency. For CPU-bound work, use processes, not async.
+
+## When to use what
+
+| Workload | Tool | Why |
+|---|---|---|
+| I/O-bound (network, files, DB) | `asyncio` | One thread, thousands of connections |
+| CPU-bound (crunching, crypto) | `multiprocessing` / `concurrent.futures.ProcessPoolExecutor` | GIL blocks threads |
+| Blocking library you can't async-ify | `asyncio.to_thread()` | Offload to the default thread pool |
+| Parallelizing blocking syscalls | `concurrent.futures.ThreadPoolExecutor` | Simple, no asyncio required |
+
+`asyncio` without I/O is pointless — a `async def` that only does CPU work gives you no concurrency.
+
+## `async def` / `await` basics
+
+```python
+import asyncio
+import httpx
+
+async def fetch(url: str) -> str:
+    async with httpx.AsyncClient() as client:
+        r = await client.get(url)
+        return r.text
+
+async def main() -> None:
+    html = await fetch("https://example.com")
+    print(len(html))
+
+asyncio.run(main())
+```
+
+Rule: never call `asyncio.run()` from inside already-running async code. It's the entry point, not a utility.
+
+## Concurrency — run coroutines in parallel
+
+### `asyncio.gather` (legacy, still common)
+
+```python
+async def fetch_all(urls: list[str]) -> list[str]:
+    return await asyncio.gather(*(fetch(u) for u in urls))
+
+# With return_exceptions — failures don't cancel siblings
+results = await asyncio.gather(*tasks, return_exceptions=True)
+for r in results:
+    if isinstance(r, Exception):
+        log.warning("one failed: %s", r)
+```
+
+### `asyncio.TaskGroup` (Python 3.11+) — preferred
+
+Structured concurrency: if any task raises, siblings are cancelled and errors are aggregated into an `ExceptionGroup`.
+
+```python
+async def fetch_all(urls: list[str]) -> list[str]:
+    async with asyncio.TaskGroup() as tg:
+        tasks = [tg.create_task(fetch(u)) for u in urls]
+    return [t.result() for t in tasks]
+
+# Error handling
+try:
+    await fetch_all(urls)
+except* httpx.ConnectError as eg:
+    log.warning("network failures: %d", len(eg.exceptions))
+except* httpx.HTTPStatusError as eg:
+    log.error("HTTP errors: %s", [e.response.status_code for e in eg.exceptions])
+```
+
+Prefer `TaskGroup` over `gather` in new code — cancellation is correct by default.
+
+## Timeouts
+
+### Python 3.11+: `asyncio.timeout`
+
+```python
+async def with_deadline():
+    async with asyncio.timeout(5.0):
+        return await slow_operation()
+
+# Nested / reschedulable
+async with asyncio.timeout(None) as cm:
+    cm.reschedule(asyncio.get_running_loop().time() + 10)
+    ...
+```
+
+### Older: `asyncio.wait_for`
+
+```python
+result = await asyncio.wait_for(slow_operation(), timeout=5.0)
+```
+
+## Cancellation
+
+Cancellation is a `CancelledError` injected at the next `await`. Rules:
+
+- **Never swallow `CancelledError`** except to run cleanup; always re-raise.
+- Cleanup in `finally` must itself be fast and cancellation-safe.
+- `asyncio.shield()` protects a critical section from outer cancellation.
+
+```python
+async def worker():
+    try:
+        while True:
+            await do_work()
+    except asyncio.CancelledError:
+        await cleanup()    # fast, idempotent
+        raise              # REQUIRED — don't swallow
+
+async def critical_write(path, data):
+    # Outer cancel won't interrupt the write
+    await asyncio.shield(write_file(path, data))
+```
+
+## Offloading blocking code
+
+Never call blocking code from an async function — it stalls the event loop for everyone.
+
+```python
+# Wrong: blocks the event loop
+async def handler(req):
+    data = requests.get(req.url).text   # ❌ blocking library in async code
+    return data
+
+# Right: offload to thread pool
+async def handler(req):
+    data = await asyncio.to_thread(requests.get, req.url)
+    return data.text
+
+# Or use an async-native library
+async def handler(req):
+    async with httpx.AsyncClient() as client:
+        r = await client.get(req.url)
+        return r.text
+```
+
+`time.sleep()` in async code is always a bug — use `await asyncio.sleep()`.
+
+## Async iteration & generators
+
+```python
+# Async iterator
+async def stream_lines(url: str) -> AsyncIterator[str]:
+    async with httpx.AsyncClient() as client:
+        async with client.stream("GET", url) as resp:
+            async for line in resp.aiter_lines():
+                yield line
+
+async for line in stream_lines(url):
+    process(line)
+
+# Async comprehension
+urls = [u async for u in stream_urls() if u.startswith("https://")]
+```
+
+## Async context managers
+
+```python
+from contextlib import asynccontextmanager
+
+@asynccontextmanager
+async def db_transaction(conn):
+    tx = await conn.begin()
+    try:
+        yield tx
+    except Exception:
+        await tx.rollback()
+        raise
+    else:
+        await tx.commit()
+
+async with db_transaction(conn) as tx:
+    await tx.execute(...)
+```
+
+## Backpressure with semaphores
+
+Limit in-flight concurrency when the downstream can't take unlimited load.
+
+```python
+async def fetch_bounded(urls: list[str], limit: int = 10) -> list[str]:
+    sem = asyncio.Semaphore(limit)
+
+    async def one(u):
+        async with sem:
+            return await fetch(u)
+
+    async with asyncio.TaskGroup() as tg:
+        tasks = [tg.create_task(one(u)) for u in urls]
+    return [t.result() for t in tasks]
+```
+
+## Anti-patterns
+
+| Anti-pattern | Fix |
+|---|---|
+| `asyncio.run()` inside async code | Pass awaitables up; `asyncio.run()` is the entry point only |
+| `time.sleep()` in async | `await asyncio.sleep()` |
+| `requests` / blocking HTTP in async | Use `httpx.AsyncClient` or `asyncio.to_thread` |
+| Swallowing `CancelledError` | Always re-raise after cleanup |
+| Fire-and-forget `asyncio.create_task(x)` without keeping a reference | Reference dropped → task can be garbage-collected mid-flight |
+| Mixing threads and asyncio naively | Use `asyncio.to_thread` / `loop.run_in_executor`, not raw `threading` |
+| Using async just because "it's modern" | Async has real overhead; pure CPU or simple scripts don't need it |
+
+## Running background tasks correctly
+
+```python
+# Wrong: task may be GC'd before it runs
+asyncio.create_task(background())
+
+# Right: keep a reference
+_background_tasks: set[asyncio.Task] = set()
+
+def schedule(coro):
+    t = asyncio.create_task(coro)
+    _background_tasks.add(t)
+    t.add_done_callback(_background_tasks.discard)
+```

package/skills/python/references/error-handling.md

@@ -0,0 +1,254 @@
+# Python Error Handling
+
+Exception design and handling patterns.
+
+## Specific exceptions only
+
+Never `except:` (bare). Never `except Exception:` except at the top of a daemon/server loop.
+
+```python
+# Good: specific exceptions, each with its own response
+def load_config(path: str) -> Config:
+    try:
+        with open(path) as f:
+            return Config.from_json(f.read())
+    except FileNotFoundError as e:
+        raise ConfigError(f"Config not found: {path}") from e
+    except json.JSONDecodeError as e:
+        raise ConfigError(f"Invalid JSON in {path}: {e}") from e
+    except PermissionError as e:
+        raise ConfigError(f"No read permission: {path}") from e
+
+# Bad: silent catch-all
+def load_config(path: str) -> Config:
+    try:
+        with open(path) as f:
+            return Config.from_json(f.read())
+    except:
+        return None  # caller has no idea what went wrong
+```
+
+## Exception chaining (`from`)
+
+Always preserve the original traceback when re-raising.
+
+```python
+def process(data: str) -> Result:
+    try:
+        parsed = json.loads(data)
+    except json.JSONDecodeError as e:
+        # `from e` attaches the original exception
+        raise ValueError(f"Bad data: {data!r}") from e
+
+# Output traceback shows both:
+#   ValueError: Bad data: '...'
+#   The above exception was the direct cause of the following:
+#   json.JSONDecodeError: ...
+```
+
+Use `raise ... from None` to **hide** the chain (rare — only when the chain is noise).
+
+## Custom exception hierarchy
+
+One base class per module/service. Subclasses for specific failures.
+
+```python
+# errors.py
+class AppError(Exception):
+    """Base exception for this application."""
+
+class ValidationError(AppError):
+    """Input validation failed."""
+
+class NotFoundError(AppError):
+    """Resource not found."""
+
+class AuthError(AppError):
+    """Authentication or authorization failed."""
+
+class ExternalServiceError(AppError):
+    """Upstream dependency failed."""
+
+# Usage
+def get_user(user_id: str) -> User:
+    user = db.find_user(user_id)
+    if not user:
+        raise NotFoundError(f"User {user_id}")
+    return user
+
+# Callers catch at the right level
+try:
+    user = get_user(uid)
+except NotFoundError:
+    return 404
+except AppError:
+    return 500  # any other app error
+except Exception:
+    logger.exception("unexpected")
+    return 500
+```
+
+## Rules
+
+| Rule | Why |
+|---|---|
+| Catch only exceptions you can handle | If you can't recover, let it propagate |
+| Never catch `BaseException` | That includes `KeyboardInterrupt` and `SystemExit` |
+| Log before re-raising | Otherwise the log line says "unexpected" when you actually expected it |
+| Specific exception types | Callers can pattern-match; `Exception` forces logs to be the only debugging aid |
+| Use `try/except/else` for 2-phase operations | `else` runs only if no exception — clearer than nesting |
+
+## `try/except/else/finally`
+
+```python
+def fetch(url: str) -> Response:
+    try:
+        response = http.get(url)
+    except TimeoutError:
+        return fallback_response()
+    except HTTPError as e:
+        logger.error("http error: %s", e)
+        raise
+    else:
+        # runs only on success — cleaner than putting this in try
+        cache.set(url, response)
+        return response
+    finally:
+        # always runs
+        release_connection()
+```
+
+## Context manager for error handling
+
+For repeated try/except patterns, wrap in a context manager.
+
+```python
+from contextlib import contextmanager
+
+@contextmanager
+def as_domain_error(target_type: type[Exception], msg: str):
+    """Re-raise any exception as target_type(msg)."""
+    try:
+        yield
+    except Exception as e:
+        raise target_type(msg) from e
+
+# Usage
+with as_domain_error(ConfigError, "Failed to load config"):
+    with open('config.json') as f:
+        return json.load(f)
+```
+
+## Validation vs exceptions
+
+For user-input validation, prefer returning a result object over raising.
+
+```python
+from dataclasses import dataclass
+
+@dataclass
+class ValidationResult:
+    valid: bool
+    errors: list[str]
+
+def validate_user(data: dict) -> ValidationResult:
+    errors = []
+    if not data.get('email'):
+        errors.append('email required')
+    if 'age' in data and not 0 <= data['age'] <= 150:
+        errors.append('age must be 0-150')
+    return ValidationResult(valid=not errors, errors=errors)
+
+# Caller
+result = validate_user(input)
+if not result.valid:
+    return {"errors": result.errors}, 400
+```
+
+Raise exceptions for **exceptional** conditions. Validation failure is **expected**.
+
+## Exception groups (Python 3.11+)
+
+For reporting multiple failures at once — concurrent tasks, batch operations, validation aggregates. Use `ExceptionGroup` and `except*`.
+
+```python
+def import_batch(records: list[dict]) -> None:
+    errors = []
+    for r in records:
+        try:
+            import_one(r)
+        except (ValidationError, DBError) as e:
+            errors.append(e)
+    if errors:
+        raise ExceptionGroup("batch import failed", errors)
+
+# Caller: except* matches by type across the group
+try:
+    import_batch(data)
+except* ValidationError as eg:
+    for e in eg.exceptions:
+        log.warning("invalid: %s", e)
+except* DBError as eg:
+    for e in eg.exceptions:
+        log.error("db failure: %s", e)
+```
+
+`asyncio.TaskGroup` (3.11+) raises `ExceptionGroup` natively when multiple child tasks fail.
+
+## Logging exceptions
+
+Use `logger.exception()` inside an except block — it auto-includes the traceback.
+
+```python
+import logging
+logger = logging.getLogger(__name__)
+
+try:
+    risky_operation()
+except ExternalServiceError:
+    # logger.exception is equivalent to logger.error(exc_info=True)
+    logger.exception("External service failed")
+    # Re-raise or handle
+    raise
+```
+
+**Never** use `print(e)` in production — print goes to stdout, not your log pipeline, and loses the traceback.
+
+## Common mistakes
+
+```python
+# ❌ Mutating `except` clause without `from`
+try: ...
+except ValueError:
+    raise TypeError("wrong type")   # original traceback lost
+
+# ✅
+try: ...
+except ValueError as e:
+    raise TypeError("wrong type") from e
+
+
+# ❌ Catching to "convert to None"
+try:
+    return lookup(key)
+except KeyError:
+    return None  # if this is your pattern, use dict.get(key) — simpler
+
+# ✅
+return mapping.get(key)
+
+
+# ❌ Overcatching hides bugs
+try:
+    data = get_user(uid)
+    data.name = new_name       # if this raises, you misclassify it as "user not found"
+except Exception:
+    return 404
+
+# ✅
+try:
+    data = get_user(uid)
+except NotFoundError:
+    return 404
+data.name = new_name     # let real bugs propagate
+```