start-vibing-stacks 2.16.0 → 2.18.0

package/stacks/python/skills/async-patterns/SKILL.md

@@ -1,103 +1,207 @@
 ---
 name: async-patterns
-version: 1.0.0
+version: 2.0.0
+description: "Async Python concurrency patterns for Python 3.13 / 3.14. Covers structured concurrency with `asyncio.TaskGroup` (3.11+), the new `asyncio.timeout()` context manager (3.11+, replaces `wait_for`), `asyncio.Runner` for one-shot scripts, AnyIO for backend-agnostic code (asyncio + trio), httpx.AsyncClient connection pooling, semaphore-based rate limiting, producer/consumer with backpressure, free-threaded mode (PEP 779 — officially supported in 3.14) for true CPU parallelism. Invoke when writing any async/await code, fan-out/fan-in flows, or deciding between threads and async."
 ---
 
-# Async Python Patterns — Concurrency & Performance
+# Async Patterns — Python 3.13/3.14
 
-**ALWAYS invoke when writing async/await code or concurrent operations.**
+**ALWAYS invoke when writing async/await code, fan-out flows, or rate-limited callers.**
 
-## Core: asyncio.gather vs TaskGroup
+## Decision Map — when to reach for what
+
+```
+Job kind?
+├── I/O-bound (HTTP, DB, file, queue)     → asyncio (one event loop)
+├── CPU-bound, single-threaded            → just call it; don't async
+├── CPU-bound, multi-thread, GIL build    → multiprocessing (workaround for GIL)
+└── CPU-bound, multi-thread, 3.14 free-th → threading on python3.14t (real parallelism)
+```
+
+Free-threaded mode is now **officially supported** in Python 3.14 (PEP 779). Ships as a separate binary (`python3.14t`); per-thread overhead ~10–15% slower than the GIL build, so only switch when you actually need parallel CPU.
+
+## Structured Concurrency — `TaskGroup` over `gather`
 
 ```python
 import asyncio
 
-# Gather — run multiple coroutines concurrently
-async def fetch_all():
-    results = await asyncio.gather(
-        fetch_users(),
-        fetch_products(),
-        fetch_orders(),
-    )
-    users, products, orders = results
-
-# TaskGroup (Python 3.11+) — structured concurrency with proper cancellation
 async def fetch_all_safe():
     async with asyncio.TaskGroup() as tg:
-        t1 = tg.create_task(fetch_users())
-        t2 = tg.create_task(fetch_products())
-    return t1.result(), t2.result()
-    # If any task fails, ALL are cancelled
+        users    = tg.create_task(fetch_users())
+        products = tg.create_task(fetch_products())
+        orders   = tg.create_task(fetch_orders())
+    # If ANY task raises, all siblings are cancelled, exception(s) re-raised as ExceptionGroup
+    return users.result(), products.result(), orders.result()
 ```
 
-## HTTP Client (httpx)
+`TaskGroup` (3.11+) is the modern default. It propagates exceptions as `ExceptionGroup` and cancels siblings on failure — exactly what you want. `asyncio.gather(...)` only swallows errors when you pass `return_exceptions=True`, which is rarely correct.
+
+```python
+# OK only when partial failure is the desired semantics
+results = await asyncio.gather(*coros, return_exceptions=True)
+ok = [r for r in results if not isinstance(r, Exception)]
+errors = [r for r in results if isinstance(r, Exception)]
+```
+
+## Timeouts — `asyncio.timeout()` over `wait_for`
+
+```python
+# 3.11+ — context manager, composes inside TaskGroup, supports deadlines
+async def fetch_with_budget():
+    try:
+        async with asyncio.timeout(5):
+            return await fetch_data()
+    except TimeoutError:
+        return default_value
+
+# Deadline-style — useful for chained operations sharing a budget
+async def call_with_deadline(deadline_ts: float):
+    async with asyncio.timeout_at(deadline_ts):
+        await step1()
+        await step2()
+```
+
+Old `asyncio.wait_for(coro, 5)` still works but doesn't compose well inside `TaskGroup` — prefer `timeout()`.
+
+## HTTP Client — httpx (reuse the client)
 
 ```python
 import httpx
 
-# REUSE client (connection pooling)
-async def fetch_data():
-    async with httpx.AsyncClient(timeout=10.0) as client:
-        response = await client.get("https://api.example.com/data")
-        response.raise_for_status()
-        return response.json()
-
-# Concurrent requests
-async def fetch_many(urls: list[str]) -> list[dict]:
-    async with httpx.AsyncClient() as client:
-        tasks = [client.get(url) for url in urls]
-        responses = await asyncio.gather(*tasks)
-        return [r.json() for r in responses]
+# REUSE — connection pooling, http/2, keepalive
+async def fetch_data(url: str) -> dict:
+    async with httpx.AsyncClient(
+        timeout=httpx.Timeout(connect=5, read=10, write=10, pool=5),
+        limits=httpx.Limits(max_connections=100, max_keepalive_connections=20),
+        http2=True,
+    ) as client:
+        r = await client.get(url)
+        r.raise_for_status()
+        return r.json()
+
+# In FastAPI — store on app.state in lifespan, reuse for the whole process
+async def lifespan(app):
+    app.state.http = httpx.AsyncClient(timeout=10.0)
+    yield
+    await app.state.http.aclose()
 ```
 
-## Semaphore (rate limiting)
+`httpx.AsyncClient` opened per-request is **wrong** — you lose the pool, pay TCP+TLS handshake every call. Reuse one per process.
+
+## Concurrent calls with bounded fan-out
 
 ```python
-# Limit concurrent operations
-sem = asyncio.Semaphore(10)  # Max 10 concurrent
+async def fetch_many(urls: list[str], client: httpx.AsyncClient) -> list[dict]:
+    sem = asyncio.Semaphore(10)                # max 10 in flight at once
 
-async def fetch_with_limit(url: str):
-    async with sem:
-        async with httpx.AsyncClient() as client:
-            return await client.get(url)
+    async def one(url):
+        async with sem:
+            r = await client.get(url)
+            r.raise_for_status()
+            return r.json()
+
+    async with asyncio.TaskGroup() as tg:
+        tasks = [tg.create_task(one(u)) for u in urls]
+    return [t.result() for t in tasks]
 ```
 
-## Queue Pattern (producer/consumer)
+Always cap fan-out — unbounded `gather`/`TaskGroup` over thousands of URLs will exhaust file descriptors, TLS sessions, or the remote rate limit.
+
+## Producer / Consumer — with backpressure
 
 ```python
-async def producer(queue: asyncio.Queue):
-    for item in data:
-        await queue.put(item)
-    await queue.put(None)  # Sentinel
+async def producer(queue: asyncio.Queue, items):
+    for item in items:
+        await queue.put(item)               # blocks if queue full → backpressure
+    for _ in range(N_CONSUMERS):
+        await queue.put(None)               # one sentinel per consumer
 
 async def consumer(queue: asyncio.Queue):
     while True:
         item = await queue.get()
-        if item is None:
-            break
-        await process(item)
-        queue.task_done()
+        try:
+            if item is None:
+                return
+            await process(item)
+        finally:
+            queue.task_done()
+
+async def main():
+    queue = asyncio.Queue(maxsize=100)      # the cap is the backpressure
+    async with asyncio.TaskGroup() as tg:
+        tg.create_task(producer(queue, items))
+        for _ in range(N_CONSUMERS):
+            tg.create_task(consumer(queue))
+```
+
+## Entry-points — `asyncio.Runner` (3.11+)
+
+```python
+import asyncio
+
+async def main(): ...
+
+# Modern — explicit lifecycle, works repeatedly, handles uvloop substitution
+with asyncio.Runner() as runner:
+    runner.run(main())
+```
+
+`asyncio.run(main())` is fine for one-shot scripts; `Runner` is better when you need to call multiple top-level coroutines or swap event loops (uvloop in production).
+
+## AnyIO — when the library must support both backends
+
+If you ship a library used by both asyncio and trio consumers, write against `anyio`:
+
+```python
+import anyio
+from anyio import create_task_group, fail_after
+
+async def fetch(url): ...
 
 async def main():
-    queue = asyncio.Queue(maxsize=100)  # Backpressure
-    await asyncio.gather(producer(queue), consumer(queue))
+    async with create_task_group() as tg:
+        with fail_after(5):                  # AnyIO's timeout
+            for url in urls:
+                tg.start_soon(fetch, url)
+
+anyio.run(main)                              # works on asyncio AND trio
 ```
 
-## Timeouts
+## Threads + async — the right escape hatch
+
+Sync code that you can't replace (legacy SDKs, blocking C extensions) must run **off the event loop**:
 
 ```python
-# Always add timeouts to external calls
-try:
-    result = await asyncio.wait_for(fetch_data(), timeout=5.0)
-except asyncio.TimeoutError:
-    logger.warning("External API timed out")
-    result = default_value
+import asyncio
+
+# Run blocking CPU/IO in the default thread pool
+result = await asyncio.to_thread(blocking_function, arg1, arg2)
+
+# Bigger pool for many concurrent blocking calls
+loop = asyncio.get_running_loop()
+with concurrent.futures.ThreadPoolExecutor(max_workers=32) as pool:
+    result = await loop.run_in_executor(pool, blocking_function, arg)
 ```
 
+For pure-CPU work on the standard GIL build, prefer `multiprocessing` or `concurrent.futures.ProcessPoolExecutor` — threads won't go faster.
+
 ## FORBIDDEN
 
-1. **`time.sleep()` in async code** — use `await asyncio.sleep()`
-2. **Sync HTTP in async context** — use `httpx.AsyncClient`, not `requests`
-3. **No timeout on external calls** — always `asyncio.wait_for()` or `httpx.Timeout`
-4. **Creating new client per request** — reuse with `async with`
-5. **Bare `asyncio.gather` without error handling** — use `return_exceptions=True` or TaskGroup
+| Pattern | Why |
+|---|---|
+| `time.sleep(x)` inside `async def` | Blocks the entire event loop — use `await asyncio.sleep(x)` |
+| `requests.get(...)` inside `async def` | Blocks event loop — use `httpx.AsyncClient` |
+| Bare `asyncio.gather(...)` without error handling | Swallows exceptions of completed tasks; use `TaskGroup` |
+| `asyncio.wait_for` instead of `asyncio.timeout()` | Older API, doesn't compose inside TaskGroup |
+| New `httpx.AsyncClient()` per request | Loses pool/keepalive — reuse |
+| Unbounded `TaskGroup` over user-provided list | DoS yourself — use Semaphore + cap |
+| Mixing `asyncio` and `trio` directly | Use `anyio` to bridge or pick one |
+| `asyncio.run` inside an existing loop | RuntimeError — use `asyncio.get_running_loop()` and `await` |
+| Banking on free-threading for an I/O-bound API | asyncio is faster; free-threading buys CPU parallelism only |
+
+## See Also
+
+- `python-patterns` — async vs sync decision; free-threading rules
+- `fastapi-patterns` — lifespan-managed http client, timeouts in routes
+- `python-performance` — when to actually reach for `python3.14t`
+- `pytest-testing` — testing async code (pytest-asyncio 1.0 + AnyIO)

package/stacks/python/skills/django-patterns/SKILL.md

@@ -1,12 +1,34 @@
 ---
 name: django-patterns
-version: 1.0.0
+version: 2.0.0
+description: "Django 5.2 LTS (April 2025, supported through April 2028) patterns for full-stack Python. Covers Composite Primary Keys (new in 5.2), `select_related`/`prefetch_related` for N+1 prevention, fat-model/thin-view discipline with custom managers, Django REST Framework viewsets/serializers/permissions, **async views** (5.0+) plus the honest caveat that the ORM remains synchronous — `sync_to_async()` is required when calling ORM from `async def`. Includes `pyproject.toml` + uv setup, migration discipline, and the upgrade timeline (Django 4.2 LTS support ends April 2026 — projects must be on 5.2)."
 ---
 
-# Django Patterns — Full-Stack Python (Django 5+)
+# Django Patterns — Django 5.2 LTS (2025–2028)
 
 **ALWAYS invoke when writing Django models, views, or serializers.**
 
+## Version Policy (2026)
+
+| Branch | Released | Support ends | Status |
+|---|---|---|---|
+| **5.2 LTS** | April 2025 | **April 2028** | ✅ Use this for new projects |
+| 5.1 | Aug 2024 | Apr 2025 | EOL |
+| 4.2 LTS | Apr 2023 | **April 2026** | ⚠️ Plan upgrade NOW if still on it |
+
+Django 5.2 brings **Composite Primary Keys**, automatic model imports in `manage.py shell`, and simplified `BoundField` overrides. It's the LTS to standardise on for the next 2 years.
+
+## Setup with uv
+
+```bash
+uv init my-django-app && cd my-django-app
+uv add django>=5.2 djangorestframework psycopg[binary] django-environ
+uv add --dev pytest pytest-django ruff mypy django-stubs
+
+uv run django-admin startproject config .
+uv run python manage.py startapp users
+```
+
 ## Model Design (Fat Models, Thin Views)
 
 ```python
@@ -77,16 +99,43 @@ class UserViewSet(viewsets.ModelViewSet):
     pagination_class = PageNumberPagination
 ```
 
-## Async Views (Django 5.0+)
+## Composite Primary Keys (new in 5.2)
 
 ```python
-# Use async when calling external APIs or heavy I/O
+from django.db.models import CompositePrimaryKey
+
+class OrderLine(models.Model):
+    pk = CompositePrimaryKey("order", "product")
+    order   = models.ForeignKey(Order,   on_delete=models.CASCADE)
+    product = models.ForeignKey(Product, on_delete=models.PROTECT)
+    qty     = models.PositiveIntegerField()
+```
+
+Use composite PKs for join tables that have no business identity of their own — saves an autoincrement column and an index.
+
+## Async Views (5.0+) — and the ORM caveat
+
+```python
+import httpx
+from django.http import JsonResponse
+from asgiref.sync import sync_to_async
+
+# OK — purely external I/O, no ORM
 async def fetch_external_data(request):
-    async with httpx.AsyncClient() as client:
-        response = await client.get('https://api.example.com/data')
+    async with httpx.AsyncClient(timeout=10.0) as client:
+        response = await client.get("https://api.example.com/data")
     return JsonResponse(response.json())
+
+# ORM is still SYNCHRONOUS — you MUST adapt it
+async def list_users(request):
+    users = await sync_to_async(list, thread_sensitive=True)(
+        User.objects.filter(is_active=True)[:50]
+    )
+    return JsonResponse({"users": [u.email for u in users]})
 ```
 
+Django itself notes: *"We're still working on async support for the ORM and other parts of Django."* Until that lands, calling ORM in `async def` without `sync_to_async()` will either deadlock or raise `SynchronousOnlyOperation`. **For DB-heavy apps in 2026, sync views remain the default**; reach for async only when most of the work is external I/O.
+
 ## Migrations Best Practices
 
 ```bash
@@ -97,10 +146,52 @@ python manage.py migrate
 python manage.py makemigrations --check --dry-run
 ```
 
+## Settings — env-driven, no surprises
+
+```python
+# config/settings.py
+import environ
+from pathlib import Path
+
+env = environ.Env(DEBUG=(bool, False))
+BASE_DIR = Path(__file__).resolve().parent.parent
+environ.Env.read_env(BASE_DIR / ".env")
+
+SECRET_KEY = env("SECRET_KEY")
+DEBUG      = env("DEBUG")
+ALLOWED_HOSTS = env.list("ALLOWED_HOSTS", default=[])
+
+DATABASES = {"default": env.db()}              # parses DATABASE_URL
+CACHES    = {"default": env.cache()}           # parses CACHE_URL
+
+# Security defaults — see api-security-python
+SECURE_HSTS_SECONDS = 31_536_000
+SECURE_HSTS_INCLUDE_SUBDOMAINS = True
+SECURE_HSTS_PRELOAD = True
+SESSION_COOKIE_SECURE = True
+SESSION_COOKIE_HTTPONLY = True
+CSRF_COOKIE_SECURE = True
+```
+
 ## FORBIDDEN
 
-1. **Logic in views** — fat models, thin views
-2. **N+1 queries** — always `select_related`/`prefetch_related`
-3. **`objects.all()` without limit** — paginate or `.only()`
-4. **Raw SQL without parameterization** — use ORM or `params=[]`
-5. **Skipping migrations** — always `makemigrations` after model changes
+| Anti-pattern | Reason |
+|---|---|
+| Logic in views | Fat models, thin views — keeps logic testable & reusable |
+| N+1 queries | Always `select_related` (FK) / `prefetch_related` (M2M) |
+| `Model.objects.all()` without `.only()`/pagination | Loads whole table into memory |
+| Raw SQL with f-strings | SQL injection — use ORM or `params=[...]` |
+| Skipping migrations | `makemigrations --check --dry-run` in CI |
+| ORM in `async def` without `sync_to_async()` | `SynchronousOnlyOperation` / deadlock |
+| Disabling `CsrfViewMiddleware` globally | Wide-open CSRF |
+| Plain `pip` for new projects | Use `uv` |
+| Staying on Django 4.2 LTS past April 2026 | Out of security support |
+| New project on Django < 5.2 | 5.2 is the current LTS — start here |
+
+## See Also
+
+- `api-security-python` — Django settings checklist (HSTS, CSRF, sessions, CORS)
+- `pydantic-validation` — DRF-Spectacular + Pydantic for OpenAPI (when DRF serializers aren't enough)
+- `pytest-testing` — `pytest-django` setup
+- `_shared/skills/postgres-patterns` — `uuidv7()`, virtual generated columns (PG18) usable from Django
+- `_shared/skills/observability` — Django + structlog + OpenTelemetry