start-vibing-stacks 2.17.0 → 2.18.0

package/stacks/python/skills/pytest-testing/SKILL.md

@@ -1,17 +1,37 @@
 ---
 name: pytest-testing
-version: 1.0.0
+version: 2.0.0
+description: "Pytest 9 (Nov 2025) testing patterns for Python 3.13/3.14. Covers conftest fixtures with autouse cleanup, parametrize + the new pytest 9 subtests API, async testing with pytest-asyncio 1.0 (May 2025 — `event_loop` fixture removed) and AnyIO for backend-agnostic tests, httpx.AsyncClient + ASGITransport as the FastAPI test client, DB rollback fixtures, mocking with `unittest.mock.AsyncMock`, parallel runs with pytest-xdist + sharding for CI, coverage with `--cov-fail-under` gate, and uv-friendly invocation. Invoke after writing any feature, fixture, or before merging."
 ---
 
-# Pytest Testing — Python Testing Patterns
+# Pytest 9 — Python Testing Patterns (2026)
 
-**ALWAYS invoke AFTER implementing any feature.**
+**ALWAYS invoke AFTER implementing a feature, before opening a PR, and as part of CI.**
+
+## Toolchain
+
+| Tool | Version | Notes |
+|---|---|---|
+| `pytest` | **9.0+** (Nov 5, 2025) | Subtests built-in; new collection internals |
+| `pytest-asyncio` | **1.0+** (May 26, 2025) | `event_loop` fixture removed; preliminary 3.14 support |
+| `anyio[pytest]` | 4.x | Backend-agnostic async tests (asyncio + trio) |
+| `pytest-cov` | 6.x | `--cov` + `--cov-fail-under` |
+| `pytest-xdist` | 3.x | Parallel + sharding for CI |
+| `httpx` | 0.27+ | `AsyncClient` + `ASGITransport` for FastAPI |
+| `pytest-mock` | optional | `mocker` fixture wrapping unittest.mock |
+
+Install via uv:
+
+```bash
+uv add --dev pytest pytest-asyncio pytest-cov pytest-xdist httpx anyio
+```
 
 ## Structure
 
 ```
 tests/
-├── conftest.py          # Shared fixtures
+├── conftest.py            # Shared fixtures (event-loop policy, DB, client)
+├── factories.py           # Test data factories (uuid + faker)
 ├── unit/
 │   ├── test_services.py
 │   └── test_models.py
@@ -22,92 +42,190 @@ tests/
     └── test_flows.py
 ```
 
-## Fixtures
+## `pyproject.toml` configuration
+
+```toml
+[tool.pytest.ini_options]
+minversion = "9.0"
+asyncio_mode = "auto"                       # @pytest.mark.asyncio not required
+asyncio_default_fixture_loop_scope = "session"
+addopts = [
+    "-ra",                                  # short summary for skip/xfail/error
+    "--strict-markers",
+    "--strict-config",
+    "--cov=app",
+    "--cov-report=term-missing",
+    "--cov-fail-under=80",
+]
+testpaths = ["tests"]
+markers = [
+    "slow: marks tests as slow (deselect with -m 'not slow')",
+    "e2e: end-to-end tests requiring external services",
+]
+```
+
+## Async client fixture (FastAPI)
+
+`pytest-asyncio` 1.0 dropped the `event_loop` fixture. Use the new `asyncio_default_fixture_loop_scope = "session"` setting (above) instead of overriding the loop manually.
 
 ```python
-import pytest
+# tests/conftest.py
+import pytest_asyncio
 from httpx import AsyncClient, ASGITransport
 from app.main import app
-from app.core.config import settings
-
-@pytest.fixture
+from app.db.session import async_session, engine
+from app.db.base import Base
+
+@pytest_asyncio.fixture(scope="session", autouse=True)
+async def _create_schema():
+    async with engine.begin() as conn:
+        await conn.run_sync(Base.metadata.create_all)
+    yield
+    async with engine.begin() as conn:
+        await conn.run_sync(Base.metadata.drop_all)
+    await engine.dispose()
+
+@pytest_asyncio.fixture
 async def client():
     transport = ASGITransport(app=app)
     async with AsyncClient(transport=transport, base_url="http://test") as ac:
         yield ac
 
-@pytest.fixture
-async def db_session():
+@pytest_asyncio.fixture
+async def db():
+    """Per-test session that always rolls back — no test pollutes another."""
     async with async_session() as session:
         yield session
-        await session.rollback()  # Cleanup
-
-@pytest.fixture
-def sample_user():
-    return {"name": "Test User", "email": f"test_{uuid4().hex[:8]}@test.com", "password": "Pass1234!"}
+        await session.rollback()
 ```
 
-## Async Tests
+## AnyIO — backend-agnostic async tests
+
+When the code under test must work for both asyncio and trio (e.g. shared library code), use AnyIO instead of pytest-asyncio:
 
 ```python
 import pytest
 
-@pytest.mark.asyncio
-async def test_create_user(client, sample_user):
-    response = await client.post("/api/v1/users", json=sample_user)
-    assert response.status_code == 201
-    data = response.json()
-    assert data["email"] == sample_user["email"]
-    assert "id" in data
-    assert "password" not in data  # Never leak passwords
-
-@pytest.mark.asyncio
-async def test_get_me_unauthorized(client):
-    response = await client.get("/api/v1/users/me")
-    assert response.status_code == 401
+pytestmark = pytest.mark.anyio        # whole module is async
+
+async def test_works_under_either_backend(anyio_backend):
+    # anyio_backend is parametrised over ['asyncio', 'trio']
+    ...
 ```
 
-## Parameterized Tests
+Configure once in `pyproject.toml`:
+
+```toml
+[tool.pytest.ini_options]
+anyio_mode = "auto"
+```
+
+## Subtests (new in pytest 9)
+
+For dataset-driven tests where you want **each row reported individually** without the parametrize ID overhead:
 
 ```python
-@pytest.mark.parametrize("email,expected", [
-    ("valid@test.com", 201),
-    ("invalid-email", 422),
-    ("", 422),
-])
-async def test_email_validation(client, email, expected):
-    response = await client.post("/api/v1/users", json={"name": "Test", "email": email, "password": "Pass1234!"})
-    assert response.status_code == expected
+def test_email_normalization(subtests):
+    cases = [("A@B.com", "a@b.com"), (" a@b.com ", "a@b.com")]
+    for raw, expected in cases:
+        with subtests.test(raw=raw):
+            assert normalize_email(raw) == expected
+```
+
+Each subtest reports as a distinct outcome — failures don't stop the others.
+
+## Parametrize — for combinatorial inputs
+
+```python
+@pytest.mark.parametrize(
+    "email,status",
+    [
+        ("valid@test.com", 201),
+        ("invalid-email",  422),
+        ("",               422),
+        ("a" * 320,        422),
+    ],
+    ids=["valid", "no-at", "empty", "too-long"],
+)
+async def test_email_validation(client, email, status):
+    body = {"name": "Test", "email": email, "password": "Pass1234!"}
+    r = await client.post("/api/v1/users", json=body)
+    assert r.status_code == status
 ```
 
 ## Mocking
 
 ```python
 from unittest.mock import AsyncMock, patch
+import pytest
 
-@pytest.mark.asyncio
+@pytest.mark.anyio
 async def test_external_api_failure(client):
     with patch("app.services.external.fetch_data", new_callable=AsyncMock) as mock:
         mock.side_effect = ConnectionError("API down")
-        response = await client.get("/api/v1/data")
-        assert response.status_code == 503
+        r = await client.get("/api/v1/data")
+        assert r.status_code == 503
+```
+
+For HTTP mocking specifically, prefer `respx` (drop-in for httpx) over hand-rolled patches.
+
+## Test data factories
+
+```python
+# tests/factories.py
+from uuid import uuid4
+from faker import Faker
+
+fake = Faker()
+
+def user_payload(**overrides):
+    return {
+        "name":     fake.name(),
+        "email":    f"{uuid4().hex[:8]}@test.com",
+        "password": "Pass1234!",
+        **overrides,
+    }
 ```
 
-## Commands
+Factories are functions, not classes — keeps them testable, composable, type-safe.
+
+## Coverage gate
 
 ```bash
-pytest                       # Run all
-pytest -x                    # Stop on first failure
-pytest --tb=short            # Short traceback
-pytest -k "test_create"      # Filter by name
-pytest --cov=app --cov-report=html  # Coverage
-pytest -n auto               # Parallel (pytest-xdist)
+pytest --cov=app --cov-report=term-missing --cov-fail-under=80
 ```
 
+Set the gate per package, raise it incrementally — never lower it. Use `--cov-config=.coveragerc` to exclude generated code (`migrations/`, `__init__.py`).
+
+## CI parallelism + sharding
+
+```bash
+# Local — use all cores
+uv run pytest -n auto
+
+# CI — split tests across N runners (matrix job in GitHub Actions)
+uv run pytest --shard-id=$SHARD_INDEX --num-shards=$TOTAL_SHARDS
+```
+
+`pytest-xdist` distributes tests across processes; `pytest-split` (or the built-in `--shard` style on newer versions) splits across CI runners.
+
 ## FORBIDDEN
 
-1. **No fixtures for cleanup** — always rollback/cleanup
-2. **Hardcoded test data** — use factories/uuid
-3. **Testing implementation** — test behavior, not internals
-4. **Skipping async tests** — use `pytest-asyncio`
-5. **No coverage in CI** — `--cov --cov-fail-under=70`
+| Anti-pattern | Reason |
+|---|---|
+| `@pytest_asyncio.fixture(loop_scope="function")` for DB-heavy suites | Recreates pool every test → slow; use `session` scope + per-test rollback |
+| Defining your own `event_loop` fixture | Removed in pytest-asyncio 1.0 — use `asyncio_default_fixture_loop_scope` |
+| Hardcoded test data (`"test@test.com"`) | Tests collide in parallel — use uuid/faker |
+| Testing private methods (`_calculate_x`) | Test public behaviour, not internals |
+| No cleanup → flaky tests | Always rollback or use isolated DB per test |
+| `print()` for debugging | Use `pytest -s` + `caplog` fixture |
+| Skipping coverage in CI | `--cov-fail-under=N` gate, raise over time |
+| Catching `Exception` then asserting | Use `pytest.raises(SpecificError)` |
+| Importing app at module top in slow tests | Lazy-import in fixtures so collection is fast |
+
+## See Also
+
+- `fastapi-patterns` — endpoints + DI under test
+- `pydantic-validation` — schema fuzzing with hypothesis
+- `async-patterns` — TaskGroup/timeout patterns the tests verify
+- `_shared/skills/playwright-automation` — for browser/E2E coverage

package/stacks/python/skills/python-patterns/SKILL.md

@@ -1,12 +1,32 @@
 ---
 name: python-patterns
-version: 1.0.0
+version: 2.0.0
+description: "Python architecture decisions for Python 3.13 (Oct 2024) / 3.14 (Oct 2025) projects. Framework selection (FastAPI / Django 5.2 LTS / Flask / scripts), async vs sync rules, free-threaded mode awareness (officially supported in 3.14 via PEP 779), modern typing (`X | None`, `TypeIs`, type-param defaults), project structure per app type, error handling, background-task choice. Pairs with the per-framework skills (fastapi-patterns, django-patterns, scripting-automation). Invoke for any new Python project, framework choice, or architectural decision."
 ---
 
-# Python Patterns — Architecture & Decision-Making
+# Python Patterns — Architecture & Decisions (3.13 / 3.14)
 
 **ALWAYS invoke when making Python architecture decisions.**
 
+## Version Policy (2026)
+
+- **Python 3.13** (Oct 7, 2024) — minimum for new projects
+- **Python 3.14** (Oct 7, 2025) — recommended; brings **officially supported free-threaded mode** (PEP 779), template strings (PEP 750), deferred annotation evaluation
+- **Package manager: `uv`** (Astral, acquired by OpenAI Mar 2026) — 10–100× faster than pip, 10–20× faster than Poetry; surpassed Poetry in monthly downloads in early 2026. Pin `uv` for new projects unless a constraint forces Poetry/pip.
+- **Lint + format: `ruff`** (one Rust binary; replaces flake8 + isort + black + pydocstyle + pyupgrade + autoflake)
+- **Type checker: `pyright`** for correctness (97.8% conformance), `ty` (Astral) when Pyright is too slow on huge codebases — still beta but 10–60× faster
+
+## Free-Threaded vs GIL — When to care
+
+| Workload | Build | Notes |
+|---|---|---|
+| Web server (I/O-bound) | Standard GIL | asyncio handles concurrency; free-threading buys little |
+| Mixed I/O + light CPU | Standard GIL | Standard build is faster per-thread |
+| **CPU-bound multi-thread** (parsing, math, ML pre-processing) | **Free-threaded 3.14** | Real parallelism — replaces the multiprocessing dance |
+| Library author | Both | Test with `python3.14t` to flag thread-safety bugs |
+
+The free-threaded interpreter ships as a separate binary (`python3.14t`). It's slower per-thread (~10–15% overhead) than the GIL build — only adopt when you actually need parallel CPU.
+
 ## Framework Selection
 
 ```
@@ -44,12 +64,22 @@ Don't:
 
 ## Type Hints (MANDATORY for public APIs)
 
+Modern syntax — `X | None` over `Optional[X]`, lowercase generics, `TypeIs` for narrowing.
+
 ```python
-from typing import Optional
+from typing import TypeIs
 
-def find_user(id: int) -> Optional[User]: ...
-def process(data: str | dict) -> None: ...
+def find_user(id: int) -> User | None: ...                     # 3.10+ union syntax
+def process(data: str | dict[str, object]) -> None: ...
 def get_items() -> list[Item]: ...
+
+# TypeIs (3.13+) — narrow types in type guards (better than TypeGuard for negative branches)
+def is_admin(user: User | Guest) -> TypeIs[User]:
+    return isinstance(user, User) and user.role == "admin"
+
+# Type parameter defaults (3.13+) — generic classes with sensible defaults
+class Repo[T = User]:
+    def find(self, id: str) -> T | None: ...
 ```
 
 ## Project Structure
@@ -125,7 +155,19 @@ async def not_found_handler(request, exc):
 ## FORBIDDEN
 
 1. **Business logic in routes/views** — use services layer
-2. **Sync libraries in async code** — blocks event loop
+2. **Sync libraries in async code** — blocks event loop (`requests`, `psycopg2` sync, `pymongo` sync, `time.sleep`)
 3. **No type hints on public APIs** — always type
-4. **Raw SQL without parameterization** — injection risk
+4. **Raw SQL without parameterization** — injection risk (use ORM bindings or `:name` / `?` placeholders)
 5. **`import *`** — explicit imports only
+6. **`Optional[X]`** — write `X | None` (3.10+ syntax)
+7. **`pip install` in new projects** — use `uv add` (uv is the 2026 default; pip still fine for legacy)
+8. **Per-tool config files (`.flake8`, `.isort.cfg`, `pyproject` for black + isort + ruff…)** — consolidate under `[tool.ruff]` in `pyproject.toml`
+9. **Banking on free-threading for an I/O-bound web app** — use asyncio; the GIL build is faster
+
+## See Also
+
+- `fastapi-patterns` / `django-patterns` / `scripting-automation` — per-application-type setup
+- `pydantic-validation` — boundary validation (Pydantic V2)
+- `pytest-testing` — pytest 9 + pytest-asyncio 1
+- `async-patterns` — asyncio.timeout, TaskGroup, AnyIO
+- `python-performance` — profiling, free-threading trade-offs

package/stacks/python/skills/python-performance/SKILL.md

@@ -1,11 +1,191 @@
 ---
 name: python-performance
-version: 1.0.0
+version: 2.0.0
+description: "Performance profiling and optimisation for Python 3.13/3.14. Covers cProfile/line-profiler/memory-profiler/py-spy choice, the experimental copy-and-patch JIT in 3.13 (PEP 744 — disabled by default), free-threaded mode in 3.14 (PEP 779 officially supported — when it actually wins vs asyncio + multiprocessing), `functools.cache` (unbounded) vs `lru_cache` (bounded), structural optimisations (set vs list membership, generators for memory, `str.join` vs `+`), bulk DB ops in SQLAlchemy/Django, async caching with redis-py, and Polars (Rust-backed dataframes, ~10× pandas) for data work. Profile FIRST, optimise SECOND."
 ---
 
-# Python Performance — Profiling & Optimization
+# Python Performance — Profiling & Optimisation (3.13 / 3.14)
 
-**ALWAYS invoke when optimizing slow Python code.**
+**ALWAYS invoke when optimising slow Python code. Profile FIRST.**
+
+## What to reach for in 2026
+
+| Symptom | Tool / Pattern |
+|---|---|
+| "Function X is hot" | `cProfile` → `snakeviz`, then `line_profiler` for line-by-line |
+| "Process eats RAM" | `memory_profiler` for line-level, `tracemalloc` for snapshots |
+| "Production is slow but we can't repro" | **`py-spy`** (sampling, no code change, attaches by PID) |
+| "I want a flame graph" | `py-spy record -o profile.svg --pid …` |
+| "Hot Python loop, can't rewrite in C" | Try **3.13 JIT** (`PYTHON_JIT=1`) — still experimental |
+| "Multi-thread CPU-bound, GIL is the wall" | **3.14 free-threaded** build (`python3.14t`) — officially supported |
+| "Tabular data crunching" | **Polars** (Rust-backed, ~10× pandas, lazy frames) |
+| "Pure-Python hot path" | `mypyc`, `cython`, `numba` — pick based on dependency tolerance |
+
+## Profiling
+
+```bash
+# CPU — cumulative time per function
+python -m cProfile -o prof.out app.py
+uv run snakeviz prof.out                          # interactive HTML view
+
+# Line-level (decorate target with @profile, no import needed)
+uv run kernprof -l -v script.py
+
+# Memory — line-level allocations
+uv run python -m memory_profiler script.py
+
+# Production-safe sampling profiler — attach by PID
+py-spy top --pid 12345
+py-spy record -o flame.svg --duration 30 --pid 12345
+```
+
+py-spy is the safest tool for prod: zero code changes, low overhead (~5%), works on a running process.
+
+## Free-threading vs JIT — when each helps
+
+```
+3.13 JIT (PEP 744)
+├── Status: experimental, OFF by default
+├── Win: hot Python bytecode loops (~5-15% on micro-benchmarks)
+└── Enable: build with --enable-experimental-jit OR run PYTHON_JIT=1 (when distro supports it)
+
+3.14 Free-threaded (PEP 779)
+├── Status: OFFICIALLY SUPPORTED
+├── Binary: python3.14t (separate from python3.14)
+├── Win: parallel CPU work across threads — no GIL
+├── Cost: ~10-15% slower per-thread vs GIL build
+└── Use when: CPU-bound multi-thread work where multiprocessing overhead is too high
+```
+
+Don't bank on either for I/O-bound web servers — asyncio dominates that case.
+
+## Caching primitives
+
+```python
+from functools import cache, lru_cache
+
+# Bounded — pick a sensible maxsize for your hot paths
+@lru_cache(maxsize=1024)
+def expensive(n: int) -> int:
+    return sum(range(n))
+
+# Unbounded — only when input space is small AND fixed
+@cache                        # 3.9+, equivalent to @lru_cache(maxsize=None) but faster
+def settings_for(env: str) -> Settings:
+    return Settings(env=env)
+
+# Async — use redis-py async; lru_cache does NOT support coroutines
+import redis.asyncio as redis
+
+cache = redis.from_url("redis://localhost", decode_responses=False)
+
+async def get_user(id: str) -> User:
+    cached = await cache.get(f"user:{id}")
+    if cached:
+        return User.model_validate_json(cached)
+    user = await db.get(User, id)
+    await cache.set(f"user:{id}", user.model_dump_json(), ex=300)
+    return user
+```
+
+## Data structures
+
+```python
+# O(n) → O(1) — set lookup wins by 100×+ on big lists
+big_list = [...]                 # 1M items
+big_set  = set(big_list)
+"target" in big_list             # SLOW
+"target" in big_set              # FAST
+
+# dict.get() over try/except for happy-path
+value = data.get("key", default)
+
+# Specialised collections
+from collections import defaultdict, Counter, deque
+counts = Counter(events)
+queue  = deque(maxlen=1000)      # bounded ring buffer
+```
+
+## Generators — memory wins
+
+```python
+# WRONG — materialises 10M dicts in RAM
+all_rows = [process(x) for x in huge_dataset]
+total    = sum(r["price"] for r in all_rows)
+
+# CORRECT — single pass, constant memory
+total = sum(process(x)["price"] for x in huge_dataset)
+```
+
+Generator expressions are not always faster wall-clock, but they **always** beat list comprehensions on memory.
+
+## String operations
+
+```python
+# O(n²) — Python recreates the string each iteration
+result = ""
+for s in strings:
+    result += s
+
+# O(n) — single allocation
+result = "".join(strings)
+
+# Building structured strings
+parts = [f"row {i}" for i in range(1000)]
+out   = "\n".join(parts)
+```
+
+## Database — bulk over loops
+
+```python
+# SQLAlchemy 2.0 async — bulk insert
+from sqlalchemy import insert
+await db.execute(insert(Item), [{"name": n} for n in names])
+await db.commit()
+
+# Django ORM
+Item.objects.bulk_create([Item(name=n) for n in names], batch_size=1000)
+
+# Avoid the N+1 trap — see django-patterns / fastapi-patterns
+```
+
+## Polars — when pandas is the bottleneck
+
+```python
+import polars as pl
+
+# Lazy — query is optimised before execution
+df = (
+    pl.scan_csv("orders.csv")
+    .filter(pl.col("amount") > 100)
+    .group_by("customer_id")
+    .agg(pl.col("amount").sum().alias("total"))
+    .sort("total", descending=True)
+    .collect(streaming=True)               # streams when bigger than RAM
+)
+```
+
+Polars is Rust-backed, multi-threaded by default, and lazy — typical 5–30× speedup over pandas on aggregation/filter pipelines, plus much lower memory.
+
+## FORBIDDEN
+
+| Anti-pattern | Reason |
+|---|---|
+| Optimising before profiling | "Premature optimisation is the root of all evil" — measure first |
+| `+` for string concat in loops | O(n²) — use `"".join()` |
+| `list` for membership testing | O(n) per lookup — use `set` |
+| Loading whole dataset in memory | Use generators / streaming / pagination |
+| One-by-one DB inserts | Use `bulk_create`/`executemany`/SQLAlchemy `insert(...)` |
+| `lru_cache` on `async def` | Doesn't cache coroutines correctly — use Redis or `aiocache` |
+| Banking on JIT for production wins today | Still experimental in 3.13 — measure on YOUR workload |
+| Switching whole app to free-threaded for "free speed" | Per-thread overhead can make I/O-bound code slower |
+
+## See Also
+
+- `python-patterns` — async vs threads vs processes decision
+- `async-patterns` — TaskGroup / Semaphore / httpx pooling
+- `_shared/skills/observability` — measure latency and memory in prod
+- `_shared/skills/postgres-patterns` — index design, EXPLAIN, AIO in PG18
 
 ## Profiling Tools