@damiangilgonzalez/harness 1.3.7

package/skills/fastapi/references/security.md

@@ -0,0 +1,330 @@
+# Securing a FastAPI service
+
+Concrete, FastAPI-specific hardening on Python 3.12+: argon2-cffi 23+ password hashing, PyJWT
+2.10+ with validated claims and **pinned algorithms**, RBAC dependencies, env-specific CORS,
+shared-store rate limiting, injection-proof queries, `SecretStr` handling, log redaction,
+security headers, and dependency auditing. For language-agnostic theory, **See Also
+`secure-coding`**.
+
+## Password hashing
+
+Use argon2id via argon2-cffi. Never MD5/SHA/plain (no work factor, GPU-crackable). bcrypt is an
+acceptable fallback where argon2 is unavailable. Verification is constant-time and raises on
+mismatch; re-hash on successful login when stored parameters fall behind current defaults.
+
+```python
+from argon2 import PasswordHasher
+from argon2.exceptions import VerifyMismatchError
+
+ph = PasswordHasher()
+
+
+def hash_password(plain: str) -> str:
+    return ph.hash(plain)
+
+
+def verify_password(stored_hash: str, plain: str) -> tuple[bool, str | None]:
+    try:
+        ph.verify(stored_hash, plain)
+    except VerifyMismatchError:
+        return False, None
+    new_hash = ph.hash(plain) if ph.check_needs_rehash(stored_hash) else None
+    return True, new_hash
+```
+
+When `verify_password` returns a non-`None` second element, persist it: the stored hash used
+weaker parameters and was just upgraded transparently on a correct login.
+
+## OAuth2 password flow + JWT
+
+`OAuth2PasswordBearer` declares the token scheme (and powers the Swagger "Authorize" button).
+The login route verifies credentials and returns a short-lived access token.
+
+```python
+from fastapi import APIRouter, Depends
+from fastapi.security import OAuth2PasswordBearer, OAuth2PasswordRequestForm
+
+from app.api.deps import DbSession
+from app.core.security import create_access_token, verify_password
+from app.exceptions import Unauthorized
+from app.services import user_service
+
+oauth2_scheme = OAuth2PasswordBearer(tokenUrl="/auth/login")
+router = APIRouter()
+
+
+@router.post("/login")
+async def login(db: DbSession, form: OAuth2PasswordRequestForm = Depends()) -> dict[str, str]:
+    user = await user_service.get_by_email(db, form.username)
+    if user is None:
+        raise Unauthorized("Invalid credentials")
+    ok, new_hash = verify_password(user.hashed_password, form.password)
+    if not ok:
+        raise Unauthorized("Invalid credentials")
+    if new_hash is not None:
+        user.hashed_password = new_hash
+        await db.flush()
+    return {"access_token": create_access_token(str(user.id)), "token_type": "bearer"}
+```
+
+Mint tokens with explicit `exp`, `iat`, `iss`, `aud`, `sub`; decode by **pinning `algorithms`**
+and requiring the claims — never trust the token header's `alg`.
+
+```python
+from datetime import UTC, datetime, timedelta
+
+import jwt
+
+from app.core.config import get_settings
+
+
+def create_access_token(subject: str) -> str:
+    s = get_settings()
+    now = datetime.now(UTC)
+    payload = {
+        "sub": subject, "iss": s.jwt_issuer, "aud": s.jwt_audience,
+        "iat": now, "exp": now + timedelta(seconds=s.access_token_ttl_seconds),
+    }
+    return jwt.encode(payload, s.jwt_secret.get_secret_value(), algorithm=s.jwt_algorithm)
+
+
+def decode_token(token: str) -> dict:
+    s = get_settings()
+    return jwt.decode(
+        token, s.jwt_secret.get_secret_value(),
+        algorithms=[s.jwt_algorithm],          # pinned — never accept the header's alg
+        audience=s.jwt_audience, issuer=s.jwt_issuer,
+        options={"require": ["exp", "iss", "aud", "sub"]},
+    )
+```
+
+Bad: `jwt.decode(token, key, algorithms=["none"])` or omitting `algorithms` entirely — both
+accept `alg=none`, making tokens forgeable with no signature. For asymmetric keys prefer RS256
+(verify with the public key; sign only on the issuer). Resolve the token into the current user
+in a dependency:
+
+```python
+from typing import Annotated
+
+import jwt
+from fastapi import Depends
+
+from app.api.deps import DbSession
+from app.core.security import decode_token, oauth2_scheme
+from app.exceptions import Unauthorized
+from app.models.user import User
+from app.services import user_service
+
+
+async def get_current_user(
+    db: DbSession, token: Annotated[str, Depends(oauth2_scheme)]
+) -> User:
+    try:
+        payload = decode_token(token)
+    except jwt.PyJWTError as exc:
+        raise Unauthorized("Invalid or expired token") from exc
+    user = await user_service.get_user(db, payload["sub"])
+    if user is None or not user.is_active:
+        raise Unauthorized("User not found or inactive")
+    return user
+
+
+CurrentUser = Annotated[User, Depends(get_current_user)]
+```
+
+## RBAC
+
+A dependency factory that reads a `role` claim and raises `Forbidden` (403) when the caller's
+role is not permitted. Compose it into a router via `dependencies=[Depends(require_roles("admin"))]`
+or as a typed parameter.
+
+```python
+from collections.abc import Callable
+from typing import Annotated
+
+from fastapi import Depends
+
+from app.core.security import decode_token, oauth2_scheme
+from app.exceptions import Forbidden, Unauthorized
+
+
+def require_roles(*allowed: str) -> Callable[..., None]:
+    async def _dep(token: Annotated[str, Depends(oauth2_scheme)]) -> None:
+        try:
+            payload = decode_token(token)
+        except Exception as exc:
+            raise Unauthorized() from exc
+        if payload.get("role") not in allowed:
+            raise Forbidden(f"Requires one of: {', '.join(allowed)}")
+    return _dep
+```
+
+For private resources, prefer returning **404 over 403** when the caller is authenticated but
+not the owner — a 403 confirms the resource exists, leaking its presence to a probing attacker.
+Use 403 only where existence is already public and the action specifically is forbidden.
+
+## CORS
+
+CORS is enforced by the browser, not a server-side authorization control — but a wrong config
+either breaks legitimate frontends or invites credentialed cross-origin abuse. Pin an explicit
+origin list per environment; the `["*"]` + credentials combination is invalid (the browser
+rejects it and Starlette refuses to echo `*` for credentialed requests).
+
+```python
+from fastapi.middleware.cors import CORSMiddleware
+
+# settings.cors_origins, e.g. dev: ["http://localhost:3000"], prod: ["https://app.example.com"]
+app.add_middleware(
+    CORSMiddleware,
+    allow_origins=settings.cors_origins,          # explicit list per env — never ["*"] with creds
+    allow_credentials=bool(settings.cors_origins),
+    allow_methods=["GET", "POST", "PUT", "PATCH", "DELETE"],
+    allow_headers=["Authorization", "Content-Type"],
+    max_age=600,                                  # cache preflight to cut OPTIONS chatter
+)
+```
+
+## Rate limiting
+
+Limits must live in a **shared store** (Redis, or the API gateway), never per-process
+counters. With multiple uvicorn workers, replicas, or serverless instances, per-process limits
+fail open — each process counts independently, so N processes allow N× the intended rate.
+
+```python
+from fastapi import Request
+from fastapi.responses import JSONResponse
+from slowapi import Limiter
+from slowapi.errors import RateLimitExceeded
+from slowapi.util import get_remote_address
+
+limiter = Limiter(key_func=get_remote_address, storage_uri="redis://redis:6379/0")
+
+
+# On the app:
+#   app.state.limiter = limiter
+#   app.add_exception_handler(RateLimitExceeded, rate_limit_handler)
+
+
+def rate_limit_handler(request: Request, exc: RateLimitExceeded) -> JSONResponse:
+    return JSONResponse(
+        status_code=429,
+        headers={"Retry-After": "60"},
+        content={"error": {"code": "rate_limited", "message": "Too many requests", "details": []}},
+    )
+
+
+# On the route:
+#   @router.post("/login")
+#   @limiter.limit("5/minute")
+#   async def login(request: Request, ...): ...
+```
+
+Rate-limit auth endpoints (`/login`, `/register`, password reset) and expensive writes first —
+those are the brute-force and abuse targets.
+
+## Injection
+
+Pydantic validates request bodies; `Query(ge=, le=)` and `Path(...)` bound query/path params.
+For the database, use SQLAlchemy expressions or bound parameters — never interpolate user input
+into SQL.
+
+```python
+from sqlalchemy import select, text
+
+from app.models.user import User
+
+
+# Bad: f-string SQL -> classic SQLi.
+async def bad(db, email: str):
+    return await db.execute(text(f"SELECT * FROM users WHERE email = '{email}'"))  # noqa: S608
+
+
+# Good: ORM expression — value is bound, never concatenated.
+async def good_orm(db, email: str):
+    return await db.execute(select(User).where(User.email == email))
+
+
+# Good: raw SQL with bound params when you truly need text().
+async def good_text(db, email: str):
+    return await db.execute(text("SELECT * FROM users WHERE email = :email"), {"email": email})
+```
+
+## Secret handling
+
+- Type every secret as `SecretStr` in `Settings`; its `repr` renders `**********`, so it cannot
+  leak via logs or tracebacks.
+- Call `.get_secret_value()` only at the point of use (signing/decoding), never to store or log.
+- Keep `.env` gitignored; commit a `.env.example` with names but no values.
+- In production read secrets from a manager (AWS Secrets Manager, Vault, Doppler) injected as
+  env vars; rotate signing keys and support a short overlap window during rotation.
+
+```python
+from app.core.config import get_settings
+
+s = get_settings()
+key = s.jwt_secret.get_secret_value()   # only here, only momentarily
+# logger.info("config", jwt_secret=s.jwt_secret)  # safe: logs "**********", not the value
+```
+
+## Log redaction
+
+Strip credentials and PII before logs are emitted. A structlog processor (or stdlib filter) that
+masks known-sensitive keys, applied app-wide:
+
+```python
+from typing import Any
+
+SENSITIVE = {"authorization", "cookie", "set-cookie", "password", "token", "access_token",
+             "refresh_token", "secret", "ssn", "credit_card"}
+
+
+def redact_processor(_logger: object, _method: str, event: dict[str, Any]) -> dict[str, Any]:
+    for key in list(event):
+        if key.lower() in SENSITIVE:
+            event[key] = "***REDACTED***"
+    return event
+
+
+# Wire into structlog.configure(processors=[..., redact_processor, JSONRenderer()])
+```
+
+## Security headers & misc
+
+```python
+from fastapi import FastAPI
+from starlette.middleware.trustedhost import TrustedHostMiddleware
+
+from app.core.config import get_settings
+
+settings = get_settings()
+in_prod = settings.environment == "production"
+
+# Disable interactive docs in prod if the API is not public.
+app = FastAPI(docs_url=None if in_prod else "/docs", redoc_url=None if in_prod else "/redoc")
+
+# Reject Host-header spoofing / DNS rebinding.
+app.add_middleware(TrustedHostMiddleware, allowed_hosts=["api.example.com", "*.example.com"])
+```
+
+Add response headers via a small middleware: `Strict-Transport-Security`
+(`max-age=63072000; includeSubDomains`), `Content-Security-Policy` tuned to the API,
+`X-Content-Type-Options: nosniff`, `X-Frame-Options: DENY`. Enforce a request body size limit at
+the proxy (e.g. nginx `client_max_body_size`) so oversized payloads never reach the app.
+
+## Dependency audit
+
+```bash
+uv run pip-audit
+uv pip compile --generate-hashes -o requirements.lock pyproject.toml
+uv pip sync requirements.lock
+```
+
+Gate CI on `pip-audit`: a non-zero exit means a dependency has a known CVE. Pin and hash the
+lockfile so installs are reproducible and tamper-evident, and enable Dependabot (or Renovate) to
+open update PRs automatically. Treat a failing audit as a build break, not a warning.
+
+## See Also
+
+- [`secure-coding`](../../secure-coding/SKILL.md) — language-agnostic injection, authz, and secret-handling rules (never leak internals through error responses).
+- [`production.md`](production.md) — TLS termination, proxy headers, observability.
+- [`postgresdb`](../../postgresdb/SKILL.md) — row-level security, DB roles/grants, and PgBouncer auth.

package/skills/fastapi/references/testing.md

@@ -0,0 +1,349 @@
+# Testing async FastAPI
+
+Deep-dive companion to `SKILL.md → Testing (embedded summary)`. The goal: fast, isolated,
+async-correct tests that hit the real ASGI app over `httpx.ASGITransport`, run against a
+**real PostgreSQL 16** inside a per-test transaction that rolls back, and gate CI on
+coverage. No `TestClient`-only patterns, no SQLite-as-Postgres pretending, no leaking state
+between tests.
+
+## Setup & config
+
+Configure pytest once in `pyproject.toml`. `asyncio_mode = "auto"` means every `async def
+test_*` runs without an explicit `@pytest.mark.asyncio`.
+
+```toml
+[tool.pytest.ini_options]
+asyncio_mode = "auto"
+testpaths = ["tests"]
+addopts = [
+    "--strict-markers",
+    "--cov=app",
+    "--cov-report=term-missing",
+    "--cov-fail-under=85",
+]
+filterwarnings = ["error"]
+```
+
+Install the test toolchain as dev dependencies:
+
+```bash
+uv add --dev pytest pytest-asyncio pytest-cov httpx respx testcontainers[postgres]
+```
+
+## Test DB strategy
+
+Test against a **real PostgreSQL 16**, not SQLite. SQLite lacks Postgres types (`UUID`,
+`JSONB`, arrays, `tsvector`), enforces constraints differently, and has different SQL
+semantics — green-on-SQLite tests routinely break on production Postgres. Spin up a
+disposable container with `testcontainers` (or point at a dedicated throwaway test DB).
+A session-scoped engine fixture creates the schema once and disposes it at the end.
+
+```python
+import pytest
+from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
+from testcontainers.postgres import PostgresContainer
+
+from app.db.base import Base
+
+
+@pytest.fixture(scope="session")
+async def engine():
+    with PostgresContainer("postgres:16-alpine") as pg:
+        url = pg.get_connection_url().replace("psycopg2", "asyncpg")
+        eng = create_async_engine(url)
+        async with eng.begin() as conn:
+            await conn.run_sync(Base.metadata.create_all)
+        yield eng
+        await eng.dispose()
+```
+
+For an existing test DB instead of a container, read its URL from an env var
+(`APP_TEST_DATABASE_URL`) and skip the `PostgresContainer` block; everything downstream is
+identical.
+
+## Transactional isolation fixture
+
+The canonical SQLAlchemy 2.0 async per-test rollback recipe. Open a connection, begin an
+outer transaction on it, and bind an `AsyncSession` with
+`join_transaction_mode="create_savepoint"`. With that mode, every `session.commit()` the
+service code runs only releases an internal SAVEPOINT — the outer transaction stays open and
+is rolled back at teardown, so the DB is pristine between tests with no `create_all` /
+`drop_all` per test. The mode is built into SQLAlchemy 2.0+, so the old
+`after_transaction_end` event listener that manually restarted savepoints is no longer
+needed.
+
+```python
+import pytest
+from sqlalchemy.ext.asyncio import AsyncSession
+
+
+@pytest.fixture
+async def db_session(engine) -> AsyncSession:
+    async with engine.connect() as connection:
+        outer = await connection.begin()
+        session = AsyncSession(
+            bind=connection,
+            expire_on_commit=False,
+            join_transaction_mode="create_savepoint",  # commits become SAVEPOINT releases
+        )
+        try:
+            yield session
+        finally:
+            await session.close()
+            await outer.rollback()  # discard everything the test wrote
+```
+
+The session's `commit()` calls never reach the database as real commits, so each test sees a
+clean schema and the rollback at teardown leaves no residue for the next test.
+
+## The async client fixture
+
+Put this in `conftest.py`. It builds the app, swaps the request `get_db` for the
+transactional test session, and drives it over `ASGITransport` (in-process, no socket).
+`base_url="http://test"` satisfies httpx's absolute-URL requirement.
+
+Build the app in its own fixture so other fixtures (auth, feature flags) can register
+`dependency_overrides` on the *same* instance the client drives — no reaching into
+`client._transport.app`.
+
+```python
+import pytest
+from fastapi import FastAPI
+from httpx import ASGITransport, AsyncClient
+
+from app.api.deps import get_db
+from app.main import create_app
+
+
+@pytest.fixture
+def app(db_session) -> FastAPI:
+    application = create_app()
+    application.dependency_overrides[get_db] = lambda: db_session
+    return application
+
+
+@pytest.fixture
+async def client(app: FastAPI) -> AsyncClient:
+    transport = ASGITransport(app=app)
+    async with AsyncClient(transport=transport, base_url="http://test") as c:
+        yield c
+    app.dependency_overrides.clear()
+```
+
+The override yields the **same** `db_session` the test sees, so assertions made directly on
+the session observe exactly what the handler wrote (before rollback).
+
+## Auth override fixture
+
+Two strategies. Use (a) for the common case — you want to test business logic, not the JWT
+machinery. Use (b) when you are specifically exercising the decode/claims path.
+
+```python
+# (a) Inject a fixed user, bypassing the real token decode.
+import pytest
+from fastapi import FastAPI
+
+from app.api.deps import get_current_user
+from app.models.user import User
+
+
+@pytest.fixture
+async def auth_client(app: FastAPI, client, db_session) -> AsyncClient:
+    user = User(email="auth@example.com", full_name="Auth", hashed_password="x")
+    db_session.add(user)
+    await db_session.flush()
+    # Override on the same app the client drives — no client._transport.app reach-in.
+    app.dependency_overrides[get_current_user] = lambda: user
+    return client
+```
+
+```python
+# (b) Mint a real JWT and send it through the real get_current_user decode path.
+import pytest
+
+from app.core.security import create_access_token
+
+
+@pytest.fixture
+async def bearer_headers(db_session) -> dict[str, str]:
+    from app.models.user import User
+
+    user = User(email="real@example.com", full_name="Real", hashed_password="x")
+    db_session.add(user)
+    await db_session.flush()
+    token = create_access_token(subject=str(user.id))
+    return {"Authorization": f"Bearer {token}"}
+```
+
+## Factories & fixtures
+
+Prefer plain async builder functions over global factory state — explicit, mypy-friendly,
+no hidden sequences. Override any field per call.
+
+```python
+from app.models.user import User
+from sqlalchemy.ext.asyncio import AsyncSession
+
+
+async def make_user(db: AsyncSession, **overrides) -> User:
+    defaults = {"email": "u@example.com", "full_name": "U", "hashed_password": "x"}
+    user = User(**{**defaults, **overrides})
+    db.add(user)
+    await db.flush()
+    return user
+```
+
+Parametrize with readable `ids=` so failures name the case:
+
+```python
+import pytest
+
+
+@pytest.mark.parametrize(
+    ("payload", "expected_field"),
+    [
+        ({"email": "bad", "full_name": "A", "password": "x" * 12}, "email"),
+        ({"email": "a@b.com", "full_name": "", "password": "x" * 12}, "full_name"),
+        ({"email": "a@b.com", "full_name": "A", "password": "short"}, "password"),
+    ],
+    ids=["bad-email", "empty-name", "short-password"],
+)
+async def test_validation_fields(client, payload, expected_field) -> None:
+    resp = await client.post("/api/v1/users", json=payload)
+    assert resp.status_code == 422
+    assert resp.json()["error"]["details"][0]["field"] == expected_field
+```
+
+## Testing endpoints
+
+Cover the happy path, the created-201 contract, the validation envelope, and domain errors.
+
+```python
+from httpx import AsyncClient
+
+from tests.factories import make_user
+
+
+async def test_list_users_pagination(client: AsyncClient, db_session) -> None:
+    for i in range(3):
+        await make_user(db_session, email=f"u{i}@example.com")
+    resp = await client.get("/api/v1/users", params={"limit": 2, "offset": 0})
+    assert resp.status_code == 200
+    assert len(resp.json()) == 2
+
+
+async def test_create_user_201_location(client: AsyncClient) -> None:
+    resp = await client.post("/api/v1/users", json={
+        "email": "new@example.com", "full_name": "New", "password": "correct-horse-battery",
+    })
+    assert resp.status_code == 201
+    assert resp.headers["Location"].startswith("/api/v1/users/")
+    assert "hashed_password" not in resp.json()
+
+
+async def test_create_user_validation_envelope(client: AsyncClient) -> None:
+    resp = await client.post("/api/v1/users", json={"email": "nope"})
+    assert resp.status_code == 422
+    body = resp.json()
+    assert body["error"]["code"] == "validation_error"
+    assert body["error"]["details"][0]["field"]
+
+
+async def test_duplicate_email_conflict(client: AsyncClient, db_session) -> None:
+    await make_user(db_session, email="dupe@example.com")
+    resp = await client.post("/api/v1/users", json={
+        "email": "dupe@example.com", "full_name": "Dupe", "password": "correct-horse-battery",
+    })
+    assert resp.status_code == 409
+    assert resp.json()["error"]["code"] == "conflict"
+```
+
+## Testing background tasks & external calls
+
+Mock outbound httpx with **respx** — never hit the network in unit tests.
+
+```python
+import httpx
+import respx
+
+
+@respx.mock
+async def test_outbound_call(client: httpx.AsyncClient) -> None:
+    route = respx.post("https://payments.example.com/charge").mock(
+        return_value=httpx.Response(200, json={"status": "ok"})
+    )
+    resp = await client.post("/api/v1/orders", json={"amount": 100})
+    assert resp.status_code == 201
+    assert route.called
+```
+
+For `BackgroundTasks`, assert the side effect by overriding the task target with a spy
+(e.g. `monkeypatch.setattr("app.services.email.send_welcome_email", spy)`) and asserting the
+spy was awaited with the expected args — the task runs after the response is sent, so make
+the override visible before the request.
+
+## TDD discipline
+
+Red → green → refactor for a new `GET /users/{id}`. Follow
+`superpowers:test-driven-development`: write the failing test first, watch it fail, then
+write the minimum to pass.
+
+```python
+# RED — write this first; it fails (404/route missing).
+async def test_get_user_by_id(client, db_session) -> None:
+    user = await make_user(db_session, email="byid@example.com")
+    resp = await client.get(f"/api/v1/users/{user.id}")
+    assert resp.status_code == 200
+    assert resp.json()["email"] == "byid@example.com"
+```
+
+```python
+# GREEN — minimal handler + service to pass.
+# app/services/user_service.py
+from uuid import UUID
+
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.exceptions import NotFoundError
+from app.models.user import User
+
+
+async def get_user(db: AsyncSession, user_id: UUID) -> User:
+    user = await db.get(User, user_id)
+    if user is None:
+        raise NotFoundError("User", str(user_id))
+    return user
+
+
+# app/api/routers/users.py
+from uuid import UUID
+
+from app.api.deps import DbSession
+from app.schemas.user import UserResponse
+
+
+@router.get("/{user_id}", response_model=UserResponse)
+async def get_user(user_id: UUID, db: DbSession) -> UserResponse:
+    return await user_service.get_user(db, user_id)
+```
+
+Refactor step: once green, extract shared lookup, add the 404-envelope test, and confirm the
+suite stays green before moving on.
+
+## Coverage gate & CI
+
+```bash
+uv run pytest --cov=app --cov-report=term-missing
+```
+
+`--cov-fail-under=85` in `pyproject.toml` is the single source of truth — do not pass a
+second threshold on the command line. Add `--cov-branch` for branch coverage; the lines that
+*must* be covered are the auth and error paths: 401, 403, 404, 409, 422, and the catch-all
+500. In CI, run `uv run pytest` after `ruff` and `mypy` so a failing lint/type check blocks
+before the (slower) test job.
+
+## See Also
+
+- [`database.md`](database.md) — the engine/session wiring these fixtures build on.
+- [`security.md`](security.md) — `create_access_token` and `get_current_user` used by the auth fixtures.
+- `superpowers:test-driven-development` — red/green/refactor discipline these tests follow.