@mrciphersmith/keryx 0.3.5 → 0.3.7

package/src/gdskills/bundled/stacks/fastapi/skills/fastapi-testing/SKILL.md

@@ -0,0 +1,146 @@
+---
+name: fastapi-testing
+description: "Use when you write, extend, or fix a FastAPI project's test suite -- driving synchronous or async HTTP requests against the running app in-process, app.dependency_overrides for auth/DB fixtures, asserting response status codes and response_model filtering, and mocking an external HTTP call. Scoped to a project that actually has a FastAPI HTTP layer under test, not a generic Python test suite with none."
+triggers:
+  - "write a pytest test for this FastAPI endpoint"
+  - "drive an in-process request against this FastAPI route in a test"
+  - "add dependency_overrides for the current user in this test"
+  - "test that this endpoint rejects a malformed request body with a 422"
+  - "mock the external API call in this FastAPI test"
+  - "write tests for this FastAPI router's additional endpoints"
+metadata:
+  origin: authored
+  category: test
+  version: "1.0.0"
+  compatible_harnesses: "claude,codex,cursor,zed,opencode"
+license: "MIT"
+---
+
+# FastAPI testing
+
+Write, extend, or fix a FastAPI project's test suite: exercising path
+operations through the real `app` instance via `TestClient`/
+`httpx.AsyncClient`, overriding dependencies the FastAPI-supported way, and
+asserting the response contract (status code, body shape, `response_model`
+filtering) rather than just that a call didn't raise. Scoped to FastAPI's
+own test-client and dependency-override conventions — for testing a plain
+Python function with no HTTP layer use `python-testing`; for reviewing test
+conventions without changing files use `review-testing-practices`.
+
+## Workflow
+
+### Step 1: Discover the project's FastAPI test conventions
+
+1. Read `pyproject.toml`/`pytest.ini` for pytest config, and check whether
+   `httpx`'s `ASGITransport` or `fastapi.testclient.TestClient` is already
+   in use — match whichever the project already uses rather than
+   introducing the other.
+2. Find the existing `app`/`client` fixture (usually in `conftest.py`) and
+   any existing `dependency_overrides` pattern for auth or the DB session —
+   reuse it rather than building a parallel one.
+3. Read 1-2 neighboring endpoint tests for: how the DB is set up for tests
+   (throwaway SQLite, a test-scoped Postgres schema, a fixture-provided
+   session), how auth is faked, and the project's assertion style.
+
+### Step 2: Plan fixtures before test cases
+
+- A `client`/`app` fixture belongs in the narrowest `conftest.py` that
+  covers every test file needing it, matching `python`'s own fixture-scope
+  guidance.
+- Plan `dependency_overrides` per test (or per fixture, yielding and
+  clearing) rather than setting them once at module import time with no
+  teardown — an override left set leaks into unrelated tests.
+- Decide the DB fixture strategy: a real throwaway test database behind a
+  `get_db` override for integration-level coverage, reserving a mocked
+  session only for the rare case where isolating from the DB entirely is
+  the actual point of the test.
+
+### Step 3: Plan test cases
+
+**Per path operation:** the success path (status code + body shape),
+the validation-failure path (a `422` when the request body fails Pydantic
+validation), any documented error path (a `404`/`403`/`409` the endpoint
+raises deliberately), and — when the endpoint requires auth — both an
+authenticated and an unauthenticated case via `dependency_overrides`.
+
+**`response_model` filtering:** when a path operation declares
+`response_model`, at least one test asserts that a field the model
+excludes (e.g. a password hash) is actually absent from the response body.
+
+**External calls:** mock at the client boundary (`httpx`/`requests` mock,
+or the project's configured HTTP-mocking library) — never mock an internal
+collaborator in the same package, matching `python-testing`'s own rule.
+
+### Step 4: Write
+
+1. Build the request through `TestClient(app)`/`AsyncClient(transport=
+   ASGITransport(app=app), base_url="http://test")`, matching the
+   project's own style from Step 1.
+2. Override dependencies with `app.dependency_overrides[real_dep] =
+   fake_dep`, clearing them in teardown (fixture `yield` + cleanup, or an
+   explicit `.clear()`), never by monkeypatching the dependency's module
+   attribute.
+3. Assert `response.status_code` and the parsed `response.json()` body,
+   not merely that the call completed without raising.
+4. Use `pytest.mark.parametrize` for input variations across the same
+   endpoint (e.g. several invalid-body shapes that should each 422).
+
+### Step 5: Run and fix
+
+```bash
+keryx test run --changed --strict
+```
+
+`src/testing/service.ts` detects the project's own test runner — do not
+hard-code `pytest` invocation flags beyond what Step 1 discovered. With no
+keryx testing config, run the project's own configured `pytest` invocation.
+
+Fix failing tests (max 3 iterations) — fix the test, not the source under
+test.
+
+### Step 6: Report
+
+```
+Generated: tests/routers/test_users.py
+  - 6 test cases: success, 422 on bad body, 404 on missing user,
+    401 without auth override, 200 with auth override, response_model
+    excludes hashed_password
+```
+
+## Rules
+
+- ALWAYS test through `TestClient`/`AsyncClient` against the real `app`,
+  not by calling the path operation function directly.
+- ALWAYS clear any `app.dependency_overrides` entry the test set, so it
+  cannot leak into a later test in the same process.
+- NEVER modify source code — only test files and `conftest.py`.
+- Mock genuinely external dependencies (a third-party HTTP API, an email
+  provider), not internal modules under the same package.
+
+## Red Flags
+
+| Rationalization | Why it is wrong |
+|---|---|
+| "I'll call `read_user(user_id=1, db=fake_db)` directly, it's faster than spinning up TestClient" | Skips FastAPI's own request parsing, dependency resolution, and response filtering — exactly where most path-operation bugs live |
+| "I'll set `app.dependency_overrides[get_current_user] = fake_user` once at the top of the file" | With no teardown, this override leaks into every other test in the same process, including ones that meant to test the unauthenticated path |
+| "I'll use `mocker.patch(\"myapp.dependencies.get_current_user\", ...)` instead of `dependency_overrides`" | Patches the function at its module attribute instead of using FastAPI's own supported override mechanism; `app.dependency_overrides` is what keeps the fake scoped to requests actually routed through the app |
+| "The endpoint didn't raise, so the test passes" | A wrong status code or a malformed body with no exception is a real bug a bare no-exception check misses; assert the actual response contract |
+| "I'll mock the DB session so the test doesn't need a real database" | A mocked ORM session only checks the code called the mock the way the mock expects; use a real throwaway test database behind a `get_db` override for anything beyond the simplest unit test |
+
+## Verification
+
+Do not report the work done until all of the following hold:
+
+- Every new/extended test exercises the path operation through
+  `TestClient`/`AsyncClient` against the real `app`, not the function
+  directly.
+- Any `app.dependency_overrides` entry set by a test is cleared afterward
+  (fixture teardown or explicit `.clear()`).
+- `keryx test run --changed --strict` — or, with no keryx testing config,
+  the project's own discovered `pytest` command — exits 0.
+- `git status` shows only test files (and `conftest.py`, if touched)
+  added or modified; no source file under test changed.
+- Every new/touched path operation identified in Step 1 has coverage for
+  its success path, its validation-failure path, and, when it requires
+  auth, both an authenticated and unauthenticated case — or the report
+  says why one is missing.

package/src/gdskills/bundled/stacks/fastapi/skills/fastapi-testing/evals.json

@@ -0,0 +1,74 @@
+{
+  "triggers": {
+    "positive": [
+      "What should the pytest suite look like for the new FastAPI order-creation route -- I want to make sure a valid POST actually persists the order.",
+      "Test this FastAPI route with TestClient to confirm it returns a 404",
+      "How do I fake being logged in when testing this FastAPI route, without hitting the real auth flow?",
+      "Test that this FastAPI endpoint returns a 422 on an invalid request body",
+      "This test keeps making a real network call to Stripe during CI -- how do I stub that out properly?",
+      "Add test coverage for this FastAPI router's signup endpoint"
+    ],
+    "negative": [
+      "Write a pytest test for this plain Python function with no HTTP layer at all",
+      "Add pytest-django coverage for this Django view, not a FastAPI route",
+      "Implement this FastAPI endpoint that creates an order",
+      "Review this FastAPI test file for testing convention violations without changing it",
+      "Fix this pytest collection error in our FastAPI test suite",
+      "Write a Jest test for this React component",
+      "Review this FastAPI diff for security issues before merging"
+    ]
+  },
+  "scenarios": [
+    {
+      "id": "dependency-overrides-not-monkeypatch",
+      "prompt": "Write a pytest test for a FastAPI endpoint `GET /me` that requires an authenticated user via a `get_current_user` dependency. How should the test supply a fake authenticated user?",
+      "strictness": "high",
+      "expected_behavior": [
+        {
+          "grader": "judge",
+          "rubric": "A correct answer overrides the dependency with `app.dependency_overrides[get_current_user] = fake_dependency`, sent through TestClient against the real app, and clears the override afterward -- rather than monkeypatching the dependency function's module attribute.",
+          "pass_criteria": [
+            "sets `app.dependency_overrides[get_current_user] = <fake>` (or the equivalent for the project's actual dependency), shown as concrete code, to supply the fake authenticated user",
+            "sends the request through `TestClient(app)` (or `httpx.AsyncClient` against the app) so the override is exercised through FastAPI's own request pipeline",
+            "clears the override afterward (e.g. `app.dependency_overrides.clear()` in a fixture teardown or at the end of the test) so it does not leak into other tests"
+          ],
+          "fail_criteria": [
+            "uses `mocker.patch`/`monkeypatch.setattr` on the `get_current_user` function's module attribute instead of `app.dependency_overrides`. Mentioning that this would be the wrong approach is not itself a failure."
+          ]
+        }
+      ],
+      "calibration": {
+        "known_right": "```python\nfrom myapp.main import app\nfrom myapp.dependencies import get_current_user\nfrom myapp.schemas import User\n\n\ndef fake_current_user() -> User:\n    return User(id=1, username=\"testuser\")\n\n\ndef test_me_returns_authenticated_user(client):\n    app.dependency_overrides[get_current_user] = fake_current_user\n    try:\n        response = client.get(\"/me\")\n    finally:\n        app.dependency_overrides.clear()\n    assert response.status_code == 200\n    assert response.json()[\"username\"] == \"testuser\"\n```\nThe fake user is supplied by setting `app.dependency_overrides[get_current_user]`, FastAPI's own supported override mechanism, so the real `/me` request still goes through `TestClient(app)` and exercises the actual request pipeline -- routing, other dependencies, response_model filtering -- with only that one dependency swapped. The `try`/`finally` clears the override afterward so it can't leak into a later test that expects the real dependency or a different fake.",
+        "known_wrong": "```python\ndef test_me_returns_authenticated_user(client, mocker):\n    mocker.patch(\"myapp.dependencies.get_current_user\", return_value=User(id=1, username=\"testuser\"))\n    response = client.get(\"/me\")\n    assert response.status_code == 200\n    assert response.json()[\"username\"] == \"testuser\"\n```\nUsing `mocker.patch` on `get_current_user` directly is a familiar pattern from regular Python unit testing -- patch the function, call the code, assert the result -- and it avoids having to learn FastAPI's own override API. Since `mocker.patch` automatically undoes itself at the end of the test via pytest-mock's fixture teardown, there's no extra cleanup to write either.",
+        "vague": "Swap in a fake user for the auth dependency using FastAPI's own mechanism for overriding it in tests, and make sure the override doesn't stick around for other tests.",
+        "subtle_wrong": "```python\ndef test_me_returns_authenticated_user(client, mocker):\n    mocker.patch(\"myapp.routers.me.get_current_user\", return_value=User(id=1, username=\"testuser\"))\n    response = client.get(\"/me\")\n    assert response.status_code == 200\n    assert response.json()[\"username\"] == \"testuser\"\n```\nPatches `get_current_user` at its point of use in the `me` router module, following the same 'patch at the point of use' guidance as any other pytest mock, rather than patching FastAPI's dependency-injection system directly -- since `Depends(get_current_user)` resolves the name from the router module's own namespace, patching it there should intercept the same call the endpoint actually makes."
+      },
+      "anti_patterns": ["mocker.patch"]
+    },
+    {
+      "id": "response-model-excludes-field",
+      "prompt": "Write a pytest test for a FastAPI `GET /users/{id}` endpoint that declares `response_model=UserRead`, where the underlying `User` object also carries a `hashed_password` field that `UserRead` does not include. What should the test verify beyond a 200 status code?",
+      "strictness": "high",
+      "expected_behavior": [
+        {
+          "grader": "judge",
+          "rubric": "A correct answer asserts, in addition to the 200 status code, that the response JSON body does not contain the hashed_password field -- verifying response_model actually filtered it out -- rather than stopping at the status code check.",
+          "pass_criteria": [
+            "asserts `response.status_code == 200` (or equivalent) as part of the test",
+            "additionally asserts that `hashed_password` (or the specific excluded field) is absent from `response.json()`, shown as concrete code such as `assert \"hashed_password\" not in response.json()`",
+            "explains that this verifies `response_model` actually filtered the field out of the response, not merely that the request succeeded"
+          ],
+          "fail_criteria": [
+            "only asserts the status code (or that the call didn't raise) without checking the response body for the excluded field. Mentioning that a field could leak is not itself a failure."
+          ]
+        }
+      ],
+      "calibration": {
+        "known_right": "```python\ndef test_get_user_excludes_hashed_password(client, seeded_user):\n    response = client.get(f\"/users/{seeded_user.id}\")\n    assert response.status_code == 200\n    body = response.json()\n    assert \"hashed_password\" not in body\n    assert body[\"username\"] == seeded_user.username\n```\nBeyond the 200 status code, the test explicitly asserts `\"hashed_password\" not in body`. `response_model=UserRead` is supposed to filter the returned `User` object down to just the fields `UserRead` declares, but that's a property of the schema definition matching the endpoint's actual intent -- a typo or a forgotten field removal in `UserRead` would otherwise ship a leaked field with every unit test still green, since a status-code-only check can't catch it. Checking `username` too confirms the response still carries the fields it should.",
+        "known_wrong": "```python\ndef test_get_user_returns_ok(client, seeded_user):\n    response = client.get(f\"/users/{seeded_user.id}\")\n    assert response.status_code == 200\n```\nA 200 status code confirms the endpoint found the user and returned a response without erroring, which is the main thing this test needs to verify. Since `response_model=UserRead` is declared on the path operation, FastAPI handles the field filtering automatically at the framework level -- that's exactly what response_model is for -- so there's no need for the test itself to re-check which fields ended up in the body.",
+        "vague": "Check more than just the status code -- make sure the response body actually reflects what response_model is supposed to filter down to.",
+        "subtle_wrong": "```python\ndef test_get_user_returns_ok(client, seeded_user):\n    response = client.get(f\"/users/{seeded_user.id}\")\n    assert response.status_code == 200\n    body = response.json()\n    assert body[\"username\"] == seeded_user.username\n    assert body[\"id\"] == seeded_user.id\n```\nChecks the status code and confirms the expected fields (`username`, `id`) are present and correct in the response body, which is the main behavior the endpoint needs to get right -- if `UserRead` is declared correctly, verifying the fields that should be there is enough evidence the schema is working as intended."
+      }
+    }
+  ]
+}

package/src/gdskills/bundled/stacks/java-kotlin-spring/agent-refs.json

@@ -0,0 +1,3 @@
+{
+  "agents": []
+}