@mrciphersmith/keryx 0.3.2 → 0.3.3

package/src/gdskills/bundled/stacks/sql-db/skills/sql-db-testing/SKILL.md

@@ -0,0 +1,129 @@
+---
+name: sql-db-testing
+description: "Use when writing or fixing tests for SQL migrations and queries at the schema/engine level -- transactional test wrapping with rollback, seed-data design for a migration or query test, testing a migration's up/down path on representative row counts, and asserting on actual query results and query counts rather than just 'no error'. Not for a language/ORM-specific test-fixture convention such as a pytest fixture, an RSpec factory, or a Laravel model factory (use that stack's own testing skill)."
+triggers:
+  - "write a test for this migration's rollback"
+  - "add a test that this backfill updates every row exactly once"
+  - "test that this query returns the right rows in the right order"
+  - "write a regression test for this N+1 fix"
+  - "set up transactional isolation so each migration test rolls back automatically"
+  - "seed representative row counts for this migration's up/down test"
+metadata:
+  origin: authored
+  category: test
+  version: "1.0.0"
+  compatible_harnesses: "claude,codex,cursor,zed,opencode"
+license: "MIT"
+---
+
+# SQL / database testing (Postgres, MySQL, generic SQL)
+
+Write, extend, or fix tests for `.sql` migrations and hand-written
+queries: transactional isolation, fixture/seed design, migration up/down
+testing at representative scale, and result-based query assertions.
+`rules/testing.mdc` carries the full rule set this skill's checklist is
+built from — read it, not just this summary, before writing tests.
+
+## Workflow
+
+### Step 1: Discover the project's test conventions
+
+1. Find how the project already isolates database tests: a
+   transaction-per-test wrapper, a truncate-and-reseed helper, or a fresh
+   container/schema per run. Match whichever is already there rather than
+   introducing a second isolation strategy.
+2. Read 1-2 existing database tests for fixture/factory helper
+   conventions, naming style, and whether the suite already asserts on
+   query counts anywhere (a query-count assertion helper is worth
+   reusing if one exists).
+3. Identify the migration runner's test hooks, if any — some runners
+   expose a way to run a migration's `up` then `down` in a test harness;
+   use it instead of hand-rolling migration invocation in the test.
+
+### Step 2: Plan test cases
+
+**Queries:** the happy-path result set (values and ordering when the
+query specifies `ORDER BY`), the empty-result case (a filter that matches
+nothing), and the boundary case for any range condition — these are where
+an off-by-one or a `JOIN` that should have been a `LEFT JOIN` surfaces.
+
+**N+1 regression tests:** alongside the result assertion, assert the
+query count didn't regress back to one-per-row — most ORMs/test harnesses
+expose a query counter or a query log to assert against.
+
+**Migrations:** the `up` path against a representative row count (not an
+empty table), the `down` path when the migration states one (running
+`up` then `down` should leave the schema equivalent to before), and
+idempotency when the migration was written to tolerate re-application.
+
+**Batched backfills:** a row count large enough to exercise at least two
+batches, asserting every row was updated exactly once — no row skipped at
+a batch boundary, none double-applied in a way that would corrupt a
+non-idempotent update.
+
+### Step 3: Write
+
+1. Wrap each test in the project's transactional-rollback isolation (or
+   its existing equivalent) per `rules/testing.mdc`; reserve a dedicated
+   torn-down schema/database only for cases a transaction can't roll back
+   cleanly (`CREATE INDEX CONCURRENTLY`, cross-transaction concurrency).
+2. Build fixtures through the project's existing factory/helper, seeding
+   only what the test needs, named descriptively
+   (`user_with_expired_token`, not `test_user_3`).
+3. Construct any "now"-relative fixture value from a fixed, injected
+   reference time — never a live `NOW()`/`CURRENT_TIMESTAMP` evaluated at
+   test-run time for a boundary case.
+4. Assert on the query's actual rows/values/ordering, and on the query
+   count for an N+1 regression test — never assert only "ran without
+   error."
+
+### Step 4: Run and fix
+
+Run the project's own database test command. Fix failing tests (max 3
+iterations) — fix the test or fixture, not the migration/query under
+test, unless the test itself correctly caught a real bug (say so in the
+report rather than silently changing the schema/query to make the test
+pass).
+
+### Step 5: Report
+
+```
+Generated: tests/migrations/test_add_status_backfill.py
+  - Up/down path tested against a 50k-row fixture, 3 batch boundaries
+  - Asserted every row updated exactly once, no row skipped or doubled
+```
+
+## Rules
+
+- ALWAYS match the project's existing isolation and fixture conventions
+  found in Step 1, not a different project's style.
+- NEVER modify the migration/query under test — only test files and
+  fixtures — unless the test caught a real bug (state that explicitly).
+- NEVER assert only that a query "ran without error" — assert on its
+  actual result rows/values/ordering.
+- Test a migration's `up` path against a representative row count, not
+  only an empty table.
+
+## Red Flags
+
+| Rationalization | Why it is wrong |
+|---|---|
+| "The test just checks the migration doesn't throw, that's enough" | A migration that silently backfills the wrong value, or skips rows at a batch boundary, throws no error — only a result assertion catches that |
+| "I'll seed with `NOW()` for the expiry fixture, it's close enough" | A live clock value makes boundary cases (a row that just expired) flaky depending on exact test-run timing; use a fixed reference time |
+| "This N+1 fix already returns the right rows, that's the test" | A correct result set today doesn't prove the query count didn't regress back to one-per-row after a later change; assert the count too |
+| "I'll test the migration against an empty table, it's faster" | An empty-table test can't catch a lock/lastingness/batch-boundary issue that only shows up at realistic row counts |
+
+## Verification
+
+Do not report the work done until all of the following hold:
+
+- Every new/changed test runs isolated via the project's transactional
+  (or equivalent) rollback strategy — no test leaves state for the next
+  test to trip over.
+- Every query test asserts on actual result rows/values/ordering, not
+  merely "no error thrown."
+- A migration test covers both the `up` path at a representative row
+  count and, when the migration states one, the `down` path.
+- `git status`/the diff shows only test and fixture files changed, unless
+  a real bug in the migration/query was found and fixed (stated in the
+  report).

package/src/gdskills/bundled/stacks/sql-db/skills/sql-db-testing/evals.json

@@ -0,0 +1,73 @@
+{
+  "triggers": {
+    "positive": [
+      "Write a test that confirms this migration's rollback leaves the schema the way it was before",
+      "Add a test fixture for a user whose invite token expired two days ago",
+      "How should I isolate these database tests from each other so one doesn't affect the next?",
+      "Write a regression test proving this order-listing query no longer issues one query per row",
+      "Write a test proving the batched backfill script for orders.legacy_status touches every row with no row skipped or double-updated",
+      "Add test cases for this query's empty-result and boundary conditions",
+      "Set up a test that seeds fifty thousand rows to check this migration at realistic scale"
+    ],
+    "negative": [
+      "Write pytest fixtures for this Django model's test suite",
+      "Add RSpec tests for this Rails ActiveRecord scope",
+      "Write Jest tests for this React component's rendering",
+      "Fix this failing Go test that uses testify assertions",
+      "Set up a mock server for this API integration test",
+      "Write end-to-end tests for this checkout flow using Playwright",
+      "Add unit tests for this Python function that formats currency"
+    ]
+  },
+  "scenarios": [
+    {
+      "id": "fixed-clock-fixture",
+      "prompt": "I'm writing a test fixture for a token that should already be expired. How should I set the expiry timestamp?",
+      "strictness": "high",
+      "expected_behavior": [
+        {
+          "grader": "judge",
+          "rubric": "A correct answer builds the expiry timestamp relative to a fixed, injected reference time controlled by the test (a constant, a frozen/injected clock) rather than a live NOW()/CURRENT_TIMESTAMP evaluated at the moment the test runs, because a live-clock-derived boundary value makes the test's outcome depend on exact timing and can flake.",
+          "pass_criteria": [
+            "Uses a fixed, test-controlled reference time (a constant timestamp, or a frozen/injected clock value) as the basis for the expiry fixture, not a live NOW()/CURRENT_TIMESTAMP.",
+            "Explains that deriving the boundary value from a live clock makes the test's pass/fail depend on when it happens to run, which is why a fixed reference time is used instead."
+          ],
+          "fail_criteria": [
+            "Recommends setting the fixture's expiry using NOW()/CURRENT_TIMESTAMP (or an equivalent live-clock call) evaluated at test-run time as the basis for the expired boundary, whether directly or via a small offset from it."
+          ]
+        }
+      ],
+      "calibration": {
+        "known_right": "Pick a fixed reference timestamp for the test -- a constant like `2026-01-01T00:00:00Z` -- and set the token's `expires_at` to something clearly before it, e.g. one day earlier. If the code under test reads the current time itself, inject or freeze that same fixed reference time into it too (a test clock, a frozen-time helper) so 'now' and the fixture's notion of 'already expired' agree deterministically. Don't derive the expiry from a live NOW()/CURRENT_TIMESTAMP call evaluated when the test actually runs -- that ties the test's pass/fail to real wall-clock timing, which is exactly the kind of thing that makes a boundary case (a token that 'just' expired) flaky depending on when CI happens to execute it.",
+        "known_wrong": "Just set it relative to right now: `expires_at = NOW() - INTERVAL '1 day'` (or your language's equivalent of 'now minus a day') -- that guarantees it's in the past relative to whenever the test happens to run, so you don't need to hardcode any timestamp or set up a fake clock for what's a pretty simple fixture.",
+        "vague": "Make sure the expiry is set to a time that's clearly in the past so the token reads as expired.",
+        "subtle_wrong": "Use a fixed reference constant for readability, something like `const REFERENCE = '2026-01-01T00:00:00Z'`, but derive the actual fixture value from `NOW() - INTERVAL '1 day'` at insert time rather than from the constant itself -- the constant documents the intent in the test file, and NOW() minus an offset gets you an expired timestamp either way without needing to wire a frozen clock into the code under test."
+      },
+      "anti_patterns": ["NOW()"]
+    },
+    {
+      "id": "n1-regression-query-count",
+      "prompt": "I fixed an N+1 query in the order-listing endpoint by switching to a single JOIN. How should I write the regression test so it doesn't silently regress back to N+1 later?",
+      "strictness": "high",
+      "expected_behavior": [
+        {
+          "grader": "judge",
+          "rubric": "A correct answer asserts both the query's actual result (rows/values) and the number of queries issued (via the test harness's or ORM's query counter/log), so a later regression back to one query per row fails the test even if the final result set still happens to be correct -- asserting the result alone, without a query-count check, is not sufficient.",
+          "pass_criteria": [
+            "Asserts the query's actual result rows/values are correct.",
+            "Additionally asserts the number of queries issued (a query count via the test harness's/ORM's counter or captured query log), not just that the final result is correct."
+          ],
+          "fail_criteria": [
+            "Tests only the final result set's correctness with no assertion on the number of queries issued, leaving a regression back to one-query-per-row undetected as long as the final rows still happen to be right."
+          ]
+        }
+      ],
+      "calibration": {
+        "known_right": "Assert two things, not one: the result rows come back correct (right orders, right associated user data, right ordering), and the number of queries the test harness observed is exactly what the single-JOIN version should issue -- most test setups expose a query counter or let you capture the executed query log around the call, so assert something like `assert query_count == 1` (or whatever the JOIN version's actual expected count is) alongside the row assertions. The result-only assertion would still pass if someone later reverted to a per-row fetch loop that happens to reconstruct the same final rows -- the query-count assertion is what actually proves the fix is still in effect, since it fails the moment the query pattern regresses back to one-per-row even though the output would still look identical.",
+        "known_wrong": "Just assert the endpoint still returns the right orders with the right user data attached -- that's the behavior that matters to callers. If the result rows are correct, the fix is working; you don't need to separately track how many queries got issued to get that, since a wrong query count with a correct final result isn't actually a bug anyone would notice.",
+        "vague": "Make sure the test proves the fix actually worked and would catch it if the query pattern ever changed back.",
+        "subtle_wrong": "Assert the result rows are correct, and add a comment above the test noting that this relies on the single-JOIN implementation so a reviewer should double check the query pattern by eye if this test is ever touched -- that's a lighter-weight way to flag the regression risk than wiring up a query counter, and it still documents the intent for whoever reads the test next."
+      }
+    }
+  ]
+}