dataplat 0.2.0__tar.gz → 0.2.2__tar.gz

dataplat-0.2.2/.github/workflows/ci.yml

@@ -0,0 +1,171 @@
+name: CI
+
+on:
+  push:
+    branches: ["**"]
+    # A tag push matches no branch, so without this a release tag would run no
+    # checks at all before release.yml publishes it.
+    tags: ["[0-9]*.[0-9]*.[0-9]*"]
+  pull_request:
+
+jobs:
+  checks:
+    runs-on: ubuntu-latest
+    strategy:
+      # The floor and the ceiling break for different reasons; report both
+      # rather than hiding one behind the other.
+      fail-fast: false
+      matrix:
+        # Lowest version requires-python allows, plus the dev pin.
+        python-version: ["3.12", "3.13"]
+    env:
+      # Overrides .python-version (3.13). Without it every uv command in the
+      # 3.12 leg would still pick the pin and we would test 3.13 twice.
+      UV_PYTHON: ${{ matrix.python-version }}
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Setup uv
+        uses: astral-sh/setup-uv@v5
+
+      - name: Setup Python
+        run: uv python install ${{ matrix.python-version }}
+
+      - name: Install dependencies
+        run: uv sync --group dev --all-extras
+
+      - name: Ruff
+        run: uv run ruff check .
+
+      - name: Ruff format
+        run: uv run ruff format --check .
+
+      - name: MyPy
+        run: uv run mypy dataplat
+
+      # Deselected rather than left to skip. With no server reachable the
+      # integration tests would skip on their own, but naming the marker keeps
+      # this leg an honest database-free signal instead of a growing pile of
+      # skips reported twice per push. Marker deselection happens after
+      # collection, so an import error under tests/integration/ still fails
+      # here and does not have to wait for the integration job.
+      - name: Pytest (unit)
+        run: uv run pytest -m "not integration"
+
+  # A separate job, not extra steps in the matrix above, for two reasons: the
+  # unit legs are the fast signal and must not queue behind a container boot,
+  # and a database per Python version would buy nothing -- the SQL these tests
+  # validate is identical on every interpreter.
+  integration:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        # An explicit major, never "latest": a silent server upgrade must not be
+        # able to change what this job proves. Adding a second major is a
+        # one-token edit here if catalog drift across majors ever earns the
+        # minutes.
+        postgres-version: ["16"]
+    env:
+      # The dev pin only -- see the job comment. The 3.12 floor is imported,
+      # linted and type-checked by `checks` above.
+      UV_PYTHON: "3.13"
+      # Matches the default in tests/integration/conftest.py, set explicitly so
+      # the published port below and the DSN cannot drift apart unnoticed.
+      DP_TEST_PG_DSN: postgresql://postgres:postgres@127.0.0.1:55432/dataplat_test
+      # The single most important line in this file. Without it an unreachable
+      # server turns every integration test into a skip: the job goes green, the
+      # badge stays green, and CI reports success while executing none of the
+      # SQL this suite exists to validate. Truthy here means the harness raises
+      # instead of skipping.
+      DP_TEST_PG_REQUIRED: "1"
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      # `docker run`, not a `services:` block. A service container is started
+      # with the image's default command and Actions gives no way to append
+      # arguments to it, so `-c shared_preload_libraries=pg_stat_statements`
+      # cannot be passed. Without that preload the extension still installs and
+      # its columns are still describable, but every read of the view raises
+      # "pg_stat_statements must be loaded via shared_preload_libraries" -- that
+      # errors the harness's own pg_stat_statements test and leaves the
+      # version-branching second half of fetch_query_history_postgres
+      # unexecuted, which is precisely the coverage this job exists for.
+      #
+      # Detached and placed before `uv sync` so initdb runs while dependencies
+      # install; by the time the wait step runs the server is normally up.
+      - name: Start PostgreSQL ${{ matrix.postgres-version }}
+        run: |
+          docker run --detach --name dp-pg-ci \
+            --publish 55432:5432 \
+            --env POSTGRES_PASSWORD=postgres \
+            --env POSTGRES_DB=dataplat_test \
+            --health-cmd "pg_isready -h 127.0.0.1 -U postgres -d dataplat_test" \
+            --health-interval 2s \
+            --health-timeout 5s \
+            --health-retries 30 \
+            postgres:${{ matrix.postgres-version }} \
+            -c shared_preload_libraries=pg_stat_statements
+
+      - name: Setup uv
+        uses: astral-sh/setup-uv@v5
+
+      - name: Setup Python
+        run: uv python install 3.13
+
+      # --all-extras is what installs psycopg (the optional "db" extra). The
+      # harness routes a missing psycopg through the same gate as an unreachable
+      # server, so under DP_TEST_PG_REQUIRED=1 dropping it is a hard error
+      # rather than a quiet skip.
+      - name: Install dependencies
+        run: uv sync --group dev --all-extras
+
+      # `-h 127.0.0.1` in the health command above is deliberate: the official
+      # entrypoint runs a temporary socket-only server while it initialises the
+      # data directory, so a Unix-socket probe reports "ready" before the real
+      # server is listening on TCP -- which is how the tests connect.
+      - name: Wait for PostgreSQL
+        run: |
+          for _ in $(seq 60); do
+            state="$(
+              docker inspect \
+                --format '{{.State.Running}}/{{.State.Health.Status}}' dp-pg-ci
+            )"
+            case "$state" in
+              true/healthy)
+                echo "PostgreSQL is accepting TCP connections."
+                exit 0
+                ;;
+              # An exited or unhealthy container (bad flag, image change, OOM)
+              # will never turn healthy: fail now with its log instead of
+              # stalling for the full two minutes.
+              false/* | */unhealthy)
+                break
+                ;;
+            esac
+            sleep 2
+          done
+          echo "::error::PostgreSQL never became healthy (state: $state)."
+          docker logs dp-pg-ci
+          exit 1
+
+      # Preloading the library only makes the view available server-wide; the
+      # extension itself is per-database and has to be created once. Nothing in
+      # the suite can do it -- the fixtures roll back every transaction.
+      - name: Enable pg_stat_statements
+        run: |
+          docker exec dp-pg-ci psql \
+            --username postgres --dbname dataplat_test \
+            --set ON_ERROR_STOP=1 \
+            --command 'CREATE EXTENSION IF NOT EXISTS pg_stat_statements'
+
+      - name: Pytest (integration)
+        run: uv run pytest -m integration
+
+      # For the failures that are about the database rather than the SQL:
+      # refused connections, a crashed backend, the OOM killer.
+      - name: PostgreSQL server log
+        if: failure()
+        run: docker logs dp-pg-ci

dataplat-0.2.2/.github/workflows/release.yml

@@ -0,0 +1,227 @@
+name: Release
+
+on:
+  push:
+    tags: ["[0-9]*.[0-9]*.[0-9]*"]
+
+jobs:
+  # Nothing downstream re-checks the tree, so this job and `integration` are the
+  # only things standing between an untested commit and PyPI. Publishing is
+  # irreversible: a version can be yanked but never replaced.
+  verify:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        # Must mirror ci.yml: the wheel advertises >=3.12, so the floor has to
+        # pass before we publish, not just the dev pin.
+        python-version: ["3.12", "3.13"]
+    env:
+      UV_PYTHON: ${{ matrix.python-version }}
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Setup uv
+        uses: astral-sh/setup-uv@v5
+
+      - name: Setup Python
+        run: uv python install ${{ matrix.python-version }}
+
+      # Before the expensive sync: a mismatch means the tag would either
+      # publish the wrong version number or die at upload against an already
+      # occupied filename.
+      - name: Assert tag matches project version
+        run: |
+          uv run --no-project python - <<'PY'
+          import os
+          import sys
+          import tomllib
+
+          tag = os.environ["GITHUB_REF_NAME"]
+          with open("pyproject.toml", "rb") as handle:
+              version = tomllib.load(handle)["project"]["version"]
+          if tag != version:
+              sys.exit(
+                  f"Tag '{tag}' does not match [project].version '{version}' in "
+                  "pyproject.toml. Bump the version, commit, and re-tag; "
+                  "refusing to publish a mismatched artifact."
+              )
+          print(f"Tag '{tag}' matches [project].version.")
+          PY
+
+      - name: Install dependencies
+        run: uv sync --group dev --all-extras
+
+      - name: Ruff
+        run: uv run ruff check .
+
+      - name: Ruff format
+        run: uv run ruff format --check .
+
+      - name: MyPy
+        run: uv run mypy dataplat
+
+      # Mirrors ci.yml: the database-backed half of the suite is the
+      # `integration` job below, which gates `build` alongside this one.
+      - name: Pytest (unit)
+        run: uv run pytest -m "not integration"
+
+  # This gate exists because the alternative is shipping SQL no server has ever
+  # parsed. dataplat's db services are thousands of lines of generated SQL, and
+  # the fake-cursor tests in `verify` prove only that execute() was called -- an
+  # invalid column, a Redshift-only construct on the PostgreSQL path, or a broken
+  # GRANT passes them all and fails on a user's warehouse. Since a bad version
+  # can only be yanked, never replaced, the few minutes this adds to tagging are
+  # cheap next to a release that has to be withdrawn.
+  #
+  # Deliberately NOT `needs: verify`: it runs concurrently with the verify
+  # matrix, so the added wall-clock is the amount by which it outlasts verify,
+  # not its full runtime. Every executable line is identical to ci.yml's
+  # integration job -- only the surrounding prose differs -- so the tag gate
+  # cannot quietly become weaker than the per-push one. (A reusable workflow
+  # would remove the duplication; two copies this size do not yet justify the
+  # extra indirection.)
+  integration:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        postgres-version: ["16"]
+    env:
+      UV_PYTHON: "3.13"
+      DP_TEST_PG_DSN: postgresql://postgres:postgres@127.0.0.1:55432/dataplat_test
+      # Load-bearing: without it an unreachable server makes every integration
+      # test skip, and this gate would wave through exactly the unexecuted SQL it
+      # was added to catch.
+      DP_TEST_PG_REQUIRED: "1"
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      # `docker run` rather than a `services:` block: a service container runs
+      # the image's default command with no way to append arguments, so
+      # shared_preload_libraries cannot be set -- and without it every read of
+      # pg_stat_statements raises "must be loaded via shared_preload_libraries".
+      # Detached and before `uv sync` so initdb overlaps the install.
+      - name: Start PostgreSQL ${{ matrix.postgres-version }}
+        run: |
+          docker run --detach --name dp-pg-ci \
+            --publish 55432:5432 \
+            --env POSTGRES_PASSWORD=postgres \
+            --env POSTGRES_DB=dataplat_test \
+            --health-cmd "pg_isready -h 127.0.0.1 -U postgres -d dataplat_test" \
+            --health-interval 2s \
+            --health-timeout 5s \
+            --health-retries 30 \
+            postgres:${{ matrix.postgres-version }} \
+            -c shared_preload_libraries=pg_stat_statements
+
+      - name: Setup uv
+        uses: astral-sh/setup-uv@v5
+
+      - name: Setup Python
+        run: uv python install 3.13
+
+      # --all-extras installs psycopg; the harness treats it as missing
+      # infrastructure, so under DP_TEST_PG_REQUIRED=1 omitting it errors.
+      - name: Install dependencies
+        run: uv sync --group dev --all-extras
+
+      # `-h 127.0.0.1` in the health command matters: the entrypoint runs a
+      # socket-only server during initdb, so a socket probe says "ready" before
+      # TCP -- the transport the tests actually use -- is listening.
+      - name: Wait for PostgreSQL
+        run: |
+          for _ in $(seq 60); do
+            state="$(
+              docker inspect \
+                --format '{{.State.Running}}/{{.State.Health.Status}}' dp-pg-ci
+            )"
+            case "$state" in
+              true/healthy)
+                echo "PostgreSQL is accepting TCP connections."
+                exit 0
+                ;;
+              # An exited or unhealthy container (bad flag, image change, OOM)
+              # will never turn healthy: fail now with its log instead of
+              # stalling for the full two minutes.
+              false/* | */unhealthy)
+                break
+                ;;
+            esac
+            sleep 2
+          done
+          echo "::error::PostgreSQL never became healthy (state: $state)."
+          docker logs dp-pg-ci
+          exit 1
+
+      # Server-wide preload only exposes the view; the extension is per-database.
+      - name: Enable pg_stat_statements
+        run: |
+          docker exec dp-pg-ci psql \
+            --username postgres --dbname dataplat_test \
+            --set ON_ERROR_STOP=1 \
+            --command 'CREATE EXTENSION IF NOT EXISTS pg_stat_statements'
+
+      - name: Pytest (integration)
+        run: uv run pytest -m integration
+
+      - name: PostgreSQL server log
+        if: failure()
+        run: docker logs dp-pg-ci
+
+  build:
+    # Both gates, so neither a lint/type/unit regression nor SQL that no server
+    # accepts can reach the publish job.
+    needs: [verify, integration]
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Setup uv
+        uses: astral-sh/setup-uv@v5
+
+      - name: Build sdist and wheel
+        run: uv build
+
+      - name: Upload artifacts
+        uses: actions/upload-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+  publish:
+    needs: build
+    runs-on: ubuntu-latest
+    environment: pypi
+    permissions:
+      id-token: write  # PyPI Trusted Publishing (OIDC)
+    steps:
+      - name: Download artifacts
+        uses: actions/download-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+      - name: Publish to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1
+
+  github-release:
+    needs: publish
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+    steps:
+      - name: Download artifacts
+        uses: actions/download-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+      - name: Create GitHub release
+        uses: softprops/action-gh-release@v2
+        with:
+          files: dist/*
+          generate_release_notes: true

{dataplat-0.2.0 → dataplat-0.2.2}/CHANGELOG.md

@@ -1,5 +1,105 @@
 # Changelog
 
+## 0.2.2
+
+Closes the six defects 0.2.1's integration suite found and pinned as expected
+failures. No expected failures remain.
+
+### Fixed
+
+- `dp db role show` claimed `Password set: yes` for every role, including a
+  passwordless `NOLOGIN` group. The attribute query read `rolpassword` from
+  `pg_roles`, whose view definition returns the literal `'********'` and can
+  never be NULL. Only `pg_authid` holds the real verifier and it is
+  superuser-only, so the field is now tri-state and reports `unknown` — with a
+  hint saying why — when the connecting role cannot read it. The privilege is
+  probed before the query rather than discovered by catching an error, since a
+  permission failure would abort the surrounding transaction.
+
+- `dp db describe <schema>` never reported `USAGE` grants. The query read
+  `information_schema.usage_privileges`, which on PostgreSQL does not cover
+  schemas at all, so the USAGE half of the union always returned nothing. The
+  PostgreSQL path now reads the schema ACL directly.
+
+- `dp db role show` under-counted a role's tables when it owned a partitioned
+  table. Both `relkind` `'r'` and `'p'` map to the label "table" and the
+  aggregation assigned rather than accumulated, so one group overwrote the
+  other while the total was summed separately and stayed right — the per-schema
+  breakdown contradicted its own total.
+
+- `dp db dbt-orphans purge` aborted the whole batch when a single relation had
+  vanished between scan and purge. The generated statement now uses
+  `IF EXISTS`, matching `top-tables`, so a missing relation is a no-op. When a
+  dependent object genuinely blocks a drop the purge still stops — that
+  all-or-nothing property is deliberate for a destructive batch — but it now
+  names the relation and everything depending on it instead of surfacing a raw
+  driver error, and records the blockage in the audit log.
+
+- `dp db describe <relation>` raised nothing but returned an invalid result for
+  a non-view relation: `pg_get_viewdef()` yields NULL rather than erroring, and
+  the PostgreSQL branch returned a view definition whose `sql` was `None`,
+  violating its own annotation. It now raises, as the Redshift branch already
+  did.
+
+- `dp db long-queries --history` died with a driver traceback on any server
+  where `pg_stat_statements` is installed but not preloaded — the most common
+  misconfiguration. The guard that was meant to catch this probed the view's
+  columns, which PostgreSQL answers from the view definition without invoking
+  the extension, so the probe always succeeded. The failure is now reported as
+  an actionable error naming `shared_preload_libraries` and the required
+  restart.
+
+### Changed
+
+- `RoleAttributes.password_set` widened from `bool` to `bool | None`, where
+  `None` means "not determinable by this connection". Relevant only if you
+  import the dataclass; the CLI renders the third state as `unknown`.
+
+- On PostgreSQL, `dp db describe <schema>` privileges now come from the schema
+  ACL rather than a `has_schema_privilege` scan. A role holding `CREATE` only
+  through membership in a granted role is no longer listed as its own row, and
+  `grantor` and `WITH GRANT OPTION` are now real values where the previous
+  CREATE half hardcoded them.
+
+## 0.2.1
+
+### Fixed
+
+- `dp db dbt-orphans` could report — and with `purge --include-unknown`, drop —
+  a relation it had never renamed. The scan matched `LIKE '%_deprecated'` with
+  no `ESCAPE`, and an unescaped `_` matches any single character, so a table
+  named `legacydeprecated` came back as a purge candidate. Found by running the
+  scan against a real PostgreSQL; `top_tables.py` had carried the escaping fix
+  for this since the beginning, one module over, which is exactly why the
+  helper now lives in one place both modules share.
+
+- `dp db describe` reported `Size 0 B` and no row estimate for every ordinary
+  table, matview and view. All five header size aggregates read from
+  `pg_partition_tree()`, which returns no rows for a relation that is neither
+  partitioned nor a partition; `SUM` over no rows is `NULL`, and a
+  `COALESCE(..., 0)` inside each subquery presented that as a confident zero.
+  Partitioned tables had the mirror-image bug and double-counted rows, because
+  `reltuples` on an analyzed parent already aggregates its partitions.
+
+- `dp db role show` under-reported a role's privileges. The recursive
+  membership walk collapsed duplicate paths with `DISTINCT ON`, and when an
+  ancestor was reachable by several equal-depth paths the surviving row — and
+  therefore whether `INHERIT` was honoured — was decided by scan order.
+  PostgreSQL reported the privilege as held while dataplat showed it missing.
+
+- `dp db long-queries` could return an empty snapshot. The scan measured age
+  with `now()`, which is `transaction_timestamp()`, so any query that started
+  after the scanning transaction began had a negative age and failed the
+  threshold filter silently.
+
+### Added
+
+- An integration suite that executes the database service layer against a real
+  PostgreSQL, in CI and locally (`tests/integration/`, see the README). It
+  found all four fixes above, plus seven further defects now pinned as strict
+  expected failures. Releases are gated on it: `build` will not run until the
+  suite passes.
+
 ## 0.2.0
 
 ### Fixed

{dataplat-0.2.0 → dataplat-0.2.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: dataplat
-Version: 0.2.0
+Version: 0.2.2
 Summary: One command to manage any shape of data platform: databases, ingestion, BI, cloud, and CI.
 Project-URL: Homepage, https://github.com/hanslemm/dataplat
 Project-URL: Repository, https://github.com/hanslemm/dataplat
@@ -337,12 +337,64 @@ uv run mypy dataplat
 CI runs those four across Python 3.12 and 3.13 — the floor the wheel
 advertises as well as the pinned dev version.
 
+### Integration tests against a real PostgreSQL
+
+Most of the suite drives a fake database cursor. That proves a code path
+*called* `execute`, never that the SQL it built is valid. The tests in
+`tests/integration/` close that gap: they run the real statements against a
+live PostgreSQL and check the results, so an invalid column reference or a
+broken `GRANT` fails here instead of on your warehouse.
+
+`uv run pytest` stays green **without Docker** — the suite skips itself when
+no server is reachable. You only need the container to actually exercise it:
+
+```bash
+docker run -d --name dp-pg-test -p 55432:5432 \
+    -e POSTGRES_PASSWORD=postgres \
+    -e POSTGRES_DB=dataplat_test \
+    postgres:16 -c shared_preload_libraries=pg_stat_statements
+
+# Once per database: pg_stat_statements is a per-database extension.
+docker exec dp-pg-test psql -U postgres -d dataplat_test \
+    -c 'CREATE EXTENSION IF NOT EXISTS pg_stat_statements'
+
+DP_TEST_PG_REQUIRED=1 uv run pytest -m integration
+```
+
+The `-c shared_preload_libraries=pg_stat_statements` is not optional for full
+coverage: without it the extension installs but every *read* of the view fails
+with `pg_stat_statements must be loaded via shared_preload_libraries`, so
+`dp db long-queries --history` stays untested.
+
+| Variable | Purpose |
+| --- | --- |
+| `DP_TEST_PG_DSN` | Connection string for the test server. Default: `postgresql://postgres:postgres@127.0.0.1:55432/dataplat_test`. |
+| `DP_TEST_PG_REQUIRED` | Truthy ⇒ an unreachable server is a hard **error**. Unset ⇒ the tests skip. |
+
+Set `DP_TEST_PG_REQUIRED=1` whenever a skip would be a lie — that is, always
+in CI. Without it a broken container makes the tests vanish and the run goes
+green having validated no SQL at all. CI runs the integration job with it set,
+against a pinned PostgreSQL major, and the release workflow runs the same job
+as a gate before publishing.
+
+Each test runs in a transaction that is rolled back afterwards, so tests never
+see each other's objects and nothing survives a failed run. To select the
+fast, database-free subset explicitly, use `-m "not integration"`.
+
+**Known gap: Redshift.** Redshift cannot be containerized, so every
+Redshift-specific code path remains fake-tested only — its SQL is generated and
+asserted against a fake cursor, never executed. Treat changes to Redshift
+paths as unverified by CI and test them against a real cluster.
+
 ## Releasing
 
 Bump `[project].version`, then tag `X.Y.Z` (bare semver, no `v` prefix) on
 `main`. The release workflow refuses to publish unless the tag matches that
-version, and runs the full check matrix first; only then does GitHub Actions
-build and publish to PyPI via Trusted Publishing. Commits follow
+version, and runs the full check matrix *plus* the integration suite against a
+real PostgreSQL first; only then does GitHub Actions build and publish to PyPI
+via Trusted Publishing. A published version can be yanked but never replaced,
+so the extra minutes buy a guarantee that the shipped SQL has at least been
+parsed by a server. Commits follow
 [Conventional Commits](https://www.conventionalcommits.org/).
 
 ## License

{dataplat-0.2.0 → dataplat-0.2.2}/README.md

@@ -292,12 +292,64 @@ uv run mypy dataplat
 CI runs those four across Python 3.12 and 3.13 — the floor the wheel
 advertises as well as the pinned dev version.
 
+### Integration tests against a real PostgreSQL
+
+Most of the suite drives a fake database cursor. That proves a code path
+*called* `execute`, never that the SQL it built is valid. The tests in
+`tests/integration/` close that gap: they run the real statements against a
+live PostgreSQL and check the results, so an invalid column reference or a
+broken `GRANT` fails here instead of on your warehouse.
+
+`uv run pytest` stays green **without Docker** — the suite skips itself when
+no server is reachable. You only need the container to actually exercise it:
+
+```bash
+docker run -d --name dp-pg-test -p 55432:5432 \
+    -e POSTGRES_PASSWORD=postgres \
+    -e POSTGRES_DB=dataplat_test \
+    postgres:16 -c shared_preload_libraries=pg_stat_statements
+
+# Once per database: pg_stat_statements is a per-database extension.
+docker exec dp-pg-test psql -U postgres -d dataplat_test \
+    -c 'CREATE EXTENSION IF NOT EXISTS pg_stat_statements'
+
+DP_TEST_PG_REQUIRED=1 uv run pytest -m integration
+```
+
+The `-c shared_preload_libraries=pg_stat_statements` is not optional for full
+coverage: without it the extension installs but every *read* of the view fails
+with `pg_stat_statements must be loaded via shared_preload_libraries`, so
+`dp db long-queries --history` stays untested.
+
+| Variable | Purpose |
+| --- | --- |
+| `DP_TEST_PG_DSN` | Connection string for the test server. Default: `postgresql://postgres:postgres@127.0.0.1:55432/dataplat_test`. |
+| `DP_TEST_PG_REQUIRED` | Truthy ⇒ an unreachable server is a hard **error**. Unset ⇒ the tests skip. |
+
+Set `DP_TEST_PG_REQUIRED=1` whenever a skip would be a lie — that is, always
+in CI. Without it a broken container makes the tests vanish and the run goes
+green having validated no SQL at all. CI runs the integration job with it set,
+against a pinned PostgreSQL major, and the release workflow runs the same job
+as a gate before publishing.
+
+Each test runs in a transaction that is rolled back afterwards, so tests never
+see each other's objects and nothing survives a failed run. To select the
+fast, database-free subset explicitly, use `-m "not integration"`.
+
+**Known gap: Redshift.** Redshift cannot be containerized, so every
+Redshift-specific code path remains fake-tested only — its SQL is generated and
+asserted against a fake cursor, never executed. Treat changes to Redshift
+paths as unverified by CI and test them against a real cluster.
+
 ## Releasing
 
 Bump `[project].version`, then tag `X.Y.Z` (bare semver, no `v` prefix) on
 `main`. The release workflow refuses to publish unless the tag matches that
-version, and runs the full check matrix first; only then does GitHub Actions
-build and publish to PyPI via Trusted Publishing. Commits follow
+version, and runs the full check matrix *plus* the integration suite against a
+real PostgreSQL first; only then does GitHub Actions build and publish to PyPI
+via Trusted Publishing. A published version can be yanked but never replaced,
+so the extra minutes buy a guarantee that the shipped SQL has at least been
+parsed by a server. Commits follow
 [Conventional Commits](https://www.conventionalcommits.org/).
 
 ## License