@yawlabs/postgres-mcp 0.10.0 → 0.11.0

package/CHANGELOG.md

@@ -7,6 +7,179 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.11.0] - 2026-08-23
+
+### Added
+
+- **`pg_io_stats`, a new tool for I/O observability.** `pg_stat_io` read /
+  write / extend / fsync counts, bytes and times per backend type and context
+  (PG16+), plus in-flight async I/O handles from `pg_aios` and the active
+  `io_method` (PG18+). Byte accounting differs by server: PG16-17 expose
+  `op_bytes` to multiply against operation counts, and PG18 removed it in favour
+  of direct `read_bytes` / `write_bytes` / `extend_bytes`. Both branches are
+  normalized to one figure, and neither ever names the other's column.
+
+- **Server-version gating.** `getServerVersionNum()` in `api.ts` caches
+  `server_version_num` per process and every version-dependent column now routes
+  through it. A failed probe returns 0, the "assume oldest" sentinel, so an
+  unknown server falls through to the conservative query rather than emitting
+  SQL referencing a column it may not have. Probe failures are deliberately not
+  cached: one transient blip during the first tool call would otherwise pin the
+  process to degraded output for its whole lifetime.
+
+- **`pg_describe_table` flags generated and identity columns.** `pg_attrdef`
+  stores generation expressions alongside plain defaults, so a generated
+  column's expression was surfacing as `default_value` with nothing marking it
+  -- an agent read that as "optional column, has a default" and wrote an INSERT
+  postgres rejects. Identity columns failed the other way: no `pg_attrdef` row
+  at all, so they reported `default_value: null, nullable: false` and the agent
+  supplied a value into `GENERATED ALWAYS AS IDENTITY`. Columns now carry
+  `generated`, `identity`, and `generation_expression`.
+
+- **PG18 constraint metadata in `pg_describe_table`.** Constraints carry
+  `validated` on every version, plus `enforced` and `has_period` on PG18+. A
+  `NOT ENFORCED` foreign key looks present but validates nothing, and a temporal
+  `WITHOUT OVERLAPS` primary key previously rendered as an ordinary one. PG18
+  also made NOT NULL a real `pg_constraint` row that can be `NOT VALID` -- its
+  `attnotnull` docs now read "possibly invalid" -- so `not_null_validated` is
+  reported on PG18+, where the question can arise.
+
+- **`pg_explain` gained the planner options that actually diagnose a slow
+  query**: `buffers`, `settings`, `verbose`, `wal`, `costs`, `timing`, plus
+  `generic_plan` (PG16+) for planning a parameterized query with no values, and
+  `memory` / `serialize` (PG17+). Options below the server's version are
+  rejected up front by name and required major, rather than as a parse error
+  from the server.
+
+- **`pg_health` reports what a health check needs.** `max_connections` and a
+  used-fraction (so a connection count is readable), `wait_event_type` /
+  `wait_event` / `backend_type` / transaction age on active queries, and a
+  `pg_stat_database` rollup: deadlocks, temp files and bytes, conflicts, cache
+  hit ratio, `stats_reset`.
+
+- **`pg_advisor` checks wraparound risk**, the classic pageable incident:
+  per-database and per-table `age(relfrozenxid)` measured against
+  `autovacuum_freeze_max_age`, with a `wraparoundThreshold` parameter mirroring
+  the existing sequence-exhaustion one. Freeze coverage via `relallfrozen` on
+  PG18+.
+
+  Multixact wraparound is checked alongside it, on the same rows and with no
+  extra round trip: `mxid_age(relminmxid)` against
+  `autovacuum_multixact_freeze_max_age`. Multixacts are consumed by row-level
+  locking (`SELECT ... FOR SHARE/UPDATE`, foreign-key checks), so a lock-heavy
+  workload can exhaust them while `relfrozenxid` still looks perfectly healthy
+  -- checking only xids reports such a cluster as clean. A row is flagged when
+  EITHER counter crosses the threshold, and `triggered_by` (`xid` /
+  `multixact` / `both`) says which, because the remediation differs.
+
+- **`pg_health` connection buckets now reconcile against `total`.** Two
+  separate ways the old breakdown lied. First, `pg_stat_activity` does not hide
+  other users' sessions from an unprivileged role -- it returns the rows with
+  `state` NULL -- so `total` was complete while `active` / `idle` counted only
+  the caller's own sessions, and an operator could read `active: 0` on a busy
+  database. A `state_unavailable` counter now makes that shortfall visible.
+  Second, `idle in transaction (aborted)`, `starting`, `fastpath function
+  call`, and `disabled` matched no filter and vanished from the breakdown; the
+  aborted state is the one that holds locks and blocks vacuum, so it gets its
+  own field, and an `other` catch-all absorbs the rest (a `NOT IN` list, so a
+  future major's new state cannot silently disappear again).
+
+- **`POSTGRES_APPLICATION_NAME`** (default `postgres-mcp`), so agent traffic is
+  identifiable in `pg_stat_activity` instead of anonymous -- while `pg_health`
+  itself reports `application_name` for every other session. An
+  `application_name` in `DATABASE_URL` still wins.
+
+### Changed
+
+- **BREAKING: `pg_seq_scan_tables` and `pg_unused_indexes` return an envelope,
+  not a bare row array.** `data` is now
+  `{rows, stats_reset, stats_reset_age_seconds}`; callers read `data.rows`.
+  The reason is `pg_unused_indexes` could tell an agent to drop a load-bearing
+  index: a scan count is meaningless without knowing when the counters were
+  reset, and if that happened an hour ago every index looks unused. Both tools
+  also report `last_idx_scan` / `last_seq_scan` on PG16+, where "not scanned
+  since March" beats a bare counter.
+
+- **BREAKING: `pg_top_queries` returns the same envelope**, for the same
+  reason -- it ranks cumulative `total_exec_time` / `calls`. Its clock is NOT
+  `pg_stat_database.stats_reset` but `pg_stat_statements_info.stats_reset`, a
+  genuinely independent reset point; using the wrong one would have been worse
+  than omitting it. The same view supplies `dealloc`, which is the subtler
+  trap: it counts how often entries for the least-executed statements were
+  evicted for exceeding `pg_stat_statements.max`, so a non-zero value means
+  the "top queries" ranking is drawn from an incomplete population. Both
+  require extension 1.9 (PostgreSQL 14) and are omitted below it rather than
+  returned as nulls that would read as "never reset".
+
+- **BREAKING: `pg_explain` with `analyze: true` now emits `BUFFERS`.** PG18
+  turns it on by default server-side; on PG15-17 it had to be requested and was
+  absent. Plans gain buffer lines, so text plans roughly double in length and
+  `POSTGRES_MAX_ROWS` truncation can fire where it previously did not. Pass
+  `buffers: false` for the old output.
+
+- **BREAKING: the Node floor is now 22.** Node 20 reached end of life; the
+  supported lines are 22, 24 and 26. The esbuild target deliberately stays at
+  `node20` -- it only controls syntax downleveling, so a lower floor keeps the
+  bundle runnable under alternate runtimes.
+
+- **Tools register through `registerTool` instead of `server.tool`.** All six
+  `server.tool` overloads are deprecated as of SDK 1.30 and are gone in the v2
+  packages. `registerTool` is also the only form that can carry `outputSchema`,
+  so this is what makes structured tool output reachable later. Tools now
+  advertise a top-level `title` as well as `annotations.title`; both are emitted,
+  since dropping either regresses hosts that read only one.
+
+- **`@modelcontextprotocol/sdk` 1.29.0 -> 1.30.0** (the final v1.x release) and
+  **`pg` ^8.14.0 -> ^8.23.0**. The pg bump makes `sslnegotiation=direct`
+  available in `DATABASE_URL`, which skips a round trip against PG17+ servers;
+  it stays opt-in because a PG16-or-older server rejects the connection.
+
+- **Documented where PostgreSQL support actually sits.** PG13 reached end of
+  life on 2025-11-13 and PG14 does so on 2026-11-12. The integration matrix
+  remains 15 / 17 / 18.
+
+### Fixed
+
+- **`POSTGRES_APPLICATION_NAME` was missing from the oam sandbox allowlist.**
+  Under `POSTGRES_MCP_SANDBOX=1`, oam removes an undeclared variable from
+  `process.env` rather than denying access, so the operator's configured name
+  would have silently vanished and the server would have reported the default
+  -- precisely the silent-misbehaviour failure the launcher's own comment warns
+  about. `PGAPPNAME` is now granted too, since the pg driver reads it as its
+  own fallback for the same setting. A new test scans the shipped bundle for
+  literal `process.env` reads and fails if any config variable is absent from
+  the allowlist, so the next one cannot ship silently.
+
+- **`pg_io_stats` had no test coverage at all.** `tools.test.ts` builds its own
+  `allTools` array separate from `index.ts`, and the new tool was added to one
+  and not the other -- exempting it from every structural check, including the
+  duplicate-name guard. Both arrays now agree, and the tool has a unit suite
+  covering its version branches.
+
+- **A version probe suspended across `shutdown()` could republish a stale
+  server version.** `shutdown()` clears the cache, but a probe already awaiting
+  its query would resolve afterwards and write the OLD server's version into
+  the cache the NEW pool uses -- gating catalog queries against the wrong
+  server, the exact failure the reset exists to prevent. A generation counter
+  now invalidates in-flight probes, matching the guard `resolveTypeNames`
+  already had.
+
+- **Four `pg_explain` tests asserted the wrong thing whenever `DATABASE_URL`
+  was set.** Their helper cleared the env var to force the version probe's
+  "unknown" sentinel, but the pool and the version cache are module-scoped and
+  survive that, so on any machine with `DATABASE_URL` exported the probe never
+  re-ran. The suite passed or failed depending on the developer's environment.
+  The helper now calls `shutdown()` on both edges.
+
+### Security
+
+- **`fast-uri` bumped past the host-confusion advisory (GHSA, high).** It
+  reaches the published artifact rather than staying a build-time concern: the
+  MCP SDK depends on `ajv`, which depends on `fast-uri`, and esbuild bundles the
+  whole graph into `dist/index.js`. "It is only a devDependency" is not the
+  right test for this package -- the bundle is what ships, and the dependency
+  tree is flattened into it. `npm audit` now reports zero vulnerabilities.
+
 ## [0.10.0] - 2026-08-08
 
 ### Added

package/README.md

@@ -167,22 +167,23 @@ The bigger leverage is multi-tool reasoning. A few real workflows:
 | `pg_query` | Run a SQL query. Writes gated by the role in `DATABASE_URL` first, `ALLOW_WRITES` second. Supports parameterized queries via `params`. Result fields include `dataTypeName` (e.g. `int4`, `jsonb`) alongside `dataTypeID`. |
 | `pg_list_schemas` | List non-system schemas. |
 | `pg_list_tables` | List tables (and optionally views) in a schema with estimated row counts. Paginated via `limit`/`offset`. |
-| `pg_describe_table` | Kind, columns, PK, outgoing FKs, incoming FKs (`referenced_by`), CHECK / UNIQUE / EXCLUDE constraints, indexes, and partition parent/children for a relation. |
+| `pg_describe_table` | Kind, columns, PK, outgoing FKs, incoming FKs (`referenced_by`), CHECK / UNIQUE / EXCLUDE constraints, indexes, and partition parent/children for a relation. Generated and identity columns are flagged (`generated`, `identity`, `generation_expression`) so an agent doesn't try to write to them. Constraints carry `validated`, plus `enforced` / `has_period` on PG18+. |
 | `pg_list_views` | List views and materialized views in a schema, including their SQL definitions. |
 | `pg_list_functions` | List functions, procedures, and aggregates in a schema with signatures and return types. |
 | `pg_list_extensions` | List installed extensions (pgvector, postgis, pg_stat_statements, etc.) with versions. |
 | `pg_search_columns` | Find columns by name pattern across all user schemas. Case-insensitive, supports SQL LIKE wildcards. |
-| `pg_explain` | `EXPLAIN` or `EXPLAIN ANALYZE` for a SQL statement. Text or JSON output. Optional `hypothetical_indexes` (requires the [HypoPG](https://github.com/HypoPG/hypopg) extension) lets you ask "what would the plan be with these indexes?" without creating them on disk. |
-| `pg_health` | Server version, database size, connection count, active queries, table count. |
-| `pg_top_queries` | Top N queries by total/mean execution time. Requires the `pg_stat_statements` extension. |
-| `pg_seq_scan_tables` | Tables with heavy sequential scans - missing-index candidates. |
-| `pg_unused_indexes` | Non-unique, non-primary indexes with low scan counts - drop candidates. |
+| `pg_explain` | `EXPLAIN` or `EXPLAIN ANALYZE` for a SQL statement. Text or JSON output. Planner options: `buffers` (on by default with `analyze`), `settings`, `verbose`, `wal`, `costs`, `timing`, plus `generic_plan` (PG16+, plan a parameterized query with no values) and `memory` / `serialize` (PG17+). Optional `hypothetical_indexes` (requires the [HypoPG](https://github.com/HypoPG/hypopg) extension) lets you ask "what would the plan be with these indexes?" without creating them on disk. |
+| `pg_health` | Server version, database size, connections against `max_connections`, active queries with wait events and transaction age, `pg_stat_database` rollup (deadlocks, temp files, cache hit ratio), table count. |
+| `pg_top_queries` | Top N queries by total/mean execution time. Requires the `pg_stat_statements` extension. Returns `stats_reset` (from `pg_stat_statements_info`, a different clock from the other stats tools) and `dealloc` on extension 1.9+ - a non-zero `dealloc` means entries were evicted past `pg_stat_statements.max`, so the ranking is drawn from an incomplete population. |
+| `pg_seq_scan_tables` | Tables with heavy sequential scans - missing-index candidates. Returns the `stats_reset` window alongside the rows, since the counters mean nothing without it. `last_seq_scan` / `last_idx_scan` on PG16+. |
+| `pg_unused_indexes` | Non-unique, non-primary indexes with low scan counts - drop candidates. Also returns `stats_reset`: a recently reset counter makes every index look unused, which is how a load-bearing index gets dropped. `last_idx_scan` on PG16+. |
+| `pg_io_stats` | I/O observability: `pg_stat_io` read/write/extend/fsync counts, bytes and times per backend type and context (PG16+), plus in-flight async I/O handles from `pg_aios` and the active `io_method` (PG18+). |
 | `pg_inspect_locks` | Who is blocking whom right now (blocked PID, blocker PID, lock type, queries). |
 | `pg_list_roles` | Database roles with login/superuser/createdb flags and group memberships. |
 | `pg_table_privileges` | Who has SELECT/INSERT/UPDATE/DELETE/etc. on a table or whole schema. |
 | `pg_table_bloat` | Tables with high dead-tuple ratios - VACUUM candidates. |
 | `pg_replication_status` | Replication slots, connected replicas, and current WAL position. |
-| `pg_advisor` | Rolled-up DBA lints in one call: sequence-exhaustion candidates, tables without a primary key, and (configurable) public tables with RLS disabled. The "what should I be looking at?" starting point. |
+| `pg_advisor` | Rolled-up DBA lints in one call: sequence-exhaustion candidates, wraparound risk for both counters (per-database and per-table `age(relfrozenxid)` against `autovacuum_freeze_max_age`, and `mxid_age(relminmxid)` against `autovacuum_multixact_freeze_max_age` -- a lock-heavy workload can exhaust multixacts while xids look healthy; `triggered_by` says which), tables without a primary key, and (configurable) public tables with RLS disabled. The "what should I be looking at?" starting point. |
 | `pg_kill` | Cancel a running query or terminate a backend connection. Requires `ALLOW_WRITES=1`. |
 
 ## Configuration
@@ -198,12 +199,25 @@ All env vars are read from the MCP server's environment:
 | `POSTGRES_MAX_ROWS` | `1000` | Cap on rows returned by `pg_query`. |
 | `POSTGRES_POOL_MAX` | `5` | Max pool connections. Set to `1` for single-threaded backends (pglite-socket, PgBouncer transaction mode). |
 | `POSTGRES_SSL_REJECT_UNAUTHORIZED` | unset | Set to `false` to skip TLS cert verification (for managed DBs using private-CA certs). Connection is still encrypted. |
+| `POSTGRES_APPLICATION_NAME` | `postgres-mcp` | Value reported in `pg_stat_activity.application_name`, so agent traffic is identifiable to whoever is watching the database. An `application_name` in `DATABASE_URL` takes precedence over this. |
 | `POSTGRES_MCP_RUNTIME` | `auto` | Which JS runtime executes the server: `auto` (prefer [oam](https://oamjs.org), fall back to Node), `oam` (require oam, fail if absent), `node` (never use oam). See [Runtime](#runtime). |
 | `OAM_BIN` | unset | Explicit path to an `oam` binary, checked before PATH and the default install locations. |
 
 ### Supported Postgres versions
 
-Tested on **PostgreSQL 15, 17 and 18** in the integration matrix. Should work on PG13+ -- a few tools (`pg_replication_status` reading `wal_status`, `pg_top_queries` reading `*_exec_time`) rely on columns that landed in PG13. PG12 and below are out of upstream support and not exercised here.
+Tested on **PostgreSQL 15, 17 and 18** in the integration matrix.
+
+Works on PG13+, but note where upstream support actually sits: **PG13 reached end of life on 2025-11-13 and PG14 does so on 2026-11-12.** PG13/14 are not exercised here and are not a compatibility target going forward. PG12 and below are further out of support and some tools rely on columns that landed in PG13 (`pg_replication_status` reading `wal_status`, `pg_top_queries` reading `*_exec_time`).
+
+Newer server versions unlock extra fields rather than being required. Every version-dependent column is gated on `server_version_num` and simply omitted on servers that predate it, so nothing errors -- you get a slightly thinner answer. The cut points that matter:
+
+| Server | What it adds |
+|--------|--------------|
+| PG16+ | `last_idx_scan` / `last_seq_scan` in the stats tools (index/table staleness rather than a bare counter), `pg_explain` `generic_plan` |
+| PG17+ | `pg_explain` `memory` and `serialize` |
+| PG18+ | Generated-column form (`stored` vs `virtual`), NOT NULL constraint validity, `conenforced` / `conperiod` constraint metadata in `pg_describe_table`, `relallfrozen` freeze coverage in `pg_advisor`. `BUFFERS` is on by default with `EXPLAIN ANALYZE` server-side |
+
+If the version probe fails, the server assumes the oldest supported shape rather than emitting SQL a server might reject.
 
 ### Runtime
 
@@ -211,7 +225,7 @@ The published `postgres-mcp` command is a small launcher that prefers the [oam](
 
 **If you do not have oam, nothing changes.** The fallback is not a re-exec: npm already started Node to run the launcher, so falling back is a plain `import()` of the server into that same process. It costs a few `existsSync` calls and no subprocess, and behaves identically to running `dist/index.js` under Node directly.
 
-**If you do have oam,** the server runs under it. Verified equivalent on both runtimes: all 21 tools register, queries return identical rows and `dataTypeName` values, and the error paths match. oam supplies every `node:` builtin the driver needs, including `net`, `tls`, `crypto`, and `dns` (SCRAM auth and the extended query protocol both work).
+**If you do have oam,** the server runs under it. Verified equivalent on both runtimes: all 22 tools register, queries return identical rows and `dataTypeName` values, and the error paths match. oam supplies every `node:` builtin the driver needs, including `net`, `tls`, `crypto`, and `dns` (SCRAM auth and the extended query protocol both work).
 
 **Startup cost, measured.** windows-arm64, 1.4 MB bundle, `postgres-mcp version` (full module init), every binary warmed first, mean of 12 runs:
 
@@ -263,6 +277,14 @@ To allow the connection while keeping traffic encrypted, add `POSTGRES_SSL_REJEC
 
 This disables certificate chain verification only -- the TCP connection is still TLS-encrypted end-to-end. For production setups where you can install the CA, prefer putting the cert in the Node trust store (`NODE_EXTRA_CA_CERTS`) over disabling verification globally.
 
+**Shaving a round trip on PG17+.** Postgres 17 added direct TLS negotiation, which skips the plaintext `SSLRequest` handshake before the TLS one. The bundled driver supports it, so append `sslnegotiation=direct` to your `DATABASE_URL`:
+
+```
+postgres://user:pass@host:5432/db?sslmode=require&sslnegotiation=direct
+```
+
+It is opt-in rather than a default because a PG16-or-older server will reject the connection outright, and the saving is one round trip per pooled connection -- worth it on a distant managed database, invisible on a local one.
+
 ## Troubleshooting
 
 **`DATABASE_URL is not set`** - Your MCP client is launching the server without the env var. On Windows especially, env vars set in bash / PowerShell profiles are not inherited by MCP servers launched via `cmd`. Put `DATABASE_URL` directly in the `env` block of `.mcp.json`.
@@ -307,7 +329,9 @@ wsl -d Ubuntu -u root bash /mnt/c/path/to/postgres-mcp/scripts/wsl-pg-setup.sh
 wsl -d Ubuntu -u root bash /mnt/c/path/to/postgres-mcp/scripts/wsl-test-matrix.sh
 ```
 
-`wsl-pg-setup.sh` installs PG17 and PG18 from the PGDG apt repo on ports 5432 and 5433, sets the `postgres` password to `postgres`, and creates `postgres_mcp_test` in each. `wsl-test-matrix.sh` rsyncs the working tree into `/root/postgres-mcp`, runs `npm ci` once, and runs the integration suite against every cluster found via `pg_lsclusters`.
+`wsl-pg-setup.sh` installs PG15, PG17 and PG18 from the PGDG apt repo (ports are auto-assigned by `pg_createcluster` -- typically 17 on 5432, 18 on 5433, 15 on 5434), sets the `postgres` password to `postgres`, and creates `postgres_mcp_test` in each. `wsl-test-matrix.sh` rsyncs the working tree into `/root/postgres-mcp`, runs `npm ci` once, and runs the integration suite against every cluster found via `pg_lsclusters`.
+
+**Running these from Git Bash instead of PowerShell?** Prefix both script invocations with `MSYS_NO_PATHCONV=1`. Git Bash rewrites the `/mnt/c/...` argument before `wsl.exe` sees it, so the script arrives as `C:/Users/<you>/scoop/apps/git/<ver>/mnt/c/...` and bash exits with "No such file or directory" having run nothing. Also avoid piping either script into `tail`/`head` -- the pipeline's exit status is the last command's, so a failing matrix reports success.
 
 Tear down when finished: `wsl --unregister Ubuntu`.
 

package/bin/postgres-mcp.mjs

@@ -179,7 +179,15 @@ function sandboxFlags() {
     }
   }
 
-  const env = ["ALLOW_WRITES","DATABASE_URL","NODE_PG_FORCE_NATIVE","PGCONNECT_TIMEOUT","PGSSLMODE","POSTGRES_CONNECTION_TIMEOUT_MS","POSTGRES_MAX_ROWS","POSTGRES_POOL_MAX","POSTGRES_SSL_REJECT_UNAUTHORIZED","POSTGRES_STATEMENT_TIMEOUT_MS","USER","USERNAME"];
+  // Every variable the SHIPPED BUNDLE reads, including the pg driver's own
+  // lookups. Adding a config env var to src/ without adding it here is a silent
+  // regression under the sandbox, not a loud one: oam removes an undeclared var
+  // from process.env rather than denying access, so the server reads undefined
+  // and quietly takes its default. POSTGRES_APPLICATION_NAME is read by
+  // getApplicationName() in src/api.ts; PGAPPNAME is pg's own env fallback for
+  // the same setting (connection-parameters.js: val('application_name', config,
+  // 'PGAPPNAME')), so omitting it would drop a name set the driver's way.
+  const env = ["ALLOW_WRITES","DATABASE_URL","NODE_PG_FORCE_NATIVE","PGAPPNAME","PGCONNECT_TIMEOUT","PGSSLMODE","POSTGRES_APPLICATION_NAME","POSTGRES_CONNECTION_TIMEOUT_MS","POSTGRES_MAX_ROWS","POSTGRES_POOL_MAX","POSTGRES_SSL_REJECT_UNAUTHORIZED","POSTGRES_STATEMENT_TIMEOUT_MS","USER","USERNAME"];
 
   const flags = ["--permission", netFlag, `--allow-env=${env.join(",")}`];
   return flags;