@mosaic-code/pg-index-check 0.0.0-stage → 0.1.1

package/LICENSE

@@ -0,0 +1,207 @@
+# Do No Harm License
+
+## 1. Preamble
+
+Most software today is developed with little to no thought of how it will be used, or the
+consequences for our society and planet.
+
+As software developers, we engineer the infrastructure of the 21st century. We recognise that our
+infrastructure has great power to shape the world and the lives of those we share it with, and we
+choose to consciously take responsibility for the social and environmental impacts of what we build.
+
+We envisage a world free from injustice, inequality, and the reckless destruction of lives and our
+planet. We reject slavery in all its forms, whether by force, indebtedness, or by algorithms that
+hack human vulnerabilities. We seek a world where humankind is at peace with our neighbours, nature,
+and ourselves. We want our work to enrich the physical, mental and spiritual wellbeing of all
+society.
+
+We build software to further this vision of a just world, or at the very least, to not put that
+vision further from reach.
+
+## 2. Definitions
+
+"License" shall mean the terms and conditions for use, reproduction, and distribution as defined by
+Sections 1 through 9 of this document.
+
+"Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is
+granting the License.
+
+"Legal Entity" shall mean the union of the acting entity and all other entities that control, are
+controlled by, or are under common control with that entity. For the purposes of this definition,
+"control" means (i) the power, direct or indirect, to cause the direction or management of such
+entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the
+outstanding shares, or (iii) beneficial ownership of such entity.
+
+"You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this
+License.
+
+"Source" form shall mean the preferred form for making modifications, including but not limited to
+software source code, documentation source, and configuration files.
+
+"Object" form shall mean any form resulting from mechanical transformation or translation of a
+Source form, including but not limited to compiled object code, generated documentation, and
+conversions to other media types.
+
+"Work" shall mean the work of authorship, whether in Source or Object form, made available under the
+License, as indicated by a copyright notice that is included in or attached to the work (an example
+is provided in the Appendix below).
+
+"Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or
+derived from) the Work and for which the editorial revisions, annotations, elaborations, or other
+modifications represent, as a whole, an original work of authorship. For the purposes of this
+License, Derivative Works shall not include works that remain separable from, or merely link (or
+bind by name) to the interfaces of, the Work and Derivative Works thereof.
+
+"Contribution" shall mean any work of authorship, including the original version of the Work and any
+modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted
+to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity
+authorized to submit on behalf of the copyright owner. For the purposes of this definition,
+"submitted" means any form of electronic, verbal, or written communication sent to the Licensor or
+its representatives, including but not limited to communication on electronic mailing lists, source
+code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor
+for the purpose of discussing and improving the Work, but excluding communication that is
+conspicuously marked or otherwise designated in writing by the copyright owner as "Not a
+Contribution."
+
+"Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a
+Contribution has been received by Licensor and subsequently incorporated within the Work.
+
+"Forests" shall mean 0.5 or more hectares of trees that were either planted more than 50 years ago
+or were not planted by humans or human made equipment.
+
+"Deforestation" shall mean the clearing, burning or destruction of 0.5 or more hectares of forests
+within a 1 year period.
+
+## 3. Grant of Copyright License
+
+Subject to the terms and conditions of this License, each Contributor hereby grants to You a
+perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to
+reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and
+distribute the Work and such Derivative Works in Source or Object form.
+
+## 4. Grant of Patent License
+
+Subject to the terms and conditions of this License, each Contributor hereby grants to You a
+perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this
+section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer
+the Work, where such license applies only to those patent claims licensable by such Contributor that
+are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s)
+with the Work to which such Contribution(s) was submitted. If You institute patent litigation
+against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or
+a Contribution incorporated within the Work constitutes direct or contributory patent infringement,
+then any patent licenses granted to You under this License for that Work shall terminate as of the
+date such litigation is filed.
+
+## 5. Redistribution
+
+You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with
+or without modifications, and in Source or Object form, provided that You meet the following
+conditions:
+
+1. You must give any other recipients of the Work or Derivative Works a copy of this License; and
+
+2. You must cause any modified files to carry prominent notices stating that You changed the
+   files; and
+
+3. You must retain, in the Source form of any Derivative Works that You distribute, all copyright,
+   patent, trademark, and attribution notices from the Source form of the Work, excluding those
+   notices that do not pertain to any part of the Derivative Works; and
+
+4. Neither the name of the copyright holder nor the names of its contributors may be used to endorse
+   or promote products derived from this software without specific prior written permission; and
+
+5. This software must not be used by any organisation, website, product, or service that:
+   1. promotes, lobbies for or derives a majority of income from:
+         1. **abuses of human rights**:
+            * human trafficking
+            * sex trafficking
+            * slavery or indentured servitude
+            * discrimination based on age, gender, gender identity, race, sexuality, religion, nationality
+            * hate speech
+         2. **environmental destruction**:
+            * the extraction or sale of fossil fuels
+            * the destruction of habitats for threatened or endangered species, including through deforestation or burning of forests
+            * the abuse, inhumane killing or neglect of animals under human control
+            * industrial processes that generate waste products that threaten life
+         3. **conflict and war**:
+            * warfare
+            * war crimes
+            * weapons manufacturing
+            * violence (except when required to protect public safety)
+         4. **addictive or destructive products and services**:
+            * gambling
+            * tobacco
+            * products that encourage adversely addictive behaviours
+
+   2. dissuades, lobbies against, or derives a majority of income from actions that discourage or frustrate:
+      * peace
+      * access to the rights set out in the Universal Declaration of Human Rights and the Convention on the Rights of the Child
+      * democratic processes
+      * peaceful assembly and association (including worker associations)
+      * a sustainable environment
+   ; and
+
+5. If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative
+   Works that You distribute must include a readable copy of the attribution notices contained
+   within such NOTICE file, excluding those notices that do not pertain to any part of the
+   Derivative Works, in at least one of the following places: within a NOTICE text file
+   distributed as part of the Derivative Works; within the Source form or documentation, if
+   provided along with the Derivative Works; or, within a display generated by the Derivative
+   Works, if and wherever such third-party notices normally appear. The contents of the NOTICE
+   file are for informational purposes only and do not modify the License. You may add Your own
+   attribution notices within Derivative Works that You distribute, alongside or as an addendum to
+   the NOTICE text from the Work, provided that such additional attribution notices cannot be
+   construed as modifying the License.
+
+You may add Your own copyright statement to Your modifications and may provide additional or
+different license terms and conditions for use, reproduction, or distribution of Your modifications,
+or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of
+the Work otherwise complies with the conditions stated in this License.
+
+## 6. Submission of Contributions
+
+Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the
+Work by You to the Licensor shall be under the terms and conditions of this License, without any
+additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify
+the terms of any separate license agreement you may have executed with Licensor regarding such
+Contributions.
+
+## 7. Trademarks
+
+This License does not grant permission to use the trade names, trademarks, service marks, or product
+names of the Licensor, except as required for reasonable and customary use in describing the origin
+of the Work and reproducing the content of the NOTICE file.
+
+## 8. Disclaimer of Warranty
+
+Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each
+Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied, including, without limitation, any warranties or conditions of
+TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely
+responsible for determining the appropriateness of using or redistributing the Work and assume any
+risks associated with Your exercise of permissions under this License.
+
+## 9. Limitation of Liability
+
+In no event and under no legal theory, whether in tort (including negligence), contract, or
+otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or
+agreed to in writing, shall any Contributor be liable to You for damages, including any direct,
+indirect, special, incidental, or consequential damages of any character arising as a result of this
+License or out of the use or inability to use the Work (including but not limited to damages for
+loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial
+damages or losses), even if such Contributor has been advised of the possibility of such damages.
+
+## 10. Accepting Warranty or Additional Liability
+
+While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee
+for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights
+consistent with this License. However, in accepting such obligations, You may act only on Your own
+behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You
+agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or
+claims asserted against, such Contributor by reason of your accepting any such warranty or
+additional liability.
+
+## Attribution
+
+Do No Harm License [Contributor Covenant], (pre 1.0),
+available at https://github.com/raisely/NoHarm

package/README.md

@@ -1,3 +1,177 @@
-# Temporary Holding Version
+# pg-index-check
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+Fail your test suite when PostgreSQL queries have no usable index.
+
+Your tests already run the queries your app makes. pg-index-check reads them back from `pg_stat_statements` after the suite, plans each one with sequential scans disabled, and reports any query that still has to read a whole table (or a whole index) to apply its filter.
+
+## Features
+
+- **Works on tiny test tables** - With `enable_seqscan = off`, a sequential scan only survives when no index can serve the query, however few rows the table has
+- **Any driver or ORM** - Pure PostgreSQL: captures statements with `pg_stat_statements`, so Prisma, Drizzle, Knex or raw `pg` all work without code changes
+- **Source locations** - Picks up `/* path:line */` comments from [pg-source-comments](https://github.com/mosaic-code-coop/pg-source-comments) and shows where each query came from
+- **Auditable allowlist** - Ignore rules need a reason, and rules that no longer match anything are reported
+
+## Requirements
+
+- PostgreSQL 16+ (uses `EXPLAIN (GENERIC_PLAN)`)
+- `pg_stat_statements` in `shared_preload_libraries` on the test database server
+- Node.js 22.9.0+, pg 8.0.0+
+
+## Installation
+
+```bash
+npm install --save-dev @mosaic-code/pg-index-check pg
+```
+
+## Quick Start
+
+### 1. Load pg_stat_statements on the test database
+
+```yaml
+# docker-compose.yml
+services:
+  postgres-test:
+    image: postgres:18-alpine
+    command: postgres -c shared_preload_libraries=pg_stat_statements
+```
+
+GitHub Actions `services:` can't pass server flags, so start Postgres with `docker compose up -d --wait` (or `docker run ... postgres -c shared_preload_libraries=pg_stat_statements`) in a step instead.
+
+### 2. Check after the test suite
+
+```typescript
+// tests/global-setup.ts
+import { assertIndexes, resetCapture } from '@mosaic-code/pg-index-check';
+
+const connectionString = process.env.TEST_DATABASE_URL!;
+
+export async function setup() {
+  // Installs the extension in the database if needed, and clears earlier statements
+  await resetCapture({ connectionString });
+}
+
+export async function teardown() {
+  // Throws an IndexCheckError listing every query without a usable index
+  await assertIndexes({
+    connectionString,
+    ignore: [{ table: 'public.countries', reason: 'fixed lookup table, ~250 rows' }],
+  });
+}
+```
+
+```typescript
+// vitest.config.ts
+export default defineConfig({
+  test: { globalSetup: ['tests/global-setup.ts'] },
+});
+```
+
+A failing check fails the run (vitest exits non-zero when teardown throws):
+
+```
+pg-index-check: 1 finding(s) across 42 statement(s)
+
+SEQ_SCAN_FILTER public.orders at src/orders/service.ts:88
+  condition: (orders.customer_id = $1)
+  fix: CREATE INDEX ON public.orders (customer_id)
+  statement: SELECT /* src/orders/service.ts:88 */ "public"."orders"."id", ... WHERE "public"."orders"."customer_id" = $1
+```
+
+## What Gets Flagged
+
+Every distinct `SELECT`, `WITH`, `UPDATE` and `DELETE` run against the current database since `resetCapture()` is planned with `EXPLAIN (GENERIC_PLAN, VERBOSE, FORMAT JSON)`.
+
+| Kind | Plan shape | Typical cause |
+|---|---|---|
+| `SEQ_SCAN_FILTER` | Sequential scan with a filter | No index on the filtered column(s) |
+| `FULL_INDEX_SCAN` | Index scan with no index condition, filtering on one column with `=` | `WHERE status = $1 ORDER BY id LIMIT n` with no index on `status` |
+| `TRAILING_INDEX_COLUMN` | Index condition with `=` that skips the index's first column (PostgreSQL 16/17 only) | Filtering on `created_at` with an index on `(org_id, created_at)` |
+| `UNPLANNABLE` | PostgreSQL could not plan the statement | Statement refers to a table the tests dropped |
+
+Not flagged:
+
+- `<>` / `NOT IN` filters, which a btree index can't serve
+- Range and `IS NULL` filters on a whole-index scan, since pagination walks the primary key on purpose
+- `TRAILING_INDEX_COLUMN` on PostgreSQL 18+, which can skip-scan
+- `INSERT`, DDL, utility statements, and queries on `pg_catalog` / `information_schema`
+
+## Configuration
+
+```typescript
+await assertIndexes({
+  connectionString,
+
+  // Findings to accept. Needs a table and/or statement, and a reason.
+  //   table: string matches exactly (`public.orders`), or a RegExp
+  //   statement: string matches if the statement contains it, or a RegExp
+  ignore: [
+    { table: 'public.countries', reason: 'fixed lookup table' },
+    { statement: /FROM "public"."audit_log"/, reason: 'admin-only export, runs nightly' },
+  ],
+
+  // 'fail' (default): unplannable statements fail the check
+  // 'report': list them in the report without failing
+  unplannable: 'fail',
+});
+```
+
+`checkIndexes()` takes the same options and returns the result without throwing:
+
+```typescript
+const { statementCount, findings, reported, staleIgnoreRules } = await checkIndexes({ connectionString });
+```
+
+## Using with pg-source-comments
+
+When the app's `pg` pool is wrapped with [pg-source-comments](https://github.com/mosaic-code-coop/pg-source-comments) 0.4.0+, findings include the source file and line of the query.
+
+- **Raw `pg` queries:** wrap the pool with `createTrackedPool`.
+- **Prisma queries** (`findMany` etc.): also wrap the client with `trackPrismaClient(new PrismaClient({ adapter: new PrismaPg(pool) }))`. Prisma sends queries to the pool after an async hop, so without this the calling code is off the stack and no location is recorded. Needs pg-source-comments 0.4.0+.
+
+`pg_stat_statements` keeps the text of the first call it sees for each distinct query, so a query made from several places shows the first location only.
+
+## Limitations
+
+- **Only tested queries are checked** - A query the test suite never runs isn't seen
+- **Joins** - An index that only a join would use (e.g. an unindexed foreign key on the inner side) isn't reported
+- **Generic plans** - Plans ignore actual parameter values. The check answers "can an index serve this?", not "will production use it?"
+- **One database** - Only statements from the database in `connectionString` are checked; `resetCapture()` clears statistics for the whole server
+
+## How It Works
+
+```
+resetCapture()       test suite runs              assertIndexes()
+     │                      │                           │
+pg_stat_statements_reset  app queries recorded   read statements for this database
+                          in pg_stat_statements  SET enable_seqscan = off (own connection)
+                                                 EXPLAIN (GENERIC_PLAN) each statement
+                                                 apply rules → ignore rules → report
+```
+
+The check runs on its own connection, so `enable_seqscan` never changes for the app's sessions.
+
+## Development
+
+```bash
+# Install dependencies
+npm install
+
+# Unit tests
+npm test
+
+# Integration tests (starts PostgreSQL 18 with docker compose)
+npm run test:integration:setup
+
+# Against another PostgreSQL version
+docker compose down && POSTGRES_VERSION=16 npm run test:integration:setup
+
+# Type check
+npm run typecheck
+
+# Build
+npm run build
+```
+
+## License
+
+[Do No Harm](https://github.com/raisely/NoHarm)