@steadybrew/postgresjs 4.5.0-rc1 โ†’ 4.5.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,1746 +1,127 @@
1
- # Postgres.js by Steady Brew
2
-
3
- A maintained fork of [Postgres.js](https://github.com/porsager/postgres), the PostgreSQL client created by [Rasmus Porsager](https://github.com/porsager), with the same API.
4
-
5
- - **Fixes reported Postgres.js bugs:** 40 open upstream issues, from queries receiving another query's rows to hung pools and crashed processes, each covered by a regression test. See [the list](#fixed-postgresjs-issues).
6
- - **Built for serverless and connection poolers:** bounded connection retries with backoff, idle connections replaced when a frozen instance resumes, pool statistics from `sql.stats()`, and an adapter for Vercel Fluid. Query timeouts and cancellable pool waits are next on the [roadmap](ROADMAP.md).
7
- - **Switch with one line:** your code and the libraries that import `postgres` keep working. See [Switching from Postgres.js](#switching-from-postgresjs).
8
-
9
- ---
1
+ <p align="center">
2
+ <img width="440" height="180" alt="Postgres.js" src="postgresjs.svg">
3
+ </p>
10
4
 
11
- <img align="left" width="440" height="180" alt="Fastest full PostgreSQL nodejs client" src="https://raw.githubusercontent.com/porsager/postgres/master/postgresjs.svg?sanitize=true">
12
-
13
- - [๐Ÿš€ Fastest full-featured node & deno client](https://github.com/porsager/postgres-benchmarks#results)
14
- - ๐Ÿท ES6 Tagged Template Strings at the core
15
- - ๐Ÿ„โ€โ™€๏ธ Simple surface API
16
- - ๐Ÿ–Š๏ธ Dynamic query support
17
-
18
- <br clear="all">
19
-
20
- ## Compatibility
5
+ # Postgres.js by Steady Brew
21
6
 
22
7
  [![test](https://github.com/steadybrew/postgresjs/actions/workflows/test.yml/badge.svg?branch=main)](https://github.com/steadybrew/postgresjs/actions/workflows/test.yml?query=branch%3Amain)
23
8
 
24
- **Version 4 runs on Node.js 22 or newer** and supports PostgreSQL 15โ€“18. Every release passes the full test matrix before it is published.
25
-
26
- | Runtime | PostgreSQL | Notes |
27
- | --- | --- | --- |
28
- | Node 22, 24 and 26, ESM and CJS | 15โ€“18 | |
29
- | Deno 1.46.3 | 17 | No Deno 2 claim |
30
- | Cloudflare Workers (workerd) | 17 | A "Stream was cancelled" rejection on socket close remains ([#1196](https://github.com/porsager/postgres/issues/1196), [#1202](https://github.com/porsager/postgres/issues/1202)) |
31
- | Bun | | Best effort, not tested in CI |
32
-
33
- [The test workflow](https://github.com/steadybrew/postgresjs/blob/main/.github/workflows/test.yml) defines which pairs run on each push. Supabase PostgreSQL and Supabase Edge are separate environments; Deno 1.46.3 coverage does not imply Deno 2 or hosted Supabase Edge.
34
-
35
- ## Switching from Postgres.js
9
+ **Keep the [Postgres.js](https://github.com/porsager/postgres) API by [Rasmus Porsager](https://github.com/porsager) and get critical fixes and new features:**
36
10
 
37
- ```sh
38
- npm install postgres@npm:@steadybrew/postgresjs@^4
39
- ```
11
+ - **Correctness fixes.** Queries get the right results, transactions stay isolated, and rollbacks are reported as failures.
12
+ - **Fewer hangs and crashes.** Fixes for stuck queries, shutdown hangs and crashes after disconnects.
13
+ - **Pool monitoring.** See busy connections, idle connections and waiting queries with `sql.stats()`.
14
+ - **Faster queries behind poolers.** Removes an extra network round trip for supported values (opt-in).
15
+ - **Vercel Fluid Compute support.** Reuse connections between requests and close idle ones before suspension to avoid connection leaks.
40
16
 
41
- This installs the fork under the name `postgres`, so `import postgres from 'postgres'` stays as it is, and so do libraries that import it, such as `drizzle-orm/postgres-js`, `drizzle-kit` and `kysely-postgres-js`. Then read [Upgrading to 4.0.0](#upgrading-to-400) for the behaviour changes.
17
+ **[50+ reported upstream bugs fixed](docs/fixed-issues.md)**, with regression tests.
42
18
 
43
- Tracing: from 4.5.0, dd-trace records the same database spans as with Postgres.js when the fork is installed under the `postgres` name. Sentry's automatic integration instruments `postgres` versions below 4 only, so widen its range before Sentry loads:
19
+ [Start a new project](#start-a-new-project) ยท [Switch from Postgres.js](#switching-from-postgresjs) ยท [Why this fork](#why-this-fork) ยท [API reference](docs/api.md) ยท [Roadmap](ROADMAP.md)
44
20
 
45
- ```js
46
- // sentry-postgres.mjs
47
- import { SENTRY_RUNTIME_INSTRUMENTATIONS } from '@sentry/server-utils/orchestrion/config'
21
+ <a id="install"></a>
48
22
 
49
- for (const instrumentation of SENTRY_RUNTIME_INSTRUMENTATIONS) {
50
- if (instrumentation.module.name === 'postgres')
51
- instrumentation.module.versionRange = '>=3.0.0 <5'
52
- }
53
- ```
23
+ ## Start a new project
54
24
 
55
25
  ```sh
56
- node --import ./sentry-postgres.mjs --import @sentry/node/import app.js
57
- ```
58
-
59
- This covers Sentry's runtime hook, not its bundler plugins. With pnpm, add `@sentry/server-utils` at the same version as `@sentry/node` so the file can import it.
60
-
61
- ## Upgrading to 4.0.0
62
-
63
- Version 4.0.0 fixes connection startup, reservations, reconnects, shutdown, transaction ownership, and built-in array handling. When upgrading from Postgres.js 3.4 or `@steadybrew/postgresjs` 3.x:
64
-
65
- - Upgrade to **Node.js 22 or newer** before installing.
66
- - Keep queries within the reservation or transaction that owns them. Reserved handles used after `release()`, and transaction handles used after completion, reject with `CONNECTION_ENDED`. A handle whose connection has closed cannot run queries on a reconnected session.
67
- - Pass a non-empty string without NUL characters to transaction `sql.prepare(name)`. Invalid names throw `INVALID_TRANSACTION_NAME`.
68
- - If you use `fetch_types: false`, review code that expects built-in arrays as raw strings: supported built-in arrays are now decoded as JavaScript arrays. SQL `NULL` elements become JavaScript `null`; quoted `"NULL"` text remains a string.
69
-
70
- See the [changelog](./CHANGELOG.md#breaking-changes-and-migration) for the full migration notes and fixes.
71
-
72
- ## Fixed Postgres.js issues
73
-
74
- Upstream issues on [porsager/postgres](https://github.com/porsager/postgres/issues) that this fork fixes, and the version that fixed them. Each fix has a test that reproduces the report. The [changelog](./CHANGELOG.md) has the details.
75
-
76
- <details>
77
- <summary>40 bugs and 4 feature requests</summary>
78
-
79
- **Wrong results and data safety**
80
-
81
- | Issue | What happens now | Since |
82
- | --- | --- | --- |
83
- | [#1082](https://github.com/porsager/postgres/issues/1082), [#729](https://github.com/porsager/postgres/issues/729) | A query whose values fail to serialize rejects on its own; the query running alongside no longer gets its error, and the next query no longer gets another query's rows | 4.3.0 |
84
- | [#1216](https://github.com/porsager/postgres/issues/1216) | A transaction handle whose connection closed can't run on a reconnected session, so it can't read another user's rows through RLS | 4.0.0 |
85
- | [#1248](https://github.com/porsager/postgres/issues/1248) | Statements in a `sql.begin()` callback after its connection closes reject instead of running outside the transaction | 4.0.0 |
86
- | [#1204](https://github.com/porsager/postgres/issues/1204) | A transaction's connection is never handed to another transaction | 4.0.0 |
87
- | [#1217](https://github.com/porsager/postgres/issues/1217) | `sql.begin()` rejects with `TRANSACTION_ROLLED_BACK` when the server rolled back the COMMIT | 4.4.0 |
88
- | [#1249](https://github.com/porsager/postgres/issues/1249), [#1226](https://github.com/porsager/postgres/issues/1226), [#1223](https://github.com/porsager/postgres/issues/1223) | The first query after a reconnect no longer rejects with an error from the closed session | 4.0.0 |
89
- | [#789](https://github.com/porsager/postgres/issues/789) | `sql.array()` infers the right array type on first use | 4.0.0 |
90
-
91
- **Hangs**
92
-
93
- | Issue | What happens now | Since |
94
- | --- | --- | --- |
95
- | [#751](https://github.com/porsager/postgres/issues/751), [#1219](https://github.com/porsager/postgres/issues/1219), [#1203](https://github.com/porsager/postgres/issues/1203) | `sql.reserve()` on a pool that hasn't opened a connection yet resolves, with or without `fetch_types` and on Cloudflare Workers | 4.0.0 |
96
- | [#1195](https://github.com/porsager/postgres/issues/1195), [#1199](https://github.com/porsager/postgres/issues/1199) | A reservation survives its connection being terminated by the server, and the pool recovers after it is released | 4.0.0 |
97
- | [#1186](https://github.com/porsager/postgres/issues/1186) | Queries queued on a reserved or transaction handle settle when its connection closes | 4.0.0 |
98
- | [#1097](https://github.com/porsager/postgres/issues/1097), [#1130](https://github.com/porsager/postgres/issues/1130), [#1242](https://github.com/porsager/postgres/issues/1242), [#861](https://github.com/porsager/postgres/issues/861) | `sql.end()` resolves after a disconnect, after a failed query and after a connection closes inside `sql.begin()`, and waits for queries queued in the pool | 4.0.0 |
99
- | [#1246](https://github.com/porsager/postgres/issues/1246) | A query sent between a socket error and its close settles | 4.1.0 |
100
- | [#1193](https://github.com/porsager/postgres/issues/1193) | A connection closed during startup settles its first query instead of reconnecting in a tight loop | 4.0.0 |
101
- | [#1234](https://github.com/porsager/postgres/issues/1234) | Concurrent queries in `sql.begin()` after a column type change roll the transaction back and return the connection to the pool | 4.4.0 |
102
-
103
- **Crashes**
104
-
105
- | Issue | What happens now | Since |
106
- | --- | --- | --- |
107
- | [#1066](https://github.com/porsager/postgres/issues/1066), [#1154](https://github.com/porsager/postgres/issues/1154), [#1208](https://github.com/porsager/postgres/issues/1208), [#1133](https://github.com/porsager/postgres/issues/1133) | A reserved or transaction handle whose backend died rejects its statements instead of throwing an uncaught exception | 4.0.0 |
108
- | [#1192](https://github.com/porsager/postgres/issues/1192), [#1205](https://github.com/porsager/postgres/issues/1205), [#1086](https://github.com/porsager/postgres/issues/1086) | A failed or cancelled type query at startup rejects the query instead of crashing the process | 4.0.0 |
109
- | [#1088](https://github.com/porsager/postgres/issues/1088) | Timers no longer fail on Cloudflare Workers | 4.0.0 |
110
-
111
- **Reconnects and failover**
112
-
113
- | Issue | What happens now | Since |
114
- | --- | --- | --- |
115
- | [#1179](https://github.com/porsager/postgres/issues/1179) | Failed connection attempts back off instead of the whole pool retrying at once | 4.0.0 |
116
- | [#1174](https://github.com/porsager/postgres/issues/1174), [#815](https://github.com/porsager/postgres/issues/815), [#988](https://github.com/porsager/postgres/issues/988) | With several hosts, retries move to the next host, a connect timeout fails over, and `prefer-standby` falls back to a primary | 4.0.0 |
117
-
118
- **Transactions, notices and subscriptions**
119
-
120
- | Issue | What happens now | Since |
121
- | --- | --- | --- |
122
- | [#823](https://github.com/porsager/postgres/issues/823), [#1189](https://github.com/porsager/postgres/issues/1189), [#1210](https://github.com/porsager/postgres/issues/1210) | `sql.begin()` no longer fails with `UNSAFE_TRANSACTION` under write backpressure or with `max_pipeline: 0` | 4.0.0 |
123
- | [#1063](https://github.com/porsager/postgres/issues/1063) | `onnotice: false` silences notices | 4.4.0 |
124
- | [#1087](https://github.com/porsager/postgres/issues/1087) | `sql.subscribe()` no longer adds a listener on each subscribe or loses changes after a transform throws | 4.4.0 |
125
-
126
- **Feature requests**
127
-
128
- | Issue | What you get | Since |
129
- | --- | --- | --- |
130
- | [#443](https://github.com/porsager/postgres/issues/443), [#908](https://github.com/porsager/postgres/issues/908) | `sql.stats()` reports connections in use and idle, and queries waiting | 4.3.0 |
131
- | [#1100](https://github.com/porsager/postgres/issues/1100) | An `onidle` callback and the `@steadybrew/postgresjs/vercel` adapter | 4.2.0 |
132
- | [#1164](https://github.com/porsager/postgres/issues/1164) | Built-in array types are decoded with `fetch_types: false` | 4.0.0 |
133
-
134
- </details>
135
-
136
- ## Getting started
137
-
138
- <br>
139
- <img height="220" width="458" alt="Good UX with Postgres.js" src="https://raw.githubusercontent.com/porsager/postgres/master/demo.gif">
140
- <br>
141
-
142
- ### Installation
143
- ```bash
144
- $ npm install @steadybrew/postgresjs
145
- ```
146
-
147
- ### Usage
148
- Create your `sql` database instance
149
- ```js
150
- // db.js
151
- import postgres from '@steadybrew/postgresjs'
152
-
153
- const sql = postgres({ /* options */ }) // will use psql environment variables
154
-
155
- export default sql
156
- ```
157
-
158
- Simply import for use elsewhere
159
- ```js
160
- // users.js
161
- import sql from './db.js'
162
-
163
- async function getUsersOver(age) {
164
- const users = await sql`
165
- select
166
- name,
167
- age
168
- from users
169
- where age > ${ age }
170
- `
171
- // users = Result [{ name: "Walter", age: 80 }, { name: 'Murray', age: 68 }, ...]
172
- return users
173
- }
174
-
175
-
176
- async function insertUser({ name, age }) {
177
- const users = await sql`
178
- insert into users
179
- (name, age)
180
- values
181
- (${ name }, ${ age })
182
- returning name, age
183
- `
184
- // users = Result [{ name: "Murray", age: 68 }]
185
- return users
186
- }
187
- ```
188
-
189
- #### ESM dynamic imports
190
-
191
- The library can be used with ESM dynamic imports as well as shown here.
192
-
193
- ```js
194
- const { default: postgres } = await import('@steadybrew/postgresjs')
195
- ```
196
-
197
- ## Table of Contents
198
-
199
- * [Upgrading to 4.0.0](#upgrading-to-400)
200
- * [Connection](#connection)
201
- * [Queries](#queries)
202
- * [Building queries](#building-queries)
203
- * [Advanced query methods](#advanced-query-methods)
204
- * [Transactions](#transactions)
205
- * [Data Transformation](#data-transformation)
206
- * [Listen & notify](#listen--notify)
207
- * [Realtime subscribe](#realtime-subscribe)
208
- * [Numbers, bigint, numeric](#numbers-bigint-numeric)
209
- * [Parameter types](#parameter-types)
210
- * [Result Array](#result-array)
211
- * [Connection details](#connection-details)
212
- * [Custom Types](#custom-types)
213
- * [Teardown / Cleanup](#teardown--cleanup)
214
- * [Error handling](#error-handling)
215
- * [TypeScript support](#typescript-support)
216
- * [Reserving connections](#reserving-connections)
217
- * [Changelog](./CHANGELOG.md)
218
-
219
-
220
- ## Connection
221
-
222
- ### `postgres([url], [options])`
223
-
224
- You can use either a `postgres://` url connection string or the options to define your database connection properties. Options in the object will override any present in the url. Options will fall back to the same environment variables as psql.
225
-
226
- ```js
227
- const sql = postgres('postgres://username:password@host:port/database', {
228
- host : '', // Postgres ip address[s] or domain name[s]
229
- port : 5432, // Postgres server port[s]
230
- database : '', // Name of database to connect to
231
- username : '', // Username of database user
232
- password : '', // Password of database user
233
- ...and more
234
- })
235
- ```
236
-
237
- More options can be found in the [Connection details section](#connection-details).
238
-
239
- ## Queries
240
-
241
- ### ```await sql`...` -> Result[]```
242
-
243
- Postgres.js utilizes [Tagged template functions](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_literals#Tagged_templates) to process query parameters **before** interpolation. Using tagged template literals benefits developers by:
244
-
245
- 1. **Enforcing** safe query generation
246
- 2. Giving the ` sql`` ` function powerful [utility](#dynamic-inserts) and [query building](#building-queries) features.
247
-
248
- Any generic value will be serialized as described in [Parameter types](#parameter-types), and replaced by a PostgreSQL protocol placeholder `$1, $2, ...`. The values are then sent separately to PostgreSQL, which handles escaping & casting.
249
-
250
- All queries will return a `Result` array, with objects mapping column names to each row.
251
-
252
- ```js
253
- const xs = await sql`
254
- insert into users (
255
- name, age
256
- ) values (
257
- 'Murray', 68
258
- )
259
-
260
- returning *
261
- `
262
-
263
- // xs = [{ user_id: 1, name: 'Murray', age: 68 }]
264
- ```
265
-
266
- > Please note that queries are first executed when `awaited` โ€“ or instantly by using [`.execute()`](#execute).
267
-
268
- ### Query parameters
269
-
270
- Parameters are automatically extracted and handled by the database so that SQL injection isn't possible. No special handling is necessary, simply use tagged template literals as usual.
271
-
272
- ```js
273
- const name = 'Mur'
274
- , age = 60
275
-
276
- const users = await sql`
277
- select
278
- name,
279
- age
280
- from users
281
- where
282
- name like ${ name + '%' }
283
- and age > ${ age }
284
- `
285
- // users = [{ name: 'Murray', age: 68 }]
286
- ```
287
-
288
- > Be careful with quotation marks here. Because Postgres infers column types, you do not need to wrap your interpolated parameters in quotes like `'${name}'`. This will cause an error because the tagged template replaces `${name}` with `$1` in the query string, leaving Postgres to do the interpolation. If you wrap that in a string, Postgres will see `'$1'` and interpret it as a string as opposed to a parameter.
289
-
290
- ### Dynamic column selection
291
-
292
- ```js
293
- const columns = ['name', 'age']
294
-
295
- await sql`
296
- select
297
- ${ sql(columns) }
298
- from users
299
- `
300
-
301
- // Which results in:
302
- select "name", "age" from users
303
- ```
304
-
305
- ### Dynamic inserts
306
-
307
- ```js
308
- const user = {
309
- name: 'Murray',
310
- age: 68
311
- }
312
-
313
- await sql`
314
- insert into users ${
315
- sql(user, 'name', 'age')
316
- }
317
- `
318
-
319
- // Which results in:
320
- insert into users ("name", "age") values ($1, $2)
321
-
322
- // The columns can also be given with an array
323
- const columns = ['name', 'age']
324
-
325
- await sql`
326
- insert into users ${
327
- sql(user, columns)
328
- }
329
- `
330
- ```
331
-
332
- **You can omit column names and simply execute `sql(user)` to get all the fields from the object as columns**. Be careful not to allow users to supply columns that you do not want to be inserted.
333
-
334
- #### Multiple inserts in one query
335
- If you need to insert multiple rows at the same time it's also much faster to do it with a single `insert`. Simply pass an array of objects to `sql()`.
336
-
337
- ```js
338
- const users = [{
339
- name: 'Murray',
340
- age: 68,
341
- garbage: 'ignore'
342
- },
343
- {
344
- name: 'Walter',
345
- age: 80
346
- }]
347
-
348
- await sql`insert into users ${ sql(users, 'name', 'age') }`
349
-
350
- // Is translated to:
351
- insert into users ("name", "age") values ($1, $2), ($3, $4)
352
-
353
- // Here you can also omit column names which will use object keys as columns
354
- await sql`insert into users ${ sql(users) }`
355
-
356
- // Which results in:
357
- insert into users ("name", "age") values ($1, $2), ($3, $4)
358
- ```
359
-
360
- ### Dynamic columns in updates
361
- This is also useful for update queries
362
- ```js
363
- const user = {
364
- id: 1,
365
- name: 'Murray',
366
- age: 68
367
- }
368
-
369
- await sql`
370
- update users set ${
371
- sql(user, 'name', 'age')
372
- }
373
- where user_id = ${ user.id }
374
- `
375
-
376
- // Which results in:
377
- update users set "name" = $1, "age" = $2 where user_id = $3
378
-
379
- // The columns can also be given with an array
380
- const columns = ['name', 'age']
381
-
382
- await sql`
383
- update users set ${
384
- sql(user, columns)
385
- }
386
- where user_id = ${ user.id }
387
- `
388
- ```
389
-
390
- ### Multiple updates in one query
391
- To create multiple updates in a single query, it is necessary to use arrays instead of objects to ensure that the order of the items correspond with the column names.
392
- ```js
393
- const users = [
394
- [1, 'John', 34],
395
- [2, 'Jane', 27],
396
- ]
397
-
398
- await sql`
399
- update users set name = update_data.name, age = (update_data.age)::int
400
- from (values ${sql(users)}) as update_data (id, name, age)
401
- where users.id = (update_data.id)::int
402
- returning users.id, users.name, users.age
403
- `
404
- ```
405
-
406
- ### Dynamic values and `where in`
407
- Value lists can also be created dynamically, making `where in` queries simple too.
408
- ```js
409
- const users = await sql`
410
- select
411
- *
412
- from users
413
- where age in ${ sql([68, 75, 23]) }
414
- `
415
- ```
416
-
417
- or
418
- ```js
419
- const [{ a, b, c }] = await sql`
420
- select
421
- *
422
- from (values ${ sql(['a', 'b', 'c']) }) as x(a, b, c)
423
- `
26
+ npm install @steadybrew/postgresjs
424
27
  ```
425
28
 
426
- ## Building queries
29
+ Or, for Drizzle or Kysely, use the alias:
427
30
 
428
- Postgres.js features a simple dynamic query builder by conditionally appending/omitting query fragments.
429
- It works by nesting ` sql`` ` fragments within other ` sql`` ` calls or fragments. This allows you to build dynamic queries safely without risking sql injections through usual string concatenation.
430
-
431
- ### Partial queries
432
- ```js
433
- const olderThan = x => sql`and age > ${ x }`
434
-
435
- const filterAge = true
436
-
437
- await sql`
438
- select
439
- *
440
- from users
441
- where name is not null ${
442
- filterAge
443
- ? olderThan(50)
444
- : sql``
445
- }
446
- `
447
- // Which results in:
448
- select * from users where name is not null
449
- // Or
450
- select * from users where name is not null and age > 50
451
- ```
452
-
453
- ### Dynamic filters
454
- ```js
455
- await sql`
456
- select
457
- *
458
- from users ${
459
- id
460
- ? sql`where user_id = ${ id }`
461
- : sql``
462
- }
463
- `
464
-
465
- // Which results in:
466
- select * from users
467
- // Or
468
- select * from users where user_id = $1
469
- ```
470
-
471
- ### Dynamic ordering
472
-
473
- ```js
474
- const id = 1
475
- const order = {
476
- username: 'asc'
477
- created_at: 'desc'
478
- }
479
- await sql`
480
- select
481
- *
482
- from ticket
483
- where account = ${ id }
484
- order by ${
485
- Object.entries(order).flatMap(([column, order], i) =>
486
- [i ? sql`,` : sql``, sql`${ sql(column) } ${ order === 'desc' ? sql`desc` : sql`asc` }`]
487
- )
488
- }
489
- `
490
- ```
491
-
492
- ### SQL functions
493
- Using keywords or calling functions dynamically is also possible by using ``` sql`` ``` fragments.
494
- ```js
495
- const date = null
496
-
497
- await sql`
498
- update users set updated_at = ${ date || sql`now()` }
499
- `
500
-
501
- // Which results in:
502
- update users set updated_at = now()
503
- ```
504
-
505
- ### Table names
506
- Dynamic identifiers like table names and column names is also supported like so:
507
- ```js
508
- const table = 'users'
509
- , column = 'id'
510
-
511
- await sql`
512
- select ${ sql(column) } from ${ sql(table) }
513
- `
514
-
515
- // Which results in:
516
- select "id" from "users"
517
- ```
518
-
519
- ### Quick primer on interpolation
520
-
521
- Here's a quick oversight over all the ways to do interpolation in a query template string:
522
-
523
- | Interpolation syntax | Usage | Example |
524
- | ------------- | ------------- | ------------- |
525
- | `${ sql`` }` | for keywords or sql fragments | ``await sql`SELECT * FROM users ${sql`order by age desc` }` `` |
526
- | `${ sql(string) }` | for identifiers | ``await sql`SELECT * FROM ${sql('table_name')` `` |
527
- | `${ sql([] or {}, ...) }` | for helpers | ``await sql`INSERT INTO users ${sql({ name: 'Peter'})}` `` |
528
- | `${ 'somevalue' }` | for values | ``await sql`SELECT * FROM users WHERE age = ${42}` `` |
529
-
530
- ## Advanced query methods
531
-
532
- ### Cursors
533
-
534
- #### ```await sql``.cursor([rows = 1], [fn])```
535
-
536
- Use cursors if you need to throttle the amount of rows being returned from a query. You can use a cursor either as an [async iterable](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/for-await...of) or with a callback function. For a callback function new results won't be requested until the promise / async callback function has resolved.
537
-
538
- ##### callback function
539
- ```js
540
- await sql`
541
- select
542
- *
543
- from generate_series(1,4) as x
544
- `.cursor(async([row]) => {
545
- // row = { x: 1 }
546
- await http.request('https://example.com/wat', { row })
547
- })
548
- ```
549
-
550
- ##### for await...of
551
- ```js
552
- // for await...of
553
- const cursor = sql`select * from generate_series(1,4) as x`.cursor()
554
-
555
- for await (const [row] of cursor) {
556
- // row = { x: 1 }
557
- await http.request('https://example.com/wat', { row })
558
- }
559
- ```
560
-
561
- A single row will be returned by default, but you can also request batches by setting the number of rows desired in each batch as the first argument to `.cursor`:
562
- ```js
563
- await sql`
564
- select
565
- *
566
- from generate_series(1,1000) as x
567
- `.cursor(10, async rows => {
568
- // rows = [{ x: 1 }, { x: 2 }, ... ]
569
- await Promise.all(rows.map(row =>
570
- http.request('https://example.com/wat', { row })
571
- ))
572
- })
573
- ```
574
-
575
- If an error is thrown inside the callback function no more rows will be requested and the outer promise will reject with the thrown error.
576
-
577
- You can close the cursor early either by calling `break` in the `for await...of` loop, or by returning the token `sql.CLOSE` from the callback function.
578
-
579
- ```js
580
- await sql`
581
- select * from generate_series(1,1000) as x
582
- `.cursor(row => {
583
- return Math.random() > 0.9 && sql.CLOSE // or sql.END
584
- })
585
- ```
586
-
587
- ### Instant iteration
588
-
589
- #### ```await sql``.forEach(fn)```
590
-
591
- If you want to handle rows returned by a query one by one, you can use `.forEach` which returns a promise that resolves once there are no more rows.
592
- ```js
593
- await sql`
594
- select created_at, name from events
595
- `.forEach(row => {
596
- // row = { created_at: '2019-11-22T14:22:00Z', name: 'connected' }
597
- })
598
-
599
- // No more rows
600
- ```
601
-
602
- ### Query Descriptions
603
- #### ```await sql``.describe() -> Result[]```
604
-
605
- Rather than executing a given query, `.describe` will return information utilized in the query process. This information can include the query identifier, column types, etc.
606
-
607
- This is useful for debugging and analyzing your Postgres queries. Furthermore, **`.describe` will give you access to the final generated query string that would be executed.**
608
-
609
- ### Rows as Array of Values
610
- #### ```sql``.values()```
611
-
612
- Using `.values` will return rows as an array of values for each column, instead of objects.
613
-
614
- This can be useful to receive identically named columns, or for specific performance/transformation reasons. The column definitions are still included on the result array, plus access to parsers for each column.
615
-
616
- ### Rows as Raw Array of Buffers
617
- #### ```sql``.raw()```
618
-
619
- Using `.raw` will return rows as an array with `Buffer` values for each column, instead of objects.
620
-
621
- This can be useful for specific performance/transformation reasons. The column definitions are still included on the result array, plus access to parsers for each column.
622
-
623
- ### Queries in Files
624
- #### `await sql.file(path, [args], [options]) -> Result[]`
625
-
626
- Using a file for a query is also supported with optional parameters to use if the file includes `$1, $2, etc`
627
-
628
- ```js
629
- const result = await sql.file('query.sql', ['Murray', 68])
630
- ```
631
-
632
- ### Multiple statements in one query
633
- #### ```await sql``.simple()```
634
-
635
- The postgres wire protocol supports ["simple"](https://www.postgresql.org/docs/current/protocol-flow.html#id-1.10.6.7.4) and ["extended"](https://www.postgresql.org/docs/current/protocol-flow.html#PROTOCOL-FLOW-EXT-QUERY) queries. "simple" queries supports multiple statements, but does not support any dynamic parameters. "extended" queries support parameters but only one statement. To use "simple" queries you can use
636
- ```sql``.simple()```. That will create it as a simple query.
637
-
638
- ```js
639
- await sql`select 1; select 2;`.simple()
640
- ```
641
-
642
- ### Copy to/from as Streams
643
-
644
- Postgres.js supports [`COPY ...`](https://www.postgresql.org/docs/14/sql-copy.html) queries, which are exposed as [Node.js streams](https://nodejs.org/api/stream.html).
645
-
646
- #### ```await sql`copy ... from stdin`.writable() -> Writable```
647
-
648
- ```js
649
- import { pipeline } from 'node:stream/promises'
650
-
651
- // Stream of users with the default tab delimitated cells and new-line delimitated rows
652
- const userStream = Readable.from([
653
- 'Murray\t68\n',
654
- 'Walter\t80\n'
655
- ])
656
-
657
- const query = await sql`copy users (name, age) from stdin`.writable()
658
- await pipeline(userStream, query);
659
- ```
660
-
661
- #### ```await sql`copy ... to stdout`.readable() -> Readable```
662
-
663
- ##### Using Stream Pipeline
664
- ```js
665
- import { pipeline } from 'node:stream/promises'
666
- import { createWriteStream } from 'node:fs'
667
-
668
- const readableStream = await sql`copy users (name, age) to stdout`.readable()
669
- await pipeline(readableStream, createWriteStream('output.tsv'))
670
- // output.tsv content: `Murray\t68\nWalter\t80\n`
671
- ```
672
-
673
- ##### Using `for await...of`
674
- ```js
675
- const readableStream = await sql`
676
- copy (
677
- select name, age
678
- from users
679
- where age = 68
680
- ) to stdout
681
- `.readable()
682
- for await (const chunk of readableStream) {
683
- // chunk.toString() === `Murray\t68\n`
684
- }
685
- ```
686
-
687
- > **NOTE** This is a low-level API which does not provide any type safety. To make this work, you must match your [`copy query` parameters](https://www.postgresql.org/docs/14/sql-copy.html) correctly to your [Node.js stream read or write](https://nodejs.org/api/stream.html) code. Ensure [Node.js stream backpressure](https://nodejs.org/en/learn/modules/backpressuring-in-streams) is handled correctly to avoid memory exhaustion.
688
-
689
- A `COPY` holds its connection until it finishes. Postgres.js cannot tell that a simple-protocol query is a `COPY` until the server answers, so it sends nothing behind a simple query whose text contains the word `copy` (`sql.unsafe()` or `sql.file()` without parameters, `.simple()`, `.readable()`, `.writable()`). Queries sent through `sql` meanwhile run on another connection or wait for one. When every connection is running a `COPY` that has not been ended, for example with `max: 1`, queries, `sql.reserve()` and `sql.begin()` through `sql` reject with `COPY_IN_PROGRESS` instead, because only that `COPY` finishing could free a connection. Statements on a `sql.reserve()` or `sql.begin()` handle reject with `COPY_IN_PROGRESS` while its `COPY` runs.
690
-
691
- ### Canceling Queries in Progress
692
-
693
- Postgres.js supports, [canceling queries in progress](https://www.postgresql.org/docs/7.1/protocol-protocol.html#AEN39000). It works by opening a new connection with a protocol level startup message to cancel the current query running on a specific connection. That means there is no guarantee that the query will be canceled, and due to the possible race conditions it might even result in canceling another query. This is fine for long running queries, but in the case of high load and fast queries it might be better to simply ignore results instead of canceling.
694
-
695
- ```js
696
- const query = sql`select pg_sleep 100`.execute()
697
- setTimeout(() => query.cancel(), 100)
698
- const result = await query
699
- ```
700
-
701
- ### Execute
702
-
703
- #### ```await sql``.execute()```
704
-
705
- The lazy Promise implementation in Postgres.js is what allows it to distinguish [Nested Fragments](#building-queries) from the main outer query. This also means that queries are always executed at the earliest in the following tick. If you have a specific need to execute the query in the same tick, you can call `.execute()`
706
-
707
- ### Unsafe raw string queries
708
-
709
- <details>
710
- <summary>Advanced unsafe use cases</summary>
711
-
712
- ### `await sql.unsafe(query, [args], [options]) -> Result[]`
713
-
714
- If you know what you're doing, you can use `unsafe` to pass any string you'd like to postgres. Please note that this can lead to SQL injection if you're not careful.
715
-
716
- ```js
717
- sql.unsafe('select ' + danger + ' from users where id = ' + dragons)
31
+ ```sh
32
+ npm install postgres@npm:@steadybrew/postgresjs@^4
718
33
  ```
719
34
 
720
- By default, `sql.unsafe` assumes the `query` string is sufficiently dynamic that prepared statements do not make sense, and so defaults them to off. If you'd like to re-enable prepared statements, you can pass `{ prepare: true }`.
721
-
722
- You can also nest `sql.unsafe` within a safe `sql` expression. This is useful if only part of your fraction has unsafe elements.
723
-
724
35
  ```js
725
- const triggerName = 'friend_created'
726
- const triggerFnName = 'on_friend_created'
727
- const eventType = 'insert'
728
- const schema_name = 'app'
729
- const table_name = 'friends'
36
+ import postgres from '@steadybrew/postgresjs' // or 'postgres' with the alias
730
37
 
731
- await sql`
732
- create or replace trigger ${sql(triggerName)}
733
- after ${sql.unsafe(eventType)} on ${sql.unsafe(`${schema_name}.${table_name}`)}
734
- for each row
735
- execute function ${sql(triggerFnName)}()
736
- `
38
+ const sql = postgres(process.env.DATABASE_URL)
737
39
 
738
- await sql`
739
- create role friend_service with login password ${sql.unsafe(`'${password}'`)}
740
- `
40
+ const minAge = 18
41
+ const users = await sql`select name, age from users where age >= ${ minAge }`
741
42
  ```
742
43
 
743
- </details>
744
-
745
- ## Transactions
746
-
747
- #### BEGIN / COMMIT `await sql.begin([options = ''], fn) -> fn()`
748
-
749
- Use `sql.begin` to start a new transaction. Postgres.js will reserve a connection for the transaction and supply a scoped `sql` instance for all transaction uses in the callback function. `sql.begin` will resolve with the returned value from the callback function.
750
-
751
- `BEGIN` is automatically sent with the optional options, and if anything fails `ROLLBACK` will be called so the connection can be released and execution can continue.
752
-
753
- ```js
754
- const [user, account] = await sql.begin(async sql => {
755
- const [user] = await sql`
756
- insert into users (
757
- name
758
- ) values (
759
- 'Murray'
760
- )
761
- returning *
762
- `
44
+ ## Switching from Postgres.js
763
45
 
764
- const [account] = await sql`
765
- insert into accounts (
766
- user_id
767
- ) values (
768
- ${ user.user_id }
769
- )
770
- returning *
771
- `
46
+ **Check first.** The [migration codemod](codemod/README.md) reports required changes, behaviour differences to review and useful fixes, with file and line. It changes nothing:
772
47
 
773
- return [user, account]
774
- })
48
+ ```sh
49
+ npx @steadybrew/postgresjs-migrate .
775
50
  ```
776
51
 
777
- Do note that you can often achieve the same result using [`WITH` queries (Common Table Expressions)](https://www.postgresql.org/docs/current/queries-with.html) instead of using transactions.
52
+ **Then migrate with Codemod:**
778
53
 
779
- It's also possible to pipeline the requests in a transaction if needed by returning an array with queries from the callback function like this:
780
-
781
- ```js
782
- const result = await sql.begin(sql => [
783
- sql`update ...`,
784
- sql`update ...`,
785
- sql`insert ...`
786
- ])
54
+ ```sh
55
+ npx codemod @steadybrew/postgresjs-migrate
787
56
  ```
788
57
 
789
- #### SAVEPOINT `await sql.savepoint([name], fn) -> fn()`
790
-
791
- ```js
792
- sql.begin('read write', async sql => {
793
- const [user] = await sql`
794
- insert into users (
795
- name
796
- ) values (
797
- 'Murray'
798
- )
799
- `
800
-
801
- const [account] = (await sql.savepoint(sql =>
802
- sql`
803
- insert into accounts (
804
- user_id
805
- ) values (
806
- ${ user.user_id }
807
- )
808
- `
809
- ).catch(err => {
810
- // Account could not be created. ROLLBACK SAVEPOINT is called because we caught the rejection.
811
- })) || []
58
+ Run it from your project root. It checks again, switches dependencies and overrides to the fork, and removes patches already covered by the fork. Your source code and imports stay unchanged. Run your package manager's install afterwards to update the lockfile.
812
59
 
813
- return [user, account]
814
- })
815
- .then(([user, account]) => {
816
- // great success - COMMIT succeeded
817
- })
818
- .catch(() => {
819
- // not so good - ROLLBACK was called
820
- })
821
- ```
60
+ <a id="upgrading-to-400"></a>
822
61
 
62
+ See the [migration guide](docs/upgrading.md) for behaviour changes, manual installation and the direct npm CLI.
823
63
 
824
- #### PREPARE TRANSACTION `sql.prepare(name)`
64
+ ### Drizzle, Kysely and tracing
825
65
 
826
- Indicates that the transactions should be prepared using the [`PREPARE TRANSACTION [NAME]`](https://www.postgresql.org/docs/current/sql-prepare-transaction.html) statement
827
- instead of being committed.
66
+ The alias above works with `drizzle-orm/postgres-js`, `drizzle-kit`, `kysely-postgres-js` and dd-trace. The codemod sets it up for existing projects, preserving `import postgres from 'postgres'`.
828
67
 
829
- The name must be a non-empty string without NUL characters; invalid names throw `INVALID_TRANSACTION_NAME`. Quotes and backslashes in valid names are preserved.
68
+ **Sentry needs an extra step:** add the [tracing preload](docs/upgrading.md#tracing) to retain automatic database spans. The codemod flags Sentry but does **not** apply this adjustment.
830
69
 
831
- ```js
832
- sql.begin('read write', async sql => {
833
- const [user] = await sql`
834
- insert into users (
835
- name
836
- ) values (
837
- 'Murray'
838
- )
839
- `
70
+ ## Security
840
71
 
841
- await sql.prepare('tx1')
842
- })
843
- ```
72
+ Published from GitHub Actions through npm trusted publishing, with provenance and maintainer approval using two-factor authentication. No stored npm token, runtime dependencies or registry install scripts. Verify with `npm audit signatures`.
844
73
 
845
- ## Data Transformation
74
+ [Changes since upstream 3.4.9](https://github.com/steadybrew/postgresjs/compare/v3.4.9...main) ยท [Report a vulnerability](SECURITY.md) ยท [Unlicense](UNLICENSE)
846
75
 
847
- Postgres.js allows for transformation of the data passed to or returned from a query by using the `transform` option.
76
+ ## Compatibility
848
77
 
849
- Built in transformation functions are:
78
+ Releases pass the [test matrix](.github/workflows/test.yml) before publishing:
850
79
 
851
- * For camelCase - `postgres.camel`, `postgres.toCamel`, `postgres.fromCamel`
852
- * For PascalCase - `postgres.pascal`, `postgres.toPascal`, `postgres.fromPascal`
853
- * For Kebab-Case - `postgres.kebab`, `postgres.toKebab`, `postgres.fromKebab`
80
+ | Runtime | PostgreSQL tested |
81
+ | --- | --- |
82
+ | Node.js 22, 24 and 26, ESM and CommonJS | 15โ€“18 |
83
+ | Deno 1.46.3 | 17 |
84
+ | Cloudflare Workers (workerd) | 17 |
854
85
 
855
- These built in transformations will only convert to/from snake_case. For example, using `{ transform: postgres.toCamel }` will convert the column names to camelCase only if the column names are in snake_case to begin with. `{ transform: postgres.fromCamel }` will convert camelCase only to snake_case.
86
+ TypeScript declarations are included; use `@types/node` 22+ or `skipLibCheck`.
856
87
 
857
- By default, using `postgres.camel`, `postgres.pascal` and `postgres.kebab` will perform a two-way transformation - both the data passed to the query and the data returned by the query will be transformed:
88
+ Older Node.js versions may work but aren't tested. The package declares `engines.node: >=22`, so package managers may warn or reject installation on older versions.
858
89
 
859
- ```js
860
- // Transform the column names to and from camel case
861
- const sql = postgres({ transform: postgres.camel })
90
+ Hosted Supabase Edge Functions are deployed in production with this fork, outside the CI matrix. Standalone Deno 2 is unverified. Bun is best effort, without CI coverage. Cloudflare Workers has a known socket-close rejection ([#1196](https://github.com/porsager/postgres/issues/1196), [#1202](https://github.com/porsager/postgres/issues/1202)).
862
91
 
863
- await sql`CREATE TABLE IF NOT EXISTS camel_case (a_test INTEGER, b_test TEXT)`
864
- await sql`INSERT INTO camel_case ${ sql([{ aTest: 1, bTest: 1 }]) }`
865
- const data = await sql`SELECT ${ sql('aTest', 'bTest') } FROM camel_case`
92
+ ## Why this fork
866
93
 
867
- console.log(data) // [ { aTest: 1, bTest: '1' } ]
868
- ```
94
+ As many of my deployments are on serverless, I ran into some Postgres.js limitations that made it harder to use it reliably. I moved a couple of projects off it and patched others to compare the pros and cons.
869
95
 
870
- To only perform half of the transformation (eg. only the transformation **to** or **from** camel case), use the other transformation functions:
96
+ But I also realised that even for a small team of one, there is a clear path to updating and fixing Postgres.js, and I decided to bring those changes, whether from local patches, parked issues or PRs, to the benefit of the wider community.
871
97
 
872
- ```js
873
- // Transform the column names only to camel case
874
- // (for the results that are returned from the query)
875
- postgres({ transform: postgres.toCamel })
98
+ The aim of this fork is to move faster while keeping rigorous review, automate broader, tedious testing, welcome community contributions, and make a library I enjoy using work better for everyone.
876
99
 
877
- await sql`CREATE TABLE IF NOT EXISTS camel_case (a_test INTEGER)`
878
- await sql`INSERT INTO camel_case ${ sql([{ a_test: 1 }]) }`
879
- const data = await sql`SELECT a_test FROM camel_case`
100
+ ### Fixed Postgres.js issues
880
101
 
881
- console.log(data) // [ { aTest: 1 } ]
882
- ```
883
-
884
- ```js
885
- // Transform the column names only from camel case
886
- // (for interpolated inserts, updates, and selects)
887
- const sql = postgres({ transform: postgres.fromCamel })
888
-
889
- await sql`CREATE TABLE IF NOT EXISTS camel_case (a_test INTEGER)`
890
- await sql`INSERT INTO camel_case ${ sql([{ aTest: 1 }]) }`
891
- const data = await sql`SELECT ${ sql('aTest') } FROM camel_case`
892
-
893
- console.log(data) // [ { a_test: 1 } ]
894
- ```
895
-
896
- > Note that Postgres.js does not rewrite the static parts of the tagged template strings. So to transform column names in your queries, the `sql()` helper must be used - eg. `${ sql('columnName') }` as in the examples above.
897
-
898
- ### Transform `undefined` Values
899
-
900
- By default, Postgres.js will throw the error `UNDEFINED_VALUE: Undefined values are not allowed` when undefined values are passed
901
-
902
- ```js
903
- // Transform the column names to and from camel case
904
- const sql = postgres({
905
- transform: {
906
- undefined: null
907
- }
908
- })
909
-
910
- await sql`CREATE TABLE IF NOT EXISTS transform_undefined (a_test INTEGER)`
911
- await sql`INSERT INTO transform_undefined ${ sql([{ a_test: undefined }]) }`
912
- const data = await sql`SELECT a_test FROM transform_undefined`
913
-
914
- console.log(data) // [ { a_test: null } ]
915
- ```
916
-
917
- To combine with the built in transform functions, spread the transform in the `transform` object:
918
-
919
- ```js
920
- // Transform the column names to and from camel case
921
- const sql = postgres({
922
- transform: {
923
- ...postgres.camel,
924
- undefined: null
925
- }
926
- })
927
-
928
- await sql`CREATE TABLE IF NOT EXISTS transform_undefined (a_test INTEGER)`
929
- await sql`INSERT INTO transform_undefined ${ sql([{ aTest: undefined }]) }`
930
- const data = await sql`SELECT ${ sql('aTest') } FROM transform_undefined`
931
-
932
- console.log(data) // [ { aTest: null } ]
933
- ```
934
-
935
- ### Custom Transform Functions
936
-
937
- To specify your own transformation functions, you can use the `column`, `value` and `row` options inside of `transform`, each an object possibly including `to` and `from` keys:
938
-
939
- * `to`: The function to transform the outgoing query column name to, i.e `SELECT ${ sql('aName') }` to `SELECT a_name` when using `postgres.toCamel`.
940
- * `from`: The function to transform the incoming query result column name to, see example below.
941
-
942
- > Both parameters are optional, if not provided, the default transformation function will be used.
943
-
944
- ```js
945
- // Implement your own functions, look at postgres.toCamel, etc
946
- // as a reference:
947
- // https://github.com/porsager/postgres/blob/4241824ffd7aa94ffb482e54ca9f585d9d0a4eea/src/types.js#L310-L328
948
- function transformColumnToDatabase() { /* ... */ }
949
- function transformColumnFromDatabase() { /* ... */ }
950
-
951
- const sql = postgres({
952
- transform: {
953
- column: {
954
- to: transformColumnToDatabase,
955
- from: transformColumnFromDatabase,
956
- },
957
- value: { /* ... */ },
958
- row: { /* ... */ }
959
- }
960
- })
961
- ```
962
-
963
- ## Listen & notify
964
-
965
- When you call `.listen`, a dedicated connection will be created to ensure that you receive notifications instantly. This connection will be used for any further calls to `.listen`. The connection will automatically reconnect according to a backoff reconnection pattern to not overload the database server.
966
-
967
- ### Listen `await sql.listen(channel, onnotify, [onlisten]) -> { state }`
968
- `.listen` takes the channel name, a function to handle each notify, and an optional function to run every time listen is registered and ready (happens on initial connect and reconnects). It returns a promise which resolves once the `LISTEN` query to Postgres completes, or if there is already a listener active. If either function throws, the error is rethrown as an uncaught exception, and the other listeners and the returned promise are not affected.
969
-
970
- ```js
971
- await sql.listen('news', payload => {
972
- const json = JSON.parse(payload)
973
- console.log(json.this) // logs 'is'
974
- })
975
- ```
976
-
977
- The optional `onlisten` method is great to use for a very simply queue mechanism:
978
-
979
- ```js
980
- await sql.listen(
981
- 'jobs',
982
- (x) => run(JSON.parse(x)),
983
- ( ) => sql`select unfinished_jobs()`.forEach(run)
984
- )
985
-
986
- function run(job) {
987
- // And here you do the work you please
988
- }
989
- ```
990
- ### Notify `await sql.notify(channel, payload) -> Result[]`
991
- Notify can be done as usual in SQL, or by using the `sql.notify` method.
992
- ```js
993
- sql.notify('news', JSON.stringify({ no: 'this', is: 'news' }))
994
- ```
995
-
996
- ## Realtime subscribe
997
-
998
- Postgres.js implements the logical replication protocol of PostgreSQL to support subscription to real-time updates of `insert`, `update` and `delete` operations.
999
-
1000
- > **NOTE** To make this work you must [create the proper publications in your database](https://www.postgresql.org/docs/current/sql-createpublication.html), enable logical replication by setting `wal_level = logical` in `postgresql.conf` and connect using either a replication or superuser.
1001
-
1002
- ### Quick start
1003
-
1004
- #### Create a publication (eg. in migration)
1005
- ```sql
1006
- CREATE PUBLICATION alltables FOR ALL TABLES
1007
- ```
1008
-
1009
- #### Subscribe to updates
1010
- ```js
1011
- const sql = postgres({ publications: 'alltables' })
1012
-
1013
- const { unsubscribe } = await sql.subscribe(
1014
- 'insert:events',
1015
- (row, { command, relation, key, old }) => {
1016
- // Callback function for each row change
1017
- // tell about new event row over eg. websockets or do something else
1018
- },
1019
- () => {
1020
- // Callback on initial connect and potential reconnects
1021
- },
1022
- (error) => {
1023
- // Callback on connection errors and changes that could not be decoded
1024
- }
1025
- )
1026
- ```
1027
-
1028
- If any of these callbacks throws, the error is rethrown as an uncaught exception, and other subscribers and the replication stream are not affected.
1029
-
1030
- `onerror` is called with errors of the replication connection, after which the subscription reconnects on a new replication slot and calls `onsubscribe` again (these are also logged with `console.error`, and changes committed while it was disconnected are not delivered; if the reconnect itself fails, it is retried with backoff, its error is logged with `console.error` and passed to `onerror`, and it never ends the process), and with errors thrown while a change is decoded by a [transform](#data-transformation) or a custom type parser. Such a change is skipped, and the replication stream and later changes carry on. A decode error goes only to the subscriptions that would have received the change, matched by operation and table; a subscription filtered by key gets every skipped change of its table, as the key is read from the decoded row, even if the table has no primary key. If a column name transform throws, each change of that table is skipped and reported until the server describes the table again, for example after `ALTER TABLE`. If one of the matching subscriptions has no `onerror`, the error is rethrown once as an uncaught exception, which ends the process under Node's default handling. `onerror` stays registered across reconnects until `unsubscribe()`.
1031
-
1032
- ### Subscribe pattern
1033
-
1034
- You can subscribe to specific operations, tables, or even rows with primary keys.
1035
-
1036
- #### `operation` `:` `schema` `.` `table` `=` `primary_key`
1037
-
1038
- **`operation`** is one of ``` * | insert | update | delete ``` and defaults to `*`
1039
-
1040
- **`schema`** defaults to `public`
1041
-
1042
- **`table`** is a specific table name and defaults to `*`
1043
-
1044
- **`primary_key`** can be used to only subscribe to specific rows
1045
-
1046
- ### Examples
1047
-
1048
- ```js
1049
- sql.subscribe('*', () => /* everything */ )
1050
- sql.subscribe('insert', () => /* all inserts */ )
1051
- sql.subscribe('*:users', () => /* all operations on the public.users table */ )
1052
- sql.subscribe('delete:users', () => /* all deletes on the public.users table */ )
1053
- sql.subscribe('update:users=1', () => /* all updates on the users row with a primary key = 1 */ )
1054
- ```
1055
-
1056
- ## Numbers, bigint, numeric
1057
-
1058
- `Number` in javascript is only able to represent 2<sup>53</sup>-1 safely which means that types in PostgreSQLs like `bigint` and `numeric` won't fit into `Number`.
1059
-
1060
- Since Node.js v10.4 we can use [`BigInt`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/BigInt) to match the PostgreSQL type `bigint` which is returned for eg. `count(*)`. Unfortunately, it doesn't work with `JSON.stringify` out of the box, so Postgres.js will return it as a string.
1061
-
1062
- If you want to use `BigInt` you can add this custom type:
1063
-
1064
- ```js
1065
- const sql = postgres({
1066
- types: {
1067
- bigint: postgres.BigInt
1068
- }
1069
- })
1070
- ```
1071
-
1072
- There is currently no guaranteed way to handle `numeric` / `decimal` types in native Javascript. **These [and similar] types will be returned as a `string`**. The best way in this case is to use [custom types](#custom-types).
1073
-
1074
- ## Parameter types
1075
-
1076
- A `$1` placeholder has a type that PostgreSQL infers from the SQL around it (the column it is compared to or inserted into, a function argument, a cast). Postgres.js has to turn each JavaScript value into text for that type, and `parameter_types` chooses how.
1077
-
1078
- Some values carry a type Postgres.js can tell from the JavaScript value: `Date`, `Buffer` / `Uint8Array`, booleans, `bigint`, and anything wrapped in `sql.typed(value, oid)`, `sql.types.*`, `sql.json` or `sql.array` with elements. They are serialized for that type and sent in a single round trip, in every mode. Everything else (strings, numbers, plain objects, `sql.typed(value)` without an OID, an empty `sql.array([])`, arrays) has no such type. `null` is always SQL `NULL`.
1079
-
1080
- ### `parameter_types: 'server'` (default)
1081
-
1082
- For a value whose type Postgres.js cannot tell, it first asks PostgreSQL for the type of the placeholder, which costs an extra round trip on every query with `prepare: false` and the first time each connection runs that SQL with `prepare: true`, and then serializes the value for that type. This has consequences worth knowing:
1083
-
1084
- - a string sent to a `json` or `jsonb` placeholder is sent as JSON, so `'{"a":1}'` is stored as a JSON string, not an object
1085
- - a string or number sent to a `date`, `timestamp` or `timestamptz` placeholder goes through `new Date(x).toISOString()`, where a date-only string such as `'2020-01-02'` is read as UTC and a date-and-time string without an offset such as `'2020-01-02T10:00'` is read in the Node.js process time zone (`TZ`)
1086
- - anything other than `true` sent to a `boolean` placeholder is stored as `false`, including the strings `'true'` and `'t'` and the number `1`
1087
- - a string sent to a `bytea` placeholder is sent as hex of its UTF-8 bytes
1088
- - a custom type registered in `types` with a `from` list runs its `serialize` function
1089
-
1090
- ### `parameter_types: 'client'`
1091
-
1092
- The JavaScript value decides how it is serialized and PostgreSQL parses the text, so a query with values is one round trip:
1093
-
1094
- - a string is sent unchanged as text
1095
- - a number is sent as its JavaScript text (`'' + n`)
1096
- - a plain object (prototype `Object.prototype` or `null`) is sent as `JSON.stringify(x)`
1097
- - any other object (Decimal, Luxon, moment, URL, class instances) is sent as `'' + x`
1098
- - `null` is SQL `NULL`; `undefined` is handled as before
1099
- - a value with a type Postgres.js can tell from the JavaScript value is serialized for that type, as with `'server'`
1100
- - an array whose elements have no such type still makes Postgres.js ask PostgreSQL for the placeholder's type first, because only PostgreSQL knows whether the placeholder is an array, JSON or a custom type; that query takes the extra round trip
1101
-
1102
- What this changes compared to `'server'`:
1103
-
1104
- | Value | Type PostgreSQL infers for the placeholder | `'server'` | `'client'` |
1105
- | --- | --- | --- | --- |
1106
- | string | json, jsonb | sent as JSON: stores a JSON string | parsed as JSON; text that is not JSON is rejected |
1107
- | string | boolean | always `false` | PostgreSQL boolean input (`'t'`, `'yes'`, `'on'`, `'1'` are true); invalid text is rejected |
1108
- | string | date, timestamp, timestamptz | `new Date(s).toISOString()`: date-only strings are read as UTC, date-and-time strings without an offset in the Node.js process time zone (`TZ`) | parsed by PostgreSQL in the PostgreSQL session's `TimeZone` setting; `date` and `timestamp` ignore an explicit offset |
1109
- | string | bytea | hex of the UTF-8 bytes | bytea text input; differs only for strings containing backslashes |
1110
- | string, number | a custom type registered in `types` with a `from` list | the type's `serialize` | sent unchanged as text |
1111
- | number | json, jsonb | `JSON.stringify` (`NaN` and `Infinity` become `null`) | the number's text (`NaN` and `Infinity` are rejected) |
1112
- | number | boolean | `false` | `1` is true, `0` is false, anything else is rejected |
1113
- | number | date, timestamp, timestamptz | `new Date(ms)` | rejected, except `Infinity`; a 6- or 8-digit number is read as a `YYMMDD` or `YYYYMMDD` date |
1114
- | plain object | json, jsonb | `JSON.stringify` or your json `serialize` | `JSON.stringify` |
1115
- | plain object | text | `[object Object]` | JSON text |
1116
- | plain object | a custom type registered in `types` with a `from` list | the type's `serialize` | JSON text, normally rejected |
1117
- | class instance | json, jsonb | `JSON.stringify`, so `toJSON` applies | `'' + x` |
1118
- | class instance | date, timestamp, timestamptz | `new Date(x)` | `'' + x`, normally rejected |
1119
-
1120
- `time`, `timetz` and `interval` placeholders have no serializer in either mode: strings and numbers are sent unchanged as text.
1121
-
1122
- In practice:
1123
-
1124
- - pass `new Date(ms)` for epoch numbers, and set `connection: { timezone }` so date strings are read in the time zone you expect
1125
- - pass `sql.json(s)` for a JSON string value, and `sql.types.name(v)` or `sql.typed(v, oid)` for a custom type; a custom type's `serialize` also runs for a whole array bound to that type, never for a plain value
1126
- - `NaN`, `Infinity` and `-0` are sent as `NaN`, `Infinity` and `0`, and PostgreSQL's input rules for the placeholder's type decide
1127
-
1128
- Run `'warn'` first to see which of your queries are affected.
1129
-
1130
- **Planned for 5.0:** `'client'` becomes the default. Pass `parameter_types: 'server'` to keep the 4.x behaviour.
1131
-
1132
- ### `parameter_types: 'warn'`
1133
-
1134
- `'warn'` behaves exactly like `'server'`, and also reports every query in which a value would be sent differently by `'client'`. The report goes to `ontypemismatch`, or to `console.warn` when it is not set:
1135
-
1136
- ```js
1137
- const sql = postgres({
1138
- parameter_types: 'warn',
1139
- ontypemismatch: ({ query, parameters }) => {
1140
- // parameters: [{ position: 2, oid: 3802, typeof: 'string' }]
1141
- // position is the $n number, oid the type PostgreSQL inferred for the placeholder
1142
- }
1143
- })
1144
- ```
1145
-
1146
- With `debug: true` each entry also carries `serverText` and `clientText`, the two texts that were compared; `clientText` is the error when the `'client'` mode would throw. A query is reported on every execution that has a difference; sample in `ontypemismatch` if that is too much. `'warn'` serializes each untyped value twice and compares the strings. The option can also be set with `?parameter_types=warn` in the connection URL or `PGPARAMETER_TYPES`.
1147
-
1148
- `'warn'` compares the serialized text, not the stored value, so a report does not always mean the stored value would change.
1149
-
1150
- These reported cases store the same value under both modes:
1151
-
1152
- - `0`, `'0'`, `'false'`, `'no'` and `'off'` sent to a `boolean` placeholder (the texts differ, both store `false`)
1153
- - a date string with an explicit offset sent to a `timestamptz` placeholder, as long as it has no more than millisecond precision (`toISOString()` truncates microseconds, so a string with microseconds stores a different value under `'server'`)
1154
- - a string without backslashes sent to a `bytea` placeholder
1155
-
1156
- These reported cases store a different value, or are rejected, under `'client'`:
1157
-
1158
- - `1`, `'t'`, `'true'`, `'yes'`, `'on'` and `'1'` sent to a `boolean` placeholder (stored `false` today, `true` under `'client'`)
1159
- - a date string with an explicit offset sent to a `timestamp` or `date` placeholder (`'server'` converts to UTC first, `'client'` drops the offset)
1160
- - a string sent to a `json` or `jsonb` placeholder, a plain object sent to a `text` placeholder, and a number sent to a date, timestamp or boolean placeholder
1161
- - a string or number sent to a custom type registered in `types` with a `from` list, which skips its `serialize`
1162
-
1163
- Date strings without an explicit offset:
1164
-
1165
- | String | Placeholder | Stored the same under both modes when |
1166
- | --- | --- | --- |
1167
- | date and time, such as `'2020-01-02T10:00'` | `timestamptz` | the Node.js process time zone and the PostgreSQL session's `TimeZone` setting match |
1168
- | date only, such as `'2020-01-02'` | `timestamptz` | the PostgreSQL session's `TimeZone` setting is UTC |
1169
- | date and time | `timestamp` or `date` | the Node.js process runs in UTC (`'server'` stores the UTC wall time, `'client'` stores the text as written) |
1170
-
1171
- ## Result Array
1172
-
1173
- The `Result` Array returned from queries is a custom array allowing for easy destructuring or passing on directly to JSON.stringify or general Array usage. It includes the following properties.
1174
-
1175
- ### .count
1176
-
1177
- The `count` property is the number of affected rows returned by the database. This is useful for insert, update and delete operations to know the number of rows since .length will be 0 in these cases if not using `RETURNING ...`.
1178
-
1179
- ### .command
1180
-
1181
- The `command` run by the query - eg. one of `SELECT`, `UPDATE`, `INSERT`, `DELETE`
1182
-
1183
- ### .columns
1184
-
1185
- The `columns` returned by the query useful to determine types, or map to the result values when using `.values()`
1186
-
1187
- ```js
1188
- {
1189
- name : String, // Column name,
1190
- type : oid, // PostgreSQL oid column type
1191
- parser: Function // The function used by Postgres.js for parsing
1192
- }
1193
- ```
1194
-
1195
- ### .statement
1196
-
1197
- The `statement` contains information about the statement implicitly created by Postgres.js.
1198
-
1199
- ```js
1200
- {
1201
- name : String, // The auto generated statement name
1202
- string : String, // The actual query string executed
1203
- types : [oid], // An array of oid expected as input parameters
1204
- columns : [Column] // Array of columns - same as Result.columns
1205
- }
1206
- ```
1207
-
1208
- ### .state
1209
-
1210
- This is the state `{ pid, secret }` of the connection that executed the query.
1211
-
1212
- ## Connection details
1213
-
1214
- ### All Postgres options
1215
-
1216
- ```js
1217
- const sql = postgres('postgres://username:password@host:port/database', {
1218
- host : '', // Postgres ip address[es] or domain name[s]
1219
- port : 5432, // Postgres server port[s]
1220
- path : '', // unix socket path (usually '/tmp')
1221
- database : '', // Name of database to connect to
1222
- username : '', // Username of database user
1223
- password : '', // Password of database user
1224
- ssl : false, // true, prefer, require, tls.connect options
1225
- sslnegotiation : null, // direct
1226
- max : 10, // Max number of connections
1227
- max_lifetime : null, // Max lifetime in seconds (more info below)
1228
- idle_timeout : 0, // Idle connection timeout in seconds
1229
- connect_timeout : 30, // Connect timeout in seconds
1230
- prepare : true, // Automatic creation of prepared statements
1231
- types : [], // Array of custom types, see more below
1232
- onnotice : fn, // Default console.log, set false to silence NOTICE
1233
- onparameter : fn, // (key, value) when server param change
1234
- onidle : fn, // (connId) when a connection returns to the pool idle
1235
- debug : fn, // Is called with (connection, query, params, types)
1236
- socket : fn, // fn returning custom socket to use
1237
- transform : {
1238
- undefined : undefined, // Transforms undefined values (eg. to null)
1239
- column : fn, // Transforms incoming column names
1240
- value : fn, // Transforms incoming row values
1241
- row : fn // Transforms entire rows
1242
- },
1243
- connection : {
1244
- application_name : 'postgres.js', // Default application_name
1245
- ... // Other connection parameters, see https://www.postgresql.org/docs/current/runtime-config-client.html
1246
- },
1247
- target_session_attrs : null, // Use 'read-write' with multiple hosts to
1248
- // ensure only connecting to primary
1249
- fetch_types : true, // Automatically fetches types on connect
1250
- // on initial connection.
1251
- parameter_types : 'server', // 'client' or 'warn', see Parameter types (PGPARAMETER_TYPES)
1252
- ontypemismatch : fn, // Called by parameter_types: 'warn', default console.warn
1253
- })
1254
- ```
1255
-
1256
- The `onnotice`, `onparameter`, `ontypemismatch`, `onnotify`, `onclose`, `onidle` and `debug` callbacks passed in the options are called synchronously from inside the driver. If one throws, the error is rethrown as an uncaught exception and the connection and its queries are not affected; under Node's default handling that ends the process.
1257
-
1258
- Note that `max_lifetime = 60 * (30 + Math.random() * 30)` by default. This resolves to an interval between 30 and 60 minutes to optimize for the benefits of prepared statements **and** working nicely with Linux's OOM killer.
1259
-
1260
- ### Dynamic passwords
1261
-
1262
- When clients need to use alternative authentication schemes such as access tokens or connections to databases with rotating passwords, provide either a synchronous or asynchronous function that will resolve the dynamic password value at connection time.
1263
-
1264
- ```js
1265
- const sql = postgres(url, {
1266
- // Other connection config
1267
- ...
1268
- // Password function for the database user
1269
- password : async () => await signer.getAuthToken(),
1270
- })
1271
- ```
1272
-
1273
- ### SSL
1274
-
1275
- Although [vulnerable to MITM attacks](https://security.stackexchange.com/a/229297/174913), a common configuration for the `ssl` option for some cloud providers is to set `rejectUnauthorized` to `false` (if `NODE_ENV` is `production`):
1276
-
1277
- ```js
1278
- const sql =
1279
- process.env.NODE_ENV === 'production'
1280
- ? // "Unless you're using a Private or Shield Heroku Postgres database, Heroku Postgres does not currently support verifiable certificates"
1281
- // https://help.heroku.com/3DELT3RK/why-can-t-my-third-party-utility-connect-to-heroku-postgres-with-ssl
1282
- postgres({ ssl: { rejectUnauthorized: false } })
1283
- : postgres()
1284
- ```
1285
-
1286
- For more information regarding `ssl` with `postgres`, check out the [Node.js documentation for tls](https://nodejs.org/dist/latest-v16.x/docs/api/tls.html#new-tlstlssocketsocket-options).
1287
-
1288
-
1289
- ### Multi-host connections - High Availability (HA)
1290
-
1291
- Multiple connection strings can be passed to `postgres()` in the form of `postgres('postgres://localhost:5432,localhost:5433', ...)`. This works the same as native the `psql` command. Read more at [multiple host URIs](https://www.postgresql.org/docs/13/libpq-connect.html#LIBPQ-MULTIPLE-HOSTS).
1292
-
1293
- Connections will be attempted in order of the specified hosts/ports. On a successful connection, all retries will be reset. This ensures that hosts can come up and down seamlessly.
1294
-
1295
- If you specify `target_session_attrs: 'primary'` or `PGTARGETSESSIONATTRS=primary` Postgres.js will only connect to the primary host, allowing for zero downtime failovers.
1296
-
1297
- ### The Connection Pool
1298
-
1299
- Connections are created lazily once a query is created. This means that simply doing const `sql = postgres(...)` won't have any effect other than instantiating a new `sql` instance.
1300
-
1301
- > No connection will be made until a query is made.
1302
-
1303
- For example:
1304
-
1305
- ```js
1306
- const sql = postgres() // no connections are opened
1307
-
1308
- await sql`...` // one connection is now opened
1309
- await sql`...` // previous opened connection is reused
1310
-
1311
- // two connections are opened now
1312
- await Promise.all([
1313
- sql`...`,
1314
- sql`...`
1315
- ])
1316
- ```
1317
-
1318
- > When there are high amount of concurrent queries, `postgres` will open as many connections as needed up until `max` number of connections is reached. By default `max` is 10. This can be changed by setting `max` in the `postgres()` call. Example - `postgres('connectionURL', { max: 20 })`.
1319
-
1320
- This means that we get a much simpler story for error handling and reconnections. Queries will be sent over the wire immediately on the next available connection in the pool. Connections are automatically taken out of the pool if you start a transaction using `sql.begin()`, and automatically returned to the pool once your transaction is done.
1321
-
1322
- Any query which was already sent over the wire will be rejected if the connection is lost. It'll automatically defer to the error handling you have for that query, and since connections are lazy it'll automatically try to reconnect the next time a query is made. The benefit of this is no weird generic "onerror" handler that tries to get things back to normal, and also simpler application code since you don't have to handle errors out of context.
1323
-
1324
- There are no guarantees about queries executing in order unless using a transaction with `sql.begin()` or setting `max: 1`. Of course doing a series of queries, one awaiting the other will work as expected, but that's just due to the nature of js async/promise handling, so it's not necessary for this library to be concerned with ordering.
1325
-
1326
- Since this library automatically creates prepared statements, it also has a default max lifetime for connections to prevent memory bloat on the database itself. This is a random interval for each connection between 45 and 90 minutes. This allows multiple connections to independently come up and down without affecting the service.
1327
-
1328
- ### Pool stats
1329
-
1330
- `sql.stats()` returns a snapshot of the pool as a plain object, ready to log, serialise from a health endpoint or map to metrics gauges:
1331
-
1332
- ```js
1333
- sql.stats()
1334
- // { max: 10, totalCount: 7, idleCount: 2, waitingCount: 3,
1335
- // activeCount: 4, availableCount: 5,
1336
- // busyCount: 3, reservedCount: 1, connectingCount: 1, closingCount: 0,
1337
- // executingCount: 4, listenCount: 1, subscribeCount: 0 }
1338
- ```
1339
-
1340
- | Field | Meaning |
1341
- |---|---|
1342
- | `max` | The `max` option |
1343
- | `totalCount` | Connections that exist: connecting, idle or in use |
1344
- | `idleCount` | Connections with a live session that are not in use. A connection released while its reserved handle still has queries in flight is idle with `executingCount` above 0 |
1345
- | `waitingCount` | Issued queries and `reserve()` or `begin()` calls that are not running yet: those in the pool queue, those held by a connection that is still opening, those pipelined behind a connection's running query, and those a reserved handle or transaction has queued behind its own connection |
1346
- | `activeCount` | `busyCount + reservedCount`, connections in use |
1347
- | `availableCount` | `idleCount` plus the slots the pool can still open (`max - totalCount - closingCount`): connections a query can get without waiting behind other work |
1348
- | `busyCount` | Pool connections running queries, including connections at the pipeline limit |
1349
- | `reservedCount` | Connections held by `reserve()` or `begin()` |
1350
- | `connectingCount` | Connections opening a session |
1351
- | `closingCount` | Connections draining or closing. They are not in `totalCount`, but still hold a pool slot until closed |
1352
- | `executingCount` | Queries the server is running right now, at most one per connection, on pool and reserved connections alike. It includes the queries the driver runs while a connection starts, and a `COPY` for its whole duration |
1353
- | `listenCount` | Connections held by the internal `sql.listen()` pool (0 or 1) |
1354
- | `subscribeCount` | Connections held by the internal `sql.subscribe()` pool (0 or 1) |
1355
-
1356
- `totalCount` and `idleCount` have the same meaning as in node-postgres. `waitingCount` is node-postgres's `waitingCount` extended to queries pipelined on connections and queued in leases, and it includes queries sent with `sql` directly, as `pool.query()` does there. Once the pool is at `max`, Postgres.js pipelines new queries onto busy connections, so waiting work can sit on a connection rather than in the pool queue; `waitingCount` counts it too, so `executingCount + waitingCount` covers all issued work that has not finished. Every snapshot satisfies `totalCount === idleCount + busyCount + reservedCount + connectingCount` and `totalCount + closingCount <= max`, the remainder being slots the pool has not opened.
1357
-
1358
- `listenCount` and `subscribeCount` are counted outside `max`, and the other fields only describe the `max` pool connections. A reserved connection's `stats()` returns the same snapshot, but `sql` inside `sql.begin()` and savepoints has no `stats()`, so call it on the main `sql`.
1359
-
1360
- The pool is saturated when `waitingCount > 0 && availableCount === 0`: every connection is in use and more work is waiting. Saturation alone is not a reason to fail a health check, because a single pipelined query on a fully used pool, including a transaction pipelining its own statements, is enough to meet it. A health endpoint should compare `waitingCount` against a limit the application chooses:
1361
-
1362
- ```js
1363
- import http from 'node:http'
1364
-
1365
- const MAX_WAITING = 100
1366
-
1367
- http.createServer((req, res) => {
1368
- const stats = sql.stats()
1369
- const overloaded = stats.availableCount === 0 && stats.waitingCount > MAX_WAITING
1370
-
1371
- res.writeHead(overloaded ? 503 : 200, { 'content-type': 'application/json' })
1372
- res.end(JSON.stringify(stats))
1373
- }).listen(8080)
1374
- ```
1375
-
1376
- How much waiting work is acceptable depends on the query durations and the latency the application can tolerate, so the limit has no universal value.
1377
-
1378
- Change events and a running total of finished queries are planned together with the diagnostics events, so for now take snapshots on a timer or on request.
1379
-
1380
- ### Connection timeout
1381
-
1382
- By default, connections will not close until `.end()` is called. However, it may be useful to have them close automatically when:
1383
-
1384
- - re-instantiating multiple ` sql`` ` instances
1385
- - using Postgres.js in a Serverless environment (Lambda, etc.)
1386
- - using Postgres.js with a database service that automatically closes connections after some time (see [`ECONNRESET` issue](https://github.com/porsager/postgres/issues/179))
1387
-
1388
- This can be done using the `idle_timeout` or `max_lifetime` options. These configuration options specify the number of seconds to wait before automatically closing an idle connection and the maximum time a connection can exist, respectively. Both are also checked when an idle connection is handed out, so an idle connection that is past either limit (for example after a serverless instance was frozen) is closed instead of being used.
1389
-
1390
- For example, to close a connection that has either been idle for 20 seconds or existed for more than 30 minutes:
1391
-
1392
- ```js
1393
- const sql = postgres({
1394
- idle_timeout: 20,
1395
- max_lifetime: 60 * 30
1396
- })
1397
- ```
1398
-
1399
- ### Vercel Fluid Compute
1400
-
1401
- On Vercel's Fluid Compute an instance can be suspended while a connection sits idle and before `idle_timeout` has closed it, which leaves the connection open on the server. `attachDatabasePool` from `@vercel/functions` keeps the instance alive until idle connections have closed. Pass it the pool through `vercelPool`:
1402
-
1403
- ```js
1404
- import postgres from '@steadybrew/postgresjs'
1405
- import { vercelPool } from '@steadybrew/postgresjs/vercel'
1406
- import { attachDatabasePool } from '@vercel/functions'
1407
-
1408
- const sql = postgres({ idle_timeout: 5 })
1409
- attachDatabasePool(vercelPool(sql))
1410
- ```
1411
-
1412
- `vercelPool` requires `idle_timeout` to be a number of seconds. It hooks into the `onidle` option, which is called with the connection id each time a connection returns to the pool idle and its idle timer starts, and it keeps any `onidle` you set yourself. If `onidle` throws, the error is rethrown as an uncaught exception and the connection is not affected. A connection that was frozen past `idle_timeout` anyway is closed when it is next handed out (see [Connection timeout](#connection-timeout)).
1413
-
1414
- ### Cloudflare Workers support
1415
-
1416
- Postgres.js has built-in support for the [TCP socket API](https://developers.cloudflare.com/workers/runtime-apis/tcp-sockets/) in Cloudflare Workers, which is [on-track](https://github.com/wintercg/proposal-sockets-api) to be standardized and adopted in Node.js and other JavaScript runtimes, such as Deno.
1417
-
1418
- You can use Postgres.js directly in a Worker, or to benefit from connection pooling and query caching, via the [Hyperdrive](https://developers.cloudflare.com/hyperdrive/learning/connect-to-postgres/#driver-examples) service available to Workers by passing the Hyperdrive `connectionString` when creating a new `postgres` client as follows:
1419
-
1420
- ```ts
1421
- // Requires Postgres.js 3.4.0 or later
1422
- import postgres from '@steadybrew/postgresjs'
1423
-
1424
- interface Env {
1425
- HYPERDRIVE: Hyperdrive;
1426
- }
1427
-
1428
- export default async fetch(req: Request, env: Env, ctx: ExecutionContext) {
1429
- // The Postgres.js library accepts a connection string directly
1430
- const sql = postgres(env.HYPERDRIVE.connectionString)
1431
- const results = await sql`SELECT * FROM users LIMIT 10`
1432
- return Response.json(results)
1433
- }
1434
- ```
1435
-
1436
- In `wrangler.toml` you will need to enable the `nodejs_compat` compatibility flag to allow Postgres.js to operate in the Workers environment:
1437
-
1438
- ```toml
1439
- compatibility_flags = ["nodejs_compat"]
1440
- ```
1441
-
1442
- ### Auto fetching of array types
1443
-
1444
- Postgres.js automatically fetches array-type information when each connection initializes.
1445
-
1446
- If you have revoked access to `pg_catalog` this feature will no longer work and will need to be disabled.
1447
-
1448
- You can disable catalog discovery by setting `fetch_types` to `false`. A fixed set of 22 common built-in array types still supports parsing and serialization, including text, numeric, boolean, JSON, date, UUID, and byte arrays. This changes earlier `fetch_types: false` behavior, which returned built-in arrays as raw strings. Other built-in, user-defined, and extension arrays still require discovery or explicit type handlers.
1449
-
1450
- SQL `NULL` elements are returned as JavaScript `null`; a quoted text element `"NULL"` remains a string. Explicit array parsers and serializers are preserved independently. Catalog discovery with `fetch_types: true` remains per connection; this change does not cache catalog metadata.
1451
-
1452
- Column, value, and row transforms apply to application query results; initialization metadata retains its original shape.
1453
-
1454
- ### Environmental variables
1455
-
1456
- It is also possible to connect to the database without a connection string or any options. Postgres.js will fall back to the common environment variables used by `psql` as in the table below:
1457
-
1458
- ```js
1459
- const sql = postgres()
1460
- ```
1461
-
1462
- | Option | Environment Variables |
1463
- | ------------------ | ------------------------ |
1464
- | `host` | `PGHOST` |
1465
- | `port` | `PGPORT` |
1466
- | `database` | `PGDATABASE` |
1467
- | `username` | `PGUSERNAME` or `PGUSER` |
1468
- | `password` | `PGPASSWORD` |
1469
- | `application_name` | `PGAPPNAME` |
1470
- | `idle_timeout` | `PGIDLE_TIMEOUT` |
1471
- | `connect_timeout` | `PGCONNECT_TIMEOUT` |
1472
- | `parameter_types` | `PGPARAMETER_TYPES` |
1473
-
1474
- ### Prepared statements
1475
-
1476
- Prepared statements will automatically be created for any queries where it can be inferred that the query is static. This can be disabled by using the `prepare: false` option. For instance โ€” this is useful when [using PGBouncer in `transaction mode`](https://github.com/porsager/postgres/issues/93#issuecomment-656290493).
1477
-
1478
- With `prepare: false`, `parameter_types: 'client'` sends queries with values in one round trip, unless a value is an untyped array; see [Parameter types](#parameter-types).
1479
-
1480
- **update**: [since 1.21.0](https://www.pgbouncer.org/2023/10/pgbouncer-1-21-0)
1481
- PGBouncer supports protocol-level named prepared statements when [configured
1482
- properly](https://www.pgbouncer.org/config.html#max_prepared_statements)
1483
-
1484
- ## Custom Types
1485
-
1486
- You can add ergonomic support for custom types, or simply use `sql.typed(value, type)` inline, where type is the PostgreSQL `oid` for the type and the correctly serialized string. _(`oid` values for types can be found in the `pg_catalog.pg_type` table.)_
1487
-
1488
- Adding Query helpers is the cleanest approach which can be done like this:
1489
-
1490
- ```js
1491
- const sql = postgres({
1492
- types: {
1493
- rect: {
1494
- // The pg_types oid to pass to the db along with the serialized value.
1495
- to : 1337,
1496
-
1497
- // An array of pg_types oids to handle when parsing values coming from the db.
1498
- from : [1337],
1499
-
1500
- //Function that transform values before sending them to the db.
1501
- serialize : ({ x, y, width, height }) => [x, y, width, height],
1502
-
1503
- // Function that transforms values coming from the db.
1504
- parse : ([x, y, width, height]) => { x, y, width, height }
1505
- }
1506
- }
1507
- })
1508
-
1509
- // Now you can use sql.typed.rect() as specified above
1510
- const [custom] = await sql`
1511
- insert into rectangles (
1512
- name,
1513
- rect
1514
- ) values (
1515
- 'wat',
1516
- ${ sql.typed.rect({ x: 13, y: 37, width: 42, height: 80 }) }
1517
- )
1518
- returning *
1519
- `
1520
-
1521
- // custom = { name: 'wat', rect: { x: 13, y: 37, width: 42, height: 80 } }
1522
-
1523
- ```
1524
-
1525
- With `parameter_types: 'client'` a type's `serialize` runs for `sql.typed.name(v)` and for a whole array bound to that type, not for a plain value; see [Parameter types](#parameter-types).
1526
-
1527
- ### Custom socket
1528
-
1529
- Easily do in-process ssh tunneling to your database by providing a custom socket for Postgres.js to use. The function (optionally async) must return a socket-like duplex stream.
1530
-
1531
- Here's a sample using [ssh2](https://github.com/mscdex/ssh2)
1532
-
1533
- ```js
1534
- import ssh2 from 'ssh2'
1535
-
1536
- const sql = postgres({
1537
- ...options,
1538
- socket: ({ host: [host], port: [port] }) => new Promise((resolve, reject) => {
1539
- const ssh = new ssh2.Client()
1540
- ssh
1541
- .on('error', reject)
1542
- .on('ready', () =>
1543
- ssh.forwardOut('127.0.0.1', 12345, host, port,
1544
- (err, socket) => err ? reject(err) : resolve(socket)
1545
- )
1546
- )
1547
- .connect(sshOptions)
1548
- })
1549
- })
1550
- ```
1551
-
1552
- ## Teardown / Cleanup
1553
-
1554
- To ensure proper teardown and cleanup on server restarts use `await sql.end()` before `process.exit()`.
1555
-
1556
- Calling `sql.end()` will reject new queries and return a Promise which resolves when all queries are finished and the underlying connections are closed. If a `{ timeout }` option is provided any pending queries will be rejected once the timeout (in seconds) is reached and the connections will be destroyed.
1557
-
1558
- With explicit resource management the pool ends itself at the end of a scope:
1559
-
1560
- ```js
1561
- async function main() {
1562
- await using sql = postgres()
1563
- await sql`select 1`
1564
- }
1565
- ```
1566
-
1567
- This calls `sql.end()` when the scope exits, so it suits scripts and jobs, not a shared client at a module's top level. The syntax needs Node.js 24+ or TypeScript 5.2+.
1568
-
1569
- #### Sample shutdown using [Prexit](https://github.com/porsager/prexit)
1570
-
1571
- ```js
1572
- import prexit from 'prexit'
1573
-
1574
- prexit(async () => {
1575
- await sql.end({ timeout: 5 })
1576
- await new Promise(r => server.close(r))
1577
- })
1578
- ```
1579
-
1580
- ## Reserving connections
1581
-
1582
- ### `await sql.reserve()`
1583
-
1584
- The `reserve` method pulls out a connection from the pool, and returns a client that wraps the single connection. This can be used for running queries on an isolated connection.
1585
-
1586
- ```ts
1587
- const reserved = await sql.reserve()
1588
- await reserved`select * from users`
1589
- await reserved.release()
1590
- ```
1591
-
1592
- ### `reserved.release()`
1593
-
1594
- Once you have finished with the reserved connection, call `release` to add it back to the pool.
1595
-
1596
- ```js
1597
- using reserved = await sql.reserve()
1598
- await reserved`select * from users`
1599
- ```
1600
-
1601
- A `using` or `await using` declaration calls `release()` when the scope exits.
1602
-
1603
- After release, queries through that reserved handle reject with `CONNECTION_ENDED`. Call `sql.reserve()` again to acquire a new handle. A reserved handle whose connection has closed cannot be reused after reconnect.
1604
-
1605
- ## Error handling
1606
-
1607
- Errors are all thrown to related queries and never globally. Errors coming from database itself are always in the [native Postgres format](https://www.postgresql.org/docs/current/errcodes-appendix.html), and the same goes for any [Node.js errors](https://nodejs.org/api/errors.html#errors_common_system_errors) eg. coming from the underlying connection.
1608
-
1609
- Query errors will contain a stored error with the origin of the query to aid in tracing errors.
1610
-
1611
- Query errors will also contain the `query` string and the `parameters`. These are not enumerable to avoid accidentally leaking confidential information in logs. To log these it is required to specifically access `error.query` and `error.parameters`, or set `debug: true` in options.
1612
-
1613
- There are also the following errors specifically for this library.
1614
-
1615
- ##### UNSAFE_TRANSACTION
1616
- > Only use sql.begin or max: 1
1617
-
1618
- To ensure statements in a transaction runs on the same connection (which is required for them to run inside the transaction), you must use [`sql.begin(...)`](#transactions) or only allow a single connection in options (`max: 1`).
1619
-
1620
- ##### UNDEFINED_VALUE
1621
- > Undefined values are not allowed
1622
-
1623
- Postgres.js won't accept `undefined` as values in tagged template queries since it becomes ambiguous what to do with the value. If you want to set something to null, use `null` explicitly.
1624
-
1625
- ##### MESSAGE_NOT_SUPPORTED
1626
- > X (X) is not supported
1627
-
1628
- Whenever a message is received from Postgres which is not supported by this library. Feel free to file an issue if you think something is missing.
1629
-
1630
- ##### MAX_PARAMETERS_EXCEEDED
1631
- > Max number of parameters (65534) exceeded
1632
-
1633
- The postgres protocol doesn't allow more than 65534 (16bit) parameters. If you run into this issue there are various workarounds such as using `sql([...])` to escape values instead of passing them as parameters.
1634
-
1635
- ##### SASL_SIGNATURE_MISMATCH
1636
- > Message type X not supported
1637
-
1638
- When using SASL authentication the server responds with a signature at the end of the authentication flow which needs to match the one on the client. This is to avoid [man-in-the-middle attacks](https://en.wikipedia.org/wiki/Man-in-the-middle_attack). If you receive this error the connection was canceled because the server did not reply with the expected signature.
1639
-
1640
- ##### NOT_TAGGED_CALL
1641
- > Query not called as a tagged template literal
1642
-
1643
- Making queries has to be done using the sql function as a [tagged template](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_literals#Tagged_templates). This is to ensure parameters are serialized and passed to Postgres as query parameters with correct types and to avoid SQL injection.
1644
-
1645
- ##### SSL_NOT_SUPPORTED
1646
- > The server does not support SSL connections
1647
-
1648
- The server answered the SSL request with a refusal while `ssl` is set to `require` (or another mode that does not fall back to plain text). The attempt fails at once for a single host. Use `ssl: 'prefer'` to fall back to an unencrypted connection, or enable SSL on the server.
1649
-
1650
- ##### TARGET_SESSION_ATTRS
1651
- > No host matched target_session_attrs=read-write: localhost:5432 is read-only
1652
-
1653
- No host satisfied [`target_session_attrs`](#connection-details). This error is the `cause` of the `CONNECTION_CLOSED` error the query rejects with, so retry logic keyed on `CONNECTION_CLOSED` keeps working; read `err.cause.code` and `err.cause.message` for the reason. The message names the requested value and each host with the reason it was rejected. With several hosts, the connection fails at once when every host answered and none matched; if any host failed for another reason (refused, closed, timed out), the client keeps retrying until `connect_timeout` instead, since a failover may be in progress. With a single host, the client keeps retrying until `connect_timeout` before failing, because a single endpoint that is read-only is often in the middle of a failover, for example a cluster DNS name that still points at the old primary.
1654
-
1655
- ##### AUTH_TYPE_NOT_IMPLEMENTED
1656
- > Auth type X not implemented
1657
-
1658
- Postgres supports many different authentication types. This one is not supported.
1659
-
1660
- ##### CONNECTION_CLOSED
1661
- > write CONNECTION_CLOSED host:port
1662
-
1663
- This error is thrown if the connection was closed, or could not be opened, without an error of its own. When the server or the network ended the session, or no host matched `target_session_attrs`, the underlying error is available as `err.cause`.
1664
-
1665
- ##### CONNECTION_ENDED
1666
- > write CONNECTION_ENDED host:port
1667
-
1668
- This error is thrown if the user has called [`sql.end()`](#teardown--cleanup) and performed a query afterward.
1669
-
1670
- A reserved `sql` used after `release()`, or a transaction `sql` used after `begin()` has committed, rolled back or prepared, also rejects with `CONNECTION_ENDED`.
1671
-
1672
- ##### CONNECTION_DESTROYED
1673
- > write CONNECTION_DESTROYED host:port
1674
-
1675
- This error is thrown for any queries that were pending when the timeout to [`sql.end({ timeout: X })`](#teardown--cleanup) was reached.
1676
-
1677
- ##### CONNECT_TIMEOUT
1678
- > write CONNECT_TIMEOUT host:port
1679
-
1680
- The timeout applies to each connection attempt and covers socket creation, TCP and TLS negotiation, authentication, session checks and type discovery. When an attempt times out, the next host is tried. The acquisition as a whole is bounded by `connect_timeout` multiplied by the number of hosts, measured from the first attempt; time spent waiting on an inherited reconnect delay is not counted. Set it with `connect_timeout` or `PGCONNECT_TIMEOUT` (30 seconds by default); zero disables both limits. The same value also bounds a graceful close: if the server does not close the socket within `connect_timeout` seconds after `end()` sends Terminate, the connection is closed anyway; zero leaves a graceful close unbounded.
1681
-
1682
- If the only attempt times out, the error is `CONNECT_TIMEOUT`. Once earlier attempts have failed, exhausting the overall budget reports the earlier error, such as the last server error or `CONNECTION_CLOSED`. With a single host, a socket or socket factory error rejects the query once; with several hosts, errors keep cycling through the hosts with backoff until the budget is spent. Queries already sent to the server are not replayed.
1683
-
1684
- ##### COPY_IN_PROGRESS
1685
- > You cannot execute queries during copy
1686
-
1687
- This error is thrown for a statement on a `sql.reserve()` or `sql.begin()` handle while its copy operation (writable / readable) runs, including statements sent before the server started it. A query, `sql.reserve()` or `sql.begin()` through `sql` gets it when every connection is running a copy operation whose writable has not been ended; otherwise it waits for a connection.
1688
-
1689
- ##### TRANSACTION_ROLLED_BACK
1690
- > The server rolled the transaction back instead of committing it
1691
-
1692
- This error is thrown by `sql.begin()` when the server answers the final `COMMIT` (or `PREPARE TRANSACTION`) with `ROLLBACK`, because the transaction was aborted by a statement whose error the callback did not surface, such as a failing query that was not awaited. Nothing was committed. When such a query's error was already recorded, it is the error's `cause`.
1693
-
1694
- ## TypeScript support
1695
-
1696
- `postgres` has TypeScript support. You can pass a row list type for your queries in this way:
1697
- ```ts
1698
- interface User {
1699
- id: number
1700
- name: string
1701
- }
1702
-
1703
- const users = await sql<User[]>`SELECT * FROM users`
1704
- users[0].id // ok => number
1705
- users[1].name // ok => string
1706
- users[0].invalid // fails: `invalid` does not exists on `User`
1707
- ```
1708
-
1709
- However, be sure to check the array length to avoid accessing properties of `undefined` rows:
1710
- ```ts
1711
- const users = await sql<User[]>`SELECT * FROM users WHERE id = ${id}`
1712
- if (!users.length)
1713
- throw new Error('Not found')
1714
- return users[0]
1715
- ```
1716
-
1717
- You can also prefer destructuring when you only care about a fixed number of rows.
1718
- In this case, we recommend you to prefer using tuples to handle `undefined` properly:
1719
- ```ts
1720
- const [user]: [User?] = await sql`SELECT * FROM users WHERE id = ${id}`
1721
- if (!user) // => User | undefined
1722
- throw new Error('Not found')
1723
- return user // => User
1724
-
1725
- // NOTE:
1726
- const [first, second]: [User?] = await sql`SELECT * FROM users WHERE id = ${id}` // fails: `second` does not exist on `[User?]`
1727
- const [first, second] = await sql<[User?]>`SELECT * FROM users WHERE id = ${id}` // don't fail : `second: User | undefined`
1728
- ```
102
+ - **Transactions keep their own connection.** Fixes connection reuse that could commit another transaction's unfinished writes or let a disconnected handle read another user's rows through RLS. [#1204](https://github.com/porsager/postgres/issues/1204), [#1216](https://github.com/porsager/postgres/issues/1216)
103
+ - **Each query gets its own result.** Fixes pipelined queries receiving another query's rows or error, and results with empty slots after a failed query. [#1082](https://github.com/porsager/postgres/issues/1082), [#1252](https://github.com/porsager/postgres/issues/1252), [#1253](https://github.com/porsager/postgres/issues/1253)
104
+ - **Rollbacks aren't reported as successful commits.** `sql.begin()` rejects when the server rolls back the commit. [#1217](https://github.com/porsager/postgres/issues/1217)
105
+ - **Fewer hangs and crashes.** Fixes reservations and shutdown hanging after disconnects, queries stuck after a slow `COPY` reader, and closed transaction handles crashing the process. [#1195](https://github.com/porsager/postgres/issues/1195), [#1097](https://github.com/porsager/postgres/issues/1097), [#1016](https://github.com/porsager/postgres/pull/1016), [#1208](https://github.com/porsager/postgres/issues/1208)
106
+ - **Recover after outages.** Failed connections [wait before retrying](docs/api.md#connection-timeout), avoiding a burst of simultaneous retries. With [multiple hosts](docs/api.md#multi-host-connections---high-availability-ha), the client tries another server and can wait for a writable primary.
1729
107
 
1730
- We do our best to type all the public API, however types are not always updated when features are added or changed. Feel free to open an issue if you have trouble with types.
108
+ [50+ bug fixes and test coverage โ†’](docs/fixed-issues.md)
1731
109
 
1732
- ## Migration tools
110
+ ### Faster queries behind poolers
1733
111
 
1734
- Postgres.js doesn't come with any migration solution since it's way out of scope, but here are some modules that support Postgres.js for migrations:
112
+ With `prepare: false`, parameterized queries can pay for an extra network round trip. The opt-in [`parameter_types: 'client'`](docs/api.md#parameter-types) removes it for supported values. In one production app behind the Supabase pooler, median query time fell from **5.3 ms to 3.1 ms** over a day on each mode. Results depend on your workload; p99 was unchanged. Use `'warn'` first to check for serialization differences.
1735
113
 
1736
- - https://github.com/porsager/postgres-shift
1737
- - https://github.com/lukeed/ley
1738
- - https://github.com/JAForbes/pgmg
114
+ ### New features
1739
115
 
1740
- ## Thank you
116
+ - **See pool pressure.** [`sql.stats()`](docs/api.md#pool-stats) reports busy and idle connections and waiting queries.
117
+ - **Handle serverless suspension.** Expired idle connections are replaced on resume; the [Vercel Fluid Compute adapter](docs/api.md#vercel-fluid-compute) keeps instances alive until idle connections close.
118
+ - **Automatic cleanup.** [`await using`](docs/api.md#teardown--cleanup) closes a pool and `using` releases a reserved connection when its scope ends. Requires Node.js 24+ or TypeScript 5.2+.
119
+ - **Arrays without startup discovery.** With [`fetch_types: false`](docs/api.md#auto-fetching-of-array-types), common built-in arrays still parse and serialize without a catalog query.
1741
120
 
1742
- A really big thank you to [@JAForbes](https://twitter.com/jmsfbs) who introduced me to Postgres and still holds my hand navigating all the great opportunities we have.
121
+ ## Documentation
1743
122
 
1744
- Thanks to [@ACXgit](https://twitter.com/andreacoiutti) for initial tests and dogfooding.
123
+ - [API reference](docs/api.md) โ€” queries, transactions, types, options and errors. Also shipped in the npm package at `docs/api.md` for editors and code assistants.
124
+ - [Migration guide](docs/upgrading.md) โ€” switching, behaviour changes and tracing.
125
+ - [Changelog](CHANGELOG.md) ยท [Roadmap](ROADMAP.md) ยท [Contributing](CONTRIBUTING.md)
1745
126
 
1746
- Also thanks to [Ryan Dahl](https://github.com/ry) for letting me have the `postgres` npm package name.
127
+ Based on [Postgres.js](https://github.com/porsager/postgres) by [Rasmus Porsager](https://github.com/porsager). Thank you to its creator and everyone contributing fixes and reproductions.