tr-pg-name-value-store 1.0.0 → 2.0.0

package/CHANGELOG.md

@@ -4,6 +4,23 @@ All notable changes to this project are documented in this file. The format is
 based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this
 project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [2.0.0] - 2026-09-07
+
+No breaking changes; every existing method, guarantee and the frozen schema are
+untouched.
+
+### Added
+
+- `list(options?)`: pages through the store's names — with their values, or
+  without (`values: false`) for a cheap walk over large documents — in byte
+  order of the name (`COLLATE "C"`, identical on every database), with keyset
+  paging (`after`, `have_more`, `limit` up to 1000) and a literal `prefix`
+  filter in which `%`, `_` and `\` match themselves. Each page is one
+  lock-free statement over its own snapshot; no index is added.
+- `count(options?)`: the number of names, optionally under the same literal
+  prefix.
+- Exported types `ListOptions`, `ListEntry`, `ListResult` and `CountOptions`.
+
 ## [1.0.0] - 2026-09-07
 
 Initial release.
@@ -35,4 +52,5 @@ Initial release.
   compose file for PostgreSQL 16 is included). Verified on PostgreSQL 9.6, 16
   and 18.
 
+[2.0.0]: https://github.com/rinne/node-tr-pg-name-value-store/releases/tag/v2.0.0
 [1.0.0]: https://github.com/rinne/node-tr-pg-name-value-store/releases/tag/v1.0.0

package/README.md

@@ -10,6 +10,9 @@ returned as the corresponding JavaScript value.
   *verified, never altered*.
 - **Multiple named stores.** A namespace gives each store its own table in the
   same database.
+- **Listing.** `list()` pages through the names (with values, or without for a
+  cheap walk) in byte order with keyset paging, and `count()` counts them, both
+  optionally under a literal prefix — so nobody needs to query the table.
 - **Exact previous values.** Every write (`set`, `remove`, `update`) resolves to
   the value it replaced. Writes to the same name are serialized across all
   processes sharing the database, so that previous value is always the one left
@@ -73,7 +76,8 @@ A stored value is any JSON-serializable JavaScript value: `number`, `string`,
 A name is any non-empty string of at most **1024 bytes of UTF-8** (so up to 1024
 ASCII characters, fewer for non-ASCII) that does not contain `U+0000`. Anything
 else throws `TypeError` before any I/O. Within those bounds names are opaque:
-case-sensitive, whitespace-significant, any punctuation or Unicode.
+case-sensitive, whitespace-significant, any punctuation or Unicode. Listing
+orders them by byte order of their UTF-8 encoding (see [Listing](#listing)).
 
 ## API
 
@@ -144,6 +148,61 @@ Writes to other names, and reads of anything, are fine.
 
 Removes every pair from this store's namespace. Other namespaces are untouched.
 
+### Listing
+
+#### `list(options?): Promise<ListResult>`
+
+Lists one page of names, in ascending **byte order** of the name, with their
+values unless `values: false`.
+
+| Option   | Type      | Default | Meaning |
+|----------|-----------|---------|---------|
+| `prefix` | `string`  | none    | Only names starting with this exact string. Literal, not a pattern: `%`, `_` and `\` match themselves. Validated like a name (≤ 1024 bytes, no `U+0000`); empty means no filter. |
+| `after`  | `string`  | none    | Keyset cursor: only names strictly after this one in byte order — normally the last name of the previous page. Need not be an existing name. Validated like `prefix`; empty means start from the beginning. |
+| `limit`  | `number`  | 1000    | Page size, a positive integer. The maximum is 1000; larger values clamp silently (keep paging while `have_more` is true). A non-integer or a value below 1 throws `TypeError`. |
+| `values` | `boolean` | `true`  | With `false`, `value` is omitted from every entry and the value column is not read, so a store of large documents can be walked cheaply. |
+
+The result:
+
+| Field       | Type          | Meaning |
+|-------------|---------------|---------|
+| `entries`   | `ListEntry[]` | The page, each `{ name, value?, updated_at }`. `value` is present (and may be `null`) whenever `values` is on; a name with no value has no entry at all. `updated_at` is a `Date` (millisecond precision) of the last write. |
+| `have_more` | `boolean`     | `true` if more names followed this page in the same snapshot; continue with `after` set to the last entry's name. |
+
+Paging loop:
+
+```ts
+const names: string[] = [];
+let after: string | undefined;
+for (;;) {
+  const page = await store.list({ prefix: 'job:', after, values: false });
+  names.push(...page.entries.map((e) => e.name));
+  if (!page.have_more) break;
+  after = page.entries.at(-1)!.name; // a page with have_more is never empty
+}
+```
+
+Each page is one lock-free statement over its own snapshot. Names written
+between two pages may or may not appear, and a name removed in between is
+simply absent (so `have_more: true` followed by an empty page is possible).
+There are no offsets: a name that exists for the whole walk is returned exactly
+once.
+
+Ordering is byte order of the UTF-8 name (`COLLATE "C"`), identical on every
+database regardless of its locale — so `'B' < '_x' < 'a'`. It differs from
+JavaScript's default string comparison only for characters outside the Basic
+Multilingual Plane, which matters if you merge pages with locally sorted data.
+On databases whose default collation is `C` the primary-key index serves
+`after` and `prefix` directly; on other databases each page is a scan and sort
+of the store's names, which is fine for the sizes a per-service store reaches.
+No index is added — the schema stays frozen.
+
+#### `count(options?): Promise<number>`
+
+Resolves to the number of names in the store, or of those starting with
+`options.prefix` (same semantics as above). A single lock-free statement that
+scans the matching names.
+
 ## Concurrency
 
 Every write to a given name — `set`, `remove`, and `update` — runs in its own
@@ -155,10 +214,11 @@ the one before it. That is what makes the *previous value* contract exact and
 what lets `update` callbacks compose (including the create-from-absent case,
 which a row lock alone cannot cover).
 
-Writes to different names never block each other. `get` is a single lock-free
-statement. `removeAll` is a single bulk `DELETE` that does not take per-name
-locks; it waits for in-flight writes on individual rows like any other statement
-would.
+Writes to different names never block each other. Reads — `get`, `list` and
+`count` — are single lock-free statements that see the latest committed state
+and neither block writes nor are blocked by them. `removeAll` is a single bulk
+`DELETE` that does not take per-name locks; it waits for in-flight writes on
+individual rows like any other statement would.
 
 Advisory-lock keys are derived in Node (SHA-256 of the namespace and name) — no
 server-side extension is required.
@@ -180,7 +240,7 @@ same data and the same per-name serialization.
 
 | Error                 | When |
 |-----------------------|------|
-| `TypeError`           | Invalid namespace; invalid name (not a non-empty string, more than 1024 bytes of UTF-8, or containing `U+0000`); a `set` value or `update` return that is not JSON-serializable; an `update` callback that is not a function. |
+| `TypeError`           | Invalid namespace; invalid name (not a non-empty string, more than 1024 bytes of UTF-8, or containing `U+0000`); a `set` value or `update` return that is not JSON-serializable; an `update` callback that is not a function; invalid `list`/`count` options (a `prefix` or `after` that is not a string within the name bounds, a `limit` that is not a positive integer, a non-boolean `values`). |
 | `SchemaMismatchError` | A table with this namespace's name already exists with a different shape. It is left untouched. Exported by the package. |
 | re-thrown value       | `update` re-throws any non-`null`/`undefined` value its callback throws. |
 

package/dist/index.d.ts

@@ -1,2 +1,2 @@
-export { PgNameValueStore, type PgNameValueStoreOptions, type UpdateCallback, } from './pg-name-value-store';
+export { PgNameValueStore, type PgNameValueStoreOptions, type UpdateCallback, type ListOptions, type ListEntry, type ListResult, type CountOptions, } from './pg-name-value-store';
 export { SchemaMismatchError } from './schema';

package/dist/pg-name-value-store.d.ts

@@ -21,6 +21,61 @@ export interface PgNameValueStoreOptions {
  * - return a non-serializable value → cancel (no change), a `TypeError` is thrown.
  */
 export type UpdateCallback = (current: unknown) => unknown | Promise<unknown>;
+/** Options for {@link PgNameValueStore.list}. */
+export interface ListOptions {
+    /**
+     * Only names that start with this exact string. Literal, not a pattern: `%`,
+     * `_` and `\` in the prefix match themselves. Validated like a name (at most
+     * 1024 bytes of UTF-8, no `U+0000`). Empty or omitted: no filter.
+     */
+    prefix?: string;
+    /**
+     * Keyset cursor: only names strictly after this one in byte order — normally
+     * the `name` of the last entry of the previous page. Need not be an existing
+     * name. Validated like `prefix`. Empty or omitted: start from the first name.
+     */
+    after?: string;
+    /**
+     * Page size. A positive integer; default and maximum 1000 (larger values are
+     * clamped silently — keep paging while `have_more` is true). A non-integer or
+     * a value below 1 throws `TypeError` before any I/O.
+     */
+    limit?: number;
+    /**
+     * Whether to include values (default `true`). With `false` the `value` field
+     * is omitted from every entry and the value column is not read at all, so a
+     * store of large documents can be walked cheaply.
+     */
+    values?: boolean;
+}
+/** One entry of a {@link ListResult}. */
+export interface ListEntry {
+    /** The name. */
+    name: string;
+    /**
+     * The stored value — present (possibly `null`) whenever the page was listed
+     * with `values` on; omitted entirely with `values: false`. A name with no
+     * value has no entry at all, so this is never `undefined` when present.
+     */
+    value?: unknown;
+    /** When the value was last written (millisecond precision). */
+    updated_at: Date;
+}
+/** Result of {@link PgNameValueStore.list}. */
+export interface ListResult {
+    /** The page, in ascending byte order of `name`. */
+    entries: ListEntry[];
+    /**
+     * `true` if more names followed this page in the same snapshot; continue with
+     * `after: entries.at(-1).name`.
+     */
+    have_more: boolean;
+}
+/** Options for {@link PgNameValueStore.count}. */
+export interface CountOptions {
+    /** Same semantics as {@link ListOptions.prefix}. */
+    prefix?: string;
+}
 /**
  * A persistent name → value store backed by a single PostgreSQL table.
  *
@@ -39,8 +94,9 @@ export type UpdateCallback = (current: unknown) => unknown | Promise<unknown>;
  * process sharing the database) by a per-(namespace, name) advisory lock. The
  * *previous value* each of them resolves to is therefore exact: it is the value
  * left by the immediately preceding write. Writes to different names never
- * block each other. {@link get} is a single lock-free statement that reads the
- * latest committed value.
+ * block each other. Reads — {@link get}, {@link list}, {@link count} — are
+ * single lock-free statements that see the latest committed state and neither
+ * block writes nor are blocked by them.
  */
 export declare class PgNameValueStore {
     #private;
@@ -113,4 +169,25 @@ export declare class PgNameValueStore {
      * are untouched.
      */
     removeAll(): Promise<void>;
+    /**
+     * Lists one page of names (with their values unless `values: false`) in
+     * ascending **byte order** of `name` (`COLLATE "C"`, so the order is the same
+     * on every database regardless of its locale). See {@link ListOptions}.
+     *
+     * Paging is keyset paging: pass the last `name` of a page as `after` to get
+     * the next one, while `have_more` is `true`. Each page is one lock-free
+     * statement over its own snapshot, so names written between two pages may or
+     * may not appear, and a name removed in between is simply absent.
+     *
+     * The primary-key index serves this query directly on databases whose
+     * default collation is `C`; elsewhere a page is a scan and sort of the
+     * store's names, which is fine for the sizes a per-service store reaches.
+     */
+    list(options?: ListOptions): Promise<ListResult>;
+    /**
+     * Resolves to the number of names in the store, or of those starting with
+     * `prefix` (same semantics as {@link ListOptions.prefix}). A single lock-free
+     * `SELECT count(*)`, which scans the matching names.
+     */
+    count(options?: CountOptions): Promise<number>;
 }

package/dist/pg-name-value-store.js

@@ -8,18 +8,51 @@ const schema_1 = require("./schema");
 // violations fail fast with a TypeError instead of a server error. 1024 bytes is
 // 1024 ASCII characters.
 const NAME_MAX_BYTES = 1024;
-function validateName(name) {
-    if (typeof name !== 'string' || name.length === 0) {
-        throw new TypeError('name must be a non-empty string');
+// Default and maximum page size of `list`. Larger requests clamp silently;
+// `have_more` tells the caller to keep paging.
+const MAX_LIST = 1000;
+/**
+ * Validates a name-like string: `name` itself (non-empty) or a `prefix`/`after`
+ * listing argument (may be empty). Rejects non-strings, over-long strings (in
+ * UTF-8 bytes) and strings containing NUL, which PostgreSQL `text` cannot hold.
+ */
+function validateNameLike(value, what, allowEmpty) {
+    if (typeof value !== 'string' || (!allowEmpty && value.length === 0)) {
+        throw new TypeError(allowEmpty ? `${what} must be a string` : `${what} must be a non-empty string`);
     }
-    const bytes = Buffer.byteLength(name, 'utf8');
+    const bytes = Buffer.byteLength(value, 'utf8');
     if (bytes > NAME_MAX_BYTES) {
-        throw new TypeError(`name must be at most ${NAME_MAX_BYTES} bytes of UTF-8 (got ${bytes})`);
+        throw new TypeError(`${what} must be at most ${NAME_MAX_BYTES} bytes of UTF-8 (got ${bytes})`);
+    }
+    if (value.includes('\0')) {
+        throw new TypeError(`${what} must not contain U+0000`);
     }
-    if (name.includes('\0')) {
-        // PostgreSQL `text` cannot hold a NUL byte; reject it before any I/O.
-        throw new TypeError('name must not contain U+0000');
+}
+function validateName(name) {
+    validateNameLike(name, 'name', false);
+}
+/** Validates `ListOptions.limit`; returns the effective (clamped) page size. */
+function validateLimit(limit) {
+    if (limit === undefined)
+        return MAX_LIST;
+    if (typeof limit !== 'number' || !Number.isInteger(limit) || limit < 1) {
+        throw new TypeError('limit must be a positive integer');
     }
+    return Math.min(limit, MAX_LIST);
+}
+function validateOptions(options, what) {
+    if (options === null || typeof options !== 'object') {
+        throw new TypeError(`${what} options must be an object`);
+    }
+}
+/**
+ * Turns a literal prefix into a `LIKE` pattern: the pattern metacharacters
+ * `\`, `%` and `_` are escaped with PostgreSQL's default escape character
+ * (backslash), then `%` is appended. Bound as a parameter, so no SQL escaping
+ * is involved.
+ */
+function likePrefix(prefix) {
+    return prefix.replace(/[\\%_]/g, '\\$&') + '%';
 }
 /**
  * Serializes a value to the JSON text bound (with an explicit `::jsonb` cast)
@@ -58,8 +91,9 @@ function serialize(value) {
  * process sharing the database) by a per-(namespace, name) advisory lock. The
  * *previous value* each of them resolves to is therefore exact: it is the value
  * left by the immediately preceding write. Writes to different names never
- * block each other. {@link get} is a single lock-free statement that reads the
- * latest committed value.
+ * block each other. Reads — {@link get}, {@link list}, {@link count} — are
+ * single lock-free statements that see the latest committed state and neither
+ * block writes nor are blocked by them.
  */
 class PgNameValueStore {
     #pool;
@@ -208,6 +242,68 @@ class PgNameValueStore {
         await this.init();
         await this.#pool.query(`DELETE FROM ${this.#table}`);
     }
+    /**
+     * Lists one page of names (with their values unless `values: false`) in
+     * ascending **byte order** of `name` (`COLLATE "C"`, so the order is the same
+     * on every database regardless of its locale). See {@link ListOptions}.
+     *
+     * Paging is keyset paging: pass the last `name` of a page as `after` to get
+     * the next one, while `have_more` is `true`. Each page is one lock-free
+     * statement over its own snapshot, so names written between two pages may or
+     * may not appear, and a name removed in between is simply absent.
+     *
+     * The primary-key index serves this query directly on databases whose
+     * default collation is `C`; elsewhere a page is a scan and sort of the
+     * store's names, which is fine for the sizes a per-service store reaches.
+     */
+    async list(options = {}) {
+        validateOptions(options, 'list');
+        const { prefix = '', after = '', values = true } = options;
+        validateNameLike(prefix, 'prefix', true);
+        validateNameLike(after, 'after', true);
+        const limit = validateLimit(options.limit);
+        if (typeof values !== 'boolean') {
+            throw new TypeError('values must be a boolean');
+        }
+        await this.init();
+        const conds = [];
+        const params = [];
+        if (prefix !== '') {
+            params.push(likePrefix(prefix));
+            conds.push(`name LIKE $${params.length}`);
+        }
+        if (after !== '') {
+            params.push(after);
+            conds.push(`name COLLATE "C" > $${params.length}`);
+        }
+        params.push(limit + 1); // one extra row tells us whether more follow
+        const where = conds.length > 0 ? `WHERE ${conds.join(' AND ')}` : '';
+        const res = await this.#pool.query(`SELECT name${values ? ', v' : ''}, updated_at FROM ${this.#table}
+       ${where}
+       ORDER BY name COLLATE "C"
+       LIMIT $${params.length}`, params);
+        const have_more = res.rows.length > limit;
+        const rows = have_more ? res.rows.slice(0, limit) : res.rows;
+        const entries = values
+            ? rows.map((r) => ({ name: r.name, value: r.v, updated_at: r.updated_at }))
+            : rows.map((r) => ({ name: r.name, updated_at: r.updated_at }));
+        return { entries, have_more };
+    }
+    /**
+     * Resolves to the number of names in the store, or of those starting with
+     * `prefix` (same semantics as {@link ListOptions.prefix}). A single lock-free
+     * `SELECT count(*)`, which scans the matching names.
+     */
+    async count(options = {}) {
+        validateOptions(options, 'count');
+        const { prefix = '' } = options;
+        validateNameLike(prefix, 'prefix', true);
+        await this.init();
+        const res = prefix === ''
+            ? await this.#pool.query(`SELECT count(*)::int AS n FROM ${this.#table}`)
+            : await this.#pool.query(`SELECT count(*)::int AS n FROM ${this.#table} WHERE name LIKE $1`, [likePrefix(prefix)]);
+        return res.rows[0].n;
+    }
     /**
      * Runs `fn` inside a transaction on a dedicated pooled connection while
      * holding the per-(namespace, name) transaction-scoped advisory lock, which

package/package.json

@@ -1,6 +1,6 @@
 {
     "name": "tr-pg-name-value-store",
-    "version": "1.0.0",
+    "version": "2.0.0",
     "description": "Persistent name/value store on PostgreSQL. JSONB values, self-maintaining never-migrated schema, atomic read-modify-write, multiple named stores per database.",
     "keywords": [
         "name-value",