zerobucket 0.1.0 → 0.4.0

package/README.md

@@ -10,18 +10,20 @@ This is the Node.js/TypeScript sibling of the [Python package](https://pypi.org/
 **It uses the exact same table**, so a Node service and a Python service can share one database
 and read each other's images.
 
-> **Status: 0.1.0.** PostgreSQL, classic mode. Dedup, object-storage tiering, SQLite and MySQL
-> are on the roadmap (see [What's not here yet](#whats-not-here-yet)).
+> **Status: early (0.4.x).** PostgreSQL **and SQLite**, each in classic and dedup modes, with S3-compatible
+> object-storage tiering. MySQL, a CLI and HEIC are on the roadmap (see [What's not here yet](#whats-not-here-yet)).
 
 ## Install
 
 ```bash
-npm install zerobucket pg sharp
+npm install zerobucket pg sharp          # PostgreSQL
+npm install zerobucket better-sqlite3 sharp   # or SQLite (see below): no database server at all
 ```
 
-`pg` (the Postgres driver) and `sharp` (image decoding/optimizing) are peer dependencies, so you
+`pg` (the Postgres driver), `better-sqlite3` (SQLite) and `sharp` (image decoding/optimizing) are peer dependencies, so you
 control their versions. If you only store PDFs through a custom validator, `sharp` is not needed.
-Requires Node 20+. Ships ESM and CommonJS with full types.
+For [object-storage tiering](#object-storage-tiering) also install `@aws-sdk/client-s3` (optional; loaded only when used).
+Requires Node 20+ (SQLite needs Node 22+, see below). Ships ESM and CommonJS with full types.
 
 ## Quick start
 
@@ -151,6 +153,10 @@ try {
 }
 ```
 
+Multi-statement operations (`putMany`, dedup `put`/`delete`, tiering) are **atomic even on a bare `pg.Client`
+with no open transaction**: ZeroBucket opens its own, or uses a `SAVEPOINT` inside yours so a failure rolls back only
+its own work and leaves your transaction usable.
+
 ## Batches
 
 `putMany` / `getMany` / `deleteMany` are best-effort per item (one bad file doesn't abort the rest):
@@ -183,6 +189,109 @@ output. Implement `ContentValidator` (`validate(data, { maxBytes })`) to store o
   operation (and once per chunk when streaming). Exceptions in your callback are swallowed.
 - The schema migration is serialized with an advisory lock, so many processes starting at once is safe.
 
+## SQLite
+
+No database server at all: one file. Ideal for small apps, single-server deployments, desktop/Electron apps, tests and prototypes.
+
+```ts
+const images = new ZeroBucket({ sqlite: "./images.db" }); // npm install better-sqlite3
+// ZeroBucket({ sqlite: ":memory:" }) for a throwaway in-process database
+```
+
+Everything in this README works on SQLite, **including dedup (`dedup: true`), object-storage tiering, streaming and
+byte ranges, hooks, the HTTP handler, optimize and validators**. The file format is identical to the Python
+package's `SQLiteBackend`, so a Node app and a Python app can share one `.db` file (verified in both directions, including
+tiering and dedup reference counts, by the conformance harness).
+
+How it behaves in Node, and why:
+
+- **Synchronous driver, event loop kept free.** `better-sqlite3` is synchronous (a local file read is microseconds). Lock
+  contention is the exception, and ZeroBucket never lets the driver sleep on a lock, which would freeze your whole server.
+  Each attempt waits ~25 ms; a `SQLITE_BUSY` is retried with async sleeps up to `busyTimeoutMs` (default 10 s, set via
+  `new ZeroBucket({ backend: new SQLiteBackend({ path, busyTimeoutMs }) })`), then fails with a clear `StorageError`.
+  A test holds the write lock from a second process and asserts the event loop keeps ticking while ZeroBucket waits.
+- **One connection, no interleaving.** Each operation is a single synchronous `BEGIN IMMEDIATE ... COMMIT`, so concurrent
+  requests can never run inside each other's transaction. WAL mode is set once, and only if the file is not already WAL.
+- **Many processes, one new file** is safe (tested with 8 processes opening a fresh database simultaneously).
+- **Tiering locks the whole database** for the duration of the upload (SQLite has no row locks): other processes' writes
+  wait, this process's writes queue, reads are never blocked. It uses its own connection, so `tierToObjectStorage` does not
+  accept `connection`, and it needs a real file (not `:memory:`). The safety guarantee is unchanged: a failed upload
+  rolls back and leaves the row untouched.
+- **`connection`** takes your own `better-sqlite3` `Database`. Multi-statement operations use a `SAVEPOINT`, so they
+  are atomic whether or not you have an open transaction.
+- **Streaming** uses ranged `substr()` reads, so a delete mid-stream raises `StorageError` (like Postgres). SQLite still
+  reads the whole blob per chunk, in native memory: this bounds Node-side memory only.
+- **Node 22+ for SQLite.** `better-sqlite3` 13 requires Node 22 or newer and crashes on older versions. It ships its Windows,
+  macOS and Linux binaries inside the npm package, so no compiler or Visual Studio is needed. Postgres users can stay on Node 20.
+  (`better-sqlite3` 12 declares Node 20 support, but it is untested here.)
+- Not available on SQLite: `migrateClassicToDedup()` (Python has it for Postgres only, too).
+
+## Deduplication
+
+Store identical bytes once. In dedup mode every `put()` still returns its own id and keeps its own filename,
+but images with the same content share a single stored blob with a reference count (think: the same logo or default
+avatar uploaded a thousand times).
+
+```ts
+const images = new ZeroBucket({ connectionString, dedup: true });
+const a = await images.put(logo, { filename: "a.png" });
+const b = await images.put(logo, { filename: "b.png" }); // different id, same blob: ref_count = 2
+await images.delete(a); // blob stays: ref_count = 1
+await images.delete(b); // last reference gone: blob removed
+```
+
+- Uses separate tables (`zerobucket_blobs` + `zerobucket_image_refs`, identical to the Python package), so dedup and
+  classic data can never be confused. A Node and a Python dedup app share one database and one set of blobs.
+- Reference counts are exact under concurrency: 20 simultaneous `put()`s of the same bytes give `ref_count == 20`, and a
+  `put()` racing the delete of the last reference never loses or leaks the blob (both are tested).
+- Batches (`putMany`, `deleteMany`) take their blob locks in sorted checksum order, so overlapping batches in
+  opposite orders cannot deadlock each other.
+- Everything else works unchanged: hooks, streaming, ranges, the HTTP handler, optimize, validators, metrics, retry.
+- Dedup **cannot be combined with `objectStorage`** (a shared blob has many owners), same rule as Python.
+
+**Moving existing data in:** `await images.migrateClassicToDedup()` copies every classic image into the dedup tables,
+keeping each id, and never modifies the classic table. It runs in small batches (bounded memory), is safe to re-run
+(already-migrated ids are skipped and counted), and refuses up front if any classic image is tiered to object storage.
+
+## Object-storage tiering
+
+For the minority of apps whose image volume outgrows what is economical to keep in a database
+(backups, replication lag, per-GB price), move individual images to any S3-compatible store (AWS S3, MinIO,
+Cloudflare R2, Backblaze B2, DigitalOcean Spaces). It is **explicit and opt-in**: `put()` never tiers
+anything by itself, and nothing changes for you until you configure it.
+
+```ts
+import { ZeroBucket, ObjectStorage } from "zerobucket";
+
+const images = new ZeroBucket({
+  connectionString: process.env.DATABASE_URL,
+  objectStorage: new ObjectStorage({ bucket: "my-images" }), // bucket must already exist
+  // endpoint: "https://<account>.r2.cloudflarestorage.com", region: "auto"   // for non-AWS stores
+});
+
+await images.tierToObjectStorage(id); // true = moved now, false = already tiered (safe to re-run a backfill)
+await images.get(id); // unchanged: get / getMany / getStream / HTTP handler all read transparently
+await images.delete(id); // removes the row, then the object
+```
+
+- **Safe by construction.** The upload runs _inside_ the database transaction that locks the row. If it fails,
+  the transaction rolls back and the row is left byte-for-byte untouched: there is never a moment when the bytes
+  exist nowhere, or when a row claims to be tiered without a finished upload.
+- **Real range requests.** Tiered images stream with genuine S3 `Range` GETs, so a `bytes=1000-1999` request fetches
+  exactly those bytes (strictly better than the Postgres `substring()` approach).
+- **Locking tradeoff.** The row lock is held for the whole upload, so concurrent _writes to that one image_ wait. Other
+  rows and plain reads are never blocked. Treat tiering as a maintenance operation, not a request-path call.
+- **Integrity checks.** A tiered object whose size no longer matches the database, or a missing/short object, raises
+  `StorageError`. It never returns wrong data and never silently truncates a stream.
+- **Same keys as Python.** Objects are named by the image's UUID with no prefix, so the Node and Python packages
+  can read, tier and delete each other's objects in one bucket (verified by the conformance harness).
+- **Deleting inside your own transaction** (`delete(id, { connection })`) deliberately does **not** remove the
+  object: we cannot know whether your transaction will commit, and deleting early would turn a rollback into data
+  loss. After you commit, remove it yourself with `objectStorage.delete(id)`.
+- Failures to remove an object after a normal delete never fail the delete (the row is already gone); they are
+  reported through `onOperation` as `object_storage_delete` events.
+- Credentials come from the AWS SDK's standard chain (env vars, shared config, IAM role) or `credentials: {...}`.
+
 ## Shared database with the Python package
 
 Same table (`zerobucket_images`), same checksum (SHA-256 hex of the stored bytes), same tiering
@@ -192,27 +301,32 @@ same corpus of valid and corrupt files identically.
 
 ## Differences from the Python package
 
-|                        | Python                             | npm                                                                                                |
-| ---------------------- | ---------------------------------- | -------------------------------------------------------------------------------------------------- |
-| API                    | sync + async clients               | one async client                                                                                   |
-| Malformed id (`"abc"`) | `StorageError` from Postgres       | treated as not found (`ImageNotFoundError`, no DB round trip), friendlier for `/images/:id` routes |
-| `optimize`             | Pillow; EXIF stripped, not applied | sharp; EXIF **applied then stripped**; transparency → **white** for JPEG                           |
-| `putMany` inserts      | pipelined                          | one transaction, statements run sequentially                                                       |
-| Formats                | JPEG, PNG, WebP, HEIC (extra)      | JPEG, PNG, WebP (HEIC not yet)                                                                     |
-| Event duration         | `duration_seconds`                 | `durationMs`                                                                                       |
-| Options                | snake_case kwargs                  | camelCase options object                                                                           |
+|                                                | Python                                                                     | npm                                                                                                |
+| ---------------------------------------------- | -------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- |
+| API                                            | sync + async clients                                                       | one async client                                                                                   |
+| Malformed id (`"abc"`)                         | `StorageError` from Postgres                                               | treated as not found (`ImageNotFoundError`, no DB round trip), friendlier for `/images/:id` routes |
+| `optimize`                                     | Pillow; EXIF stripped, not applied                                         | sharp; EXIF **applied then stripped**; transparency → **white** for JPEG                           |
+| `putMany` inserts                              | pipelined                                                                  | one transaction, statements run sequentially                                                       |
+| Formats                                        | JPEG, PNG, WebP, HEIC (extra)                                              | JPEG, PNG, WebP (HEIC not yet)                                                                     |
+| Event duration                                 | `duration_seconds`                                                         | `durationMs`                                                                                       |
+| `migrateClassicToDedup`                        | loads the whole table into memory; fails if re-run or if any row is tiered | batched, safe to re-run, refuses tiered rows with a clear message                                  |
+| Dedup `putMany` / `deleteMany` locking         | arbitrary order (overlapping batches can deadlock)                         | sorted checksum order (cannot deadlock)                                                            |
+| SQLite API                                     | synchronous `SQLiteBackend` + separate async backend                       | one async client; the sync driver runs on the event loop, lock waits are async (never block it)    |
+| SQLite `onOperation` / busy retry              | none                                                                       | `onOperation` events and async `SQLITE_BUSY` retry (`busyTimeoutMs`)                               |
+| SQLite stream deleted mid-read                 | sync adapter keeps reading a WAL snapshot (async adapter raises)           | raises `StorageError`, like Postgres; use `connection` in a transaction for a snapshot             |
+| `connection=` without an open transaction      | multi-statement ops rely on the driver's implicit transaction              | always atomic (own transaction or savepoint)                                                       |
+| `delete(id, connection=...)` on a tiered image | deletes the S3 object immediately                                          | leaves the object until you delete it after commit (a rolled-back transaction can never lose data) |
+| S3 delete failure after `delete()`             | raises after the row is gone                                               | never fails the delete; reported via `onOperation`                                                 |
+| Options                                        | snake_case kwargs                                                          | camelCase options object                                                                           |
 
 ## What's not here yet
 
-Planned, in order: object-storage **tiering** (S3-compatible), **dedup** mode, **SQLite**, **MySQL/MariaDB**.
-Until tiering lands: reading a row that the Python package tiered to S3 throws a clear `StorageError`
-(never wrong data), and **deleting** such a row from Node removes the database row but leaves the S3
-object behind (the same documented orphan behaviour as Python without `object_storage`).
-Delete tiered images from Python if you use tiering.
+Planned, in order: **MySQL/MariaDB**, the `zerobucket` **CLI**, and **HEIC**.
+Tiering and dedup cannot be combined (same rule as the Python package).
 
 ## Runtime notes
 
-Node 20+. Bun/Deno: expected to work, not yet tested. Edge runtimes (Cloudflare Workers, Vercel Edge) cannot open raw
+Node 20+ for Postgres, Node 22+ for SQLite. Bun/Deno: expected to work, not yet tested. Edge runtimes (Cloudflare Workers, Vercel Edge) cannot open raw
 Postgres TCP connections through `pg` without a proxy such as Hyperdrive; use the Node runtime for these routes.
 In serverless, keep `poolMaxSize` small (1-2) or inject a pool shared across invocations.
 

package/dist/{client-C_W0t8My.d.cts → client-WZ4VRCPi.d.cts}

@@ -63,6 +63,22 @@ interface Queryable {
         rowCount: number | null;
     }>;
 }
+/**
+ * The slice of a `better-sqlite3` Database that ZeroBucket's SQLite backend uses. Pass your own
+ * Database handle as `connection` to make a call part of YOUR SQLite transaction.
+ */
+interface SqliteDatabaseLike {
+    prepare(sql: string): {
+        all(...params: unknown[]): any[];
+        get(...params: unknown[]): any;
+        run(...params: unknown[]): {
+            changes: number;
+        };
+    };
+    exec(sql: string): unknown;
+}
+/** What `connection` accepts: a `pg` client for the Postgres backend, a `better-sqlite3` Database for SQLite. */
+type Connection = Queryable | SqliteDatabaseLike;
 /** The row shape a storage backend persists (after validation and checksumming). */
 interface PreparedRow {
     data: Buffer;
@@ -99,7 +115,7 @@ interface StoredRecord {
 }
 type StoredRecordMetadata = Omit<StoredRecord, "data">;
 interface CallOptions {
-    connection?: Queryable;
+    connection?: Connection;
 }
 interface StreamOptions$1 extends CallOptions {
     chunkSize: number;
@@ -128,9 +144,65 @@ interface StorageBackend {
     /** Returns the ids that were actually deleted. */
     deleteMany(ids: string[], opts?: CallOptions): Promise<string[]>;
     exists(id: string, opts?: CallOptions): Promise<boolean>;
+    /** null = no such image, false = already tiered (no-op), true = tiered now. Needs object storage configured. */
+    tierToObjectStorage(id: string, opts?: CallOptions): Promise<boolean | null>;
+    /** Dedup-mode instances only: copy the classic table into the dedup tables (non-destructive). */
+    migrateClassicToDedup?(): Promise<{
+        imagesMigrated: number;
+        distinctBlobsCreated: number;
+        duplicateReferencesFound: number;
+        alreadyMigrated: number;
+    }>;
     close(): Promise<void>;
 }
 
+/** The surface the Postgres adapter needs. Implement this to plug in a non-S3 store (or a test double). */
+interface ObjectStorageLike {
+    readonly bucket: string;
+    upload(key: string, data: Buffer, opts: {
+        mimeType: string;
+    }): Promise<void>;
+    download(key: string): Promise<Buffer>;
+    /** Inclusive byte range, fetched with a real HTTP Range request. */
+    downloadRange(key: string, start: number, endInclusive: number): Promise<Buffer>;
+    /** Idempotent: deleting a missing key is not an error. */
+    delete(key: string): Promise<void>;
+    exists(key: string): Promise<boolean>;
+}
+interface ObjectStorageOptions {
+    /** Must already exist. */
+    bucket: string;
+    /** Set for anything that is not AWS S3 itself (MinIO, R2, B2, Spaces, ...). */
+    endpoint?: string;
+    /** Default: AWS_REGION / AWS_DEFAULT_REGION, else "us-east-1". Many S3-compatible services ignore it but the SDK requires one. */
+    region?: string;
+    /** Default: the SDK's standard chain (env vars, shared config, IAM role, ...). */
+    credentials?: {
+        accessKeyId: string;
+        secretAccessKey: string;
+        sessionToken?: string;
+    };
+    /** Path-style addressing (`endpoint/bucket/key`). Default: true when `endpoint` is set, else false. */
+    forcePathStyle?: boolean;
+    /** Bring your own configured S3Client (advanced). */
+    client?: unknown;
+}
+declare class ObjectStorage implements ObjectStorageLike {
+    private readonly opts;
+    readonly bucket: string;
+    private clientPromise;
+    constructor(opts: ObjectStorageOptions);
+    private connection;
+    upload(key: string, data: Buffer, { mimeType }: {
+        mimeType: string;
+    }): Promise<void>;
+    download(key: string): Promise<Buffer>;
+    downloadRange(key: string, start: number, endInclusive: number): Promise<Buffer>;
+    private get;
+    delete(key: string): Promise<void>;
+    exists(key: string): Promise<boolean>;
+}
+
 interface OptimizeOptions {
     /** Downscale (aspect ratio preserved, Lanczos) if wider than this. Never upscales. */
     maxWidth?: number;
@@ -188,7 +260,13 @@ interface ZeroBucketOptions<C = unknown> {
     connectionString?: string;
     /** Bring your own pg.Pool. It is NOT closed by close(). */
     pool?: Pool;
-    /** Advanced: inject a custom StorageBackend instead of building a PostgresBackend. */
+    /**
+     * Use SQLite instead of PostgreSQL: a database file path (or ":memory:"). Needs `npm install better-sqlite3`.
+     * `dedup`, `objectStorage`, `autoMigrate` and `onOperation` apply to it too. Same file format as the Python
+     * package's SQLiteBackend, so both can share one `.db` file.
+     */
+    sqlite?: string;
+    /** Advanced: inject a custom StorageBackend instead of building one from `connectionString` / `pool` / `sqlite`. */
     backend?: StorageBackend;
     /** Max accepted size in bytes. Default 8 MiB. */
     maxBytes?: number;
@@ -204,6 +282,19 @@ interface ZeroBucketOptions<C = unknown> {
     poolTimeoutMs?: number;
     /** Fire-and-forget observability. Exceptions thrown inside it are swallowed. */
     onOperation?: (event: OperationEvent) => void;
+    /**
+     * S3-compatible object storage, enabling `tierToObjectStorage()` and transparent
+     * reads of images the Python or Node package tiered. Configure it with the same
+     * bucket and credentials used to tier. Without it, reading a tiered image throws
+     * a clear StorageError.
+     */
+    objectStorage?: ObjectStorageLike;
+    /**
+     * Content-addressed mode: identical bytes are stored once and shared by reference, each image
+     * keeping its own id and filename. Cannot be combined with `objectStorage`. Uses separate tables
+     * (zerobucket_blobs / zerobucket_image_refs); see `migrateClassicToDedup()` for existing data.
+     */
+    dedup?: boolean;
     /**
      * Authorization hook for get / getMany (per id) / getStream / streamTo / metadata.
      * Return false to deny. A hook that THROWS fails closed: the exception
@@ -222,7 +313,7 @@ interface PutBase<C> {
     /** Store non-image content through the same machinery. Incompatible with `optimize`. */
     validator?: ContentValidator;
     /** Join your own transaction. ZeroBucket then does not commit, roll back or retry. */
-    connection?: Queryable;
+    connection?: Connection;
     /** Passed to beforePut. */
     context?: C;
 }
@@ -232,7 +323,7 @@ interface PutManyOptions<C = unknown> extends Omit<PutBase<C>, "filename"> {
     filenames?: (string | null | undefined)[];
 }
 interface GetOptions<C = unknown> {
-    connection?: Queryable;
+    connection?: Connection;
     context?: C;
 }
 interface StreamOptions<C = unknown> extends GetOptions<C> {
@@ -302,6 +393,21 @@ declare class ZeroBucket<C = unknown> {
     delete(imageId: string, options?: Pick<CallOptions, "connection">): Promise<boolean>;
     /** Delete many in one query. Results are in input order; a missing id is `deleted: false`, not an error. */
     deleteMany(imageIds: string[], options?: Pick<CallOptions, "connection">): Promise<BatchDeleteResult[]>;
+    /**
+     * Move an image's bytes from Postgres into object storage; reads, streams and
+     * deletes keep working transparently. Returns true if tiered now, false if it was
+     * already tiered (safe to re-run in a backfill). Throws ImageNotFoundError if missing.
+     *
+     * The upload runs inside the database transaction: if it fails, the row is left
+     * completely untouched. Explicit and opt-in: put() never tiers automatically.
+     * Not gated by beforeGet/beforePut (it is a maintenance operation, as in the Python package).
+     */
+    tierToObjectStorage(imageId: string, options?: Pick<CallOptions, "connection">): Promise<boolean>;
+    /**
+     * One-time, non-destructive copy of classic-mode images into a dedup instance's tables, keeping every
+     * id. Call it on an instance created with `dedup: true`. Safe to re-run. See PostgresBackend for details.
+     */
+    migrateClassicToDedup(): ReturnType<NonNullable<StorageBackend["migrateClassicToDedup"]>>;
     /** Release the pool (if ZeroBucket created it). */
     close(): Promise<void>;
 }
@@ -316,4 +422,4 @@ declare function readInput(image: ImageInput, filename: string | undefined, maxB
     filename: string | undefined;
 }>;
 
-export { type BatchDeleteResult as B, type CallOptions as C, DEFAULT_MAX_BYTES as D, type GetOptions as G, type Image as I, type OperationEvent as O, type PreparedRow as P, type Queryable as Q, type StorageBackend as S, type ValidatedContent as V, type WritableLike as W, ZeroBucket as Z, type StoredRecord as a, type StoredRecordMetadata as b, type StreamOptions$1 as c, type ContentValidator as d, type BatchGetResult as e, type BatchPutResult as f, type BeforeGetHook as g, type BeforePutHook as h, type ImageInput as i, type ImageMetadata as j, type OptimizationResult as k, type OptimizeOptions as l, type PutManyOptions as m, type PutOptions as n, type StreamOptions as o, type ZeroBucketOptions as p, optimizeImage as q, readInput as r };
+export { type BatchDeleteResult as B, type CallOptions as C, DEFAULT_MAX_BYTES as D, type GetOptions as G, type Image as I, type OperationEvent as O, type PreparedRow as P, type Queryable as Q, type StorageBackend as S, type ValidatedContent as V, type WritableLike as W, ZeroBucket as Z, type ObjectStorageLike as a, type StoredRecord as b, type StoredRecordMetadata as c, type StreamOptions$1 as d, type ContentValidator as e, type BatchGetResult as f, type BatchPutResult as g, type BeforeGetHook as h, type BeforePutHook as i, type Connection as j, type ImageInput as k, type ImageMetadata as l, ObjectStorage as m, type ObjectStorageOptions as n, type OptimizationResult as o, type OptimizeOptions as p, type PutManyOptions as q, type PutOptions as r, type SqliteDatabaseLike as s, type StreamOptions as t, type ZeroBucketOptions as u, optimizeImage as v, readInput as w };

package/dist/{client-C_W0t8My.d.ts → client-WZ4VRCPi.d.ts}

@@ -63,6 +63,22 @@ interface Queryable {
         rowCount: number | null;
     }>;
 }
+/**
+ * The slice of a `better-sqlite3` Database that ZeroBucket's SQLite backend uses. Pass your own
+ * Database handle as `connection` to make a call part of YOUR SQLite transaction.
+ */
+interface SqliteDatabaseLike {
+    prepare(sql: string): {
+        all(...params: unknown[]): any[];
+        get(...params: unknown[]): any;
+        run(...params: unknown[]): {
+            changes: number;
+        };
+    };
+    exec(sql: string): unknown;
+}
+/** What `connection` accepts: a `pg` client for the Postgres backend, a `better-sqlite3` Database for SQLite. */
+type Connection = Queryable | SqliteDatabaseLike;
 /** The row shape a storage backend persists (after validation and checksumming). */
 interface PreparedRow {
     data: Buffer;
@@ -99,7 +115,7 @@ interface StoredRecord {
 }
 type StoredRecordMetadata = Omit<StoredRecord, "data">;
 interface CallOptions {
-    connection?: Queryable;
+    connection?: Connection;
 }
 interface StreamOptions$1 extends CallOptions {
     chunkSize: number;
@@ -128,9 +144,65 @@ interface StorageBackend {
     /** Returns the ids that were actually deleted. */
     deleteMany(ids: string[], opts?: CallOptions): Promise<string[]>;
     exists(id: string, opts?: CallOptions): Promise<boolean>;
+    /** null = no such image, false = already tiered (no-op), true = tiered now. Needs object storage configured. */
+    tierToObjectStorage(id: string, opts?: CallOptions): Promise<boolean | null>;
+    /** Dedup-mode instances only: copy the classic table into the dedup tables (non-destructive). */
+    migrateClassicToDedup?(): Promise<{
+        imagesMigrated: number;
+        distinctBlobsCreated: number;
+        duplicateReferencesFound: number;
+        alreadyMigrated: number;
+    }>;
     close(): Promise<void>;
 }
 
+/** The surface the Postgres adapter needs. Implement this to plug in a non-S3 store (or a test double). */
+interface ObjectStorageLike {
+    readonly bucket: string;
+    upload(key: string, data: Buffer, opts: {
+        mimeType: string;
+    }): Promise<void>;
+    download(key: string): Promise<Buffer>;
+    /** Inclusive byte range, fetched with a real HTTP Range request. */
+    downloadRange(key: string, start: number, endInclusive: number): Promise<Buffer>;
+    /** Idempotent: deleting a missing key is not an error. */
+    delete(key: string): Promise<void>;
+    exists(key: string): Promise<boolean>;
+}
+interface ObjectStorageOptions {
+    /** Must already exist. */
+    bucket: string;
+    /** Set for anything that is not AWS S3 itself (MinIO, R2, B2, Spaces, ...). */
+    endpoint?: string;
+    /** Default: AWS_REGION / AWS_DEFAULT_REGION, else "us-east-1". Many S3-compatible services ignore it but the SDK requires one. */
+    region?: string;
+    /** Default: the SDK's standard chain (env vars, shared config, IAM role, ...). */
+    credentials?: {
+        accessKeyId: string;
+        secretAccessKey: string;
+        sessionToken?: string;
+    };
+    /** Path-style addressing (`endpoint/bucket/key`). Default: true when `endpoint` is set, else false. */
+    forcePathStyle?: boolean;
+    /** Bring your own configured S3Client (advanced). */
+    client?: unknown;
+}
+declare class ObjectStorage implements ObjectStorageLike {
+    private readonly opts;
+    readonly bucket: string;
+    private clientPromise;
+    constructor(opts: ObjectStorageOptions);
+    private connection;
+    upload(key: string, data: Buffer, { mimeType }: {
+        mimeType: string;
+    }): Promise<void>;
+    download(key: string): Promise<Buffer>;
+    downloadRange(key: string, start: number, endInclusive: number): Promise<Buffer>;
+    private get;
+    delete(key: string): Promise<void>;
+    exists(key: string): Promise<boolean>;
+}
+
 interface OptimizeOptions {
     /** Downscale (aspect ratio preserved, Lanczos) if wider than this. Never upscales. */
     maxWidth?: number;
@@ -188,7 +260,13 @@ interface ZeroBucketOptions<C = unknown> {
     connectionString?: string;
     /** Bring your own pg.Pool. It is NOT closed by close(). */
     pool?: Pool;
-    /** Advanced: inject a custom StorageBackend instead of building a PostgresBackend. */
+    /**
+     * Use SQLite instead of PostgreSQL: a database file path (or ":memory:"). Needs `npm install better-sqlite3`.
+     * `dedup`, `objectStorage`, `autoMigrate` and `onOperation` apply to it too. Same file format as the Python
+     * package's SQLiteBackend, so both can share one `.db` file.
+     */
+    sqlite?: string;
+    /** Advanced: inject a custom StorageBackend instead of building one from `connectionString` / `pool` / `sqlite`. */
     backend?: StorageBackend;
     /** Max accepted size in bytes. Default 8 MiB. */
     maxBytes?: number;
@@ -204,6 +282,19 @@ interface ZeroBucketOptions<C = unknown> {
     poolTimeoutMs?: number;
     /** Fire-and-forget observability. Exceptions thrown inside it are swallowed. */
     onOperation?: (event: OperationEvent) => void;
+    /**
+     * S3-compatible object storage, enabling `tierToObjectStorage()` and transparent
+     * reads of images the Python or Node package tiered. Configure it with the same
+     * bucket and credentials used to tier. Without it, reading a tiered image throws
+     * a clear StorageError.
+     */
+    objectStorage?: ObjectStorageLike;
+    /**
+     * Content-addressed mode: identical bytes are stored once and shared by reference, each image
+     * keeping its own id and filename. Cannot be combined with `objectStorage`. Uses separate tables
+     * (zerobucket_blobs / zerobucket_image_refs); see `migrateClassicToDedup()` for existing data.
+     */
+    dedup?: boolean;
     /**
      * Authorization hook for get / getMany (per id) / getStream / streamTo / metadata.
      * Return false to deny. A hook that THROWS fails closed: the exception
@@ -222,7 +313,7 @@ interface PutBase<C> {
     /** Store non-image content through the same machinery. Incompatible with `optimize`. */
     validator?: ContentValidator;
     /** Join your own transaction. ZeroBucket then does not commit, roll back or retry. */
-    connection?: Queryable;
+    connection?: Connection;
     /** Passed to beforePut. */
     context?: C;
 }
@@ -232,7 +323,7 @@ interface PutManyOptions<C = unknown> extends Omit<PutBase<C>, "filename"> {
     filenames?: (string | null | undefined)[];
 }
 interface GetOptions<C = unknown> {
-    connection?: Queryable;
+    connection?: Connection;
     context?: C;
 }
 interface StreamOptions<C = unknown> extends GetOptions<C> {
@@ -302,6 +393,21 @@ declare class ZeroBucket<C = unknown> {
     delete(imageId: string, options?: Pick<CallOptions, "connection">): Promise<boolean>;
     /** Delete many in one query. Results are in input order; a missing id is `deleted: false`, not an error. */
     deleteMany(imageIds: string[], options?: Pick<CallOptions, "connection">): Promise<BatchDeleteResult[]>;
+    /**
+     * Move an image's bytes from Postgres into object storage; reads, streams and
+     * deletes keep working transparently. Returns true if tiered now, false if it was
+     * already tiered (safe to re-run in a backfill). Throws ImageNotFoundError if missing.
+     *
+     * The upload runs inside the database transaction: if it fails, the row is left
+     * completely untouched. Explicit and opt-in: put() never tiers automatically.
+     * Not gated by beforeGet/beforePut (it is a maintenance operation, as in the Python package).
+     */
+    tierToObjectStorage(imageId: string, options?: Pick<CallOptions, "connection">): Promise<boolean>;
+    /**
+     * One-time, non-destructive copy of classic-mode images into a dedup instance's tables, keeping every
+     * id. Call it on an instance created with `dedup: true`. Safe to re-run. See PostgresBackend for details.
+     */
+    migrateClassicToDedup(): ReturnType<NonNullable<StorageBackend["migrateClassicToDedup"]>>;
     /** Release the pool (if ZeroBucket created it). */
     close(): Promise<void>;
 }
@@ -316,4 +422,4 @@ declare function readInput(image: ImageInput, filename: string | undefined, maxB
     filename: string | undefined;
 }>;
 
-export { type BatchDeleteResult as B, type CallOptions as C, DEFAULT_MAX_BYTES as D, type GetOptions as G, type Image as I, type OperationEvent as O, type PreparedRow as P, type Queryable as Q, type StorageBackend as S, type ValidatedContent as V, type WritableLike as W, ZeroBucket as Z, type StoredRecord as a, type StoredRecordMetadata as b, type StreamOptions$1 as c, type ContentValidator as d, type BatchGetResult as e, type BatchPutResult as f, type BeforeGetHook as g, type BeforePutHook as h, type ImageInput as i, type ImageMetadata as j, type OptimizationResult as k, type OptimizeOptions as l, type PutManyOptions as m, type PutOptions as n, type StreamOptions as o, type ZeroBucketOptions as p, optimizeImage as q, readInput as r };
+export { type BatchDeleteResult as B, type CallOptions as C, DEFAULT_MAX_BYTES as D, type GetOptions as G, type Image as I, type OperationEvent as O, type PreparedRow as P, type Queryable as Q, type StorageBackend as S, type ValidatedContent as V, type WritableLike as W, ZeroBucket as Z, type ObjectStorageLike as a, type StoredRecord as b, type StoredRecordMetadata as c, type StreamOptions$1 as d, type ContentValidator as e, type BatchGetResult as f, type BatchPutResult as g, type BeforeGetHook as h, type BeforePutHook as i, type Connection as j, type ImageInput as k, type ImageMetadata as l, ObjectStorage as m, type ObjectStorageOptions as n, type OptimizationResult as o, type OptimizeOptions as p, type PutManyOptions as q, type PutOptions as r, type SqliteDatabaseLike as s, type StreamOptions as t, type ZeroBucketOptions as u, optimizeImage as v, readInput as w };

package/dist/http.d.cts

@@ -1,5 +1,5 @@
 import { IncomingMessage, ServerResponse } from 'node:http';
-import { Z as ZeroBucket } from './client-C_W0t8My.cjs';
+import { Z as ZeroBucket } from './client-WZ4VRCPi.cjs';
 import 'pg';
 
 /**

package/dist/http.d.ts

@@ -1,5 +1,5 @@
 import { IncomingMessage, ServerResponse } from 'node:http';
-import { Z as ZeroBucket } from './client-C_W0t8My.js';
+import { Z as ZeroBucket } from './client-WZ4VRCPi.js';
 import 'pg';
 
 /**