zerobucket 0.8.0 → 0.8.2

package/README.md

@@ -123,6 +123,12 @@ await images.get(id, { context: { userId } }); // throws AccessDeniedError if th
 **A hook that throws fails closed**: the original exception propagates; it is never treated as
 "allow". Denied calls never reach the database. `exists()` is deliberately not gated.
 
+**Your hook receives the id exactly as the caller supplied it**, before ZeroBucket validates it, so it can be a
+malformed string such as one taken straight from a URL. Check the format before using it in SQL (for example against a
+UUID pattern, returning `true` for a malformed id so ZeroBucket answers "not found"); otherwise a request for
+`/images/garbage` becomes a database error and a `500`. The [example app](https://github.com/KedarGhadyalji/ZeroBucket/tree/main/examples/node-integration)
+shows a working hook, plus transactional uploads and serving.
+
 ## Streaming and ranges
 
 ```ts
@@ -236,6 +242,7 @@ zerobucket tier IMAGE_ID --bucket my-images                         # move one i
 zerobucket tier --older-than 90 --min-size 500000 --bucket my-images --dry-run   # preview a bulk move
 zerobucket tier --all --limit 1000 --bucket my-images --endpoint-url https://<acct>.r2.cloudflarestorage.com
 zerobucket migrate --to-dedup   # PostgreSQL: copy classic images into the dedup tables (non-destructive, re-runnable)
+zerobucket migrate --to-dedup --skip-tiered   # ...skipping images tiered to object storage instead of refusing
 ```
 
 - **Exit codes:** `0` success, `1` problems found or an operation failed, `2` usage or connection error. That makes
@@ -390,7 +397,11 @@ await images.delete(b); // last reference gone: blob removed
 
 **Moving existing data in:** `await images.migrateClassicToDedup()` copies every classic image into the dedup tables,
 keeping each id, and never modifies the classic table. It runs in small batches (bounded memory), is safe to re-run
-(already-migrated ids are skipped and counted), and refuses up front if any classic image is tiered to object storage.
+(already-migrated ids are skipped and counted), and refuses up front if any classic image is tiered to object storage
+(their bytes are not in the database to copy, and dedup mode has no tiering), changing nothing. If you would rather
+migrate everything else, pass `{ skipTiered: true }` (CLI: `--skip-tiered`): tiered images are left untouched in the
+classic table, counted in `tieredSkipped`, and **will not exist in the dedup tables**, so only use it if you can live
+without them there.
 
 ## Object-storage tiering
 
@@ -440,30 +451,52 @@ same corpus of valid and corrupt files identically.
 
 ## Differences from the Python package
 
-|                                                | Python                                                                     | npm                                                                                                              |
-| ---------------------------------------------- | -------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- |
-| API                                            | sync + async clients                                                       | one async client                                                                                                 |
-| Malformed id (`"abc"`)                         | `StorageError` from Postgres                                               | treated as not found (`ImageNotFoundError`, no DB round trip), friendlier for `/images/:id` routes               |
-| `optimize`                                     | Pillow; EXIF stripped, not applied                                         | sharp; EXIF **applied then stripped**; transparency → **white** for JPEG                                         |
-| `putMany` inserts                              | pipelined                                                                  | one transaction, statements run sequentially                                                                     |
-| Formats                                        | JPEG, PNG, WebP, HEIC (`[heic]` extra)                                     | JPEG, PNG, WebP, HEIC (optional `libheif-js`)                                                                    |
-| Convert TO HEIC with `optimize`                | yes (pillow-heif)                                                          | no (no HEVC encoder for Node); converting FROM HEIC works; `optimize: true` on a HEIC needs an explicit `format` |
-| Event duration                                 | `duration_seconds`                                                         | `durationMs`                                                                                                     |
-| `migrateClassicToDedup`                        | loads the whole table into memory; fails if re-run or if any row is tiered | batched, safe to re-run, refuses tiered rows with a clear message                                                |
-| Dedup `putMany` / `deleteMany` locking         | arbitrary order (overlapping batches can deadlock)                         | sorted checksum order (cannot deadlock)                                                                          |
-| SQLite API                                     | synchronous `SQLiteBackend` + separate async backend                       | one async client; the sync driver runs on the event loop, lock waits are async (never block it)                  |
-| SQLite `onOperation` / busy retry              | none                                                                       | `onOperation` events and async `SQLITE_BUSY` retry (`busyTimeoutMs`)                                             |
-| SQLite stream deleted mid-read                 | sync adapter keeps reading a WAL snapshot (async adapter raises)           | raises `StorageError`, like Postgres; use `connection` in a transaction for a snapshot                           |
-| CLI                                            | PostgreSQL, classic mode only; `verify` crashes on tiered rows             | PostgreSQL **and** SQLite, classic **and** dedup; `info`/`verify` understand tiered rows; `migrate --to-dedup`   |
-| `connection=` without an open transaction      | multi-statement ops rely on the driver's implicit transaction              | always atomic (own transaction or savepoint)                                                                     |
-| `delete(id, connection=...)` on a tiered image | deletes the S3 object immediately                                          | leaves the object until you delete it after commit (a rolled-back transaction can never lose data)               |
-| S3 delete failure after `delete()`             | raises after the row is gone                                               | never fails the delete; reported via `onOperation`                                                               |
-| Options                                        | snake_case kwargs                                                          | camelCase options object                                                                                         |
-
-## What's not here yet
-
-Planned for MySQL/MariaDB: a MySQL-specific operations guide, and `migrate --to-dedup` (PostgreSQL-only today, as in the
-Python package). Tiering and dedup cannot be combined (same rule as the Python package).
+Python column = zerobucket on PyPI **0.25.1 or later** (several of its behaviours below were fixed in 0.25.1; older versions
+differ).
+
+|                                              | Python                                                                | npm                                                                                                                                |
+| -------------------------------------------- | --------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- |
+| API                                          | sync + async clients                                                  | one async client                                                                                                                   |
+| Malformed id (`"abc"`)                       | `StorageError` from Postgres                                          | treated as not found (`ImageNotFoundError`, no DB round trip), friendlier for `/images/:id` routes                                 |
+| `optimize`                                   | Pillow; EXIF stripped, not applied                                    | sharp; EXIF **applied then stripped**; transparency → **white** for JPEG                                                           |
+| `putMany` inserts                            | pipelined                                                             | one transaction, statements run sequentially                                                                                       |
+| Formats                                      | JPEG, PNG, WebP, HEIC (`[heic]` extra)                                | JPEG, PNG, WebP, HEIC (optional `libheif-js`)                                                                                      |
+| Convert TO HEIC with `optimize`              | yes (pillow-heif)                                                     | no (no HEVC encoder for Node); converting FROM HEIC works; `optimize: true` on a HEIC needs an explicit `format`                   |
+| Event duration                               | `duration_seconds`                                                    | `durationMs`                                                                                                                       |
+| `migrateClassicToDedup`, tiered classic rows | always **skips** them and reports `tiered_rows_skipped`               | **refuses** by default (changes nothing); `skipTiered: true` / `--skip-tiered` skips and reports like Python                       |
+| `migrateClassicToDedup`, other behaviour     | batched, safe to re-run (server-side cursor)                          | batched, safe to re-run (keyset paging, advisory lock so concurrent runs cannot race)                                              |
+| SQLite API                                   | synchronous `SQLiteBackend` + separate async backend                  | one async client; the sync driver runs on the event loop, lock waits are async (never block it)                                    |
+| SQLite `onOperation` / busy retry            | none                                                                  | `onOperation` events and async `SQLITE_BUSY` retry (`busyTimeoutMs`)                                                               |
+| SQLite stream deleted mid-read               | sync adapter keeps reading a WAL snapshot (async adapter raises)      | raises `StorageError`, like Postgres; use `connection` in a transaction for a snapshot                                             |
+| MySQL `onOperation` / retry                  | sync `MySQLBackend` yes; `AsyncMySQLBackend` none                     | yes (same client for everything)                                                                                                   |
+| CLI                                          | PostgreSQL, classic mode only; `verify` skips tiered rows with a note | PostgreSQL, SQLite **and** MySQL/MariaDB, classic **and** dedup; `verify --bucket` also checks tiered images; `migrate --to-dedup` |
+| `connection=` without an open transaction    | multi-statement ops rely on the driver's implicit transaction         | always atomic (own transaction or savepoint)                                                                                       |
+| S3 delete failure after `delete()`           | raises after the row is gone                                          | never fails the delete; reported via `onOperation`                                                                                 |
+| Options                                      | snake_case kwargs                                                     | camelCase options object                                                                                                           |
+
+Both packages now leave a tiered image's object in place on `delete(id, connection=...)` (a rolled-back transaction can
+never lose data), take dedup batch locks in sorted checksum order (overlapping batches cannot deadlock), and skip the
+MySQL `ALTER TABLE` at startup when the schema is current.
+
+## Limitations (read before using in production)
+
+- **Images live in your database.** That is the point (one backup, one transaction, no extra service), but it means
+  this is not built for very large files or high-volume media: every image goes through the database and counts toward
+  its size, its backups and its replication. The default size limit is 8 MiB (`maxBytes`). For large video or
+  millions of big files, use object storage directly.
+- **Backups and space need attention.** A whole-database dump now includes every image, and deleting images does not
+  give disk space back on its own (PostgreSQL needs autovacuum, MySQL `OPTIMIZE TABLE`, SQLite `VACUUM`). SQLite
+  files must be backed up with `.backup`, not copied. See the
+  [Operations Guide](https://github.com/KedarGhadyalji/ZeroBucket/blob/main/docs/OPERATIONS.md).
+- **Dedup and tiering cannot be combined** (a shared blob has many owners): `dedup: true` with `objectStorage` throws.
+- **`migrate --to-dedup` is PostgreSQL-only**, as in the Python package. Tiered classic images cannot be migrated (see
+  Deduplication for `skipTiered`).
+- **HEIC can be read and converted from, but not written.** There is no HEVC encoder for Node, so `optimize` cannot
+  produce HEIC.
+- **SQLite allows one writer at a time**; lock waits are async and never block the event loop, but write-heavy
+  workloads belong on PostgreSQL or MySQL/MariaDB.
+- **Tested versions:** CI runs PostgreSQL 16 and MariaDB 10.11; MySQL 8.0.46 was also run by hand. Other versions are
+  expected to behave the same but have not been run.
 
 ## Runtime notes
 

package/dist/bin.js

@@ -875,13 +875,13 @@ var PostgresBackend = class {
    * dedup instance's tables, preserving every id (external references keep working) and
    * deduplicating identical content on the way. The classic table is never modified.
    *
-   * Differences from the Python version, all deliberate: rows are copied in small batches (bounded
-   * memory instead of loading the whole table); re-running is safe (already-migrated ids are
-   * skipped and reported, not an error); and classic rows already tiered to object storage are
-   * refused up front with a clear message (their bytes are not in the database to copy).
+   * Rows are copied in small batches (bounded memory instead of loading the whole table);
+   * re-running is safe (already-migrated ids are skipped and reported, not an error). Classic rows
+   * already tiered to object storage are refused up front with a clear message (their bytes are
+   * not in the database to copy) unless `skipTiered: true`, which skips and counts them instead.
    * Still all-or-nothing: one transaction.
    */
-  async migrateClassicToDedup() {
+  async migrateClassicToDedup(opts = {}) {
     if (!this.dedup) {
       throw new StorageError(
         "migrateClassicToDedup() must be called on an instance created with dedup: true."
@@ -900,6 +900,7 @@ var PostgresBackend = class {
           "No classic zerobucket_images table found in this database -- nothing to migrate."
         );
       }
+      let tieredSkipped = 0;
       const hasTierCol = (await client.query(
         "SELECT 1 FROM information_schema.columns WHERE table_name = 'zerobucket_images' AND column_name = 'storage_backend'"
       )).rowCount;
@@ -907,12 +908,14 @@ var PostgresBackend = class {
         const tiered = (await client.query(
           "SELECT count(*)::int AS n FROM zerobucket_images WHERE storage_backend <> 'postgres'"
         )).rows[0].n;
-        if (tiered > 0) {
+        if (tiered > 0 && !opts.skipTiered) {
           throw new StorageError(
-            `${tiered} classic image(s) are tiered to object storage, so their bytes are not in the database to migrate. Dedup mode does not support tiering. Migrate only an un-tiered table, or bring those images back first.`
+            `${tiered} classic image(s) are tiered to object storage, so their bytes are not in the database to migrate. Dedup mode does not support tiering. Migrate only an un-tiered table, bring those images back first, or pass skipTiered: true (CLI: --skip-tiered) to skip them and migrate the rest.`
           );
         }
+        tieredSkipped = tiered;
       }
+      const onlyInDb = hasTierCol && tieredSkipped > 0;
       const seen = /* @__PURE__ */ new Set();
       let migrated = 0;
       let already = 0;
@@ -921,11 +924,11 @@ var PostgresBackend = class {
       for (; ; ) {
         const page = cursor ? await client.query(
           `SELECT id, data, mime_type, original_filename, size_bytes, width, height, checksum_sha256, created_at::text AS created_at_txt
-               FROM zerobucket_images WHERE (created_at, id) > ($1::timestamptz, $2::uuid) ORDER BY created_at, id LIMIT ${BATCH}`,
+               FROM zerobucket_images WHERE (created_at, id) > ($1::timestamptz, $2::uuid)${onlyInDb ? " AND storage_backend = 'postgres'" : ""} ORDER BY created_at, id LIMIT ${BATCH}`,
           [cursor.createdAt, cursor.id]
         ) : await client.query(
           `SELECT id, data, mime_type, original_filename, size_bytes, width, height, checksum_sha256, created_at::text AS created_at_txt
-               FROM zerobucket_images ORDER BY created_at, id LIMIT ${BATCH}`
+               FROM zerobucket_images${onlyInDb ? " WHERE storage_backend = 'postgres'" : ""} ORDER BY created_at, id LIMIT ${BATCH}`
         );
         if (page.rows.length === 0) break;
         const last = page.rows[page.rows.length - 1];
@@ -975,7 +978,8 @@ var PostgresBackend = class {
         imagesMigrated: migrated,
         distinctBlobsCreated: seen.size,
         duplicateReferencesFound: migrated - seen.size,
-        alreadyMigrated: already
+        alreadyMigrated: already,
+        tieredSkipped
       };
     } catch (exc) {
       try {
@@ -2995,6 +2999,7 @@ Commands:
   init       Create the schema (tables and indexes) if missing.
   migrate    Apply schema migrations (currently: same as init).
              --to-dedup  copy every classic image into the dedup tables (PostgreSQL only; non-destructive)
+             --skip-tiered  with --to-dedup: skip images tiered to object storage instead of refusing
   info       Show storage stats: count, size, breakdown by format.
   verify     Re-checksum stored images to detect corruption.
                --sample N        check a random sample of N images instead of every image
@@ -3024,6 +3029,7 @@ var OPTIONS = {
   help: { type: "boolean", short: "h" },
   version: { type: "boolean", short: "v" },
   "to-dedup": { type: "boolean" },
+  "skip-tiered": { type: "boolean" },
   sample: { type: "string" },
   bucket: { type: "string" },
   "endpoint-url": { type: "string" },
@@ -3038,7 +3044,7 @@ var OPTIONS = {
 };
 var COMMAND_OPTIONS = {
   init: [],
-  migrate: ["to-dedup"],
+  migrate: ["to-dedup", "skip-tiered"],
   info: [],
   verify: [
     "sample",
@@ -3155,6 +3161,8 @@ async function cmdInit(v, env, io) {
   return 0;
 }
 async function cmdMigrate(v, env, io) {
+  if (v["skip-tiered"] && !v["to-dedup"])
+    throw usageError("Error: --skip-tiered only applies with --to-dedup.");
   if (v["to-dedup"]) {
     const target = resolveTarget(v, env);
     if (target.kind !== "postgres")
@@ -3162,7 +3170,9 @@ async function cmdMigrate(v, env, io) {
     const backend = openBackend(target, { dedup: true, autoMigrate: true });
     await connect(backend, "connect");
     try {
-      const r = await backend.migrateClassicToDedup();
+      const r = await backend.migrateClassicToDedup({
+        skipTiered: Boolean(v["skip-tiered"])
+      });
       io.out(
         `Migrated ${num(r.imagesMigrated)} image(s) into ${num(r.distinctBlobsCreated)} distinct blob(s).`
       );
@@ -3170,6 +3180,10 @@ async function cmdMigrate(v, env, io) {
         `  Duplicate references found: ${num(r.duplicateReferencesFound)}`
       );
       io.out(`  Already migrated (skipped): ${num(r.alreadyMigrated)}`);
+      if (r.tieredSkipped > 0)
+        io.out(
+          `  Tiered to object storage (skipped, NOT in the dedup tables): ${num(r.tieredSkipped)}`
+        );
       io.out("The classic table was not modified.");
       return 0;
     } catch (exc) {

package/dist/{client-XQ22b1l3.d.cts → client-l4jPIFJI.d.cts}

@@ -191,6 +191,24 @@ interface StorageInfo {
         storedBytes: number;
     };
 }
+interface MigrateToDedupOptions {
+    /**
+     * Classic rows already tiered to object storage cannot be migrated (their bytes are not in the
+     * database, and dedup mode has no tiering). By default the migration REFUSES, changing nothing.
+     * With `skipTiered: true` it skips them instead, leaves them untouched in the classic table, and
+     * reports the count in `tieredSkipped`. Skipped images will not exist in the dedup tables.
+     */
+    skipTiered?: boolean;
+}
+interface MigrateToDedupResult {
+    imagesMigrated: number;
+    distinctBlobsCreated: number;
+    duplicateReferencesFound: number;
+    /** Ids already present in the dedup tables (a re-run): skipped, never double-counted. */
+    alreadyMigrated: number;
+    /** Tiered classic rows skipped because `skipTiered` was set; always 0 otherwise. */
+    tieredSkipped: number;
+}
 interface StorageBackend {
     /** Resolve once the backend is connected and migrated. Safe to call repeatedly. */
     ready(): Promise<void>;
@@ -213,12 +231,7 @@ interface StorageBackend {
     /** null = no such image, false = already tiered (no-op), true = tiered now. Needs object storage configured. */
     tierToObjectStorage(id: string, opts?: CallOptions): Promise<boolean | null>;
     /** Dedup-mode instances only: copy the classic table into the dedup tables (non-destructive). */
-    migrateClassicToDedup?(): Promise<{
-        imagesMigrated: number;
-        distinctBlobsCreated: number;
-        duplicateReferencesFound: number;
-        alreadyMigrated: number;
-    }>;
+    migrateClassicToDedup?(opts?: MigrateToDedupOptions): Promise<MigrateToDedupResult>;
     /** Maintenance: storage statistics, or null if the tables do not exist yet. */
     getInfo?(): Promise<StorageInfo | null>;
     /** Maintenance: list ids matching the filters (oldest first, or random when sampling). */
@@ -487,7 +500,7 @@ declare class ZeroBucket<C = unknown> {
      * One-time, non-destructive copy of classic-mode images into a dedup instance's tables, keeping every
      * id. Call it on an instance created with `dedup: true`. Safe to re-run. See PostgresBackend for details.
      */
-    migrateClassicToDedup(): ReturnType<NonNullable<StorageBackend["migrateClassicToDedup"]>>;
+    migrateClassicToDedup(opts?: MigrateToDedupOptions): Promise<MigrateToDedupResult>;
     /** Release the pool (if ZeroBucket created it). */
     close(): Promise<void>;
 }
@@ -502,4 +515,4 @@ declare function readInput(image: ImageInput, filename: string | undefined, maxB
     filename: string | undefined;
 }>;
 
-export { optimizeImage as A, type BatchDeleteResult as B, type CallOptions as C, DEFAULT_MAX_BYTES as D, readInput as E, type GetOptions as G, type ImageListOptions as I, type MysqlPoolLike as M, type OperationEvent as O, type PreparedRow as P, type Queryable as Q, type StorageBackend as S, type ValidatedContent as V, type WritableLike as W, ZeroBucket as Z, type ObjectStorageLike as a, type StoredRecord as b, type StoredRecordMetadata as c, type StreamOptions$1 as d, type StorageInfo as e, type ImageListEntry as f, type ContentValidator as g, type BatchGetResult as h, type BatchPutResult as i, type BeforeGetHook as j, type BeforePutHook as k, type Connection as l, type Image as m, type ImageInput as n, type ImageMetadata as o, type MysqlPoolConnectionLike as p, type MysqlQueryable as q, ObjectStorage as r, type ObjectStorageOptions as s, type OptimizationResult as t, type OptimizeOptions as u, type PutManyOptions as v, type PutOptions as w, type SqliteDatabaseLike as x, type StreamOptions as y, type ZeroBucketOptions as z };
+export { type StreamOptions as A, type BatchDeleteResult as B, type CallOptions as C, DEFAULT_MAX_BYTES as D, type ZeroBucketOptions as E, optimizeImage as F, type GetOptions as G, readInput as H, type ImageListOptions as I, type MigrateToDedupOptions as M, type OperationEvent as O, type PreparedRow as P, type Queryable as Q, type StorageBackend as S, type ValidatedContent as V, type WritableLike as W, ZeroBucket as Z, type ObjectStorageLike as a, type StoredRecord as b, type StoredRecordMetadata as c, type StreamOptions$1 as d, type StorageInfo as e, type ImageListEntry as f, type MigrateToDedupResult as g, type MysqlPoolLike as h, type ContentValidator as i, type BatchGetResult as j, type BatchPutResult as k, type BeforeGetHook as l, type BeforePutHook as m, type Connection as n, type Image as o, type ImageInput as p, type ImageMetadata as q, type MysqlPoolConnectionLike as r, type MysqlQueryable as s, ObjectStorage as t, type ObjectStorageOptions as u, type OptimizationResult as v, type OptimizeOptions as w, type PutManyOptions as x, type PutOptions as y, type SqliteDatabaseLike as z };

package/dist/{client-XQ22b1l3.d.ts → client-l4jPIFJI.d.ts}

@@ -191,6 +191,24 @@ interface StorageInfo {
         storedBytes: number;
     };
 }
+interface MigrateToDedupOptions {
+    /**
+     * Classic rows already tiered to object storage cannot be migrated (their bytes are not in the
+     * database, and dedup mode has no tiering). By default the migration REFUSES, changing nothing.
+     * With `skipTiered: true` it skips them instead, leaves them untouched in the classic table, and
+     * reports the count in `tieredSkipped`. Skipped images will not exist in the dedup tables.
+     */
+    skipTiered?: boolean;
+}
+interface MigrateToDedupResult {
+    imagesMigrated: number;
+    distinctBlobsCreated: number;
+    duplicateReferencesFound: number;
+    /** Ids already present in the dedup tables (a re-run): skipped, never double-counted. */
+    alreadyMigrated: number;
+    /** Tiered classic rows skipped because `skipTiered` was set; always 0 otherwise. */
+    tieredSkipped: number;
+}
 interface StorageBackend {
     /** Resolve once the backend is connected and migrated. Safe to call repeatedly. */
     ready(): Promise<void>;
@@ -213,12 +231,7 @@ interface StorageBackend {
     /** null = no such image, false = already tiered (no-op), true = tiered now. Needs object storage configured. */
     tierToObjectStorage(id: string, opts?: CallOptions): Promise<boolean | null>;
     /** Dedup-mode instances only: copy the classic table into the dedup tables (non-destructive). */
-    migrateClassicToDedup?(): Promise<{
-        imagesMigrated: number;
-        distinctBlobsCreated: number;
-        duplicateReferencesFound: number;
-        alreadyMigrated: number;
-    }>;
+    migrateClassicToDedup?(opts?: MigrateToDedupOptions): Promise<MigrateToDedupResult>;
     /** Maintenance: storage statistics, or null if the tables do not exist yet. */
     getInfo?(): Promise<StorageInfo | null>;
     /** Maintenance: list ids matching the filters (oldest first, or random when sampling). */
@@ -487,7 +500,7 @@ declare class ZeroBucket<C = unknown> {
      * One-time, non-destructive copy of classic-mode images into a dedup instance's tables, keeping every
      * id. Call it on an instance created with `dedup: true`. Safe to re-run. See PostgresBackend for details.
      */
-    migrateClassicToDedup(): ReturnType<NonNullable<StorageBackend["migrateClassicToDedup"]>>;
+    migrateClassicToDedup(opts?: MigrateToDedupOptions): Promise<MigrateToDedupResult>;
     /** Release the pool (if ZeroBucket created it). */
     close(): Promise<void>;
 }
@@ -502,4 +515,4 @@ declare function readInput(image: ImageInput, filename: string | undefined, maxB
     filename: string | undefined;
 }>;
 
-export { optimizeImage as A, type BatchDeleteResult as B, type CallOptions as C, DEFAULT_MAX_BYTES as D, readInput as E, type GetOptions as G, type ImageListOptions as I, type MysqlPoolLike as M, type OperationEvent as O, type PreparedRow as P, type Queryable as Q, type StorageBackend as S, type ValidatedContent as V, type WritableLike as W, ZeroBucket as Z, type ObjectStorageLike as a, type StoredRecord as b, type StoredRecordMetadata as c, type StreamOptions$1 as d, type StorageInfo as e, type ImageListEntry as f, type ContentValidator as g, type BatchGetResult as h, type BatchPutResult as i, type BeforeGetHook as j, type BeforePutHook as k, type Connection as l, type Image as m, type ImageInput as n, type ImageMetadata as o, type MysqlPoolConnectionLike as p, type MysqlQueryable as q, ObjectStorage as r, type ObjectStorageOptions as s, type OptimizationResult as t, type OptimizeOptions as u, type PutManyOptions as v, type PutOptions as w, type SqliteDatabaseLike as x, type StreamOptions as y, type ZeroBucketOptions as z };
+export { type StreamOptions as A, type BatchDeleteResult as B, type CallOptions as C, DEFAULT_MAX_BYTES as D, type ZeroBucketOptions as E, optimizeImage as F, type GetOptions as G, readInput as H, type ImageListOptions as I, type MigrateToDedupOptions as M, type OperationEvent as O, type PreparedRow as P, type Queryable as Q, type StorageBackend as S, type ValidatedContent as V, type WritableLike as W, ZeroBucket as Z, type ObjectStorageLike as a, type StoredRecord as b, type StoredRecordMetadata as c, type StreamOptions$1 as d, type StorageInfo as e, type ImageListEntry as f, type MigrateToDedupResult as g, type MysqlPoolLike as h, type ContentValidator as i, type BatchGetResult as j, type BatchPutResult as k, type BeforeGetHook as l, type BeforePutHook as m, type Connection as n, type Image as o, type ImageInput as p, type ImageMetadata as q, type MysqlPoolConnectionLike as r, type MysqlQueryable as s, ObjectStorage as t, type ObjectStorageOptions as u, type OptimizationResult as v, type OptimizeOptions as w, type PutManyOptions as x, type PutOptions as y, type SqliteDatabaseLike as z };

package/dist/http.d.cts

@@ -1,5 +1,5 @@
 import { IncomingMessage, ServerResponse } from 'node:http';
-import { Z as ZeroBucket } from './client-XQ22b1l3.cjs';
+import { Z as ZeroBucket } from './client-l4jPIFJI.cjs';
 import 'pg';
 
 /**

package/dist/http.d.ts

@@ -1,5 +1,5 @@
 import { IncomingMessage, ServerResponse } from 'node:http';
-import { Z as ZeroBucket } from './client-XQ22b1l3.js';
+import { Z as ZeroBucket } from './client-l4jPIFJI.js';
 import 'pg';
 
 /**

package/dist/index.cjs

@@ -980,13 +980,13 @@ var PostgresBackend = class {
    * dedup instance's tables, preserving every id (external references keep working) and
    * deduplicating identical content on the way. The classic table is never modified.
    *
-   * Differences from the Python version, all deliberate: rows are copied in small batches (bounded
-   * memory instead of loading the whole table); re-running is safe (already-migrated ids are
-   * skipped and reported, not an error); and classic rows already tiered to object storage are
-   * refused up front with a clear message (their bytes are not in the database to copy).
+   * Rows are copied in small batches (bounded memory instead of loading the whole table);
+   * re-running is safe (already-migrated ids are skipped and reported, not an error). Classic rows
+   * already tiered to object storage are refused up front with a clear message (their bytes are
+   * not in the database to copy) unless `skipTiered: true`, which skips and counts them instead.
    * Still all-or-nothing: one transaction.
    */
-  async migrateClassicToDedup() {
+  async migrateClassicToDedup(opts = {}) {
     if (!this.dedup) {
       throw new StorageError(
         "migrateClassicToDedup() must be called on an instance created with dedup: true."
@@ -1005,6 +1005,7 @@ var PostgresBackend = class {
           "No classic zerobucket_images table found in this database -- nothing to migrate."
         );
       }
+      let tieredSkipped = 0;
       const hasTierCol = (await client.query(
         "SELECT 1 FROM information_schema.columns WHERE table_name = 'zerobucket_images' AND column_name = 'storage_backend'"
       )).rowCount;
@@ -1012,12 +1013,14 @@ var PostgresBackend = class {
         const tiered = (await client.query(
           "SELECT count(*)::int AS n FROM zerobucket_images WHERE storage_backend <> 'postgres'"
         )).rows[0].n;
-        if (tiered > 0) {
+        if (tiered > 0 && !opts.skipTiered) {
           throw new StorageError(
-            `${tiered} classic image(s) are tiered to object storage, so their bytes are not in the database to migrate. Dedup mode does not support tiering. Migrate only an un-tiered table, or bring those images back first.`
+            `${tiered} classic image(s) are tiered to object storage, so their bytes are not in the database to migrate. Dedup mode does not support tiering. Migrate only an un-tiered table, bring those images back first, or pass skipTiered: true (CLI: --skip-tiered) to skip them and migrate the rest.`
           );
         }
+        tieredSkipped = tiered;
       }
+      const onlyInDb = hasTierCol && tieredSkipped > 0;
       const seen = /* @__PURE__ */ new Set();
       let migrated = 0;
       let already = 0;
@@ -1026,11 +1029,11 @@ var PostgresBackend = class {
       for (; ; ) {
         const page = cursor ? await client.query(
           `SELECT id, data, mime_type, original_filename, size_bytes, width, height, checksum_sha256, created_at::text AS created_at_txt
-               FROM zerobucket_images WHERE (created_at, id) > ($1::timestamptz, $2::uuid) ORDER BY created_at, id LIMIT ${BATCH}`,
+               FROM zerobucket_images WHERE (created_at, id) > ($1::timestamptz, $2::uuid)${onlyInDb ? " AND storage_backend = 'postgres'" : ""} ORDER BY created_at, id LIMIT ${BATCH}`,
           [cursor.createdAt, cursor.id]
         ) : await client.query(
           `SELECT id, data, mime_type, original_filename, size_bytes, width, height, checksum_sha256, created_at::text AS created_at_txt
-               FROM zerobucket_images ORDER BY created_at, id LIMIT ${BATCH}`
+               FROM zerobucket_images${onlyInDb ? " WHERE storage_backend = 'postgres'" : ""} ORDER BY created_at, id LIMIT ${BATCH}`
         );
         if (page.rows.length === 0) break;
         const last = page.rows[page.rows.length - 1];
@@ -1080,7 +1083,8 @@ var PostgresBackend = class {
         imagesMigrated: migrated,
         distinctBlobsCreated: seen.size,
         duplicateReferencesFound: migrated - seen.size,
-        alreadyMigrated: already
+        alreadyMigrated: already,
+        tieredSkipped
       };
     } catch (exc) {
       try {
@@ -3739,13 +3743,13 @@ var ZeroBucket = class _ZeroBucket {
    * One-time, non-destructive copy of classic-mode images into a dedup instance's tables, keeping every
    * id. Call it on an instance created with `dedup: true`. Safe to re-run. See PostgresBackend for details.
    */
-  async migrateClassicToDedup() {
+  async migrateClassicToDedup(opts) {
     if (!this.backend.migrateClassicToDedup) {
       throw new StorageError(
         "This backend does not support migrateClassicToDedup()."
       );
     }
-    return this.backend.migrateClassicToDedup();
+    return this.backend.migrateClassicToDedup(opts);
   }
   /** Release the pool (if ZeroBucket created it). */
   close() {