backupdrill 0.1.2 → 0.2.0

package/README.md

@@ -6,10 +6,12 @@ Backups stream through the CLI on your machine (or your CI) straight to your buc
 
 > Want this on a schedule, with **restore drills that prove your backup actually restores**, plus alerts and reports? That's the hosted service → **[backupdrill.com](https://backupdrill.com)**. This CLI is the DIY layer; the cloud adds scheduling, drills, alerting, and reports on top.
 
+🎥 Prefer watching? [Where do your backups actually go — explained in about a minute](https://backupdrill.com/guides/test-supabase-backup-restore#explainer-video).
+
 ## Why
 
 - Supabase's own database restore only brings back `storage.objects` **metadata**, not your actual Storage files.
-- Pro-plan backups keep 7 days and can't be pulled into your own infrastructure; PITR is $100/mo per project.
+- Pro-plan backups keep 7 days and can't be pulled into your own infrastructure; PITR is $100/month per project.
 - A `pg_dump` you've never restored is a guess.
 
 This CLI gets your database into a bucket **you** control, with a manifest you can check.
@@ -106,19 +108,19 @@ With those set, `backupdrill backup` also copies every Storage file into your bu
 
 ## Egress & cost — read this before scheduling daily backups
 
-A backup pulls data **out** of Supabase, which counts as **egress on your Supabase bill**. Verified rates (2026-07-04, [egress docs](https://supabase.com/docs/guides/platform/manage-your-usage/egress) + [pricing](https://supabase.com/pricing)):
+A backup pulls data **out** of Supabase, which counts as **egress on your Supabase bill**. Verified rates (2026-07-10, [egress docs](https://supabase.com/docs/guides/platform/manage-your-usage/egress) + [pricing](https://supabase.com/pricing)):
 
 - Backup traffic (pg_dump + first-time file downloads) is **uncached egress: $0.09/GB** above your plan's included allowance.
 - Included uncached egress: **Free 5 GB · Pro 250 GB · Team 250 GB** per month.
 - **Spend Cap is ON by default (Pro/Team).** If a backup pushes you past the quota, Supabase **throttles your whole project's egress** for the rest of the month and the backup fails — turn Spend Cap OFF to be billed the overage instead.
-- **Free is a hard cap:** exceed 5 GB and the project is restricted (never billed). A large daily backup can't complete on Free.
+- **Free never bills overage.** Exceeding the included 5 GB/month starts Supabase's Fair Use process: a notification to your billing email, then a grace period, then egress restrictions if usage stays over (the restriction clears at the next billing cycle or when you upgrade). Sustained large backups are not viable on Free.
 
 Rough cost (Pro, Spend Cap OFF, backups = your only egress):
 
 | DB + Storage | Weekly | Daily |
 |---|---|---|
-| 10 GB | $0/mo (within 250 GB) | ~$4.50/mo |
-| 50 GB | ~$0/mo | **~$112/mo** ⚠ |
+| 10 GB | $0/month (within 250 GB) | ~$4.50/month |
+| 50 GB | ~$0/month | **~$112/month** ⚠ |
 
 Cost scales with **size × frequency**. Run `backupdrill estimate` to project it for *your* project, and prefer weekly (or a smaller scope) once your data is large. The compressed dump is smaller than raw DB size, so real DB egress is below the `estimate` upper bound; Storage egress is exact.
 
@@ -146,6 +148,16 @@ backupdrill drill --verify-all-files # also checksum every Storage file, not jus
 
 Backups record your installed Postgres extensions in the manifest, and the drill pre-installs them in the sandbox before restoring — including switching to a pgvector-enabled sandbox image when your database uses `vector` columns.
 
+### Run your own checks against the restored copy
+
+The drill's structural checks (tables, rows, checksums) can't know what "working" means for *your* app — RLS behavior and business workflows only have answers in your code. `--check-cmd` hands you the restored sandbox before it's destroyed:
+
+```bash
+backupdrill drill --check-cmd "npm run smoke"   # BACKUPDRILL_SANDBOX_URL points at the restored copy
+```
+
+Your command runs after all structural checks pass, with `BACKUPDRILL_SANDBOX_URL` set to the sandbox's connection string. Exit 0 means pass; the result shows up in the report as an `app checks` line, and a non-zero exit fails the drill (the report keeps structural and app-level failures clearly separate). Entirely optional — without the flag, nothing changes and the report has no `app checks` line. There's a 10-minute timeout, and `--keep` leaves the sandbox container running when a drill fails so you can inspect it (`docker rm -f <id>` when done).
+
 When you actually need to recover, `restore` puts the database back into a target you name and pulls the Storage files down to a local folder:
 
 ```bash

package/dist/drill.d.ts

@@ -1,5 +1,13 @@
 import type { BackupConfig } from "./config.js";
 import type { Manifest } from "./manifest.js";
+/**
+ * 无放回随机抽样:部分 Fisher-Yates,只洗出前 n 位(O(n) 次交换,不必全 shuffle)。
+ * 此前恒取 manifest 顺序的前 CAP 个 → 超过 CAP 的项目里排序靠后的文件永远不被校验;
+ * 随机抽样让每次演练覆盖不同子集,损坏的文件长期必然被抽中。
+ * 随机源用 Math.random 足够:演练是一次性进程,无可复现性要求——校验失败的文件名
+ * 会原样列进报告,定位问题不依赖重放同一批样本。
+ */
+export declare function sampleStorageFiles<T>(files: T[], n: number): T[];
 export interface DrillCheck {
     name: string;
     pass: boolean;
@@ -25,14 +33,30 @@ export declare function classifyPostDataErrors(stderr: string): PostDataResult;
  * 归档器致命错误等)绝不能谎报"全部恢复"——记为通用失败。
  */
 export declare function postDataResult(code: number | null, stderr: string): PostDataResult;
+export declare const APP_CHECK_TIMEOUT_MS: number;
+/**
+ * 语义层钩子:结构检查全绿后、销毁沙箱前,把沙箱连接串经 BACKUPDRILL_SANDBOX_URL
+ * 交给用户命令执行(RLS 行为、业务流可用性——判据只存在于应用侧,引擎不解释
+ * 命令输出,只裁决退出码:0 = pass)。
+ * 子进程输出定向到 stderr:stdout 是 CLI 的机器可读 JSON 通道,不能被污染。
+ * 红线:托管 worker 绝不能把用户输入传到这里(任意代码执行);此钩子只属于
+ * 用户在自己机器上运行的 CLI(见 PRD §1.4 演练后应用自检钩子)。
+ */
+export declare function runAppCheck(command: string, connString: string, timeoutMs?: number): Promise<DrillCheck>;
 /**
  * 演练核心:把一份本地 dump 恢复进临时 Postgres 并对照 manifest 校验,然后销毁容器。
  * 与 S3 解耦,便于离线测试。`preChecks` 用于把下载阶段的校验(如归档校验和)带进报告。
  */
-export declare function drillDump(dumpPath: string, manifest: Manifest, snapshot: string, preChecks?: DrillCheck[]): Promise<DrillReport>;
+export declare function drillDump(dumpPath: string, manifest: Manifest, snapshot: string, preChecks?: DrillCheck[], opts?: {
+    appCheckCommand?: string;
+    keepSandboxOnFailure?: boolean;
+}): Promise<DrillReport>;
 /** 演练一份快照:从桶里下载→drillDump(含 Storage 校验)→报告。 */
 export declare function runDrill(config: BackupConfig, opts?: {
     snapshot?: string;
     verifyAllFiles?: boolean;
+    /** CLI 专用。红线:托管 worker 绝不能传入(在 worker 上执行用户命令 = RCE)。 */
+    appCheckCommand?: string;
+    keepSandboxOnFailure?: boolean;
 }): Promise<DrillReport>;
 export {};

package/dist/drill.js

@@ -10,6 +10,23 @@ import { log } from "./log.js";
 const execFileAsync = promisify(execFile);
 // Storage 校验默认抽样上限(避免每次演练重下全部文件产生大量 egress);超限如实记录
 const STORAGE_SAMPLE_CAP = 100;
+/**
+ * 无放回随机抽样:部分 Fisher-Yates,只洗出前 n 位(O(n) 次交换,不必全 shuffle)。
+ * 此前恒取 manifest 顺序的前 CAP 个 → 超过 CAP 的项目里排序靠后的文件永远不被校验;
+ * 随机抽样让每次演练覆盖不同子集,损坏的文件长期必然被抽中。
+ * 随机源用 Math.random 足够:演练是一次性进程,无可复现性要求——校验失败的文件名
+ * 会原样列进报告,定位问题不依赖重放同一批样本。
+ */
+export function sampleStorageFiles(files, n) {
+    if (files.length <= n)
+        return files;
+    const pool = [...files];
+    for (let i = 0; i < n; i++) {
+        const j = i + Math.floor(Math.random() * (pool.length - i));
+        [pool[i], pool[j]] = [pool[j], pool[i]];
+    }
+    return pool.slice(0, n);
+}
 /**
  * 沙箱镜像选择:alpine 镜像只带 contrib 扩展,装不了 pgvector——含 vector 列的
  * 备份在裸镜像里恢复必然硬崩("type vector does not exist")。manifest 记录了
@@ -264,7 +281,7 @@ async function verifyRestored(connString, manifest) {
  */
 async function verifyStorage(s3, config, manifest, verifyAll) {
     const files = manifest.storage.files;
-    const sample = verifyAll ? files : files.slice(0, STORAGE_SAMPLE_CAP);
+    const sample = verifyAll ? files : sampleStorageFiles(files, STORAGE_SAMPLE_CAP);
     // 备份写入时的键:<snapshot base>/storage/<bucket>/<key>,base 从 dump.key 反推
     const base = manifest.dump.key.replace(/\/dump\.pgcustom$/, "");
     const mismatches = [];
@@ -279,25 +296,75 @@ async function verifyStorage(s3, config, manifest, verifyAll) {
             mismatches.push(`${f.bucket}/${f.key} (unreadable)`);
         }
     }
-    const scope = verifyAll
-        ? `all ${files.length}`
-        : `${sample.length}/${files.length} sampled`;
+    // 抽样时如实说明这是随机样本(N of M);全量(--verify-all-files 或文件数 ≤ 上限)说 all
+    const scope = sample.length === files.length
+        ? `all ${files.length} files`
+        : `random sample of ${sample.length} of ${files.length} files`;
     return {
         name: "storage file integrity",
         pass: mismatches.length === 0,
         detail: mismatches.length === 0
-            ? `${scope} files match their manifest checksums`
-            : `${mismatches.length} of ${scope} failed: ${mismatches.slice(0, 5).join(", ")}`,
+            ? `${scope} match their manifest checksums`
+            : `${mismatches.length} failed in ${scope}: ${mismatches.slice(0, 5).join(", ")}`,
     };
 }
+// ── 应用自检钩子(语义层)────────────────────────────────────────────
+// 挂死的 smoke test 不能永远挡住沙箱销毁;10 分钟对"冒烟"级检查绰绰有余
+export const APP_CHECK_TIMEOUT_MS = 10 * 60 * 1000;
+/**
+ * 语义层钩子:结构检查全绿后、销毁沙箱前,把沙箱连接串经 BACKUPDRILL_SANDBOX_URL
+ * 交给用户命令执行(RLS 行为、业务流可用性——判据只存在于应用侧,引擎不解释
+ * 命令输出,只裁决退出码:0 = pass)。
+ * 子进程输出定向到 stderr:stdout 是 CLI 的机器可读 JSON 通道,不能被污染。
+ * 红线:托管 worker 绝不能把用户输入传到这里(任意代码执行);此钩子只属于
+ * 用户在自己机器上运行的 CLI(见 PRD §1.4 演练后应用自检钩子)。
+ */
+export function runAppCheck(command, connString, timeoutMs = APP_CHECK_TIMEOUT_MS) {
+    const started = Date.now();
+    return new Promise((resolve) => {
+        const proc = spawn(command, {
+            shell: true,
+            stdio: ["ignore", 2, 2],
+            env: { ...process.env, BACKUPDRILL_SANDBOX_URL: connString },
+            timeout: timeoutMs,
+            killSignal: "SIGKILL",
+        });
+        proc.on("error", (error) => {
+            resolve({
+                name: "app checks",
+                pass: false,
+                detail: `command failed to start: ${error.message}`,
+            });
+        });
+        proc.on("close", (code, signal) => {
+            const seconds = Math.round((Date.now() - started) / 100) / 10;
+            if (signal) {
+                resolve({
+                    name: "app checks",
+                    pass: false,
+                    detail: `terminated by ${signal} after ${seconds}s (timeout ${timeoutMs / 1000}s)`,
+                });
+                return;
+            }
+            resolve({
+                name: "app checks",
+                pass: code === 0,
+                detail: code === 0
+                    ? `command exited 0 in ${seconds}s`
+                    : `command exited ${code} after ${seconds}s`,
+            });
+        });
+    });
+}
 // ── 主流程 ──────────────────────────────────────────────────────────
 /**
  * 演练核心:把一份本地 dump 恢复进临时 Postgres 并对照 manifest 校验,然后销毁容器。
  * 与 S3 解耦,便于离线测试。`preChecks` 用于把下载阶段的校验(如归档校验和)带进报告。
  */
-export async function drillDump(dumpPath, manifest, snapshot, preChecks = []) {
+export async function drillDump(dumpPath, manifest, snapshot, preChecks = [], opts = {}) {
     const checks = [...preChecks];
     const pg = await startEphemeralPostgres(sandboxImage(manifest));
+    let pass = false; // 异常路径视为失败,供 finally 决定 --keep 是否保留沙箱
     try {
         // 旧 manifest(≤0.1.1)没有 extensions 字段 → 不装任何扩展,行为与从前一致
         const extensions = manifest.database.extensions ?? [];
@@ -345,9 +412,16 @@ export async function drillDump(dumpPath, manifest, snapshot, preChecks = []) {
         });
         const verify = await verifyRestored(pg.connString, manifest);
         checks.push(...verify.checks);
+        // 语义层钩子:只在结构层全绿时才跑——结构已失败时再跑应用检查,只会把
+        // 一次失败报成两层失败,误导排查方向。没配置则不跑、报告里不出现该行。
+        if (opts.appCheckCommand && checks.every((c) => c.pass)) {
+            log.step("Running app checks…");
+            checks.push(await runAppCheck(opts.appCheckCommand, pg.connString));
+        }
+        pass = checks.every((c) => c.pass);
         return {
             snapshot,
-            pass: checks.every((c) => c.pass),
+            pass,
             checks,
             restoreSeconds,
             restoredTableCount: verify.tableCount,
@@ -355,8 +429,15 @@ export async function drillDump(dumpPath, manifest, snapshot, preChecks = []) {
         };
     }
     finally {
-        log.step("Tearing down ephemeral Postgres…");
-        await execFileAsync("docker", ["rm", "-f", pg.containerId]).catch(() => { });
+        if (opts.keepSandboxOnFailure && !pass) {
+            // --rm 容器不 docker rm 就一直活着;用户排查完 stop/rm 时 --rm 自动清理
+            log.warn(`--keep: sandbox left running for inspection — ${pg.connString} ` +
+                `(remove with: docker rm -f ${pg.containerId.slice(0, 12)})`);
+        }
+        else {
+            log.step("Tearing down ephemeral Postgres…");
+            await execFileAsync("docker", ["rm", "-f", pg.containerId]).catch(() => { });
+        }
     }
 }
 /** 演练一份快照:从桶里下载→drillDump(含 Storage 校验)→报告。 */
@@ -397,7 +478,10 @@ export async function runDrill(config, opts = {}) {
             log.step("Verifying Storage files…");
             preChecks.push(await verifyStorage(s3, config, manifest, opts.verifyAllFiles ?? false));
         }
-        return await drillDump(dumpPath, manifest, snapshot, preChecks);
+        return await drillDump(dumpPath, manifest, snapshot, preChecks, {
+            appCheckCommand: opts.appCheckCommand,
+            keepSandboxOnFailure: opts.keepSandboxOnFailure,
+        });
     }
     finally {
         await rm(workdir, { recursive: true, force: true }).catch(() => { });

package/dist/estimate.d.ts

@@ -1,6 +1,6 @@
 import type { BackupConfig } from "./config.js";
 /**
- * Supabase egress 定价(2026-07-04 经核实,来源见 README「Egress & cost」)。
+ * Supabase egress 定价(2026-07-10 经核实,来源见 README「Egress & cost」)。
  * 单价 = 超出计划包含额度后的每 GB 价;included = 各计划每月包含的 egress。
  * 备份 = 把数据拉出 Supabase = egress,计入用户账单。这是 PRD §4.3 的上线否决项:
  * 让用户"我们帮你算过账"而不是收到意外账单。

package/dist/estimate.js

@@ -59,10 +59,10 @@ export async function runEstimate(config, pricing) {
     console.error(`  Plan: ${pricing.planLabel} — ${pricing.includedGb} GB egress included, ` +
         `then $${pricing.pricePerGb}/GB`);
     console.error("");
-    console.error("  Frequency   Runs/mo   Monthly egress   Overage   Est. cost/mo");
-    console.error("  ─────────   ───────   ──────────────   ───────   ────────────");
+    console.error("  Frequency   Runs/month   Monthly egress   Overage   Est. cost/month");
+    console.error("  ─────────   ──────────   ──────────────   ───────   ───────────────");
     for (const r of projection.rows) {
-        console.error(`  ${r.frequency.padEnd(9)}   ${String(r.runsPerMonth).padStart(7)}   ` +
+        console.error(`  ${r.frequency.padEnd(9)}   ${String(r.runsPerMonth).padStart(10)}   ` +
             `${(r.monthlyEgressGb + " GB").padStart(14)}   ` +
             `${(r.overageGb + " GB").padStart(7)}   ` +
             `$${r.monthlyCost.toFixed(2)}`);
@@ -76,11 +76,13 @@ export async function runEstimate(config, pricing) {
     const weekly = projection.rows.find((r) => r.frequency === "weekly");
     console.error("");
     if (pricing.hardCap) {
-        // Free:超额=限流,备份直接失败,daily 大库根本跑不完
+        // Free:超额不计费,走 Fair Use 通知 → 宽限期 → 限制;持续超额的大备份在 Free 跑不通
         if (daily && daily.overageGb > 0) {
-            log.warn(`${pricing.planLabel} egress is a HARD CAP (${pricing.includedGb} GB). ` +
-                `Daily backups (~${daily.monthlyEgressGb} GB/mo) will blow the cap and ` +
-                `get your project throttled — the backup itself will fail. ` +
+            log.warn(`${pricing.planLabel} plan egress is never billed, but exceeding the included ` +
+                `${pricing.includedGb} GB/month starts Supabase's Fair Use process: a ` +
+                `notification, then a grace period, then egress restrictions. Daily backups ` +
+                `(~${daily.monthlyEgressGb} GB/month) would keep you permanently over the ` +
+                `line — sustained large backups are not viable on Free. ` +
                 `Use weekly at most, or upgrade to Pro.`);
         }
     }
@@ -93,9 +95,9 @@ export async function runEstimate(config, pricing) {
             "numbers above are what you'd pay.");
         const worst = daily?.monthlyCost ?? 0;
         if (worst >= SUBSCRIPTION_FLOOR_USD) {
-            log.warn(`At daily frequency the egress overage alone (~$${worst.toFixed(0)}/mo) is ` +
+            log.warn(`At daily frequency the egress overage alone (~$${worst.toFixed(0)}/month) is ` +
                 `in the range of a paid backup subscription. Consider weekly ` +
-                `(~$${(weekly?.monthlyCost ?? 0).toFixed(0)}/mo overage) or a smaller scope.`);
+                `(~$${(weekly?.monthlyCost ?? 0).toFixed(0)}/month overage) or a smaller scope.`);
         }
     }
     return projection;

package/dist/index.js

@@ -7,9 +7,10 @@ import { runDrill } from "./drill.js";
 import { runRestore } from "./restore.js";
 import { log } from "./log.js";
 import { TOOL_VERSION } from "./version.js";
-// Supabase egress 定价表(2026-07-04 核实,来源见 README「Egress & cost」)。
+// Supabase egress 定价表(2026-07-10 核实,来源见 README「Egress & cost」)。
 // 备份流量 100% 未缓存,故用 uncached 单价 $0.09/GB;included 用 uncached 额度。
-// Free 硬上限(超额即限流,不计费);Pro/Team 由 Spend Cap 决定超额限流还是计费。
+// Free 永不计费:超额走 Fair Use 流程(通知 → 宽限期 → 限制),持续大备份不可行;
+// Pro/Team 由 Spend Cap 决定超额限流还是计费。
 const EGRESS_PRICING = {
     free: { planLabel: "Free", includedGb: 5, pricePerGb: 0.09, hardCap: true },
     pro: { planLabel: "Pro", includedGb: 250, pricePerGb: 0.09, hardCap: false },
@@ -74,6 +75,9 @@ program
     .option("--prefix <prefix>", "key prefix inside the bucket (default: backupdrill)")
     .option("--project-name <name>", "project name used in the object key")
     .option("--verify-all-files", "checksum every Storage file, not just a sample")
+    .option("--check-cmd <command>", "after structural checks pass, run your own smoke test against the restored sandbox " +
+    "(connection string in BACKUPDRILL_SANDBOX_URL; exit 0 = pass, reported as 'app checks')")
+    .option("--keep", "if the drill fails, keep the sandbox container running for inspection")
     .action(async (opts) => {
     try {
         const config = await loadConfig({
@@ -93,6 +97,8 @@ program
         const report = await runDrill(config, {
             snapshot: opts.snapshot,
             verifyAllFiles: opts.verifyAllFiles,
+            appCheckCommand: opts.checkCmd,
+            keepSandboxOnFailure: opts.keep,
         });
         console.error("");
         for (const c of report.checks) {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "backupdrill",
-  "version": "0.1.2",
+  "version": "0.2.0",
   "description": "Open-source CLI to back up a Supabase project — Postgres database and Storage files — to your own S3/R2/B2 bucket, with a checksummed manifest.",
   "type": "module",
   "bin": {