backupdrill 2.5.0 → 2.6.0

package/README.md

@@ -158,7 +158,9 @@ backupdrill estimate --plan pro   # measures your DB + Storage, prints the proje
 <prefix>/<project>/<timestamp>/
   ├── dump.pgcustom              # pg_dump --format=custom (public schema by default)
   ├── storage/<bucket>/<key>     # your Storage files (if configured)
+  ├── RECOVERY.md                # how to restore this snapshot
   └── manifest.json              # schemas, tables, row counts, file list, sizes + sha256
+<prefix>/<project>.drills/<finished-at>-<passed|failed>-<tag>.json  # only with drill --record: one file per drill
 ```
 
 ## Restoring & drilling
@@ -261,6 +263,80 @@ Both pings of a run carry the same `rid`, so Healthchecks pairs them even if two
 
 A ping never changes the result of a run. Each request times out after 10 seconds and is retried twice on network errors and 5xx responses; if it still fails, the CLI prints a warning and carries on, with the same exit code it would have had without a ping URL.
 
+## Prometheus metrics
+
+`backupdrill exporter` serves [Prometheus](https://prometheus.io) metrics about the backups in your bucket, and about the restore drills you run with `--record`. It reads the bucket each time Prometheus scrapes it and never writes, so a key with read-only access to the bucket is enough.
+
+```bash
+# reads the same BACKUPDRILL_S3_* and BACKUPDRILL_PROJECT_NAME settings as backup and drill
+backupdrill exporter                  # http://127.0.0.1:10059/metrics
+backupdrill exporter --host 0.0.0.0   # when Prometheus runs on another machine
+```
+
+It listens on `127.0.0.1` unless you pass `--host`, because every scrape costs a few requests against your bucket and the port shouldn't end up open to the internet by accident. `--port` changes the port.
+
+Drill results only show up if the drill leaves them in the bucket. Add `--record` to your drill job: when the drill finishes, it writes one small JSON file to `<prefix>/<project>.drills/`, named after the time it finished, whether it passed and a random tag (`2026-10-20T04-12-09-311Z-passed-3f9a1c07.json`). Each drill writes a file under its own name instead of updating a shared one, so the order in which two overlapping drills finish writing doesn't matter. Only the outcome, the time, the snapshot name and the restore time are stored, never the check details. A drill that stops early (Docker isn't running, the bucket is empty) is recorded as not passed. `--record` needs write access to the bucket; if the write fails, the drill prints a warning and its own result and exit code stay the same. Without `--record`, `drill` only reads the bucket, as before.
+
+```bash
+backupdrill drill --record
+```
+
+Point Prometheus at it. One exporter watches one project. The exporter doesn't add a project label, so set one in the scrape config:
+
+```yaml
+scrape_configs:
+  - job_name: backupdrill
+    scrape_interval: 1m
+    static_configs:
+      - targets: ["backup-host:10059"]
+        labels:
+          project: my-app
+```
+
+For a second project, run a second exporter with `--project-name` and a different `--port`.
+
+| Metric | Meaning |
+|---|---|
+| `backupdrill_up` | 1 if the bucket was read, 0 if not (the reason goes to the exporter's log) |
+| `backupdrill_scrape_duration_seconds` | How long reading the bucket took |
+| `backupdrill_build_info{version}` | CLI version, always 1 |
+| `backupdrill_snapshots` | Snapshot folders for the project, complete or not |
+| `backupdrill_backup_last_success_timestamp_seconds` | When the newest complete snapshot finished (`createdAt` in its `manifest.json`); 0 if there is none |
+| `backupdrill_latest_snapshot_dump_bytes` | Size of the database dump in that snapshot |
+| `backupdrill_latest_snapshot_storage_files`, `_storage_bytes` | Storage files in that snapshot and their total size; 0 for a database-only backup |
+| `backupdrill_latest_snapshot_tables`, `_estimated_rows` | Tables captured, and the row count Postgres estimated at backup time |
+| `backupdrill_drill_last_run_timestamp_seconds` | When the last recorded drill finished, passed or not; 0 if none |
+| `backupdrill_drill_last_run_passed` | 1 if that drill passed, 0 if not |
+| `backupdrill_drill_last_success_timestamp_seconds` | When the last recorded drill that passed finished; 0 if none |
+| `backupdrill_drill_last_success_restore_duration_seconds` | How long the database restore took in that drill |
+
+A snapshot counts as complete once its `manifest.json` exists, because the backup writes it last. While there is no complete snapshot, the `latest_snapshot` metrics are left out. `backupdrill_drill_last_run_passed` appears once a drill has been recorded, and the restore duration once a recorded drill has passed. A manifest dated more than 10 minutes in the future counts as a failed read (`backupdrill_up 0`), and a drill file named with such a time is skipped with a warning in the log: either would hold back the alerts that work out an age from that timestamp. Check the clock on the machine that wrote it.
+
+Alert rules to start from. Save them as `backupdrill-alerts.yml`, list that file under `rule_files` in `prometheus.yml` and reload Prometheus; `promtool check rules` passes on them:
+
+```yaml
+groups:
+  - name: backupdrill
+    rules:
+      - alert: BackupdrillBucketUnreadable
+        expr: backupdrill_up == 0
+        for: 15m
+      - alert: SupabaseBackupMissing
+        # daily backups; use 8 * 86400 for weekly
+        expr: time() - backupdrill_backup_last_success_timestamp_seconds > 26 * 3600
+        for: 15m
+      - alert: SupabaseBackupShrank
+        expr: backupdrill_latest_snapshot_dump_bytes < 0.7 * max_over_time(backupdrill_latest_snapshot_dump_bytes[7d])
+      - alert: SupabaseRestoreDrillFailed
+        expr: backupdrill_drill_last_run_passed == 0
+      - alert: SupabaseRestoreDrillMissing
+        # weekly drills
+        expr: time() - backupdrill_drill_last_success_timestamp_seconds > 8 * 86400
+        for: 1h
+```
+
+What a scrape costs: one list request per 1,000 snapshot folders and one per 1,000 drill files, plus reads of the newest `manifest.json` and the newest passing drill file. The manifest is fetched with `If-None-Match`, so when it hasn't changed the bucket answers `304 Not Modified` and nothing is downloaded. That matters once you back up Storage files, since the manifest lists every file. At a one-minute interval this comes to about 86,000 list requests and 86,000 small reads a month; check your storage provider's request pricing. Each read of the bucket stops half a second before Prometheus's scrape timeout (10 seconds if there's no Prometheus header), so a slow bucket shows up as `backupdrill_up 0` instead of a failed scrape.
+
 ## Roadmap
 
 - [x] Database backup → your bucket, with checksummed manifest
@@ -270,6 +346,7 @@ A ping never changes the result of a run. Each request times out after 10 second
 - [x] Storage-file integrity check inside the drill (checksums, sampled by default)
 - [x] `restore` — recover a snapshot into a fresh Supabase project: database through the drill-grade engine with verification, Storage uploaded back with per-file reconciliation (local download remains an option)
 - [x] GitHub Action wrapper
+- [x] `exporter` — Prometheus metrics for backups and recorded drills
 
 For scheduling, automated restore drills, and alerting, use the hosted service at [backupdrill.com](https://backupdrill.com).
 
@@ -278,9 +355,10 @@ For scheduling, automated restore drills, and alerting, use the hosted service a
 `backupdrill` follows [semantic versioning](https://semver.org/). From **1.0.0** onwards the
 following is the stable public surface — breaking it requires a major version bump:
 
-- the commands (`backup`, `drill`, `restore`, `estimate`) and their flags
+- the commands (`backup`, `drill`, `restore`, `estimate`, `exporter`) and their flags
 - the `BACKUPDRILL_*` environment variables and the config-file keys
-- the layout written to your bucket, and the shape of `manifest.json`
+- the layout written to your bucket, and the shape of `manifest.json` and the drill record files
+- the names and meaning of the metrics `exporter` serves
 
 `manifest.json` carries a `schemaVersion` so the format can evolve safely. A snapshot written by
 a newer release is **refused with a clear message** rather than parsed on a best-effort basis —

package/dist/deadline.d.ts

@@ -0,0 +1,9 @@
+/**
+ * 到点就放弃,不管 run 停没停。只把 AbortSignal 交给 SDK 不够:它在两次重试之间按
+ * Retry-After 睡觉时不看 signal,503 带 Retry-After: 2 就能把 1 秒的时限拖到 2 秒多。
+ * signal 仍然传下去;结束时(成功、失败、超时)一律 abort,让并行中还没回来的请求收手,
+ * 不然一个先失败、一个挂着的 Promise.all 会把挂着的那个连接一直留着。
+ */
+export declare function withDeadline<T>(run: (signal: AbortSignal) => Promise<T>, ms: number): Promise<T>;
+/** 超时类错误(withDeadline 的、AbortSignal 的)统一认出来,好给一句人话 */
+export declare function isTimeout(error: unknown): boolean;

package/dist/deadline.js

@@ -0,0 +1,29 @@
+/**
+ * 到点就放弃,不管 run 停没停。只把 AbortSignal 交给 SDK 不够:它在两次重试之间按
+ * Retry-After 睡觉时不看 signal,503 带 Retry-After: 2 就能把 1 秒的时限拖到 2 秒多。
+ * signal 仍然传下去;结束时(成功、失败、超时)一律 abort,让并行中还没回来的请求收手,
+ * 不然一个先失败、一个挂着的 Promise.all 会把挂着的那个连接一直留着。
+ */
+export async function withDeadline(run, ms) {
+    const controller = new AbortController();
+    let timer;
+    const deadline = new Promise((_, reject) => {
+        timer = setTimeout(() => {
+            const error = new Error(`timed out after ${ms}ms`);
+            error.name = "TimeoutError";
+            reject(error);
+        }, ms);
+    });
+    try {
+        return await Promise.race([run(controller.signal), deadline]);
+    }
+    finally {
+        clearTimeout(timer);
+        controller.abort();
+    }
+}
+/** 超时类错误(withDeadline 的、AbortSignal 的)统一认出来,好给一句人话 */
+export function isTimeout(error) {
+    const name = error?.name;
+    return name === "TimeoutError" || name === "AbortError";
+}

package/dist/drill-record.d.ts

@@ -0,0 +1,53 @@
+import { type S3Client } from "@aws-sdk/client-s3";
+import { type BackupConfig } from "./config.js";
+import type { DrillReport } from "./drill.js";
+/**
+ * `drill --record` 写进桶里的演练结论。`exporter` 只读桶,演练结果不落桶它就无从得知。
+ *
+ * 每次演练写一个新对象,从不覆盖:
+ *   <prefix>/<project>.drills/<结束时间>-<passed|failed>-<8 位随机>.json
+ * 为什么不是一份可覆盖的"最近一次":两次演练重叠时,先结束的那次可能后写进桶,把更新的
+ * "没通过"盖成更早的"通过",失败告警就没了;先读后写也挡不住(不是原子的),而各家
+ * S3 兼容存储对条件写入的支持参差不齐。每次一个对象,"哪次最新"留给读的一方按名字判断,
+ * 写入顺序就无所谓了。
+ *
+ * 为什么放在 <project>.drills/,和快照目录并排而不是在它里面:resolveSnapshot(包括已经发出去的
+ * 旧版本)按字典序取 <project>/ 下最大的目录当最新快照,里面多一个 drills/ 会被当成快照。
+ *
+ * 文件名本身带着结束时间和结论,exporter 列一下目录就知道最近一次和最近一次通过是哪个;
+ * 只为了恢复用时才读一次通过的那份。只记结论,不记检查明细:失败明细里可能有 pg_restore
+ * 回显的表数据,而桶的读权限可能比演练日志给得更宽(exporter 只要只读 key)。
+ */
+/** 版本规则同 manifest:缺省不接受,更高的版本明确报错让用户升级,不按旧结构硬读。 */
+export declare const DRILL_RECORD_SCHEMA_VERSION = 1;
+export interface DrillRecord {
+    schemaVersion: number;
+    tool: "backupdrill-cli";
+    toolVersion: string;
+    /** 演练结束的时间(ISO8601),与文件名里的时间相同 */
+    finishedAt: string;
+    pass: boolean;
+    /** 演练的快照名;演练在选定快照之前就失败了(没装 Docker、桶里没有快照)时没有 */
+    snapshot?: string;
+    /** 恢复用时,只有通过的演练才记:没通过时这个数可能只是半程,拿来估恢复时间会误导 */
+    restoreSeconds?: number;
+}
+/** 演练记录所在的"目录",带尾斜杠 */
+export declare function drillRecordsPrefix(config: BackupConfig): string;
+export declare function drillRecordName(record: Pick<DrillRecord, "finishedAt" | "pass">, unique?: string): string;
+/** 从文件名读出结束时间(毫秒)和结论;不是记录文件的名字返回 null */
+export declare function parseDrillRecordName(name: string): {
+    finishedAt: number;
+    pass: boolean;
+} | null;
+/** report 为 null = 演练中途抛错、没有报告;这也算一次没通过的演练。 */
+export declare function buildDrillRecord(report: DrillReport | null, finishedAt: Date): DrillRecord;
+/** 写一个新对象,返回它的 key。不读桶:只需要写权限 */
+export declare function writeDrillRecord(s3: S3Client, config: BackupConfig, record: DrillRecord, abortSignal?: AbortSignal): Promise<string>;
+/**
+ * CLI 的 `drill --record`:写不进去只警告,演练本身的结论和退出码不变。
+ * 常见原因是给演练配的是只读 key;这种情况下 exporter 的演练时间会一直不动,告警规则会发现。
+ */
+export declare function recordDrillOutcome(config: BackupConfig, report: DrillReport | null, s3?: S3Client, timeoutMs?: number): Promise<void>;
+/** 读桶里的演练记录。结构不对就报错:exporter 宁可报 up=0,也不报一个读错的结论。 */
+export declare function parseDrillRecord(json: string, file: string): DrillRecord;

package/dist/drill-record.js

@@ -0,0 +1,126 @@
+import { randomBytes } from "node:crypto";
+import { PutObjectCommand } from "@aws-sdk/client-s3";
+import { requireBucket } from "./config.js";
+import { isTimeout, withDeadline } from "./deadline.js";
+import { log } from "./log.js";
+import { targetClient } from "./snapshots.js";
+import { TOOL_VERSION } from "./version.js";
+/**
+ * `drill --record` 写进桶里的演练结论。`exporter` 只读桶,演练结果不落桶它就无从得知。
+ *
+ * 每次演练写一个新对象,从不覆盖:
+ *   <prefix>/<project>.drills/<结束时间>-<passed|failed>-<8 位随机>.json
+ * 为什么不是一份可覆盖的"最近一次":两次演练重叠时,先结束的那次可能后写进桶,把更新的
+ * "没通过"盖成更早的"通过",失败告警就没了;先读后写也挡不住(不是原子的),而各家
+ * S3 兼容存储对条件写入的支持参差不齐。每次一个对象,"哪次最新"留给读的一方按名字判断,
+ * 写入顺序就无所谓了。
+ *
+ * 为什么放在 <project>.drills/,和快照目录并排而不是在它里面:resolveSnapshot(包括已经发出去的
+ * 旧版本)按字典序取 <project>/ 下最大的目录当最新快照,里面多一个 drills/ 会被当成快照。
+ *
+ * 文件名本身带着结束时间和结论,exporter 列一下目录就知道最近一次和最近一次通过是哪个;
+ * 只为了恢复用时才读一次通过的那份。只记结论,不记检查明细:失败明细里可能有 pg_restore
+ * 回显的表数据,而桶的读权限可能比演练日志给得更宽(exporter 只要只读 key)。
+ */
+/** 版本规则同 manifest:缺省不接受,更高的版本明确报错让用户升级,不按旧结构硬读。 */
+export const DRILL_RECORD_SCHEMA_VERSION = 1;
+/** 演练记录所在的"目录",带尾斜杠 */
+export function drillRecordsPrefix(config) {
+    const storage = requireBucket(config);
+    return `${storage.prefix}/${config.projectName}.drills/`;
+}
+/**
+ * 时间戳写法同快照目录名(把 ISO 里的 : 和 . 换成 -),定长,按名字排序就是按时间排序。
+ * 末尾的随机段:两次演练在同一毫秒结束、结论又相同时,只靠时间和结论会撞名、后写的覆盖先写的
+ */
+const RECORD_NAME = /^(\d{4}-\d{2}-\d{2})T(\d{2})-(\d{2})-(\d{2})-(\d{3})Z-(passed|failed)-[0-9a-f]{8}\.json$/;
+export function drillRecordName(record, unique = randomBytes(4).toString("hex")) {
+    return `${record.finishedAt.replace(/[:.]/g, "-")}-${record.pass ? "passed" : "failed"}-${unique}.json`;
+}
+/** 从文件名读出结束时间(毫秒)和结论;不是记录文件的名字返回 null */
+export function parseDrillRecordName(name) {
+    const m = RECORD_NAME.exec(name);
+    if (!m)
+        return null;
+    const finishedAt = Date.parse(`${m[1]}T${m[2]}:${m[3]}:${m[4]}.${m[5]}Z`);
+    return Number.isFinite(finishedAt) ? { finishedAt, pass: m[6] === "passed" } : null;
+}
+/** report 为 null = 演练中途抛错、没有报告;这也算一次没通过的演练。 */
+export function buildDrillRecord(report, finishedAt) {
+    return {
+        schemaVersion: DRILL_RECORD_SCHEMA_VERSION,
+        tool: "backupdrill-cli",
+        toolVersion: TOOL_VERSION,
+        finishedAt: finishedAt.toISOString(),
+        pass: report?.pass === true,
+        ...(report ? { snapshot: report.snapshot } : {}),
+        ...(report?.pass ? { restoreSeconds: report.restoreSeconds } : {}),
+    };
+}
+/** 写一个新对象,返回它的 key。不读桶:只需要写权限 */
+export async function writeDrillRecord(s3, config, record, abortSignal) {
+    const key = drillRecordsPrefix(config) + drillRecordName(record);
+    await s3.send(new PutObjectCommand({
+        Bucket: requireBucket(config).bucket,
+        Key: key,
+        Body: JSON.stringify(record, null, 2),
+        ContentType: "application/json",
+    }), { abortSignal });
+    return key;
+}
+/** 记一笔最多等这么久:桶接了请求却不回应时,演练不能卡在这里,后面还要退出、发监控 ping */
+const RECORD_TIMEOUT_MS = 30_000;
+/**
+ * CLI 的 `drill --record`:写不进去只警告,演练本身的结论和退出码不变。
+ * 常见原因是给演练配的是只读 key;这种情况下 exporter 的演练时间会一直不动,告警规则会发现。
+ */
+export async function recordDrillOutcome(config, report, s3 = targetClient(config), timeoutMs = RECORD_TIMEOUT_MS) {
+    const record = buildDrillRecord(report, new Date());
+    try {
+        const key = await withDeadline((signal) => writeDrillRecord(s3, config, record, signal), timeoutMs);
+        log.ok(`Drill result recorded → s3://${requireBucket(config).bucket}/${key}`);
+    }
+    catch (error) {
+        const reason = isTimeout(error)
+            ? `no answer from the bucket within ${timeoutMs / 1000}s`
+            : error.message;
+        log.warn(`Could not record the drill result (${reason}). ` +
+            `--record needs write access to the bucket. The drill's own result is unchanged.`);
+    }
+}
+function malformed(file, what) {
+    throw new Error(`${file} is malformed: ${what}`);
+}
+/** 读桶里的演练记录。结构不对就报错:exporter 宁可报 up=0,也不报一个读错的结论。 */
+export function parseDrillRecord(json, file) {
+    let parsed;
+    try {
+        parsed = JSON.parse(json);
+    }
+    catch (error) {
+        throw new Error(`${file} is not valid JSON: ${error.message}`);
+    }
+    if (typeof parsed !== "object" || parsed === null || Array.isArray(parsed)) {
+        malformed(file, "root is not an object");
+    }
+    const r = parsed;
+    if (typeof r.schemaVersion !== "number" || !Number.isInteger(r.schemaVersion) || r.schemaVersion < 1) {
+        malformed(file, `schemaVersion ${JSON.stringify(r.schemaVersion)}`);
+    }
+    if (r.schemaVersion > DRILL_RECORD_SCHEMA_VERSION) {
+        throw new Error(`${file} uses schema version ${r.schemaVersion}, but this BackupDrill understands up to ` +
+            `${DRILL_RECORD_SCHEMA_VERSION}. Upgrade the CLI (npm i -g backupdrill@latest) to read it.`);
+    }
+    if (typeof r.finishedAt !== "string" || !Number.isFinite(Date.parse(r.finishedAt))) {
+        malformed(file, "finishedAt");
+    }
+    if (typeof r.pass !== "boolean")
+        malformed(file, "pass");
+    if (r.snapshot !== undefined && typeof r.snapshot !== "string")
+        malformed(file, "snapshot");
+    if (r.restoreSeconds !== undefined &&
+        (typeof r.restoreSeconds !== "number" || !Number.isFinite(r.restoreSeconds) || r.restoreSeconds < 0)) {
+        malformed(file, "restoreSeconds");
+    }
+    return r;
+}

package/dist/exporter.d.ts

@@ -0,0 +1,57 @@
+import { type Server } from "node:http";
+import { type S3Client } from "@aws-sdk/client-s3";
+import { type BackupConfig } from "./config.js";
+/**
+ * `backupdrill exporter`:按 Prometheus 文本格式暴露桶里备份与演练的状态。
+ * 规矩照 https://prometheus.io/docs/instrumenting/writing_exporters/ :
+ *  - 每次抓取现读桶,不按自己的定时器预取,不报旧值;
+ *  - 读不了桶时照样回 200,backupdrill_up 为 0,其余业务指标不出(半套数据比没有更误导);
+ *  - 指标不带标签(一个 exporter 进程只看一个项目,项目名属于 Prometheus 那边的 target 标签),
+ *    连接串、key、桶名一律不进指标。
+ */
+/** 10059:Prometheus wiki「Default port allocations」上 10058 之后的第一个空位(2026-10-20 查)。 */
+export declare const DEFAULT_EXPORTER_PORT = 10059;
+/** 最新一份完整快照里 exporter 要用的数,不留整份 manifest(Storage 文件多时 manifest 很大) */
+export interface SnapshotSummary {
+    createdAt: number;
+    dumpBytes: number;
+    storageFiles: number;
+    storageBytes: number;
+    tables: number;
+    estimatedRows: number;
+}
+export interface BucketState {
+    /** <prefix>/<project>/ 下的快照目录数,含没写完 manifest 的 */
+    snapshots: number;
+    /** 最新一份有 manifest.json 的快照;一份都没有时为 null */
+    latest: SnapshotSummary | null;
+    /** 最近一次记下的演练(drill --record);没有时为 null */
+    lastDrill: {
+        finishedAt: number;
+        pass: boolean;
+    } | null;
+    /** 最近一次通过的演练;restoreSeconds 来自它的记录内容 */
+    lastPassedDrill: {
+        finishedAt: number;
+        restoreSeconds?: number;
+    } | null;
+}
+/**
+ * 读桶的函数。每次调用都现读;唯一记住的是最新一份 manifest 的 ETag 和摘要:
+ * 下次带 If-None-Match 去问,没变就是 304、不传正文。快照写完就不再改,而一份 manifest 列着
+ * 每个 Storage 文件,可能有几十 MB,每 30 秒整份下载一遍会变成用户账单上的流量。
+ */
+export declare function createBucketReader(s3: S3Client, config: BackupConfig): (abortSignal: AbortSignal) => Promise<BucketState>;
+/** state 为 null = 这次没读成桶。全部是 gauge:都是"某一刻的状态",没有一个会只增不减。 */
+export declare function renderMetrics(state: BucketState | null, scrapeSeconds: number): string;
+/**
+ * 读桶的时限跟着 Prometheus 走:它在请求头里告诉我们这次抓取最多等多久,留半秒余量,
+ * 好让超时的这次还能回一个 up=0,而不是被 Prometheus 当成 exporter 挂了。
+ */
+export declare function scrapeTimeoutMs(header: string | string[] | undefined): number;
+export declare function createExporterServer(read: (abortSignal: AbortSignal) => Promise<BucketState>): Server;
+/** CLI 的 `backupdrill exporter`:起 HTTP 服务,直到进程被停掉。 */
+export declare function startExporter(config: BackupConfig, opts: {
+    host: string;
+    port: number;
+}): Promise<Server>;

package/dist/exporter.js

@@ -0,0 +1,305 @@
+import { createServer } from "node:http";
+import { performance } from "node:perf_hooks";
+import { GetObjectCommand, ListObjectsV2Command } from "@aws-sdk/client-s3";
+import { requireBucket } from "./config.js";
+import { withDeadline } from "./deadline.js";
+import { drillRecordsPrefix, parseDrillRecord, parseDrillRecordName } from "./drill-record.js";
+import { log } from "./log.js";
+import { parseManifest } from "./manifest.js";
+import { listSnapshotPrefixes, targetClient } from "./snapshots.js";
+import { TOOL_VERSION } from "./version.js";
+/**
+ * `backupdrill exporter`:按 Prometheus 文本格式暴露桶里备份与演练的状态。
+ * 规矩照 https://prometheus.io/docs/instrumenting/writing_exporters/ :
+ *  - 每次抓取现读桶,不按自己的定时器预取,不报旧值;
+ *  - 读不了桶时照样回 200,backupdrill_up 为 0,其余业务指标不出(半套数据比没有更误导);
+ *  - 指标不带标签(一个 exporter 进程只看一个项目,项目名属于 Prometheus 那边的 target 标签),
+ *    连接串、key、桶名一律不进指标。
+ */
+/** 10059:Prometheus wiki「Default port allocations」上 10058 之后的第一个空位(2026-10-20 查)。 */
+export const DEFAULT_EXPORTER_PORT = 10059;
+/** 没有 Prometheus 超时头时(手动 curl)给读桶留的时间,与 Prometheus 默认 scrape_timeout 一致 */
+const DEFAULT_SCRAPE_TIMEOUT_MS = 10_000;
+const MAX_SCRAPE_TIMEOUT_MS = 5 * 60_000;
+/**
+ * 记录里的时间最多可以比这台机器的时钟快多少。备份、演练、exporter 可能在不同机器上跑,
+ * 几分钟的时钟差是正常的;再多就不可信。
+ */
+const CLOCK_SKEW_TOLERANCE_MS = 10 * 60_000;
+function httpStatusOf(error) {
+    return error.$metadata?.httpStatusCode;
+}
+async function readObject(s3, bucket, key, abortSignal, ifNoneMatch) {
+    try {
+        const res = await s3.send(new GetObjectCommand({ Bucket: bucket, Key: key, IfNoneMatch: ifNoneMatch }), { abortSignal });
+        return { status: "ok", text: await res.Body.transformToString(), etag: res.ETag };
+    }
+    catch (error) {
+        const status = httpStatusOf(error);
+        if (status === 304)
+            return { status: "not-modified" };
+        if (status === 404 || error.name === "NoSuchKey")
+            return { status: "missing" };
+        throw error;
+    }
+}
+function summarize(text, key) {
+    const manifest = parseManifest(text);
+    // parseManifest 不校验 createdAt(演练和恢复用不到),这里要拿它当备份时间,自己把关
+    const createdAt = Date.parse(manifest.createdAt);
+    if (!Number.isFinite(createdAt)) {
+        throw new Error(`${key} is malformed: createdAt`);
+    }
+    assertNotInFuture(createdAt, key, "createdAt");
+    const summary = {
+        createdAt,
+        dumpBytes: manifest.dump.bytes,
+        storageFiles: manifest.storage?.fileCount ?? 0,
+        storageBytes: manifest.storage?.totalBytes ?? 0,
+        tables: manifest.database.tableCount,
+        estimatedRows: manifest.database.estimatedRowTotal,
+    };
+    // 这些数会原样写进指标文本。parseManifest 只管演练和恢复要用的字段(estimatedRowTotal 就不查),
+    // 一个字符串或带换行的值能让整次抓取在 Prometheus 那边解析失败,甚至塞进假样本
+    for (const [field, value] of Object.entries(summary)) {
+        if (typeof value !== "number" || !Number.isFinite(value) || value < 0) {
+            throw new Error(`${key} is malformed: ${field} is not a non-negative number`);
+        }
+    }
+    return summary;
+}
+/**
+ * 时间在将来的记录会让"距今多久"的告警永远不响(time() - 将来的时间是负数)。
+ * 报成读桶失败(up=0),让人去看是记录被改了还是哪台机器的时钟不对。
+ */
+function assertNotInFuture(at, key, field) {
+    if (at > Date.now() + CLOCK_SKEW_TOLERANCE_MS) {
+        throw new Error(`${key} has ${field} ${new Date(at).toISOString()}, more than 10 minutes in the future ` +
+            `(check the clock on the machine that wrote it)`);
+    }
+}
+/** <prefix>/<project>.drills/ 下的对象名,分页列完 */
+async function listDrillRecordNames(s3, bucket, prefix, abortSignal) {
+    const names = [];
+    let token;
+    do {
+        const res = await s3.send(new ListObjectsV2Command({ Bucket: bucket, Prefix: prefix, Delimiter: "/", ContinuationToken: token }), { abortSignal });
+        for (const object of res.Contents ?? []) {
+            if (object.Key)
+                names.push(object.Key.slice(prefix.length));
+        }
+        token = res.IsTruncated ? res.NextContinuationToken : undefined;
+    } while (token);
+    return names;
+}
+/**
+ * 最近一次演练和最近一次通过的演练。结束时间和结论都在文件名里,列一下目录就够;
+ * 只为了恢复用时读一次通过的那份。时间在将来的记录(写它的机器时钟快了,或被改过)跳过并警告:
+ * 它会一直排在最前面,让"距今多久"的告警算出负数。记录从不删除,所以不能让它把读桶整个弄成失败。
+ */
+async function readDrills(s3, config, abortSignal) {
+    const bucket = requireBucket(config).bucket;
+    const prefix = drillRecordsPrefix(config);
+    const now = Date.now();
+    let lastDrill = null;
+    let lastPassed = null;
+    for (const name of await listDrillRecordNames(s3, bucket, prefix, abortSignal)) {
+        const parsed = parseDrillRecordName(name);
+        if (!parsed)
+            continue;
+        if (parsed.finishedAt > now + CLOCK_SKEW_TOLERANCE_MS) {
+            log.warn(`Ignoring ${prefix}${name}: it is dated more than 10 minutes in the future (check the clock on the machine that ran the drill).`);
+            continue;
+        }
+        if (!lastDrill || parsed.finishedAt > lastDrill.finishedAt)
+            lastDrill = { ...parsed, name };
+        if (parsed.pass && (!lastPassed || parsed.finishedAt > lastPassed.finishedAt)) {
+            lastPassed = { finishedAt: parsed.finishedAt, name };
+        }
+    }
+    let lastPassedDrill = null;
+    if (lastPassed) {
+        const key = prefix + lastPassed.name;
+        const read = await readObject(s3, bucket, key, abortSignal);
+        // 刚列出来就不见了:生命周期规则正好删了它,或者有人手动删了。按读失败处理,下次抓取再看
+        if (read.status !== "ok")
+            throw new Error(`${key} disappeared while it was being read`);
+        const record = parseDrillRecord(read.text, key);
+        if (!record.pass || Date.parse(record.finishedAt) !== lastPassed.finishedAt) {
+            throw new Error(`${key} is malformed: its content does not match its name`);
+        }
+        lastPassedDrill = { finishedAt: lastPassed.finishedAt, restoreSeconds: record.restoreSeconds };
+    }
+    return {
+        lastDrill: lastDrill ? { finishedAt: lastDrill.finishedAt, pass: lastDrill.pass } : null,
+        lastPassedDrill,
+    };
+}
+/**
+ * 读桶的函数。每次调用都现读;唯一记住的是最新一份 manifest 的 ETag 和摘要:
+ * 下次带 If-None-Match 去问,没变就是 304、不传正文。快照写完就不再改,而一份 manifest 列着
+ * 每个 Storage 文件,可能有几十 MB,每 30 秒整份下载一遍会变成用户账单上的流量。
+ */
+export function createBucketReader(s3, config) {
+    const storage = requireBucket(config);
+    const root = `${storage.prefix}/${config.projectName}/`;
+    let remembered = null;
+    return async (abortSignal) => {
+        const prefixes = (await listSnapshotPrefixes(s3, storage.bucket, root, abortSignal))
+            .sort()
+            .reverse();
+        // manifest 是快照的完成标记,最后写。目录在、manifest 不在 = 备份还在跑或中途失败了,
+        // 往前找上一份完整的。不设上限:找太久就撞上抓取超时,报 up=0,这本身就该有人去看
+        let latest = null;
+        for (const prefix of prefixes) {
+            const key = `${prefix}manifest.json`;
+            const known = remembered?.key === key ? remembered : null;
+            const read = await readObject(s3, storage.bucket, key, abortSignal, known?.etag);
+            if (read.status === "missing")
+                continue;
+            if (read.status === "not-modified" && known) {
+                // 内容没变,但"是不是在将来"取决于现在几点:本机时钟被校正回来之后要重新判断
+                assertNotInFuture(known.summary.createdAt, key, "createdAt");
+                latest = known.summary;
+            }
+            else if (read.status === "ok") {
+                latest = summarize(read.text, key);
+                remembered = read.etag ? { key, etag: read.etag, summary: latest } : null;
+            }
+            else {
+                // 没带 If-None-Match 却收到 304:对端行为不对,别猜
+                throw new Error(`unexpected 304 Not Modified for ${key}`);
+            }
+            break;
+        }
+        const drills = await readDrills(s3, config, abortSignal);
+        return { snapshots: prefixes.length, latest, ...drills };
+    };
+}
+const escapeHelp = (s) => s.replace(/\\/g, "\\\\").replace(/\n/g, "\\n");
+const escapeLabel = (s) => s.replace(/\\/g, "\\\\").replace(/"/g, '\\"').replace(/\n/g, "\\n");
+const seconds = (ms) => ms / 1000;
+function gauge(name, help, value) {
+    return value === null ? null : { name, help, samples: [{ value }] };
+}
+/** state 为 null = 这次没读成桶。全部是 gauge:都是"某一刻的状态",没有一个会只增不减。 */
+export function renderMetrics(state, scrapeSeconds) {
+    const latest = state?.latest ?? null;
+    const lastDrill = state?.lastDrill ?? null;
+    const lastPassed = state?.lastPassedDrill ?? null;
+    const families = [
+        gauge("backupdrill_up", "Whether the last read of the backup bucket worked (1) or failed (0).", state ? 1 : 0),
+        gauge("backupdrill_scrape_duration_seconds", "How long this scrape took to read the backup bucket.", scrapeSeconds),
+        {
+            name: "backupdrill_build_info",
+            help: "The backupdrill CLI version running this exporter. Always 1.",
+            samples: [{ labels: { version: TOOL_VERSION }, value: 1 }],
+        },
+    ];
+    if (state) {
+        families.push(gauge("backupdrill_snapshots", "Snapshot folders under <prefix>/<project>/, including ones without a manifest.json yet.", state.snapshots), gauge("backupdrill_backup_last_success_timestamp_seconds", "When the newest complete snapshot finished (its manifest.json createdAt), as a Unix timestamp. 0 if there is none.", latest ? seconds(latest.createdAt) : 0), gauge("backupdrill_latest_snapshot_dump_bytes", "Size of the database dump (pg_dump custom format) in the newest complete snapshot.", latest?.dumpBytes ?? null), gauge("backupdrill_latest_snapshot_storage_files", "Supabase Storage files in the newest complete snapshot. 0 for a database-only backup.", latest?.storageFiles ?? null), gauge("backupdrill_latest_snapshot_storage_bytes", "Total size of the Supabase Storage files in the newest complete snapshot.", latest?.storageBytes ?? null), gauge("backupdrill_latest_snapshot_tables", "Tables, partitioned tables and materialized views captured in the newest complete snapshot.", latest?.tables ?? null), gauge("backupdrill_latest_snapshot_estimated_rows", "Row count of the newest complete snapshot, estimated from Postgres statistics (n_live_tup) at backup time.", latest?.estimatedRows ?? null), gauge("backupdrill_drill_last_run_timestamp_seconds", "When the last drill recorded with --record finished, passed or not, as a Unix timestamp. 0 if none is recorded.", lastDrill ? seconds(lastDrill.finishedAt) : 0), gauge("backupdrill_drill_last_run_passed", "Whether the last recorded drill passed (1) or not (0).", lastDrill ? (lastDrill.pass ? 1 : 0) : null), gauge("backupdrill_drill_last_success_timestamp_seconds", "When the last passing drill recorded with --record finished, as a Unix timestamp. 0 if none is recorded.", lastPassed ? seconds(lastPassed.finishedAt) : 0), gauge("backupdrill_drill_last_success_restore_duration_seconds", "How long the database restore took in the last passing recorded drill.", lastPassed?.restoreSeconds ?? null));
+    }
+    const lines = [];
+    for (const family of families) {
+        if (!family)
+            continue;
+        lines.push(`# HELP ${family.name} ${escapeHelp(family.help)}`, `# TYPE ${family.name} gauge`);
+        for (const sample of family.samples) {
+            const labels = sample.labels
+                ? `{${Object.entries(sample.labels)
+                    .map(([k, v]) => `${k}="${escapeLabel(v)}"`)
+                    .join(",")}}`
+                : "";
+            lines.push(`${family.name}${labels} ${sample.value}`);
+        }
+    }
+    return lines.join("\n") + "\n";
+}
+/**
+ * 读桶的时限跟着 Prometheus 走:它在请求头里告诉我们这次抓取最多等多久,留半秒余量,
+ * 好让超时的这次还能回一个 up=0,而不是被 Prometheus 当成 exporter 挂了。
+ */
+export function scrapeTimeoutMs(header) {
+    const value = Number(Array.isArray(header) ? header[0] : header);
+    if (!Number.isFinite(value) || value <= 0)
+        return DEFAULT_SCRAPE_TIMEOUT_MS;
+    // AbortSignal.timeout 只收整数毫秒(1.1 秒算出来是 600.0000000000001,直接抛错);
+    // 上限防一个离谱的头把读桶拖成几小时
+    return Math.min(Math.max(1, Math.floor(Math.max(value - 0.5, value / 2) * 1000)), MAX_SCRAPE_TIMEOUT_MS);
+}
+const LANDING_PAGE = `<!doctype html>
+<html lang="en">
+<head><meta charset="utf-8"><title>backupdrill exporter</title></head>
+<body>
+<h1>backupdrill exporter</h1>
+<p>Version ${TOOL_VERSION}. Prometheus metrics for the backups and recorded restore drills in one project's bucket.</p>
+<p><a href="/metrics">Metrics</a></p>
+</body>
+</html>
+`;
+export function createExporterServer(read) {
+    return createServer((req, res) => {
+        // 处理函数是 async:里面任何没接住的异常都会变成未处理的 rejection,把整个进程带走。
+        // 端口对 Prometheus 开着,一个怪请求不能让 exporter 停掉
+        handle(req, res, read).catch((error) => {
+            log.warn(`Request failed: ${error.message}`);
+            if (!res.headersSent)
+                res.writeHead(500, { "Content-Type": "text/plain; charset=utf-8" });
+            res.end();
+        });
+    });
+}
+async function handle(req, res, read) {
+    // 只要路径,不用 new URL:"GET // HTTP/1.1" 这种请求行会让它抛 Invalid URL
+    const path = (req.url ?? "/").split("?")[0];
+    if (req.method !== "GET") {
+        res.writeHead(405, { Allow: "GET", "Content-Type": "text/plain; charset=utf-8" });
+        res.end("Method Not Allowed\n");
+        return;
+    }
+    if (path === "/") {
+        res.writeHead(200, { "Content-Type": "text/html; charset=utf-8" });
+        res.end(LANDING_PAGE);
+        return;
+    }
+    if (path !== "/metrics") {
+        res.writeHead(404, { "Content-Type": "text/plain; charset=utf-8" });
+        res.end("Not Found\n");
+        return;
+    }
+    const started = performance.now();
+    const timeoutMs = scrapeTimeoutMs(req.headers["x-prometheus-scrape-timeout-seconds"]);
+    let state = null;
+    try {
+        state = await withDeadline(read, timeoutMs);
+    }
+    catch (error) {
+        const e = error;
+        const reason = e.name === "TimeoutError" || e.name === "AbortError"
+            ? `timed out after ${(timeoutMs / 1000).toFixed(1)}s`
+            : e.message;
+        log.warn(`Could not read the backup bucket: ${reason}`);
+    }
+    // 微秒够用,免得样本里出现 0.021267749999999978 这种浮点尾巴
+    const body = renderMetrics(state, Number(((performance.now() - started) / 1000).toFixed(6)));
+    res.writeHead(200, { "Content-Type": "text/plain; version=0.0.4; charset=utf-8" });
+    res.end(body);
+}
+/** CLI 的 `backupdrill exporter`:起 HTTP 服务,直到进程被停掉。 */
+export async function startExporter(config, opts) {
+    const storage = requireBucket(config);
+    const server = createExporterServer(createBucketReader(targetClient(config), config));
+    await new Promise((resolve, reject) => {
+        server.once("error", reject);
+        server.listen(opts.port, opts.host, () => {
+            server.off("error", reject);
+            resolve();
+        });
+    });
+    // 报实际绑定的端口:--port 0 时由系统挑
+    const port = server.address().port;
+    const shownHost = opts.host.includes(":") ? `[${opts.host}]` : opts.host;
+    log.ok(`Serving metrics for s3://${storage.bucket}/${storage.prefix}/${config.projectName}/ ` +
+        `at http://${shownHost}:${port}/metrics`);
+    return server;
+}

package/dist/index.js

@@ -8,6 +8,8 @@ import { runDrill } from "./drill.js";
 import { createInterface } from "node:readline/promises";
 import { Writable } from "node:stream";
 import { runRestore, NO_SOURCE_DATABASE } from "./restore.js";
+import { recordDrillOutcome } from "./drill-record.js";
+import { DEFAULT_EXPORTER_PORT, startExporter } from "./exporter.js";
 /** 隐藏式秘密输入(PRD §5.3.1):终端不回显;无 TTY(CI/脚本)时明确要求走环境变量。 */
 async function promptSecret(label) {
     if (!process.stdin.isTTY) {
@@ -114,9 +116,12 @@ program
     "verify data and business invariants here, not RLS behavior")
     .option("--keep", "if the drill fails, keep the sandbox container running for inspection")
     .option(PING_FLAG, PING_HELP)
+    .option("--record", "write the outcome as a small JSON file in <prefix>/<project>.drills/, where " +
+    "`backupdrill exporter` reads it. Needs write access to the bucket")
     .action(async (opts) => {
     const ping = createPinger(await resolvePingUrl({ flag: opts.pingUrl, configPath: opts.config }));
     await ping.start();
+    // config 放在 try 外面:出错时 /fail 要用它抹敏感值,--record 要用它记一笔"没通过"
     let config;
     try {
         if (opts.checkCmd !== undefined && !String(opts.checkCmd).trim()) {
@@ -152,6 +157,8 @@ program
             `${report.restoredTableCount} tables / ${report.restoredRowTotal.toLocaleString()} rows ` +
             `restored in ${report.restoreSeconds}s`);
         process.stdout.write(JSON.stringify({ ok: report.pass, report }) + "\n");
+        if (opts.record)
+            await recordDrillOutcome(config, report);
         const outcome = drillOutcome(report);
         if (outcome.pass) {
             await ping.success(outcome.text);
@@ -165,6 +172,9 @@ program
         const message = error.message;
         log.error(message);
         process.exitCode = 1;
+        // 没装 Docker、桶里没快照这类中途失败也算一次没通过的演练;配置都没读成就无处可记
+        if (opts.record && config)
+            await recordDrillOutcome(config, null);
         await ping.fail(`backupdrill ${TOOL_VERSION}: drill failed. ${pingMessageOf(error)}`, sensitiveValuesOf(config));
     }
 });
@@ -297,6 +307,44 @@ program
         process.exitCode = 1;
     }
 });
+program
+    .command("exporter")
+    .description("Serve Prometheus metrics about the backups (and drills run with --record) in your bucket")
+    .option("-c, --config <path>", "path to a JSON config file", "backupdrill.config.json")
+    .option("--bucket <name>", "bucket the backups live in")
+    .option("--endpoint <url>", "S3-compatible endpoint (for R2/B2)")
+    .option("--region <region>", "bucket region")
+    .option("--prefix <prefix>", "key prefix inside the bucket (default: backupdrill)")
+    .option("--project-name <name>", "project name used in the object key")
+    .option("--host <address>", "address to listen on; use 0.0.0.0 when Prometheus runs on another machine", "127.0.0.1")
+    .option("--port <port>", "port to listen on", String(DEFAULT_EXPORTER_PORT))
+    .action(async (opts) => {
+    try {
+        const port = Number(opts.port);
+        if (!Number.isInteger(port) || port < 1 || port > 65535) {
+            throw new Error(`--port must be a whole number from 1 to 65535, got "${opts.port}"`);
+        }
+        const config = await loadConfig({
+            configPath: opts.config,
+            overrides: {
+                projectName: opts.projectName,
+                // exporter 只读桶,不连源库;给 databaseUrl 占位以通过校验
+                databaseUrl: "postgresql://unused",
+                storage: {
+                    bucket: opts.bucket,
+                    endpoint: opts.endpoint,
+                    region: opts.region,
+                    prefix: opts.prefix,
+                },
+            },
+        });
+        await startExporter(config, { host: opts.host, port });
+    }
+    catch (error) {
+        log.error(error.message);
+        process.exitCode = 1;
+    }
+});
 program.parseAsync().catch((error) => {
     log.error(error.message);
     process.exitCode = 1;

package/dist/snapshots.d.ts

@@ -2,6 +2,11 @@ import { S3Client } from "@aws-sdk/client-s3";
 import { type BackupConfig } from "./config.js";
 /** 读备份桶(用户自己的桶)的 S3 客户端。drill/restore 共用。 */
 export declare function targetClient(config: BackupConfig): S3Client;
+/**
+ * 列出 root(带尾斜杠)下的全部快照目录,返回带尾斜杠的前缀,未排序。只看 CommonPrefixes,
+ * root 下直接放的对象不算快照。
+ */
+export declare function listSnapshotPrefixes(s3: S3Client, bucket: string, root: string, abortSignal?: AbortSignal): Promise<string[]>;
 /** 定位快照:显式指定,否则取字典序最大(时间戳格式天然可排序)。返回带尾斜杠的前缀。 */
 export declare function resolveSnapshot(s3: S3Client, config: BackupConfig, explicit?: string): Promise<string>;
 export declare function getObjectText(s3: S3Client, bucket: string, key: string): Promise<string>;

package/dist/snapshots.js

@@ -18,27 +18,35 @@ export function targetClient(config) {
         },
     });
 }
-/** 定位快照:显式指定,否则取字典序最大(时间戳格式天然可排序)。返回带尾斜杠的前缀。 */
-export async function resolveSnapshot(s3, config, explicit) {
-    const storage = requireBucket(config);
-    const root = `${storage.prefix}/${config.projectName}/`;
-    if (explicit)
-        return `${root}${explicit}/`;
+/**
+ * 列出 root(带尾斜杠)下的全部快照目录,返回带尾斜杠的前缀,未排序。只看 CommonPrefixes,
+ * root 下直接放的对象不算快照。
+ */
+export async function listSnapshotPrefixes(s3, bucket, root, abortSignal) {
     const prefixes = [];
     let token;
     do {
         const res = await s3.send(new ListObjectsV2Command({
-            Bucket: storage.bucket,
+            Bucket: bucket,
             Prefix: root,
             Delimiter: "/",
             ContinuationToken: token,
-        }));
+        }), { abortSignal });
         for (const cp of res.CommonPrefixes ?? []) {
             if (cp.Prefix)
                 prefixes.push(cp.Prefix);
         }
         token = res.IsTruncated ? res.NextContinuationToken : undefined;
     } while (token);
+    return prefixes;
+}
+/** 定位快照:显式指定,否则取字典序最大(时间戳格式天然可排序)。返回带尾斜杠的前缀。 */
+export async function resolveSnapshot(s3, config, explicit) {
+    const storage = requireBucket(config);
+    const root = `${storage.prefix}/${config.projectName}/`;
+    if (explicit)
+        return `${root}${explicit}/`;
+    const prefixes = await listSnapshotPrefixes(s3, storage.bucket, root);
     if (!prefixes.length) {
         throw new Error(`No snapshots found under s3://${storage.bucket}/${root}. Run a backup first.`);
     }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "backupdrill",
-  "version": "2.5.0",
+  "version": "2.6.0",
   "description": "Open-source CLI to back up a Supabase project — Postgres database and Storage files — to your own S3/R2/B2 bucket, with a checksummed manifest.",
   "type": "module",
   "bin": {