@rehearsal-db/core 0.1.0-beta.5 → 0.1.0-beta.7

package/docs/tutorial.md

@@ -1,96 +1,107 @@
-# Tutorial: rehearse a migration in a new project
+# Safe hands-on tutorial
 
-This walkthrough uses a fictional `widgets` application. It demonstrates the complete
-consumer workflow without importing private or hosted data.
+This tutorial runs a complete PostgreSQL rehearsal in a temporary fictional project. It
+does not use your application, production data, or a hosted database.
 
-## Create the application migration history
+Allow about five minutes. You need Node.js 24 and a running Docker-compatible engine.
 
-The project begins with one historical migration:
+## 1. Prepare Rehearsal
 
-```sql
-create table public.widgets (
-  id bigint generated always as identity primary key,
-  name text not null
-);
+If you do not already have this repository, clone it:
+
+```bash
+git clone https://github.com/Ddupasquier/rehearsal-db.git
+cd rehearsal-db
 ```
 
-Place it at `supabase/migrations/20260101000000_create_widgets.sql`. Add a second,
-candidate migration:
+Then run:
 
-```sql
-alter table public.widgets add column description text;
+```bash
+npm ci --ignore-scripts
+docker pull postgres:17-alpine
 ```
 
-Place that at `supabase/migrations/20260101000100_add_widget_description.sql`.
+If you already have this repository open, use that checkout.
 
-## Configure the isolated runtime
+## 2. Make a temporary project
 
-Run `npx rehearsal init --write`, then edit the result. Give the runtime unique local
-ports and a unique project ID. Its `rehearsalConfig` must point at a dedicated, unlinked
-Supabase config. Never reuse a hosted project reference or production environment file.
+From the Rehearsal repository root, run this block exactly:
 
-Keep the generated `.rehearsal` directory ignored. It contains local artifacts and
-runtime state, not source code.
+```bash
+rehearsal_repo=$PWD
+tutorial_dir=$(mktemp -d)
+cp -R tests/fixtures/postgresql-project "$tutorial_dir/app"
+cd "$tutorial_dir/app"
+npm install --ignore-scripts --no-save "$rehearsal_repo"
+```
 
-## Build a synthetic baseline
+The temporary project contains:
 
-The baseline contains the historical migration bundle, a one-row sanitized data stream,
-and a manifest binding their checksums. Baseline construction is deliberately separate
-from runtime execution: the application owns extraction and policy; the engine accepts
-only a completed, verified artifact.
+- one historical migration that creates a `widgets` table;
+- one candidate migration that adds a `description` column;
+- one synthetic row;
+- a reviewed sanitization policy;
+- an application proof that checks the migrated database.
 
-Write the safe rows to `rehearsal/synthetic-data.ndjson` and the exact represented
-statements to `rehearsal/migration-ledger.json`, then run:
+## 3. Create the baseline
+
+Open the guide:
 
 ```bash
-npx rehearsal baseline create \
-  --records=rehearsal/synthetic-data.ndjson \
-  --ledger=rehearsal/migration-ledger.json
+npx rehearsal
 ```
 
-For an executable sample, inspect `tests/fixtures/rehearsal-project` in the repository.
-The maintained fixture proof invokes this public command through the packed npm tarball
-and never contacts a hosted service.
+The project begins at Stage 3 of 4. Choose **Create the baseline**. Accept the detected
+record and ledger files, review the summary, then confirm creation.
 
-## Prove planning is non-mutating
+The project should advance to Stage 4 of 4 with every item checked.
 
-```bash
-npx rehearsal doctor
-npx rehearsal explain
-npx rehearsal run --dry-run
-npx rehearsal candidates --json
+## 4. Run the migration
+
+Choose **Run a rehearsal**. The guide should show exactly one candidate:
+
+```text
+20260101000100_add_widget_description.sql
 ```
 
-`explain` and `run --dry-run` return the same execution plan. At this point no local
-database has been restored.
+Confirm it. Rehearsal creates a loopback-only PostgreSQL container, restores the baseline,
+applies the candidate, and runs the fixture proof.
 
-## Run and inspect
+Success ends with an application-proof message and suggests exercising the local
+application before verification.
 
-Copy the exact digest from `candidates`:
+## 5. Try the runtime commands
+
+Exit the guide, then run:
 
 ```bash
-npx rehearsal run --confirm-candidates=<sha256>
-npx rehearsal inspect migrations
 npx rehearsal status
+npx rehearsal verify
+npx rehearsal reset
+npx rehearsal stop
+npx rehearsal discard
 ```
 
-The historical migration should be `represented_by_baseline`; the description migration
-should be `applied_to_current_runtime`. Test normal creates, updates, and deletes through
-your local application. They affect only this disposable database.
+`discard` removes only the labeled tutorial container and volume. The temporary project
+directory remains on disk and can be deleted when you no longer need it.
+
+## What you proved
 
-## Prove failure behavior
+You used the same packaged CLI a normal project installs. Rehearsal verified the baseline,
+approved an exact migration, ran it in a disposable database, tested the result, and
+cleaned up only its own runtime.
 
-Add a timestamped migration containing invalid SQL, rerun `candidates`, and use its new
-digest. The command must fail with `migration_candidate_failure`, remove the untrusted
-runtime, and never produce a successful receipt.
+Next, follow [Getting started](getting-started.md) in your own project. Start with synthetic
+rows until the workflow and application proof are reliable.
 
-Then restore the valid migration set and rerun:
+## Supabase check
+
+If Supabase CLI 2.117.0 is installed, the repository also has a fully automated Supabase
+proof:
 
 ```bash
-npx rehearsal reset
-npx rehearsal verify
-npx rehearsal stop
+cd "$rehearsal_repo"
+npm run test:fixture
 ```
 
-That cycle—plan, confirm exact bytes, run, exercise the app, reset—is the normal Rehearsal
-workflow.
+The PostgreSQL equivalent is `npm run test:fixture:postgresql`.

package/package.json

@@ -1,8 +1,8 @@
 {
   "name": "@rehearsal-db/core",
-  "version": "0.1.0-beta.5",
+  "version": "0.1.0-beta.7",
   "private": false,
-  "description": "Safely rehearse Supabase migrations against sanitized, production-shaped PostgreSQL data.",
+  "description": "Safely rehearse PostgreSQL and Supabase migrations against sanitized, production-shaped data.",
   "repository": {
     "type": "git",
     "url": "git+https://github.com/Ddupasquier/rehearsal-db.git"
@@ -39,6 +39,7 @@
   "files": [
     "scripts/lib/environment/local_supabase.mjs",
     "scripts/lib/rehearsal",
+    "scripts/lib/runtime",
     "scripts/operations/database/manage_rehearsal_database.mjs",
     "scripts/operations/rehearsal/rehearsal_cli.mjs",
     "docs/*.md",
@@ -55,6 +56,7 @@
     "test:unit": "vitest run tests/scripts",
     "test:docs": "vitest run tests/contracts/documentationCommands.test.mjs",
     "test:fixture": "node scripts/operations/rehearsal/prove_independent_fixture.mjs",
+    "test:fixture:postgresql": "node scripts/operations/rehearsal/prove_postgresql_fixture.mjs",
     "check": "npm run check:syntax && npm run test && npm run format:check && npm run package:audit",
     "check:syntax": "find scripts tests -type f -name '*.mjs' -exec node --check {} +",
     "format": "prettier --write .",

package/scripts/lib/environment/local_supabase.mjs

@@ -62,14 +62,13 @@ export const localCommandSucceeds = (
     stdio: "ignore",
   }).status === 0;
 
-export const ensureLocalContainerRuntime = ({ cwd = process.cwd() } = {}) => {
+export const ensureLocalContainerRuntime = ({
+  cwd = process.cwd(),
+  autoStartColima = true,
+} = {}) => {
   if (localCommandSucceeds("docker", ["info"], { cwd })) return;
-  if (localCommandSucceeds("colima", ["version"], { cwd })) {
-    runLocalCommand(
-      "colima",
-      ["start", "--cpu", "4", "--memory", "4", "--disk", "40"],
-      { cwd },
-    );
+  if (autoStartColima && localCommandSucceeds("colima", ["version"], { cwd })) {
+    runLocalCommand("colima", ["start"], { cwd });
   }
   if (!localCommandSucceeds("docker", ["info"], { cwd })) {
     throw new Error(
@@ -118,8 +117,9 @@ export const startLocalSupabase = ({
   exclude = [],
   environment = {},
   applyMigrations = true,
+  autoStartColima = true,
 } = {}) => {
-  ensureLocalContainerRuntime({ cwd });
+  ensureLocalContainerRuntime({ cwd, autoStartColima });
   const startArguments = withWorkdir(
     ["start", ...(exclude.length ? ["--exclude", exclude.join(",")] : [])],
     workdir,

package/scripts/lib/rehearsal/baseline_artifact.mjs

@@ -478,7 +478,7 @@ export const listIncompleteBaselineBuilds = async ({ artifactRoot }) => {
   }
 };
 
-export const pruneBaselineGenerations = async ({
+export const planBaselineGenerationPrune = async ({
   artifactRoot,
   retain = 2,
 }) => {
@@ -516,7 +516,36 @@ export const pruneBaselineGenerations = async ({
   const removed = entries
     .map((entry) => entry.name)
     .filter((name) => !retained.has(name));
-  for (const name of removed) {
+  return { retained: [...retained], removed };
+};
+
+export const pruneBaselineGenerations = async ({
+  artifactRoot,
+  retain = 2,
+  expectedRemoved,
+}) => {
+  const root = assertArtifactRoot(artifactRoot);
+  const plan = await planBaselineGenerationPrune({
+    artifactRoot: root,
+    retain,
+  });
+  const generationsRoot = join(root, generationsDirectoryName);
+  if (
+    expectedRemoved &&
+    (expectedRemoved.length !== plan.removed.length ||
+      expectedRemoved.some((name, index) => name !== plan.removed[index]))
+  ) {
+    throw new Error(
+      "Rehearsal baseline generations changed after the cleanup preview.",
+    );
+  }
+  for (const name of plan.removed) {
+    const active = await resolveActiveBaselinePaths({ artifactRoot: root });
+    if (basename(active.generationDirectory) === name) {
+      throw new Error(
+        "Refusing to prune the active Rehearsal baseline generation.",
+      );
+    }
     const target = assertInsideRoot(
       generationsRoot,
       join(generationsRoot, name),
@@ -525,7 +554,7 @@ export const pruneBaselineGenerations = async ({
     await makeArtifactTreeWritable(target);
     await rm(target, { recursive: true, force: true });
   }
-  return { retained: [...retained], removed };
+  return plan;
 };
 
 export const removeBaselineArtifactRoot = async ({ artifactRoot }) => {

package/scripts/lib/rehearsal/cleanup.mjs

@@ -0,0 +1,381 @@
+/**
+ * Purpose: Preview and apply narrowly scoped Rehearsal cleanup without global
+ * Docker pruning or deletion of unowned database resources.
+ */
+
+import { spawnSync } from "node:child_process";
+import { createHash } from "node:crypto";
+import { stat } from "node:fs/promises";
+import {
+  planBaselineGenerationPrune,
+  pruneBaselineGenerations,
+} from "./baseline_artifact.mjs";
+import { loadRehearsalConfig } from "./configuration.mjs";
+import { createCleanProcessEnvironment } from "./process_environment.mjs";
+
+const SUPABASE_IMAGE_PREFIX = "public.ecr.aws/supabase/";
+
+const commandResult = (command, args, { projectRoot }) =>
+  spawnSync(command, args, {
+    cwd: projectRoot,
+    encoding: "utf8",
+    env: createCleanProcessEnvironment(),
+    maxBuffer: 32 * 1024 * 1024,
+    stdio: ["ignore", "pipe", "pipe"],
+  });
+
+const runDocker = (args, { projectRoot, allowFailure = false }) => {
+  const result = commandResult("docker", args, { projectRoot });
+  if (!allowFailure && (result.error || result.status !== 0)) {
+    const detail = [result.stdout, result.stderr]
+      .filter(Boolean)
+      .join("\n")
+      .trim();
+    throw new Error(
+      `docker ${args.join(" ")} failed${detail ? `:\n${detail}` : "."}`,
+    );
+  }
+  return result;
+};
+
+const nonEmptyLines = (value) =>
+  String(value ?? "")
+    .split("\n")
+    .map((line) => line.trim())
+    .filter(Boolean);
+
+const dockerAvailable = (projectRoot) =>
+  runDocker(["info"], { projectRoot, allowFailure: true }).status === 0;
+
+export const parsePosixDiskUsage = (output) => {
+  const columns = nonEmptyLines(output).at(-1)?.split(/\s+/u) ?? [];
+  if (columns.length < 6) return null;
+  const totalKiB = Number(columns[1]);
+  const usedKiB = Number(columns[2]);
+  const availableKiB = Number(columns[3]);
+  const usedPercent = Number.parseInt(columns[4], 10);
+  if (![totalKiB, usedKiB, availableKiB, usedPercent].every(Number.isFinite)) {
+    return null;
+  }
+  return {
+    totalBytes: totalKiB * 1024,
+    usedBytes: usedKiB * 1024,
+    availableBytes: availableKiB * 1024,
+    usedPercent,
+  };
+};
+
+const inspectContainerDisk = (projectRoot) => {
+  const context = commandResult("docker", ["context", "show"], {
+    projectRoot,
+  });
+  if (context.status !== 0 || context.stdout.trim() !== "colima") return null;
+  const result = commandResult(
+    "colima",
+    ["ssh", "--", "df", "-Pk", "/var/lib/docker"],
+    { projectRoot },
+  );
+  const usage = result.status === 0 ? parsePosixDiskUsage(result.stdout) : null;
+  return usage ? { backend: "colima", ...usage } : null;
+};
+
+const inspectDockerUsage = (projectRoot) => {
+  if (!dockerAvailable(projectRoot)) {
+    return { available: false, resources: [], disk: null };
+  }
+  const result = runDocker(["system", "df", "--format", "{{json .}}"], {
+    projectRoot,
+  });
+  return {
+    available: true,
+    resources: nonEmptyLines(result.stdout).map((line) => JSON.parse(line)),
+    disk: inspectContainerDisk(projectRoot),
+  };
+};
+
+const inspectUsedImageIds = (projectRoot) => {
+  const containers = nonEmptyLines(
+    runDocker(["container", "ls", "--all", "--quiet"], { projectRoot }).stdout,
+  );
+  if (containers.length === 0) return new Set();
+  return new Set(
+    nonEmptyLines(
+      runDocker(
+        ["container", "inspect", "--format", "{{.Image}}", ...containers],
+        { projectRoot },
+      ).stdout,
+    ),
+  );
+};
+
+const splitImageTag = (value) => {
+  const separator = value.lastIndexOf(":");
+  if (separator <= value.lastIndexOf("/")) return null;
+  return {
+    repository: value.slice(0, separator),
+    tag: value.slice(separator + 1),
+  };
+};
+
+export const selectOlderUnusedSupabaseImages = ({ images, usedImageIds }) => {
+  const byRepository = new Map();
+  for (const image of images) {
+    for (const fullTag of image.RepoTags ?? []) {
+      const parsed = splitImageTag(fullTag);
+      if (!parsed?.repository.startsWith(SUPABASE_IMAGE_PREFIX)) continue;
+      const entries = byRepository.get(parsed.repository) ?? [];
+      entries.push({
+        id: image.Id,
+        createdAt: image.Created,
+        bytes: image.Size,
+        fullTag,
+      });
+      byRepository.set(parsed.repository, entries);
+    }
+  }
+
+  const candidatesById = new Map();
+  for (const entries of byRepository.values()) {
+    const newestId = [...entries].sort(
+      (left, right) => Date.parse(right.createdAt) - Date.parse(left.createdAt),
+    )[0]?.id;
+    for (const entry of entries) {
+      if (entry.id === newestId || usedImageIds.has(entry.id)) continue;
+      const existing = candidatesById.get(entry.id) ?? {
+        id: entry.id,
+        createdAt: entry.createdAt,
+        bytes: entry.bytes,
+        tags: [],
+      };
+      existing.tags.push(entry.fullTag);
+      candidatesById.set(entry.id, existing);
+    }
+  }
+  return [...candidatesById.values()]
+    .map((entry) => ({ ...entry, tags: [...new Set(entry.tags)].sort() }))
+    .sort((left, right) => left.createdAt.localeCompare(right.createdAt));
+};
+
+const inspectUnusedImages = (projectRoot) => {
+  const imageIds = [
+    ...new Set(
+      nonEmptyLines(
+        runDocker(["image", "ls", "--quiet", "--no-trunc"], {
+          projectRoot,
+        }).stdout,
+      ),
+    ),
+  ];
+  if (imageIds.length === 0) return [];
+  const images = nonEmptyLines(
+    runDocker(["image", "inspect", "--format", "{{json .}}", ...imageIds], {
+      projectRoot,
+    }).stdout,
+  ).map((line) => JSON.parse(line));
+  return selectOlderUnusedSupabaseImages({
+    images,
+    usedImageIds: inspectUsedImageIds(projectRoot),
+  });
+};
+
+const runtimeDetected = async ({ config, paths, projectRoot }) => {
+  try {
+    await stat(paths.runtimeWorkdir);
+    return true;
+  } catch (error) {
+    if (error?.code !== "ENOENT") throw error;
+  }
+  if (!dockerAvailable(projectRoot)) return false;
+  if (config.runtime.target === "postgresql") {
+    const containers = runDocker(
+      [
+        "container",
+        "ls",
+        "--all",
+        "--filter",
+        `label=com.rehearsal-db.project=${config.runtime.projectId}`,
+        "--quiet",
+      ],
+      { projectRoot },
+    );
+    const volumes = runDocker(
+      [
+        "volume",
+        "ls",
+        "--filter",
+        `label=com.rehearsal-db.project=${config.runtime.projectId}`,
+        "--quiet",
+      ],
+      { projectRoot },
+    );
+    return (
+      nonEmptyLines(containers.stdout).length > 0 ||
+      nonEmptyLines(volumes.stdout).length > 0
+    );
+  }
+  const containers = runDocker(
+    [
+      "container",
+      "ls",
+      "--all",
+      "--filter",
+      `label=com.supabase.cli.project=${config.runtime.projectId}`,
+      "--quiet",
+    ],
+    { projectRoot },
+  );
+  const volumes = nonEmptyLines(
+    runDocker(["volume", "ls", "--format", "{{.Name}}"], { projectRoot })
+      .stdout,
+  );
+  return (
+    nonEmptyLines(containers.stdout).length > 0 ||
+    volumes.some((name) => name.endsWith(`_${config.runtime.projectId}`))
+  );
+};
+
+const inspectBaselineCleanup = async ({ artifactRoot, retain }) => {
+  try {
+    return {
+      available: true,
+      ...(await planBaselineGenerationPrune({ artifactRoot, retain })),
+    };
+  } catch (error) {
+    if (error?.code === "ENOENT") {
+      return { available: false, retained: [], removed: [] };
+    }
+    throw error;
+  }
+};
+
+const cleanupIdentity = (plan) => ({
+  projectId: plan.projectId,
+  retainBaselineGenerations: plan.retainBaselineGenerations,
+  baselineGenerations: plan.baselines.removed,
+  runtime: plan.runtime.included && plan.runtime.detected,
+  images: plan.images.candidates.map(({ id, tags }) => ({ id, tags })),
+});
+
+const cleanupDigest = (plan) =>
+  createHash("sha256")
+    .update(JSON.stringify(cleanupIdentity(plan)))
+    .digest("hex");
+
+export const planRehearsalCleanup = async ({
+  projectRoot = process.cwd(),
+  configPath,
+  includeRuntime = false,
+  includeImages = false,
+  inspection = { inspectDockerUsage, runtimeDetected },
+} = {}) => {
+  const loaded = await loadRehearsalConfig({ projectRoot, configPath });
+  const docker = inspection.inspectDockerUsage(loaded.projectRoot);
+  const baselines = await inspectBaselineCleanup({
+    artifactRoot: loaded.paths.artifactDirectory,
+    retain: loaded.config.cleanup.retainBaselineGenerations,
+  });
+  const images = {
+    requested: includeImages,
+    inspected: includeImages && docker.available,
+    candidates:
+      includeImages && docker.available
+        ? inspectUnusedImages(loaded.projectRoot)
+        : [],
+  };
+  const plan = {
+    project: loaded.config.project.name,
+    projectId: loaded.config.runtime.projectId,
+    target: loaded.config.runtime.target,
+    retainBaselineGenerations: loaded.config.cleanup.retainBaselineGenerations,
+    baselines,
+    runtime: {
+      included: includeRuntime,
+      detected: await inspection.runtimeDetected({ ...loaded }),
+    },
+    images,
+    docker,
+  };
+  return Object.freeze({ ...plan, digest: cleanupDigest(plan) });
+};
+
+const removePlannedImages = ({ candidates, projectRoot }) => {
+  if (candidates.length === 0) return [];
+  const usedImageIds = inspectUsedImageIds(projectRoot);
+  const removed = [];
+  for (const candidate of candidates) {
+    if (usedImageIds.has(candidate.id)) {
+      throw new Error(
+        `Cleanup stopped because a container began using ${candidate.id}. Preview the cleanup again.`,
+      );
+    }
+    for (const tag of candidate.tags) {
+      const inspection = runDocker(
+        ["image", "inspect", "--format", "{{.Id}}", tag],
+        { projectRoot, allowFailure: true },
+      );
+      if (inspection.status !== 0) continue;
+      if (inspection.stdout.trim() !== candidate.id) {
+        throw new Error(
+          `Cleanup stopped because ${tag} changed after the preview. Preview the cleanup again.`,
+        );
+      }
+      runDocker(["image", "rm", tag], { projectRoot });
+      removed.push(tag);
+    }
+  }
+  return removed;
+};
+
+export const applyRehearsalCleanup = async ({
+  plan,
+  confirmation,
+  projectRoot = process.cwd(),
+  configPath,
+  removeRuntime = async () => undefined,
+  inspection,
+} = {}) => {
+  if (confirmation !== plan?.digest) {
+    throw new Error(
+      "Cleanup requires the exact --confirm-cleanup digest from a fresh preview. Nothing was removed.",
+    );
+  }
+  const refreshed = await planRehearsalCleanup({
+    projectRoot,
+    configPath,
+    includeRuntime: plan.runtime.included,
+    includeImages: plan.images.inspected,
+    ...(inspection ? { inspection } : {}),
+  });
+  if (refreshed.digest !== plan.digest) {
+    throw new Error(
+      "Cleanup targets changed after the preview. Nothing was removed; preview the cleanup again.",
+    );
+  }
+  if (plan.images.requested && !refreshed.images.inspected) {
+    throw new Error(
+      "Docker is unavailable, so Rehearsal cannot verify or remove unused images.",
+    );
+  }
+  if (refreshed.runtime.included && refreshed.runtime.detected) {
+    await removeRuntime();
+  }
+  const loaded = await loadRehearsalConfig({ projectRoot, configPath });
+  const baselines = refreshed.baselines.available
+    ? await pruneBaselineGenerations({
+        artifactRoot: loaded.paths.artifactDirectory,
+        retain: refreshed.retainBaselineGenerations,
+        expectedRemoved: refreshed.baselines.removed,
+      })
+    : { retained: [], removed: [] };
+  const images = removePlannedImages({
+    candidates: refreshed.images.candidates,
+    projectRoot: loaded.projectRoot,
+  });
+  return {
+    project: refreshed.project,
+    projectId: refreshed.projectId,
+    baselines,
+    runtimeRemoved: refreshed.runtime.included && refreshed.runtime.detected,
+    imagesRemoved: images,
+  };
+};