@boxctl/migrate 1.1.2 → 2.0.1

package/OVERVIEW.md

@@ -0,0 +1,79 @@
+# @boxctl/migrate — Overview
+
+## What this is
+
+A minimal, roll-forward only SQL migration tool for MySQL/MariaDB accessed over a Unix socket. Built for a specific production setup — a rootless Podman-based server stack where direct socket access is available and simplicity is more valuable than features.
+
+If you need rollbacks, multi-database support, or a general-purpose migration framework, use something else. This tool does one thing.
+
+---
+
+## Philosophy
+
+Roll-forward only. Migrations are append-only history. If something goes wrong, you write a new migration that fixes it — you do not rewind. This is not a limitation, it is a deliberate constraint that keeps the mental model simple and the tool small.
+
+The loose baseline is golang-migrate. Not in implementation, but in behavior and honesty about what it does. The dirty flag mechanism is taken directly from golang-migrate's approach: record intent before acting, not outcome after.
+
+---
+
+## How migrations work
+
+Each migration is a single `.sql` file. The filename is a millisecond-precision UTC timestamp followed by a sanitized name. Lexicographic sort on the filename is the execution order — no separate ordering metadata, no manifest file.
+
+When `up` runs a migration it does three things in order:
+
+1. Insert a record into the `migrations` table with `dirty = true`
+2. Execute the SQL
+3. On success, update the record to `dirty = false`
+
+This order matters. If the process is killed mid-migration, or the SQL fails, the record exists and is dirty. Nothing is silently lost. The dirty flag is evidence that something went wrong, not an absence of evidence that something succeeded.
+
+---
+
+## The dirty flag
+
+A dirty migration means the schema is in an unknown state. `up` will refuse to run while any dirty records exist. This is intentional — running further migrations on top of an unknown state compounds the problem.
+
+Resolution is manual. The developer inspects what happened, fixes the schema directly if needed, then clears or removes the dirty record. The tool does not try to be clever about recovery because recovery from a failed DDL migration in MySQL is inherently a human problem.
+
+---
+
+## Why no down command
+
+MySQL and MariaDB do not support transactional DDL. `CREATE TABLE`, `ALTER TABLE`, `DROP TABLE` and similar statements are auto-committed the moment they execute. A rollback command gives a false sense of safety — you can run a `.down.sql` file but you are not rolling back, you are applying a new forward change that happens to be the inverse. That is just another migration.
+
+Rolling forward with a corrective migration is honest about what is actually happening. The down command was removed to eliminate the illusion.
+
+---
+
+## Why no checksums
+
+Checksums on applied migration files detect modifications after the fact, but in a roll-forward only system there is nothing actionable to do with that information. You cannot re-apply or roll back. A modified applied migration is a developer problem to reason about — the tool blocking `up` on a comment edit adds friction with no safety benefit.
+
+---
+
+## The migrations table
+
+Five columns. `id` for ordering within the same second. `name` as the unique identifier matching the filename without extension. `dirty` as the state flag. `applied_at` as the timestamp. Nothing else.
+
+The table is created automatically on first run of `up` or `status`. No separate setup step required.
+
+---
+
+## Environment and connectivity
+
+Connection details are read from an env file. The default is `.env` in the working directory. A different file can be specified with `--env=<file>`, which makes it practical to use the same tool across development, staging, and production without changing anything except which env file is loaded.
+
+Required variables are `DB_SOCK`, `DB_NAME`, `DB_USER`, and `DB_PASS`. Connection is over Unix socket only — no TCP, no host, no port. This matches the production setup this tool was built for.
+
+---
+
+## What the create command does and does not do
+
+`create` is filesystem only. It does not touch the database or read the env file. It creates a single `.sql` file with a millisecond-precision timestamp prefix and a two-line comment header. The `refs:` line in the header is a convention for documenting dependencies between migrations — which other migration files this one relates to or builds on. It is not enforced by the tool.
+
+---
+
+## Scope
+
+Three commands. One direction. One database driver. One connection method. The tool is not intended to grow beyond this.

package/README.md

@@ -1,6 +1,6 @@
 # @boxctl/migrate
 
-Minimal SQL migration tool for MySQL/MariaDB over Unix socket.
+Minimal roll-forward SQL migration tool for MySQL/MariaDB over Unix socket.
 
 > **Note:** This tool was created for very specific needs. If it doesn't fit yours, don't use it.
 
@@ -12,22 +12,60 @@ npx @boxctl/migrate <command>
 
 ### Commands
 
-- `create <name>` — Create a new migration
+- `create <name>` — Create a new migration file
 - `up` — Run pending migrations
-- `down` — Rollback the last migration
 - `status` — Show migration status
 
+### Options
+
+- `--env=<file>` — Specify env file (default: .env)
+- `--help` — Show help message
+
 ## Setup
 
 Create a `.env` file in your project root:
 
 ```
-DB_SOCKET=/var/run/mysqld/mysqld.sock
+DB_SOCK=/var/run/mysqld/mysqld.sock
 DB_NAME=your_database
 DB_USER=username
 DB_PASS=********
 ```
 
+Use a different env file:
+
+```bash
+npx @boxctl/migrate up --env=.env.staging
+```
+
+## Migrations
+
+Migrations are stored in `./migrations/` as single `.sql` files:
+
+- `YYYYMMDDHHmmssSSS_name.sql` — Migration to apply
+
+Each file is created with a header:
+
+```sql
+-- 20260317120000000_create_users.sql
+-- refs:
+```
+
+Use `refs:` to document any other migration files this one depends on or modifies.
+
+> **Important:** This is a roll-forward only system. There is no down command. Write migrations accordingly.
+
+## How it works
+
+For each pending migration `up` will:
+
+1. Insert a record with `dirty = true`
+2. Execute the SQL
+3. On success — update to `dirty = false`
+4. On failure — leave as `dirty = true` and stop
+
+A dirty migration means something went wrong. Fix the issue manually then resolve the dirty flag in the database before running `up` again.
+
 ## Requirements
 
 - Node.js >= 24

package/migrate.js

@@ -1,11 +1,26 @@
 // migrate.js
+const [, , command, ...args] = process.argv;
+
+if (command === "--help" || command === "-h" || !command) {
+    console.log("");
+    console.log("  Usage: migrate <command> [options]");
+    console.log("");
+    console.log("  Commands:");
+    console.log("    create <name>   Create a new migration file");
+    console.log("    up              Run all pending migrations");
+    console.log("    status          Show applied and pending migrations");
+    console.log("");
+    console.log("  Options:");
+    console.log("    --env=<file>    Specify env file (default: .env)");
+    console.log("    --help          Show this help message");
+    console.log("");
+    process.exit(command ? 0 : 1);
+}
+
 import { create } from "./src/commands/create.js";
 import { up } from "./src/commands/up.js";
-import { down } from "./src/commands/down.js";
 import { status } from "./src/commands/status.js";
-import db from "./src/db.js";
-
-const [, , command, ...args] = process.argv;
+import { closeDb } from "./src/db.js";
 
 async function main() {
     switch (command) {
@@ -17,42 +32,20 @@ async function main() {
             await up();
             break;
         }
-        case "down": {
-            await down(args[0] ?? 1);
-            break;
-        }
         case "status": {
             await status();
             break;
         }
-        default: {
-            console.log("");
-            console.log("  Usage: migrate <command> [options]");
-            console.log("");
-            console.log("  Commands:");
-            console.log("    create <name>   Create a new migration file pair");
-            console.log("    up              Run all pending migrations");
-            console.log(
-                "    down [n]        Revert last n migrations (default: 1)",
-            );
-            console.log(
-                "    status          Show applied and pending migrations",
-            );
-            console.log("");
-            process.exit(command ? 1 : 0);
-        }
+        default:
+            throw new Error(`Unknown command: ${command}`);
     }
 }
 
 main()
     .catch((err) => {
-        console.error("Unexpected error:", err.message);
+        console.error(err.message);
         process.exit(1);
     })
     .finally(async () => {
-        try {
-            await db.end();
-        } catch {
-            // connection may already be closed
-        }
+        await closeDb();
     });

package/package.json

@@ -1,7 +1,8 @@
 {
     "name": "@boxctl/migrate",
-    "version": "1.1.2",
-    "description": "Minimal SQL migration tool for MySQL/MariaDB over Unix socket",
+    "version": "2.0.1",
+    "license": "MIT",
+    "description": "Minimal roll-forward SQL migration tool for MySQL/MariaDB over Unix socket.",
     "type": "module",
     "bin": {
         "migrate": "./migrate.js"

package/src/commands/create.js

@@ -6,11 +6,9 @@ const MIGRATIONS_DIR = "./migrations";
 
 export async function create(name) {
     if (!name || name.trim() === "") {
-        console.error("Usage: migrate create <name>");
-        process.exit(1);
+        throw new Error("Usage: migrate create <name>");
     }
 
-    // sanitize name: lowercase, spaces to underscores, strip special chars
     const safeName = name
         .trim()
         .toLowerCase()
@@ -18,34 +16,33 @@ export async function create(name) {
         .replace(/[^a-z0-9_]/g, "");
 
     if (safeName === "") {
-        console.error("Migration name is invalid after sanitization.");
-        process.exit(1);
+        throw new Error("Migration name is invalid after sanitization.");
     }
 
     const now = new Date();
-    const timestamp = now
-        .toISOString()
-        .replace(/[-T:Z]/g, "")
-        .replace(".", "")
-        .slice(0, 17);
+    const pad = (n, l = 2) => String(n).padStart(l, "0");
+    const timestamp =
+        now.getUTCFullYear() +
+        pad(now.getUTCMonth() + 1) +
+        pad(now.getUTCDate()) +
+        pad(now.getUTCHours()) +
+        pad(now.getUTCMinutes()) +
+        pad(now.getUTCSeconds()) +
+        pad(now.getUTCMilliseconds(), 3);
+
     const baseName = `${timestamp}_${safeName}`;
 
     if (!existsSync(MIGRATIONS_DIR)) {
         mkdirSync(MIGRATIONS_DIR, { recursive: true });
     }
 
-    const upFile = join(MIGRATIONS_DIR, `${baseName}.up.sql`);
-    const downFile = join(MIGRATIONS_DIR, `${baseName}.down.sql`);
+    const file = join(MIGRATIONS_DIR, `${baseName}.sql`);
 
-    if (existsSync(upFile) || existsSync(downFile)) {
-        console.error(`Migration files for "${baseName}" already exist.`);
-        process.exit(1);
+    if (existsSync(file)) {
+        throw new Error(`Migration file "${baseName}.sql" already exists.`);
     }
 
-    writeFileSync(upFile, `-- migrate up: ${safeName}\n`);
-    writeFileSync(downFile, `-- migrate down: ${safeName}\n`);
+    writeFileSync(file, `-- ${baseName}.sql\n-- refs:\n`);
 
-    console.log(`Created:`);
-    console.log(`  ${upFile}`);
-    console.log(`  ${downFile}`);
+    console.log(`Created: ${file}`);
 }

package/src/commands/status.js

@@ -1,69 +1,80 @@
 // src/commands/status.js
 import { readdirSync, existsSync } from "fs";
-import { join } from "path";
-import db from "../db.js";
-import { bootstrap } from "../bootstrap.js";
+import getDb from "../db.js";
 
 const MIGRATIONS_DIR = "./migrations";
 
+async function bootstrap(db) {
+    await db.query(`
+        CREATE TABLE IF NOT EXISTS migrations (
+            id         INT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
+            name       VARCHAR(255) NOT NULL UNIQUE,
+            dirty      BOOLEAN DEFAULT FALSE,
+            applied_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
+        )
+    `);
+}
+
+function formatDate(d) {
+    return new Date(d).toISOString().replace("T", " ").slice(0, 19);
+}
+
 export async function status() {
-    await bootstrap();
+    const db = await getDb();
+    await bootstrap(db);
 
-    const [rows] = await db.query(
-        "SELECT name, ran_at FROM migrations ORDER BY ran_at ASC, id ASC",
+    const [dbRows] = await db.query(
+        "SELECT name, dirty, applied_at FROM migrations ORDER BY applied_at ASC, id ASC",
     );
 
-    const applied = new Map(rows.map((r) => [r.name, r.ran_at]));
+    const dbMap = new Map(dbRows.map((r) => [r.name, r]));
 
     const files = existsSync(MIGRATIONS_DIR)
         ? readdirSync(MIGRATIONS_DIR)
-              .filter((f) => f.endsWith(".up.sql"))
+              .filter((f) => f.endsWith(".sql"))
               .sort()
-              .map((f) => f.replace(".up.sql", ""))
+              .map((f) => f.replace(".sql", ""))
         : [];
 
-    // migrations on disk but not in db
-    const pending = files.filter((name) => !applied.has(name));
-
-    // migrations in db but no file on disk (deleted/lost)
-    const missing = [...applied.keys()].filter((name) => !files.includes(name));
+    const fileSet = new Set(files);
+    const allNames = [...new Set([...files, ...dbMap.keys()])];
 
-    if (applied.size === 0 && pending.length === 0) {
+    if (allNames.length === 0) {
         console.log("No migrations found.");
         return;
     }
 
-    const PAD = 42;
+    const dirty = dbRows.filter((r) => r.dirty);
 
-    console.log("");
-    console.log(`  ${"Migration".padEnd(PAD)} ${"Status".padEnd(12)} Ran At`);
-    console.log(`  ${"─".repeat(PAD)} ${"─".repeat(12)} ${"─".repeat(20)}`);
-
-    for (const name of files) {
-        if (applied.has(name)) {
-            const ranAt = new Date(applied.get(name))
-                .toISOString()
-                .replace("T", " ")
-                .slice(0, 19);
-            console.log(
-                `  ${name.padEnd(PAD)} ${"applied".padEnd(12)} ${ranAt}`,
-            );
-        } else {
-            console.log(`  ${name.padEnd(PAD)} ${"pending".padEnd(12)}`);
-        }
+    if (dirty.length > 0) {
+        console.log("\n  ! DIRTY MIGRATIONS\n");
+        console.table(dirty.map((r) => ({ Migration: r.name, "Applied At": formatDate(r.applied_at) })));
     }
 
-    for (const name of missing) {
-        const ranAt = new Date(applied.get(name))
-            .toISOString()
-            .replace("T", " ")
-            .slice(0, 19);
-        console.log(`  ${name.padEnd(PAD)} ${"! missing".padEnd(12)} ${ranAt}`);
-    }
+    const rows = allNames.map((name) => {
+        const rec = dbMap.get(name);
+        if (!rec) return { Migration: name, Status: "pending", "Applied At": "" };
+        if (rec.dirty) return { Migration: name, Status: "dirty", "Applied At": formatDate(rec.applied_at) };
+        if (!fileSet.has(name)) return { Migration: name, Status: "missing", "Applied At": formatDate(rec.applied_at) };
+        return { Migration: name, Status: "applied", "Applied At": formatDate(rec.applied_at) };
+    });
 
     console.log("");
-    console.log(
-        `  ${applied.size} applied, ${pending.length} pending${missing.length > 0 ? `, ${missing.length} missing file(s)` : ""}`,
-    );
-    console.log("");
+    console.table(rows);
+
+    const counts = {
+        applied: rows.filter((r) => r.Status === "applied").length,
+        pending: rows.filter((r) => r.Status === "pending").length,
+        dirty: dirty.length,
+        missing: rows.filter((r) => r.Status === "missing").length,
+    };
+
+    const summary = [
+        `${counts.applied} applied`,
+        `${counts.pending} pending`,
+        ...(counts.dirty > 0 ? [`${counts.dirty} dirty`] : []),
+        ...(counts.missing > 0 ? [`${counts.missing} missing`] : []),
+    ].join(", ");
+
+    console.log(`  ${summary}\n`);
 }

package/src/commands/up.js

@@ -1,30 +1,38 @@
 // src/commands/up.js
 import { readdirSync, readFileSync, existsSync } from "fs";
 import { join } from "path";
-import db from "../db.js";
-import { bootstrap } from "../bootstrap.js";
+import getDb from "../db.js";
 
 const MIGRATIONS_DIR = "./migrations";
 
-function isSqlEmpty(sql) {
-    const lines = sql.split("\n");
-    return lines.every((line) => {
-        const trimmed = line.trim();
-        return trimmed === "" || trimmed.startsWith("--");
-    });
+async function bootstrap(db) {
+    await db.query(`
+        CREATE TABLE IF NOT EXISTS migrations (
+            id         INT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
+            name       VARCHAR(255) NOT NULL UNIQUE,
+            dirty      BOOLEAN DEFAULT FALSE,
+            applied_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
+        )
+    `);
 }
 
 export async function up() {
-    await bootstrap();
+    const db = await getDb();
+    await bootstrap(db);
+
+    const [dirtyRows] = await db.query("SELECT name FROM migrations WHERE dirty = TRUE");
+    if (dirtyRows.length > 0) {
+        const names = dirtyRows.map((r) => `  - ${r.name}`).join("\n");
+        throw new Error(`Dirty migrations detected, resolve before running up:\n${names}`);
+    }
 
     if (!existsSync(MIGRATIONS_DIR)) {
         console.log("No migrations directory found. Nothing to run.");
         return;
     }
 
-    // get all .up.sql files, sorted ascending by filename (timestamp prefix ensures order)
     const files = readdirSync(MIGRATIONS_DIR)
-        .filter((f) => f.endsWith(".up.sql"))
+        .filter((f) => f.endsWith(".sql"))
         .sort();
 
     if (files.length === 0) {
@@ -32,12 +40,11 @@ export async function up() {
         return;
     }
 
-    // get already applied migrations from db
-    const [rows] = await db.query("SELECT name FROM migrations");
+    const [rows] = await db.query("SELECT name FROM migrations WHERE dirty = FALSE");
     const applied = new Set(rows.map((r) => r.name));
 
     const pending = files
-        .map((f) => ({ file: f, name: f.replace(".up.sql", "") }))
+        .map((f) => ({ file: f, name: f.replace(".sql", "") }))
         .filter((m) => !applied.has(m.name));
 
     if (pending.length === 0) {
@@ -48,37 +55,17 @@ export async function up() {
     console.log(`Running ${pending.length} migration(s)...`);
 
     for (const migration of pending) {
-        const downFile = join(MIGRATIONS_DIR, `${migration.name}.down.sql`);
+        const sql = readFileSync(join(MIGRATIONS_DIR, migration.file), "utf8").trim();
 
-        if (!existsSync(downFile)) {
-            console.error(`Missing down file for migration: ${migration.name}`);
-            console.error(`Expected: ${downFile}`);
-            process.exit(1);
-        }
+        console.log(`  Applying: ${migration.name}`);
 
-        const sql = readFileSync(
-            join(MIGRATIONS_DIR, migration.file),
-            "utf8",
-        ).trim();
-
-        if (isSqlEmpty(sql)) {
-            console.error(
-                `Migration ${migration.name} contains no SQL (only comments/whitespace).`,
-            );
-            console.error(`Add SQL or delete the migration files.`);
-            process.exit(1);
-        }
+        await db.query("INSERT INTO migrations (name, dirty) VALUES (?, TRUE)", [migration.name]);
 
-        console.log(`  Applying: ${migration.name}`);
         try {
             await db.query(sql);
-            await db.query("INSERT INTO migrations (name) VALUES (?)", [
-                migration.name,
-            ]);
+            await db.query("UPDATE migrations SET dirty = FALSE WHERE name = ?", [migration.name]);
         } catch (err) {
-            console.error(`Migration failed: ${migration.name}`);
-            console.error(err.message);
-            process.exit(1);
+            throw new Error(`Migration failed: ${migration.name}\n${err.message}`);
         }
     }
 

package/src/db.js

@@ -1,24 +1,34 @@
 // src/db.js
 import mysql from "mysql2/promise";
-import { env } from "./env.js";
 
 let connection;
 
-try {
-    connection = await mysql.createConnection({
-        socketPath: env.DB_SOCK,
-        database: env.DB_NAME,
-        user: env.DB_USER,
-        password: env.DB_PASS,
-        multipleStatements: true,
-    });
-} catch (err) {
-    console.error(`Failed to connect to database:`);
-    console.error(`  Socket: ${env.DB_SOCK}`);
-    console.error(`  Database: ${env.DB_NAME}`);
-    console.error(`  User: ${env.DB_USER}`);
-    console.error(`  Error: ${err.message}`);
-    process.exit(1);
+export default async function getDb() {
+    if (connection) return connection;
+
+    const { env } = await import("./env.js");
+
+    try {
+        connection = await mysql.createConnection({
+            socketPath: env.DB_SOCK,
+            database: env.DB_NAME,
+            user: env.DB_USER,
+            password: env.DB_PASS,
+            multipleStatements: true,
+        });
+    } catch (err) {
+        console.error(`Failed to connect to database: ${err.message}`);
+        process.exit(1);
+    }
+
+    return connection;
 }
 
-export default connection;
+export async function closeDb() {
+    if (!connection) return;
+    try {
+        await connection.end();
+    } catch {
+        // connection may already be closed
+    }
+}

package/src/env.js

@@ -2,18 +2,41 @@
 import { readFileSync, existsSync } from "fs";
 import { parse } from "dotenv";
 
+function parseArgs() {
+    const args = process.argv.slice(2);
+    let envFile = null;
+
+    for (let i = 0; i < args.length; i++) {
+        const arg = args[i];
+        if (arg === "--env" && i + 1 < args.length) {
+            envFile = args[i + 1];
+            break;
+        }
+        if (arg.startsWith("--env=")) {
+            envFile = arg.replace("--env=", "");
+            break;
+        }
+    }
+
+    return envFile;
+}
+
 function loadEnv() {
+    const envArg = parseArgs();
+
     let envFile;
-    let isProd;
 
-    if (existsSync(".env.development")) {
-        envFile = ".env.development";
-        isProd = false;
+    if (envArg) {
+        envFile = envArg;
     } else if (existsSync(".env")) {
         envFile = ".env";
-        isProd = true;
     } else {
-        console.error("No .env.development or .env file found.");
+        console.error("No .env file found. Use --env to specify a different file.");
+        process.exit(1);
+    }
+
+    if (!existsSync(envFile)) {
+        console.error(`Env file not found: ${envFile}`);
         process.exit(1);
     }
 
@@ -24,9 +47,7 @@ function loadEnv() {
     const missing = required.filter((k) => !parsed[k]);
 
     if (missing.length > 0) {
-        console.error(
-            `Missing required variables in ${envFile}: ${missing.join(", ")}`,
-        );
+        console.error(`Missing required variables in ${envFile}: ${missing.join(", ")}`);
         process.exit(1);
     }
 
@@ -35,7 +56,6 @@ function loadEnv() {
         DB_NAME: parsed.DB_NAME,
         DB_USER: parsed.DB_USER,
         DB_PASS: parsed.DB_PASS,
-        isProd,
         envFile,
     };
 }