@oalo-ai/cli 0.1.1 → 0.2.0

package/man/oalo.1

@@ -1,4 +1,4 @@
-.TH OALO 1 "" "oalo 0.1.0" "OALO CLI"
+.TH OALO 1 "" "oalo 0.1.1" "OALO CLI"
 .SH NAME
 oalo \- wire a local app into your Oalo workspace
 .SH SYNOPSIS
@@ -368,25 +368,51 @@ Each one says what was seen: which process, how many restarts, a memory kill wit
 .PP
 Exit codes: 0 no open incident; 1 an incident is open, or the request failed; 2 bad arguments or no CLI token.
 .SS db
-All of an env's data as plain SQL (every app's own tables and the Oalo models), to keep a copy or take it anywhere.
+All of an env's data as plain SQL (every app's own tables and the Oalo models), to keep a copy or take it anywhere. Or load an existing Postgres database into an env.
 .PP
 \fBoalo db dump [\-\-env dev|prod] [\-\-out <file.sql>] [\-\-force]\fR
 .br
+\fBoalo db restore <file> \-\-env dev|prod [\-\-schema <name>] [\-\-check] [\-\-yes]\fR
+.br
+\fBoalo db restore \-\-from\-env <VAR> \-\-env dev|prod [\-\-schema <name>] [\-\-check] [\-\-yes]\fR
+.br
 .TP
 \fB\-\-env dev|prod\fR
-Which env's data. Default prod.
+dump: which env's data. Default prod. restore: which env the data goes into, always on the command line.
 .TP
 \fB\-\-out <file.sql>\fR
 Where to write it. Default <workspace>\-<env>\-<date>.sql here.
 .TP
 \fB\-\-force\fR
 Replace the file if it exists.
+.TP
+\fB\-\-from\-env <VAR>\fR
+restore: copy from a running database instead of a file. VAR is the environment variable that holds its postgres:// or postgresql:// URL. Only its host is ever shown.
+.TP
+\fB\-\-schema <name>\fR
+restore: the source schema to load. Default public, or the only one when there's just one.
+.TP
+\fB\-\-check\fR
+restore: do everything but the write, and report what would happen. Nothing is written, so it never asks.
+.TP
+\fB\-\-yes\fR
+restore: go ahead with prod without asking. Needed for prod when there's no terminal to ask at.
 .IP \(bu 2
 Workspace admins only: it holds every member's data. Restore it into any Postgres with psql \-f.
 .IP \(bu 2
 Your code is in git (oalo deploy \-\-pull); this is the data. Together they are everything you need to run your apps elsewhere.
+.IP \(bu 2
+restore loads a dump file made by pg_dump (like Heroku's latest.dump), up to 10 GB, or copies from a database URL. Admins only, and one restore at a time per Base.
+.IP \(bu 2
+The tables land in the env's schema, owned by the app's database login, so its Procfile \fBrelease:\fR migrations can alter them. Your app reads them at DATABASE_URL.
+.IP \(bu 2
+It's refused if a table name already exists there, and the names are listed. A failed restore leaves the env as it was.
+.IP \(bu 2
+The URL is never a flag value, since that stays in your shell history. Put it in a variable and pass \-\-from\-env VAR.
+.IP \(bu 2
+Run it with \-\-check first. prod asks before it writes; without a terminal it needs \-\-yes. Stop writes to the old database before the copy: writes made after it starts are lost.
 .PP
-Exit codes: 0 a whole dump was written; 1 refused, unreachable, or cut off (the partial file is removed); 2 bad arguments, the file exists, or no CLI token.
+Exit codes: 0 a whole dump was written, or the restore (or check) passed; 1 refused, unreachable, cut off (a partial dump is removed), or the restore failed or was declined; 2 bad arguments, the file exists or is missing, or no CLI token.
 .SS metrics
 A number from the workspace's data: count, sum or average over a model's rows, events or contacts, grouped or over time.
 .PP
@@ -911,6 +937,41 @@ How a server app starts: its Procfile (\fBweb:\fR, workers, \fBrelease:\fR), els
 Admins and members with Developer access only. Progress streams while it runs.
 .PP
 Exit codes: 0 deployed; 1 refused or failed; 2 bad arguments, no CLI token, or not in a git repo.
+.SS migrate
+Move an app from Heroku, Railway, Render, Fly.io or Primcloud: its Procfile, sizes, health path and variables, worked out from its files and that platform's CLI.
+.PP
+\fBoalo migrate <heroku|railway|render|fly|primcloud> [\-\-from\-app <name>] [\-\-env\-file <file>] [\-\-json]\fR
+.br
+\fBoalo migrate <platform> \-\-apply [\-\-app <slug>]\fR
+.br
+.TP
+\fB\-\-from\-app <name>\fR
+The app or service on the old platform (a Heroku or Fly app, a Railway or Render service). Default: what that platform's CLI is linked to here, or fly.toml's app.
+.TP
+\fB\-\-env\-file <file>\fR
+More variables, from a .env file (NAME=value lines). They win over what the platform's CLI reported. For Primcloud, and for Fly secrets, which can't be read back.
+.TP
+\fB\-\-apply\fR
+Write the Procfile and .oalo/app.json (never over an existing Procfile; app.json only gains keys it lacks), and set the variables on the app's prod env. Without it, nothing changes.
+.TP
+\fB\-\-no\-cli\fR
+Don't run the old platform's CLI; read only the files here and \-\-env\-file.
+.TP
+\fB\-\-app <slug>\fR
+The Oalo app the variables go to. Default: the app this folder is linked to (oalo deploy \-\-new links it).
+.TP
+\fB\-\-json\fR
+The plan as JSON: names only, never a value.
+.IP \(bu 2
+Run it in the app's repo. It reads fly.toml, railway.json or railway.toml, render.yaml, app.json and the Procfile, and asks the platform's CLI for variables (heroku, railway), secret names (fly) and dyno sizes (heroku) when it's installed and signed in.
+.IP \(bu 2
+Variables: PORT and each platform's own (HEROKU_*, RAILWAY_*, RENDER_*, FLY_*) are left out, and DATABASE_URL becomes EXTERNAL_DATABASE_URL, since DATABASE_URL on Oalo is the Base's own Postgres. Values are never printed or written to a file; \-\-apply sends them to prod without restarting the app.
+.IP \(bu 2
+It says what doesn't carry over: cron jobs, volumes and disks, private addresses (*.railway.internal, *.flycast), extra replicas, build commands, an Aptfile, a VM bigger than large.
+.IP \(bu 2
+The order: oalo migrate <platform> (read the plan), \-\-apply, commit, oalo deploy \-\-new, oalo migrate <platform> \-\-apply again (now the app exists, it sets the variables), then the data (oalo db restore) and oalo deploy \-\-env prod. \fBoalo docs migrating\-from\-<platform>\fR walks through it.
+.PP
+Exit codes: 0 the plan was shown, or applied; 1 a config file couldn't be read, or the variables couldn't be set; 2 bad arguments, an unknown platform, or no CLI token.
 .SS env
 An app's environment variables on Oalo (API keys, settings), per env: list, set, unset, rotate\-token.
 .PP

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@oalo-ai/cli",
-  "version": "0.1.1",
+  "version": "0.2.0",
   "description": "Wire a local app into your OALO workspace: link a token, generate typed bindings, and serve the build surface over MCP.",
   "type": "module",
   "bin": {

package/plugin/skills/oalo/SKILL.md

@@ -263,6 +263,29 @@ oalo deploy --app shop --env prod    # build and swap the live release
 6. **Rolling back** (`oalo deploy --rollback`, MCP: `oalo_releases`) changes what
    live visitors get: only when the person asks. `--history` lists the
    releases.
+7. **Bringing a database in** (from Heroku, Railway, Render, Fly.io or
+   Primcloud): `oalo db restore <dump file> --env prod --check`, then the same
+   without `--check` once the person says go. The dump comes from the old
+   platform (Heroku: `heroku pg:backups:capture`, then `heroku
+   pg:backups:download` gives `latest.dump`). Or the person puts the old
+   database's URL in an environment variable and you pass `--from-env VAR`.
+   Never write a database URL into a command or a flag. Always name `--env`.
+   Have the person stop writes to the old database first: anything written
+   after the copy starts is lost. `oalo docs migrating-from-heroku` (or
+   `-railway`, `-render`, `-fly`, `-primcloud`) has each platform's steps.
+8. **Moving an app from another platform** (Heroku, Railway, Render, Fly.io,
+   Primcloud): in the app's repo, run `oalo migrate <platform> --from-app
+   <name>` first (the app or service's name on the old platform, the Heroku
+   app say) and show the person the plan; it changes nothing. Only `--apply`
+   writes the Procfile and `.oalo/app.json` and sets the variables, and it
+   sets them only once the app exists, so the order is `--apply`, commit,
+   `oalo deploy --new`, then the same `oalo migrate ... --apply` again. The
+   variables go to the app's prod settings with nothing restarted or deployed,
+   so this doesn't put it live; the preview keeps its own (`--env dev`, test
+   keys). It never prints a value. Don't copy variables by hand with `oalo env
+   set NAME=value`: for values it can't read (Fly secrets, Primcloud) the
+   person puts them in a .env file and you pass `--env-file`. Then the data
+   (rule 7), and live only when the person says so.
 
 ## An app's environment variables
 

package/src/commands/db.js

@@ -1,7 +1,24 @@
-import { createWriteStream, existsSync, rmSync, statSync } from "node:fs";
+import { createReadStream, createWriteStream, existsSync, rmSync, statSync } from "node:fs";
+import { Readable, Transform } from "node:stream";
+import http from "node:http";
+import https from "node:https";
+import { createInterface } from "node:readline/promises";
 import { resolve as resolveConfig, controlPlane, HOW_TO_CONNECT } from "../config.js";
 import { dim } from "../events.js";
 
+const USAGE =
+  "usage: oalo db dump [--env dev|prod] [--out <file.sql>]\n" +
+  "       oalo db restore <file> --env dev|prod [--schema <name>] [--check] [--yes]\n" +
+  "       oalo db restore --from-env <VAR> --env dev|prod [--schema <name>] [--check] [--yes]";
+
+/** `oalo db dump` and `oalo db restore`. */
+export async function db(flags, rest, opts = {}) {
+  const sub = rest[1];
+  if (sub === "dump") return dump(flags, opts);
+  if (sub === "restore") return restore(flags, rest, opts);
+  return fail(USAGE, 2);
+}
+
 /**
  * `oalo db dump [--env prod] [--out file.sql]`: all of an env's data as plain
  * SQL (every app's own tables and the Oalo models, with their rows), so it can
@@ -9,9 +26,7 @@ import { dim } from "../events.js";
  * with pg_dump's own "dump complete" line; a dump without it is cut off, and
  * the file is removed rather than left looking whole.
  */
-export async function db(flags, rest, opts = {}) {
-  const sub = rest[1];
-  if (sub !== "dump") return fail("usage: oalo db dump [--env dev|prod] [--out <file.sql>]", 2);
+async function dump(flags, opts) {
   const env = flags.env === "dev" ? "dev" : "prod";
   const { url, token } = controlPlane(resolveConfig(flags));
   if (!url || !token) return fail(`oalo db dump needs a CLI token.\n\n${HOW_TO_CONNECT}`, 2);
@@ -52,6 +67,323 @@ export async function db(flags, rest, opts = {}) {
   console.log(`${out}  ${dim(`${mb} MB, the ${env} data as SQL. Restore it with: psql "$DATABASE_URL" -f ${out}`)}`);
 }
 
+export const RESTORE_LIMIT = 10 * 1024 ** 3;
+
+/**
+ * `oalo db restore <file> --env dev|prod` or `--from-env VAR`: load an
+ * existing Postgres database (a dump file, or a URL the Base copies from) into
+ * one env of this Base. The reverse of dump. The URL comes from an environment
+ * variable, never a flag (shell history), and is never printed: only its host
+ * is named. A live restore asks first at a terminal and needs --yes without
+ * one; --check writes nothing, so it never asks.
+ *
+ * opts (for tests): fetch, env (the environment variables), isTTY (stdin is a
+ * terminal), confirm(question) -> answer, progress (stderr is a terminal).
+ */
+async function restore(flags, rest, opts) {
+  const vars = opts.env || process.env;
+  const env = flags.env;
+  if (env !== "dev" && env !== "prod") return fail("oalo db restore needs --env dev or --env prod: which env the data goes into.", 2);
+
+  const file = rest[2];
+  const fromEnv = flags["from-env"];
+  if (file && fromEnv !== undefined) return fail("pick one: a dump file, or --from-env VAR for a database URL.", 2);
+  if (!file && fromEnv === undefined) return fail(USAGE, 2);
+  if (flags.schema !== undefined && (typeof flags.schema !== "string" || !flags.schema)) return fail("--schema needs the source schema's name, like --schema public", 2);
+  const check = !!flags.check;
+
+  let source;
+  if (file) {
+    let st;
+    try {
+      st = statSync(file);
+    } catch {
+      return fail(`${file} doesn't exist.`, 2);
+    }
+    if (!st.isFile()) return fail(`${file} isn't a file. Give the dump file's path.`, 2);
+    if (st.size > RESTORE_LIMIT) return fail(`${file} is ${size(st.size)}, over the 10 GB a restore can take.`, 1);
+    source = { file, bytes: st.size };
+  } else {
+    const name = typeof fromEnv === "string" ? fromEnv : "";
+    if (!/^[A-Za-z_][A-Za-z0-9_]*$/.test(name)) return fail("--from-env takes the name of the environment variable that holds the database URL, like --from-env SOURCE_DATABASE_URL", 2);
+    const value = (vars[name] || "").trim();
+    if (!value) return fail(`${name} is empty or not set. Put the database URL in it, then run this again.`, 2);
+    const host = pgHost(value);
+    if (!host) return fail(`${name} doesn't hold a database URL. It needs one that starts with postgres:// or postgresql:// and names a host.`, 2);
+    source = { url: value, host };
+  }
+
+  const { url, token } = controlPlane(resolveConfig(flags));
+  if (!url || !token) return fail(`oalo db restore needs a CLI token.\n\n${HOW_TO_CONNECT}`, 2);
+
+  // A live restore asks first. --check writes nothing, so it never does.
+  if (env === "prod" && !check && !flags.yes) {
+    const tty = opts.isTTY ?? process.stdin.isTTY;
+    if (!tty) return fail("restoring into prod needs --yes when there's no terminal to ask at. Run it with --check first, then add --yes.", 2);
+    const question = source.file ? `Restore ${size(source.bytes)} into prod? [y/N] ` : `Restore the database at ${source.host} into prod? [y/N] `;
+    const answer = await (opts.confirm || ask)(question);
+    if (!/^(y|yes)$/i.test(String(answer || "").trim())) return fail("Nothing was restored.", 1);
+  }
+
+  const scrub = scrubber(source.url);
+  const q = new URLSearchParams({ env, ...(typeof flags.schema === "string" ? { schema: flags.schema } : {}), ...(check ? { check: "true" } : {}) });
+  const endpoint = `${url.replace(/\/$/, "")}/api/cli/v1/db/restore?${q}`;
+  console.error(dim(`${check ? "checking" : "restoring"} ${source.file ? `${source.file} (${size(source.bytes)})` : `the database at ${source.host}`} ${check ? "against" : "into"} ${env} on ${url}`));
+
+  let res;
+  let upload = null;
+  try {
+    if (source.file) {
+      upload = uploadBody(source, opts.progress ?? process.stderr.isTTY);
+      const headers = { authorization: `Bearer ${token}`, "content-type": "application/octet-stream", "content-length": String(source.bytes) };
+      res = opts.fetch
+        ? await opts.fetch(endpoint, { method: "POST", headers, body: upload.web(), duplex: "half" })
+        : await sendFile(endpoint, headers, upload.stream);
+    } else {
+      res = await (opts.fetch || fetch)(endpoint, {
+        method: "POST",
+        headers: { authorization: `Bearer ${token}`, "content-type": "application/json" },
+        body: JSON.stringify({ secret_url: source.url }),
+      });
+    }
+  } catch (e) {
+    upload?.stop();
+    const why = scrub(e.cause?.message || e.message);
+    return fail(`${upload ? `the upload to ${url} didn't go through` : `could not reach ${url}`} (${why}). Nothing was restored.`, 1);
+  }
+
+  // A refusal can come before the upload is done: read it, then stop sending.
+  if (!res.ok) {
+    upload?.pause();
+    const body = await res.json().catch(() => ({}));
+    upload?.stop();
+    res.close?.();
+    return fail(scrub(body?.error?.message || `the restore was refused (HTTP ${res.status})`), 1);
+  }
+
+  let last = null;
+  try {
+    last = await follow(res, (m) => (upload ? upload.note(() => console.error(dim(`  ${scrub(m)}`))) : console.error(dim(`  ${scrub(m)}`))));
+  } catch {
+    last = null;
+  } finally {
+    upload?.stop();
+    res.close?.();
+  }
+
+  if (!last) {
+    return fail("the connection closed before the restore finished. A restore that fails leaves the env as it was; run it again with --check to see where it stands.", 1);
+  }
+  if (last.error) {
+    console.error(`restore failed: ${scrub(last.error.message || last.error.code || "unknown error")}`);
+    const names = last.error.conflicts || [];
+    if (names.length) console.error(dim(`  already in ${env}: ${names.join(", ")}`));
+    process.exitCode = 1;
+    return;
+  }
+  summary(last, env, scrub);
+}
+
+/** The summary of a finished restore, or of a check. */
+function summary(r, env, scrub) {
+  if (r.message) console.log(scrub(r.message));
+  const tables = r.tables || [];
+  const w = Math.max(0, ...tables.map((t) => String(t.name).length));
+  for (const t of tables) {
+    const rows = `${Number(t.rows || 0).toLocaleString("en-US")} ${t.rows === 1 ? "row" : "rows"}`;
+    console.log(`  ${String(t.name).padEnd(w)}  ${dim(`${rows}, ${size(t.bytes || 0)}`)}`);
+  }
+  if (r.extensions?.length) console.log(dim(`extensions: ${r.extensions.join(", ")}`));
+  if (r.skipped?.length) console.log(dim(`skipped: ${r.skipped.join(", ")}`));
+  for (const x of r.warnings || []) console.log(`warning: ${scrub(x)}`);
+  if (r.owner) console.log(dim(`owner: ${r.owner}, the app's database login, so its release: migrations can alter them`));
+  if (r.check) {
+    const names = r.conflicts || [];
+    if (names.length) {
+      console.log(`already in ${env}, so the restore would be refused: ${names.join(", ")}`);
+      process.exitCode = 1;
+    }
+    console.log(dim("Nothing was written."));
+  }
+}
+
+/** Print each progress message; return the final line (done or error), or null. */
+async function follow(res, print) {
+  const decoder = new TextDecoder();
+  let buf = "";
+  let last = null;
+  const take = (raw) => {
+    raw = raw.trim();
+    if (!raw) return;
+    let msg;
+    try {
+      msg = JSON.parse(raw);
+    } catch {
+      return;
+    }
+    if (msg.done || msg.error) last = msg;
+    else if (msg.phase !== "waiting" && msg.message) print(msg.message);
+  };
+  for await (const part of res.body) {
+    buf += decoder.decode(part, { stream: true });
+    let i;
+    while ((i = buf.indexOf("\n")) >= 0) {
+      take(buf.slice(0, i));
+      buf = buf.slice(i + 1);
+    }
+  }
+  take(buf + decoder.decode());
+  return last;
+}
+
+/**
+ * The file as a request body, read as it's sent and counted. At a terminal one
+ * stderr line shows how far it got; progress messages print above it.
+ */
+function uploadBody({ file, bytes }, tty) {
+  const total = size(bytes);
+  let sent = 0;
+  let shown = "";
+  let active = false;
+  let done = false;
+  const clear = () => active && !done && process.stderr.write("\r\x1b[K");
+  const draw = (force) => {
+    if (!tty || done) return;
+    const line = `uploading ${size(sent)} of ${total}`;
+    if (line === shown && !force) return;
+    shown = line;
+    active = true;
+    process.stderr.write(`\r\x1b[K${dim(line)}`);
+  };
+  const src = createReadStream(file);
+  const stream = new Transform({
+    transform(chunk, _enc, cb) {
+      sent += chunk.length;
+      draw();
+      cb(null, chunk);
+    },
+    flush(cb) {
+      if (active && !done) process.stderr.write(`\r\x1b[K${dim(`uploaded ${total}`)}\n`);
+      done = true;
+      cb();
+    },
+  });
+  src.on("error", (e) => stream.destroy(e));
+  src.pipe(stream);
+  if (!tty) console.error(dim(`uploading ${total}`));
+  return {
+    stream,
+    web: () => Readable.toWeb(stream),
+    /** Print a line without breaking the progress line. */
+    note(print) {
+      clear();
+      print();
+      if (active) draw(true);
+    },
+    pause: () => src.unpipe(stream),
+    /** Stop reading the file (a refusal, an error, or the end). */
+    stop() {
+      clear();
+      done = true;
+      src.destroy();
+      stream.destroy();
+    },
+  };
+}
+
+/**
+ * POST the file with node:http and `expect: 100-continue`: the body goes once
+ * the server says to (or after CONTINUE_WAIT_MS, for one that never does), so
+ * a refusal made from the headers (too big, not enough disk) arrives before
+ * any bytes are sent. fetch can't wait like this, and when a server refuses
+ * mid-upload and closes, it fails with EPIPE and the server's message is lost.
+ * The reply is returned as a web Response; `close()` ends the request.
+ */
+export const CONTINUE_WAIT_MS = 3000;
+
+function sendFile(endpoint, headers, body) {
+  return new Promise((resolve, reject) => {
+    const u = new URL(endpoint);
+    let replied = false;
+    let started = false;
+    const req = (u.protocol === "https:" ? https : http).request(u, { method: "POST", headers: { ...headers, expect: "100-continue" } });
+    const start = () => {
+      if (started || replied) return;
+      started = true;
+      body.pipe(req);
+    };
+    const timer = setTimeout(start, CONTINUE_WAIT_MS);
+    req.on("continue", () => (clearTimeout(timer), start()));
+    req.on("response", (incoming) => {
+      replied = true;
+      clearTimeout(timer);
+      const h = new Headers();
+      for (const [k, v] of Object.entries(incoming.headers)) h.set(k, Array.isArray(v) ? v.join(", ") : String(v));
+      const res = new Response(Readable.toWeb(incoming), { status: incoming.statusCode, headers: h });
+      res.close = () => req.destroy();
+      resolve(res);
+    });
+    // After the reply has come, a write error (the server stopped reading) is expected.
+    req.on("error", (e) => {
+      clearTimeout(timer);
+      if (!replied) reject(e);
+    });
+    body.on("error", (e) => req.destroy(e));
+    req.flushHeaders();
+  });
+}
+
+/** The host of a postgres:// or postgresql:// URL, or null when it isn't one. */
+export function pgHost(value) {
+  if (!/^postgres(ql)?:\/\//i.test(value)) return null;
+  try {
+    return new URL(value).hostname || null;
+  } catch {
+    return null;
+  }
+}
+
+/**
+ * Removes a database URL's credentials from text the server or the network
+ * stack wrote, before it's printed: the URL itself, its password, and the
+ * user:password part of any postgres URL.
+ */
+function scrubber(secret) {
+  let password = "";
+  let host = "";
+  if (secret) {
+    try {
+      const u = new URL(secret);
+      password = decodeURIComponent(u.password || "");
+      host = u.hostname;
+    } catch {}
+  }
+  return (text) => {
+    let s = String(text ?? "");
+    if (secret) s = s.split(secret).join(`the database at ${host}`);
+    s = s.replace(/(postgres(?:ql)?:\/\/)[^\s@/"']*@/gi, "$1***@");
+    if (password.length >= 4) s = s.split(password).join("***");
+    return s;
+  };
+}
+
+/** Bytes as people read them: 1.2 GB, 340 MB, 12 KB. */
+export function size(n) {
+  if (n >= 1024 ** 3) return `${(n / 1024 ** 3).toFixed(1)} GB`;
+  if (n >= 1024 ** 2) return `${n >= 10 * 1024 ** 2 ? Math.round(n / 1024 ** 2) : (n / 1024 ** 2).toFixed(1)} MB`;
+  if (n >= 1024) return `${Math.round(n / 1024)} KB`;
+  return `${n} B`;
+}
+
+async function ask(question) {
+  const rl = createInterface({ input: process.stdin, output: process.stderr });
+  try {
+    return await rl.question(question);
+  } finally {
+    rl.close();
+  }
+}
+
 function fail(message, code) {
   console.error(message);
   process.exitCode = code;