@newly/cli 0.2.163 → 0.2.165

package/README.md

@@ -157,9 +157,141 @@ Set a backend secret. **Dev only.** Redeploy to apply.
 - `--project <id>`
 
 ### `newly db url`
-Print a readonly URL for the Newly DB. **Dev only.**
+Print a direct PostgreSQL connection URL for the selected database.
+- `--env <dev|prod>` (default `dev`)
+- `--role <readonly|owner>` (default `readonly`)
 - `--project <id>`
 
+```bash
+newly db url --env prod --role owner --project PROJECT_ID
+```
+
+Owner access allows direct SQL, migrations, exports and logical backups using
+PostgreSQL tools such as `psql` and `pg_dump`. Save the URL in your own secret
+manager before an outage. Once saved, it connects directly to the database;
+Newly's API, worker and CLI do not need to be running. The database provider and
+network must remain available. Retrieving a URL still requires Newly to be
+available. This exports the existing app owner credential; changing its password
+also affects the deployed API/Auth services. It does not grant a provider account
+or provider snapshot controls.
+
+The URL contains a password. New credential exports require the project owner's
+user credential in either environment; managed agent credentials cannot export
+them. Responses are not cached and credentials do not enter operation results.
+Readonly access retains the role's existing grants; it does not add access to
+custom app schemas. The legacy dev readonly command keeps its existing access
+rules.
+
+### Database inspection, backups and restore
+
+These commands keep database provider credentials on the platform. Production
+requires the project owner's user credential from `newly login`; managed agent
+sessions remain dev-only. Dev requires an admin or owner. All commands accept
+`--project` and `--json`; `--env` defaults to `dev`.
+
+```bash
+newly db migrations --env prod --json
+newly db migrations --env prod --services api,auth
+newly db backup status --env prod
+newly db backup list --env prod
+newly db backup create --env prod --name before-catalog --retention-hours 48
+```
+
+Migration inspection compares live ledgers, read in a read-only transaction,
+with the selected services' local journals and SQL hashes. It reports applied
+and pending migrations, edited SQL, missing local entries and skipped
+migrations. A discrepancy returns a nonzero exit code. This is a comparison
+with the current checkout, not a deployment dry run or an application
+compatibility check; it does not apply or select migrations.
+
+Manual snapshots expire after 24 hours by default; `--retention-hours` accepts
+1 to 168. At most five manual snapshots and three restore previews can exist per
+database. Newly does not manage backup schedules or take automatic predeployment
+snapshots. `backup status` reports `managed_backup_scheduling: false` and
+`automatic_predeploy_snapshot: false`.
+
+Users can automate the manual command from their own cron or CI job:
+
+```bash
+newly db backup create --env prod --project PROJECT_ID \
+  --name owner-backup-job --retention-hours 24
+```
+
+The command runs without an interactive prompt. Supply `NEWLY_API_URL` and an
+owner user credential in `NEWLY_API_TOKEN` through the job's configuration and
+secret store, or use the owner's authenticated CLI profile. Managed agent
+credentials cannot operate on production. Users own the job, its schedule,
+credentials, retention, and monitoring. Leave room below the five-snapshot limit
+for expiration cleanup and extra manual snapshots; alert on nonzero exit codes
+and expired credentials. Inspect a pending or uncertain operation before retrying.
+This automation requires Newly and the database provider to be available. For
+independent access during a Newly outage, save the owner connection URL beforehand
+and run SQL clients or `pg_dump` from infrastructure you control.
+
+See [Database backups](https://docs.newly.app/backend/database-backups) for token
+setup, a GitHub Actions example, failure handling, and independent database exports.
+
+Restore starts with a separate copy. Its result includes the preview operation
+ID, migration ledgers and whether current API/Auth/read-only database
+credentials work against the copy. Inspect that result before deciding to apply:
+
+```bash
+newly db restore prepare SNAPSHOT_ID --env prod --schemas api,auth
+newly db operation PREVIEW_OPERATION_ID --env prod
+newly db restore apply PREVIEW_OPERATION_ID --env prod \
+  --confirm-project PROJECT_ID --accept-data-loss
+# Or remove an unapplied copy and its preview compute:
+newly db restore discard PREVIEW_OPERATION_ID --env prod
+```
+
+Applying switches the whole database, including Auth and every app schema, to
+the snapshot state. Writes after the snapshot can be lost and connections
+restart. App traffic is not paused. The deployed application, secrets, object
+storage and queues are not rolled back. The owner controls when to apply and
+validates application compatibility afterward.
+
+An existing deployment blocks restore. Database operations hold the backend row
+lock to block new deployments or teardown during the switch. After verifying the
+switch, Newly attempts to delete the previous branch; snapshots remain listed.
+If `cleanup_required` is reported, discard remaining unapplied previews and run
+`newly db restore cleanup PREVIEW_OPERATION_ID --env prod` to remove the old
+branch. Discard can remove stale or failed previews and never the live branch.
+Pause user-owned
+automation and coordinate application writes before applying a restore, then
+validate the restored application before resuming them. If an external provider
+backup schedule exists, restore refuses to switch until it is disabled through
+its original management tool. Newly does not copy, disable, or configure that
+schedule; the owner manages it after the restore.
+
+Operations run asynchronously. A timeout or lost response does not cancel a
+provider operation; inspect the printed operation ID with `newly db operation`
+and check database state before submitting another request. A successful restore
+confirms the provider database switch, not the application's behavior.
+
+Production URL export (including read-only), owner export in either environment,
+production restore apply and previous-branch cleanup require
+the project owner's CLI key plus a fresh email verification code. The CLI
+prompts for a six-digit code sent to the account's verified email. Each code is
+bound to one key, project, environment, action and request, expires after five
+minutes and can be consumed once. Web/desktop sessions cannot perform these
+actions. Never share a code with an agent. Manual snapshot creation stays
+noninteractive for user-owned automation.
+
+To invalidate saved owner and read-only URLs and update API/Auth database credentials:
+
+```bash
+newly db credentials rotate --env prod --project PROJECT_ID \
+  --confirm-project PROJECT_ID --accept-connection-restart
+```
+
+Rotation also requires fresh email authorization. It resets both passwords and
+closes existing connections using these roles; export new URLs afterward and
+update external jobs. A snapshot retains
+its original database credentials, so pre-rotation snapshots can fail restore
+compatibility checks. Export and rotation events have durable audit records
+without URLs or verification codes. One database operation can be queued or
+running per project; wait and inspect results before retrying a busy request.
+
 ### `newly email`
 Retired. It never had a server to talk to; send mail from your API service with
 your own provider. Running it prints that and exits non-zero.

package/dist/commands/database.js

@@ -0,0 +1,192 @@
+import { randomUUID } from 'node:crypto';
+import { ApiClient, friendlyApiError } from '../api.js';
+import { requireCredentials, resolveProject } from '../config.js';
+import { readLocalMigrations, compareMigrations } from '../migration-plan.js';
+import { authorizeDatabaseAction } from '../database-authorization.js';
+export async function runDatabaseOperation(client, projectId, env, operation, options = {}) {
+    const operationId = options.operationId ?? randomUUID();
+    const root = `/newly_backend/database/${encodeURIComponent(projectId)}/operations`;
+    options.onStarted?.(operationId);
+    await client.post(root, { ...operation, env, operation_id: operationId });
+    const deadline = Date.now() + (options.timeoutMs ?? 300_000);
+    const sleep = options.sleep ?? ((ms) => new Promise((done) => setTimeout(done, ms)));
+    for (;;) {
+        const result = await client.get(`${root}/${operationId}?env=${env}`);
+        if (result.status === 'succeeded' && result.data)
+            return result.data;
+        if (result.status === 'failed')
+            throw new Error(result.message ?? 'Database operation failed');
+        if (result.status !== 'pending')
+            throw new Error('Unexpected database operation response');
+        if (Date.now() >= deadline)
+            throw new Error(`Database operation ${operationId} is still pending. Check it with newly db operation ${operationId} --env ${env} --project ${projectId}`);
+        await sleep(2000);
+    }
+}
+function connection(opts) {
+    if (opts.env !== 'dev' && opts.env !== 'prod')
+        throw new Error('--env must be dev or prod');
+    const creds = requireCredentials();
+    return { client: new ApiClient(creds.apiUrl, creds.token), projectId: resolveProject(process.cwd(), opts.project) };
+}
+function common(command) {
+    return command.option('--env <env>', 'dev or prod (production requires the owner’s user credential)', 'dev').option('--project <id>', 'project id').option('--json', 'print structured JSON');
+}
+const handleError = (error) => { console.error(friendlyApiError(error)); process.exitCode = 1; };
+export function registerDatabaseHelpers(db) {
+    common(db.command('migrations').description('Compare live migration ledgers with this checkout; does not apply SQL'))
+        .option('--services <names>', 'comma-separated services from backend/newly.services.json')
+        .action(async (opts) => {
+        try {
+            const { client, projectId } = connection(opts);
+            const local = readLocalMigrations(process.cwd(), opts.services);
+            if (!local.length)
+                throw new Error('No selected service has a Drizzle migration journal and schema');
+            const data = await runDatabaseOperation(client, projectId, opts.env, { action: 'migrations', schemas: local.map((set) => set.schema) });
+            const ledgers = data.ledgers;
+            const services = local.map((set) => {
+                if (!ledgers?.[set.schema])
+                    throw new Error(`Inspection did not return ledger ${set.schema}`);
+                return compareMigrations(set, ledgers[set.schema]);
+            });
+            if (opts.json)
+                console.log(JSON.stringify({ observed_at: data.observed_at, env: opts.env, services }, null, 2));
+            else {
+                console.log(`Migration inspection (${opts.env}), observed ${String(data.observed_at)}`);
+                for (const service of services) {
+                    console.log(`\n${service.service} (${service.schema}): applied ${service.applied.length}, pending ${service.pending.length}`);
+                    console.log(`Applied: ${service.applied.map((row) => row.tag ?? row.created_at).join(', ') || 'none'}`);
+                    console.log(`Would execute from this checkout: ${service.would_execute?.join(', ') || (service.problems.length ? 'cannot determine — ledger differs' : 'none')}`);
+                    for (const problem of service.problems)
+                        console.log(`Problem: ${problem}`);
+                }
+                console.log('\nThis is a ledger comparison, not a deployment dry run or a compatibility check. Deployment uses the files submitted at that time.');
+            }
+            if (services.some((service) => service.problems.length))
+                process.exitCode = 1;
+        }
+        catch (error) {
+            handleError(error);
+        }
+    });
+    common(db.command('operation').argument('<id>', 'database operation id').description('Read the status or result of a database operation'))
+        .action(async (id, opts) => {
+        try {
+            if (!/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/.test(id))
+                throw new Error('Operation id must be a lowercase UUID');
+            const { client, projectId } = connection(opts);
+            const result = await client.get(`/newly_backend/database/${encodeURIComponent(projectId)}/operations/${id}?env=${opts.env}`);
+            console.log(JSON.stringify(result, null, 2));
+            if (result.status === 'failed')
+                process.exitCode = 1;
+        }
+        catch (error) {
+            handleError(error);
+        }
+    });
+    const backup = db.command('backup').description('Inspect and create expiring database snapshots')
+        .addHelpText('after', '\nSnapshots are created on demand. Newly does not schedule backups or create predeployment snapshots.\nYou can run backup create from your own scheduler; production requires an owner user credential and Newly availability.');
+    for (const action of ['status', 'list']) {
+        common(backup.command(action).description(action === 'status' ? 'Show recovery history, snapshot availability and automation support' : 'List available database snapshots, including previous branches'))
+            .action(async (opts) => {
+            try {
+                const { client, projectId } = connection(opts);
+                console.log(JSON.stringify(await runDatabaseOperation(client, projectId, opts.env, { action: `backup-${action}` }), null, 2));
+            }
+            catch (error) {
+                handleError(error);
+            }
+        });
+    }
+    common(backup.command('create').description('Create a snapshot and wait for the provider to finish'))
+        .option('--name <name>', 'snapshot label (up to 80 characters)')
+        .option('--retention-hours <hours>', 'expires after 1 to 168 hours', '24')
+        .action(async (opts) => {
+        try {
+            const hours = Number(opts.retentionHours);
+            if (!Number.isInteger(hours) || hours < 1 || hours > 168)
+                throw new Error('--retention-hours must be an integer from 1 to 168');
+            if (opts.name !== undefined && (!opts.name.trim() || opts.name.length > 80 || [...opts.name].some((char) => char.charCodeAt(0) < 32)))
+                throw new Error('--name must be 1 to 80 characters without control characters');
+            const { client, projectId } = connection(opts);
+            const data = await runDatabaseOperation(client, projectId, opts.env, { action: 'backup-create', name: opts.name, retention_seconds: hours * 3600 }, { onStarted: (id) => console.error(`Backup operation: ${id}`) });
+            console.log(JSON.stringify(data, null, 2));
+        }
+        catch (error) {
+            handleError(error);
+        }
+    });
+    const restore = db.command('restore').description('Prepare a restored copy, inspect it, and explicitly switch the database');
+    common(restore.command('prepare').argument('<snapshot-id>').description('Restore a snapshot into an isolated copy; keeps the live database in place'))
+        .option('--schemas <schemas>', 'comma-separated schemas whose migration ledgers to inspect', 'api,auth')
+        .action(async (snapshotId, opts) => {
+        try {
+            if (!/^[a-z0-9-]{1,60}$/.test(snapshotId))
+                throw new Error('Invalid snapshot id');
+            const { client, projectId } = connection(opts);
+            console.log(JSON.stringify(await runDatabaseOperation(client, projectId, opts.env, { action: 'restore-prepare', snapshot_id: snapshotId, schemas: opts.schemas.split(',').map((schema) => schema.trim()) }, { onStarted: (id) => console.error(`Restore preview operation: ${id}`) }), null, 2));
+        }
+        catch (error) {
+            handleError(error);
+        }
+    });
+    common(restore.command('apply').argument('<preview-operation-id>').description('Switch the whole database to the prepared copy; discards later writes and restarts connections'))
+        .requiredOption('--confirm-project <id>', 'type the exact project id to confirm the database switch')
+        .option('--accept-data-loss', 'confirm that all writes since the snapshot can be lost')
+        .action(async (previewId, opts) => {
+        try {
+            if (!/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/.test(previewId))
+                throw new Error('Invalid restore preview operation id');
+            const { client, projectId } = connection(opts);
+            if (opts.confirmProject !== projectId || opts.acceptDataLoss !== true)
+                throw new Error('Applying a restore requires --confirm-project matching the project id and --accept-data-loss');
+            const operationId = randomUUID();
+            const stepUp = opts.env === 'prod' ? await authorizeDatabaseAction(client, projectId, opts.env, 'restore-apply', operationId) : undefined;
+            console.log(JSON.stringify(await runDatabaseOperation(client, projectId, opts.env, { action: 'restore-apply', preview_operation_id: previewId, confirm_project: opts.confirmProject, accept_data_loss: true, step_up: stepUp }, { operationId, onStarted: (id) => console.error(`Restore apply operation: ${id}`) }), null, 2));
+        }
+        catch (error) {
+            handleError(error);
+        }
+    });
+    common(restore.command('discard').argument('<preview-operation-id>').description('Delete an unapplied restored copy and its preview compute'))
+        .action(async (previewId, opts) => {
+        try {
+            if (!/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/.test(previewId))
+                throw new Error('Invalid restore preview operation id');
+            const { client, projectId } = connection(opts);
+            console.log(JSON.stringify(await runDatabaseOperation(client, projectId, opts.env, { action: 'restore-discard', preview_operation_id: previewId }, { onStarted: (id) => console.error(`Restore discard operation: ${id}`) }), null, 2));
+        }
+        catch (error) {
+            handleError(error);
+        }
+    });
+    common(restore.command('cleanup').argument('<preview-operation-id>').description('Remove the previous branch if an applied restore reported cleanup_required'))
+        .action(async (previewId, opts) => {
+        try {
+            const { client, projectId } = connection(opts);
+            const operationId = randomUUID();
+            const stepUp = await authorizeDatabaseAction(client, projectId, opts.env, 'restore-cleanup', operationId);
+            console.log(JSON.stringify(await runDatabaseOperation(client, projectId, opts.env, { action: 'restore-cleanup', preview_operation_id: previewId, step_up: stepUp }, { operationId }), null, 2));
+        }
+        catch (error) {
+            handleError(error);
+        }
+    });
+    const credentials = db.command('credentials').description('Manage the runtime owner credential shared by API and Auth');
+    common(credentials.command('rotate').description('Invalidate saved owner URLs and publish a new database password to API and Auth'))
+        .requiredOption('--confirm-project <id>', 'exact project id')
+        .option('--accept-connection-restart', 'accept that resetting the password restarts database connections')
+        .action(async (opts) => {
+        try {
+            const { client, projectId } = connection(opts);
+            if (opts.confirmProject !== projectId || !opts.acceptConnectionRestart)
+                throw new Error('Rotation requires --confirm-project matching the project and --accept-connection-restart');
+            const operationId = randomUUID();
+            const stepUp = await authorizeDatabaseAction(client, projectId, opts.env, 'credentials-rotate', operationId);
+            console.log(JSON.stringify(await runDatabaseOperation(client, projectId, opts.env, { action: 'credentials-rotate', confirm_project: projectId, accept_connection_restart: true, step_up: stepUp }, { operationId, onStarted: (id) => console.error(`Credential rotation operation: ${id}`) }), null, 2));
+        }
+        catch (error) {
+            handleError(error);
+        }
+    });
+}

package/dist/commands/deploy.js

@@ -106,6 +106,7 @@ export async function runBundleDeploy(projectId, client, dir, env, allServices,
         secrets: s.secrets,
     }));
     const bundle = await buildDeployBundle({
+        projectId,
         projectDir: dir,
         services: bundleServices,
         run: runCmd,
@@ -144,7 +145,7 @@ export function registerDeploy(program) {
         .option('--region <region>', "Newly server region hint for first-time dev provisioning (defaults to 'auto')", 'auto')
         .option('--project <id>', 'project id (defaults to the linked directory)')
         .option('--service <names>', 'comma-separated service names from backend/newly.services.json')
-        .option('--git-sha <sha>', 'prod git revision to deploy from canonical project code', 'HEAD')
+        .option('--git-sha <sha>', 'legacy prod deploys only; ignored when the project deploys your local backend/ folder', 'HEAD')
         .option('--yes', 'skip the prod confirmation prompt', false)
         .action(async (opts) => {
         try {

package/dist/commands/link.js

@@ -10,6 +10,7 @@ export function registerLink(program) {
         writeProjectLink(dir, {
             projectId,
             pulledSha: existing?.projectId === projectId ? existing.pulledSha : null,
+            pulledBackendHash: existing?.projectId === projectId ? existing.pulledBackendHash : undefined,
             pulledAt: new Date().toISOString(),
         });
         console.log(`Linked ${dir} to project ${projectId}. You can now run \`newly deploy\`, \`newly status\`, etc. here.`);

package/dist/commands/misc.js

@@ -1,5 +1,6 @@
 import { ApiClient, friendlyApiError } from '../api.js';
 import { requireCredentials, resolveProject } from '../config.js';
+import { registerDatabaseHelpers } from './database.js';
 export function assertDevOnly(env, command) {
     if (env !== 'dev') {
         throw new Error(`\`newly ${command}\` is dev-only from the CLI. Prod ${command} requires the Newly web app (a browser session).`);
@@ -85,8 +86,16 @@ export async function setSecret(client, projectId, env, service, key, value) {
         value,
     });
 }
-export async function getDbUrl(client, projectId, env) {
-    const out = await client.get(`/newly_backend/readonly_db_url?project_id=${encodeURIComponent(projectId)}&env=${env}`);
+export async function getDbUrl(client, projectId, env, role = 'readonly') {
+    if (role === 'owner' || env === 'prod') {
+        const exportId = randomUUID();
+        const proof = await authorizeDatabaseAction(client, projectId, env, `${role}-export`, exportId);
+        const out = await client.post(`/newly_backend/database/${encodeURIComponent(projectId)}/connection`, { env, role, export_id: exportId, step_up: proof });
+        return out.database_url;
+    }
+    const out = await client.get(env === 'dev' && role === 'readonly'
+        ? `/newly_backend/readonly_db_url?project_id=${encodeURIComponent(projectId)}&env=${env}`
+        : `/newly_backend/database/${encodeURIComponent(projectId)}/connection?env=${env}&role=${role}`);
     return out.database_url ?? out.url ?? JSON.stringify(out);
 }
 function makeClient(project) {
@@ -162,15 +171,19 @@ export function registerMisc(program) {
         }
     });
     const db = program.command('db').description('Database helpers');
+    registerDatabaseHelpers(db);
     db.command('url')
-        .description('Print a readonly URL for the Newly DB')
+        .description('Print a direct database connection URL; owner role permits database management')
         .option('--env <env>', 'dev or prod', 'dev')
+        .option('--role <role>', 'readonly or owner (owner export requires the project owner’s user credential)', 'readonly')
         .option('--project <id>', 'project id')
         .action(async (opts) => {
         try {
-            assertDevOnly(opts.env, 'db url');
+            assertKnownEnv(opts.env, 'db url');
+            if (opts.role !== 'readonly' && opts.role !== 'owner')
+                throw new Error('--role must be readonly or owner');
             const { client, projectId } = makeClient(opts.project);
-            console.log(await getDbUrl(client, projectId, opts.env));
+            console.log(await getDbUrl(client, projectId, opts.env, opts.role));
         }
         catch (e) {
             console.error(friendlyApiError(e));
@@ -178,3 +191,5 @@ export function registerMisc(program) {
         }
     });
 }
+import { randomUUID } from 'node:crypto';
+import { authorizeDatabaseAction } from '../database-authorization.js';

package/dist/commands/pull.js

@@ -1,7 +1,9 @@
-import { readdirSync } from 'node:fs';
+import { existsSync, readdirSync } from 'node:fs';
+import { join } from 'node:path';
 import { ApiClient, friendlyApiError } from '../api.js';
 import { requireCredentials, resolveProject, writeProjectLink } from '../config.js';
-import { extractSnapshot, pruneStaleTopLevel, snapshotTopLevelEntries } from '../archive.js';
+import { extractSnapshot, listSnapshot, pruneStaleTopLevel, snapshotTopLevelEntries } from '../archive.js';
+import { hashBackendSource } from '../deploy-bundle.js';
 export function assertPullTarget(dir, force) {
     const entries = readdirSync(dir).filter((e) => e !== '.newly');
     if (entries.length > 0 && !force) {
@@ -38,10 +40,12 @@ export function registerPull(program) {
             }
             await extractTarball(data, dir);
             const pulledBranch = res.headers.get('x-newly-branch');
+            const pulledPaths = new Set((await listSnapshot(data)).map((name) => name.replace(/^\.?\//, '')));
             writeProjectLink(dir, {
                 projectId,
                 pulledSha: res.headers.get('x-newly-head-sha'),
                 pulledAt: new Date().toISOString(),
+                pulledBackendHash: existsSync(join(dir, 'backend')) ? hashBackendSource(dir, pulledPaths) : undefined,
             });
             console.log(`Pulled${pulledBranch ? ` branch ${pulledBranch}` : ''} into ${dir} and linked to project ${projectId}.`);
         }

package/dist/database-authorization.js

@@ -0,0 +1,23 @@
+import { createInterface } from 'node:readline/promises';
+export async function authorizeDatabaseAction(client, projectId, env, action, resourceId, readCode) {
+    const response = await client.post(`/newly_backend/database/${encodeURIComponent(projectId)}/step-up`, { env, action, resource_id: resourceId });
+    if (typeof response.challenge_id !== 'string')
+        throw new Error('Database authorization did not return a challenge id');
+    console.error(`Database authorization: ${response.challenge_id}`);
+    let code;
+    if (readCode)
+        code = await readCode();
+    else {
+        const input = createInterface({ input: process.stdin, output: process.stderr, terminal: false });
+        try {
+            code = await input.question('Enter the verification code sent to your account email: ');
+        }
+        finally {
+            input.close();
+        }
+    }
+    code = code.trim();
+    if (!/^\d{6}$/.test(code))
+        throw new Error('A six-digit email code is required. Credential export and production restore require fresh authorization.');
+    return { challenge_id: response.challenge_id, code };
+}

package/dist/deploy-bundle.js

@@ -1,9 +1,11 @@
 import { spawn } from 'node:child_process';
+import { createHash } from 'node:crypto';
 import { existsSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, rmSync, writeFileSync, } from 'node:fs';
 import { tmpdir } from 'node:os';
 import { join } from 'node:path';
 import { Buffer } from 'node:buffer';
 import { create as tarCreate } from 'tar';
+import { readProjectLink } from './config.js';
 export const NEWLY_BACKEND_DEPLOY_BUNDLE_CONTENT_TYPE = 'application/vnd.newly.deploy-bundle+gzip';
 export const NEWLY_BACKEND_DEPLOY_MODE_BUNDLE = 'bundle';
 const MANIFEST_PATH = 'newly-deploy.json';
@@ -17,7 +19,7 @@ export class DeployBuildError extends Error {
     }
 }
 export async function buildDeployBundle(options) {
-    const { projectDir, services, run, log } = options;
+    const { projectId, projectDir, services, run, log } = options;
     const bunVersion = await run('bun', ['--version'], projectDir);
     if (bunVersion.code !== 0) {
         throw new DeployBuildError('newly deploy needs bun on your PATH (https://bun.sh)');
@@ -26,11 +28,11 @@ export async function buildDeployBundle(options) {
     const servicesDir = join(bundleDir, 'services');
     mkdirSync(servicesDir, { recursive: true });
     try {
+        const gitSha = await resolveGitSha(run, projectDir, projectId);
         const manifestServices = [];
         for (const service of services) {
             manifestServices.push(await buildService(service, projectDir, servicesDir, run, log));
         }
-        const gitSha = await resolveGitSha(run, projectDir);
         const manifest = { version: 1, git_sha: gitSha, services: manifestServices };
         writeFileSync(join(bundleDir, MANIFEST_PATH), JSON.stringify(manifest, null, 2));
         const bundle = await tarBundle(bundleDir);
@@ -110,12 +112,40 @@ function stageMigrations(dir, serviceDir, name) {
     }
     return `services/${name}/drizzle`;
 }
-async function resolveGitSha(run, projectDir) {
-    const result = await run('git', ['rev-parse', 'HEAD'], projectDir);
-    if (result.code !== 0)
+async function resolveGitSha(run, projectDir, projectId) {
+    const link = readProjectLink(projectDir);
+    if (link?.projectId === projectId && link.pulledSha && link.pulledBackendHash) {
+        try {
+            if (link.pulledBackendHash === hashBackendSource(projectDir))
+                return link.pulledSha;
+        }
+        catch {
+        }
+    }
+    const head = await run('git', ['rev-parse', 'HEAD'], projectDir);
+    const sha = head.code === 0 ? head.output.trim() : '';
+    if (!sha)
+        return null;
+    const tracked = await run('git', ['ls-files', '--error-unmatch', '--', 'backend'], projectDir);
+    if (tracked.code !== 0)
         return null;
-    const sha = result.output.trim();
-    return sha || null;
+    const changes = await run('git', ['--no-optional-locks', 'status', '--porcelain', '--', 'backend'], projectDir);
+    return changes.code === 0 && changes.output.trim() === '' ? sha : null;
+}
+export function hashBackendSource(projectDir, only) {
+    const digest = (data) => createHash('sha256').update(data).digest('hex');
+    const files = backendSourceFiles(projectDir, 'backend').filter((path) => !only || only.has(path));
+    return digest(files.sort().map((path) => `${path}\0${digest(readFileSync(join(projectDir, path)))}`).join('\n'));
+}
+function backendSourceFiles(projectDir, rel) {
+    return readdirSync(join(projectDir, rel), { withFileTypes: true }).flatMap((entry) => {
+        if (entry.name.startsWith('.') || entry.name === 'node_modules')
+            return [];
+        const path = `${rel}/${entry.name}`;
+        if (entry.isDirectory())
+            return backendSourceFiles(projectDir, path);
+        return entry.isFile() ? [path] : [];
+    });
 }
 async function tarBundle(bundleDir) {
     const chunks = [];

package/dist/migration-plan.js

@@ -0,0 +1,74 @@
+import { createHash } from 'node:crypto';
+import { existsSync, readFileSync } from 'node:fs';
+import { join, relative, resolve } from 'node:path';
+import { readBackendServices, assertKnownServices, parseServiceFilter } from './services-manifest.js';
+export function readLocalMigrations(dir, servicesFilter) {
+    const services = readBackendServices(dir);
+    const selected = parseServiceFilter(servicesFilter);
+    assertKnownServices(services, selected);
+    const root = resolve(dir, 'backend');
+    const seenSchemas = new Set();
+    return services.filter((service) => !selected.length || selected.includes(service.name)).flatMap((service) => {
+        const serviceDir = resolve(root, service.path);
+        const path = relative(root, serviceDir);
+        if (path.startsWith('..') || path === '')
+            throw new Error(`Service ${service.name} must have a path inside backend/`);
+        const drizzle = join(serviceDir, 'drizzle');
+        const journal = join(drizzle, 'meta', '_journal.json');
+        if (!existsSync(journal)) {
+            if (service.schema)
+                throw new Error(`Service ${service.name} declares a schema but has no migration journal`);
+            return [];
+        }
+        if (!service.schema || !/^[a-z_][a-z0-9_]{0,62}$/.test(service.schema) || service.schema.startsWith('pg_') || service.schema === 'information_schema')
+            throw new Error(`Service ${service.name} needs a valid migration schema`);
+        if (seenSchemas.has(service.schema))
+            throw new Error(`More than one migration journal targets schema ${service.schema}`);
+        seenSchemas.add(service.schema);
+        const parsed = JSON.parse(readFileSync(journal, 'utf8'));
+        if (!Array.isArray(parsed.entries))
+            throw new Error(`${journal} has no entries array`);
+        let previous = -1;
+        const tags = new Set();
+        const migrations = parsed.entries.map((entry) => {
+            if (typeof entry.tag !== 'string' || !/^[A-Za-z0-9_-]+$/.test(entry.tag) || tags.has(entry.tag) || typeof entry.when !== 'number' || !Number.isSafeInteger(entry.when) || entry.when <= previous)
+                throw new Error(`Invalid or unordered migration journal for ${service.name}`);
+            tags.add(entry.tag);
+            previous = entry.when;
+            const sql = readFileSync(join(drizzle, `${entry.tag}.sql`), 'utf8');
+            return { tag: entry.tag, when: entry.when, hash: createHash('sha256').update(sql).digest('hex') };
+        });
+        return [{ service: service.name, schema: service.schema, migrations }];
+    });
+}
+export function compareMigrations(local, ledger) {
+    const expected = new Map(local.migrations.map((migration) => [migration.when, migration]));
+    const applied = new Set(ledger.rows.map((row) => Number(row.created_at)));
+    const head = ledger.rows.length ? Math.max(...applied) : -1;
+    const problems = [];
+    const seen = new Set();
+    const rows = ledger.rows.map((row) => {
+        const when = Number(row.created_at);
+        if (!Number.isSafeInteger(when) || when < 0)
+            problems.push(`Invalid ledger timestamp: ${row.created_at}`);
+        if (seen.has(when))
+            problems.push(`Duplicate ledger timestamp: ${row.created_at}`);
+        seen.add(when);
+        const migration = expected.get(when);
+        if (!migration)
+            problems.push(`Applied migration is absent from this checkout: ${row.created_at}`);
+        else if (migration.hash !== row.hash)
+            problems.push(`Applied SQL hash differs: ${migration.tag}`);
+        return { ...row, tag: migration?.tag ?? null, hash_matches: migration ? migration.hash === row.hash : null };
+    });
+    const pending = local.migrations.filter((migration) => !applied.has(migration.when));
+    for (const migration of pending) {
+        if (migration.when <= head)
+            problems.push(`Migration was skipped before the applied head: ${migration.tag}`);
+    }
+    return {
+        service: local.service, schema: local.schema, ledger_exists: ledger.exists,
+        applied: rows, pending: pending.map((migration) => migration.tag), problems,
+        would_execute: problems.length ? null : pending.map((migration) => migration.tag),
+    };
+}

package/dist/version.js

@@ -1 +1 @@
-export const CLI_VERSION = '0.2.163';
+export const CLI_VERSION = '0.2.165';

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@newly/cli",
-  "version": "0.2.163",
+  "version": "0.2.165",
   "description": "Develop, provision, and deploy Newly backends from your terminal",
   "homepage": "https://newly.app",
   "repository": {