mnemonad-cli 0.3.0 → 0.3.2

package/README.md

@@ -118,6 +118,7 @@ Every command prints the installed CLI version as its first line of output.
 | `--pinata-jwt <jwt>` | Pinata JWT for IPFS offload of large files (or `PINATA_JWT` env var) |
 | `--manifest` | Write/use `.mnemonad` manifest for faster change detection. With no stream-id given, every command recovers it from an existing `.mnemonad` in the folder instead of requiring it explicitly |
 | `--force-snapshot` | Push a full snapshot regardless of prior history (repairs a corrupt stream) |
+| `--detach` | `push`: build the next version here, send it from a background process, and return right away. With `--index`, indexing and building move to the background too. Existing streams only. See "Push in the background" below |
 | `--index` | `push`/`watch`: update the folder's search index before every push (takes `--db`/`--model`/`--chunk-size`/`--chunk-overlap`); if indexing fails, nothing is pushed |
 | `--poll-interval <s>` | `watch`: seconds between remote version checks (default: `2`) |
 | `--debounce <ms>` | `watch`: quiet period in ms before pushing after a local change (default: `1000`) |
@@ -230,7 +231,7 @@ mnemonad pull 123 ~/restored --version 1
 ### Inspect a stream
 
 ```bash
-# check your wallet + chain (no stream id needed)
+# check your wallet, its MON balance, and the chain (no stream id needed)
 mnemonad info
 
 # full stream metadata + local sync status
@@ -240,6 +241,7 @@ mnemonad info 123 ~/my-data
 ```
   chain:           testnet
   your wallet:     0x06bc6420b37a4898424429dcfa236f0065e12279
+  balance:         4.2183 MON
 
 stream: 0x68d9736257f0d2da3c848666a4b36c4b9f18eda7e4dfba655599c47560d8a4b5
   owner:           0x06bc6420b37a4898424429dcfa236f0065e12279 (you)
@@ -349,6 +351,69 @@ mnemonad burn 123
 mnemonad burn 123 --yes
 ```
 
+### Push in the background
+
+A push spends most of its time waiting for the chain. `--detach` does everything that can
+fail for your own reasons first, here and now: the owner check, unlocking, `--index`, "no
+changes", and the balance against the estimated cost. Then it hands only the sending to a
+background process and returns:
+
+```bash
+mnemonad push ~/my-data --manifest --detach
+#   version 8 prepared (diff, gzip compressed, 2.1 KB)
+#   estimated cost: 0.0041 MON
+#   stream: 0x68d9…
+#   sending in the background: job 8f3c1a2b (pid 41210)
+#   log:   ~/.cache/mnemonad/jobs/8f3c1a2b/push.log
+#   check: mnemonad info ~/my-data --manifest
+```
+
+- **The version is frozen when the command runs.** Edits made after it returns go into the
+  next push, not this one.
+- **Existing streams only.** It needs a stream id, or `--manifest` on a folder that has a
+  `.mnemonad`. Create a stream with a normal push first. `--passkey` can't be used: the browser
+  tab that signs for it closes when the command exits.
+- **The next command waits for it.** A `push`, `pull`, `compact` or `watch` of the same stream
+  waits until the background push has finished, so nothing builds on, or restores, the version
+  before it. A push from the same wallet to another stream waits too, so the two don't race for
+  one nonce. `diff` only notes that a push is still sending.
+- **Results:** `mnemonad info <path>` shows the last background push for the stream, and when
+  it failed, why. If one fails, the next command on that stream also prints a warning, once.
+- **Nothing half-lands.** The background process refuses to send a version whose base is gone
+  (the stream changed after it was prepared), and the contract rejects any write that doesn't
+  land at the index it names. A failed background push leaves the stream as it was. Push again.
+- **Secrets stay in memory.** The key, the password and a Pinata JWT are handed to the
+  background process over a pipe. They are never written to the job folder, the log, or the
+  process list.
+- If the background process is killed mid-send, the job is reported as failed the next time a
+  command looks at it. A large push that was staged over several transactions, or paid for an
+  IPFS upload, can't be resumed. A paid but unused upload can be refunded after the gateway's
+  time limit, with the SDK's `MnemonadUploadGateway.refund()` (the CLI has no command for it).
+
+**With `--index`, indexing goes to the background too.** The updated index is part of the
+version being pushed, so the version can only be built once indexing is done, and sent once
+it is built. `push --detach --index` therefore hands off all three, and returns after only the
+checks that need no folder scan: the signer owns the stream, the credential unlocks it, and the
+wallet isn't empty.
+
+```bash
+mnemonad push ~/my-data --manifest --index --detach
+#   stream: 0x68d9…
+#   indexing, building and sending in the background: job 3b9e07c1 (pid 41388)
+```
+
+The trade-offs, compared with a plain `--detach`:
+
+- No version number, size or cost estimate is printed. `mnemonad info` shows them once the job
+  has built the version, and shows "no changes, nothing pushed" if the folder already matched.
+- An indexing error, a model download that fails, and "no changes" show up in the job's status,
+  not in the terminal.
+- The folder is not frozen when the command runs. The job reads it when it starts building, so
+  edits made before then are part of this version.
+
+Jobs live in `~/.cache/mnemonad/jobs/` (set `MNEMONAD_JOBS_DIR` to move them). Finished ones are
+removed after 14 days.
+
 ### Repair a corrupt stream with a force snapshot
 
 If a diff patch was pushed against a stale base (e.g. a race condition in `watch`),

package/bin/mnemonad.js

@@ -20,6 +20,7 @@ import { dirname, join } from 'node:path';
 import { push, pull, info, diff, watch, compact, burn, buildIndex, search } from '../lib/commands/index.js';
 import { parseQualifiedStreamId } from '../lib/commands/shared.js';
 import { closePasskeyBridge } from '../lib/passkeyBridge.js';
+import { runPushJob } from '../lib/jobs.js';
 import defaultConfig from '../mnemonad.config.js';
 
 const PKG_ROOT = join(dirname(fileURLToPath(import.meta.url)), '..');
@@ -104,6 +105,11 @@ Options:
                             With no stream-id given, every command recovers it from an
                             existing .mnemonad in <path> instead of requiring it explicitly
   --force-snapshot          Push a full snapshot regardless of prior history (repairs corrupt streams)
+  --detach                  push: build the next version here, send it from a background process,
+                            and return right away. Existing streams only (an id, or --manifest).
+                            Later pushes and pulls of that stream wait for it; \`mnemonad info\`
+                            shows how it went. With --index, indexing and building the version
+                            move to the background too
   --index                   push/watch: update the folder's search index (search_index.db)
                             before every push, so the stream is searchable right after a pull.
                             Takes --db/--model/--chunk-size/--chunk-overlap like \`index\`
@@ -162,6 +168,7 @@ function parseArgs(argv) {
         index: false,
         rebuild: false,
         forceSnapshot: false,
+        detach: false,
         pollInterval: 2,
         debounce: 1000,
         pushOnly: false,
@@ -281,6 +288,8 @@ function parseArgs(argv) {
             args.rebuild = true;
         } else if (flag === '--force-snapshot') {
             args.forceSnapshot = true;
+        } else if (flag === '--detach') {
+            args.detach = true;
         } else if (flag === '--poll-interval' && i + 1 < raw.length) {
             args.pollInterval = Number(raw[++i]);
         } else if (flag === '--debounce' && i + 1 < raw.length) {
@@ -363,6 +372,9 @@ try {
         await buildIndex(args);
     } else if (args.command === 'search') {
         await search(args);
+    } else if (args.command === '__push-job') {
+        // The detached half of `push --detach` (lib/jobs.js) — never typed by a person.
+        await runPushJob(args.path);
     } else {
         console.error('Unknown command:', args.command);
         console.log(USAGE);

package/lib/commands/compact.js

@@ -12,6 +12,7 @@ import {
     resolveOffloadParams,
     discoverStreamId,
 } from './shared.js';
+import { settleStreamJobs } from '../jobs.js';
 
 /**
  * Monad has no rebate/refund concept — truncating a stream's history here gives back no
@@ -28,6 +29,7 @@ export async function compact(args) {
     if (!args.streamId) {
         throw new Error('stream-id is required for compact');
     }
+    await settleStreamJobs(args, { reason: 'before replacing the history with a fresh snapshot' });
 
     const fsFolder = new FSFolder(destPath, makeExcludes(args));
 

package/lib/commands/diff.js

@@ -12,6 +12,7 @@ import {
     folderExists,
     discoverStreamId,
 } from './shared.js';
+import { settleStreamJobs } from '../jobs.js';
 
 export function compareFileMaps(localFiles, remoteFiles) {
     const added = [];
@@ -55,6 +56,7 @@ export async function diff(args) {
     if (!args.streamId) {
         throw new Error('stream-id is required for diff');
     }
+    await settleStreamJobs(args, { wait: false });
 
     const client = await makeChainClient(args);
 

package/lib/commands/info.js

@@ -1,5 +1,6 @@
 import Mnemonad, { kekMethodOf } from 'mnemonad';
 import { MonadSync } from 'monadsync';
+import { formatEther } from 'viem';
 import { makeChainClient } from '../chainClient.js';
 import {
     resolvePath,
@@ -12,12 +13,16 @@ import {
     readManifest,
     streamIdsEqual,
 } from './shared.js';
+import { streamKey, latestJob, describeJob } from '../jobs.js';
 
 export async function info(args) {
     const client = await makeChainClient(args);
 
     console.log('  chain:          ', args.chain);
     console.log('  your wallet:    ', client.account?.address || '(none)');
+    if (client.account) {
+        console.log('  balance:        ', await walletBalance(client));
+    }
 
     // Peek at .mnemonad whenever a path is given, regardless of --manifest and regardless
     // of whether an id was given — info is read-only, so surfacing what's there costs
@@ -45,6 +50,14 @@ export async function info(args) {
         }
     }
 
+    // The last background push (`push --detach`) for this stream, whichever way its id is
+    // known — even with no --manifest, the folder's own manifest names the stream.
+    const jobStreamId = args.streamId || manifest?.streamId;
+    if (jobStreamId) {
+        const job = latestJob(streamKey({ chain: args.chain, contractAddress: args.contractAddress, streamId: jobStreamId }));
+        if (job) console.log('  background push:', describeJob(job));
+    }
+
     if (!args.streamId) {
         return;
     }
@@ -161,3 +174,18 @@ export async function info(args) {
         console.log('  latest remote:   version', totalVersions);
     }
 }
+
+/**
+ * The signing wallet's native balance, in the chain's own unit (MON on Monad, ETH on a local
+ * Hardhat node). A failed RPC read is shown as such rather than failing `info` — the balance
+ * is a convenience next to the stream report, not something the rest depends on.
+ */
+async function walletBalance(client) {
+    const symbol = client.chain.nativeCurrency?.symbol || 'MON';
+    try {
+        const wei = await client.publicClient.getBalance({ address: client.account.address });
+        return `${formatEther(wei)} ${symbol}`;
+    } catch {
+        return '(could not read)';
+    }
+}

package/lib/commands/pull.js

@@ -11,6 +11,7 @@ import {
     formatStreamId,
     discoverStreamId,
 } from './shared.js';
+import { settleStreamJobs } from '../jobs.js';
 
 export async function pull(args) {
     // Resolved up front (throws if missing) so a missing path fails fast, before any chain
@@ -24,6 +25,10 @@ export async function pull(args) {
         throw new Error('stream-id is required for pull');
     }
 
+    // Pulling while a background push of this folder is still sending would restore the
+    // version before it — and delete the very files that push is uploading.
+    await settleStreamJobs(args, { reason: 'so the pull includes it' });
+
     const client = await makeChainClient(args);
 
     const mn = new Mnemonad({

package/lib/commands/push.js

@@ -19,8 +19,12 @@ import {
     streamIdsEqual,
     discoverStreamId,
     userError,
+    formatBytes,
+    formatPath,
 } from './shared.js';
 import { updateIndex, checkIndex, indexDbName } from './buildIndex.js';
+import { streamKey, waitForJobs, settleStreamJobs, startPushJob } from '../jobs.js';
+import { formatEther } from 'viem';
 
 export async function push(args) {
     const destPath = resolvePath(args.path, { mustExist: true });
@@ -31,12 +35,35 @@ export async function push(args) {
     await discoverStreamId(args, destPath);
     let streamId = args.streamId;
 
+    if (args.detach) {
+        // Creating a stream prints the id the user has to keep, and needs signatures that have
+        // to be seen to succeed — nothing worth handing off. --passkey's signer is a browser
+        // tab that closes when this process exits.
+        if (!streamId) {
+            throw userError(
+                '--detach only pushes to an existing stream. Pass its id, or --manifest on a folder\n' +
+                '  that has a .mnemonad, or push once without --detach to create the stream first.'
+            );
+        }
+        if (args.passkey) {
+            throw userError('--detach can\'t be combined with --passkey: the browser tab signing for it closes when this command exits.');
+        }
+    }
+
+    // A background push still sending to this stream means its next version isn't on chain
+    // yet — building on top of the current tip now would only lose the race to it.
+    await settleStreamJobs(args, { reason: 'before building the next version on top of it' });
+
     // Before anything is scanned or signed: the push below then picks the updated index up
     // like any other file. A failure stops the push outright — a push that silently went out
     // without the index asked for would leave the stream unsearchable with no sign of why.
     if (args.rebuild && !args.index) {
         throw userError('--rebuild only applies with --index (it starts the search index over)');
     }
+    if (args.detach && args.index) {
+        await detachPrepare(args, destPath, streamId);
+        return;
+    }
     if (args.index) {
         console.log(args.rebuild ? '  rebuilding search index...' : '  updating search index...');
         try {
@@ -67,6 +94,12 @@ export async function push(args) {
     const client = await makeChainClient(args);
     requireWalletClient(client);
 
+    // Same wallet, different stream: two processes sending at once would race for one nonce.
+    await waitForJobs(
+        { signer: client.account.address, chain: args.chain },
+        { reason: 'so the two pushes don\'t race for the same wallet nonce' },
+    );
+
     const baseParams = {
         publicClient: client.publicClient,
         walletClient: client.walletClient,
@@ -139,6 +172,11 @@ export async function push(args) {
         }
     }
 
+    if (args.detach) {
+        await detachPush({ args, mn, monadSync, fsFolder, client, streamId, destPath, existingVersions });
+        return;
+    }
+
     console.log('  building patch + pushing to chain...');
     const result = await monadSync.push(fsFolder);
 
@@ -156,6 +194,159 @@ export async function push(args) {
     }
 }
 
+/**
+ * `push --detach --index`: indexing can't run here and still be "detached" — the updated index
+ * is part of the version being pushed, so building the patch has to wait for it, and the
+ * sending waits for the patch. So this hands off all three, and keeps only the checks that
+ * need no folder scan and no chain replay: the signer owns the stream, the credential unlocks
+ * it, and the wallet isn't empty. Everything else — "no changes", an indexing error, the
+ * version number — shows up in the job (`mnemonad info`).
+ *
+ * Unlike a plain `--detach`, the folder isn't frozen here: the job reads it when its own turn
+ * comes, so edits made before then go into this version.
+ */
+async function detachPrepare(args, destPath, streamId) {
+    const client = await makeChainClient(args);
+    requireWalletClient(client);
+    await waitForJobs(
+        { signer: client.account.address, chain: args.chain },
+        { reason: 'so the two pushes don\'t race for the same wallet nonce' },
+    );
+
+    // Flag combinations are checked here too, not left for the job to trip over.
+    resolveEncryptMethod(args);
+
+    const mn = new Mnemonad({
+        publicClient: client.publicClient,
+        walletClient: client.walletClient,
+        contractAddress: client.contractAddress,
+        id: streamId,
+        ...resolveOffloadParams(args, client.account),
+    });
+    await assertStreamOwner(mn, client.account.address);
+    await unlockStream(mn, { password: args.password, signer: client.walletClient });
+
+    const balance = await client.publicClient.getBalance({ address: client.account.address });
+    if (balance === 0n) {
+        throw userError(`${client.account.address} has no MON to pay for the push. Nothing was started.`);
+    }
+
+    const job = await startPushJob(
+        {
+            mode: 'prepare',
+            streamId: formatStreamId(streamId),
+            streamKey: streamKey({ chain: args.chain, contractAddress: args.contractAddress, streamId }),
+            chain: args.chain,
+            signer: client.account.address,
+            folder: destPath,
+            baseLength: null,
+            version: null,
+            manifest: args.manifest ? {} : null,
+            args: {
+                chain: args.chain,
+                rpcUrl: args.rpcUrl,
+                contractAddress: args.contractAddress,
+                gatewayUrl: args.gatewayUrl,
+                presignUrl: args.presignUrl,
+                exclude: args.exclude,
+                compress: args.compress,
+                forceSnapshot: args.forceSnapshot,
+                searchDb: args.searchDb,
+                model: args.model,
+                chunkSize: args.chunkSize,
+                chunkOverlap: args.chunkOverlap,
+                rebuild: args.rebuild,
+            },
+        },
+        null,
+        {
+            key: args.key || process.env.MNEMONAD_KEY || null,
+            phrase: args.phrase,
+            password: args.password,
+            pinataJwt: args.pinataJwt,
+        },
+    );
+
+    console.log('  stream:', formatStreamId(streamId));
+    console.log('  indexing, building and sending in the background: job', job.id, '(pid ' + job.pid + ')');
+    console.log('  log:  ', formatPath(job.dir + '/push.log'));
+    console.log('  check: mnemonad info', formatPath(destPath) + (args.manifest ? ' --manifest' : ''));
+}
+
+/**
+ * `push --detach`: everything up to a finished patch runs here, so every error a user can act
+ * on still surfaces before this command exits. Only the sending — transactions, receipts, an
+ * IPFS upload — goes to a detached child (see lib/jobs.js), and this process exits as soon
+ * as it has started.
+ *
+ * The patch is frozen now: files edited after this command returns go into the next push,
+ * not this one. The manifest's file hashes are taken at the same moment for the same reason.
+ */
+async function detachPush({ args, mn, monadSync, fsFolder, client, streamId, destPath, existingVersions }) {
+    console.log('  building patch...');
+    const prepared = await monadSync.preparePush(fsFolder);
+
+    // A single-item patch is the common case and the only one estimatePushCost models; a
+    // segmented one (over 8 MB) just goes without an estimate rather than a wrong one.
+    let cost = null;
+    if (prepared.itemCount === 1) {
+        try {
+            cost = await mn.estimatePushCost(prepared.patchBytes);
+        } catch {
+            // No estimate is not a reason to stop — the child surfaces any real send error.
+        }
+    }
+    if (cost) {
+        const balance = await client.publicClient.getBalance({ address: client.account.address });
+        if (balance < cost.total) {
+            throw userError(
+                `not enough MON to push: about ${formatEther(cost.total)} needed, ` +
+                `${formatEther(balance)} in ${client.account.address}. Nothing was sent.`
+            );
+        }
+    }
+
+    const files = args.manifest ? await hashLocalTree(fsFolder) : null;
+    const job = await startPushJob(
+        {
+            streamId: formatStreamId(streamId),
+            streamKey: streamKey({ chain: args.chain, contractAddress: args.contractAddress, streamId }),
+            chain: args.chain,
+            signer: client.account.address,
+            folder: destPath,
+            baseLength: prepared.baseLength,
+            version: prepared.version,
+            kind: prepared.kind,
+            bytes: prepared.patchBytes.length,
+            manifest: files ? { files } : null,
+            // Everything the child needs to rebuild the same chain client — never a secret.
+            args: {
+                chain: args.chain,
+                rpcUrl: args.rpcUrl,
+                contractAddress: args.contractAddress,
+                gatewayUrl: args.gatewayUrl,
+                presignUrl: args.presignUrl,
+            },
+        },
+        prepared.patchBytes,
+        {
+            key: args.key || process.env.MNEMONAD_KEY || null,
+            phrase: args.phrase,
+            password: args.password,
+            pinataJwt: args.pinataJwt,
+        },
+    );
+
+    const patchType = existingVersions === 0 || prepared.kind === 'patch' ? 'full snapshot' : 'diff';
+    const compressedNote = args.compress !== false ? ', gzip compressed' : '';
+    console.log('  version', prepared.version, 'prepared (' + patchType + compressedNote + ', ' + formatBytes(prepared.patchBytes.length) + ')');
+    if (cost) console.log('  estimated cost:', formatEther(cost.total), 'MON');
+    console.log('  stream:', formatStreamId(streamId));
+    console.log('  sending in the background: job', job.id, '(pid ' + job.pid + ')');
+    console.log('  log:  ', formatPath(job.dir + '/push.log'));
+    console.log('  check: mnemonad info', formatPath(destPath) + (args.manifest ? ' --manifest' : ''));
+}
+
 /**
  * A folder that already carries an index, pushed without `--index` after its files changed,
  * would publish an index that no longer matches them — searches keep returning the old text's

package/lib/commands/watch.js

@@ -18,6 +18,7 @@ import {
     discoverStreamId,
     userError,
 } from './shared.js';
+import { settleStreamJobs } from '../jobs.js';
 import { makeIndexer, updateIndex } from './buildIndex.js';
 
 function timestamp() {
@@ -37,6 +38,7 @@ export async function watch(args) {
     // no need to require it again.
     await discoverStreamId(args, cwd);
     if (!args.streamId) throw new Error('stream-id is required for watch');
+    await settleStreamJobs(args, { reason: 'before watching' });
     const debounceMs = args.debounce ?? 5000;
     const pollIntervalMs = (args.pollInterval ?? 2) * 1000;
     const pushEnabled = !args.pullOnly;

package/lib/jobs.js

@@ -0,0 +1,459 @@
+/**
+ * Background push jobs — what `mnemonad push --detach` leaves behind.
+ *
+ * A detached push builds its patch in the foreground (every check that can fail for the
+ * user's own reasons runs there: ownership, unlocking, indexing, "nothing to push", the
+ * balance), then hands only the sending to a detached child process and exits. Each job is a
+ * folder under `~/.cache/mnemonad/jobs/<id>/`:
+ *
+ *   job.json   — what to send where, and the job's status (see JOB_STATUS)
+ *   patch.bin  — the prepared patch, frozen when the command ran; deleted once sent
+ *   push.log   — the child's own output
+ *
+ * Secrets (a key, a phrase, a password, a Pinata JWT) never touch this folder or the child's
+ * argv: the parent writes them to the child's stdin and closes it.
+ *
+ * Correctness never depends on the locking below. The child refuses to send a patch whose
+ * base version is gone (`MonadSync.commitPush`), and the contract rejects any write that
+ * doesn't land at the index it names. The waiting here only keeps the next command from
+ * racing a job it would lose to anyway — or, for a pull, from overwriting the folder with the
+ * version before the one still being sent.
+ */
+import { spawn } from 'node:child_process';
+import { openSync, closeSync, readdirSync, readFileSync, writeFileSync, renameSync, rmSync } from 'node:fs';
+import { mkdir, readFile, writeFile, rm } from 'node:fs/promises';
+import { randomBytes } from 'node:crypto';
+import { homedir } from 'node:os';
+import { dirname, join } from 'node:path';
+import { fileURLToPath } from 'node:url';
+import Mnemonad from 'mnemonad';
+import { MonadSync } from 'monadsync';
+import { makeChainClient, requireWalletClient } from './chainClient.js';
+import { FSFolder } from './FSFolder.js';
+import {
+    formatStreamId,
+    formatPath,
+    unlockStream,
+    assertStreamOwner,
+    resolveOffloadParams,
+    writeManifest,
+    makeExcludes,
+    hashLocalTree,
+    localTreeHash,
+    userError,
+} from './commands/shared.js';
+import { updateIndex } from './commands/buildIndex.js';
+
+const BIN_PATH = join(dirname(fileURLToPath(import.meta.url)), '..', 'bin', 'mnemonad.js');
+
+/** pending: written, child not started yet · waiting: queued behind an earlier job from the
+ *  same signer · preparing: indexing and building the patch (`push --detach --index` only) ·
+ *  sending: transactions in flight · done / failed: finished. */
+export const JOB_STATUS = {
+    PENDING: 'pending', WAITING: 'waiting', PREPARING: 'preparing', SENDING: 'sending', DONE: 'done', FAILED: 'failed',
+};
+
+const ACTIVE = new Set([JOB_STATUS.PENDING, JOB_STATUS.WAITING, JOB_STATUS.PREPARING, JOB_STATUS.SENDING]);
+
+/** "version 8", or what a job that hasn't built its version yet is doing instead. */
+function versionLabel(job) {
+    return job.version != null ? `version ${job.version}` : 'the next version';
+}
+
+/** A job still `pending` with no pid after this long never got its child started. */
+const SPAWN_GRACE_MS = 60_000;
+
+/** Finished jobs older than this are removed the next time a job is created. */
+const KEEP_FINISHED_MS = 14 * 24 * 60 * 60 * 1000;
+
+const POLL_MS = 1000;
+
+/** `MNEMONAD_JOBS_DIR` overrides the location (the tests use a temp dir). */
+export function jobsRoot() {
+    return process.env.MNEMONAD_JOBS_DIR || join(homedir(), '.cache', 'mnemonad', 'jobs');
+}
+
+/**
+ * Identifies a stream across commands: the same token id is a different stream on another
+ * chain or registry.
+ *
+ * @param {{chain: string, contractAddress: ?string, streamId: bigint|string}} where
+ */
+export function streamKey({ chain, contractAddress, streamId }) {
+    return `${chain}|${String(contractAddress || '').toLowerCase()}|${formatStreamId(streamId)}`;
+}
+
+function signerKey({ chain, signer }) {
+    return `${chain}|${String(signer || '').toLowerCase()}`;
+}
+
+function isPidAlive(pid) {
+    if (!pid) return false;
+    try {
+        process.kill(pid, 0);
+        return true;
+    } catch (err) {
+        // EPERM: the process exists, it's just not ours to signal.
+        return err.code === 'EPERM';
+    }
+}
+
+function writeJobSync(dir, job) {
+    // Synchronous on purpose: callers read-modify-write job.json from two processes, and a
+    // torn async write would leave a half-written file for the other side to parse.
+    const tmp = join(dir, `job.json.${process.pid}.tmp`);
+    writeFileSync(tmp, JSON.stringify(job, null, 2));
+    renameSync(tmp, join(dir, 'job.json'));
+}
+
+function readJobSync(dir) {
+    try {
+        return JSON.parse(readFileSync(join(dir, 'job.json'), 'utf8'));
+    } catch {
+        return null;
+    }
+}
+
+/**
+ * Re-reads a job and settles it as failed when it claims to be running but nothing is: the
+ * child was killed, the machine slept through a crash, or the spawn never happened.
+ */
+function refreshJob(dir) {
+    const job = readJobSync(dir);
+    if (!job || !ACTIVE.has(job.status)) return job;
+
+    const neverStarted = !job.pid && Date.now() - Date.parse(job.createdAt) > SPAWN_GRACE_MS;
+    if (neverStarted || (job.pid && !isPidAlive(job.pid))) {
+        const settled = {
+            ...job,
+            status: JOB_STATUS.FAILED,
+            error: neverStarted
+                ? 'the background process never started'
+                : 'the background process exited before finishing',
+            finishedAt: new Date().toISOString(),
+            reported: false,
+        };
+        writeJobSync(dir, settled);
+        return settled;
+    }
+    return job;
+}
+
+/** Every job on disk, oldest first, with dead ones settled as failed. */
+export function listJobs() {
+    let names;
+    try {
+        names = readdirSync(jobsRoot());
+    } catch {
+        return [];
+    }
+    const jobs = [];
+    for (const name of names) {
+        const dir = join(jobsRoot(), name);
+        const job = refreshJob(dir);
+        if (job) jobs.push({ ...job, dir });
+    }
+    return jobs.sort((a, b) => a.createdAt.localeCompare(b.createdAt) || a.id.localeCompare(b.id));
+}
+
+export function isActive(job) {
+    return ACTIVE.has(job.status);
+}
+
+/** Jobs still running for a stream (`{streamKey}`) or a signer (`{signer, chain}`). */
+export function activeJobs(selector) {
+    return listJobs().filter((job) => isActive(job) && matches(job, selector));
+}
+
+function matches(job, selector) {
+    if (selector.streamKey) return job.streamKey === selector.streamKey;
+    return signerKey(job) === signerKey(selector);
+}
+
+/** The most recent job for a stream, active or not. */
+export function latestJob(key) {
+    const jobs = listJobs().filter((job) => job.streamKey === key);
+    return jobs.length ? jobs[jobs.length - 1] : null;
+}
+
+/**
+ * Blocks until no job matching `selector` is still running, printing one line while it
+ * waits. Returns how many it waited for.
+ *
+ * @param {{streamKey: string}|{signer: string, chain: string}} selector
+ * @param {Object} [opts]
+ * @param {string} [opts.reason] - why this command has to wait, for the printed line
+ */
+export async function waitForJobs(selector, { reason = 'before continuing' } = {}) {
+    let first = activeJobs(selector);
+    if (!first.length) return 0;
+
+    const job = first[first.length - 1];
+    console.log(
+        `  waiting for background push ${job.id} (${versionLabel(job)}) to finish ${reason}...\n` +
+        `  log: ${formatPath(join(job.dir, 'push.log'))}`
+    );
+    while (first.length) {
+        await new Promise((r) => setTimeout(r, POLL_MS));
+        first = activeJobs(selector);
+    }
+    return 1;
+}
+
+/**
+ * Prints a failed job's error once — the next command on that stream after the failure —
+ * then marks it reported, so a failure is never silent past one run but doesn't nag forever.
+ * `mnemonad info` keeps showing it regardless (see describeJob).
+ */
+export function reportFailedJob(key) {
+    const job = latestJob(key);
+    if (!job || job.status !== JOB_STATUS.FAILED || job.reported) return;
+    console.log(
+        `  warning: background push ${job.id} (${versionLabel(job)}) failed: ${job.error}\n` +
+        `           nothing from it reached the stream. Log: ${formatPath(join(job.dir, 'push.log'))}`
+    );
+    writeJobSync(job.dir, { ...readJobSync(job.dir), reported: true });
+}
+
+/**
+ * What every command that touches a stream calls once its id is known: reports a failed
+ * background push (once), then either waits for a running one (`wait: true` — push, pull,
+ * compact, watch: anything that would build on, or overwrite with, the version before it) or
+ * just says one is running (diff, which only reads).
+ *
+ * @param {Object} args - parsed CLI args; needs `chain`, `contractAddress`, `streamId`
+ * @param {Object} [opts]
+ * @param {boolean} [opts.wait=true]
+ * @param {string} [opts.reason] - see waitForJobs
+ */
+export async function settleStreamJobs(args, { wait = true, reason } = {}) {
+    if (!args.streamId) return;
+    const key = streamKey({ chain: args.chain, contractAddress: args.contractAddress, streamId: args.streamId });
+    reportFailedJob(key);
+    if (wait) {
+        await waitForJobs({ streamKey: key }, { reason });
+        return;
+    }
+    const running = activeJobs({ streamKey: key });
+    if (running.length) {
+        const job = running[running.length - 1];
+        console.log(
+            `  note: background push ${job.id} (${versionLabel(job)}) is still running —\n` +
+            '        this compares against the chain as it is before that version lands'
+        );
+    }
+}
+
+/** One line for `mnemonad info`. */
+export function describeJob(job) {
+    const log = formatPath(join(job.dir, 'push.log'));
+    switch (job.status) {
+        case JOB_STATUS.DONE:
+            return job.noChanges
+                ? `no changes, nothing pushed ${job.finishedAt} (job ${job.id})`
+                : `version ${job.version} sent ${job.finishedAt} (job ${job.id})`;
+        case JOB_STATUS.FAILED:
+            return `${versionLabel(job)} FAILED: ${job.error} (job ${job.id}, log: ${log})`;
+        case JOB_STATUS.WAITING:
+            return `${versionLabel(job)} queued behind another push from the same wallet (job ${job.id}, log: ${log})`;
+        case JOB_STATUS.PREPARING:
+            return `indexing and building the next version (job ${job.id}, pid ${job.pid ?? '?'}, log: ${log})`;
+        default:
+            return `${versionLabel(job)} still sending (job ${job.id}, pid ${job.pid ?? '?'}, log: ${log})`;
+    }
+}
+
+function pruneFinishedJobs() {
+    for (const job of listJobs()) {
+        if (isActive(job) || !job.finishedAt) continue;
+        if (Date.now() - Date.parse(job.finishedAt) > KEEP_FINISHED_MS) {
+            rmSync(job.dir, { recursive: true, force: true });
+        }
+    }
+}
+
+/**
+ * Writes a job and starts its detached child.
+ *
+ * @param {Object} job - everything the child needs except secrets; see push.js's detachPush.
+ *   `mode: 'prepare'` (push.js's detachPrepare) has the child index and build the patch too.
+ * @param {?Uint8Array} patchBytes - the prepared patch; null in `prepare` mode
+ * @param {{key?: ?string, phrase?: ?string, password?: ?string, pinataJwt?: ?string}} secrets
+ * @returns {Promise<{id: string, dir: string, pid: number}>}
+ */
+export async function startPushJob(job, patchBytes, secrets) {
+    pruneFinishedJobs();
+
+    const id = randomBytes(4).toString('hex');
+    const dir = join(jobsRoot(), id);
+    await mkdir(dir, { recursive: true });
+    if (patchBytes) await writeFile(join(dir, 'patch.bin'), patchBytes);
+    writeJobSync(dir, { ...job, id, status: JOB_STATUS.PENDING, pid: null, createdAt: new Date().toISOString() });
+
+    const logFd = openSync(join(dir, 'push.log'), 'a');
+    let child;
+    try {
+        child = spawn(process.execPath, [BIN_PATH, '__push-job', dir], {
+            detached: true,
+            stdio: ['pipe', logFd, logFd],
+            windowsHide: true,
+        });
+    } finally {
+        closeSync(logFd);
+    }
+
+    writeJobSync(dir, { ...readJobSync(dir), pid: child.pid });
+    child.stdin.end(JSON.stringify(secrets));
+    child.unref();
+    return { id, dir, pid: child.pid };
+}
+
+async function readStdin() {
+    const parts = [];
+    for await (const part of process.stdin) parts.push(part);
+    const raw = Buffer.concat(parts).toString('utf8');
+    return raw ? JSON.parse(raw) : {};
+}
+
+function log(...parts) {
+    console.log(new Date().toISOString(), ...parts);
+}
+
+/**
+ * The detached child: `mnemonad __push-job <dir>`. Never typed by a person — not in --help.
+ *
+ * Waits its turn behind earlier jobs from the same signer (two processes sending with one key
+ * would race for the same nonce), then sends the prepared patch from a fresh, never-replayed
+ * Mnemonad instance, and on success writes the folder's .mnemonad exactly like a foreground
+ * `push --manifest` would have.
+ *
+ * @param {string} dir - the job folder
+ */
+export async function runPushJob(dir) {
+    if (!dir) throw userError('__push-job needs a job folder');
+    const secrets = await readStdin();
+
+    let job = readJobSync(dir);
+    if (!job) throw userError(`no job at ${dir}`);
+    const update = (fields) => {
+        job = { ...readJobSync(dir), ...fields };
+        writeJobSync(dir, job);
+    };
+    update({ pid: process.pid });
+
+    try {
+        const earlier = () => activeJobs({ signer: job.signer, chain: job.chain })
+            .filter((other) => other.createdAt < job.createdAt || (other.createdAt === job.createdAt && other.id < job.id));
+        if (earlier().length) {
+            update({ status: JOB_STATUS.WAITING });
+            log(`waiting for earlier background push(es) from ${job.signer}...`);
+            while (earlier().length) await new Promise((r) => setTimeout(r, POLL_MS));
+        }
+
+        const args = { ...job.args, key: secrets.key ?? null, phrase: secrets.phrase ?? null, password: secrets.password ?? null, pinataJwt: secrets.pinataJwt ?? null, passkey: false };
+        const client = await makeChainClient(args);
+        requireWalletClient(client);
+
+        const mn = new Mnemonad({
+            publicClient: client.publicClient,
+            walletClient: client.walletClient,
+            contractAddress: client.contractAddress,
+            id: job.streamId,
+            ...resolveOffloadParams(args, client.account),
+        });
+        await unlockStream(mn, { password: args.password, signer: client.walletClient });
+
+        let sync;
+        let prepared;
+        let files = job.manifest?.files || null;
+        if (job.mode === 'prepare') {
+            update({ status: JOB_STATUS.PREPARING, startedAt: new Date().toISOString() });
+            await assertStreamOwner(mn, client.account.address);
+            const built = await prepareInBackground(job, args, mn);
+            if (!built) {
+                update({ status: JOB_STATUS.DONE, noChanges: true, version: mn.length, finishedAt: new Date().toISOString() });
+                log('no changes, nothing to push');
+                return;
+            }
+            ({ sync, prepared, files } = built);
+            update({ version: prepared.version, baseLength: prepared.baseLength, kind: prepared.kind, bytes: prepared.patchBytes.length });
+        } else {
+            sync = new MonadSync({ mnemonad: mn });
+            prepared = { patchBytes: new Uint8Array(await readFile(join(dir, 'patch.bin'))), baseLength: job.baseLength };
+        }
+
+        update({ status: JOB_STATUS.SENDING, sendingAt: new Date().toISOString() });
+        log(`sending version ${job.version} of ${job.streamId} on ${job.chain}...`);
+        const result = await sync.commitPush(prepared);
+
+        if (job.manifest) {
+            await writeManifest(job.folder, { streamId: job.streamId, version: result.version, files });
+        }
+        update({ status: JOB_STATUS.DONE, version: result.version, finishedAt: new Date().toISOString() });
+        log(`version ${result.version} pushed`);
+    } catch (err) {
+        const error = err.code === 'STREAM_MOVED'
+            ? `the stream changed after this push was prepared (it was built on version ${job.baseLength}) — push again`
+            : err.message;
+        update({ status: JOB_STATUS.FAILED, error, finishedAt: new Date().toISOString(), reported: false });
+        log('failed:', err.stack || err.message);
+        process.exitCode = 1;
+    } finally {
+        await rm(join(dir, 'patch.bin'), { force: true });
+    }
+}
+
+/**
+ * The part of a foreground push that `push --detach --index` hands off along with the sending:
+ * update the search index, replay the chain, and build the patch — same steps, same order, as
+ * push.js. Runs after any earlier job from the same wallet has landed, so it builds on the
+ * real tip.
+ *
+ * @returns {Promise<?{sync: MonadSync, prepared: Object, files: ?Object}>} null when the folder
+ *   already matches the latest version
+ */
+async function prepareInBackground(job, args, mn) {
+    log('updating search index...');
+    try {
+        await updateIndex(job.folder, args, { quiet: true });
+    } catch (err) {
+        throw new Error(`indexing failed, so nothing was pushed: ${err.message}`);
+    }
+
+    const fsFolder = new FSFolder(job.folder, makeExcludes(args));
+    const sync = new MonadSync({ mnemonad: mn, compress: args.compress === false ? false : 'gzip' });
+
+    if (args.forceSnapshot) {
+        // Same escape hatch push.js uses — see monadsync/README.md "Repairing a corrupt chain".
+        await mn.initialize();
+        sync._isInitialized = true;
+        sync._lastSnapshot = null;
+        sync._replayedCount = mn.length;
+    } else {
+        await sync.initialize();
+        if (mn.length > 0) {
+            const localHash = await localTreeHash(job.folder);
+            const remoteHash = await sync.getTreeHash(mn.length);
+            if (localHash && remoteHash.length === localHash.length && localHash.every((b, i) => b === remoteHash[i])) {
+                return null;
+            }
+        }
+    }
+
+    log('building patch...');
+    const prepared = await sync.preparePush(fsFolder);
+    const files = job.manifest ? await hashLocalTree(fsFolder) : null;
+    return { sync, prepared, files };
+}
+
+/** For tests: blocks until a job has finished, and returns it. */
+export async function waitForJob(dir, { timeoutMs = 60_000 } = {}) {
+    const deadline = Date.now() + timeoutMs;
+    for (;;) {
+        const job = refreshJob(dir);
+        if (job && !isActive(job)) return job;
+        if (Date.now() > deadline) throw new Error(`job ${dir} still ${job?.status} after ${timeoutMs} ms`);
+        await new Promise((r) => setTimeout(r, 200));
+    }
+}
+

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "mnemonad-cli",
-  "version": "0.3.0",
+  "version": "0.3.2",
   "description": "CLI to sync local folders to versioned, diffed on-chain streams on Monad, backed by Mnemonad + monadsync.",
   "repository": {
     "type": "git",
@@ -26,7 +26,7 @@
     "@sqliteai/sqlite-vector": "^1.1.2",
     "better-sqlite3": "^13.0.3",
     "mnemonad": "^0.0.1",
-    "monadsync": "^0.0.1",
+    "monadsync": "^0.0.2",
     "open": "^11.0.4",
     "viem": "^2.56.3",
     "ws": "^8.22.0"