@getopencrawl/cli 0.4.8 → 0.4.10

package/README.md

@@ -47,17 +47,18 @@ or forbidden, 64 invalid arguments, 130 interrupted.
 ## Updates and distribution
 
 TypeScript compiled with Bun 1.3.14 produces a self-contained binary. Users need
-Node 22+ and npm for the bootstrap, never Bun. npm platform packages cover macOS
+Node 22.15+ and npm for the bootstrap, never Bun. npm platform packages cover macOS
 arm64/x64, Linux glibc arm64/x64 and Windows x64. Alpine/musl and Windows arm64
 are not supported in v0. Platform packages are optional dependencies at the exact
 launcher version; there are no install scripts.
 
-After a command, the launcher starts a detached updater at most once per day.
+Before ordinary commands, the launcher checks for updates at most once per day.
+It awaits the update in the invoking process; hooks never check for updates.
 It checks the npm `next` release channel, installs into a private user-owned cache with
 npm integrity verification and install scripts disabled, then atomically selects
-the complete release. The next invocation uses it. No sudo or global-directory
+the complete release. The command uses the selected binary. No sudo or global-directory
 writes are needed. Offline/update failures preserve the working copy and retry
-after 15 minutes. Concurrent updater processes share a lock; updates never
+after 15 minutes. Concurrent invocations share an update lock; updates never
 downgrade. An unused installation updates when it is next invoked, not on a timer.
 
 `OPENCRAWL_NO_UPDATE=1` disables automatic updates for reproducible agent runs.
@@ -101,12 +102,35 @@ background upgrades; Glen's small npm entry point for agents.
 
 ### Session capture
 
-After signing in, run `opencrawl collect install` to install Claude Code and Codex hooks and the background transcript collector. Review the commands in Codex `/hooks` or desktop Settings > Hooks. The same local configuration covers desktop and terminal sessions.
+After signing in, run `opencrawl collect install` to install Claude Code and Codex hooks and the on-demand transcript uploader. Review the commands in Codex `/hooks` or desktop Settings > Hooks. The same local configuration covers desktop and terminal sessions.
 
-`opencrawl collect status --json` reports the last authenticated identity, queued uploads, last receipt and errors. `collect sync` reconciles active transcripts; `collect flush` retries queued uploads; `collect uninstall` removes OpenCrawl hooks and stops collection while preserving queued data.
+`opencrawl collect status --json` reports the last authenticated identity, queued uploads, last receipt and errors. `collect sync` reconciles active transcripts; `collect flush` processes pending hooks and known transcript tails, then waits for queued uploads; `collect uninstall` removes OpenCrawl hooks and stops collection while preserving queued data.
 
 Messages, tool arguments/results and attachments are collected with native session IDs and a stable installation ID. System/developer prompts are excluded. The hosted service determines the current team from login and validates human attribution. Existing collectors and queued personal-scope uploads automatically use that team after updating, without reinstalling or resetting. Self-hosted users can supply `OPENCRAWL_SESSIONS_TOKEN` and `--api` for the portable receiver. See [capture architecture and limits](../docs/session-capture.md).
 
+The harness runs each hook asynchronously and owns its process. The hook saves
+its notice locally, then competes for one nonblocking uploader lock. The winner
+parses permitted transcripts, spools immutable batches and uploads; other hooks
+exit after saving their notices. On Unix the shell and Node dispatcher replace
+themselves with the native CLI using exec, keeping the harness-owned PID. Capture
+never detaches a child, starts a service or launches a replacement worker.
+Foreground flush/backfill waits for upload ownership and confirmed receipts.
+
+No systemd, launchd, cron, or installed daemon is required. Reinstall removes
+legacy OpenCrawl timers. Existing installations upgrade their owned hook settings
+on the next CLI invocation; restart existing sessions to load async settings.
+Legacy synchronous hooks remain local-only until those settings reload.
+The uploader polls known transcripts for 30 seconds after activity, including
+late writes after Stop, and retries failures with backoff while the harness lets
+it run. Later hooks or explicit sync/flush recover retained work after process
+loss. Harness teardown may cancel an async hook; no delivery is possible after
+a sandbox destroys unsent files. An idle interval is not a final-session receipt.
+`collect status` reports pending hooks separately from queued batches.
+
+Cloud image setup uses `collect install --harness claude-code --defer-auth
+--backfill skip` without a runtime credential or identity lookup. The first async
+hook binds identity before parsing or uploading, without blocking the agent.
+
 ### Optional history import
 
 `collect install` asks terminal users whether to import past Claude Code sessions,
@@ -185,7 +209,7 @@ a separate setup session are supported. Environment variables are visible to
 commands in that environment; don't paste the credential into the chat or repo.
 
 The installer detects `CLAUDE_CODE_REMOTE=true`, installs Claude hooks only and
-skips systemd. `--harness claude-code --no-background` also selects this explicitly.
+uses the same service-free uploader as local installations. `--harness claude-code` selects just Claude hooks; `--no-background` remains an accepted compatibility flag.
 The setup installs the CLI at
 `$HOME/.local/share/opencrawl-cloud/node_modules/.bin/opencrawl`. After a user
 message, tool call and image, `collect status` must report a stored receipt and

package/bin/opencrawl.cjs

@@ -6,7 +6,7 @@ const fs = require('node:fs');
 const path = require('node:path');
 const os = require('node:os');
 const {createRequire} = require('node:module');
-const {spawn, spawnSync} = require('node:child_process');
+const {spawnSync} = require('node:child_process');
 const {randomUUID} = require('node:crypto');
 const {parseArgs} = require('node:util');
 const ROOT = path.resolve(__dirname, '..');
@@ -123,14 +123,10 @@ async function installRelease(version, {registry = REGISTRY} = {}) {
     fs.renameSync(stage, destination);
   } finally { fs.rmSync(stage, {recursive: true, force: true}); }
 }
-function scheduleUpdate() {
+async function automaticUpdate() {
   if (process.env.OPENCRAWL_NO_UPDATE === '1') return;
-  try {
-    if (read(path.join(cacheRoot(), 'update.json'))?.checkedAt > Date.now() - DAY) return;
-    const child = spawn(process.execPath, [__filename, '--internal-update'], {detached: true, stdio: 'ignore', windowsHide: true});
-    child.on('error', () => {});
-    child.unref();
-  } catch {} // updating can never make an ordinary command fail
+  try { await update(); }
+  catch { try { write(path.join(cacheRoot(), 'update.json'), {checkedAt: Date.now() - DAY + 15 * 60_000}); } catch {} }
 }
 async function main(args = process.argv.slice(2)) {
   if (args[0] === '--internal-update') {
@@ -140,9 +136,9 @@ async function main(args = process.argv.slice(2)) {
   const selected = activeRoot();
   // Delegate the launcher too, so later releases can fix bootstrap behavior.
   if (selected !== ROOT) {
-    const delegated = spawnSync(process.execPath, [path.join(selected, 'bin/opencrawl.cjs'), ...args], {stdio: 'inherit', windowsHide: true, env: {...process.env, OPENCRAWL_LAUNCHER: process.env.OPENCRAWL_LAUNCHER || __filename, OPENCRAWL_NODE: process.execPath}});
-    if (delegated.error) throw new Error('Could not start the cached launcher.');
-    return delegated.signal === 'SIGINT' ? 130 : delegated.status ?? 1;
+    process.env.OPENCRAWL_LAUNCHER ||= __filename;
+    process.env.OPENCRAWL_NODE = process.execPath;
+    return require(path.join(selected, 'bin/opencrawl.cjs')).main(args);
   }
   let parsed;
   try { parsed = parseArgs({args, allowPositionals: true, options: {json: {type: 'boolean'}, 'no-browser': {type: 'boolean'}, api: {type: 'string'}, version: {type: 'boolean', short: 'v'}, help: {type: 'boolean', short: 'h'}}}); }
@@ -152,9 +148,9 @@ async function main(args = process.argv.slice(2)) {
     process.stdout.write(args.includes('--json') ? JSON.stringify({schemaVersion: 1, data}) + '\n' : data.busy ? 'An update is already running.\n' : data.updated ? `Installed opencrawl ${data.version}. Your next invocation will use it.\n` : `opencrawl ${data.version} is current.\n`);
     return 0;
   }
+  if (!(args[0] === 'collect' && ['hook','worker'].includes(args[1]))) await automaticUpdate();
   const root = activeRoot();
   const result = spawnSync(binary(root), args, {stdio: 'inherit', windowsHide: true, env: {...process.env, OPENCRAWL_LAUNCHER: process.env.OPENCRAWL_LAUNCHER || __filename, OPENCRAWL_NODE: process.execPath}});
-  scheduleUpdate();
   if (result.error) throw new Error('Could not start the OpenCrawl binary. Reinstall @getopencrawl/cli.');
   if (result.signal === 'SIGINT') return 130;
   return result.status ?? 1;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@getopencrawl/cli",
-  "version": "0.4.8",
+  "version": "0.4.10",
   "description": "OpenCrawl CLI: sign in now, query the index as features become available",
   "type": "module",
   "license": "UNLICENSED",
@@ -17,17 +17,17 @@
     "directory": "cli"
   },
   "engines": {
-    "node": ">=22"
+    "node": ">=22.15"
   },
   "publishConfig": {
     "access": "public",
     "registry": "https://registry.npmjs.org"
   },
   "optionalDependencies": {
-    "@getopencrawl/cli-darwin-arm64": "0.4.8",
-    "@getopencrawl/cli-darwin-x64": "0.4.8",
-    "@getopencrawl/cli-linux-arm64": "0.4.8",
-    "@getopencrawl/cli-linux-x64": "0.4.8",
-    "@getopencrawl/cli-win32-x64": "0.4.8"
+    "@getopencrawl/cli-darwin-arm64": "0.4.10",
+    "@getopencrawl/cli-darwin-x64": "0.4.10",
+    "@getopencrawl/cli-linux-arm64": "0.4.10",
+    "@getopencrawl/cli-linux-x64": "0.4.10",
+    "@getopencrawl/cli-win32-x64": "0.4.10"
   }
 }