c8ctl-plugin-nano 1.3.3 → 1.4.0

package/README.md

@@ -30,6 +30,30 @@ c8ctl nano start|status|stop|restart|logs|clean|set|config
 on `localhost` (round-robin partition ownership), tracks them in a state file,
 and waits until every node is reachable.
 
+## Installation
+
+This is a plugin for the [Camunda 8 CLI](https://www.npmjs.com/package/@camunda8/cli)
+(`c8ctl`). Install the CLI, then load the plugin from npm:
+
+```bash
+# 1. Install the Camunda 8 CLI (once); requires Node.js 18+
+npm install -g @camunda8/cli
+
+# 2. Load this plugin from the npm registry
+c8ctl load plugin c8ctl-plugin-nano
+
+# 3. Verify it's available
+c8ctl nano --help
+```
+
+The prebuilt Nano BPM server binary for your platform is pulled in automatically
+as an npm `optionalDependency`, so there is nothing to compile. To pull a newer
+release later, run `c8ctl nano update` (see
+[Updating to a new release](#updating-to-a-new-release-update)).
+
+> Loading from a local checkout instead? Use
+> `c8ctl load plugin --from file:///path/to/c8ctl-nano`.
+
 ## Usage
 
 ```bash
@@ -113,6 +137,35 @@ c8ctl nano stop --purge       # stop and remove engine data in one step
 
 `clean` refuses to run while any node is alive.
 
+### Stress / throughput runs: bounding disk and RAM
+
+The engine journal (`journal.jsonl`) is **append-only** — there is currently no
+compaction or rotation — and the read-model retains every terminal instance by
+default. Under sustained high load (tens of thousands of PI/s) this fills the
+disk quickly. Two `start` flags keep a long run bounded:
+
+```bash
+# Pure throughput: no journal / read-model on disk at all (in-memory engine).
+# State is lost on stop/restart, and instances live in RAM — so cap them.
+c8ctl nano start --in-memory --history-max 50000
+
+# Exercise the disk path but cap the read model's terminal-instance history.
+# (The journal still grows append-only; watch free space.)
+c8ctl nano start --history-max 50000
+```
+
+- `--in-memory` (alias `--no-journal`) routes the engine to an in-memory journal
+  and a `:memory:` read store — nothing is written under `NANOBPMN_DATA_DIR`.
+- `--history-max <n>` sets `NANOBPMN_HISTORY_MAX_INSTANCES`, continuously pruning
+  all but the most recent *n* terminal instances from the read model (`0`/unset =
+  unbounded). Works in both storage modes.
+
+`c8ctl nano status` reports the active storage mode (`in-memory` vs `on-disk`)
+and the history cap.
+
+> ⚠️ With `--in-memory`, restart recovers nothing, and Raft/replicated logs are
+> not persisted. Use it for stress/throughput testing, not durability testing.
+
 ## Configuration (`set` / `config`)
 
 Persistent settings are stored in `<state home>/config.json`:
@@ -370,13 +423,13 @@ ProcessOS is the optimization-plane server that analyses a running Nano BPM
 engine. The plugin can manage a single local ProcessOS instance with the same
 start/stop/status/logs lifecycle as `nano`.
 
-> **ProcessOS is a closed alpha.** The operational commands (`start`, `stop`,
+> **ProcessOS is a closed beta.** The operational commands (`start`, `stop`,
 > `status`, `logs`, `restart`) stay locked with a *"not available yet"* notice
 > until you opt in. Only `set` and `config` work before then. Opt in either by
 > setting the download URL you were given by the Nano BPM team, or by pointing
 > the plugin at a binary you already have.
 
-### Quick install (closed-alpha invitees)
+### Quick install (closed-beta invitees)
 
 If you were given a ProcessOS download URL, this one-liner installs the Camunda 8
 CLI (`@camunda8/cli`) and this plugin, then configures the download URL:
@@ -391,7 +444,7 @@ curl -fsSL https://gist.githubusercontent.com/jwulf/9015a7c660b274c568d80e85c391
 
 
 ```bash
-# Closed-alpha channel: persist the download URL, then start
+# Closed-beta channel: persist the download URL, then start
 c8ctl processos set download-url <url you were given>
 c8ctl processos start            # fetches processos-<os>-<arch> on first run
 
@@ -423,7 +476,7 @@ c8ctl processos stop
 
 ### Automatic update notice
 
-When you're on the closed-alpha channel (download URL configured), the
+When you're on the closed-beta channel (download URL configured), the
 plugin checks for newer ProcessOS builds in the background and prints a short
 one-line notice (at most **once per day**) when the published version is newer
 than the one you're running. It compares your installed binary's version against
@@ -473,7 +526,7 @@ Settings persist under a `processos` key in the same `config.json` as `nano`.
 
 ```bash
 c8ctl processos set bin <path>          # path to the downloaded ProcessOS binary
-c8ctl processos set download-url <url>  # closed-alpha binary download URL (enables ProcessOS)
+c8ctl processos set download-url <url>  # closed-beta binary download URL (enables ProcessOS)
 c8ctl processos set port <n>            # listen port (default 8090)
 c8ctl processos set nano-url <url>      # target Nano BPM engine (default http://localhost:8080)
 c8ctl processos set data-dir <path>     # PROCESSOS_DATA_DIR (default <stateHome>/processos-data)
@@ -489,7 +542,7 @@ configured download URL (`set download-url` / `$PROCESSOS_DOWNLOAD_URL`). Typed
 settings (`port`, `nano-url`, `data-dir`) always
 win over generic `env` passthrough values when launching.
 
-## Installing
+## Installing from a local checkout (development)
 
 ```bash
 c8ctl load plugin --from file:///path/to/c8ctl-nano
@@ -501,6 +554,8 @@ Then verify it shows up:
 c8ctl help | grep nano
 ```
 
+For the normal npm install, see [Installation](#installation) above.
+
 ## Distribution & releasing
 
 Releases are automated with **semantic-release** (`.github/workflows/release.yml`,

package/c8ctl-plugin.js

@@ -19,6 +19,7 @@
  *
  * Usage:
  *   c8ctl nano start [<nodes>] [--port <basePort>] [--partitions <n>] [--rf <n>]
+ *                    [--in-memory] [--history-max <n>]
  *   c8ctl nano status
  *   c8ctl nano stop [--purge]
  *   c8ctl nano logs [<nodeId>] [--follow]
@@ -119,7 +120,7 @@ const UPDATE_CACHE_FILE = 'update-check.json';
 const UPDATE_CHECK_TTL_MS = 24 * 60 * 60 * 1000;
 const UPDATE_NOTIFY_TTL_MS = 24 * 60 * 60 * 1000;
 
-// ProcessOS is a closed alpha distributed out-of-band: the binary lives in an
+// ProcessOS is a closed beta distributed out-of-band: the binary lives in an
 // S3 bucket whose base URL is handed to enabled users via PROCESSOS_DOWNLOAD_URL.
 // `<base>/processos-<os>-<arch>[.exe]` is the per-platform binary and
 // `<base>/version.json` is the {version,commit,updated} metadata the CI writes
@@ -322,6 +323,8 @@ function parseRequest(args, flags) {
     purge: Boolean(flags?.purge),
     force: Boolean(flags?.force),
     capture: Boolean(flags?.capture),
+    inMemory: Boolean(flags?.['in-memory'] || flags?.['no-journal']),
+    historyMax: intFlag('history-max'),
     workspace: Boolean(flags?.workspace),
     check: Boolean(flags?.check),
     binary: flags?.binary,
@@ -496,6 +499,8 @@ async function startCluster(req) {
   // Raft is required for replication; auto-enable when RF > 1, allow override.
   const raft = req.raft === undefined ? rf > 1 : Boolean(req.raft);
   const capture = Boolean(req.capture);
+  const inMemory = Boolean(req.inMemory);
+  const historyMax = req.historyMax;
 
   if (partitions < nodeCount) {
     logger.warn(
@@ -506,6 +511,21 @@ async function startCluster(req) {
   if (req.rf && req.rf > nodeCount) {
     logger.warn(`--rf ${req.rf} clamped to node count (${nodeCount}).`);
   }
+  if (inMemory) {
+    logger.warn(
+      'In-memory mode: no journal or read-model is written to disk. Engine state is ' +
+        'lost on stop/restart, and every retained instance lives in RAM' +
+        (historyMax === undefined
+          ? ' — pair with --history-max <N> to bound RAM under sustained load.'
+          : '.'),
+    );
+    if (raft || rf > 1) {
+      logger.warn(
+        'In-memory mode with Raft/replication: replicated logs are not persisted; ' +
+          'a restarted node recovers nothing.',
+      );
+    }
+  }
 
   const binary = findBinary(req);
 
@@ -548,7 +568,9 @@ async function startCluster(req) {
 
   logger.info(
     `Starting Nano BPM cluster: ${nodeCount} node(s), ${partitions} partition(s), ` +
-      `RF=${rf}${raft ? ', Raft on' : ''}${capture ? ', trace capture on' : ''}`,
+      `RF=${rf}${raft ? ', Raft on' : ''}${capture ? ', trace capture on' : ''}` +
+      `${inMemory ? ', in-memory (no disk)' : ''}` +
+      `${historyMax !== undefined ? `, history-max=${historyMax}` : ''}`,
   );
   logger.info(`Binary:    ${binary}`);
   logger.info(`Workspace: ${workspaceDir} (models/, workers/)`);
@@ -558,7 +580,7 @@ async function startCluster(req) {
     const port = ports[id];
     const dataDir = join(getDataDir(), `node-${id}`);
     const logFile = join(getLogDir(), `node-${id}.log`);
-    mkdirSync(dataDir, { recursive: true });
+    if (!inMemory) mkdirSync(dataDir, { recursive: true });
 
     const env = {
       ...process.env,
@@ -567,7 +589,6 @@ async function startCluster(req) {
       NANOBPMN_NODES: nodesEnv,
       NANOBPMN_PARTITIONS: String(partitions),
       NANOBPMN_RF: String(rf),
-      NANOBPMN_DATA_DIR: dataDir,
       // Default to async durability (group-commit) for throughput; the user can
       // override per the spread of process.env above by exporting
       // NANOBPMN_DURABILITY (e.g. "sync") before running.
@@ -584,6 +605,22 @@ async function startCluster(req) {
       // outside the per-node data dir so "nano clean" never wipes it.
       NANOBPMN_WORKSPACE_DIR: workspaceDir,
     };
+    // Storage axis: an on-disk journal + read-model under the per-node data dir
+    // (default), or a fully in-memory engine (in-memory journal + :memory: read
+    // store) when --in-memory is set. In in-memory mode, scrub any inherited
+    // path vars so nothing leaks back to disk.
+    if (inMemory) {
+      delete env.NANOBPMN_DATA_DIR;
+      delete env.NANOBPMN_JOURNAL;
+      delete env.NANOBPMN_READ_DB;
+    } else {
+      env.NANOBPMN_DATA_DIR = dataDir;
+    }
+    // Bound retained terminal instances in the read model when requested. Works
+    // in both storage modes (caps disk growth on-disk; caps RAM in-memory).
+    if (historyMax !== undefined) {
+      env.NANOBPMN_HISTORY_MAX_INSTANCES = String(historyMax);
+    }
     if (raft) env.NANOBPMN_RAFT = '1';
     // Trace capture: a single flag enables the Tier 2 recorded-input (stimuli)
     // log AND auto-enables Tier 1 variable capture, so historical replay /
@@ -614,7 +651,7 @@ async function startCluster(req) {
       process.exit(1);
     }
 
-    nodes.push({ id, port, pid: child.pid, url: peers[id], dataDir, logFile });
+    nodes.push({ id, port, pid: child.pid, url: peers[id], dataDir: inMemory ? null : dataDir, logFile });
     logger.info(`  node ${id}: pid ${child.pid} → ${peers[id]} (log: ${logFile})`);
   }
 
@@ -627,6 +664,8 @@ async function startCluster(req) {
     rf,
     raft,
     capture,
+    inMemory,
+    historyMax: historyMax ?? null,
     basePort,
     nodes,
   };
@@ -663,7 +702,7 @@ async function printSummary(state) {
   console.log('');
   console.log(
     `Nano BPM cluster is up: ${state.nodes.length} node(s), ${state.partitions} partition(s), ` +
-      `RF=${state.rf}${state.raft ? ', Raft on' : ''}`,
+      `RF=${state.rf}${state.raft ? ', Raft on' : ''}${state.inMemory ? ', in-memory (no disk)' : ''}`,
   );
   console.log('');
   for (const n of state.nodes) {
@@ -872,7 +911,14 @@ async function statusCluster(req) {
   console.log(`  binary:    ${state.binary}`);
   console.log(`  version:   ${binaryVersion(state.binary) ?? 'unknown'}`);
   console.log(`  workspace: ${state.workspaceDir || getWorkspaceDir()}`);
-  console.log(`  data:      ${getDataDir()}`);
+  const historyNote =
+    state.historyMax != null ? `, history-max ${state.historyMax}` : '';
+  if (state.inMemory) {
+    console.log(`  storage:   in-memory (no journal/read-model on disk${historyNote})`);
+  } else {
+    console.log(`  storage:   on-disk${historyNote}`);
+    console.log(`  data:      ${getDataDir()}`);
+  }
   console.log('');
   console.log('  NODE  PORT   PID       PROCESS   HEALTH    URL');
   for (const c of checks) {
@@ -1438,7 +1484,7 @@ function getProcessosNanoUrl() {
 }
 
 /**
- * The closed-alpha download URL: env var (PROCESSOS_DOWNLOAD_URL) wins, then the
+ * The closed-beta download URL: env var (PROCESSOS_DOWNLOAD_URL) wins, then the
  * persisted `processos set download-url` config value. Null when neither is set.
  */
 function getProcessosDownloadUrl() {
@@ -1699,7 +1745,7 @@ async function resolveProcessosBinary(req) {
 }
 
 /**
- * Whether ProcessOS is enabled for this user. It is a closed alpha, so the
+ * Whether ProcessOS is enabled for this user. It is a closed beta, so the
  * operational commands stay locked until the user either has the binary on
  * their system (configured path / cached download / local build) or has been
  * given a PROCESSOS_DOWNLOAD_URL to fetch it from.
@@ -1710,16 +1756,16 @@ function processosEnabled(req) {
     if (findConfiguredProcessosBinary(req)) return true;
   } catch {
     // A configured-but-missing path still means the user opted in; let the real
-    // not-found error surface from the command rather than the closed-alpha gate.
+    // not-found error surface from the command rather than the closed-beta gate.
     return true;
   }
   return false;
 }
 
-function printProcessosClosedAlpha() {
+function printProcessosClosedBeta() {
   const logger = getLogger();
   logger.error(
-    'ProcessOS is in closed alpha and is not available yet.\n' +
+    'ProcessOS is in closed beta and is not available yet.\n' +
       '\n' +
       'To enable it, set the download URL you were given by the Nano BPM team:\n' +
       '  c8ctl processos set download-url <url>   # persists it for this machine\n' +
@@ -1815,7 +1861,7 @@ function printProcessosUpdateNotice(current, latest) {
  * Best-effort, non-blocking ProcessOS update check. Triggers a background
  * version.json fetch when the cache is stale and prints a notice (at most once
  * per day) when the published version is newer than the installed one. Only
- * meaningful when a download URL is configured (the closed-alpha channel).
+ * meaningful when a download URL is configured (the closed-beta channel).
  */
 function maybeNotifyProcessosUpdate(req) {
   try {
@@ -2114,7 +2160,7 @@ function printProcessosSetUsage() {
   const logger = getLogger();
   logger.info('Usage: c8ctl processos set <field> <value>');
   logger.info('  bin <path>          Path to the downloaded ProcessOS binary');
-  logger.info('  download-url <url>  Closed-alpha binary download URL (enables ProcessOS)');
+  logger.info('  download-url <url>  Closed-beta binary download URL (enables ProcessOS)');
   logger.info('  port <n>            Listen port (default 8090)');
   logger.info('  nano-url <url>      Target Nano BPM engine URL (default http://localhost:8080)');
   logger.info('  data-dir <path>     ProcessOS data directory');
@@ -2229,14 +2275,14 @@ function showProcessosConfig() {
     }
   }
   console.log('');
-  console.log('  closed-alpha channel:');
+  console.log('  closed-beta channel:');
   const dlUrl = getProcessosDownloadUrl();
   const dlSource = process.env.PROCESSOS_DOWNLOAD_URL && String(process.env.PROCESSOS_DOWNLOAD_URL).trim()
     ? '  (from $PROCESSOS_DOWNLOAD_URL)'
     : cfg.downloadUrl
       ? '  (from "processos set download-url")'
       : '';
-  console.log(`    download url   ${dlUrl ? dlUrl + dlSource : '(not set — ProcessOS is a closed alpha; "c8ctl processos set download-url <url>" to enable)'}`);
+  console.log(`    download url   ${dlUrl ? dlUrl + dlSource : '(not set — ProcessOS is a closed beta; "c8ctl processos set download-url <url>" to enable)'}`);
   const cached = getProcessosCachedBinaryPath();
   const meta = readProcessosBinaryMeta();
   console.log(`    cached binary  ${existsSync(cached) ? cached : '(none — downloaded on first "processos start")'}`);
@@ -2262,7 +2308,7 @@ function printProcessosUsage() {
   console.log('  c8ctl processos set bin <path> | download-url <url> | port <n> | nano-url <url> | data-dir <path> | env KEY=VALUE');
   console.log('  c8ctl processos config');
   console.log('');
-  console.log('ProcessOS is a closed alpha. Enable it with the download URL you were given:');
+  console.log('ProcessOS is a closed beta. Enable it with the download URL you were given:');
   console.log('  c8ctl processos set download-url <url>   # plugin downloads + runs the matching binary');
   console.log('or point the plugin at a binary you already have: "c8ctl processos set bin <path>".');
   console.log('By default ProcessOS spawns its own internal pilot Nano engine (the plugin auto-wires the nano');
@@ -2311,6 +2357,7 @@ export const metadata = {
         },
         { command: 'c8ctl nano start 3 --port 9000', description: 'Start 3 nodes on ports 9000..9002' },
         { command: 'c8ctl nano start --capture', description: 'Start with trace capture for historical replay/analysis' },
+        { command: 'c8ctl nano start --in-memory --history-max 50000', description: 'Stress mode: no disk journal, cap retained instances in RAM' },
         { command: 'c8ctl nano status', description: 'Show cluster status and per-node health' },
         { command: 'c8ctl nano pause 1', description: 'Freeze node 1 (SIGSTOP) to simulate a node failure' },
         { command: 'c8ctl nano resume 1', description: 'Resume node 1 (SIGCONT) to bring it back online' },
@@ -2328,7 +2375,7 @@ export const metadata = {
     processos: {
       description: 'Manage a local ProcessOS instance — start, status, stop, logs, config',
       examples: [
-        { command: 'c8ctl processos set download-url <url>', description: 'Enable the closed alpha + auto-download the matching binary' },
+        { command: 'c8ctl processos set download-url <url>', description: 'Enable the closed beta + auto-download the matching binary' },
         { command: 'c8ctl processos set bin <path>', description: 'Point the plugin at a ProcessOS binary you already have' },
         { command: 'c8ctl processos start', description: 'Start ProcessOS against the local Nano BPM engine' },
         { command: 'c8ctl processos start --nano-url http://localhost:8080', description: 'Start against a specific engine' },
@@ -2353,6 +2400,9 @@ export const commands = {
       rf: { type: 'string', description: 'Replication factor; >1 enables Raft (default 1)' },
       raft: { type: 'boolean', description: 'Force per-partition Raft on/off (default: on when rf>1)' },
       capture: { type: 'boolean', description: 'start: enable trace capture (recorded-input replay) on every node' },
+      'in-memory': { type: 'boolean', description: 'start: run with NO on-disk journal/read-model (in-memory engine; state lost on restart). Alias: --no-journal' },
+      'no-journal': { type: 'boolean', description: 'start: alias for --in-memory' },
+      'history-max': { type: 'string', description: 'start: cap retained terminal instances in the read model (NANOBPMN_HISTORY_MAX_INSTANCES; 0/unset = unbounded)' },
       follow: { type: 'boolean', description: 'logs: stream output (tail -F)', short: 'f' },
       purge: { type: 'boolean', description: 'stop: also delete per-node engine data' },
       force: { type: 'boolean', description: 'start: stop any existing cluster first' },
@@ -2435,12 +2485,12 @@ export const commands = {
         return;
       }
 
-      // ProcessOS is a closed alpha: gate the operational commands until the
+      // ProcessOS is a closed beta: gate the operational commands until the
       // user has opted in (download URL set or a binary on their system).
       // `set`/`config` stay open so users can configure/inspect at any time.
       const ungated = req.subcommand === 'set' || req.subcommand === 'config';
       if (!ungated && !processosEnabled(req)) {
-        printProcessosClosedAlpha();
+        printProcessosClosedBeta();
         process.exit(1);
       }
 
@@ -2484,7 +2534,7 @@ export const commands = {
 
 function printUsage() {
   console.log('Usage:');
-  console.log('  c8ctl nano start [<nodes>] [--port <basePort>] [--partitions <n>] [--rf <n>] [--raft] [--capture] [--binary <path>]');
+  console.log('  c8ctl nano start [<nodes>] [--port <basePort>] [--partitions <n>] [--rf <n>] [--raft] [--capture] [--in-memory] [--history-max <n>] [--binary <path>]');
   console.log('  c8ctl nano status [--port <port>]');
   console.log('  c8ctl nano stop [--purge]');
   console.log('  c8ctl nano logs [<nodeId>] [--follow]');
@@ -2516,6 +2566,8 @@ function printUsage() {
   console.log('  --rf <n>             Replication factor; >1 enables Raft (default 1)');
   console.log('  --raft               Force Raft on (default: on iff rf>1)');
   console.log('  --capture            start: enable trace capture (recorded-input replay) on every node');
+  console.log('  --in-memory          start: run with NO on-disk journal/read-model (alias --no-journal; state lost on restart)');
+  console.log('  --history-max <n>    start: cap retained terminal instances in the read model (0/unset = unbounded)');
   console.log('  --binary <path>      Path to the nanobpmn server binary (overrides "set bin")');
   console.log('  --purge              stop: also delete per-node engine data');
   console.log('  --force              start: stop any existing cluster first');

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "c8ctl-plugin-nano",
-  "version": "1.3.3",
+  "version": "1.4.0",
   "type": "module",
   "description": "c8ctl plugin to start, inspect, and stop a local Nano BPM (nanobpmn) cluster",
   "main": "c8ctl-plugin.js",
@@ -49,10 +49,10 @@
     "semantic-release": "^25.0.3"
   },
   "optionalDependencies": {
-    "@nanobpm/c8ctl-plugin-nano-darwin-arm64": "1.3.3",
-    "@nanobpm/c8ctl-plugin-nano-darwin-x64": "1.3.3",
-    "@nanobpm/c8ctl-plugin-nano-linux-x64": "1.3.3",
-    "@nanobpm/c8ctl-plugin-nano-linux-arm64": "1.3.3",
-    "@nanobpm/c8ctl-plugin-nano-win32-x64": "1.3.3"
+    "@nanobpm/c8ctl-plugin-nano-darwin-arm64": "1.4.0",
+    "@nanobpm/c8ctl-plugin-nano-darwin-x64": "1.4.0",
+    "@nanobpm/c8ctl-plugin-nano-linux-x64": "1.4.0",
+    "@nanobpm/c8ctl-plugin-nano-linux-arm64": "1.4.0",
+    "@nanobpm/c8ctl-plugin-nano-win32-x64": "1.4.0"
   }
 }