@tikoci/centrs 0.1.8 → 0.1.9-next.0

package/docs/errors/cmr/outcome-unknown.md

@@ -0,0 +1,29 @@
+# `cmr/outcome-unknown`
+
+`execute --cmr` sent the command to a CMR client, but no final status came back
+for it. The command **may have run, may still be running, or may not have
+run**.
+
+## Typical trigger
+
+- CMR reported `failed` with `timed out (6)` (the `timeout=` centrs passes, from
+  `--timeout`) or `std failure: timeout (13)` (CMR's own ~60 s ceiling per call).
+- centrs stopped waiting: its local deadline passed, the operation was
+  cancelled, or the controller connection dropped after the request was sent.
+- The client changed its CMR key (`ids`) during the call, or the controller's
+  inventory could not be re-read afterwards to confirm which client answered.
+
+On RouterOS 7.26beta1, a timed-out or cancelled `run-script` **kept running on
+the client**, and its side effects landed afterwards (#441, #450). A timeout is
+not a rollback.
+
+## Fix
+
+- Check the client's state before doing anything else. Read the menu the
+  command changed, directly or through the controller.
+- Do not re-run a command that is not idempotent until you know it did not run.
+  centrs never retries a CMR dispatch.
+- For long work, raise `--timeout` (up to 55 s per call). Anything needing more
+  than CMR's ~60 s ceiling belongs in a script or scheduler entry on the client.
+
+See [`commands/execute/README.md`](../../../commands/execute/README.md) → CMR.

package/docs/errors/cmr/output-truncated.md

@@ -0,0 +1,16 @@
+# `cmr/output-truncated`
+
+A warning: CMR clipped a client's command output at its cap (about 60 KiB on
+7.26beta1) and appended `...(+N B)`. The command **completed**; only the
+returned text is incomplete. The envelope's `meta.truncated` records it.
+
+## Typical trigger
+
+A command that prints a lot, such as a large `print` or `/export`, run with
+`execute --cmr`.
+
+## Fix
+
+Narrow the output on the client: `print` with `where` and `proplist=`, or
+`:put` only the fields you need. For a full configuration, save it to a file on
+the client instead of printing it.

package/docs/errors/cmr/policy-denied.md

@@ -0,0 +1,31 @@
+# `cmr/policy-denied`
+
+A CMR client ran the command and refused one of its statements with
+`not enough permissions (9)`.
+
+CMR runs `run-script` scripts on each client with only the `read`, `write`,
+`test` and `reboot` policies, and without a user session. Validation cannot
+catch this: the client's `:parse` accepts the command, and the policy check
+happens only when it runs.
+
+The refused statement did not run, and the script stopped there. Statements
+before it **did** run: CMR does not roll them back. Their output, followed by
+the diagnostic, is in `context.output`.
+
+## Typical trigger
+
+- `/user add`, `/user group add` (needs `policy`).
+- `/tool sniffer` (needs `sniff`).
+
+Reading a sensitive value is not this error: it succeeds and returns `*` or an
+empty value.
+
+## Fix
+
+Check what the earlier statements changed. Then run only the refused
+statement, and the ones after it, on the device directly with a user that has
+the policy:
+
+```sh
+centrs execute <device> -- '<command>'
+```

package/docs/errors/cmr/selection-incomplete.md

@@ -0,0 +1,18 @@
+# `cmr/selection-incomplete`
+
+centrs asked the controller, for every `/cmr/device` row, whether it matches
+`--labels`. At least one row got no answer, so the selection is not known and
+**nothing was run**.
+
+## Typical trigger
+
+A per-device preflight call timed out, was cancelled, or lost its connection
+before the controller either refused it (`no devices selected`, so not a
+member) or returned a row (a member).
+
+## Fix
+
+- Check that the controller is reachable and responsive, then retry.
+- Raise `--timeout` if the controller is slow.
+- centrs never runs a partial selection, because a smaller set that reports
+  success would hide the clients it skipped.

package/docs/errors/cmr/validation-incomplete.md

@@ -0,0 +1,23 @@
+# `cmr/validation-incomplete`
+
+The validation result from a CMR client was incomplete, so the command was
+**not** sent to it. The other members are unaffected.
+
+## Typical trigger
+
+centrs validates on each client with `:put [:parse <command>]; :put "<nonce>"`
+through `run-script`. The result is rejected before it is classified when:
+
+- the status was not `success` (for example `timed out (6)`);
+- the output was clipped by CMR's ~60 KiB cap (`...(+N B)`);
+- the nonce line that proves the output is complete is missing;
+- the `:parse` return was empty or was not parse output (IL).
+
+An empty or clipped result is not a pass. Without the completeness check,
+missing output would have read as "no diagnostic".
+
+## Fix
+
+- Make sure the client is connected and responsive, then retry.
+- Very large commands can exceed the output cap once `:parse` echoes them back.
+  Split the work into smaller commands.

package/docs/errors/routeros/version-unsupported.md

@@ -9,6 +9,10 @@ error.
 Distinct from `routeros/unsupported-capability`, where the chosen *protocol*
 cannot carry the operation at all regardless of device version.
 
+`execute --cmr` raises it with `capability: cmr-controller` when the target has
+no `/cmr/device` and runs RouterOS older than 7.26beta1, where CMR first
+shipped. That check reads the version only after the missing menu, not before.
+
 ## Context fields
 
 - `capability` — stable slug of the missing operation.

package/docs/errors/transport/write-outcome-unknown.md

@@ -0,0 +1,24 @@
+# `transport/write-outcome-unknown`
+
+A warning on a failed write. The request had already been sent when the
+transport failed (a dropped connection, a REST 5xx, a timeout), so the router
+may have run it. The error beside this warning says how the call ended, not
+whether the write happened.
+
+## Typical trigger
+
+- A connection closed or reset after an `execute` or `api` write was sent.
+- A REST 5xx or a timeout on a write.
+
+centrs never retries a write that it has sent, in single-target or fan-out
+mode, because replaying an `add`, a script run or a counter change would apply
+it twice (#448). In fan-out, a write is retried only when a retryable
+transport failure (`transport/network` or `transport/connection-closed`) ends
+it *before* it was sent, for example during remote validation or a native API
+login. Any other failure, sent or not, is not retried.
+
+## Fix
+
+Check the router's state before running the write again. For example, read
+back the menu you changed with `centrs retrieve`. Re-run only what did not
+apply.

package/docs/errors/usage/missing-labels.md

@@ -0,0 +1,19 @@
+# `usage/missing-labels`
+
+`centrs execute --cmr` was given no `--labels` expression.
+
+## Typical trigger
+
+`--cmr` runs the command on the CMR clients a label expression selects, and the
+expression is required so the blast radius is always explicit.
+
+## Fix
+
+Pass RouterOS label syntax: a comma list where plain labels are OR, `+label`
+must match, `-label` excludes, and `all` selects every device, including the
+controller:
+
+```sh
+centrs execute ctl --cmr --labels all -- ':put [/system/identity/get name]'
+centrs execute ctl --cmr --labels site-a,+edge,-ctl --yes -- '/ip/dns/set servers=192.0.2.53'
+```

package/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "@tikoci/centrs",
-	"version": "0.1.8",
+	"version": "0.1.9-next.0",
 	"main": "src/index.ts",
 	"module": "src/index.ts",
 	"repository": {

package/src/api-fanout.ts

@@ -47,6 +47,7 @@ import {
 	buildFanoutResolveFailure,
 	commonVia,
 	defaultFanoutSleep,
+	mayRetryUnsentWrite,
 	resolveFanoutConcurrency,
 	runFanout,
 	summarizeFanout,
@@ -105,7 +106,10 @@ export interface ApiFanoutInternals {
 		env: Record<string, string | undefined>,
 		config?: Record<string, string | undefined>,
 	) => Promise<SelectionExpansion>;
-	execute?: (resolved: ResolvedApiRequest) => Promise<ApiSuccessEnvelope>;
+	execute?: (
+		resolved: ResolvedApiRequest,
+		trace: ValidationTrace,
+	) => Promise<ApiSuccessEnvelope>;
 	sleep?: (ms: number) => Promise<void>;
 }
 
@@ -318,6 +322,8 @@ export async function apiFanout(
 				),
 				member,
 			),
+		mayRetry: (resolved) =>
+			mayRetryUnsentWrite(requestSummary.write, traces.get(resolved)),
 		sleep,
 	});
 

package/src/api.ts

@@ -15,6 +15,7 @@
  * structured validation, confirmation, and result-envelope surface.
  */
 
+import { assertNotCmrDashboardPath, cmrRunScriptWarnings } from "./core/cmr.ts";
 import type {
 	CentrsEnvelope,
 	CentrsErrorEnvelope,
@@ -28,6 +29,7 @@ import type {
 	Warning,
 } from "./core/envelope.ts";
 import { buildTip, notReachedReason } from "./core/envelope.ts";
+import { sentWriteWarnings } from "./core/fanout.ts";
 import {
 	inspectArgumentNames,
 	inspectChildrenOrEmpty,
@@ -393,6 +395,11 @@ export async function runResolvedApi(
 			resolved === request
 				? protocolRequest
 				: buildProtocolApiRequest(resolved),
+			{
+				onSent: () => {
+					trace.dispatched = true;
+				},
+			},
 		);
 		const routerOsFailure = apiRouterOsFailureFromResult(resolved, result);
 		if (routerOsFailure) {
@@ -563,7 +570,10 @@ async function* streamResolvedApi(
 		const protocolRequest = buildProtocolApiRequest(resolved);
 		for await (const reply of backend.stream(protocolRequest, {
 			signal: controller.signal,
-			onListening,
+			onListening: () => {
+				trace.dispatched = true;
+				onListening?.();
+			},
 			onCancelUnacknowledged: () => {
 				cancelUnacknowledged = true;
 			},
@@ -602,7 +612,7 @@ async function* streamResolvedApi(
 			done,
 		);
 	} catch (error) {
-		const envelope = buildApiErrorEnvelopeFromResolved(resolved, error);
+		const envelope = buildApiErrorEnvelopeFromResolved(resolved, error, trace);
 		envelope.meta.validation = validation;
 		envelope.warnings = [...envelope.warnings, ...cancellationWarnings()];
 		envelope.tips = [...envelope.tips, ...tips];
@@ -1029,9 +1039,10 @@ export async function resolveApiRequest(
 		insecure,
 		yes: request.yes ?? false,
 		verbose: request.verbose ?? false,
-		warnings: connection
-			? [...connection.warnings]
-			: (cdbResolution?.warnings ?? []),
+		warnings: [
+			...(connection ? connection.warnings : (cdbResolution?.warnings ?? [])),
+			...(verb === "run" ? cmrRunScriptWarnings(normalized.path) : []),
+		],
 	};
 }
 
@@ -1718,7 +1729,14 @@ export function buildApiErrorEnvelopeFromResolved(
 	return {
 		ok: false,
 		error: serializeCentrsError(centrsError),
-		warnings: [...resolved.warnings],
+		warnings: [
+			...resolved.warnings,
+			...sentWriteWarnings(
+				isApiMutating(resolved.method, resolved.path),
+				centrsError,
+				trace,
+			),
+		],
 		tips: [],
 		meta: metaFromResolved(
 			resolved,
@@ -2020,6 +2038,7 @@ export function validateApiRequestShape(request: ApiRequest): void {
 		});
 	}
 	const normalized = normalizeApiEndpoint(request.endpoint);
+	assertNotCmrDashboardPath(normalized.path);
 	if (pathTokens(normalized.path).length === 0) {
 		throw new CentrsError({
 			code: "input/invalid-path",

package/src/cli/execute.ts

@@ -1,5 +1,9 @@
 import { fanoutExitCode } from "../core/fanout.ts";
-import { asCentrsError, formatCentrsErrorText } from "../errors.ts";
+import {
+	asCentrsError,
+	CentrsError,
+	formatCentrsErrorText,
+} from "../errors.ts";
 import {
 	buildExecuteErrorEnvelope,
 	type ExecuteOutputFormat,
@@ -8,6 +12,11 @@ import {
 	executeOutputFormats,
 	renderExecuteEnvelope,
 } from "../execute.ts";
+import {
+	buildExecuteCmrErrorEnvelope,
+	executeCmr,
+	renderExecuteCmrEnvelope,
+} from "../execute-cmr.ts";
 import {
 	buildExecuteFanoutErrorEnvelope,
 	executeFanout,
@@ -54,6 +63,17 @@ export const executeCommand: CliCommandMetadata = {
 				"Pin the protocol selector; no silent downgrade when set. A bare MAC target defaults to mac-telnet.",
 		},
 		...selectionCommandOptions,
+		{
+			flag: "--cmr",
+			description:
+				"Run on CMR clients through their controller: the <target> is the controller (RouterOS 7.26beta1+, `cmr` package). Needs `--labels`; native-api or rest-api only.",
+		},
+		{
+			flag: "--labels",
+			valueName: "<expr>",
+			description:
+				"With `--cmr`: RouterOS label expression selecting clients (comma list; plain = OR, `+` AND, `-` AND NOT, `all`).",
+		},
 		{
 			flag: "--host",
 			valueName: "<host|url>",
@@ -147,6 +167,8 @@ export const executeCommand: CliCommandMetadata = {
 
 interface ExecuteCliArgs extends ExecuteRequest {
 	help?: boolean;
+	cmr?: boolean;
+	labels?: string;
 	format?: ExecuteOutputFormat;
 	selectionFlags?: SelectionFlags;
 	targetPositionals?: readonly string[];
@@ -194,6 +216,12 @@ export function parseExecuteCliArgs(args: readonly string[]): ExecuteCliArgs {
 			case "--yes":
 				parsed.yes = true;
 				break;
+			case "--cmr":
+				parsed.cmr = true;
+				break;
+			case "--labels":
+				parsed.labels = expectValue(args, ++index, arg);
+				break;
 			case "--validate":
 				parsed.validate = true;
 				break;
@@ -322,6 +350,14 @@ export async function runExecuteCli(args: readonly string[]): Promise<number> {
 				selectionFlags.quickchr[0] ?? "",
 			);
 		}
+		if (parsed.cmr || parsed.labels !== undefined) {
+			return await runExecuteCmrCli(
+				parsed,
+				selectionFlags,
+				targetPositionals,
+				args,
+			);
+		}
 		if (isFanoutMode(selectionFlags, targetPositionals.length)) {
 			return await runExecuteFanoutCli(
 				parsed,
@@ -450,6 +486,101 @@ async function runExecuteFanoutCli(
 	}
 }
 
+/**
+ * `execute --cmr`: the single target is the CMR controller; `--labels` picks
+ * the clients. Fan-out selectors would mix two selection sources, so they are
+ * refused rather than combined (#450).
+ */
+async function runExecuteCmrCli(
+	parsed: ExecuteCliArgs,
+	selectionFlags: SelectionFlags,
+	targetPositionals: readonly string[],
+	args: readonly string[],
+): Promise<number> {
+	const format = inferExecuteFormat(args, parsed);
+	const controller = new AbortController();
+	// First Ctrl-C cancels (unsent members become `cmr/cancelled`); a second
+	// one exits at once in case a controller call does not end promptly.
+	const onSigint = (): void => {
+		if (controller.signal.aborted) process.exit(130);
+		controller.abort();
+	};
+	process.on("SIGINT", onSigint);
+	try {
+		if (!parsed.cmr) {
+			throw new CentrsError({
+				code: "usage/conflicting-flags",
+				summary:
+					"`--labels` selects CMR clients and only applies with `--cmr`.",
+				remediation:
+					"Add `--cmr` and name the controller as the target, or drop `--labels`.",
+			});
+		}
+		if (
+			hasTargetSelector({ ...selectionFlags, quickchr: [] }) ||
+			selectionFlags.quickchr.length > 1 ||
+			targetPositionals.length > 1
+		) {
+			throw new CentrsError({
+				code: "usage/conflicting-flags",
+				summary:
+					"`--cmr` takes one controller target; CDB fan-out selectors and several targets cannot be combined with it.",
+				remediation:
+					"Name one controller (a positional, `--host` or one `--quickchr`) and pick clients with `--labels`.",
+				context: {
+					positionals: targetPositionals.length,
+					quickchr: selectionFlags.quickchr.length,
+				},
+			});
+		}
+		if (selectionFlags.quickchr.length === 1) {
+			parsed.quickchr = selectionFlags.quickchr[0];
+		}
+		if (
+			!parsed.targetInput &&
+			parsed.quickchr === undefined &&
+			parsed.host === undefined
+		) {
+			throw missingTargetError({
+				command: "execute",
+				summary:
+					"`centrs execute --cmr` requires the CMR controller as its <target>.",
+				remediation:
+					'Name the controller, e.g. `centrs execute ctl --cmr --labels all -- ":put ok"`.',
+			});
+		}
+		const envelope = await executeCmr({
+			...parsed,
+			labels: parsed.labels ?? "",
+			concurrency: selectionFlags.concurrency,
+			signal: controller.signal,
+		});
+		const resolvedFormat = envelope.meta.operation?.request.format ?? format;
+		console.log(
+			renderExecuteCmrEnvelope(envelope, resolvedFormat, {
+				verbose: parsed.verbose ?? false,
+			}),
+		);
+		return fanoutExitCode(envelope);
+	} catch (error) {
+		console.error(
+			renderExecuteCmrEnvelope(
+				buildExecuteCmrErrorEnvelope(
+					{ targetInput: parsed.targetInput, via: parsed.via },
+					error,
+				),
+				format,
+				{
+					verbose: parsed.verbose ?? false,
+				},
+			),
+		);
+		return 1;
+	} finally {
+		process.off("SIGINT", onSigint);
+	}
+}
+
 function inferExecuteFormat(
 	args: readonly string[],
 	parsed?: ExecuteCliArgs,

package/src/core/cmr.ts

@@ -0,0 +1,104 @@
+/**
+ * CMR facts shared by the surfaces that can reach `/cmr/device/run-script`
+ * directly (`api`, plain `execute`) and by `execute --cmr` (#450).
+ */
+
+import { CentrsError } from "../errors.ts";
+import type { Warning } from "./envelope.ts";
+
+/** The controller command that runs a script on CMR clients. */
+export const CMR_RUN_SCRIPT_PATH = "/cmr/device/run-script";
+
+/**
+ * The console-only live view. Grounded on 7.26beta1 CHR: over the native API
+ * `/cmr/device/dashboard` closed the API session; over REST it never returned,
+ * and every other REST call on the controller then waited 10–30 s.
+ */
+const CMR_DASHBOARD = ["cmr", "device", "dashboard"] as const;
+const DASHBOARD_PATH = "/cmr/device/dashboard";
+
+/**
+ * Whether a structured RouterOS path (an `api` endpoint, or a single
+ * `execute` command's path and verb) is the dashboard. Each segment may be an
+ * abbreviation, as RouterOS accepts (`/cmr/dev/dash`). This needs no router, so
+ * it refuses the plain spellings before anything is dialed.
+ */
+export function isCmrDashboardPath(path: string): boolean {
+	const segments = path
+		.split("/")
+		.filter((segment) => segment.length > 0)
+		.map((segment) => segment.toLowerCase());
+	return (
+		segments.length === CMR_DASHBOARD.length &&
+		segments.every((segment, k) =>
+			(CMR_DASHBOARD[k] as string).startsWith(segment),
+		)
+	);
+}
+
+/**
+ * Whether a `:parse` return invokes the dashboard. The device resolves every
+ * spelling to one invocation: abbreviations, console spacing, a relative menu
+ * change across `;` or a newline, and `:do`/`:if`/`[...]` bodies. Strings stay
+ * `=` values and comments are dropped (7.26beta1 CHR). An invocation follows
+ * the start of the IL, a `;` or another `(evl `. A string value sits after `=`.
+ */
+export function ilInvokesCmrDashboard(il: string): boolean {
+	return /(?:^|;|\(evl )\(evl \/cmr\/device\/dashboard/.test(il);
+}
+
+function consoleOnlyError(input: string): CentrsError {
+	return new CentrsError({
+		code: "cmr/console-only",
+		summary: `\`${DASHBOARD_PATH}\` (in any spelling) is an interactive console view, not an API command. Over the native API it closes the session; over REST it holds the controller's REST service for every caller until it times out.`,
+		remediation:
+			"Open it in WinBox 4 or WebFig, or run it in `centrs terminal <controller>`. For the data behind it, read `/cmr/device`, `/cmr/alert` or `/cmr/upgrade/job` with `centrs retrieve`.",
+		context: { path: DASHBOARD_PATH, input },
+	});
+}
+
+/** Refuse a structured dashboard path before anything is dialed. */
+export function assertNotCmrDashboardPath(path: string): void {
+	if (isCmrDashboardPath(path)) throw consoleOnlyError(path);
+}
+
+/** Refuse a command whose accepted `:parse` return invokes the dashboard. */
+export function assertNotCmrDashboardIl(il: string, command: string): void {
+	if (ilInvokesCmrDashboard(il)) throw consoleOnlyError(command);
+}
+
+function withoutTrailingSlashes(path: string): string {
+	// A loop, not `/\/+$/`: that regex is quadratic on a long run of `/`.
+	let end = path.length;
+	while (end > 0 && path[end - 1] === "/") end -= 1;
+	return path.slice(0, end);
+}
+
+const clientScriptNotValidated: Warning = {
+	code: "cmr/client-script-not-validated",
+	message:
+		"Validation covered the controller request only; the `script=` sent to CMR clients was not validated on them. Use `centrs execute <controller> --cmr --labels <expr> -- '<command>'` to validate on each client first.",
+	context: { path: CMR_RUN_SCRIPT_PATH },
+};
+
+/**
+ * A caller-addressed `run-script` is validated as a controller request only:
+ * `/console/inspect` checks its arguments, and the `script=` value is an opaque
+ * string to the controller's `:parse`. The script itself then runs without validation
+ * on every selected client, so say so next to the `passed` gate.
+ */
+export function cmrRunScriptWarnings(commandPath: string): Warning[] {
+	if (withoutTrailingSlashes(commandPath) !== CMR_RUN_SCRIPT_PATH) return [];
+	return [clientScriptNotValidated];
+}
+
+/**
+ * Script-shaped `execute` input is opaque to the canonicalizer, so a
+ * `run-script` inside it (console spacing, a `:do {}` body) is found by text.
+ * A false positive costs only a warning.
+ */
+export function cmrScriptWarnings(script: string): Warning[] {
+	return /cmr[\s/]+device[\s/]+run-script\b/.test(script)
+		? [clientScriptNotValidated]
+		: [];
+}

package/src/core/envelope.ts

@@ -109,6 +109,17 @@ export interface EnvelopeTargetMeta {
 	source?: SettingSource;
 	/** Provenance per resolved field once the resolver is wired (WP-0c). */
 	sources?: Record<string, SettingSource>;
+	/**
+	 * `execute --cmr`: the CMR client this result belongs to, as the controller
+	 * knows it. `id` is the controller's `/cmr/device` row; `deviceKey` is the
+	 * device half of its `ids`. The client is never dialed directly.
+	 */
+	cmr?: {
+		controller: string;
+		id: string;
+		deviceKey?: string;
+		address?: string;
+	};
 }
 
 /**
@@ -153,6 +164,11 @@ export interface ValidationTrace {
 	 * to judge.
 	 */
 	offlineVerdict?: "pass" | "warn";
+	/**
+	 * Set immediately before the RouterOS request is sent. A failure after this
+	 * point may have run on the router, so a write must not be replayed (#448).
+	 */
+	dispatched?: boolean;
 }
 
 export interface EnvelopeValidationMeta {