@checkstack/backend-api 0.15.3 → 0.17.0

package/CHANGELOG.md

@@ -1,5 +1,117 @@
 # @checkstack/backend-api
 
+## 0.17.0
+
+### Minor Changes
+
+- f23f3c9: Add `correlationMiddleware` to `@checkstack/backend-api` and apply it
+  to every plugin/core router so each request carries a stable
+  `x-correlation-id` (read from the inbound header, or freshly minted
+  via `crypto.randomUUID()` when absent) and an auto-injected child
+  logger bound with `{ correlationId, pluginId, userId? }`. The ID is
+  echoed back on the response header so the caller can correlate their
+  client-side trace to the server logs.
+
+  The `Logger` interface in `@checkstack/backend-api` now formally
+  documents the structured-metadata convention (`logger.info("msg",
+{ ...meta })`) alongside the long-standing varargs shape. Winston's
+  splat handling already routes both shapes through the same vararg
+  slot, so existing call sites are unaffected. A new optional
+  `Logger.child(meta)` method captures the metadata-binding contract the
+  new middleware relies on; production loggers always implement it,
+  minimal test mocks may omit it (the middleware falls back gracefully).
+
+  `RpcContext` grew two optional `Headers` bags, `requestHeaders` and
+  `responseHeaders`, populated by the outer Hono `/api/*` and `/rest/*`
+  handlers in `@checkstack/backend`. They are write-through observation
+  points for middleware; an `RpcContext` constructed without them (S2S
+  clients, tests) keeps working — the echo is a silent no-op and the ID
+  is still bound onto the child logger for server-side correlation.
+
+  The scaffolding template in `@checkstack/scripts` was updated so any
+  new plugin generated via `bun run create` wires the middleware in the
+  expected `.use(correlationMiddleware).use(autoAuthMiddleware)` order
+  out of the box.
+
+### Patch Changes
+
+- Updated dependencies [f23f3c9]
+- Updated dependencies [f23f3c9]
+  - @checkstack/common@0.11.0
+  - @checkstack/healthcheck-common@1.1.2
+  - @checkstack/signal-common@0.2.4
+  - @checkstack/cache-api@0.3.4
+  - @checkstack/queue-api@0.3.4
+
+## 0.16.0
+
+### Minor Changes
+
+- a06b899: Dead-code audit cleanup and a small platform of shared notification helpers.
+
+  **Removed (dead code)**
+
+  - `core/backend/src/plugin-manager/deregistration-guard.ts` deleted. The exported `assertCanDeregister()` was never called and was a less-complete version of the dependents+isUninstallable checks already done inline by `previewUninstallOriginator` / `uninstallOriginator` in `plugin-manager-orchestrator.ts`.
+  - `createMockQueueFactory` deprecated alias removed from `@checkstack/test-utils-backend`. Use `createMockQueueManager` directly.
+
+  **New shared helpers**
+
+  - `@checkstack/backend-api` now exports `requestTimeoutMs()` — a Zod field builder for outbound HTTP request timeouts (1s..60s, default 10s). Replaces hand-rolled `configNumber({}).min(1000).max(60_000).default(10_000)` in `integration-webhook-backend`, `integration-script-backend`, and `healthcheck-script-backend`'s inline collector.
+  - `@checkstack/notification-common` now exports `SubjectStatusSchema` / `SubjectStatus`, mirroring the existing `ImportanceSchema`.
+  - `@checkstack/notification-backend` now exports:
+    - `SUBJECT_STATUS_EMOJI` / `IMPORTANCE_EMOJI` — the shared status / importance emoji maps that Discord, Slack, Teams, Webex and Telegram previously each redefined inline.
+    - `postJson(opts)` — a timeout-bounded `fetch` wrapper that handles non-2xx logging and error mapping for webhook-style POSTs. Returns `{ ok: true, response } | { ok: false, error }`.
+
+  **Migrated to shared helpers**
+
+  - Discord, Slack, Gotify, Pushover notification backends now use `postJson`. Outer try/catch + per-plugin error mapping deleted (~140 LOC).
+  - Discord, Slack, Teams, Telegram, Webex notification backends now use `IMPORTANCE_EMOJI`. Discord, Slack, Teams use `SUBJECT_STATUS_EMOJI`.
+  - Teams, Webex, Backstage, Telegram kept their inline fetch/Bot logic: their error strings surface server response bodies to operators, or the transport isn't raw `fetch` (Telegram uses `grammy`'s `Bot`).
+
+  **API surface tightening**
+
+  - Per-plugin test-only re-exports in 6 notification backends (Pushover, Gotify, Backstage, Slack, Discord, Teams) and the `CertificateInfo` interface in `healthcheck-tls-backend/strategy.ts` are now JSDoc-tagged `@internal`. No behaviour change; signals that downstream consumers must not depend on them.
+
+- a06b899: Extract shared `EsmScriptRunner` + `ShellScriptRunner` utilities, fix HIGH-severity privilege amplification in the integration TS provider, and harden the integration shell setupGuide example.
+
+  **SECURITY FIX (HIGH)**
+
+  The integration TS provider (`@checkstack/integration-script-backend` → `scriptProvider`) previously executed user scripts via `new Function(script)` in the satellite's main V8 isolate. A user with `integrationAccess.manage` could read `globalThis.process.env` directly (`DATABASE_URL`, `JWT_SECRET`, queue credentials, signing keys, …) and exfiltrate them through `result.id` — which round-trips into `delivery_logs.externalId` and is readable via the `getDeliveryLog` ORPC procedure. The same permission grants no legitimate API to those secrets; this was a privilege amplification.
+
+  The provider now runs user scripts in a fresh Bun subprocess (matching the healthcheck inline-script collector model). The subprocess receives only a curated `SAFE_ENV_VARS` whitelist (`PATH`, `HOME`, `USER`, `LANG`, `LC_ALL`, `LC_CTYPE`, `TZ`, `TMPDIR`, `HOSTNAME`, `SHELL`) — backend secrets are no longer visible to user code. Filesystem reads, network calls (`fetch`), and the rest of the Node/Bun standard library continue to work, just in an isolated process.
+
+  **BREAKING CHANGE (`@checkstack/integration-script-backend`)**
+
+  User scripts can no longer read the satellite process's environment variables (`process.env.DATABASE_URL` etc. return `undefined`). Scripts that legitimately need configuration should accept it via the provider's `script` field input, not by introspecting the host environment. The full Node/Bun stdlib remains available; only the env scrub is new.
+
+  **REFACTOR — new shared utilities in `@checkstack/backend-api`**
+
+  Both the healthcheck and integration plugins had near-identical inline implementations of "run a user script in a subprocess sandbox" (ESM path) and "run a user shell script through `sh -c`" (shell path). These are now canonical, single-source-of-truth utilities:
+
+  - **`defaultEsmScriptRunner.run({ script, context, timeoutMs, helperModuleName?, helperFunctionName? })`** — writes the user module to a fresh `mkdtemp` directory along with a generated runner module, spawns a Bun subprocess with `pickSafeEnv()`, parses the result back through a UUID-tagged stderr marker, and tears everything down in `finally` (`clearTimeout` + `proc.kill()` + recursive `rm`). The optional `helperModuleName` / `helperFunctionName` pair drops a sibling `_helpers.mjs` file and rewrites `import { <fn> } from "<module>"` to point at it (this is the trick that makes `@checkstack/healthcheck` / `@checkstack/integration` resolve at runtime even though they're not real npm packages).
+  - **`defaultShellScriptRunner.run({ script, timeoutMs, cwd?, env? })`** — invokes `sh -c <script>` via `Bun.spawn` with `SAFE_ENV_VARS` (user-supplied `env` merged on top), `Promise.race` timeout with `proc.kill()` on expiry, and the same `clearTimeout` + `proc.kill()` cleanup in `finally`.
+
+  Both runners expose `EsmScriptRunner` / `ShellScriptRunner` interfaces so tests can inject mocks without touching the spawn path. The four call sites (`plugins/healthcheck-script-backend/src/inline-script-collector.ts`, `strategy.ts` and `plugins/integration-script-backend/src/provider.ts`, `shell-provider.ts`) collapse from full inline implementations to ~8-line adapters.
+
+  **FIXES**
+
+  - Integration shell provider's `setupGuide` example replaced the unsafe `curl -d "{\"title\": \"$PAYLOAD_TITLE\"}"` JSON interpolation with a `jq -n --arg title "$PAYLOAD_TITLE" '{title: $title}'` pattern. The previous example demonstrated a shell-injection vulnerability whenever event payload values contained shell-special or JSON-special characters (which they can, since payloads come from other plugins / events / GitOps reconciles).
+  - The shared shell runner adds `clearTimeout` + idempotent `proc?.kill()` in `finally`, fixing a leaked event-loop timer in the integration shell provider's previous inline implementation.
+
+  **TESTS**
+
+  - New `core/backend-api/src/esm-script-runner.test.ts` covering `normaliseUserScript` + `rewriteHelperImports` across both healthcheck and integration helper-module names, including regex-metacharacter escape coverage.
+  - The plugin-local `inline-script-normaliser.test.ts` was deleted; the same coverage (plus more) lives at the canonical location with the utility.
+  - Integration TS provider console-logging tests updated: in the subprocess model, `console.warn` and `console.error` both write to stderr (Bun matches Node), so the provider forwards every stderr line to `logger.error`. `console.log({…})` uses Bun's native `util.inspect` format rather than `JSON.stringify`, so the JSON-logging test now asserts on substring presence instead of strict serialisation.
+
+  2047 tests pass, lint + typecheck clean.
+
+### Patch Changes
+
+- @checkstack/cache-api@0.3.3
+- @checkstack/queue-api@0.3.3
+- @checkstack/healthcheck-common@1.1.1
+
 ## 0.15.3
 
 ### Patch Changes

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@checkstack/backend-api",
-  "version": "0.15.3",
+  "version": "0.17.0",
   "license": "Elastic-2.0",
   "type": "module",
   "main": "./src/index.ts",
@@ -11,9 +11,9 @@
   },
   "dependencies": {
     "@checkstack/common": "0.10.0",
-    "@checkstack/healthcheck-common": "1.1.0",
-    "@checkstack/cache-api": "0.3.1",
-    "@checkstack/queue-api": "0.3.1",
+    "@checkstack/healthcheck-common": "1.1.1",
+    "@checkstack/cache-api": "0.3.3",
+    "@checkstack/queue-api": "0.3.3",
     "@checkstack/signal-common": "0.2.3",
     "@orpc/client": "^1.13.14",
     "@orpc/contract": "^1.13.14",

package/src/base-strategy-config.ts

@@ -24,3 +24,22 @@ export const baseStrategyConfigSchema = z.object({
  * Base config type that all strategy configs must satisfy.
  */
 export type BaseStrategyConfig = z.infer<typeof baseStrategyConfigSchema>;
+
+/**
+ * Shared timeout field for outbound HTTP requests (webhooks, integration
+ * deliveries, inline scripts). Standardises bounds and default across plugins
+ * so operators see a consistent UX in the config UI.
+ *
+ * Bounds: `1_000ms` to `60_000ms`, default `10_000ms`.
+ *
+ * @example
+ * ```typescript
+ * const provider = z.object({
+ *   url: configString({}).url(),
+ *   timeout: requestTimeoutMs().describe("Request timeout in milliseconds"),
+ * });
+ * ```
+ */
+export function requestTimeoutMs() {
+  return configNumber({}).min(1000).max(60_000).default(10_000);
+}

package/src/correlation-middleware.test.ts

@@ -0,0 +1,191 @@
+import { describe, expect, it, mock, type Mock } from "bun:test";
+import { call, implement } from "@orpc/server";
+import { z } from "zod";
+import { proc } from "@checkstack/common";
+import {
+  autoAuthMiddleware,
+  CORRELATION_ID_HEADER,
+  correlationMiddleware,
+  RpcContext,
+} from "./rpc";
+import { createMockRpcContext } from "./test-utils";
+import { Logger } from "./types";
+
+const echoLoggerProcedure = proc({
+  userType: "anonymous",
+  operationType: "query",
+  access: [],
+}).output(
+  z.object({
+    correlationIdFromMeta: z.string().optional(),
+    pluginIdFromMeta: z.string().optional(),
+    userIdFromMeta: z.string().optional(),
+  }),
+);
+
+const testContract = {
+  echo: echoLoggerProcedure,
+};
+
+type LastChildMeta = Record<string, unknown> | undefined;
+
+function buildRouterCapturingChildMeta(captureRef: { value: LastChildMeta }) {
+  const os = implement(testContract)
+    .$context<RpcContext>()
+    .use(correlationMiddleware)
+    .use(autoAuthMiddleware);
+
+  return os.router({
+    echo: os.echo.handler(({ context }) => {
+      // The capture happens in the child() mock on the test's logger.
+      // We just return the metadata that the middleware should have bound.
+      void context.logger;
+      return {
+        correlationIdFromMeta: captureRef.value?.correlationId as
+          | string
+          | undefined,
+        pluginIdFromMeta: captureRef.value?.pluginId as string | undefined,
+        userIdFromMeta: captureRef.value?.userId as string | undefined,
+      };
+    }),
+  });
+}
+
+function buildContextWithCapture(
+  overrides: Partial<RpcContext> = {},
+): {
+  context: RpcContext;
+  capture: { value: LastChildMeta };
+  childMock: Mock<(meta: Record<string, unknown>) => Logger>;
+} {
+  const capture: { value: LastChildMeta } = { value: undefined };
+  const childMock = mock((meta: Record<string, unknown>): Logger => {
+    capture.value = meta;
+    return {
+      info: mock(),
+      error: mock(),
+      warn: mock(),
+      debug: mock(),
+    };
+  });
+  const baseContext = createMockRpcContext(overrides);
+  // Replace the logger so child() captures the bound metadata.
+  baseContext.logger = {
+    info: mock(),
+    error: mock(),
+    warn: mock(),
+    debug: mock(),
+    child: childMock,
+  };
+  return { context: baseContext, capture, childMock };
+}
+
+const UUID_V4_REGEX =
+  /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i;
+
+describe("correlationMiddleware", () => {
+  it("generates a UUID v4 when no x-correlation-id header is present", async () => {
+    const { context, capture } = buildContextWithCapture();
+    const router = buildRouterCapturingChildMeta(capture);
+
+    await call(router.echo, {}, { context });
+
+    expect(typeof capture.value?.correlationId).toBe("string");
+    expect(capture.value?.correlationId).toMatch(UUID_V4_REGEX);
+  });
+
+  it("uses the incoming x-correlation-id header when present", async () => {
+    const incoming = "11111111-2222-4333-8444-555555555555";
+    const { context, capture } = buildContextWithCapture({
+      requestHeaders: new Headers({ [CORRELATION_ID_HEADER]: incoming }),
+    });
+    const router = buildRouterCapturingChildMeta(capture);
+
+    await call(router.echo, {}, { context });
+
+    expect(capture.value?.correlationId).toBe(incoming);
+  });
+
+  it("ignores an empty x-correlation-id header and generates a fresh ID", async () => {
+    const { context, capture } = buildContextWithCapture({
+      requestHeaders: new Headers({ [CORRELATION_ID_HEADER]: "" }),
+    });
+    const router = buildRouterCapturingChildMeta(capture);
+
+    await call(router.echo, {}, { context });
+
+    expect(capture.value?.correlationId).toMatch(UUID_V4_REGEX);
+  });
+
+  it("echoes the correlation ID on responseHeaders when available", async () => {
+    const responseHeaders = new Headers();
+    const { context, capture } = buildContextWithCapture({
+      responseHeaders,
+    });
+    const router = buildRouterCapturingChildMeta(capture);
+
+    await call(router.echo, {}, { context });
+
+    const echoed = responseHeaders.get(CORRELATION_ID_HEADER);
+    expect(echoed).not.toBeNull();
+    expect(echoed).toBe(capture.value?.correlationId as string);
+  });
+
+  it("binds pluginId from context.pluginMetadata onto the child logger", async () => {
+    const { context, capture } = buildContextWithCapture({
+      pluginMetadata: { pluginId: "fancy-plugin" },
+    });
+    const router = buildRouterCapturingChildMeta(capture);
+
+    await call(router.echo, {}, { context });
+
+    expect(capture.value?.pluginId).toBe("fancy-plugin");
+  });
+
+  it("binds userId when the user has an id field", async () => {
+    const { context, capture } = buildContextWithCapture({
+      user: { type: "user", id: "user-abc", accessRules: ["*"] },
+    });
+    const router = buildRouterCapturingChildMeta(capture);
+
+    await call(router.echo, {}, { context });
+
+    expect(capture.value?.userId).toBe("user-abc");
+  });
+
+  it("omits userId when the user is a service (no id field)", async () => {
+    const { context, capture } = buildContextWithCapture({
+      user: { type: "service", pluginId: "internal-svc" },
+    });
+    const router = buildRouterCapturingChildMeta(capture);
+
+    await call(router.echo, {}, { context });
+
+    expect("userId" in (capture.value ?? {})).toBe(false);
+  });
+
+  it("falls back to the base logger when child() is not implemented", async () => {
+    // Reproduce a minimal mock logger without `.child` (e.g. a test mock
+    // that hasn't been updated). The middleware must not throw and the
+    // request must still complete.
+    const context = createMockRpcContext();
+    context.logger = {
+      info: mock(),
+      error: mock(),
+      warn: mock(),
+      debug: mock(),
+      // intentionally no child
+    };
+
+    const os = implement(testContract)
+      .$context<RpcContext>()
+      .use(correlationMiddleware)
+      .use(autoAuthMiddleware);
+    const router = os.router({
+      echo: os.echo.handler(() => ({})),
+    });
+
+    const result = await call(router.echo, {}, { context });
+    expect(result).toEqual({});
+  });
+});

package/src/esm-script-runner.test.ts

@@ -0,0 +1,169 @@
+import { describe, expect, it } from "bun:test";
+import {
+  normaliseUserScript,
+  rewriteHelperImports,
+} from "./esm-script-runner";
+
+/**
+ * The shared ESM-script runner applies two text transforms to user
+ * source before writing it to a temp `.mjs` file:
+ *
+ *  1. `normaliseUserScript` — wraps legacy `return X;` style scripts in
+ *     an async IIFE so they're valid as a module's default export,
+ *     while leaving real ESM modules (those with top-level
+ *     `import`/`export`) untouched.
+ *
+ *  2. `rewriteHelperImports` — rewrites `from "<helperModuleName>"`
+ *     to point at a runtime helper file written next to the user
+ *     script.
+ *
+ * Both are easy to regress (a regex tweak that gets greedier than
+ * intended, an edge case with comments, …) and both directly affect
+ * whether user scripts run at all. Keep this suite tight.
+ *
+ * These were previously tested under
+ * `plugins/healthcheck-script-backend/src/inline-script-normaliser.test.ts`;
+ * the utilities now live in `@checkstack/backend-api` and are shared by
+ * both healthcheck (inline-script-collector) and integration
+ * (script-provider).
+ */
+
+describe("normaliseUserScript", () => {
+  it("wraps a bare `return` body in an async IIFE default export", () => {
+    const out = normaliseUserScript("return { success: true };");
+    expect(out).toContain("export default await (async () => {");
+    expect(out).toContain("return { success: true };");
+    expect(out).toContain("})();");
+  });
+
+  it("wraps a no-return body (side-effect script) the same way", () => {
+    const out = normaliseUserScript("console.log('hi');");
+    expect(out).toContain("export default await (async () => {");
+    expect(out).toContain("console.log('hi');");
+  });
+
+  it("leaves a real ESM module (with top-level `import`) untouched", () => {
+    const src = `import { loadavg } from "node:os";\nexport default { success: true };`;
+    expect(normaliseUserScript(src)).toBe(src);
+  });
+
+  it("leaves a module with only `export default` (no import) untouched", () => {
+    const src = `export default { success: true };`;
+    expect(normaliseUserScript(src)).toBe(src);
+  });
+
+  it("leaves a script with `export` after leading whitespace untouched", () => {
+    // Top-level matters regardless of leading whitespace — the regex is
+    // anchored with `^\s*` per-line via the `m` flag.
+    const src = `   export default 42;`;
+    expect(normaliseUserScript(src)).toBe(src);
+  });
+
+  it("treats indented `return`-style scripts as legacy (wraps them)", () => {
+    // Per current behaviour: only top-of-line `export`/`import` qualifies
+    // as ESM. Anything else gets wrapped. This is intentional — a script
+    // that contains the substring "import" inside a comment shouldn't be
+    // misclassified as ESM and have its `return` left dangling.
+    const src = `  // imports are great\n  return true;`;
+    expect(normaliseUserScript(src)).toContain("export default await");
+  });
+
+  it("appends a trailing newline before `})` so a trailing `//` comment doesn't swallow the brace", () => {
+    const out = normaliseUserScript("return true; // trailing comment");
+    // Confirm the wrapper structure didn't get clobbered by the comment.
+    expect(out).toMatch(/return true; \/\/ trailing comment\s*\n\}\)\(\)/);
+  });
+});
+
+describe("rewriteHelperImports", () => {
+  const HELPER_URL = "file:///tmp/checkstack-script-abc/_helpers.mjs";
+
+  it("rewrites a named import from the helper module", () => {
+    const out = rewriteHelperImports({
+      userScript: `import { defineHealthCheck } from "@checkstack/healthcheck";`,
+      helperModuleName: "@checkstack/healthcheck",
+      helperUrl: HELPER_URL,
+    });
+    expect(out).toBe(`import { defineHealthCheck } from "${HELPER_URL}";`);
+  });
+
+  it("works with single-quoted import specs too", () => {
+    const out = rewriteHelperImports({
+      userScript: `import { defineHealthCheck } from '@checkstack/healthcheck';`,
+      helperModuleName: "@checkstack/healthcheck",
+      helperUrl: HELPER_URL,
+    });
+    expect(out).toBe(`import { defineHealthCheck } from "${HELPER_URL}";`);
+  });
+
+  it("rewrites a side-effect import too", () => {
+    const out = rewriteHelperImports({
+      userScript: `import "@checkstack/healthcheck";`,
+      helperModuleName: "@checkstack/healthcheck",
+      helperUrl: HELPER_URL,
+    });
+    expect(out).toBe(`import "${HELPER_URL}";`);
+  });
+
+  it("leaves other imports alone", () => {
+    const src = `import { loadavg } from "node:os";\nimport fs from "node:fs/promises";`;
+    expect(
+      rewriteHelperImports({
+        userScript: src,
+        helperModuleName: "@checkstack/healthcheck",
+        helperUrl: HELPER_URL,
+      }),
+    ).toBe(src);
+  });
+
+  it("rewrites multiple occurrences", () => {
+    const src = `
+      import { defineHealthCheck } from "@checkstack/healthcheck";
+      import type { HealthCheckScriptResult } from "@checkstack/healthcheck";
+    `;
+    const out = rewriteHelperImports({
+      userScript: src,
+      helperModuleName: "@checkstack/healthcheck",
+      helperUrl: HELPER_URL,
+    });
+    expect(out.match(/@checkstack\/healthcheck/g)).toBeNull();
+    expect([...out.matchAll(new RegExp(HELPER_URL, "g"))].length).toBe(2);
+  });
+
+  it("doesn't rewrite the package name if it appears in a string literal", () => {
+    // Conservative regex: it only matches the spec position of an import
+    // statement (`from "..."` / `import "..."`). A string containing the
+    // package name elsewhere must be left alone.
+    const src = `console.log("Look up @checkstack/healthcheck on npm");`;
+    expect(
+      rewriteHelperImports({
+        userScript: src,
+        helperModuleName: "@checkstack/healthcheck",
+        helperUrl: HELPER_URL,
+      }),
+    ).toBe(src);
+  });
+
+  it("rewrites the integration helper module too (regex is parameterised)", () => {
+    // The shared runner is used by both healthcheck and integration
+    // plugins; the regex is built from the supplied `helperModuleName`.
+    // Sanity check that the integration variant works.
+    const out = rewriteHelperImports({
+      userScript: `import { defineIntegration } from "@checkstack/integration";`,
+      helperModuleName: "@checkstack/integration",
+      helperUrl: HELPER_URL,
+    });
+    expect(out).toBe(`import { defineIntegration } from "${HELPER_URL}";`);
+  });
+
+  it("escapes regex metacharacters in the helper module name", () => {
+    // Defence-in-depth: if someone ever passes a name with special
+    // chars, we don't want it interpreted as a regex.
+    const out = rewriteHelperImports({
+      userScript: `import x from "weird.pkg+name";`,
+      helperModuleName: "weird.pkg+name",
+      helperUrl: HELPER_URL,
+    });
+    expect(out).toBe(`import x from "${HELPER_URL}";`);
+  });
+});