@checkstack/healthcheck-script-backend 0.5.0 → 0.6.0

package/CHANGELOG.md

@@ -1,5 +1,123 @@
 # @checkstack/healthcheck-script-backend
 
+## 0.6.0
+
+### Minor Changes
+
+- 270ef29: Extend in-UI script testing to health-check collectors, and add
+  load-from-run replay for automation script tests.
+
+  - Health-check collectors: a new `testCollectorScript` RPC runs the
+    inline-script (TypeScript) collector and the shell `script` collector
+    against an editable, auto-seeded sample context using the same
+    sandboxed runner the real collector uses. Surfaces beneath the
+    collector script fields in the collector editor (both marked
+    `x-script-testable`). Gated by `healthcheck.configuration.manage`.
+  - Automation replay: a new `getRunScopeForReplay` RPC reconstructs an
+    editable test context from a real run (trigger + persisted artifacts,
+    plus the durable scope snapshot when the run is still in-flight), and
+    the script-test panel gains a "Load from run" picker that seeds the
+    sample context from a past run.
+
+  Note: health-check executions do not persist the script / config /
+  check / system that produced a result, so there is no health-check
+  replay - auto-seed is the only context source for collector tests. This
+  is by design; see the feature plan.
+
+- 270ef29: Activate npm packages in script execution: thread the managed
+  `resolutionRoot` into every user-script call site so an allowlisted package
+  can actually be `import`ed.
+
+  - `@checkstack/backend-api`: the ESM runner now always writes a per-run
+    `bunfig.toml` with `[install] auto = "disable"` and runs with that dir as
+    CWD. Without this Bun silently auto-installs any imported package from the
+    registry (verified), defeating the allowlist; with it, imports resolve
+    only against the reconciled `current/node_modules` (when a `resolutionRoot`
+    is set) and otherwise fail fast.
+  - `@checkstack/script-packages-backend`: `resolveResolutionRoot` /
+    `resolveResolutionRootFromStore` / `resolveResolutionRootForHost` decide a
+    host's resolution-root status (`none` / `ready` / `notReady`) from the
+    local `<store>/current`.
+  - `run_script` (integration-script-backend), the inline-script collector
+    (healthcheck-script-backend, core + satellite), and the in-UI `testScript`
+    / `testCollectorScript` endpoints all resolve the root per run and pass it
+    to the runner; `run_script` surfaces a clear "npm packages not ready"
+    error when configured-but-unsynced. Shell paths are unaffected (no module
+    resolution).
+
+  An opt-in end-to-end test (`CHECKSTACK_E2E_NETWORK=1`) proves an allowlisted
+  package imports successfully through the real `run_script` action execute
+  path, with non-network degradation tests running always.
+
+  BREAKING CHANGES: `@checkstack/backend-api`'s `defaultEsmScriptRunner` now
+  always disables Bun auto-install for the user subprocess. A script that
+  previously relied on Bun silently fetching an un-vendored package from the
+  registry at import time will now fail to resolve it. This is intentional -
+  package availability is governed by the admin allowlist - but any caller
+  depending on the old implicit auto-install behavior must add the package to
+  the allowlist instead. The new `EsmScriptRunOptions.resolutionRoot` field is
+  optional and additive (defaults to today's `os.tmpdir()` behavior when
+  unset), so the runner API itself is source-compatible.
+
+- 270ef29: Secrets platform Phase 3: just-in-time secret delivery to satellites + source-side masking, and central-execution injection for healthcheck collectors.
+
+  - New satellite WS messages `request_run_secrets` / `run_secrets`: just
+    before a satellite runs a collector that declares a `secretEnv`, it asks
+    core for that collector's resolved env; core resolves ONLY the secrets the
+    collector's OWN persisted assignment declares (least-privilege — the
+    satellite cannot choose) and replies with the env map (or a clear error).
+    The satellite injects it memory-only for the run and drops it on
+    completion. Secrets never ride the persisted assignment and never touch
+    disk.
+  - Source-side masking: the satellite runs `maskSecrets` over the collector's
+    stdout/stderr/result/error using the run's delivered values BEFORE the
+    result leaves the satellite (defense in depth).
+  - `CollectorStrategy.execute` gains an optional `secretEnv`. The
+    inline-script and shell collectors inject it into the runner
+    (`process.env` / `$VAR`) and mask the values out of their output.
+  - Healthcheck collectors running centrally (the queue executor) also resolve
+    - inject `secretEnv` via `secretResolverRef`, closing the gap where a
+      centrally-run secretEnv collector got no secrets. A missing required
+      secret fails the run clearly in all paths.
+
+### Patch Changes
+
+- Updated dependencies [270ef29]
+- Updated dependencies [270ef29]
+- Updated dependencies [270ef29]
+- Updated dependencies [270ef29]
+- Updated dependencies [270ef29]
+- Updated dependencies [270ef29]
+- Updated dependencies [b995afb]
+- Updated dependencies [b995afb]
+- Updated dependencies [b995afb]
+- Updated dependencies [270ef29]
+- Updated dependencies [270ef29]
+- Updated dependencies [270ef29]
+- Updated dependencies [270ef29]
+- Updated dependencies [270ef29]
+- Updated dependencies [b995afb]
+- Updated dependencies [b995afb]
+- Updated dependencies [270ef29]
+- Updated dependencies [270ef29]
+- Updated dependencies [270ef29]
+- Updated dependencies [b995afb]
+- Updated dependencies [270ef29]
+- Updated dependencies [b995afb]
+- Updated dependencies [270ef29]
+- Updated dependencies [270ef29]
+- Updated dependencies [b995afb]
+- Updated dependencies [270ef29]
+- Updated dependencies [270ef29]
+- Updated dependencies [270ef29]
+- Updated dependencies [270ef29]
+- Updated dependencies [270ef29]
+- Updated dependencies [b995afb]
+  - @checkstack/backend-api@0.19.0
+  - @checkstack/healthcheck-common@1.4.0
+  - @checkstack/script-packages-backend@0.2.0
+  - @checkstack/secrets-common@0.1.0
+
 ## 0.5.0
 
 ### Minor Changes

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@checkstack/healthcheck-script-backend",
-  "version": "0.5.0",
+  "version": "0.6.0",
   "type": "module",
   "main": "src/index.ts",
   "checkstack": {
@@ -14,15 +14,17 @@
     "pack": "bunx @checkstack/scripts plugin-pack"
   },
   "dependencies": {
-    "@checkstack/backend-api": "0.17.1",
-    "@checkstack/common": "0.11.0",
-    "@checkstack/healthcheck-common": "1.2.0"
+    "@checkstack/backend-api": "0.18.0",
+    "@checkstack/common": "0.12.0",
+    "@checkstack/healthcheck-common": "1.3.0",
+    "@checkstack/script-packages-backend": "0.1.0",
+    "@checkstack/secrets-common": "0.0.1"
   },
   "devDependencies": {
     "@types/bun": "^1.0.0",
     "typescript": "^5.0.0",
     "@checkstack/tsconfig": "0.0.7",
-    "@checkstack/scripts": "0.3.3"
+    "@checkstack/scripts": "0.3.4"
   },
   "description": "Checkstack healthcheck-script-backend plugin",
   "author": {

package/src/execute-collector.test.ts

@@ -294,3 +294,71 @@ describe("ExecuteCollector", () => {
     });
   });
 });
+
+describe("ExecuteCollector secret env injection + source-side masking", () => {
+  // Capture the env the collector passes to the transport client. Typed
+  // so we can assert without indexing the loosely-typed mock.calls tuple.
+  const makeRecordingClient = (
+    response: { stdout: string; stderr: string },
+  ): {
+    client: ScriptTransportClient;
+    getEnv: () => Record<string, string> | undefined;
+  } => {
+    let capturedEnv: Record<string, string> | undefined;
+    return {
+      getEnv: () => capturedEnv,
+      client: {
+        exec: async (input) => {
+          capturedEnv = input.env;
+          return {
+            exitCode: 0,
+            stdout: response.stdout,
+            stderr: response.stderr,
+            timedOut: false,
+          };
+        },
+      },
+    };
+  };
+
+  it("injects the delivered secretEnv into the shell exec and masks it from output", async () => {
+    const { client, getEnv } = makeRecordingClient({
+      stdout: "value=sh-secret-987",
+      stderr: "warn sh-secret-987",
+    });
+    const collector = new ExecuteCollector();
+
+    const result = await collector.execute({
+      config: {
+        script: "echo value=$TOKEN",
+        timeout: 5000,
+        secretEnv: { TOKEN: "${{ secrets.tok }}" },
+      },
+      client,
+      pluginId: "test",
+      secretEnv: { TOKEN: "sh-secret-987" },
+    });
+
+    // The exec received the injected env.
+    expect(getEnv()?.TOKEN).toBe("sh-secret-987");
+    // The secret is redacted from stdout/stderr (source-side).
+    expect(result.result.stdout).toBe("value=****");
+    expect(result.result.stderr).toBe("warn ****");
+    expect(JSON.stringify(result)).not.toContain("sh-secret-987");
+  });
+
+  it("does not inject or mask when no secretEnv is delivered", async () => {
+    const { client, getEnv } = makeRecordingClient({
+      stdout: "no secrets",
+      stderr: "",
+    });
+    const collector = new ExecuteCollector();
+    const result = await collector.execute({
+      config: { script: "echo hi", timeout: 5000 },
+      client,
+      pluginId: "test",
+    });
+    expect(getEnv()?.TOKEN).toBeUndefined();
+    expect(result.result.stdout).toBe("no secrets");
+  });
+});

package/src/execute-collector.ts

@@ -2,6 +2,7 @@ import {
   Versioned,
   z,
   configString,
+  withConfigMeta,
   type HealthCheckRunForAggregation,
   type CollectorResult,
   type CollectorStrategy,
@@ -21,6 +22,10 @@ import {
 } from "@checkstack/healthcheck-common";
 import { pluginMetadata } from "./plugin-metadata";
 import type { ScriptTransportClient } from "./transport-client";
+import {
+  secretEnvMappingSchema,
+  maskScriptRunOutput,
+} from "@checkstack/secrets-common";
 
 // ============================================================================
 // RUN-CONTEXT ENV INJECTION
@@ -82,6 +87,7 @@ function shellQuote(arg: string): string {
 const executeConfigSchemaV2 = z.object({
   script: configString({
     "x-editor-types": ["shell"],
+    "x-script-testable": true,
   }).describe(
     "Shell script source. Executed via `sh -c`, so pipes, redirects, `if`/`for`/`while`, variable expansion, command substitution etc. all work. Exit non-zero to fail the check.",
   ),
@@ -95,6 +101,11 @@ const executeConfigSchemaV2 = z.object({
     .describe(
       "Extra environment variables to expose to the script. Merged on top of the safe-vars whitelist (PATH, HOME, ...).",
     ),
+  secretEnv: withConfigMeta(secretEnvMappingSchema, { "x-secret-env": true })
+    .optional()
+    .describe(
+      'Secret → env mapping, e.g. { "API_TOKEN": "${{ secrets.token }}" }. Only the named secrets are resolved and injected for this run (read via process.env.API_TOKEN / $API_TOKEN); on a satellite they are delivered just-in-time over the encrypted channel, never persisted. Values are masked out of the collector output.',
+    ),
   timeout: z
     .number()
     .int()
@@ -243,20 +254,25 @@ export class ExecuteCollector implements CollectorStrategy<
     config,
     client,
     runContext,
+    secretEnv,
   }: {
     config: ExecuteConfig;
     client: ScriptTransportClient;
     pluginId: string;
     runContext?: CollectorRunContext;
+    secretEnv?: Record<string, string>;
   }): Promise<CollectorResult<ExecuteResult>> {
     const startTime = Date.now();
 
-    // Merge run-context metadata under the user-supplied env so a user
-    // `config.env` key always wins on collision (matches the runner's
-    // existing merge order against the safe-vars whitelist).
-    const env = runContext
-      ? { ...runContextEnv(runContext), ...config.env }
-      : config.env;
+    // Merge run-context metadata under the user-supplied env, then the
+    // resolved secret env on top (a declared secret env name wins, matching
+    // the central action's layering). User `config.env` still wins over the
+    // run-context vars.
+    const env = {
+      ...(runContext ? runContextEnv(runContext) : {}),
+      ...config.env,
+      ...secretEnv,
+    };
 
     const response = await client.exec({
       script: config.script,
@@ -265,20 +281,33 @@ export class ExecuteCollector implements CollectorStrategy<
       timeout: config.timeout,
     });
 
+    // Source-side masking: redact the delivered secret values from the
+    // captured output BEFORE it leaves the satellite (core masks again on
+    // receipt). A script echoing a secret it was given is masked here.
+    const maskValues = Object.values(secretEnv ?? {});
+    const masked = maskScriptRunOutput({
+      output: {
+        stdout: response.stdout,
+        stderr: response.stderr,
+        error: response.error,
+      },
+      values: maskValues,
+    });
+
     const executionTimeMs = Date.now() - startTime;
     const success = response.exitCode === 0 && !response.timedOut;
 
     return {
       result: {
         exitCode: response.exitCode,
-        stdout: response.stdout,
-        stderr: response.stderr,
+        stdout: masked.stdout,
+        stderr: masked.stderr,
         executionTimeMs,
         success,
         timedOut: response.timedOut,
       },
       error:
-        response.error ??
+        masked.error ??
         (success ? undefined : `Exit code: ${response.exitCode}`),
     };
   }

package/src/index.ts

@@ -1,8 +1,15 @@
 import { createBackendPlugin, coreServices } from "@checkstack/backend-api";
+import {
+  resolveResolutionRootFromStore,
+  resolveScriptPackagesDir,
+} from "@checkstack/script-packages-backend";
 import { ScriptHealthCheckStrategy } from "./strategy";
 import { pluginMetadata } from "./plugin-metadata";
 import { ExecuteCollector } from "./execute-collector";
-import { InlineScriptCollector } from "./inline-script-collector";
+import {
+  InlineScriptCollector,
+  defaultInlineScriptExecutor,
+} from "./inline-script-collector";
 
 export default createBackendPlugin({
   metadata: pluginMetadata,
@@ -18,7 +25,17 @@ export default createBackendPlugin({
         const strategy = new ScriptHealthCheckStrategy();
         healthCheckRegistry.register(strategy);
         collectorRegistry.register(new ExecuteCollector());
-        collectorRegistry.register(new InlineScriptCollector());
+        // The inline TS collector resolves the managed npm-package tree from
+        // the local store - identical on core and satellites (same
+        // `<dataDir>/script-packages/current` convention). No RPC needed, so
+        // it works in the satellite runtime too. Execution safety is
+        // guaranteed by the runner (auto-install disabled); this just points
+        // it at the synced tree when present.
+        collectorRegistry.register(
+          new InlineScriptCollector(defaultInlineScriptExecutor, () =>
+            resolveResolutionRootFromStore(resolveScriptPackagesDir()),
+          ),
+        );
       },
     });
   },

package/src/inline-script-collector.test.ts

@@ -2,6 +2,7 @@ import { describe, it, expect } from "bun:test";
 import {
   InlineScriptCollector,
   type InlineScriptConfig,
+  type InlineScriptExecutor,
 } from "./inline-script-collector";
 import type { ScriptTransportClient } from "./transport-client";
 
@@ -404,3 +405,55 @@ describe("InlineScriptCollector", () => {
     });
   });
 });
+
+describe("InlineScriptCollector secret env injection + source-side masking", () => {
+  it("injects the delivered secretEnv into the executor and masks it from output", async () => {
+    const calls: Array<{ secretEnv?: Record<string, string> }> = [];
+    const mockExecutor: InlineScriptExecutor = {
+      execute: async (input) => {
+        calls.push({ secretEnv: input.secretEnv });
+        // Simulate a script that echoes its injected secret.
+        return {
+          result: { success: true, message: "token is gh_realSecret123" },
+          stdout: "logged gh_realSecret123",
+          stderr: "",
+          timedOut: false,
+        };
+      },
+    };
+    const collector = new InlineScriptCollector(mockExecutor);
+
+    const result = await collector.execute({
+      config: createConfig({
+        secretEnv: { API_TOKEN: "${{ secrets.tok }}" },
+      }),
+      client: mockClient,
+      pluginId: "script",
+      secretEnv: { API_TOKEN: "gh_realSecret123" },
+    });
+
+    // The executor received the injected env.
+    expect(calls[0]?.secretEnv).toEqual({ API_TOKEN: "gh_realSecret123" });
+    // The secret is redacted from the result message (source-side).
+    expect(result.result.message).toBe("token is ****");
+    expect(JSON.stringify(result)).not.toContain("gh_realSecret123");
+  });
+
+  it("does not mask anything when no secretEnv is delivered", async () => {
+    const mockExecutor: InlineScriptExecutor = {
+      execute: async () => ({
+        result: { success: true, message: "nothing secret" },
+        stdout: "plain output",
+        stderr: "",
+        timedOut: false,
+      }),
+    };
+    const collector = new InlineScriptCollector(mockExecutor);
+    const result = await collector.execute({
+      config: createConfig(),
+      client: mockClient,
+      pluginId: "script",
+    });
+    expect(result.result.message).toBe("nothing secret");
+  });
+});

package/src/inline-script-collector.ts

@@ -2,6 +2,7 @@ import {
   Versioned,
   z,
   configString,
+  withConfigMeta,
   requestTimeoutMs,
   defaultEsmScriptRunner,
   type EsmScriptRunResult,
@@ -25,6 +26,11 @@ import {
 import { pluginMetadata } from "./plugin-metadata";
 import type { ScriptTransportClient } from "./transport-client";
 import { extractErrorMessage } from "@checkstack/common";
+import {
+  secretEnvMappingSchema,
+  maskScriptRunOutput,
+} from "@checkstack/secrets-common";
+import type { ResolutionRootStatus } from "@checkstack/script-packages-backend";
 
 // ============================================================================
 // EXECUTOR ADAPTER
@@ -58,6 +64,10 @@ export interface InlineScriptExecutor {
     config: Record<string, unknown>;
     timeoutMs: number;
     runContext?: CollectorRunContext;
+    /** Managed npm-package resolution root for this run, if ready. */
+    resolutionRoot?: string;
+    /** Resolved secret env injected into the runner for this run only. */
+    secretEnv?: Record<string, string>;
   }): Promise<InlineScriptExecutionResult>;
 }
 
@@ -67,8 +77,15 @@ export interface InlineScriptExecutor {
  * health-check runtime surface) and the virtual `@checkstack/healthcheck`
  * module / global `defineHealthCheck` helper.
  */
-const defaultInlineScriptExecutor: InlineScriptExecutor = {
-  async execute({ script, config, timeoutMs, runContext }) {
+export const defaultInlineScriptExecutor: InlineScriptExecutor = {
+  async execute({
+    script,
+    config,
+    timeoutMs,
+    runContext,
+    resolutionRoot,
+    secretEnv,
+  }) {
     const res: EsmScriptRunResult = await defaultEsmScriptRunner.run({
       script,
       context: {
@@ -80,6 +97,11 @@ const defaultInlineScriptExecutor: InlineScriptExecutor = {
       timeoutMs,
       helperModuleName: "@checkstack/healthcheck",
       helperFunctionName: "defineHealthCheck",
+      ...(resolutionRoot ? { resolutionRoot } : {}),
+      // Inject the resolved secrets as process.env for THIS run only.
+      ...(secretEnv && Object.keys(secretEnv).length > 0
+        ? { env: secretEnv }
+        : {}),
     });
     return res;
   },
@@ -92,9 +114,15 @@ const defaultInlineScriptExecutor: InlineScriptExecutor = {
 const inlineScriptConfigSchema = z.object({
   script: configString({
     "x-editor-types": ["typescript"],
+    "x-script-testable": true,
   }).describe(
     "TypeScript/JavaScript module. Use `import { ... } from \"node:os\"` to pull in Node built-ins. The recommended pattern is `export default defineHealthCheck({ success, message?, value? })` — `defineHealthCheck` is provided by `@checkstack/healthcheck` and asserts the return shape at the type level. Throwing also signals failure.",
   ),
+  secretEnv: withConfigMeta(secretEnvMappingSchema, { "x-secret-env": true })
+    .optional()
+    .describe(
+      'Secret → env mapping, e.g. { "API_TOKEN": "${{ secrets.token }}" }. Only the named secrets are resolved and injected for this run (read via process.env.API_TOKEN / $API_TOKEN); on a satellite they are delivered just-in-time over the encrypted channel, never persisted. Values are masked out of the collector output.',
+    ),
   timeout: requestTimeoutMs().describe("Maximum execution time in milliseconds"),
 });
 
@@ -254,30 +282,65 @@ export class InlineScriptCollector implements CollectorStrategy<
   });
 
   private executor: InlineScriptExecutor;
+  private getResolutionRoot?: () => Promise<ResolutionRootStatus>;
 
-  constructor(executor: InlineScriptExecutor = defaultInlineScriptExecutor) {
+  constructor(
+    executor: InlineScriptExecutor = defaultInlineScriptExecutor,
+    getResolutionRoot?: () => Promise<ResolutionRootStatus>,
+  ) {
     this.executor = executor;
+    this.getResolutionRoot = getResolutionRoot;
   }
 
   async execute({
     config,
     runContext,
+    secretEnv,
   }: {
     config: InlineScriptConfig;
     client: ScriptTransportClient;
     pluginId: string;
     runContext?: CollectorRunContext;
+    secretEnv?: Record<string, string>;
   }): Promise<CollectorResult<InlineScriptResult>> {
     const startTime = Date.now();
 
+    // Resolve the managed npm-package root. `notReady` -> fail clearly;
+    // `none` -> unset (no packages); `ready` -> point the runner at it.
+    const rootStatus = await this.getResolutionRoot?.();
+    if (rootStatus?.mode === "notReady") {
+      return {
+        result: {
+          success: false,
+          message: rootStatus.reason,
+          executionTimeMs: Date.now() - startTime,
+          timedOut: false,
+        },
+        error: rootStatus.reason,
+      };
+    }
+    const resolutionRoot =
+      rootStatus?.mode === "ready" ? rootStatus.root : undefined;
+
+    // Source-side masking values: the run's delivered secret values.
+    const maskValues = Object.values(secretEnv ?? {});
+
     let exec: InlineScriptExecutionResult;
     try {
-      exec = await this.executor.execute({
+      const raw = await this.executor.execute({
         script: config.script,
         config: config as unknown as Record<string, unknown>,
         timeoutMs: config.timeout,
         runContext,
+        ...(resolutionRoot ? { resolutionRoot } : {}),
+        ...(secretEnv ? { secretEnv } : {}),
       });
+      // Redact the delivered secret values from the captured output BEFORE
+      // any of it leaves the satellite (defense in depth: core masks again
+      // on receipt). A script echoing a secret it was given is masked here.
+      // `raw` carries `stdout`/`stderr` (required by ScriptRunOutput) plus
+      // `timedOut`, so the masked result keeps the full shape.
+      exec = maskScriptRunOutput({ output: raw, values: maskValues });
     } catch (error) {
       const executionTimeMs = Date.now() - startTime;
       const message = extractErrorMessage(error);

package/tsconfig.json

@@ -12,6 +12,12 @@
     },
     {
       "path": "../../core/healthcheck-common"
+    },
+    {
+      "path": "../../core/script-packages-backend"
+    },
+    {
+      "path": "../../core/secrets-common"
     }
   ]
 }