@checkstack/healthcheck-backend 1.13.1 → 1.15.0

package/CHANGELOG.md

@@ -1,5 +1,133 @@
 # @checkstack/healthcheck-backend
 
+## 1.15.0
+
+### Minor Changes
+
+- faf98f5: Security: config secrets (health-check strategy/collector credentials such as
+  SSH passwords, DB credentials, HTTP auth, and integration connection
+  credentials) ride ONE shared, domain-agnostic extraction channel instead of
+  being stored as plaintext or re-implemented per plugin.
+
+  New primitive and shared service:
+
+  - `configSecret({ id })` (in `@checkstack/backend-api`) declares an
+    extraction-channel secret keyed by a STABLE `id`, independent of field name or
+    position, so renaming or reordering a field never orphans its value. Use it
+    (not `configString({ "x-secret": true })`) for any credential whose config is
+    relayed to a satellite, projected to AI, or diffed by GitOps. `validateSecretIds`
+    rejects, at plugin registration, an `x-secret` field with no `id`, a duplicate
+    `id`, or a secret nested in an un-keyable container (array / record / tuple /
+    map) - so a mis-keyable schema fails boot rather than at run time.
+  - `ConfigSecretChannel` (in `@checkstack/secrets-backend`) is the single
+    extract / inflate / collect / redact / merge / delete / prune implementation.
+    Health-checks and integration connections both BIND it to their own scope
+    (marker prefix + internal-secret key layout); neither re-implements the walk.
+
+  Lifecycle (both bindings):
+
+  - **Write**: an inline value is extracted into the encrypted internal secret
+    store; the stored config keeps only an opaque marker. `${{ secrets.NAME }}`
+    references are stored verbatim and resolve through the active backend (local
+    or Vault) at run time.
+  - **Read**: configuration and connection reads strip `x-secret` values and
+    internal markers while keeping `${{ secrets.NAME }}` references visible; the
+    AI `getConfigurations` tool and create/update responses are redacted too. A
+    value never reaches a browser or an AI model context.
+  - **Run**: the core executor inflates markers/references in memory just before
+    the client is built. Satellites receive markers only and fetch values
+    just-in-time over the authenticated WS channel, per run, never persisted, then
+    fail CLOSED if any marker/reference survives resolution.
+  - **No orphan**: clearing a secret, removing a field/collector, swapping an
+    inline value for a reference, updating a connection, or deleting a
+    configuration/connection deletes the now-unreferenced internal secret. Cleanup
+    is schema-free (scans markers by prefix) and best-effort on delete, so it works
+    even when the owning plugin is uninstalled and never blocks a delete.
+  - **Forged-marker safe**: extract/inflate key each internal secret by the
+    SCHEMA leaf's stable `id`, never by an id parsed out of a stored marker string,
+    so a crafted marker can never resolve or delete another scope's secret.
+
+  Health-checks additionally get an idempotent, advisory-locked backfill that
+  moves pre-existing plaintext values into the internal store, and per-config-id
+  locking so concurrent writers across pods can never leave a dangling marker.
+  Integration connection credentials keep their released `__connref__:` marker
+  prefix and key layout (id equals the flat field name), so existing stored
+  connections are byte-compatible.
+
+  BREAKING CHANGES:
+
+  - Configuration and connection reads no longer include `x-secret` field values
+    (clients must treat blank-on-save as keep-existing; the bundled editors
+    already do).
+  - Satellites must be upgraded together with the core: an old satellite cannot
+    resolve the markers a new core stores, so its credentialed checks fail until
+    upgraded.
+
+### Patch Changes
+
+- Updated dependencies [faf98f5]
+- Updated dependencies [faf98f5]
+  - @checkstack/ai-backend@0.10.6
+  - @checkstack/backend-api@0.29.0
+  - @checkstack/secrets-backend@0.3.0
+  - @checkstack/secrets-common@0.3.0
+  - @checkstack/common@0.20.0
+  - @checkstack/healthcheck-common@1.12.0
+  - @checkstack/satellite-backend@0.8.0
+  - @checkstack/automation-backend@0.10.8
+  - @checkstack/catalog-backend@1.6.6
+  - @checkstack/incident-backend@1.9.4
+  - @checkstack/command-backend@0.2.18
+  - @checkstack/gitops-backend@0.5.18
+  - @checkstack/script-packages-backend@0.3.22
+  - @checkstack/status-page-backend@0.4.5
+  - @checkstack/gitops-common@0.7.1
+  - @checkstack/sdk@0.122.1
+  - @checkstack/ai-common@0.6.4
+  - @checkstack/cache-api@0.3.17
+  - @checkstack/catalog-common@2.6.1
+  - @checkstack/incident-common@1.7.1
+  - @checkstack/maintenance-common@1.8.1
+  - @checkstack/notification-common@1.5.1
+  - @checkstack/queue-api@0.3.17
+  - @checkstack/signal-common@0.2.15
+  - @checkstack/status-page-common@0.5.1
+  - @checkstack/cache-utils@0.2.22
+
+## 1.14.0
+
+### Minor Changes
+
+- e819276: Fix JSONPath collector assertions: the executor previously evaluated every
+  assertion with a flat field lookup, so a `Body (JSONPath)` assertion compared
+  against `undefined` and the configured path was silently ignored (`Exists`
+  always failed, `Not Exists` always passed). The executor now parses the source
+  field as JSON and extracts the configured path via `jsonpath-plus` (with
+  expression evaluation disabled - filter/script expressions are rejected).
+  Fail-closed: a non-JSON body, missing expression, or invalid path fails the
+  assertion with a diagnostic, never the collection.
+
+  Also adds `isEmpty` / `isNotEmpty` to the JSONPath operator set (and the
+  AssertionBuilder), treating `[]`, `{}`, `""`, and missing values as empty - so
+  "no errors reported" is a single `$.errors Is Empty` assertion, and "key exists
+  but is empty" is `Exists` + `Is Empty` on the same path.
+
+### Patch Changes
+
+- Updated dependencies [e819276]
+- Updated dependencies [e819276]
+  - @checkstack/ai-backend@0.10.5
+  - @checkstack/backend-api@0.28.0
+  - @checkstack/automation-backend@0.10.7
+  - @checkstack/catalog-backend@1.6.5
+  - @checkstack/incident-backend@1.9.3
+  - @checkstack/satellite-backend@0.7.8
+  - @checkstack/command-backend@0.2.17
+  - @checkstack/gitops-backend@0.5.17
+  - @checkstack/script-packages-backend@0.3.21
+  - @checkstack/secrets-backend@0.2.17
+  - @checkstack/status-page-backend@0.4.4
+
 ## 1.13.1
 
 ### Patch Changes

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@checkstack/healthcheck-backend",
-  "version": "1.13.1",
+  "version": "1.15.0",
   "license": "Elastic-2.0",
   "type": "module",
   "main": "src/index.ts",
@@ -14,45 +14,46 @@
     "lint:code": "eslint . --max-warnings 0"
   },
   "dependencies": {
-    "@checkstack/backend-api": "0.27.1",
-    "@checkstack/ai-backend": "0.10.4",
-    "@checkstack/ai-common": "0.6.3",
-    "@checkstack/script-packages-backend": "0.3.20",
-    "@checkstack/cache-api": "0.3.16",
-    "@checkstack/cache-utils": "0.2.21",
-    "@checkstack/catalog-backend": "1.6.4",
-    "@checkstack/catalog-common": "2.6.0",
-    "@checkstack/command-backend": "0.2.16",
-    "@checkstack/common": "0.19.0",
-    "@checkstack/gitops-backend": "0.5.16",
-    "@checkstack/gitops-common": "0.7.0",
-    "@checkstack/healthcheck-common": "1.11.0",
-    "@checkstack/secrets-common": "0.2.8",
-    "@checkstack/secrets-backend": "0.2.16",
-    "@checkstack/incident-backend": "1.9.2",
-    "@checkstack/incident-common": "1.7.0",
-    "@checkstack/automation-backend": "0.10.6",
-    "@checkstack/maintenance-common": "1.8.0",
-    "@checkstack/notification-common": "1.5.0",
-    "@checkstack/queue-api": "0.3.16",
-    "@checkstack/satellite-backend": "0.7.7",
-    "@checkstack/sdk": "0.120.0",
-    "@checkstack/signal-common": "0.2.14",
-    "@checkstack/status-page-backend": "0.4.3",
-    "@checkstack/status-page-common": "0.5.0",
+    "@checkstack/ai-backend": "0.10.6",
+    "@checkstack/ai-common": "0.6.4",
+    "@checkstack/automation-backend": "0.10.8",
+    "@checkstack/backend-api": "0.29.0",
+    "@checkstack/cache-api": "0.3.17",
+    "@checkstack/cache-utils": "0.2.22",
+    "@checkstack/catalog-backend": "1.6.6",
+    "@checkstack/catalog-common": "2.6.1",
+    "@checkstack/command-backend": "0.2.18",
+    "@checkstack/common": "0.20.0",
+    "@checkstack/gitops-backend": "0.5.18",
+    "@checkstack/gitops-common": "0.7.1",
+    "@checkstack/healthcheck-common": "1.12.0",
+    "@checkstack/incident-backend": "1.9.4",
+    "@checkstack/incident-common": "1.7.1",
+    "@checkstack/maintenance-common": "1.8.1",
+    "@checkstack/notification-common": "1.5.1",
+    "@checkstack/queue-api": "0.3.17",
+    "@checkstack/satellite-backend": "0.8.0",
+    "@checkstack/script-packages-backend": "0.3.22",
+    "@checkstack/sdk": "0.122.1",
+    "@checkstack/secrets-backend": "0.3.0",
+    "@checkstack/secrets-common": "0.3.0",
+    "@checkstack/signal-common": "0.2.15",
+    "@checkstack/status-page-backend": "0.4.5",
+    "@checkstack/status-page-common": "0.5.1",
     "@hono/zod-validator": "^0.7.6",
+    "@orpc/contract": "^1.14.4",
+    "@orpc/server": "^1.14.4",
     "drizzle-orm": "^0.45.0",
     "hono": "^4.12.25",
+    "jsonpath-plus": "^10.4.0",
     "tdigest": "^0.1.2",
     "yaml": "^2.6.1",
-    "zod": "^4.2.1",
-    "@orpc/contract": "^1.14.4",
-    "@orpc/server": "^1.14.4"
+    "zod": "^4.2.1"
   },
   "devDependencies": {
     "@checkstack/drizzle-helper": "0.0.6",
-    "@checkstack/scripts": "0.7.0",
-    "@checkstack/test-utils-backend": "0.1.50",
+    "@checkstack/scripts": "0.7.1",
+    "@checkstack/test-utils-backend": "0.1.52",
     "@checkstack/tsconfig": "0.0.7",
     "@types/bun": "^1.0.0",
     "@types/tdigest": "^0.1.5",

package/src/ai/healthcheck-propose.ts

@@ -103,9 +103,17 @@ const UNASSIGNED_GUIDANCE =
 export async function validateHealthcheckDraft({
   input,
   rpcClient,
+  existingConfigurationId,
 }: {
   input: HealthcheckProposeInput;
   rpcClient: RpcClient;
+  /**
+   * When validating an UPDATE, the id of the config being edited. The server
+   * restores that config's stored secrets before validating, so a kept secret
+   * (redacted out of the read) does not fail a required-secret check. Omit for
+   * a create draft, where blank secrets stay required.
+   */
+  existingConfigurationId?: string;
 }): Promise<{ strategy: HealthCheckStrategyDto; collectors: CollectorDto[] }> {
   const healthcheckClient = rpcClient.forPlugin(HealthCheckApi);
 
@@ -116,6 +124,7 @@ export async function validateHealthcheckDraft({
     config: input.config,
     intervalSeconds: input.intervalSeconds,
     collectors: input.collectors,
+    ...(existingConfigurationId ? { existingConfigurationId } : {}),
   });
   if (!validation.valid) {
     throw new HealthcheckProposeValidationError(

package/src/ai/healthcheck-update.ts

@@ -82,10 +82,14 @@ export function createHealthcheckUpdateTool(): RegisteredAiTool<
     }
     const merged = mergeConfig({ existing, body: input.body });
     // Deep-validate the merged result (throws with structured detail if invalid,
-    // including bad assertion fields/operators).
+    // including bad assertion fields/operators). Pass the id so the server
+    // restores stored secrets before validating - `existing.config` is
+    // redacted, so a kept `x-secret` field arrives blank and would otherwise
+    // fail a required-secret check even for a name-only edit.
     const { strategy } = await validateHealthcheckDraft({
       input: merged,
       rpcClient,
+      existingConfigurationId: input.id,
     });
     // before -> after diff over the updatable fields only (the existing config
     // also carries id/timestamps/paused that are not part of an update).

package/src/collector-assertions.test.ts

@@ -0,0 +1,276 @@
+import { describe, expect, it } from "bun:test";
+import type { CollectorAssertion } from "@checkstack/healthcheck-common";
+import { evaluateCollectorAssertions } from "./collector-assertions";
+
+/** The HTTP Request collector's result shape, as the executor sees it. */
+const httpResult = (body: string): Record<string, unknown> => ({
+  statusCode: 200,
+  statusText: "OK",
+  responseTimeMs: 12,
+  body,
+  bodyLength: body.length,
+  success: true,
+});
+
+const jsonPathAssertion = (
+  jsonPath: string,
+  operator: string,
+  value?: unknown,
+): CollectorAssertion => ({
+  field: "body.$",
+  jsonPath,
+  operator,
+  ...(value === undefined ? {} : { value }),
+});
+
+describe("evaluateCollectorAssertions", () => {
+  describe("plain field assertions (regression)", () => {
+    it("passes and fails on a direct result field", () => {
+      const result = httpResult("{}");
+
+      expect(
+        evaluateCollectorAssertions({
+          assertions: [{ field: "statusCode", operator: "equals", value: 200 }],
+          result,
+        }),
+      ).toBeUndefined();
+
+      expect(
+        evaluateCollectorAssertions({
+          assertions: [{ field: "statusCode", operator: "equals", value: 404 }],
+          result,
+        }),
+      ).toBe("statusCode equals 404");
+    });
+
+    it("returns undefined for no assertions", () => {
+      expect(
+        evaluateCollectorAssertions({
+          assertions: undefined,
+          result: httpResult(""),
+        }),
+      ).toBeUndefined();
+      expect(
+        evaluateCollectorAssertions({ assertions: [], result: httpResult("") }),
+      ).toBeUndefined();
+    });
+
+    it("returns the FIRST failing assertion in configured order", () => {
+      const failure = evaluateCollectorAssertions({
+        assertions: [
+          { field: "success", operator: "isFalse" },
+          { field: "statusCode", operator: "equals", value: 404 },
+        ],
+        result: httpResult("{}"),
+      });
+      expect(failure).toBe("success isFalse");
+    });
+  });
+
+  describe("JSONPath assertions: extraction actually happens", () => {
+    it("exists fails when the key is missing and passes when present", () => {
+      expect(
+        evaluateCollectorAssertions({
+          assertions: [jsonPathAssertion("$.error", "exists")],
+          result: httpResult(`{"status":"ok"}`),
+        }),
+      ).toBe("body.$ $.error exists");
+
+      expect(
+        evaluateCollectorAssertions({
+          assertions: [jsonPathAssertion("$.error", "exists")],
+          result: httpResult(`{"error":""}`),
+        }),
+      ).toBeUndefined();
+    });
+
+    it("notExists passes for a missing key and fails for a present one", () => {
+      expect(
+        evaluateCollectorAssertions({
+          assertions: [jsonPathAssertion("$.error", "notExists")],
+          result: httpResult(`{"status":"ok"}`),
+        }),
+      ).toBeUndefined();
+
+      expect(
+        evaluateCollectorAssertions({
+          assertions: [jsonPathAssertion("$.error", "notExists")],
+          result: httpResult(`{"error":"boom"}`),
+        }),
+      ).toBe("body.$ $.error notExists");
+    });
+
+    it("equals compares the EXTRACTED value, not the whole body", () => {
+      expect(
+        evaluateCollectorAssertions({
+          assertions: [jsonPathAssertion("$.data.id", "equals", "42")],
+          result: httpResult(`{"data":{"id":42},"noise":"x"}`),
+        }),
+      ).toBeUndefined();
+
+      expect(
+        evaluateCollectorAssertions({
+          assertions: [jsonPathAssertion("$.data.id", "equals", "43")],
+          result: httpResult(`{"data":{"id":42}}`),
+        }),
+      ).toBe("body.$ $.data.id equals 43");
+    });
+
+    it('asserts "error exists but is empty" via exists + isEmpty', () => {
+      const assertions = [
+        jsonPathAssertion("$.error", "exists"),
+        jsonPathAssertion("$.error", "isEmpty"),
+      ];
+
+      // Present and empty string: both pass.
+      expect(
+        evaluateCollectorAssertions({
+          assertions,
+          result: httpResult(`{"error":""}`),
+        }),
+      ).toBeUndefined();
+
+      // Missing entirely: exists fails (isEmpty alone would have passed).
+      expect(
+        evaluateCollectorAssertions({
+          assertions,
+          result: httpResult(`{"status":"ok"}`),
+        }),
+      ).toBe("body.$ $.error exists");
+
+      // Present and non-empty: isEmpty fails.
+      expect(
+        evaluateCollectorAssertions({
+          assertions,
+          result: httpResult(`{"error":"boom"}`),
+        }),
+      ).toBe("body.$ $.error isEmpty");
+    });
+  });
+
+  describe("empty arrays in JSON responses", () => {
+    it("isEmpty passes for [] and fails for a populated array", () => {
+      expect(
+        evaluateCollectorAssertions({
+          assertions: [jsonPathAssertion("$.errors", "isEmpty")],
+          result: httpResult(`{"errors":[]}`),
+        }),
+      ).toBeUndefined();
+
+      expect(
+        evaluateCollectorAssertions({
+          assertions: [jsonPathAssertion("$.errors", "isEmpty")],
+          result: httpResult(`{"errors":["boom"]}`),
+        }),
+      ).toBe("body.$ $.errors isEmpty");
+    });
+
+    it("isNotEmpty passes for a populated array and fails for []", () => {
+      expect(
+        evaluateCollectorAssertions({
+          assertions: [jsonPathAssertion("$.items", "isNotEmpty")],
+          result: httpResult(`{"items":[1]}`),
+        }),
+      ).toBeUndefined();
+
+      expect(
+        evaluateCollectorAssertions({
+          assertions: [jsonPathAssertion("$.items", "isNotEmpty")],
+          result: httpResult(`{"items":[]}`),
+        }),
+      ).toBe("body.$ $.items isNotEmpty");
+    });
+
+    it("isEmpty handles empty objects and nested paths", () => {
+      expect(
+        evaluateCollectorAssertions({
+          assertions: [jsonPathAssertion("$.meta.warnings", "isEmpty")],
+          result: httpResult(`{"meta":{"warnings":{}}}`),
+        }),
+      ).toBeUndefined();
+
+      expect(
+        evaluateCollectorAssertions({
+          assertions: [jsonPathAssertion("$.meta.warnings", "isEmpty")],
+          result: httpResult(`{"meta":{"warnings":{"w1":"x"}}}`),
+        }),
+      ).toBe("body.$ $.meta.warnings isEmpty");
+    });
+
+    it("array length is assertable via the .length property", () => {
+      expect(
+        evaluateCollectorAssertions({
+          assertions: [
+            jsonPathAssertion("$.items.length", "greaterThan", "2"),
+          ],
+          result: httpResult(`{"items":[1,2,3]}`),
+        }),
+      ).toBeUndefined();
+
+      expect(
+        evaluateCollectorAssertions({
+          assertions: [
+            jsonPathAssertion("$.items.length", "greaterThan", "5"),
+          ],
+          result: httpResult(`{"items":[1,2,3]}`),
+        }),
+      ).toBe("body.$ $.items.length greaterThan 5");
+    });
+  });
+
+  describe("fail-closed diagnostics", () => {
+    it("fails with a diagnostic when the body is not valid JSON", () => {
+      expect(
+        evaluateCollectorAssertions({
+          assertions: [jsonPathAssertion("$.error", "notExists")],
+          result: httpResult("<html>not json</html>"),
+        }),
+      ).toBe('body.$ $.error notExists (field "body" is not valid JSON)');
+    });
+
+    it("fails with a diagnostic when the source field is absent", () => {
+      expect(
+        evaluateCollectorAssertions({
+          assertions: [jsonPathAssertion("$.error", "exists")],
+          result: { statusCode: 200 },
+        }),
+      ).toBe('body.$ $.error exists (field "body" has no value)');
+    });
+
+    it("fails with a diagnostic when the JSONPath expression is missing", () => {
+      expect(
+        evaluateCollectorAssertions({
+          assertions: [{ field: "body.$", jsonPath: "  ", operator: "exists" }],
+          result: httpResult("{}"),
+        }),
+      ).toBe("body.$ exists (missing JSONPath expression)");
+    });
+
+    it("rejects filter/script expressions instead of evaluating them", () => {
+      const failure = evaluateCollectorAssertions({
+        assertions: [
+          jsonPathAssertion(`$.items[?(@.x==1)]`, "exists"),
+        ],
+        result: httpResult(`{"items":[{"x":1}]}`),
+      });
+      expect(failure).toContain("invalid JSONPath");
+    });
+  });
+
+  describe("structured (non-string) source fields", () => {
+    it("evaluates JSONPath against an already-parsed object field", () => {
+      expect(
+        evaluateCollectorAssertions({
+          assertions: [
+            {
+              field: "details.$",
+              jsonPath: "$.nodes",
+              operator: "isEmpty",
+            },
+          ],
+          result: { details: { nodes: [] } },
+        }),
+      ).toBeUndefined();
+    });
+  });
+});