@checkstack/integration-backend 0.1.30 → 0.3.0

package/CHANGELOG.md

@@ -1,5 +1,193 @@
 # @checkstack/integration-backend
 
+## 0.3.0
+
+### Minor Changes
+
+- 270ef29: Fix suspend/resume durability + complete the run-wide secret-masking guarantee.
+
+  A panel review confirmed several defects in the automation dispatch engine's suspend/resume durability and in the run-wide masking choke point. These survived because the unit suite stubbed the seam under test; the fixes ship with tests that exercise the real suspend / sweep / resume paths.
+
+  Suspend/resume durability:
+
+  - **Stalled sweeper no longer re-runs intentional waits.** `findStalledRunIds` now joins `automation_runs` and returns only `status = 'running'` runs, and suspend-finalisation no longer clobbers the run's `lastActionPath` checkpoint to `null`. Previously any wait longer than the stale window (>60s) was re-walked from the top every sweep cycle, re-firing pre-wait side effects and leaking wait locks. The wait-aware sweeps now also run before the stalled-run sweep.
+  - **Stalled recovery refuses a run holding a live wait lock.** `recoverStalledRun` now only recovers a genuinely-`running` run with no wait lock; a crash-mid-wait recovery is left to the wait/resume paths instead of re-walking from the top and creating a duplicate lock + duplicate delay job.
+  - **Cancelled runs can no longer resurrect.** `resumeRun` guards on `status === 'waiting'` (mirroring `checkWaitUntil`) and drops any stale lock for a non-waiting run, so `wakeWaitingRuns` / delay-expiry / a racing queue job can't wake a cancelled or terminal run. `cancelActiveRuns` (restart mode) now deletes the cancelled runs' wait locks + run-state in the same operation.
+  - **Concurrency check-then-create is serialized.** The `mode` check + `createRun` now run under a transaction-scoped advisory lock keyed on `(automationId, scope)`, so two concurrent fires can't both pass a `single`-mode "no active run" check and double-run.
+
+  Masking guarantee (now genuinely covers scope + artifacts):
+
+  - **The run-wide masking choke point now also masks the durable scope snapshot and produced artifacts.** The `RunSecretRegistry` is threaded into `RunStateStore.upsert` (masks `scopeSnapshot`) and `ArtifactStore.record` (masks `data`) so a resolved connection credential threaded into `scope.variables` or surfaced into an artifact is redacted before persist - and therefore cannot reach a read-only user via `getRunScopeForReplay`. **GUARANTEE CHANGE**: run-wide masking now covers step output, run error, scope snapshot, and artifact data for every action.
+  - **`testConnection` / `testProviderConnection` mask provider errors.** These RPCs run outside a dispatch run, so they build a per-call mask set from the resolved/submitted connection config and run any provider error through it before returning, so a provider error echoing a token can't cross back to the browser.
+  - **Short secrets surface a warning.** `setSecret` now warns when a value is shorter than `MIN_MASKABLE_LENGTH` (4) that it cannot be auto-redacted (the threshold is intentionally not lowered).
+
+  Internal:
+
+  - `@checkstack/backend-api`: `withXactLock`'s `fn` now receives the transaction handle `tx` so a critical section can run on the locked connection; the doc clarifies why running on the pool inside the lock window is still safe. The incident dedup caller's comment is corrected accordingly. `RunStore` gains `findWaitLocksByRun`.
+
+- 270ef29: Secrets platform Phase 5b: route integration connection credentials through the ONE secrets channel.
+
+  Connection credentials now resolve through the same secrets channel as
+  everything else, so a credential can originate from Vault and there is no
+  parallel credential-resolution code to drift. Two entry forms, both walked
+  by the shared `walkSecretFields` machinery (acting only on the provider
+  `connectionSchema`'s `x-secret` fields):
+
+  - Reference form: a `${{ secrets.NAME }}` template resolves through the
+    ACTIVE backend (local or Vault) via `secretResolverRef`.
+  - Inline form: an operator-typed value is extracted into an internal
+    secret on the local backend; the stored config keeps only a reference
+    marker, resolved via `internalSecretsRef`.
+
+  The `ConnectionStore` public API is unchanged: `listConnections` /
+  `getConnection` stay redacted; `getConnectionWithCredentials` inflates via
+  the unified channel. A one-time, idempotent, parity-verified, REVERSIBLE
+  migration (backup ConfigService entry per connection; rewrites only after
+  the platform copy reads back identically) moves existing inline
+  credentials onto the platform without breaking live connections.
+
+  `secrets-backend` exports `walkSecretFields` (the shared schema-walk behind
+  `resolveSecretsBySchema`, reused for the migration extract + inflate).
+
+  BREAKING CHANGES: a connection's stored credential fields may now hold a
+  `${{ secrets.NAME }}` reference or an internal-reference marker instead of
+  an inline value. Resolution is transparent (`getConnectionWithCredentials`
+  returns the same plaintext); a legacy inline value still resolves until the
+  one-time migration converts it.
+
+### Patch Changes
+
+- 270ef29: Secrets platform Phase 5: internal-secret consolidation (registry token) + connection-credential leak hardening.
+
+  - New `internalSecretsRef`: platform-internal secrets (not user-managed
+    named secrets) stored under a reserved `__internal__:` prefix, ALWAYS on
+    the local (always-writable, AES-GCM) backend so internal writes never
+    break when Vault is the active backend. Excluded from the user-facing
+    Secrets list.
+  - The script-package registry auth token is consolidated onto
+    `internalSecretsRef`. The `authSecretRef` column now holds a stable
+    marker; a one-time, idempotent, parity-verified migration moves legacy
+    inline ciphertext into the platform and only rewrites the column once the
+    platform copy reads back identically (legacy value never dropped early).
+    Resolution stays backward-compatible with legacy ciphertext.
+  - Integration: `createConnection` / `updateConnection` now return the
+    redacted connection preview instead of echoing the submitted credential
+    fields back in the response (leak hardening). Non-breaking — the frontend
+    refetches the redacted list and ignores the returned preview.
+
+  NOTE: integration connection-credential STORAGE is intentionally NOT
+  migrated onto the secrets platform. Connection creds are co-mingled
+  secret/non-secret config stored per-provider via `ConfigService` (which
+  already uses the same AES-GCM crypto + per-field redaction); splitting them
+  out would require per-provider schema-walking and a lossy migration across
+  live integrations for no real gain. The `ConnectionStore` API + storage are
+  unchanged.
+
+- Updated dependencies [270ef29]
+- Updated dependencies [270ef29]
+- Updated dependencies [270ef29]
+- Updated dependencies [b995afb]
+- Updated dependencies [270ef29]
+- Updated dependencies [270ef29]
+- Updated dependencies [270ef29]
+- Updated dependencies [b995afb]
+- Updated dependencies [270ef29]
+- Updated dependencies [270ef29]
+- Updated dependencies [270ef29]
+- Updated dependencies [270ef29]
+- Updated dependencies [270ef29]
+- Updated dependencies [270ef29]
+- Updated dependencies [b995afb]
+  - @checkstack/backend-api@0.19.0
+  - @checkstack/secrets-backend@0.1.0
+  - @checkstack/secrets-common@0.1.0
+  - @checkstack/command-backend@0.1.32
+  - @checkstack/queue-api@0.3.7
+
+## 0.2.0
+
+### Minor Changes
+
+- 41c77f4: feat(automation): one-time migration of webhook subscriptions + remove legacy integration backend
+
+  **BREAKING CHANGES** (platform is in BETA — no major bump):
+
+  - `IntegrationProvider` no longer carries `config` (subscription
+    config) or `deliver`. The interface now models a connection provider
+    only: connection schema + `getConnectionOptions` + `testConnection`.
+  - The legacy subscription / delivery-log / event endpoints
+    (`listSubscriptions`, `createSubscription`, `getDeliveryLogs`,
+    `listEventTypes`, …) are removed from `integrationContract`.
+  - `delivery-coordinator`, `hook-subscriber`, `event-registry`, and the
+    `integrationEventExtensionPoint` are deleted. Plugins that
+    previously called `integrationEvents.registerEvent(...)` now
+    register their hooks as automation triggers via
+    `automationTriggerExtensionPoint.registerTrigger(...)`.
+  - Frontend pages `IntegrationsPage` and `DeliveryLogsPage` are gone;
+    the integration plugin's only remaining UI is connection
+    management. Subscription management lives under `/automation/...`.
+  - `webhook_subscriptions` and `delivery_logs` tables stay in the
+    database for one release as a safety net (no code reads or writes
+    them), and will be dropped in a follow-up migration.
+
+  **New**:
+
+  - `jira.create_issue`, `teams.post_message`, `webex.post_message`,
+    `webhook.send`, `integration-script.run_shell`, and
+    `integration-script.run_script` actions registered against the
+    Automation Platform with matching `*.message`, `*.delivery`,
+    `shell.result`, and `script.result` artifact types. The script
+    plugin exposes **two** actions — `run_shell` runs bash via the
+    shared `ShellScriptRunner` (Monaco `shell` editor), `run_script`
+    runs an ESM module in a Bun subprocess via `EsmScriptRunner`
+    (Monaco `typescript` editor + `defineIntegration` helper) — to
+    preserve the legacy provider split. `jira.create_issue` keeps the
+    dynamic field-mapping dropdown (driven by
+    `JIRA_RESOLVERS.FIELD_OPTIONS`).
+  - One-time data migration runs on boot in
+    `automation-backend.afterPluginsReady`. It reads
+    `webhook_subscriptions` via a new service RPC
+    `IntegrationApi.listLegacySubscriptions`, translates each row into
+    a single-trigger / single-action automation (marked with
+    `managed_by = "migrated-subscription:<id>"`), and is idempotent
+    across restarts.
+  - Failed translations are recorded in a new
+    `automation_migration_failures` table and surfaced via
+    `AutomationApi.listMigrationFailures` /
+    `acknowledgeMigrationFailure` so admins can review and re-create
+    failed entries by hand.
+
+### Patch Changes
+
+- 41c77f4: feat(jira): register Jira automation actions + `jira.issue` artifact type
+
+  Adds three Jira actions to the Automation platform:
+
+  - `jira.create_issue` — produces the new `jira.issue` artifact type
+    (`issueKey`, `projectKey`, `issueUrl`, `id`, `status?`)
+  - `jira.transition_issue` — consumes `jira.issue` (or accepts an
+    explicit `issueKey`), idempotent against already-applied transitions
+  - `jira.add_comment` — consumes `jira.issue` (or accepts an explicit
+    `issueKey`)
+
+  Extends the Jira client with `getTransitions`, `getIssueStatus`,
+  `transitionIssue` (handles 204 No Content, comment in ADF for Cloud /
+  plain text for Data Center), and `addComment`. Adds a new
+  `JIRA_RESOLVERS.TRANSITION_OPTIONS` cascading dropdown driven by
+  `connectionId` + `issueKey`. `@checkstack/integration-backend` now
+  re-exports the `ConnectionStore` interface so action plugins can take
+  it as a typed dep.
+
+- Updated dependencies [41c77f4]
+- Updated dependencies [6d52276]
+- Updated dependencies [35bc682]
+  - @checkstack/integration-common@0.6.0
+  - @checkstack/common@0.12.0
+  - @checkstack/backend-api@0.18.0
+  - @checkstack/command-backend@0.1.31
+  - @checkstack/signal-common@0.2.5
+  - @checkstack/queue-api@0.3.6
+
 ## 0.1.30
 
 ### Patch Changes

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@checkstack/integration-backend",
-  "version": "0.1.30",
+  "version": "0.3.0",
   "license": "Elastic-2.0",
   "type": "module",
   "main": "src/index.ts",
@@ -15,21 +15,23 @@
     "test": "bun test"
   },
   "dependencies": {
-    "@checkstack/integration-common": "0.5.0",
-    "@checkstack/backend-api": "0.17.0",
-    "@checkstack/signal-common": "0.2.4",
-    "@checkstack/queue-api": "0.3.4",
-    "@checkstack/common": "0.11.0",
-    "@checkstack/command-backend": "0.1.29",
+    "@checkstack/integration-common": "0.6.0",
+    "@checkstack/backend-api": "0.18.0",
+    "@checkstack/signal-common": "0.2.5",
+    "@checkstack/queue-api": "0.3.6",
+    "@checkstack/common": "0.12.0",
+    "@checkstack/command-backend": "0.1.31",
+    "@checkstack/secrets-common": "0.0.1",
+    "@checkstack/secrets-backend": "0.0.1",
     "drizzle-orm": "^0.45.0",
     "zod": "^4.2.1",
     "@orpc/server": "^1.13.2"
   },
   "devDependencies": {
     "@checkstack/drizzle-helper": "0.0.5",
-    "@checkstack/scripts": "0.3.3",
+    "@checkstack/scripts": "0.3.4",
     "@checkstack/tsconfig": "0.0.7",
-    "@checkstack/test-utils-backend": "0.1.29",
+    "@checkstack/test-utils-backend": "0.1.31",
     "@types/node": "^20.0.0",
     "drizzle-kit": "^0.31.10",
     "typescript": "^5.0.0"

package/src/connection-credentials-migration.test.ts

@@ -0,0 +1,171 @@
+import { describe, it, expect } from "bun:test";
+import { z } from "zod";
+import { configString } from "@checkstack/backend-api";
+import { createMockLogger } from "@checkstack/test-utils-backend";
+import type {
+  Logger,
+  ConfigService,
+} from "@checkstack/backend-api";
+import type {
+  SecretResolverService,
+  InternalSecretsService,
+} from "@checkstack/secrets-backend";
+import {
+  migrateConnectionCredentials,
+  type ConnectionForMigration,
+} from "./connection-credentials-migration";
+import { inflateConnectionCredentials } from "./connection-credentials";
+
+const schema = z.object({
+  baseUrl: z.string(),
+  apiToken: configString({ "x-secret": true }),
+});
+
+function fakeInternal(): InternalSecretsService & { store: Map<string, string> } {
+  const store = new Map<string, string>();
+  const key = (parts: string[]) => parts.join("|");
+  return {
+    store,
+    set: async ({ parts, value }) => {
+      store.set(key(parts), value);
+    },
+    get: async ({ parts }) => store.get(key(parts)),
+    delete: async ({ parts }) => {
+      store.delete(key(parts));
+    },
+  };
+}
+
+const noopResolver: SecretResolverService = {
+  resolveSecret: async () => "",
+  resolveBySchema: async ({ value }) => ({ resolved: value, warnings: [] }),
+  resolveForRun: async () => ({
+    env: {},
+    masking: { size: 0, maskText: (t) => t, maskDeep: (v) => v },
+  }),
+};
+
+/** Minimal in-memory ConfigService for the backup entries. */
+function fakeConfigService(): ConfigService {
+  const store = new Map<string, unknown>();
+  return {
+    get: async (id) => store.get(id) as never,
+    getRedacted: async (id) => store.get(id) as never,
+    set: async (id, _s, _v, data) => {
+      store.set(id, data);
+    },
+    delete: async (id) => {
+      store.delete(id);
+    },
+    list: async () => [...store.keys()],
+  };
+}
+
+const PROVIDER = "integration-jira.jira";
+
+describe("migrateConnectionCredentials", () => {
+  it("migrates an inline credential, backs it up, and is parity-verified + reversible", async () => {
+    const internal = fakeInternal();
+    const configService = fakeConfigService();
+    const persisted = new Map<string, Record<string, unknown>>();
+
+    const conn: ConnectionForMigration = {
+      providerId: PROVIDER,
+      connectionId: "c1",
+      schema,
+      schemaVersion: 1,
+      config: { baseUrl: "https://x", apiToken: "inline-secret-1" },
+    };
+
+    const result = await migrateConnectionCredentials({
+      configService,
+      internalSecrets: internal,
+      secretResolver: noopResolver,
+      logger: createMockLogger() as Logger,
+      loadConnections: async () => [conn],
+      persistConfig: async ({ connectionId, config }) => {
+        persisted.set(connectionId, config);
+      },
+    });
+
+    expect(result).toEqual({ total: 1, migrated: 1, skipped: 0, failed: 0 });
+
+    // Stored config is reference-ized (no plaintext).
+    const stored = persisted.get("c1")!;
+    expect(JSON.stringify(stored)).not.toContain("inline-secret-1");
+
+    // Reversible: a backup of the original raw config exists.
+    const backup = await configService.get(
+      "integration_connection_credbackup_integration-jira_jira_c1",
+      z.object({ config: z.record(z.string(), z.unknown()) }),
+      1,
+    );
+    expect((backup as { config: Record<string, unknown> }).config.apiToken).toBe(
+      "inline-secret-1",
+    );
+
+    // Parity: inflating the stored form yields the original plaintext.
+    const { config: inflated } = await inflateConnectionCredentials({
+      providerId: PROVIDER,
+      connectionId: "c1",
+      config: stored,
+      schema,
+      deps: { internalSecrets: internal, secretResolver: noopResolver },
+    });
+    expect(inflated).toEqual(conn.config);
+  });
+
+  it("is idempotent: re-running over the already-migrated (marker) config does nothing", async () => {
+    const internal = fakeInternal();
+    const configService = fakeConfigService();
+
+    // First pass.
+    let stored: Record<string, unknown> = {
+      baseUrl: "https://x",
+      apiToken: "inline-secret-2",
+    };
+    const run = (config: Record<string, unknown>) =>
+      migrateConnectionCredentials({
+        configService,
+        internalSecrets: internal,
+        secretResolver: noopResolver,
+        logger: createMockLogger() as Logger,
+        loadConnections: async () => [
+          { providerId: PROVIDER, connectionId: "c2", schema, schemaVersion: 1, config },
+        ],
+        persistConfig: async ({ config: c }) => {
+          stored = c;
+        },
+      });
+
+    const first = await run(stored);
+    expect(first.migrated).toBe(1);
+    const second = await run(stored); // now reference-ized
+    expect(second.migrated).toBe(0);
+    expect(second.skipped).toBe(1);
+  });
+
+  it("skips a connection with no inline secret (already a reference)", async () => {
+    const internal = fakeInternal();
+    const result = await migrateConnectionCredentials({
+      configService: fakeConfigService(),
+      internalSecrets: internal,
+      secretResolver: noopResolver,
+      logger: createMockLogger() as Logger,
+      loadConnections: async () => [
+        {
+          providerId: PROVIDER,
+          connectionId: "c3",
+          schema,
+          schemaVersion: 1,
+          config: { baseUrl: "https://x", apiToken: "${{ secrets.jira }}" },
+        },
+      ],
+      persistConfig: async () => {
+        throw new Error("should not persist a reference-only connection");
+      },
+    });
+    expect(result.skipped).toBe(1);
+    expect(result.migrated).toBe(0);
+  });
+});

package/src/connection-credentials-migration.ts

@@ -0,0 +1,182 @@
+/**
+ * One-time, idempotent, parity-verified, REVERSIBLE migration that moves
+ * inline connection credentials onto the secrets platform.
+ *
+ * For each existing connection of each provider that has a
+ * `connectionSchema`:
+ *   1. Back up the current raw config to a backup ConfigService entry
+ *      (only if no backup exists yet) so the pre-migration value is
+ *      recoverable.
+ *   2. Extract inline `x-secret` values into internal secrets (local
+ *      backend) via `extractInlineCredentials`, producing a config whose
+ *      secret fields are internal-ref markers.
+ *   3. Verify parity: inflate the rewritten config back through the unified
+ *      channel and confirm every secret field resolves to the SAME value as
+ *      the original. Only if parity holds, rewrite the stored config.
+ *
+ * Idempotent: a config already reference-ized (markers / `${{ }}`) extracts
+ * nothing and is left as-is. Guarded across boots by the per-connection
+ * backup entry + the marker form. Never drops a value until the platform
+ * copy reads back identically.
+ */
+import { z } from "zod";
+import type { ConfigService, Logger } from "@checkstack/backend-api";
+import { extractErrorMessage } from "@checkstack/common";
+import type {
+  SecretResolverService,
+  InternalSecretsService,
+} from "@checkstack/secrets-backend";
+import {
+  extractInlineCredentials,
+  inflateConnectionCredentials,
+} from "./connection-credentials";
+
+const BACKUP_VERSION = 1;
+
+/** Backup ConfigService key for a connection's pre-migration raw config. */
+function backupKey(providerId: string, connectionId: string): string {
+  const sanitized = providerId.replaceAll(".", "_");
+  return `integration_connection_credbackup_${sanitized}_${connectionId}`;
+}
+
+const BackupSchema = z.object({
+  config: z.record(z.string(), z.unknown()),
+  migratedAt: z.coerce.date(),
+});
+
+export interface ConnectionForMigration {
+  providerId: string;
+  connectionId: string;
+  /** Provider connection schema (Zod) used to find x-secret fields. */
+  schema: z.ZodTypeAny;
+  /** Provider connectionSchema version (for ConfigService set). */
+  schemaVersion: number;
+  /** Current raw stored config. */
+  config: Record<string, unknown>;
+}
+
+export interface MigrationResult {
+  total: number;
+  migrated: number;
+  skipped: number;
+  failed: number;
+}
+
+/**
+ * Run the credential migration over the given connections.
+ *
+ * `loadConnections` yields every connection (with its provider schema) so
+ * this stays decoupled from the store internals; `persistConfig` writes the
+ * rewritten config back through the store's normal path.
+ */
+export async function migrateConnectionCredentials(params: {
+  configService: ConfigService;
+  internalSecrets: InternalSecretsService;
+  secretResolver: SecretResolverService;
+  logger: Logger;
+  loadConnections: () => Promise<ConnectionForMigration[]>;
+  persistConfig: (input: {
+    providerId: string;
+    connectionId: string;
+    schemaVersion: number;
+    config: Record<string, unknown>;
+  }) => Promise<void>;
+}): Promise<MigrationResult> {
+  const {
+    configService,
+    internalSecrets,
+    secretResolver,
+    logger,
+    loadConnections,
+    persistConfig,
+  } = params;
+
+  const connections = await loadConnections();
+  const result: MigrationResult = {
+    total: connections.length,
+    migrated: 0,
+    skipped: 0,
+    failed: 0,
+  };
+
+  for (const conn of connections) {
+    try {
+      // 1. Back up the original raw config (idempotent — only once).
+      const bKey = backupKey(conn.providerId, conn.connectionId);
+      const existingBackup = await configService.get(
+        bKey,
+        BackupSchema,
+        BACKUP_VERSION,
+      );
+      if (!existingBackup) {
+        await configService.set(bKey, BackupSchema, BACKUP_VERSION, {
+          config: conn.config,
+          migratedAt: new Date(),
+        });
+      }
+
+      // 2. Extract inline secret values into internal secrets.
+      const { config: rewritten, extracted } = await extractInlineCredentials({
+        providerId: conn.providerId,
+        connectionId: conn.connectionId,
+        config: conn.config,
+        schema: conn.schema,
+        internalSecrets,
+      });
+
+      if (extracted === 0) {
+        // Already reference-ized (or no secret fields) — nothing to do.
+        result.skipped++;
+        continue;
+      }
+
+      // 3. Parity check: inflate the rewritten config and confirm the
+      //    secret fields resolve back to the ORIGINAL values before we
+      //    overwrite the stored config (reversible guarantee).
+      const original = await inflateConnectionCredentials({
+        providerId: conn.providerId,
+        connectionId: conn.connectionId,
+        config: conn.config,
+        schema: conn.schema,
+        deps: { secretResolver, internalSecrets },
+      });
+      const roundTripped = await inflateConnectionCredentials({
+        providerId: conn.providerId,
+        connectionId: conn.connectionId,
+        config: rewritten,
+        schema: conn.schema,
+        deps: { secretResolver, internalSecrets },
+      });
+      if (
+        JSON.stringify(original.config) !== JSON.stringify(roundTripped.config)
+      ) {
+        logger.error(
+          `Credential migration parity check FAILED for connection ${conn.connectionId}; leaving it unchanged.`,
+        );
+        result.failed++;
+        continue;
+      }
+
+      // Parity proven — rewrite the stored config to the reference form.
+      await persistConfig({
+        providerId: conn.providerId,
+        connectionId: conn.connectionId,
+        schemaVersion: conn.schemaVersion,
+        config: rewritten,
+      });
+      result.migrated++;
+    } catch (error) {
+      logger.error(
+        `Credential migration failed for connection ${conn.connectionId}: ${extractErrorMessage(error)}`,
+      );
+      result.failed++;
+    }
+  }
+
+  if (result.migrated > 0 || result.failed > 0) {
+    logger.info(
+      `Connection credential migration: ${result.migrated} migrated, ${result.skipped} skipped, ${result.failed} failed (of ${result.total}).`,
+    );
+  }
+  return result;
+}