@checkstack/incident-backend 1.1.5 → 1.2.0

package/CHANGELOG.md

@@ -1,5 +1,44 @@
 # @checkstack/incident-backend
 
+## 1.2.0
+
+### Minor Changes
+
+- ba07ae2: Quiet down notification spam on flapping systems, auto-open incidents when a check goes critical, and let operators land directly on the broken checks.
+
+  Notification policy lives **per healthcheck assignment** (one row per `system × configuration`). Different checks on the same system are fully independent — disabling a setting on one check does not affect the others. Defaults preserve existing behaviour for `suppressDeEscalations`; **auto-incident defaults to on** for new and existing assignments.
+
+  - **`suppressDeEscalations`** (off by default). When on, transitions from a worse state to a better-but-still-failing state (e.g. `unhealthy → degraded`) no longer fire a notification. Escalations and full recoveries to `healthy` are unaffected. Resolved per assignment (the just-ran check is the one driving any aggregate transition).
+  - **`autoOpenIncidentOnUnhealthy`** (on by default). Either of two independent triggers can open the auto-incident:
+    - **`sustainedUnhealthyTrigger`** (default 30 min) — opens when the check stays continuously unhealthy for the configured duration. Catches real outages.
+    - **`flappingTrigger`** (default 3 transitions in 60 min) — opens when the check flips to unhealthy that many times in the window. Catches persistent flapping where each unhealthy phase is too brief for the sustained trigger.
+      Each trigger can be individually disabled. One incident per system: triggering checks attach to an existing active auto-incident.
+  - **`useNotificationSuppression`** (on by default, only meaningful when auto-open is on). Controls whether the auto-opened incident is created with `suppressNotifications: true` — leaving this off opens the incident but still pings operators on each transition.
+  - **`skipDuringMaintenance`** (on by default). No auto-incident is opened while the system has an active maintenance window with suppression. The system is intentionally down and shouldn't trip the on-call.
+  - **`autoCloseAfterMinutes`** (default 30). Auto-close cooldown is now per-assignment and snapshotted per-incident at open time — later policy edits don't alter in-flight incidents. Setting `null` ("Never auto-close") leaves the incident for manual resolution.
+  - **Require-recovery rule.** After any auto-incident closes (manual or auto), no new auto-incident can open until the check has logged at least one healthy run. Prevents a "operator dismissed but it's still broken" loop.
+  - **Auto-close worker** ticks every 60s and resolves auto-opened incidents whose systems have been healthy for their per-row `cooldownMinutes`. Rows with `null` cooldown are skipped entirely. Per-incident: failed close attempts are logged but never abort the sweep.
+  - **`incidentResolved` hook subscriber** syncs the auto-incident mapping when an operator manually resolves the incident, so the require-recovery rule sees the close immediately.
+  - **Platform-wide defaults.** New admin RPCs `getPlatformNotificationDefaults` / `setPlatformNotificationDefaults` (under the existing `healthcheck.configuration.{read,manage}` access rules) let operators set notification policy once for the whole instance. Per-assignment rows with `notificationPolicy: null` inherit the platform defaults at read time. UI: a "Notification defaults" button in the Assignment IDE opens a modal editor. The per-assignment Notifications panel shows an inheritance banner — "Using platform defaults" (read-only) with an "Override" button, or "Custom override" with a "Use platform defaults" button to revert. The all-or-nothing model keeps the mental model simple: each assignment is either fully inherited or fully overridden.
+  - **New service-level RPCs** on the incident plugin (`createAutoIncident`, `resolveAutoIncident`) let other plugins open/close incidents without a user context. Reused by the healthcheck auto-incident flow.
+  - **Health-state notification CTA** now deep-links to `?filter=failing` on the system detail page for non-recovery transitions (label changes to "View failing checks"). The system overview gains an `All / Failing / Healthy` segmented filter wired to the same `?filter=…` param.
+  - **Notification bell badge** now counts collapse groups instead of raw rows, so the number matches what the user sees in the notifications list. Built on `COUNT(DISTINCT COALESCE(collapse_key, id))` — notifications without a collapse key still each count as one.
+  - **`statusFilter` on `getHistory` / `getDetailedHistory`** lets the run-history page and the drawer's Recent Runs panel filter to `All / Healthy / Failing` via shared pills, with the page resetting to the first page on filter change.
+  - **Pagination defaults aligned with selector options.** Several pages defaulted to a page size (5 or 20) that wasn't in the dropdown's options (`[10, 25, 50, 100]`), so the page-size `<Select>` rendered empty. The drawer's Recent Runs now defaults to 10; the Run History, History List, and Delivery Logs pages now default to 25.
+
+  Includes Drizzle migrations adding the `notification_policy` jsonb column to `system_health_checks`, plus two new tables: `health_check_unhealthy_transitions` (for threshold counting) and `health_check_auto_incidents` (for mapping back to incident ids during auto-close).
+
+### Patch Changes
+
+- Updated dependencies [ba07ae2]
+  - @checkstack/incident-common@1.3.0
+  - @checkstack/backend-api@0.17.1
+  - @checkstack/cache-api@0.3.5
+  - @checkstack/catalog-backend@1.1.6
+  - @checkstack/command-backend@0.1.30
+  - @checkstack/integration-backend@0.1.30
+  - @checkstack/cache-utils@0.2.10
+
 ## 1.1.5
 
 ### Patch Changes

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@checkstack/incident-backend",
-  "version": "1.1.5",
+  "version": "1.2.0",
   "license": "Elastic-2.0",
   "type": "module",
   "main": "src/index.ts",
@@ -14,27 +14,27 @@
     "lint:code": "eslint . --max-warnings 0"
   },
   "dependencies": {
-    "@checkstack/backend-api": "0.16.0",
-    "@checkstack/cache-api": "0.3.3",
-    "@checkstack/cache-utils": "0.2.8",
-    "@checkstack/incident-common": "1.2.1",
-    "@checkstack/catalog-common": "2.2.1",
-    "@checkstack/catalog-backend": "1.1.4",
-    "@checkstack/notification-common": "1.1.1",
-    "@checkstack/auth-common": "0.7.0",
-    "@checkstack/command-backend": "0.1.28",
-    "@checkstack/signal-common": "0.2.3",
-    "@checkstack/integration-backend": "0.1.28",
-    "@checkstack/integration-common": "0.4.0",
-    "@checkstack/common": "0.10.0",
+    "@checkstack/backend-api": "0.17.0",
+    "@checkstack/cache-api": "0.3.4",
+    "@checkstack/cache-utils": "0.2.9",
+    "@checkstack/incident-common": "1.2.2",
+    "@checkstack/catalog-common": "2.2.2",
+    "@checkstack/catalog-backend": "1.1.5",
+    "@checkstack/notification-common": "1.2.0",
+    "@checkstack/auth-common": "0.7.1",
+    "@checkstack/command-backend": "0.1.29",
+    "@checkstack/signal-common": "0.2.4",
+    "@checkstack/integration-backend": "0.1.29",
+    "@checkstack/integration-common": "0.5.0",
+    "@checkstack/common": "0.11.0",
     "drizzle-orm": "^0.45.0",
     "zod": "^4.2.1",
     "@orpc/server": "^1.13.2"
   },
   "devDependencies": {
     "@checkstack/drizzle-helper": "0.0.5",
-    "@checkstack/scripts": "0.3.2",
-    "@checkstack/test-utils-backend": "0.1.28",
+    "@checkstack/scripts": "0.3.3",
+    "@checkstack/test-utils-backend": "0.1.29",
     "@checkstack/tsconfig": "0.0.7",
     "@types/bun": "^1.0.0",
     "drizzle-kit": "^0.31.10",

package/src/router.ts

@@ -395,6 +395,96 @@ export function createRouter(
         return { suppressed };
       }),
 
+    createAutoIncident: os.createAutoIncident.handler(
+      async ({ input, context }) => {
+        // No user context for service-initiated incidents; createdBy
+        // stays null and the timeline shows the originating plugin via
+        // the hook payload.
+        const result = await service.createIncident(input);
+
+        await cache.invalidateForMutation({
+          incidentId: result.id,
+          systemIds: result.systemIds,
+        });
+
+        await signalService.broadcast(INCIDENT_UPDATED, {
+          incidentId: result.id,
+          systemIds: result.systemIds,
+          action: "created",
+        });
+
+        await context.emitHook(incidentHooks.incidentCreated, {
+          incidentId: result.id,
+          systemIds: result.systemIds,
+          title: result.title,
+          description: result.description,
+          severity: result.severity,
+          status: result.status,
+          createdAt: result.createdAt.toISOString(),
+        });
+
+        const systemNames = await resolveSystemNames(result.systemIds);
+        await notifyAffectedSystems({
+          catalogClient,
+          notificationClient,
+          logger,
+          incidentId: result.id,
+          incidentTitle: result.title,
+          systemIds: result.systemIds,
+          systemNames,
+          action: "created",
+          severity: result.severity,
+        });
+
+        return { id: result.id };
+      },
+    ),
+
+    resolveAutoIncident: os.resolveAutoIncident.handler(
+      async ({ input, context }) => {
+        const result = await service.resolveIncident(input.id, input.message);
+        // Idempotent: a missing or already-resolved incident is treated
+        // as success so the auto-close worker can be re-run safely.
+        if (!result) {
+          return { success: true };
+        }
+
+        await cache.invalidateForMutation({
+          incidentId: result.id,
+          systemIds: result.systemIds,
+        });
+
+        await signalService.broadcast(INCIDENT_UPDATED, {
+          incidentId: result.id,
+          systemIds: result.systemIds,
+          action: "resolved",
+        });
+
+        await context.emitHook(incidentHooks.incidentResolved, {
+          incidentId: result.id,
+          systemIds: result.systemIds,
+          title: result.title,
+          severity: result.severity,
+          resolvedAt: new Date().toISOString(),
+        });
+
+        const systemNames = await resolveSystemNames(result.systemIds);
+        await notifyAffectedSystems({
+          catalogClient,
+          notificationClient,
+          logger,
+          incidentId: result.id,
+          incidentTitle: result.title,
+          systemIds: result.systemIds,
+          systemNames,
+          action: "resolved",
+          severity: result.severity,
+        });
+
+        return { success: true };
+      },
+    ),
+
     addLink: os.addLink.handler(async ({ input }) => {
       // Verify incident exists so the FK violation surfaces as NOT_FOUND.
       const incident = await service.getIncident(input.incidentId);