@checkstack/notification-backend 1.1.0 → 1.3.0

package/CHANGELOG.md

@@ -1,5 +1,207 @@
 # @checkstack/notification-backend
 
+## 1.3.0
+
+### Minor Changes
+
+- ba07ae2: Quiet down notification spam on flapping systems, auto-open incidents when a check goes critical, and let operators land directly on the broken checks.
+
+  Notification policy lives **per healthcheck assignment** (one row per `system × configuration`). Different checks on the same system are fully independent — disabling a setting on one check does not affect the others. Defaults preserve existing behaviour for `suppressDeEscalations`; **auto-incident defaults to on** for new and existing assignments.
+
+  - **`suppressDeEscalations`** (off by default). When on, transitions from a worse state to a better-but-still-failing state (e.g. `unhealthy → degraded`) no longer fire a notification. Escalations and full recoveries to `healthy` are unaffected. Resolved per assignment (the just-ran check is the one driving any aggregate transition).
+  - **`autoOpenIncidentOnUnhealthy`** (on by default). Either of two independent triggers can open the auto-incident:
+    - **`sustainedUnhealthyTrigger`** (default 30 min) — opens when the check stays continuously unhealthy for the configured duration. Catches real outages.
+    - **`flappingTrigger`** (default 3 transitions in 60 min) — opens when the check flips to unhealthy that many times in the window. Catches persistent flapping where each unhealthy phase is too brief for the sustained trigger.
+      Each trigger can be individually disabled. One incident per system: triggering checks attach to an existing active auto-incident.
+  - **`useNotificationSuppression`** (on by default, only meaningful when auto-open is on). Controls whether the auto-opened incident is created with `suppressNotifications: true` — leaving this off opens the incident but still pings operators on each transition.
+  - **`skipDuringMaintenance`** (on by default). No auto-incident is opened while the system has an active maintenance window with suppression. The system is intentionally down and shouldn't trip the on-call.
+  - **`autoCloseAfterMinutes`** (default 30). Auto-close cooldown is now per-assignment and snapshotted per-incident at open time — later policy edits don't alter in-flight incidents. Setting `null` ("Never auto-close") leaves the incident for manual resolution.
+  - **Require-recovery rule.** After any auto-incident closes (manual or auto), no new auto-incident can open until the check has logged at least one healthy run. Prevents a "operator dismissed but it's still broken" loop.
+  - **Auto-close worker** ticks every 60s and resolves auto-opened incidents whose systems have been healthy for their per-row `cooldownMinutes`. Rows with `null` cooldown are skipped entirely. Per-incident: failed close attempts are logged but never abort the sweep.
+  - **`incidentResolved` hook subscriber** syncs the auto-incident mapping when an operator manually resolves the incident, so the require-recovery rule sees the close immediately.
+  - **Platform-wide defaults.** New admin RPCs `getPlatformNotificationDefaults` / `setPlatformNotificationDefaults` (under the existing `healthcheck.configuration.{read,manage}` access rules) let operators set notification policy once for the whole instance. Per-assignment rows with `notificationPolicy: null` inherit the platform defaults at read time. UI: a "Notification defaults" button in the Assignment IDE opens a modal editor. The per-assignment Notifications panel shows an inheritance banner — "Using platform defaults" (read-only) with an "Override" button, or "Custom override" with a "Use platform defaults" button to revert. The all-or-nothing model keeps the mental model simple: each assignment is either fully inherited or fully overridden.
+  - **New service-level RPCs** on the incident plugin (`createAutoIncident`, `resolveAutoIncident`) let other plugins open/close incidents without a user context. Reused by the healthcheck auto-incident flow.
+  - **Health-state notification CTA** now deep-links to `?filter=failing` on the system detail page for non-recovery transitions (label changes to "View failing checks"). The system overview gains an `All / Failing / Healthy` segmented filter wired to the same `?filter=…` param.
+  - **Notification bell badge** now counts collapse groups instead of raw rows, so the number matches what the user sees in the notifications list. Built on `COUNT(DISTINCT COALESCE(collapse_key, id))` — notifications without a collapse key still each count as one.
+  - **`statusFilter` on `getHistory` / `getDetailedHistory`** lets the run-history page and the drawer's Recent Runs panel filter to `All / Healthy / Failing` via shared pills, with the page resetting to the first page on filter change.
+  - **Pagination defaults aligned with selector options.** Several pages defaulted to a page size (5 or 20) that wasn't in the dropdown's options (`[10, 25, 50, 100]`), so the page-size `<Select>` rendered empty. The drawer's Recent Runs now defaults to 10; the Run History, History List, and Delivery Logs pages now default to 25.
+
+  Includes Drizzle migrations adding the `notification_policy` jsonb column to `system_health_checks`, plus two new tables: `health_check_unhealthy_transitions` (for threshold counting) and `health_check_auto_incidents` (for mapping back to incident ids during auto-close).
+
+### Patch Changes
+
+- @checkstack/backend-api@0.17.1
+- @checkstack/auth-backend@0.4.30
+- @checkstack/cache-api@0.3.5
+- @checkstack/queue-api@0.3.5
+- @checkstack/cache-utils@0.2.10
+
+## 1.2.0
+
+### Minor Changes
+
+- f23f3c9: Add per-channel notification delivery-attempt tracking
+  (Phase 8 of the v1 polishing plan). The external dispatch loop now
+  persists one row per `strategy.send(...)` call into a new
+  `notification_delivery_attempts` table - both successes and
+  failures - so silent delivery breakage (misconfigured webhooks, dead
+  channels) becomes queryable instead of buried in logs.
+
+  - `@checkstack/notification-backend` adds the
+    `notification_delivery_attempts` table, the matching Drizzle
+    migration, and a new `dispatchWithAttempt` helper that wraps every
+    external `strategy.send(...)` with duration measurement and
+    best-effort row persistence. The insert is intentionally
+    fire-and-forget: if writing the attempt row itself errors, the
+    dispatch loop logs and continues so visibility tracking can never
+    introduce a _new_ silent failure.
+  - `@checkstack/notification-common` exports a new
+    `DeliveryAttemptSchema` zod schema, the
+    `ListDeliveryAttemptsInputSchema =
+PaginationInput.extend({ notificationId })` input, and a new
+    `getDeliveryAttempts` procedure on the contract. The procedure is
+    gated by the existing `notificationAccess.admin`
+    (`notification:manage`) access rule - no new permission was
+    introduced.
+  - `@checkstack/notification-frontend` adds a minimal admin-only
+    `DeliveryAttemptsPage` (route id `notification.deliveryAttempts`,
+    path `/notifications/delivery-attempts`) and an "Open inspector"
+    link from the Notification Settings page for users with
+    `notification:manage`. No client-side `isAdmin` gate - the FORBIDDEN
+    case is rendered via the standard error-state branch on the page,
+    enforced by the contract.
+
+  Visibility only: there is no retry mechanism in this phase. A
+  `failure` row is a final outcome an admin actions manually
+  (re-trigger the source event, fix the misconfigured channel).
+  Automated retries are deferred to v1.1.
+
+  Strategy errors thrown during `send(...)` are persisted via
+  `extractErrorMessage(error)` so secrets potentially embedded in raw
+  error objects (webhook URLs, OAuth tokens reachable from the strategy
+  send context) are not stored verbatim.
+
+  See the new
+  `docs/src/content/docs/backend/notification-delivery.md` page for the
+  full surface description.
+
+### Patch Changes
+
+- f23f3c9: Add `correlationMiddleware` to `@checkstack/backend-api` and apply it
+  to every plugin/core router so each request carries a stable
+  `x-correlation-id` (read from the inbound header, or freshly minted
+  via `crypto.randomUUID()` when absent) and an auto-injected child
+  logger bound with `{ correlationId, pluginId, userId? }`. The ID is
+  echoed back on the response header so the caller can correlate their
+  client-side trace to the server logs.
+
+  The `Logger` interface in `@checkstack/backend-api` now formally
+  documents the structured-metadata convention (`logger.info("msg",
+{ ...meta })`) alongside the long-standing varargs shape. Winston's
+  splat handling already routes both shapes through the same vararg
+  slot, so existing call sites are unaffected. A new optional
+  `Logger.child(meta)` method captures the metadata-binding contract the
+  new middleware relies on; production loggers always implement it,
+  minimal test mocks may omit it (the middleware falls back gracefully).
+
+  `RpcContext` grew two optional `Headers` bags, `requestHeaders` and
+  `responseHeaders`, populated by the outer Hono `/api/*` and `/rest/*`
+  handlers in `@checkstack/backend`. They are write-through observation
+  points for middleware; an `RpcContext` constructed without them (S2S
+  clients, tests) keeps working — the echo is a silent no-op and the ID
+  is still bound onto the child logger for server-side correlation.
+
+  The scaffolding template in `@checkstack/scripts` was updated so any
+  new plugin generated via `bun run create` wires the middleware in the
+  expected `.use(correlationMiddleware).use(autoAuthMiddleware)` order
+  out of the box.
+
+- f23f3c9: Sweep every paginated `*-common` contract onto the canonical
+  `PaginationInput` / `PaginatedResult` from `@checkstack/common` and
+  remove the now-unused legacy exports.
+
+  **BREAKING CHANGE** - `@checkstack/common` drops the deprecated
+  `PaginationInputSchema`, `paginatedOutput`, and `PaginatedResponse`
+  symbols. Callers must consume `PaginationInput` (input) and
+  `PaginatedResult(itemSchema)` (output) instead. The canonical input is
+  `{ limit (1-100, default 20), offset (>= 0, default 0) }`; the
+  canonical output envelope is
+  `{ items, total, limit, offset }`.
+
+  **BREAKING CHANGE** - `@checkstack/notification-common` migrates
+  `getNotifications` off the legacy `PaginationInputSchema`
+  (`{ limit, offset, unreadOnly }` with output `{ notifications, total }`)
+  onto `ListNotificationsInputSchema =
+PaginationInput.extend({ unreadOnly })` and
+  `PaginatedResult(NotificationSchema)`. The output key changes from
+  `notifications` to `items`, and `limit` / `offset` are now echoed on
+  the response. The `PaginationInput` type alias previously exported
+  from `notification-common` is removed - use `ListNotificationsInput`
+  or the canonical `PaginationInput` from `@checkstack/common`.
+
+  **BREAKING CHANGE** - `@checkstack/integration-common` migrates
+  `listSubscriptions` (inline `{ page, pageSize, ... }` -> output
+  `{ subscriptions, total }`) and `getDeliveryLogs` (via
+  `DeliveryLogQueryInputSchema` `{ subscriptionId?, eventType?, status?,
+page, pageSize }` -> output `{ logs, total }`) onto the canonical
+  `PaginationInput.extend({...})` input and
+  `PaginatedResult(itemSchema)` output. External callers must switch
+  from `{ page, pageSize }` to `{ limit, offset }` and read response
+  items from `data.items` (no more `data.subscriptions` / `data.logs`).
+
+  The matching `*-backend` handlers were updated to consume the new
+  input shape (`offset` arithmetic in lieu of `(page - 1) * pageSize`)
+  and to echo `limit` / `offset` on the response. The `*-frontend` call
+  sites in `NotificationsPage`, `NotificationBell`, `IntegrationsPage`,
+  and `DeliveryLogsPage` were updated to send the new input shape and
+  read `data.items`.
+
+- f23f3c9: Phase 9 of the v1 polishing plan: tighten the plugin loader's boot-time
+  hook policy and backfill notification-router test coverage.
+
+  `@checkstack/backend` adopts an explicit per-hook policy for the two
+  boot-time hooks the plugin loader emits. `pluginInitialized` now
+  **halts the boot** if a subscriber throws — a failing subscriber here
+  means a downstream never wired itself against the freshly initialised
+  plugin, and continuing past that would leave the platform serving
+  traffic in a half-wired state. `accessRulesRegistered` keeps its
+  log-and-continue behaviour but escalates to `error` level and emits a
+  summary count if any subscriber failed; boot-blocking this hook would
+  let one misbehaving plugin DOS every other plugin on the same
+  instance. The policy is documented inline at each emit site and in a
+  new `docs/src/content/docs/backend/plugin-hook-policy.md` page.
+  **BREAKING CHANGE**: subscribers to `pluginInitialized` that
+  previously threw silently (logged and swallowed) now halt platform
+  boot. Audit subscribers and ensure they handle their own internal
+  errors before throwing.
+
+  `@checkstack/notification-backend` ships a real
+  `core/notification-backend/src/router.test.ts` covering the dispatch
+  fan-out (`notifyForSubscription`: zero subscribers, multi-recipient
+  insert, `excludeUserIds`, plus NOT_FOUND/FORBIDDEN guard rails), the
+  canonical paginated read on `getNotifications` (envelope shape,
+  `unreadOnly` filter propagation, null→undefined column mapping), the
+  service-only `createGroup` upsert behaviour (happy path + idempotent
+  re-create), and the multi-strategy `sendTransactional` path with a
+  focused fallback-style assertion: when one strategy throws, the
+  dispatch loop continues to the next and surfaces the failure as a
+  per-strategy `success: false` row instead of short-circuiting. No
+  runtime changes to the notification router.
+
+- Updated dependencies [f23f3c9]
+- Updated dependencies [f23f3c9]
+- Updated dependencies [f23f3c9]
+- Updated dependencies [f23f3c9]
+  - @checkstack/common@0.11.0
+  - @checkstack/backend-api@0.17.0
+  - @checkstack/auth-backend@0.4.29
+  - @checkstack/notification-common@1.2.0
+  - @checkstack/auth-common@0.7.1
+  - @checkstack/signal-common@0.2.4
+  - @checkstack/cache-api@0.3.4
+  - @checkstack/queue-api@0.3.4
+  - @checkstack/cache-utils@0.2.9
+
 ## 1.1.0
 
 ### Minor Changes

package/drizzle/0007_funny_hobgoblin.sql

@@ -0,0 +1,14 @@
+CREATE TYPE "notification_delivery_status" AS ENUM('success', 'failure');--> statement-breakpoint
+CREATE TABLE "notification_delivery_attempts" (
+	"id" uuid PRIMARY KEY DEFAULT gen_random_uuid() NOT NULL,
+	"notification_id" uuid NOT NULL,
+	"strategy_qualified_id" text NOT NULL,
+	"attempted_at" timestamp DEFAULT now() NOT NULL,
+	"status" "notification_delivery_status" NOT NULL,
+	"error_message" text,
+	"duration_ms" integer NOT NULL
+);
+--> statement-breakpoint
+ALTER TABLE "notification_delivery_attempts" ADD CONSTRAINT "notification_delivery_attempts_notification_id_notifications_id_fk" FOREIGN KEY ("notification_id") REFERENCES "notifications"("id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint
+CREATE INDEX "notification_delivery_attempts_notification_idx" ON "notification_delivery_attempts" USING btree ("notification_id");--> statement-breakpoint
+CREATE INDEX "notification_delivery_attempts_attempted_at_idx" ON "notification_delivery_attempts" USING btree ("attempted_at");