@checkstack/healthcheck-backend 0.17.0 → 0.18.0

package/CHANGELOG.md

@@ -1,5 +1,142 @@
 # @checkstack/healthcheck-backend
 
+## 0.18.0
+
+### Minor Changes
+
+- 8d1ef12: ## Anomaly Detection & UI Improvements
+
+  ### Anomaly Detection Enhancements (Phase 2)
+
+  - **`@checkstack/anomaly-backend`**: Implemented background baseline analyzer jobs and anomaly trend deviation detection mechanics.
+  - **`@checkstack/anomaly-common`**: Added new baseline statistical logic and inference rules.
+  - **`@checkstack/anomaly-frontend`**: Added new Anomaly Widget and refactored system detail rendering to be more human-readable.
+  - **`@checkstack/dashboard-frontend`**: Refined the global anomaly widget and fixed hardcoded access gating to render appropriately.
+  - **`@checkstack/healthcheck-backend`**: Connected executor telemetry to the anomaly pipeline.
+  - **`@checkstack/healthcheck-frontend`**: Reconciled baseline display consistency in Drawer and charts.
+
+  ### Notification Identifiers
+
+  - **`@checkstack/incident-backend`**: Resolved system IDs to human-readable System Names within Incident notifications to eliminate ID-only alert content.
+  - **`@checkstack/maintenance-backend`**: Adopted the same resolution strategy for Maintenance notifications to keep parity.
+
+  ### UI Experience
+
+  - **`@checkstack/incident-frontend`**: Fixed the "Back to X" BackLink to properly use `react-router` hook `useNavigate` instead of doing a full application reload.
+  - **`@checkstack/healthcheck-frontend`**: Implemented `useNavigate` for seamless SPA back-linking.
+  - **`@checkstack/integration-frontend`**: Updated connections and delivery logs links to navigate without hard reloads.
+
+- 8d1ef12: ## Per-entity caching with single-flight + safe invalidation across the dashboard hot paths
+
+  ### `@checkstack/cache-api`
+
+  - **Breaking** for backend implementors: `CacheProvider` now requires `deleteByPrefix(prefix: string): Promise<number>` for family-level invalidation. The in-memory provider implements it; downstream providers (Redis, etc.) must add it before upgrading.
+  - `createScopedCache` forwards `deleteByPrefix` and keeps prefixes scoped to the calling plugin.
+
+  ### `@checkstack/cache-utils` (new package)
+
+  High-level read-through caching helpers built on `CacheProvider`:
+
+  - `createCachedScope({ cacheManager, pluginId })` returns a scope with `wrap`, `wrapMany`, `invalidate`, and `invalidatePrefix`.
+  - **Single-flight**: concurrent cache misses for the same key share one loader.
+  - **Per-entity bulk caching** via `wrapMany` so list/bulk RPCs cache by id rather than by the full input shape — overlapping callers share entries and invalidation stays exact.
+  - **Race-safe invalidation** via per-key epoch counters: a loader started before a mutation cannot repopulate the cache with stale data after the mutation invalidates it. The mutation invariant is `db.write → cache.invalidate (await) → signals.emit`.
+  - Cache failures fall through to the loader so a cache outage cannot break reads.
+
+  ### `@checkstack/backend`
+
+  - The internal null `CacheProvider` (used when no cache backend is configured) now implements the new `deleteByPrefix` method as a no-op. Patch bump only — no behavior change for existing callers.
+
+  ### `@checkstack/healthcheck-backend`
+
+  - `getSystemHealthStatus` and `getBulkSystemHealthStatus` now read through a per-system cache (`healthcheck:status:<systemId>`), eliminating N database queries per dashboard refresh for unchanged systems.
+  - Mutation paths (configuration CRUD, system associations, satellite ingest, queue-driven check runs, system/satellite removal hooks) invalidate affected keys before broadcasting their signals so frontend refetches always observe fresh data.
+
+  ### `@checkstack/incident-backend`
+
+  - `listIncidents`, `getIncident`, `getIncidentsForSystem`, and `getBulkIncidentsForSystems` now read through a scoped cache:
+    - per-incident at `incident:<id>`
+    - per-system at `system:<systemId>`
+    - per-filter-shape at `list:<stable-stringify(filters)>` for the few list shapes the dashboard polls
+  - Mutations (`createIncident`, `updateIncident`, `addUpdate`, `resolveIncident`, `deleteIncident`) invalidate the incident, every affected system, and every cached list before broadcasting `INCIDENT_UPDATED`.
+  - The catalog `systemDeleted` cleanup hook drops that system's cached entries.
+
+  ### `@checkstack/maintenance-backend`
+
+  - `listMaintenances`, `getMaintenance`, `getMaintenancesForSystem`, and `getBulkMaintenancesForSystems` use the same per-entity / per-system / per-filter-shape pattern as incidents.
+  - Mutations (`createMaintenance`, `updateMaintenance`, `addUpdate`, `closeMaintenance`, `deleteMaintenance`) invalidate before broadcasting `MAINTENANCE_UPDATED`.
+
+  ### `@checkstack/catalog-backend`
+
+  - Topology reads (`getEntities`, `getSystems`, `getSystem`, `getGroups`, `getSystemGroupIds`) cache under the `entity:` family (25s TTL).
+  - Views (`getViews`) and per-system contacts (`getSystemContacts`) cache in their own families.
+  - System / group / membership mutations drop the entire `entity:` family (every reader joins the same tables); view and contact mutations drop only their respective scopes.
+
+  ### `@checkstack/slo-backend`
+
+  - `listObjectives`, `getObjective`, `getObjectivesForSystem`, and `getBulkObjectivesForSystems` cache results including the expensive `engine.computeStatus` output.
+  - Per-entity caching for the bulk handler so dashboards with overlapping system sets share entries.
+  - Mutations (`createObjective`, `updateObjective`, `deleteObjective`) invalidate before broadcasting `SLO_STATUS_CHANGED`.
+
+  ### `@checkstack/anomaly-backend`
+
+  - New `router-cache.ts` adds a cache scope distinct from the existing detector baseline cache, keyed by stable filter hash.
+  - `getAnomalies` and `getAnomalyBaselines` cache through this scope (15s TTL).
+  - The detector invalidates the router cache before broadcasting `ANOMALY_STATE_CHANGED` on every state transition (suspicious/anomaly/recovered).
+  - Config mutations also invalidate.
+
+  ### `@checkstack/notification-backend`
+
+  - `getUnreadCount`, `getNotifications`, and `getSubscriptions` cache per-user.
+  - `markAsRead`, `deleteNotification`, `notifyUsers`, and `notifyGroups` invalidate every affected user's cache before sending realtime signals to that user.
+  - `subscribe` and `unsubscribe` invalidate the user's subscription cache.
+
+  ### `@checkstack/announcement-backend`
+
+  - `getActiveAnnouncements` caches per-user (or anonymous) and per-`includeDismissed` flag (45s TTL — admin-driven, slowly changing).
+  - `listAllAnnouncements` caches under a single key.
+  - `dismissAnnouncement` only drops that user's cache; `createAnnouncement`, `updateAnnouncement`, `deleteAnnouncement` drop every user's cache before broadcasting `ANNOUNCEMENT_UPDATED`.
+  - The auth `userDeleted` cleanup hook drops that user's cached entries.
+
+### Patch Changes
+
+- Updated dependencies [8d1ef12]
+- Updated dependencies [8d1ef12]
+- Updated dependencies [8d1ef12]
+- Updated dependencies [8d1ef12]
+  - @checkstack/healthcheck-common@0.12.0
+  - @checkstack/common@0.7.0
+  - @checkstack/cache-api@0.2.0
+  - @checkstack/cache-utils@0.2.0
+  - @checkstack/catalog-backend@0.7.0
+  - @checkstack/backend-api@0.13.0
+  - @checkstack/satellite-backend@0.2.16
+  - @checkstack/catalog-common@1.5.2
+  - @checkstack/command-backend@0.1.20
+  - @checkstack/gitops-backend@0.2.4
+  - @checkstack/gitops-common@0.2.1
+  - @checkstack/incident-common@0.4.9
+  - @checkstack/integration-backend@0.1.20
+  - @checkstack/maintenance-common@0.4.11
+  - @checkstack/signal-common@0.1.10
+  - @checkstack/queue-api@0.2.14
+
+## 0.17.1
+
+### Patch Changes
+
+- c4e7560: Fix data integrity, cache invalidation, and mobile UI issues
+
+  - **Centralized mutation cache invalidation**: Every mutation now automatically invalidates its plugin's query cache on success via the shared `createProcedureHook` in `orpc-query.tsx`. This ensures all views stay in sync without requiring individual components to remember manual `invalidateQueries` calls.
+  - **Fixed oRPC query key matching**: Query keys use nested arrays (`[["pluginId"]]`) to correctly match oRPC's `[pathArray, options]` key structure. Fixed the broken flat-string pattern in `SystemBadgeDataProvider`.
+  - **Fixed hourly aggregation duplication**: Added `NULLS NOT DISTINCT` to the `health_check_aggregates` unique constraint so local runs (`source_id = NULL`) correctly conflict-match instead of creating duplicate hourly buckets. Includes a migration to clean up existing duplicates.
+  - **Fixed modal scrolling on mobile**: Added `max-height` + `overflow-y-auto` to `ConfirmationModal`, and refactored `Dialog` from translate-centering to flex-centering with `dvh` units for reliable mobile scroll containment.
+  - @checkstack/catalog-common@1.5.1
+  - @checkstack/incident-common@0.4.8
+  - @checkstack/maintenance-common@0.4.10
+  - @checkstack/satellite-backend@0.2.15
+  - @checkstack/catalog-backend@0.6.1
+
 ## 0.17.0
 
 ### Minor Changes

package/drizzle/0011_fluffy_sphinx.sql

@@ -0,0 +1,16 @@
+DROP INDEX "health_check_aggregates_bucket_unique";--> statement-breakpoint
+
+-- Clean up duplicate local (source_id IS NULL) hourly buckets that were
+-- created due to the missing NULLS NOT DISTINCT clause. Keep the row with
+-- the highest id (most recent insert) for each bucket group.
+DELETE FROM "health_check_aggregates" a
+USING "health_check_aggregates" b
+WHERE a.source_id IS NULL
+  AND b.source_id IS NULL
+  AND a.configuration_id = b.configuration_id
+  AND a.system_id = b.system_id
+  AND a.bucket_start = b.bucket_start
+  AND a.bucket_size = b.bucket_size
+  AND a.id < b.id;--> statement-breakpoint
+
+ALTER TABLE "health_check_aggregates" ADD CONSTRAINT "health_check_aggregates_bucket_unique" UNIQUE NULLS NOT DISTINCT("configuration_id","system_id","bucket_start","bucket_size","source_id");

package/drizzle/meta/0011_snapshot.json

@@ -0,0 +1,441 @@
+{
+  "id": "446eaec3-7894-4425-b7db-00c7761ca83f",
+  "prevId": "743f0b98-66a5-462e-8c35-4b2eb3093010",
+  "version": "7",
+  "dialect": "postgresql",
+  "tables": {
+    "public.health_check_aggregates": {
+      "name": "health_check_aggregates",
+      "schema": "",
+      "columns": {
+        "id": {
+          "name": "id",
+          "type": "uuid",
+          "primaryKey": true,
+          "notNull": true,
+          "default": "gen_random_uuid()"
+        },
+        "configuration_id": {
+          "name": "configuration_id",
+          "type": "uuid",
+          "primaryKey": false,
+          "notNull": true
+        },
+        "system_id": {
+          "name": "system_id",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": true
+        },
+        "bucket_start": {
+          "name": "bucket_start",
+          "type": "timestamp",
+          "primaryKey": false,
+          "notNull": true
+        },
+        "bucket_size": {
+          "name": "bucket_size",
+          "type": "bucket_size",
+          "typeSchema": "public",
+          "primaryKey": false,
+          "notNull": true
+        },
+        "run_count": {
+          "name": "run_count",
+          "type": "integer",
+          "primaryKey": false,
+          "notNull": true
+        },
+        "healthy_count": {
+          "name": "healthy_count",
+          "type": "integer",
+          "primaryKey": false,
+          "notNull": true
+        },
+        "degraded_count": {
+          "name": "degraded_count",
+          "type": "integer",
+          "primaryKey": false,
+          "notNull": true
+        },
+        "unhealthy_count": {
+          "name": "unhealthy_count",
+          "type": "integer",
+          "primaryKey": false,
+          "notNull": true
+        },
+        "latency_sum_ms": {
+          "name": "latency_sum_ms",
+          "type": "integer",
+          "primaryKey": false,
+          "notNull": false
+        },
+        "avg_latency_ms": {
+          "name": "avg_latency_ms",
+          "type": "integer",
+          "primaryKey": false,
+          "notNull": false
+        },
+        "min_latency_ms": {
+          "name": "min_latency_ms",
+          "type": "integer",
+          "primaryKey": false,
+          "notNull": false
+        },
+        "max_latency_ms": {
+          "name": "max_latency_ms",
+          "type": "integer",
+          "primaryKey": false,
+          "notNull": false
+        },
+        "p95_latency_ms": {
+          "name": "p95_latency_ms",
+          "type": "integer",
+          "primaryKey": false,
+          "notNull": false
+        },
+        "aggregated_result": {
+          "name": "aggregated_result",
+          "type": "jsonb",
+          "primaryKey": false,
+          "notNull": false
+        },
+        "tdigest_state": {
+          "name": "tdigest_state",
+          "type": "jsonb",
+          "primaryKey": false,
+          "notNull": false
+        },
+        "source_id": {
+          "name": "source_id",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": false
+        },
+        "source_label": {
+          "name": "source_label",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": false
+        }
+      },
+      "indexes": {},
+      "foreignKeys": {
+        "health_check_aggregates_configuration_id_health_check_configurations_id_fk": {
+          "name": "health_check_aggregates_configuration_id_health_check_configurations_id_fk",
+          "tableFrom": "health_check_aggregates",
+          "tableTo": "health_check_configurations",
+          "columnsFrom": [
+            "configuration_id"
+          ],
+          "columnsTo": [
+            "id"
+          ],
+          "onDelete": "cascade",
+          "onUpdate": "no action"
+        }
+      },
+      "compositePrimaryKeys": {},
+      "uniqueConstraints": {
+        "health_check_aggregates_bucket_unique": {
+          "name": "health_check_aggregates_bucket_unique",
+          "nullsNotDistinct": true,
+          "columns": [
+            "configuration_id",
+            "system_id",
+            "bucket_start",
+            "bucket_size",
+            "source_id"
+          ]
+        }
+      },
+      "policies": {},
+      "checkConstraints": {},
+      "isRLSEnabled": false
+    },
+    "public.health_check_configurations": {
+      "name": "health_check_configurations",
+      "schema": "",
+      "columns": {
+        "id": {
+          "name": "id",
+          "type": "uuid",
+          "primaryKey": true,
+          "notNull": true,
+          "default": "gen_random_uuid()"
+        },
+        "name": {
+          "name": "name",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": true
+        },
+        "strategy_id": {
+          "name": "strategy_id",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": true
+        },
+        "config": {
+          "name": "config",
+          "type": "jsonb",
+          "primaryKey": false,
+          "notNull": true
+        },
+        "collectors": {
+          "name": "collectors",
+          "type": "jsonb",
+          "primaryKey": false,
+          "notNull": false
+        },
+        "interval_seconds": {
+          "name": "interval_seconds",
+          "type": "integer",
+          "primaryKey": false,
+          "notNull": true
+        },
+        "is_template": {
+          "name": "is_template",
+          "type": "boolean",
+          "primaryKey": false,
+          "notNull": false,
+          "default": false
+        },
+        "paused": {
+          "name": "paused",
+          "type": "boolean",
+          "primaryKey": false,
+          "notNull": true,
+          "default": false
+        },
+        "created_at": {
+          "name": "created_at",
+          "type": "timestamp",
+          "primaryKey": false,
+          "notNull": true,
+          "default": "now()"
+        },
+        "updated_at": {
+          "name": "updated_at",
+          "type": "timestamp",
+          "primaryKey": false,
+          "notNull": true,
+          "default": "now()"
+        }
+      },
+      "indexes": {},
+      "foreignKeys": {},
+      "compositePrimaryKeys": {},
+      "uniqueConstraints": {},
+      "policies": {},
+      "checkConstraints": {},
+      "isRLSEnabled": false
+    },
+    "public.health_check_runs": {
+      "name": "health_check_runs",
+      "schema": "",
+      "columns": {
+        "id": {
+          "name": "id",
+          "type": "uuid",
+          "primaryKey": true,
+          "notNull": true,
+          "default": "gen_random_uuid()"
+        },
+        "configuration_id": {
+          "name": "configuration_id",
+          "type": "uuid",
+          "primaryKey": false,
+          "notNull": true
+        },
+        "system_id": {
+          "name": "system_id",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": true
+        },
+        "status": {
+          "name": "status",
+          "type": "health_check_status",
+          "typeSchema": "public",
+          "primaryKey": false,
+          "notNull": true
+        },
+        "latency_ms": {
+          "name": "latency_ms",
+          "type": "integer",
+          "primaryKey": false,
+          "notNull": false
+        },
+        "result": {
+          "name": "result",
+          "type": "jsonb",
+          "primaryKey": false,
+          "notNull": false
+        },
+        "source_id": {
+          "name": "source_id",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": false
+        },
+        "source_label": {
+          "name": "source_label",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": false
+        },
+        "timestamp": {
+          "name": "timestamp",
+          "type": "timestamp",
+          "primaryKey": false,
+          "notNull": true,
+          "default": "now()"
+        }
+      },
+      "indexes": {},
+      "foreignKeys": {
+        "health_check_runs_configuration_id_health_check_configurations_id_fk": {
+          "name": "health_check_runs_configuration_id_health_check_configurations_id_fk",
+          "tableFrom": "health_check_runs",
+          "tableTo": "health_check_configurations",
+          "columnsFrom": [
+            "configuration_id"
+          ],
+          "columnsTo": [
+            "id"
+          ],
+          "onDelete": "cascade",
+          "onUpdate": "no action"
+        }
+      },
+      "compositePrimaryKeys": {},
+      "uniqueConstraints": {},
+      "policies": {},
+      "checkConstraints": {},
+      "isRLSEnabled": false
+    },
+    "public.system_health_checks": {
+      "name": "system_health_checks",
+      "schema": "",
+      "columns": {
+        "system_id": {
+          "name": "system_id",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": true
+        },
+        "configuration_id": {
+          "name": "configuration_id",
+          "type": "uuid",
+          "primaryKey": false,
+          "notNull": true
+        },
+        "enabled": {
+          "name": "enabled",
+          "type": "boolean",
+          "primaryKey": false,
+          "notNull": true,
+          "default": true
+        },
+        "state_thresholds": {
+          "name": "state_thresholds",
+          "type": "jsonb",
+          "primaryKey": false,
+          "notNull": false
+        },
+        "retention_config": {
+          "name": "retention_config",
+          "type": "jsonb",
+          "primaryKey": false,
+          "notNull": false
+        },
+        "satellite_ids": {
+          "name": "satellite_ids",
+          "type": "jsonb",
+          "primaryKey": false,
+          "notNull": false
+        },
+        "include_local": {
+          "name": "include_local",
+          "type": "boolean",
+          "primaryKey": false,
+          "notNull": true,
+          "default": true
+        },
+        "created_at": {
+          "name": "created_at",
+          "type": "timestamp",
+          "primaryKey": false,
+          "notNull": true,
+          "default": "now()"
+        },
+        "updated_at": {
+          "name": "updated_at",
+          "type": "timestamp",
+          "primaryKey": false,
+          "notNull": true,
+          "default": "now()"
+        }
+      },
+      "indexes": {},
+      "foreignKeys": {
+        "system_health_checks_configuration_id_health_check_configurations_id_fk": {
+          "name": "system_health_checks_configuration_id_health_check_configurations_id_fk",
+          "tableFrom": "system_health_checks",
+          "tableTo": "health_check_configurations",
+          "columnsFrom": [
+            "configuration_id"
+          ],
+          "columnsTo": [
+            "id"
+          ],
+          "onDelete": "cascade",
+          "onUpdate": "no action"
+        }
+      },
+      "compositePrimaryKeys": {
+        "system_health_checks_system_id_configuration_id_pk": {
+          "name": "system_health_checks_system_id_configuration_id_pk",
+          "columns": [
+            "system_id",
+            "configuration_id"
+          ]
+        }
+      },
+      "uniqueConstraints": {},
+      "policies": {},
+      "checkConstraints": {},
+      "isRLSEnabled": false
+    }
+  },
+  "enums": {
+    "public.bucket_size": {
+      "name": "bucket_size",
+      "schema": "public",
+      "values": [
+        "hourly",
+        "daily"
+      ]
+    },
+    "public.health_check_status": {
+      "name": "health_check_status",
+      "schema": "public",
+      "values": [
+        "healthy",
+        "unhealthy",
+        "degraded"
+      ]
+    }
+  },
+  "schemas": {},
+  "sequences": {},
+  "roles": {},
+  "policies": {},
+  "views": {},
+  "_meta": {
+    "columns": {},
+    "schemas": {},
+    "tables": {}
+  }
+}

package/drizzle/meta/_journal.json

@@ -78,6 +78,13 @@
       "when": 1776599270689,
       "tag": "0010_colorful_shinobi_shaw",
       "breakpoints": true
+    },
+    {
+      "idx": 11,
+      "version": "7",
+      "when": 1777354405576,
+      "tag": "0011_fluffy_sphinx",
+      "breakpoints": true
     }
   ]
 }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@checkstack/healthcheck-backend",
-  "version": "0.17.0",
+  "version": "0.18.0",
   "type": "module",
   "main": "src/index.ts",
   "checkstack": {
@@ -14,18 +14,20 @@
   },
   "dependencies": {
     "@checkstack/backend-api": "0.12.0",
-    "@checkstack/catalog-backend": "0.5.4",
-    "@checkstack/catalog-common": "1.4.1",
+    "@checkstack/cache-api": "0.1.0",
+    "@checkstack/cache-utils": "0.1.0",
+    "@checkstack/catalog-backend": "0.6.1",
+    "@checkstack/catalog-common": "1.5.1",
     "@checkstack/command-backend": "0.1.19",
     "@checkstack/common": "0.6.5",
     "@checkstack/gitops-backend": "0.2.3",
     "@checkstack/gitops-common": "0.2.0",
     "@checkstack/healthcheck-common": "0.11.0",
-    "@checkstack/incident-common": "0.4.7",
+    "@checkstack/incident-common": "0.4.8",
     "@checkstack/integration-backend": "0.1.19",
-    "@checkstack/maintenance-common": "0.4.9",
+    "@checkstack/maintenance-common": "0.4.10",
     "@checkstack/queue-api": "0.2.13",
-    "@checkstack/satellite-backend": "0.2.13",
+    "@checkstack/satellite-backend": "0.2.15",
     "@checkstack/signal-common": "0.1.9",
     "@hono/zod-validator": "^0.7.6",
     "drizzle-orm": "^0.45.0",

package/src/cache.ts

@@ -0,0 +1,70 @@
+import type { CacheManager } from "@checkstack/cache-api";
+import {
+  createCachedScope,
+  type CachedScope,
+} from "@checkstack/cache-utils";
+import type { Logger } from "@checkstack/backend-api";
+import type { HealthCheckService } from "./service";
+
+/**
+ * TTL chosen to be slightly shorter than the dashboard's 30s `staleTime` so
+ * that signal-driven invalidation almost always wins, and TTL only acts as
+ * a safety net for paths that forget to invalidate.
+ */
+const STATUS_TTL_MS = 15_000;
+
+/**
+ * Per-entity cache helpers for the healthcheck plugin. Wrapping reads
+ * goes through {@link wrapSystemHealthStatus}; mutations should call
+ * {@link invalidateSystem} after the DB write but before emitting any
+ * signal so that frontend refetches see fresh data.
+ */
+export interface HealthCheckCache {
+  /** Read-through cache for one system's health status. */
+  wrapSystemHealthStatus: (
+    systemId: string,
+    loader: () => ReturnType<HealthCheckService["getSystemHealthStatus"]>,
+  ) => ReturnType<HealthCheckService["getSystemHealthStatus"]>;
+
+  /** Invalidate a single system's cached status. */
+  invalidateSystem: (systemId: string) => Promise<void>;
+
+  /**
+   * Invalidate every system's cached status. Used when the change can
+   * affect many systems at once (e.g. a configuration update with
+   * cross-system fan-out, or a plugin reload).
+   */
+  invalidateAllSystems: () => Promise<number>;
+
+  /** Underlying scope, exposed for advanced callers. */
+  scope: CachedScope;
+}
+
+const STATUS_KEY_PREFIX = "status:";
+const statusKey = (systemId: string): string =>
+  `${STATUS_KEY_PREFIX}${systemId}`;
+
+export function createHealthCheckCache({
+  cacheManager,
+  logger,
+}: {
+  cacheManager: CacheManager;
+  logger: Logger;
+}): HealthCheckCache {
+  const scope = createCachedScope({
+    cacheManager,
+    pluginId: "healthcheck",
+    defaultTtlMs: STATUS_TTL_MS,
+    onError: (op, error) => {
+      logger.warn(`healthcheck cache ${op} failed: ${String(error)}`);
+    },
+  });
+
+  return {
+    wrapSystemHealthStatus: (systemId, loader) =>
+      scope.wrap(statusKey(systemId), loader),
+    invalidateSystem: (systemId) => scope.invalidate(statusKey(systemId)),
+    invalidateAllSystems: () => scope.invalidatePrefix(STATUS_KEY_PREFIX),
+    scope,
+  };
+}

package/src/hooks.ts

@@ -42,4 +42,17 @@ export const healthCheckHooks = {
     systemId: string;
     configurationId: string;
   }>("healthcheck.assignment.changed"),
+
+  /**
+   * Emitted when a single health check execution finishes.
+   * This is used by the anomaly detection engine to run the inline fast detector.
+   */
+  checkCompleted: createHook<{
+    systemId: string;
+    configurationId: string;
+    status: string;
+    latencyMs: number | undefined;
+    result: Record<string, unknown> | undefined;
+    timestamp: string;
+  }>("healthcheck.check.completed"),
 } as const;

package/src/index.ts

@@ -35,6 +35,7 @@ import { GitOpsApi } from "@checkstack/gitops-common";
 import { healthCheckHooks } from "./hooks";
 import { registerSearchProvider } from "@checkstack/command-backend";
 import { resolveRoute } from "@checkstack/common";
+import { createHealthCheckCache } from "./cache";
 
 // =============================================================================
 // Integration Event Payload Schemas
@@ -101,6 +102,9 @@ export default createBackendPlugin({
     let gitopsHealthCheckRegistry: HealthCheckRegistry | undefined;
     let gitopsCollectorRegistry: CollectorRegistry | undefined;
     let gitopsQueueManager: QueueManager | undefined;
+    let healthCheckCache:
+      | ReturnType<typeof createHealthCheckCache>
+      | undefined;
 
     const kindRegistry = env.getExtensionPoint(entityKindExtensionPoint);
     registerHealthcheckGitOpsKinds({
@@ -144,6 +148,7 @@ export default createBackendPlugin({
         rpcClient: coreServices.rpcClient,
         queueManager: coreServices.queueManager,
         signalService: coreServices.signalService,
+        cacheManager: coreServices.cacheManager,
       },
       // Phase 2: Register router and setup worker
       init: async ({
@@ -155,6 +160,7 @@ export default createBackendPlugin({
         rpcClient,
         queueManager,
         signalService,
+        cacheManager,
       }) => {
         logger.debug("🏥 Initializing Health Check Backend...");
 
@@ -176,6 +182,13 @@ export default createBackendPlugin({
         // Create gitops client for provenance lock checks
         const gitOpsClient = rpcClient.forPlugin(GitOpsApi);
 
+        // Per-entity status cache shared between the router, queue executor,
+        // and afterPluginsReady cleanup hooks. Mutations / new check results
+        // invalidate by systemId BEFORE emitting signals so frontend
+        // refetches see fresh data.
+        const cache = createHealthCheckCache({ cacheManager, logger });
+        healthCheckCache = cache;
+
         // Setup queue-based health check worker
         await setupHealthCheckWorker({
           db: database,
@@ -188,6 +201,7 @@ export default createBackendPlugin({
           maintenanceClient,
           incidentClient,
           getEmitHook: () => storedEmitHook,
+          cache,
         });
 
         // Setup retention job for tiered storage (daily aggregation)
@@ -203,6 +217,7 @@ export default createBackendPlugin({
           collectorRegistry,
           gitOpsClient,
           getEmitHook: () => storedEmitHook,
+          cache,
         });
         rpc.registerRouter(healthCheckRouter, healthCheckContract);
 
@@ -272,6 +287,7 @@ export default createBackendPlugin({
               `Cleaning up health check associations for deleted system: ${payload.systemId}`,
             );
             await service.removeAllSystemAssociations(payload.systemId);
+            await healthCheckCache?.invalidateSystem(payload.systemId);
           },
           { mode: "work-queue", workerGroup: "system-cleanup" },
         );
@@ -284,6 +300,9 @@ export default createBackendPlugin({
               `Scrubbing satellite ${payload.satelliteId} from health check associations`,
             );
             await service.scrubSatelliteFromAssociations(payload.satelliteId);
+            // Satellite removal can change the includedness of many systems'
+            // checks; invalidate everything since we don't know which.
+            await healthCheckCache?.invalidateAllSystems();
           },
           { mode: "work-queue", workerGroup: "satellite-cleanup" },
         );

package/src/queue-executor.test.ts

@@ -5,6 +5,14 @@ import {
   bootstrapHealthChecks,
   type HealthCheckJobPayload,
 } from "./queue-executor";
+import type { HealthCheckCache } from "./cache";
+
+const passthroughCache: HealthCheckCache = {
+  wrapSystemHealthStatus: (_systemId, loader) => loader(),
+  invalidateSystem: async () => {},
+  invalidateAllSystems: async () => 0,
+  scope: {} as HealthCheckCache["scope"],
+};
 import {
   createMockLogger,
   createMockQueueManager,
@@ -188,6 +196,7 @@ describe("Queue-Based Health Check Executor", () => {
           typeof setupHealthCheckWorker
         >[0]["incidentClient"],
         getEmitHook: () => undefined,
+        cache: passthroughCache,
       });
 
       expect(mockLogger.debug).toHaveBeenCalledWith(
@@ -381,6 +390,7 @@ describe("Queue-Based Health Check Executor", () => {
           typeof setupHealthCheckWorker
         >[0]["incidentClient"],
         getEmitHook: () => undefined,
+        cache: passthroughCache,
       });
 
       // Execute a paused health check

package/src/queue-executor.ts

@@ -31,12 +31,45 @@ import { resolveRoute, type InferClient, extractErrorMessage} from "@checkstack/
 import { HealthCheckService } from "./service";
 import { healthCheckHooks } from "./hooks";
 import { incrementHourlyAggregate } from "./realtime-aggregation";
+import type { HealthCheckCache } from "./cache";
 
 type Db = SafeDatabase<typeof schema>;
 type CatalogClient = InferClient<typeof CatalogApi>;
 type MaintenanceClient = InferClient<typeof MaintenanceApi>;
 type IncidentClient = InferClient<typeof IncidentApi>;
 
+/**
+ * Emit the checkCompleted hook if available.
+ * Extracted to avoid duplicating the hook emission pattern across success/error paths.
+ */
+async function emitCheckCompletedHook({
+  getEmitHook,
+  systemId,
+  configurationId,
+  status,
+  latencyMs,
+  result,
+}: {
+  getEmitHook: () => EmitHookFn | undefined;
+  systemId: string;
+  configurationId: string;
+  status: string;
+  latencyMs: number | undefined;
+  result: Record<string, unknown> | undefined;
+}): Promise<void> {
+  const emitHook = getEmitHook();
+  if (emitHook) {
+    await emitHook(healthCheckHooks.checkCompleted, {
+      systemId,
+      configurationId,
+      status,
+      latencyMs,
+      result,
+      timestamp: new Date().toISOString(),
+    });
+  }
+}
+
 /**
  * Payload for health check queue jobs
  */
@@ -227,6 +260,7 @@ async function executeHealthCheckJob(props: {
   maintenanceClient: MaintenanceClient;
   incidentClient: IncidentClient;
   getEmitHook: () => EmitHookFn | undefined;
+  cache: HealthCheckCache;
 }): Promise<void> {
   const {
     payload,
@@ -239,6 +273,7 @@ async function executeHealthCheckJob(props: {
     maintenanceClient,
     incidentClient,
     getEmitHook,
+    cache,
   } = props;
   const { configId, systemId } = payload;
 
@@ -524,6 +559,10 @@ async function executeHealthCheckJob(props: {
         `Health check ${configId} for system ${systemId} failed: ${finalError}`,
       );
 
+      // Invalidate the per-system status cache before broadcasting so any
+      // frontend that refetches in response to the signal gets fresh data.
+      await cache.invalidateSystem(systemId);
+
       await signalService.broadcast(HEALTH_CHECK_RUN_COMPLETED, {
         systemId,
         systemName,
@@ -603,6 +642,10 @@ async function executeHealthCheckJob(props: {
       `Ran health check ${configId} for system ${systemId}: ${result.status}`,
     );
 
+    // Invalidate the per-system status cache before broadcasting so any
+    // frontend that refetches in response to the signal gets fresh data.
+    await cache.invalidateSystem(systemId);
+
     // Broadcast enriched signal for realtime frontend updates (e.g., terminal feed)
     await signalService.broadcast(HEALTH_CHECK_RUN_COMPLETED, {
       systemId,
@@ -613,6 +656,15 @@ async function executeHealthCheckJob(props: {
       latencyMs: result.latencyMs,
     });
 
+    await emitCheckCompletedHook({
+      getEmitHook,
+      systemId,
+      configurationId: configId,
+      status: result.status,
+      latencyMs: result.latencyMs,
+      result: (result.metadata?.collectors as Record<string, unknown>) ?? undefined,
+    });
+
     // Check if aggregated state changed and notify subscribers
     const newState = await service.getSystemHealthStatus(systemId);
     if (newState.status !== previousStatus) {
@@ -722,6 +774,10 @@ async function executeHealthCheckJob(props: {
       // Use IDs as fallback
     }
 
+    // Invalidate the per-system status cache before broadcasting so any
+    // frontend that refetches in response to the signal gets fresh data.
+    await cache.invalidateSystem(systemId);
+
     // Broadcast enriched failure signal for realtime frontend updates
     await signalService.broadcast(HEALTH_CHECK_RUN_COMPLETED, {
       systemId,
@@ -731,6 +787,15 @@ async function executeHealthCheckJob(props: {
       status: "unhealthy",
     });
 
+    await emitCheckCompletedHook({
+      getEmitHook,
+      systemId,
+      configurationId: configId,
+      status: "unhealthy",
+      latencyMs: undefined,
+      result: undefined,
+    });
+
     // Check if aggregated state changed and notify subscribers
     const newState = await service.getSystemHealthStatus(systemId);
     if (newState.status !== previousStatus) {
@@ -806,6 +871,7 @@ export async function setupHealthCheckWorker(props: {
   maintenanceClient: MaintenanceClient;
   incidentClient: IncidentClient;
   getEmitHook: () => EmitHookFn | undefined;
+  cache: HealthCheckCache;
 }): Promise<void> {
   const {
     db,
@@ -818,6 +884,7 @@ export async function setupHealthCheckWorker(props: {
     maintenanceClient,
     incidentClient,
     getEmitHook,
+    cache,
   } = props;
 
   const queue =
@@ -837,6 +904,7 @@ export async function setupHealthCheckWorker(props: {
         maintenanceClient,
         incidentClient,
         getEmitHook,
+        cache,
       });
     },
     {

package/src/router.test.ts

@@ -3,6 +3,14 @@ import { createHealthCheckRouter } from "./router";
 import { createMockRpcContext } from "@checkstack/backend-api";
 import { call } from "@orpc/server";
 import { z } from "zod";
+import type { HealthCheckCache } from "./cache";
+
+const passthroughCache: HealthCheckCache = {
+  wrapSystemHealthStatus: (_systemId, loader) => loader(),
+  invalidateSystem: async () => {},
+  invalidateAllSystems: async () => 0,
+  scope: {} as HealthCheckCache["scope"],
+};
 
 describe("HealthCheck Router", () => {
   const mockUser = {
@@ -60,6 +68,7 @@ describe("HealthCheck Router", () => {
     collectorRegistry: mockCollectorRegistry as never,
     gitOpsClient: mockGitOpsClient as never,
     getEmitHook: () => undefined,
+    cache: passthroughCache,
   });
 
   it("getStrategies returns strategies from registry", async () => {

package/src/router.ts

@@ -15,6 +15,7 @@ import * as schema from "./schema";
 import { toJsonSchemaWithChartMeta } from "./schema-utils";
 import type { InferClient } from "@checkstack/common";
 import { GitOpsApi } from "@checkstack/gitops-common";
+import type { HealthCheckCache } from "./cache";
 
 /**
  * Creates the healthcheck router using contract-based implementation.
@@ -28,8 +29,9 @@ export const createHealthCheckRouter = (opts: {
   collectorRegistry: CollectorRegistry;
   gitOpsClient: InferClient<typeof GitOpsApi>;
   getEmitHook: () => ((hook: { id: string }, payload: Record<string, unknown>) => Promise<void>) | undefined;
+  cache: HealthCheckCache;
 }) => {
-  const { database, registry, collectorRegistry, getEmitHook } = opts;
+  const { database, registry, collectorRegistry, getEmitHook, cache } = opts;
   // Create service instance once - shared across all handlers
   const service = new HealthCheckService(database, registry, collectorRegistry);
 
@@ -112,7 +114,12 @@ export const createHealthCheckRouter = (opts: {
     }),
 
     createConfiguration: os.createConfiguration.handler(async ({ input }) => {
-      return service.createConfiguration(input);
+      const created = await service.createConfiguration(input);
+      // A new configuration could be associated with any system later; the
+      // safe move is to drop every per-system status cache so the next read
+      // recomputes from fresh DB state.
+      await cache.invalidateAllSystems();
+      return created;
     }),
 
     updateConfiguration: os.updateConfiguration.handler(async ({ input }) => {
@@ -123,22 +130,27 @@ export const createHealthCheckRouter = (opts: {
           message: "Configuration not found",
         });
       }
+      // Configuration update affects every system that has it associated.
+      await cache.invalidateAllSystems();
       return config;
     }),
 
     deleteConfiguration: os.deleteConfiguration.handler(async ({ input }) => {
       await enforceNotGitOpsLocked("Healthcheck", input);
       await service.deleteConfiguration(input);
+      await cache.invalidateAllSystems();
     }),
 
     pauseConfiguration: os.pauseConfiguration.handler(async ({ input }) => {
       await enforceNotGitOpsLocked("Healthcheck", input);
       await service.pauseConfiguration(input);
+      await cache.invalidateAllSystems();
     }),
 
     resumeConfiguration: os.resumeConfiguration.handler(async ({ input }) => {
       await enforceNotGitOpsLocked("Healthcheck", input);
       await service.resumeConfiguration(input);
+      await cache.invalidateAllSystems();
     }),
 
     getSystemConfigurations: os.getSystemConfigurations.handler(
@@ -163,6 +175,7 @@ export const createHealthCheckRouter = (opts: {
         satelliteIds: input.body.satelliteIds,
         includeLocal: input.body.includeLocal,
       });
+      await cache.invalidateSystem(input.systemId);
 
       // If enabling the health check, schedule it immediately
       if (input.body.enabled) {
@@ -195,6 +208,7 @@ export const createHealthCheckRouter = (opts: {
     disassociateSystem: os.disassociateSystem.handler(async ({ input }) => {
       await enforceNotGitOpsLocked("System", input.systemId);
       await service.disassociateSystem(input.systemId, input.configId);
+      await cache.invalidateSystem(input.systemId);
 
       // Notify subscribers that assignments changed
       const emitHook = getEmitHook();
@@ -248,24 +262,30 @@ export const createHealthCheckRouter = (opts: {
     ),
     getSystemHealthStatus: os.getSystemHealthStatus.handler(
       async ({ input }) => {
-        return service.getSystemHealthStatus(input.systemId);
+        return cache.wrapSystemHealthStatus(input.systemId, () =>
+          service.getSystemHealthStatus(input.systemId),
+        );
       },
     ),
 
     getBulkSystemHealthStatus: os.getBulkSystemHealthStatus.handler(
       async ({ input }) => {
+        // Per-entity caching: each system's status is cached individually
+        // and invalidated by id on mutations, so dashboards with overlapping
+        // (but non-identical) system sets share cache entries. See
+        // ./cache.ts for the key/TTL/invalidation contract.
         const statuses: Record<
           string,
           Awaited<ReturnType<typeof service.getSystemHealthStatus>>
         > = {};
-
-        // Fetch health status for each system in parallel
         await Promise.all(
           input.systemIds.map(async (systemId) => {
-            statuses[systemId] = await service.getSystemHealthStatus(systemId);
+            statuses[systemId] = await cache.wrapSystemHealthStatus(
+              systemId,
+              () => service.getSystemHealthStatus(systemId),
+            );
           }),
         );
-
         return { statuses };
       },
     ),
@@ -289,6 +309,15 @@ export const createHealthCheckRouter = (opts: {
     ingestSatelliteResult: os.ingestSatelliteResult.handler(
       async ({ input }) => {
         await service.ingestSatelliteResult(input);
+        // A satellite result writes a new run for this system, so the
+        // cached aggregate status is now stale.
+        await cache.invalidateSystem(input.systemId);
+      },
+    ),
+
+    getRunsForAnalysis: os.getRunsForAnalysis.handler(
+      async ({ input }) => {
+        return service.getRunsForAnalysis(input);
       },
     ),
   });

package/src/schema-utils.ts

@@ -60,6 +60,14 @@ function addHealthResultMeta(
         jsonField["x-chart-unit"] = healthMeta["x-chart-unit"];
       if (healthMeta["x-jsonpath"])
         jsonField["x-jsonpath"] = healthMeta["x-jsonpath"];
+      if (healthMeta["x-anomaly-enabled"] !== undefined)
+        jsonField["x-anomaly-enabled"] = healthMeta["x-anomaly-enabled"];
+      if (healthMeta["x-anomaly-direction"])
+        jsonField["x-anomaly-direction"] = healthMeta["x-anomaly-direction"];
+      if (healthMeta["x-anomaly-sensitivity"] !== undefined)
+        jsonField["x-anomaly-sensitivity"] = healthMeta["x-anomaly-sensitivity"];
+      if (healthMeta["x-anomaly-confirmation-window"] !== undefined)
+        jsonField["x-anomaly-confirmation-window"] = healthMeta["x-anomaly-confirmation-window"];
     }
 
     // Recurse into nested objects and arrays

package/src/schema.ts

@@ -8,7 +8,7 @@ import {
   uuid,
   timestamp,
   primaryKey,
-  uniqueIndex,
+  unique,
 } from "drizzle-orm/pg-core";
 import type {
   StateThresholds,
@@ -182,13 +182,15 @@ export const healthCheckAggregates = pgTable(
     sourceLabel: text("source_label"),
   },
   (t) => ({
-    // Unique constraint includes sourceId for per-region aggregation
-    bucketUnique: uniqueIndex("health_check_aggregates_bucket_unique").on(
+    // Unique constraint includes sourceId for per-region aggregation.
+    // NULLS NOT DISTINCT ensures local runs (sourceId=NULL) correctly
+    // conflict-match instead of creating duplicate rows per hour.
+    bucketUnique: unique("health_check_aggregates_bucket_unique").on(
       t.configurationId,
       t.systemId,
       t.bucketStart,
       t.bucketSize,
       t.sourceId,
-    ),
+    ).nullsNotDistinct(),
   }),
 );

package/src/service.ts

@@ -1091,6 +1091,53 @@ export class HealthCheckService {
     return assignments;
   }
 
+  async getRunsForAnalysis(props: {
+    startDate: Date;
+    limitPerAssignment?: number;
+  }) {
+    const { startDate, limitPerAssignment = 200 } = props;
+
+    // Fetch all active associations
+    const activeAssignments = await this.db
+      .select({
+        systemId: systemHealthChecks.systemId,
+        configurationId: systemHealthChecks.configurationId,
+      })
+      .from(systemHealthChecks)
+      .where(eq(systemHealthChecks.enabled, true));
+
+    const results = [];
+
+    // For each assignment, fetch the recent runs
+    // This endpoint is used specifically for cross-plugin background jobs
+    for (const assignment of activeAssignments) {
+      const runs = await this.db
+        .select({
+          result: healthCheckRuns.result,
+        })
+        .from(healthCheckRuns)
+        .where(
+          and(
+            eq(healthCheckRuns.systemId, assignment.systemId),
+            eq(healthCheckRuns.configurationId, assignment.configurationId),
+            gte(healthCheckRuns.timestamp, startDate),
+          ),
+        )
+        .orderBy(desc(healthCheckRuns.timestamp))
+        .limit(limitPerAssignment);
+
+      results.push({
+        systemId: assignment.systemId,
+        configurationId: assignment.configurationId,
+        runs: runs.map((r) => ({
+          result: r.result,
+        })),
+      });
+    }
+
+    return results;
+  }
+
   /**
    * Ingest a health check result from a satellite.
    * Stores the run with source attribution (sourceId + sourceLabel)
@@ -1116,7 +1163,9 @@ export class HealthCheckService {
       sourceLabel,
     } = props;
 
-    const resultRecord = result ? { ...result } as Record<string, unknown> : {};
+    const resultRecord = result
+      ? ({ ...result } as Record<string, unknown>)
+      : {};
 
     await this.db.insert(healthCheckRuns).values({
       configurationId: configId,