@checkstack/queue-api 0.2.18 → 0.3.0

package/CHANGELOG.md

@@ -1,5 +1,127 @@
 # @checkstack/queue-api
 
+## 0.3.0
+
+### Minor Changes
+
+- aa89bc5: Replace the bespoke `registerInfrastructureTab()` registry with a standard
+  slot-extension contract (`InfrastructureTabsSlot` from
+  `@checkstack/infrastructure-common`). Plugins now contribute infrastructure
+  tabs via `createSlotExtension`, depending only on the slot owner.
+
+  The slot system in `@checkstack/frontend-api` gains a second type parameter
+  on `createSlot<TContext, TMetadata>` so extensions can declare typed static
+  metadata at registration time (label, icon, access rules, ordering for the
+  infrastructure tab bar). A new `useSlotExtensions(slot)` hook returns typed
+  extensions and subscribes to plugin lifecycle changes.
+
+  Each tab body now stacks a **Runtime** sub-section (live state, read-only)
+  on top of a **Configuration** sub-section (settings, gated by `canUpdate`).
+
+  **Queue runtime panel.** Surfaces aggregated counts (pending / processing /
+  completed / failed) plus three sub-tabs of recent jobs: **Active**, **Recent
+  failed** (with the failure message), and **Recent completed** (with
+  duration). Job payloads are deliberately not surfaced — they may carry
+  secrets and need a separate manage-access gate to be shown.
+
+  To support this, `Queue<T>` gains a required `listJobs(opts)` method
+  returning `JobSummary[]` (no payloads), and `QueueStats` gains a
+  `scope: "instance" | "cluster"` field. The in-memory queue keeps rolling
+  ring buffers (200 entries) for completed/failed history and tracks active
+  jobs by id; BullMQ uses native `getJobs`. `QueueManager.listJobs` aggregates
+  across queues and sorts (most-recent-first for terminal states, FIFO for
+  active/waiting/delayed).
+
+  **Cache runtime panel.** Lists the top N entries by size (or by recency) so
+  operators can debug a cache filling up. Values are deliberately omitted —
+  PII / secret risk. Backends opt in via an optional `listEntries?` method on
+  `CacheProvider`; non-supporting backends return `{ supported: false }` and
+  the UI renders a "not supported by this backend" hint. The in-memory cache
+  implements it using its existing per-entry byte tracking.
+
+  `CacheStats` also gains `scope: "instance" | "cluster"`.
+
+  **Multi-instance scope warning.** A new `<InstanceScopeBanner>` component in
+  `@checkstack/ui` renders a yellow banner above any runtime panel whose
+  backend reports `scope: "instance"` — i.e. in-memory queue or cache running
+  in a horizontally scaled deployment. The banner explains the metrics are
+  local to the responding replica and recommends switching to a clustered
+  backend (Redis-backed queue / cache) for cluster-wide visibility.
+
+  **Bug fix — stable cache provider proxy.** `CacheManagerImpl.getProvider()`
+  now returns a single stable proxy that delegates to whatever provider is
+  currently active. Previously, consumers of `createCachedScope` (and any
+  direct `cacheManager.getProvider()` caller) captured the active provider
+  reference at plugin-init time. After any `setActiveBackend` call — including
+  saving the same memory config in the new Cache tab, which reconstructs the
+  in-memory cache — those scopes wrote to an orphaned old provider while the
+  runtime panel read stats from the new (empty) one, making the runtime panel
+  appear to report 0 keys. With the proxy, all consumers share a single stable
+  identity and writes always land in the active provider.
+
+  **Bytes tracking on the in-memory cache.** `InMemoryCache.getStats().sizeBytes`
+  now returns a running approximation (UTF-8 bytes of the key plus
+  `v8.serialize(value).byteLength`, with a JSON fallback) that's kept in sync
+  across all eviction paths. Treat the number as a sanity gauge; it doesn't
+  include `Map` per-entry overhead.
+
+  **Pagination.** Both `Queue<T>.listJobs` and `CacheProvider.listEntries?`
+  are offset-paginated. Inputs gain an `offset: number`; outputs change to
+  `{ items, total: number | null, hasMore: boolean }`. `total` is nullable
+  so backends that can't compute it cheaply still paginate via `hasMore`.
+  The UI uses the existing `<Pagination>` component with a 25-row default
+  page size. `QueueManager.listJobs` aggregates by over-fetching
+  `[0, offset+limit)` per queue, merge-sorting, then slicing the window —
+  optimal for the single-queue case, acceptable for the multi-queue case
+  within the UI's reasonable page-depth bounds. BullMQ uses native offset
+  ranges via `getJobs(types, start, end)` plus `getJobCounts` for `total`.
+
+  **Pending tab.** The Queue runtime panel exposes a virtual `"pending"`
+  state (waiting ∪ delayed, FIFO). It's now the default sub-tab, since
+  "what's queued up?" is the most common question. Per-row state is shown
+  when viewing the combined list.
+
+  **Recurring schedules visible under Pending.** Cron- and interval-based
+  recurring jobs (e.g. healthchecks) are surfaced under Pending/Delayed
+  between fires, with a `nextRunAt` countdown column and a "(recurring)"
+  label. `JobSummary` gains optional `nextRunAt: Date` and `recurring:
+boolean` fields. The in-memory queue synthesises these rows from its
+  `recurringJobs` registry; BullMQ already materialises the next fire of
+  each scheduler as a delayed job and we now surface its trigger time and
+  the `repeatJobKey`-derived `recurring` flag.
+
+  **Bug fix — drop hook emits with no listeners.** `EventBus.emit` no
+  longer enqueues a job when zero listeners (distributed or instance-local)
+  are registered for the hook. Previously, hooks like
+  `core.plugin.initialized` — emitted on every plugin init but subscribed
+  to by nothing in the core repo — accumulated one waiting job per emit
+  forever. The in-memory queue's `processNext` short-circuits when there
+  are zero consumer groups, so its post-loop cleanup never ran for these
+  orphaned jobs. The fix drops the emit at the source and logs a debug
+  line. Note: in distributed deployments using a Redis-backed queue, this
+  means a subscriber on another replica won't receive an event if no
+  replica that emits it has a local listener. Plugins needing cross-process
+  delivery must register their listener on every replica that should
+  receive the hook.
+
+  **Breaking notes (treated as minor under beta semantics)**:
+
+  - `@checkstack/infrastructure-common` removes `registerInfrastructureTab`
+    and `getInfrastructureTabs`; former callers must register an extension
+    into `InfrastructureTabsSlot`.
+  - `@checkstack/queue-api`'s `Queue<T>` interface requires the new
+    `listJobs(opts)` method returning `ListJobsResult` (paginated). Both
+    bundled queue backends (memory, BullMQ) are updated; out-of-tree
+    implementations will need to add it.
+  - `QueueStats` and `CacheStats` add a required `scope` field.
+  - `CacheProvider.listEntries?` (when implemented) now returns
+    `ListEntriesResult` instead of `CacheEntrySummary[]`.
+  - `JobState` adds a `"pending"` variant.
+
+### Patch Changes
+
+- @checkstack/backend-api@0.15.1
+
 ## 0.2.18
 
 ### Patch Changes

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@checkstack/queue-api",
-  "version": "0.2.18",
+  "version": "0.3.0",
   "license": "Elastic-2.0",
   "checkstack": {
     "type": "tooling"
@@ -8,12 +8,12 @@
   "type": "module",
   "main": "src/index.ts",
   "dependencies": {
-    "@checkstack/backend-api": "0.14.1",
+    "@checkstack/backend-api": "0.15.0",
     "zod": "^4.0.0"
   },
   "devDependencies": {
-    "@checkstack/tsconfig": "0.0.6",
-    "@checkstack/scripts": "0.1.2"
+    "@checkstack/tsconfig": "0.0.7",
+    "@checkstack/scripts": "0.3.0"
   },
   "scripts": {
     "typecheck": "tsgo -b",

package/src/queue-plugin.ts

@@ -1,5 +1,11 @@
 import { z } from "zod";
-import type { Queue, QueueStats, RecurringSchedule } from "./queue";
+import type {
+  Queue,
+  QueueStats,
+  RecurringSchedule,
+  ListJobsOptions,
+  ListJobsResult,
+} from "./queue";
 import type { Migration, Logger } from "@checkstack/backend-api";
 
 export interface QueuePlugin<Config = unknown> {
@@ -89,6 +95,12 @@ export interface QueueManager {
    */
   getAggregatedStats(): Promise<QueueStats>;
 
+  /**
+   * Aggregate {@link Queue.listJobs} across all queues, sorted into a single
+   * paginated list. Used by the Infrastructure runtime panel.
+   */
+  listJobs(opts: ListJobsOptions): Promise<ListJobsResult>;
+
   /**
    * List all recurring jobs across all queues.
    * Used for migration preview.

package/src/queue.ts

@@ -152,6 +152,24 @@ export interface Queue<T = unknown> {
    * Get queue statistics
    */
   getStats(): Promise<QueueStats>;
+
+  /**
+   * List jobs in a particular state for the Infrastructure runtime panel,
+   * paginated.
+   *
+   * Implementations MUST:
+   * - Return summaries only (no payloads).
+   * - Honour `opts.offset` and `opts.limit`.
+   * - For `completed` / `failed`: return the most recent first.
+   * - For `active` / `waiting` / `delayed`: return the oldest first
+   *   (FIFO order — what users typically want to see).
+   * - Set `total` to a cheap exact count when possible, or `null` when not.
+   * - Set `hasMore` correctly so the UI can disable the "next" control.
+   *
+   * Backends without affordable history (e.g. fire-and-forget transports)
+   * may return `{ items: [], total: 0, hasMore: false }` for terminal states.
+   */
+  listJobs(opts: ListJobsOptions): Promise<ListJobsResult>;
 }
 
 export interface QueueStats {
@@ -163,4 +181,87 @@ export interface QueueStats {
    * Number of active consumer groups
    */
   consumerGroups: number;
+  /**
+   * Whether these stats reflect the local process only (`"instance"`) or
+   * the entire deployment cluster (`"cluster"`). Backends that share state
+   * across replicas (e.g. BullMQ on Redis) report `"cluster"`. Backends
+   * that hold state in-process (e.g. the in-memory queue) report
+   * `"instance"`; in horizontally scaled deployments, each replica returns
+   * its own numbers.
+   */
+  scope: "instance" | "cluster";
+}
+
+/**
+ * Lifecycle states a job can be in. Mirrors BullMQ for consistency, with
+ * one virtual addition: `"pending"` is the union of `"waiting"` and
+ * `"delayed"`. Most operators think of "pending work" as both — the
+ * Infrastructure UI exposes it as a single tab, and the existing
+ * `QueueStats.pending` already aggregates both counts.
+ */
+export type JobState =
+  | "pending"
+  | "waiting"
+  | "active"
+  | "delayed"
+  | "completed"
+  | "failed";
+
+/**
+ * Inspection summary for a single job. Carries no payload — payloads can
+ * contain secrets, so they're surfaced (if at all) behind a separate
+ * manage-access RPC. The Infrastructure runtime panel uses these summaries
+ * for the Active / Failed / Completed sub-tables.
+ */
+export interface JobSummary {
+  id: string;
+  /** Optional job/queue name for display. */
+  name?: string;
+  state: JobState;
+  /** Time the job was first enqueued. */
+  enqueuedAt: Date;
+  /** Time the job started processing, if applicable. */
+  startedAt?: Date;
+  /** Time the job finished (completed or failed), if applicable. */
+  finishedAt?: Date;
+  /** Attempts made so far (1-based on completion, 0 before first run). */
+  attempts: number;
+  /** Failure message, only set when `state === "failed"`. */
+  failedReason?: string;
+  /**
+   * For recurring schedules and delayed jobs: the absolute time of the
+   * next planned execution. Lets the UI render a "Next run" column for
+   * pending/delayed rows so cron-scheduled work (e.g. healthchecks) is
+   * visible between fires.
+   */
+  nextRunAt?: Date;
+  /**
+   * Whether this row represents a recurring schedule (cron / interval)
+   * rather than a one-shot enqueue. Used by the UI to label such rows
+   * distinctly so operators can tell scheduled work from ad-hoc work.
+   */
+  recurring?: boolean;
+}
+
+/**
+ * Options for {@link Queue.listJobs}. Offset-based pagination — backends
+ * that natively use cursors (BullMQ, Redis SCAN, etc.) translate internally.
+ */
+export interface ListJobsOptions {
+  state: JobState;
+  /** Zero-based offset into the result set. */
+  offset: number;
+  /** Maximum summaries to return. Implementations should cap at a reasonable upper bound. */
+  limit: number;
+}
+
+/**
+ * Paginated result for {@link Queue.listJobs}.
+ */
+export interface ListJobsResult {
+  items: JobSummary[];
+  /** Total job count for the requested state, or `null` if the backend can't compute it cheaply. */
+  total: number | null;
+  /** Whether more items exist past `offset + items.length`. */
+  hasMore: boolean;
 }