@checkstack/backend 0.8.1 → 0.9.0

package/CHANGELOG.md

@@ -1,5 +1,285 @@
 # @checkstack/backend
 
+## 0.9.0
+
+### Minor Changes
+
+- 50e5f5f: Add `bunx @checkstack/scripts dev` — a local Checkstack dev server for
+  plugin authors that runs from the plugin's own repo without a monorepo
+  checkout.
+
+  Mechanics:
+
+  - The dev command spawns `core/backend`'s production entry as a child
+    process with three env vars wired in:
+    - `CHECKSTACK_DEV_PLUGIN_PATH=<cwd>` — backend skips filesystem
+      discovery and imports the plugin at this path as a manual plugin.
+    - `CHECKSTACK_DEV_EXTRA_PLUGIN_PATHS=<JSON array>` — additional
+      backend plugins co-loaded as manual plugins. The dev command walks
+      the plugin under dev's `package.json#dependencies` recursively to
+      discover every `@checkstack/*-backend` package and pass their
+      module paths through. Auto-includes
+      `@checkstack/queue-memory-backend` +
+      `@checkstack/cache-memory-backend` when no other queue/cache
+      provider is in the dep graph, so `coreServices.queueManager` /
+      `coreServices.cacheManager` always have a registered strategy on
+      boot. Without this co-loading, plugins that depend on
+      `healthcheck-backend`, `notification-backend`, etc. would hit
+      unregistered services and the boot would deadlock.
+    - `CHECKSTACK_DEV_AUTH=true` — backend registers a synthetic
+      `AuthService` that auto-grants every registered access rule.
+      Refused when `NODE_ENV=production` so accidental misuse is loud.
+  - A file watcher under the plugin's `./src` triggers a full backend
+    restart (debounced) on save. Bun's startup is sub-second for a single
+    plugin, so the loop stays tight.
+  - For frontend plugins (or bundle primaries with a `-frontend`
+    sibling), the dev command additionally spawns a Vite dev server on
+    port 5173 (configurable via `--frontend-port`). Vite serves
+    `core/frontend`'s new `dev-main.tsx` shell — the same App.tsx,
+    loadPlugins(), ThemeProvider, etc. that ship in production. The
+    plugin module is mounted via a `virtual:checkstack-dev-plugin` alias
+    Vite resolves at config time. React Fast Refresh works for component
+    edits.
+  - On boot, the dev command validates the plugin's `package.json`
+    against the same `installPackageMetadataSchema` the runtime install
+    pipeline uses, so missing required fields fail fast.
+
+  Reuses 100% of the production boot code path — no parallel dev backend
+  to drift from. New code surfaces:
+
+  - `core/backend/src/services/dev-auth.ts` — the synthetic auth service.
+    Inert unless `CHECKSTACK_DEV_AUTH=true`.
+  - `core/scripts/src/commands/dev-server.ts` — the CLI command.
+  - `core/scripts/src/commands/dev-deps-resolver.ts` — pure function that
+    walks the plugin's deps and resolves the co-load set; covered by 8
+    unit tests.
+  - `core/scripts/src/commands/dev-frontend.ts` — Vite spawn helper.
+  - `core/frontend/src/dev-main.tsx` — frontend dev-shell entry.
+
+  `@checkstack/scripts` now depends on `@checkstack/backend`,
+  `@checkstack/frontend`, `@checkstack/frontend-api`, `@checkstack/ui`,
+  `vite`, and `@vitejs/plugin-react` so a `bunx` invocation pulls in
+  everything needed for the dev server in one shot.
+
+  Replaces the previous "three patterns" plugin-development guide with a
+  single `bun run dev` workflow.
+
+  A new ESLint rule branch in `no-extraneous-runtime-deps` ignores
+  `virtual:` module specifiers (resolved by bundler aliases at runtime,
+  not installed from npm).
+
+  Scaffold templates updated for one-click compatibility — `bun run create`
+  now produces plugin packages that pass the dev-server's
+  `installPackageMetadataSchema` gate and ship `dev` / `pack` scripts plus
+  `@checkstack/scripts` in devDependencies, so a freshly scaffolded plugin
+  runs `bun run dev` without any further file edits. Required metadata
+  (`description`, `author`, `license: "Elastic-2.0"`, `checkstack.pluginId`)
+  is filled in by the scaffold; `@checkstack/scripts plugin-pack
+--validate-only` accepts the rendered package.json directly. Templates
+  also reformatted from one-line JSON-in-handlebars to readable
+  multi-line.
+
+  New scaffold tests in `core/scripts/src/templates.test.ts` render each
+  template type and assert: dev-server validation passes, `dev` script
+  present (backend/frontend), `pack` script present, `@checkstack/scripts`
+  in devDependencies.
+
+  In addition, the new `dev-internals.ts`, `dev-lifecycle.ts`,
+  `dev-deps-resolver.ts`, and refactored `dev-frontend.ts` ship 58
+  unit tests covering arg parsing, package.json validation, backend
+  entry resolution, frontend-spawn decision, child env construction,
+  the debounce watcher, the spawn → restart → shutdown lifecycle (with
+  hard-kill SIGKILL fallback), the dev-auth service, and the bundle
+  sibling resolver — all driven through injectable seams so no real
+  process / Postgres / Vite is needed at test time.
+
+- 50e5f5f: Runtime plugin system: install + uninstall plugins from npm, GitHub releases
+  (including private GitHub Enterprise instances), or tarball uploads at
+  runtime, with multi-package bundles, dependency-derived compatibility checks,
+  multi-instance coordination via a Postgres artifact store, and
+  single-coordinator destructive cleanup.
+
+  Highlights:
+
+  - New `PluginSource` discriminated union and `PluginInstaller` /
+    `PluginInstallerRegistry` interfaces in `@checkstack/backend-api`. The
+    GitHub variant accepts an optional `apiBaseUrl` so deployments backed by
+    GitHub Enterprise can install from `https://ghe.example.com/api/v3`
+    instead of `api.github.com`.
+  - New `installPackageMetadataSchema` (Zod) in `@checkstack/common` validates
+    every plugin's `package.json` at install time. Required fields: `name`,
+    `version`, `description`, `author`, `license`, `checkstack.type`,
+    `checkstack.pluginId`. Optional: `checkstack.bundle`,
+    `checkstack.usageInstructions`, `checkstack.allowInstallScripts`.
+  - New `pluginManagerContract` in `@checkstack/pluginmanager-common` with
+    `list`, `previewInstall`, `install`, `previewUninstall`, `uninstall`, and
+    `events` procedures.
+  - New `@checkstack/pluginmanager-frontend` admin UI: installed-plugins list
+    with per-row uninstall (typed-confirmation modal, schema/configs/cascade
+    toggles), install page with NPM / Tarball Upload / GitHub Release tabs
+    (Catalog tab disabled — coming soon), and an events page surfacing the
+    install/uninstall audit log.
+  - New `bunx @checkstack/scripts plugin-pack` CLI for plugin authors —
+    per-package mode produces an npm-shaped tarball; `--bundle` mode produces
+    an outer tarball containing every sibling declared in
+    `package.json#checkstack.bundle`. Published to npm so external authors
+    can `bunx` it directly without a workspace checkout.
+  - Compatibility derived from `package.json#dependencies` ranges
+    (`semver.satisfies` against the platform's loaded `@checkstack/*`
+    versions) — no separate `compatibility` field.
+  - Multi-instance: originator persists artifacts + `plugins` rows + broadcasts
+    install/uninstall; receiving instances do in-process register/unregister
+    only. Destructive ops (drop schema, delete plugin_configs, delete
+    artifacts, delete `plugins` rows) run exactly once on the originator.
+  - Fresh-instance bootstrap: `loadPlugins()` hydrates any
+    `is_uninstallable=true` plugin missing from `node_modules` from the
+    artifact store before normal Phase 1 register.
+  - New schema: `plugin_artifacts` (tarball storage), `plugin_install_events`
+    (audit/error log). `plugins` extended with `version`, `metadata`,
+    `source`, `bundle_id`, `is_primary`. Local plugin sync now writes
+    `version` from each plugin's `package.json` so the admin UI shows real
+    versions instead of `—`.
+  - Tarball-upload endpoint (`POST /api/pluginmanager/upload-tarball`) for
+    the install UI; access-gated by `pluginmanager.plugin.manage`.
+  - Plugin Manager menu link added to the user menu (main grid, alongside
+    Profile / Notification Settings / etc.).
+
+  Cross-cutting changes:
+
+  - Backend request/response logging now flows through `rootLogger` (winston)
+    instead of `hono/logger`. 5xx responses include the response body inline
+    so swallowed early-return errors are visible in the log.
+  - The `/api/:pluginId/*` dispatcher now logs which core service is missing
+    or which `pluginId` had no metadata when it 500s.
+  - New `registerCorePluginMetadata` on `PluginManager` for core routers
+    (like the plugin manager itself) that need their metadata visible to the
+    RPC dispatcher without going through the full plugin lifecycle.
+  - ESLint: `unicorn/no-null` is now disabled globally. Drizzle distinguishes
+    between `null` (writes a real SQL NULL) and `undefined` (skip the column
+    on insert), so treating them as interchangeable produced latent bugs at
+    the persistence boundary. The bulk of the patch-bumped packages above
+    reflect lint-fix touches that landed when this rule was relaxed.
+  - Workspace-wide license normalization to `Elastic-2.0` (matches
+    `LICENSE.md`). Every `package.json` in the workspace now declares the
+    same SPDX identifier; the patch bumps capture this.
+
+  Plugin packages (every `plugins/*`): added a `pack` npm script
+  (`bunx @checkstack/scripts plugin-pack`), mirrored each plugin's
+  `pluginId` from `plugin-metadata.ts` into `package.json#checkstack.pluginId`
+  so install-time validation passes, stubbed any missing required metadata
+  fields (`description`, `author`, `license`), and added
+  `checkstack.bundle` to multi-package plugin primaries (telegram, rcon, ssh,
+  jira, queue-bullmq, queue-memory, cache-memory).
+
+  Breaking changes:
+
+  - The legacy single-method `PluginInstaller` interface (`install(packageName)`)
+    is removed. Callers must use `coreServices.pluginInstallerRegistry`.
+  - The old `pluginAdminContract` and `createPluginAdminRouter` are removed.
+    Replaced by `pluginManagerContract` in `@checkstack/pluginmanager-common`
+    and `createPluginManagerRouter` in `core/backend`.
+  - `@checkstack/test-utils-backend` no longer exports
+    `createMockPluginInstaller` / `MockPluginInstaller` (the legacy interface
+    it shimmed is gone).
+
+  Note: bumps are limited to `minor` (for packages with new public API
+  surface) and `patch` (for downstream consumers, license normalization,
+  and lint fixes). No `major` bumps despite the `PluginInstaller` removal —
+  the legacy interface had no third-party consumers in the wild before this
+  runtime plugin system landed, and the contract surface is the same shape
+  modulo the rename.
+
+### Patch Changes
+
+- Updated dependencies [50e5f5f]
+  - @checkstack/auth-common@0.6.5
+  - @checkstack/backend-api@0.15.0
+  - @checkstack/common@0.8.0
+  - @checkstack/drizzle-helper@0.0.5
+  - @checkstack/pluginmanager-common@0.2.0
+  - @checkstack/queue-api@0.2.18
+  - @checkstack/signal-backend@0.2.3
+  - @checkstack/api-docs-common@0.1.11
+  - @checkstack/cache-api@0.2.4
+  - @checkstack/signal-common@0.2.1
+
+## 0.8.2
+
+### Patch Changes
+
+- 302cd3f: fix: resilient startup routing + /health and /ready endpoints
+
+  Three fixes that together eliminate startup-race errors during boot and
+  hot-reload, plus a new readiness API for plugins.
+
+  1. **TrieRouter swap (root cause).** Hono's default `SmartRouter` freezes
+     its matcher on the first request — any later `app.add()` throws
+     `MESSAGE_MATCHER_IS_ALREADY_BUILT`. Plugins register routes during
+     `init()` (and at runtime via `loadSinglePlugin`), so an early request
+     during boot would silently lock the matcher with only the module-load
+     routes, and every later route registration would fail. The backend
+     now uses `TrieRouter`, which is incremental — routes can be added at
+     any time, including after thousands of requests have been served.
+     This also future-proofs runtime plugin install.
+
+  2. **Init gating + fail-loud.** Non-bypass requests now `await` an
+     `initPromise` (with a 30s timeout that returns 503 + Retry-After) so
+     no traffic reaches Hono before plugins finish registering routes.
+     Init failures crash the process via `process.exit(1)` so docker/k8s
+     restart cleanly instead of silently serving a half-initialized
+     backend.
+
+  3. **`/assets/*` fall-through.** The production frontend asset handler
+     now calls `next()` instead of `c.notFound()` on miss, so
+     plugin-asset routes registered later (`/assets/plugins/:pluginName/*`)
+     actually get a chance to match.
+
+  ### New: platform endpoints under `/.checkstack/*`
+
+  - `GET /.checkstack/health` — liveness, always 200 once the process is up.
+  - `GET /.checkstack/ready` — readiness, 503 until init completes and all
+    critical probes pass; 200 otherwise. Returns `{ ready, checks: [...] }`
+    with per-probe status, message/error and duration.
+
+  The leading `.checkstack/` prefix namespaces platform-level endpoints
+  away from plugin `/api/*`, runtime frontend assets, and the SPA wildcard,
+  leaving room for additional operator endpoints in the future.
+
+  ### New: plugin readiness API
+
+  Plugins can contribute readiness probes via the new
+  `coreServices.readinessRegistry` service:
+
+  ```ts
+  registerInit({
+    deps: { readiness: coreServices.readinessRegistry },
+    async init({ readiness }) {
+      readiness.register({
+        name: "queue.connected",
+        critical: true,
+        check: async () => ({
+          ok: pool.isConnected(),
+          message: pool.isConnected() ? undefined : "queue pool not connected",
+        }),
+      });
+    },
+  });
+  ```
+
+  Probes run in parallel, throwing probes are reported as `ok: false`,
+  and non-critical probes don't block readiness.
+
+- Updated dependencies [302cd3f]
+  - @checkstack/backend-api@0.14.1
+  - @checkstack/cache-api@0.2.3
+  - @checkstack/queue-api@0.2.17
+  - @checkstack/signal-backend@0.2.2
+  - @checkstack/api-docs-common@0.1.10
+  - @checkstack/auth-common@0.6.4
+  - @checkstack/common@0.7.0
+  - @checkstack/drizzle-helper@0.0.4
+  - @checkstack/signal-common@0.2.0
+
 ## 0.8.1
 
 ### Patch Changes

package/drizzle/0001_slim_mordo.sql

@@ -0,0 +1,34 @@
+CREATE TABLE "plugin_artifacts" (
+	"id" uuid PRIMARY KEY DEFAULT gen_random_uuid() NOT NULL,
+	"plugin_name" text NOT NULL,
+	"version" text NOT NULL,
+	"bundle_id" uuid,
+	"tarball" text NOT NULL,
+	"content_hash" text NOT NULL,
+	"size_bytes" integer NOT NULL,
+	"created_at" timestamp DEFAULT now() NOT NULL
+);
+--> statement-breakpoint
+CREATE TABLE "plugin_install_events" (
+	"id" uuid PRIMARY KEY DEFAULT gen_random_uuid() NOT NULL,
+	"plugin_name" text,
+	"bundle_id" uuid,
+	"action" text NOT NULL,
+	"phase" text NOT NULL,
+	"status" text NOT NULL,
+	"source" jsonb,
+	"error" text,
+	"instance_id" text NOT NULL,
+	"user_id" text,
+	"created_at" timestamp DEFAULT now() NOT NULL
+);
+--> statement-breakpoint
+ALTER TABLE "plugins" ADD COLUMN "version" text DEFAULT '' NOT NULL;--> statement-breakpoint
+ALTER TABLE "plugins" ADD COLUMN "metadata" jsonb DEFAULT '{}'::jsonb NOT NULL;--> statement-breakpoint
+ALTER TABLE "plugins" ADD COLUMN "source" jsonb;--> statement-breakpoint
+ALTER TABLE "plugins" ADD COLUMN "bundle_id" uuid;--> statement-breakpoint
+ALTER TABLE "plugins" ADD COLUMN "is_primary" boolean DEFAULT false NOT NULL;--> statement-breakpoint
+CREATE UNIQUE INDEX "plugin_artifacts_name_version_idx" ON "plugin_artifacts" USING btree ("plugin_name","version");--> statement-breakpoint
+CREATE INDEX "plugin_artifacts_content_hash_idx" ON "plugin_artifacts" USING btree ("content_hash");--> statement-breakpoint
+CREATE INDEX "plugin_install_events_name_created_idx" ON "plugin_install_events" USING btree ("plugin_name","created_at");--> statement-breakpoint
+CREATE INDEX "plugin_install_events_status_created_idx" ON "plugin_install_events" USING btree ("status","created_at");