@checkstack/backend 0.15.0 → 0.16.0

package/CHANGELOG.md

@@ -1,5 +1,172 @@
 # @checkstack/backend
 
+## 0.16.0
+
+### Minor Changes
+
+- 9dcc848: Add the AI platform: a transport-agnostic tool spine, an OAuth Authorization Server + read-only MCP server, a propose/apply flow with audit log, a streaming in-app chat agent, per-conversation permission modes, per-integration spend caps, and user-scoped tool authorization.
+
+  Two new packages, `@checkstack/ai-common` (the `AiTool` contract, `read`/`mutate`/`destructive` effect classification, the `ai.*` access rules, the OpenAI-compatible connection shape, and the wire contracts) and `@checkstack/ai-backend` (the tool registry, extension points, principal-to-tool resolver, shared zod-to-JSON-Schema serializer, and all transports). The OpenAI-compatible integration provider registers through the existing integration provider extension point, so its API key is stored in the Secrets Vault and configured in the generic Connections UI.
+
+  What ships:
+
+  - Tool spine and extension points: `aiToolExtensionPoint.registerTool` (hand-authored composite tools) and `aiToolProjectionExtensionPoint.expose` (opt-in projections of existing oRPC procedures). Authorization mirrors `autoAuthMiddleware` exactly - a tool is surfaced only when every `requiredAccessRules` entry is satisfied, so a scope-narrowed principal can only ever see fewer tools.
+  - OAuth + MCP: Checkstack can act as its own OAuth 2.1 Authorization Server (authorization code + PKCE, consent screen, Dynamic Client Registration) and expose a read-only MCP server over Streamable HTTP at `/api/ai/mcp`. Off by default, enabled by the admin `ai.mcp-oauth` setting. A Bearer OAuth-token branch is added to the auth strategy; token scopes are intersected live with the bound user's access rules on every call. A shared-Postgres rate limiter throttles the DCR endpoint per client IP. `getMcpOAuthSettings` / `setMcpOAuthSettings` contracts added to `@checkstack/auth-common`. A minimal OAuth consent page (`/auth/oauth-consent`) renders the requesting client and scopes.
+  - Propose/apply + audit: a transport-agnostic two-step service - `propose` re-checks authz, runs the tool's `dryRun` without mutating, and returns a single-use proposal token (the `proposed` audit row IS the token store, 10-minute TTL, atomic single-use); `apply` re-parses the server-stored payload, re-checks authz, and atomically commits. The `ai_tool_calls` audit table records every call across both transports with a SHA-256 args hash (never raw arguments) and stamps who proposed and who applied. An `ai.toolCalled` event carries metadata only.
+  - In-app chat: a server-side, provider-agnostic Vercel AI SDK agent loop (OpenAI, Azure, OpenRouter, Ollama, vLLM, LM Studio, ...). The model provider is built on the backend from the integration credentials, so the API key never leaves the backend. The loop offers only resolver-allowed tools, auto-runs read tools (re-entering the live router as the logged-in user) and routes mutating / destructive tools through propose/apply. Durable conversation persistence (`ai_conversations`, `ai_messages`, owner-scoped RPCs) plus a streaming chat UI with a confirm-card component and per-integration model picker.
+  - Per-conversation permission mode (Claude-Code-style approve/auto), a durable `permission_mode` column on `ai_conversations` (default `approve`). `read` always auto-runs in both modes; `mutate` inherits the mode (auto-applies server-side in `auto`, confirm-carded in `approve`); `destructive` ALWAYS requires the human `applyTool` in both modes. Security invariant (structural + tested): the mode is consulted only on the `mutate` branch, so no `(effect, mode)` pair routes a destructive tool to auto-apply.
+  - Per-integration LLM spend cap (optional `spendCap` = `tokenBudget` + `windowMinutes`, default OFF). Spend is tracked in a shared-Postgres `ai_spend` ledger; enforcement is a rolling-window SUM run before each turn (HTTP 429 over budget). Per-principal tool rate-limit budgets are a rolling COUNT over `ai_tool_calls`, enforced on both transports. An absent / empty / incomplete `spendCap` is treated as "no cap" rather than rejected.
+  - Full tool-call replay: `ai_messages.model_messages` (jsonb) persists the canonical AI-SDK `ResponseMessage[]` per turn and replays them verbatim on the next turn; legacy rows fall back to text-only replay.
+  - Enforced no-secret-leak scrubbing: `appendMessage` runs `scrubContent` on every write, redacting credential-shaped keys and high-confidence credential values; a canary regression test asserts injected secrets are stripped. A hardening test suite asserts no secret appears in any AI-surface DTO and that handler-side authz holds when the model misbehaves.
+  - Provider correctness: the chat provider uses `@ai-sdk/openai-compatible`'s `chatModel` (plain `/chat/completions`), so OpenAI-compatible gateways (OpenRouter, DeepSeek, Ollama, vLLM) no longer reject turns with `invalid_prompt`; `@ai-sdk/openai` is removed.
+
+  BREAKING CHANGES:
+
+  - The `AiTool` contract (`@checkstack/ai-common`) gained a `TRpc` type parameter, and both `dryRun` and `execute` now receive a USER-SCOPED `rpcClient` arg bound to the originating user. Every plugin procedure a tool calls re-enters the live router AS THAT USER, so handler-side authorization (access rules AND per-resource/team scope) is enforced exactly as a direct UI/RPC call - closing a prior privilege-escalation where tools captured a trusted service client at construction. A hand-authored tool MUST resolve its plugin client from this per-call arg and MUST NOT capture a trusted service client at factory scope. Tool factories that previously took `{ rpcClient }` should drop that parameter.
+  - `AiToolProjectionExtensionPoint.expose` no longer takes a second `pluginMetadata` argument; the owning metadata lives on `input.sourcePluginMetadata`. Callers must drop the second argument.
+
+  State and scale: conversations, messages, the audit log, proposal tokens, the rate-limit counter, and the spend ledger all live in shared Postgres, so every pod answers identically and the agent loop is resumable on any pod. The only pod-local state is the live MCP connection registry (bookkeeping, never a source of truth). Cross-pod conversation readback, the spend cap, and the tool budget are verified by env-gated two-pod integration tests.
+
+  This is a beta minor.
+
+- 9dcc848: Automations now run as a configured service account, removing implicit god-mode from the dispatch path.
+
+  BREAKING: every automation must declare a `runAs` application (service account). Previously every automation action ran as the trusted service client, bypassing all access-rule, per-resource, and team-scope checks - so an automation could touch any team's data. Now each automation runs as a bounded `application` principal, and every data-access call an action makes is authorized exactly as that identity. An automation with no `runAs` fails to run with a clear error rather than falling back to the trusted client; legacy automations must be assigned a service account before they run again.
+
+  What changed:
+
+  - New top-level field `runAs` on automations (a `run_as_application_id` column + create/update inputs; `AutomationSchema.runAs`). Required on create; GitOps sets it via the `run-as` metadata label.
+  - A new `coreServices.rpcClientAs(applicationId)` mints a short-lived, backend-signed app-principal token; the auth service resolves it LIVE to an `application` principal (reusing `enrichApplicationPrincipal`), so it flows through full `autoAuthMiddleware` enforcement. The dispatch engine threads this client into every action's `execute` as the required `context.rpcClient`.
+  - Bind authority (anti-escalation): a user may only bind an application whose access rules are a subset of their own (`isApplicationBindable`); `getBindableApplications` lists only bindable apps, and the create/update handlers enforce the check.
+  - `notification.sendTransactional` moves from service-only to access-gated (`notification.send`, a new access rule), so an automation's `runAs` can call the built-in `notify_user` / `notification.send` actions; trusted services still bypass via short-circuit.
+  - A "Run as (Service Account)" picker in the automation editor, populated from `getBindableApplications` (server-side filtered to bindable apps), seeding from the loaded `runAs` on edit and passing it into create + update. First-class teaching UX: an inline info banner, a blocked Save with an inline hint until one is chosen, and an empty state linking to the Applications admin + docs when none are bindable.
+
+  State and scale: `runAs` resolution is a pure read over shared tables; the app-principal token is self-contained and verified statelessly, so the per-run client is correct under horizontal scale.
+
+  This is a beta minor.
+
+- 9dcc848: Add environments as a first-class catalog primitive, with per-environment health-check fan-out, config templating, per-environment reactive health, and script run-context exposure.
+
+  - Catalog primitive: an environment is a sibling of groups - a named, instance-global record carrying free-form custom fields (baseUrl, region, tier, ...) that any system can belong to many-to-many. New `environments` + `systems_environments` tables, `EnvironmentSchema` + create/update schemas, `EntityService` environment CRUD and membership joins, RPC endpoints gated by a new `catalogAccess.environment` access rule, a GitOps `Environment` kind + `System.environments` extension, and frontend management (an `EnvironmentEditor`, an Environments management panel, and a per-system environment picker). The Environments card's Add/Edit/Delete affordances are gated on `catalogAccess.environment.manage`.
+  - Per-environment fan-out: run identity becomes `(systemId, configurationId, environmentId)`. Runs, aggregates, and state transitions gain a nullable `environmentId`. The health-check assignment gains an `environmentIds` selector with three modes (All / Specific / None; `null` and `[]` are distinct). The queue executor resolves the effective environment set via the catalog `resolveSystemEnvironments` read and executes one isolated run per environment.
+  - Config templating: a new `x-templatable` config-field marker renders a string field through the template engine at execute time, against `{ environment, check, system }`. A shared `renderTemplatableConfig` and a `renderTemplatePreview` helper (re-exported from `@checkstack/template-engine`) keep editor previews identical to the run-time render. The HTTP collector's `url`, `headers[].value`, and `body` are templatable, rendered per environment (the strategy client build moves inside the per-env loop); the `url`'s `.url()` validation moves post-render. Secrets resolve before templating; a field marked both secret and `x-templatable` is rejected at plugin load. `DynamicForm` shows a live "Preview" line, and the catalog `EnvironmentPreviewPicker` ("Preview as: <environment>") drives it in the collector editor (only when the schema has a templatable field).
+  - Script run-context: `CollectorRunContext` gains an optional `environment` field (`{ id, name, fields }`, metadata only). Shell collectors receive `CHECKSTACK_ENV_ID` / `_NAME` / `CHECKSTACK_ENV_<FIELD>` vars; inline TS collectors read `globalThis.context.environment`; the editor test panel mirrors both. The env-less path is unchanged.
+  - Per-environment reactive health (see BREAKING below), env-keyed read/write paths, env-qualified serialization locks, an optional `trigger.payload.environmentId`, per-environment isolation, and an `ENVIRONMENT_RESOLUTION_FAILED` signal when catalog resolution degrades to a single env-less run.
+
+  BREAKING CHANGES: the reactive `health` entity's id-shape and cardinality change. It now encodes two views: per-environment (id `"<systemId>::<environmentId>"`) and a system rollup (id `"<systemId>"`, the worst status across environments + env-less runs). The rollup PRESERVES the pre-existing system-level contract - dashboards, status badges, and automations referencing health by `systemId` keep working without re-authoring - but the entity's contract surface changed (new id-shape, higher cardinality, new payload field), so it is flagged breaking. `getBulkHealthState` parses env-qualified ids and keys results by the original id.
+
+  State and scale: membership and custom fields live only in catalog Postgres and are re-read every tick via the cross-plugin RPC; env-keyed health reads from shared `health_check_runs` / aggregates / transitions (compute-on-read). Every pod resolves the same effective set and the same per-environment health. No pod-local environment state.
+
+  Also: `unwrapSchema` in `zod-config.ts` loops instead of single-pass-stripping so multi-layer wrappers (`.optional().default()`) still resolve `x-templatable` meta. The env-less `{{ environment.* }}` run notice logs at `debug` (a legitimate recurring configuration), while the post-render HTTP `.url()` check still fails a genuinely-broken empty render with a clear "Rendered URL is invalid" error.
+
+  This is a beta minor.
+
+- 9dcc848: Move primary navigation into a left sidebar, and serve the user guide in-app.
+
+  Feature navigation (a ~20-item user-menu dropdown) now lives in a persistent left sidebar (a slide-over drawer on mobile), grouped by section with the active route highlighted; the user menu keeps only account actions. A route opts into the sidebar with new `nav` metadata (`{ group, icon, label?, order?, accessRule? }`) on its registration, co-located with path + access + title. The sidebar filters entries with the same access check as page guards. `@checkstack/common` gains `isAccessRuleSatisfied` and a centralized set of in-app doc slugs (`APP_DOC_SLUGS` + `docsPath`, with a test asserting each resolves to a real docs page); `@checkstack/auth-frontend` exports `useAccessRules`.
+
+  The backend now serves the Astro Starlight docs build same-origin at `/checkstack/*` (the same artifact deployed to GitHub Pages), so the user guide is available inside the app including for self-hosted / air-gapped installs (served verbatim, no rebuild, no link rewriting; from `CHECKSTACK_DOCS_DIST`, before the SPA catch-all, degrading gracefully when absent; the Docker image builds and ships `docs/dist`; Vite proxies `/checkstack` in dev). The "Docs" link is a shell-owned external sidebar entry under the Documentation group (book icon), opening `/checkstack/user-guide/` in a new tab; the group renders even when no plugin route contributes to it.
+
+  BREAKING (plugin authors): `UserMenuItemsSlot` is no longer the way to add navigation - registering a top user-menu item no longer surfaces it anywhere. Add `nav` to the page's route instead. `UserMenuItemsBottomSlot` (account items) is unchanged. All bundled plugins have been migrated.
+
+  This is a beta minor.
+
+- 9dcc848: Stop the spurious "Plugin unknown is not using new API. Skipping." startup warning.
+
+  `@checkstack/signal-backend` is a host-consumed library (the backend imports `SignalServiceImpl` and `createWebSocketHandler` directly), but its `package.json` declared `checkstack.type: "backend"`, so plugin discovery inserted it as a runtime backend plugin and the loader tried to read a default `register()` export it does not have - logging the offending package as the literal `unknown`.
+
+  - Reclassify `@checkstack/signal-backend` to `checkstack.type: "tooling"` (like `@checkstack/backend-api`), so it is no longer discovered or registered as a backend plugin. No runtime behavior change - the SignalService and WebSocket handler are still instantiated and registered directly by the host backend.
+  - Harden the loader's skip diagnostic so it can never render `unknown`: it resolves the offending plugin by its database-row package name (falling back to the on-disk path) and tells operators to set `checkstack.type` to `"tooling"` for host-consumed libraries.
+
+  This is a beta minor.
+
+- 9dcc848: Align workspace dependency versions and migrate React Router to v7.
+
+  BREAKING CHANGES (React Router v7): All frontend packages now depend on `react-router-dom@^7.16.0`. Previously the workspace declared four divergent ranges (`^6.20.0`, `^6.22.0`, `^7.1.1`, `^7.14.2`), which resolved both `react-router@6` and `react-router@7` into a single bundle. Everything is now unified on v7. The public imports the app uses (`BrowserRouter`, `Routes`, `Route`, `Link`, `NavLink`, `MemoryRouter`, `useNavigate`, `useParams`, `useSearchParams`, `useLocation`) are unchanged between v6 and v7, so no source rewrites were required - but any out-of-tree plugin still on react-router v6 should upgrade to v7 (see the React Router v6 -> v7 upgrade guide) to share the host's single router instance via the import map.
+
+  Other unified ranges (no API change): `react` -> `^18.3.1`, the `@orpc/*` family (`contract`, `server`, `client`, `tanstack-query`, `openapi`, `zod`) -> `^1.14.4`, and `better-auth` -> `^1.6.13`.
+
+  Removed the pre-rename `@orpc/react-query` leftover from `@checkstack/frontend-api`; its `createRouterUtils` / `RouterUtils` / `ProcedureUtils` now come from `@orpc/tanstack-query` (the package already in use).
+
+  Stale in-range runtime deps pulled up to current published versions: `hono` `^4.12.23`, `@tanstack/react-query` (+devtools) `^5.100.14`, `date-fns` `^4.4.0`, `jose` `^6.2.3`, `tar` `^7.5.16`, `semver` `^7.8.1`, `@xyflow/react` `^12.11.0`.
+
+### Patch Changes
+
+- 9dcc848: Assorted bug fixes and small hardening across the platform.
+
+  - announcement-backend: `updateAnnouncement` now invalidates the active-announcements and admin-list caches (it was missing the `invalidateAllActive` / `invalidateListAll` calls), so an edited announcement no longer stays stale up to the 45s TTL.
+  - anomaly-backend: anomaly/drift state transitions (confirmations, recoveries, self-resolutions) now log at `debug` instead of info/warn - they are already surfaced via the `ANOMALY_STATE_CHANGED` signal, so logging them louder just added noise; genuine failure paths stay `warn`.
+  - backend: the `/api/:pluginId/*` dispatcher now populates `requestHeaders` on the per-request RPC context, so a handler that re-enters the router as the originating user (e.g. an AI tool's user-scoped client) can forward the caller's session cookie / bearer - previously the loopback failed with "Authentication required". Guarded by a real end-to-end integration test. The HTTP server idle timeout is also raised (default 255s, configurable via `CHECKSTACK_SERVER_IDLE_TIMEOUT_SECONDS`, clamped 0-255, reset on each streamed chunk) so long AI chat SSE turns are not severed mid-stream.
+  - backend: a request for an unknown plugin id (`/api/<unknown>/...`) now returns `404 Not Found` instead of `500` (and logs at warn, not error, since it is a client request) - an unknown _procedure_ on a known plugin already 404'd. The in-app docs namespace `/checkstack/*` now serves Starlight's own `404.html` with a real 404 status for a missing doc, instead of falling through to the SPA catch-all and 200-ing the app shell. Both guarded by tests.
+  - automation-common: remove polynomial-time backtracking from `toShellEnvKey`'s underscore-trim (CodeQL `js/polynomial-redos`); a negative look-behind anchors the trailing run, keeping the trim linear.
+  - common + script-packages-common: the pure transport-safe sandbox-policy schema (`sandboxPolicySchema` and its sub-schemas + inferred types) moved to `@checkstack/common` (the neutral base), removing two inverted deps that existed only to reach the shape; `@checkstack/backend-api` continues to re-export it. The schema is no longer exported from `@checkstack/script-packages-common`. Pure refactor, no behavior change.
+  - catalog-backend: reject duplicate system names (a `CONFLICT` on create/rename, enforced by a pre-write check AND a new DB unique index on `systems.name`, migration 0004 which first resolves pre-existing duplicates by suffixing).
+  - catalog-frontend: detail-page cleanups (use `<NotFound />` not `<AccessDenied />` on the not-found branch, a readable key/value metadata list via `normalizeMetadata`, runtime locale via `formatDate`); and stop the browse view re-rendering on every health report (adopt a new statuses report only when a value actually changed, via `healthStatusesEqual`, so rows stay stable and interactive).
+  - healthcheck-backend: fix the daily-rollup retention step failing with an `ON CONFLICT` mismatch (SQLSTATE 42P10) after `environmentId` joined the `health_check_aggregates` unique constraint - the rollup now groups by (day, environmentId, sourceId) and uses a single exported conflict-target constant (`DAILY_AGGREGATE_CONFLICT_TARGET`) kept in lock-step with the schema by a unit test.
+  - automation-frontend: the service-account picker's "Learn more" links are now absolute URLs to the deployed Astro docs site (they 404ed as in-app relative paths). The Monaco script editor double-init crash is fixed (serialized cold init, a guarded `monacoGuard` accessor, theme/type effects gated on `apiReady`).
+  - auth-frontend: bound the desktop user-menu popover height (`max-h-[var(--radix-popover-content-available-height)]` + `overflow-y-auto`) so it no longer clips on short viewports, and fold the standalone `Account > Profile` item into a focusable name/email header (`profileHref` on `UserMenu`); the now-empty `Account` group no longer renders.
+  - satellite-frontend: picked up via the sidebar-nav migration (account-only user menu).
+
+  (Related UI fixes - the Monaco editor following the app theme, the `DynamicOptionsField` no-flash fix, the shared `Spinner`, GFM tables, and the user-menu popover bound - land their `@checkstack/ui` bump in the UI/perf changesets where `@checkstack/ui` is already minored.)
+
+  This is a beta patch.
+
+- 9dcc848: Fix the external/published-install dev loop and scaffolded-plugin first-boot (the #251 published-tarball integration lane).
+
+  Backend:
+
+  - `CHECKSTACK_DEV_AUTH=true` now actually takes effect for plugin APIs: dev auth is registered as a FACTORY (not a plain instance) so `ServiceRegistry.get()` reaches it instead of resolving the real auth factory first - previously every plugin API request under dev auth 401ed.
+  - `CHECKSTACK_DEV_AUTH=true` no longer fatally crashes boot: dev-auth passes through real S2S tokens in `authenticate()` and mints a real plugin-scoped service token in `getCredentials()`, registered per plugin so each carries its own id - so boot-time backend-to-backend calls (e.g. `notification.registerSubscriptionSpec`) are accepted. A `PORT` env override is added to the backend entry point.
+  - A dev-loaded plugin's Drizzle migrations now run on boot: `loadPlugins` accepts an optional `manualPluginPaths` map and the dev path supplies `CHECKSTACK_DEV_PLUGIN_PATH` so the plugin's `drizzle/` migrations run (previously manual plugins booted with no tables).
+
+  dev-server:
+
+  - The Vite frontend dev server now starts from a published install: `@checkstack/frontend` is resolved from a candidate list (the plugin first, then dev-server's own install), a `checkstack.bundle`-referenced `-frontend` sibling is picked up by scanning sibling dirs, and resolution failures yield a clean `undefined`/`Error` instead of Bun's non-`Error` throw (which had surfaced as "An error occurred").
+  - The dev shell is now styled from a published install: `@checkstack/frontend` moves `tailwindcss`, `autoprefixer`, and `tailwindcss-animate` to dependencies and exports a `./tailwind-preset` subpath; the dev server assembles the PostCSS chain from that preset + autoprefixer and injects the plugin-under-dev's source globs into Tailwind's `content` (so a plugin author's custom utility classes compile), degrading gracefully if the toolchain can't be loaded. (`@checkstack/frontend` now declares an `exports` field, a BREAKING change for any consumer importing an undeclared subpath; nothing in the platform imports it as a module, and a `./*` passthrough preserves filesystem-style subpath access. The `@checkstack/frontend` minor bump for this lands in the version-alignment / frontend-bundle-perf changesets.)
+  - `--help` output corrected from the stale `checkstack-scripts dev` to `checkstack-dev`, with a note that the binary ships in `@checkstack/dev-server`.
+
+  scripts (scaffold):
+
+  - The standalone backend template now ships a generated `drizzle/0000_init` migration creating the example `items` table, and `drizzle.config.ts` `out` points at `./drizzle` (the folder the loader reads), so a scaffolded plugin serves its API on first `bun install && bun run dev` instead of 500ing with "relation \"items\" does not exist".
+  - The `common` template's `definePluginMetadata({ pluginId })` now renders the bare base name (not `<base>-common`), matching the backend's `checkstack.pluginId` and `/api/<pluginId>/*` route, fixing "Plugin metadata not found in registry".
+
+  This is a beta patch.
+
+- 9dcc848: Input-validation and error-mapping hardening found by a fuzzing pass against the built container.
+
+  - backend: a Postgres driver error caused by bad client input no longer surfaces as a `500`. The `/api` and `/rest` dispatchers now map the relevant SQLSTATE classes to the correct status - `22P02`/`22003`/`22001`/`22007` (malformed/out-of-range/over-long/bad-date value), `23502`/`23503`/`23514` (missing/dangling/check-failed) to `400`, and `23505` (unique violation) to `409` - and log them at `warn` (client mistake), not `error`. The client-facing message is generic so column/constraint names are never leaked; genuine unknown faults still log at `error` and 500. Previously a `where id = $1` with a non-uuid `$1` (or an over-long string, or a foreign-key miss in `addSystemToGroup`) reached the driver and 500'd, making routine probing look like a server outage and burying real 500s.
+  - slo-common: **fixes a stored cluster-wide DoS.** `windowDays` was accepted up to `2^53`, but the SLO engine derives window boundaries with `Date(now - windowDays * 86_400_000)` - a large value overflows past the max representable `Date` and yields `Invalid Date`. That objective committed fine, then every subsequent read of the system's objectives threw `RangeError: Invalid time value` during serialization (a 500 readable by anyone with SLO read access, on any pod). `windowDays` is now bounded to 1..3650 days at the contract, the GitOps `kind: SLO` spec, and the update path via a single shared `SloWindowDaysSchema`, so the poison row can never be created.
+  - slo-common + healthcheck-common: SLO `getDailySnapshots` and the healthcheck history endpoints (`getHistory`, `getDetailedHistory`, `getAggregatedHistory`, `getDetailedAggregatedHistory`, `getRunsForAnalysis`) declared their `startDate`/`endDate` params as `z.date()`, which a `/rest/...` string param can never satisfy - so those endpoints 400'd on the entire REST surface. They now use `z.coerce.date()`, accepting both the REST string shape and the native RPC `Date`.
+  - healthcheck-common: `intervalSeconds` was `z.number().min(1)` with no `.int()` and no upper bound, so a fractional or out-of-range value reached the DB and failed at insert (the column is a 32-bit int). It is now `.int().min(1).max(2_592_000)` (1 second .. 30 days), applied to both create and update (the update schema is the create partial).
+  - catalog-common: system/group/environment names were bare `z.string()` (environment was `.min(1)` only), so empty, whitespace-only, and 100KB+ names reached the DB - the huge ones surfaced as 500s when parameter binding blew up. Names are now `trim().min(1).max(200)` via a shared schema.
+
+    **BREAKING:** `getSystemContacts` is now `userType: "authenticated"` (was `"public"`). System contacts carry PII (user id, name, email); the public read leaked them to anonymous status-page visitors. Anonymous callers now receive `401` for this one endpoint; the system detail page already renders "No contacts assigned" for anonymous viewers, so the UI degrades gracefully. All other catalog reads remain public.
+
+  - catalog-frontend: the system detail page skips the `getSystemContacts` request entirely for anonymous viewers (it would now `401`) and falls back to the empty state.
+
+  This is a beta release: the breaking contact-visibility change ships as a minor bump per the beta versioning policy, not a major.
+
+- Updated dependencies [9dcc848]
+- Updated dependencies [9dcc848]
+- Updated dependencies [9dcc848]
+- Updated dependencies [9dcc848]
+- Updated dependencies [9dcc848]
+- Updated dependencies [9dcc848]
+- Updated dependencies [9dcc848]
+- Updated dependencies [9dcc848]
+- Updated dependencies [9dcc848]
+- Updated dependencies [9dcc848]
+- Updated dependencies [9dcc848]
+  - @checkstack/auth-common@0.8.0
+  - @checkstack/backend-api@0.21.0
+  - @checkstack/signal-backend@0.3.0
+  - @checkstack/common@0.13.0
+  - @checkstack/cache-api@0.3.9
+  - @checkstack/queue-api@0.3.9
+  - @checkstack/api-docs-common@0.1.16
+  - @checkstack/pluginmanager-common@0.2.5
+  - @checkstack/signal-common@0.2.6
+
 ## 0.15.0
 
 ### Minor Changes

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@checkstack/backend",
-  "version": "0.15.0",
+  "version": "0.16.0",
   "license": "Elastic-2.0",
   "checkstack": {
     "type": "backend"
@@ -16,27 +16,27 @@
   "dependencies": {
     "@checkstack/api-docs-common": "0.1.15",
     "@checkstack/auth-common": "0.7.2",
-    "@checkstack/backend-api": "0.18.0",
+    "@checkstack/backend-api": "0.20.0",
     "@checkstack/common": "0.12.0",
     "@checkstack/drizzle-helper": "0.0.5",
-    "@checkstack/cache-api": "0.3.6",
-    "@checkstack/queue-api": "0.3.6",
-    "@checkstack/signal-backend": "0.2.10",
+    "@checkstack/cache-api": "0.3.8",
+    "@checkstack/queue-api": "0.3.8",
+    "@checkstack/signal-backend": "0.2.12",
     "@checkstack/signal-common": "0.2.5",
     "@checkstack/pluginmanager-common": "0.2.4",
     "@hono/zod-validator": "^0.7.6",
-    "@orpc/client": "^1.13.14",
-    "@orpc/contract": "^1.13.14",
-    "@orpc/openapi": "^1.13.2",
-    "@orpc/server": "^1.13.2",
-    "@orpc/zod": "^1.13.2",
-    "better-auth": "^1.4.7",
+    "@orpc/client": "^1.14.4",
+    "@orpc/contract": "^1.14.4",
+    "@orpc/openapi": "^1.14.4",
+    "@orpc/server": "^1.14.4",
+    "@orpc/zod": "^1.14.4",
+    "better-auth": "^1.6.13",
     "drizzle-orm": "^0.45.0",
-    "hono": "^4.12.14",
-    "jose": "^6.1.3",
+    "hono": "^4.12.23",
+    "jose": "^6.2.3",
     "pg": "^8.11.0",
-    "semver": "^7.6.3",
-    "tar": "^7.4.3",
+    "semver": "^7.8.1",
+    "tar": "^7.5.16",
     "winston": "^3.19.0",
     "zod": "^4.2.1"
   },
@@ -46,7 +46,7 @@
     "@types/semver": "^7.5.0",
     "@checkstack/tsconfig": "0.0.7",
     "@checkstack/scripts": "0.3.4",
-    "@checkstack/test-utils-backend": "0.1.31",
+    "@checkstack/test-utils-backend": "0.1.33",
     "drizzle-kit": "^0.31.10"
   }
 }

package/src/index.ts

@@ -35,7 +35,6 @@ import {
   type WebSocketData,
 } from "@checkstack/signal-backend";
 import type {
-  AuthService,
   BackendPlugin,
   WsConnectionHandlers,
 } from "@checkstack/backend-api";
@@ -295,6 +294,61 @@ app.get("/.well-known/jwks.json", async (c) => {
   return c.json(jwks);
 });
 
+// Serve the in-app user guide: the SAME Astro Starlight static build deployed
+// to GitHub Pages, mounted same-origin at `/checkstack/*` (the docs build has
+// `base: "/checkstack"` and all cross-links are `/checkstack/...`, so it works
+// verbatim - no rebuild, no link rewriting). Registered BEFORE the SPA
+// catch-all below so doc paths win; gated on the dist existing so a deployment
+// without docs degrades gracefully (the path 404s as before). Note: this is
+// `/checkstack/*`, distinct from the platform's `/.checkstack/*` (leading dot).
+const docsDistPath =
+  process.env.CHECKSTACK_DOCS_DIST ??
+  path.resolve(import.meta.dir, "../../../docs/dist");
+if (fs.existsSync(docsDistPath)) {
+  rootLogger.info(`📚 Serving in-app user guide from: ${docsDistPath}`);
+  const serveDocsFile = async (c: Context, filePath: string) => {
+    const file = Bun.file(filePath);
+    const content = await file.arrayBuffer();
+    c.header("Content-Type", file.type);
+    return c.body(content);
+  };
+  app.get("/checkstack/*", async (c, next) => {
+    // Map `/checkstack/<rest>` -> `<docsDist>/<rest>`, resolving a directory or
+    // trailing-slash/pretty URL to its `index.html` (Starlight emits
+    // `<slug>/index.html`). Fall through on a miss so non-doc `/checkstack/...`
+    // paths aren't swallowed.
+    const rel = c.req.path.replace(/^\/checkstack\/?/, "");
+    if (rel.includes("..")) return next();
+
+    let filePath = path.join(docsDistPath, rel);
+    const isDir =
+      rel === "" ||
+      rel.endsWith("/") ||
+      (fs.existsSync(filePath) && fs.statSync(filePath).isDirectory());
+    if (isDir) {
+      filePath = path.join(docsDistPath, rel, "index.html");
+    } else if (!fs.existsSync(filePath)) {
+      // Pretty URL with no trailing slash (e.g. /checkstack/user-guide).
+      const asIndex = path.join(docsDistPath, rel, "index.html");
+      if (fs.existsSync(asIndex)) filePath = asIndex;
+    }
+
+    if (fs.existsSync(filePath) && fs.statSync(filePath).isFile()) {
+      return serveDocsFile(c, filePath);
+    }
+    // Unknown `/checkstack/*` path: this namespace IS the docs site (the
+    // platform's own endpoints live under `/.checkstack/`, with a dot), so serve
+    // Starlight's own 404 page with a real 404 status instead of falling through
+    // to the SPA catch-all (which would 200 the app shell for a missing doc).
+    const notFoundPage = path.join(docsDistPath, "404.html");
+    if (fs.existsSync(notFoundPage)) {
+      c.status(404);
+      return serveDocsFile(c, notFoundPage);
+    }
+    return c.text("Not Found", 404);
+  });
+}
+
 // Production: Serve frontend static files when CHECKSTACK_FRONTEND_DIST is set
 // Must be registered at module load time before Hono's router is built
 const frontendDistPath = process.env.CHECKSTACK_FRONTEND_DIST;
@@ -630,13 +684,35 @@ const init = async () => {
     rootLogger.warn(
       "🛠 Dev auth ENABLED — every access rule is auto-granted. Do NOT use in production.",
     );
-    const devAuthService: AuthService = createDevAuthService({
-      getAllAccessRules: () => pluginManager.getAllAccessRules(),
-    });
-    pluginManager.registerService(coreServices.auth, devAuthService);
+    // Register the dev auth service as a FACTORY, not a plain instance.
+    // `registerCoreServices` already registered the real (token/strategy-
+    // based) auth as a factory for `coreServices.auth`, and
+    // `ServiceRegistry.get()` resolves factories BEFORE instances — so a
+    // plain `registerService(coreServices.auth, devAuthService)` would be
+    // shadowed by the production factory and the dev bypass would never take
+    // effect (every plugin API request would 401 with "Authentication
+    // required"). Registering the dev auth as a factory makes `get()` reach
+    // it. This stays entirely inside the dev-flag-gated path: the production
+    // auth factory is left in place and untouched whenever CHECKSTACK_DEV_AUTH
+    // is not set.
+    // Scoped per plugin (like the production auth factory) so each plugin's
+    // S2S credentials are minted as `{ service: <its pluginId> }`.
+    pluginManager
+      .getRegistry()
+      .registerFactory(coreServices.auth, (metadata) =>
+        createDevAuthService({
+          getAllAccessRules: () => pluginManager.getAllAccessRules(),
+          pluginId: metadata.pluginId,
+        }),
+      );
   }
 
   const manualPlugins: BackendPlugin[] = [];
+  // Maps a manually-loaded plugin's id to its on-disk dir so the loader can
+  // run its Drizzle migrations (in `<dir>/drizzle`). Without this, manual
+  // plugins get `pluginPath: ""` and their migrations are skipped — which is
+  // why a freshly-scaffolded plugin booted with no `items` table.
+  const manualPluginPaths = new Map<string, string>();
   if (devPluginPath) {
     rootLogger.info(`🛠 Dev mode — loading plugin from ${devPluginPath}`);
 
@@ -677,6 +753,9 @@ const init = async () => {
         );
       }
       manualPlugins.push(pluginExport);
+      // `CHECKSTACK_DEV_PLUGIN_PATH` is the plugin's repo dir (the dev server
+      // sets it to the plugin cwd), so its migrations live at `<dir>/drizzle`.
+      manualPluginPaths.set(pluginExport.metadata.pluginId, devPluginPath);
     } catch (error) {
       throw new Error(
         `Failed to import dev plugin from ${devPluginPath}: ${extractErrorMessage(error)}`,
@@ -686,6 +765,7 @@ const init = async () => {
 
   await pluginManager.loadPlugins(app, manualPlugins, {
     skipDiscovery: !!devPluginPath,
+    manualPluginPaths,
   });
 
   // 4. Wire up auth client for access-based signal filtering
@@ -967,8 +1047,24 @@ const fetch = async (
   return app.fetch(req, server);
 };
 
+// Bun closes a connection that stays idle (no bytes sent or received) for
+// `idleTimeout` seconds. The default is 10s, which severs long agentic chat
+// turns: the AI assistant streams an SSE response that can pause >10s between
+// chunks while a slow provider "thinks" or a tool runs, surfacing as
+// "Error in input stream" on the client. Raise it to Bun's maximum (255s) so a
+// multi-step turn is not killed; each streamed chunk resets the idle timer, so
+// only a single >255s silent gap would still time out.
+const IDLE_TIMEOUT_SECONDS = (() => {
+  const raw = Number(process.env.CHECKSTACK_SERVER_IDLE_TIMEOUT_SECONDS);
+  // Bun clamps idleTimeout to [0, 255]; keep within range and fall back to max.
+  return Number.isFinite(raw) && raw >= 0 && raw <= 255 ? raw : 255;
+})();
+
 export default {
-  port: 3000,
+  // Listen port. Defaults to 3000; overridable via PORT so a second instance
+  // (e.g. an isolated E2E stack) can run alongside a dev server on another port.
+  port: Number(process.env.PORT) || 3000,
+  idleTimeout: IDLE_TIMEOUT_SECONDS,
   fetch,
   websocket: {
     // Type template for ws.data

package/src/plugin-manager/api-router.ts

@@ -24,6 +24,7 @@ import type { EventBus } from "@checkstack/backend-api";
 import type { PluginMetadata } from "@checkstack/common";
 import { rootLogger } from "../logger";
 import { extractErrorMessage } from "@checkstack/common";
+import { mapPgErrorToHttp } from "./pg-http-errors";
 
 interface RouteHandlerDeps {
   registry: ServiceRegistry;
@@ -139,14 +140,20 @@ async function resolveRequestContext({
     deps.pluginMetadataRegistry.get(pluginId);
 
   if (!pluginMetadata) {
-    (logger as Logger).error(
+    // No metadata for this pluginId. The common cause is a client requesting an
+    // unknown plugin (typo / probing) - a 404, not a 500. A genuine
+    // misconfiguration (a core router that forgot
+    // pluginManager.registerCorePluginMetadata()) surfaces the same way, so we
+    // log it for diagnosis, but at warn level since the dominant case is a
+    // client error and erroring on every bad path would be noise.
+    (logger as Logger).warn(
       `${pathname}: no plugin metadata registered for pluginId='${pluginId}'. ` +
-        `Regular plugins populate this during register(); core routers must call ` +
-        `pluginManager.registerCorePluginMetadata().`,
+        `Either the plugin id is unknown (client typo / probe), or a core ` +
+        `router did not call pluginManager.registerCorePluginMetadata().`,
     );
     return {
       ok: false,
-      response: c.json({ error: "Plugin metadata not found in registry" }, 500),
+      response: c.json({ error: "Not Found" }, 404),
     };
   }
 
@@ -163,6 +170,11 @@ async function resolveRequestContext({
     cachePluginRegistry: cachePluginRegistry as CachePluginRegistry,
     cacheManager: cacheManager as CacheManager,
     user,
+    // The incoming request's headers, so a handler can forward the caller's OWN
+    // auth (session cookie / bearer) when it re-enters the router as the same
+    // user - e.g. an AI tool's user-scoped rpcClient (proposeTool/applyTool).
+    // Also read by correlationMiddleware for the inbound correlation id.
+    requestHeaders: c.req.raw.headers,
     emitHook,
   };
 
@@ -203,6 +215,35 @@ function logHandlerError({
   }
 }
 
+/**
+ * Shared interceptor catch path for both the RPC and REST handlers. A Postgres
+ * driver error caused by bad client input (bad uuid, out-of-range int, over-long
+ * string, constraint violation) is mapped to the correct 4xx `ORPCError` and
+ * logged at warn (it is a client mistake, not a server fault). Everything else
+ * keeps the existing error-level logging + rethrow so genuine 500s stay loud.
+ */
+function rethrowAsHttpError({
+  error,
+  pathname,
+  logger,
+  protocolLabel,
+}: {
+  error: unknown;
+  pathname: string;
+  logger: Logger | undefined;
+  protocolLabel: string;
+}): never {
+  const mapped = mapPgErrorToHttp(error);
+  if (mapped) {
+    (logger ?? rootLogger).warn(
+      `${protocolLabel} ${pathname}: ${mapped.code} (${extractErrorMessage(error)})`,
+    );
+    throw mapped;
+  }
+  logHandlerError({ error, pathname, logger, protocolLabel });
+  throw error;
+}
+
 /**
  * Creates the API route handler for Hono.
  * Serves oRPC's native RPC wire protocol at /api/:pluginId/*.
@@ -242,13 +283,12 @@ export function createApiRouteHandler({
             try {
               return await next(rest);
             } catch (error) {
-              logHandlerError({
+              rethrowAsHttpError({
                 error,
                 pathname,
                 logger,
                 protocolLabel: "RPC",
               });
-              throw error;
             }
           },
         ],
@@ -321,13 +361,12 @@ export function createRestRouteHandler({
             try {
               return await next(rest);
             } catch (error) {
-              logHandlerError({
+              rethrowAsHttpError({
                 error,
                 pathname,
                 logger,
                 protocolLabel: "REST",
               });
-              throw error;
             }
           },
         ],