@voltro/plugin-clickhouse 0.80.0 → 0.81.0

package/CHANGELOG.md

@@ -39,6 +39,96 @@ _Changes staged for the next release accumulate here (rolled up from
 
 ---
 
+## [0.81.0] — 2026-09-25
+
+### ⚠ BREAKING
+
+- **The compose baseline gives each api app its own database** — `@voltro/cli`
+
+  `voltro baseline sync` now gives every api app in a `project.json` its own database, login and migrate service. The primary api (the app named `api`, else the first api app) keeps the project's database; every other api app connects to `<project>_<app>` with `<PROJECT>_<APP>_DB_PASSWORD`, and each api app gets `VOLTRO_API_<APP>_URL` for its sibling api apps by service name. Migration: sync, move each secondary app's own tables into its new database, and set the new password variables in production.
+
+  **`voltro update` carries you across this** — codemod `0.81.0/03_compose-baseline-one-database-per-api-app`. If you pin versions by hand and never run it, print the notes without changing anything: `voltro update --codemods-only --from <your current version> --dry-run` (this one ships in 0.81.0).
+- **A crashed process blocks its idempotency key for 30 seconds, not a day** — `@voltro/protocol`, `@voltro/runtime`, `@voltro/client`, `@voltro/voltro`
+
+  A process that died inside an idempotent mutation left its key `in_flight` for the whole replay window — 24 hours by default — so every retry with that key was refused with `MutationInFlight` (REST: `409`), including the retry of a workflow step whose callee had simply restarted. A running claim now holds on a 30-second lease that the server renews while the handler runs, over the socket and on REST alike; a dead process stops renewing, and the next send with the key takes over and runs the mutation once — the dead call's transaction never committed. A completed response still replays for the whole window. A takeover is conditional on the claim still being stale, so a claim renewed a moment earlier is never taken from a live handler. `ctx.apis` calls treat `MutationInFlight` as no answer yet: the same key is re-sent when the callee says, until it answers or the call's deadline passes (then `ApiUnavailable`). BREAKING only for an app that implements `IdempotencyStore` itself: `claim` takes the lease, and `renew(scope, key, now)` is required.
+
+  **`voltro update` carries you across this** — codemod `0.81.0/05_idempotency-store-renews-its-lease`. If you pin versions by hand and never run it, print the notes without changing anything: `voltro update --codemods-only --from <your current version> --dry-run` (this one ships in 0.81.0).
+- **Each api app owns its database schema; a second app on it refuses to boot** — `@voltro/database`, `@voltro/cli`
+
+  The first boot that migrates a database records which app owns it (`_voltro_schema_owner`). A different app booting against the same database and schema now refuses — naming both apps — before it plans anything, under `voltro dev`, `voltro serve` and `VOLTRO_AUTO_MIGRATE=0` alike; `voltro db apply` refuses the same way and claims an unclaimed database — as its first write, after every other check has passed, so an apply refused for drift or a blocked operation leaves the database as it found it. `voltro data` export and import treat the owner row as the deployment's own: it never travels to another database. Two apps on one schema each diffed it against their own declarations and planned to drop the other's tables. Migration: give every api app its own database (`DB_URL`) or schema (`DB_SCHEMA`) and reach the other app's data through `apis:` and `service:` procedures; `voltro doctor` lists api apps of a project that resolve the same target. `voltro db claim --yes` hands a schema to a RENAMED app.
+
+  **`voltro update` carries you across this** — codemod `0.81.0/01_one-database-per-app`. If you pin versions by hand and never run it, print the notes without changing anything: `voltro update --codemods-only --from <your current version> --dry-run` (this one ships in 0.81.0).
+- **ReactionDefinition is the union of the table form and the query form** — `@voltro/runtime`, `@voltro/voltro`
+
+  With reactions that watch a live query's rows, `ReactionDefinition` is now `EdgeReactionDefinition | LevelReactionDefinition`. Code holding a value of that type and reading a member only the table form has — `watch.table`, `watch.on`, `guards.dedupeKey`, `when(event)` — no longer compiles: type it `EdgeReactionDefinition`, or narrow with `isLevelReaction(def)` first. `defineReaction({ … })` is unchanged: an object literal resolves to its own form, and a value typed as the union is still accepted and validated by its `watch`.
+
+  **`voltro update` carries you across this** — codemod `0.81.0/04_reaction-definition-is-a-union`. If you pin versions by hand and never run it, print the notes without changing anything: `voltro update --codemods-only --from <your current version> --dry-run` (this one ships in 0.81.0).
+- **Store calls on a table the app does not declare now fail** — `@voltro/runtime`, `@voltro/cli`
+
+  Every store verb — in handlers, startups, schedules and workflows — now fails with `UndeclaredTableAccess` on a table that is neither declared by the app (its `table()`s, a plugin's `extendSchema.tables`) nor framework-owned (`_voltro_*`), in every environment. A string table name used to reach another app's table past its guards, audit and tenant rules. `store.raw(sql)` is unchanged. Migration: call the owning app through `apis:` and a `service:` procedure, or declare the table if it is this app's own.
+
+  **`voltro update` carries you across this** — codemod `0.81.0/02_store-refuses-undeclared-tables`. If you pin versions by hand and never run it, print the notes without changing anything: `voltro update --codemods-only --from <your current version> --dry-run` (this one ships in 0.81.0).
+
+### Added
+
+- **A plugin can bring app-shaped building blocks with `sources`** — `@voltro/protocol`, `@voltro/cli`
+
+  `definePlugin({ sources: fileURLToPath(new URL('./app', import.meta.url)) })` points at a directory of the same file conventions an app uses — queries and mutations with their executors, `schema.ts` tables, `*.workflow.tsx`, `*.cron.tsx`, subscribers, reactions — and every app that installs the plugin discovers them exactly as it discovers its own: the same codegen, browser-safety guard, access gate, migrations, and both boot paths. A reusable building block (a ledger, an audit trail, an approval flow) is shared between apps of a project without copying it; each installing app gets its own copy of the block's tables in its own database. `voltro build` keys the block's modules by plugin and path, so the serve bundle finds them wherever the block lives after a deploy.
+- **A reaction can watch the rows of a live query, including another app's** — `@voltro/runtime`, `@voltro/cli`
+
+  `defineReaction({ watch: { query: 'deployments.pending', api: 'cloud' }, key: (row) => row.id, act: { kind: 'workflow', workflow: 'deploy.run' } })` reacts to every ROW of a live query's current result — the set of open work — instead of to each change of a table. Every delivery is the whole set, so a row that appeared while the reacting process was down, restarting or deploying is acted on when it comes back; nothing is lost. The workflow's `idempotencyKey` makes each row's act exactly-once across every replica, so no claim table is involved and a start that failed is tried again on the next delivery or on `resync` (default 5 minutes). `resync` and `stuckAfter` take milliseconds or a duration from `'30s'` to `'8 days'`. With `api:` the query is another app's `service:` query, subscribed over `ctx.apis` — push without the watched app knowing its watcher; without it, one of this app's own queries, read through the same guards and row filter as its REST projection. A row still in the set long after its workflow started is reported as `stuck` and never restarted. `voltro_reaction_pending`, `voltro_reaction_oldest_seconds` and `voltro_reaction_connected` expose the queue per reaction. The table form (`watch: { table }`) is unchanged; `runReaction` now takes the table form's type, `EdgeReactionDefinition`.
+- **Apps of one project call each other through typed service procedures** — `@voltro/voltro`, `@voltro/protocol`, `@voltro/runtime`, `@voltro/client`, `@voltro/cli`, `@voltro/testing`, `@voltro/workflow`
+
+  An api app can now call another app of the same project, typed, over the socket, without touching its tables. The called app declares `service: { callers: ['worker'] }` on a query, mutation or action and lists the caller under `serviceCallers: { worker: {} }`; codegen emits those procedures into `serviceGroup.generated.ts` and never into the browser's `rpcGroup.generated.ts`. The calling app binds it under `apis: { cloud: { package: '@acme/api' } }` and calls `ctx.apis.cloud.deployments.activate(input)` (Promise) or `.effect(input)` (Effect), with every input, output and typed error inferred from the callee's declarations. A service authenticates with a per-edge credential (`VOLTRO_API_<BINDING>_TOKEN` on the caller, `VOLTRO_SERVICE_CALLER_<CALLER>_TOKEN` on the callee, `_NEXT` for rotation); `voltro dev` derives and mints it into both apps' `.env.local`, production supplies it (`voltro secret generate service`) and a missing one fails boot. A caller name is letters, digits and dashes, at most 56 characters, so its `service:<name>` identity fits an `id()` column. The called app plants an `actors` row for each accepted caller at boot (`service:<name>`, `kind: 'serviceAccount'`), so `audit()`-stamped writes name the calling app instead of failing on the foreign key. A service subject holds no scopes: it reaches exactly the procedures that name it, is refused everywhere else — framework built-ins included — and no user, API key or `admin:full` session reaches a service procedure. A write made inside a workflow step carries an idempotency key derived from the step, so a step retried after a lost reply replays instead of writing twice; an app with `serviceCallers` deduplicates mutations by default. A call without an answer fails with `ApiUnavailable` within a deadline (10 s by default); calls are measured as `voltro_api_call_duration_seconds` and `voltro_api_call_errors_total`. `voltro doctor` reports a caller listed without a procedure, a procedure naming an unlisted caller and a binding the callee does not accept. `@voltro/testing` adds `makeServiceApiMock`, which decodes every input and output through the callee's real schemas.
+
+  `apiSurface: compatible` — the replaced golden lines in `@voltro/protocol` (and the `@voltro/voltro` aggregate re-exporting it) are `AnyCheckSpec` and `DeclaredAccess` gaining the member `ServiceAccessSpec`. Every existing member is unchanged and an app keeps writing the guards it wrote; the framework is the only producer of the new member, which it normalises from `service:`. Only a switch exhaustive over guard specs needs an arm.
+- **startTestApps boots several apps of one project, each service edge wired** — `@voltro/testing`
+
+  `startTestApps({ apps, edges })` from `@voltro/testing` boots the apps of one project as real `voltro` processes (`dev` by default, `serve` with `mode: 'serve'`), each on a free port — every callee before its callers, because a caller imports the contract its callee's codegen writes — and resolves once every app answers `/internal/readiness`; a cycle of edges is refused before anything starts. Every listed edge gets a freshly minted credential written to both ends, under the variables the apps read (`VOLTRO_API_<BINDING>_TOKEN` / `_URL` on the caller, `VOLTRO_SERVICE_CALLER_<CALLER>_TOKEN` on the callee); `apps.token(caller, callee)` returns it so a test can call the callee as the caller. An app that exits or is not ready in time stops the others and the call rejects with its log. Processes rather than one process, because apps with their own tables cannot share one process's table registry.
+- **makeTestApp requests carry the caller's address** — `@voltro/testing`
+
+  A route that reads `ctx.remoteAddr` (a per-client rate limit, an audit field) could not be tested per client: every `makeTestApp` request arrived without an address. A request's options now take `remoteAddr`, the address the connection comes from, defaulting to `127.0.0.1`. The route sees it as `ctx.remoteAddr`, resolved by the function the server's listener uses, and a `publicApi` procedure's plugin interceptors see the same address, as on the server. `makeTestApp({ security: { trustedProxies, clientAddressHeader } })` takes the api's own settings, so a forwarding header is believed in a test exactly when it would be in production.
+
+  `markRestInvocation` takes the request's address as an optional third argument, and `restInvocationOf` returns it beside the spec; both only gain an optional member, so existing calls and reads are unchanged.
+
+### Fixed
+
+- **Codegen removes an rpc group the app no longer has procedures for** — `@voltro/cli`
+
+  An app whose last client-reachable procedure went away — deleted, or marked `internal: true` — kept its `rpcGroup.generated.ts`: codegen reported "no rpc primitives found — nothing to write" and left the old file, a typed contract for procedures the server no longer served. The next `voltro dev` or `voltro codegen` now removes it, as it already removed an empty `serviceGroup.generated.ts` and `apis.generated.ts`. A hand-written file of that name is left alone.
+- **The auto-launched dashboard binds `VOLTRO_DASHBOARD_PORT`** — `@voltro/cli`
+
+  `VOLTRO_DASHBOARD_PORT` was read by `voltro dashboard` only. The dashboard that `voltro dev` and `voltro start` launch on their own — api and web app alike — always came up on `:5179`, and the launcher wrote `5179` into the child's `VOLTRO_DASHBOARD_PORT`, overriding the value the developer had set. The launcher and `voltro dashboard` now resolve the port through one reader: the variable, else `5179`. A value that is not a port (`1..65535`) is refused with a warning naming it instead of silently becoming `5179`.
+- **The db pool boot line counts a connection outside the pool once** — `@voltro/cli`
+
+  With change data capture on, the line read `max=5 … + 1 = 6` and then, separately, `PLUS 1 outside the pool (CDC LISTEN consumer)`, which reads as 7 when the process holds 6. The outside connection is now named once, inside the sum it belongs to: `max=5 per replica (DB_MAX_CONNECTIONS) + 1 outside the pool (CDC LISTEN consumer) = 6 per process`.
+- **A declared `utf8mb4` store reaches the workflow engine's tables** — `@voltro/database`, `@voltro/workflow`, `@voltro/cli`
+
+  `store: { charset: 'utf8mb4' }` converted every table the app declares and left the tables the framework creates without declaring them in the database default: the durable workflow engine's `cluster_*` storage, which holds every workflow call's input and result as JSON text, and the deploy runner's `_voltro_deploys` / `_voltro_migration_ops`. On a `utf8mb3` database a workflow whose input carried an emoji was refused there. Every table the framework creates is now created in the declared charset: the runner's ledgers directly, and the workflow engine's tables by the process that creates them on its first boot, while they are still empty. A workflow table a booting process finds is never rewritten at boot; tables from before the declaration are converted by `voltro db apply` (also when the plan has nothing to do, and with `--plan`) once the deploy has closed and by the `voltro dev` auto-migrate, keeping type, nullability and column CHECKs, and `voltro db plan` lists them beside the plan (under `--json` too, on the log, never in the plan file). The database's own default is left alone. The step only ever widens, and a deploy that fails and is undone never runs it.
+- **The MySQL/MariaDB pool keeps its connections instead of reconnecting** — `@voltro/sql-mysql`
+
+  The driver pool was created with `maxIdle: 0`, so every connection released to it was closed within a second. A process reconnected after each second of quiet: a TCP handshake per burst, plus a TLS one where configured. Measured on MySQL and MariaDB: a query after two quiet seconds ran on a new connection. The pool now keeps the connections it opened, up to the limit.
+- **Workflows started by a reaction or an incoming webhook start again** — `@voltro/cli`
+
+  A workflow started by a `*.reaction.ts` act or an incoming `*.webhook.tsx` was refused with `Workflow placement requires explicit caller identity`, under `voltro dev` and `voltro serve`: neither path named a caller, and placement refuses a start without one. Both now start as the framework itself, placed in the changed row's tenant when it has one.
+- **The first boot on an empty database no longer warns about replica observations** — `@voltro/cli`
+
+  `voltro dev` started publishing this replica's observations before its auto-migrate, so the first write on an empty database hit a table that did not exist yet and printed `could not record this replica's observation — fleet views will report this replica as stale` on the first boot of every new project. The publisher now writes its first observation once the schema is applied — at once on `voltro serve`, whose fingerprint check runs before it — and never when the boot stops earlier.
+- **A 405 on a shared REST path names every method the path serves** — `@voltro/protocol`
+
+  A `DELETE` to a path served by `GET` and `POST` answered `405` with `Allow: POST`: each route refused with its own method, and the last refusal won. `Allow` and the error body's `allow` now list every method declared on the path, `GET, POST`, as the REST documentation already promised.
+- **A large REST surface no longer holds the boot for seconds registering routes** — `@voltro/runtime`, `@voltro/protocol`
+
+  Every REST route used to be mounted for all 34 HTTP methods the router knows, at its path and at its sub-path, and each insert compares itself against every entry already held. A public REST surface of about 1000 procedures therefore held about 34 000 router entries, and building them kept the thread busy for over a second after the port was bound — long enough for a liveness probe to time out. A path is now mounted for exactly the methods its routes declare: the same 1000 procedures hold 4 000 entries. Behaviour is unchanged. A method the path does not declare still reaches its routes and gets the precise `405` with `Allow`, a path nobody declared is still a `404`, and `HEAD` still answers from `GET`. For this, a REST route now declares its real `method` instead of `'*'` and sets the new `PluginHttpRoute.answersOtherMethods: true`. A plugin route that declares one method without that flag never sees another method, as before. A server of your own that mounts `restRoutesToHttpRoutes` output by method must hand the other methods to routes with `answersOtherMethods`, or they become `404` instead of `405`.
+- **The testing guide's tsconfig example no longer sets baseUrl** — `@voltro/cli`
+
+  The `paths` alias example in the testing guide set `"baseUrl": "."`, an option TypeScript 7 removed. It now maps `paths` alone, which resolves against the tsconfig's own directory. `voltro test` still reads a `baseUrl` an older config sets.
+- **voltro update lists a note's code and config lines before prose** — `@voltro/cli`
+
+  A manual note lists the lines it found in code and config first, under `── found in code or config (N lines)`. Lines in Markdown and text files follow in their own `── mentioned in prose (N lines)` block, and a note that matched only prose says `nothing in code or config matched`. Past the lines shown, the rest are named by file with a count instead of a bare `… and N more`. Before, the lines were sorted by path, and 40 lines of documentation hid a Helm env entry, the one line to change. The guide `voltro agents-md` copies into `agent-docs/` is no longer searched. It names every change of a release, so refreshing it made notes report "found" for changes the project never used.
+
+---
+
 ## [0.80.0] — 2026-09-24
 
 ### ⚠ BREAKING

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@voltro/plugin-clickhouse",
-  "version": "0.80.0",
+  "version": "0.81.0",
   "description": "ClickHouse AnalyticsSink. Native HTTP ingest into a ClickHouse cluster — self-hosted OR ClickHouse Cloud. Production-grade OLAP for high-cardinality events with millisecond aggregate queries over billions of rows.",
   "keywords": [
     "voltro",
@@ -34,8 +34,8 @@
   },
   "dependencies": {
     "@clickhouse/client": "^1.23.1",
-    "@voltro/logger": "0.80.0",
-    "@voltro/runtime": "0.80.0"
+    "@voltro/logger": "0.81.0",
+    "@voltro/runtime": "0.81.0"
   },
   "peerDependencies": {
     "effect": "^3.22.2"