@voltro/plugin-audit 0.32.0 → 0.33.0

package/CHANGELOG.md

@@ -39,6 +39,211 @@ _Changes staged for the next release accumulate here (rolled up from
 
 ---
 
+## [0.33.0] — 2026-08-11
+
+### ⚠ BREAKING
+
+- **@voltro/protocol, @voltro/runtime, @voltro/voltro** — `CoordinatedScheduleHandle` gained `wake()`, `currentIntervalMs()` and `isArmed()`.
+
+  The type change that carries the poller work in this release (see *A coordinated tick is a FLOOR*). Two of the three shapes it touches are NOT breaking and are listed here so the classification is checkable rather than asserted:
+
+  - the effect parameter was **widened** — it may now return a tick outcome, and an existing `() => Promise<void>` still satisfies it; - `Coordinator.tryClaim` gained an **optional** third parameter (the caller's bucket width), so an existing implementation still conforms.
+
+  (The plugin-facing `scheduleCoordinated` also gained an OPTIONAL fourth argument, `{ disarmWhenIdle }` — additive, and how a plugin opts its own task out of polling entirely.)
+
+  What breaks is code that **constructs** a handle rather than receiving one: a hand-written test double of `PluginBindContext`, which is the ordinary way to unit-test a plugin's `bindDataStore`. Four of the framework's own suites carried one, and three of those compiled only because the stub was cast — which is also why the two new members must be REQUIRED rather than optional. An optional `wake()` would let a caller subscribe a change channel to a handle that silently has none, and a poller that never wakes is the failure this release exists to remove, arriving quietly.
+
+  The codemod is `manual`: the object literal needing the two fields carries no importable symbol and usually sits behind an `as never`, so no transform can tell it apart from an unrelated literal in the same test file. It is gated on the app mentioning `scheduleCoordinated` at all.
+- **@voltro/runtime** — `@effect/opentelemetry` is now an **optional peer** of `@voltro/runtime` instead of a dependency. **If you export traces or metrics, install it:**
+
+  ```sh
+  pnpm add @effect/opentelemetry
+  ```
+
+  If you do not (no `FRAMEWORK_TRACING`, no `FRAMEWORK_METRICS`, no `OTEL_EXPORTER_OTLP_*`), nothing changes and your install gets 24 lines quieter.
+
+  It is reached from one dynamic `import()`, only when tracing is on, and it declares seven non-optional OpenTelemetry peers of which we supply five. So every `pnpm install` of every consumer ended with an unmet-peer block describing a condition that broke nothing. Declaring the two missing peers as real dependencies was the wrong direction — one of them is `@opentelemetry/sdk-trace-web`, the BROWSER tracer — and 0.64.0 is the current stable, so there is no upstream release marking them optional to wait for.
+
+  The reporting consumer's argument is what decided it: *"a check that is loud on every upgrade teaches people to skip the output, and the next warning in that block is the one that matters. We read past this one for four releases."*
+
+  A boot with tracing enabled and the package absent fails with a message naming this install line — a startup failure, not a silent loss of telemetry.
+
+  **`voltro update` carries you across this** — codemod `0.33.0/01_opentelemetry-optional-peer`.
+
+### Added
+
+- **@voltro/cli** — `voltro agents-md` now reports which `@voltro/cli` it seeded from, and warns when that is not the one the project installs.
+
+  ```
+  seeded from @voltro/cli 0.31.0 (project has 0.32.0; whats-new describes 0.31.0;
+    modules COPIED into ./agent-docs)
+  WARN the `voltro` binary that ran is 0.31.0, but this project installs 0.32.0 —
+    everything just written describes the OLDER version.
+  ```
+
+  A consumer reported a freshly-seeded `agent-docs/whats-new.md` one release behind their installed version, twice. The published packages are correct (verified with `npm pack`), so the content came from a different `@voltro/cli` than the one they installed — the command reads its templates relative to the RUNNING binary, and a globally-installed `voltro`, a stale `dist`, or a parent workspace's copy all produce exactly that, with output that looked identical either way.
+
+  It does not refuse and does not pick a cli for you: running the workspace binary against a checkout is legitimate and common.
+- **@voltro/cli** — `voltro db encrypt-column <table>.<column>` — the data migration `.encrypted()` always needed.
+
+  `.encrypted()` encrypts on WRITE, so adding it to a populated column converts nothing that is already there, and there was no supported way to convert it. A consumer carried three plaintext credential columns for months with no next step: *"`.encrypted()` braucht einen Cipher UND eine Datenmigration der bestehenden Zeilen; gemeldet, nicht behoben."*
+
+  ```sh
+  voltro db encrypt-column integrations.webhookSecret --dry-run
+  voltro db encrypt-column integrations.webhookSecret employees.meilisearchKey --yes
+  ```
+
+  Five guards, each for a way a naive version succeeds and destroys data:
+
+  - **Idempotent** — an already-ciphertext value is skipped, so an interrupted run is resumed by running it again. Double encryption is unrecoverable without the key history. - **Round-trip verified before the write** — every value is decrypted back in-process first, so a broken cipher fails with nothing written. - **Key checked against what the column already holds** — a *different* key round-trips fine, so the check above cannot see it. Resuming with the wrong key would leave a column readable with neither key alone. - **Width pre-flight** — ciphertext is `49 + 4×ceil(bytes/3)` characters, so a 64-char key needs 137 and a `varchar(100)` fails partway. Refuses with both numbers and the `.maxLength()` to set. Measured in BYTES: `'ä'.repeat(10)` is 10 characters and 20 bytes. - **`--yes` required**, `--dry-run` shows the counts, and no value — plaintext or ciphertext — is ever printed.
+
+  Verified against a real postgres: the conversion, the re-run no-op, both refusals writing nothing, and a decrypt back to the original including multi-byte content.
+
+### Changed
+
+- **@voltro/runtime** — `_voltro_schedule_claims` swaps its `(scheduleName, bucket)` index for `(scheduleName, claimedAt)`.
+
+  A consumer read `pg_stat_user_indexes` on their live table and measured, over its whole lifetime:
+
+  ```
+  _voltro_schedule_claims_pkey                   348 978 scans
+  _voltro_schedule_claims_claimedAt_idx            3 949
+  _voltro_schedule_claims_scheduleName_bucket_idx      4
+  ```
+
+  Four. It was declared "for the case where you would rather ask by field", and nothing ever asks by field — every read of this table goes through the primary key, which *is* `<scheduleName>@<bucket>`. An index nothing uses is not free: it is written on every INSERT, into a table written once per tick per schedule.
+
+  `(scheduleName, claimedAt)` is the shape of a query that now exists — the per-schedule prune a winning claim runs (`WHERE scheduleName = ? AND claimedAt < ?`). The `claimedAt` index stays: the retention sweep's cutoff spans every schedule and needs it leading, which the composite cannot provide.
+
+  No codemod: a `_voltro_*` change rides the declarative differ on `voltro db apply` and on a `voltro dev` boot, on every dialect.
+
+  The same measurement corrected something the reporter had said in an earlier round and we had repeated back to them — that both indexes went unused. The primary key is used constantly. That makes the finding sharper rather than weaker: the ability to answer this question in one lookup is not merely available, it is demonstrably in use on the same table, and the one read path that needed it was the one not taking it.
+
+### Fixed
+
+- **@voltro/database** — A column ADDED with a `reference()` now gets its foreign key in the same plan.
+
+  `ADD COLUMN` emits no `REFERENCES` clause on any dialect, and the planner's FK branch lived only in the path for a column present on both sides — so adding a `reference()` column to an existing table planned an `add-column` and nothing else. The constraint appeared on the SECOND `voltro db apply`, when the column was live and the diff finally saw a live column with no FK.
+
+  Two applies converged, so the state was reachable, which is why this survived as a low-priority note for a long time. It is worse under `voltro dev`: the boot diff refuses to record a fingerprint while the re-plan is non-empty, so an app whose only pending change was such a column re-planned on every boot and never converged.
+
+  Both callers share one `addForeignKeyOps` builder now, and the existing dependency tiering already orders `add-column` before `add-foreign-key`.
+- **@voltro/runtime, @voltro/protocol, @voltro/workflow, @voltro/cli** — A coordinated tick is a FLOOR now, and a claim no longer outlives its bucket.
+
+  A consumer's `_voltro_schedule_claims` reached **86 214 rows / 33 MB** on two days of uptime and took their deployment down: ten of a fifteen-slot pooler pinned on the claim read, an SSR render measured at **300 490 ms** behind them, every page in three frontends unusable, and a `rollout restart` that could not complete because the surge pod could not get a connection. Two hours of their own measurement produced the diagnosis, and both halves of it were right.
+
+  **Where the rows came from.** They declare one workflow, have never started it, use no flow control and no offloaded inference. Over one hour, with two replicas:
+
+  ```
+  voltro.ai.inference          1 259 rows/h    (250 ms ticks)   framework
+  voltro.workflow.admission    1 247 rows/h    (1 s ticks)      framework
+  their own eight schedules       18 rows/h
+  ```
+
+  99.3 % of the ledger was the framework polling two structurally empty queues. A fixed interval has no way to learn that, so:
+
+  - **`scheduleCoordinated`'s effect may now REPORT its tick.** Return `{ idle: true }` and the runner backs off toward a ceiling; return `{ idle: true, nextDueInMs }` and it arms for that instant instead — which is what keeps a `debounce` window from being slept through. Returning nothing keeps the fixed interval, so every existing plugin task ticks exactly as before. - **Where an arrival is guaranteed to wake it, an idle task STOPS ENTIRELY** (`{ disarmWhenIdle: true }`). Both framework tasks do, on any deployment where a peer replica's write is visible locally — Postgres LISTEN/NOTIFY, or a broadcast broker. Measured against a real Postgres on a deployment that uses neither queue: **2 claim rows in five minutes**, one per task, both at boot. Where that guarantee does not hold, the ceiling (`VOLTRO_POLL_CEILING_MS`, default 30 s) is the correct behaviour and is what they get. - **`handle.wake()` runs a tick now.** Both framework queues are tables with the framework's own CDC triggers on them, so an enqueue already produces a change event on every replica; both dispatchers subscribe to it. The idle case gets ~120× cheaper and the busy case gets FASTER — work starts on the INSERT rather than up to a tick later. - The claim bucket stays floored by the BASE interval. Replicas do not share a backoff state, and two replicas computing different keys for one moment would both win.
+
+  **The cadence is declarable.** `scheduling: { admissionDrainMs, inferenceTickMs, cancelSweepMs, pollCeilingMs }` in `app.config.ts`, each with a matching `VOLTRO_*` env var that overrides it — the same ordering as `VOLTRO_TENANT_ISOLATION` over `tenancy.isolation`. They were internal constants, and a number the framework picks on a user's behalf belongs somewhere they can read it without reading our source. One resolver, called by both boot paths, so there is no second default to drift.
+
+  **Why the rows never left.** A claim answers one question about one bucket and was already answered the moment the bucket passed. A winning claim now deletes that schedule's own predecessors, so the table's size is a small multiple of the number of schedules rather than a function of uptime. How far back it prunes scales with the caller's bucket width — a cron keeps ~68 minutes of them (its firings carry their own instant, so a stalled one can re-present an old bucket), a 250 ms task ~1 minute (it recomputes its bucket at tick time, so an old one is unreachable). Deleting too early is a double fire; that grace is the whole safety argument. The 24-hour retention sweep stays as the backstop for a schedule that was renamed or deleted, which the per-schedule prune can never revisit.
+
+  Where reactivity is absent — a non-Postgres dialect with no broadcast broker — a remote replica's enqueue produces no local event and the ceiling is the whole latency budget. `VOLTRO_POLL_CEILING_MS` is there for that case and documented as such.
+- **@voltro/logger** — The pretty log format now prints a nested `Error`'s `message`. It did not, and the JSON format did.
+
+  `Error.prototype.message` is non-enumerable, so `JSON.stringify(err)` emits the metadata and drops the message. `expandCauseForJson` has existed for a long time to solve exactly that — and it was wired into `jsonFormat` only. The section heading above it said "(JSON path)", which was literally accurate.
+
+  `voltro dev` prints the pretty format. What a consumer saw when their boot died on a saturated pooler:
+
+  ```
+  auto-migrate failed — aborting boot
+  err={"failure":{"cause":{"length":117,…,"code":"XX000"},"message":"PgClient: Failed to connect"}}
+  ```
+
+  `length: 117` is the length of a message that is not there. Recovered by hand, it was `(EMAXCONNSESSION) max clients reached in session mode - max clients are limited to pool_size: 15` — the whole diagnosis in one sentence, naming the fix.
+
+  Both the field tail and the plain-object cause branch expand now. The fix is in the formatter, not at the reporting call site: every `log.error('…', { err })` anywhere had the same hole.
+- **@voltro/cli** — Every `@voltro/*` package now exports its own `package.json`, so `require('@voltro/cli/package.json').version` works.
+
+  It threw. Node has enforced this since 12: a package with an `exports` field exposes only what that field lists, and none of the 77 packages listed `"./package.json"`.
+
+  Reported by a consumer for whom it was the instruction WE gave for settling whether a security command had been running on stale code — so the verification step for a security question could not run at all. Both the workspace `exports` and the shipped `publishConfig.exports` are fixed, and a guard sweeps every package so a new one cannot ship without it.
+- **@voltro/cli, @voltro/sql-postgres** — The `db pool:` boot line now counts the connections this process holds OUTSIDE the pool, and names them.
+
+  It reported `max × replicas` and called that the connection count. A consumer sizing a per-pod budget against a pooler measured the gap:
+
+  > `LISTEN` läuft außerhalb von `dbMaxConnections` (eine pro Pod, gemessen sogar > 3). Der echte Bedarf ist `dbMaxConnections + 1`.
+
+  Their measurement was right and their conclusion was one short. The framework opens a standalone connection in three places, and a full deployment holds all three:
+
+  | Process | Connection | When | |---|---|---| | api `voltro serve` | CDC `LISTEN` consumer | `changeStrategy: 'cdc'` | | web `voltro start` | ISR invalidator `LISTEN` | a page declares `cacheInvalidatesOn` | | web `voltro start` | postgres ISR cache client | `SSR_CACHE=postgres` |
+
+  The third is not a `LISTEN`, which is why counting `LISTEN` rows in `pg_stat_activity` undercounts, and why `+1` could not have been documented as a constant: the count is per PROCESS and only the process knows what it armed.
+
+  The line says `No connections outside the pool in this process` when there are none — silence about it is what made "counted, zero" indistinguishable from "not counted". The `maxConnections` docstring, which promised `+1` as if it were the deployment's number, is corrected. Production-hardening docs (both languages) gain the table plus the `maxSurge` arithmetic a rolling update needs.
+- **@voltro/cli** — The retention sweep is registered on every dialect — it was postgres-only, and silently.
+
+  `wireRetentionSweep` opened with `if (dialect !== 'postgres') return`, so on mariadb, mysql, mssql and sqlite **none of its seven policies was registered, nothing was ever deleted, and the boot printed no armed-policies line** — so there was nothing to notice either. A consumer on MariaDB 11.8.8 measured it by reading the published bundle rather than their logs:
+
+  ```
+  _voltro_schedule_claims   109 520 rows   32.8 MB   over 20 days
+  _voltro_schedule_runs      17 507 rows    7.4 MB
+  ```
+
+  Their seven `VOLTRO_*_TTL_HOURS` variables were inert — read only inside the branch that never ran — and two of them were already set in their Helm chart.
+
+  **The gate was aimed at the right thing and applied to the wrong scope.** What is postgres-specific is the fast DELETE (`"camelCase"` quoting, `DELETE … RETURNING`), which is one branch of one function that has always had a portable fallback beside it. Gating the REGISTRATION on it turned a performance choice into a feature that does not exist. The dialect check now sits on the branch it describes.
+
+  Two things came out with it:
+
+  - **The fallback deleted row by row.** Acceptable while the path was unreachable; against the reporter's backlog it is 20 000 round trips per sweep pass. It reads a bounded batch of ids and issues ONE set-based delete for them — still bounded, so the DELETE never grows to lock the whole backlog. - **Two tests asserted the defect as intended behaviour**, with reasoning that was internally consistent and rested on the premise that was itself the bug (*"the sweep is postgres-only, and announcing a delete that will not happen is the mirror image of the defect"*). Both are inverted now and run across all five dialects.
+
+  This is the third turn of the same screw, and the reporter's framing is the one to keep: we fixed *a standing delete that never introduces itself*, then shipped *one that introduces itself and does not run* — and beside both of those sat one that silently did not exist.
+- **@voltro/cli** — `voltro db scan-credentials` no longer reports the framework's own redaction markers as credentials, and no longer claims a match was a *key*.
+
+  A consumer with correctly-redacting plugins got:
+
+  ```
+  ✗  _voltro_row_history.data — 69 of 149 row(s) match a credential-shaped key
+         matched (rows per needle, may overlap): token (69)
+  ```
+
+  All 69 rows were `"_omitted": ["token"]` — `@voltro/plugin-versioning`'s record that a `.serverOnly().sensitive('secret')` column was deliberately left OUT of the snapshot. The scan matched the proof that nothing is stored there, called it a credential, and printed *purge them AND rotate the credentials* underneath.
+
+  Two changes. The headline says what the predicate does — it is a substring match over the whole serialized column, so it finds a credential-shaped **name** anywhere in the value, which it always did. And each hit is now EXPLAINED: a bounded second pass (500 matched rows per target) reads them back in-process and separates a JSON **key** from a **redaction marker** (`_omitted`, `__redacted`). Values are never printed and never logged.
+
+  A target whose every matched row is a marker reports as explained and exits `0`. The bar is deliberately high — every matched row examined, every one a marker and nothing else. A capped read-back, one real key, or one row that will not parse as JSON keeps the target a finding and still exits `1`.
+
+  The shape mattered more than the one key name: the more columns an app classifies correctly, the more markers it writes, and the redder the scan turned.
+- **@voltro/runtime, @voltro/cli** — `advisoryLock` scheduling no longer reads the whole `_voltro_schedule_claims` table to answer whether one claim row exists, and that table is now swept on the scale it fills.
+
+  The existence check ran `SELECT "id" FROM "_voltro_schedule_claims"` with no `WHERE` and no `LIMIT`, then filtered in JavaScript — twice per claim attempt (the fast path, and the re-read that separates "lost the race" from "the claims table is broken"), on every replica, for every schedule firing. A consumer measured ten concurrent copies of that scan holding every connection of a 15-slot pooler, with an SSR render behind them at **300 490 ms**. It is a primary-key lookup bounded to one row now (`id` *is* the claim key).
+
+  The pool-acquire bound added in 0.32.0 turns that from a hang into an error; it does not stop the scan from filling the pool. Both are needed.
+
+  `VOLTRO_SCHEDULE_CLAIMS_TTL_HOURS` also defaults to **24 hours** instead of 30 days. The 30-day default was copied from the framework's history tables (`_voltro_schedule_runs` and friends), and a claim row is a lock ledger — it answers a question about one firing instant and nothing reads yesterday's. At the 1 557 rows/hour that consumer measured, a 30-day window reaches ~1.1 million rows before the first one ages out. Raise it deliberately if you need to; the number to reason about is the longest a replica may be paused and still be trusted not to re-fire a bucket it already lost.
+
+  The boot announcement can now express an age under a day (`older than 1h`); it previously rounded every TTL to whole days, so an operator setting one hour read their own policy back as `older than 0d`.
+- **@voltro/runtime** — A coordinated periodic task armed below one second now runs at the interval it was given.
+
+  `scheduleCoordinated` floors the wall clock to its `intervalMs` and races on that instant; the claim key truncated it to second precision. A task at 250 ms therefore produced four bucket instants per second that collapsed to one key — the first tick won and the other three were dropped as "lost the claim". Measured: 1 of 4.
+
+  `voltro.ai.inference` is armed at 250 ms and was dispatching once per second, on every multi-replica deployment, with nothing above `warn` to say so.
+
+  This is the defect the coordinator's own comment describes at minute precision (6-field crons firing once a minute), one decimal place down; that comment was written before `scheduleCoordinated` existed, and `scheduleCoordinated` is the caller that goes below a second.
+
+  Milliseconds join the claim key only when non-zero, so every cron key is byte-identical to before — load-bearing during a rolling deploy, where old and new replicas computing different keys for one firing would both win and double-fire.
+
+  Note the consequence for table size: a sub-second task now writes claim rows at its true rate. Bounded by `VOLTRO_SCHEDULE_CLAIMS_TTL_HOURS` (24 h), and `ai.tickIntervalMs` raises the interval if you want fewer.
+
+### Internal (no consumer-facing effect)
+
+- **@voltro/cli** — No separate consumer-facing note on purpose: this refines the per-needle breakdown described in the UNRELEASED 0.32.0 section, and that section — which is what a reader will actually see — carries the correction. Documenting it twice would describe one change as two.
+
+  The refinement: the per-needle counts OVERLAP and do not sum to the hit count (a row holding both a token and a secret is counted by both). The output line says so now, because two numbers printed under a total invite being added up, and a reader who adds them and gets more than the total loses confidence in the whole report.
+
+---
+
 ## [0.32.0] — 2026-08-10
 
 ### ⚠ BREAKING

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@voltro/plugin-audit",
-  "version": "0.32.0",
+  "version": "0.33.0",
   "description": "Audit plugin — ships the `audit()` schema mixin (createdAt/updatedAt/createdBy/updatedBy → Actor) plus an optional mutation interceptor that records every call to a configurable sink (console / memory / custom function).",
   "keywords": [
     "voltro",
@@ -27,7 +27,8 @@
       "types": "./dist/mixin.d.ts",
       "import": "./dist/mixin.js",
       "default": "./dist/mixin.js"
-    }
+    },
+    "./package.json": "./package.json"
   },
   "main": "./dist/index.js",
   "module": "./dist/index.js",
@@ -37,9 +38,9 @@
     "node": ">=24.0.0"
   },
   "dependencies": {
-    "@voltro/database": "0.32.0",
-    "@voltro/logger": "0.32.0",
-    "@voltro/protocol": "0.32.0"
+    "@voltro/database": "0.33.0",
+    "@voltro/logger": "0.33.0",
+    "@voltro/protocol": "0.33.0"
   },
   "peerDependencies": {
     "effect": "^3.22.0"