@voltro/cli 0.1.0

package/templates/agent-docs/database/scaling.md

@@ -0,0 +1,331 @@
+# database.scaling
+
+> Transparent read-replica routing with read-your-writes (RYW) consistency, multi-instance RYW via Redis, region-aware replica selection, and the per-dialect adapter surface.
+
+
+
+---
+
+<!-- source: en/database/read-replicas.md -->
+## Read replicas
+
+_Transparent read-replica routing with read-your-writes (RYW) consistency, multi-instance RYW via Redis, region-aware replica selection, and the per-dialect adapter surface._
+
+The framework's `DataStore` can wrap a primary connection + a list
+of read replicas behind one transparent surface. Writes go to the
+primary. Reads query to a replica chosen by power-of-two-choices,
+EXCEPT when the same subject just wrote — RYW (read-your-writes)
+pins their next reads to the primary for a TTL window so they
+never see stale data after their own changes.
+
+This is **completely opt-in**. Without `DB_REPLICA_URLS` set, the
+framework returns the primary store directly — no `ReplicatedDataStore`
+wrapper, no per-call routing overhead, no cost.
+
+## Quick start
+
+Add replica URLs:
+
+```sh
+DB_PRIMARY_URL=postgresql://app:app@primary.db.acme.com/app
+DB_REPLICA_URLS=postgresql://app:app@replica.db.acme.com/app
+voltro start
+```
+
+Verify in the boot log:
+
+```
+[voltro:dev] read replicas: 1 configured, ryw policy 'fallback', dialect=postgres, region=unknown
+```
+
+That's it. Reads now flow to the replica unless RYW pins them back.
+Writes always go to primary.
+
+## Read-your-writes (RYW)
+
+After a mutation commits, the subject who wrote is **pinned to
+primary** for the next 30 seconds (configurable). This guarantees
+they see their own change on the next read, even if the replica
+hasn't caught up yet.
+
+How it's implemented under the hood per dialect:
+
+| Dialect | Capture surface | Probe surface |
+|---|---|---|
+| Postgres | `pg_current_wal_lsn()` | `pg_last_wal_replay_lsn()` |
+| MySQL / MariaDB | `@@global.gtid_executed` (MySQL) / `@@global.gtid_current_pos` (MariaDB) | same, on the replica |
+| MSSQL | `end_of_log_lsn` from `sys.dm_hadr_database_replica_states` (AG only) | `last_hardened_lsn` from the same DMV |
+| SQLite | n/a (single-process — replicas not applicable) |
+
+The framework calls `capturePrimaryPosition()` immediately after
+every successful write, stores it keyed by the subject, and consults
+the cache on every subsequent read by that subject. No `position` →
+fall through to the replica selector.
+
+### RYW policies
+
+Three modes via `RYW_POLICY` env or `rywPolicy:` in config:
+
+| Policy | Behaviour |
+|---|---|
+| `fallback` (default) | Pin to primary for the TTL window. Simple, low-tail-latency, no replica wait. |
+| `wait` | Poll the replica until it catches up (max `rywMaxWaitMs`). Higher consistency at cost of latency. |
+| `off` | No RYW — every read goes to a replica regardless of write history. For analytics workloads where staleness is OK. |
+
+For most apps, `fallback` is correct: it prevents the "I just edited
+this, why is it gone" UX without paying the replica-poll cost. Reach
+for `wait` only when stronger consistency outweighs latency.
+
+## Replica selection
+
+By default the framework uses **power-of-two-choices** (P2C): pick
+two replicas at random, send the read to the one with fewer
+in-flight queries. Standard load-balancer trick — robust under skew,
+no global LRU/LFU state needed.
+
+### Region-aware selection (multi-region)
+
+When replicas span regions, set `DB_REPLICA_REGIONS` to a
+comma-separated list aligned with `DB_REPLICA_URLS`:
+
+```sh
+DB_REPLICA_URLS=postgres://repl-use1.acme/,postgres://repl-euw1.acme/
+DB_REPLICA_REGIONS=us-east-1,eu-west-1
+```
+
+The framework auto-detects the current process's region from these
+env vars (in priority order):
+
+1. `VOLTRO_REGION` — explicit override
+2. `AWS_REGION` — AWS ECS / Lambda / EC2
+3. `FLY_REGION` — Fly.io
+4. `RAILWAY_REGION` — Railway
+5. `GCP_REGION` / `CLOUD_RUN_REGION` — Google Cloud
+
+The selector then wraps P2C with a locality preference: **same-region
+replicas are tried first**, other-region replicas only when
+same-region is empty or unavailable. Boot log confirms:
+
+```
+[voltro:dev] read replicas: 2 configured, ryw policy 'fallback', dialect=postgres, region=us-east-1, replica-locality=on
+```
+
+On a wrong-region read, you pay 80-150ms cross-region latency. On a
+same-region read, 1-2ms. The selector does its best to keep you in
+the second bucket.
+
+If `DB_REPLICA_REGIONS` count mismatches `DB_REPLICA_URLS` count,
+the framework warns + disables locality (degrades to plain P2C
+across all replicas — correct but slower).
+
+## Multi-instance RYW: Redis
+
+The default RYW store is a per-process `Map`. Multi-instance
+deployments (k8s with `replicas: 3`, multi-pod ECS, etc.) need a
+shared store, otherwise a write on instance A doesn't pin reads on
+instance B.
+
+Wire Redis:
+
+```sh
+RYW_STORE=redis REDIS_URL=redis://host:6379
+voltro start
+```
+
+The Redis store keeps a local mirror per instance, kept warm via
+pub/sub — reads stay sync (`Map.get`), writes propagate to peers
+within one Redis round-trip. Sub-millisecond on localhost, 1-2ms
+across a VPC.
+
+What you get:
+
+- Cross-instance coherent RYW: A's write pins B's reads for the TTL.
+- Server-side TTL via PSETEX — entries expire automatically even
+  when no `get` triggers lazy eviction.
+- Self-publish loopback filtering: an instance ignores its own pub/sub
+  broadcasts to avoid double-applying.
+
+What you don't get:
+
+- Strong consistency. Between the moment A commits + the moment B
+  receives the pub/sub broadcast, B's reads could still hit a
+  stale replica. The window is one Redis round-trip — small but
+  nonzero. For strict-consistency workloads use `RYW_POLICY=wait`
+  on top of Redis store.
+
+## Performance
+
+Routing overhead is sub-microsecond per call. The micro-bench in
+`packages/runtime/src/replicatedDataStore.perf.test.ts` measures
+~10µs per routing decision including the AsyncLocalStorage context
+push/pop. The actual SQL query dominates wall-clock by orders of
+magnitude.
+
+If you see latency regressions, run that test in your fork:
+
+```sh
+pnpm -F @voltro/runtime test -- replicatedDataStore.perf
+```
+
+It logs the per-call cost and fails if you've accidentally turned
+the routing path into an O(n²) walk or added an unnecessary
+allocation per query.
+
+## Dialect support matrix
+
+| Dialect | Routing | RYW | Region-aware | E2E validated |
+|---|---|---|---|---|
+| Postgres | ✓ | streaming-replication LSN | ✓ | docker-compose (Phase 4 / C10) |
+| MySQL 8+ | ✓ | GTID | ✓ | docker-compose (Phase 5) |
+| MariaDB 10.6+ | ✓ | GTID (via mysql adapter) | ✓ | shares MySQL adapter |
+| MSSQL 2019+ | ✓ AG only | AG LSN | ✓ | adapter validated, AG cluster setup is operator-managed |
+| SQLite | n/a | n/a | n/a | single-process, no replication possible |
+
+MSSQL routing requires Always-On Availability Group cluster setup
+(certs + endpoints + listener + Pacemaker on Linux OR Windows
+Failover Clustering). That's a deployment-grade infrastructure
+setup, not a framework concern; the adapter validates against any
+SQL Server but real replica routing needs AG. Without AG, both
+capture + probe return the zero-LSN sentinel and the framework
+gracefully degrades to "treat every read like a primary read"
+(safe, just slower than it could be).
+
+## Anti-patterns
+
+- **Don't run `RYW_STORE=redis` without `REDIS_URL`** — the framework
+  warns + falls back to memory store. Multi-instance deployments
+  silently lose RYW coherence.
+- **Don't set `DB_REPLICA_REGIONS` without `DB_REPLICA_URLS`** — the
+  region map is paired by order with the URL list. Without URLs the
+  region list has nothing to bind to and is ignored.
+- **Don't use `wait` policy with high write-rate subjects** — every
+  read after a write polls the replica with a sub-second timeout.
+  The tail latency is bounded by `rywMaxWaitMs`, but for hot subjects
+  this can starve.
+- **Don't manually pin reads to a specific replica from handler code.**
+  The framework's routing layer is the single decision point — if you
+  bypass it, you lose RYW correctness + observability counters.
+
+
+
+---
+
+<!-- source: en/database/multi-replica.md -->
+## Multi-replica reactivity
+
+_Cross-replica change fan-out behind a load balancer — native LISTEN/NOTIFY (postgres) and binlog CDC (mysql + mariadb), plus @voltro/plugin-broadcast (Redis / NATS) to close the gap for every other dialect._
+
+Behind a load balancer, each live client WebSocket lives on exactly **one** replica. Replica A holds clients 1–100, replica B holds 101–200, and so on. A mutation runs on whichever replica received the request, and inline-emits the change to **that replica's** local subscribers. The other replicas never learn of the write — so their clients go stale until they refetch.
+
+This page is about closing that gap: making a write on **any** replica reach the live subscriptions on **every** replica.
+
+## The three tiers
+
+How a change crosses replica boundaries depends on the dialect:
+
+| Tier | Dialects | Mechanism | Cross-replica? |
+|---|---|---|---|
+| Native DB fan-out | postgres | `LISTEN/NOTIFY` — the database IS the bus | ✓ |
+| Native DB fan-out | mysql · mariadb | binlog CDC (ROW format) — every replica tails the binlog | ✓ |
+| Pub/sub bus | mssql · cockroachdb · planetscale · azure-sql | `@voltro/plugin-broadcast` (Redis / NATS) | ✓ (when wired) |
+| Single-instance | sqlite · `memory` | in-process only | n/a (single process by design) |
+
+Postgres (`LISTEN/NOTIFY`) and MySQL/MariaDB (ROW-format binlog CDC) have a native cross-instance path: every replica learns of a write directly from the database. For **the other SQL dialects there is no native cross-instance path** — reactivity silently degrades to single-instance. `@voltro/plugin-broadcast` closes that with a pub/sub message bus. (PlanetScale is MySQL-based but Vitess doesn't expose a raw binlog to external readers, so it stays on the bus tier.)
+
+## @voltro/plugin-broadcast
+
+A first-class, provider-pluggable plugin that fans out app-mutation change events to every replica over a pub/sub broker. Two backends ship: **Redis** and **NATS**.
+
+```ts
+// app.config.ts
+import { broadcastPlugin } from '@voltro/plugin-broadcast'
+
+export default {
+  type: 'api' as const,
+  name: 'api',
+  plugins: [
+    broadcastPlugin(),  // reads BROADCAST_URL / BROADCAST_PROVIDER / REDIS_URL from env
+  ],
+}
+```
+
+```sh
+# Redis (RESP pub/sub)
+BROADCAST_URL=redis://localhost:6379
+
+# or NATS
+BROADCAST_URL=nats://localhost:4222
+
+# explicit provider override (otherwise inferred from the URL scheme)
+BROADCAST_PROVIDER=redis
+```
+
+If `BROADCAST_URL` is unset the plugin falls back to an in-process **memory** provider — useful in tests, but **single-process**: a real cross-replica bus needs a broker URL. The boot banner warns when this happens.
+
+You can also pass a pre-built provider or an explicit name:
+
+```ts
+import { broadcastPlugin, redisProvider } from '@voltro/plugin-broadcast'
+
+broadcastPlugin({ provider: redisProvider({ url: process.env.REDIS_URL! }) })
+broadcastPlugin({ provider: 'nats', url: 'nats://nats:4222' })
+```
+
+`ioredis` and `nats` are **optional dependencies** — installed only for the backend you use. The plugin dynamically imports the driver, so an app on `memory` never pulls either.
+
+## Mechanism — additive, not a replacement
+
+The bus is **additive** to the inline emit path. Local reactivity must survive a broker outage:
+
+1. A local change (via `store.onChange`) publishes `{ origin: replicaId, event }` to the channel `voltro:changes`.
+2. Every replica subscribes. On a message whose `origin` is **not** this replica, it injects the change event into the local store emitter — the same `injectExternalChange` seam the postgres `LISTEN/NOTIFY` consumer uses.
+3. The writer's **own** broadcast is skipped (it already inline-emitted locally) — so there is no double-emit and no dedup table.
+
+Because the inline path is never removed, a broker outage degrades **cross-replica** fan-out only — local reactivity keeps working, and the framework logs a warning. The bus reconnects when the broker returns.
+
+## The honest caveat — app-mutation changes only
+
+The bus carries changes that flow through **`ctx.store`** (the framework's mutation path). It does **not** capture **out-of-band DB writes** — a `psql` session, a cron job, or a second service writing the same database directly. Those changes never hit `store.onChange`, so they never reach the bus.
+
+Only two mechanisms observe out-of-band writes:
+
+- **postgres `LISTEN/NOTIFY`** — the DB itself fires on any committed change (the framework's triggers fan out every write, regardless of who made it).
+- **mysql / mariadb binlog CDC** — the ROW-format binlog records every committed row change; both engines share one reader.
+
+If your app shares its database with other writers and needs them to drive reactivity, choose postgres, mysql, or mariadb. If all writes go through the Voltro app (the common case), the broadcast bus is the right tool for non-native dialects.
+
+## What about polling a changelog table?
+
+**Polling is not built.** It's documented here only as the absolute last resort.
+
+A changelog table that every replica polls (`SELECT … WHERE seq > :last`) would observe out-of-band writes on any dialect — but at a cost the pub/sub bus avoids entirely: ~150 ms latency (poll interval) vs ~1 ms, constant DB load from every replica on every tick, and a changelog table to vacuum. Pub/sub beats it on every axis, is dialect-agnostic, and Voltro already runs Redis pub/sub for read-your-writes consistency. If you genuinely need out-of-band-write reactivity on a dialect that can't do `LISTEN/NOTIFY` or binlog, that's the signal to move to postgres — not to bolt on polling.
+
+## Boot banner
+
+`voltro dev` / `voltro start` prints the resolved reactivity tier so `voltro logs --tail 50` answers "how do writes cross replicas here" without reading source:
+
+```
+[voltro:dev] reactivity: cross-instance via native LISTEN/NOTIFY (postgres)
+```
+
+```
+[voltro:dev] reactivity: cross-instance via @voltro/plugin-broadcast (redis) for dialect 'planetscale'
+```
+
+```
+[voltro:dev] reactivity: cross-instance fan-out is OFF for dialect 'mssql'. A write on one replica
+will NOT reach clients on other replicas. Add @voltro/plugin-broadcast (Redis / NATS) to close the gap…
+```
+
+When BOTH a native path and the broadcast plugin are wired (e.g. postgres + broadcast), both stay active — the own-origin skip dedups, so there's no double-emit. The native path is the primary; the bus is harmless redundancy.
+
+## Where it lives in the codebase
+
+- `voltro/packages/plugin-broadcast` — the plugin, the `BroadcastProvider` contract, the redis / nats / memory providers, `attachBroadcastBus`.
+- `voltro/packages/database/src/dataStore.ts` — `DataStore.injectExternalChange` (the cross-instance seam).
+- `voltro/packages/cli/src/dev.ts` `wireBroadcastBus()` — tier selection + boot banner.
+
+## See also
+
+- [SQL dialects](./dialects) — the per-dialect parity table (CDC row).
+- [Read replicas](./read-replicas) — read routing + read-your-writes consistency (a different axis: which replica serves a *read*, not how a *write* fans out).