@voltro/cli 0.52.0 → 0.53.0

package/templates/agent-docs/plugins/queue.md

@@ -0,0 +1,129 @@
+# Queue (Kafka interop)
+
+> Consume and produce against an existing Kafka — Schema-decoded consumers (at-least-once, serial per partition, retry + dead-letter), batched producing via a handler service or transactionally through the outbox, and a kafkaSink for cdc-out.
+
+
+
+---
+
+<!-- source: en/plugins/queue.md -->
+## Queue (Kafka interop)
+
+_Consume and produce against an existing Kafka — Schema-decoded consumers (at-least-once, serial per partition, retry + dead-letter), batched producing via a handler service or transactionally through the outbox, and a kafkaSink for cdc-out._
+
+`@voltro/plugin-queue` is the door to queues **somebody else owns**: a Voltro
+backend consuming and producing against an adopter's existing Kafka. The
+boundary with the built-ins, in one line each: the [outbox](/docs/data/outbox)
+is *your own* durable side-effects, a [workflow](/docs/workflows/overview) is
+*your own* orchestration — this plugin is interop with foreign infrastructure.
+Kafka first; the provider contract is cut so SQS/RabbitMQ can be later
+implementations.
+
+```ts
+// app.config.ts
+import { queuePlugin } from '@voltro/plugin-queue'
+
+export default {
+  // …
+  plugins: [
+    queuePlugin({ brokers: ['kafka-1:9092', 'kafka-2:9092'] }),
+  ],
+}
+```
+
+## Consuming: `*.consumer.ts`
+
+```ts
+// src/consumers/orders.consumer.ts
+import { Schema } from 'effect'
+import { defineQueueConsumer } from '@voltro/plugin-queue'
+
+export const orders = defineQueueConsumer({
+  topic: 'orders',
+  schema: Schema.Struct({ orderId: Schema.String, total: Schema.Number }),
+  handler: async (order, ctx) => {
+    // MUST be idempotent — delivery is at-least-once. A unique-column
+    // upsert is the standard shape:
+    await ctx.store.upsert('orders_mirror',
+      { orderId: order.orderId, total: order.total },
+      { conflictColumns: ['orderId'] })
+  },
+})
+```
+
+Both boot paths discover `*.consumer.ts`; the plugin starts every registered
+consumer at activation and stops them at shutdown. The semantics, precisely:
+
+- **Ordering: serial per partition.** Parallelism exists only ACROSS
+  partitions — concurrency inside one would destroy ordering and commit
+  semantics both. Retry backoff deliberately BLOCKS the partition.
+- **Commit after the handler, per message.** A process killed mid-batch
+  redelivers exactly the unhandled tail — never the whole batch, never a
+  skipped message.
+- **Decode failures dead-letter IMMEDIATELY** (to `<topic>.dlq`, with
+  `x-voltro-dlq-*` reason headers) — a deterministic failure retried forever
+  is an infinite loop with extra steps, and a poison message must release
+  its partition.
+- **Handler failures retry with backoff, then dead-letter** after
+  `maxAttempts` (default 3).
+- **A rebalance is not a failure.** A partition revoked mid-batch or
+  mid-retry stops processing without a retry-counter increment or a DLQ
+  publish — the new owner redelivers.
+- **Handlers are NOT wrapped in a transaction** (the same documented
+  boundary as HTTP route handlers). A handler needing atomic multi-writes
+  opens `ctx.store.transactional` itself — and stays idempotent either way.
+- **Replica coordination is Kafka's own.** Every replica joins the same
+  consumer group and the broker assigns partitions — no advisory lock, unlike
+  [schedules](/docs/scheduling/coordination), which coordinate through the
+  claim table because no broker exists to do it for them.
+
+## Producing
+
+Two paths, one rule: transactional-with-a-write goes through the outbox.
+
+```ts
+// Inside a mutation — commits or rolls back WITH the domain write:
+await ctx.outbox.enqueue('queue.produce', {
+  topic: 'orders',
+  messages: [{ key: order.id, value: JSON.stringify(order) }],
+})
+```
+
+```ts
+// src/queue.outbox.ts — the bridge (once per app):
+import { queueOutboxHandler } from '@voltro/plugin-queue'
+export default queueOutboxHandler()
+```
+
+The outbox runner delivers after commit — batched (`messages` is an array →
+one transport round-trip), at-least-once, retried, dead-lettered. One
+durability path: the existing outbox, not a second one. For fire-and-forget
+producing without a surrounding write, `yield* QueueService` in a handler and
+call `produce(topic, messages)` directly.
+
+Topic creation is EXPLICIT (`provider.ensureTopics([...])`) — your Kafka is
+foreign infrastructure, and whether a client may create topics is your
+policy. A consumer started against a topic that does not exist yet warns and
+retries in the background (it connects once the topic appears), never
+aborting the boot.
+
+## cdc-out to Kafka
+
+`kafkaSink` plugs table-change mirroring ([plugin-cdc-out](/docs/plugins/cdc-out))
+into the SAME provider: message key = the row id (one row's changes stay
+ordered in one partition), value = the change record, and the
+`x-voltro-delivery-key` header carries cdc-out's at-least-once dedupe handle.
+
+```ts
+import { cdcOutPlugin } from '@voltro/plugin-cdc-out'
+import { kafkaSink } from '@voltro/plugin-queue'
+
+cdcOutPlugin({ sinks: [{ table: 'orders', sink: kafkaSink({ topic: 'orders.cdc' }) }] })
+```
+
+## Observability
+
+Per-topic counters (consumed / retried / dead-lettered / produced + the last
+error) on `GET /_voltro/inspect/plugins/queue/consumers` and in the
+dashboards' Queue panel. Message headers carry `traceparent` through to
+`ctx.traceparent` for cross-system trace continuity.

package/templates/agent-docs/plugins.md

@@ -42,7 +42,9 @@ The framework ships some plugins; you write your own; the contract is small enou
 - [plugin-logship](/docs/plugins/logship) — ship structured logs to Better Stack / Axiom / Loki / any HTTP sink; batched, redacted, fail-soft
 - [plugin-moderation](/docs/plugins/moderation) — moderate user content before commit: keyword or AI provider, block / flag via interceptor + in-handler redact
 - [plugin-search](/docs/plugins/search) — keep an external index (Typesense / Meilisearch / Algolia) in sync via the ChangeEvent tap; tenant-scoped `search.query` + hook
-- [plugin-cdc-out](/docs/plugins/cdc-out) — declarative reverse-ETL: mirror table changes outward to a webhook sink (or any custom `CdcSink`) through a durable outbox; ordered per pipe, at-least-once from enqueue, dead-lettered
+- [plugin-cdc-out](/docs/plugins/cdc-out) — declarative reverse-ETL: mirror table changes outward to a webhook or Kafka sink (or any custom `CdcSink`) through a durable outbox; ordered per pipe, at-least-once from enqueue, dead-lettered
+- [plugin-queue](/docs/plugins/queue) — Kafka interop: Schema-decoded consumers (`*.consumer.ts`; at-least-once, serial per partition, retry + dead-letter), batched producing via a service or transactionally through the outbox, and a `kafkaSink` for cdc-out
+- [plugin-comments](/docs/plugins/comments) — comment threads on any app entity: replies, resolve/reopen, tenant-safe @-mentions with notifications, reactions, unread — live over the reactive engine, with the ejectable `<CommentsThread>` UI
 - [plugin-governance](/docs/plugins/governance) — data governance: retention TTL sweep, GDPR export + erasure, consent ledger, field encryption
 - [plugin-openapi](/docs/plugins/openapi) — OpenAPI 3.1 spec + Swagger-UI docs generated from your `defineRestRoute` descriptors and (opt-in) rpc procedures
 - [plugin-row-history](/docs/plugins/row-history) — full row history + time-travel (`rowHistory` / `rowAsOf` / `restoreAsOf` / `diffVersions`); what-changed-to-what on every write
@@ -74,13 +76,15 @@ Status legend: ✓ shipped · ◐ partial · — planned.
 | `@voltro/plugin-postgis` | ✓ | Postgres-native `geography` / `geometry` columns + spatial predicates (`ST_DWithin`, `ST_Contains`, `ST_Intersects`); GiST indexes via `.expressionIndex(..., { kind: 'gist' })`. No `ST_Distance` projection yet. Postgres-only by design (fails loud elsewhere). [→ details](/docs/plugins/postgis) |
 | `@voltro/plugin-broadcast` | ✓ | Cross-replica reactivity — fans out app-mutation change events to every replica over a pub/sub bus (Redis / NATS). Closes the single-instance gap for every non-postgres dialect. [→ details](/docs/plugins/broadcast) |
 | `@voltro/plugin-webhooks` | ✓ | Incoming + outgoing webhooks — `defineIncomingWebhook` (signature verify + idempotency, Stripe/GitHub/Slack presets) and `defineEvent` (durable delivery workflow, HMAC signing, retries, filters). [→ details](/docs/plugins/webhooks) |
+| `@voltro/plugin-comments` | ✓ | Comment threads anchored to any entity — fail-closed access delegation (`viaEntity`/`scope`), replies, resolve/reopen, tenant-safe mentions (validated twice, delivered via plugin-notifications incl. digests), reactions, per-subject unread, live `comments.list`, `<CommentsThread>` in `@voltro/ui`. [→ details](/docs/plugins/comments) |
+| `@voltro/plugin-queue` | ✓ | Kafka interop — `defineQueueConsumer` (`*.consumer.ts`, Schema-decoded, at-least-once, serial per partition, retry + DLQ with reason headers), batched producing via `QueueService` or transactionally through the outbox (`queueOutboxHandler`), `kafkaSink` for cdc-out, per-topic counters in the dashboards. [→ details](/docs/plugins/queue) |
 | `@voltro/plugin-auth-social` | ✓ | First-party social login — Sign in with Google / GitHub / Apple with no identity vendor: authorize URL + code exchange + JWKS-verified ID tokens, mandatory PKCE (S256) and `state`, an explicit account-linking policy (`never` by default), Apple's signed-JWT client secret / one-time name / private-relay email all handled; sessions via `issueUserSession`. [→ details](/docs/plugins/auth-social) |
 | `@voltro/plugin-auth-{workos,kinde,clerk,auth0,supabase,oidc}` | ✓ | Six IdP adapters over the shared `jwtBearerStrategy` — JWKS verify + claims→tenant mapping; WorkOS additionally ships hosted-login OAuth primitives (`workosAuthorizationUrl` / `workosAuthenticateWithCode`) for a redirect-based SSO login flow. [→ details](/docs/authentication/external-idp) |
-| `@voltro/plugin-analytics-postgres` | ✓ | First-party lite — events on the main DataStore, cross-dialect (postgres / mysql / mariadb / mssql / sqlite / turso). [→ details](/docs/plugins/analytics#voltroplugin-analytics-postgres) |
-| `@voltro/plugin-duckdb` | ✓ | Embedded DuckDB sidecar — real OLAP performance, no external service. [→ details](/docs/plugins/analytics#voltroplugin-duckdb) |
-| `@voltro/plugin-clickhouse` | ✓ | Production OLAP via the official ClickHouse client. [→ details](/docs/plugins/analytics#voltroplugin-clickhouse) |
-| `@voltro/plugin-tinybird` | ✓ | Hosted ClickHouse via Events API + Pipes. [→ details](/docs/plugins/analytics#voltroplugin-tinybird) |
-| `@voltro/plugin-posthog` | ✓ | Product analytics — track-only; compose with another sink for reads. [→ details](/docs/plugins/analytics#voltroplugin-posthog) |
+| `@voltro/plugin-analytics-postgres` | ✓ | First-party lite — events on the main DataStore, cross-dialect (postgres / mysql / mariadb / mssql / sqlite / turso). [→ details](/docs/plugins/analytics) |
+| `@voltro/plugin-duckdb` | ✓ | Embedded DuckDB sidecar — real OLAP performance, no external service. [→ details](/docs/plugins/analytics) |
+| `@voltro/plugin-clickhouse` | ✓ | Production OLAP via the official ClickHouse client. [→ details](/docs/plugins/analytics) |
+| `@voltro/plugin-tinybird` | ✓ | Hosted ClickHouse via Events API + Pipes. [→ details](/docs/plugins/analytics) |
+| `@voltro/plugin-posthog` | ✓ | Product analytics — track-only; compose with another sink for reads. [→ details](/docs/plugins/analytics) |
 | `@voltro/plugin-atlassian` | ✓ | `JiraService` + `ConfluenceService` over the Atlassian REST / Greenhopper / Agile APIs — PAT **or** OAuth 2.0 (3LO) auth, transient retry, SSRF guard, comment-write, signature-verified inbound webhooks, avatar proxy, per-tenant cache. [→ details](/docs/plugins/atlassian) |
 | `@voltro/plugin-deactivation` | ✓ | `deactivation()` schema mixin — `deactivatedAt` + `deactivatedBy` (→ Actor); subject can't log in but data stays visible. [→ details](/docs/plugins/deactivation) |
 | `@voltro/plugin-prometheus` | ✓ | Prometheus exporter — `GET /metrics` in text exposition format over the unified Metrics-API (Effect `MetricRegistry`); counters / histograms / gauges + custom metrics, optional bearer gate + node process metrics. [→ details](/docs/plugins/prometheus) |
@@ -880,7 +884,7 @@ A plugin can mount plain HTTP routes beside the rpc surface (`@voltro/plugin-sto
 
 - **The full method union is first-class.** `method` is `'*' | 'GET' | 'POST' | 'PUT' | 'PATCH' | 'DELETE' | 'HEAD' | 'OPTIONS'`. **HEAD is admitted wherever GET is** (RFC 9110) — the GET handler runs and the transport drops the body; you never mount a second route for it. A wrong method stays a precise `405` with an `Allow:` header, including when several routes share one path.
 - **The body read is capped** — 8 MiB by default, the same cap as every other surface (`http.maxBodyBytes` in `app.config.ts`, env `VOLTRO_MAX_BODY_BYTES`), and the read is binary-clean. A route that takes more declares its own `maxBodyBytes`; routes **sharing a path share one body read**, so the widest override in the group applies to the group. Oversize answers `413` for both `Content-Length` and chunked requests.
-- **Binary streaming responses** return `byteStream` on the `PluginHttpRouteResult` — a `ReadableStream<Uint8Array>` (or lazy thunk) with optional `contentLength` / `contentDisposition`, piped without buffering and never compressed. It is the plugin-route spelling of the REST surface's [`bytes()`](/docs/data/rest-routes#binary-downloads--bytes).
+- **Binary streaming responses** return `byteStream` on the `PluginHttpRouteResult` — a `ReadableStream<Uint8Array>` (or lazy thunk) with optional `contentLength` / `contentDisposition`, piped without buffering and never compressed. It is the plugin-route spelling of the REST surface's [`bytes()`](/docs/data/rest-routes#binary-downloads-bytes).
 - **Buffered responses are compression-negotiated** (brotli/gzip, compressible types only) by the listener — nothing to declare; see [Security → compression](/docs/security/overview).
 
 A state-changing plugin route is origin-checked unless it declares `originGuard: 'exempt'`, and `req.remoteAddr` is the trusted-proxy-resolved client address — both covered with examples in [Security](/docs/security/overview#routes-that-a-third-party-legitimately-posts-to).

package/templates/agent-docs/reference.md

@@ -225,6 +225,18 @@ const { data } = useSubscription(
 )
 ```
 
+### Offline semantics with the local-first mirror
+
+With `@voltro/local-first`'s query mirror bound (see
+[the sync engine](/docs/local-first/overview#the-sync-engine-query-mirror--durable-outbox)),
+`useSubscription`'s behaviour extends offline WITHOUT a second API: a cold
+start seeds `data` (and `revision`) from the device's mirrored rows for the
+subject's partition, so `loading` resolves against local data when the server
+is unreachable; the first live event replaces it, and a reconnect inside the
+resume window continues with deltas from the mirrored revision. Offline
+WRITES ride [`useOutbox`](/docs/ui/client-utilities/use-outbox) — durable
+with `outboxPersistence()`, conflict resolution via `resolveConflict`.
+
 ## `useMutation(apiName, rpcTag)`
 
 Calls a `defineMutation` RPC.
@@ -592,7 +604,7 @@ window.history.forward()   // forward
 
 ## `useBlocker()`
 
-Hold a pending navigation so you can prompt before the user leaves — the unsaved-changes guard.
+Hold a pending navigation so you can prompt before the user leaves — the unsaved-changes guard. It guards SPA navigations, the browser's **Back/Forward gestures** (popstate: the router reverts the already-moved URL and offers `retry`/`reset` — this also covers ESC inside an [intercepting-route overlay](/docs/routing/intercepting-routes)), and full-page unloads via `beforeunload`.
 
 ```tsx
 import { useBlocker } from '@voltro/web'
@@ -637,7 +649,7 @@ setParams((p) => { p.set('page', '2'); return p })   // patch one param
 setParams({ page: '2' }, { push: true })             // distinct history entry
 ```
 
-Writes default to a history replace; pass `{ push: true }` for a Back entry or `{ scroll: false }` to keep scroll. See [Navigation](/docs/routing/navigation#reading--writing-search-params).
+Writes default to a history replace; pass `{ push: true }` for a Back entry or `{ scroll: false }` to keep scroll. See [Navigation](/docs/routing/navigation#reading-writing-search-params).
 
 `useSetSearchParams(searchParams)` — pass the schema to get the **typed** setter. Object form replaces the query (a left-out field decodes to its default on the next read); the updater form receives the current **decoded** params, so a merge is an explicit spread:
 
@@ -715,7 +727,7 @@ Precisely, it returns `LoaderData<T>`. For every ordinary loader that IS `T`. Fo
 a loader that returned `defer()`, `LoaderData<T>` flattens the two buckets into
 one object — eager fields as values, deferred fields as `Promise<T>` — so the
 compiler tells you which fields have to be rendered through
-[`<Await>`](/docs/routing/loaders-and-meta#deferring-slow-data-defer--await):
+[`<Await>`](/docs/routing/loaders-and-meta#deferring-slow-data-defer):
 
 ```tsx
 export const loader = async ({ query }) => defer(
@@ -856,7 +868,7 @@ that must react to router-pushed query changes without a reload re-render throug
 
 Prefer the typed form where the page declares a `searchParams` schema export —
 `useSearchParams(searchParams)` returns the decoded shape instead of a raw
-`URLSearchParams`. See [Routing hooks](/docs/reference/hooks-routing#usesearchparams--usesetsearchparams).
+`URLSearchParams`. See [Routing hooks](/docs/reference/hooks-routing#usesearchparams-usesetsearchparams).
 
 ## Reading cookies