actor-ts 0.9.1 → 0.10.0

package/CHANGELOG.md

@@ -9,6 +9,388 @@ breaking.  See `ROADMAP.md` for what's coming, and `README.md` →
 
 ## [Unreleased]
 
+## [0.10.0] — 2026-07-08
+
+### Added — Typed WebSocket & MQTT
+
+- **Typed WebSocket routing** (#1) — a `websocket(path, actorRef)` directive
+  in the HTTP routing DSL binds a `WebSocketServerActor<TOut, TIn>`.  The
+  hub receives codec-decoded messages (JSON by default; `rawCodec()` for
+  binary), replies to the sending connection with `this.reply(...)` or
+  fans out with `this.broadcast(...)`, and gets `onClientConnected` /
+  `onClientDisconnected` / `onInvalidMessage` hooks.  The framework spawns
+  an internal session actor per connection and solves the first-frame
+  race by construction (listeners attach synchronously at upgrade; the
+  mailbox is the buffer).  Works on all three HTTP backends — Fastify
+  (`@fastify/websocket`), Express (`ws` upgrade handling), and Hono
+  (per-runtime: Bun/Node/Deno).  `withMiddleware(...)` runs at upgrade
+  time, so `BearerTokenAuth` / `IpAllowlist` gate the handshake.
+- **`WebSocketClientActor<TOut, TIn>`** (#1) — the typed client half, built
+  on `BrokerActor`: reconnect-with-backoff, outbound buffering across
+  reconnects, circuit breaker, and HOCON settings.  Other actors push a
+  typed send with `ref.tell(wsSend(msg))`.
+- **`actor-ts.http.websocket` HOCON block** — server-side WebSocket
+  defaults (`maxFrameBytes`, `onOversizeFrame`, `onInvalidMessage`,
+  `maxBufferedBytes`, `onBackpressure`); route options override HOCON,
+  which overrides built-in defaults.
+- **Subclass-first, typed `MqttActor<T, TSelf>`** (#345) — the MQTT
+  counterpart to `WebSocketClientActor`.  Extend it, declare
+  subscriptions in the constructor with `this.subscribe(topic, { qos })`,
+  handle inbound traffic in `onMessage(msg)`, and publish with
+  `this.publish(topic, payload)`.  Lifecycle events (inbound / connected /
+  disconnected) run on the actor thread via the mailbox.  Hooks:
+  `onMessage`, `onConnected`, `onDisconnected`, `onDecodeError`,
+  `onSelfMessage`.  Still externally controllable via
+  `ref.tell({ kind: 'publish' | 'subscribe' | 'unsubscribe', … })`.
+- **`MqttOptions` fluent builder** (#345) — `MqttOptions.create()
+  .withBrokerUrl(…).withClientId(…).withQos(…)…`; feeds the same
+  three-layer settings merge (constructor > HOCON
+  `actor-ts.io.broker.mqtt` > built-in defaults).  (As of #346/#348 this is
+  the primary way to construct; a plain `MqttOptionsType` object works too —
+  see the options note under *Changed*.)
+- **Typed MQTT payloads** (#345) — inbound `MqttMessage<T>` carries a
+  lazily-decoding `MqttPayload<T>` (`.bytes` / `.text()` / `.entity<U=T>()`,
+  successes cached).  A pluggable `MqttCodec<T>` seam (default
+  `mqttJsonCodec()`) decodes `entity()` and encodes non-string publishes;
+  `publish(topic, { … })` encodes an entity, `publish(topic, string |
+  Uint8Array)` sends raw bytes.  Decode failures surface via
+  `onDecodeError`.  `MqttClientLike` / `MqttModuleLike` are exported as
+  test seams for the `mqttModule()` override.
+
+### Changed — Options overhaul & MQTT
+
+- **Fluent options, framework-wide** (#346, #348) — every configurable
+  constructor and factory takes a fluent options builder **or** a plain
+  settings object, interchangeably: `new MqttActor(MqttOptions.create()
+  .withClientId('x'))` behaves identically to `new MqttActor({ clientId:
+  'x' })`.  A builder *is* its settings — `OptionsBuilder` stores each field
+  as an own property, so a builder instance reads and spreads exactly like a
+  plain settings object (no separate resolve step; consumers just read the
+  argument).  Each configurable type exposes **three names from one
+  `XOptions.ts` file**: `XOptionsType` (the plain object), `XOptionsBuilder`
+  (the fluent builder, `XOptions.create().withField(…)`), and `XOptions` — the
+  **union** of the two that every consumer signature accepts (`options:
+  XOptions`), plus a value alias so `XOptions.create()` keeps working.  There
+  is no separate "Settings" concept (the former `XSettings` interface is now
+  `XOptionsType`, co-located in `XOptions.ts`).  HOCON resolution is
+  unchanged — the builder / plain object only supplies the
+  highest-precedence explicit layer, and unset fields still fall through
+  to config then defaults.  Naming lockstep with no divergence: builder
+  method `withX` ⇔ settings field `x` ⇔ HOCON leaf `x`.  Affected
+  (non-exhaustive): `ActorSystem.create(name, ActorSystemOptions
+  .create()…)`; `TestKit.create` / `new TestProbe` (`TestKitOptions` /
+  `TestProbeOptions`); every broker actor (`MqttOptions`, `KafkaOptions`,
+  `AmqpOptions`, `NatsOptions`, `JetStreamOptions`, `RedisStreamsOptions`,
+  `SseOptions`, `TcpSocketOptions`, `UdpSocketOptions`, `GrpcClientOptions`,
+  `GrpcServerOptions`); HTTP/WS (`WebSocketClientOptions`,
+  `WebSocketRouteOptions`, `ExpressBackendOptions`, `HonoBackendOptions`);
+  cache (`RedisCacheOptions`, `MemcachedCacheOptions`); persistence
+  journals / snapshot stores / durable-state stores / object-storage
+  backends / projections / plugin registrations; cluster / sharding /
+  singleton / client / pub-sub / router / downing / failure detectors;
+  leases, seed providers + discovery, observability adapters
+  (`OtelAdapterOptions`, `PromClientAdapterOptions`), `WorkerClusterOptions`,
+  `DistributedDataOptions`, and `ProducerControllerOptions`.  Migration:
+  `new X({ a, b })` still works, or use `new X(XOptions.create()
+  .withA(a).withB(b))`; the positional "context" args that were never
+  settings (a system name, a `Cluster`, a sharding entity + type name)
+  stay positional.  **BREAKING** (pre-1.0 hard cut): the builder class
+  `XOptions` is renamed `XOptionsBuilder` and the settings interface
+  `XSettings` is renamed `XOptionsType`; `XOptions` is now the accepted-input
+  union.  Everyday call sites — `XOptions.create()…` and plain objects — are
+  unaffected; only code that referenced the old `XSettings` type name or the
+  builder *class* by name needs updating.
+- **BREAKING: renamed settings fields + HOCON keys** (#348) — to keep the
+  builder-method ⇔ settings-field ⇔ HOCON-leaf names in lockstep, six
+  fields were renamed: MQTT `defaultQos` → `qos` (`withQos`) and
+  `keepAliveSec` → `keepAlive` (`withKeepAlive`); JetStream `ackTimeoutMs`
+  → `ackTimeout`; ClusterClient `log` → `logger`; DistributedData
+  `gossipIntervalMs` → `gossipInterval`; ProducerController
+  `resendTimeoutMs` → `resendTimeout`.  Update any plain settings objects
+  and HOCON keys using the old names.
+- **BREAKING: `MqttActor` is now abstract** (#345) — you subclass it and
+  override `onMessage` instead of spawning it directly and driving it only
+  with `tell`.  Migration: `class MyClient extends MqttActor<T> { … }` and
+  spawn the subclass.  A pure external-router setup needs a trivial
+  subclass with an empty `onMessage`.
+- **BREAKING: `MqttMessage.payload` is a `MqttPayload<T>` wrapper**, no
+  longer a raw `Uint8Array` (#345).  Migration: `msg.payload` →
+  `msg.payload.bytes`; `new TextDecoder().decode(msg.payload)` →
+  `msg.payload.text()`; JSON reads → `msg.payload.entity()`.
+- **BREAKING: `MqttOptionsType.subscriptions` and the `MqttSubscription`
+  type are removed** (#345) — they were never HOCON-expressible (targets
+  are actor refs).  Migration: move `subscriptions: [{ topic, target }]`
+  into the subclass constructor as `this.subscribe(topic, { target })`, or
+  send `ref.tell({ kind: 'subscribe', topic, target })`.
+- **BREAKING: `subscribe`/`unsubscribe` command `target` semantics**
+  (#345) — a `subscribe` command with no `target` now delivers to the
+  actor's own `onMessage` (previously `target` was required).  An
+  `unsubscribe` command with no `target` now removes only the *foreign*
+  targets and leaves the actor's own subscription intact (previously it
+  dropped the whole topic).
+
+### Fixed — MQTT
+
+- **MQTT runtime subscriptions are re-applied after a reconnect** (#345) —
+  previously only the (now-removed) `settings.subscriptions` were
+  re-subscribed on reconnect, so subscriptions added at runtime silently
+  stopped receiving after a drop.  The unified registry is now re-applied
+  on every (re)connect.
+- **MQTT `subscribe` while disconnected reaches the broker on connect**
+  (#345) — previously it updated only the local routing map and never
+  issued the broker SUBSCRIBE.
+- **MQTT terminated fan-out targets are cleaned up** (#345) — subscriber
+  refs are deathwatched; when one stops it is pruned from the registry and
+  a broker UNSUBSCRIBE fires once the pattern has no consumers left.
+
+### Removed — legacy frame-level WebSocket API
+
+- **BREAKING: the legacy frame-level WebSocket API** — `WebSocketActor`
+  (client), `ServerWebSocketActor` (server bridge), and the
+  `serverWebSocketActorOf` / `bunWebSocketHandlers` adapters are gone.
+  They worked at the raw text/binary frame level and needed ~150 lines of
+  backend-specific boilerplate to stand up a server.  Replace a
+  `WebSocketActor` with `WebSocketClientActor`, and a hand-rolled server
+  plugin with a `websocket(path, ref)` route + `WebSocketServerActor`.
+  The client HOCON key `actor-ts.io.broker.websocket` is unchanged.
+
+### Security
+
+- **WebSocket DoS hardening carried into the new stack** (#1) — inbound
+  frames are size-capped (`maxFrameBytes`, default 1 MiB) *before* the
+  codec decodes them; oversize frames close 1009 (or drop) and
+  undecodable frames close 1003 (or drop / hook) per policy.  Slow-consumer
+  backpressure closes/drops past `maxBufferedBytes`.
+
+- **DurableState revision tampering** (#116) — `ObjectStorageDurableStateStore.load()`
+  previously trusted the `revision` value inside the body JSON, so an
+  attacker with write access to the underlying bucket could roll back
+  state past CAS checks.  Two-track integrity fix: encrypted bodies use
+  AES-GCM with `revision` as AAD (already wired); unencrypted bodies
+  gain an opt-in HMAC-SHA256 over `{ revision, etag }` with per-pid
+  HKDF-derived subkeys.  Set `integrity: { mode: 'hmac-sha256', integrityKey }`
+  on the store + `requireIntegrity: true` to refuse legacy un-tagged
+  bodies on the read path.
+- **ClusterClient ask-ID predictability** (#120) — `nextAskId()` used
+  `Date.now() + counter`, predictable enough that a MitM on the
+  TCP socket could pre-compute likely IDs and inject forged
+  `cluster-client-reply` frames.  Switched to `crypto.randomUUID()`
+  (122 bits of entropy per call).
+- **Master-key rotation sweep race** (#109) — `reEncryptObjectStorage()`
+  had no durable progress token, so a crash forced the resumed run
+  to re-list and re-GET every object from scratch (a 24-hour sweep =
+  a 24-hour wasted re-walk).  Worse: if the operator dropped a
+  retired key from the keyring too soon, the sweep would only
+  notice mid-corpus, leaving the bucket half-rewritten.  Added two
+  opt-in options: `progress: ReEncryptProgressStore` for durable
+  resume tokens (file/Redis/object-storage-backed) and
+  `verifyKeyringCompleteness: boolean` (default `true`) for a
+  pre-sweep sample that refuses to start when a body's key version
+  is absent from `active`/`retired`.
+- **LeaseMajority split-brain** (#142) — a slow `lease.acquire()`
+  that the local defence-in-depth timeout had given up on could
+  later resolve `true` and write `decision=surviveSet`, letting
+  both sides of an equal partition claim victory.  Three layered
+  fixes: (1) monotonic `acquireEpoch` so a late result with a
+  stale epoch is dropped; (2) fire-and-forget `lease.release()`
+  on abandon to undo any wire-side success after the local
+  give-up, with fail-safe-on-rejection (refuse to claim majority
+  on the same view); (3) optional fencing tokens — `Lease.acquireWithToken?():
+  Promise<{ token: string } | null>` with `KubernetesLease`
+  returning `<resourceVersion>/<leaseTransitions>` and
+  `InMemoryLease` a monotonic per-name version stamp.
+
+### Added — Persistence, HTTP & observability
+
+- **PostgreSQL persistence backend** (#323) — `PostgresJournal`,
+  `PostgresSnapshotStore`, and `PostgresDurableStateStore` (the first
+  SQL-backed durable-state store) on top of the `pg` driver, registered
+  via `registerPostgresPlugins(ext, …)` which selects the journal +
+  snapshot store by config plugin ID and returns the durable-state-store
+  handle (the object-storage-plugin pattern — `PersistenceExtension` has
+  no durable-state registry).  Optimistic concurrency (per-pid
+  `SELECT MAX(seq)` inside a transaction plus a primary-key
+  unique-violation `23505` backstop; revision CAS via
+  `ON CONFLICT`/`UPDATE … WHERE revision`), an indexed tags join table,
+  and auto-created schema (`autoCreateTables`, default on).  `pg` is an
+  optional peer-dependency, lazy-imported; the backend defines its own
+  minimal client shapes so the framework stays dependency-free.  Ships
+  with an in-process fake-pool unit suite and a live `postgres:latest`
+  Docker suite wired into the integration-brokers CI matrix.
+- **MariaDB persistence backend** (#324) — sibling of #323 for
+  MariaDB / MySQL via the official `mariadb` connector: `MariaDbJournal`,
+  `MariaDbSnapshotStore`, `MariaDbDurableStateStore`, and
+  `registerMariaDbPlugins`.  A separate implementation with the MariaDB
+  dialect (`?` placeholders, `INSERT IGNORE` for the tag dedup,
+  `ON DUPLICATE KEY UPDATE` snapshot upsert, a derived-table-wrapped
+  `keepN` prune, `ER_DUP_ENTRY`/1062 concurrency backstop, and
+  `LONGTEXT`/`VARCHAR(255)`/`BIGINT` columns).  Optional `mariadb`
+  peer-dep; in-process fake-pool suite + live `mariadb:latest` Docker
+  suite in CI.
+- **Configurable compression level** (#322) — `CompressionConfig` gains
+  an optional `level` (gzip 0–9, zstd 1–22) threaded through the codec to
+  the object-storage snapshot + durable-state stores.  Out-of-range values
+  are clamped; the level is encoder-only and is NOT written to the wire
+  (the ATS1 manifest records only the algorithm), so changing it needs no
+  migration — old bodies keep decoding, new bodies use the new level, and
+  the two mix freely in one bucket.
+- **Real-network multi-node integration tests** (#313) — new
+  `tests/integration/` subtree with a Docker-compose setup that
+  brings up 5 cluster-node containers + 1 controller container
+  on a shared bridge network and runs partition / heal /
+  membership-convergence scenarios over a real TCP stack.  All
+  fault injection happens inside each container's network
+  namespace via `iptables` + `tc netem`, so no host privileged
+  mode is required — just `NET_ADMIN` on each cluster-node
+  container.  Two npm scripts ship: `bun run test:integration`
+  (build + up + auto-exit on the controller's status) and
+  `bun run test:integration:teardown`.  The same command works
+  locally on Docker Desktop and in
+  `.github/workflows/integration.yml`; the workflow is
+  triggered by pushes to `main`, manual dispatch, and a nightly
+  schedule.  Fifteen scenarios covering the cluster's load-bearing
+  primitives:
+  - **01** — membership convergence (smoke test)
+  - **02** — 2:3 split-brain with partition + heal verification
+  - **03** — Receptionist gossip-convergence over a shared
+    `ServiceKey` across all 5 nodes, with partition + heal
+  - **04** — DistributedData `LWWRegister` quorum reads/writes
+    during a 50ms `tc-netem` egress latency storm — proves
+    `majority`-consistency operations survive a real network
+    slowdown
+  - **05** — Cluster Singleton failover after the host node
+    `cluster.leave()`s; new leader's manager spawns the
+    singleton, proxies from every remaining node converge
+  - **06** — Cluster Sharding rebalance: 30 entities warmed up,
+    victim node leaves, 8 ex-victim entities relocate to
+    surviving regions via the coordinator's HandOff path
+  - **07** — Concurrent `GCounter` increments from all 5 nodes
+    converge to the exact expected total (proves CRDT merge +
+    `ddata-gossip` wire path under write pressure)
+  - **08** — Receptionist `Subscribe` continuous-listing
+    notifications fire on register / deregister, observable
+    from every cluster node within gossip-propagation latency
+  - **09** — External `ClusterClient` (NOT a cluster member)
+    makes 100 sequential asks against `/user/echo`; exercises
+    the #120 `randomUUID` ask-id path end-to-end
+  - **10** — Management HTTP auth end-to-end: 401 without
+    token, 200 with valid token, 404 with valid token + fake
+    address, /health stays anonymous (probe contract)
+  - **11** — `PersistentActor` event-sourcing + snapshot +
+    replay: 5 increments → snapshot at seq=3 → kill → respawn
+    triggers `recover()` → snapshot-load + replay restores
+    state.  Two-kill cycle verifies determinism.
+  - **12** — `DistributedPubSub` topic fan-out: 15 events
+    published from two different nodes, all 5 subscribers
+    receive both bursts in order
+  - **13** — `CoordinatedShutdown` pipeline progresses through
+    early (`BeforeServiceUnbind`) + late
+    (`BeforeActorSystemTerminate`) phases on a victim node;
+    markers POST'd to a peer observer verify both fired in
+    chronological order
+  - **14** — Bounded mailbox + `actor_mailbox_dropped_total`
+    metric: bombard a slow actor with 15 000 messages, verify
+    ~5 000 drops are counted in the Prometheus output with
+    correct `{class, path, reason}` labels
+  - **15** — `DnsSeedProvider` against docker's embedded DNS:
+    resolves every peer hostname, validates IPv4 shape and
+    `<systemName>@<host>:<port>` stamping
+- **Backend `remoteAddress` wiring** (#312 follow-up) — the
+  Fastify, Express, and Hono backends now populate
+  `HttpRequest.remoteAddress` from the socket peer
+  (`req.ip` / `req.socket.remoteAddress` on Fastify+Express;
+  best-effort across `c.req.raw` / `c.env.requestIP` on Hono).
+  `IpAllowlist` works end-to-end on real socket peers — the
+  pre-existing `getClientIp` override is no longer required for
+  default deployments behind direct connections.
+- **HTTP route middleware framework** (#312) — new
+  `withMiddleware(mw, route)` builder + `Middleware` type
+  `(req, next) => Promise<HttpResponse> | HttpResponse`.  Middlewares
+  compose outside-in; nested wraps run in declaration order.  The
+  HTTP cache primitives (`rateLimit`, `idempotent`, `cached`) are
+  unchanged, but new orthogonal concerns (auth, allowlists, custom
+  logging, request tracing) can hang off the same hook.
+- **`BearerTokenAuth({ tokens })`** (#312) — built-in middleware
+  that 401s every request lacking a `Authorization: Bearer <token>`
+  header from the configured (rotatable) shared-secret list.
+  Constant-time comparison so an attacker probing tokens can't
+  distinguish "first character wrong" from "last character wrong"
+  by timing.  Rejection includes `WWW-Authenticate: Bearer realm=...`.
+- **`IpAllowlist({ allow })`** (#312) — built-in middleware for
+  CIDR-based network-level isolation.  Parses IPv4 + IPv6 CIDRs
+  (including IPv4-mapped IPv6 like `::ffff:10.0.0.1` so a dual-
+  stack socket peer matches an IPv4 CIDR).  Fail-secure: no
+  resolvable client IP means 403.  Trust-source is explicit:
+  default reads `req.remoteAddress` (the socket peer); operators
+  behind a trusted proxy must opt-in to header trust via the
+  `getClientIp` extractor.
+- **`HttpRequest.remoteAddress?: string`** (#312) — optional new
+  field on the request shape.  Backends should populate from the
+  underlying socket where available.  Consumers that need to
+  trust `x-forwarded-for` must do so explicitly (see
+  `IpAllowlist`'s `getClientIp`).
+- **`managementRoutes`** gains `auth`, `ipAllowlist`, and
+  `authProtectHealth` settings (#312).  By default the auth
+  middleware wraps the privileged subtree (`/cluster/*`,
+  `/metrics`) but leaves `/health` and `/ready` anonymous —
+  standard Kubernetes liveness/readiness probes can't easily
+  attach an Authorization header.  Set `authProtectHealth: true`
+  when the deployment can present credentials on probes.  The
+  IP-allowlist wraps EVERYTHING (network-level isolation
+  precedes any application policy).
+- **`JsonLogger`** (#311) — structured-logging logger that emits one
+  `\n`-delimited JSON object per record to `process.stdout` (or an
+  injected `JsonLogSink`).  Every record carries `ts` (ISO-8601),
+  `level`, optional `source`, `msg`, the merged static + dynamic
+  MDC, and positional `...args` under an `args` array.  Errors
+  serialise as `{ name, message, stack }`; circular refs,
+  `BigInt`, and functions are sanitised so a log call never throws.
+  Drop-in for log-aggregation pipelines (Loki, ELK, Datadog,
+  CloudWatch, etc.) via the standard stdout-pipe path.
+- **`otelLogger({ api })`** (#311) — bridge to
+  `@opentelemetry/api-logs` for OTLP-Logs pipelines.  Optional peer
+  dep (structural-typed on the OTel surface, like `otelTracer`).
+  Maps severity to OTel's standard severity-number range, attaches
+  the actor's path on `source`, merges static + dynamic MDC into
+  `attributes`, and the SDK auto-links the active span's
+  `traceId`/`spanId` when tracing is enabled in the same process.
+
+### Changed — Bounded mailbox default
+
+- **Bounded mailbox is now the default** (#310) — every actor spawned
+  without an explicit `Props.withMailbox(...)` gets a
+  `BoundedMailbox` with `capacity = 10_000` and `overflow = 'drop-head'`.
+  The pre-#310 unbounded shape was a classic Akka-anti-pattern in
+  disguise: a runaway producer could absorb the JVM, ahem, the V8
+  heap, until OOM.  10 000 is high enough that a well-tuned actor
+  never hits it on a normal traffic spike; if it does, the actor's
+  throughput is mismatched and the bound makes that operationally
+  visible.  Drops are emitted as the `actor_mailbox_dropped_total`
+  Counter (labels `class`, `path`, `reason`).  Opt back into unbounded
+  per-actor via `Props.withMailbox(() => new Mailbox())`; keep the
+  bounded shape but change the capacity via `Props.withMailboxCapacity(n)`.
+
+### Fixed — Compression
+
+- **zstd compression on runtimes without native zstd** (#321) — the
+  compression codec wired the `fzstd` peer-dependency as a compressor, but
+  `fzstd` is decompression-only (it has no `compress`), so
+  `compression: { algorithm: 'zstd' }` threw `fzstd.compress is not a
+  function` on any runtime without native zstd (i.e. not Bun and not
+  Node ≥22.15) — and the eager peer-dep probe passed anyway.  zstd
+  resolution is now split by direction: compress is native-only with a
+  clear "needs Bun / Node ≥22.15" error, decompress keeps the `fzstd`
+  fallback so a non-native runtime can still READ zstd bodies written
+  elsewhere, and `probeCompressionAvailability('zstd')` now checks the
+  compress path so the misconfig surfaces at plugin-init, not on first
+  persist.
+- **Object-storage compression docs were inaccurate** — the docs
+  described `gzip` / `brotli` / `deflate` with a `level` field and
+  `Content-Encoding`-header-driven decode, none of which matched the
+  implementation.  Corrected across EN + DE to the real `none` / `gzip` /
+  `zstd` set, the ATS1-manifest-driven decode, the now-real `level`
+  option, and the per-direction zstd runtime support.
+
 ## [0.9.1] — 2026-05-15
 
 Docs-only patch release covering the first round of post-v0.9.0