cent 4.0.0 → 5.0.0

data/README.md

@@ -1,260 +1,807 @@
 # cent
 
-![Build Status](https://github.com/centrifugal/rubycent/actions/workflows/main.yml/badge.svg)
+Ruby SDK for the [HTTP server API](https://centrifugal.dev/docs/server/server_api)
+of [Centrifugo](https://github.com/centrifugal/centrifugo): every API method,
+including those of Centrifugo PRO, with typed results; batches, automatic
+batching and retries; connection, subscription and server API tokens; the
+proxy handlers and the endpoints client SDKs call, as Rack apps.
 
-Ruby client for the [Centrifugo](https://centrifugal.dev) server HTTP API.
+```ruby
+gem "cent", "~> 5.0"   # json is the only dependency
+```
 
-- `Cent::Client` — call server API methods (publish, broadcast, presence, history, …).
-- `Cent::Notary` — issue connection and subscription JWTs.
+| | Version |
+|---|---|
+| Ruby | 3.4 or newer |
+| Centrifugo | v6, any release |
+| Map and shared poll methods | Centrifugo v6.8.0 or newer |
+| `ApiError#temporary?`, `TooManyRequestsError`, refused requests reported as such rather than as internal errors | Centrifugo v6.10.0 or newer |
+| Ed25519 tokens verified without JWKS, by `client.token.ed25519_public_key` | Centrifugo v6.10.2 or newer |
+| Methods marked "Centrifugo PRO only" | Centrifugo PRO v6 |
 
-Works with Centrifugo **v4 and newer** (tested against v6.7.1). Ruby 3.0+.
+cent 5 replaces cent 4: see [MIGRATING.md](MIGRATING.md) - an application
+which needs time to move can stay on `gem "cent", "~> 4.0"`. cent implements the
+[Centrifugo server SDK specification](https://github.com/centrifugal/gocent/blob/master/spec.md),
+version 0.4.
 
-## Installation
+## Example
 
 ```ruby
-gem 'cent', '~> 4.0'
-```
+require "cent"
+
+client = Cent::Client.new(api_endpoint: "http://localhost:8000/api", api_key: ENV.fetch("CENTRIFUGO_API_KEY"))
 
-```sh
-$ bundle install
+res = client.publish(channel: "news", data: { text: "hello" })
+puts "published at offset #{res.offset}"
 ```
 
-## API client
+There is a method for every API method, generated from Centrifugo's
+`api.proto` with its documentation. It takes the fields of the method's
+request as keyword arguments and returns its result - a `Data` object - raising
+unless Centrifugo did everything asked. A nested message is its class or a
+Hash of its fields: `since: { offset: 5, epoch: "xyz" }`. A batch and commands
+have the same methods: `batch.publish(...)`, `Cent::Command.publish(...)`.
+
+A field `api.proto` marks required is a required keyword - of the method, or of
+a nested message's class - and must have a value: given `nil`, `""`, `0` or a
+message with nothing set, the call raises `Cent::InvalidRequestError` naming
+it, as in `limit_strategy.rate_limit.policies[0].rate is required`, before
+anything is sent. A field at its zero value is not sent, nor a message with
+nothing set - but for the updates of `device_update`, sent whenever given:
+`timezone_update: { timezone: "" }` clears the timezone, and leaving
+`timezone_update` out changes nothing.
+
+A field with a fixed set of values has a constant per value, named as the
+specification's table pins it for every SDK, then the value:
+`provider: Cent::PROVIDER_FCM`, `op: Cent::OP_ADD`, `Cent::FILTER_NODE_OP_AND`
+for `FilterNode`'s `op`, and in a proxy handler
+`req.encoding == Cent::Proxy::ENCODING_BINARY` or
+`event.type == Cent::Proxy::CHANNEL_EVENT_TYPE_OCCUPIED`. The fields stay strings: a
+value Centrifugo adds can be sent before cent names it.
+
+Create the client once and share it - a constant, an initializer: it is safe
+for concurrent use from threads, and keeps its connections to Centrifugo alive
+between calls. Under a fiber scheduler - Async, Falcon - calls wait without
+blocking other fibers.
+
+A result converts to a Hash and to JSON whole, nested results included:
+`res.to_h`, `res.to_json`, `render json: res`.
+
+## Rails
+
+The client, configured once in an initializer and closed when the process
+exits, so that calls in progress finish:
 
 ```ruby
-client = Cent::Client.new(api_key: 'your-api-key')
-# Or pointing at a remote Centrifugo:
-client = Cent::Client.new(
-  api_key: 'your-api-key',
-  endpoint: 'https://centrifugo.example.com/api',
-  timeout: 5
+# config/initializers/centrifugo.rb
+CENTRIFUGO = Cent::Client.new(
+  api_endpoint: ENV.fetch("CENTRIFUGO_API_ENDPOINT", "http://localhost:8000/api"),
+  api_key: Rails.application.credentials.dig(:centrifugo, :api_key),
+  request_timeout: 5
 )
+at_exit { CENTRIFUGO.close(timeout: 5) }
 ```
 
-Every method returns the parsed response body from Centrifugo:
+`api_key: nil` means no key, so the initializer loads where credentials are
+absent - `rails assets:precompile` in a Docker build, say. Under Puma's
+`preload_app!` or a Sidekiq swarm the client is shared across the fork: a
+forked process starts its own connections and batch threads on its first call.
+In tests, `at_exit` handlers run in reverse order, so one registered after
+`minitest/autorun` closes the client before the tests run: close it in
+production only, or from Puma's `on_worker_shutdown`.
 
-- On success the body has a `"result"` key: `{ "result" => { ... } }`.
-- On an API-level failure (e.g. unknown channel, namespace not found) `Cent::ResponseError` is raised with Centrifugo's numeric `code` and `message`.
-- On a transport problem (network failure, timeout, non-2xx HTTP, malformed JSON) a `Cent::Error` subclass is raised.
+Publishing from a model or a job, and rendering a result:
 
-`batch` and `broadcast` are different — see their sections below.
+```ruby
+class Message < ApplicationRecord
+  after_create_commit do
+    CENTRIFUGO.publish(channel: "chat:#{room_id}", data: { id:, text:, user: user.name },
+                       idempotency_key: "message-#{id}")
+  end
+end
 
-### Customizing the connection
+class HistoryController < ApplicationController
+  def show
+    render json: CENTRIFUGO.history(channel: "chat:#{params[:room_id]}", limit: 50, reverse: true)
+  end
+end
+```
 
-The initializer yields the underlying [`Faraday::Connection`](https://lostisland.github.io/faraday/) so you can adjust headers, timeouts, adapter, etc.
+Token endpoints, as plain actions behind your authentication - the token
+functions are all you need from cent here:
 
 ```ruby
-Cent::Client.new(api_key: 'k') do |conn|
-  conn.headers['User-Agent'] = 'my-app/1.0'
-  conn.options.open_timeout  = 3
-  conn.options.timeout       = 7
-  conn.adapter :typhoeus
+# config/initializers/centrifugo.rb
+CENTRIFUGO_KEY = Cent::Token::Key.hmac(Rails.application.credentials.dig(:centrifugo, :token_secret))
+
+# app/controllers/centrifugo_controller.rb
+class CentrifugoController < ApplicationController
+  # 403, not the 401 authenticate_user! answers: a client SDK stops asking only
+  # on 403, and retries anything else.
+  before_action { head :forbidden unless current_user }
+
+  def connection_token
+    token = Cent::Token.connection(CENTRIFUGO_KEY, user: current_user.id.to_s, expires_at: 15.minutes.from_now)
+    render_token(token)
+  end
+
+  def subscription_token
+    channel = params.require(:channel)
+    # 403 tells the client SDK to stop asking, and to unsubscribe.
+    return head :forbidden unless current_user.can_read?(channel)
+
+    render_token(Cent::Token.subscription(CENTRIFUGO_KEY, user: current_user.id.to_s, channel:,
+                                                          expires_at: 15.minutes.from_now))
+  end
+
+  private
+
+  def render_token(token)
+    response.headers["Cache-Control"] = "no-store"   # a token is never cached
+    render json: { token: }
+  end
 end
+
+# config/routes.rb
+post "centrifugo/connection_token", to: "centrifugo#connection_token"
+post "centrifugo/subscription_token", to: "centrifugo#subscription_token"
 ```
 
-### Publishing
+A client SDK treats a 403 from a token endpoint as final - it disconnects, or
+unsubscribes - and retries anything else, a 401 or 500 included: a signed-out
+user's browser answered 401 would ask again and again.
+
+A proxy handler, which Centrifugo calls on an internal network, is mounted as
+it is. Its block hands the request to a module of your app: the module is
+reloaded in development, where the initializer is not, and its methods can
+`return` early, which a block cannot:
 
 ```ruby
-client.publish(channel: 'chat', data: { text: 'hello' })
-# => {"result" => {"offset" => 1, "epoch" => "xyz"}}
-
-client.publish(
-  channel:         'chat',
-  data:            { text: 'hello' },
-  skip_history:    false,
-  tags:            { 'author' => '42' },
-  idempotency_key: 'my-idempotency-key',
-  delta:           true
-)
-```
+# config/initializers/centrifugo.rb
+REPORT = ->(_http, err) { Rails.error.report(err) }
+
+SUBSCRIBE_HANDLER = Cent::Proxy::SubscribeHandler.new(on_error: REPORT) do |req, http|
+  CentrifugoProxy.subscribe(req, http)
+end
 
-See [publish](https://centrifugal.dev/docs/server/server_api#publish).
+# app/services/centrifugo_proxy.rb
+module CentrifugoProxy
+  def self.subscribe(req, _http)
+    return {} if req.channel.start_with?("public:")
+    raise Cent::Proxy::ProxyError.new(403, "permission denied") unless Room.readable_by?(req.user, req.channel)
 
-### Broadcast
+    { info: { "role" => Room.role(req.user, req.channel) } }
+  end
+end
 
-```ruby
-response = client.broadcast(channels: %w[chat:1 chat:2], data: { text: 'hi' })
-# response => { "result" => { "responses" => [ {"result" => {...}}, {"result" => {...}} ] } }
+# config/routes.rb
+mount SUBSCRIBE_HANDLER, at: "/centrifugo/subscribe"
 ```
 
-The outer call only raises `Cent::ResponseError` if the whole broadcast is rejected (e.g. malformed request). Per-channel failures are delivered as individual entries in `response["result"]["responses"]`, each of which may contain an `"error"` key — **these are not raised**. Walk the array to check them:
+The module's methods are plain to test:
+`CentrifugoProxy.subscribe(Cent::Proxy::SubscribeRequest.new(user: "42", channel: "room:1"), nil)`.
+
+## Pages and positions
+
+The methods which return a page at a time - `history`, `device_list`,
+`device_topic_list`, `user_topic_list` and `map_read_state` - each have an
+`_all` method taking the same arguments, which reads page after page: with a
+block it yields every item, without one it returns an `Enumerator`. A page is
+read when the iteration reaches it, so `first(10)` or a `break` reads no more.
+An iteration lasts as long as your loop, so it takes no `timeout:`;
+`page_timeout:` bounds each page's request instead, replacing
+`request_timeout` for it:
 
 ```ruby
-response['result']['responses'].each_with_index do |r, i|
-  warn "channel #{i} failed: #{r['error']['message']}" if r['error']
+client.history_all(channel: "news", limit: 100) do |item|
+  archive(item.data)
+  last = item.position   # resume with since: last, or give a client recover_since
 end
+client.device_list_all(filter: { users: ["42"] }, page_timeout: 30).each_slice(50) { |devices| notify(devices) }
 ```
 
-### Subscribe / Unsubscribe
+`history_all` needs a `limit` of 1 or more, the page size, and reads forward
+only. It yields each publication as a `Cent::HistoryItem`, whose fields read
+as the publication's and whose `position` is where it is in the stream;
+`map_read_state_all` yields each entry as a `Cent::MapStateItem`, its
+`position` the state's, which `map_read_stream` follows. `map_read_state_all`
+reads every page after the first at the first one's revision, so that a map reset between pages raises
+`Cent::UnrecoverablePositionError` rather than mixing two states: start again
+then. Items changed while iterating - a device registered meanwhile - may or
+may not be seen.
+
+The results of `publish`, `history`, `map_publish`, `map_remove`,
+`map_read_state` and `map_read_stream` give their offset and epoch as a `Cent::StreamPosition`: what came after a
+publication is
 
 ```ruby
-client.subscribe(user: '42', channel: 'chat')
-client.unsubscribe(user: '42', channel: 'chat')
+pos = client.publish(channel: "news", data: { text: "hi" }).position
+client.history(channel: "news", since: pos, limit: 100)
 ```
 
-### Disconnect / Refresh
+A position whose epoch is no longer the stream's raises
+`Cent::UnrecoverablePositionError`: the stream was reset, and reading it from
+its start is the only recovery.
+
+## Payloads
+
+`data`, `info`, `meta` and the other payloads are JSON values, given as the
+Ruby values `JSON.generate` writes - `Hash`, `Array`, `String`, `Integer`,
+`Float`, `true`, `false`, `nil`, Symbols as strings - and returned as
+`JSON.parse` gives them, Hashes with String keys:
 
 ```ruby
-client.disconnect(user: '42')
-client.disconnect(user: '42', whitelist: %w[keep-this-client-id])
+client.publish(channel: "news", data: { text: "hi", tags: %w[a b] })
 
-client.refresh(user: '42', expired: true)
+pub = client.history(channel: "news", limit: 1).publications.first
+pub.data["text"]
+client.publish(channel: "archive", data: pub.data)   # exactly the same JSON value
 ```
 
-### Presence / Presence stats
+A payload from a reply has String keys, as JSON has: `pub.data["text"]`, not
+`pub.data[:text]`, so that it goes back out as the same JSON value. To read it
+with Symbols, convert a copy - `pub.data.transform_keys(&:to_sym)`, or
+`pub.data.with_indifferent_access` in Rails - and send the original.
+
+A String is a JSON string. JSON you already have, or encode with rules of your
+own - dates, decimals, a faster encoder - goes in a `JSON::Fragment`, sent as it
+is, also inside a Hash:
 
 ```ruby
-client.presence(channel: 'chat')
-client.presence_stats(channel: 'chat')
+client.publish(channel: "news", data: JSON::Fragment.new(row["payload_json"]))
+client.publish(channel: "news", data: { meta:, doc: JSON::Fragment.new(json) })
 ```
 
-### History
+A payload which is not JSON - NaN, a `Time`, a binary (ASCII-8BIT) string,
+invalid UTF-8, a Hash with Integer keys - raises `Cent::InvalidRequestError`
+naming where it is, before anything is sent. Binary data goes base64-encoded
+in the `b64` fields, such as `b64data`.
+
+Write integers which may pass 2^53 - a 64-bit ID, say - as JSON strings:
+browsers and JavaScript backends decode JSON numbers as doubles, and round
+larger integers silently.
+
+## Timeouts
+
+`request_timeout` - ten seconds by default - bounds every call. A call's
+`timeout:`, in seconds, replaces it for that call, longer or shorter; nil
+leaves `request_timeout`. Neither can be turned off: zero is refused, as a
+negative is - set a large value for long calls. Ruby has
+no deadline a call runs under which cent could read: a `Timeout.timeout` or an
+Async task's timeout around a call can only cut it short, with its own error.
+An automatic batch request lasts as long as the longest of its calls' timeouts,
+and so may outlast a shorter one, whose call still returns at its timeout:
 
 ```ruby
-client.history(channel: 'chat', limit: 10)
-client.history(channel: 'chat', limit: 10, reverse: true)
-client.history(channel: 'chat', limit: 10, since: { 'offset' => 5, 'epoch' => 'xyz' })
-client.history_remove(channel: 'chat')
+client.info(timeout: 0.5)
 ```
 
-### Channels
+The timeout bounds the whole call: connecting and TLS, sending, and the
+reply, however slowly a server - a proxy in trouble, say - sends it. However a
+call times out, it raises `Cent::TimeoutError`. `broadcast_with_retry` makes
+several requests in one call: its `timeout:` bounds them all, waits between
+them included, and `attempt_timeout:` each one.
+
+## Errors
+
+| Error | Meaning |
+|---|---|
+| `Cent::ApiError` | Centrifugo refused the request or failed to carry it out: `code`, `server_message`, `temporary?`. Known codes are subclasses: `UnknownChannelError`, `NotFoundError`, `BadRequestError`, `NotAvailableError`, `TooManyRequestsError`, `InternalError`... |
+| `Cent::BroadcastFailedChannelsError`, `Cent::BatchFailedCommandsError` | some parts of a broadcast or batch failed - see below |
+| `Cent::HTTPError` | the request did not reach the API method: `status_code`, `body`, `headers`; `UnauthorizedError` for rejected credentials |
+| `Cent::DecodeError` | the reply was not Centrifugo's - usually `api_endpoint` points elsewhere |
+| `Cent::InvalidRequestError` | the client could tell the request is wrong and did not send it |
+| `Cent::InvalidConfigError` | the client is refused as configured: not created, or a call given an endpoint by `api_endpoint_func` or a token by `bearer_token_func` which cannot be sent. The message names the option and the fix |
+| `Cent::FunctionError` | `api_endpoint_func` or `bearer_token_func` raised: `function` is the option's name, which the message names too, `cause` is its exception, and `Cent.retryable?` judges it by that |
+| `Cent::NetworkError`, `Cent::TimeoutError`, `Cent::ClosedError` | as their names say |
+
+All are `Cent::Error`. Centrifugo adds error codes over time: an unknown code
+is still an `ApiError`, so rescue the codes you handle and treat the rest by
+class.
+
+## Retries
+
+`Cent.retryable?(err)` reports whether a new attempt, with time left, may
+succeed: the error came from a temporary condition - a broker being
+unavailable, a rate limit, the network - not from the request.
+
+A temporary error does not mean nothing was done: after a timeout, or even an
+internal error, the publication may have happened. So retry a publication only
+with an `idempotency_key`, and back off between a bounded number of attempts:
 
 ```ruby
-client.channels
-client.channels(pattern: 'chat:*')
+backoff = Cent.exponential_backoff(0.2, 5.0)
+attempt = 0
+begin
+  attempt += 1
+  # The same key on every attempt: a publication which happened in an
+  # earlier attempt, even one reported as failed, is not published again.
+  client.publish(channel: "orders:42", data:, idempotency_key: "order-42-paid")
+rescue Cent::Error => e
+  raise unless Cent.retryable?(e) && attempt < 3
+
+  sleep(backoff.(attempt))
+  retry
+end
 ```
 
-### Info
+Choosing keys:
+
+- One key per publication or broadcast, reused only to retry that same one. A
+  broadcast needs one key, not one per channel.
+- Centrifugo remembers a key per channel, for five minutes by default. A retry
+  within that time is answered with the first result and not published again -
+  and so is a *different* publication into the same channel under the same
+  key. A key built from an ID must name the event too: `order-42-paid`, not
+  `order-42`.
+- The memory, Redis and PostgreSQL brokers keep keys. The NATS broker ignores
+  them, so with it a retry publishes again.
+
+Retry a broadcast with `broadcast_with_retry` rather than a loop: see below.
+
+## Broadcast and batch
+
+`broadcast` publishes the same data into many channels in one request; a batch
+sends many commands at once. Both raise unless every part succeeded:
 
 ```ruby
-client.info
+client.broadcast(channels: ["user:1", "user:2"], data: { text: "hello" }, idempotency_key: message_id)
+
+batch = client.new_batch
+pub = batch.publish(channel: "news", data:)
+stats = batch.presence_stats(channel: "news")
+batch.send
+pub.value.offset        # each command has its own result, or raises its own error
+stats.value.num_clients
 ```
 
-### Batch
+Parts succeed or fail on their own. When Centrifugo carried the request out
+and some parts failed, up to all, it raises `BroadcastFailedChannelsError` or
+`BatchFailedCommandsError`: its `failed` lists them, and its `result` holds
+every part's outcome. Any other error means the request failed as a whole - the
+network, the credentials, an invalid request - and no part got a reply; after a
+timeout or a network error Centrifugo may still have carried it out.
 
-Send many commands in one HTTP request — Centrifugo processes them sequentially (or in parallel with `parallel: true`) and returns one reply per command in the same order.
+A partial error is not an error of its parts: `rescue Cent::UnknownChannelError`
+does not catch a broadcast in which a channel was unknown, nor one into a
+single channel - use `publish` for one channel. To go on despite some failed
+parts:
 
 ```ruby
-response = client.batch(commands: [
-  { 'publish'        => { 'channel' => 'a', 'data' => { 'x' => 1 } } },
-  { 'publish'        => { 'channel' => 'b', 'data' => { 'x' => 2 } } },
-  { 'presence_stats' => { 'channel' => 'a' } }
-])
-# => { "replies" => [ {"publish" => {...}}, {"publish" => {...}}, {"presence_stats" => {...}} ] }
+begin
+  client.broadcast(channels:, data:)
+rescue Cent::BroadcastFailedChannelsError => e
+  logger.warn("broadcast failed in #{e.result.failed.map(&:channel)}")
+end
 ```
 
-Two things about batch are different from every other method:
+Retrying:
+
+- a **broadcast**: `broadcast_with_retry` does it. It sends again only the
+  channels which may still succeed, all attempts with one `idempotency_key` -
+  made for the call when none is given - and returns every channel's last
+  outcome, as `broadcast` does:
+
+  ```ruby
+  res = client.broadcast_with_retry(channels:, data:, max_attempts: 5, attempt_timeout: 2, timeout: 10)
+  ```
+
+  By default it retries once, after about 100 milliseconds. Its block, when
+  given, is called after each failed attempt which will be retried, with a
+  `Cent::BroadcastAttempt` - the channels to retry and their errors: it returns
+  the seconds to wait before the next attempt - or nil for the default
+  backoff - and may log or count meanwhile; it raises to stop. A `HTTPError`
+  in `attempt.error` carries the response's `headers`: wait at least its
+  `retry-after` to honour one.
+
+  ```ruby
+  backoff = Cent.exponential_backoff(0.2, 5.0)
+  client.broadcast_with_retry(channels:, data:) do |attempt|
+    logger.info("attempt #{attempt.attempt}: #{attempt.retry.size} channels to retry")
+    backoff.(attempt.attempt)
+  end
+  ```
+
+  `res.idempotency_key` is the key the attempts used: to retry the broadcast
+  later, after it gave up - from a job, say - send it with that key. Repeating
+  `broadcast` on any error would instead send it to every channel again, those
+  which already got it included.
+
+  It waits between attempts with `sleep`, in the thread calling it. In a web
+  request, give it a short `timeout:`, so that a Centrifugo which is down does
+  not hold the request for every attempt's `request_timeout`. Retry for longer
+  from a job - Sidekiq, Active Job - with an `idempotency_key` of its own, so
+  that the job retries under the same key whichever way the call failed:
+
+  ```ruby
+  class BroadcastJob
+    include Sidekiq::Job
+
+    def perform(channels, data, key)
+      CENTRIFUGO.broadcast_with_retry(channels:, data:, idempotency_key: key, max_attempts: 3)
+    rescue Cent::BroadcastFailedChannelsError => e
+      retried = e.retryable_channels
+      self.class.perform_in(30, retried, data, key) unless retried.empty?
+      # Failed for good - an unknown channel, say: no retry fixes it.
+      lost = e.failed.reject { retried.include?(it.channel) }
+      Sidekiq.logger.error("broadcast #{key} failed in #{lost.map(&:channel)}: #{e.message}") unless lost.empty?
+    end
+  end
+  ```
+- a **batch**: build a new one, with the same keys, from the commands
+  `e.failed` lists with a retryable error. A batch is sent once.
+
+`new_batch` runs the commands one by one, in order; `parallel: true` runs them
+concurrently. Centrifugo PRO sends a batch's publications to its broker
+together: each channel's publications still take effect in order, but
+publications into different channels may not, unless the namespace sets
+`publication_grouping_disabled`.
+
+## Automatic batching
+
+With `auto_batch`, calls made concurrently from one process go out together
+once `max_in_flight` requests are in flight, with no delay otherwise, since
+nothing waits for a timer. It is for processes which publish a lot at once - a
+Sidekiq process at concurrency 10 or more whose jobs publish, an app on Falcon -
+and does nothing for a Puma web process of a few threads: leave it off there.
 
-1. **No `result` wrapper.** The response is `{ "replies" => [...] }` at the top level. This matches Centrifugo's wire format.
-2. **Per-command errors are not raised.** Each entry in `replies` may instead be `{ "error" => { "code" => ..., "message" => ... } }`. Raising on the first would make partial-success responses impossible to inspect — so the caller is expected to walk the array:
+```ruby
+# A Sidekiq process at concurrency 16: about half the threads which publish.
+client = Cent::Client.new(api_endpoint:, api_key:, auto_batch: Cent::AutoBatch.new(max_in_flight: 8))
+```
 
-   ```ruby
-   response['replies'].each_with_index do |reply, i|
-     if reply['error']
-       warn "command #{i} failed: #{reply['error']['code']} #{reply['error']['message']}"
-     end
-   end
-   ```
+Set `max_in_flight` to about half the threads - or fibers - which call the
+client at once. It batches only calls made while that many requests are in
+flight: above the concurrency, it never does; just below, it batches calls one
+at a time, and only limits them. Under heavy publishing it sends a fraction of
+the requests, for up to twice the publications a second, with Centrifugo, Redis
+and your process doing about half the work per publication.
 
-   `Cent::ResponseError` is still raised if Centrifugo rejects the batch request as a whole (e.g. malformed top-level body).
+Every call still gets its own result or error, and keeps its own timeout. Calls
+sharing a batch share its request, though: if the request itself fails - the
+network, a body too large for a proxy in front of Centrifugo (nginx refuses
+bodies over 1 MB by default) - every call in it fails.
 
-### Error handling
+A batch belongs to no single caller: it carries no caller's `Fiber[]` values. `batch_context`
+gets the callers' fiber storages and a block sending the batch - to tag its
+logs, or start a tracing span linking each caller's:
 
 ```ruby
-begin
-  response = client.publish(channel: 'chat', data: 'hi')
-rescue Cent::ResponseError => e
-  # Centrifugo rejected the request (e.g. unknown channel, namespace not found).
-  puts "Centrifugo error #{e.code}: #{e.message}"
-rescue Cent::TimeoutError
-  # request timed out
-rescue Cent::NetworkError
-  # connection refused / DNS failure / etc.
-rescue Cent::UnauthorizedError => e
-  # HTTP 401 — API key is wrong
-rescue Cent::TransportError => e
-  # other 4xx/5xx — e.status has the HTTP code
-rescue Cent::DecodeError
-  # response body wasn't valid JSON
+batch_context = lambda do |callers, &send_batch|
+  Rails.logger.tagged("batch of #{callers.filter_map { it[:request_id] }.join(",")}") { send_batch.call }
 end
 ```
 
-All of the above inherit from `Cent::Error`, so you can rescue that single class if you don't need to discriminate.
+## Authentication
 
-## Token generation
+`api_key` is sent in the `X-API-Key` header. `bearer_token_func` instead sends a
+token as `Authorization: Bearer`, for Centrifugo PRO's JWKS authentication - it
+is called for every request, so return a cached token. Leave both unset when
+mutual TLS, set up in `http_options`, or a proxy authenticates instead.
+
+To sign the tokens yourself, `Cent::Token::ApiTokenSource` issues them and reuses
+each for half its lifetime:
 
 ```ruby
-notary = Cent::Notary.new(secret: 'hmac-secret')                       # HS256
-notary = Cent::Notary.new(secret: rsa_private_key, algorithm: 'RS256') # RSA
-notary = Cent::Notary.new(secret: ec_private_key,  algorithm: 'ES256') # ECDSA
+api_token_source = Cent::Token::ApiTokenSource.new(
+  key: api_key,   # its own key - never one of the client tokens
+  audience: "centrifugo-api", scope: ["centrifugo:api"]
+)
+client = Cent::Client.new(api_endpoint: "https://centrifugo.example.com/api",
+                          bearer_token_func: api_token_source.method(:token))
 ```
 
-### Connection token
+Centrifugo PRO accepts a token signed with any key of `http_api.jwks.endpoint`,
+and checks the audience, issuer and scope only when configured to. So keep API
+tokens apart from the connection and subscription tokens your users hold - or
+any user can call the server API with their connection token:
+
+- publish the API keys with `Cent::Endpoints::JWKS.new(*api_keys)` at an
+  endpoint of their own, not the one of `client.token.jwks_public_endpoint`;
+- set `http_api.jwks.audience` and `http_api.jwks.scope` to values only API
+  tokens carry - `Cent::Token.api` refuses claims with neither.
 
-Used by clients to establish a real-time connection. See [authentication](https://centrifugal.dev/docs/server/authentication).
+The API keys rotate as client keys do: see [Key rotation with JWKS](#key-rotation-with-jwks).
+
+Rejected credentials make every call raise `Cent::UnauthorizedError`.
+
+## HTTP, tracing and metrics
+
+cent sends requests with Net::HTTP, from Ruby itself, keeping connections
+alive between calls and never following redirects. TLS and other settings of the connections go in `http_options`:
 
 ```ruby
-notary.issue_connection_token(sub: '42')
-notary.issue_connection_token(sub: '42', exp: Time.now.to_i + 600, info: { name: 'Alex' })
-
-# With any of the standard/Centrifugo claims:
-notary.issue_connection_token(
-  sub: '42', exp: 1735689600, iat: 1735686000, jti: SecureRandom.uuid,
-  aud: 'centrifugo', iss: 'my-app',
-  info: { role: 'admin' }, meta: { tenant: 'acme' },
-  channels: %w[user:42 news],
-  subs: { 'room:1' => { 'data' => { 'welcome' => true } } }
-)
+client = Cent::Client.new(api_endpoint:, http_options: { ca_file: "/etc/ssl/centrifugo-ca.pem",
+                                                         cert: client_cert, key: client_key })
 ```
 
-### Subscription token
+`http_client` sends requests another way - another HTTP library, or a fake in
+tests: a callable taking the URL, the headers, the body and the seconds left,
+and returning `[status, headers, body]`. WebMock and VCR see the requests of
+the default transport too, as any Net::HTTP request.
 
-Used by clients to subscribe to a channel that requires token authorization. See [channel token auth](https://centrifugal.dev/docs/server/channel_token_auth).
+Tracing comes from the HTTP library: OpenTelemetry's Net::HTTP
+instrumentation, `opentelemetry-instrumentation-net_http`, traces every
+request. `on_request` is told of every request - method, commands and how many
+failed, duration, status, error - for metrics without a dependency of cent's:
 
 ```ruby
-notary.issue_channel_token(sub: '42', channel: 'private-chat', exp: 1735689600)
-notary.issue_channel_token(
-  sub: '42', channel: 'private-chat',
-  info: { role: 'writer' },
-  override: { 'presence' => { 'value' => true } }
-)
+on_request = ->(info) { StatsD.measure("centrifugo.#{info.method}", info.duration * 1000) }
 ```
 
-## Migrating from v3
+It must not raise: what it raises goes to the caller in place of the call's
+outcome.
+
+## Shutdown
 
-v4 changes some aspects of the library. We expect smooth migration for happy path though.
+`client.close` refuses new calls with `ClosedError` and waits for the ones in
+progress, automatic batches included, so that what was accepted is sent before
+the process exits - from `at_exit`, Puma's `on_worker_shutdown` or Sidekiq's
+`on(:shutdown)`. With `timeout:` it raises `Cent::TimeoutError` when calls are
+still in progress then, and they go on. It may be called again, concurrently
+too: each call waits the same way. A `broadcast_with_retry` stops at its next
+attempt, which raises `ClosedError`.
 
-- **Centrifugo v5+ is required.** v3 of this gem spoke the legacy `POST /api` JSON-RPC-style protocol; v4 uses the current per-method endpoints (`POST /api/publish`, `POST /api/broadcast`, …) and sends the API key as `X-API-Key` instead of `Authorization: apikey <key>`.
-- **Error handling is unchanged for the common case.** `Cent::ResponseError` still exists and is still raised when Centrifugo returns a top-level API error. Existing `rescue Cent::ResponseError => e` blocks using `e.code` / `e.message` keep working. The new additions are typed transport errors — `Cent::TimeoutError`, `Cent::NetworkError`, `Cent::TransportError`, `Cent::UnauthorizedError`, `Cent::DecodeError` — all subclassed under `Cent::Error`.
-- **Keyword arg rename**: `Cent::Notary#issue_channel_token(client:)` → `issue_channel_token(sub:)` to match Centrifugo's standard `sub` JWT claim.
-- **Ruby 3.0+** is required (was 2.5+).
-- **New methods** added for common Centrifugo operations: `subscribe`, `refresh`, `history_remove`, `batch`.
-- **Richer kwargs** on existing methods (e.g. `publish` now accepts `tags`, `skip_history`, `idempotency_key`, `delta`, `version`, `version_epoch`, `b64data`; `history` accepts `limit`, `since`, `reverse`; `channels` accepts `pattern`).
+## HTTP handlers
 
-## Development
+The proxy handlers and the endpoints client SDKs call are Rack apps, which
+Rails, Sinatra, Hanami and every Rack server mount:
 
-```sh
-$ bin/setup                # install dependencies
-$ bundle exec rspec        # run unit tests
-$ bundle exec rubocop      # lint
+```ruby
+# config/routes.rb
+mount CONNECT_HANDLER, at: "/centrifugo/connect"
+```
+
+Each takes a block and calls it with what the request asks for, and with a
+`Cent::HTTPRequest`: the method, headers, query and the server's own request
+in `raw` - the Rack env. Mounted in Rails, a handler runs behind the app's
+middleware, so the env carries the session: for the connect proxy, with
+`Cookie` in Centrifugo's `http_headers`, Devise's user is
+`http.raw["warden"].user`.
+
+Proxy handlers are mounted in every framework: Centrifugo calls them, and the
+user is in the request. The endpoints client SDKs call - tokens, shared poll
+signatures - suit an app without its own authentication layer: plain Rack,
+Sinatra, Roda, a token service. In Rails, write those as controller actions
+calling `Cent::Token`: see [Rails](#rails).
+
+Any other server calls the framework-free core, `handle(method, headers, body)`,
+which returns a `Cent::HTTPResponse`.
+
+Pass every handler an `on_error:`, told of the request and the error whenever
+the handler answers 500 or 400. Without it, an error in your block is answered
+with 500 and logged nowhere: the handler catches it to answer Centrifugo, so it
+never reaches your framework's error reporting.
+
+## Proxy
+
+`Cent::Proxy` is the backend side of Centrifugo's HTTP proxy, which calls your
+application when a client connects, subscribes, publishes or makes an RPC
+call: a handler per proxy type, taking a block.
+
+```ruby
+REPORT = ->(_http, err) { logger.error(err) }
+
+SUBSCRIBE_HANDLER = Cent::Proxy::SubscribeHandler.new(on_error: REPORT) { |req, http| Proxy.subscribe(req, http) }
+
+module Proxy
+  # req: a Cent::Proxy::SubscribeRequest; http: the Cent::HTTPRequest, its
+  # headers those Centrifugo passed on.
+  def self.subscribe(req, _http)
+    raise Cent::Proxy::ProxyError.new(403, "permission denied") unless can_read?(req.user, req.channel)
+
+    {}   # a Cent::Proxy::SubscribeResult, or a Hash of its fields
+  end
+end
+```
+
+Return a result to accept the request, raise `Cent::Proxy::ProxyError` to
+reject it with an error the client receives, or `Cent::Proxy::Disconnect` to
+close the client's connection; any other error fails the request, and goes to
+`on_error` to be logged - without `on_error`, nowhere; `on_error` itself must
+not raise. Error codes are in [400, 1999], disconnect codes in [4000, 4999],
+and a disconnect reason is at most 32 bytes. A block returning nil fails the
+request too: Centrifugo would take a response with nothing in it as accepting
+a subscribe, publish, RPC, map publish or map remove as sent, as refusing a
+connect unless anonymous connections are allowed, and as an expired refresh or
+sub refresh. A block calling a method,
+as here, keeps the logic where it can `return` early and be tested alone; see
+[Rails](#rails) for where each part goes there.
+
+Keep proxy endpoints reachable by Centrifugo only: whoever can call them can
+claim to be any user. The ways to do it:
+
+- Serve them on a network only Centrifugo reaches, not the one your public
+  routes are on.
+- Have Centrifugo add a secret header to every proxy request, with the proxy's
+  `static_headers` option, and check it in your application before the
+  handler - a Rack middleware or a route constraint - answering 403 without it.
+- Use TLS where the traffic leaves a trusted network.
+
+A handler's own share of the protection is its bound on a request's body,
+`max_body:` (16 MiB by default), and the deadline it passes on.
+
+## Tokens
+
+`Cent::Token` issues the JWTs Centrifugo authenticates clients with -
+connection and subscription tokens, with Centrifugo's claims, signed with
+HMAC, RSA (2048 bits or more), ECDSA (P-256, P-384, P-521) or Ed25519 from
+Ruby's OpenSSL - and `Cent::Endpoints` serves
+the endpoints the `getToken` callbacks of client SDKs call, in a Rack app or
+Sinatra (in Rails, a controller action: see [Rails](#rails)):
+
+```ruby
+KEY = Cent::Token::Key.hmac(ENV.fetch("CENTRIFUGO_TOKEN_SECRET"))   # client.token.hmac_secret_key
+
+CONNECTION_TOKEN = Cent::Endpoints::ConnectionToken.new(on_error: REPORT) do |req|
+  user = session_user(req.raw)
+  raise Cent::Endpoints::ForbiddenError unless user   # 403: the client stops asking
+
+  Cent::Token.connection(KEY, user: user.id.to_s, expires_at: Time.now + 3600)
+end
+```
+
+`Cent::Endpoints::ForbiddenError` answers 403, which client SDKs report as
+their `UnauthorizedError`, and stop asking; it is not the API client's
+`Cent::UnauthorizedError`, which means Centrifugo refused your API key.
+
+An expiry is required: a `Time`, or `:never` for a token accepted forever.
+`Cent::Endpoints::SubscriptionToken` serves subscription tokens the same way,
+its block given the channel the client asks for. To rotate keys without
+restarting Centrifugo, see [Key rotation with JWKS](#key-rotation-with-jwks).
+
+## Key rotation with JWKS
+
+By default Centrifugo verifies tokens with a single key from its
+configuration: `client.token.hmac_secret_key`, or an RSA, ECDSA or Ed25519
+public key. Changing that key means restarting Centrifugo, and from that moment
+it refuses every token signed with the old key. To rotate keys without that,
+your application can serve the keys itself, as a JSON Web Key Set (JWKS) at
+Centrifugo's `client.token.jwks_public_endpoint`. Each key in the set has an
+ID, and Centrifugo verifies a token with the key its `kid` header names:
+
+```ruby
+# An ID for each key, never reused for another key: Centrifugo caches keys by ID.
+key = Cent::Token::Key.from_pem(File.read("keys/2026-10.pem")).with_key_id("2026-10")
+
+JWKS = Cent::Endpoints::JWKS.new(key)   # mounted at the URL of jwks_public_endpoint
+token = Cent::Token.connection(key, user: "42", expires_at:)
 ```
 
-### Running integration tests
+```json
+{"client": {"token": {"jwks_public_endpoint": "https://app.example.com/centrifugo/jwks"}}}
+```
 
-Integration tests under `spec/integration/` exercise a real Centrifugo server. They're skipped unless `CENTRIFUGO_API_URL` is set.
+Only RSA, ECDSA and Ed25519 keys can go in a set: an HMAC secret has no public
+part to publish. Once `jwks_public_endpoint` is set, Centrifugo verifies tokens
+with the set only, and refuses tokens signed with the key it was configured
+with before. So moving to a JWKS is a one-time restart.
+
+How Centrifugo reads the set:
+
+- when a token names a key ID Centrifugo doesn't know, it fetches the set
+  again, at most once every 5 seconds;
+- it keeps the keys it fetched for 5 minutes (an hour before Centrifugo
+  v6.10.2), so a key dropped from the set stops working within that time;
+- when the endpoint can't be reached, it keeps using the keys it already has;
+- a token whose key isn't in the set is refused as expired, and the client SDK
+  asks for a new token.
+
+Rotate a key in three steps. Each step is one deploy of your application, and
+every instance should finish a step before any instance moves to the next:
+
+1. **Publish the new key alongside the current one,** and keep signing with
+   the current one: `Cent::Endpoints::JWKS.new(current, new)`.
+2. **Sign with the new key.** Centrifugo fetches the set the first time it sees
+   the new ID.
+3. **Drop the old key** once every token it signed has expired: wait longer
+   than the longest `expires_at` you give.
+
+If a key leaks, do all three steps now, without waiting in step 3. Within 5
+minutes Centrifugo refuses new connections with the leaked key's tokens.
+Clients that connected before stay connected until their tokens expire:
+disconnect them with the server API if needed.
+
+Centrifugo PRO's API tokens rotate the same way, from a set of their own at
+`http_api.jwks.endpoint` (see [Authentication](#authentication)).
+`Cent::Token::ApiTokenSource.new(current_key: -> { ... })` takes the key to sign
+with from a callable, so it can switch to the new key at step 2. Each API token
+is reused for half its `ttl`, so in step 3 wait at least `ttl` after the
+switch.
+
+## Shared poll signatures
+
+Clients tracking keys of a [shared poll](https://centrifugal.dev/docs/server/shared_poll)
+channel need a signature from the backend. `Cent::SharedPoll.sign` makes one,
+and `Cent::Endpoints::SharedPollSignature` serves the endpoint a subscription's
+`getSignature` callback calls - in Rails, an action answering
+`{signature:, keys:}` does the same:
 
-```sh
-$ docker compose up -d
-$ CENTRIFUGO_API_URL=http://localhost:8000/api CENTRIFUGO_API_KEY=api_key bundle exec rspec spec/integration
+```ruby
+SIGNATURE = Cent::Endpoints::SharedPollSignature.new(on_error: REPORT) do |channel, keys, req|
+  user = session_user(req.raw)
+  allowed = readable_keys(user, channel, keys)   # the rest are revoked
+  [allowed, Cent::SharedPoll.sign(SECRET, user: user.id.to_s, channel:, keys: allowed, expires_at: Time.now + 3600)]
+end
 ```
 
-### Testing across Faraday / JWT versions
+Shared poll subscriptions are experimental in Centrifugo, and this part
+follows their changes.
 
-```sh
-$ bundle exec appraisal install         # generate gemfiles/*.gemfile.lock
-$ bundle exec appraisal rspec           # run the full matrix locally
+## Commands for async consumers
+
+`Cent::Command.publish(...)` - and a method like it for every API method -
+checks a call as a client would and encodes it, for Centrifugo's
+[async consumers](https://centrifugal.dev/docs/server/consumers): its `method`
+and `payload` for an outbox table, or `to_json` for a Kafka message. `payload`
+is JSON text: the outbox's `payload` column is `jsonb`, and ActiveRecord's json
+types take the object, so parse it - given the text, they store a JSON string,
+which consumers drop:
+
+```ruby
+cmd = Cent::Command.publish(channel: "news", data: { text: "hi" })
+CentrifugoOutbox.create!(method: cmd.method, payload: JSON.parse(cmd.payload), partition: 0)
 ```
 
-## License
+`as_json` gives the command with its payload as a Hash, so a command inside
+another value - rendered by Rails, or in a job's arguments - is an object too.
+
+A consumer keeps commands in order only within a partition: to keep a
+channel's commands in order, derive their partition from the channel the same
+way in every writer.
+
+## Streaming AI output
+
+Publish a model's output into a channel of its own, in a namespace with history
+and recovery, from one thread - each publish waiting for the previous one - in
+chunks of what arrived every 50 ms or so, not a publication per token. See
+[Centrifugo for AI apps](https://centrifugal.dev/docs/getting-started/ai_apps),
+which also covers sizing history for a whole response.
+
+## Testing
+
+Test code which uses cent without Centrifugo: give the client an
+`http_client` answering as Centrifugo does - each method at `/api/<method>`,
+its reply `{"result": {...}}` or `{"error": {"code": ..., "message": ...}}`:
+
+```ruby
+REPLIES = {
+  "publish" => { result: { offset: 1, epoch: "e" } },
+  # One response per channel, in order.
+  "broadcast" => { result: { responses: [{ result: { offset: 1 } },
+                                         { error: { code: 102, message: "unknown channel" } }] } },
+  # One reply per command, in order.
+  "batch" => { replies: [{ publish: { offset: 1 } }, { error: { code: 102, message: "unknown channel" } }] }
+}.freeze
+
+centrifugo = ->(url, _headers, _body, _timeout) { [200, {}, JSON.generate(REPLIES.fetch(url.split("/").last))] }
+client = Cent::Client.new(api_endpoint: "http://centrifugo.test/api", http_client: centrifugo)
+```
+
+A test double of your own returns cent's results and raises its errors,
+partial failures included:
+
+```ruby
+ok = Cent::ChannelResult.new(channel: "news", result: Cent::PublishResult.new(offset: 1))
+bad = Cent::ChannelResult.new(channel: "unknown:x", error: Cent::ApiError.for_code(102, "unknown channel"))
+raise Cent::BroadcastFailedChannelsError.new([bad], 2, Cent::BroadcastResult.new(channels: [ok, bad], replied: true))
+```
+
+and for a batch, `Cent::Pending.resolved(result)` or
+`Cent::Pending.resolved(error:)` as each command's outcome, with
+`Cent::BatchFailedCommandsError.new(failed, total, Cent::BatchResult.new(replied: true, failed:))`
+of `Cent::FailedCommand.new(index:, method:, error:)`.
+
+## More
+
+- The API reference, every class, method and field, from the YARD docs:
+  [rubydoc.info/gems/cent](https://rubydoc.info/gems/cent), or `yard server`
+  locally.
+- Docs for agents and LLMs: [centrifugal.dev/llms.txt](https://centrifugal.dev/llms.txt),
+  an index of the current (v6) documentation.
+- Development: see [AGENTS.md](AGENTS.md); `bundle exec rake check` runs what
+  CI does but the integration tests, `bundle exec rake test:integration` those.
 
-MIT — see [LICENSE.txt](LICENSE.txt).
+MIT license. Centrifugo PRO, which the methods marked "Centrifugo PRO only"
+call, is commercial software under its own
+[license](https://centrifugal.dev/license).