monkrb 0.19.0 → 0.20.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 5745b05b44d3f2953a037262d97c7a211ebea46b72f9872e6d626c58886338bf
-  data.tar.gz: f6a970fb8f3e724e992238c7c89f230e215cba4b9d36aadc6122bab452b36ea5
+  metadata.gz: 0346b062053981ee7d06c7d3cc9b563c429c1b81443e449b683e53a8080005a1
+  data.tar.gz: 8793cbc85210dc77a03b41e3adc39f58fed92488ef2662d2222c9136f5c55f4f
 SHA512:
-  metadata.gz: 9f16caa7f5d287d95cf1adaf65f72a1da511f9a02671ab06ac51634c4f2a44af2ae76c232b7931ec6118f461116f9fc0ce89d08a56bd1e23eeb8f076f4ff834e
-  data.tar.gz: 8083b153954fee1475e08b778772bdb8592ff597feae9367bf6d589bf22c5859792ea3bf9bc74e5c1baef0567568ee5302b87d1e82addc65a5703081988b7ec7
+  metadata.gz: 711bddca3ff4fbe17dd0b776e3a759ede7a810304e7c42ad96e5e2dfb6c37e9ad5b77ebad8c5f6224a3c993544722e55067ebffc57c1d20f7436c18021b5afae
+  data.tar.gz: e79bb8a6e8d1b77c8bf0227ce0ceeaf16a3ec87b733ce063e8dc32ae7ebd0b24d19915440fb8603f1fe453c74ee91fbe04c68e41e41c84060e6f2d75eaca7353

data/CHANGELOG.md

@@ -4,6 +4,95 @@ All notable changes to this project are documented here. Format is loosely
 [Keep a Changelog](https://keepachangelog.com/); versions are as released
 in `lib/monk/version.rb`.
 
+## 0.20.0 - 2026-10-05
+
+### Fixed
+
+- A route that raises now logs why. With no `error` handler for the
+  exception, `log/<env>.log` gets `ERROR GET /path raised Class: message`
+  and the first 10 backtrace lines; before, only `GET /path -> 500` was
+  logged, in development too. An exception an `error` handler takes care
+  of logs one `INFO ... (handled)` line.
+- `Monk::Persistence::Pg` recovers from a dropped connection (a Postgres
+  restart, a failover, `pg_terminate_backend`). Before, every request
+  that touched the database answered `500` until the process restarted.
+  `checkout` now checks the connection first, about 1 µs when it's
+  healthy, and reconnects a dead one. A block is never run twice: a drop
+  in the middle of one fails that request, and the next checkout
+  reconnects. While Postgres is down, `checkout` raises
+  `PG::ConnectionBad` at once.
+- `Monk::WebSocket::PgFanout`'s publisher reconnects after a dropped
+  connection. Before, every `Monk::Live.patch`/`batch` in that Ractor
+  raised `PG::ConnectionBad` from then on, and no open page updated. A
+  failed `pg_notify` isn't retried; the client's resync recovers it.
+- `Monk::WebSocket::PgFanout`'s `LISTEN` connection reconnects after a
+  drop, with backoff (500 ms doubling to 30 s). Before, its Ractor ended
+  silently and the WebSocket process never got another process's
+  broadcast again. Once listening again it closes every open socket with
+  `1011`, so each page reconnects and resyncs the broadcasts it missed. A
+  notify that can't be decoded or relayed is logged and skipped instead
+  of ending the listener.
+- `Monk::WebSocket::RedisFanout`'s subscriber resubscribes after Redis
+  drops its connection, the same way `PgFanout`'s `LISTEN` does now:
+  backoff, then every open socket closed with `1011` so pages resync.
+  Before, it ended silently. A message without an origin separator is
+  logged and skipped; it used to reach sockets as `nil`.
+- `Monk::WebSocket::Connection#close` called from a thread other than the
+  handler's (a relay, the `reverify_interval` check) could leave the
+  socket open, so the client never saw the close. It now shuts the
+  socket down; `Server.serve` closes it.
+
+- `Monk::Persistence::Pg` connections and `PgFanout`'s publisher get
+  `connect_timeout: 5` unless the app sets one, so a reconnect to a host
+  that doesn't answer fails instead of waiting indefinitely.
+- A WebSocket socket's own connections are closed when the socket ends:
+  its Postgres connection (`authenticate:` without `db_pool:`, or a
+  handler that queries) and a `PgFanout`/`RedisFanout` publisher (a
+  handler that broadcasts). Before, they stayed open until the garbage
+  collector happened to run, which a mostly idle server may not do for a
+  long time, so closed pages kept counting against Postgres's
+  `max_connections`. New: `Monk::Persistence.disconnect_all` closes every
+  connection the calling Ractor opened, and `PgFanout.disconnect_publisher`
+  / `RedisFanout.disconnect_publisher` its publisher; `Server.serve` calls
+  them as each socket ends.
+- `monk new --postgres` writes `db/migrate/.keep`. The directory was
+  created empty, and git doesn't keep an empty directory, so a fresh
+  clone of the app had no `db/migrate/`.
+- An app scaffolded without `--auth` and without `--live` crashed at boot
+  in `bin/websocket_server` (`authenticate must be false, true or
+  :optional, got nil`). The template now passes `false`; fix an existing
+  app's line to `authenticate = !!(defined?(Monk::Auth) && Monk::Auth.config)`.
+- A `checkout` block that ran a raw `BEGIN` and raised no longer leaves
+  its transaction open for the next checkout in that Ractor: it's rolled
+  back.
+
+### Added
+
+- `monk new --auth` checks WebSocket sessions through a pool:
+  `config/auth.rb` declares `Monk::Persistence::Pg.pool(:auth, size: 4)`,
+  and both `bin/websocket_server` templates start it and pass
+  `db_pool: :auth`. A new app's WebSocket process holds 4 connections
+  however many pages are open, instead of one per open page.
+
+- Connection pools for Ractors that shouldn't each hold a connection
+  (`docs/guides/persistence.md`, "Pools"): `Monk::Persistence::Pg.pool(:auth,
+  size: 4)` declares one, `start_pools!(:auth)` starts it in the process that
+  uses it, and `pool(:auth).call(Contacts, :include?, owner, email)` runs that
+  method in one of its worker Ractors, from any Ractor. Errors keep their
+  class (`rescue PG::UniqueViolation` works), calls time out, a full queue
+  refuses calls, dying workers are replaced, and `call_async` doesn't wait.
+  `Monk.boot` lists running pools.
+- `Monk::WebSocket::Server.new(db_pool:)` checks sessions through a pool:
+  open sockets no longer hold a Postgres connection each (1,000 sockets on
+  4 connections).
+- `Monk::Live.authorize(..., db_pool:)` runs a rule in a pool.
+- `Monk::WebSocket::PgFanout.new(registry, db_pool:)` publishes through a
+  pool of size 1: one publishing connection per process, and
+  `Monk::Live.patch` no longer waits for (or raises because of) Postgres.
+- `Monk::WebSocket::Registry#close_all(code, reason)` closes every
+  registered connection's socket with that code, through a
+  `Monk::WebSocket::CloseRequest` sent to each port.
+
 ## 0.19.0 - 2026-10-01
 
 ### Fixed

data/README.md

@@ -87,6 +87,6 @@ More (server modes, Docker) in [`docs/development.md`](docs/development.md).
 
 ## Status
 
-Monk is **pre-1.0** (see `lib/monk/version.rb` and the [changelog](CHANGELOG.md)): the API may still change between minor versions. The core is described in [`docs/history/core-plan.md`](docs/history/core-plan.md). There's no open roadmap issue at the moment; new work is proposed and tracked as it comes up. Done so far, each with its own design doc and plan: persistence, migrations, HTML templating and static assets, auth and sessions, WebSocket with Redis fan-out, log levels, live updates (`Monk::Live`, as of 2026-09-19), deployment support (Dockerfile scaffolding, `docs/guides/deploying.md`), the RubyGems release as `monkrb` (both 2026-09-22), and `Monk::WebSocket::PgFanout`/`monk new --live --postgres` — the same cross-process fan-out over Postgres `LISTEN`/`NOTIFY` instead of Redis, for an app that doesn't want a second piece of infrastructure (`docs/design/live-pg-fanout.md`, 2026-09-24), the built-in mailer `Monk::Mail` (0.17, 2026-09-26, ADR 0012), and background jobs on Postgres, `Monk::Jobs`, with mail sent from jobs (0.18, 2026-09-29, ADRs 0013 and 0014). In 0.19 (2026-10-01): `monk new` lays the app out under `app/`, one directory per role (ADR 0015); a WebSocket fanout listens only in the process that holds the sockets (`#listen!`); `authenticate: :optional` lets visitors' sockets in anonymously, for Live's rules to decide; and the WebSocket server stops cleanly on `TERM` (`docs/design/websocket.md`).
+Monk is **pre-1.0** (see `lib/monk/version.rb` and the [changelog](CHANGELOG.md)): the API may still change between minor versions. The core is described in [`docs/history/core-plan.md`](docs/history/core-plan.md). There's no open roadmap issue at the moment; new work is proposed and tracked as it comes up. Done so far, each with its own design doc and plan: persistence, migrations, HTML templating and static assets, auth and sessions, WebSocket with Redis fan-out, log levels, live updates (`Monk::Live`, as of 2026-09-19), deployment support (Dockerfile scaffolding, `docs/guides/deploying.md`), the RubyGems release as `monkrb` (both 2026-09-22), and `Monk::WebSocket::PgFanout`/`monk new --live --postgres` — the same cross-process fan-out over Postgres `LISTEN`/`NOTIFY` instead of Redis, for an app that doesn't want a second piece of infrastructure (`docs/design/live-pg-fanout.md`, 2026-09-24), the built-in mailer `Monk::Mail` (0.17, 2026-09-26, ADR 0012), and background jobs on Postgres, `Monk::Jobs`, with mail sent from jobs (0.18, 2026-09-29, ADRs 0013 and 0014). In 0.19 (2026-10-01): `monk new` lays the app out under `app/`, one directory per role (ADR 0015); a WebSocket fanout listens only in the process that holds the sockets (`#listen!`); `authenticate: :optional` lets visitors' sockets in anonymously, for Live's rules to decide; and the WebSocket server stops cleanly on `TERM` (`docs/design/websocket.md`). In 0.20 (2026-10-05): Monk recovers from a dropped database connection (a Postgres restart, a failover) without a process restart. `checkout` reconnects a dead connection before running its block, never retrying the block itself; both fanouts' listeners reconnect and then make every open page resync; and a route that raises logs why (`docs/history/plan-pg-reconnect.md`). Also new: connection pools for Ractors that shouldn't each hold a connection (`Monk::Persistence::Pg.pool`). The WebSocket server's session checks, Live's subscribe rules and `PgFanout`'s publisher can run through one, so open pages share a few connections instead of holding one each: 1,000 open sockets on 4 connections (`docs/guides/persistence.md`, "Pools"; `docs/history/plan-pg-pool.md`).
 
 [Made with Love ❤️, Ruby 💎 and AI 🤖](docs/ai_usage_disclaimer.md)

data/lib/monk/base.rb

@@ -12,6 +12,11 @@ module Monk
     EMPTY_ARRAY = [].freeze
     private_constant :EMPTY_ARRAY
 
+    # How much of an unhandled exception's backtrace #log_exception
+    # writes -- the same depth Monk::Jobs keeps for a failed job.
+    BACKTRACE_LINES = 10
+    private_constant :BACKTRACE_LINES
+
     # Action name -> path suffix (appended to the resource path) and verb(s)
     # it dispatches on, for #resources. :update registers both PATCH and PUT --
     # PATCH is the canonical partial-update verb, PUT accepted too since
@@ -162,6 +167,7 @@ module Monk
             [200, context.headers, [body]]
           rescue StandardError => e
             handler = error_handlers.find { |matcher, _| matcher.is_a?(Class) && e.is_a?(matcher) }
+            log_exception(env, e, handled: !handler.nil?)
             if handler
               context.status = 500
               body = context.instance_exec(context, &handler.last)
@@ -173,6 +179,20 @@ module Monk
         end
       end
 
+      # The request line alone ("GET /x -> 500") says nothing about why.
+      # Unhandled: an error, with the first backtrace lines. Handled by an
+      # `error` block: the app chose the response, so one info line --
+      # an error(NotFound) { halt 404 } shouldn't fill the log with ERRORs.
+      def log_exception(env, error, handled:)
+        line = "#{env["REQUEST_METHOD"]} #{env["PATH_INFO"]} raised #{error.class}: #{error.message}"
+        if handled
+          Monk::Log.info("#{line} (handled)")
+        else
+          trace = Array(error.backtrace).first(BACKTRACE_LINES).map { |frame| "\n  #{frame}" }
+          Monk::Log.error("#{line}#{trace.join}")
+        end
+      end
+
       def log_request(env, status, start)
         duration_ms = ((Process.clock_gettime(Process::CLOCK_MONOTONIC) - start) * 1000).round(1)
         line = "#{Monk::Log.timestamp} #{env["REQUEST_METHOD"]} #{env["PATH_INFO"]} -> #{status} (#{duration_ms}ms)"

data/lib/monk/jobs/adapters/pg.rb

@@ -247,10 +247,12 @@ module Monk
         # Ractor's connection is still running on the server, then
         # reconnects, so the worker's next query doesn't wait for the
         # abandoned one (Phase 0.3) or land inside its open transaction.
+        # The cancel stays here: that connection is alive, just busy, so
+        # checkout's own probe wouldn't reset it.
         def reset_connection
           conn = Monk::Persistence::Pg[@db_name]
           conn.cancel
-          conn.reset
+          Monk::Persistence::Pg.revive(conn)
           nil
         end
 

data/lib/monk/live/policy.rb

@@ -7,8 +7,16 @@ module Monk
     # the matching rule says `anonymous: true`: a block like
     # `topic == "contacts:#{subject}"` would otherwise let a nil subject
     # in through "contacts:". A block that raises fails closed.
+    #
+    # A rule with db_pool: runs its block in that Monk::Persistence::Pg
+    # pool (docs/history/plan-pg-pool.md), not in the socket's own Ractor,
+    # so a rule that queries the database doesn't open a connection per
+    # open page. The block crosses to the pool as it is: rules are
+    # shareable procs already. If the pool itself fails (not started in
+    # this process, timed out, full), the rule denies and the reason is
+    # logged.
     module Policy
-      Rule = Data.define(:pattern, :anonymous, :check) do
+      Rule = Data.define(:pattern, :anonymous, :check, :db_pool) do
         def matches?(topic)
           if pattern.end_with?("*")
             topic.start_with?(pattern.chomp("*"))
@@ -18,14 +26,17 @@ module Monk
         end
       end
 
-      def self.build_rule(pattern, anonymous, block)
+      def self.build_rule(pattern, anonymous, block, db_pool = nil)
         unless pattern.is_a?(String) && !pattern.strip.empty? && valid_wildcard?(pattern)
           raise ArgumentError,
             "pattern must be a topic or a topic prefix ending in a single *, got #{pattern.inspect}"
         end
         raise ArgumentError, "authorize needs a block: { |subject, topic| ... }" unless block
+        unless db_pool.nil? || db_pool.is_a?(Symbol)
+          raise ArgumentError, "db_pool must be a pool name (a Symbol), got #{db_pool.inspect}"
+        end
 
-        Rule.new(pattern: pattern.dup.freeze, anonymous: anonymous, check: shareable(block))
+        Rule.new(pattern: pattern.dup.freeze, anonymous: anonymous, check: shareable(block), db_pool: db_pool)
       end
 
       def self.allowed?(rules, subject, topic)
@@ -33,11 +44,35 @@ module Monk
         return false unless rule
         return false if subject.nil? && !rule.anonymous
 
-        rule.check.call(subject, topic) ? true : false
+        run(rule, subject, topic) ? true : false
+      rescue Monk::PersistenceTimeoutError, Monk::PoolFullError, Monk::PoolNotStartedError, Monk::UnknownPoolError,
+             Monk::PoolWorkerDiedError, Monk::PoolStoppedError => e
+        log_pool_failure(rule, topic, e)
+        false
       rescue StandardError
         false
       end
 
+      # What a pooled rule runs in the pool's worker.
+      def self.check(block, subject, topic) = block.call(subject, topic)
+
+      def self.run(rule, subject, topic)
+        return rule.check.call(subject, topic) unless rule.db_pool
+
+        Monk::Persistence::Pg.pool(rule.db_pool).call(Policy, :check, rule.check, subject, topic)
+      end
+      private_class_method :run
+
+      def self.log_pool_failure(rule, topic, error)
+        Monk::Log.error(
+          "Monk::Live: denied #{topic} (rule #{rule.pattern}): its pool #{rule.db_pool.inspect} failed: " \
+          "#{error.class}: #{error.message}",
+        )
+      rescue StandardError
+        nil
+      end
+      private_class_method :log_pool_failure
+
       def self.valid_wildcard?(pattern)
         pattern.count("*").zero? || (pattern.count("*") == 1 && pattern.end_with?("*"))
       end

data/lib/monk/live/session.rb

@@ -95,6 +95,13 @@ module Monk
         seq = 0
         loop do
           payload = @port.receive
+          # Registry#close_all: broadcasts were missed, so the client must
+          # reconnect and resync (ADR 0011). run's read loop ends, and
+          # cleanup unregisters every topic.
+          if payload.is_a?(Monk::WebSocket::CloseRequest)
+            return @connection.close(code: payload.code, reason: payload.reason)
+          end
+
           seq += 1
           @connection.write(payload.start_with?("{") ? "{\"seq\":#{seq},#{payload.byteslice(1..)}" : payload)
         end

data/lib/monk/live.rb

@@ -74,9 +74,11 @@ module Monk
 
       # Boot-time, main Ractor. See Monk::Live::Policy for the semantics:
       # deny by default, first matching rule wins, anonymous subjects need
-      # `anonymous: true`. The block must be Ractor-shareable.
-      def authorize(pattern, anonymous: false, &block)
-        @rules = Ractor.make_shareable([*@rules, Policy.build_rule(pattern, anonymous, block)])
+      # `anonymous: true`. The block must be Ractor-shareable. db_pool:
+      # names a Monk::Persistence::Pg pool to run the block in, for a rule
+      # that queries the database; the WebSocket process must start it.
+      def authorize(pattern, anonymous: false, db_pool: nil, &block)
+        @rules = Ractor.make_shareable([*@rules, Policy.build_rule(pattern, anonymous, block, db_pool)])
       end
 
       def authorized?(subject, topic) = Policy.allowed?(rules, subject, topic)

data/lib/monk/persistence/errors.rb

@@ -4,4 +4,36 @@ module Monk
 
   class UnknownPersistenceError < StandardError
   end
+
+  # Monk::Persistence::Pg.pool(name) for a pool never declared.
+  class UnknownPoolError < StandardError
+  end
+
+  # Monk::Persistence::Pg.pool(name) for a pool declared but not started
+  # in this process (start_pools!).
+  class PoolNotStartedError < StandardError
+  end
+
+  # A pool call's method returned a value that can't be copied to the
+  # caller's Ractor (a PG::Result, say).
+  class PoolReturnError < StandardError
+  end
+
+  # A pool's queue is full: as many calls as its queue: option allows are
+  # already waiting for a worker.
+  class PoolFullError < StandardError
+  end
+
+  # The pool worker running a call died (not an exception from the called
+  # method, which the caller gets): the call may or may not have finished.
+  class PoolWorkerDiedError < StandardError
+  end
+
+  # The pool has stopped: calls waiting when it stopped, and calls after.
+  class PoolStoppedError < StandardError
+  end
+
+  # start_pools! couldn't connect a pool's workers.
+  class PoolStartError < StandardError
+  end
 end

data/lib/monk/persistence/pg.rb

@@ -36,11 +36,102 @@ module Monk
           .tap { |types| %w[json jsonb].each { |name| types.register_type(0, name, nil, JsonDecoder) } },
       )
 
+      # Without a connect_timeout, libpq waits indefinitely for a host that
+      # doesn't answer -- and #checkout's revive does so holding the
+      # Ractor's slot. The same 5 s as Registry's checkout timeout. An
+      # app's own connect_timeout: wins.
+      DEFAULT_CONNECT_TIMEOUT = 5
+
+      # Every diagnostic field libpq reports for an error (SQLSTATE,
+      # constraint, table, detail, hint, ...).
+      DIAG_FIELDS = PG.constants.grep(/\APG_DIAG_/).map { |name| PG.const_get(name) }.freeze
+
+      # What a PG::Error's #result becomes when the error leaves a pool's
+      # worker (#make_portable): a PG::Result is tied to its connection and
+      # can't be copied to another Ractor. Keeps what reading an error
+      # needs; anything else raises NoMethodError.
+      ErrorResult = Data.define(:fields, :error_message, :result_status, :res_status) do
+        def self.from(result)
+          fields = DIAG_FIELDS.to_h { |code| [code, result.error_field(code)] }.compact
+          new(
+            fields: fields, error_message: result.error_message,
+            result_status: result.result_status, res_status: result.res_status,
+          )
+        end
+
+        def error_field(code) = fields[code]
+        alias_method :result_error_field, :error_field
+        alias_method :result_error_message, :error_message
+      end
+
       class << self
+        # A copy of an exception about to leave a pool's worker
+        # (Pool::Portable): a PG::Error holds its connection and result,
+        # which can't be copied to another Ractor. The connection goes (no
+        # use in another Ractor anyway), and the result becomes an
+        # ErrorResult, so error_field still answers. The error keeps its
+        # class, so `rescue PG::UniqueViolation` matches as it would here.
+        def make_portable(error)
+          return unless error.is_a?(PG::Error)
+
+          result = error.result
+          error.instance_variable_set(:@connection, nil)
+          error.instance_variable_set(:@result, result && ErrorResult.from(result))
+        end
+
+        # Detects a connection the server has closed (a restart, a
+        # failover, pg_terminate_backend). libpq still reports
+        # CONNECTION_OK, but the socket is readable: a healthy idle
+        # connection's isn't. So the common case costs one zero-timeout
+        # select, about 1 µs, and nothing sent.
+        #
+        # A readable socket is either the server's FATAL before it closes,
+        # or a NOTIFY on a connection that LISTENs. Reading on to tell
+        # them apart races the close: the FATAL can arrive a moment before
+        # the end of file, and a second select in between sees nothing.
+        # So a readable socket gets an empty query, one round trip, which
+        # fails on a closed connection and leaves pending notifies queued.
+        #
+        # A network path that dies without closing the socket (a NAT
+        # timing out an idle connection) leaves it unreadable, and this
+        # can't see it: the next query waits for TCP to give up.
+        #
+        # Public: PgFanout probes its own connections with it.
+        def alive?(conn)
+          return false unless conn.status == PG::CONNECTION_OK
+          return true unless conn.socket_io.wait_readable(0)
+
+          conn.exec("")
+          true
+        rescue PG::Error, IOError
+          false
+        end
+
+        # Same connection object, new backend: the result type map set by
+        # #connect survives. Raises PG::ConnectionBad if Postgres is still
+        # unreachable; the next checkout tries again.
+        def revive(conn)
+          conn.reset
+        end
+
         private
 
+        # A block that ran a raw BEGIN and raised leaves the connection in
+        # a transaction, and the next checkout in this Ractor would run
+        # inside it. (conn.transaction rolls back by itself.) Skipped for
+        # PQTRANS_ACTIVE, a query still running: a ROLLBACK would first
+        # wait for it. Monk::Jobs cancels those itself.
+        def release(conn)
+          return unless conn.status == PG::CONNECTION_OK
+          return unless [PG::PQTRANS_INTRANS, PG::PQTRANS_INERROR].include?(conn.transaction_status)
+
+          conn.exec("ROLLBACK")
+        rescue PG::Error
+          nil
+        end
+
         def connect(**opts)
-          conn = PG.connect(**opts)
+          conn = PG.connect(connect_timeout: DEFAULT_CONNECT_TIMEOUT, **opts)
           conn.type_map_for_results = PG::BasicTypeMapForResults.new(conn, registry: RESULT_TYPES)
           conn
         end