monkrb 0.21.0 → 0.22.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 715184d6d6d9b3d6e50ab43d88d3c8ab7c56db310d0d30d90d67150b9dc28e0d
-  data.tar.gz: 16265638cdde3786734a98c8bd0a743b26e4dca7ea3f253b74c32407ec0ae7f1
+  metadata.gz: fa788432dda8104f066b4230e406e690df36c72528f57de126001519576fa0e6
+  data.tar.gz: 36352b5fa73ff475ca37141e8d5aeca9f0cd487845f1521f62bc3048650a57a2
 SHA512:
-  metadata.gz: '097a3550e18dfe987ade8ec24705073b1d0d129fa7303d557acf70ba896186c423d18bb780cbe1e2d6704d7ec8a3706ae163761b7e56225bf7fd651fe0f9163b'
-  data.tar.gz: c1d3e26f194892e893fbdc92a30699fee4b2fbd0424a74bb063b5ca86eda29f76f4909f07e5c44d7211f78d242a1f01f6f36f3d356b37c203d861ffbba97bf83
+  metadata.gz: 03ecd1b575dcb40a8a1a35c4b5b0a2b1c8f867369ffda064d6138fbde7f8db8c586d9d4e302d2ee471a71a29b9c50133ee8a91eaf4c49dec88ca64e0158aecab
+  data.tar.gz: c14485432530c8e4f14ee1ff630208a191ebffb4ec6c1c1fd6326562aba05555a4173e865fe73b913027f896a530166ab0d115468d28dc15efdade92fc49d0c5

data/CHANGELOG.md

@@ -4,6 +4,60 @@ All notable changes to this project are documented here. Format is loosely
 [Keep a Changelog](https://keepachangelog.com/); versions are as released
 in `lib/monk/version.rb`.
 
+## 0.22.0 - 2026-10-13
+
+### Added
+
+- **Recurring jobs.** `Monk::Jobs.recurring "purge_sessions",
+  PurgeSessions, cron: "0 3 * * *"`, declared in `config/recurring.rb`,
+  is enqueued by `bin/jobs` at each occurrence, once however many job
+  processes run. Five-field cron in UTC, plus `@hourly` to `@yearly`. A
+  new task first runs at its next occurrence, missed occurrences run
+  once, and `skip_if_pending: true` skips a run while the previous one
+  hasn't finished. `Monk::Jobs.run_scheduler!(at:)` runs the schedule in
+  tests. One checkpoint row per task in the new `monk_recurring_tasks`
+  table (ADR 0016, `docs/guides/jobs.md`, "Recurring jobs").
+  `monk add jobs` writes `config/recurring.rb` with a commented example
+  and the table's migration, and the base `config/load.rb` loads
+  `config/recurring.rb` after `app/`. An app that already has jobs:
+  `docs/guides/upgrading.md`, 0.22; until then its tests that call
+  `Monk::Jobs.clear!` fail on the missing table.
+- **A login-email rate limit in `Monk::Auth`.**
+  `Monk::Auth.allow_login_request?(email)` allows one login email per
+  address per `login_request_interval:` (60 seconds), counted in a new
+  `login_requests` table, so across every process. The scaffolded login
+  routes use it instead of the per-process `RateLimiter`.
+  `Monk::Auth.clear_login_requests!` resets it in tests.
+- **Auth cleans up after itself.** `Monk::Auth.sweep!` deletes in batches
+  and covers `login_requests`; the new `Monk::Auth::Sweep` job runs it,
+  and `monk add jobs`'s `config/recurring.rb` schedules it nightly
+  (`monk_auth_sweep`) when auth is installed.
+- **An index on `sessions (subject)`**, in the same new auth migration,
+  `create_login_requests`. An app that already has auth:
+  `docs/guides/upgrading.md`, 0.22, "Auth housekeeping".
+
+## 0.21.1 - 2026-10-09
+
+### Fixed
+
+- **`monk add websocket --transport=postgres` publishes through the
+  `:notify` pool.** `config/websocket.rb` declares
+  `Monk::Persistence::Pg.pool(:notify, size: 1)` and builds
+  `PgFanout.new(..., db_pool: :notify)`, so a process holds one publishing
+  connection however many Ractors publish. Before, every `bin/server`
+  worker opened its own. `AppWebSocket.start!` starts the pool (with
+  Redis it does nothing): `config/load.rb` calls it at the end, which
+  covers `bin/server`, `bin/jobs`, `bin/console` and the tests, and
+  `bin/websocket_server` calls it too. An existing app: replace
+  `config/websocket.rb` with
+  `monk help websocket --file config/websocket.rb --transport=postgres`
+  and add the last lines of `monk help base --file config/load.rb`.
+- **The scaffolded `Dockerfile` builds with `monk add postgres`.** Its
+  builder stage installs `build-essential`: `irb`'s dependencies (`erb`,
+  `io-console`, `prism`) compile native extensions. The image that runs
+  still has no compiler. An existing app: add the `apt-get` line from
+  `monk help base --file Dockerfile`.
+
 ## 0.21.0 - 2026-10-09
 
 ### Changed (breaking)

data/README.md

@@ -21,7 +21,7 @@ bin/server           # -> http://localhost:9292/hello
 monk add --list      # the modules you can add: postgres, auth, jobs, live...
 ```
 
-`monk new` scaffolds the app's own `Gemfile` with `gem "monkrb", "~> 0.21", require: "monk"`.
+`monk new` scaffolds the app's own `Gemfile` with `gem "monkrb", "~> 0.22", require: "monk"`.
 
 `monk new` writes a working skeleton (an HTML home page, a `/hello` route, a `/api/hello` JSON route, `public/`, its tests, a `SETUP.md` and an `AGENTS.md` for coding agents), with the app's own code under `app/`: `app/app.rb` and `app/routes/` for routes, `app/views/`, and one directory per role (`models/`, `presenters/`, `helpers/`, `mailers/`, `broadcasts/`, `jobs/`). `monk add` adds Postgres, Redis, email, auth, background jobs, WebSocket and live updates, to a new app or an old one, each with its wiring, commented examples of how to use it, and a test (`monk new my_app --with auth,jobs` does both at once): see [`docs/guides/scaffolding.md`](docs/guides/scaffolding.md), which also says where each kind of code goes.
 
@@ -60,9 +60,9 @@ Everything beyond the core is opt-in (`require "monk"` alone loads none of it).
 | Logging | request log per environment, `Monk::Log.debug`/`info`/`warn`/`error` | — | [`logging.md`](docs/guides/logging.md) |
 | Persistence | `Monk::Persistence::Pg`: raw `pg`, per-Ractor connections, hash-based `Model` | `require "monk/persistence/pg"` (+ `.../pg/model`); needs the `pg` gem | [`persistence.md`](docs/guides/persistence.md) |
 | Migrations | plain `.sql` up/down pairs, `Migrator` | `require "monk/persistence/pg/migrator"`; needs the `pg` gem | [`migrations.md`](docs/guides/migrations.md) |
-| Auth and sessions | `Monk::Auth`: passwordless tokens, Bearer or cookie + CSRF | `require "monk/auth"`; needs the `pg` gem and a registered Postgres connection | [`auth.md`](docs/guides/auth.md) |
+| Auth and sessions | `Monk::Auth`: passwordless tokens, Bearer or cookie + CSRF, a per-address login-email rate limit, nightly cleanup with jobs | `require "monk/auth"`; needs the `pg` gem and a registered Postgres connection | [`auth.md`](docs/guides/auth.md) |
 | Email | `Monk::Mail`: text/HTML email over SMTP or a local relay, one `MAIL_URL`, sent from any worker Ractor, or from a background job with `deliver_later` | `require "monk/mail"`; `deliver_later` is there once `monk/jobs` is loaded too; SMTP needs the `net-smtp` gem | [`mail.md`](docs/guides/mail.md) |
-| Background jobs | `Monk::Jobs`: a queue in Postgres, run by `bin/jobs` on worker Ractors; retries with backoff, scheduled jobs, enqueue inside the app's own transaction, `drain!` for tests | `require "monk/jobs"` (+ `require "monk/jobs/runtime"` in the job process); needs the `pg` gem and a registered Postgres connection | [`jobs.md`](docs/guides/jobs.md) |
+| Background jobs | `Monk::Jobs`: a queue in Postgres, run by `bin/jobs` on worker Ractors; retries with backoff, scheduled jobs, recurring jobs on a cron schedule (UTC), enqueue inside the app's own transaction, `drain!` for tests | `require "monk/jobs"` (+ `require "monk/jobs/runtime"` in the job process); needs the `pg` gem and a registered Postgres connection | [`jobs.md`](docs/guides/jobs.md) |
 | WebSocket | `Monk::WebSocket`: RFC 6455 server as its own process, Redis or Postgres fan-out | `require "monk/websocket"`; fan-out: `require "monk/websocket/redis_fanout"` (+ the `redis` gem) or `require "monk/websocket/pg_fanout"` (+ the `pg` gem) | [`websocket.md`](docs/guides/websocket.md) |
 | Live updates | `Monk::Live`: server-rendered HTML patches pushed to open tabs | `require "monk/live"`; needs the WebSocket server and, across processes, Redis or Postgres | [`live.md`](docs/guides/live.md) |
 | Scaffolding | `monk new` and `monk add`: one generator per module, for a new app or an existing one; text or `--json` output | — (the `monk` command: `monk new NAME --with ...`, `monk add MODULE`, `monk add --list`) | [`scaffolding.md`](docs/guides/scaffolding.md) |
@@ -88,6 +88,6 @@ More (server modes, Docker) in [`docs/development.md`](docs/development.md).
 
 ## Status
 
-Monk is **pre-1.0** (see `lib/monk/version.rb` and the [changelog](CHANGELOG.md)): the API may still change between minor versions. The core is described in [`docs/history/core-plan.md`](docs/history/core-plan.md). Features to add are listed in [`docs/roadmap.md`](docs/roadmap.md). Done so far, each with its own design doc and plan: persistence, migrations, HTML templating and static assets, auth and sessions, WebSocket with Redis fan-out, log levels, live updates (`Monk::Live`, as of 2026-09-19), deployment support (Dockerfile scaffolding, `docs/guides/deploying.md`), the RubyGems release as `monkrb` (both 2026-09-22), and `Monk::WebSocket::PgFanout`/`monk new --live --postgres` — the same cross-process fan-out over Postgres `LISTEN`/`NOTIFY` instead of Redis, for an app that doesn't want a second piece of infrastructure (`docs/design/live-pg-fanout.md`, 2026-09-24), the built-in mailer `Monk::Mail` (0.17, 2026-09-26, ADR 0012), and background jobs on Postgres, `Monk::Jobs`, with mail sent from jobs (0.18, 2026-09-29, ADRs 0013 and 0014). In 0.19 (2026-10-01): `monk new` lays the app out under `app/`, one directory per role (ADR 0015); a WebSocket fanout listens only in the process that holds the sockets (`#listen!`); `authenticate: :optional` lets visitors' sockets in anonymously, for Live's rules to decide; and the WebSocket server stops cleanly on `TERM` (`docs/design/websocket.md`). In 0.20 (2026-10-05): Monk recovers from a dropped database connection (a Postgres restart, a failover) without a process restart. `checkout` reconnects a dead connection before running its block, never retrying the block itself; both fanouts' listeners reconnect and then make every open page resync; and a route that raises logs why (`docs/history/plan-pg-reconnect.md`). Also new: connection pools for Ractors that shouldn't each hold a connection (`Monk::Persistence::Pg.pool`). The WebSocket server's session checks, Live's subscribe rules and `PgFanout`'s publisher can run through one, so open pages share a few connections instead of holding one each: 1,000 open sockets on 4 connections (`docs/guides/persistence.md`, "Pools"; `docs/history/plan-pg-pool.md`). In 0.21 (2026-10-09): scaffolding by module. `monk new` makes the core app, and `monk add` adds a module (Postgres, Redis, mail, auth, jobs, WebSocket, Live) to any app, new or existing, with its wiring, commented examples, a test, and its sections of `SETUP.md` and an `AGENTS.md` for coding agents; `--json` for agents and scripts. Redis and WebSocket become modules of their own, the WebSocket transport is chosen once per app, and `monk new`'s module flags are gone (ADR 0017, `docs/guides/scaffolding.md`, `docs/history/plan-scaffold.md`).
+Monk is **pre-1.0** (see `lib/monk/version.rb` and the [changelog](CHANGELOG.md)): the API may still change between minor versions. The core is described in [`docs/history/core-plan.md`](docs/history/core-plan.md). Features to add are listed in [`docs/roadmap.md`](docs/roadmap.md). Done so far, each with its own design doc and plan: persistence, migrations, HTML templating and static assets, auth and sessions, WebSocket with Redis fan-out, log levels, live updates (`Monk::Live`, as of 2026-09-19), deployment support (Dockerfile scaffolding, `docs/guides/deploying.md`), the RubyGems release as `monkrb` (both 2026-09-22), and `Monk::WebSocket::PgFanout`/`monk new --live --postgres` — the same cross-process fan-out over Postgres `LISTEN`/`NOTIFY` instead of Redis, for an app that doesn't want a second piece of infrastructure (`docs/design/live-pg-fanout.md`, 2026-09-24), the built-in mailer `Monk::Mail` (0.17, 2026-09-26, ADR 0012), and background jobs on Postgres, `Monk::Jobs`, with mail sent from jobs (0.18, 2026-09-29, ADRs 0013 and 0014). In 0.19 (2026-10-01): `monk new` lays the app out under `app/`, one directory per role (ADR 0015); a WebSocket fanout listens only in the process that holds the sockets (`#listen!`); `authenticate: :optional` lets visitors' sockets in anonymously, for Live's rules to decide; and the WebSocket server stops cleanly on `TERM` (`docs/design/websocket.md`). In 0.20 (2026-10-05): Monk recovers from a dropped database connection (a Postgres restart, a failover) without a process restart. `checkout` reconnects a dead connection before running its block, never retrying the block itself; both fanouts' listeners reconnect and then make every open page resync; and a route that raises logs why (`docs/history/plan-pg-reconnect.md`). Also new: connection pools for Ractors that shouldn't each hold a connection (`Monk::Persistence::Pg.pool`). The WebSocket server's session checks, Live's subscribe rules and `PgFanout`'s publisher can run through one, so open pages share a few connections instead of holding one each: 1,000 open sockets on 4 connections (`docs/guides/persistence.md`, "Pools"; `docs/history/plan-pg-pool.md`). In 0.21 (2026-10-09): scaffolding by module. `monk new` makes the core app, and `monk add` adds a module (Postgres, Redis, mail, auth, jobs, WebSocket, Live) to any app, new or existing, with its wiring, commented examples, a test, and its sections of `SETUP.md` and an `AGENTS.md` for coding agents; `--json` for agents and scripts. Redis and WebSocket become modules of their own, the WebSocket transport is chosen once per app, and `monk new`'s module flags are gone (ADR 0017, `docs/guides/scaffolding.md`, `docs/history/plan-scaffold.md`). In 0.22 (2026-10-13): recurring jobs on a cron schedule, `Monk::Jobs.recurring` in `config/recurring.rb`, enqueued once per occurrence however many `bin/jobs` run (ADR 0016, `docs/guides/jobs.md`); auth housekeeping, with a login-email rate limit in the database, an index on `sessions (subject)` and a nightly cleanup job (`docs/guides/auth.md`); and an upgrade guide for apps made with an earlier Monk (`docs/guides/upgrading.md`).
 
 [Made with Love ❤️, Ruby 💎 and AI 🤖](docs/ai_usage_disclaimer.md)

data/lib/monk/auth/errors.rb

@@ -13,4 +13,9 @@ module Monk
 
   class MissingAuthDeliveryError < StandardError
   end
+
+  # Monk::Auth.clear_login_requests! outside MONK_ENV=test: it would lift
+  # the login-email rate limit for every address.
+  class ClearLoginRequestsOutsideTestsError < StandardError
+  end
 end

data/lib/monk/auth/sweep.rb

@@ -0,0 +1,23 @@
+# Monk::Auth::Sweep. Loaded at the end of whichever of monk/auth and
+# monk/jobs is required second, once both are defined (docs/adr/0017), so
+# it requires neither itself.
+
+module Monk
+  module Auth
+    # Monk::Auth.sweep! as a job: deletes expired login tokens and sessions
+    # and old login requests. config/recurring.rb (monk add jobs) runs it
+    # nightly when auth is installed (docs/history/plan-auth-housekeeping.md
+    # decision 1):
+    #
+    #   Monk::Jobs.recurring "monk_auth_sweep", Monk::Auth::Sweep, cron: "17 3 * * *" if defined?(Monk::Auth::Sweep)
+    class Sweep < Monk::Job
+      def self.perform
+        counts = Monk::Auth.sweep!
+        Monk::Log.info(
+          "Monk::Auth::Sweep: deleted #{counts[:sessions]} sessions, #{counts[:login_tokens]} login tokens, " \
+          "#{counts[:login_requests]} login requests",
+        )
+      end
+    end
+  end
+end

data/lib/monk/auth.rb

@@ -16,6 +16,34 @@ module Monk
   module Auth
     REQUIRED_CONFIG_KEYS = %i[db_name secret login_ttl session_ttl].freeze
 
+    # The login-email rate limit (docs/history/plan-auth-housekeeping.md decision
+    # 5): records a request unless one was recorded for the address less
+    # than $2 seconds ago. One statement, so racing requests for one
+    # address let exactly one through; a refused request changes nothing.
+    LOGIN_REQUEST_SQL = <<~SQL.freeze
+      INSERT INTO login_requests (email) VALUES ($1)
+      ON CONFLICT (email) DO UPDATE SET requested_at = now()
+        WHERE login_requests.requested_at < now() - $2::int * interval '1 second'
+      RETURNING 1
+    SQL
+
+    SWEEP_BATCH_SIZE = 10_000
+
+    # sweep!'s statements, one batch each. Tokens and sessions expire by
+    # the app's clock, as redeem and verify check them; login requests by
+    # the database's, as allow_login_request? does.
+    SWEEP_LOGIN_TOKENS_SQL = <<~SQL.freeze
+      DELETE FROM login_tokens WHERE id IN (SELECT id FROM login_tokens WHERE expires_at < $1 LIMIT $2)
+    SQL
+    SWEEP_SESSIONS_SQL = <<~SQL.freeze
+      DELETE FROM sessions WHERE id IN (SELECT id FROM sessions WHERE expires_at < $1 LIMIT $2)
+    SQL
+    SWEEP_LOGIN_REQUESTS_SQL = <<~SQL.freeze
+      DELETE FROM login_requests WHERE email IN (
+        SELECT email FROM login_requests WHERE requested_at < now() - $1::int * interval '1 second' LIMIT $2
+      )
+    SQL
+
     class << self
       # Called from Base#freeze! (Seam B), via Monk.freeze_hooks. Freezes
       # the value, not the module -- Monk::Auth is always Ractor.shareable?
@@ -37,15 +65,17 @@ module Monk
       # callback_path: is where the app's own callback route redeems a
       # token (GET "#{callback_path}/:token"). Monk doesn't define that
       # route, but builds links to it: login_link below, and so
-      # Monk::Auth::SendLoginLink.
+      # Monk::Auth::SendLoginLink. login_request_interval: is the seconds
+      # allow_login_request? makes an address wait between login emails.
       def configure(
         db_name: nil, secret: nil, login_ttl: nil, session_ttl: nil, redirect_allowlist: [], secure: true,
-        deliver: nil, callback_path: "/auth/callback"
+        deliver: nil, callback_path: "/auth/callback", login_request_interval: 60
       )
         config = {
           db_name: db_name, secret: secret, login_ttl: login_ttl, session_ttl: session_ttl,
           redirect_allowlist: redirect_allowlist, secure: secure, deliver: deliver,
           callback_path: checked_callback_path(callback_path),
+          login_request_interval: checked_login_request_interval(login_request_interval),
         }
 
         REQUIRED_CONFIG_KEYS.each do |key|
@@ -86,6 +116,37 @@ module Monk
         raw
       end
 
+      # The login-email rate limit, for the login route to check before it
+      # sends or enqueues the link: true, and recorded, if no link was
+      # requested for this address in the last login_request_interval
+      # seconds; false otherwise. Per address, across every process. The
+      # address is stripped and downcased for the limit only.
+      #
+      #   halt(429) unless Monk::Auth.allow_login_request?(email)
+      def allow_login_request?(email)
+        ensure_configured!
+        unless email.is_a?(String) && !email.strip.empty?
+          raise ArgumentError, "Monk::Auth.allow_login_request? needs an email address, got #{email.inspect}"
+        end
+
+        params = [email.strip.downcase, config[:login_request_interval]]
+        Monk::Persistence::Pg.checkout(config[:db_name]) { |c| c.exec_params(LOGIN_REQUEST_SQL, params).ntuples == 1 }
+      end
+
+      # Test-only: forgets every login request, so the next test can log in
+      # again at once. Refuses unless MONK_ENV is test.
+      def clear_login_requests!
+        unless Monk.env.test?
+          raise Monk::ClearLoginRequestsOutsideTestsError,
+            "Monk::Auth.clear_login_requests! lifts the login-email rate limit for every address, so it only " \
+            "runs with MONK_ENV=test (this is #{Monk.env})"
+        end
+
+        ensure_configured!
+        Monk::Persistence::Pg.checkout(config[:db_name]) { |c| c.exec("DELETE FROM login_requests") }
+        nil
+      end
+
       # The magic link for a raw token: the app's public_url
       # (config/settings.rb, a trusted origin -- never request headers like
       # X-Forwarded-Host, which any direct client can spoof), then
@@ -214,15 +275,28 @@ module Monk
         rows.size
       end
 
+      # Deletes expired login tokens and sessions, and login requests older
+      # than login_request_interval; returns how many of each. At most
+      # batch_size rows per statement, so a large backlog (the first sweep
+      # of an app that never swept) holds no lock for long. Run nightly by
+      # Monk::Auth::Sweep (docs/history/plan-auth-housekeeping.md decision 3).
       # Deliberately not grown onto Model: a `<` comparison is real
       # query-DSL scope for a hygiene task (docs/design/auth-sessions.md).
-      def sweep!
+      def sweep!(batch_size: SWEEP_BATCH_SIZE)
         ensure_configured!
+        unless batch_size.is_a?(Integer) && batch_size.positive?
+          raise ArgumentError, "batch_size: must be a positive Integer, got #{batch_size.inspect}"
+        end
+
         Monk::Persistence::Pg.checkout(config[:db_name]) do |conn|
           now = Time.now
-          login_tokens_deleted = conn.exec_params("DELETE FROM login_tokens WHERE expires_at < $1", [now]).cmd_tuples
-          sessions_deleted = conn.exec_params("DELETE FROM sessions WHERE expires_at < $1", [now]).cmd_tuples
-          { login_tokens: login_tokens_deleted, sessions: sessions_deleted }
+          {
+            login_tokens: delete_in_batches(conn, SWEEP_LOGIN_TOKENS_SQL, [now], batch_size),
+            sessions: delete_in_batches(conn, SWEEP_SESSIONS_SQL, [now], batch_size),
+            login_requests: delete_in_batches(
+              conn, SWEEP_LOGIN_REQUESTS_SQL, [config[:login_request_interval]], batch_size,
+            ),
+          }
         end
       end
 
@@ -279,6 +353,26 @@ module Monk
         path.chomp("/").freeze
       end
 
+      # Runs a batched DELETE until one deletes less than a full batch; the
+      # batch size is its last parameter.
+      def delete_in_batches(conn, sql, params, batch_size)
+        total = 0
+        loop do
+          deleted = conn.exec_params(sql, params + [batch_size]).cmd_tuples
+          total += deleted
+          return total if deleted < batch_size
+        end
+      end
+
+      def checked_login_request_interval(seconds)
+        unless seconds.is_a?(Integer) && seconds.positive?
+          raise Monk::InvalidAuthConfigError,
+            "Monk::Auth.configure(login_request_interval:) must be a positive Integer of seconds, got #{seconds.inspect}"
+        end
+
+        seconds
+      end
+
       def hash_token(raw)
         Digest::SHA256.hexdigest(raw)
       end
@@ -328,3 +422,4 @@ end
 # Monk::Auth::SendLoginLink, when jobs are loaded too -- either order works
 # (lib/monk/jobs.rb ends the same way, docs/adr/0017).
 require_relative "auth/send_login_link" if Monk.const_defined?(:Jobs)
+require_relative "auth/sweep" if Monk.const_defined?(:Jobs)

data/lib/monk/generators/auth.rb

@@ -9,6 +9,7 @@ Monk::Generator.define(:auth) do
   copy "app/routes/auth.rb", role: :example
   copy "test/auth_test.rb", role: :test
   migration "create_auth_tables"
+  migration "create_login_requests"
 
   env :development, { "AUTH_SECRET" => "change-me-dev-secret" }
   env :test, { "AUTH_SECRET" => "change-me-test-secret" }

data/lib/monk/generators/jobs.rb

@@ -4,11 +4,12 @@ Monk::Generator.define(:jobs) do
   installed_if "config/jobs.rb"
   depends_on :postgres
 
-  copy "config/jobs.rb", role: :wiring
+  copy "config/jobs.rb", "config/recurring.rb", role: :wiring
   copy "bin/jobs", role: :wiring, executable: true
   copy "app/jobs/hello_job.rb", "app/routes/jobs.rb", role: :example
   copy "test/jobs_test.rb", role: :test
   migration "create_jobs_tables"
+  migration "create_recurring_tables"
 
   queues = { "JOBS_WORKERS" => "2", "JOBS_QUEUES" => "mailers,default" }
   env :development, queues
@@ -18,4 +19,5 @@ Monk::Generator.define(:jobs) do
   agents_section "jobs/agents.md"
   next_step "bin/jobs", note: "beside bin/server"
   example "jobs-enqueue", todo: "uncomment, or enqueue your own jobs from any route"
+  example "jobs-recurring", todo: "uncomment to run the demo job every hour"
 end

data/lib/monk/jobs/adapters/pg.rb

@@ -2,11 +2,13 @@ require "json"
 require "time"
 require_relative "../../persistence/pg"
 require_relative "../claim"
+require_relative "../firing"
 
 module Monk
   module Jobs
     module Adapters
-      # The queue on Postgres: monk_jobs + monk_job_payloads + monk_processes
+      # The queue on Postgres: monk_jobs + monk_job_payloads + monk_processes,
+      # plus monk_recurring_tasks for the scheduler
       # (docs/adr/0013-jobs-narrow-state-table-plus-payloads.md). Holds only
       # the registered database's name and freezes itself, so the one
       # instance is shareable with every Ractor; each Ractor reaches the
@@ -151,9 +153,70 @@ module Monk
           WHERE id = $1 AND state = 'running' AND locked_by = $2
         SQL
 
+        # The scheduler's one statement per occurrence (docs/history/plan-scheduled-jobs.md
+        # decisions 5-8). ins, reinit and advance touch the task's row in
+        # mutually exclusive cases; advance is a compare-and-set on the
+        # checkpoint, so of several processes firing the same occurrence
+        # exactly one moves it and enqueues. The job's id is drawn from the
+        # sequence first, so the same UPDATE records it as last_job_id;
+        # that burns an id on calls that don't enqueue, harmless on a
+        # BIGINT. The pending check sits inside advance, so READ
+        # COMMITTED's recheck sees the row a racing UPDATE left. $10 stands
+        # in for now() under Monk::Jobs.run_scheduler!(at:).
+        FIRE_RECURRING_SQL = <<~SQL.freeze
+          WITH gate AS (
+            SELECT $3::timestamptz <= COALESCE($10::timestamptz, now()) AS due
+          ), prev AS (
+            SELECT last_occurrence_at FROM monk_recurring_tasks WHERE key = $1
+          ), seq AS (
+            SELECT nextval(pg_get_serial_sequence('monk_jobs', 'id')) AS id
+          ), ins AS (
+            INSERT INTO monk_recurring_tasks (key, cron, last_occurrence_at)
+            SELECT $1, $2, $3 FROM gate WHERE due
+            ON CONFLICT (key) DO NOTHING
+            RETURNING key
+          ), reinit AS (
+            UPDATE monk_recurring_tasks t
+            SET cron = $2, last_occurrence_at = $3, last_job_id = NULL, updated_at = now()
+            FROM gate
+            WHERE gate.due AND t.key = $1 AND t.cron <> $2
+            RETURNING t.key
+          ), advance AS (
+            UPDATE monk_recurring_tasks t
+            SET last_occurrence_at = $3, updated_at = now(),
+              last_job_id = CASE
+                WHEN $4::boolean AND EXISTS (
+                  SELECT 1 FROM monk_jobs j WHERE j.id = t.last_job_id AND j.state <> 'failed'
+                ) THEN t.last_job_id
+                ELSE seq.id
+              END
+            FROM gate, seq
+            WHERE gate.due AND t.key = $1 AND t.cron = $2 AND t.last_occurrence_at < $3
+            RETURNING t.last_job_id IS DISTINCT FROM seq.id AS skipped, seq.id AS job_id
+          ), job AS (
+            INSERT INTO monk_jobs (id, queue, priority, max_attempts, run_at, state) OVERRIDING SYSTEM VALUE
+            SELECT job_id, $5::text, $6::smallint, $7::smallint, now(), 'available' FROM advance WHERE NOT skipped
+            RETURNING id
+          ), payload AS (
+            INSERT INTO monk_job_payloads (job_id, job_class, args)
+            SELECT id, $8::text, $9::jsonb FROM job
+          )
+          SELECT CASE
+              WHEN NOT (SELECT due FROM gate) THEN 'not_due'
+              WHEN EXISTS (SELECT 1 FROM ins) THEN 'initialized'
+              WHEN EXISTS (SELECT 1 FROM reinit) THEN 'reinitialized'
+              WHEN EXISTS (SELECT 1 FROM advance WHERE skipped) THEN 'skipped_pending'
+              WHEN EXISTS (SELECT 1 FROM advance) THEN 'enqueued'
+              ELSE 'already_handled'
+            END AS result,
+            (SELECT id FROM job) AS job_id,
+            (SELECT extract(epoch FROM last_occurrence_at)::float8 FROM prev) AS previous_checkpoint
+        SQL
+
         CLEAR_SQL = <<~SQL.freeze
           DELETE FROM monk_jobs;
           DELETE FROM monk_processes;
+          DELETE FROM monk_recurring_tasks;
         SQL
 
         attr_reader :db_name
@@ -197,6 +260,30 @@ module Monk
           row && row["state"].to_sym
         end
 
+        # Handles one occurrence of a recurring task: initializes its
+        # checkpoint, enqueues its job, or finds the occurrence handled or
+        # not yet due (Firing). occurrence and clock are Times; clock, set
+        # only by Monk::Jobs.run_scheduler!, replaces the database clock.
+        def fire_recurring(key:, cron:, occurrence:, skip_if_pending:, job_class:, queue:, priority:, max_attempts:,
+                           args:, clock: nil)
+          params = [
+            key, cron, occurrence.utc.iso8601(6), skip_if_pending, queue, priority, max_attempts, job_class, args,
+            clock&.utc&.iso8601(6),
+          ]
+          row = with_connection { |c| c.exec_params(FIRE_RECURRING_SQL, params).first }
+          checkpoint = row["previous_checkpoint"]
+          Firing.new(
+            result: row["result"].to_sym, job_id: row["job_id"] && Integer(row["job_id"]),
+            previous_checkpoint: checkpoint && Time.at(Float(checkpoint).to_r, in: "UTC"),
+          )
+        end
+
+        # Whether monk_recurring_tasks exists, checked once when bin/jobs
+        # starts (docs/history/plan-scheduled-jobs.md decision 9).
+        def recurring_table?
+          with_connection { |c| !c.exec("SELECT to_regclass('monk_recurring_tasks')::text").getvalue(0, 0).nil? }
+        end
+
         # Moves up to `limit` due scheduled jobs to available; returns how many.
         def stage_due(limit = 500)
           with_connection { |c| c.exec_params(STAGE_SQL, [limit]).cmd_tuples }
@@ -256,7 +343,8 @@ module Monk
           nil
         end
 
-        # Every job (payloads by cascade) and every process row. Behind
+        # Every job (payloads by cascade), every process row and every
+        # recurring task's checkpoint. Behind
         # Monk::Jobs.clear!, which only runs in tests.
         def clear!
           with_connection { |c| c.exec(CLEAR_SQL) }

data/lib/monk/jobs/cron.rb

@@ -0,0 +1,180 @@
+require_relative "errors"
+
+module Monk
+  module Jobs
+    # A recurring task's schedule (docs/history/plan-scheduled-jobs.md decision 3):
+    # five fields -- minute, hour, day of month, month, day of week -- each
+    # `*`, numbers, lists (`,`), ranges (`-`) and steps (`/`), plus the
+    # shortcuts in SHORTCUTS. No seconds, no names, no time zones: every
+    # time is UTC.
+    #
+    # When both day fields are restricted, a day matching either counts; a
+    # field starting with `*` (`*/2` too) doesn't restrict, as in Vixie
+    # cron. Day of week 7 is Sunday, like 0.
+    #
+    # Frozen Arrays of Integers all the way down, so a parsed expression is
+    # shareable with every Ractor.
+    Cron = Data.define(:expression, :minutes, :hours, :days, :months, :weekdays, :any_day, :any_weekday)
+
+    class Cron
+      SHORTCUTS = {
+        "@hourly" => "0 * * * *", "@daily" => "0 0 * * *", "@weekly" => "0 0 * * 0",
+        "@monthly" => "0 0 1 * *", "@yearly" => "0 0 1 1 *",
+      }.freeze
+
+      FIELDS = [["minute", 0..59], ["hour", 0..23], ["day of month", 1..31], ["month", 1..12],
+                ["day of week", 0..7],].freeze
+
+      PART = %r{\A(\*|\d+(?:-\d+)?)(?:/(\d+))?\z}
+
+      # The longest month each month can have: February counts its 29th.
+      MONTH_DAYS = [nil, 31, 29, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31].freeze
+
+      MINUTE = 60
+      HOUR = 3600
+      DAY = 86_400
+
+      # Neither lookup can go further than this without having found an
+      # occurrence: February 29th is at most 8 years away.
+      MAX_YEARS = 9
+
+      def self.parse(expression)
+        unless expression.is_a?(String)
+          raise InvalidCronError, "a cron expression is a String, got #{expression.inspect}"
+        end
+
+        normalized = expression.split.join(" ")
+        fields = shortcut(normalized).split
+        unless fields.size == 5
+          raise InvalidCronError,
+            "#{expression.inspect} has #{fields.size} field(s); a cron expression has 5 fields " \
+            "(minute hour day-of-month month day-of-week) or is one of #{SHORTCUTS.keys.join(", ")}"
+        end
+
+        minutes, hours, days, months, weekdays = fields.zip(FIELDS).map do |text, (name, range)|
+          parse_field(text, name, range, expression)
+        end
+        cron = new(
+          expression: normalized.freeze, minutes:, hours:, days:, months:,
+          weekdays: weekdays.map { |day| day % 7 }.uniq.sort.freeze,
+          any_day: fields[2].start_with?("*"), any_weekday: fields[4].start_with?("*"),
+        )
+        cron.check_matches!
+        Ractor.make_shareable(cron)
+      end
+
+      def self.shortcut(normalized)
+        return normalized unless normalized.start_with?("@")
+
+        SHORTCUTS.fetch(normalized) do
+          raise InvalidCronError, "unknown cron shortcut #{normalized.inspect}; known: #{SHORTCUTS.keys.join(", ")}"
+        end
+      end
+
+      def self.parse_field(text, name, range, expression)
+        invalid = ->(problem) { raise InvalidCronError, "#{expression.inspect}: #{name} #{problem}" }
+
+        text.split(",", -1).flat_map do |part|
+          match = PART.match(part) || invalid.call("#{part.inspect} isn't a number, range or step")
+          base, step = match.captures
+          invalid.call("#{part.inspect}: a step goes on * or a range (a-b/n)") if step && !base.match?(/[*-]/)
+          invalid.call("#{part.inspect}: the step is 0") if step&.to_i&.zero?
+
+          from, to = base == "*" ? range.minmax : base.split("-").map(&:to_i).then { |a, b| [a, b || a] }
+          [from, to].reject { |value| range.cover?(value) }.each do |value|
+            invalid.call("#{value} is out of range (#{range.min}-#{range.max})")
+          end
+          invalid.call("range #{part.inspect} runs backwards") if from > to
+
+          (from..to).step(step ? step.to_i : 1).to_a
+        end.uniq.sort.freeze
+      end
+      private_class_method :shortcut, :parse_field
+
+      def to_s = expression
+
+      # The latest occurrence at or before `time`, as a UTC Time on a whole
+      # minute. Steps field by field, so a yearly expression takes a handful
+      # of steps, not a year of minutes.
+      def previous(time)
+        time = minute_of(time)
+        limit = time.year - MAX_YEARS
+        loop do
+          raise InvalidCronError, "#{expression.inspect}: no occurrence found" if time.year < limit
+
+          time =
+            if !months.include?(time.month)
+              month = months.rfind { |m| m < time.month }
+              month ? end_of_month(time.year, month) : end_of_month(time.year - 1, months.last)
+            elsif !day?(time)
+              day_of(time) - MINUTE
+            elsif !hours.include?(time.hour)
+              hour = hours.rfind { |h| h < time.hour }
+              hour ? day_of(time) + (hour * HOUR) + (59 * MINUTE) : day_of(time) - MINUTE
+            elsif !minutes.include?(time.min)
+              minute = minutes.rfind { |m| m < time.min }
+              minute ? hour_of(time) + (minute * MINUTE) : hour_of(time) - MINUTE
+            else
+              return time
+            end
+        end
+      end
+
+      # The first occurrence after `time`, as a UTC Time on a whole minute.
+      def next_after(time)
+        time = minute_of(time) + MINUTE
+        limit = time.year + MAX_YEARS
+        loop do
+          raise InvalidCronError, "#{expression.inspect}: no occurrence found" if time.year > limit
+
+          time =
+            if !months.include?(time.month)
+              month = months.find { |m| m > time.month }
+              month ? Time.utc(time.year, month) : Time.utc(time.year + 1, months.first)
+            elsif !day?(time)
+              day_of(time) + DAY
+            elsif !hours.include?(time.hour)
+              hour = hours.find { |h| h > time.hour }
+              hour ? day_of(time) + (hour * HOUR) : day_of(time) + DAY
+            elsif !minutes.include?(time.min)
+              minute = minutes.find { |m| m > time.min }
+              minute ? hour_of(time) + (minute * MINUTE) : hour_of(time) + HOUR
+            else
+              return time
+            end
+        end
+      end
+
+      # A day of month on its own (day of week *) must exist in one of the
+      # months, or the expression never matches: `0 0 30 2 *`. With day of
+      # week restricted, some week always has a matching day.
+      def check_matches!
+        return unless any_weekday && !any_day
+        return if months.any? { |month| days.first <= MONTH_DAYS[month] }
+
+        raise InvalidCronError, "#{expression.inspect} never matches: none of its months has day #{days.first}"
+      end
+
+      private
+
+      def minute_of(time)
+        utc = time.getutc
+        Time.utc(utc.year, utc.month, utc.day, utc.hour, utc.min)
+      end
+
+      def day_of(time) = Time.utc(time.year, time.month, time.day)
+
+      def hour_of(time) = Time.utc(time.year, time.month, time.day, time.hour)
+
+      def end_of_month(year, month)
+        (month == 12 ? Time.utc(year + 1, 1, 1) : Time.utc(year, month + 1, 1)) - MINUTE
+      end
+
+      def day?(time)
+        in_month = days.include?(time.day)
+        in_week = weekdays.include?(time.wday)
+        any_day || any_weekday ? in_month && in_week : in_month || in_week
+      end
+    end
+  end
+end