solid_queue-rate_limit 0.1.0 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f903a1ef2190109dff2e08ad8e57357534a45228f2079097ede82055e095aac2
-  data.tar.gz: 395d34ca629026778e2c6cb96838bedf46250ff4e40d2231e6a63987e49c8288
+  metadata.gz: 494dc86905183d3c7e43d93794d8529a7cee72b8039422137c98601c1c3f9793
+  data.tar.gz: 1926904a64c22e7bccc4f94066ffe308fd9a250700931ee5082b710fcb9dc607
 SHA512:
-  metadata.gz: 8eadc6cd4a9276e3ffd0f8246c28be9c5da25dbe77398abbd0d2f0ac6c4f08814e1447fd69935e3994a8a17d4ee0767e80b84641be72e7c9d6548c01e87ba5ac
-  data.tar.gz: 6296c9c4d330e516b4ee0c89f57adc84409418b9fea6ba13671010f0e5c4369fa2eb84b6d9894574bbf6957407a64950db605021b40770bafc750794134dac68
+  metadata.gz: 01afb44f70f3b73d76b29966687ee7da5876e41c9f70523f9c56f2f4741dcd2d88d6c0d1255a3ff70cd728e966fff93634b5e49f8f72b8626ada29b5272e1e68
+  data.tar.gz: b496de40a83ea8169222ddc5c2bc4e3f3900daeff6c55c71b7fc883e9e816ced70535bb10fc1cf70fe54e83c7af715424de3b873c3e69a2d9eb04632363c0517

data/CHANGELOG.md

@@ -6,6 +6,30 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 
 ## [Unreleased]
 
+## [0.2.0] - 2026-11-06
+
+Adds a budget for each key with `key:` and `group:`, `limits_rate` and `limits_concurrency` on one class, a cleanup for idle budgets, and a retry for claims and dispatches that deadlock. Upgrading takes code changes but no migration: every `limits_rate` needs a `key:`, `to:` now includes the burst, and Solid Queue 1.7.2 is required. See [UPGRADING.md](https://github.com/dtcristo/solid_queue-rate_limit/blob/main/UPGRADING.md).
+
+### Added
+
+- `key:` and `group:` for `limits_rate`, which take what `limits_concurrency` takes. `key:` gives each value a budget of its own, such as one for each account, and `group:` shares a budget between the classes that name it. The key is built like Solid Queue's concurrency key, worked out once when Solid Queue enqueues the job, and kept by its retries. Upgrading needs no migration. ([#19](https://github.com/dtcristo/solid_queue-rate_limit/pull/19), [#24](https://github.com/dtcristo/solid_queue-rate_limit/pull/24))
+- `rate_limit_key` and `rate_limit_group` on job classes. ([#19](https://github.com/dtcristo/solid_queue-rate_limit/pull/19))
+- `SolidQueue::RateLimitState.clear_idle_in_batches(batch_size: 500, sleep_between_batches: 0)`, which deletes the state rows of budgets that are full and have no slots ahead, so per-key rows don't pile up. Clearing never changes what's admitted, and a claim that runs while it clears inserts the row again. The README has a `recurring.yml` entry to run it every hour. ([#20](https://github.com/dtcristo/solid_queue-rate_limit/pull/20))
+- `keys` in the `defer_rate_limited.solid_queue` payload and its log line, listing the budgets that deferred jobs. ([#19](https://github.com/dtcristo/solid_queue-rate_limit/pull/19))
+- `limits_rate` and `limits_concurrency` on one class, in either order and through inheritance. A job over the rate waits as a scheduled job, and gives its concurrency slot back and wakes the next blocked job for its key once the claim commits. When the dispatcher brings it back, it takes a slot again or is blocked. `limits_concurrency(on_conflict: :discard)` with `limits_rate` still raises `ArgumentError`. The README has a recipe for one job at a time for each account under a global rate. ([#22](https://github.com/dtcristo/solid_queue-rate_limit/pull/22))
+- README recipes that combine `limits_rate` with [Solid Queue Coalesce](https://github.com/dtcristo/solid_queue-coalesce) 0.2: collapsing calls for each account with `limits_concurrency` under a global rate, with a table of which control acts at each step, and batching calls for a bulk API under a quota. ([#23](https://github.com/dtcristo/solid_queue-rate_limit/pull/23))
+
+### Changed
+
+- **Breaking:** `limits_rate`'s `to:` is the most a budget admits in any `per`, burst included, as `to:` is in Rails' `rate_limit` and `limits_concurrency`. It was the steady rate, with the burst on top, so `to: 100, per: 1.minute, burst: 10` could admit 109 in a minute. After a burst, jobs are now spaced `per / (to - burst + 1)` apart, and a backlog drains at `to - burst + 1` per `per`. To keep a limit as 0.1.0 enforced it, add `burst - 1` to `to`. A `burst` above `to` raises `ArgumentError`, and the README's recipes and backlog rule use the new meaning. ([#25](https://github.com/dtcristo/solid_queue-rate_limit/pull/25))
+- **Breaking:** `limits_rate` requires `key:`, as `limits_concurrency` and Solid Queue Coalesce's `coalesce` do, and `group:` defaults to Solid Queue's `->(*) { self.class.name }`. Add `key: :global` for one budget per class, stored as `JobClass/global`, or `key: ->(*) {}` to keep the 0.1.0 budget, stored as `JobClass`. A `nil` or `false` key turns the limit off, as it turns off `limits_concurrency`, and a job whose key comes out blank isn't limited, as a job whose concurrency key is blank isn't. With `:global`, jobs enqueued by 0.1.0 drain on the old budget while new ones use the new one, so the limit can admit up to twice its rate until they're gone. Every limited job stores its key, so each claim with limited jobs reads their keys in one more query. ([#24](https://github.com/dtcristo/solid_queue-rate_limit/pull/24))
+- **Breaking:** Requires Solid Queue 1.7.2 or newer within 1.7, because giving a deferred job's concurrency slot back relies on 1.7.2 recording which semaphore a job holds. ([#22](https://github.com/dtcristo/solid_queue-rate_limit/pull/22))
+- A job that still has a concurrency key after its class dropped `limits_concurrency` is checked against the rate, instead of being admitted without a check. ([#22](https://github.com/dtcristo/solid_queue-rate_limit/pull/22))
+- `defer_rate_limited.solid_queue` includes jobs of classes that also declare `limits_concurrency`. Each blocked job a deferral wakes instruments Solid Queue's `release_blocked.solid_queue`. ([#22](https://github.com/dtcristo/solid_queue-rate_limit/pull/22))
+- `name:` still raises `ArgumentError`, and now links to [#14](https://github.com/dtcristo/solid_queue-rate_limit/issues/14), which tracks multiple limits. ([#19](https://github.com/dtcristo/solid_queue-rate_limit/pull/19))
+- Enqueueing a limited job before the migration has run raises `SolidQueue::RateLimitState::PendingMigrations` from every path into Solid Queue, including Solid Queue Coalesce's coalesced enqueues, which skipped the check when that gem's patch loaded after this one's. ([#23](https://github.com/dtcristo/solid_queue-rate_limit/pull/23))
+- A claim whose first pass deadlocks, or a dispatcher batch that deadlocks, runs once more instead of raising, so a deadlock no longer stops the worker or the dispatcher. On MySQL, claims that insert a new budget's row at the same time can deadlock, which `key:` makes common, and so can a claim that defers jobs and a dispatcher batch. A second deadlock still raises. ([#19](https://github.com/dtcristo/solid_queue-rate_limit/pull/19), [#21](https://github.com/dtcristo/solid_queue-rate_limit/pull/21))
+
 ## [0.1.0] - 2026-11-04
 
 The first release adds `limits_rate`, so workers claim a job class's jobs no faster than a declared rate, and jobs over it wait as scheduled jobs. Install it with `bin/rails generate solid_queue:rate_limit:install`, and there's nothing to upgrade from. The [README](https://github.com/dtcristo/solid_queue-rate_limit/blob/main/README.md#what-the-limit-guarantees) states what the limit guarantees and how to pick a `burst` for a backlog.
@@ -18,3 +42,7 @@ The first release adds `limits_rate`, so workers claim a job class's jobs no fas
 - The `defer_rate_limited.solid_queue` event, with `class_name`, `job_ids` and `size`, logged at debug. ([#12](https://github.com/dtcristo/solid_queue-rate_limit/pull/12))
 - Requires Solid Queue 1.7. ([#11](https://github.com/dtcristo/solid_queue-rate_limit/pull/11))
 - A README with recipes, the guarantee and a rule for picking `burst`, and [design notes](https://github.com/dtcristo/solid_queue-rate_limit/blob/main/docs/design.md). ([#13](https://github.com/dtcristo/solid_queue-rate_limit/pull/13))
+
+[Unreleased]: https://github.com/dtcristo/solid_queue-rate_limit/compare/v0.2.0...HEAD
+[0.2.0]: https://github.com/dtcristo/solid_queue-rate_limit/compare/v0.1.0...v0.2.0
+[0.1.0]: https://github.com/dtcristo/solid_queue-rate_limit/releases/tag/v0.1.0

data/README.md

@@ -5,11 +5,11 @@
   </p>
 </div>
 
-Solid Queue Rate Limit adds `limits_rate` to Active Job classes. Workers claim a class's jobs no faster than the rate you declare, and a job over the limit waits as a scheduled job instead of holding a worker thread, sleeping in `perform` or retrying after a 429.
+Solid Queue Rate Limit adds `limits_rate` to Active Job classes. Workers claim no more of a class's jobs in any period than you declare, and a job over the limit waits as a scheduled job instead of holding a worker thread, sleeping in `perform` or retrying after a 429.
 
 ```ruby
 class SyncStripeJob < ApplicationJob
-  limits_rate to: 100, per: 1.minute, burst: 10
+  limits_rate to: 100, per: 1.minute, burst: 10, key: :global
 
   def perform(account_id)
     Account.find(account_id).sync_to_stripe!
@@ -19,6 +19,8 @@ end
 
 The limit is checked when a worker claims the job, inside Solid Queue's own claim transaction, so it holds across any number of workers and processes on SQLite, PostgreSQL and MySQL. Unlimited jobs in the same queue keep running while limited ones wait.
 
+To collapse repeated calls into one job before the limit sees them, pair it with [Solid Queue Coalesce](https://github.com/dtcristo/solid_queue-coalesce). The [last two recipes](#collapse-calls-for-each-account-and-run-them-one-at-a-time-under-a-global-rate) use both.
+
 ## Table of contents
 
 - [Getting started](#getting-started)
@@ -27,6 +29,7 @@ The limit is checked when a worker claims the job, inside Solid Queue's own clai
 - [What the limit guarantees](#what-the-limit-guarantees)
 - [Picking a burst for a backlog](#picking-a-burst-for-a-backlog)
 - [Things to know](#things-to-know)
+- [Clearing idle budgets](#clearing-idle-budgets)
 - [Compatibility](#compatibility)
 - [`limits_rate` options](#limits_rate-options)
 - [Rules and limitations](#rules-and-limitations)
@@ -69,9 +72,9 @@ Until the migration has run, everything else works as before, and enqueuing a jo
 
 Each process caches whether the table exists, so restart workers and web processes after you migrate. A deploy usually does that for you.
 
-The gem relies on private Solid Queue methods, so its gemspec allows one minor version of Solid Queue at a time.
+The gem relies on private Solid Queue methods, so its gemspec allows one minor version of Solid Queue at a time, from 1.7.2.
 
-[CHANGELOG.md](CHANGELOG.md) lists what changed in each version.
+To upgrade from an earlier version, follow [UPGRADING.md](UPGRADING.md). [CHANGELOG.md](CHANGELOG.md) lists what changed in each version.
 
 ## Recipes
 
@@ -80,8 +83,14 @@ Start from what the API on the other end allows.
 - [A quota per minute](#stay-under-a-quota-per-minute)
 - [A fixed gap between calls](#space-jobs-out-evenly)
 - [A high rate, with a backlog that has to keep up](#drain-a-backlog-at-the-full-rate)
+- [A quota for each account](#give-each-account-a-budget)
+- [One quota for several job classes](#share-a-budget-between-classes)
+- [A quota for each account, used by several job classes](#share-a-budget-for-each-account-between-classes)
+- [One job at a time for each account, under one quota](#run-one-job-at-a-time-for-each-account-under-a-global-rate)
+- [One pending and one running job for each account, under one quota](#collapse-calls-for-each-account-and-run-them-one-at-a-time-under-a-global-rate)
+- [A bulk API, with calls batched while they wait](#batch-calls-under-a-quota)
 
-`Account`, `Address`, `Webhook` and the other names in these recipes stand in for your own code.
+`Account`, `Address`, `Shop`, `Webhook` and the other names in these recipes stand in for your own code.
 
 ### Stay under a quota per minute
 
@@ -89,8 +98,8 @@ Use this when an API allows a number of calls in any minute and you want to stay
 
 ```ruby
 class SyncStripeCustomerJob < ApplicationJob
-  # At most 10 + 90 = 100 claims in any minute
-  limits_rate to: 90, per: 1.minute, burst: 10
+  # At most 100 claims in any minute, 10 of them at once
+  limits_rate to: 100, per: 1.minute, burst: 10, key: :global
 
   def perform(account_id)
     Account.find(account_id).sync_to_stripe!
@@ -98,7 +107,7 @@ class SyncStripeCustomerJob < ApplicationJob
 end
 ```
 
-A quiet budget lets 10 jobs through at once, then one every 0.67 seconds. The limit isn't a rolling-window counter, so `to:` alone isn't the most a minute can hold. In any window of `w` seconds, workers claim at most `burst + floor(w / (per / to))` jobs. For a quota of `N` per `per`, keep `to + burst` at or below `N`. See [What the limit guarantees](#what-the-limit-guarantees).
+A quiet budget lets 10 jobs through at once, then one every 60 / 91 seconds, about 0.66. That spacing leaves room for the burst, so no minute holds more than 100, and a backlog drains at 91 a minute. See [What the limit guarantees](#what-the-limit-guarantees).
 
 ### Space jobs out evenly
 
@@ -107,7 +116,7 @@ Use this when an API wants a gap between calls, and two calls close together wou
 ```ruby
 class GeocodeAddressJob < ApplicationJob
   # The default burst: 1 never claims two jobs less than 5 seconds apart
-  limits_rate to: 1, per: 5.seconds
+  limits_rate to: 1, per: 5.seconds, key: :global
 
   def perform(address_id)
     Address.find(address_id).geocode!
@@ -123,8 +132,9 @@ Use this when a backlog builds up and has to drain at the declared rate.
 
 ```ruby
 class SendWebhookJob < ApplicationJob
+  # At most 10 in any second, draining a backlog at 10 - 6 + 1 = 5 a second.
   # 5 × (1 + 0.1) / 1 = 5.5, rounded up, for the default 1-second dispatcher poll and 0.1-second worker poll
-  limits_rate to: 5, per: 1.second, burst: 6
+  limits_rate to: 10, per: 1.second, burst: 6, key: :global
 
   def perform(webhook_id)
     Webhook.find(webhook_id).deliver!
@@ -132,7 +142,7 @@ class SendWebhookJob < ApplicationJob
 end
 ```
 
-If 6 jobs at once is too many for the API, make the dispatcher poll more often instead. With a 0.2-second poll, `5 × (0.2 + 0.1) / 1 = 1.5`, which rounds up to 2, so `limits_rate to: 5, per: 1.second, burst: 2` is enough:
+If 6 jobs at once is too many for the API, make the dispatcher poll more often instead. With a 0.2-second poll, `limits_rate to: 10, per: 1.second, burst: 3, key: :global` is enough, and leaves more of the 10 for the backlog, which drains at `10 - 3 + 1 = 8` a second. `8 × (0.2 + 0.1) / 1 = 2.4`, which rounds up to 3:
 
 ```yaml
 # config/queue.yml
@@ -144,11 +154,143 @@ production:
 
 A shorter poll means more queries from the dispatcher. Solid Queue's [configuration docs](https://github.com/rails/solid_queue/blob/v1.7.0/README.md#configuration) cover `polling_interval` and `batch_size`.
 
+### Give each account a budget
+
+Use this when an API's quota is per account, per API key or per tenant, and one busy account shouldn't use up the others' share.
+
+```ruby
+class SyncShopOrdersJob < ApplicationJob
+  # At most 2 calls in any second for each shop
+  limits_rate to: 2, per: 1.second, burst: 2, key: ->(shop) { shop }
+
+  def perform(shop)
+    shop.sync_orders!
+  end
+end
+```
+
+`key:` gets the job's arguments, as in `limits_concurrency`. A record becomes its class name and id, so the shop with id 42 gets a budget of its own, stored under `SyncShopOrdersJob/Shop/42`. A Proc that returns `nil` sends the job to the class's bare budget, `SyncShopOrdersJob`. `burst: 2` follows the [backlog rule](#picking-a-burst-for-a-backlog), and applies to each shop, whose backlog drains at 1 a second.
+
+### Share a budget between classes
+
+Use this when several job classes call one API, and together they have to stay under its quota.
+
+```ruby
+class SyncStripeCustomerJob < ApplicationJob
+  limits_rate to: 100, per: 1.minute, burst: 10, group: "Stripe", key: :global
+
+  def perform(account_id)
+    Account.find(account_id).sync_to_stripe!
+  end
+end
+
+class RefundStripeChargeJob < ApplicationJob
+  limits_rate to: 100, per: 1.minute, burst: 10, group: "Stripe", key: :global
+
+  def perform(charge_id)
+    Charge.find(charge_id).refund!
+  end
+end
+```
+
+Both classes draw on one budget, stored under `Stripe/global`. Declare the same limit in every class of a group, because each job is charged with its own class's settings. See [Rules and limitations](#rules-and-limitations).
+
+### Share a budget for each account between classes
+
+Use this when several job classes call an API whose quota is per account.
+
+```ruby
+class SyncStripeConnectAccountJob < ApplicationJob
+  # At most 20 calls in any second for each account, with a 0.2-second dispatcher poll
+  limits_rate to: 20, per: 1.second, burst: 5, group: "StripeConnect", key: ->(account) { account }
+
+  def perform(account)
+    account.sync_to_stripe!
+  end
+end
+
+class CreateStripeTransferJob < ApplicationJob
+  limits_rate to: 20, per: 1.second, burst: 5, group: "StripeConnect", key: ->(account, _amount) { account }
+
+  def perform(account, amount)
+    account.transfer!(amount)
+  end
+end
+```
+
+The account with id 7 gets one budget for both classes, stored under `StripeConnect/Account/7`. Each class's Proc gets its own job's arguments, so they can take different ones, as long as they return the same key for the same account. With the [0.2-second dispatcher poll](#drain-a-backlog-at-the-full-rate), `burst: 5` follows the [backlog rule](#picking-a-burst-for-a-backlog), and each account's backlog drains at 16 a second.
+
+### Run one job at a time for each account, under a global rate
+
+Use this when two syncs for one account mustn't run at once, and all accounts together have to stay under an API's quota.
+
+```ruby
+class SyncStripeAccountJob < ApplicationJob
+  # One sync at a time for each account, and at most 100 claims in any minute across all of them
+  limits_concurrency key: ->(account) { account }
+  limits_rate to: 100, per: 1.minute, burst: 10, group: "Stripe", key: :global
+
+  def perform(account)
+    account.sync_to_stripe!
+  end
+end
+```
+
+Solid Queue gives each account one concurrency slot, and the gem checks the rate when a worker claims the job. A job over the rate gives its concurrency slot back while it waits, so the account's next blocked job can take it, and takes one again when it comes back. If there's none free by then, the job is blocked, as any job would be. `limits_concurrency` with `on_conflict: :discard` can't be combined with `limits_rate`, because a job that came back to a taken concurrency slot would be discarded.
+
+### Collapse calls for each account and run them one at a time, under a global rate
+
+Use this when an account's sync can be asked for many times, only the latest needs to run, two syncs for one account mustn't overlap, and all accounts together have to stay under an API's quota. It needs [Solid Queue Coalesce](https://github.com/dtcristo/solid_queue-coalesce).
+
+```ruby
+class SyncShopProductsJob < ApplicationJob
+  # One pending and one running sync for each shop, and at most 2 claims in any second across them
+  coalesce key: ->(shop) { shop }
+  limits_concurrency key: ->(shop) { shop }
+  limits_rate to: 2, per: 1.second, burst: 2, group: "Shopify", key: :global
+
+  def perform(shop)
+    shop.sync_products!
+  end
+end
+```
+
+Each control acts at a different step of the job's life:
+
+| Step | What happens | Control |
+| --- | --- | --- |
+| `perform_later` | The call joins the shop's pending job, or creates one. The job stores the concurrency and rate limit keys of the call that created it | `coalesce` |
+| The job becomes ready | It takes the shop's concurrency slot, or is blocked until the running sync finishes | `limits_concurrency` |
+| A worker claims it | The budget admits it, or it waits as a scheduled job and gives its concurrency slot back | `limits_rate` |
+| `perform` starts | The coalesce key is free, so the next call creates the next pending job | `coalesce` |
+| `perform` finishes | The concurrency slot is free, and the shop's blocked job becomes ready | `limits_concurrency` |
+
+A job waiting on the rate still holds its coalesce key, so calls keep joining it. With `debounce:`, a call can move it later than its rate check, but never earlier. With `on_full: :dispatch`, a full job that's waiting becomes ready, and the next claim sends it back to wait.
+
+### Batch calls under a quota
+
+Use this when an API has a bulk endpoint with a quota on requests, and calls arrive one item at a time. It needs [Solid Queue Coalesce](https://github.com/dtcristo/solid_queue-coalesce).
+
+```ruby
+class PushInventoryJob < ApplicationJob
+  # Up to 100 SKUs in each request, and at most 10 requests in any minute for each shop
+  coalesce key: ->(shop, _sku) { shop }, on_conflict: :collect, max_calls: 100
+  limits_rate to: 10, per: 1.minute, burst: 2, key: ->(shop, _sku) { shop }
+
+  def perform_calls(calls)
+    shop = calls.first.arguments.first
+    shop.push_inventory(calls.map { |call| call.arguments.last }.uniq)
+  end
+end
+```
+
+Each call joins the shop's pending job until it holds 100, and the limit is checked when a worker claims that job. A job waiting on the rate hasn't started, so calls keep joining it, and batches grow as the budget runs out. The limit counts claims, so make one bulk request in each run. A retry runs all of the job's calls again, and is charged again.
+
 ## How waiting works
 
-When a worker claims jobs, the gem checks each limited job against its class's budget, in queue order, inside Solid Queue's claim transaction. A job the budget admits is claimed and runs like any other. A job over the limit moves from `solid_queue_ready_executions` to `solid_queue_scheduled_executions`, with a time to check again. It doesn't take a worker thread, and Active Job doesn't see a retry. When that time comes, the dispatcher makes it ready as usual, and the next claim checks it again.
+When a worker claims jobs, the gem checks each limited job against its budget, in queue order, inside Solid Queue's claim transaction. A job the budget admits is claimed and runs like any other. A job over the limit moves from `solid_queue_ready_executions` to `solid_queue_scheduled_executions`, with a time to check again. It doesn't take a worker thread, and Active Job doesn't see a retry. When that time comes, the dispatcher makes it ready as usual, and the next claim checks it again.
 
-The checks are staggered. Each waiting job gets the next free slot, one interval `per / to` after the last slot handed out, so 100 waiting jobs come back at 100 different times instead of all on the next poll. Slots stop 5 minutes ahead. A job past that checks again in 5 minutes without taking a slot. If waiting jobs get discarded, or you loosen the limit, the slots they held come free within 5 minutes.
+The checks are staggered. Each waiting job gets the next free slot, one interval `per / (to - burst + 1)` after the last slot handed out, so 100 waiting jobs come back at 100 different times instead of all on the next poll. Each budget hands out its own slots, so a backlog on one account's key doesn't push back another account's jobs. Slots stop 5 minutes ahead. A job past that checks again in 5 minutes without taking a slot. If waiting jobs get discarded, or you loosen the limit, the slots they held come free within 5 minutes.
 
 A slot is a hint, and never charges the budget. A job that comes back and finds the budget spent waits again, with the next free slot.
 
@@ -160,13 +302,16 @@ Unlimited jobs keep running. Only limited jobs move to scheduled executions, and
 
 The limit counts claims at the worker gate, the moment a worker takes a job from `solid_queue_ready_executions`. It doesn't count job starts, finishes or the API calls a job makes.
 
-With `T = per / to`, in any window of `w` seconds, workers claim at most `burst + floor(w / T)` of a class's jobs. With `burst: 1` that's even spacing, at least `T` apart. With a larger burst it isn't an exact rolling-window quota. `limits_rate to: 100, per: 1.minute, burst: 10` allows 110 claims in a minute that follows a quiet spell.
+Workers claim at most `to` of the jobs that share a budget in any window of `per`, starting at any moment. A quiet budget admits up to `burst` of them at once, and then one every `T = per / (to - burst + 1)`, which is how fast a backlog drains. With `burst: 1` that's even spacing, at least `per / to` apart. `limits_rate to: 100, per: 1.minute, burst: 10, key: :global` allows 10 claims at once, and 100 in any minute.
+
+The limit isn't a rolling-window counter. It spaces claims so that a burst at the start of a window and the claims after it add up to `to`. So a larger burst leaves less of `to` for the rest of the window: `to: 100, burst: 100` admits 100 at once, then one a minute.
 
 Some things fall outside it:
 
 - A claim is charged when it happens, whatever happens next. A worker that crashes after claiming a job has used that budget.
 - A job that comes back to ready is charged again. That covers a `retry_on` retry, a failed job you retry, and a claim released when a worker shuts down.
 - A job can start a moment after its claim, make several API calls, or make one late in a long `perform`. Leave headroom if the API counts something else.
+- A window that spans a change to the limit can hold more than either limit allows. See [`limits_rate` options](#limits_rate-options). So can a group whose classes declare different limits. See [Rules and limitations](#rules-and-limitations).
 - Each worker reads the time from its own clock. Skew between hosts can let claims through closer together than `T`, so keep their clocks in sync, as Solid Queue's scheduled jobs already need.
 - It's not FIFO. Jobs are checked in queue order within a claim, but a waiting job can be overtaken by one that arrives after the budget recovers, or one whose slot comes sooner.
 
@@ -174,35 +319,50 @@ Some things fall outside it:
 
 A job that reaches a worker while the budget has room runs straight away, whatever the burst. A backlog is different. Its jobs come back from waiting through the dispatcher, which promotes due jobs once per `polling_interval` (1 second by default), and a worker claims them shortly after.
 
-`burst` also sets how early a job can reach the gate and still get in. Each burst above 1 allows one more interval `per / to`. With `burst: 1` there's no slack. A waiting job that arrives even a millisecond before the interval since the last claim has passed waits again, behind the rest of the backlog. Dispatch and claim delays vary by tens of milliseconds, so this happens to a good share of waiting jobs. Polling more often doesn't help, because the delays still vary.
+`burst` also sets how early a job can reach the gate and still get in. Each burst above 1 allows one more interval `T`. With `burst: 1` there's no slack. A waiting job that arrives even a millisecond before the interval since the last claim has passed waits again, behind the rest of the backlog. Dispatch and claim delays vary by tens of milliseconds, so this happens to a good share of waiting jobs. Polling more often doesn't help, because the delays still vary.
 
-For a backlog to drain at the full rate, `burst` has to cover two things. A dispatcher poll makes about `to × polling_interval / per` jobs due at once, and one claim takes them together. On top of that, the slack has to cover how much a worker's claim delay varies, which is up to the worker's own `polling_interval`. So set `burst` to at least `to × (dispatcher polling_interval + worker polling_interval) / per`, rounded up, and to at least 2. With Solid Queue's defaults of 1 and 0.1 seconds, that's `burst: 2` for 1 job a second, and `burst: 6` for 5. The cost is that a quiet budget lets that many jobs through at once.
+A backlog drains at one job every `T`, that is `to - burst + 1` per `per`. To drain at that rate, `burst` has to cover two things. A dispatcher poll makes about `polling_interval / T` jobs due at once, and one claim takes them together. On top of that, the slack has to cover how much a worker's claim delay varies, which is up to the worker's own `polling_interval`. So set `burst` to at least `(dispatcher polling_interval + worker polling_interval) / T`, rounded up, and to at least 2.
 
-This is a rule of thumb that fits the measurements below and the recipe test's simulation, not a bound. Slow claims or busy workers can need more. In the simulation, 100 jobs a second with a 0.1-second dispatcher poll fell behind with `burst: 11` and kept up with `burst: 20`.
+For a limit of `to` per `per`, with `p` for the two polling intervals added up, that's `burst` of at least `(to + 1) × p / (per + p)`, rounded up, and at least 2. With Solid Queue's defaults of 1 and 0.1 seconds, that's `burst: 2` for `to: 2, per: 1.second`, which drains at 1 a second, and `burst: 6` for `to: 10`, which drains at 5. If the result is more than `to`, `per` is too short for the polls. Make the dispatcher poll more often, or accept a backlog that drains more slowly. The cost of a larger burst is that a quiet budget lets that many jobs through at once, and the backlog drains more slowly.
+
+This is a rule of thumb that fits the measurements below and the recipe test's simulation, not a bound. Slow claims or busy workers can need more. In the simulation, a backlog draining at 100 jobs a second with a 0.1-second dispatcher poll fell behind with `to: 110, burst: 11` and kept up with `to: 119, burst: 20`.
 
 These numbers come from 30 to 60 waiting jobs, with one worker polling every 0.1 seconds, on SQLite:
 
-| Limit | Dispatcher poll | `burst` | Backlog drained at |
-| --- | --- | --- | --- |
-| 1 per second | 1 s | 1 | 0.55 per second |
-| 1 per second | 1 s | 2 | 0.95 per second |
-| 1 per second | 0.1 s | 1 | 0.53 per second |
-| 1 per second | 0.1 s | 2 | 1.0 per second |
-| 2 per second | 1 s | 2 | 1.53 per second |
-| 2 per second | 1 s | 3 | 1.9 per second |
-| 5 per second | 1 s | 5 | 4.61 per second |
-| 5 per second | 1 s | 6 | 4.91 per second |
-| 1 per 3 seconds | 1 s | 1 | 0.30 per second |
-| 1 per 3 seconds | 1 s | 2 | 0.33 per second |
+| Limit | `burst` | Dispatcher poll | Drains at, at most | Backlog drained at |
+| --- | --- | --- | --- | --- |
+| 1 per second | 1 | 1 s | 1 per second | 0.55 per second |
+| 2 per second | 2 | 1 s | 1 per second | 0.95 per second |
+| 1 per second | 1 | 0.1 s | 1 per second | 0.53 per second |
+| 2 per second | 2 | 0.1 s | 1 per second | 1.0 per second |
+| 3 per second | 2 | 1 s | 2 per second | 1.53 per second |
+| 4 per second | 3 | 1 s | 2 per second | 1.9 per second |
+| 9 per second | 5 | 1 s | 5 per second | 4.61 per second |
+| 10 per second | 6 | 1 s | 5 per second | 4.91 per second |
+| 1 per 3 seconds | 1 | 1 s | 1 per 3 seconds | 0.30 per second |
+| 2 per 3 seconds | 2 | 1 s | 1 per 3 seconds | 0.33 per second |
 
 ## Things to know
 
 - It's a separate gem that adds to Solid Queue from the outside, with its own table. It only works with the Solid Queue adapter, because it checks the limit inside Solid Queue's claim.
-- The limit counts claims, not API calls, and isn't an exact rolling-window quota once `burst` is above 1. See [What the limit guarantees](#what-the-limit-guarantees).
-- With `burst: 1`, a backlog drains slower than the rate. See [Picking a burst for a backlog](#picking-a-burst-for-a-backlog).
+- It works with [Solid Queue Coalesce](https://github.com/dtcristo/solid_queue-coalesce), which coalesces calls when they're enqueued, before the limit checks the job at claim. See [Collapse calls for each account and run them one at a time, under a global rate](#collapse-calls-for-each-account-and-run-them-one-at-a-time-under-a-global-rate).
+- The limit counts claims, not API calls. A larger `burst` leaves less of `to` for a backlog, which drains at `to - burst + 1` per `per`. See [What the limit guarantees](#what-the-limit-guarantees).
+- With `burst: 1`, a backlog drains slower than `to` per `per`. See [Picking a burst for a backlog](#picking-a-burst-for-a-backlog).
 - A waiting job is a scheduled job with its own `scheduled_at` kept, and checks again at most 5 minutes later. See [How waiting works](#how-waiting-works).
-- Each job class has its own budget, and so does each subclass. See [Rules and limitations](#rules-and-limitations).
-- When the app is eager-loaded and no class declares `limits_rate`, claims run exactly Solid Queue's queries. Otherwise each claim reads the class names of the jobs it locked, and locks one row per limited class.
+- Each job class has its own budgets, and so does each subclass, unless it declares `group:`. See [Rules and limitations](#rules-and-limitations).
+- When the app is eager-loaded and no class declares `limits_rate`, claims run exactly Solid Queue's queries. Otherwise each claim reads the class names of the jobs it locked, reads the limited jobs' keys in one more query, and locks one row per budget. When it defers jobs that hold a concurrency slot, it loads them, gives their concurrency slots back and wakes the next blocked job for each.
+
+## Clearing idle budgets
+
+A budget gets a row in `solid_queue_rate_limits` the first time a claim needs it, and the row stays. With `key:`, that's a row for every account, shop or API key your jobs have used. `SolidQueue::RateLimitState.clear_idle_in_batches` deletes the rows of budgets that are full and have no slots handed out ahead. Such a row admits exactly as a missing one does, so clearing never changes what's admitted, and a claim that runs while it clears inserts the row again. Nothing runs it for you, so add it to your `recurring.yml`, as Solid Queue's installer does for finished jobs:
+
+```yml
+clear_solid_queue_rate_limits:
+  command: "SolidQueue::RateLimitState.clear_idle_in_batches(sleep_between_batches: 0.3)"
+  schedule: every hour at minute 27
+```
+
+It takes `batch_size:`, 500 by default, and `sleep_between_batches:`, like Solid Queue's `clear_finished_in_batches`.
 
 ## Compatibility
 
@@ -210,8 +370,9 @@ These numbers come from 30 to 60 waiting jobs, with one worker polling every 0.1
 | --- | --- |
 | Ruby | 3.2 and newer |
 | Rails | 7.1, 7.2, 8.0 and 8.1. CI also runs Rails main, except on Ruby 3.2 |
-| Solid Queue | 1.7 |
+| Solid Queue | 1.7.2 and newer within 1.7 |
 | Database | MySQL, PostgreSQL and SQLite |
+| [Solid Queue Coalesce](https://github.com/dtcristo/solid_queue-coalesce) | Optional. CI tests 0.2 alongside |
 
 CI runs Ruby 3.2, 3.3, 3.4 and 4.0 against all three databases.
 
@@ -227,24 +388,50 @@ CI runs Ruby 3.2, 3.3, 3.4 and 4.0 against all three databases.
 ## `limits_rate` options
 
 ```ruby
-limits_rate to:, per:, burst: 1
+limits_rate to:, per:, key:, burst: 1, group: ->(*) { self.class.name }
+```
+
+- `to` is required, and is a positive integer: the most jobs any `per` admits.
+- `per` is required, and is a positive duration.
+- `key` is required, and identifies the jobs that share a budget, as in `limits_concurrency`.
+- `burst` is a positive integer, 1 by default, and at most `to`: how many jobs a quiet budget admits at once. After a burst, workers admit jobs `per / (to - burst + 1)` apart, which has to be at least 1 millisecond.
+- `group` shares the budget with every class that names the same group. It takes the same values as `key`. Without it, the budget belongs to the class.
+
+`key` and `group` work like the ones `limits_concurrency` takes, and the gem builds a budget's key the way Solid Queue builds a concurrency key: the group, or the class's name, then the key's value, joined with `/`.
+
+```ruby
+limits_rate to: 1, per: 1.second, key: :global                                       # Literal class-wide key: JobClass/global
+limits_rate to: 1, per: 1.second, key: "global"                                      # Same literal component
+limits_rate to: 1, per: 1.second, key: ->(account_id) { account_id }                 # Distinct per account
+limits_rate to: 1, per: 1.second, key: ->(account) { account }                       # Active Record class and ID
+limits_rate to: 1, per: 1.second, key: ->(account_id, kind) { [ account_id, kind ] } # Composite key
 ```
 
-- `to` is required, and is a positive integer: how many jobs a `per` admits.
-- `per` is required, and is a positive duration. Workers admit jobs `per / to` apart, which has to be at least 1 millisecond.
-- `burst` is a positive integer, 1 by default: how many jobs a quiet budget admits at once.
+- A Symbol or String is a literal. `key: :account_id` doesn't read a parameter called `account_id`.
+- A Proc receives the job's arguments.
+- The result is joined into a string, so `1` and `"1"` give the same key.
+- An array's items are joined with `/`, so `[ "a/b" ]` and `[ "a", "b" ]` give the same key.
+- A Proc that returns `nil` gives the bare class-wide key, `JobClass`, or the group's, so all of those jobs share a budget.
+- A declaration of `nil` or `false` disables the limit, as it disables `limits_concurrency`, so `key: nil` isn't a class-wide key. For one key per class, use a literal such as `:global`, or a Proc that returns `nil`. Other unsupported declaration types, such as `key: 123`, give the class-wide key.
+- A key that comes out blank, such as with `group: nil` and a Proc that returns `nil`, leaves the job unlimited, as a blank concurrency key does.
+
+Except for `key:`, an invalid option raises `ArgumentError` when the class loads. The class gets `rate_limit_to`, `rate_limit_per`, `rate_limit_burst`, `rate_limit_key`, `rate_limit_group` and `rate_limited?` to read them back.
 
-An invalid option raises `ArgumentError` when the class loads. The class gets `rate_limit_to`, `rate_limit_per`, `rate_limit_burst` and `rate_limited?` to read them back.
+The key is computed once, when Solid Queue enqueues the job, and stored with it, so a claim never deserializes arguments or loads records. A key Proc that raises fails the enqueue. `perform_now` and other adapters never run it.
 
-Changing a limit and deploying applies to the next claim. A looser limit doesn't pay off all the debt the old one built up. The claim caps it at `burst` intervals of the new limit, rounded up to the microsecond, so new admissions start within one new interval and a microsecond. Jobs already waiting keep the next checks the old limit gave them, and new waiting jobs get slots after those, so a backlog can take up to 5 minutes to speed up. A tighter limit starts from the budget as the old one left it.
+Changing a limit and deploying applies to the next claim. A looser limit doesn't pay off all the debt the old one built up. The claim caps it at `burst` intervals of the new limit, rounded up to the microsecond, so new admissions start within one new interval and a microsecond. Jobs already waiting keep the next checks the old limit gave them, and new waiting jobs get slots after those, so a backlog can take up to 5 minutes to speed up. A tighter limit starts from the budget as the old one left it, so a window that spans the deploy can hold more than the new limit allows, until one new `per` has passed.
 
 ## Rules and limitations
 
-- A class has one limit, and its budget belongs to that class. A second `limits_rate` in the same class raises `ArgumentError`.
-- A subclass inherits its parent's limit, with a budget of its own. Declaring `limits_rate` on `ApplicationJob` gives each job class its own budget, not one shared between them. A subclass can redeclare the limit.
-- `key:`, `group:` and `name:`, for keyed, shared and multiple limits, raise `ArgumentError` until [#3](https://github.com/dtcristo/solid_queue-rate_limit/issues/3).
-- `limits_rate` and `limits_concurrency` on the same class raise `ArgumentError` in either order, and through inheritance, until [#2](https://github.com/dtcristo/solid_queue-rate_limit/issues/2). A job that still has a concurrency key, after its class switched from one to the other, is admitted without a rate check, because it holds its semaphore while it waits.
-- The budget is stored under the class's name. A job enqueued under an alias, such as a constant kept after a rename, shares the class's budget. On MySQL, whose default collation ignores case, two classes whose names differ only in case share one budget.
+- A class has one limit. A second `limits_rate` in the same class raises `ArgumentError`. `name:`, for multiple limits, raises `ArgumentError` until [#14](https://github.com/dtcristo/solid_queue-rate_limit/issues/14).
+- A subclass inherits its parent's limit, `key:` and `group:`. Without a group, it has budgets of its own, so declaring `limits_rate` on `ApplicationJob` gives each job class its own budget, not one shared between them. A subclass can redeclare the limit, and then has only the `key:` and `group:` it declares. Redeclaring with `key: nil` turns the limit off.
+- Every class in a group has to declare the same `to:`, `per:` and `burst:`. The gem doesn't check, as Solid Queue doesn't for concurrency groups. Each job is charged with its own class's settings against the shared budget, so with mixed settings a group can admit more than the stricter one allows.
+- Keys aren't escaped, as concurrency keys aren't. A group named after a job class shares that class's budget, and `group: "Stripe", key: "a/b"` shares one with `group: "Stripe/a", key: "b"`. On MySQL, whose default collation ignores case, keys that differ only in case share one budget.
+- A key longer than `solid_queue_rate_limits.key` allows, 255 characters on MySQL, raises `ActiveRecord::ValueTooLong` at enqueue, as an over-long concurrency key does. `perform_later` raises it wrapped in `SolidQueue::Job::EnqueueError`, and `perform_all_later` raises it as it is. PostgreSQL and SQLite have no limit.
+- A retry keeps the key its job was enqueued with. That covers a `retry_on` retry, a failed job you retry, and a retry after the arguments failed to deserialize.
+- A job keeps the key it was enqueued with while it waits, even if its class changes `key:` or `group:`. A job enqueued without a key, before its class declared a limit or by 0.1.0, uses the bare class-wide budget, `JobClass`, and gets the current key if it's retried.
+- `limits_rate` combines with `limits_concurrency`, in either order and through inheritance. A job over the rate gives its concurrency slot back and wakes the next blocked job for its key once the claim commits, and takes one again when the dispatcher brings it back, or is blocked. `limits_concurrency` with `on_conflict: :discard` raises `ArgumentError` with `limits_rate`, because a job that came back to a taken concurrency slot would be discarded. A job that still has a concurrency key after its class dropped `limits_concurrency` is checked like any other.
+- Without `group:`, the budget's key starts with the class's name. A job enqueued under an alias, such as a constant kept after a rename, keeps the key it was enqueued with, and shares the class's budget. On MySQL, two classes whose names differ only in case share one budget.
 - A job whose class doesn't load, or whose `class_name` isn't a job class, is claimed without a check and fails as it would without the gem.
 - A class that drops its limit has its waiting jobs claimed when they next check, at most 5 minutes later.
 
@@ -254,14 +441,14 @@ The design, and the costs behind these rules, are in [the design notes](https://
 
 | Event | When | Payload |
 | --- | --- | --- |
-| `defer_rate_limited.solid_queue` | A claim sent a class's jobs to wait | `class_name`, `job_ids`, `size` |
+| `defer_rate_limited.solid_queue` | A claim sent a class's jobs to wait | `class_name`, `keys`, `job_ids`, `size` |
 
-A claim instruments one event for each class it deferred, inside its transaction. If the claim then rolls back, the event has still fired. A job can wait more than once, so `size` counts deferrals, not distinct jobs.
+A claim instruments one event for each class it deferred, inside its transaction. If the claim then rolls back, the event has still fired. `keys` lists the budgets that deferred them, once each. A job can wait more than once, so `size` counts deferrals, not distinct jobs. Jobs of a class that also declares `limits_concurrency` are included, and each blocked job one of them wakes instruments Solid Queue's own `release_blocked.solid_queue`.
 
 `SolidQueue::LogSubscriber` logs it at debug level:
 
 ```text
-SolidQueue-1.7.2 Defer rate-limited jobs (0.1ms)  class_name: "SyncStripeJob", job_ids: [1, 2], size: 2
+SolidQueue-1.7.2 Defer rate-limited jobs (0.1ms)  class_name: "SyncStripeJob", keys: ["SyncStripeJob"], job_ids: [1, 2], size: 2
 ```
 
 A backlog can defer jobs on every claim, so the line isn't built at all above debug.
@@ -280,7 +467,7 @@ Tests that run jobs through Solid Queue need the table in the test queue databas
 
 ## Roadmap
 
-[#10](https://github.com/dtcristo/solid_queue-rate_limit/issues/10) tracks what comes after 0.1.0, including keyed and shared limits, combining with `limits_concurrency`, and feedback from an API's 429 responses.
+[#10](https://github.com/dtcristo/solid_queue-rate_limit/issues/10) tracks what's planned, including multiple limits per job and feedback from an API's 429 responses.
 
 ## License
 

data/UPGRADING.md

@@ -0,0 +1,18 @@
+# Upgrading from version 0.1.0 to 0.2.0
+
+There's no migration. Update Solid Queue and every `limits_rate` declaration, then deploy.
+
+1. Update Solid Queue to 1.7.2 or newer within 1.7. The gemspec requires it.
+2. Add `key:` to every `limits_rate`. A declaration without it raises `ArgumentError` when its class loads.
+3. If a declaration sets `burst` above 1, add `burst - 1` to `to` to keep the limit 0.1.0 enforced. `to:` is now the most a budget admits in any `per`, burst included. A declaration with the default `burst: 1` needs no change.
+
+| 0.1.0 | 0.2.0, same limit and budget |
+| --- | --- |
+| `limits_rate to: 10, per: 1.minute` | `limits_rate to: 10, per: 1.minute, key: ->(*) {}` |
+| `limits_rate to: 100, per: 1.minute, burst: 10` | `limits_rate to: 109, per: 1.minute, burst: 10, key: ->(*) {}` |
+
+`key: ->(*) {}` keeps 0.1.0's budget, stored as the class name, so jobs enqueued by 0.1.0 share it with new ones. `key: :global` reads better, but it's a new budget, stored as `JobClass/global`. Jobs enqueued by 0.1.0 drain on the old budget while new ones use the new one, so the limit can admit up to twice its rate until they're gone. To avoid that, drain those jobs before you switch, or keep `key: ->(*) {}`.
+
+A key for each value, such as `key: ->(account) { account }`, gives each value a budget of its own. Jobs enqueued by 0.1.0 have no key and drain on the old class-wide budget, which doesn't count against the new ones.
+
+Deploy every worker together. A 0.1.0 worker checks every job against its class's old budget, whatever key the job stores, so while both versions run, a job with a new key can be admitted on both budgets.

data/app/models/solid_queue/rate_limit_state.rb

@@ -19,12 +19,25 @@ module SolidQueue
       end
     end
 
+    # A full budget with no slots ahead admits as a missing row does, so it can go
+    scope :idle, ->(now = Time.current) { where(tat: ...now, deferred_until: [ nil, ...now ]) }
+
     class << self
       # Until the app has run the gem's migration everything else works as before.
       def migrated?
         @migrated ||= table_exists?
       end
 
+      # The delete checks each row again, because a claim can charge it once it's been
+      # picked. By key, so on MySQL it locks rows in the order claims do.
+      def clear_idle_in_batches(batch_size: 500, sleep_between_batches: 0)
+        loop do
+          records_deleted = idle.where(key: idle.limit(batch_size).pluck(:key)).delete_all
+          sleep(sleep_between_batches) if sleep_between_batches > 0
+          break if records_deleted == 0
+        end
+      end
+
       # Returns the state of each key, by key, locked until the claim commits. A new
       # key starts with a full budget. They're inserted and locked in key order, so
       # claims that share keys can't deadlock on them.
@@ -32,16 +45,27 @@ module SolidQueue
         keys = keys.sort
 
         # On PostgreSQL even a skipped insert takes a value from the id sequence
-        if (missing = keys - where(key: keys).pluck(:key)).any?
-          insert_all(missing.map { |key| { key: key, tat: Time.current } })
-        end
-
+        insert_missing(keys - where(key: keys).pluck(:key))
         states = where(key: keys).order(:key).lock.index_by(&:key)
 
-        # Under a case-insensitive collation, like MySQL's default, a key can match
-        # a row spelled differently, and then the two share a budget
-        keys.index_with { |key| states[key] || states.fetch(where(key: key).pick(:key)) }
+        keys.index_with { |key| states[key] || lock_one(key, states) }
       end
+
+      private
+        def insert_missing(keys)
+          insert_all(keys.map { |key| { key: key, tat: Time.current } }) if keys.any?
+        end
+
+        # Under a case-insensitive collation, like MySQL's default, a key can match
+        # a row spelled differently, and then the two share a budget. A row the cleanup
+        # deleted since the check is inserted again, out of key order.
+        def lock_one(key, states)
+          until (state = where(key: key).lock.first)
+            insert_missing([ key ])
+          end
+
+          states[state.key] ||= state
+        end
     end
 
     # Charges one admission and returns true, or returns false if the budget is spent.

data/app/models/solid_queue/ready_execution/rate_limited.rb

@@ -8,38 +8,60 @@ module SolidQueue
       class_methods do
         # Moves the candidates whose class is over its rate to scheduled executions,
         # leaving their jobs' own scheduled_at as it is, and returns them. Called inside
-        # the claim's transaction, after the candidates have been locked.
+        # the claim's transaction, after the candidates have been locked. The deferred
+        # jobs that hold a semaphore are kept for release_concurrency_locks.
         def defer_rate_limited(executions)
           jobs = Job.where(id: executions.map(&:job_id)).select(:id, :class_name, :queue_name, :priority, :concurrency_key).index_by(&:id)
           job_classes = jobs.values.map(&:class_name).uniq.index_with { |class_name| rate_limited_job_class(class_name) }.compact
 
-          # A job with a concurrency key holds its semaphore, so it can't wait here
-          limited = executions.select { |execution| (job = jobs[execution.job_id]) && job.concurrency_key.nil? && job_classes[job.class_name] }
-          return [] if limited.empty?
+          limited = executions.select { |execution| (job = jobs[execution.job_id]) && job_classes[job.class_name] }
+          keys = limited.any? ? rate_limit_keys(limited, jobs, job_classes) : {}
+          return [] if keys.empty?
 
           next_checks = if RateLimitState.migrated?
-            check_rate_limits(limited, jobs, job_classes)
+            check_rate_limits(keys, jobs, job_classes)
           else
             log_pending_rate_limit_migrations
-            limited.to_h { |execution| [ execution, 1.minute.from_now ] }
+            keys.transform_values { 1.minute.from_now }
           end
 
-          schedule_rate_limited(next_checks, jobs)
+          schedule_rate_limited(next_checks, jobs, keys)
+          hold_concurrency_locks(next_checks.keys.map(&:job_id).select { |job_id| jobs[job_id].concurrency_key })
           next_checks.keys
         end
 
+        # A deferred job gives its semaphore back while it waits, so the semaphore can't expire
+        # and let a second job with its key run. It takes one again when it's dispatched.
+        #
+        # Called once the claim has committed, as Solid Queue unblocks a finished job's next
+        # one after its own transaction. Inside the claim, the release and the wake would lock
+        # semaphores and insert ready executions while holding ready executions and rate
+        # limits, and on MySQL that deadlocks with other claims and the dispatcher.
+        #
+        # The claim has committed, so an error is logged rather than raised, and the claim
+        # returns what it claimed.
+        def release_concurrency_locks
+          ActiveSupport::IsolatedExecutionState.delete(:solid_queue_rate_limit_releases)&.each do |job|
+            job.unblock_next_blocked_job
+          rescue StandardError, ScriptError => error
+            raise if error.is_a?(ActiveRecord::ActiveRecordError) && connection.transaction_open?
+
+            SolidQueue.logger&.error("Couldn't release the semaphore of job #{job.id}, or the next job blocked on #{job.concurrency_key}: #{error.class}: #{error.message}")
+          end
+        end
+
         private
           # Locks every limit the candidates need, in key order, then charges them in queue
-          # order. Returns when each rejected candidate should look again. The time is
-          # read once the locks are held, so claims that share a limit see it move on.
-          # A limit's key is its class's name, which a job enqueued under an alias shares.
-          def check_rate_limits(executions, jobs, job_classes)
-            states = RateLimitState.lock_all(executions.map { |execution| job_classes[jobs[execution.job_id].class_name].name }.uniq)
+          # order, each with its own class's settings. Returns when each rejected candidate
+          # should look again. The time is read once the locks are held, so claims that
+          # share a limit see it move on.
+          def check_rate_limits(keys, jobs, job_classes)
+            states = RateLimitState.lock_all(keys.values.uniq)
             now = Time.current
 
-            next_checks = executions.each_with_object({}) do |execution, deferred|
+            next_checks = keys.keys.each_with_object({}) do |execution, deferred|
               job_class = job_classes[jobs[execution.job_id].class_name]
-              state = states[job_class.name]
+              state = states[keys[execution]]
 
               unless state.admit(job_class.rate_limit_interval, job_class.rate_limit_burst, now)
                 deferred[execution] = state.defer(job_class.rate_limit_interval, job_class.rate_limit_burst, now)
@@ -50,18 +72,37 @@ module SolidQueue
             next_checks
           end
 
-          def schedule_rate_limited(next_checks, jobs)
+          # Returns each candidate's key, stored with its job at enqueue. A job without one,
+          # enqueued before its class declared a limit or by 0.1.0, uses the class's name. A
+          # blank key isn't limited, as a blank concurrency key isn't.
+          def rate_limit_keys(executions, jobs, job_classes)
+            stored = Job.where(id: executions.map(&:job_id)).pluck(:id, :arguments).to_h { |id, arguments| [ id, arguments[ActiveJob::RateControls::RATE_LIMIT_KEY] ] }
+
+            executions.index_with { |execution| stored[execution.job_id] || job_classes[jobs[execution.job_id].class_name].name }.reject { |_, key| key.empty? }
+          end
+
+          def schedule_rate_limited(next_checks, jobs, keys)
             if next_checks.any?
               ScheduledExecution.insert_all(next_checks.map do |execution, scheduled_at|
                 jobs[execution.job_id].slice(:queue_name, :priority).merge(job_id: execution.job_id, scheduled_at: scheduled_at)
               end)
 
               next_checks.keys.group_by { |execution| jobs[execution.job_id].class_name }.each do |class_name, deferred|
-                SolidQueue.instrument(:defer_rate_limited, class_name: class_name, job_ids: deferred.map(&:job_id), size: deferred.size)
+                SolidQueue.instrument(:defer_rate_limited, class_name: class_name, keys: keys.values_at(*deferred).uniq, job_ids: deferred.map(&:job_id), size: deferred.size)
               end
             end
           end
 
+          # Loaded inside the claim, so each job keeps the semaphore it held when it was deferred,
+          # even if the dispatcher brings it back and it takes another before the release. Going
+          # through them by concurrency key, as dispatch_all_one_by_one does, makes claims that
+          # release the same semaphores take them in the same order.
+          def hold_concurrency_locks(job_ids)
+            if job_ids.any?
+              ActiveSupport::IsolatedExecutionState[:solid_queue_rate_limit_releases] = Job.where(id: job_ids).sort_by { |job| [ job.concurrency_key, job.id ] }
+            end
+          end
+
           # A class that can't be loaded fails its jobs when they run, as without the gem
           def rate_limited_job_class(class_name)
             job_class = class_name.constantize

data/lib/active_job/rate_controls.rb

@@ -5,35 +5,53 @@ module ActiveJob
   module RateControls
     extend ActiveSupport::Concern
 
+    DEFAULT_RATE_LIMIT_GROUP = ->(*) { self.class.name }
     MIN_RATE_LIMIT_INTERVAL = 0.001
 
+    # Stored in the job's serialized Active Job data, which Active Job ignores when running the job
+    RATE_LIMIT_KEY = "solid_queue_rate_limit_key"
+
     included do
       class_attribute :rate_limit_to, instance_accessor: false
       class_attribute :rate_limit_per, instance_accessor: false
       class_attribute :rate_limit_burst, instance_accessor: false
+      class_attribute :rate_limit_key, instance_accessor: false
+      class_attribute :rate_limit_group, default: DEFAULT_RATE_LIMIT_GROUP, instance_accessor: false
     end
 
     module ClassMethods
-      # Admits this class's jobs at worker claim no faster than +to+ per +per+,
-      # using GCRA. A job over the limit waits as a scheduled job, without taking
-      # a worker thread or a retry, and is checked again when it's dispatched.
+      # Admits at most +to+ of this class's jobs at worker claim in any +per+, using
+      # GCRA. A job over the limit waits as a scheduled job, without taking a worker
+      # thread or a retry, and is checked again when it's dispatched.
       #
       #   class SyncStripeJob < ApplicationJob
-      #     limits_rate to: 100, per: 1.minute, burst: 10
+      #     limits_rate to: 100, per: 1.minute, burst: 10, key: :global
       #   end
       #
       # ==== Options
       #
-      # * +:to+ - Required. A positive integer, how many jobs a +per+ admits.
-      # * +:per+ - Required. A positive duration. Admissions are spaced <tt>per / to</tt>
+      # * +:to+ - Required. A positive integer, the most jobs any +per+ admits.
+      # * +:per+ - Required. A positive duration.
+      # * +:key+ - Required. A String or Symbol literal, or a Proc that receives the job's
+      #   arguments, as in +limits_concurrency+. Each value gets a budget of its own. A
+      #   +nil+ or +false+ key turns the limit off, as it turns off +limits_concurrency+.
+      # * +:burst+ - How many jobs a fully recovered budget admits at once, up to +to+.
+      #   Defaults to 1. After a burst, admissions are spaced <tt>per / (to - burst + 1)</tt>
       #   apart, which has to be at least 1 ms.
-      # * +:burst+ - How many jobs a fully recovered budget admits at once. Defaults to 1.
+      # * +:group+ - Shares the budget with every class that names the same group. Takes
+      #   the same values as +:key+. Defaults to the class's name.
+      #
+      #   class SyncStripeCustomerJob < ApplicationJob
+      #     limits_rate to: 100, per: 1.minute, group: "Stripe", key: ->(account) { account }
+      #   end
       #
-      # A subclass has its own budget, and can redeclare the limit. An invalid option
-      # raises +ArgumentError+. +perform_now+ and other adapters don't enforce it.
-      def limits_rate(to:, per:, burst: 1, **options)
-        if (planned = options.keys & %i[ key group name ]).any?
-          raise ArgumentError, "`#{planned.first}` isn't supported yet. Follow https://github.com/dtcristo/solid_queue-rate_limit/issues/3 for shared, keyed and multiple limits"
+      # A subclass inherits the limit, key and group. Without a group it has its own
+      # budget, and it can redeclare the limit. Except for +:key+, an invalid option raises
+      # +ArgumentError+.
+      # +perform_now+ and other adapters don't enforce it, and never compute the key.
+      def limits_rate(to:, per:, key:, burst: 1, group: DEFAULT_RATE_LIMIT_GROUP, **options)
+        if options.key?(:name)
+          raise ArgumentError, "`name` isn't supported yet. Follow https://github.com/dtcristo/solid_queue-rate_limit/issues/14 for multiple limits"
         elsif options.any?
           raise ArgumentError, "unknown keyword#{"s" if options.many?}: #{options.keys.map(&:inspect).join(", ")}"
         end
@@ -50,42 +68,97 @@ module ActiveJob
           raise ArgumentError, "`burst` must be a positive integer"
         end
 
-        if per.to_r / to < MIN_RATE_LIMIT_INTERVAL
-          raise ArgumentError, "`limits_rate to: #{to}, per: #{per.inspect}` would admit jobs less than 1 ms apart, and they have to be at least 1 ms apart"
+        if burst > to
+          raise ArgumentError, "`burst` can't be more than `to`"
+        end
+
+        if rate_limit_interval_for(to, per, burst) < MIN_RATE_LIMIT_INTERVAL
+          raise ArgumentError, "`limits_rate to: #{to}, per: #{per.inspect}, burst: #{burst}` would admit jobs less than 1 ms apart, and they have to be at least 1 ms apart"
         end
 
         if @rate_limit_declared
           raise ArgumentError, "#{name || "This class"} already declares `limits_rate`"
         end
 
-        ensure_rate_limit_compatible_with_concurrency_controls(rate_limited: true)
+        ensure_rate_limit_compatible_with_concurrency_controls(rate_limited: key)
 
         @rate_limit_declared = SolidQueue::RateLimit.declared = true
         self.rate_limit_to = to
         self.rate_limit_per = per
         self.rate_limit_burst = burst
+        self.rate_limit_key = key
+        self.rate_limit_group = group
       end
 
       def rate_limited?
-        rate_limit_to.present?
+        rate_limit_key ? true : false
       end
 
-      # The seconds between admissions
+      # The seconds between admissions after a burst, so that any +per+ admits at most +to+
       def rate_limit_interval # :nodoc:
-        rate_limit_per.to_r / rate_limit_to
+        rate_limit_interval_for(rate_limit_to, rate_limit_per, rate_limit_burst)
       end
 
-      def limits_concurrency(**) # :nodoc:
-        ensure_rate_limit_compatible_with_concurrency_controls(concurrency_limited: true)
+      def limits_concurrency(on_conflict: :block, **) # :nodoc:
+        ensure_rate_limit_compatible_with_concurrency_controls(discards_on_conflict: on_conflict == :discard)
         super
       end
 
       private
-        def ensure_rate_limit_compatible_with_concurrency_controls(rate_limited: rate_limited?, concurrency_limited: concurrency_key.present?)
-          if rate_limited && concurrency_limited
-            raise ArgumentError, "`limits_rate` can't be combined with `limits_concurrency` yet. Follow https://github.com/dtcristo/solid_queue-rate_limit/issues/2"
+        def rate_limit_interval_for(to, per, burst)
+          per.to_r / (to - burst + 1)
+        end
+
+        # A deferred job that comes back to a taken slot would be discarded, dropping work already accepted
+        def ensure_rate_limit_compatible_with_concurrency_controls(rate_limited: rate_limited?, discards_on_conflict: concurrency_key.present? && concurrency_on_conflict == :discard)
+          if rate_limited && discards_on_conflict
+            raise ArgumentError, "`limits_rate` can't be combined with `limits_concurrency(on_conflict: :discard)`"
           end
         end
     end
+
+    # The key is computed once, when Solid Queue enqueues the job, and stored with it.
+    # A claim reads it without deserializing arguments, and a retry keeps it. Every
+    # path into Solid Queue serializes the job before writing it, including ones other
+    # gems add, so this is where a limited job finds out the schema is missing.
+    def serialize # :nodoc:
+      if self.class.rate_limited? && self.class.queue_adapter_name == "solid_queue"
+        raise SolidQueue::RateLimitState::PendingMigrations unless SolidQueue::RateLimitState.migrated?
+
+        @rate_limit_key ||= rate_limit_key.tap { |key| ensure_rate_limit_key_fits(key) }
+      end
+
+      @rate_limit_key ? super.merge(RATE_LIMIT_KEY => @rate_limit_key) : super
+    end
+
+    def deserialize(job_data) # :nodoc:
+      super
+      @rate_limit_key = job_data[RATE_LIMIT_KEY]
+    end
+
+    def rate_limit_key # :nodoc:
+      if self.class.rate_limited?
+        param = compute_concurrency_parameter(self.class.rate_limit_key)
+
+        case param
+        when ActiveRecord::Base
+          [ rate_limit_group, param.class.name, param.id ]
+        else
+          [ rate_limit_group, param ]
+        end.compact.join("/")
+      end
+    end
+
+    private
+      # Raised here rather than at claim, where it would fail every claim that took the job
+      def ensure_rate_limit_key_fits(key)
+        if (limit = SolidQueue::RateLimitState.columns_hash["key"].limit) && key.length > limit
+          raise ActiveRecord::ValueTooLong, "#{self.class.name}'s rate limit key is #{key.length} characters long, and solid_queue_rate_limits.key allows #{limit}"
+        end
+      end
+
+      def rate_limit_group
+        compute_concurrency_parameter(self.class.rate_limit_group)
+      end
   end
 end

data/lib/solid_queue/rate_limit/patches.rb

@@ -10,22 +10,34 @@ module SolidQueue
       #
       # A pass that deferred jobs leaves threads idle that could run jobs behind them,
       # so the claim looks again, each time in a transaction of its own.
+      #
+      # Claims that insert a new budget's row at once can deadlock on MySQL, and a
+      # deadlock in the first pass would stop the worker, so that pass runs once more.
       module Claiming
         MAX_CLAIM_PASSES = 5
 
         private
           def select_and_lock(queue_relation, process_id, limit)
-            return super unless RateLimit.enforced? && !connection_pool.with_connection(&:transaction_open?)
+            return super unless RateLimit.enforced?
+            return releasing_concurrency_locks { super } if connection_pool.with_connection(&:transaction_open?)
 
             claimed = []
+            retried = false
 
             MAX_CLAIM_PASSES.times do |pass|
-              claimed += super(queue_relation, process_id, limit - claimed.size)
+              begin
+                # Before looking again, so the jobs it wakes can be claimed
+                claimed += releasing_concurrency_locks { super(queue_relation, process_id, limit - claimed.size) }
+              rescue ActiveRecord::Deadlocked
+                # Whatever earlier passes claimed has been committed
+                break if pass.positive?
+                raise if retried
+
+                retried = true
+                retry
+              end
+
               break unless claimed.size < limit && rate_limit_deferred_in_pass?
-            rescue ActiveRecord::Deadlocked
-              # Whatever earlier passes claimed has been committed
-              raise if pass.zero?
-              break
             end
 
             claimed
@@ -45,35 +57,44 @@ module SolidQueue
             end
           end
 
+          # Gives back the semaphores of the jobs a pass deferred once it has committed. A pass
+          # that fails has rolled them back, so they're forgotten rather than left for the next.
+          def releasing_concurrency_locks
+            yield.tap { release_concurrency_locks }
+          ensure
+            ActiveSupport::IsolatedExecutionState.delete(:solid_queue_rate_limit_releases)
+          end
+
           def rate_limit_deferred_in_pass?
             ActiveSupport::IsolatedExecutionState.delete(:solid_queue_rate_limit_deferred)
           end
       end
 
-      # Prepended rather than added to the adapter, because RecurringTask calls
-      # Job.enqueue directly. A limited job can't be admitted without the schema,
-      # so it's better to find out when it's enqueued.
-      module Enqueuing
-        def enqueue_all(active_jobs)
-          ensure_rate_limitable if active_jobs.any? { |active_job| active_job.class.rate_limited? }
-          super
-        end
+      # A claim that defers jobs inserts scheduled executions while it holds ready ones,
+      # and the dispatcher inserts ready executions while it holds scheduled ones, so on
+      # MySQL the two can deadlock. The dispatcher doesn't rescue errors, and would stop,
+      # so a deadlocked batch runs once more.
+      module Dispatching
+        def dispatch_next_batch(batch_size)
+          return super unless RateLimit.enforced? && !connection_pool.with_connection(&:transaction_open?)
 
-        def enqueue(active_job, **)
-          ensure_rate_limitable if active_job.class.rate_limited?
-          super
-        end
+          retried = false
 
-        private
-          def ensure_rate_limitable
-            raise RateLimitState::PendingMigrations unless RateLimitState.migrated?
+          begin
+            super
+          rescue ActiveRecord::Deadlocked
+            raise if retried
+
+            retried = true
+            retry
           end
+        end
       end
 
       def self.apply
-        SolidQueue::Job.singleton_class.prepend Enqueuing
         SolidQueue::ReadyExecution.include SolidQueue::ReadyExecution::RateLimited
         SolidQueue::ReadyExecution.singleton_class.prepend Claiming
+        SolidQueue::ScheduledExecution.singleton_class.prepend Dispatching
       end
     end
   end
@@ -84,6 +105,6 @@ class SolidQueue::LogSubscriber
   subscribe_log_level :defer_rate_limited, :debug
 
   def defer_rate_limited(event) # :nodoc:
-    debug formatted_event(event, action: "Defer rate-limited jobs", **event.payload.slice(:class_name, :job_ids, :size))
+    debug formatted_event(event, action: "Defer rate-limited jobs", **event.payload.slice(:class_name, :keys, :job_ids, :size))
   end
 end

data/lib/solid_queue/rate_limit/version.rb

@@ -1,5 +1,5 @@
 module SolidQueue
   module RateLimit
-    VERSION = "0.1.0"
+    VERSION = "0.2.0"
   end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: solid_queue-rate_limit
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.2.0
 platform: ruby
 authors:
 - David Cristofaro
@@ -13,16 +13,22 @@ dependencies:
   name: solid_queue
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 1.7.2
+    - - "<"
       - !ruby/object:Gem::Version
-        version: 1.7.0
+        version: '1.8'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
-        version: 1.7.0
+        version: 1.7.2
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '1.8'
 - !ruby/object:Gem::Dependency
   name: appraisal
   requirement: !ruby/object:Gem::Requirement
@@ -135,6 +141,20 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
+- !ruby/object:Gem::Dependency
+  name: solid_queue-coalesce
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.2.0
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.2.0
 - !ruby/object:Gem::Dependency
   name: rdoc
   requirement: !ruby/object:Gem::Requirement
@@ -175,6 +195,7 @@ files:
 - CHANGELOG.md
 - MIT-LICENSE
 - README.md
+- UPGRADING.md
 - app/models/solid_queue/rate_limit_state.rb
 - app/models/solid_queue/ready_execution/rate_limited.rb
 - lib/active_job/rate_controls.rb
@@ -206,7 +227,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 4.0.20
+rubygems_version: 4.0.18
 specification_version: 4
 summary: Rate limits for Solid Queue jobs.
 test_files: []