idempotency_lock 0.1.3 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +75 -0
- data/README.md +174 -26
- data/lib/generators/idempotency_lock/install_generator.rb +2 -9
- data/lib/generators/idempotency_lock/migration_generator_helpers.rb +20 -0
- data/lib/generators/idempotency_lock/templates/add_owner_token_to_idempotency_locks.rb.erb +9 -0
- data/lib/generators/idempotency_lock/templates/create_idempotency_locks.rb.erb +8 -4
- data/lib/generators/idempotency_lock/upgrade_generator.rb +23 -0
- data/lib/idempotency_lock/arguments.rb +59 -0
- data/lib/idempotency_lock/lock.rb +60 -38
- data/lib/idempotency_lock/railtie.rb +6 -6
- data/lib/idempotency_lock/result.rb +6 -2
- data/lib/idempotency_lock/version.rb +1 -1
- data/lib/idempotency_lock.rb +153 -60
- data/sig/idempotency_lock.rbs +53 -1
- metadata +13 -8
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: '0952806b286b015b244bbf5b11a3f1382007d202ba6195b16b0df7d7126fc8e4'
|
|
4
|
+
data.tar.gz: 67fe502a6e14b2c0e54090d2127e2e823b466dda34be01e0275826e9b8365414
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: cba6fda111af891eaad41c400060d7b9f613ac7e3b2d0a3c685a708294986e79f6617b71d6542598fc0c20a79ab7c82907acb1b614a010dfa851448c26f59879
|
|
7
|
+
data.tar.gz: 4ac42ca1a5a69e0856345eae7cb4cc8e7b06a15f69d48fe573db69e0616de3c4691ad2f8818a48300275a617185a76add5e8dbbb130640588dce3ca39c3bcedd
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,80 @@
|
|
|
1
1
|
## [Unreleased]
|
|
2
2
|
|
|
3
|
+
## [0.3.0] - 2026-11-07
|
|
4
|
+
|
|
5
|
+
### Upgrading
|
|
6
|
+
|
|
7
|
+
- **Run the upgrade migration before deploying:** `rails generate idempotency_lock:upgrade && rails db:migrate`
|
|
8
|
+
(adds the `owner_token` column).
|
|
9
|
+
- Minimum supported ActiveRecord / ActiveSupport is now **7.1** (7.0 is end-of-life).
|
|
10
|
+
- Invalid arguments now raise `ArgumentError` instead of being silently accepted
|
|
11
|
+
(see "Fixed" below).
|
|
12
|
+
|
|
13
|
+
### Fixed
|
|
14
|
+
|
|
15
|
+
- **A process could release a lock it no longer held.** If a lock's TTL lapsed
|
|
16
|
+
mid-block and another process claimed it, the original holder's release
|
|
17
|
+
(`temporarily`'s auto-release, or `once` with `:unlock` / `:raise` / a callable)
|
|
18
|
+
deleted the new holder's lock, allowing concurrent execution. Locks now carry
|
|
19
|
+
a random owner token and releases only delete the caller's own lock.
|
|
20
|
+
- Invalid lock names (`nil`, blank, non-String/Symbol, or over 255 characters)
|
|
21
|
+
were reported as `skipped?` without running the block. They now raise
|
|
22
|
+
`ArgumentError`.
|
|
23
|
+
- An unknown `on_error` value (e.g. a typo like `:unlok`) silently kept the lock.
|
|
24
|
+
It now raises `ArgumentError`. Invalid `ttl:` and `wait:` values also raise.
|
|
25
|
+
- Non-`StandardError` exceptions (`Interrupt`, `SignalException`,
|
|
26
|
+
`Sidekiq::Shutdown`, ...) bypassed `on_error`, leaving the lock held even with
|
|
27
|
+
`:unlock`. The lock is now released for every strategy except `:keep`.
|
|
28
|
+
- A lock conflict that reached the database (two callers racing on a new name)
|
|
29
|
+
inside a caller's transaction aborted that transaction on PostgreSQL
|
|
30
|
+
(`PG::InFailedSqlTransaction`). Acquisition now runs in a savepoint.
|
|
31
|
+
- `IdempotencyLock.logger = nil` was ignored in Rails apps (it fell back to
|
|
32
|
+
`Rails.logger`), and the lookup crashed when `Rails` was defined without a
|
|
33
|
+
`logger` method.
|
|
34
|
+
- Fractional `ttl:` values were truncated to whole seconds.
|
|
35
|
+
|
|
36
|
+
### Changed
|
|
37
|
+
|
|
38
|
+
- Requiring the gem no longer loads `ActiveRecord::Base` eagerly; the model is
|
|
39
|
+
autoloaded on first use, so host-app ActiveRecord configuration is respected.
|
|
40
|
+
The gem now requires `active_record` / `active_support` itself.
|
|
41
|
+
- Expired-lock claims use the owner token (instead of `updated_at`) as the
|
|
42
|
+
optimistic-lock version, avoiding timestamp-precision edge cases.
|
|
43
|
+
- `Lock.acquire` returns the owner token (or `nil`) instead of `true`/`false`;
|
|
44
|
+
`Lock.release` accepts an optional `owner_token:`. Truthiness is unchanged.
|
|
45
|
+
- Dropped the `uniqueness` validation on `Lock#name`; the unique index already
|
|
46
|
+
enforces it, saving a `SELECT` per acquisition.
|
|
47
|
+
- `Result#inspect` shows the class of the value rather than its contents, to
|
|
48
|
+
avoid leaking sensitive data into logs and error trackers.
|
|
49
|
+
- `temporarily` wait loop uses a monotonic clock.
|
|
50
|
+
- The Railtie no longer overwrites a configured logger.
|
|
51
|
+
|
|
52
|
+
### Added
|
|
53
|
+
|
|
54
|
+
- `on_error:` for `temporarily` (`:raise` or a callable).
|
|
55
|
+
- `rails generate idempotency_lock:upgrade` generator.
|
|
56
|
+
- `on_error:` accepts any object responding to `call`, not only Procs.
|
|
57
|
+
- RBS signatures for the public API.
|
|
58
|
+
- CI: Ruby 3.2–3.4 × ActiveRecord 7.1–8.1, plus PostgreSQL and MySQL jobs;
|
|
59
|
+
read-only token permissions and SHA-pinned actions.
|
|
60
|
+
|
|
61
|
+
### Documentation
|
|
62
|
+
|
|
63
|
+
- Clarified that `once` is *at most once*, not exactly once, and documented
|
|
64
|
+
error-swallowing, interrupt, transaction, and clock-skew behavior.
|
|
65
|
+
- Fixed the Sidekiq example (it used `:unlock`, which swallows the error so
|
|
66
|
+
Sidekiq never retries) and the webhook example (now verifies the signature
|
|
67
|
+
before taking a lock). Added a Security Considerations section.
|
|
68
|
+
|
|
69
|
+
## [0.2.0] - 2026-01-24
|
|
70
|
+
|
|
71
|
+
### Added
|
|
72
|
+
|
|
73
|
+
- `IdempotencyLock.temporarily` - Execute a block with an ephemeral lock that is
|
|
74
|
+
automatically released when the block completes (success or error)
|
|
75
|
+
- `wait:` parameter for `temporarily` - Poll for lock availability with configurable
|
|
76
|
+
timeout (e.g., `wait: 5.seconds`)
|
|
77
|
+
|
|
3
78
|
## [0.1.3] - 2025-12-06
|
|
4
79
|
|
|
5
80
|
### Changed
|
data/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# IdempotencyLock
|
|
2
2
|
|
|
3
|
-
A simple, robust idempotency solution for Ruby on Rails applications.
|
|
3
|
+
A simple, robust idempotency solution for Ruby on Rails applications. Guarantees an operation runs **at most once** per lock name (or once per TTL window) across processes and machines, using database-backed locks with support for TTL expiration, multiple error handling strategies, and clean return value handling.
|
|
4
4
|
|
|
5
5
|
## Installation
|
|
6
6
|
|
|
@@ -23,6 +23,17 @@ rails generate idempotency_lock:install
|
|
|
23
23
|
rails db:migrate
|
|
24
24
|
```
|
|
25
25
|
|
|
26
|
+
Requires Ruby 3.2+ and Rails / ActiveRecord 7.1+. Tested on SQLite, PostgreSQL, and MySQL.
|
|
27
|
+
|
|
28
|
+
### Upgrading from 0.2.x or earlier
|
|
29
|
+
|
|
30
|
+
Version 0.3.0 adds an `owner_token` column. Generate and run the upgrade migration **before** deploying 0.3.0:
|
|
31
|
+
|
|
32
|
+
```bash
|
|
33
|
+
rails generate idempotency_lock:upgrade
|
|
34
|
+
rails db:migrate
|
|
35
|
+
```
|
|
36
|
+
|
|
26
37
|
## Usage
|
|
27
38
|
|
|
28
39
|
### Basic Usage
|
|
@@ -52,6 +63,15 @@ result.value # the return value of the block
|
|
|
52
63
|
result.error # the exception if one was raised
|
|
53
64
|
```
|
|
54
65
|
|
|
66
|
+
### At-most-once semantics
|
|
67
|
+
|
|
68
|
+
The lock is taken **before** the block runs. That makes `once` *at most once*, not *exactly once*:
|
|
69
|
+
|
|
70
|
+
- If the block raises, `on_error` decides whether the lock is kept (no retry) or released (retry allowed). See [Error Handling](#error-handling).
|
|
71
|
+
- If the process dies mid-block (crash, `kill -9`, OOM), the lock stays in place and the operation will not run again until the lock expires (`ttl:`) or is released manually. Use a `ttl:` if you would rather risk a second run than never finishing.
|
|
72
|
+
|
|
73
|
+
Lock names must be non-blank Strings or Symbols of at most 255 characters; anything else raises `ArgumentError`.
|
|
74
|
+
|
|
55
75
|
### TTL / Expiration
|
|
56
76
|
|
|
57
77
|
Sometimes you want "run once per hour" instead of "run once forever":
|
|
@@ -67,51 +87,131 @@ IdempotencyLock.once("hourly-cleanup", ttl: 1.hour) do
|
|
|
67
87
|
TempFile.cleanup_old
|
|
68
88
|
end
|
|
69
89
|
|
|
70
|
-
# TTL also accepts
|
|
90
|
+
# TTL also accepts seconds
|
|
71
91
|
IdempotencyLock.once("periodic-task", ttl: 3600) do
|
|
72
92
|
some_periodic_work
|
|
73
93
|
end
|
|
74
94
|
```
|
|
75
95
|
|
|
96
|
+
Expiry is checked against the clock of the server doing the check, so keep server clocks synchronized (NTP). Clock skew can shorten or lengthen a TTL window: a server whose clock runs ahead can see a lock as expired, and take it over, while the previous holder is still running. Pick TTLs with a margin well above your expected clock drift. (When several callers race to claim the *same* expired lock, the owner-token check still guarantees only one succeeds.)
|
|
97
|
+
|
|
98
|
+
### Temporary Locks (Auto-Release)
|
|
99
|
+
|
|
100
|
+
Use `temporarily` when you want exclusive execution but don't need permanent idempotency. The lock is released when the block finishes, whether it succeeds, raises, or exits early:
|
|
101
|
+
|
|
102
|
+
```ruby
|
|
103
|
+
IdempotencyLock.temporarily("process-payment-#{payment.id}", ttl: 5.minutes) do
|
|
104
|
+
PaymentProcessor.charge(payment)
|
|
105
|
+
end
|
|
106
|
+
# Lock is now released - another call can acquire it
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
This is useful for:
|
|
110
|
+
- Preventing concurrent processing of the same resource
|
|
111
|
+
- Mutex-style synchronization across processes/servers
|
|
112
|
+
- Operations that should be exclusive but retriable
|
|
113
|
+
|
|
114
|
+
#### Always set a TTL
|
|
115
|
+
|
|
116
|
+
> **Without `ttl:`, a lock held by a process that is killed (SIGKILL, OOM, host failure) is never released**, and every later call is skipped until you run `IdempotencyLock.release(name)`.
|
|
117
|
+
|
|
118
|
+
Set `ttl:` comfortably longer than the block's worst-case runtime. If the block outlives its TTL, another process may acquire the lock and run concurrently. The original holder will not release the new holder's lock when it finishes.
|
|
119
|
+
|
|
120
|
+
#### Waiting for Lock Availability
|
|
121
|
+
|
|
122
|
+
By default, `temporarily` returns immediately if the lock is held. Use `wait:` to poll until the lock becomes available:
|
|
123
|
+
|
|
124
|
+
```ruby
|
|
125
|
+
# Wait up to 5 seconds for lock, then skip if still unavailable
|
|
126
|
+
result = IdempotencyLock.temporarily("exclusive-job", ttl: 10.minutes, wait: 5.seconds) do
|
|
127
|
+
process_job
|
|
128
|
+
end
|
|
129
|
+
|
|
130
|
+
if result.skipped?
|
|
131
|
+
puts "Could not acquire lock within timeout"
|
|
132
|
+
end
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
#### Errors in temporary locks
|
|
136
|
+
|
|
137
|
+
By default an exception raised in the block is captured in the `Result` (`result.error?`) and **not re-raised**. Pass `on_error: :raise` to re-raise it after the lock is released, or a callable to be notified:
|
|
138
|
+
|
|
139
|
+
```ruby
|
|
140
|
+
IdempotencyLock.temporarily("sync-account-#{id}", ttl: 1.minute, on_error: :raise) do
|
|
141
|
+
AccountSync.run(id)
|
|
142
|
+
end
|
|
143
|
+
```
|
|
144
|
+
|
|
76
145
|
### Error Handling
|
|
77
146
|
|
|
78
|
-
Control what happens when
|
|
147
|
+
Control what happens when the block raises a `StandardError`:
|
|
79
148
|
|
|
80
149
|
```ruby
|
|
81
|
-
# :keep (default) - Keep the lock
|
|
150
|
+
# :keep (default) - Keep the lock: the operation will NOT be retried
|
|
82
151
|
IdempotencyLock.once("risky-op", on_error: :keep) do
|
|
83
152
|
might_fail
|
|
84
153
|
end
|
|
85
154
|
|
|
86
|
-
# :unlock - Release the lock so
|
|
155
|
+
# :unlock - Release the lock so a later call can retry
|
|
87
156
|
IdempotencyLock.once("retriable-op", on_error: :unlock) do
|
|
88
157
|
external_api_call
|
|
89
158
|
end
|
|
90
159
|
|
|
91
|
-
# :raise -
|
|
160
|
+
# :raise - Release the lock, then re-raise the exception
|
|
92
161
|
IdempotencyLock.once("critical-op", on_error: :raise) do
|
|
93
162
|
must_succeed_or_alert
|
|
94
163
|
end
|
|
95
164
|
|
|
96
|
-
#
|
|
165
|
+
# Any callable - Release the lock, then call it with the exception
|
|
97
166
|
IdempotencyLock.once("custom-error", on_error: ->(e) { Bugsnag.notify(e) }) do
|
|
98
167
|
something_important
|
|
99
168
|
end
|
|
100
169
|
```
|
|
101
170
|
|
|
171
|
+
Things to know:
|
|
172
|
+
|
|
173
|
+
- **Errors are swallowed unless you use `:raise`.** With `:keep`, `:unlock`, or a callable, `once` returns a `Result` with `error?` set and does not raise. In a background job, that means the job framework sees success and **will not retry**. Use `on_error: :raise` when you rely on job retries.
|
|
174
|
+
- **The default (`:keep`) means a transient failure permanently suppresses the operation** (or until the TTL expires). Choose `:unlock` / `:raise` if retrying is safe.
|
|
175
|
+
- **Interrupts and other non-`StandardError` exceptions** (e.g. `Interrupt`, `SignalException`, `Sidekiq::Shutdown` during a deploy) are always re-raised. The lock is kept for `:keep` and released for every other strategy; callables are not invoked for them.
|
|
176
|
+
- **An early `return`/`break`/`next` out of the block counts as success.** The lock is kept.
|
|
177
|
+
- An unknown `on_error` value raises `ArgumentError` before anything runs.
|
|
178
|
+
|
|
179
|
+
### Database Transactions
|
|
180
|
+
|
|
181
|
+
The lock row is written through the current database connection, so if you call `once` inside a transaction:
|
|
182
|
+
|
|
183
|
+
- The lock is not visible to other processes until your transaction commits. A concurrent caller blocks on the unique index until then, then skips.
|
|
184
|
+
- **If your transaction rolls back, the lock is rolled back too**, even though any side effects (emails sent, APIs called) already happened. Prefer to call `once` outside transactions, or wrap only database work that should roll back together with the lock.
|
|
185
|
+
|
|
186
|
+
Acquisition uses a savepoint, so a lock conflict never aborts your surrounding transaction (relevant on PostgreSQL).
|
|
187
|
+
|
|
102
188
|
### Manual Lock Management
|
|
103
189
|
|
|
104
190
|
```ruby
|
|
105
191
|
# Check if an operation is locked
|
|
106
192
|
IdempotencyLock.locked?("my-operation") # => true/false
|
|
107
193
|
|
|
108
|
-
#
|
|
194
|
+
# Force-release a lock regardless of who holds it
|
|
195
|
+
# (useful for testing or manual intervention)
|
|
109
196
|
IdempotencyLock.release("my-operation")
|
|
110
197
|
|
|
111
198
|
# Clean up all expired locks (good for a periodic job)
|
|
112
199
|
IdempotencyLock.cleanup_expired
|
|
113
200
|
```
|
|
114
201
|
|
|
202
|
+
Locks without a TTL are never removed by `cleanup_expired`.
|
|
203
|
+
|
|
204
|
+
### Logging
|
|
205
|
+
|
|
206
|
+
Logs go to `Rails.logger` by default. To use a different logger, or disable logging:
|
|
207
|
+
|
|
208
|
+
```ruby
|
|
209
|
+
IdempotencyLock.logger = Logger.new($stdout)
|
|
210
|
+
IdempotencyLock.logger = nil # disable
|
|
211
|
+
```
|
|
212
|
+
|
|
213
|
+
Lock names and exception messages are logged, so avoid putting secrets or personal data in lock names.
|
|
214
|
+
|
|
115
215
|
### Alias
|
|
116
216
|
|
|
117
217
|
The `wrap` method is available as an alias for `once`:
|
|
@@ -124,16 +224,26 @@ IdempotencyLock.wrap("operation-name") { do_something }
|
|
|
124
224
|
|
|
125
225
|
### Preventing Duplicate Webhook Processing
|
|
126
226
|
|
|
227
|
+
Verify the webhook signature **before** taking a lock. Otherwise anyone can send forged requests that claim lock names (and grow the table) for events that never happened.
|
|
228
|
+
|
|
127
229
|
```ruby
|
|
128
230
|
class WebhooksController < ApplicationController
|
|
129
231
|
def stripe
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
232
|
+
event = Stripe::Webhook.construct_event(
|
|
233
|
+
request.body.read,
|
|
234
|
+
request.headers["Stripe-Signature"],
|
|
235
|
+
Rails.application.credentials.stripe_webhook_secret
|
|
236
|
+
)
|
|
237
|
+
|
|
238
|
+
# Namespace the key; expire it after Stripe's retry window.
|
|
239
|
+
# on_error: :raise releases the lock and returns a 500, so Stripe retries.
|
|
240
|
+
IdempotencyLock.once("stripe-webhook:#{event.id}", ttl: 7.days, on_error: :raise) do
|
|
241
|
+
StripeWebhookProcessor.process(event)
|
|
134
242
|
end
|
|
135
|
-
|
|
243
|
+
|
|
136
244
|
head :ok
|
|
245
|
+
rescue JSON::ParserError, Stripe::SignatureVerificationError
|
|
246
|
+
head :bad_request
|
|
137
247
|
end
|
|
138
248
|
end
|
|
139
249
|
```
|
|
@@ -164,14 +274,32 @@ class ImportantJob
|
|
|
164
274
|
include Sidekiq::Job
|
|
165
275
|
|
|
166
276
|
def perform(record_id)
|
|
167
|
-
|
|
168
|
-
|
|
277
|
+
# :raise releases the lock and re-raises, so Sidekiq's retry runs it again.
|
|
278
|
+
# (With :unlock the error would be swallowed and Sidekiq would never retry.)
|
|
279
|
+
IdempotencyLock.once("important-job-#{record_id}", on_error: :raise) do
|
|
169
280
|
ImportantService.process(record_id)
|
|
170
281
|
end
|
|
171
282
|
end
|
|
172
283
|
end
|
|
173
284
|
```
|
|
174
285
|
|
|
286
|
+
### Exclusive Resource Processing
|
|
287
|
+
|
|
288
|
+
```ruby
|
|
289
|
+
# Ensure only one process handles a payment at a time
|
|
290
|
+
def process_payment(payment_id)
|
|
291
|
+
IdempotencyLock.temporarily("payment-#{payment_id}", ttl: 5.minutes, wait: 10.seconds) do
|
|
292
|
+
Payment.find(payment_id).process!
|
|
293
|
+
end
|
|
294
|
+
end
|
|
295
|
+
```
|
|
296
|
+
|
|
297
|
+
## Security Considerations
|
|
298
|
+
|
|
299
|
+
- **Lock names derived from user input:** if an attacker can choose lock names, they can claim names in advance to block legitimate operations, or create rows without limit. Authenticate/verify input first (e.g. webhook signatures), namespace names (`"stripe-webhook:#{id}"`), and use a `ttl:` plus a periodic `cleanup_expired` so the table stays bounded.
|
|
300
|
+
- **`Result#inspect`** shows only the class of the block's return value, not its contents, so inspecting a result in logs or error trackers does not leak data. `result.value` still holds the real value.
|
|
301
|
+
- All queries use bound parameters.
|
|
302
|
+
|
|
175
303
|
## Database Schema
|
|
176
304
|
|
|
177
305
|
The gem creates an `idempotency_locks` table with:
|
|
@@ -179,8 +307,9 @@ The gem creates an `idempotency_locks` table with:
|
|
|
179
307
|
| Column | Type | Description |
|
|
180
308
|
|--------|------|-------------|
|
|
181
309
|
| `name` | string (max 255) | Unique identifier for the lock |
|
|
310
|
+
| `owner_token` | string (36) | Random token identifying the current holder |
|
|
182
311
|
| `expires_at` | datetime | When the lock expires (null = never) |
|
|
183
|
-
| `executed_at` | datetime | When the
|
|
312
|
+
| `executed_at` | datetime | When the lock was acquired (i.e. when the block started) |
|
|
184
313
|
| `created_at` | datetime | Standard Rails timestamp |
|
|
185
314
|
| `updated_at` | datetime | Standard Rails timestamp |
|
|
186
315
|
|
|
@@ -188,30 +317,49 @@ Indexes:
|
|
|
188
317
|
- Unique index on `name` (provides atomicity)
|
|
189
318
|
- Index on `expires_at` (partial index on PostgreSQL/SQLite, regular index on MySQL)
|
|
190
319
|
|
|
191
|
-
**Note on name length:**
|
|
320
|
+
**Note on name length:** modern MySQL (InnoDB with the default DYNAMIC row format) can index 255 `utf8mb4` characters. Very old MySQL setups limited to 767-byte index keys need the column reduced to 191 characters in the migration. If you do that, keep your lock names within that length too.
|
|
192
321
|
|
|
193
322
|
## How It Works
|
|
194
323
|
|
|
195
|
-
1. **Lock Acquisition**:
|
|
324
|
+
1. **Lock Acquisition**: Inserts a row with the given name and a fresh random owner token. The unique index ensures only one process succeeds. The insert runs in a savepoint so a conflict never breaks a surrounding transaction.
|
|
196
325
|
|
|
197
|
-
2. **TTL Support**: If a TTL is provided, the lock includes an `expires_at` timestamp.
|
|
326
|
+
2. **TTL Support**: If a TTL is provided, the lock includes an `expires_at` timestamp. An expired lock is claimed with a conditional `UPDATE ... WHERE owner_token = <observed token>`, so if several processes race to claim the same expired lock, exactly one wins. Whether a lock counts as expired depends on the claiming server's clock (see [TTL / Expiration](#ttl--expiration)).
|
|
198
327
|
|
|
199
|
-
3. **
|
|
328
|
+
3. **Owner-Safe Release**: Releases delete the row only if it still carries the releasing process's owner token. A process whose lock expired and was taken over cannot delete the new holder's lock.
|
|
200
329
|
|
|
201
|
-
4. **
|
|
330
|
+
4. **Error Handling**: Strategies control whether the lock is released on error, allowing for retry semantics.
|
|
202
331
|
|
|
203
|
-
|
|
332
|
+
5. **Return Values**: The block's return value is captured and returned in the `Result` object.
|
|
333
|
+
|
|
334
|
+
## Concurrency
|
|
204
335
|
|
|
205
336
|
The gem relies on database-level uniqueness constraints for atomicity. This means:
|
|
206
337
|
|
|
207
|
-
- ✅ Safe across multiple processes (web servers, job workers)
|
|
208
|
-
- ✅ Safe across
|
|
209
|
-
- ✅ Safe in multi-threaded environments
|
|
338
|
+
- ✅ Safe across multiple processes (web servers, job workers) and machines
|
|
339
|
+
- ✅ Safe across threads (each thread uses its own database connection)
|
|
210
340
|
- ✅ Survives application restarts
|
|
341
|
+
- ⚠️ Mutual exclusion from `temporarily` holds only while the TTL has not expired (as judged by each server's clock), so pick a TTL longer than the block's runtime plus expected clock drift
|
|
211
342
|
|
|
212
343
|
## Development
|
|
213
344
|
|
|
214
|
-
After checking out the repo, run `bin/setup` to install dependencies. Then
|
|
345
|
+
After checking out the repo, run `bin/setup` to install dependencies. Then run `rake` to run the tests and RuboCop.
|
|
346
|
+
|
|
347
|
+
The suite uses SQLite by default. To run it against PostgreSQL or MySQL:
|
|
348
|
+
|
|
349
|
+
```bash
|
|
350
|
+
BUNDLE_WITH=postgres bundle install
|
|
351
|
+
BUNDLE_WITH=postgres DATABASE_URL=postgres://user:pass@localhost/idempotency_lock_test bundle exec rake test
|
|
352
|
+
|
|
353
|
+
BUNDLE_WITH=mysql bundle install
|
|
354
|
+
BUNDLE_WITH=mysql DATABASE_URL=mysql2://user:pass@127.0.0.1/idempotency_lock_test bundle exec rake test
|
|
355
|
+
```
|
|
356
|
+
|
|
357
|
+
To test a specific ActiveRecord version, use the gemfiles in `gemfiles/`:
|
|
358
|
+
|
|
359
|
+
```bash
|
|
360
|
+
BUNDLE_GEMFILE=gemfiles/activerecord_7.1.gemfile bundle install
|
|
361
|
+
BUNDLE_GEMFILE=gemfiles/activerecord_7.1.gemfile bundle exec rake test
|
|
362
|
+
```
|
|
215
363
|
|
|
216
364
|
## Contributing
|
|
217
365
|
|
|
@@ -1,16 +1,13 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
require "rails/generators/active_record"
|
|
3
|
+
require_relative "migration_generator_helpers"
|
|
5
4
|
|
|
6
5
|
module IdempotencyLock
|
|
7
6
|
module Generators
|
|
8
7
|
# Generator to create the idempotency_locks table migration.
|
|
9
8
|
# Run with: rails generate idempotency_lock:install
|
|
10
9
|
class InstallGenerator < Rails::Generators::Base
|
|
11
|
-
include
|
|
12
|
-
|
|
13
|
-
source_root File.expand_path("templates", __dir__)
|
|
10
|
+
include MigrationGeneratorHelpers
|
|
14
11
|
|
|
15
12
|
desc "Creates the idempotency_locks table migration"
|
|
16
13
|
|
|
@@ -20,10 +17,6 @@ module IdempotencyLock
|
|
|
20
17
|
"db/migrate/create_idempotency_locks.rb"
|
|
21
18
|
)
|
|
22
19
|
end
|
|
23
|
-
|
|
24
|
-
def migration_version
|
|
25
|
-
"[#{ActiveRecord::VERSION::MAJOR}.#{ActiveRecord::VERSION::MINOR}]"
|
|
26
|
-
end
|
|
27
20
|
end
|
|
28
21
|
end
|
|
29
22
|
end
|
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "rails/generators"
|
|
4
|
+
require "rails/generators/active_record"
|
|
5
|
+
|
|
6
|
+
module IdempotencyLock
|
|
7
|
+
module Generators
|
|
8
|
+
# Shared setup for the gem's migration generators.
|
|
9
|
+
module MigrationGeneratorHelpers
|
|
10
|
+
def self.included(base)
|
|
11
|
+
base.include ActiveRecord::Generators::Migration
|
|
12
|
+
base.source_root File.expand_path("templates", __dir__)
|
|
13
|
+
end
|
|
14
|
+
|
|
15
|
+
def migration_version
|
|
16
|
+
"[#{ActiveRecord::VERSION::MAJOR}.#{ActiveRecord::VERSION::MINOR}]"
|
|
17
|
+
end
|
|
18
|
+
end
|
|
19
|
+
end
|
|
20
|
+
end
|
|
@@ -0,0 +1,9 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
class AddOwnerTokenToIdempotencyLocks < ActiveRecord::Migration<%= migration_version %>
|
|
4
|
+
def change
|
|
5
|
+
# Random token identifying the current lock holder, so a process can only
|
|
6
|
+
# release a lock it still owns. Required by idempotency_lock >= 0.3.0.
|
|
7
|
+
add_column :idempotency_locks, :owner_token, :string, limit: 36
|
|
8
|
+
end
|
|
9
|
+
end
|
|
@@ -3,10 +3,15 @@
|
|
|
3
3
|
class CreateIdempotencyLocks < ActiveRecord::Migration<%= migration_version %>
|
|
4
4
|
def change
|
|
5
5
|
create_table :idempotency_locks do |t|
|
|
6
|
-
#
|
|
7
|
-
#
|
|
6
|
+
# Must match IdempotencyLock::MAX_NAME_LENGTH (255). Modern MySQL
|
|
7
|
+
# (InnoDB, DYNAMIC row format) can index 255 utf8mb4 characters; very old
|
|
8
|
+
# MySQL setups limited to 767-byte index keys need 191 here instead.
|
|
8
9
|
t.string :name, null: false, limit: 255
|
|
10
|
+
# Random token identifying the current lock holder, so a process can only
|
|
11
|
+
# release a lock it still owns.
|
|
12
|
+
t.string :owner_token, limit: 36
|
|
9
13
|
t.datetime :expires_at
|
|
14
|
+
# When the lock was acquired (i.e. when the block started running).
|
|
10
15
|
t.datetime :executed_at
|
|
11
16
|
|
|
12
17
|
t.timestamps
|
|
@@ -16,11 +21,10 @@ class CreateIdempotencyLocks < ActiveRecord::Migration<%= migration_version %>
|
|
|
16
21
|
|
|
17
22
|
# Partial indexes are more efficient but not supported by MySQL.
|
|
18
23
|
# Use a regular index for MySQL, partial index for PostgreSQL/SQLite.
|
|
19
|
-
if
|
|
24
|
+
if connection.adapter_name.downcase.include?("mysql")
|
|
20
25
|
add_index :idempotency_locks, :expires_at
|
|
21
26
|
else
|
|
22
27
|
add_index :idempotency_locks, :expires_at, where: "expires_at IS NOT NULL"
|
|
23
28
|
end
|
|
24
29
|
end
|
|
25
30
|
end
|
|
26
|
-
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require_relative "migration_generator_helpers"
|
|
4
|
+
|
|
5
|
+
module IdempotencyLock
|
|
6
|
+
module Generators
|
|
7
|
+
# Generator for existing installs upgrading to 0.3.0+: adds the
|
|
8
|
+
# owner_token column used to make lock release owner-safe.
|
|
9
|
+
# Run with: rails generate idempotency_lock:upgrade
|
|
10
|
+
class UpgradeGenerator < Rails::Generators::Base
|
|
11
|
+
include MigrationGeneratorHelpers
|
|
12
|
+
|
|
13
|
+
desc "Adds the owner_token column required by idempotency_lock >= 0.3.0"
|
|
14
|
+
|
|
15
|
+
def create_migration_file
|
|
16
|
+
migration_template(
|
|
17
|
+
"add_owner_token_to_idempotency_locks.rb.erb",
|
|
18
|
+
"db/migrate/add_owner_token_to_idempotency_locks.rb"
|
|
19
|
+
)
|
|
20
|
+
end
|
|
21
|
+
end
|
|
22
|
+
end
|
|
23
|
+
end
|
|
@@ -0,0 +1,59 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "active_support/core_ext/object/blank"
|
|
4
|
+
|
|
5
|
+
module IdempotencyLock
|
|
6
|
+
# Validation for the public API's arguments. Invalid input raises
|
|
7
|
+
# ArgumentError up front instead of being mistaken for "lock already held".
|
|
8
|
+
# @api private
|
|
9
|
+
module Arguments
|
|
10
|
+
module_function
|
|
11
|
+
|
|
12
|
+
# @return [String] the name as a String
|
|
13
|
+
def normalize_name!(name)
|
|
14
|
+
unless name.is_a?(String) || name.is_a?(Symbol)
|
|
15
|
+
raise ArgumentError, "Lock name must be a String or Symbol, got #{name.class}"
|
|
16
|
+
end
|
|
17
|
+
|
|
18
|
+
name = name.to_s
|
|
19
|
+
# blank? (not strip.empty?) so Unicode whitespace such as U+00A0 is
|
|
20
|
+
# rejected here, matching the model's presence validation.
|
|
21
|
+
raise ArgumentError, "Lock name must not be blank" if name.blank?
|
|
22
|
+
if name.length > MAX_NAME_LENGTH
|
|
23
|
+
raise ArgumentError, "Lock name is too long (#{name.length} > #{MAX_NAME_LENGTH} characters)"
|
|
24
|
+
end
|
|
25
|
+
|
|
26
|
+
name
|
|
27
|
+
end
|
|
28
|
+
|
|
29
|
+
# ActiveSupport::Duration reports is_a?(Numeric) as true, so this accepts
|
|
30
|
+
# both plain seconds and durations like 5.minutes. NaN and Infinity are
|
|
31
|
+
# rejected: they can't be added to a Time.
|
|
32
|
+
def validate_ttl!(ttl)
|
|
33
|
+
return if ttl.nil? || (finite_number?(ttl) && ttl.positive?)
|
|
34
|
+
|
|
35
|
+
raise ArgumentError, "ttl must be a positive number of seconds or a Duration, got #{ttl.inspect}"
|
|
36
|
+
end
|
|
37
|
+
|
|
38
|
+
# @return [Float, nil] the wait in seconds
|
|
39
|
+
# NaN and Infinity are rejected: NaN would silently skip without waiting,
|
|
40
|
+
# and Infinity would poll forever.
|
|
41
|
+
def normalize_wait!(wait)
|
|
42
|
+
return nil if wait.nil?
|
|
43
|
+
return wait.to_f if finite_number?(wait) && !wait.negative?
|
|
44
|
+
|
|
45
|
+
raise ArgumentError, "wait must be a non-negative number of seconds or a Duration, got #{wait.inspect}"
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
def finite_number?(value)
|
|
49
|
+
value.is_a?(Numeric) && value.to_f.finite?
|
|
50
|
+
end
|
|
51
|
+
|
|
52
|
+
def validate_on_error!(on_error, strategies)
|
|
53
|
+
return if strategies.include?(on_error) || on_error.respond_to?(:call)
|
|
54
|
+
|
|
55
|
+
allowed = strategies.map(&:inspect).join(", ")
|
|
56
|
+
raise ArgumentError, "on_error must be one of #{allowed} or a callable, got #{on_error.inspect}"
|
|
57
|
+
end
|
|
58
|
+
end
|
|
59
|
+
end
|
|
@@ -1,26 +1,31 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require "securerandom"
|
|
4
|
+
|
|
3
5
|
module IdempotencyLock
|
|
4
6
|
# ActiveRecord model for storing idempotency locks.
|
|
5
7
|
#
|
|
6
8
|
# This model expects a database table with the following schema:
|
|
7
9
|
# - name: string (unique, indexed, max 255 chars)
|
|
10
|
+
# - owner_token: string (random token identifying the current holder)
|
|
8
11
|
# - expires_at: datetime (nullable, indexed)
|
|
9
|
-
# - executed_at: datetime
|
|
12
|
+
# - executed_at: datetime (when the lock was acquired / the block started)
|
|
10
13
|
# - created_at: datetime
|
|
11
14
|
# - updated_at: datetime
|
|
12
15
|
#
|
|
13
|
-
# Use the provided
|
|
14
|
-
# rails generate idempotency_lock:install
|
|
16
|
+
# Use the provided generators to create the migration:
|
|
17
|
+
# rails generate idempotency_lock:install # new installs
|
|
18
|
+
# rails generate idempotency_lock:upgrade # upgrading from < 0.3.0
|
|
15
19
|
#
|
|
16
20
|
class Lock < ActiveRecord::Base
|
|
17
21
|
self.table_name = "idempotency_locks"
|
|
18
22
|
|
|
19
|
-
|
|
20
|
-
# unique index limits (MySQL utf8mb4 = 191, most others = 255+).
|
|
21
|
-
MAX_NAME_LENGTH = 255
|
|
23
|
+
MAX_NAME_LENGTH = IdempotencyLock::MAX_NAME_LENGTH
|
|
22
24
|
|
|
23
|
-
|
|
25
|
+
# Uniqueness is enforced by the unique index on `name`, not by a
|
|
26
|
+
# validation, so acquiring a lock is a single INSERT rather than a
|
|
27
|
+
# SELECT followed by an INSERT.
|
|
28
|
+
validates :name, presence: true, length: { maximum: MAX_NAME_LENGTH }
|
|
24
29
|
|
|
25
30
|
# Check if this lock has expired
|
|
26
31
|
# @return [Boolean]
|
|
@@ -36,62 +41,79 @@ module IdempotencyLock
|
|
|
36
41
|
|
|
37
42
|
class << self
|
|
38
43
|
# Attempt to acquire a lock for the given name.
|
|
39
|
-
# Returns true if the lock was acquired, false otherwise.
|
|
40
44
|
#
|
|
41
45
|
# @param name [String] unique identifier for the operation
|
|
42
46
|
# @param expires_at [Time, nil] when the lock should expire (nil = never)
|
|
43
47
|
# @param now [Time] current time reference (injectable for testing)
|
|
44
|
-
# @return [
|
|
48
|
+
# @return [String, nil] an owner token if the lock was acquired, nil otherwise.
|
|
49
|
+
# Pass the token to {release} so only the holder can release the lock.
|
|
45
50
|
def acquire(name, expires_at: nil, now: Time.current)
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
create!(name: name, expires_at: expires_at, executed_at: now)
|
|
49
|
-
return true
|
|
50
|
-
rescue ActiveRecord::RecordNotUnique, ActiveRecord::RecordInvalid
|
|
51
|
-
# Lock exists, check if it's expired
|
|
52
|
-
end
|
|
51
|
+
token = SecureRandom.uuid
|
|
52
|
+
return token if insert_lock(name, token, expires_at, now)
|
|
53
53
|
|
|
54
|
-
|
|
55
|
-
existing = find_by(name: name)
|
|
56
|
-
return false if existing.nil?
|
|
57
|
-
return false unless existing.expired?(now: now)
|
|
58
|
-
|
|
59
|
-
# Optimistic locking: use the observed updated_at in the WHERE clause.
|
|
60
|
-
# This prevents a race where two processes both see an expired lock and
|
|
61
|
-
# both try to claim it. Only one UPDATE will match; the other returns 0.
|
|
62
|
-
# We use updated_at rather than expires_at because it's system-managed
|
|
63
|
-
# and always moves forward, making it more reliable for concurrency control.
|
|
64
|
-
updated = where(name: name, updated_at: existing.updated_at)
|
|
65
|
-
.update_all(expires_at: expires_at, executed_at: now, updated_at: now)
|
|
66
|
-
|
|
67
|
-
updated.positive?
|
|
54
|
+
claim_expired(name, token, expires_at, now) ? token : nil
|
|
68
55
|
end
|
|
69
56
|
|
|
70
|
-
# Release a lock by name
|
|
57
|
+
# Release a lock by name.
|
|
58
|
+
#
|
|
59
|
+
# When `owner_token` is given, the lock is only removed if it is still
|
|
60
|
+
# held by that owner. This prevents a holder whose lock expired (and was
|
|
61
|
+
# claimed by someone else) from deleting the new holder's lock.
|
|
71
62
|
#
|
|
72
63
|
# @param name [String] unique identifier for the operation
|
|
64
|
+
# @param owner_token [String, nil] token returned by {acquire}; nil forces release
|
|
73
65
|
# @return [Boolean] true if a lock was removed
|
|
74
|
-
def release(name)
|
|
75
|
-
|
|
76
|
-
|
|
66
|
+
def release(name, owner_token: nil)
|
|
67
|
+
scope = where(name: name)
|
|
68
|
+
scope = scope.where(owner_token: owner_token) if owner_token
|
|
69
|
+
scope.delete_all.positive?
|
|
77
70
|
end
|
|
78
71
|
|
|
79
72
|
# Check if a valid (non-expired) lock exists for the given name
|
|
80
73
|
#
|
|
81
74
|
# @param name [String] unique identifier for the operation
|
|
82
75
|
# @return [Boolean]
|
|
83
|
-
def locked?(name)
|
|
76
|
+
def locked?(name, now: Time.current)
|
|
84
77
|
lock = find_by(name: name)
|
|
85
78
|
return false if lock.nil?
|
|
86
79
|
|
|
87
|
-
lock.valid_lock?
|
|
80
|
+
lock.valid_lock?(now: now)
|
|
88
81
|
end
|
|
89
82
|
|
|
90
83
|
# Clean up expired locks
|
|
91
84
|
#
|
|
92
85
|
# @return [Integer] number of locks removed
|
|
93
|
-
def cleanup_expired
|
|
94
|
-
where("expires_at IS NOT NULL AND expires_at < ?",
|
|
86
|
+
def cleanup_expired(now: Time.current)
|
|
87
|
+
where("expires_at IS NOT NULL AND expires_at < ?", now).delete_all
|
|
88
|
+
end
|
|
89
|
+
|
|
90
|
+
private
|
|
91
|
+
|
|
92
|
+
# Insert a new lock row. Runs inside a savepoint so that a unique
|
|
93
|
+
# violation does not abort an enclosing transaction (PostgreSQL marks the
|
|
94
|
+
# whole transaction as failed after any error otherwise).
|
|
95
|
+
def insert_lock(name, token, expires_at, now)
|
|
96
|
+
transaction(requires_new: true) do
|
|
97
|
+
create!(name: name, owner_token: token, expires_at: expires_at, executed_at: now)
|
|
98
|
+
end
|
|
99
|
+
true
|
|
100
|
+
rescue ActiveRecord::RecordNotUnique
|
|
101
|
+
false
|
|
102
|
+
end
|
|
103
|
+
|
|
104
|
+
# Claim an existing lock if it has expired.
|
|
105
|
+
#
|
|
106
|
+
# Uses the observed owner_token as an optimistic-lock version: if two
|
|
107
|
+
# processes both see the same expired lock, only one UPDATE matches and
|
|
108
|
+
# the other affects 0 rows. Every acquisition writes a fresh random
|
|
109
|
+
# token, so the version always changes (no timestamp precision issues).
|
|
110
|
+
def claim_expired(name, token, expires_at, now)
|
|
111
|
+
existing = find_by(name: name)
|
|
112
|
+
return false if existing.nil? || !existing.expired?(now: now)
|
|
113
|
+
|
|
114
|
+
where(name: name, owner_token: existing.owner_token)
|
|
115
|
+
.update_all(owner_token: token, expires_at: expires_at, executed_at: now, updated_at: now)
|
|
116
|
+
.positive?
|
|
95
117
|
end
|
|
96
118
|
end
|
|
97
119
|
end
|
|
@@ -1,15 +1,15 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
module IdempotencyLock
|
|
4
|
-
# Rails integration for IdempotencyLock.
|
|
5
|
-
#
|
|
4
|
+
# Rails integration for IdempotencyLock: registers the generators.
|
|
5
|
+
#
|
|
6
|
+
# Logging defaults to Rails.logger (see IdempotencyLock.logger), so no
|
|
7
|
+
# initializer is needed and an explicitly configured logger, including nil,
|
|
8
|
+
# is never overwritten.
|
|
6
9
|
class Railtie < Rails::Railtie
|
|
7
10
|
generators do
|
|
8
11
|
require_relative "../generators/idempotency_lock/install_generator"
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
initializer "idempotency_lock.configure" do
|
|
12
|
-
IdempotencyLock.logger = Rails.logger
|
|
12
|
+
require_relative "../generators/idempotency_lock/upgrade_generator"
|
|
13
13
|
end
|
|
14
14
|
end
|
|
15
15
|
end
|
|
@@ -41,11 +41,15 @@ module IdempotencyLock
|
|
|
41
41
|
@executed && !error?
|
|
42
42
|
end
|
|
43
43
|
|
|
44
|
-
#
|
|
44
|
+
# Human-readable representation for debugging. Shows only the class of the
|
|
45
|
+
# value, not its contents, so inspecting a Result (in logs or error
|
|
46
|
+
# trackers) does not leak sensitive return values.
|
|
47
|
+
#
|
|
48
|
+
# @return [String]
|
|
45
49
|
def inspect
|
|
46
50
|
parts = ["executed=#{@executed}"]
|
|
47
51
|
parts << "skipped=#{@skipped}" if @skipped
|
|
48
|
-
parts << "value=#{@value.
|
|
52
|
+
parts << "value=#{@value.class}" unless @value.nil?
|
|
49
53
|
parts << "error=#{@error.class}" if @error
|
|
50
54
|
"#<IdempotencyLock::Result #{parts.join(" ")}>"
|
|
51
55
|
end
|
data/lib/idempotency_lock.rb
CHANGED
|
@@ -1,14 +1,18 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require "active_record"
|
|
4
|
+
require "active_support/time"
|
|
5
|
+
|
|
3
6
|
require_relative "idempotency_lock/version"
|
|
4
7
|
require_relative "idempotency_lock/result"
|
|
5
|
-
require_relative "idempotency_lock/
|
|
8
|
+
require_relative "idempotency_lock/arguments"
|
|
6
9
|
|
|
7
10
|
# Database-backed idempotency locks for Ruby on Rails.
|
|
8
11
|
#
|
|
9
|
-
#
|
|
10
|
-
#
|
|
11
|
-
#
|
|
12
|
+
# Guarantees an operation runs *at most once* per lock name (or once per TTL
|
|
13
|
+
# window) across processes and machines, using a unique database index for
|
|
14
|
+
# atomicity. Supports TTL expiration, multiple error handling strategies, and
|
|
15
|
+
# clean return value handling.
|
|
12
16
|
#
|
|
13
17
|
# @example Basic usage
|
|
14
18
|
# result = IdempotencyLock.once("send-welcome-email-user-123") do
|
|
@@ -22,52 +26,108 @@ require_relative "idempotency_lock/lock"
|
|
|
22
26
|
module IdempotencyLock
|
|
23
27
|
class Error < StandardError; end
|
|
24
28
|
|
|
25
|
-
# Error handling strategies
|
|
29
|
+
# Error handling strategies for `once`
|
|
26
30
|
ON_ERROR_UNLOCK = :unlock # Remove lock so operation can be retried
|
|
27
31
|
ON_ERROR_KEEP_LOCKED = :keep # Keep lock in place (default)
|
|
28
|
-
ON_ERROR_RAISE = :raise #
|
|
32
|
+
ON_ERROR_RAISE = :raise # Release lock, then re-raise
|
|
33
|
+
ON_ERROR_STRATEGIES = [ON_ERROR_KEEP_LOCKED, ON_ERROR_UNLOCK, ON_ERROR_RAISE].freeze
|
|
34
|
+
|
|
35
|
+
# Error handling strategies for `temporarily` (the lock is always released)
|
|
36
|
+
TEMPORARY_ON_ERROR_STRATEGIES = [nil, ON_ERROR_RAISE].freeze
|
|
37
|
+
|
|
38
|
+
# Maximum length for lock names. Matches the column limit in the generated
|
|
39
|
+
# migration and stays within unique index limits on common databases.
|
|
40
|
+
MAX_NAME_LENGTH = 255
|
|
41
|
+
|
|
42
|
+
# Interval between attempts when `temporarily` is called with `wait:`
|
|
43
|
+
WAIT_RETRY_INTERVAL = 0.1 # 100ms
|
|
44
|
+
|
|
45
|
+
# Loaded on first use so that requiring the gem does not force
|
|
46
|
+
# ActiveRecord::Base to load before the host application is configured.
|
|
47
|
+
autoload :Lock, File.expand_path("idempotency_lock/lock", __dir__)
|
|
29
48
|
|
|
30
49
|
class << self
|
|
31
|
-
# Execute a block
|
|
50
|
+
# Execute a block at most once for the given operation name.
|
|
32
51
|
#
|
|
33
|
-
#
|
|
34
|
-
#
|
|
35
|
-
#
|
|
36
|
-
# @
|
|
52
|
+
# The lock is taken *before* the block runs. If the block raises, `on_error`
|
|
53
|
+
# decides whether the lock is kept (no retry) or released (retry allowed).
|
|
54
|
+
#
|
|
55
|
+
# @param name [String, Symbol] unique identifier for this operation
|
|
56
|
+
# @param ttl [ActiveSupport::Duration, Numeric, nil] time-to-live for the lock
|
|
57
|
+
# @param on_error [Symbol, #call] :keep (default), :unlock, :raise, or a callable
|
|
58
|
+
# @yield The block to execute (at most once per lock name / TTL window)
|
|
37
59
|
# @return [Result] containing execution status and return value
|
|
38
|
-
|
|
39
|
-
|
|
60
|
+
# @raise [ArgumentError] for a missing block, invalid name, ttl, or on_error
|
|
61
|
+
def once(name, ttl: nil, on_error: ON_ERROR_KEEP_LOCKED, &block)
|
|
62
|
+
raise ArgumentError, "Block required" unless block
|
|
40
63
|
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
64
|
+
name = Arguments.normalize_name!(name)
|
|
65
|
+
Arguments.validate_ttl!(ttl)
|
|
66
|
+
Arguments.validate_on_error!(on_error, ON_ERROR_STRATEGIES)
|
|
44
67
|
|
|
45
|
-
|
|
68
|
+
token = acquire_lock(name, ttl)
|
|
69
|
+
unless token
|
|
46
70
|
log_debug("Lock already exists for '#{name}', skipping execution")
|
|
47
71
|
return Result.new(executed: false, skipped: true)
|
|
48
72
|
end
|
|
49
73
|
|
|
50
74
|
log_debug("Lock acquired for '#{name}', executing block")
|
|
51
|
-
|
|
75
|
+
run_once(name, token, on_error, &block)
|
|
52
76
|
end
|
|
53
77
|
|
|
54
78
|
# Convenience alias for `once`
|
|
55
79
|
alias wrap once
|
|
56
80
|
|
|
57
|
-
#
|
|
81
|
+
# Execute a block while holding a lock that is always released when the
|
|
82
|
+
# block finishes (success, error, or non-local exit).
|
|
83
|
+
#
|
|
84
|
+
# Unlike `once`, this is mutex-style exclusion, not idempotency.
|
|
85
|
+
#
|
|
86
|
+
# Without a `ttl:`, a lock held by a process that is killed (e.g. SIGKILL,
|
|
87
|
+
# OOM, host failure) is never released. Pass a `ttl:` longer than the
|
|
88
|
+
# block's worst-case runtime for crash protection.
|
|
89
|
+
#
|
|
90
|
+
# @param name [String, Symbol] unique identifier for this lock
|
|
91
|
+
# @param ttl [ActiveSupport::Duration, Numeric, nil] time-to-live for crash protection
|
|
92
|
+
# @param wait [ActiveSupport::Duration, Numeric, nil] how long to wait for the lock
|
|
93
|
+
# @param on_error [nil, Symbol, #call] nil (default) captures the error in the
|
|
94
|
+
# Result; :raise re-raises it; a callable is invoked with it
|
|
95
|
+
# @yield The block to execute while holding the lock
|
|
96
|
+
# @return [Result] containing execution status and return value
|
|
97
|
+
# @raise [ArgumentError] for a missing block, invalid name, ttl, wait, or on_error
|
|
98
|
+
def temporarily(name, ttl: nil, wait: nil, on_error: nil, &block)
|
|
99
|
+
raise ArgumentError, "Block required" unless block
|
|
100
|
+
|
|
101
|
+
name = Arguments.normalize_name!(name)
|
|
102
|
+
Arguments.validate_ttl!(ttl)
|
|
103
|
+
Arguments.validate_on_error!(on_error, TEMPORARY_ON_ERROR_STRATEGIES)
|
|
104
|
+
wait_seconds = Arguments.normalize_wait!(wait)
|
|
105
|
+
|
|
106
|
+
token = acquire_with_wait(name, ttl, wait_seconds)
|
|
107
|
+
unless token
|
|
108
|
+
log_debug("Lock already held for '#{name}', skipping execution")
|
|
109
|
+
return Result.new(executed: false, skipped: true)
|
|
110
|
+
end
|
|
111
|
+
|
|
112
|
+
log_debug("Temporary lock acquired for '#{name}', executing block")
|
|
113
|
+
run_temporarily(name, token, on_error, &block)
|
|
114
|
+
end
|
|
115
|
+
|
|
116
|
+
# Force-release a lock regardless of who holds it (useful for testing or
|
|
117
|
+
# manual intervention).
|
|
58
118
|
#
|
|
59
|
-
# @param name [String] the lock name to release
|
|
119
|
+
# @param name [String, Symbol] the lock name to release
|
|
60
120
|
# @return [Boolean] true if a lock was released
|
|
61
121
|
def release(name)
|
|
62
|
-
Lock.release(name)
|
|
122
|
+
Lock.release(Arguments.normalize_name!(name))
|
|
63
123
|
end
|
|
64
124
|
|
|
65
125
|
# Check if an operation is currently locked
|
|
66
126
|
#
|
|
67
|
-
# @param name [String] the lock name to check
|
|
127
|
+
# @param name [String, Symbol] the lock name to check
|
|
68
128
|
# @return [Boolean] true if locked and not expired
|
|
69
129
|
def locked?(name)
|
|
70
|
-
Lock.locked?(name)
|
|
130
|
+
Lock.locked?(Arguments.normalize_name!(name))
|
|
71
131
|
end
|
|
72
132
|
|
|
73
133
|
# Clean up all expired locks
|
|
@@ -77,67 +137,100 @@ module IdempotencyLock
|
|
|
77
137
|
Lock.cleanup_expired
|
|
78
138
|
end
|
|
79
139
|
|
|
80
|
-
#
|
|
140
|
+
# Set the logger. Set to nil to disable logging.
|
|
81
141
|
attr_writer :logger
|
|
82
142
|
|
|
143
|
+
# The configured logger, or Rails.logger if none has been set.
|
|
83
144
|
def logger
|
|
84
|
-
@logger
|
|
145
|
+
return @logger if defined?(@logger)
|
|
146
|
+
|
|
147
|
+
Rails.logger if defined?(Rails) && Rails.respond_to?(:logger)
|
|
85
148
|
end
|
|
86
149
|
|
|
87
150
|
private
|
|
88
151
|
|
|
89
|
-
def
|
|
90
|
-
|
|
152
|
+
def run_once(name, token, on_error)
|
|
153
|
+
value = yield
|
|
154
|
+
Result.new(executed: true, value: value)
|
|
155
|
+
rescue StandardError => e
|
|
156
|
+
handle_error(name, token, e, on_error)
|
|
157
|
+
rescue Exception # rubocop:disable Lint/RescueException
|
|
158
|
+
# Interrupt, SignalException, Sidekiq::Shutdown, etc. The block did not
|
|
159
|
+
# finish, so apply the release policy before propagating.
|
|
160
|
+
release_after_abort(name, token, on_error)
|
|
161
|
+
raise
|
|
162
|
+
end
|
|
163
|
+
|
|
164
|
+
def run_temporarily(name, token, on_error)
|
|
165
|
+
begin
|
|
166
|
+
value = yield
|
|
167
|
+
rescue StandardError => e
|
|
168
|
+
error = e
|
|
169
|
+
ensure
|
|
170
|
+
Lock.release(name, owner_token: token)
|
|
171
|
+
log_debug("Temporary lock released for '#{name}'")
|
|
172
|
+
end
|
|
173
|
+
return Result.new(executed: true, value: value) unless error
|
|
91
174
|
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
175
|
+
log_error("Exception in temporary lock block '#{name}': #{error.class} - #{error.message}")
|
|
176
|
+
raise error if on_error == ON_ERROR_RAISE
|
|
177
|
+
|
|
178
|
+
on_error&.call(error)
|
|
179
|
+
Result.new(executed: true, error: error)
|
|
180
|
+
end
|
|
181
|
+
|
|
182
|
+
def acquire_lock(name, ttl)
|
|
183
|
+
now = Time.current
|
|
184
|
+
Lock.acquire(name, expires_at: ttl && (now + ttl), now: now)
|
|
185
|
+
end
|
|
186
|
+
|
|
187
|
+
def acquire_with_wait(name, ttl, wait_seconds)
|
|
188
|
+
token = acquire_lock(name, ttl)
|
|
189
|
+
return token if token || wait_seconds.nil?
|
|
190
|
+
|
|
191
|
+
deadline = monotonic_now + wait_seconds
|
|
192
|
+
loop do
|
|
193
|
+
remaining = deadline - monotonic_now
|
|
194
|
+
return nil unless remaining.positive?
|
|
195
|
+
|
|
196
|
+
sleep([WAIT_RETRY_INTERVAL, remaining].min)
|
|
197
|
+
token = acquire_lock(name, ttl)
|
|
198
|
+
return token if token
|
|
98
199
|
end
|
|
99
200
|
end
|
|
100
201
|
|
|
101
|
-
def
|
|
102
|
-
|
|
103
|
-
Result.new(executed: true, value: value)
|
|
104
|
-
rescue StandardError => e
|
|
105
|
-
handle_error(name, e, on_error)
|
|
202
|
+
def monotonic_now
|
|
203
|
+
Process.clock_gettime(Process::CLOCK_MONOTONIC)
|
|
106
204
|
end
|
|
107
205
|
|
|
108
|
-
def handle_error(name, exception, on_error)
|
|
206
|
+
def handle_error(name, token, exception, on_error)
|
|
109
207
|
log_error("Exception in idempotent block '#{name}': #{exception.class} - #{exception.message}")
|
|
110
208
|
|
|
111
209
|
case on_error
|
|
112
|
-
when
|
|
113
|
-
|
|
114
|
-
when
|
|
115
|
-
|
|
116
|
-
|
|
210
|
+
when ON_ERROR_KEEP_LOCKED
|
|
211
|
+
log_debug("Lock kept for '#{name}' despite error (on_error: :keep)")
|
|
212
|
+
when ON_ERROR_UNLOCK
|
|
213
|
+
release_owned(name, token, "due to error (on_error: :unlock)")
|
|
214
|
+
when ON_ERROR_RAISE
|
|
215
|
+
release_owned(name, token, "re-raising error (on_error: :raise)")
|
|
216
|
+
raise exception
|
|
217
|
+
else
|
|
218
|
+
release_owned(name, token, "calling error handler")
|
|
219
|
+
on_error.call(exception)
|
|
117
220
|
end
|
|
118
|
-
end
|
|
119
221
|
|
|
120
|
-
def handle_unlock_error(name, exception)
|
|
121
|
-
Lock.release(name)
|
|
122
|
-
log_debug("Lock released for '#{name}' due to error (on_error: :unlock)")
|
|
123
222
|
Result.new(executed: true, error: exception)
|
|
124
223
|
end
|
|
125
224
|
|
|
126
|
-
def
|
|
127
|
-
|
|
128
|
-
log_debug("Lock released for '#{name}', re-raising error (on_error: :raise)")
|
|
129
|
-
raise exception
|
|
130
|
-
end
|
|
225
|
+
def release_after_abort(name, token, on_error)
|
|
226
|
+
return if on_error == ON_ERROR_KEEP_LOCKED
|
|
131
227
|
|
|
132
|
-
|
|
133
|
-
log_debug("Lock kept for '#{name}' despite error (on_error: :keep)")
|
|
134
|
-
Result.new(executed: true, error: exception)
|
|
228
|
+
release_owned(name, token, "after the block was aborted")
|
|
135
229
|
end
|
|
136
230
|
|
|
137
|
-
def
|
|
138
|
-
Lock.release(name)
|
|
139
|
-
|
|
140
|
-
Result.new(executed: true, error: exception)
|
|
231
|
+
def release_owned(name, token, reason)
|
|
232
|
+
Lock.release(name, owner_token: token)
|
|
233
|
+
log_debug("Lock released for '#{name}' #{reason}")
|
|
141
234
|
end
|
|
142
235
|
|
|
143
236
|
def log_debug(message)
|
data/sig/idempotency_lock.rbs
CHANGED
|
@@ -1,4 +1,56 @@
|
|
|
1
1
|
module IdempotencyLock
|
|
2
2
|
VERSION: String
|
|
3
|
-
|
|
3
|
+
|
|
4
|
+
class Error < StandardError
|
|
5
|
+
end
|
|
6
|
+
|
|
7
|
+
ON_ERROR_UNLOCK: :unlock
|
|
8
|
+
ON_ERROR_KEEP_LOCKED: :keep
|
|
9
|
+
ON_ERROR_RAISE: :raise
|
|
10
|
+
ON_ERROR_STRATEGIES: Array[Symbol]
|
|
11
|
+
TEMPORARY_ON_ERROR_STRATEGIES: Array[Symbol?]
|
|
12
|
+
MAX_NAME_LENGTH: Integer
|
|
13
|
+
WAIT_RETRY_INTERVAL: Float
|
|
14
|
+
|
|
15
|
+
interface _ErrorHandler
|
|
16
|
+
def call: (StandardError) -> untyped
|
|
17
|
+
end
|
|
18
|
+
|
|
19
|
+
type lock_name = String | Symbol
|
|
20
|
+
type duration = Numeric
|
|
21
|
+
type on_error = :keep | :unlock | :raise | _ErrorHandler
|
|
22
|
+
type temporary_on_error = :raise | _ErrorHandler | nil
|
|
23
|
+
|
|
24
|
+
def self.once: [T] (lock_name name, ?ttl: duration?, ?on_error: on_error) { () -> T } -> Result[T]
|
|
25
|
+
def self.wrap: [T] (lock_name name, ?ttl: duration?, ?on_error: on_error) { () -> T } -> Result[T]
|
|
26
|
+
def self.temporarily: [T] (lock_name name, ?ttl: duration?, ?wait: duration?, ?on_error: temporary_on_error) { () -> T } -> Result[T]
|
|
27
|
+
def self.release: (lock_name name) -> bool
|
|
28
|
+
def self.locked?: (lock_name name) -> bool
|
|
29
|
+
def self.cleanup_expired: () -> Integer
|
|
30
|
+
def self.logger: () -> untyped
|
|
31
|
+
def self.logger=: (untyped logger) -> untyped
|
|
32
|
+
|
|
33
|
+
class Result[T]
|
|
34
|
+
attr_reader value: T?
|
|
35
|
+
attr_reader error: StandardError?
|
|
36
|
+
|
|
37
|
+
def initialize: (executed: bool, ?value: T?, ?skipped: bool, ?error: StandardError?) -> void
|
|
38
|
+
def executed?: () -> bool
|
|
39
|
+
def skipped?: () -> bool
|
|
40
|
+
def error?: () -> bool
|
|
41
|
+
def success?: () -> bool
|
|
42
|
+
def inspect: () -> String
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
class Lock
|
|
46
|
+
MAX_NAME_LENGTH: Integer
|
|
47
|
+
|
|
48
|
+
def self.acquire: (String name, ?expires_at: Time?, ?now: Time) -> String?
|
|
49
|
+
def self.release: (String name, ?owner_token: String?) -> bool
|
|
50
|
+
def self.locked?: (String name, ?now: Time) -> bool
|
|
51
|
+
def self.cleanup_expired: (?now: Time) -> Integer
|
|
52
|
+
|
|
53
|
+
def expired?: (?now: Time) -> bool
|
|
54
|
+
def valid_lock?: (?now: Time) -> bool
|
|
55
|
+
end
|
|
4
56
|
end
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: idempotency_lock
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.3.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- John Nagro
|
|
@@ -15,32 +15,33 @@ dependencies:
|
|
|
15
15
|
requirements:
|
|
16
16
|
- - ">="
|
|
17
17
|
- !ruby/object:Gem::Version
|
|
18
|
-
version: '7.
|
|
18
|
+
version: '7.1'
|
|
19
19
|
type: :runtime
|
|
20
20
|
prerelease: false
|
|
21
21
|
version_requirements: !ruby/object:Gem::Requirement
|
|
22
22
|
requirements:
|
|
23
23
|
- - ">="
|
|
24
24
|
- !ruby/object:Gem::Version
|
|
25
|
-
version: '7.
|
|
25
|
+
version: '7.1'
|
|
26
26
|
- !ruby/object:Gem::Dependency
|
|
27
27
|
name: activesupport
|
|
28
28
|
requirement: !ruby/object:Gem::Requirement
|
|
29
29
|
requirements:
|
|
30
30
|
- - ">="
|
|
31
31
|
- !ruby/object:Gem::Version
|
|
32
|
-
version: '7.
|
|
32
|
+
version: '7.1'
|
|
33
33
|
type: :runtime
|
|
34
34
|
prerelease: false
|
|
35
35
|
version_requirements: !ruby/object:Gem::Requirement
|
|
36
36
|
requirements:
|
|
37
37
|
- - ">="
|
|
38
38
|
- !ruby/object:Gem::Version
|
|
39
|
-
version: '7.
|
|
39
|
+
version: '7.1'
|
|
40
40
|
description: |
|
|
41
|
-
A simple, robust idempotency solution for Rails applications.
|
|
42
|
-
|
|
43
|
-
|
|
41
|
+
A simple, robust idempotency solution for Rails applications. Guarantees an
|
|
42
|
+
operation runs at most once per key (or per TTL window) across processes using
|
|
43
|
+
database-backed locks, with TTL expiration, owner-safe release, multiple error
|
|
44
|
+
handling strategies, and clean return value handling.
|
|
44
45
|
email:
|
|
45
46
|
- john@noreastergroup.com
|
|
46
47
|
executables: []
|
|
@@ -53,8 +54,12 @@ files:
|
|
|
53
54
|
- README.md
|
|
54
55
|
- Rakefile
|
|
55
56
|
- lib/generators/idempotency_lock/install_generator.rb
|
|
57
|
+
- lib/generators/idempotency_lock/migration_generator_helpers.rb
|
|
58
|
+
- lib/generators/idempotency_lock/templates/add_owner_token_to_idempotency_locks.rb.erb
|
|
56
59
|
- lib/generators/idempotency_lock/templates/create_idempotency_locks.rb.erb
|
|
60
|
+
- lib/generators/idempotency_lock/upgrade_generator.rb
|
|
57
61
|
- lib/idempotency_lock.rb
|
|
62
|
+
- lib/idempotency_lock/arguments.rb
|
|
58
63
|
- lib/idempotency_lock/lock.rb
|
|
59
64
|
- lib/idempotency_lock/railtie.rb
|
|
60
65
|
- lib/idempotency_lock/result.rb
|