redis_queued_locks 0.0.39 → 0.0.40
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.rubocop.yml +3 -0
- data/CHANGELOG.md +14 -0
- data/README.md +341 -69
- data/lib/redis_queued_locks/acquier/acquire_lock/try_to_lock.rb +1 -1
- data/lib/redis_queued_locks/acquier/acquire_lock.rb +7 -6
- data/lib/redis_queued_locks/acquier/clear_dead_requests.rb +52 -0
- data/lib/redis_queued_locks/acquier/extend_lock_ttl.rb +3 -2
- data/lib/redis_queued_locks/acquier/locks.rb +4 -2
- data/lib/redis_queued_locks/acquier/queues.rb +4 -2
- data/lib/redis_queued_locks/acquier/release_all_locks.rb +13 -12
- data/lib/redis_queued_locks/acquier/release_lock.rb +24 -5
- data/lib/redis_queued_locks/acquier.rb +1 -0
- data/lib/redis_queued_locks/client.rb +83 -16
- data/lib/redis_queued_locks/version.rb +2 -2
- metadata +2 -1
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: b46bd07dab8fc9ce3228eb7bb804569a99dfc8f7f65a1eaf085c3af9061ae09e
|
4
|
+
data.tar.gz: 2c9470b5dd4b41a112c93bb891f33ff68f930dc47fd78a37516540249ded415d
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: a7e0740247300cf79385c477a66bc823621888104887de33c3868cf4d06a2087ae6b2ea972e8844fd148e8e3b0da9468cfaf276f68f5d1be7e3b80f72e0c31fe
|
7
|
+
data.tar.gz: d920cda1d4f89da24123df0cf95595c87c21e1e4ed18495765ae0f78c1714af325f94e8e88a7971e4bb10250dde4f392095e8d19f010e9a5f0341d3d3201dc0c
|
data/.rubocop.yml
CHANGED
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,19 @@
|
|
1
1
|
## [Unreleased]
|
2
2
|
|
3
|
+
## [0.0.40] - 2024-04-01
|
4
|
+
### Added
|
5
|
+
- `RedisQueuedLocks::Client#clear_dead_requests` implementation;
|
6
|
+
- Logger and instrumentation are passed everywhere where any changes in Redis (with locks and queus)
|
7
|
+
are expected;
|
8
|
+
- New config `is_timed_by_default` (boolean, `false` by default) that reflects the `timed` option of `#lock` and `#lock!` methods;
|
9
|
+
- Ther result of `#unlock` is changed: added `:lock_res` and `:queue_res` result data in order to reflect
|
10
|
+
what happened inside (`:released` or `:nothing_to_release`) and to adopt to the case when you trying
|
11
|
+
to unlock the non-existent lock;
|
12
|
+
- A lot of documentation updates;
|
13
|
+
- Github CI Workflow;
|
14
|
+
### Changed
|
15
|
+
- `:rel_key_cnt` result of `#clear_locks` is more accurate now;
|
16
|
+
|
3
17
|
## [0.0.39] - 2024-03-31
|
4
18
|
### Added
|
5
19
|
- Logging:
|
data/README.md
CHANGED
@@ -1,6 +1,6 @@
|
|
1
1
|
# RedisQueuedLocks · [![Gem Version](https://badge.fury.io/rb/redis_queued_locks.svg)](https://badge.fury.io/rb/redis_queued_locks)
|
2
2
|
|
3
|
-
Distributed locks with "lock acquisition queue" capabilities based on the Redis Database.
|
3
|
+
<a href="https://redis.io/docs/manual/patterns/distributed-locks/">Distributed locks</a> with "lock acquisition queue" capabilities based on the Redis Database.
|
4
4
|
|
5
5
|
Provides flexible invocation flow, parametrized limits (lock request ttl, lock ttls, queue ttls, fast failing, etc), logging and instrumentation.
|
6
6
|
|
@@ -32,6 +32,7 @@ Each lock request is put into the request queue (each lock is hosted by it's own
|
|
32
32
|
- [locks_info](#locks_info---get-list-of-locks-with-their-info)
|
33
33
|
- [queues_info](#queues_info---get-list-of-queues-with-their-info)
|
34
34
|
- [clear_dead_requests](#clear_dead_requests)
|
35
|
+
- [Logging](#logging)
|
35
36
|
- [Instrumentation](#instrumentation)
|
36
37
|
- [Instrumentation Events](#instrumentation-events)
|
37
38
|
- [Roadmap](#roadmap)
|
@@ -43,14 +44,19 @@ Each lock request is put into the request queue (each lock is hosted by it's own
|
|
43
44
|
|
44
45
|
### Requirements
|
45
46
|
|
47
|
+
<sup>\[[back to top](#table-of-contents)\]</sup>
|
48
|
+
|
46
49
|
- Redis Version: `~> 7.x`;
|
47
50
|
- Redis Protocol: `RESP3`;
|
48
51
|
- gem `redis-client`: `~> 0.20`;
|
52
|
+
- Ruby: `>= 3.1`;
|
49
53
|
|
50
54
|
---
|
51
55
|
|
52
56
|
### Experience
|
53
57
|
|
58
|
+
<sup>\[[back to top](#table-of-contents)\]</sup>
|
59
|
+
|
54
60
|
- Battle-tested on huge ruby projects in production: `~1500` locks-per-second are obtained and released on an ongoing basis;
|
55
61
|
- Works well with `hiredis` driver enabled (it is enabled by default on our projects where `redis_queued_locks` are used);
|
56
62
|
|
@@ -58,6 +64,8 @@ Each lock request is put into the request queue (each lock is hosted by it's own
|
|
58
64
|
|
59
65
|
### Algorithm
|
60
66
|
|
67
|
+
<sup>\[[back to top](#table-of-contents)\]</sup>
|
68
|
+
|
61
69
|
> Each lock request is put into the request queue (each lock is hosted by it's own queue separately from other queues) and processed in order of their priority (FIFO). Each lock request lives some period of time (RTTL) which guarantees that the request queue will never be stacked.
|
62
70
|
|
63
71
|
**Soon**: detailed explanation.
|
@@ -66,6 +74,8 @@ Each lock request is put into the request queue (each lock is hosted by it's own
|
|
66
74
|
|
67
75
|
### Installation
|
68
76
|
|
77
|
+
<sup>\[[back to top](#table-of-contents)\]</sup>
|
78
|
+
|
69
79
|
```ruby
|
70
80
|
gem 'redis_queued_locks'
|
71
81
|
```
|
@@ -84,6 +94,8 @@ require 'redis_queued_locks'
|
|
84
94
|
|
85
95
|
### Setup
|
86
96
|
|
97
|
+
<sup>\[[back to top](#table-of-contents)\]</sup>
|
98
|
+
|
87
99
|
```ruby
|
88
100
|
require 'redis_queued_locks'
|
89
101
|
|
@@ -105,6 +117,8 @@ rq_lock_client.lock("some-lock") { puts "Hello, lock!" }
|
|
105
117
|
|
106
118
|
### Configuration
|
107
119
|
|
120
|
+
<sup>\[[back to top](#table-of-contents)\]</sup>
|
121
|
+
|
108
122
|
```ruby
|
109
123
|
redis_client = RedisClient.config.new_pool # NOTE: provide your own RedisClient instance
|
110
124
|
|
@@ -131,6 +145,10 @@ clinet = RedisQueuedLocks::Client.new(redis_client) do |config|
|
|
131
145
|
# - lock request timeout. after this timeout your lock request in queue will be requeued with new position (at the end of the queue);
|
132
146
|
config.default_queue_ttl = 15
|
133
147
|
|
148
|
+
# (boolean) (default: false)
|
149
|
+
# - should be all blocks of code are timed by default;
|
150
|
+
config.is_timed_by_default = false
|
151
|
+
|
134
152
|
# (default: 100)
|
135
153
|
# - how many items will be released at a time in RedisQueuedLocks::Client#clear_locks logic (uses SCAN);
|
136
154
|
# - affects the performancs of your Redis and Ruby Application (configure thoughtfully);
|
@@ -167,6 +185,7 @@ clinet = RedisQueuedLocks::Client.new(redis_client) do |config|
|
|
167
185
|
# - "[redis_queued_locks.dead_score_reached__reset_acquier_position]" (logs "lock_key", "queue_ttl", "acq_id");
|
168
186
|
# - "[redis_queued_locks.lock_obtained]" (logs "lockkey", "queue_ttl", "acq_id", "acq_time");
|
169
187
|
# - "[redis_queued_locks.fail_fast_or_limits_reached__dequeue] (logs "lock_key", "queue_ttl", "acq_id");
|
188
|
+
# - "[redis_queued_locks.expire_lock]" # (logs "lock_key", "queue_ttl", "acq_id");
|
170
189
|
# - by default uses VoidLogger that does nothing;
|
171
190
|
config.logger = RedisQueuedLocks::Logging::VoidLogger
|
172
191
|
|
@@ -182,7 +201,7 @@ clinet = RedisQueuedLocks::Client.new(redis_client) do |config|
|
|
182
201
|
# - "[redis_queued_locks.try_lock.exit__queue_ttl_reached]" (logs "lock_key", "queue_ttl", "acq_id");
|
183
202
|
# - "[redis_queued_locks.try_lock.exit__no_first]" (logs "lock_key", "queue_ttl", "acq_id", "first_acq_id_in_queue", "<current_lock_data>");
|
184
203
|
# - "[redis_queued_locks.try_lock.exit__lock_still_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "first_acq_id_in_queue", "locked_by_acq_id", "<current_lock_data>");
|
185
|
-
# - "[redis_queued_locks.try_lock.
|
204
|
+
# - "[redis_queued_locks.try_lock.obtain__free_to_acquire]" (logs "lock_key", "queue_ttl", "acq_id");
|
186
205
|
config.log_lock_try = false
|
187
206
|
end
|
188
207
|
```
|
@@ -191,6 +210,8 @@ end
|
|
191
210
|
|
192
211
|
### Usage
|
193
212
|
|
213
|
+
<sup>\[[back to top](#table-of-contents)\]</sup>
|
214
|
+
|
194
215
|
- [lock](#lock---obtain-a-lock)
|
195
216
|
- [lock!](#lock---exeptional-lock-obtaining)
|
196
217
|
- [lock_info](#lock_info)
|
@@ -211,6 +232,8 @@ end
|
|
211
232
|
|
212
233
|
#### #lock - obtain a lock
|
213
234
|
|
235
|
+
<sup>\[[back to top](#usage)\]</sup>
|
236
|
+
|
214
237
|
- If block is passed the obtained lock will be released after the block execution or the lock's ttl (what will happen first);
|
215
238
|
- If block is not passed the obtained lock will be released after lock's ttl;
|
216
239
|
- If block is passed the block's yield result will be returned;
|
@@ -237,48 +260,64 @@ def lock(
|
|
237
260
|
)
|
238
261
|
```
|
239
262
|
|
240
|
-
- `lock_name` - `[String]`
|
263
|
+
- `lock_name` - (required) `[String]`
|
241
264
|
- Lock name to be obtained.
|
242
|
-
- `ttl` [Integer]
|
243
|
-
- Lock's time to live (in milliseconds)
|
244
|
-
-
|
265
|
+
- `ttl` - (optional) - [Integer]
|
266
|
+
- Lock's time to live (in milliseconds);
|
267
|
+
- pre-configured in `config[:default_lock_ttl]`;
|
268
|
+
- `queue_ttl` - (optional) `[Integer]`
|
245
269
|
- Lifetime of the acuier's lock request. In seconds.
|
246
|
-
-
|
247
|
-
|
248
|
-
-
|
270
|
+
- pre-configured in `config[:default_queue_ttl]`;
|
271
|
+
- `timeout` - (optional) `[Integer,NilClass]`
|
272
|
+
- Time period a client should try to acquire the lock (in seconds). Nil means "without timeout".
|
273
|
+
- pre-configured in `config[:try_to_lock_timeout]`;
|
274
|
+
- `timed` - (optiona) `[Boolean]`
|
249
275
|
- Limit the invocation time period of the passed block of code by the lock's TTL.
|
250
|
-
-
|
276
|
+
- pre-configured in `config[:is_timed_by_default]`;
|
277
|
+
- `false` by default;
|
278
|
+
- `retry_count` - (optional) `[Integer,NilClass]`
|
251
279
|
- How many times we should try to acquire a lock. Nil means "infinite retries".
|
252
|
-
-
|
280
|
+
- pre-configured in `config[:retry_count]`;
|
281
|
+
- `retry_delay` - (optional) `[Integer]`
|
253
282
|
- A time-interval between the each retry (in milliseconds).
|
254
|
-
-
|
255
|
-
|
256
|
-
-
|
257
|
-
-
|
258
|
-
- `
|
259
|
-
-
|
260
|
-
-
|
283
|
+
- pre-configured in `config[:retry_delay]`;
|
284
|
+
- `retry_jitter` - (optional) `[Integer]`
|
285
|
+
- Time-shift range for retry-delay (in milliseconds);
|
286
|
+
- pre-configured in `config[:retry_jitter]`;
|
287
|
+
- `instrumenter` - (optional) `[#notify]`
|
288
|
+
- See RedisQueuedLocks::Instrument::ActiveSupport for example;
|
289
|
+
- See [Instrumentation](#instrumentation) section of docs;
|
290
|
+
- pre-configured in `config[:isntrumenter]` with void notifier (`RedisQueuedLocks::Instrumenter::VoidNotifier`);
|
291
|
+
- `raise_errors` - (optional) `[Boolean]`
|
292
|
+
- Raise errors on library-related limits such as timeout or retry count limit;
|
293
|
+
- `false` by default;
|
294
|
+
- `fail_fast` - (optional) `[Boolean]`
|
261
295
|
- Should the required lock to be checked before the try and exit immidietly if lock is
|
262
296
|
already obtained;
|
263
297
|
- Should the logic exit immidietly after the first try if the lock was obtained
|
264
298
|
by another process while the lock request queue was initially empty;
|
265
|
-
- `
|
299
|
+
- `false` by default;
|
300
|
+
- `identity` - (optional) `[String]`
|
266
301
|
- An unique string that is unique per `RedisQueuedLock::Client` instance. Resolves the
|
267
302
|
collisions between the same process_id/thread_id/fiber_id/ractor_id identifiers on different
|
268
303
|
pods or/and nodes of your application;
|
269
304
|
- It is calculated once during `RedisQueuedLock::Client` instantiation and stored in `@uniq_identity`
|
270
305
|
ivar (accessed via `uniq_dentity` accessor method);
|
271
|
-
-
|
306
|
+
- Identity calculator is pre-configured in `config[:uniq_identifier]`;
|
307
|
+
- `meta` - (optional) `[NilClass,Hash<String|Symbol,Any>]`
|
272
308
|
- A custom metadata wich will be passed to the lock data in addition to the existing data;
|
273
309
|
- Custom metadata can not contain reserved lock data keys (such as `lock_key`, `acq_id`, `ts`, `ini_ttl`, `rem_ttl`);
|
274
|
-
- `
|
310
|
+
- `nil` by default (means "no metadata");
|
311
|
+
- `instrument` - (optional) `[NilClass,Any]`
|
275
312
|
- Custom instrumentation data wich will be passed to the instrumenter's payload with :instrument key;
|
276
|
-
- `
|
277
|
-
|
278
|
-
-
|
279
|
-
-
|
313
|
+
- `nil` by default (means "no custom instrumentation data");
|
314
|
+
- `logger` - (optional) `[::Logger,#debug]`
|
315
|
+
- Logger object used for loggin internal mutation oeprations and opertioan results / process progress;
|
316
|
+
- pre-configured in `config[:logger]` with void logger `RedisQueuedLocks::Logging::VoidLogger`;
|
317
|
+
- `log_lock_try` - (optional) `[Boolean]`
|
280
318
|
- should be logged the each try of lock acquiring (a lot of logs can be generated depending on your retry configurations);
|
281
|
-
-
|
319
|
+
- pre-configured in `config[:log_lock_try]`;
|
320
|
+
- `false` by default;
|
282
321
|
- `block` - `[Block]`
|
283
322
|
- A block of code that should be executed after the successfully acquired lock.
|
284
323
|
- If block is **passed** the obtained lock will be released after the block execution or it's ttl (what will happen first);
|
@@ -286,8 +325,25 @@ def lock(
|
|
286
325
|
|
287
326
|
Return value:
|
288
327
|
|
289
|
-
- If block is passed the block's yield result will be returned
|
290
|
-
|
328
|
+
- If block is passed the block's yield result will be returned:
|
329
|
+
```ruby
|
330
|
+
result = rql.lock("my_lock") { 1 + 1 }
|
331
|
+
result # => 2
|
332
|
+
```
|
333
|
+
- If block is not passed the lock information will be returned:
|
334
|
+
```ruby
|
335
|
+
result = rql.lock("my_lock")
|
336
|
+
result # =>
|
337
|
+
{
|
338
|
+
ok: true,
|
339
|
+
result: {
|
340
|
+
lock_key: "rql:lock:my_lock",
|
341
|
+
acq_id: "rql:acq:26672/2280/2300/2320/70ea5dbf10ea1056",
|
342
|
+
ts: 1711909612.653696,
|
343
|
+
ttl: 10000
|
344
|
+
}
|
345
|
+
}
|
346
|
+
```
|
291
347
|
- Lock information result:
|
292
348
|
- Signature: `[yield, Hash<Symbol,Boolean|Hash<Symbol,Numeric|String>>]`
|
293
349
|
- Format: `{ ok: true/false, result: <Symbol|Hash<Symbol,Hash>> }`;
|
@@ -298,11 +354,23 @@ Return value:
|
|
298
354
|
result: {
|
299
355
|
lock_key: String, # acquierd lock key ("rql:lock:your_lock_name")
|
300
356
|
acq_id: String, # acquier identifier ("process_id/thread_id/fiber_id/ractor_id/identity")
|
301
|
-
ts:
|
357
|
+
ts: Float, # time (epoch) when lock was obtained (float, Time#to_f)
|
302
358
|
ttl: Integer # lock's time to live in milliseconds (integer)
|
303
359
|
}
|
304
360
|
}
|
305
361
|
```
|
362
|
+
```ruby
|
363
|
+
# example:
|
364
|
+
{
|
365
|
+
ok: true,
|
366
|
+
result: {
|
367
|
+
lock_key: "rql:lock:my_lock",
|
368
|
+
acq_id: "rql:acq:26672/2280/2300/2320/70ea5dbf10ea1056",
|
369
|
+
ts: 1711909612.653696,
|
370
|
+
ttl: 10000
|
371
|
+
}
|
372
|
+
}
|
373
|
+
```
|
306
374
|
- for failed lock obtaining:
|
307
375
|
```ruby
|
308
376
|
{ ok: false, result: :timeout_reached }
|
@@ -312,10 +380,61 @@ Return value:
|
|
312
380
|
{ ok: false, result: :unknown }
|
313
381
|
```
|
314
382
|
|
383
|
+
Examples:
|
384
|
+
|
385
|
+
- obtain a lock:
|
386
|
+
|
387
|
+
```ruby
|
388
|
+
rql.lock("my_lock") { print "Hello!" }
|
389
|
+
```
|
390
|
+
|
391
|
+
- obtain a lock with custom lock TTL:
|
392
|
+
|
393
|
+
```ruby
|
394
|
+
rql.lock("my_lock", ttl: 5_000) { print "Hello!" } # for 5 seconds
|
395
|
+
```
|
396
|
+
|
397
|
+
- obtain a lock and limit the passed block of code TTL with lock's TTL:
|
398
|
+
|
399
|
+
```ruby
|
400
|
+
rql.lock("my_lock", ttl: 5_000, timed: true) { sleep(4) }
|
401
|
+
# => OK
|
402
|
+
|
403
|
+
rql.lock("my_lock", ttl: 5_000, timed: true) { sleep(6) }
|
404
|
+
# => fails with RedisQueuedLocks::TimedLockTimeoutError
|
405
|
+
```
|
406
|
+
|
407
|
+
- infinite lock obtaining (no retry limit, no timeout limit):
|
408
|
+
|
409
|
+
```ruby
|
410
|
+
rql.lock("my_lock", retry_count: nil, timeout: nil)
|
411
|
+
```
|
412
|
+
|
413
|
+
- try to obtain with a custom waiting timeout:
|
414
|
+
|
415
|
+
```ruby
|
416
|
+
# First Ruby Process:
|
417
|
+
rql.lock("my_lock", ttl: 5_000) { sleep(4) } # acquire a long living lock
|
418
|
+
|
419
|
+
# Another Ruby Process:
|
420
|
+
rql.lock("my_lock", timeout: 2) # try to acquire but wait for a 2 seconds maximum
|
421
|
+
# =>
|
422
|
+
{ ok: false, result: :timeout_reached }
|
423
|
+
```
|
424
|
+
|
425
|
+
- obtain a lock and immediatly continue working (the lock will live in the background in Redis with the passed ttl)
|
426
|
+
|
427
|
+
```ruby
|
428
|
+
rql.lock("my_lock", ttl: 6_500) # blocks execution until the lock is obtained
|
429
|
+
puts "Let's go" # will be called immediately after the lock is obtained
|
430
|
+
```
|
431
|
+
|
315
432
|
---
|
316
433
|
|
317
434
|
#### #lock! - exceptional lock obtaining
|
318
435
|
|
436
|
+
<sup>\[[back to top](#usage)\]</sup>
|
437
|
+
|
319
438
|
- fails when (and with):
|
320
439
|
- (`RedisQueuedLocks::LockAlreadyObtainedError`) when `fail_fast` is `true` and lock is already obtained;
|
321
440
|
- (`RedisQueuedLocks::LockAcquiermentTimeoutError`) `timeout` limit reached before lock is obtained;
|
@@ -346,6 +465,8 @@ See `#lock` method [documentation](#lock---obtain-a-lock).
|
|
346
465
|
|
347
466
|
#### #lock_info
|
348
467
|
|
468
|
+
<sup>\[[back to top](#usage)\]</sup>
|
469
|
+
|
349
470
|
- get the lock information;
|
350
471
|
- returns `nil` if lock does not exist;
|
351
472
|
- lock data (`Hash<String,String|Integer>`):
|
@@ -391,6 +512,14 @@ rql.lock_info("your_lock_name")
|
|
391
512
|
|
392
513
|
#### #queue_info
|
393
514
|
|
515
|
+
<sup>\[[back to top](#usage)\]</sup>
|
516
|
+
|
517
|
+
Returns an information about the required lock queue by the lock name. The result
|
518
|
+
represnts the ordered lock request queue that is ordered by score (Redis Sets) and shows
|
519
|
+
lock acquirers and their position in queue. Async nature with redis communcation can lead
|
520
|
+
the situation when the queue becomes empty during the queue data extraction. So sometimes
|
521
|
+
you can receive the lock queue info with empty queue value (an empty array).
|
522
|
+
|
394
523
|
- get the lock queue information;
|
395
524
|
- queue represents the ordered set of lock key reqests:
|
396
525
|
- set is ordered by score in ASC manner (inside the Redis Set);
|
@@ -403,14 +532,6 @@ rql.lock_info("your_lock_name")
|
|
403
532
|
- `"acq_id"` - `string` - acquier identifier (process_id/thread_id/fiber_id/ractor_id/identity by default);
|
404
533
|
- `"score"` - `float`/`epoch` - time when the lock request was made (epoch);
|
405
534
|
|
406
|
-
```
|
407
|
-
| Returns an information about the required lock queue by the lock name. The result
|
408
|
-
| represnts the ordered lock request queue that is ordered by score (Redis sets) and shows
|
409
|
-
| lock acquirers and their position in queue. Async nature with redis communcation can lead
|
410
|
-
| the situation when the queue becomes empty during the queue data extraction. So sometimes
|
411
|
-
| you can receive the lock queue info with empty queue value (an empty array).
|
412
|
-
```
|
413
|
-
|
414
535
|
```ruby
|
415
536
|
rql.queue_info("your_lock_name")
|
416
537
|
|
@@ -430,6 +551,8 @@ rql.queue_info("your_lock_name")
|
|
430
551
|
|
431
552
|
#### #locked?
|
432
553
|
|
554
|
+
<sup>\[[back to top](#usage)\]</sup>
|
555
|
+
|
433
556
|
- is the lock obtaied or not?
|
434
557
|
|
435
558
|
```ruby
|
@@ -450,26 +573,47 @@ rql.queued?("your_lock_name") # => true/false
|
|
450
573
|
|
451
574
|
#### #unlock - release a lock
|
452
575
|
|
576
|
+
<sup>\[[back to top](#usage)\]</sup>
|
577
|
+
|
453
578
|
- release the concrete lock with lock request queue;
|
454
579
|
- queue will be relased first;
|
455
|
-
|
456
|
-
|
457
|
-
|
458
|
-
|
459
|
-
|
460
|
-
- `
|
461
|
-
|
580
|
+
- accepts:
|
581
|
+
- `lock_name` - (required) `[String]` - the lock name that should be released.
|
582
|
+
- `:logger` - (optional) `[::Logger,#debug]`
|
583
|
+
- custom logger object;
|
584
|
+
- pre-configured in `config[:logger]`;
|
585
|
+
- `:instrumenter` - (optional) `[#notify]`
|
586
|
+
- custom instrumenter object;
|
587
|
+
- pre-configured in `config[:instrumetner]`;
|
588
|
+
- `:instrument` - (optional) `[NilClass,Any]`;
|
589
|
+
- custom instrumentation data wich will be passed to the instrumenter's payload with :instrument key;
|
590
|
+
- `nil` by default (no additional data);
|
591
|
+
- if you try to unlock non-existent lock you will receive `ok: true` result with operation timings
|
592
|
+
and `:nothing_to_release` result factor inside;
|
462
593
|
|
463
594
|
Return:
|
464
|
-
- `[Hash<Symbol,Numeric|String
|
595
|
+
- `[Hash<Symbol,Boolean|Hash<Symbol,Numeric|String|Symbol>>]` (`{ ok: true/false, result: Hasn }`);
|
596
|
+
- `:result` format;
|
597
|
+
- `:rel_time` - `Float` - time spent to process redis commands (in seconds);
|
598
|
+
- `:rel_key` - `String` - released lock key (RedisQueudLocks-internal lock key name from Redis);
|
599
|
+
- `:rel_queue` - `String` - released lock queue key (RedisQueuedLocks-internal queue key name from Redis);
|
600
|
+
- `:queue_res` - `Symbol` - `:released` (or `:nothing_to_release` if the required queue does not exist);
|
601
|
+
- `:lock_res` - `Symbol` - `:released` (or `:nothing_to_release` if the required lock does not exist);
|
602
|
+
|
603
|
+
Consider that `lock_res` and `queue_res` can have different value because of the async nature of invoked Redis'es commands.
|
465
604
|
|
466
605
|
```ruby
|
606
|
+
rql.unlock("your_lock_name")
|
607
|
+
|
608
|
+
# =>
|
467
609
|
{
|
468
610
|
ok: true,
|
469
611
|
result: {
|
470
612
|
rel_time: 0.02, # time spent to lock release (in seconds)
|
471
613
|
rel_key: "rql:lock:your_lock_name", # released lock key
|
472
614
|
rel_queue: "rql:lock_queue:your_lock_name" # released lock key queue
|
615
|
+
queue_res: :released, # or :nothing_to_release
|
616
|
+
lock_res: :released # or :nothing_to_release
|
473
617
|
}
|
474
618
|
}
|
475
619
|
```
|
@@ -478,25 +622,38 @@ Return:
|
|
478
622
|
|
479
623
|
#### #clear_locks - release all locks and lock queues
|
480
624
|
|
625
|
+
<sup>\[[back to top](#usage)\]</sup>
|
626
|
+
|
481
627
|
- release all obtained locks and related lock request queues;
|
482
628
|
- queues will be released first;
|
629
|
+
- accepts:
|
630
|
+
- `:batch_size` - (optional) `[Integer]`
|
631
|
+
- the size of batch of locks and lock queus that should be cleared under the one pipelined redis command at once;
|
632
|
+
- pre-configured in `config[:lock_release_batch_size]`;
|
633
|
+
- `:logger` - (optional) `[::Logger,#debug]`
|
634
|
+
- custom logger object;
|
635
|
+
- has a preconfigured value in `config[:logger]`;
|
636
|
+
- `:instrumenter` - (optional) `[#notify]`
|
637
|
+
- custom instrumenter object;
|
638
|
+
- has a preconfigured value in `config[:isntrumenter]`;
|
639
|
+
- `:instrument` - (optional) `[NilClass,Any]`
|
640
|
+
- custom instrumentation data wich will be passed to the instrumenter's payload with :instrument key;
|
483
641
|
|
484
|
-
|
485
|
-
|
486
|
-
|
487
|
-
|
488
|
-
- `
|
489
|
-
- the size of batch of locks and lock queus that should be cleared under the one pipelined redis command at once;
|
490
|
-
|
491
|
-
Return:
|
492
|
-
- `[Hash<Symbol,Numeric>]` - Format: `{ ok: true/false, result: Hash<Symbol,Numeric> }`;
|
642
|
+
- returns:
|
643
|
+
- `[Hash<Symbol,Numeric>]` - Format: `{ ok: true, result: Hash<Symbol,Numeric> }`;
|
644
|
+
- result data:
|
645
|
+
- `:rel_time` - `Numeric` - time spent to release all locks and related queus;
|
646
|
+
- `:rel_key_cnt` - `Integer` - the number of released Redis keys (queues+locks);
|
493
647
|
|
494
648
|
```ruby
|
649
|
+
rql.clear_locks
|
650
|
+
|
651
|
+
# =>
|
495
652
|
{
|
496
653
|
ok: true,
|
497
654
|
result: {
|
498
|
-
rel_time: 3.07,
|
499
|
-
rel_key_cnt:
|
655
|
+
rel_time: 3.07,
|
656
|
+
rel_key_cnt: 1234
|
500
657
|
}
|
501
658
|
}
|
502
659
|
```
|
@@ -505,7 +662,18 @@ Return:
|
|
505
662
|
|
506
663
|
#### #extend_lock_ttl
|
507
664
|
|
508
|
-
|
665
|
+
<sup>\[[back to top](#usage)\]</sup>
|
666
|
+
|
667
|
+
- extends lock ttl by the required number of milliseconds;
|
668
|
+
- expects the lock name and the number of milliseconds;
|
669
|
+
- accepts:
|
670
|
+
- `lock_name` - (required) `[String]`
|
671
|
+
- the lock name which ttl should be extended;
|
672
|
+
- `milliseconds` - (required) `[Integer]`
|
673
|
+
- how many milliseconds should be added to the lock's TTL;
|
674
|
+
- `:logger` - (optional) `[::Logger,#debug]`
|
675
|
+
- custom logger object;
|
676
|
+
- pre-configured in `config[:logger]`;
|
509
677
|
- returns `{ ok: true, result: :ttl_extended }` when ttl is extended;
|
510
678
|
- returns `{ ok: false, result: :async_expire_or_no_lock }` when lock not found or lock is expired during
|
511
679
|
some steps of invocation (see **Important** section below);
|
@@ -535,6 +703,8 @@ rql.extend_lock_ttl("my_lock", 5_000) # NOTE: add 5_000 milliseconds
|
|
535
703
|
|
536
704
|
#### #locks - get list of obtained locks
|
537
705
|
|
706
|
+
<sup>\[[back to top](#usage)\]</sup>
|
707
|
+
|
538
708
|
- uses redis `SCAN` under the hood;
|
539
709
|
- accepts:
|
540
710
|
- `:scan_size` - `Integer` - (`config[:key_extraction_batch_size]` by default);
|
@@ -565,6 +735,8 @@ rql.locks # or rql.locks(scan_size: 123)
|
|
565
735
|
|
566
736
|
#### #queues - get list of lock request queues
|
567
737
|
|
738
|
+
<sup>\[[back to top](#usage)\]</sup>
|
739
|
+
|
568
740
|
- uses redis `SCAN` under the hood;
|
569
741
|
- accepts
|
570
742
|
- `:scan_size` - `Integer` - (`config[:key_extraction_batch_size]` by default);
|
@@ -595,6 +767,8 @@ rql.queues # or rql.queues(scan_size: 123)
|
|
595
767
|
|
596
768
|
#### #keys - get list of taken locks and queues
|
597
769
|
|
770
|
+
<sup>\[[back to top](#usage)\]</sup>
|
771
|
+
|
598
772
|
- uses redis `SCAN` under the hood;
|
599
773
|
- accepts:
|
600
774
|
`:scan_size` - `Integer` - (`config[:key_extraction_batch_size]` by default);
|
@@ -626,6 +800,8 @@ rql.keys # or rql.keys(scan_size: 123)
|
|
626
800
|
|
627
801
|
#### #locks_info - get list of locks with their info
|
628
802
|
|
803
|
+
<sup>\[[back to top](#usage)\]</sup>
|
804
|
+
|
629
805
|
- uses redis `SCAN` under the hod;
|
630
806
|
- accepts `scan_size:`/`Integer` option (`config[:key_extraction_batch_size]` by default);
|
631
807
|
- returns `Set<Hash<Symbol,Any>>` (see [#lock_info](#lock_info) and examples below for details).
|
@@ -658,6 +834,8 @@ rql.locks_info # or rql.locks_info(scan_size: 123)
|
|
658
834
|
|
659
835
|
#### #queues_info - get list of queues with their info
|
660
836
|
|
837
|
+
<sup>\[[back to top](#usage)\]</sup>
|
838
|
+
|
661
839
|
- uses redis `SCAN` under the hod;
|
662
840
|
- accepts `scan_size:`/`Integer` option (`config[:key_extraction_batch_size]` by default);
|
663
841
|
- returns `Set<Hash<Symbol,Any>>` (see [#queue_info](#queue_info) and examples below for details).
|
@@ -685,12 +863,91 @@ rql.queues_info # or rql.qeuues_info(scan_size: 123)
|
|
685
863
|
|
686
864
|
#### #clear_dead_requests
|
687
865
|
|
688
|
-
|
866
|
+
<sup>\[[back to top](#usage)\]</sup>
|
867
|
+
|
868
|
+
In some cases your lock requests may become "dead". It can happen when your processs
|
869
|
+
that are enqueeud to the lock queue is failed unexpectedly (for some reason) before the lock acquire moment
|
870
|
+
and when no any other process does not need this lock anymore. For this case your lock will be cleared only when any process
|
871
|
+
will try to acquire this lock again (cuz lock acquirement triggers the removement of expired requests).
|
872
|
+
|
873
|
+
In order to help with these dead requests you may periodically call `#clear_dead_requests`
|
874
|
+
with corresponding `dead_ttl` option, that is pre-configured by default via `config[:dead_request_ttl]`.
|
875
|
+
|
876
|
+
An option is required because of it is no any **fast** way to understand which request
|
877
|
+
is dead now and is it really dead cuz each request queue can host their requests with
|
878
|
+
a custom queue ttl for each request differently.
|
879
|
+
|
880
|
+
Accepts:
|
881
|
+
- `:dead_ttl` - (optional) `[Integer]`
|
882
|
+
- lock request ttl after which a lock request is considered dead;
|
883
|
+
- has a preconfigured value in `config[:dead_request_ttl]` (1 day by default);
|
884
|
+
- `:sacn_size` - (optional) `[Integer]`
|
885
|
+
- the batch of scanned keys for Redis'es SCAN command;
|
886
|
+
- has a preconfigured valie in `config[:key_extraction_batch_size]`;
|
887
|
+
- `:logger` - (optional) `[::Logger,#debug]`
|
888
|
+
- custom logger object;
|
889
|
+
- pre-configured in `config[:logger]`;
|
890
|
+
- `:instrumenter` - (optional) `[#notify]`
|
891
|
+
- custom instrumenter object;
|
892
|
+
- pre-configured in `config[:isntrumenter]`;
|
893
|
+
- `:instrument` - (optional) `[NilClass,Any]`
|
894
|
+
- custom instrumentation data wich will be passed to the instrumenter's payload with :instrument key;
|
895
|
+
- `nil` by default (no additional data);
|
896
|
+
|
897
|
+
Returns: `{ ok: true, processed_queues: Set<String> }` returns the list of processed lock queues;
|
898
|
+
|
899
|
+
```ruby
|
900
|
+
rql.clear_dead_requests(dead_ttl: 60 * 60 * 1000) # 1 hour in milliseconds
|
901
|
+
|
902
|
+
# =>
|
903
|
+
{
|
904
|
+
ok: true,
|
905
|
+
processed_queues: [
|
906
|
+
"rql:lock_queue:some-lock-123",
|
907
|
+
"rql:lock_queue:some-lock-456",
|
908
|
+
"rql:lock_queue:your-other-lock",
|
909
|
+
...
|
910
|
+
]
|
911
|
+
}
|
912
|
+
```
|
913
|
+
|
914
|
+
---
|
915
|
+
|
916
|
+
## Logging
|
917
|
+
|
918
|
+
<sup>\[[back to top](#table-of-contents)\]</sup>
|
919
|
+
|
920
|
+
- default logs (raised from `#lock`/`#lock!`):
|
921
|
+
|
922
|
+
```ruby
|
923
|
+
"[redis_queued_locks.start_lock_obtaining]" # (logs "lock_key", "queue_ttl", "acq_id");
|
924
|
+
"[redis_queued_locks.start_try_to_lock_cycle]" # (logs "lock_key", "queue_ttl", "acq_id");
|
925
|
+
"[redis_queued_locks.dead_score_reached__reset_acquier_position]" # (logs "lock_key", "queue_ttl", "acq_id");
|
926
|
+
"[redis_queued_locks.lock_obtained]" # (logs "lockkey", "queue_ttl", "acq_id", "acq_time");
|
927
|
+
"[redis_queued_locks.fail_fast_or_limits_reached__dequeue]" # (logs "lock_key", "queue_ttl", "acq_id");
|
928
|
+
"[redis_queued_locks.expire_lock]" # (logs "lock_key", "queue_ttl", "acq_id");
|
929
|
+
```
|
930
|
+
|
931
|
+
- additional logs (raised from `#lock`/`#lock!` with `confg[:log_lock_try] == true`):
|
932
|
+
|
933
|
+
```ruby
|
934
|
+
"[redis_queued_locks.try_lock.start]" # (logs "lock_key", "queue_ttl", "acq_id");
|
935
|
+
"[redis_queued_locks.try_lock.rconn_fetched]" # (logs "lock_key", "queue_ttl", "acq_id");
|
936
|
+
"[redis_queued_locks.try_lock.acq_added_to_queue]" # (logs "lock_key", "queue_ttl", "acq_id)";
|
937
|
+
"[redis_queued_locks.try_lock.remove_expired_acqs]" # (logs "lock_key", "queue_ttl", "acq_id");
|
938
|
+
"[redis_queued_locks.try_lock.get_first_from_queue]" # (logs "lock_key", "queue_ttl", "acq_id", "first_acq_id_in_queue");
|
939
|
+
"[redis_queued_locks.try_lock.exit__queue_ttl_reached]" # (logs "lock_key", "queue_ttl", "acq_id");
|
940
|
+
"[redis_queued_locks.try_lock.exit__no_first]" # (logs "lock_key", "queue_ttl", "acq_id", "first_acq_id_in_queue", "<current_lock_data>");
|
941
|
+
"[redis_queued_locks.try_lock.exit__lock_still_obtained]" # (logs "lock_key", "queue_ttl", "acq_id", "first_acq_id_in_queue", "locked_by_acq_id", "<current_lock_data>");
|
942
|
+
"[redis_queued_locks.try_lock.obtain__free_to_acquire]" # (logs "lock_key", "queue_ttl", "acq_id");
|
943
|
+
```
|
689
944
|
|
690
945
|
---
|
691
946
|
|
692
947
|
## Instrumentation
|
693
948
|
|
949
|
+
<sup>\[[back to top](#table-of-contents)\]</sup>
|
950
|
+
|
694
951
|
- [Instrumentation Events](#instrumentation-events)
|
695
952
|
|
696
953
|
An instrumentation layer is incapsulated in `instrumenter` object stored in [config](#configuration) (`RedisQueuedLocks::Client#config[:instrumenter]`).
|
@@ -702,8 +959,8 @@ Instrumenter object should provide `notify(event, payload)` method with the foll
|
|
702
959
|
|
703
960
|
`redis_queued_locks` provides two instrumenters:
|
704
961
|
|
705
|
-
- `RedisQueuedLocks::Instrument::ActiveSupport` -
|
706
|
-
that instrument events via
|
962
|
+
- `RedisQueuedLocks::Instrument::ActiveSupport` - **ActiveSupport::Notifications** instrumenter
|
963
|
+
that instrument events via **ActiveSupport::Notifications** API;
|
707
964
|
- `RedisQueuedLocks::Instrument::VoidNotifier` - instrumenter that does nothing;
|
708
965
|
|
709
966
|
By default `RedisQueuedLocks::Client` is configured with the void notifier (which means "instrumentation is disabled").
|
@@ -712,17 +969,20 @@ By default `RedisQueuedLocks::Client` is configured with the void notifier (whic
|
|
712
969
|
|
713
970
|
### Instrumentation Events
|
714
971
|
|
972
|
+
<sup>\[[back to top](#instrumentation-events)\]</sup>
|
973
|
+
|
715
974
|
List of instrumentation events
|
716
975
|
|
717
|
-
- `redis_queued_locks.lock_obtained
|
718
|
-
- `redis_queued_locks.lock_hold_and_release
|
719
|
-
- `redis_queued_locks.explicit_lock_release
|
720
|
-
- `redis_queued_locks.explicit_all_locks_release
|
976
|
+
- `redis_queued_locks.lock_obtained`;
|
977
|
+
- `redis_queued_locks.lock_hold_and_release`;
|
978
|
+
- `redis_queued_locks.explicit_lock_release`;
|
979
|
+
- `redis_queued_locks.explicit_all_locks_release`;
|
721
980
|
|
722
981
|
Detalized event semantics and payload structure:
|
723
982
|
|
724
983
|
- `"redis_queued_locks.lock_obtained"`
|
725
984
|
- a moment when the lock was obtained;
|
985
|
+
- raised from `#lock`/`#lock!`;
|
726
986
|
- payload:
|
727
987
|
- `:ttl` - `integer`/`milliseconds` - lock ttl;
|
728
988
|
- `:acq_id` - `string` - lock acquier identifier;
|
@@ -730,9 +990,10 @@ Detalized event semantics and payload structure:
|
|
730
990
|
- `:ts` - `integer`/`epoch` - the time when the lock was obtaiend;
|
731
991
|
- `:acq_time` - `float`/`milliseconds` - time spent on lock acquiring;
|
732
992
|
- `:instrument` - `nil`/`Any` - custom data passed to the `lock`/`lock!` method as `:instrument` attribute;
|
993
|
+
|
733
994
|
- `"redis_queued_locks.lock_hold_and_release"`
|
734
|
-
- an event signalizes about the "hold+and+release" process
|
735
|
-
|
995
|
+
- an event signalizes about the "hold+and+release" process is finished;
|
996
|
+
- raised from `#lock`/`#lock!` when invoked with a block of code;
|
736
997
|
- payload:
|
737
998
|
- `:hold_time` - `float`/`milliseconds` - lock hold time;
|
738
999
|
- `:ttl` - `integer`/`milliseconds` - lock ttl;
|
@@ -741,15 +1002,19 @@ Detalized event semantics and payload structure:
|
|
741
1002
|
- `:ts` - `integer`/`epoch` - the time when lock was obtained;
|
742
1003
|
- `:acq_time` - `float`/`milliseconds` - time spent on lock acquiring;
|
743
1004
|
- `:instrument` - `nil`/`Any` - custom data passed to the `lock`/`lock!` method as `:instrument` attribute;
|
1005
|
+
|
744
1006
|
- `"redis_queued_locks.explicit_lock_release"`
|
745
1007
|
- an event signalizes about the explicit lock release (invoked via `RedisQueuedLock#unlock`);
|
1008
|
+
- raised from `#unlock`;
|
746
1009
|
- payload:
|
747
1010
|
- `:at` - `float`/`epoch` - the time when the lock was released;
|
748
1011
|
- `:rel_time` - `float`/`milliseconds` - time spent on lock releasing;
|
749
1012
|
- `:lock_key` - `string` - released lock (lock name);
|
750
1013
|
- `:lock_key_queue` - `string` - released lock queue (lock queue name);
|
1014
|
+
|
751
1015
|
- `"redis_queued_locks.explicit_all_locks_release"`
|
752
1016
|
- an event signalizes about the explicit all locks release (invoked via `RedisQueuedLock#clear_locks`);
|
1017
|
+
- raised from `#clear_locks`;
|
753
1018
|
- payload:
|
754
1019
|
- `:rel_time` - `float`/`milliseconds` - time spent on "realese all locks" operation;
|
755
1020
|
- `:at` - `float`/`epoch` - the time when the operation has ended;
|
@@ -759,8 +1024,10 @@ Detalized event semantics and payload structure:
|
|
759
1024
|
|
760
1025
|
## Roadmap
|
761
1026
|
|
1027
|
+
<sup>\[[back to top](#table-of-contents)\]</sup>
|
1028
|
+
|
762
1029
|
- Semantic Error objects for unexpected Redis errors;
|
763
|
-
- better specs
|
1030
|
+
- better specs with 100% test coverage;
|
764
1031
|
- per-block-holding-the-lock sidecar `Ractor` and `in progress queue` in RedisDB that will extend
|
765
1032
|
the acquired lock for long-running blocks of code (that invoked "under" the lock
|
766
1033
|
whose ttl may expire before the block execution completes). It only makes sense for non-`timed` locks;
|
@@ -769,14 +1036,15 @@ Detalized event semantics and payload structure:
|
|
769
1036
|
- structured logging (separated docs);
|
770
1037
|
- GitHub Actions CI;
|
771
1038
|
- `RedisQueuedLocks::Acquier::Try.try_to_lock` - detailed successful result analization;
|
772
|
-
- better code stylization
|
773
|
-
- dead requests cleanup;
|
1039
|
+
- better code stylization (+ some refactorings);
|
774
1040
|
- statistics with UI;
|
775
1041
|
|
776
1042
|
---
|
777
1043
|
|
778
1044
|
## Contributing
|
779
1045
|
|
1046
|
+
<sup>\[[back to top](#table-of-contents)\]</sup>
|
1047
|
+
|
780
1048
|
- Fork it ( https://github.com/0exp/redis_queued_locks )
|
781
1049
|
- Create your feature branch (`git checkout -b feature/my-new-feature`)
|
782
1050
|
- Commit your changes (`git commit -am '[feature_context] Add some feature'`)
|
@@ -785,8 +1053,12 @@ Detalized event semantics and payload structure:
|
|
785
1053
|
|
786
1054
|
## License
|
787
1055
|
|
1056
|
+
<sup>\[[back to top](#table-of-contents)\]</sup>
|
1057
|
+
|
788
1058
|
Released under MIT License.
|
789
1059
|
|
790
1060
|
## Authors
|
791
1061
|
|
1062
|
+
<sup>\[[back to top](#table-of-contents)\]</sup>
|
1063
|
+
|
792
1064
|
[Rustam Ibragimov](https://github.com/0exp)
|
@@ -200,7 +200,7 @@ module RedisQueuedLocks::Acquier::AcquireLock::TryToLock
|
|
200
200
|
if log_lock_try
|
201
201
|
run_non_critical do
|
202
202
|
logger.debug do
|
203
|
-
"[redis_queued_locks.try_lock.
|
203
|
+
"[redis_queued_locks.try_lock.obtain__free_to_acquire] " \
|
204
204
|
"lock_key => '#{lock_key}' " \
|
205
205
|
"queue_ttl => #{queue_ttl} " \
|
206
206
|
"acq_id => '#{acquier_id}'"
|
@@ -23,11 +23,11 @@ module RedisQueuedLocks::Acquier::AcquireLock
|
|
23
23
|
# @since 0.1.0
|
24
24
|
extend RedisQueuedLocks::Utilities
|
25
25
|
|
26
|
-
# @return [Integer] Redis
|
26
|
+
# @return [Integer] Redis time error (in milliseconds).
|
27
27
|
#
|
28
28
|
# @api private
|
29
29
|
# @since 0.1.0
|
30
|
-
|
30
|
+
REDIS_TIMESHIFT_ERROR = 2
|
31
31
|
|
32
32
|
class << self
|
33
33
|
# @param redis [RedisClient]
|
@@ -146,9 +146,6 @@ module RedisQueuedLocks::Acquier::AcquireLock
|
|
146
146
|
ractor_id,
|
147
147
|
identity
|
148
148
|
)
|
149
|
-
# NOTE:
|
150
|
-
# - think aobut the redis expiration error
|
151
|
-
# - (ttl - REDIS_EXPIRE_ERROR).yield_self { |val| (val == 0) ? ttl : val }
|
152
149
|
lock_ttl = ttl
|
153
150
|
lock_key = RedisQueuedLocks::Resource.prepare_lock_key(lock_name)
|
154
151
|
lock_key_queue = RedisQueuedLocks::Resource.prepare_lock_queue(lock_name)
|
@@ -324,7 +321,11 @@ module RedisQueuedLocks::Acquier::AcquireLock
|
|
324
321
|
if block_given?
|
325
322
|
begin
|
326
323
|
yield_time = ::Process.clock_gettime(::Process::CLOCK_MONOTONIC)
|
327
|
-
|
324
|
+
|
325
|
+
ttl_shift = (
|
326
|
+
(yield_time - acq_process[:acq_end_time]) * 1000 - REDIS_TIMESHIFT_ERROR
|
327
|
+
).ceil(2)
|
328
|
+
|
328
329
|
yield_with_expire(
|
329
330
|
redis,
|
330
331
|
logger,
|
@@ -0,0 +1,52 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
# @api private
|
4
|
+
# @since 0.1.0
|
5
|
+
module RedisQueuedLocks::Acquier::ClearDeadRequests
|
6
|
+
class << self
|
7
|
+
# @param redis_client [RedisClient]
|
8
|
+
# @param scan_size [Integer]
|
9
|
+
# @param dead_ttl [Integer] In milliseconds
|
10
|
+
# @param logger [::Logger,#debug]
|
11
|
+
# @param instrumenter [#notify]
|
12
|
+
# @param instrument [NilClass,Any]
|
13
|
+
# @return [Hash<Symbol,Boolean|Hash<Symbol,Set<String>>>]
|
14
|
+
#
|
15
|
+
# @api private
|
16
|
+
# @since 0.1.0
|
17
|
+
def clear_dead_requests(redis_client, scan_size, dead_ttl, logger, instrumenter, instrument)
|
18
|
+
dead_score = RedisQueuedLocks::Resource.acquier_dead_score(dead_ttl / 1000.0)
|
19
|
+
|
20
|
+
result = Set.new.tap do |processed_queues|
|
21
|
+
redis_client.with do |rconn|
|
22
|
+
each_lock_queue(rconn, scan_size) do |lock_queue|
|
23
|
+
rconn.call('ZREMRANGEBYSCORE', lock_queue, '-inf', dead_score)
|
24
|
+
processed_queues << lock_queue
|
25
|
+
end
|
26
|
+
end
|
27
|
+
end
|
28
|
+
|
29
|
+
RedisQueuedLocks::Data[ok: true, result: { processed_queues: result }]
|
30
|
+
end
|
31
|
+
|
32
|
+
private
|
33
|
+
|
34
|
+
# @param redis_client [RedisClient]
|
35
|
+
# @param scan_size [Integer]
|
36
|
+
# @yield [lock_queue]
|
37
|
+
# @yieldparam lock_queue [String]
|
38
|
+
# @yieldreturn [void]
|
39
|
+
# @return [Enumerator]
|
40
|
+
#
|
41
|
+
# @api private
|
42
|
+
# @since 0.1.0
|
43
|
+
def each_lock_queue(redis_client, scan_size, &block)
|
44
|
+
redis_client.scan(
|
45
|
+
'MATCH',
|
46
|
+
RedisQueuedLocks::Resource::LOCK_QUEUE_PATTERN,
|
47
|
+
count: scan_size,
|
48
|
+
&block
|
49
|
+
)
|
50
|
+
end
|
51
|
+
end
|
52
|
+
end
|
@@ -16,14 +16,15 @@ module RedisQueuedLocks::Acquier::ExtendLockTTL
|
|
16
16
|
# @param redis_client [RedisClient]
|
17
17
|
# @param lock_name [String]
|
18
18
|
# @param milliseconds [Integer]
|
19
|
+
# @param logger [::Logger,#debug]
|
19
20
|
# @return [Hash<Symbol,Boolean|Symbol>]
|
20
21
|
#
|
21
22
|
# @api private
|
22
23
|
# @since 0.1.0
|
23
|
-
def extend_lock_ttl(redis_client, lock_name, milliseconds)
|
24
|
+
def extend_lock_ttl(redis_client, lock_name, milliseconds, logger)
|
24
25
|
lock_key = RedisQueuedLocks::Resource.prepare_lock_key(lock_name)
|
25
26
|
|
26
|
-
# NOTE: EVAL signature -> <lua script>, (keys
|
27
|
+
# NOTE: EVAL signature -> <lua script>, (number of keys), *(keys), *(arguments)
|
27
28
|
result = redis_client.call('EVAL', EXTEND_LOCK_PTTL, 1, lock_key, milliseconds)
|
28
29
|
# TODO: upload scripts to the redis
|
29
30
|
|
@@ -12,8 +12,10 @@ module RedisQueuedLocks::Acquier::Locks
|
|
12
12
|
# @api private
|
13
13
|
# @since 0.1.0
|
14
14
|
def locks(redis_client, scan_size:, with_info:)
|
15
|
-
|
16
|
-
|
15
|
+
redis_client.with do |rconn|
|
16
|
+
lock_keys = scan_locks(rconn, scan_size)
|
17
|
+
with_info ? extract_locks_info(rconn, lock_keys) : lock_keys
|
18
|
+
end
|
17
19
|
end
|
18
20
|
|
19
21
|
private
|
@@ -12,8 +12,10 @@ module RedisQueuedLocks::Acquier::Queues
|
|
12
12
|
# @api private
|
13
13
|
# @since 0.1.0
|
14
14
|
def queues(redis_client, scan_size:, with_info:)
|
15
|
-
|
16
|
-
|
15
|
+
redis_client.with do |rconn|
|
16
|
+
lock_queues = scan_queues(rconn, scan_size)
|
17
|
+
with_info ? extract_queues_info(rconn, lock_queues) : lock_queues
|
18
|
+
end
|
17
19
|
end
|
18
20
|
|
19
21
|
private
|
@@ -15,17 +15,20 @@ module RedisQueuedLocks::Acquier::ReleaseAllLocks
|
|
15
15
|
# Redis connection client.
|
16
16
|
# @param batch_size [Integer]
|
17
17
|
# The number of lock keys that should be released in a time.
|
18
|
-
# @param isntrumenter [#notify]
|
19
|
-
# See RedisQueuedLocks::Instrument::ActiveSupport for example.
|
20
18
|
# @param logger [::Logger,#debug]
|
21
19
|
# - Logger object used from `configuration` layer (see config[:logger]);
|
22
20
|
# - See RedisQueuedLocks::Logging::VoidLogger for example;
|
21
|
+
# @param isntrumenter [#notify]
|
22
|
+
# See RedisQueuedLocks::Instrument::ActiveSupport for example.
|
23
|
+
# @option instrument [NilClass,Any]
|
24
|
+
# - Custom instrumentation data wich will be passed to the instrumenter's payload
|
25
|
+
# with :instrument key;
|
23
26
|
# @return [RedisQueuedLocks::Data,Hash<Symbol,Any>]
|
24
|
-
# Format: { ok: true
|
27
|
+
# Format: { ok: true, result: Hash<Symbol,Numeric> }
|
25
28
|
#
|
26
29
|
# @api private
|
27
30
|
# @since 0.1.0
|
28
|
-
def release_all_locks(redis, batch_size, instrumenter,
|
31
|
+
def release_all_locks(redis, batch_size, logger, instrumenter, instrument)
|
29
32
|
rel_start_time = ::Process.clock_gettime(::Process::CLOCK_MONOTONIC)
|
30
33
|
fully_release_all_locks(redis, batch_size) => { ok:, result: }
|
31
34
|
time_at = Time.now.to_f
|
@@ -36,13 +39,13 @@ module RedisQueuedLocks::Acquier::ReleaseAllLocks
|
|
36
39
|
instrumenter.notify('redis_queued_locks.explicit_all_locks_release', {
|
37
40
|
at: time_at,
|
38
41
|
rel_time: rel_time,
|
39
|
-
|
42
|
+
rel_key_cnt: result[:rel_key_cnt]
|
40
43
|
})
|
41
44
|
end
|
42
45
|
|
43
46
|
RedisQueuedLocks::Data[
|
44
47
|
ok: true,
|
45
|
-
result: { rel_key_cnt: result[:
|
48
|
+
result: { rel_key_cnt: result[:rel_key_cnt], rel_time: rel_time }
|
46
49
|
]
|
47
50
|
end
|
48
51
|
|
@@ -52,7 +55,8 @@ module RedisQueuedLocks::Acquier::ReleaseAllLocks
|
|
52
55
|
#
|
53
56
|
# @param redis [RedisClient]
|
54
57
|
# @param batch_size [Integer]
|
55
|
-
# @return [RedisQueuedLocks::Data,Hash<Symbol,
|
58
|
+
# @return [RedisQueuedLocks::Data,Hash<Symbol,Boolean|Hash<Symbol,Integer>>]
|
59
|
+
# - Exmaple: { ok: true, result: { rel_key_cnt: 12345 } }
|
56
60
|
#
|
57
61
|
# @api private
|
58
62
|
# @since 0.1.0
|
@@ -66,8 +70,7 @@ module RedisQueuedLocks::Acquier::ReleaseAllLocks
|
|
66
70
|
count: batch_size
|
67
71
|
) do |lock_queue|
|
68
72
|
# TODO: reduce unnecessary iterations
|
69
|
-
pipeline.call('
|
70
|
-
pipeline.call('EXPIRE', RedisQueuedLocks::Resource.lock_key_from_queue(lock_queue), '0')
|
73
|
+
pipeline.call('EXPIRE', lock_queue, '0')
|
71
74
|
end
|
72
75
|
|
73
76
|
# Step B: release all locks
|
@@ -82,9 +85,7 @@ module RedisQueuedLocks::Acquier::ReleaseAllLocks
|
|
82
85
|
end
|
83
86
|
end
|
84
87
|
|
85
|
-
|
86
|
-
|
87
|
-
RedisQueuedLocks::Data[ok: true, result: { rel_keys: rel_keys }]
|
88
|
+
RedisQueuedLocks::Data[ok: true, result: { rel_key_cnt: result.sum }]
|
88
89
|
end
|
89
90
|
end
|
90
91
|
end
|
@@ -23,8 +23,8 @@ module RedisQueuedLocks::Acquier::ReleaseLock
|
|
23
23
|
# @param logger [::Logger,#debug]
|
24
24
|
# - Logger object used from `configuration` layer (see config[:logger]);
|
25
25
|
# - See RedisQueuedLocks::Logging::VoidLogger for example;
|
26
|
-
# @return [RedisQueuedLocks::Data,Hash<Symbol,
|
27
|
-
# Format: { ok: true/false, result: Hash<
|
26
|
+
# @return [RedisQueuedLocks::Data,Hash<Symbol,Boolean<Hash<Symbol,Numeric|String|Symbol>>]
|
27
|
+
# Format: { ok: true/false, result: Hash<Symbol,Numeric|String|Symbol> }
|
28
28
|
#
|
29
29
|
# @api private
|
30
30
|
# @since 0.1.0
|
@@ -49,7 +49,13 @@ module RedisQueuedLocks::Acquier::ReleaseLock
|
|
49
49
|
|
50
50
|
RedisQueuedLocks::Data[
|
51
51
|
ok: true,
|
52
|
-
result: {
|
52
|
+
result: {
|
53
|
+
rel_time: rel_time,
|
54
|
+
rel_key: lock_key,
|
55
|
+
rel_queue: lock_key_queue,
|
56
|
+
queue_res: result[:queue],
|
57
|
+
lock_res: result[:lock]
|
58
|
+
}
|
53
59
|
]
|
54
60
|
end
|
55
61
|
|
@@ -60,7 +66,14 @@ module RedisQueuedLocks::Acquier::ReleaseLock
|
|
60
66
|
# @param redis [RedisClient]
|
61
67
|
# @param lock_key [String]
|
62
68
|
# @param lock_key_queue [String]
|
63
|
-
# @return [RedisQueuedLocks::Data,Hash<Symbol,
|
69
|
+
# @return [RedisQueuedLocks::Data,Hash<Symbol,Boolean|Hash<Symbol,Symbol>>]
|
70
|
+
# Format: {
|
71
|
+
# ok: true/false,
|
72
|
+
# result: {
|
73
|
+
# queue: :released/:nothing_to_release,
|
74
|
+
# lock: :released/:nothing_to_release
|
75
|
+
# }
|
76
|
+
# }
|
64
77
|
#
|
65
78
|
# @api private
|
66
79
|
# @since 0.1.0
|
@@ -72,7 +85,13 @@ module RedisQueuedLocks::Acquier::ReleaseLock
|
|
72
85
|
end
|
73
86
|
end
|
74
87
|
|
75
|
-
RedisQueuedLocks::Data[
|
88
|
+
RedisQueuedLocks::Data[
|
89
|
+
ok: true,
|
90
|
+
result: {
|
91
|
+
queue: (result[0] != 0) ? :released : :nothing_to_release,
|
92
|
+
lock: (result[1] != 0) ? :released : :nothing_to_release
|
93
|
+
}
|
94
|
+
]
|
76
95
|
end
|
77
96
|
end
|
78
97
|
end
|
@@ -20,6 +20,8 @@ class RedisQueuedLocks::Client
|
|
20
20
|
setting :uniq_identifier, -> { RedisQueuedLocks::Resource.calc_uniq_identity }
|
21
21
|
setting :logger, RedisQueuedLocks::Logging::VoidLogger
|
22
22
|
setting :log_lock_try, false
|
23
|
+
setting :dead_request_ttl, (1 * 24 * 60 * 60 * 1000) # NOTE: 1 day in milliseconds
|
24
|
+
setting :is_timed_by_default, false
|
23
25
|
|
24
26
|
validate('retry_count') { |val| val == nil || (val.is_a?(::Integer) && val >= 0) }
|
25
27
|
validate('retry_delay') { |val| val.is_a?(::Integer) && val >= 0 }
|
@@ -32,6 +34,8 @@ class RedisQueuedLocks::Client
|
|
32
34
|
validate('uniq_identifier', :proc)
|
33
35
|
validate('logger') { |val| RedisQueuedLocks::Logging.valid_interface?(val) }
|
34
36
|
validate('log_lock_try', :boolean)
|
37
|
+
validate('dead_request_ttl') { |val| val.is_a?(::Integer) && val > 0 }
|
38
|
+
validate('is_timed_by_default', :boolean)
|
35
39
|
end
|
36
40
|
|
37
41
|
# @return [RedisClient]
|
@@ -40,15 +44,14 @@ class RedisQueuedLocks::Client
|
|
40
44
|
# @since 0.1.0
|
41
45
|
attr_reader :redis_client
|
42
46
|
|
47
|
+
# NOTE: attr_access here is chosen intentionally in order to have an ability to change
|
48
|
+
# uniq_identity value for debug purposes in runtime;
|
43
49
|
# @return [String]
|
44
50
|
#
|
45
51
|
# @api private
|
46
52
|
# @since 0.1.0
|
47
53
|
attr_accessor :uniq_identity
|
48
54
|
|
49
|
-
# NOTE: attr_access is chosen intentionally in order to have an ability to change
|
50
|
-
# uniq_identity values for debug purposes in runtime;
|
51
|
-
|
52
55
|
# @param redis_client [RedisClient]
|
53
56
|
# Redis connection manager, which will be used for the lock acquierment and distribution.
|
54
57
|
# It should be an instance of RedisClient.
|
@@ -119,7 +122,7 @@ class RedisQueuedLocks::Client
|
|
119
122
|
ttl: config[:default_lock_ttl],
|
120
123
|
queue_ttl: config[:default_queue_ttl],
|
121
124
|
timeout: config[:try_to_lock_timeout],
|
122
|
-
timed:
|
125
|
+
timed: config[:is_timed_by_default],
|
123
126
|
retry_count: config[:retry_count],
|
124
127
|
retry_delay: config[:retry_delay],
|
125
128
|
retry_jitter: config[:retry_jitter],
|
@@ -167,7 +170,7 @@ class RedisQueuedLocks::Client
|
|
167
170
|
ttl: config[:default_lock_ttl],
|
168
171
|
queue_ttl: config[:default_queue_ttl],
|
169
172
|
timeout: config[:try_to_lock_timeout],
|
170
|
-
timed:
|
173
|
+
timed: config[:is_timed_by_default],
|
171
174
|
retry_count: config[:retry_count],
|
172
175
|
retry_delay: config[:retry_delay],
|
173
176
|
retry_jitter: config[:retry_jitter],
|
@@ -191,20 +194,38 @@ class RedisQueuedLocks::Client
|
|
191
194
|
raise_errors: true,
|
192
195
|
identity:,
|
193
196
|
fail_fast:,
|
197
|
+
logger:,
|
198
|
+
log_lock_try:,
|
194
199
|
meta:,
|
195
200
|
instrument:,
|
196
201
|
&block
|
197
202
|
)
|
198
203
|
end
|
199
204
|
|
200
|
-
# @param lock_name [String]
|
201
|
-
#
|
205
|
+
# @param lock_name [String] The lock name that should be released.
|
206
|
+
# @option logger [::Logger,#debug]
|
207
|
+
# @option instrumenter [#notify]
|
208
|
+
# @option instrument [NilClass,Any]
|
202
209
|
# @return [RedisQueuedLocks::Data, Hash<Symbol,Any>]
|
203
|
-
# Format: {
|
210
|
+
# Format: {
|
211
|
+
# ok: true/false,
|
212
|
+
# result: {
|
213
|
+
# rel_time: Integer, # <millisecnds>
|
214
|
+
# rel_key: String, # lock key
|
215
|
+
# rel_queue: String, # lock queue
|
216
|
+
# queue_res: Symbol, # :released or :nothing_to_release
|
217
|
+
# lock_res: Symbol # :released or :nothing_to_release
|
218
|
+
# }
|
219
|
+
# }
|
204
220
|
#
|
205
221
|
# @api public
|
206
222
|
# @since 0.1.0
|
207
|
-
def unlock(
|
223
|
+
def unlock(
|
224
|
+
lock_name,
|
225
|
+
logger: config[:logger],
|
226
|
+
instrumenter: config[:instrumenter],
|
227
|
+
instrument: nil
|
228
|
+
)
|
208
229
|
RedisQueuedLocks::Acquier::ReleaseLock.release_lock(
|
209
230
|
redis_client,
|
210
231
|
lock_name,
|
@@ -268,26 +289,41 @@ class RedisQueuedLocks::Client
|
|
268
289
|
#
|
269
290
|
# @api public
|
270
291
|
# @since 0.1.0
|
271
|
-
def extend_lock_ttl(lock_name, milliseconds)
|
292
|
+
def extend_lock_ttl(lock_name, milliseconds, logger: config[:logger])
|
272
293
|
RedisQueuedLocks::Acquier::ExtendLockTTL.extend_lock_ttl(
|
273
294
|
redis_client,
|
274
295
|
lock_name,
|
275
|
-
milliseconds
|
296
|
+
milliseconds,
|
297
|
+
logger
|
276
298
|
)
|
277
299
|
end
|
278
300
|
|
301
|
+
# Releases all queues and locks.
|
302
|
+
# Returns:
|
303
|
+
# - :rel_time - (milliseconds) - time spent to release all locks and queues;
|
304
|
+
# - :rel_key_cnt - (integer) - the number of released redis keys (queus+locks);
|
305
|
+
#
|
279
306
|
# @option batch_size [Integer]
|
280
|
-
# @
|
281
|
-
#
|
307
|
+
# @option logger [::Logger,#debug]
|
308
|
+
# @option instrumenter [#notify]
|
309
|
+
# @option instrument [NilClass,Any]
|
310
|
+
# @return [RedisQueuedLocks::Data,Hash<Symbol,Boolean|Hash<Symbol,Numeric>>]
|
311
|
+
# Example: { ok: true, result { rel_key_cnt: 100, rel_time: 0.01 } }
|
282
312
|
#
|
283
313
|
# @api public
|
284
314
|
# @since 0.1.0
|
285
|
-
def clear_locks(
|
315
|
+
def clear_locks(
|
316
|
+
batch_size: config[:lock_release_batch_size],
|
317
|
+
logger: config[:logger],
|
318
|
+
instrumenter: config[:instrumenter],
|
319
|
+
instrument: nil
|
320
|
+
)
|
286
321
|
RedisQueuedLocks::Acquier::ReleaseAllLocks.release_all_locks(
|
287
322
|
redis_client,
|
288
323
|
batch_size,
|
289
|
-
|
290
|
-
|
324
|
+
logger,
|
325
|
+
instrumenter,
|
326
|
+
instrument
|
291
327
|
)
|
292
328
|
end
|
293
329
|
|
@@ -364,5 +400,36 @@ class RedisQueuedLocks::Client
|
|
364
400
|
def keys(scan_size: config[:key_extraction_batch_size])
|
365
401
|
RedisQueuedLocks::Acquier::Keys.keys(redis_client, scan_size:)
|
366
402
|
end
|
403
|
+
|
404
|
+
# @option dead_ttl [Integer]
|
405
|
+
# - the time period (in millsiecnds) after whcih the lock request is
|
406
|
+
# considered as dead;
|
407
|
+
# - `config[:dead_request_ttl]` is used by default;
|
408
|
+
# @option scan_size [Integer]
|
409
|
+
# The batch of scanned keys for Redis'es SCAN command.
|
410
|
+
# @option logger [::Logger,#debug]
|
411
|
+
# @option instrumenter [#notify]
|
412
|
+
# @option instrument [NilClass,Any]
|
413
|
+
# @return [Hash<Symbol,Boolean|Hash<Symbol,Set<String>>>]
|
414
|
+
# Format: { ok: true, result: { processed_queus: Set<String> } }
|
415
|
+
#
|
416
|
+
# @api public
|
417
|
+
# @since 0.1.0
|
418
|
+
def clear_dead_requests(
|
419
|
+
dead_ttl: config[:dead_request_ttl],
|
420
|
+
scan_size: config[:key_extraction_batch_size],
|
421
|
+
logger: config[:logger],
|
422
|
+
instrumenter: config[:instrumenter],
|
423
|
+
instrument: nil
|
424
|
+
)
|
425
|
+
RedisQueuedLocks::Acquier::ClearDeadRequests.clear_dead_requests(
|
426
|
+
redis_client,
|
427
|
+
scan_size,
|
428
|
+
dead_ttl,
|
429
|
+
logger,
|
430
|
+
instrumenter,
|
431
|
+
instrument
|
432
|
+
)
|
433
|
+
end
|
367
434
|
end
|
368
435
|
# rubocop:enable Metrics/ClassLength
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: redis_queued_locks
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.0.
|
4
|
+
version: 0.0.40
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Rustam Ibragimov
|
@@ -61,6 +61,7 @@ files:
|
|
61
61
|
- lib/redis_queued_locks/acquier/acquire_lock/try_to_lock.rb
|
62
62
|
- lib/redis_queued_locks/acquier/acquire_lock/with_acq_timeout.rb
|
63
63
|
- lib/redis_queued_locks/acquier/acquire_lock/yield_with_expire.rb
|
64
|
+
- lib/redis_queued_locks/acquier/clear_dead_requests.rb
|
64
65
|
- lib/redis_queued_locks/acquier/extend_lock_ttl.rb
|
65
66
|
- lib/redis_queued_locks/acquier/is_locked.rb
|
66
67
|
- lib/redis_queued_locks/acquier/is_queued.rb
|