redis_queued_locks 1.2.0 → 1.3.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +17 -0
- data/README.md +221 -51
- data/lib/redis_queued_locks/acquier/acquire_lock/try_to_lock.rb +235 -8
- data/lib/redis_queued_locks/acquier/acquire_lock/yield_expire.rb +115 -0
- data/lib/redis_queued_locks/acquier/acquire_lock.rb +195 -84
- data/lib/redis_queued_locks/acquier/lock_info.rb +17 -1
- data/lib/redis_queued_locks/acquier/locks.rb +9 -0
- data/lib/redis_queued_locks/client.rb +29 -2
- data/lib/redis_queued_locks/errors.rb +4 -0
- data/lib/redis_queued_locks/logging.rb +1 -1
- data/lib/redis_queued_locks/resource.rb +6 -0
- data/lib/redis_queued_locks/version.rb +2 -2
- metadata +3 -3
- data/lib/redis_queued_locks/acquier/acquire_lock/yield_with_expire.rb +0 -72
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 5cac51b8b6c2a4986c188d7b419e11a282c4a18d6a32689000a2c9127f77eec3
|
4
|
+
data.tar.gz: d159e30574e503a714081dfda22d9085553818573ad968f745e302efe93edfde
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 6ee000a0470297832a593a8719a192feb5fdf583003d49f8dfb3335dad4c63aff45730cc8d4481a02eb41f629e60d929349fcc1842025117bd5b35304a09c8cd
|
7
|
+
data.tar.gz: c7e52ef09012e29a2ac1495fedc287c7a51c5fee8ceac466f78d5de548e8231b7a5c8fc29626db28ca82629ac0294b4ded991cbf5902f97a76619279bed4e353
|
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,22 @@
|
|
1
1
|
## [Unreleased]
|
2
2
|
|
3
|
+
## [1.3.1] - 2024-05-10
|
4
|
+
### Fixed
|
5
|
+
- `:meta` attribute type validation of `#lock`/`#lock!` was incorrect;
|
6
|
+
### Added
|
7
|
+
- documentation updates and clarifications;
|
8
|
+
|
9
|
+
## [1.3.0] - 2024-05-08
|
10
|
+
### Added
|
11
|
+
- **Major Feature**: support for **Reentrant Locks**;
|
12
|
+
- The result of lock obtaining now includes `:process` key that shows the type of logical process that obtains the lock
|
13
|
+
(`:lock_obtaining`, `:extendable_conflict_work_through`, `:conflict_work_through`, `:dead_locking`);
|
14
|
+
- Added reentrant lock details to `RedisQueuedLocks::Client#lock_info` and `RedisQueuedLocks::Client#locks` method results;
|
15
|
+
- Documentation updates;
|
16
|
+
### Changed
|
17
|
+
- Logging: `redis_queued_locks.fail_fast_or_limits_reached__dequeue` log is renamed to `redis_queued_locks.fail_fast_or_limits_reached_or_deadlock__dequeue`
|
18
|
+
in order to reflect the lock conflict failures too;
|
19
|
+
|
3
20
|
## [1.2.0] - 2024-04-27
|
4
21
|
### Added
|
5
22
|
- Documentation updates;
|
data/README.md
CHANGED
@@ -1,11 +1,11 @@
|
|
1
1
|
# RedisQueuedLocks · ![Gem Version](https://img.shields.io/gem/v/redis_queued_locks) ![build](https://github.com/0exp/redis_queued_locks/actions/workflows/build.yml/badge.svg??branch=master)
|
2
2
|
|
3
|
-
<a href="https://redis.io/docs/manual/patterns/distributed-locks/">Distributed locks</a> with "lock acquisition queue" capabilities based on the Redis Database.
|
4
|
-
|
5
|
-
Provides flexible invocation flow, parametrized limits (lock request ttl, lock ttls, queue ttls, fast failing, etc), logging and instrumentation.
|
3
|
+
<a href="https://redis.io/docs/manual/patterns/distributed-locks/">Distributed locks</a> with "prioritized lock acquisition queue" capabilities based on the Redis Database.
|
6
4
|
|
7
5
|
Each lock request is put into the request queue (each lock is hosted by it's own queue separately from other queues) and processed in order of their priority (FIFO). Each lock request lives some period of time (RTTL) (with requeue capabilities) which guarantees the request queue will never be stacked.
|
8
6
|
|
7
|
+
Provides flexible invocation flow, parametrized limits (lock request ttl, lock ttls, queue ttls, lock attempts limit, fast failing, etc), logging and instrumentation.
|
8
|
+
|
9
9
|
---
|
10
10
|
|
11
11
|
## Table of Contents
|
@@ -32,6 +32,7 @@ Each lock request is put into the request queue (each lock is hosted by it's own
|
|
32
32
|
- [locks_info](#locks_info---get-list-of-locks-with-their-info)
|
33
33
|
- [queues_info](#queues_info---get-list-of-queues-with-their-info)
|
34
34
|
- [clear_dead_requests](#clear_dead_requests)
|
35
|
+
- [Dead locks and Reentrant locks](#dead-locks-and-reentrant-locks)
|
35
36
|
- [Logging](#logging)
|
36
37
|
- [Instrumentation](#instrumentation)
|
37
38
|
- [Instrumentation Events](#instrumentation-events)
|
@@ -57,7 +58,7 @@ Each lock request is put into the request queue (each lock is hosted by it's own
|
|
57
58
|
|
58
59
|
<sup>\[[back to top](#table-of-contents)\]</sup>
|
59
60
|
|
60
|
-
- Battle-tested on huge ruby projects in production: `~
|
61
|
+
- Battle-tested on huge ruby projects in production: `~3000` locks-per-second are obtained and released on an ongoing basis;
|
61
62
|
- Works well with `hiredis` driver enabled (it is enabled by default on our projects where `redis_queued_locks` are used);
|
62
63
|
|
63
64
|
---
|
@@ -149,6 +150,20 @@ clinet = RedisQueuedLocks::Client.new(redis_client) do |config|
|
|
149
150
|
# - should be all blocks of code are timed by default;
|
150
151
|
config.is_timed_by_default = false
|
151
152
|
|
153
|
+
# (symbol) (default: :wait_for_lock)
|
154
|
+
# - Global default conflict strategy mode;
|
155
|
+
# - Can be customized in methods `#lock` and `#lock` via `:conflict_strategy` attribute (see method signatures of #lock and #lock! methods);
|
156
|
+
# - Conflict strategy is a logical behavior for cases when the process that obtained the lock want to acquire this lock again;
|
157
|
+
# - Realizes "reentrant locks" abstraction (same process conflict / same process deadlock);
|
158
|
+
# - By default uses `:wait_for_lock` strategy (classic way);
|
159
|
+
# - Strategies:
|
160
|
+
# - `:work_through` - continue working under the lock <without> lock's TTL extension;
|
161
|
+
# - `:extendable_work_through` - continue working under the lock <with> lock's TTL extension;
|
162
|
+
# - `:wait_for_lock` - (default) - work in classic way (with timeouts, retry delays, retry limits, etc - in classic way :));
|
163
|
+
# - `:dead_locking` - fail with deadlock exception;
|
164
|
+
# - See "Dead locks and Reentrant Locks" documentation section in REDME.md for details;
|
165
|
+
config.default_conflict_strategy = :wait_for_lock
|
166
|
+
|
152
167
|
# (default: 100)
|
153
168
|
# - how many items will be released at a time in #clear_locks and in #clear_dead_requests (uses SCAN);
|
154
169
|
# - affects the performance of your Redis and Ruby Application (configure thoughtfully);
|
@@ -185,13 +200,17 @@ clinet = RedisQueuedLocks::Client.new(redis_client) do |config|
|
|
185
200
|
# (default: RedisQueuedLocks::Logging::VoidLogger)
|
186
201
|
# - the logger object;
|
187
202
|
# - should implement `debug(progname = nil, &block)` (minimal requirement) or be an instance of Ruby's `::Logger` class/subclass;
|
203
|
+
# - supports `SemanticLogger::Logger` (see "semantic_logger" gem)
|
188
204
|
# - at this moment the only debug logs are realised in following cases:
|
189
205
|
# - "[redis_queued_locks.start_lock_obtaining]" (logs "lock_key", "queue_ttl", "acq_id");
|
190
206
|
# - "[redis_queued_locks.start_try_to_lock_cycle]" (logs "lock_key", "queue_ttl", "acq_id");
|
191
207
|
# - "[redis_queued_locks.dead_score_reached__reset_acquier_position]" (logs "lock_key", "queue_ttl", "acq_id");
|
192
|
-
# - "[redis_queued_locks.lock_obtained]" (logs "
|
193
|
-
# - "[redis_queued_locks.
|
194
|
-
# - "[redis_queued_locks.
|
208
|
+
# - "[redis_queued_locks.lock_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "acq_time");
|
209
|
+
# - "[redis_queued_locks.extendable_reentrant_lock_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "acq_time");
|
210
|
+
# - "[redis_queued_locks.reentrant_lock_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "acq_time");
|
211
|
+
# - "[redis_queued_locks.fail_fast_or_limits_reached_or_deadlock__dequeue]" (logs "lock_key", "queue_ttl", "acq_id");
|
212
|
+
# - "[redis_queued_locks.expire_lock]" (logs "lock_key", "queue_ttl", "acq_id");
|
213
|
+
# - "[redis_queued_locks.decrease_lock]" (logs "lock_key", "decreased_ttl", "queue_ttl", "acq_id");
|
195
214
|
# - by default uses VoidLogger that does nothing;
|
196
215
|
config.logger = RedisQueuedLocks::Logging::VoidLogger
|
197
216
|
|
@@ -201,6 +220,10 @@ clinet = RedisQueuedLocks::Client.new(redis_client) do |config|
|
|
201
220
|
# - it adds following logs in addition to the existing:
|
202
221
|
# - "[redis_queued_locks.try_lock.start]" (logs "lock_key", "queue_ttl", "acq_id");
|
203
222
|
# - "[redis_queued_locks.try_lock.rconn_fetched]" (logs "lock_key", "queue_ttl", "acq_id");
|
223
|
+
# - "[redis_queued_locks.try_lock.same_process_conflict_detected]" (logs "lock_key", "queue_ttl", "acq_id");
|
224
|
+
# - "[redis_queued_locks.try_lock.same_process_conflict_analyzed]" (logs "lock_key", "queue_ttl", "acq_id", "spc_status");
|
225
|
+
# - "[redis_queued_locks.try_lock.reentrant_lock__extend_and_work_through]" (logs "lock_key", "queue_ttl", "acq_id", "spc_status", "last_ext_ttl", "last_ext_ts");
|
226
|
+
# - "[redis_queued_locks.try_lock.reentrant_lock__work_through]" (logs "lock_key", "queue_ttl", "acq_id", "spc_status", last_spc_ts);
|
204
227
|
# - "[redis_queued_locks.try_lock.acq_added_to_queue]" (logs "lock_key", "queue_ttl", "acq_id)";
|
205
228
|
# - "[redis_queued_locks.try_lock.remove_expired_acqs]" (logs "lock_key", "queue_ttl", "acq_id");
|
206
229
|
# - "[redis_queued_locks.try_lock.get_first_from_queue]" (logs "lock_key", "queue_ttl", "acq_id", "first_acq_id_in_queue");
|
@@ -240,10 +263,14 @@ end
|
|
240
263
|
|
241
264
|
<sup>\[[back to top](#usage)\]</sup>
|
242
265
|
|
243
|
-
-
|
244
|
-
- If block is
|
245
|
-
-
|
246
|
-
-
|
266
|
+
- `#lock` - obtain a lock;
|
267
|
+
- If block is passed:
|
268
|
+
- the obtained lock will be released after the block execution or the lock's ttl (what will happen first);
|
269
|
+
- if you want to timeout (fail with timeout) the block execution with lock's TTL use `timed: true` option;
|
270
|
+
- the block's result will be returned;
|
271
|
+
- If block is not passed:
|
272
|
+
- the obtained lock will be released after lock's ttl;
|
273
|
+
- the lock information will be returned (hash with technical info that contains: lock key, acquier identifier, acquirement timestamp, lock's ttl, type of obtaining process, etc);
|
247
274
|
|
248
275
|
```ruby
|
249
276
|
def lock(
|
@@ -257,6 +284,7 @@ def lock(
|
|
257
284
|
retry_jitter: config[:retry_jitter],
|
258
285
|
raise_errors: false,
|
259
286
|
fail_fast: false,
|
287
|
+
conflict_strategy: config[:default_conflict_strategy],
|
260
288
|
identity: uniq_identity, # (attr_accessor) calculated during client instantiation via config[:uniq_identifier] proc;
|
261
289
|
meta: nil,
|
262
290
|
instrument: nil,
|
@@ -295,7 +323,7 @@ def lock(
|
|
295
323
|
- See [Instrumentation](#instrumentation) section of docs;
|
296
324
|
- pre-configured in `config[:isntrumenter]` with void notifier (`RedisQueuedLocks::Instrumenter::VoidNotifier`);
|
297
325
|
- `raise_errors` - (optional) `[Boolean]`
|
298
|
-
- Raise errors on library-related limits such as timeout or retry count limit;
|
326
|
+
- Raise errors on library-related limits (such as timeout or retry count limit) and on lock conflicts (such as same-process dead locks);
|
299
327
|
- `false` by default;
|
300
328
|
- `fail_fast` - (optional) `[Boolean]`
|
301
329
|
- Should the required lock to be checked before the try and exit immidietly if lock is
|
@@ -303,6 +331,17 @@ def lock(
|
|
303
331
|
- Should the logic exit immidietly after the first try if the lock was obtained
|
304
332
|
by another process while the lock request queue was initially empty;
|
305
333
|
- `false` by default;
|
334
|
+
- `conflict_strategy` - (optional) - `[Symbol]``
|
335
|
+
- The conflict strategy mode for cases when the process that obtained the lock
|
336
|
+
want to acquire this lock again;
|
337
|
+
- By default uses `:wait_for_lock` strategy;
|
338
|
+
- pre-confured in `config[:default_conflict_strategy]`;
|
339
|
+
- Strategies:
|
340
|
+
- `:work_through` - continue working under the lock **without** lock's TTL extension;
|
341
|
+
- `:extendable_work_through` - continue working under the lock **with** lock's TTL extension;
|
342
|
+
- `:wait_for_lock` - (default) - work in classic way (with timeouts, retry delays, retry limits, etc - in classic way :));
|
343
|
+
- `:dead_locking` - fail with deadlock exception;
|
344
|
+
- See [Dead locks and Reentrant locks](#dead-locks-and-reentrant-locks) readme section for details;
|
306
345
|
- `identity` - (optional) `[String]`
|
307
346
|
- An unique string that is unique per `RedisQueuedLock::Client` instance. Resolves the
|
308
347
|
collisions between the same process_id/thread_id/fiber_id/ractor_id identifiers on different
|
@@ -348,14 +387,21 @@ Return value:
|
|
348
387
|
lock_key: "rql:lock:my_lock",
|
349
388
|
acq_id: "rql:acq:26672/2280/2300/2320/70ea5dbf10ea1056",
|
350
389
|
ts: 1711909612.653696,
|
351
|
-
ttl: 10000
|
390
|
+
ttl: 10000,
|
391
|
+
process: :lock_obtaining
|
352
392
|
}
|
353
393
|
}
|
354
394
|
```
|
355
395
|
- Lock information result:
|
356
396
|
- Signature: `[yield, Hash<Symbol,Boolean|Hash<Symbol,Numeric|String>>]`
|
357
397
|
- Format: `{ ok: true/false, result: <Symbol|Hash<Symbol,Hash>> }`;
|
358
|
-
-
|
398
|
+
- Includes the `:process` key that describes a logical type of the lock obtaining process. Possible values:
|
399
|
+
- `:lock_obtaining` - classic lock obtaining proces. Default behavior (`conflict_strategy: :wait_for_lock`);
|
400
|
+
- `:extendable_conflict_work_through` - reentrant lock acquiring process with lock's TTL extension. Suitable for `conflict_strategy: :extendable_work_through`;
|
401
|
+
- `:conflict_work_through` - reentrant lock acquiring process without lock's TTL extension. Suitable for `conflict_strategy: :work_through`;
|
402
|
+
- `:dead_locking` - current process tries to acquire a lock that is already acquired by them. Suitalbe for `conflict_startegy: :dead_locking`;
|
403
|
+
- For more details see [Dead locks and Reentrant locks](#dead-locks-and-reentrant-locks) readme section;
|
404
|
+
- For successful lock obtaining:
|
359
405
|
```ruby
|
360
406
|
{
|
361
407
|
ok: true,
|
@@ -363,10 +409,12 @@ Return value:
|
|
363
409
|
lock_key: String, # acquierd lock key ("rql:lock:your_lock_name")
|
364
410
|
acq_id: String, # acquier identifier ("process_id/thread_id/fiber_id/ractor_id/identity")
|
365
411
|
ts: Float, # time (epoch) when lock was obtained (float, Time#to_f)
|
366
|
-
ttl: Integer # lock's time to live in milliseconds (integer)
|
412
|
+
ttl: Integer, # lock's time to live in milliseconds (integer)
|
413
|
+
process: Symbol # which logical process has acquired the lock (:lock_obtaining, :extendable_conflict_work_through, :conflict_work_through, :conflict_dead_lock)
|
367
414
|
}
|
368
415
|
}
|
369
416
|
```
|
417
|
+
|
370
418
|
```ruby
|
371
419
|
# example:
|
372
420
|
{
|
@@ -375,14 +423,16 @@ Return value:
|
|
375
423
|
lock_key: "rql:lock:my_lock",
|
376
424
|
acq_id: "rql:acq:26672/2280/2300/2320/70ea5dbf10ea1056",
|
377
425
|
ts: 1711909612.653696,
|
378
|
-
ttl: 10000
|
426
|
+
ttl: 10000,
|
427
|
+
process: :lock_obtaining # for custom conflict strategies may be: :conflict_dead_lock, :conflict_work_through, :extendable_conflict_work_through
|
379
428
|
}
|
380
429
|
}
|
381
430
|
```
|
382
|
-
-
|
431
|
+
- For failed lock obtaining:
|
383
432
|
```ruby
|
384
433
|
{ ok: false, result: :timeout_reached }
|
385
434
|
{ ok: false, result: :retry_count_reached }
|
435
|
+
{ ok: false, result: :conflict_dead_lock } # see <conflict_strategy> option for details (:dead_locking strategy)
|
386
436
|
{ ok: false, result: :fail_fast_no_try } # see <fail_fast> option
|
387
437
|
{ ok: false, result: :fail_fast_after_try } # see <fail_fast> option
|
388
438
|
{ ok: false, result: :unknown }
|
@@ -498,26 +548,30 @@ rql.lock("my_lock", queue_ttl: 5, timeout: 10_000, retry_count: nil)
|
|
498
548
|
|
499
549
|
<sup>\[[back to top](#usage)\]</sup>
|
500
550
|
|
551
|
+
- `#lock!` - exceptional lock obtaining;
|
501
552
|
- fails when (and with):
|
502
553
|
- (`RedisQueuedLocks::LockAlreadyObtainedError`) when `fail_fast` is `true` and lock is already obtained;
|
503
554
|
- (`RedisQueuedLocks::LockAcquiermentTimeoutError`) `timeout` limit reached before lock is obtained;
|
504
555
|
- (`RedisQueuedLocks::LockAcquiermentRetryLimitError`) `retry_count` limit reached before lock is obtained;
|
556
|
+
- (`RedisQueuedLocks::ConflictLockObtainError`) when `conflict_strategy: :dead_locking` is used and the "same-process-dead-lock" is happened (see [Dead locks and Reentrant locks](#dead-locks-and-reentrant-locks) for details);
|
505
557
|
|
506
558
|
```ruby
|
507
559
|
def lock!(
|
508
560
|
lock_name,
|
509
561
|
ttl: config[:default_lock_ttl],
|
510
562
|
queue_ttl: config[:default_queue_ttl],
|
511
|
-
timeout: config[:
|
563
|
+
timeout: config[:try_to_lock_timeout],
|
564
|
+
timed: config[:is_timed_by_default],
|
512
565
|
retry_count: config[:retry_count],
|
513
566
|
retry_delay: config[:retry_delay],
|
514
567
|
retry_jitter: config[:retry_jitter],
|
515
|
-
identity: uniq_identity,
|
516
568
|
fail_fast: false,
|
569
|
+
identity: uniq_identity,
|
517
570
|
meta: nil,
|
518
|
-
instrument: nil,
|
519
571
|
logger: config[:logger],
|
520
572
|
log_lock_try: config[:log_lock_try],
|
573
|
+
instrument: nil,
|
574
|
+
conflict_strategy: config[:default_conflict_strategy],
|
521
575
|
&block
|
522
576
|
)
|
523
577
|
```
|
@@ -535,27 +589,36 @@ See `#lock` method [documentation](#lock---obtain-a-lock).
|
|
535
589
|
- lock data (`Hash<String,String|Integer>`):
|
536
590
|
- `"lock_key"` - `string` - lock key in redis;
|
537
591
|
- `"acq_id"` - `string` - acquier identifier (process_id/thread_id/fiber_id/ractor_id/identity);
|
538
|
-
- `"ts"` - `
|
592
|
+
- `"ts"` - `numeric`/`epoch` - the time when lock was obtained;
|
539
593
|
- `"init_ttl"` - `integer` - (milliseconds) initial lock key ttl;
|
540
594
|
- `"rem_ttl"` - `integer` - (milliseconds) remaining lock key ttl;
|
541
|
-
-
|
595
|
+
- `<custom metadata>`- `string`/`integer` - custom metadata passed to the `lock`/`lock!` methods via `meta:` keyword argument (see [lock]((#lock---obtain-a-lock)) method documentation);
|
596
|
+
- additional keys for **reentrant locks** and **extendable reentrant locks**:
|
597
|
+
- for any type of reentrant locks:
|
598
|
+
- `"spc_cnt"` - `integer` - how many times the lock was obtained as reentrant lock;
|
599
|
+
- for non-extendable reentrant locks:
|
600
|
+
- `"l_spc_ts"` - `numeric`/`epoch` - timestamp of the last **non-extendable** reentrant lock obtaining;
|
601
|
+
- for extendalbe reentrant locks:
|
602
|
+
- `"spc_ext_ttl"` - `integer` - (milliseconds) sum of TTL of the each **extendable** reentrant lock (the total TTL extension time);
|
603
|
+
- `"l_spc_ext_ini_ttl"` - `integer` - (milliseconds) TTL of the last reentrant lock;
|
604
|
+
- `"l_spc_ext_ts"` - `numeric`/`epoch` - timestamp of the last extendable reentrant lock obtaining;
|
542
605
|
|
543
606
|
```ruby
|
544
|
-
# without custom metadata
|
607
|
+
# <without custom metadata>
|
545
608
|
rql.lock_info("your_lock_name")
|
546
609
|
|
547
610
|
# =>
|
548
611
|
{
|
549
612
|
"lock_key" => "rql:lock:your_lock_name",
|
550
613
|
"acq_id" => "rql:acq:123/456/567/678/374dd74324",
|
551
|
-
"ts" => 123456789,
|
552
|
-
"ini_ttl" =>
|
553
|
-
"rem_ttl" =>
|
614
|
+
"ts" => 123456789.12345,
|
615
|
+
"ini_ttl" => 5_000,
|
616
|
+
"rem_ttl" => 4_999
|
554
617
|
}
|
555
618
|
```
|
556
619
|
|
557
620
|
```ruby
|
558
|
-
# with custom metadata
|
621
|
+
# <with custom metadata>
|
559
622
|
rql.lock("your_lock_name", meta: { "kek" => "pek", "bum" => 123 })
|
560
623
|
rql.lock_info("your_lock_name")
|
561
624
|
|
@@ -563,14 +626,41 @@ rql.lock_info("your_lock_name")
|
|
563
626
|
{
|
564
627
|
"lock_key" => "rql:lock:your_lock_name",
|
565
628
|
"acq_id" => "rql:acq:123/456/567/678/374dd74324",
|
566
|
-
"ts" => 123456789,
|
567
|
-
"ini_ttl" =>
|
568
|
-
"rem_ttl" =>
|
629
|
+
"ts" => 123456789.12345,
|
630
|
+
"ini_ttl" => 5_000,
|
631
|
+
"rem_ttl" => 4_999,
|
569
632
|
"kek" => "pek",
|
570
633
|
"bum" => "123" # NOTE: returned as a raw string directly from Redis
|
571
634
|
}
|
572
635
|
```
|
573
636
|
|
637
|
+
```ruby
|
638
|
+
# <for reentrant locks>
|
639
|
+
# (see `conflict_strategy:` kwarg attribute of #lock/#lock! methods and `config.default_conflict_strategy` config)
|
640
|
+
|
641
|
+
rql.lock("your_lock_name", ttl: 5_000)
|
642
|
+
rql.lock("your_lock_name", ttl: 3_000)
|
643
|
+
rql.lock("your_lock_name", ttl: 2_000)
|
644
|
+
rql.lock_info("your_lock_name")
|
645
|
+
|
646
|
+
# =>
|
647
|
+
{
|
648
|
+
"lock_key" => "rql:lock:your_lock_name",
|
649
|
+
"acq_id" => "rql:acq:123/456/567/678/374dd74324",
|
650
|
+
"ts" => 123456789.12345,
|
651
|
+
"ini_ttl" => 5_000,
|
652
|
+
"rem_ttl" => 9_444,
|
653
|
+
# ==> keys for any type of reentrant lock:
|
654
|
+
"spc_count" => 2, # how many times the lock was obtained as reentrant lock
|
655
|
+
# ==> keys for extendable reentarnt locks with `:extendable_work_through` strategy:
|
656
|
+
"spc_ext_ttl" => 5_000, # sum of TTL of the each <extendable> reentrant lock (3_000 + 2_000)
|
657
|
+
"l_spc_ext_ini_ttl" => 2_000, # TTL of the last <extendable> reentrant lock
|
658
|
+
"l_spc_ext_ts" => 123456792.12345, # timestamp of the last <extendable> reentrant lock obtaining
|
659
|
+
# ==> keys for non-extendable locks with `:work_through` strategy:
|
660
|
+
"l_spc_ts" => 123456.789 # timestamp of the last <non-extendable> reentrant lock obtaining
|
661
|
+
}
|
662
|
+
```
|
663
|
+
|
574
664
|
---
|
575
665
|
|
576
666
|
#### #queue_info
|
@@ -770,6 +860,7 @@ rql.extend_lock_ttl("my_lock", 5_000) # NOTE: add 5_000 milliseconds
|
|
770
860
|
|
771
861
|
<sup>\[[back to top](#usage)\]</sup>
|
772
862
|
|
863
|
+
- get list of obtained locks;
|
773
864
|
- uses redis `SCAN` under the hood;
|
774
865
|
- accepts:
|
775
866
|
- `:scan_size` - `Integer` - (`config[:key_extraction_batch_size]` by default);
|
@@ -802,6 +893,7 @@ rql.locks # or rql.locks(scan_size: 123)
|
|
802
893
|
|
803
894
|
<sup>\[[back to top](#usage)\]</sup>
|
804
895
|
|
896
|
+
- get list of lock request queues;
|
805
897
|
- uses redis `SCAN` under the hood;
|
806
898
|
- accepts
|
807
899
|
- `:scan_size` - `Integer` - (`config[:key_extraction_batch_size]` by default);
|
@@ -834,6 +926,7 @@ rql.queues # or rql.queues(scan_size: 123)
|
|
834
926
|
|
835
927
|
<sup>\[[back to top](#usage)\]</sup>
|
836
928
|
|
929
|
+
- get list of taken locks and queues;
|
837
930
|
- uses redis `SCAN` under the hood;
|
838
931
|
- accepts:
|
839
932
|
`:scan_size` - `Integer` - (`config[:key_extraction_batch_size]` by default);
|
@@ -867,6 +960,7 @@ rql.keys # or rql.keys(scan_size: 123)
|
|
867
960
|
|
868
961
|
<sup>\[[back to top](#usage)\]</sup>
|
869
962
|
|
963
|
+
- get list of locks with their info;
|
870
964
|
- uses redis `SCAN` under the hod;
|
871
965
|
- accepts `scan_size:`/`Integer` option (`config[:key_extraction_batch_size]` by default);
|
872
966
|
- returns `Set<Hash<Symbol,Any>>` (see [#lock_info](#lock_info) and examples below for details).
|
@@ -875,7 +969,9 @@ rql.keys # or rql.keys(scan_size: 123)
|
|
875
969
|
- `:status` - `Symbol`- `:released` or `:alive`
|
876
970
|
- the lock may become relased durign the lock info extraction process;
|
877
971
|
- `:info` for `:released` keys is empty (`{}`);
|
878
|
-
- `:info` - `Hash<String,Any>`
|
972
|
+
- `:info` - `Hash<String,Any>`
|
973
|
+
- lock data stored in the lock key in Redis;
|
974
|
+
- See [#lock_info](#lock_info) for details;
|
879
975
|
|
880
976
|
```ruby
|
881
977
|
rql.locks_info # or rql.locks_info(scan_size: 123)
|
@@ -901,6 +997,7 @@ rql.locks_info # or rql.locks_info(scan_size: 123)
|
|
901
997
|
|
902
998
|
<sup>\[[back to top](#usage)\]</sup>
|
903
999
|
|
1000
|
+
- get list of queues with their info;
|
904
1001
|
- uses redis `SCAN` under the hod;
|
905
1002
|
- accepts `scan_size:`/`Integer` option (`config[:key_extraction_batch_size]` by default);
|
906
1003
|
- returns `Set<Hash<Symbol,Any>>` (see [#queue_info](#queue_info) and examples below for details).
|
@@ -979,6 +1076,24 @@ rql.clear_dead_requests(dead_ttl: 60 * 60 * 1000) # 1 hour in milliseconds
|
|
979
1076
|
|
980
1077
|
---
|
981
1078
|
|
1079
|
+
## Dead locks and Reentrant locks
|
1080
|
+
|
1081
|
+
<sup>\[[back to top](#table-of-contents)\]</sup>
|
1082
|
+
|
1083
|
+
- **this documentation section is in progress**;
|
1084
|
+
- (little details for a context of the current implementation and feautres):
|
1085
|
+
- at this moment we support only **reentrant locks**: they works via customizable conflict strategy behavior
|
1086
|
+
(`:wait_for_lock` (default), `:work_through`, `:extendable_work_through`, `:dead_locking`);
|
1087
|
+
- by default behavior (`:wait_for_lock`) your lock obtaining process will work in a classic way (limits, retries, etc);
|
1088
|
+
- `:work_through`, `:extendable_work_through` works with limits too (timeouts, delays, etc), but the decision of
|
1089
|
+
"is your lock are obtained or not" is made as you work with **reentrant locks** (your process continues to use the lock without/without
|
1090
|
+
lock's TTL accordingly);
|
1091
|
+
- for current implementation details check:
|
1092
|
+
- [Configuration](#configuration) documentation: see `config.default_conflict_strategy` config docs;
|
1093
|
+
- [#lock](#lock---obtain-a-lock) method documentation: see `conflict_strategy` attribute docs and the method result data;
|
1094
|
+
|
1095
|
+
---
|
1096
|
+
|
982
1097
|
## Logging
|
983
1098
|
|
984
1099
|
<sup>\[[back to top](#table-of-contents)\]</sup>
|
@@ -989,9 +1104,12 @@ rql.clear_dead_requests(dead_ttl: 60 * 60 * 1000) # 1 hour in milliseconds
|
|
989
1104
|
"[redis_queued_locks.start_lock_obtaining]" # (logs "lock_key", "queue_ttl", "acq_id");
|
990
1105
|
"[redis_queued_locks.start_try_to_lock_cycle]" # (logs "lock_key", "queue_ttl", "acq_id");
|
991
1106
|
"[redis_queued_locks.dead_score_reached__reset_acquier_position]" # (logs "lock_key", "queue_ttl", "acq_id");
|
992
|
-
"[redis_queued_locks.lock_obtained]" # (logs "
|
993
|
-
"[redis_queued_locks.
|
1107
|
+
"[redis_queued_locks.lock_obtained]" # (logs "lock_key", "queue_ttl", "acq_id", "acq_time");
|
1108
|
+
"[redis_queued_locks.extendable_reentrant_lock_obtained]" # (logs "lock_key", "queue_ttl", "acq_id", "acq_time");
|
1109
|
+
"[redis_queued_locks.reentrant_lock_obtained]" # (logs "lock_key", "queue_ttl", "acq_id", "acq_time");
|
1110
|
+
"[redis_queued_locks.fail_fast_or_limits_reached_or_deadlock__dequeue]" # (logs "lock_key", "queue_ttl", "acq_id");
|
994
1111
|
"[redis_queued_locks.expire_lock]" # (logs "lock_key", "queue_ttl", "acq_id");
|
1112
|
+
"[redis_queued_locks.decrease_lock]" # (logs "lock_key", "decreased_ttl", "queue_ttl", "acq_id");
|
995
1113
|
```
|
996
1114
|
|
997
1115
|
- additional logs (raised from `#lock`/`#lock!` with `confg[:log_lock_try] == true`):
|
@@ -999,6 +1117,11 @@ rql.clear_dead_requests(dead_ttl: 60 * 60 * 1000) # 1 hour in milliseconds
|
|
999
1117
|
```ruby
|
1000
1118
|
"[redis_queued_locks.try_lock.start]" # (logs "lock_key", "queue_ttl", "acq_id");
|
1001
1119
|
"[redis_queued_locks.try_lock.rconn_fetched]" # (logs "lock_key", "queue_ttl", "acq_id");
|
1120
|
+
"[redis_queued_locks.try_lock.same_process_conflict_detected]" # (logs "lock_key", "queue_ttl", "acq_id");
|
1121
|
+
"[redis_queued_locks.try_lock.same_process_conflict_analyzed]" # (logs "lock_key", "queue_ttl", "acq_id", "spc_status");
|
1122
|
+
"[redis_queued_locks.try_lock.reentrant_lock__extend_and_work_through]" # (logs "lock_key", "queue_ttl", "acq_id", "spc_status", "last_ext_ttl", "last_ext_ts");
|
1123
|
+
"[redis_queued_locks.try_lock.reentrant_lock__work_through]" # (logs "lock_key", "queue_ttl", "acq_id", "spc_status", last_spc_ts);
|
1124
|
+
"[redis_queued_locks.try_lock.single_process_lock_conflict__dead_lock]" # (logs "lock_key", "queue_ttl", "acq_id", "spc_status", "last_spc_ts");
|
1002
1125
|
"[redis_queued_locks.try_lock.acq_added_to_queue]" # (logs "lock_key", "queue_ttl", "acq_id)";
|
1003
1126
|
"[redis_queued_locks.try_lock.remove_expired_acqs]" # (logs "lock_key", "queue_ttl", "acq_id");
|
1004
1127
|
"[redis_queued_locks.try_lock.get_first_from_queue]" # (logs "lock_key", "queue_ttl", "acq_id", "first_acq_id_in_queue");
|
@@ -1040,7 +1163,10 @@ By default `RedisQueuedLocks::Client` is configured with the void notifier (whic
|
|
1040
1163
|
List of instrumentation events
|
1041
1164
|
|
1042
1165
|
- `redis_queued_locks.lock_obtained`;
|
1166
|
+
- `redis_queued_locks.extendable_reentrant_lock_obtained`;
|
1167
|
+
- `redis_queued_locks.reentrant_lock_obtained`;
|
1043
1168
|
- `redis_queued_locks.lock_hold_and_release`;
|
1169
|
+
- `redis_queued_locks.reentrant_lock_hold_completes`;
|
1044
1170
|
- `redis_queued_locks.explicit_lock_release`;
|
1045
1171
|
- `redis_queued_locks.explicit_all_locks_release`;
|
1046
1172
|
|
@@ -1053,9 +1179,31 @@ Detalized event semantics and payload structure:
|
|
1053
1179
|
- `:ttl` - `integer`/`milliseconds` - lock ttl;
|
1054
1180
|
- `:acq_id` - `string` - lock acquier identifier;
|
1055
1181
|
- `:lock_key` - `string` - lock name;
|
1056
|
-
- `:ts` - `
|
1182
|
+
- `:ts` - `numeric`/`epoch` - the time when the lock was obtaiend;
|
1183
|
+
- `:acq_time` - `float`/`milliseconds` - time spent on lock acquiring;
|
1184
|
+
- `:instrument` - `nil`/`Any` - custom data passed to the `#lock`/`#lock!` method as `:instrument` attribute;
|
1185
|
+
|
1186
|
+
- `"redis_queued_locks.extendable_reentrant_lock_obtained"`
|
1187
|
+
- an event signalizes about the "extendable reentrant lock" obtaining is happened;
|
1188
|
+
- raised from `#lock`/`#lock!` when the lock was obtained as reentrant lock;
|
1189
|
+
- payload:
|
1190
|
+
- `:lock_key` - `string` - lock name;
|
1191
|
+
- `:ttl` - `integer`/`milliseconds` - last lock ttl by reentrant locking;
|
1192
|
+
- `:acq_id` - `string` - lock acquier identifier;
|
1193
|
+
- `:ts` - `numeric`/`epoch` - the time when the lock was obtaiend as extendable reentrant lock;
|
1057
1194
|
- `:acq_time` - `float`/`milliseconds` - time spent on lock acquiring;
|
1058
|
-
- `:instrument` - `nil`/`Any` - custom data passed to the
|
1195
|
+
- `:instrument` - `nil`/`Any` - custom data passed to the `#lock`/`#lock!` method as `:instrument` attribute;
|
1196
|
+
|
1197
|
+
- `"redis_queued_locks.reentrant_lock_obtained"`
|
1198
|
+
- an event signalizes about the "reentrant lock" obtaining is happened (without TTL extension);
|
1199
|
+
- raised from `#lock`/`#lock!` when the lock was obtained as reentrant lock;
|
1200
|
+
- payload:
|
1201
|
+
- `:lock_key` - `string` - lock name;
|
1202
|
+
- `:ttl` - `integer`/`milliseconds` - last lock ttl by reentrant locking;
|
1203
|
+
- `:acq_id` - `string` - lock acquier identifier;
|
1204
|
+
- `:ts` - `numeric`/`epoch` - the time when the lock was obtaiend as reentrant lock;
|
1205
|
+
- `:acq_time` - `float`/`milliseconds` - time spent on lock acquiring;
|
1206
|
+
- `:instrument` - `nil`/`Any` - custom data passed to the `#lock`/`#lock!` method as `:instrument` attribute;
|
1059
1207
|
|
1060
1208
|
- `"redis_queued_locks.lock_hold_and_release"`
|
1061
1209
|
- an event signalizes about the "hold+and+release" process is finished;
|
@@ -1065,9 +1213,22 @@ Detalized event semantics and payload structure:
|
|
1065
1213
|
- `:ttl` - `integer`/`milliseconds` - lock ttl;
|
1066
1214
|
- `:acq_id` - `string` - lock acquier identifier;
|
1067
1215
|
- `:lock_key` - `string` - lock name;
|
1068
|
-
- `:ts` - `
|
1216
|
+
- `:ts` - `numeric`/`epoch` - the time when lock was obtained;
|
1069
1217
|
- `:acq_time` - `float`/`milliseconds` - time spent on lock acquiring;
|
1070
|
-
- `:instrument` - `nil`/`Any` - custom data passed to the
|
1218
|
+
- `:instrument` - `nil`/`Any` - custom data passed to the `#lock`/`#lock!` method as `:instrument` attribute;
|
1219
|
+
|
1220
|
+
- `"redis_queued_locks.reentrant_lock_hold_completes"`
|
1221
|
+
- an event signalizes about the "reentrant lock hold" is complete (both extendable and non-extendable);
|
1222
|
+
- lock re-entering can happen many times and this event happens for each of them separately;
|
1223
|
+
- raised from `#lock`/`#lock!` when the lock was obtained as reentrant lock;
|
1224
|
+
- payload:
|
1225
|
+
- `:hold_time` - `float`/`milliseconds` - lock hold time;
|
1226
|
+
- `:ttl` - `integer`/`milliseconds` - last lock ttl by reentrant locking;
|
1227
|
+
- `:acq_id` - `string` - lock acquier identifier;
|
1228
|
+
- `:ts` - `numeric`/`epoch` - the time when the lock was obtaiend as reentrant lock;
|
1229
|
+
- `:lock_key` - `string` - lock name;
|
1230
|
+
- `:acq_time` - `float`/`milliseconds` - time spent on lock acquiring;
|
1231
|
+
- `:instrument` - `nil`/`Any` - custom data passed to the `#lock`/`#lock!` method as `:instrument` attribute;
|
1071
1232
|
|
1072
1233
|
- `"redis_queued_locks.explicit_lock_release"`
|
1073
1234
|
- an event signalizes about the explicit lock release (invoked via `RedisQueuedLock#unlock`);
|
@@ -1092,20 +1253,29 @@ Detalized event semantics and payload structure:
|
|
1092
1253
|
|
1093
1254
|
<sup>\[[back to top](#table-of-contents)\]</sup>
|
1094
1255
|
|
1095
|
-
- **
|
1096
|
-
-
|
1097
|
-
-
|
1098
|
-
-
|
1099
|
-
-
|
1100
|
-
|
1101
|
-
|
1102
|
-
|
1103
|
-
- support for
|
1104
|
-
-
|
1105
|
-
-
|
1106
|
-
-
|
1107
|
-
|
1108
|
-
-
|
1256
|
+
- **Major**:
|
1257
|
+
- support for Random Access strategy (non-queued behavior);
|
1258
|
+
- lock request prioritization;
|
1259
|
+
- **strict redlock algorithm support** (support for many `RedisClient` instances);
|
1260
|
+
- `#lock_series` - acquire a series of locks:
|
1261
|
+
```ruby
|
1262
|
+
rql.lock_series('lock_a', 'lock_b', 'lock_c') { puts 'locked' }
|
1263
|
+
```
|
1264
|
+
- support for `Dragonfly` database backend (https://github.com/dragonflydb/dragonfly) (https://www.dragonflydb.io/);
|
1265
|
+
- **Minor**:
|
1266
|
+
- Semantic error objects for unexpected Redis errors;
|
1267
|
+
- change all `::Process.clock_gettime(::Process::CLOCK_MONOTONIC)` milliseconds-related invocations to
|
1268
|
+
`::Process.clock_gettime(::Process::CLOCK_MONOTONIC, :millisecond)` in order to reduce time-related math operations count;
|
1269
|
+
- **Experimental feature**: (non-`timed` locks): per-ruby-block-holding-the-lock sidecar `Ractor` and `in progress queue` in RedisDB that will extend
|
1270
|
+
the acquired lock for long-running blocks of code (that invoked "under" the lock
|
1271
|
+
whose ttl may expire before the block execution completes). It makes sense for non-`timed` locks *only*;
|
1272
|
+
- better code stylization (+ some refactorings);
|
1273
|
+
- `RedisQueuedLocks::Acquier::Try.try_to_lock` - detailed successful result analization;
|
1274
|
+
- Support for LIFO strategy;
|
1275
|
+
- better specs with 100% test coverage (total specs rework);
|
1276
|
+
- statistics with UI;
|
1277
|
+
- JSON log formatter;
|
1278
|
+
- `go`-lang implementation;
|
1109
1279
|
|
1110
1280
|
---
|
1111
1281
|
|