redis_queued_locks 1.2.0 → 1.3.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +11 -0
- data/README.md +206 -47
- data/lib/redis_queued_locks/acquier/acquire_lock/try_to_lock.rb +235 -8
- data/lib/redis_queued_locks/acquier/acquire_lock/yield_expire.rb +115 -0
- data/lib/redis_queued_locks/acquier/acquire_lock.rb +194 -83
- data/lib/redis_queued_locks/acquier/lock_info.rb +17 -1
- data/lib/redis_queued_locks/acquier/locks.rb +9 -0
- data/lib/redis_queued_locks/client.rb +29 -2
- data/lib/redis_queued_locks/errors.rb +4 -0
- data/lib/redis_queued_locks/logging.rb +1 -1
- data/lib/redis_queued_locks/resource.rb +6 -0
- data/lib/redis_queued_locks/version.rb +2 -2
- metadata +4 -4
- data/lib/redis_queued_locks/acquier/acquire_lock/yield_with_expire.rb +0 -72
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 06ddeeb8f9ef0bd0c24742e68e83d10b237f8233ca5183c7f9612e7aebadd874
|
4
|
+
data.tar.gz: df9a7ba009357fc3fa80efc9b5837cec56f10cb9004f54df19e5d08fd06f7f26
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 722aff2cb859419913d8f47c6118acaab378db70e49034290043cb1f9c3b4df115d15020d37effe815399d389fb360b68f2692a50537165043eb8a659428b6ba
|
7
|
+
data.tar.gz: ef370212d73abc96b1fed6962e274b616bb9b1ed232576ebee0b1e2e2b98d62cbdbe7dbb113ac32b28e489d4142a3a2f91b7f96be1be6c11d285cb948711bfee
|
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,16 @@
|
|
1
1
|
## [Unreleased]
|
2
2
|
|
3
|
+
## [1.3.0] - 2024-05-08
|
4
|
+
### Added
|
5
|
+
- **Major Feature**: support for **Reentrant Locks**;
|
6
|
+
- The result of lock obtaining now includes `:process` key that shows the type of logical process that obtains the lock
|
7
|
+
(`:lock_obtaining`, `:extendable_conflict_work_through`, `:conflict_work_through`, `:dead_locking`);
|
8
|
+
- Added reentrant lock details to `RedisQueuedLocks::Client#lock_info` and `RedisQueuedLocks::Client#locks` method results;
|
9
|
+
- Documentation updates;
|
10
|
+
### Changed
|
11
|
+
- Logging: `redis_queued_locks.fail_fast_or_limits_reached__dequeue` log is renamed to `redis_queued_locks.fail_fast_or_limits_reached_or_deadlock__dequeue`
|
12
|
+
in order to reflect the lock conflict failures too;
|
13
|
+
|
3
14
|
## [1.2.0] - 2024-04-27
|
4
15
|
### Added
|
5
16
|
- Documentation updates;
|
data/README.md
CHANGED
@@ -1,11 +1,11 @@
|
|
1
1
|
# RedisQueuedLocks · ![Gem Version](https://img.shields.io/gem/v/redis_queued_locks) ![build](https://github.com/0exp/redis_queued_locks/actions/workflows/build.yml/badge.svg??branch=master)
|
2
2
|
|
3
|
-
<a href="https://redis.io/docs/manual/patterns/distributed-locks/">Distributed locks</a> with "lock acquisition queue" capabilities based on the Redis Database.
|
4
|
-
|
5
|
-
Provides flexible invocation flow, parametrized limits (lock request ttl, lock ttls, queue ttls, fast failing, etc), logging and instrumentation.
|
3
|
+
<a href="https://redis.io/docs/manual/patterns/distributed-locks/">Distributed locks</a> with "prioritized lock acquisition queue" capabilities based on the Redis Database.
|
6
4
|
|
7
5
|
Each lock request is put into the request queue (each lock is hosted by it's own queue separately from other queues) and processed in order of their priority (FIFO). Each lock request lives some period of time (RTTL) (with requeue capabilities) which guarantees the request queue will never be stacked.
|
8
6
|
|
7
|
+
Provides flexible invocation flow, parametrized limits (lock request ttl, lock ttls, queue ttls, lock attempts limit, fast failing, etc), logging and instrumentation.
|
8
|
+
|
9
9
|
---
|
10
10
|
|
11
11
|
## Table of Contents
|
@@ -32,6 +32,7 @@ Each lock request is put into the request queue (each lock is hosted by it's own
|
|
32
32
|
- [locks_info](#locks_info---get-list-of-locks-with-their-info)
|
33
33
|
- [queues_info](#queues_info---get-list-of-queues-with-their-info)
|
34
34
|
- [clear_dead_requests](#clear_dead_requests)
|
35
|
+
- [Dead locks and Reentrant locks](#dead-locks-and-reentrant-locks)
|
35
36
|
- [Logging](#logging)
|
36
37
|
- [Instrumentation](#instrumentation)
|
37
38
|
- [Instrumentation Events](#instrumentation-events)
|
@@ -57,7 +58,7 @@ Each lock request is put into the request queue (each lock is hosted by it's own
|
|
57
58
|
|
58
59
|
<sup>\[[back to top](#table-of-contents)\]</sup>
|
59
60
|
|
60
|
-
- Battle-tested on huge ruby projects in production: `~
|
61
|
+
- Battle-tested on huge ruby projects in production: `~3000` locks-per-second are obtained and released on an ongoing basis;
|
61
62
|
- Works well with `hiredis` driver enabled (it is enabled by default on our projects where `redis_queued_locks` are used);
|
62
63
|
|
63
64
|
---
|
@@ -149,6 +150,19 @@ clinet = RedisQueuedLocks::Client.new(redis_client) do |config|
|
|
149
150
|
# - should be all blocks of code are timed by default;
|
150
151
|
config.is_timed_by_default = false
|
151
152
|
|
153
|
+
# (symbol) (default: :wait_for_lock)
|
154
|
+
# - The conflict strategy mode for cases when the process that obtained the lock want to acquire this lock again;
|
155
|
+
# - Realizes "reentrant locks" abstraction (same process conflict / same process deadlock);
|
156
|
+
# - By default uses `:wait_for_lock` strategy (classic way);
|
157
|
+
# - Strategies:
|
158
|
+
# - `:work_through` - continue working under the lock <without> lock's TTL extension;
|
159
|
+
# - `:extendable_work_through` - continue working under the lock <with> lock's TTL extension;
|
160
|
+
# - `:wait_for_lock` - (default) - work in classic way (with timeouts, retry delays, retry limits, etc - in classic way :));
|
161
|
+
# - `:dead_locking` - fail with deadlock exception;
|
162
|
+
# - Can be customized in methods via `:conflict_strategy` attribute (see method signatures of #lock and #lock! methods);
|
163
|
+
# - See "Dead locks and Reentrant Locks" documentation section in REDME.md for details;
|
164
|
+
config.default_conflict_strategy = :wait_for_lock
|
165
|
+
|
152
166
|
# (default: 100)
|
153
167
|
# - how many items will be released at a time in #clear_locks and in #clear_dead_requests (uses SCAN);
|
154
168
|
# - affects the performance of your Redis and Ruby Application (configure thoughtfully);
|
@@ -185,13 +199,17 @@ clinet = RedisQueuedLocks::Client.new(redis_client) do |config|
|
|
185
199
|
# (default: RedisQueuedLocks::Logging::VoidLogger)
|
186
200
|
# - the logger object;
|
187
201
|
# - should implement `debug(progname = nil, &block)` (minimal requirement) or be an instance of Ruby's `::Logger` class/subclass;
|
202
|
+
# - supports `SemanticLogger::Logger` (see "semantic_logger" gem)
|
188
203
|
# - at this moment the only debug logs are realised in following cases:
|
189
204
|
# - "[redis_queued_locks.start_lock_obtaining]" (logs "lock_key", "queue_ttl", "acq_id");
|
190
205
|
# - "[redis_queued_locks.start_try_to_lock_cycle]" (logs "lock_key", "queue_ttl", "acq_id");
|
191
206
|
# - "[redis_queued_locks.dead_score_reached__reset_acquier_position]" (logs "lock_key", "queue_ttl", "acq_id");
|
192
|
-
# - "[redis_queued_locks.lock_obtained]" (logs "
|
193
|
-
# - "[redis_queued_locks.
|
194
|
-
# - "[redis_queued_locks.
|
207
|
+
# - "[redis_queued_locks.lock_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "acq_time");
|
208
|
+
# - "[redis_queued_locks.extendable_reentrant_lock_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "acq_time");
|
209
|
+
# - "[redis_queued_locks.reentrant_lock_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "acq_time");
|
210
|
+
# - "[redis_queued_locks.fail_fast_or_limits_reached_or_deadlock__dequeue]" (logs "lock_key", "queue_ttl", "acq_id");
|
211
|
+
# - "[redis_queued_locks.expire_lock]" (logs "lock_key", "queue_ttl", "acq_id");
|
212
|
+
# - "[redis_queued_locks.decrease_lock]" (logs "lock_key", "decreased_ttl", "queue_ttl", "acq_id");
|
195
213
|
# - by default uses VoidLogger that does nothing;
|
196
214
|
config.logger = RedisQueuedLocks::Logging::VoidLogger
|
197
215
|
|
@@ -201,6 +219,10 @@ clinet = RedisQueuedLocks::Client.new(redis_client) do |config|
|
|
201
219
|
# - it adds following logs in addition to the existing:
|
202
220
|
# - "[redis_queued_locks.try_lock.start]" (logs "lock_key", "queue_ttl", "acq_id");
|
203
221
|
# - "[redis_queued_locks.try_lock.rconn_fetched]" (logs "lock_key", "queue_ttl", "acq_id");
|
222
|
+
# - "[redis_queued_locks.try_lock.same_process_conflict_detected]" (logs "lock_key", "queue_ttl", "acq_id");
|
223
|
+
# - "[redis_queued_locks.try_lock.same_process_conflict_analyzed]" (logs "lock_key", "queue_ttl", "acq_id", "spc_status");
|
224
|
+
# - "[redis_queued_locks.try_lock.reentrant_lock__extend_and_work_through]" (logs "lock_key", "queue_ttl", "acq_id", "spc_status", "last_ext_ttl", "last_ext_ts");
|
225
|
+
# - "[redis_queued_locks.try_lock.reentrant_lock__work_through]" (logs "lock_key", "queue_ttl", "acq_id", "spc_status", last_spc_ts);
|
204
226
|
# - "[redis_queued_locks.try_lock.acq_added_to_queue]" (logs "lock_key", "queue_ttl", "acq_id)";
|
205
227
|
# - "[redis_queued_locks.try_lock.remove_expired_acqs]" (logs "lock_key", "queue_ttl", "acq_id");
|
206
228
|
# - "[redis_queued_locks.try_lock.get_first_from_queue]" (logs "lock_key", "queue_ttl", "acq_id", "first_acq_id_in_queue");
|
@@ -257,6 +279,7 @@ def lock(
|
|
257
279
|
retry_jitter: config[:retry_jitter],
|
258
280
|
raise_errors: false,
|
259
281
|
fail_fast: false,
|
282
|
+
conflict_strategy: config[:default_conflict_strategy],
|
260
283
|
identity: uniq_identity, # (attr_accessor) calculated during client instantiation via config[:uniq_identifier] proc;
|
261
284
|
meta: nil,
|
262
285
|
instrument: nil,
|
@@ -295,7 +318,7 @@ def lock(
|
|
295
318
|
- See [Instrumentation](#instrumentation) section of docs;
|
296
319
|
- pre-configured in `config[:isntrumenter]` with void notifier (`RedisQueuedLocks::Instrumenter::VoidNotifier`);
|
297
320
|
- `raise_errors` - (optional) `[Boolean]`
|
298
|
-
- Raise errors on library-related limits such as timeout or retry count limit;
|
321
|
+
- Raise errors on library-related limits (such as timeout or retry count limit) and on lock conflicts (such as same-process dead locks);
|
299
322
|
- `false` by default;
|
300
323
|
- `fail_fast` - (optional) `[Boolean]`
|
301
324
|
- Should the required lock to be checked before the try and exit immidietly if lock is
|
@@ -303,6 +326,17 @@ def lock(
|
|
303
326
|
- Should the logic exit immidietly after the first try if the lock was obtained
|
304
327
|
by another process while the lock request queue was initially empty;
|
305
328
|
- `false` by default;
|
329
|
+
- `conflict_strategy` - (optional) - `[Symbol]``
|
330
|
+
- The conflict strategy mode for cases when the process that obtained the lock
|
331
|
+
want to acquire this lock again;
|
332
|
+
- By default uses `:wait_for_lock` strategy;
|
333
|
+
- pre-confured in `config[:default_conflict_strategy]`;
|
334
|
+
- Strategies:
|
335
|
+
- `:work_through` - continue working under the lock **without** lock's TTL extension;
|
336
|
+
- `:extendable_work_through` - continue working under the lock **with** lock's TTL extension;
|
337
|
+
- `:wait_for_lock` - (default) - work in classic way (with timeouts, retry delays, retry limits, etc - in classic way :));
|
338
|
+
- `:dead_locking` - fail with deadlock exception;
|
339
|
+
- See [Dead locks and Reentrant locks](#dead-locks-and-reentrant-locks) readme section for details;
|
306
340
|
- `identity` - (optional) `[String]`
|
307
341
|
- An unique string that is unique per `RedisQueuedLock::Client` instance. Resolves the
|
308
342
|
collisions between the same process_id/thread_id/fiber_id/ractor_id identifiers on different
|
@@ -348,14 +382,21 @@ Return value:
|
|
348
382
|
lock_key: "rql:lock:my_lock",
|
349
383
|
acq_id: "rql:acq:26672/2280/2300/2320/70ea5dbf10ea1056",
|
350
384
|
ts: 1711909612.653696,
|
351
|
-
ttl: 10000
|
385
|
+
ttl: 10000,
|
386
|
+
process: :lock_obtaining
|
352
387
|
}
|
353
388
|
}
|
354
389
|
```
|
355
390
|
- Lock information result:
|
356
391
|
- Signature: `[yield, Hash<Symbol,Boolean|Hash<Symbol,Numeric|String>>]`
|
357
392
|
- Format: `{ ok: true/false, result: <Symbol|Hash<Symbol,Hash>> }`;
|
358
|
-
-
|
393
|
+
- Includes the `:process` key that describes a logical type of the lock obtaining process. Possible values:
|
394
|
+
- `:lock_obtaining` - classic lock obtaining proces. Default behavior (`conflict_strategy: :wait_for_lock`);
|
395
|
+
- `:extendable_conflict_work_through` - reentrant lock acquiring process with lock's TTL extension. Suitable for `conflict_strategy: :extendable_work_through`;
|
396
|
+
- `:conflict_work_through` - reentrant lock acquiring process without lock's TTL extension. Suitable for `conflict_strategy: :work_through`;
|
397
|
+
- `:dead_locking` - current process tries to acquire a lock that is already acquired by them. Suitalbe for `conflict_startegy: :dead_locking`;
|
398
|
+
- For more details see [Dead locks and Reentrant locks](#dead-locks-and-reentrant-locks) readme section;
|
399
|
+
- For successful lock obtaining:
|
359
400
|
```ruby
|
360
401
|
{
|
361
402
|
ok: true,
|
@@ -363,10 +404,12 @@ Return value:
|
|
363
404
|
lock_key: String, # acquierd lock key ("rql:lock:your_lock_name")
|
364
405
|
acq_id: String, # acquier identifier ("process_id/thread_id/fiber_id/ractor_id/identity")
|
365
406
|
ts: Float, # time (epoch) when lock was obtained (float, Time#to_f)
|
366
|
-
ttl: Integer # lock's time to live in milliseconds (integer)
|
407
|
+
ttl: Integer, # lock's time to live in milliseconds (integer)
|
408
|
+
process: Symbol # which logical process has acquired the lock (:lock_obtaining, :extendable_conflict_work_through, :conflict_work_through, :conflict_dead_lock)
|
367
409
|
}
|
368
410
|
}
|
369
411
|
```
|
412
|
+
|
370
413
|
```ruby
|
371
414
|
# example:
|
372
415
|
{
|
@@ -375,14 +418,16 @@ Return value:
|
|
375
418
|
lock_key: "rql:lock:my_lock",
|
376
419
|
acq_id: "rql:acq:26672/2280/2300/2320/70ea5dbf10ea1056",
|
377
420
|
ts: 1711909612.653696,
|
378
|
-
ttl: 10000
|
421
|
+
ttl: 10000,
|
422
|
+
process: :lock_obtaining # for custom conflict strategies may be: :conflict_dead_lock, :conflict_work_through, :extendable_conflict_work_through
|
379
423
|
}
|
380
424
|
}
|
381
425
|
```
|
382
|
-
-
|
426
|
+
- For failed lock obtaining:
|
383
427
|
```ruby
|
384
428
|
{ ok: false, result: :timeout_reached }
|
385
429
|
{ ok: false, result: :retry_count_reached }
|
430
|
+
{ ok: false, result: :conflict_dead_lock } # see <conflict_strategy> option for details (:dead_locking strategy)
|
386
431
|
{ ok: false, result: :fail_fast_no_try } # see <fail_fast> option
|
387
432
|
{ ok: false, result: :fail_fast_after_try } # see <fail_fast> option
|
388
433
|
{ ok: false, result: :unknown }
|
@@ -508,16 +553,18 @@ def lock!(
|
|
508
553
|
lock_name,
|
509
554
|
ttl: config[:default_lock_ttl],
|
510
555
|
queue_ttl: config[:default_queue_ttl],
|
511
|
-
timeout: config[:
|
556
|
+
timeout: config[:try_to_lock_timeout],
|
557
|
+
timed: config[:is_timed_by_default],
|
512
558
|
retry_count: config[:retry_count],
|
513
559
|
retry_delay: config[:retry_delay],
|
514
560
|
retry_jitter: config[:retry_jitter],
|
515
|
-
identity: uniq_identity,
|
516
561
|
fail_fast: false,
|
562
|
+
identity: uniq_identity,
|
517
563
|
meta: nil,
|
518
|
-
instrument: nil,
|
519
564
|
logger: config[:logger],
|
520
565
|
log_lock_try: config[:log_lock_try],
|
566
|
+
instrument: nil,
|
567
|
+
conflict_strategy: config[:default_conflict_strategy],
|
521
568
|
&block
|
522
569
|
)
|
523
570
|
```
|
@@ -535,27 +582,36 @@ See `#lock` method [documentation](#lock---obtain-a-lock).
|
|
535
582
|
- lock data (`Hash<String,String|Integer>`):
|
536
583
|
- `"lock_key"` - `string` - lock key in redis;
|
537
584
|
- `"acq_id"` - `string` - acquier identifier (process_id/thread_id/fiber_id/ractor_id/identity);
|
538
|
-
- `"ts"` - `
|
585
|
+
- `"ts"` - `numeric`/`epoch` - the time when lock was obtained;
|
539
586
|
- `"init_ttl"` - `integer` - (milliseconds) initial lock key ttl;
|
540
587
|
- `"rem_ttl"` - `integer` - (milliseconds) remaining lock key ttl;
|
541
|
-
-
|
588
|
+
- `<custom metadata>`- `string`/`integer` - custom metadata passed to the `lock`/`lock!` methods via `meta:` keyword argument (see [lock]((#lock---obtain-a-lock)) method documentation);
|
589
|
+
- additional keys for **reentrant locks** and **extendable reentrant locks**:
|
590
|
+
- for any type of reentrant locks:
|
591
|
+
- `"spc_cnt"` - `integer` - how many times the lock was obtained as reentrant lock;
|
592
|
+
- for non-extendable reentrant locks:
|
593
|
+
- `"l_spc_ts"` - `numeric`/`epoch` - timestamp of the last **non-extendable** reentrant lock obtaining;
|
594
|
+
- for extendalbe reentrant locks:
|
595
|
+
- `"spc_ext_ttl"` - `integer` - (milliseconds) sum of TTL of the each **extendable** reentrant lock (the total TTL extension time);
|
596
|
+
- `"l_spc_ext_ini_ttl"` - `integer` - (milliseconds) TTL of the last reentrant lock;
|
597
|
+
- `"l_spc_ext_ts"` - `numeric`/`epoch` - timestamp of the last extendable reentrant lock obtaining;
|
542
598
|
|
543
599
|
```ruby
|
544
|
-
# without custom metadata
|
600
|
+
# <without custom metadata>
|
545
601
|
rql.lock_info("your_lock_name")
|
546
602
|
|
547
603
|
# =>
|
548
604
|
{
|
549
605
|
"lock_key" => "rql:lock:your_lock_name",
|
550
606
|
"acq_id" => "rql:acq:123/456/567/678/374dd74324",
|
551
|
-
"ts" => 123456789,
|
552
|
-
"ini_ttl" =>
|
553
|
-
"rem_ttl" =>
|
607
|
+
"ts" => 123456789.12345,
|
608
|
+
"ini_ttl" => 5_000,
|
609
|
+
"rem_ttl" => 4_999
|
554
610
|
}
|
555
611
|
```
|
556
612
|
|
557
613
|
```ruby
|
558
|
-
# with custom metadata
|
614
|
+
# <with custom metadata>
|
559
615
|
rql.lock("your_lock_name", meta: { "kek" => "pek", "bum" => 123 })
|
560
616
|
rql.lock_info("your_lock_name")
|
561
617
|
|
@@ -563,14 +619,41 @@ rql.lock_info("your_lock_name")
|
|
563
619
|
{
|
564
620
|
"lock_key" => "rql:lock:your_lock_name",
|
565
621
|
"acq_id" => "rql:acq:123/456/567/678/374dd74324",
|
566
|
-
"ts" => 123456789,
|
567
|
-
"ini_ttl" =>
|
568
|
-
"rem_ttl" =>
|
622
|
+
"ts" => 123456789.12345,
|
623
|
+
"ini_ttl" => 5_000,
|
624
|
+
"rem_ttl" => 4_999,
|
569
625
|
"kek" => "pek",
|
570
626
|
"bum" => "123" # NOTE: returned as a raw string directly from Redis
|
571
627
|
}
|
572
628
|
```
|
573
629
|
|
630
|
+
```ruby
|
631
|
+
# <for reentrant locks>
|
632
|
+
# (see `conflict_strategy:` kwarg attribute of #lock/#lock! methods and `config.default_conflict_strategy` config)
|
633
|
+
|
634
|
+
rql.lock("your_lock_name", ttl: 5_000)
|
635
|
+
rql.lock("your_lock_name", ttl: 3_000)
|
636
|
+
rql.lock("your_lock_name", ttl: 2_000)
|
637
|
+
rql.lock_info("your_lock_name")
|
638
|
+
|
639
|
+
# =>
|
640
|
+
{
|
641
|
+
"lock_key" => "rql:lock:your_lock_name",
|
642
|
+
"acq_id" => "rql:acq:123/456/567/678/374dd74324",
|
643
|
+
"ts" => 123456789.12345,
|
644
|
+
"ini_ttl" => 5_000,
|
645
|
+
"rem_ttl" => 9_444,
|
646
|
+
# ==> keys for any type of reentrant lock:
|
647
|
+
"spc_count" => 2, # how many times the lock was obtained as reentrant lock
|
648
|
+
# ==> keys for extendable reentarnt locks with `:extendable_work_through` strategy:
|
649
|
+
"spc_ext_ttl" => 5_000, # sum of TTL of the each <extendable> reentrant lock (3_000 + 2_000)
|
650
|
+
"l_spc_ext_ini_ttl" => 2_000, # TTL of the last <extendable> reentrant lock
|
651
|
+
"l_spc_ext_ts" => 123456792.12345 # timestamp of the last <extendable> reentrant lock obtaining
|
652
|
+
# ==> keys for non-extendable locks with `:work_through` strategy:
|
653
|
+
"l_spc_ts" => 123456.789 # timestamp of the last <non-extendable> reentrant lock obtaining
|
654
|
+
}
|
655
|
+
```
|
656
|
+
|
574
657
|
---
|
575
658
|
|
576
659
|
#### #queue_info
|
@@ -770,6 +853,7 @@ rql.extend_lock_ttl("my_lock", 5_000) # NOTE: add 5_000 milliseconds
|
|
770
853
|
|
771
854
|
<sup>\[[back to top](#usage)\]</sup>
|
772
855
|
|
856
|
+
- get list of obtained locks;
|
773
857
|
- uses redis `SCAN` under the hood;
|
774
858
|
- accepts:
|
775
859
|
- `:scan_size` - `Integer` - (`config[:key_extraction_batch_size]` by default);
|
@@ -802,6 +886,7 @@ rql.locks # or rql.locks(scan_size: 123)
|
|
802
886
|
|
803
887
|
<sup>\[[back to top](#usage)\]</sup>
|
804
888
|
|
889
|
+
- get list of lock request queues;
|
805
890
|
- uses redis `SCAN` under the hood;
|
806
891
|
- accepts
|
807
892
|
- `:scan_size` - `Integer` - (`config[:key_extraction_batch_size]` by default);
|
@@ -834,6 +919,7 @@ rql.queues # or rql.queues(scan_size: 123)
|
|
834
919
|
|
835
920
|
<sup>\[[back to top](#usage)\]</sup>
|
836
921
|
|
922
|
+
- get list of taken locks and queues;
|
837
923
|
- uses redis `SCAN` under the hood;
|
838
924
|
- accepts:
|
839
925
|
`:scan_size` - `Integer` - (`config[:key_extraction_batch_size]` by default);
|
@@ -867,6 +953,7 @@ rql.keys # or rql.keys(scan_size: 123)
|
|
867
953
|
|
868
954
|
<sup>\[[back to top](#usage)\]</sup>
|
869
955
|
|
956
|
+
- get list of locks with their info;
|
870
957
|
- uses redis `SCAN` under the hod;
|
871
958
|
- accepts `scan_size:`/`Integer` option (`config[:key_extraction_batch_size]` by default);
|
872
959
|
- returns `Set<Hash<Symbol,Any>>` (see [#lock_info](#lock_info) and examples below for details).
|
@@ -875,7 +962,9 @@ rql.keys # or rql.keys(scan_size: 123)
|
|
875
962
|
- `:status` - `Symbol`- `:released` or `:alive`
|
876
963
|
- the lock may become relased durign the lock info extraction process;
|
877
964
|
- `:info` for `:released` keys is empty (`{}`);
|
878
|
-
- `:info` - `Hash<String,Any>`
|
965
|
+
- `:info` - `Hash<String,Any>`
|
966
|
+
- lock data stored in the lock key in Redis;
|
967
|
+
- See [#lock_info](#lock_info) for details;
|
879
968
|
|
880
969
|
```ruby
|
881
970
|
rql.locks_info # or rql.locks_info(scan_size: 123)
|
@@ -901,6 +990,7 @@ rql.locks_info # or rql.locks_info(scan_size: 123)
|
|
901
990
|
|
902
991
|
<sup>\[[back to top](#usage)\]</sup>
|
903
992
|
|
993
|
+
- get list of queues with their info;
|
904
994
|
- uses redis `SCAN` under the hod;
|
905
995
|
- accepts `scan_size:`/`Integer` option (`config[:key_extraction_batch_size]` by default);
|
906
996
|
- returns `Set<Hash<Symbol,Any>>` (see [#queue_info](#queue_info) and examples below for details).
|
@@ -979,6 +1069,20 @@ rql.clear_dead_requests(dead_ttl: 60 * 60 * 1000) # 1 hour in milliseconds
|
|
979
1069
|
|
980
1070
|
---
|
981
1071
|
|
1072
|
+
## Dead locks and Reentrant locks
|
1073
|
+
|
1074
|
+
<sup>\[[back to top](#table-of-contents)\]</sup>
|
1075
|
+
|
1076
|
+
- **documentation is in progress**
|
1077
|
+
- (little details for a context of current implementation and feautres):
|
1078
|
+
- at this moment we support only **reentrant locks**: they works via customizable conflict strategy behavior;
|
1079
|
+
- in non-reentrant conflict cases your lock obtaining process will work in a classic way (some dead lock conflict => work in "wait for lock" style);
|
1080
|
+
- for current implementation details check:
|
1081
|
+
- (Configuration)[#configuration] documentation: see `config.default_conflict_strategy` config docs;
|
1082
|
+
- (#lock)[#lock] method documentation: see `conflict_strategy` attribute docs;
|
1083
|
+
|
1084
|
+
---
|
1085
|
+
|
982
1086
|
## Logging
|
983
1087
|
|
984
1088
|
<sup>\[[back to top](#table-of-contents)\]</sup>
|
@@ -989,9 +1093,12 @@ rql.clear_dead_requests(dead_ttl: 60 * 60 * 1000) # 1 hour in milliseconds
|
|
989
1093
|
"[redis_queued_locks.start_lock_obtaining]" # (logs "lock_key", "queue_ttl", "acq_id");
|
990
1094
|
"[redis_queued_locks.start_try_to_lock_cycle]" # (logs "lock_key", "queue_ttl", "acq_id");
|
991
1095
|
"[redis_queued_locks.dead_score_reached__reset_acquier_position]" # (logs "lock_key", "queue_ttl", "acq_id");
|
992
|
-
"[redis_queued_locks.lock_obtained]" # (logs "
|
993
|
-
"[redis_queued_locks.
|
1096
|
+
"[redis_queued_locks.lock_obtained]" # (logs "lock_key", "queue_ttl", "acq_id", "acq_time");
|
1097
|
+
"[redis_queued_locks.extendable_reentrant_lock_obtained]" # (logs "lock_key", "queue_ttl", "acq_id", "acq_time");
|
1098
|
+
"[redis_queued_locks.reentrant_lock_obtained]" # (logs "lock_key", "queue_ttl", "acq_id", "acq_time");
|
1099
|
+
"[redis_queued_locks.fail_fast_or_limits_reached_or_deadlock__dequeue]" # (logs "lock_key", "queue_ttl", "acq_id");
|
994
1100
|
"[redis_queued_locks.expire_lock]" # (logs "lock_key", "queue_ttl", "acq_id");
|
1101
|
+
"[redis_queued_locks.decrease_lock]" # (logs "lock_key", "decreased_ttl", "queue_ttl", "acq_id");
|
995
1102
|
```
|
996
1103
|
|
997
1104
|
- additional logs (raised from `#lock`/`#lock!` with `confg[:log_lock_try] == true`):
|
@@ -999,6 +1106,11 @@ rql.clear_dead_requests(dead_ttl: 60 * 60 * 1000) # 1 hour in milliseconds
|
|
999
1106
|
```ruby
|
1000
1107
|
"[redis_queued_locks.try_lock.start]" # (logs "lock_key", "queue_ttl", "acq_id");
|
1001
1108
|
"[redis_queued_locks.try_lock.rconn_fetched]" # (logs "lock_key", "queue_ttl", "acq_id");
|
1109
|
+
"[redis_queued_locks.try_lock.same_process_conflict_detected]" # (logs "lock_key", "queue_ttl", "acq_id");
|
1110
|
+
"[redis_queued_locks.try_lock.same_process_conflict_analyzed]" # (logs "lock_key", "queue_ttl", "acq_id", "spc_status");
|
1111
|
+
"[redis_queued_locks.try_lock.reentrant_lock__extend_and_work_through]" # (logs "lock_key", "queue_ttl", "acq_id", "spc_status", "last_ext_ttl", "last_ext_ts");
|
1112
|
+
"[redis_queued_locks.try_lock.reentrant_lock__work_through]" # (logs "lock_key", "queue_ttl", "acq_id", "spc_status", last_spc_ts);
|
1113
|
+
"[redis_queued_locks.try_lock.single_process_lock_conflict__dead_lock]" # (logs "lock_key", "queue_ttl", "acq_id", "spc_status", "last_spc_ts");
|
1002
1114
|
"[redis_queued_locks.try_lock.acq_added_to_queue]" # (logs "lock_key", "queue_ttl", "acq_id)";
|
1003
1115
|
"[redis_queued_locks.try_lock.remove_expired_acqs]" # (logs "lock_key", "queue_ttl", "acq_id");
|
1004
1116
|
"[redis_queued_locks.try_lock.get_first_from_queue]" # (logs "lock_key", "queue_ttl", "acq_id", "first_acq_id_in_queue");
|
@@ -1040,7 +1152,10 @@ By default `RedisQueuedLocks::Client` is configured with the void notifier (whic
|
|
1040
1152
|
List of instrumentation events
|
1041
1153
|
|
1042
1154
|
- `redis_queued_locks.lock_obtained`;
|
1155
|
+
- `redis_queued_locks.extendable_reentrant_lock_obtained`;
|
1156
|
+
- `redis_queued_locks.reentrant_lock_obtained`;
|
1043
1157
|
- `redis_queued_locks.lock_hold_and_release`;
|
1158
|
+
- `redis_queued_locks.reentrant_lock_hold_completes`;
|
1044
1159
|
- `redis_queued_locks.explicit_lock_release`;
|
1045
1160
|
- `redis_queued_locks.explicit_all_locks_release`;
|
1046
1161
|
|
@@ -1053,9 +1168,31 @@ Detalized event semantics and payload structure:
|
|
1053
1168
|
- `:ttl` - `integer`/`milliseconds` - lock ttl;
|
1054
1169
|
- `:acq_id` - `string` - lock acquier identifier;
|
1055
1170
|
- `:lock_key` - `string` - lock name;
|
1056
|
-
- `:ts` - `
|
1171
|
+
- `:ts` - `numeric`/`epoch` - the time when the lock was obtaiend;
|
1172
|
+
- `:acq_time` - `float`/`milliseconds` - time spent on lock acquiring;
|
1173
|
+
- `:instrument` - `nil`/`Any` - custom data passed to the `#lock`/`#lock!` method as `:instrument` attribute;
|
1174
|
+
|
1175
|
+
- `"redis_queued_locks.extendable_reentrant_lock_obtained"`
|
1176
|
+
- an event signalizes about the "extendable reentrant lock" obtaining is happened;
|
1177
|
+
- raised from `#lock`/`#lock!` when the lock was obtained as reentrant lock;
|
1178
|
+
- payload:
|
1179
|
+
- `:lock_key` - `string` - lock name;
|
1180
|
+
- `:ttl` - `integer`/`milliseconds` - last lock ttl by reentrant locking;
|
1181
|
+
- `:acq_id` - `string` - lock acquier identifier;
|
1182
|
+
- `:ts` - `numeric`/`epoch` - the time when the lock was obtaiend as extendable reentrant lock;
|
1057
1183
|
- `:acq_time` - `float`/`milliseconds` - time spent on lock acquiring;
|
1058
|
-
- `:instrument` - `nil`/`Any` - custom data passed to the
|
1184
|
+
- `:instrument` - `nil`/`Any` - custom data passed to the `#lock`/`#lock!` method as `:instrument` attribute;
|
1185
|
+
|
1186
|
+
- `"redis_queued_locks.reentrant_lock_obtained"`
|
1187
|
+
- an event signalizes about the "reentrant lock" obtaining is happened (without TTL extension);
|
1188
|
+
- raised from `#lock`/`#lock!` when the lock was obtained as reentrant lock;
|
1189
|
+
- payload:
|
1190
|
+
- `:lock_key` - `string` - lock name;
|
1191
|
+
- `:ttl` - `integer`/`milliseconds` - last lock ttl by reentrant locking;
|
1192
|
+
- `:acq_id` - `string` - lock acquier identifier;
|
1193
|
+
- `:ts` - `numeric`/`epoch` - the time when the lock was obtaiend as reentrant lock;
|
1194
|
+
- `:acq_time` - `float`/`milliseconds` - time spent on lock acquiring;
|
1195
|
+
- `:instrument` - `nil`/`Any` - custom data passed to the `#lock`/`#lock!` method as `:instrument` attribute;
|
1059
1196
|
|
1060
1197
|
- `"redis_queued_locks.lock_hold_and_release"`
|
1061
1198
|
- an event signalizes about the "hold+and+release" process is finished;
|
@@ -1065,9 +1202,22 @@ Detalized event semantics and payload structure:
|
|
1065
1202
|
- `:ttl` - `integer`/`milliseconds` - lock ttl;
|
1066
1203
|
- `:acq_id` - `string` - lock acquier identifier;
|
1067
1204
|
- `:lock_key` - `string` - lock name;
|
1068
|
-
- `:ts` - `
|
1205
|
+
- `:ts` - `numeric`/`epoch` - the time when lock was obtained;
|
1069
1206
|
- `:acq_time` - `float`/`milliseconds` - time spent on lock acquiring;
|
1070
|
-
- `:instrument` - `nil`/`Any` - custom data passed to the
|
1207
|
+
- `:instrument` - `nil`/`Any` - custom data passed to the `#lock`/`#lock!` method as `:instrument` attribute;
|
1208
|
+
|
1209
|
+
- `"redis_queued_locks.reentrant_lock_hold_completes"`
|
1210
|
+
- an event signalizes about the "reentrant lock hold" is complete (both extendable and non-extendable);
|
1211
|
+
- lock re-entering can happen many times and this event happens for each of them separately;
|
1212
|
+
- raised from `#lock`/`#lock!` when the lock was obtained as reentrant lock;
|
1213
|
+
- payload:
|
1214
|
+
- `:hold_time` - `float`/`milliseconds` - lock hold time;
|
1215
|
+
- `:ttl` - `integer`/`milliseconds` - last lock ttl by reentrant locking;
|
1216
|
+
- `:acq_id` - `string` - lock acquier identifier;
|
1217
|
+
- `:ts` - `numeric`/`epoch` - the time when the lock was obtaiend as reentrant lock;
|
1218
|
+
- `:lock_key` - `string` - lock name;
|
1219
|
+
- `:acq_time` - `float`/`milliseconds` - time spent on lock acquiring;
|
1220
|
+
- `:instrument` - `nil`/`Any` - custom data passed to the `#lock`/`#lock!` method as `:instrument` attribute;
|
1071
1221
|
|
1072
1222
|
- `"redis_queued_locks.explicit_lock_release"`
|
1073
1223
|
- an event signalizes about the explicit lock release (invoked via `RedisQueuedLock#unlock`);
|
@@ -1092,20 +1242,29 @@ Detalized event semantics and payload structure:
|
|
1092
1242
|
|
1093
1243
|
<sup>\[[back to top](#table-of-contents)\]</sup>
|
1094
1244
|
|
1095
|
-
- **
|
1096
|
-
-
|
1097
|
-
-
|
1098
|
-
-
|
1099
|
-
-
|
1100
|
-
|
1101
|
-
|
1102
|
-
|
1103
|
-
- support for
|
1104
|
-
-
|
1105
|
-
-
|
1106
|
-
-
|
1107
|
-
|
1108
|
-
-
|
1245
|
+
- **Major**:
|
1246
|
+
- support for Random Access strategy (non-queued behavior);
|
1247
|
+
- lock request prioritization;
|
1248
|
+
- **strict redlock algorithm support** (support for many `RedisClient` instances);
|
1249
|
+
- `#lock_series` - acquire a series of locks:
|
1250
|
+
```ruby
|
1251
|
+
rql.lock_series('lock_a', 'lock_b', 'lock_c') { puts 'locked' }
|
1252
|
+
```
|
1253
|
+
- support for `Dragonfly` database backend (https://github.com/dragonflydb/dragonfly) (https://www.dragonflydb.io/);
|
1254
|
+
- **Minor**:
|
1255
|
+
- Semantic error objects for unexpected Redis errors;
|
1256
|
+
- change all `::Process.clock_gettime(::Process::CLOCK_MONOTONIC)` milliseconds-related invocations to
|
1257
|
+
`::Process.clock_gettime(::Process::CLOCK_MONOTONIC, :millisecond)` in order to reduce time-related math operations count;
|
1258
|
+
- **Experimental feature**: (non-`timed` locks): per-ruby-block-holding-the-lock sidecar `Ractor` and `in progress queue` in RedisDB that will extend
|
1259
|
+
the acquired lock for long-running blocks of code (that invoked "under" the lock
|
1260
|
+
whose ttl may expire before the block execution completes). It makes sense for non-`timed` locks *only*;
|
1261
|
+
- better code stylization (+ some refactorings);
|
1262
|
+
- `RedisQueuedLocks::Acquier::Try.try_to_lock` - detailed successful result analization;
|
1263
|
+
- Support for LIFO strategy;
|
1264
|
+
- better specs with 100% test coverage (total specs rework);
|
1265
|
+
- statistics with UI;
|
1266
|
+
- JSON log formatter;
|
1267
|
+
- `go`-lang implementation;
|
1109
1268
|
|
1110
1269
|
---
|
1111
1270
|
|